[libssh] fatal: dumb http transport does not support shallow capabilities

  • Done
  • quality assurance status badge
Details
3 participants
  • Leo Famulari
  • Ludovic Courtès
  • Maxim Cournoyer
Owner
unassigned
Submitted by
Maxim Cournoyer
Severity
normal
M
M
Maxim Cournoyer wrote on 20 Oct 2018 04:22
(name . bug-guix)(address . bug-guix@gnu.org)(name . Leo Famulari)(address . leo@famulari.name)
87zhv94as1.fsf@gmail.com
Hello,

Today I stumbled upon libssh that wouldn't build, due to the following
git error:

Toggle snippet (3 lines)
fatal: dumb http transport does not support shallow capabilities"

There are now substitutes served from Berlin, so it's a bit more
difficult to reproduce, but the following worked for Ludovic and myself:

guix build --check /gnu/store/f0i7bdcg1lksr9dhz0cidi3ym8r8a5wl-libssh-0.7.6-checkout.drv

The output looks like:

Toggle snippet (6 lines)
guix build --check /gnu/store/f0i7bdcg1lksr9dhz0cidi3ym8r8a5wl-libssh-0.7.6-checkout.drv
building /gnu/store/f0i7bdcg1lksr9dhz0cidi3ym8r8a5wl-libssh-0.7.6-checkout.drv...
Initialized empty Git repository in /gnu/store/gqyjgpvzqy55dzibm59530fsx21dpiz4-libssh-0.7.6-checkout/.git/
fatal: dumb http transport does not support shallow capabilities

Leo, would you have an idea?

Thanks,

Maxim
L
L
Ludovic Courtès wrote on 20 Oct 2018 16:05
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)(address . 33100@debbugs.gnu.org)
87in1w90iu.fsf@gnu.org
Hello!

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

Toggle quote (5 lines)
> guix build --check /gnu/store/f0i7bdcg1lksr9dhz0cidi3ym8r8a5wl-libssh-0.7.6-checkout.drv
> building /gnu/store/f0i7bdcg1lksr9dhz0cidi3ym8r8a5wl-libssh-0.7.6-checkout.drv...
> Initialized empty Git repository in /gnu/store/gqyjgpvzqy55dzibm59530fsx21dpiz4-libssh-0.7.6-checkout/.git/
> fatal: dumb http transport does not support shallow capabilities

On closer inspection, it seems that there’s nothing “fatal” here: if you
let it run for a while (1 or 2 minutes), it ends up silently cloning the
whole repo and the derivation build eventually succeeds.

Could you check if that’s the case for you?

berlin.guixsd.org has a substitute for this checkout, which suggests it
went fine there.

Thanks,
Ludo’.
M
M
Maxim Cournoyer wrote on 21 Oct 2018 05:24
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 33100-done@debbugs.gnu.org)
87va5w3rtj.fsf@gmail.com
Hi,

ludo@gnu.org (Ludovic Courtès) writes:

Toggle quote (13 lines)
> Hello!
>
> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>> guix build --check /gnu/store/f0i7bdcg1lksr9dhz0cidi3ym8r8a5wl-libssh-0.7.6-checkout.drv
>> building /gnu/store/f0i7bdcg1lksr9dhz0cidi3ym8r8a5wl-libssh-0.7.6-checkout.drv...
>> Initialized empty Git repository in /gnu/store/gqyjgpvzqy55dzibm59530fsx21dpiz4-libssh-0.7.6-checkout/.git/
>> fatal: dumb http transport does not support shallow capabilities
>
> On closer inspection, it seems that there’s nothing “fatal” here: if you
> let it run for a while (1 or 2 minutes), it ends up silently cloning the
> whole repo and the derivation build eventually succeeds.

It did end up working fine, although it took a large amout of time for
doing what seems to be a checkout (4 min 46 s). I did some experiments
and this is really the time it took to do a full clone of the libssh
project.

Toggle snippet (14 lines)
time git clone git://git.libssh.org/projects/libssh.git libssh
Cloning into 'libssh'...
remote: Enumerating objects: 28264, done.
remote: Counting objects: 100% (28264/28264), done.
remote: Compressing objects: 100% (11718/11718), done.
remote: Total 28264 (delta 20985), reused 21830 (delta 16350)s
Receiving objects: 100% (28264/28264), 5.21 MiB | 263.00 KiB/s, done.
Resolving deltas: 100% (20985/20985), done.

real 4m19.419s
user 0m3.272s
sys 0m0.540s

It's a bit of a shame, given that the shallow clone takes about 2
seconds (!):

Toggle snippet (14 lines)
time git clone --depth 1 git://git.libssh.org/projects/libssh.git libssh
Cloning into 'libssh'...
remote: Enumerating objects: 367, done.
remote: Counting objects: 100% (367/367), done.
remote: Compressing objects: 100% (358/358), done.
remote: Total 367 (delta 39), reused 53 (delta 1)
Receiving objects: 100% (367/367), 704.23 KiB | 728.00 KiB/s, done.
Resolving deltas: 100% (39/39), done.

real 0m2.028s
user 0m0.160s
sys 0m0.071s

Based on the discussion here:
means that the libssh git server doesn't support the newer "smart HTTP
transport" and the git client bails out (IIUC). At least in our case the
guile-ssh library seems to already correctly fallback to doing a full
clone.

Perhaps just clearer messages would have helped here also ('Failed to do
a shallow git clone due to ~error message~, falling back to a full clone').

Such a change would need to be made in guile-git IIUC.

I'll close this bug now; thank you!

Maxim
Closed
L
L
Leo Famulari wrote on 21 Oct 2018 21:10
20181021191018.GA18445@jasmine.lan
On Sat, Oct 20, 2018 at 11:24:24PM -0400, Maxim Cournoyer wrote:
Toggle quote (10 lines)
> ludo@gnu.org (Ludovic Courtès) writes:
> > On closer inspection, it seems that there’s nothing “fatal” here: if you
> > let it run for a while (1 or 2 minutes), it ends up silently cloning the
> > whole repo and the derivation build eventually succeeds.
>
> It did end up working fine, although it took a large amout of time for
> doing what seems to be a checkout (4 min 46 s). I did some experiments
> and this is really the time it took to do a full clone of the libssh
> project.

Great, you figured it out :)

More explanation:

Git has a few different server protocols: git, dumb HTTP, smart HTTP,
and SSH:


The dumb HTTP protocol is slow and does not show any progress status or
other informative message while it works, but if you monitor your
network traffic you can see it working.

For obvious reasons, it's rare to see the dumb HTTP protocol deployed
nowadays, but you may still find it on legacy installations.
-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEsFFZSPHn08G5gDigJkb6MLrKfwgFAlvMzxcACgkQJkb6MLrK
fwhhIBAAmTI0bBzOCc/7cZ7QeiqvvhdijEbhVi9lEejD7We7LPqNxlwOvabF8xqo
5RUvXl1EzsQNgZ+eOiJvd5UmtTzeFHJ40dSicN+7EGyK2hzCt43IroRjKf51Vjl2
f7zPqSldMGs6ZV6EEjFvkERo5V6A7Tg3fLVO4ivNucnJbmK/FgL/OY/7SOWFQwTS
nqPLKav98zo58LNZRpyEPm4uxkkOW/w8svPuRRxIbC82ris8YueIpUk7CHfyqwJW
foVSurk3OsGECqOiicY0xNkSl5+qAKsnu3OLdUJSauPW74Y18MCD0foy4DrGyFWz
fCNgNbts8G+KThPjb3VGskoUqvEjaM9egAfcW1b9Sk+LkjIknfO+fdpPcjjL3PrA
1EiClmc8KqonzhSL8RlFx0fHpYGza+Z6c7n1R6gJH332BcsO0nJTQpVi4l1UsFBO
U5Xp3fXGdF39lQSCvpAdePFJWUrY14S5Mby3pTOgIawB0uH1Jx5aoj9AY/Mw2fUz
xFoRt/VyqSKoWLDCQGyORAYVlLeJPgEltIgysu7Nw7p1VV0R5RQc+U1/TYHeUawc
3cvrEb3SK3UxsC/g7ZWTrxOJH/gJT+VUYquZjzACsC0efDkgd5UoAnbe1rN3JJFn
KNL62ROCQZaFkvdc69XO8PGMydeykvTskn+lnAFWXfKfgoO1pz4=
=cHdi
-----END PGP SIGNATURE-----


M
M
Maxim Cournoyer wrote on 21 Oct 2018 23:42
(name . Leo Famulari)(address . leo@famulari.name)(address . 33100@debbugs.gnu.org)
87ftwz3rkd.fsf@gmail.com
Hello Leo,

Leo Famulari <leo@famulari.name> writes:

Toggle quote (27 lines)
> On Sat, Oct 20, 2018 at 11:24:24PM -0400, Maxim Cournoyer wrote:
>> ludo@gnu.org (Ludovic Courtès) writes:
>> > On closer inspection, it seems that there’s nothing “fatal” here: if you
>> > let it run for a while (1 or 2 minutes), it ends up silently cloning the
>> > whole repo and the derivation build eventually succeeds.
>>
>> It did end up working fine, although it took a large amout of time for
>> doing what seems to be a checkout (4 min 46 s). I did some experiments
>> and this is really the time it took to do a full clone of the libssh
>> project.
>
> Great, you figured it out :)
>
> More explanation:
>
> Git has a few different server protocols: git, dumb HTTP, smart HTTP,
> and SSH:
>
> https://git-scm.com/book/en/v2/Git-on-the-Server-The-Protocols
>
> The dumb HTTP protocol is slow and does not show any progress status or
> other informative message while it works, but if you monitor your
> network traffic you can see it working.
>
> For obvious reasons, it's rare to see the dumb HTTP protocol deployed
> nowadays, but you may still find it on legacy installations.

Thanks for the extra bits of information :)

Cheers,

Maxim
L
L
Ludovic Courtès wrote on 22 Oct 2018 11:55
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)(address . 33100-done@debbugs.gnu.org)
875zxupapd.fsf@gnu.org
Hello,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
Toggle quote (42 lines)
> It did end up working fine, although it took a large amout of time for
> doing what seems to be a checkout (4 min 46 s). I did some experiments
> and this is really the time it took to do a full clone of the libssh
> project.
>
> time git clone git://git.libssh.org/projects/libssh.git libssh
> Cloning into 'libssh'...
> remote: Enumerating objects: 28264, done.
> remote: Counting objects: 100% (28264/28264), done.
> remote: Compressing objects: 100% (11718/11718), done.
> remote: Total 28264 (delta 20985), reused 21830 (delta 16350)s
> Receiving objects: 100% (28264/28264), 5.21 MiB | 263.00 KiB/s, done.
> Resolving deltas: 100% (20985/20985), done.
>
> real 4m19.419s
> user 0m3.272s
> sys 0m0.540s
>
>
> It's a bit of a shame, given that the shallow clone takes about 2
> seconds (!):
>
> time git clone --depth 1 git://git.libssh.org/projects/libssh.git libssh
> Cloning into 'libssh'...
> remote: Enumerating objects: 367, done.
> remote: Counting objects: 100% (367/367), done.
> remote: Compressing objects: 100% (358/358), done.
> remote: Total 367 (delta 39), reused 53 (delta 1)
> Receiving objects: 100% (367/367), 704.23 KiB | 728.00 KiB/s, done.
> Resolving deltas: 100% (39/39), done.
>
> real 0m2.028s
> user 0m0.160s
> sys 0m0.071s
>
> Based on the discussion here:
> https://github.com/CocoaPods/CocoaPods/issues/6270, it would seem this
> means that the libssh git server doesn't support the newer "smart HTTP
> transport" and the git client bails out (IIUC). At least in our case the
> guile-ssh library seems to already correctly fallback to doing a full
> clone.

Switching to the git:// transport would seem like a reasonable
workaround—we’d lose encryption and authentication, but the latter is
covered by the content hash in the ‘origin’ anyway.

WDYT, Leo?

Toggle quote (3 lines)
> Perhaps just clearer messages would have helped here also ('Failed to do
> a shallow git clone due to ~error message~, falling back to a full clone').

I agree, and that’s something to suggest to the Git folks. :-)

Thanks,
Ludo’.
Closed
L
L
Leo Famulari wrote on 22 Oct 2018 19:07
(name . Ludovic Courtès)(address . ludo@gnu.org)
20181022170734.GB6300@jasmine.lan
On Mon, Oct 22, 2018 at 11:55:26AM +0200, Ludovic Courtès wrote:
Toggle quote (6 lines)
> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
> > It did end up working fine, although it took a large amout of time for
> > doing what seems to be a checkout (4 min 46 s). I did some experiments
> > and this is really the time it took to do a full clone of the libssh
> > project.

[...]

Toggle quote (3 lines)
> > It's a bit of a shame, given that the shallow clone takes about 2
> > seconds (!):

Yeah, it's incredibly slow. The repo is not even 10 MB. At first, I too
thought the HTTPS clone had stalled.

Protocol duration
------------------------
https:// 217 sec
git:// 10 sec
git:// shallow 1.5 sec

And of course, the shallow clone is 3.6 MB instead of 10 MB.

Toggle quote (6 lines)
> Switching to the git:// transport would seem like a reasonable
> workaround—we’d lose encryption and authentication, but the latter is
> covered by the content hash in the ‘origin’ anyway.
>
> WDYT, Leo?

Overall, I think the slowness doesn't matter too much, since we offer
substitutes for the patched source code. However, here is a patch.
Please feel free to apply it!

I'll ask the libssh team to support "smart" HTTP Git.
From 683713dd8f5d67e3f077d5d13c23e5556d8ad779 Mon Sep 17 00:00:00 2001
From: Leo Famulari <leo@famulari.name>
Date: Mon, 22 Oct 2018 13:00:55 -0400
Subject: [PATCH] gnu: libssh: Fetch the source code more efficiently.

* gnu/packages/ssh.scm (libssh)[source]: Use the git:// protocol.
---
gnu/packages/ssh.scm | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)

Toggle diff (23 lines)
diff --git a/gnu/packages/ssh.scm b/gnu/packages/ssh.scm
index e806fc80f..b93cb03a1 100644
--- a/gnu/packages/ssh.scm
+++ b/gnu/packages/ssh.scm
@@ -71,8 +71,14 @@
(source (origin
(method git-fetch)
(uri (git-reference
- (url "https://git.libssh.org/projects/libssh.git")
- (commit (string-append "libssh-" version))))
+ ;; git.libssh.org does not support the fast "smart" HTTP
+ ;; Git protocol. The "dumb" HTTP Git protocol is extremely
+ ;; slow, and does not support shallow clones, so we use the
+ ;; plain Git protocol despite its flaws. This offers an
+ ;; incredible speedup and reduces the size of the the
+ ;; source by more than half.
+ (url "git://git.libssh.org/projects/libssh.git")
+ (commit (string-append "libssh-" version))))
(patches (search-patches "libssh-hostname-parser-bug.patch"))
(sha256
(base32
--
2.19.1
-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEsFFZSPHn08G5gDigJkb6MLrKfwgFAlvOA9YACgkQJkb6MLrK
fwj2nRAAyULtNR3TnVyjEzaCsgkSZOISogcmCR7po6YqjG0dRCTTmdzu/q4rfot0
10QKPLQOM8c7mFCeYCc2LJDaXEaeYnjiCrtwfd10/PyvHSyZeJBnDDo5xxaT42UQ
T3AbC2acA2R/F41L4upH3tW0Xa0szFu7T8A1qZ+NjZjlJZ/hivO/rsVKiL03PCTi
ICfPFijxkh//8OIAva4BYrqdcHurgKfwxuvFl/mt7ynR4NcppIL/YI9L5j9vq7mZ
WeNJW+uCi59GTJU5CUWgvPqZ5QSwkgtKUBpSW4XoyBZzyQ7tNEUP0rMy2iF8y0AT
4ohqKMzR3/7anJ93HFfwcoQue+F9bMVGFP5Rk0/Bdmv2kxFxNE9mx2llDFbJAbQF
gVJjesMoVUHnZWLp9dszWNfdK38wcu51E+9fU8DUsayYkfmSUbvgkv5OnxLc4ruE
pNb/yXlEhYm21oJcl7Ju2W/yRZh26owS0n35yU/pAX6XyrsljLbuj0e0nal5/ZJV
wh5O+swksHcRC4Pp4KKf43uKi2dHCuTnpY90sWWhAPUyb+tJb8I8FTCvIM4j/fuN
vvEIoY3nqOM0amx7o7R1KCH83E3f1RaQFW8bgdaEONOfpJgnnQ36NdeR5vYtpye4
TlB96uIbw6SlFWcBdLNR+wokzAycGDwzmJy8mA1qCiSaBPTGRY4=
=K/th
-----END PGP SIGNATURE-----


Closed
M
M
Maxim Cournoyer wrote on 24 Oct 2018 15:01
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 33100-done@debbugs.gnu.org)
87o9bj33e4.fsf@gmail.com
Hi Ludovic,

ludo@gnu.org (Ludovic Courtès) writes:

[...]

Toggle quote (5 lines)
>> Perhaps just clearer messages would have helped here also ('Failed to do
>> a shallow git clone due to ~error message~, falling back to a full clone').
>
> I agree, and that’s something to suggest to the Git folks. :-)

Upon further inspection, we can do better on our side, specifically by
printing a message of the fallback occurring when calling git-fetch. The
patch below implements this simple change.

What do you think?
From 06ba66d1949ba59573518f471ad3cbacefea6ea2 Mon Sep 17 00:00:00 2001
From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
Date: Wed, 24 Oct 2018 08:49:50 -0400
Subject: [PATCH] build: git-fetch: Print message when falling back to a full
checkout.

Otherwise the user might believe that git-fetch stalled, observing the lack of
output following a 'fatal' git error message (see:

* guix/build/git.scm (git-fetch): Print message when falling back to a full
checkout.
---
guix/build/git.scm | 1 +
1 file changed, 1 insertion(+)

Toggle diff (14 lines)
diff --git a/guix/build/git.scm b/guix/build/git.scm
index 14d415a6f..df8f1548b 100644
--- a/guix/build/git.scm
+++ b/guix/build/git.scm
@@ -45,6 +45,7 @@ recursively. Return #t on success, #f otherwise."
(if (zero? (system* git-command "fetch" "--depth" "1" "origin" commit))
(invoke git-command "checkout" "FETCH_HEAD")
(begin
+ (format #t "Failed to do a shallow fetch; retrying a full fetch...~%")
(invoke git-command "fetch" "origin")
(invoke git-command "checkout" commit)))
(when recursive?
--
2.19.0
Maxim
Closed
L
L
Ludovic Courtès wrote on 24 Oct 2018 16:11
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)(address . 33100-done@debbugs.gnu.org)
87va5r1ll0.fsf@gnu.org
Hi,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

Toggle quote (19 lines)
> Upon further inspection, we can do better on our side, specifically by
> printing a message of the fallback occurring when calling git-fetch. The
> patch below implements this simple change.
>
> What do you think?
>
> From 06ba66d1949ba59573518f471ad3cbacefea6ea2 Mon Sep 17 00:00:00 2001
> From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
> Date: Wed, 24 Oct 2018 08:49:50 -0400
> Subject: [PATCH] build: git-fetch: Print message when falling back to a full
> checkout.
>
> Otherwise the user might believe that git-fetch stalled, observing the lack of
> output following a 'fatal' git error message (see:
> https://debbugs.gnu.org/33100).
>
> * guix/build/git.scm (git-fetch): Print message when falling back to a full
> checkout.

Oh indeed, I had overlooked that.

Go for it!

Thanks,
Ludo’.
Closed
L
L
Leo Famulari wrote on 24 Oct 2018 16:15
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
20181024141523.GB5988@jasmine.lan
On Wed, Oct 24, 2018 at 09:01:07AM -0400, Maxim Cournoyer wrote:
Toggle quote (13 lines)
> From 06ba66d1949ba59573518f471ad3cbacefea6ea2 Mon Sep 17 00:00:00 2001
> From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
> Date: Wed, 24 Oct 2018 08:49:50 -0400
> Subject: [PATCH] build: git-fetch: Print message when falling back to a full
> checkout.
>
> Otherwise the user might believe that git-fetch stalled, observing the lack of
> output following a 'fatal' git error message (see:
> https://debbugs.gnu.org/33100).
>
> * guix/build/git.scm (git-fetch): Print message when falling back to a full
> checkout.

I like it, but it doesn't seem to actually print anything for me when I
trigger the failing case, for example by fetching the libssh source over
HTTP.
-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEsFFZSPHn08G5gDigJkb6MLrKfwgFAlvQfnsACgkQJkb6MLrK
fwhSphAA3awEUDSHPvFghTiah6x2ryHL7zBvj8uw8lzWqfiWaIEDfcJUbiKhdq6D
3cyzMPXJOzwGRA0iQy52rsRrBo5NT33Q3T9guKg86Je3+DcQP1e+iBpJaOHr4S3B
lL38G34H1+jD7JVTROXN7Hh5EJf4vy7cMKAjU11R+/cVFrpwLfOCD+V4yUn75Rrs
IIni4vi391TsOGUdJzZCZzEwlhP7B7x3X0NWGmIwM1ii//1BTiabqcAcD/GpCZ5m
yT7ZTawJuOcz9sBwC1evpQFH3ZG08FB9RE862bRmV3JzNRb7c8f0fq5BJWwnrh76
1dp2Ad+7ANPNVfIn2z/wOQtX/Gl3inqz9gCUefiFq9qGDpPWXMLRXdplWGAW1mOy
2rC2VIl1OB7Cvg/VxYyYjkVeOwnhpRewS1/B7S7xuk60KZNY01vrOE0FabgHi5D6
e8UNvBKCp0AgzsoM69SH8DlKAQk5kK606gl2/qyyMGD7Fz5EsXTP6bbsOsc3nLzE
08MLu8CtacwCEJFD5XUScNmF2B+HvGq2MPJaf118qBJcSFXeuaZZpUv/MRL6UJHt
EBSxhuDNEIYWa5cMmkrFA8C2/aHaITFH3hDik1JIpDxYN5IWCOB6GJsE9C2WWHLw
2nnu1hzZV2BWyEL67KRAXYaGiKikeC34Mtc2qRgcs7/UYc7idkw=
=mxbW
-----END PGP SIGNATURE-----


Closed
L
L
Ludovic Courtès wrote on 24 Oct 2018 23:33
(name . Leo Famulari)(address . leo@famulari.name)
87h8hbyqr4.fsf@gnu.org
Leo Famulari <leo@famulari.name> skribis:

Toggle quote (18 lines)
> On Wed, Oct 24, 2018 at 09:01:07AM -0400, Maxim Cournoyer wrote:
>> From 06ba66d1949ba59573518f471ad3cbacefea6ea2 Mon Sep 17 00:00:00 2001
>> From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
>> Date: Wed, 24 Oct 2018 08:49:50 -0400
>> Subject: [PATCH] build: git-fetch: Print message when falling back to a full
>> checkout.
>>
>> Otherwise the user might believe that git-fetch stalled, observing the lack of
>> output following a 'fatal' git error message (see:
>> https://debbugs.gnu.org/33100).
>>
>> * guix/build/git.scm (git-fetch): Print message when falling back to a full
>> checkout.
>
> I like it, but it doesn't seem to actually print anything for me when I
> trigger the failing case, for example by fetching the libssh source over
> HTTP.

If might be that current-output-port is fully buffered. What if you
add:

(setvbuf (current-output-port) 'line)

before the ‘format’ call?

Thanks,
Ludo’, who is found guilty of not actually running the code.
Closed
L
L
Leo Famulari wrote on 25 Oct 2018 03:02
(name . Ludovic Courtès)(address . ludo@gnu.org)
20181025010241.GA16007@jasmine.lan
On Wed, Oct 24, 2018 at 11:33:03PM +0200, Ludovic Courtès wrote:
Toggle quote (7 lines)
> If might be that current-output-port is fully buffered. What if you
> add:
>
> (setvbuf (current-output-port) 'line)
>
> before the ‘format’ call?

Yeah, this lets' the message through.
Closed
M
M
Maxim Cournoyer wrote on 25 Oct 2018 03:44
(name . Ludovic Courtès)(address . ludo@gnu.org)
87k1m63ilz.fsf@gmail.com
ludo@gnu.org (Ludovic Courtès) writes:

Toggle quote (30 lines)
> Leo Famulari <leo@famulari.name> skribis:
>
>> On Wed, Oct 24, 2018 at 09:01:07AM -0400, Maxim Cournoyer wrote:
>>> From 06ba66d1949ba59573518f471ad3cbacefea6ea2 Mon Sep 17 00:00:00 2001
>>> From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
>>> Date: Wed, 24 Oct 2018 08:49:50 -0400
>>> Subject: [PATCH] build: git-fetch: Print message when falling back to a full
>>> checkout.
>>>
>>> Otherwise the user might believe that git-fetch stalled, observing the lack of
>>> output following a 'fatal' git error message (see:
>>> https://debbugs.gnu.org/33100).
>>>
>>> * guix/build/git.scm (git-fetch): Print message when falling back to a full
>>> checkout.
>>
>> I like it, but it doesn't seem to actually print anything for me when I
>> trigger the failing case, for example by fetching the libssh source over
>> HTTP.
>
> If might be that current-output-port is fully buffered. What if you
> add:
>
> (setvbuf (current-output-port) 'line)
>
> before the ‘format’ call?
>
> Thanks,
> Ludo’, who is found guilty of not actually running the code.

What is preferable, between your solution or using (force-output)?

I'll send a revised patch accordingly.

Thank you,

Maxim
Closed
L
L
Ludovic Courtès wrote on 25 Oct 2018 14:24
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
87lg6musc4.fsf@gnu.org
Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

Toggle quote (34 lines)
> ludo@gnu.org (Ludovic Courtès) writes:
>
>> Leo Famulari <leo@famulari.name> skribis:
>>
>>> On Wed, Oct 24, 2018 at 09:01:07AM -0400, Maxim Cournoyer wrote:
>>>> From 06ba66d1949ba59573518f471ad3cbacefea6ea2 Mon Sep 17 00:00:00 2001
>>>> From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
>>>> Date: Wed, 24 Oct 2018 08:49:50 -0400
>>>> Subject: [PATCH] build: git-fetch: Print message when falling back to a full
>>>> checkout.
>>>>
>>>> Otherwise the user might believe that git-fetch stalled, observing the lack of
>>>> output following a 'fatal' git error message (see:
>>>> https://debbugs.gnu.org/33100).
>>>>
>>>> * guix/build/git.scm (git-fetch): Print message when falling back to a full
>>>> checkout.
>>>
>>> I like it, but it doesn't seem to actually print anything for me when I
>>> trigger the failing case, for example by fetching the libssh source over
>>> HTTP.
>>
>> If might be that current-output-port is fully buffered. What if you
>> add:
>>
>> (setvbuf (current-output-port) 'line)
>>
>> before the ‘format’ call?
>>
>> Thanks,
>> Ludo’, who is found guilty of not actually running the code.
>
> What is preferable, between your solution or using (force-output)?

I’d go for line buffering since you only need to do it once for all.

Thanks!
Ludo’.
Closed
M
M
Maxim Cournoyer wrote on 27 Oct 2018 04:05
(name . Ludovic Courtès)(address . ludo@gnu.org)
878t2kuosf.fsf@gmail.com
Hello!

ludo@gnu.org (Ludovic Courtès) writes:

Toggle quote (18 lines)
>>>> I like it, but it doesn't seem to actually print anything for me when I
>>>> trigger the failing case, for example by fetching the libssh source over
>>>> HTTP.
>>>
>>> If might be that current-output-port is fully buffered. What if you
>>> add:
>>>
>>> (setvbuf (current-output-port) 'line)
>>>
>>> before the ‘format’ call?
>>>
>>> Thanks,
>>> Ludo’, who is found guilty of not actually running the code.
>>
>> What is preferable, between your solution or using (force-output)?
>
> I’d go for line buffering since you only need to do it once for all.

I finally got around to reproducing the problem and testing the fix; it
was costly to build using --no-substitutes.

Is it OK to push this patch into master?

Thanks,

Maxim
From 44782db3f63a29cdbd98cddb77eab8a473806765 Mon Sep 17 00:00:00 2001
From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
Date: Wed, 24 Oct 2018 08:49:50 -0400
Subject: [PATCH] git-download: Print a message when falling back to a full
fetch.

Otherwise the user might believe that git-fetch stalled, observing the lack of
output following a 'fatal' git error message (see:

* guix/build/git.scm (git-fetch): Print message when falling back to a full
fetch.
---
guix/build/git.scm | 2 ++
1 file changed, 2 insertions(+)

Toggle diff (15 lines)
diff --git a/guix/build/git.scm b/guix/build/git.scm
index 14d415a6f..2d1700a9b 100644
--- a/guix/build/git.scm
+++ b/guix/build/git.scm
@@ -45,6 +45,8 @@ recursively. Return #t on success, #f otherwise."
(if (zero? (system* git-command "fetch" "--depth" "1" "origin" commit))
(invoke git-command "checkout" "FETCH_HEAD")
(begin
+ (setvbuf (current-output-port) 'line)
+ (format #t "Failed to do a shallow fetch; retrying a full fetch...~%")
(invoke git-command "fetch" "origin")
(invoke git-command "checkout" commit)))
(when recursive?
--
2.19.0
Closed
L
L
Ludovic Courtès wrote on 27 Oct 2018 16:45
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
877ei3a1nn.fsf@gnu.org
Hi Maxim,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

Toggle quote (25 lines)
> ludo@gnu.org (Ludovic Courtès) writes:
>
>>>>> I like it, but it doesn't seem to actually print anything for me when I
>>>>> trigger the failing case, for example by fetching the libssh source over
>>>>> HTTP.
>>>>
>>>> If might be that current-output-port is fully buffered. What if you
>>>> add:
>>>>
>>>> (setvbuf (current-output-port) 'line)
>>>>
>>>> before the ‘format’ call?
>>>>
>>>> Thanks,
>>>> Ludo’, who is found guilty of not actually running the code.
>>>
>>> What is preferable, between your solution or using (force-output)?
>>
>> I’d go for line buffering since you only need to do it once for all.
>
> I finally got around to reproducing the problem and testing the fix; it
> was costly to build using --no-substitutes.
>
> Is it OK to push this patch into master?

Definitely, thank you!

Ludo’.
Closed
M
M
Maxim Cournoyer wrote on 29 Oct 2018 03:45
(name . Ludovic Courtès)(address . ludo@gnu.org)
87k1m1v5b1.fsf@gmail.com
Hello,

ludo@gnu.org (Ludovic Courtès) writes:

[...]

Toggle quote (6 lines)
>> Is it OK to push this patch into master?
>
> Definitely, thank you!
>
> Ludo’.

Pushed as commit 2f18b7329! Thank you,

Maxim
Closed
?