guix build --subsitute-urls does not override guix-daemon run with --no-subsitutes

  • Done
  • quality assurance status badge
Details
4 participants
  • Drashne
  • Leo Famulari
  • Maxim Cournoyer
  • Ricardo Wurmus
Owner
unassigned
Submitted by
Drashne
Severity
normal
D
D
Drashne wrote on 8 Oct 2017 19:13
(name . bug-guix@gnu.org)(address . bug-guix@gnu.org)
xvi7dt6njIocawq9JtdiRt4d7d9CfJez9xVJzwykKMD0l-6Wdnfl9tRo2dy_b_vSeHPebsCzMgOSuBHd4RcsE0k1Ky__zO_6FZX8hQ2jkd4=@protonmail.com
From the kind people on #guix, I've heard that "guix build --subsitute-urls" should override "guix-daemon --no-subsitutes", but it seems it's not doing so for me.

Here's the situation I ran in to:

While doing a "./pre-inst-env guix pull" I got an error about:

output path `/gnu/store/53lj4z9cavl7n27r89zjnvyd8fk854kj-libgit2-0.26.0.tar.gz' should have sha256 hash `1fdk9yhwvl1w1z71ykzcvgh4nsf8scxcbclz5anh98zpplmhmisa', instead has `
1b3figbhp5l83vd37vq6j2narrq4yl9pfw6mw0px0dzb1hz3jqka'

So I tried "./pre-inst-env guix build --source libgit2 --substitute-urls=[https://mirror.hydra.gnu.org"](https://mirror.hydra.gnu.org)
which had the wrong hash.

Attached is the full log of that failed attempt. This was done while guix-daemon was run with the --no-subsitutes option.

Then I killed guix-daemon and restarted it without --no-subsitutes, and did the same thing and it worked (log of the success also attached).
Attachment: file
sh-4.3$ ./pre-inst-env guix build --source libgit2 --substitute-urls=https://mirror.hydra.gnu.org
The following derivations will be built:
/gnu/store/5szrmzmfgxk6pylk5fh9bk8apj4x8axf-libgit2-0.26.0.tar.xz.drv
/gnu/store/mgh4yjxkxfyqmc7c61vwq4vs8v837602-libgit2-0.26.0.tar.gz.drv
@ build-started /gnu/store/mgh4yjxkxfyqmc7c61vwq4vs8v837602-libgit2-0.26.0.tar.gz.drv - x86_64-linux /var/log/guix/drvs/mg//h4yjxkxfyqmc7c61vwq4vs8v837602-libgit2-0.26.0.tar
.gz.drv.bz2

Starting download of /gnu/store/53lj4z9cavl7n27r89zjnvyd8fk854kj-libgit2-0.26.0.tar.gz
v0.26.0 4.2MiB/s 00:01 | 4.5MiB transferred
output path `/gnu/store/53lj4z9cavl7n27r89zjnvyd8fk854kj-libgit2-0.26.0.tar.gz' should have sha256 hash `1fdk9yhwvl1w1z71ykzcvgh4nsf8scxcbclz5anh98zpplmhmisa', instead has `
1b3figbhp5l83vd37vq6j2narrq4yl9pfw6mw0px0dzb1hz3jqka'
@ build-failed /gnu/store/mgh4yjxkxfyqmc7c61vwq4vs8v837602-libgit2-0.26.0.tar.gz.drv - 1 output path `/gnu/store/53lj4z9cavl7n27r89zjnvyd8fk854kj-libgit2-0.26.0.tar.gz' shou
ld have sha256 hash `1fdk9yhwvl1w1z71ykzcvgh4nsf8scxcbclz5anh98zpplmhmisa', instead has `1b3figbhp5l83vd37vq6j2narrq4yl9pfw6mw0px0dzb1hz3jqka'
cannot build derivation `/gnu/store/5szrmzmfgxk6pylk5fh9bk8apj4x8axf-libgit2-0.26.0.tar.xz.drv': 1 dependencies couldn't be built
guix build: error: build failed: build of `/gnu/store/5szrmzmfgxk6pylk5fh9bk8apj4x8axf-libgit2-0.26.0.tar.xz.drv' failed
sh-4.3$ ./pre-inst-env guix build --source libgit2 --substitute-urls=https://mirror.hydra.gnu.org
substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%
2.8 MB will be downloaded:
/gnu/store/s62d5lbr6sb7x0mxhhdwf13in7yi8mbc-libgit2-0.26.0.tar.xz
substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%
@ substituter-started /gnu/store/s62d5lbr6sb7x0mxhhdwf13in7yi8mbc-libgit2-0.26.0.tar.xz /gnu/store/vir3lrwqy50pr8fkaf3m091dgbrja2n6-guix-0.13.0/libexec/guix/substitute
libgit2-0.26.0.tar.xz 2.7MiB 2.3MiB/s 00:01 [####################] 100.0%

@ substituter-succeeded /gnu/store/s62d5lbr6sb7x0mxhhdwf13in7yi8mbc-libgit2-0.26.0.tar.xz
/gnu/store/s62d5lbr6sb7x0mxhhdwf13in7yi8mbc-libgit2-0.26.0.tar.xz
L
L
Leo Famulari wrote on 9 Oct 2017 21:20
(name . Drashne)(address . drashne@protonmail.com)(address . 28749-done@debbugs.gnu.org)
20171009192034.GB16949@jasmine.lan
On Sun, Oct 08, 2017 at 01:13:16PM -0400, Drashne wrote:
Toggle quote (4 lines)
> From the kind people on #guix, I've heard that "guix build
> --subsitute-urls" should override "guix-daemon --no-subsitutes", but
> it seems it's not doing so for me.

The documentation of guix-daemon [0] says this on the subject:

"When the daemon runs with --no-substitutes, clients can still
explicitly enable substitution via the set-build-options remote
procedure call (see The Store)."

So, there is a way for unprivileged users to enable substitution for
themselves even when the local administrator has disabled substitution,
but it's not via the --substitute-urls mechanism.

I'm closing this bug because I think it's mostly a case of having
received mistaken advice on #guix.

[0]

Toggle quote (11 lines)
> Here's the situation I ran in to:
>
> While doing a "./pre-inst-env guix pull" I got an error about:
>
> output path `/gnu/store/53lj4z9cavl7n27r89zjnvyd8fk854kj-libgit2-0.26.0.tar.gz' should have sha256 hash `1fdk9yhwvl1w1z71ykzcvgh4nsf8scxcbclz5anh98zpplmhmisa', instead has `
> 1b3figbhp5l83vd37vq6j2narrq4yl9pfw6mw0px0dzb1hz3jqka'
>
> So I tried "./pre-inst-env guix build --source libgit2 --substitute-urls=[https://mirror.hydra.gnu.org"](https://mirror.hydra.gnu.org)
> but it redirected me to https://codeload.github.com/libgit2/libgit2/tar.gz/v0.26.0
> which had the wrong hash.

We are discussing how to handle unstable upstream sources more
gracefully here:

-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEsFFZSPHn08G5gDigJkb6MLrKfwgFAlnbzAIACgkQJkb6MLrK
fwghAA/+O5V1E5GiZWo8S0iKJK37JugDTNigtTRqW5taBW9/zqeYCq1V4M3yHsmS
YWRdDk3b7F6QbtzkO2/lER9jXOu+lNS3YC/mmhp2r/X0nptaXeoCSt0ntJHHZxsp
tX9HtpQ8ISQWY6btnUnDtT4RQa72pSQf+FynBqyHjTlveM+swkyS5NzPLWVuH32L
CFxaOJ9Q5G5NUv9Qe6aTkaA97x0Af1I2n7EGVzJlfSQdoCvp91w/JAb21hhX8/TR
6fjz0rAZ96WudAx+8MW2zWPrEyLL2AyIWeAJvDACTgRkLTTXjYaEJDhpNmljYz4v
wtuz7B8jbVnBA7jo4ul+bmNSsksGGkgkiO8CUgVvs0Cwhfw8MunNtCL46HWqZYdM
l7krmspep+db4ALyGEFrzpcgK0n4ZVsZmNxgRh9ryRHsdPx9xNUAhXDCO7F9JIq7
Z2PH6Y4iz1I0ja3kYMyfL6cwU2ydB5O7pRoJVhZV2RV6hBw4MGrUD43Fq4xFpi1c
wuukHG+SSJ3dpmRVEWqNCARsst66OevhaojksuT7FMTn26unrvxAg3QatJN+0KOd
6c245wxZKhrZXLjAyl/UWAQ6+vw7CdC0ZqArxzPCJqpkNU/IvGJwQ55cUT/wieKa
Xwn+lONibWInW3jDGAAs4hEFTL/iH/pJ9xCX8L0ztvItRwN50es=
=UW/W
-----END PGP SIGNATURE-----


Closed
M
M
Maxim Cournoyer wrote on 13 Oct 2017 15:59
(address . 28749@debbugs.gnu.org)
87tvz3f8sp.fsf@gmail.com
Leo Famulari <leo@famulari.name> writes:

Toggle quote (18 lines)
> On Sun, Oct 08, 2017 at 01:13:16PM -0400, Drashne wrote:
>> From the kind people on #guix, I've heard that "guix build
>> --subsitute-urls" should override "guix-daemon --no-subsitutes", but
>> it seems it's not doing so for me.
>
> The documentation of guix-daemon [0] says this on the subject:
>
> "When the daemon runs with --no-substitutes, clients can still
> explicitly enable substitution via the set-build-options remote
> procedure call (see The Store)."
>
> So, there is a way for unprivileged users to enable substitution for
> themselves even when the local administrator has disabled substitution,
> but it's not via the --substitute-urls mechanism.
>
> I'm closing this bug because I think it's mostly a case of having
> received mistaken advice on #guix.

Eh, I'm sorry I was the one suggesting to open this bug report in the
first place!

Although, I would argue that the current behavior is
non-intuitive. While true that the manual skim about how one can achieve
this, the reference to "The Store" is not helpful; it doesn't even
mention the "set-build-options" procedure. Also, leaving the command
line to plug directly into Guix's API from Guile is inconvenient at best.

It seems to me that the current behavior of other options that affect
the guix-daemon operation are that user options override the
corresponding guix-daemon defaults; maybe that's what lead me and others
to think that --substitute-urls should attempt to do what the user
desires?

Otherwise, we could at least give advice on the output of a Guix command
when the user passed --substitute-urls when the guix-daemon substitutes
were disabled to make this clear(er).

My 2 cents,

Maxim
L
L
Leo Famulari wrote on 13 Oct 2017 23:54
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
20171013215430.GA1951@jasmine.lan
On Fri, Oct 13, 2017 at 09:59:18AM -0400, Maxim Cournoyer wrote:
Toggle quote (12 lines)
> Although, I would argue that the current behavior is
> non-intuitive. While true that the manual skim about how one can achieve
> this, the reference to "The Store" is not helpful; it doesn't even
> mention the "set-build-options" procedure. Also, leaving the command
> line to plug directly into Guix's API from Guile is inconvenient at best.
>
> It seems to me that the current behavior of other options that affect
> the guix-daemon operation are that user options override the
> corresponding guix-daemon defaults; maybe that's what lead me and others
> to think that --substitute-urls should attempt to do what the user
> desires?

Yeah, maybe it should be changed to be consistent with the behavior of
the other options.

Toggle quote (4 lines)
> Otherwise, we could at least give advice on the output of a Guix command
> when the user passed --substitute-urls when the guix-daemon substitutes
> were disabled to make this clear(er).

Agreed, we should at least do that.
-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEsFFZSPHn08G5gDigJkb6MLrKfwgFAlnhNhMACgkQJkb6MLrK
fwjwFQ//ZjzqlADa/Cn9MP/E+3fZRr80gg6Y7Nhpi68TDCTL+GAcU/0wizLrQNBV
oFStODCHL3l3mXIL4HYhAPKVqqBCnNVNN/glTvH+eLE9uciRB4o/fKtWRQUqK/Yp
q1a4jWvSp+vLEcZavW+q+u0e/QFW3+S8iZwG4vO3ydLTHNTZECop0dV4tJLpfmpe
32RgFHiDnW8Jd4Gjuj7eEWM6FRnpH012hpz4B7nMGJhGdBu88dkftygTbwKvcaQE
Gxnz3d0eVPgCVtN5lTVF0as+MsJ2L+Kfj/NpqOwnv0vzXvMBQxH8qY+KSGpVWquF
evlvkTAAj3LXJJMHmdcahyGqDzYCv2GnptmOlncwa21PuXoRTpVBDvrt7pFdNsht
DuUZLMPvefx59uc8KRiXa3/nUzReip13SOjlxgnprdIWtFdRFuW+Rt3z2YmwZa2t
0GD/9dTz3pK0rmYeDNOCT9eBXZAeGC1RfQrMZ+2MSWy4Rj2uUlcBQkgT2AbYsD9I
lW7WDiJZJ1g4U0C1CeM4/hqtxRyodc4EXt+epYAfagfc/Mjtc0pFsTcYwqel9I/v
NiHldipTrx4+puy1dGVapDsUoq3NrWQhpy5Or3xZIBAw/ZixDDAAds8/81XBZMD3
GrhwgQ8nLLWbUN/BPW4ZCMSqsXUH1OBTWPU5+s5NSlQTiXOQKRE=
=DJbi
-----END PGP SIGNATURE-----


R
R
Ricardo Wurmus wrote on 14 Oct 2017 15:23
(name . Leo Famulari)(address . leo@famulari.name)
871sm5uala.fsf@elephly.net
Leo Famulari <leo@famulari.name> writes:

Toggle quote (16 lines)
> On Fri, Oct 13, 2017 at 09:59:18AM -0400, Maxim Cournoyer wrote:
>> Although, I would argue that the current behavior is
>> non-intuitive. While true that the manual skim about how one can achieve
>> this, the reference to "The Store" is not helpful; it doesn't even
>> mention the "set-build-options" procedure. Also, leaving the command
>> line to plug directly into Guix's API from Guile is inconvenient at best.
>>
>> It seems to me that the current behavior of other options that affect
>> the guix-daemon operation are that user options override the
>> corresponding guix-daemon defaults; maybe that's what lead me and others
>> to think that --substitute-urls should attempt to do what the user
>> desires?
>
> Yeah, maybe it should be changed to be consistent with the behavior of
> the other options.

I don’t know. Substitute sources have to authorized before downloaded
substitutes are accepted by the daemon. This authorization happens as
the root user, as it constitutes a system-wide change.

When the daemon is run by the root user to disable substitutes
system-wide, maybe we should not let users override that decision, just
like we don’t let them override from what server binaries are to be
accepted.

I’m not convinced by the reasoning above, but I’d like to offer this
thought for consideration anyway.

Toggle quote (6 lines)
>> Otherwise, we could at least give advice on the output of a Guix command
>> when the user passed --substitute-urls when the guix-daemon substitutes
>> were disabled to make this clear(er).
>
> Agreed, we should at least do that.

Yes, this is a good idea.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC
L
L
Leo Famulari wrote on 14 Oct 2017 18:41
(name . Ricardo Wurmus)(address . rekado@elephly.net)
20171014164156.GA2074@jasmine.lan
On Sat, Oct 14, 2017 at 03:23:45PM +0200, Ricardo Wurmus wrote:
Toggle quote (4 lines)
> I don’t know. Substitute sources have to authorized before downloaded
> substitutes are accepted by the daemon. This authorization happens as
> the root user, as it constitutes a system-wide change.

I was thinking of situations where the subsitute signing key is
authorized, but substitutes are disabled system-wide.

I don't have a use case for this configuration but, to me, it doesn't
seem far-fetched for multi-user systems. Maybe the administrator is
willing to let users trust substitutes, but doesn't want to do it for
the privileged Guix installation.
-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEsFFZSPHn08G5gDigJkb6MLrKfwgFAlniPlEACgkQJkb6MLrK
fwj1nRAAqheeh8ofKsPo4T7ekPnyvFBA8ao/B3RI66ffHvzn/b2NtyaCMwus+MOX
Y/n0bwo/Xtls6jCWErQxwIDgaHzzVWKeOA+GkzKNqH01dX9UI74gVrO0mUeJqQgy
75EU2awFg5/qGgo1sMhegZOFfHAgRcyK2rbmvkCiXWIeZ9rRqgMGYI3gQ+b1ct0a
YxhWkuYgQGU6xSoieJ/GEqlgdtpbfnOpITETGM20EcxP2B2Vetg0F+DGOr3vl98x
gmT7roOkVJHaK2/bvzTPZ9D57qR8IESNyIyJrHX4o9AAar2iDHTJjlJkmQ323reZ
JFPJFwj9P3mJZNp1PrW4tLlxHyjl6++P+HTFQd5c5A3e1gn6AGQX97XN6hdTqDA2
6fzR8mh5q6eJkPp743cxvb2q6JrafzhCRhLHWI5u7aBnd13U9wFIGnlQ2gm30lo/
O5OnyVtvnoNktgtMvuXembt3K8eYH0qRrk3K5i8gjgy2K7kqmtkbTdGIm+ND277C
toiD1lS4phx4dyxfdBSkc7fiuyK3Iv4LRwiwnGZOeezXIUo/jc8WrxViDYd7OdPC
yq9/r6GKv55iKrucnBReg8SO7nBQHkB1qOnne0TguRz9IF3OIPaF7THsCPq15yav
xxwLy36d+sC19w1bl2mVPVvE/K+2gv5Omk4E63CVx4jjIfEaziY=
=B9in
-----END PGP SIGNATURE-----


?