Document how to interact with a remote daemon

  • Done
  • quality assurance status badge
Details
2 participants
  • Ludovic Courtès
  • Ricardo Wurmus
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
wishlist
L
L
Ludovic Courtès wrote on 20 Apr 2015 14:47
Interacting with a remote daemon
(address . bug-guix@gnu.org)(name . Ricardo Wurmus)(address . ricardo.wurmus@mdc-berlin.de)
87a8y3q84k.fsf@gnu.org

At some point I think it would make sense to enhance Guix such that
RPCs can be made over SSH, so that explicit logging on to a management
machine is no longer necessary.

We should see exactly how much support is needed.

Is ‘socat’ enough, as suggested in
That means there would be no authentication, but maybe that’s acceptable
in a cluster that is isolated on its own LAN anyway.

Option #2 would be to augment (guix store) and guix-daemon so that there
is built-in support for TCP sockets, which would be slightly more
convenient.

Option #3 would be to make (guix store) SSH-capable, with Guile-SSH.
But it sounds like SSH would incur relatively high latency when
connecting to the daemon.

There remains the problem that /var/guix/profiles/per-user/$USER would
have to be writable to this to be useful.

Thoughts?

Ludo’.
L
L
Ludovic Courtès wrote on 20 Apr 2015 15:19
control message for bug #20381
(address . control@debbugs.gnu.org)
878udnq6mn.fsf@gnu.org
severity 20381 wishlist
R
R
Ricardo Wurmus wrote on 10 Jul 2015 12:42
Interacting with a remote daemon
(address . 20381@debbugs.gnu.org)
idj7fq847ke.fsf@bimsb-sys02.mdc-berlin.net
I just tried the socat idea[1] with some success.

On the guix-builder host where guix-daemon is running and the NFS share
holding ‘/gnu’ (with $localstatedir set to ‘/gnu/var’) is mounted as
read-write I executed this:

/root/.guix-profile/bin/socat TCP4-LISTEN:9999 UNIX:/gnu/var/guix/daemon-socket/socket

On a cluster node where /gnu is mounted read-only I ran this:

socat UNIX-LISTEN:/home/rwurmus/foo TCP4:guix-builder:9999 &
export GUIX_DAEMON_SOCKET=$HOME/foo

At this point I could use

guix build hello
guix environment hello

which is really great!

To make the “guix” command available on cluster nodes I just installed
it into my default user profile as ‘~/.guix-profile/bin/guix’. The
problem with this is that profile commands don’t work as the regular
“guix” package as installed with $localstatedir set to ‘/var’. This can
be fixed, of course, (e.g. by creating a slightly different “guix”
package with the appropriate configure flags set) but it’s still a minor
annoyance. It would be great if $localstatedir could be overridden at
runtime or if it could default to whatever the daemon uses.

This would probably work fine if I limited the socket forwarding to just
the cluster nodes, because only there user ids are guaranteed to be
correct (not on workstations). On workstations that are not centrally
managed this will not work, as the user ids could be arbitrary and it
would thus allow anyone to change anyone else’s profile by creating a
local account with the appropriate uid.

I prefer the socat approach over just running “guix” remotely through an
SSH connection, because with socat the “guix” command can actually be
used to spawn a new local shell with “guix environment”, which is very
useful. I don’t think this would work if “guix” were just run
remotely. (Please correct me if I’m wrong about this.)

~~ Ricardo
L
L
Ludovic Courtès wrote on 10 Jul 2015 19:48
(name . Ricardo Wurmus)(address . ricardo.wurmus@mdc-berlin.de)(address . 20381@debbugs.gnu.org)
87h9pbaoot.fsf@gnu.org
Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:

Toggle quote (20 lines)
> I just tried the socat idea[1] with some success.
>
> On the guix-builder host where guix-daemon is running and the NFS share
> holding ‘/gnu’ (with $localstatedir set to ‘/gnu/var’) is mounted as
> read-write I executed this:
>
> /root/.guix-profile/bin/socat TCP4-LISTEN:9999 UNIX:/gnu/var/guix/daemon-socket/socket
>
> On a cluster node where /gnu is mounted read-only I ran this:
>
> socat UNIX-LISTEN:/home/rwurmus/foo TCP4:guix-builder:9999 &
> export GUIX_DAEMON_SOCKET=$HOME/foo
>
> At this point I could use
>
> guix build hello
> guix environment hello
>
> which is really great!

Excellent, thanks for testing!

Toggle quote (8 lines)
> To make the “guix” command available on cluster nodes I just installed
> it into my default user profile as ‘~/.guix-profile/bin/guix’. The
> problem with this is that profile commands don’t work as the regular
> “guix” package as installed with $localstatedir set to ‘/var’. This can
> be fixed, of course, (e.g. by creating a slightly different “guix”
> package with the appropriate configure flags set) but it’s still a minor
> annoyance.

What about installing Guix in /gnu/bin (say) and sharing it over NFS?

I would avoid installing Guix in a profile, because if things go wrong,
you may find yourself unable to do anything. In practice, you can
always roll-back by hand (it’s simply a matter of switching the
profiles/per-user/$USER symlink), but still.

Toggle quote (3 lines)
> It would be great if $localstatedir could be overridden at runtime or
> if it could default to whatever the daemon uses.

Actually it can be overridden via the intentionally-undocumented
NIX_STATE_DIR environment variable (see (guix config).)

Toggle quote (7 lines)
> This would probably work fine if I limited the socket forwarding to just
> the cluster nodes, because only there user ids are guaranteed to be
> correct (not on workstations). On workstations that are not centrally
> managed this will not work, as the user ids could be arbitrary and it
> would thus allow anyone to change anyone else’s profile by creating a
> local account with the appropriate uid.

The only problem would be with ‘guix package’, which you haven’t
mentioned yet. :-) For ‘guix package’ to work,
/gnu/var/guix/profiles/per-user must be shared read-write (over NFS)
with correct UID mapping.

Toggle quote (6 lines)
> I prefer the socat approach over just running “guix” remotely through an
> SSH connection, because with socat the “guix” command can actually be
> used to spawn a new local shell with “guix environment”, which is very
> useful. I don’t think this would work if “guix” were just run
> remotely. (Please correct me if I’m wrong about this.)

Indeed, that would only allow you to spawn a shell on the machine where
the ‘guix’ command is executed (which, in your case, is not a compute
node AIUI.)

I think we should have a “Cluster Setup” section in the manual to
explain all this. Would you like to give it a try?

Ludo’.
R
R
Ricardo Wurmus wrote on 10 Jul 2015 22:24
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 20381@debbugs.gnu.org)
87k2u7iwwt.fsf@mdc-berlin.de
Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (13 lines)
> What about installing Guix in /gnu/bin (say) and sharing it over NFS?
>
> I would avoid installing Guix in a profile, because if things go wrong,
> you may find yourself unable to do anything. In practice, you can
> always roll-back by hand (it’s simply a matter of switching the
> profiles/per-user/$USER symlink), but still.
>
>> It would be great if $localstatedir could be overridden at runtime or
>> if it could default to whatever the daemon uses.
>
> Actually it can be overridden via the intentionally-undocumented
> NIX_STATE_DIR environment variable (see (guix config).)

Oh, nice. Installing Guix somewhere into /gnu is actually a pretty good
idea. I’ll try that and play with NIX_STATE_DIR as well.

Toggle quote (12 lines)
>> This would probably work fine if I limited the socket forwarding to just
>> the cluster nodes, because only there user ids are guaranteed to be
>> correct (not on workstations). On workstations that are not centrally
>> managed this will not work, as the user ids could be arbitrary and it
>> would thus allow anyone to change anyone else’s profile by creating a
>> local account with the appropriate uid.
>
> The only problem would be with ‘guix package’, which you haven’t
> mentioned yet. :-) For ‘guix package’ to work,
> /gnu/var/guix/profiles/per-user must be shared read-write (over NFS)
> with correct UID mapping.

Correct. I haven’t tried ‘guix package’ at all because I just assumed
it would work.

Toggle quote (3 lines)
> I think we should have a “Cluster Setup” section in the manual to
> explain all this. Would you like to give it a try?

Sounds like a good idea. I can give it a try but I’ll be on vacation
for a while and can only get around to writing in a couple of weeks.
But I think I’m a good candidate for drafting this section, given that
I’ve got a cluster to play with :)

Thanks for your helpful recommendations!

~~ Ricardo
R
R
Ricardo Wurmus wrote on 24 Mar 2016 16:23
Interacting with a remote daemon
(address . control@debbugs.gnu.org)
idjfuvfrjao.fsf@bimsb-sys02.mdc-berlin.net
retitle 20381 Document how to interact with a remote daemon
owner !
L
L
Ludovic Courtès wrote on 25 Apr 2017 12:24
(address . 20381-done@debbugs.gnu.org)(name . Ricardo Wurmus)(address . ricardo.wurmus@mdc-berlin.de)
8737cwdbt5.fsf@gnu.org
Hello!

ludo@gnu.org (Ludovic Courtès) skribis:

Toggle quote (21 lines)
>
> At some point I think it would make sense to enhance Guix such that
> RPCs can be made over SSH, so that explicit logging on to a management
> machine is no longer necessary.
>
> We should see exactly how much support is needed.
>
> Is ‘socat’ enough, as suggested in
> <https://lists.gnu.org/archive/html/guix-devel/2015-02/msg00680.html>?
> That means there would be no authentication, but maybe that’s acceptable
> in a cluster that is isolated on its own LAN anyway.
>
> Option #2 would be to augment (guix store) and guix-daemon so that there
> is built-in support for TCP sockets, which would be slightly more
> convenient.
>
> Option #3 would be to make (guix store) SSH-capable, with Guile-SSH.
> But it sounds like SSH would incur relatively high latency when
> connecting to the daemon.

I think this is fixed by this series of commits:

285f63e80 * store: Support 'ssh://' URIs in 'GUIX_DAEMON_SOCKET'.
e53783372 * ssh: Decompose 'connect-to-remote-daemon'.
615c5298f * ssh: Move 'open-ssh-session' to (guix ssh).
ba97e454b * offload: Avoid using '_' as a 'match' pattern.
3dff90ce3 * store: Add support for remote connections via 'guix://' URIs.
3b5cd17a9 * store: Add 'system-error-to-connection-error' macro.
1397b422e * store: 'GUIX_DAEMON_SOCKET' can now be a URI.

with the caveats given here:


Closing!

Ludo’.
Closed
?