Hi Maxim, Maxim Cournoyer skribis: > ludo@gnu.org (Ludovic Courtès) writes: [...] >> But wait! “guix build nss-certs --check -K” fails, and the diff is: >> >> $ LANGUAGE= diff -ur /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2{,-check} >> Only in /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2-check/etc/ssl/certs: AC_Raíz_Certicámara_S.A.:2.15.7.126.82.147.123.224.21.227.87.240.105.140.203.236.12.pem >> Only in /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2/etc/ssl/certs: AC_Ra?z_Certic?mara_S.A.:2.15.7.126.82.147.123.224.21.227.87.240.105.140.203.236.12.pem >> diff -ur /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2/etc/ssl/certs/ae8153b9.0 /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2-check/etc/ssl/certs/ae8153b9.0 >> --- /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2/etc/ssl/certs/ae8153b9.0 1970-01-01 01:00:01.000000000 +0100 >> +++ /gnu/store/3ql0vilc0zv6ra42ghi04787vrg6bb71-nss-certs-3.30.2-check/etc/ssl/certs/ae8153b9.0 1970-01-01 01:00:01.000000000 +0100 >> @@ -3,10 +3,10 @@ >> # distrust= >> # openssl-trust=codeSigning emailProtection serverAuth >> -----BEGIN CERTIFICATE----- >> -MIIHyTCCBbGgAwIBAgIBATANBgkqhkiG9w0BAQUFADB9MQswCQYDVQQGEwJJTDEW >> +MIIHhzCCBW+gAwIBAgIBLTANBgkqhkiG9w0BAQsFADB9MQswCQYDVQQGEwJJTDEW > > Can this be explained by locale alone? That is troubling. Yes it’s troubling, it deserves more investigation. >> There are two ways to create nars. One is via the ‘export-paths’ RPC >> (implemented in the daemon in C++), which does not interpret file names >> and thus leaves them untouched. The other one is via ‘write-file’ from >> (guix serialization), which is written in Scheme and thus converts file >> names from locale encoding (specifically, ‘scandir’ does that.) >> >> ‘guix publish’ uses the latter, so ‘guix publish’ is sensitive to locale >> settings, which is pretty bad. >> >> Guile currently does not allow us to specify whether/how file names >> should be decoded, but possible solutions have been discussed for 2.2. >> >> In the meantime, solutions are: >> >> 1. To run ‘guix publish’ in a UTF-8 locale, which apparently was not >> the case. > > I'm surprised by that. Wouldn't a utf8 locale be the default? Users are free to choose their favorite locale. Also, on a foreign distro where locales are not properly set up, you end up in the C locale with US-ASCII encoding (as was the case here). >> 2. Add to (guix build syscalls) a separate locale-independent >> ‘scandir’ implementation and use that. > > If the general solution is to fix it in Guile, the workaround proposed > in 1. seems preferable. I implemented ‘scandir/utf-8’ and used that in ‘write-file’ (patches attached). Unfortunately that’s not enough since libguile procedures like ‘open-file’ still do locale-dependent conversion, so we’d need to duplicate those as well, which is not great. But on second thought, I think the problem is not in the ‘write-file’ call that ‘guix publish’ makes: if it were, ‘scandir’ would return bogus file names (with question marks), but then trying to open them would fail, so ‘write-file’ wouldn’t produce a bogus nar. So I think the culprit is ‘restore-file-set’ (used in ‘guix offload’ when retrieving store items from a build machine): this one reads file names as UTF-8, via ‘read-store-path’, but then when it tries to create those files using Guile’s primitives, their names can be be converted, possibly with those question marks popping up. Here ‘restore-file-set’ can’t notice that Guile changed the file names behind its back. The short-term fix is to ensure guix-daemon itself runs in a UTF-8 locale. :-/ I’ve restarted guix-daemon and ‘guix publish’ in a UTF-8 locale on hydra.gnu.org. Thanks, Ludo’.