Bashisms when building 0.11.0

  • Done
  • quality assurance status badge
Details
6 participants
  • elaexuotee
  • Leo Famulari
  • Maxim Cournoyer
  • Tobias Geerinckx-Rice
  • pelzflorian (Florian Pelz)
  • Vincent Legoll
Owner
unassigned
Submitted by
pelzflorian (Florian Pelz)
Severity
normal
P
P
pelzflorian (Florian Pelz) wrote on 24 Dec 2016 08:26
(address . bug-guix@gnu.org)
c3eb8da7-acf5-f661-48d1-fd70d322560d@pelzflorian.de
Hello,

Guix' testsuite uses the non-POSIX bashism `type -P guile` in
tests/guix-packages.sh which causes a test failure when using
Dash to provide /bin/sh instead of Bash. Additionally gnu/local.mk uses
brace expansion which is not part of POSIX and does not work on Dash.

When building Guix on Parabola GNU/Linux-libre modified to use Dash as
/bin/sh,[1] this causes the Guix build to fail. For building, the guix
0.11.0 package from AUR[2] was used. The attached “patch” allows for a
successful build but is not what we want.

Guix should probably not use bashisms in its Makefile includes and
either not use bashisms in its test suite or make sure the test suite is
executed explicitly with Bash and not /bin/sh.

Regards,
Florian Pelz

diff -aur guix-0.11.0.pristine/gnu/local.mk guix-0.11.0.new/gnu/local.mk
--- guix-0.11.0.pristine/gnu/local.mk 2016-12-23 16:24:51.501094452 +0100
+++ guix-0.11.0.new/gnu/local.mk 2016-12-23 17:26:55.651112436 +0100
@@ -903,7 +903,10 @@
# Those files must remain executable, so they remain executable once
# imported into the store.
set-bootstrap-executable-permissions:
- chmod +x $(DESTDIR)$(bootstrapdir)/*/{bash,mkdir,tar,xz}
+ chmod +x $(DESTDIR)$(bootstrapdir)/*/bash
+ chmod +x $(DESTDIR)$(bootstrapdir)/*/mkdir
+ chmod +x $(DESTDIR)$(bootstrapdir)/*/tar
+ chmod +x $(DESTDIR)$(bootstrapdir)/*/xz
DISTCLEANFILES = \
$(nodist_bootstrap_x86_64_linux_DATA) \
diff -aur guix-0.11.0.pristine/Makefile.am guix-0.11.0.new/Makefile.am
--- guix-0.11.0.pristine/Makefile.am 2016-12-23 16:24:49.007702057 +0100
+++ guix-0.11.0.new/Makefile.am 2016-12-23 16:25:39.372217348 +0100
@@ -275,7 +275,6 @@
tests/guix-download.sh \
tests/guix-gc.sh \
tests/guix-hash.sh \
- tests/guix-package.sh \
tests/guix-package-net.sh \
tests/guix-system.sh \
tests/guix-archive.sh \
L
L
Leo Famulari wrote on 13 Feb 2019 02:46
(name . pelzflorian (Florian Pelz))(address . pelzflorian@pelzflorian.de)(address . 25258@debbugs.gnu.org)
20190213014630.GA27743@jasmine.lan
On Sat, Dec 24, 2016 at 08:26:27AM +0100, pelzflorian (Florian Pelz) wrote:
Toggle quote (14 lines)
> Guix' testsuite uses the non-POSIX bashism `type -P guile` in
> tests/guix-packages.sh which causes a test failure when using
> Dash to provide /bin/sh instead of Bash. Additionally gnu/local.mk uses
> brace expansion which is not part of POSIX and does not work on Dash.
>
> When building Guix on Parabola GNU/Linux-libre modified to use Dash as
> /bin/sh,[1] this causes the Guix build to fail. For building, the guix
> 0.11.0 package from AUR[2] was used. The attached “patch” allows for a
> successful build but is not what we want.
>
> Guix should probably not use bashisms in its Makefile includes and
> either not use bashisms in its test suite or make sure the test suite is
> executed explicitly with Bash and not /bin/sh.

About two years have passed on this bug report without comment. Is it a
"wontfix"?
-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEsFFZSPHn08G5gDigJkb6MLrKfwgFAlxjdvYACgkQJkb6MLrK
fwhVjA/9HYwM0S71t09ZpDz++mEUNiirbBpUsCBCnhoRkBv9mPg8h+U6/HLueN4W
qO54jXMM38StL9ql+b/doCJLtcEs97N2nHU/xHCFPhKventP25PtFvWLlb/wrK7Q
Ip7s1OOkHS/cFBYH1rnhlr0OWh/snENHuI99qXazYw/uH37kflCYzFp96Mb7Cs9k
g10aWi4PPEGww+8Q/Zb5T0zlyW0YeZGbLWCDY1ouV8+DTKhw6AfUHagd36u3CNyc
TiJUIfZXsUIte+5suuXnDOX4zvMzNBbMXWOKIOZHjytmMt4pPxwY8OmVyzb/nkHW
nXlteJGjvPk6ZJS7WcAj3QrZKVp3iST2t/+njV9yU75Us9wzhapmgLl3FH4v3qDD
Lg1qU2R3QocwM1G0Tu9bm7k5wlfd78k/mTLGzd01ng5VWsYs81fYHzLSlx43fEZW
YqPGOYHdQK6NzMT9Y7L0feSuuxj8edyQd7FmT0XlWPHnAQuHqEqohbbDdxiun5SS
/R4/f8vKlu9GVDO6mdeVlocbWMFvglr0braNz2So4r1ybDk4g40emYgyRg2eWRpX
ShTAA9ksFHbKAfzqzYx1SzpSRBUHIsnhgelltyrEKWg0SuVt2Hu5dtt6IiL+7w/c
7J6aY1lDfjg0bVko8xbSPM5tNHqfPigP7O+4648r0I+BPpNl+fI=
=0i4O
-----END PGP SIGNATURE-----


L
L
Leo Famulari wrote on 22 Mar 2020 21:28
(name . pelzflorian (Florian Pelz))(address . pelzflorian@pelzflorian.de)(address . 25258-done@debbugs.gnu.org)
20200322202832.GA18995@jasmine.lan
On Tue, Feb 12, 2019 at 08:46:30PM -0500, Leo Famulari wrote:
Toggle quote (3 lines)
> About two years have passed on this bug report without comment. Is it a
> "wontfix"?

Based on the lack of interest I've closed this bug.
Closed
E
E
elaexuotee wrote on 16 May 2020 06:50
Unarchive 25258
(address . control@debbugs.gnu.org)
2130W7IZNZBIV.3O4EVOCAUP4JS@wilsonb.com
unarchive 25258
E
E
elaexuotee wrote on 16 May 2020 08:33
Bashisms in make rules
(address . 25258@debbugs.gnu.org)
3QC37OOAPFHT8.3ELLIYWH54G6D@wilsonb.com
Resurrecting this, since it cause me significant grief, and is a simple
workaround which I believe is worth putting in the documentation. First, here
is a short summary of the problem:

Currently, some of our make rules contain bashisms. As of commit 408ae72c, this
ends up generating broken texinfo translations under doc/*.texi, eventually
failing with "@ref reference to nonexistent node..." errors.

From a pristine repository (e.g. after running `git clean -xfd'), running make
shows a bunch of errors from the broken rules, but since they happen to be
multi-line scripts, the errors fail to propogate up to make. Thus make leaves
around the broken texinfo files. The real kicker is that subsequent runs pick
up the broken files and make fails just as above, but without encountering or
showing the errors from the broken rules. This makes tracking down the error
significantly more painful and non-obvious.

Anyway, the source of the problem is a combination of our use of bashisms and
autoconf trying really hard to use the least-common-denominator shell.
Autoconf's configure script sets make's SHELL by first looking for /bin/sh and
only finding sh on PATH as a last resort. Thus, even when running under `guix
environment --pure guix' make will run with /bin/sh instead of
$GUIX_ENVIRONMENT/bin/sh.

On a Guix System this ends up running make with the sh of the system profile
which, at the moment, resolves to bash. However, on a foreign distribution,
/bin/sh often resolves to a non-bash shell; in particular, it is common these
days for it to be a symlink to dash. Hence, lots of pain.

Once we know the source of the error, the fix is simple. We can explicitly tell
make to run it's rules with bash:

$ make SHELL=$(command -v bash)

In a guix environment, this should pick up the bash in $GUIX_ENVIRONMENT.
Better yet, we can do this once, by telling *configure* what shell to use:

$ CONFIG_SHELL=$(command -v bash) ./configure

and all subsequent runs of make will Just Work from within the environment.

Does this make sense? Am I missing something obvious? If not, would it make
sense to include the above ./configure invocation in our documentation?

Ideally, we could modify Makefile.am or configure.ac in some way to
automatically fix the problem; however, if we want to support build exterior to
a guix build environment, I suspect the "most correct" course of action is to
simply remove our bashisms.

That said, we could tell automake to *not* explicity set make's SHELL by a
simple one-liner in `configure.ac':

AM_SUBST_NOTMAKE([SHELL])

This will cause make to use it's baked-in default instead of the one autoconf
detects. Luckily, for guix builds make correctly references the profile's sh:

$ strings $GUIX_ENVIRONMENT/bin/make | grep bin/sh
/gnu/store/<hash>-bash-<version>/bin/sh

and non-guix make is likely to point to something relevant for that
distribution anyway. That said, I am sure automake and autoconf set make's
SHELL for a reason, so perhaps this solution is too heavy of a hammer.
Thoughts?

Anyway, this turned into a lot longer of an email than I anticipated. Thanks
for taking the time to read through it.
Attachment: signature.asc
V
V
Vincent Legoll wrote on 17 May 2020 18:42
(address . 25258@debbugs.gnu.org)
4c16cf7b-0a55-b7f2-4c49-a34ce1f815a1@gmail.com
Hello,

Having done a few bashismectomies lately (in guix-install.sh), I
can sympathetize with you.

I don't know guix team's stance on that kind ow work (I still have not
polished my work enough for review). But it looks like you stumbled on
some painful items, that ought to be resolved one way or the other.

It would be nice to hear that those are actionable items that will
receive positive feedback before doing substantial work. (Looks like
I should have asked that earlier, before falling into the rabbit
hole...)

I personally think we should try to support using guix as widely as
possible/practical on foreign distros.

So, WDGCT (What Do Guix Core Think) ?

Thanks

--
Vincent Legoll
T
T
Tobias Geerinckx-Rice wrote on 17 May 2020 19:34
Re: bug#25258: Bashisms in make rules
(name . Vincent Legoll)(address . vincent.legoll@gmail.com)(address . 25258@debbugs.gnu.org)
87y2pqtqh0.fsf@nckx
Vincent,

Vincent Legoll ???
Toggle quote (7 lines)
> I don't know guix team's stance on that kind ow work (I still
> have not
> polished my work enough for review). But it looks like you
> stumbled on
> some painful items, that ought to be resolved one way or the
> other.

The only reason for Guix to have any shell code at all is to run
in places where Guile isn't (yet) available. Therefore it should
be portable between GNU systems, or it might as well be Guile. I
don't believe GNU implies bash.

Toggle quote (7 lines)
> It would be nice to hear that those are actionable items that
> will
> receive positive feedback before doing substantial work. (Looks
> like
> I should have asked that earlier, before falling into the rabbit
> hole...)

Big upz from me. Thanks for being proactive :-)

The challenge is keeping new bashisms out, so sudo please try to
include some automated tests. Maybe you can use shellcheck; our
test suite could use a GHC dependency.

Kind regards,

T G-R
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQT12iAyS4c9C3o4dnINsP+IT1VteQUCXsF1uwAKCRANsP+IT1Vt
eVeaAQDd2OOdrR3PboPIdEh/rUpnvj2Lckkggkg0NxWp5CdPFAD/d80gsSrKNttQ
AwZUWDD/jLSaEDlsvI8HZzPj5GMIZgk=
=3blj
-----END PGP SIGNATURE-----

M
M
Maxim Cournoyer wrote on 11 Jul 2022 15:49
control message for bug #25258
(address . control@debbugs.gnu.org)
87fsj7ep2y.fsf@gmail.com
retitle 25258 Bashisms in test suite cause 3 tests to fail when using a POSIX Bourne shell
quit
?