[PATCH] build-system/asdf: Add option to compress programs.

  • Done
  • quality assurance status badge
Details
3 participants
  • Katherine Cox-Buday
  • Ludovic Courtès
  • Pierre Neidhardt
Owner
unassigned
Submitted by
Pierre Neidhardt
Severity
normal
P
P
Pierre Neidhardt wrote on 2 Sep 2019 15:50
(address . guix-patches@gnu.org)
20190902135020.14473-1-mail@ambrevar.xyz
* guix/build/lisp-utils.scm (build-program): Add `compress?' key argument.
(generate-executable-for-system): Same.
(generate-executable): Same.
---
guix/build/lisp-utils.scm | 14 ++++++++++++--
1 file changed, 12 insertions(+), 2 deletions(-)

Toggle diff (60 lines)
diff --git a/guix/build/lisp-utils.scm b/guix/build/lisp-utils.scm
index 97bc6197a3..c7a589c902 100644
--- a/guix/build/lisp-utils.scm
+++ b/guix/build/lisp-utils.scm
@@ -220,12 +220,19 @@ Also load TEST-ASD-FILE if necessary."
"Return a lisp keyword for the concatenation of STRINGS."
(string->symbol (apply string-append ":" strings)))
-(define (generate-executable-for-system type system)
+(define* (generate-executable-for-system type system #:key compress?)
"Use LISP to generate an executable, whose TYPE can be 'asdf:image-op or
'asdf:program-op. The latter will always be standalone. Depends on having
created a \"SYSTEM-exec\" system which contains the entry program."
(lisp-eval-program
`((require :asdf)
+ ;; Only SBCL supports compression as of 2019-09-02.
+ ,(if (and compress? (string=? (%lisp-type) "sbcl"))
+ '(defmethod asdf:perform ((o asdf:image-op) (c asdf:system))
+ (uiop:dump-image (asdf:output-file o c)
+ :executable t
+ :compression t))
+ '())
(asdf:operate ',type ,(string-append system "-exec")))))
(define (generate-executable-wrapper-system system dependencies)
@@ -339,6 +346,7 @@ which are not nested."
(dependency-prefixes (list (library-output outputs)))
(dependencies (list (basename program)))
entry-program
+ compress?
#:allow-other-keys)
"Generate an executable program containing all DEPENDENCIES, and which will
execute ENTRY-PROGRAM. The result is placed in PROGRAM. When executed, it
@@ -350,6 +358,7 @@ retained."
#:dependencies dependencies
#:dependency-prefixes dependency-prefixes
#:entry-program entry-program
+ #:compress? compress?
#:type 'asdf:program-op)
(let* ((name (basename program))
(bin-directory (dirname program)))
@@ -382,6 +391,7 @@ DEPENDENCY-PREFIXES to ensure references to those libraries are retained."
dependency-prefixes
entry-program
type
+ compress?
#:allow-other-keys)
"Generate an executable by using asdf operation TYPE, containing whithin the
image all DEPENDENCIES, and running ENTRY-PROGRAM in the case of an
@@ -405,7 +415,7 @@ references to those libraries are retained."
`(((,bin-directory :**/ :*.*.*)
(,bin-directory :**/ :*.*.*)))))))
- (generate-executable-for-system type name)
+ (generate-executable-for-system type name #:compress? compress?)
(let* ((after-store-prefix-index
(string-index out-file #\/
--
2.23.0
P
P
Pierre Neidhardt wrote on 2 Sep 2019 15:54
Re: bug#37269: Acknowledgement ([PATCH] build-system/asdf: Add option to compress programs.)
87pnki3is4.fsf@ambrevar.xyz
This patch could be used by StumpWM and Next browser to compress the
binary.
For Next, this compresses the binary from ~100 MiB to ~23 MiB. There is
no real drawback, the startup time is still well under 100 ms.

Andy, Katherine, thoughts?

--
Pierre Neidhardt
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAl1tHysACgkQm9z0l6S7
zH8OYAf/aptgrtKTfA5ibwMwDP9WSaL0XVvpY6ngA8xuRqKyUZWluWFLv1MA/8+E
ap6Sjitvwtr2MGt3L8ZXQMoHqHIqkrZUy8tAwjCTY2c5KOe2LT87DY5p+qDiYSfY
ZxumZheAVy+okmdr70jVaDFjvzP6wqUXDAWJNeBEyZlLSkTtCsZt2ihDSD3m4Zak
lCEgEGFuQM8R55nxX2tPrfjnPJuZ6feZgOnd1Y2ssc8EYQDy6NMmvuajmxQUX1Oc
//m4AjHrn1m1zOCpXIjWiM0dpj3jwSu7fAL/E5qUL9itROtpaoVnwlHYO5K7hmWa
91Wzu67s/CtxmDqNYTkpASPfauHl8w==
=CNMM
-----END PGP SIGNATURE-----

K
K
Katherine Cox-Buday wrote on 3 Sep 2019 16:12
(name . Pierre Neidhardt)(address . mail@ambrevar.xyz)
874l1t8o4s.fsf@gmail.com
Pierre Neidhardt <mail@ambrevar.xyz> writes:

Toggle quote (5 lines)
> This patch could be used by StumpWM and Next browser to compress the
> binary. For Next, this compresses the binary from ~100 MiB to ~23 MiB.
> There is no real drawback, the startup time is still well under
> 100 ms.

This is a great idea and something I lost when I moved from
self-compiling SBCL to Guix's version!

I am also not aware of any drawbacks, and I love that you implemented
this as an option in the build-system. Should we default it to true?

Also, as an aside, I am tracking Next; thank you for this project!

--
Katherine
P
P
Pierre Neidhardt wrote on 3 Sep 2019 16:56
(name . Katherine Cox-Buday)(address . cox.katherine.e@gmail.com)
87pnkhxwb4.fsf@ambrevar.xyz
Hi,

Toggle quote (3 lines)
> I am also not aware of any drawbacks, and I love that you implemented
> this as an option in the build-system. Should we default it to true?

SBCL does not do it by default, so maybe we should respect the default
behaviour of the compiler instead.

I'll go ahead and apply the patch then. Update SBCL to 1.5.6 and use
CLISP or ECL to build it instead of CCL which is not bootstrappable.

Toggle quote (2 lines)
> Also, as an aside, I am tracking Next; thank you for this project!

Thanks! Lots of good stuff will come this month!
(1.3.1 released today!)

Cheers!

--
Pierre Neidhardt
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAl1ufy8ACgkQm9z0l6S7
zH8Sygf+IUDCj9Q9ekaw8s4dExvoMBWVzFB0LLo8ynWBXam1YL/IwU+1axD4crNz
amlYDTfncKPMuYxINPW2EbduGAW+ZiFro+TOvS/efL/5fJVjMAAb5fBVsrS/FXmN
L0NXpzhBPMMLWwcocEk07SVVaQQn4w6C4kroK2VEjJlSk+zTYmmxsCU8SHBR/oEG
TUSt9qyYvNou2BLe+HXBlTVCSz3Q7PphnMjgzEbv1a9hjDJM9LZA247EgRm7dgv3
e1gdhVmzxCiLRLje2b7fXuW5tkgAPZTjf4qXfQo8oYUPUbZmhx1UjSCnbu/6t3dn
zj7c/K0IoNS2uMPt8aFICsjKLOAQCw==
=zhn+
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 4 Sep 2019 14:07
Re: [bug#37269] Acknowledgement ([PATCH] build-system/asdf: Add option to compress programs.)
(name . Katherine Cox-Buday)(address . cox.katherine.e@gmail.com)
874l1s6z9r.fsf@gnu.org
Hello!

Katherine Cox-Buday <cox.katherine.e@gmail.com> skribis:

Toggle quote (13 lines)
> Pierre Neidhardt <mail@ambrevar.xyz> writes:
>
>> This patch could be used by StumpWM and Next browser to compress the
>> binary. For Next, this compresses the binary from ~100 MiB to ~23 MiB.
>> There is no real drawback, the startup time is still well under
>> 100 ms.
>
> This is a great idea and something I lost when I moved from
> self-compiling SBCL to Guix's version!
>
> I am also not aware of any drawbacks, and I love that you implemented
> this as an option in the build-system. Should we default it to true?

Compressed files are opaque to the garbage collector scanner and to
grafting, so we could end up in situations like:


(Which would be nice to address, BTW! :-))

Thus, I think we’ll have keep compression turned off.

Ludo’.
P
P
Pierre Neidhardt wrote on 4 Sep 2019 14:54
87sgpcdxxt.fsf@ambrevar.xyz
Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (9 lines)
> Compressed files are opaque to the garbage collector scanner and to
> grafting, so we could end up in situations like:
>
> https://issues.guix.gnu.org/issue/33848
>
> (Which would be nice to address, BTW! :-))
>
> Thus, I think we’ll have keep compression turned off.

Actually, in this case it doesn't. I tried with Next and the resulting
package keeps all references to the Common Lisp libraries.
I believe that SBCL only compresses the image, not the sources (which are
embedded as well).

Anyways, I won't turn it off for now.

--
Pierre Neidhardt
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAl1vs+4ACgkQm9z0l6S7
zH8mtQf/SLEFUi3bo2K7iABnQc5gC3VI4rTvcIratga1n/xbgrlExv/82s5u2yG/
bb8Y91qt//+Pm1UTIQao9ACje++lCTVf1VYxMuVakm8eIt9u343MigjBpAJbGJvJ
Rdo8KbT46q1HYQJWDpOUfN36vMIcWh7X7dqpyfNmayE1o6zT/zAB3StiIIcjzeh3
N4pilhxtUunNV4NT7c7pqJNtn5RD6pRnge4VIG2XZjrf3p89qIloU8f5hipAzd8m
utYlaWAhL8ttmWDIU7tpRVkdXkh9ehnZkkTd4U9wu2LGkEngK0giB6YZooTBmlHW
0F3cfEFBgk5xfovAJ5aU0nm9nGM0cQ==
=W6LV
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 5 Sep 2019 10:33
(name . Pierre Neidhardt)(address . mail@ambrevar.xyz)
87lfv3i1l7.fsf@gnu.org
Pierre Neidhardt <mail@ambrevar.xyz> skribis:

Toggle quote (16 lines)
> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Compressed files are opaque to the garbage collector scanner and to
>> grafting, so we could end up in situations like:
>>
>> https://issues.guix.gnu.org/issue/33848
>>
>> (Which would be nice to address, BTW! :-))
>>
>> Thus, I think we’ll have keep compression turned off.
>
> Actually, in this case it doesn't. I tried with Next and the resulting
> package keeps all references to the Common Lisp libraries.
> I believe that SBCL only compresses the image, not the sources (which are
> embedded as well).

OK. We’d need to see exactly what it does if we decide to enable
it—better safe than sorry!

Thanks,
Ludo’.
P
P
Pierre Neidhardt wrote on 5 Sep 2019 15:26
control message for bug #37269
(address . control@debbugs.gnu.org)
87y2z2an70.fsf@ambrevar.xyz
close 37269
quit
--
Pierre Neidhardt
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAl1xDRMACgkQm9z0l6S7
zH/AuQgAkq8+SPS/6gQs4wOPVwiKBJXDP8iGeVSO+lrKQRFlU2RvmNrApqZ94vc7
NbfUox5h+rZdExCFEJuB4a5R8ccA0XZNXdo0lkjILVDSi1nCEJQWskRXkMmmuZUc
SAeng2E9x/JqWQMMYlbji10IILsTgUEzylQWU6PM7bn0HKvmJctVvhWZNORh2Mc6
8RXVlnO4vCdy70T76YOk9e5v/Ayfqu7FpW4N6GTg16cUVSFge9FuORjZj/kv5OWT
QskTrQbL/AlBs9H+p5F/zyjQHVYBKug4JZ1qj7NeHgbZJYZoeFPwjTDoh356+kr9
nyxSO4no6f0tYqxHGG7PHkdzRa+4ug==
=yNoO
-----END PGP SIGNATURE-----

?