[PATCH 0/6] Allow for a cryptographic hash function migration

Done

Details

3 participants

Leo Famulari
Ludovic Courtès
Marius Bakke

Owner: unassigned

Submitted by: Ludovic Courtès

Severity: normal

Ludovic Courtès wrote on 18 May 2020 23:31

Recipients:(address . guix-patches@gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:20200518213116.23978-1-ludo@gnu.org

Hello Guix!

This is a first stab at preparing for a possible migration to

hash algorithms other than SHA256 (there’s no rush AFAIK, but it’s

good to be prepared). The first bits just improve tests and

adjust the tools, which doesn’t hurt.

The last bit adds a ‘sha512’ field to <origin>, which we could

eventually provide instead of or in addition to ‘sha256’, possibly

as a base64-encoded string so that it’s not too large.

Another option would be to create a <hash> data type that specifies

its algorithm and its value. We’d replace the ‘sha256’ field with

a ‘hash’ field of that type (in a backward-compatible way). Thinking

about it, this is perhaps the better option.

Other bits that would need to be adjusted include importer, updaters,

and ‘guix publish’.

Thoughts?

Ludo’.

Ludovic Courtès (6):

tests: Test 'add-to-store' with several hash algorithms.

tests: Test fixed-output derivations with several hash algorithms.

guix hash, guix download: Add '--hash'.

guix hash, guix download: Support base64 format.

packages: Add 'sha512' optional field to <origin>.

packages: Add 'base64' macro.

doc/guix.texi | 25 +++++++++++++++--

guix/packages.scm | 56 +++++++++++++++++++++++----------------

guix/scripts/download.scm | 26 +++++++++++++-----

guix/scripts/hash.scm | 35 +++++++++++++++++-------

tests/derivations.scm | 39 ++++++++++++++++-----------

tests/guix-hash.sh | 7 ++++-

tests/packages.scm | 26 ++++++++++++++++++

tests/store.scm | 12 +++++++++

8 files changed, 170 insertions(+), 56 deletions(-)

2.26.2

Ludovic Courtès wrote on 18 May 2020 23:32

[PATCH 1/6] tests: Test 'add-to-store' with several hash algorithms.

Recipients:(address . 41382@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:20200518213244.24165-1-ludo@gnu.org

* tests/store.scm ("add-to-store"): New test.

---

tests/store.scm | 12 ++++++++++++

1 file changed, 12 insertions(+)

Toggle diff (25 lines)diff --git a/tests/store.scm b/tests/store.scm
index 0af099c1ad..f007846dc1 100644
--- a/tests/store.scm
+++ b/tests/store.scm
@@ -115,6 +115,18 @@
                                 (passwd:name (getpwuid (getuid)))))))
     (list (stat:uid s) (stat:perms s))))
 
+(test-equal "add-to-store"
+  '("sha1" "sha256" "sha512")
+  (let* ((file    (search-path %load-path "guix.scm"))
+         (content (call-with-input-file file get-bytevector-all)))
+    (map (lambda (hash-algo)
+           (let ((file (add-to-store %store "guix.scm" #f hash-algo file)))
+             (and (direct-store-path? file)
+                  (bytevector=? (call-with-input-file file get-bytevector-all)
+                                content)
+                  hash-algo)))
+         '("sha1" "sha256" "sha512"))))
+
 (test-equal "add-data-to-store"
   #vu8(1 2 3 4 5)
   (call-with-input-file (add-data-to-store %store "data" #vu8(1 2 3 4 5))
-- 
2.26.2

Ludovic Courtès wrote on 18 May 2020 23:32

[PATCH 2/6] tests: Test fixed-output derivations with several hash algorithms.

Recipients:(address . 41382@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:20200518213244.24165-2-ludo@gnu.org

* tests/derivations.scm ("fixed-output derivation"): Test several hash
algorithms.
---
 tests/derivations.scm | 39 ++++++++++++++++++++++++---------------
 1 file changed, 24 insertions(+), 15 deletions(-)

Toggle diff (57 lines)diff --git a/tests/derivations.scm b/tests/derivations.scm
index ef6cec6c76..a409fa99f0 100644
--- a/tests/derivations.scm
+++ b/tests/derivations.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020 Ludovic Courtès <ludo@gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -324,20 +324,29 @@
                                  #:hash hash #:hash-algo 'sha256)))
     (fixed-output-derivation? drv)))
 
-(test-assert "fixed-output derivation"
-  (let* ((builder    (add-text-to-store %store "my-fixed-builder.sh"
-                                        "echo -n hello > $out" '()))
-         (hash       (sha256 (string->utf8 "hello")))
-         (drv        (derivation %store "fixed"
-                                 %bash `(,builder)
-                                 #:sources `(,builder) ;optional
-                                 #:hash hash #:hash-algo 'sha256))
-         (succeeded? (build-derivations %store (list drv))))
-    (and succeeded?
-         (let ((p (derivation->output-path drv)))
-           (and (equal? (string->utf8 "hello")
-                        (call-with-input-file p get-bytevector-all))
-                (bytevector? (query-path-hash %store p)))))))
+(test-equal "fixed-output derivation"
+  '(sha1 sha256 sha512)
+  (map (lambda (hash-algorithm)
+         (let* ((builder (add-text-to-store %store "my-fixed-builder.sh"
+                                            "echo -n hello > $out" '()))
+                (sha256  (sha256 (string->utf8 "hello")))
+                (hash    (bytevector-hash
+                          (string->utf8 "hello")
+                          (lookup-hash-algorithm hash-algorithm)))
+                (drv     (derivation %store
+                                     (string-append
+                                      "fixed-" (symbol->string hash-algorithm))
+                                     %bash `(,builder)
+                                     #:sources `(,builder) ;optional
+                                     #:hash hash
+                                     #:hash-algo hash-algorithm)))
+           (build-derivations %store (list drv))
+           (let ((p (derivation->output-path drv)))
+             (and (bytevector=? (string->utf8 "hello")
+                                (call-with-input-file p get-bytevector-all))
+                  (bytevector? (query-path-hash %store p))
+                  hash-algorithm))))
+       '(sha1 sha256 sha512)))
 
 (test-assert "fixed-output derivation: output paths are equal"
   (let* ((builder1   (add-text-to-store %store "fixed-builder1.sh"
-- 
2.26.2

Ludovic Courtès wrote on 18 May 2020 23:32

[PATCH 3/6] guix hash, guix download: Add '--hash'.

Recipients:(address . 41382@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:20200518213244.24165-3-ludo@gnu.org

* guix/scripts/download.scm (%default-options): Add 'hash-algorithm'.
(show-help, %options): Add "--hash".
(guix-download): Honor it.
* guix/scripts/hash.scm (%default-options): Add 'hash-algorithm'.
(show-help, %options): Add "--hash".
(guix-hash): Honor it.
* tests/guix-hash.sh: Test '-H sha512'.
* doc/guix.texi (Invoking guix download): Document it.
(Invoking guix hash): Document it.
---
 doc/guix.texi             | 15 +++++++++++++++
 guix/scripts/download.scm | 14 ++++++++++++--
 guix/scripts/hash.scm     | 21 +++++++++++++++++----
 tests/guix-hash.sh        |  6 +++++-
 4 files changed, 49 insertions(+), 7 deletions(-)

Toggle diff (166 lines)diff --git a/doc/guix.texi b/doc/guix.texi
index eef5b703fe..0cf006770e 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -9018,6 +9018,11 @@ Certificates}), unless @option{--no-check-certificate} is used.
 The following options are available:
 
 @table @code
+@item --hash=@var{algorithm}
+@itemx -H @var{algorithm}
+Compute a hash using the specified @var{algorithm}.  @xref{Invoking guix
+hash}, for more information.
+
 @item --format=@var{fmt}
 @itemx -f @var{fmt}
 Write the hash in the format specified by @var{fmt}.  For more
@@ -9057,6 +9062,16 @@ following options:
 
 @table @code
 
+@item --hash=@var{algorithm}
+@itemx -H @var{algorithm}
+Compute a hash using the specified @var{algorithm}, @code{sha256} by
+default.
+
+@var{algorithm} must the name of a cryptographic hash algorithm
+supported by Libgcrypt @i{via} Guile-Gcrypt---e.g., @code{sha512} or
+@code{sha3-256} (@pxref{Hash Functions,,, guile-gcrypt, Guile-Gcrypt
+Reference Manual}).
+
 @item --format=@var{fmt}
 @itemx -f @var{fmt}
 Write the hash in the format specified by @var{fmt}.
diff --git a/guix/scripts/download.scm b/guix/scripts/download.scm
index 22cd75ea0b..b4446c2e2f 100644
--- a/guix/scripts/download.scm
+++ b/guix/scripts/download.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2012, 2013, 2015, 2016, 2017 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2012, 2013, 2015, 2016, 2017, 2020 Ludovic Courtès <ludo@gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -77,6 +77,7 @@
 (define %default-options
   ;; Alist of default option values.
   `((format . ,bytevector->nix-base32-string)
+    (hash-algorithm . ,(hash-algorithm sha256))
     (verify-certificate? . #t)
     (download-proc . ,download-to-store*)))
 
@@ -89,6 +90,8 @@ Supported formats: 'nix-base32' (default), 'base32', and 'base16'
 ('hex' and 'hexadecimal' can be used as well).\n"))
   (format #t (G_ "
   -f, --format=FMT       write the hash in the given format"))
+  (format #t (G_ "
+  -H, --hash=ALGORITHM   use the given hash ALGORITHM"))
   (format #t (G_ "
       --no-check-certificate
                          do not validate the certificate of HTTPS servers "))
@@ -119,6 +122,13 @@ Supported formats: 'nix-base32' (default), 'base32', and 'base16'
 
                   (alist-cons 'format fmt-proc
                               (alist-delete 'format result))))
+        (option '(#\H "hash") #t #f
+                (lambda (opt name arg result)
+                  (match (lookup-hash-algorithm (string->symbol arg))
+                    (#f
+                     (leave (G_ "~a: unknown hash algorithm~%") arg))
+                    (algo
+                     (alist-cons 'hash-algorithm algo result)))))
         (option '("no-check-certificate") #f #f
                 (lambda (opt name arg result)
                   (alist-cons 'verify-certificate? #f result)))
@@ -175,7 +185,7 @@ Supported formats: 'nix-base32' (default), 'base32', and 'base16'
                       (or path
                           (leave (G_ "~a: download failed~%")
                                  arg))
-                    port-sha256))
+                    (cute port-hash (assoc-ref opts 'hash-algorithm) <>)))
            (fmt   (assq-ref opts 'format)))
       (format #t "~a~%~a~%" path (fmt hash))
       #t)))
diff --git a/guix/scripts/hash.scm b/guix/scripts/hash.scm
index b8b2158195..cfc4420260 100644
--- a/guix/scripts/hash.scm
+++ b/guix/scripts/hash.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2012, 2013, 2014, 2016, 2017 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2012, 2013, 2014, 2016, 2017, 2020 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2013 Nikita Karetnikov <nikita@karetnikov.org>
 ;;; Copyright © 2016 Jan Nieuwenhuizen <janneke@gnu.org>
 ;;; Copyright © 2018 Tim Gesthuizen <tim.gesthuizen@yahoo.de>
@@ -42,7 +42,8 @@
 
 (define %default-options
   ;; Alist of default option values.
-  `((format . ,bytevector->nix-base32-string)))
+  `((format . ,bytevector->nix-base32-string)
+    (hash-algorithm . ,(hash-algorithm sha256))))
 
 (define (show-help)
   (display (G_ "Usage: guix hash [OPTION] FILE
@@ -53,6 +54,8 @@ and 'hexadecimal' can be used as well).\n"))
   (format #t (G_ "
   -x, --exclude-vcs      exclude version control directories"))
   (format #t (G_ "
+  -H, --hash=ALGORITHM   use the given hash ALGORITHM"))
+  (format #t (G_ "
   -f, --format=FMT       write the hash in the given format"))
   (format #t (G_ "
   -r, --recursive        compute the hash on FILE recursively"))
@@ -69,6 +72,13 @@ and 'hexadecimal' can be used as well).\n"))
   (list (option '(#\x "exclude-vcs") #f #f
                 (lambda (opt name arg result)
                   (alist-cons 'exclude-vcs? #t result)))
+        (option '(#\H "hash") #t #f
+                (lambda (opt name arg result)
+                  (match (lookup-hash-algorithm (string->symbol arg))
+                    (#f
+                     (leave (G_ "~a: unknown hash algorithm~%") arg))
+                    (algo
+                     (alist-cons 'hash-algorithm algo result)))))
         (option '(#\f "format") #t #f
                 (lambda (opt name arg result)
                   (define fmt-proc
@@ -139,8 +149,11 @@ and 'hexadecimal' can be used as well).\n"))
               (force-output port)
               (get-hash))
             (match file
-              ("-" (port-sha256 (current-input-port)))
-              (_   (call-with-input-file file port-sha256))))))
+              ("-" (port-hash (assoc-ref opts 'hash-algorithm)
+                              (current-input-port)))
+              (_   (call-with-input-file file
+                     (cute port-hash (assoc-ref opts 'hash-algorithm)
+                           <>)))))))
 
     (match args
       ((file)
diff --git a/tests/guix-hash.sh b/tests/guix-hash.sh
index 190c9e7f8a..1c595b49ed 100644
--- a/tests/guix-hash.sh
+++ b/tests/guix-hash.sh
@@ -1,5 +1,5 @@
 # GNU Guix --- Functional package management for GNU
-# Copyright © 2013, 2014, 2016 Ludovic Courtès <ludo@gnu.org>
+# Copyright © 2013, 2014, 2016, 2020 Ludovic Courtès <ludo@gnu.org>
 # Copyright © 2016 Jan Nieuwenhuizen <janneke@gnu.org>
 #
 # This file is part of GNU Guix.
@@ -31,6 +31,10 @@ test `echo -n | guix hash -` = 0mdqa9w1p6cmli6976v4wi0sw9r4p5prkj7lzfd1877wk11c9
 test `guix hash -f nix-base32 /dev/null` = 0mdqa9w1p6cmli6976v4wi0sw9r4p5prkj7lzfd1877wk11c9c73
 test `guix hash -f hex /dev/null` = e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
 test `guix hash -f base32 /dev/null` = 4oymiquy7qobjgx36tejs35zeqt24qpemsnzgtfeswmrw6csxbkq
+test `guix hash -H sha512 -f hex /dev/null` = cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e
+
+if guix hash -H abcd1234 /dev/null;
+then false; else true; fi
 
 mkdir "$tmpdir"
 echo -n executable > "$tmpdir/exe"
-- 
2.26.2

Ludovic Courtès wrote on 18 May 2020 23:32

[PATCH 4/6] guix hash, guix download: Support base64 format.

Recipients:(address . 41382@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:20200518213244.24165-4-ludo@gnu.org

* guix/scripts/download.scm (show-help, %options): Support "base64"
format.
* guix/scripts/hash.scm (show-help, %options): Likewise.
* tests/guix-hash.sh: Test it.
* doc/guix.texi (Invoking guix hash): Document it.
---
 doc/guix.texi             |  2 +-
 guix/scripts/download.scm | 12 ++++++++----
 guix/scripts/hash.scm     | 14 +++++++++-----
 tests/guix-hash.sh        |  1 +
 4 files changed, 19 insertions(+), 10 deletions(-)

Toggle diff (108 lines)diff --git a/doc/guix.texi b/doc/guix.texi
index 0cf006770e..fdd9622211 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -9076,7 +9076,7 @@ Reference Manual}).
 @itemx -f @var{fmt}
 Write the hash in the format specified by @var{fmt}.
 
-Supported formats: @code{nix-base32}, @code{base32}, @code{base16}
+Supported formats: @code{base64}, @code{nix-base32}, @code{base32}, @code{base16}
 (@code{hex} and @code{hexadecimal} can be used as well).
 
 If the @option{--format} option is not specified, @command{guix hash}
diff --git a/guix/scripts/download.scm b/guix/scripts/download.scm
index b4446c2e2f..589f62da9d 100644
--- a/guix/scripts/download.scm
+++ b/guix/scripts/download.scm
@@ -23,6 +23,7 @@
   #:use-module (gcrypt hash)
   #:use-module (guix base16)
   #:use-module (guix base32)
+  #:autoload   (guix base64) (base64-encode)
   #:use-module ((guix download) #:hide (url-fetch))
   #:use-module ((guix build download)
                 #:select (url-fetch))
@@ -84,10 +85,11 @@
 (define (show-help)
   (display (G_ "Usage: guix download [OPTION] URL
 Download the file at URL to the store or to the given file, and print its
-file name and the hash of its contents.
-
-Supported formats: 'nix-base32' (default), 'base32', and 'base16'
-('hex' and 'hexadecimal' can be used as well).\n"))
+file name and the hash of its contents.\n"))
+  (newline)
+  (display (G_ "\
+Supported formats: 'base64', 'nix-base32' (default), 'base32',
+and 'base16' ('hex' and 'hexadecimal' can be used as well).\n"))
   (format #t (G_ "
   -f, --format=FMT       write the hash in the given format"))
   (format #t (G_ "
@@ -111,6 +113,8 @@ Supported formats: 'nix-base32' (default), 'base32', and 'base16'
                 (lambda (opt name arg result)
                   (define fmt-proc
                     (match arg
+                      ("base64"
+                       base64-encode)
                       ("nix-base32"
                        bytevector->nix-base32-string)
                       ("base32"
diff --git a/guix/scripts/hash.scm b/guix/scripts/hash.scm
index cfc4420260..9b4f419a24 100644
--- a/guix/scripts/hash.scm
+++ b/guix/scripts/hash.scm
@@ -20,12 +20,13 @@
 ;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
 
 (define-module (guix scripts hash)
-  #:use-module (guix base32)
   #:use-module (gcrypt hash)
   #:use-module (guix serialization)
   #:use-module (guix ui)
   #:use-module (guix scripts)
   #:use-module (guix base16)
+  #:use-module (guix base32)
+  #:autoload   (guix base64) (base64-encode)
   #:use-module (ice-9 binary-ports)
   #:use-module (rnrs files)
   #:use-module (ice-9 match)
@@ -47,10 +48,11 @@
 
 (define (show-help)
   (display (G_ "Usage: guix hash [OPTION] FILE
-Return the cryptographic hash of FILE.
-
-Supported formats: 'nix-base32' (default), 'base32', and 'base16' ('hex'
-and 'hexadecimal' can be used as well).\n"))
+Return the cryptographic hash of FILE.\n"))
+  (newline)
+  (display (G_ "\
+Supported formats: 'base64', 'nix-base32' (default), 'base32',
+and 'base16' ('hex' and 'hexadecimal' can be used as well).\n"))
   (format #t (G_ "
   -x, --exclude-vcs      exclude version control directories"))
   (format #t (G_ "
@@ -83,6 +85,8 @@ and 'hexadecimal' can be used as well).\n"))
                 (lambda (opt name arg result)
                   (define fmt-proc
                     (match arg
+                      ("base64"
+                       base64-encode)
                       ("nix-base32"
                        bytevector->nix-base32-string)
                       ("base32"
diff --git a/tests/guix-hash.sh b/tests/guix-hash.sh
index 1c595b49ed..3538b9aeda 100644
--- a/tests/guix-hash.sh
+++ b/tests/guix-hash.sh
@@ -32,6 +32,7 @@ test `guix hash -f nix-base32 /dev/null` = 0mdqa9w1p6cmli6976v4wi0sw9r4p5prkj7lz
 test `guix hash -f hex /dev/null` = e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
 test `guix hash -f base32 /dev/null` = 4oymiquy7qobjgx36tejs35zeqt24qpemsnzgtfeswmrw6csxbkq
 test `guix hash -H sha512 -f hex /dev/null` = cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e
+test `guix hash -H sha1 -f base64 /dev/null` = "2jmj7l5rSw0yVb/vlWAYkK/YBwk="
 
 if guix hash -H abcd1234 /dev/null;
 then false; else true; fi
-- 
2.26.2

Ludovic Courtès wrote on 18 May 2020 23:32

[PATCH 5/6] packages: Add 'sha512' optional field to <origin>.

Recipients:(address . 41382@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:20200518213244.24165-5-ludo@gnu.org

* guix/packages.scm (<origin>)[sha512]: New field.
(print-origin): Honor it.
(origin->derivation): Likewise.
* tests/packages.scm ("package-source-derivation, origin, sha512"): New
test.
* doc/guix.texi (origin Reference): Document 'sha512'.
---
 doc/guix.texi      |  8 +++++++-
 guix/packages.scm  | 25 ++++++++++++++-----------
 tests/packages.scm | 26 ++++++++++++++++++++++++++
 3 files changed, 47 insertions(+), 12 deletions(-)

Toggle diff (122 lines)diff --git a/doc/guix.texi b/doc/guix.texi
index fdd9622211..50d7eb7a43 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -5966,10 +5966,16 @@ specified in the @code{uri} field as a @code{git-reference} object; a
 @end table
 
 @item @code{sha256}
-A bytevector containing the SHA-256 hash of the source.  Typically the
+@itemx @code{sha512}
+A bytevector containing the SHA-256 (respectively SHA-512) hash of the source.  Typically the
 @code{base32} form is used here to generate the bytevector from a
 base-32 string.
 
+One of these fields must be a bytevector while the others can be
+@code{#f}.  When several hashes are provided, the ``strongest'' is used
+when computing the underlying fixed-output derivation
+(@pxref{Derivations}).
+
 You can obtain this information using @code{guix download}
 (@pxref{Invoking guix download}) or @code{guix hash} (@pxref{Invoking
 guix hash}).
diff --git a/guix/packages.scm b/guix/packages.scm
index 3fff50a6e8..7cf4c9c3e6 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -164,6 +164,7 @@
   (uri       origin-uri)                          ; string
   (method    origin-method)                       ; procedure
   (sha256    origin-sha256)                       ; bytevector
+  (sha512    origin-sha512 (default #f))          ; bytevector | #f
   (file-name origin-file-name (default #f))       ; optional file name
 
   ;; Patches are delayed so that the 'search-patch' calls are made lazily,
@@ -189,9 +190,9 @@
 (define (print-origin origin port)
   "Write a concise representation of ORIGIN to PORT."
   (match origin
-    (($ <origin> uri method sha256 file-name patches)
+    (($ <origin> uri method sha256 sha512 file-name patches)
      (simple-format port "#<origin ~s ~a ~s ~a>"
-                    uri (bytevector->base32-string sha256)
+                    uri (bytevector->base32-string (or sha512 sha256))
                     (force patches)
                     (number->string (object-address origin) 16)))))
 
@@ -1381,18 +1382,20 @@ unless you know what you are doing."
                              #:optional (system (%current-system)))
   "Return the derivation corresponding to ORIGIN."
   (match origin
-    (($ <origin> uri method sha256 name (= force ()) #f)
+    (($ <origin> uri method sha256 sha512 name (= force ()) #f)
      ;; No patches, no snippet: this is a fixed-output derivation.
-     (method uri 'sha256 sha256 name #:system system))
-    (($ <origin> uri method sha256 name (= force (patches ...)) snippet
+     (let ((algorithm (if sha512 'sha512 'sha256)))
+       (method uri algorithm (or sha512 sha256) name #:system system)))
+    (($ <origin> uri method sha256 sha512 name (= force (patches ...)) snippet
         (flags ...) inputs (modules ...) guile-for-build)
      ;; Patches and/or a snippet.
-     (mlet %store-monad ((source (method uri 'sha256 sha256 name
-                                         #:system system))
-                         (guile  (package->derivation (or guile-for-build
-                                                          (default-guile))
-                                                      system
-                                                      #:graft? #f)))
+     (mlet* %store-monad ((algorithm -> (if sha512 'sha512 'sha256))
+                          (source (method uri algorithm (or sha512 sha256)
+                                          name #:system system))
+                          (guile  (package->derivation (or guile-for-build
+                                                           (default-guile))
+                                                       system
+                                                       #:graft? #f)))
        (patch-and-repack source patches
                          #:inputs inputs
                          #:snippet snippet
diff --git a/tests/packages.scm b/tests/packages.scm
index c528d2080c..06d41b5ce7 100644
--- a/tests/packages.scm
+++ b/tests/packages.scm
@@ -51,6 +51,7 @@
   #:use-module (srfi srfi-34)
   #:use-module (srfi srfi-35)
   #:use-module (srfi srfi-64)
+  #:use-module (rnrs bytevectors)
   #:use-module (rnrs io ports)
   #:use-module (ice-9 vlist)
   #:use-module (ice-9 regex)
@@ -497,6 +498,31 @@
                      (search-path %load-path "guix/base32.scm")
                    get-bytevector-all)))))
 
+(test-equal "package-source-derivation, origin, sha512"
+  "hello"
+  (let* ((bash    (search-bootstrap-binary "bash" (%current-system)))
+         (builder (add-text-to-store %store "my-fixed-builder.sh"
+                                     "echo -n hello > $out" '()))
+         (method  (lambda* (url hash-algo hash #:optional name
+                                #:rest rest)
+                    (and (eq? hash-algo 'sha512)
+                         (raw-derivation name bash (list builder)
+                                         #:sources (list builder)
+                                         #:hash hash
+                                         #:hash-algo hash-algo))))
+         (source  (origin
+                    (method method)
+                    (uri "unused://")
+                    (file-name "origin-sha512")
+                    (sha256 (bytevector-hash (string->utf8 "hello")
+                                             (hash-algorithm sha256)))
+                    (sha512 (bytevector-hash (string->utf8 "hello")
+                                             (hash-algorithm sha512)))))
+         (drv    (package-source-derivation %store source))
+         (output (derivation->output-path drv)))
+    (build-derivations %store (list drv))
+    (call-with-input-file output get-string-all)))
+
 (unless (network-reachable?) (test-skip 1))
 (test-equal "package-source-derivation, snippet"
   "OK"
-- 
2.26.2

Ludovic Courtès wrote on 18 May 2020 23:32

[PATCH 6/6] packages: Add 'base64' macro.

Recipients:(address . 41382@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:20200518213244.24165-6-ludo@gnu.org

* guix/packages.scm (define-compile-time-decoder): New macro.
(base32): Redefine in terms of it.
(base64): New macro.
---
 guix/packages.scm | 31 +++++++++++++++++++------------
 1 file changed, 19 insertions(+), 12 deletions(-)

Toggle diff (59 lines)diff --git a/guix/packages.scm b/guix/packages.scm
index 7cf4c9c3e6..724d7693c7 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -28,6 +28,7 @@
   #:use-module (guix monads)
   #:use-module (guix gexp)
   #:use-module (guix base32)
+  #:autoload   (guix base64) (base64-decode)
   #:use-module (guix grafts)
   #:use-module (guix derivations)
   #:use-module (guix memoization)
@@ -62,6 +63,7 @@
             origin-snippet
             origin-modules
             base32
+            base64
 
             package
             package?
@@ -198,19 +200,24 @@
 
 (set-record-type-printer! <origin> print-origin)
 
-(define-syntax base32
-  (lambda (s)
-    "Return the bytevector corresponding to the given Nix-base32
+(define-syntax-rule (define-compile-time-decoder name string->bytevector)
+  "Define NAME as a macro that runs STRING->BYTEVECTOR at macro expansion time
+if possible."
+  (define-syntax name
+    (lambda (s)
+      "Return the bytevector corresponding to the given textual
 representation."
-    (syntax-case s ()
-      ((_ str)
-       (string? (syntax->datum #'str))
-       ;; A literal string: do the conversion at expansion time.
-       (with-syntax ((bv (nix-base32-string->bytevector
-                          (syntax->datum #'str))))
-         #''bv))
-      ((_ str)
-       #'(nix-base32-string->bytevector str)))))
+      (syntax-case s ()
+        ((_ str)
+         (string? (syntax->datum #'str))
+         ;; A literal string: do the conversion at expansion time.
+         (with-syntax ((bv (string->bytevector (syntax->datum #'str))))
+           #''bv))
+        ((_ str)
+         #'(string->bytevector str))))))
+
+(define-compile-time-decoder base32 nix-base32-string->bytevector)
+(define-compile-time-decoder base64 base64-decode)
 
 (define (origin-actual-file-name origin)
   "Return the file name of ORIGIN, either its 'file-name' field or the file
-- 
2.26.2

Ludovic Courtès wrote on 19 May 2020 16:42

Re: [bug#41382] [PATCH 0/6] Allow for a cryptographic hash function migration

Recipients:(address . 41382@debbugs.gnu.org)

Message-ID:871rnggf4d.fsf@gnu.org

Hello,

Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (5 lines)

> Another option would be to create a <hash> data type that specifies

> its algorithm and its value. We’d replace the ‘sha256’ field with

> a ‘hash’ field of that type (in a backward-compatible way). Thinking

> about it, this is perhaps the better option.

Here’s a v2 that does that: instead of adding a ‘sha512’ field to

<origin>, it replaces the ‘sha256’ field with ‘hash’ and introduces a

<content-hash> data type (similar to the <uuid> data type we have).

One can now write things like:

(origin

;; …

(hash (content-hash (base64 "…") sha512)))

Since it’s a bit verbose, one can also pass a literal string directly,

in which case it’s base32-decoded:

(origin

;; …

(hash (content-hash "…")))

‘content-hash’ uses macrology to validate as much as possible at

macro-expansion time.

There’s a compatibility ‘origin’ macro intended to allow people to keep

writing:

(origin

(url …)

(method …)

(sha256 …))

and to automatically “convert” the ‘sha256’ field specification to a

‘content-hash’. Due to the way identifiers are matched, there are cases

where we can’t preserve the illusion of compatibility, as can be seen

with the patch below. Perhaps that’s acceptable, though.

Thoughts?

Thanks,

Ludo’.

Attachment: 0001-packages-Introduce-content-hash-and-use-it-in-origin.patch

Marius Bakke wrote on 19 May 2020 20:00

Recipients:(address . guix-devel@gnu.org)

Message-ID:87wo57reil.fsf@devup.no

Ludovic,

(+ guix-devel)

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (44 lines)> Hello,
>
> Ludovic Courtès <ludo@gnu.org> skribis:
>
>> Another option would be to create a <hash> data type that specifies
>> its algorithm and its value.  We’d replace the ‘sha256’ field with
>> a ‘hash’ field of that type (in a backward-compatible way).  Thinking
>> about it, this is perhaps the better option.
>
> Here’s a v2 that does that: instead of adding a ‘sha512’ field to
> <origin>, it replaces the ‘sha256’ field with ‘hash’ and introduces a
> <content-hash> data type (similar to the <uuid> data type we have).
>
> One can now write things like:
>
>   (origin
>     ;; …
>     (hash (content-hash (base64 "…") sha512)))
>
> Since it’s a bit verbose, one can also pass a literal string directly,
> in which case it’s base32-decoded:
>
>   (origin
>     ;; …
>     (hash (content-hash "…")))
>
> ‘content-hash’ uses macrology to validate as much as possible at
> macro-expansion time.
>
> There’s a compatibility ‘origin’ macro intended to allow people to keep
> writing:
>
>   (origin
>     (url …)
>     (method …)
>     (sha256 …))
>
> and to automatically “convert” the ‘sha256’ field specification to a
> ‘content-hash’.  Due to the way identifiers are matched, there are cases
> where we can’t preserve the illusion of compatibility, as can be seen
> with the patch below.  Perhaps that’s acceptable, though.
>
> Thoughts?

This is a great initiative, and the patches LGTM.

I think that if we are to move away from SHA256, we should go with

something that is immune to length extension attacks[0] such as BLAKE2/3

or SHA-3 (Keccak).

Although I don't know any Guile implementations of those as of yet.

SHA512 does not improve much security-wise IMO, but maybe it's

worthwhile as s stop-gap.

0: https://en.wikipedia.org/wiki/Length_extension_attack

Leo Famulari wrote on 19 May 2020 20:20

Recipients:(name . Marius Bakke)(address . mbakke@fastmail.com)

Message-ID:20200519182001.GA19528@jasmine.lan

On Tue, May 19, 2020 at 08:00:34PM +0200, Marius Bakke wrote:

Toggle quote (2 lines)

> This is a great initiative, and the patches LGTM.

Toggle quote (4 lines)

> I think that if we are to move away from SHA256, we should go with

> something that is immune to length extension attacks[0] such as BLAKE2/3

> or SHA-3 (Keccak).

I think we are so far from needing to move away from SHA256 that we
can't know what the best replacement will be when the time comes. It
will probably be quite a long time, maybe decades.

Ludovic Courtès wrote on 21 May 2020 22:46

Recipients:(name . Marius Bakke)(address . mbakke@fastmail.com)

Message-ID:875zcpc8yt.fsf@gnu.org

Hi!

Marius Bakke <mbakke@fastmail.com> skribis:

[...]

Toggle quote (9 lines)

>> and to automatically “convert” the ‘sha256’ field specification to a

>> ‘content-hash’. Due to the way identifiers are matched, there are cases

>> where we can’t preserve the illusion of compatibility, as can be seen

>> with the patch below. Perhaps that’s acceptable, though.

>> Thoughts?

> This is a great initiative, and the patches LGTM.

Great, thanks for taking a look.

Toggle quote (4 lines)

> I think that if we are to move away from SHA256, we should go with

> something that is immune to length extension attacks[0] such as BLAKE2/3

> or SHA-3 (Keccak).

That makes sense to me.

I think we have time to think about it. When we choose to switch, we

should change all the tools (importers, ‘guix download’, etc.) and

documentation to default to the new hash so migration can happen

consistently.

Toggle quote (2 lines)

> Although I don't know any Guile implementations of those as of yet.

Libgcrypt supports them, so we can definitely use them. I realize we

also need to extend nix/libutil/hash.{cc,hh}.

Toggle quote (3 lines)

> SHA512 does not improve much security-wise IMO, but maybe it's

> worthwhile as s stop-gap.

Yeah, I’m not sure. We should definitely keep an eye on what others are

doing and what crypto folks recommend.

Ludo’.

Ludovic Courtès wrote on 22 May 2020 01:43

Recipients:(name . Marius Bakke)(address . mbakke@fastmail.com)

Message-ID:87wo54c0rf.fsf@gnu.org

Pushed the whole series:

ce0be5675b packages: Introduce <content-hash> and use it in <origin>.

56f7ca6e7c packages: Add 'base64' macro.

0e4e9c8e76 guix hash, guix download: Support base64 format.

18ae1ec3ec guix hash, guix download: Add '--hash'.

9418aaa00d tests: Test fixed-output derivations with several hash algorithms.

73b27eaa64 tests: Test 'add-to-store' with several hash algorithms.

You’ll have to recompile due to the ABI change:

make clean-go && make

I realized several tests needed to be adjusted for proper syntax-case

matching, which I did in ce0be5675b.

Ludo’.

Closed

Your comment

This issue is archived.

To comment on this conversation send an email to 41382@debbugs.gnu.org

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date