guix graph gives duplicate nodes

  • Done
  • quality assurance status badge
Details
3 participants
  • Björn Höfling
  • Hartmut Goebel
  • Ludovic Courtès
Owner
unassigned
Submitted by
Hartmut Goebel
Severity
normal
H
H
Hartmut Goebel wrote on 5 Mar 2018 10:37
(name . bug-guix)(address . bug-guix@gnu.org)
86fb9895-768a-bea0-154c-070861c0bb1d@crazy-compilers.com
Hi,

"guix graph" delivers the same package with different IDs. Here is an
example with a node delivered twice. (For plasma-workspace, which I'm
working on, this package was even listed four times).

$ ./pre-inst-env guix graph -t package -b graphviz qtbase | grep
autoconf-wrapper
  "59511552" [label = "autoconf-wrapper-2.69", shape = box, fontname =
Helvetica];
  "59511744" [label = "autoconf-wrapper-2.69", shape = box, fontname =
Helvetica];

--
Regards
Hartmut Goebel

| Hartmut Goebel | h.goebel@crazy-compilers.com |
| www.crazy-compilers.com | compilers which you thought are impossible |
L
L
Ludovic Courtès wrote on 5 Mar 2018 18:19
(name . Hartmut Goebel)(address . h.goebel@crazy-compilers.com)(address . 30710@debbugs.gnu.org)
87d10i8mpb.fsf@gnu.org
Hello,

Hartmut Goebel <h.goebel@crazy-compilers.com> skribis:

Toggle quote (11 lines)
> "guix graph" delivers the same package with different IDs. Here is an
> example with a node delivered twice. (For plasma-workspace, which I'm
> working on, this package was even listed four times).
>
> $ ./pre-inst-env guix graph -t package -b graphviz qtbase | grep
> autoconf-wrapper
>   "59511552" [label = "autoconf-wrapper-2.69", shape = box, fontname =
> Helvetica];
>   "59511744" [label = "autoconf-wrapper-2.69", shape = box, fontname =
> Helvetica];

This is expected. Strictly speaking, we’re talking about two different
package objects, hence the different IDs.

Now, there are cases were we have multiple package objects mapping to a
single derivation. That’s OK. You can use “guix graph -t bag” or
similar if you want nodes that correspond to derivations.

HTH!

Ludo’.
H
H
Hartmut Goebel wrote on 6 Mar 2018 21:28
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 30710@debbugs.gnu.org)
c9c92f41-65ea-3ac8-daa8-2cc9b7fbddf5@crazy-compilers.com
Hi,
Toggle quote (3 lines)
> This is expected. Strictly speaking, we’re talking about two different
> package objects, hence the different IDs.

I wonder

a) whether it is useful to have different graph nodes for the same package.

This is about usability of the resulting graph, esp. since this is the
default graph output format. Does it help *users* for their analysis? Or
is this some *expert* insight?

b) how there can be different package objects for the same package

To my understanding, e.g. (gnu packages base) is loaded once, defining
package object abcd once and assigning it to a variable. All packages
referring to abcd use the some package object. So there should be only
*one* package object.

--
Regards
Hartmut Goebel

| Hartmut Goebel | h.goebel@crazy-compilers.com |
| www.crazy-compilers.com | compilers which you thought are impossible |
L
L
Ludovic Courtès wrote on 7 Mar 2018 16:18
(name . Hartmut Goebel)(address . h.goebel@crazy-compilers.com)(address . 30710@debbugs.gnu.org)
87efkvsylg.fsf@gnu.org
Hello,

Hartmut Goebel <h.goebel@crazy-compilers.com> skribis:

Toggle quote (11 lines)
>> This is expected. Strictly speaking, we’re talking about two different
>> package objects, hence the different IDs.
>
> I wonder
>
> a) whether it is useful to have different graph nodes for the same package.
>
> This is about usability of the resulting graph, esp. since this is the
> default graph output format. Does it help *users* for their analysis? Or
> is this some *expert* insight?

As explained in “Invoking guix graph”, the tool provides different graph
types, each at its own abstraction level.

The package graph is high-level, but as a side-effect it has this
artifact.

To developers it’s actually useful to see the graph of package objects.
There are cases where, with functions that return packages, you would
notice that you’re generating lots of package objects for the same
underlying derivation, which is super inefficient (in particular it
defeats memoization).

Most of the time, there’s exactly one package object for each
derivation; if not, that’s usually a bug.

Toggle quote (7 lines)
> b) how there can be different package objects for the same package
>
> To my understanding, e.g. (gnu packages base) is loaded once, defining
> package object abcd once and assigning it to a variable. All packages
> referring to abcd use the some package object. So there should be only
> *one* package object.

(eq? foo (package (inherit foo))) => #false

Yet, these two packages map to the very same derivation.

HTH,
Ludo’.
L
L
Ludovic Courtès wrote on 8 Mar 2018 10:08
control message for bug #30710
(address . control@debbugs.gnu.org)
87sh9bnddw.fsf@gnu.org
tags 30710 notabug
close 30710
B
B
Björn Höfling wrote on 9 Mar 2018 11:09
Re: bug#30710: guix graph gives duplicate nodes
(name . Ludovic Courtès)(address . ludo@gnu.org)
20180309110917.79b14a36@alma-ubu
On Wed, 07 Mar 2018 16:18:51 +0100
ludo@gnu.org (Ludovic Courtès) wrote:

Toggle quote (43 lines)
> Hello,
>
> Hartmut Goebel <h.goebel@crazy-compilers.com> skribis:
>
> >> This is expected. Strictly speaking, we’re talking about two
> >> different package objects, hence the different IDs.
> >
> > I wonder
> >
> > a) whether it is useful to have different graph nodes for the same
> > package.
> >
> > This is about usability of the resulting graph, esp. since this is
> > the default graph output format. Does it help *users* for their
> > analysis? Or is this some *expert* insight?
>
> As explained in “Invoking guix graph”, the tool provides different
> graph types, each at its own abstraction level.
>
> The package graph is high-level, but as a side-effect it has this
> artifact.
>
> To developers it’s actually useful to see the graph of package
> objects. There are cases where, with functions that return packages,
> you would notice that you’re generating lots of package objects for
> the same underlying derivation, which is super inefficient (in
> particular it defeats memoization).
>
> Most of the time, there’s exactly one package object for each
> derivation; if not, that’s usually a bug.
>
> > b) how there can be different package objects for the same package
> >
> > To my understanding, e.g. (gnu packages base) is loaded once,
> > defining package object abcd once and assigning it to a variable.
> > All packages referring to abcd use the some package object. So
> > there should be only *one* package object.
>
> (eq? foo (package (inherit foo))) => #false
>
> Yet, these two packages map to the very same derivation.


This thing really took me a while to think about the graph system in
general and this concrete case.

Speaking about this concrete case: If you inspect the output of

`guix graph qt` you find these interesting lines:

"64168128" [label = "autoconf-wrapper-2.69", shape = box, fontname = Helvetica
"64167936" [label = "autoconf-wrapper-2.69", shape = box, fontname = Helvetica

"52941184" -> "64168128" [color = darkseagreen];
"52940800" -> "64167936" [color = blue];

"52941184" [label = "automake-1.15.1", shape = box, fontname = Helvetica];
"52940800" [label = "libtool-2.4.6", shape = box, fontname = Helvetica];


If you look into gnu/packages/autotools.scm, you see that
autoconf-wrapper is not a package, but a package-factory:

(define* (autoconf-wrapper #:optional (autoconf autoconf))

Now the package definitions of "automake" and "libtool" each use the
same fragment of code in their native-inputs, but a different "package"
in the eq?-sense, although they basically want the same thing:

`(("autoconf" ,(autoconf-wrapper))

As ludo stated above: "Most of the time, there’s exactly one package
object for each derivation; if not, that’s usually a bug."

This looks to me like a bug.

Correction:

(define autoconf-wrapper-default (autoconf-wrapper))

And then use this singular package as native-inputs to libtool and automake.

Furthermore, when I search:

find . -name "*.scm" -exec grep -H "autoconf-wrapper" "{}" ";" | less

I find about 10 packages that use the fabrik, but all in the default way.

So instead of:

#:export (autoconf-wrapper))


We could just

(define-public autoconf-wrapper-default (autoconf-wrapper))

and use that.

Or, if noone is using this fabrik, just drop that and make a normal package out of it.

WDYT? Reopen this one?

Björn
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iEYEARECAAYFAlqiXU0ACgkQvyhstlk+X/2guACggmTt9KBh6POkgal0Y6oFuiEy
2B0An1rE5zvKJHIFxz1BiCKNqP1+tYIu
=wghw
-----END PGP SIGNATURE-----


L
L
Ludovic Courtès wrote on 9 Mar 2018 23:59
(name . Björn Höfling)(address . bjoern.hoefling@bjoernhoefling.de)
87d10cyhwx.fsf@gnu.org
Björn Höfling <bjoern.hoefling@bjoernhoefling.de> skribis:

Toggle quote (43 lines)
> If you look into gnu/packages/autotools.scm, you see that
> autoconf-wrapper is not a package, but a package-factory:
>
> (define* (autoconf-wrapper #:optional (autoconf autoconf))
>
> Now the package definitions of "automake" and "libtool" each use the
> same fragment of code in their native-inputs, but a different "package"
> in the eq?-sense, although they basically want the same thing:
>
> `(("autoconf" ,(autoconf-wrapper))
>
> As ludo stated above: "Most of the time, there’s exactly one package
> object for each derivation; if not, that’s usually a bug."
>
> This looks to me like a bug.
>
> Correction:
>
> (define autoconf-wrapper-default (autoconf-wrapper))
>
> And then use this singular package as native-inputs to libtool and automake.
>
> Furthermore, when I search:
>
> find . -name "*.scm" -exec grep -H "autoconf-wrapper" "{}" ";" | less
>
> I find about 10 packages that use the fabrik, but all in the default way.
>
> So instead of:
>
> #:export (autoconf-wrapper))
>
>
> We could just
>
> (define-public autoconf-wrapper-default (autoconf-wrapper))
>
> and use that.
>
> Or, if noone is using this fabrik, just drop that and make a normal package out of it.
>
> WDYT? Reopen this one?

Good catch! I implemented what you suggest above in commit
464f5447396fcec9b43f7eab71d5d42b522a157f.

Thank you!

Ludo’.
Closed
H
H
Hartmut Goebel wrote on 10 Mar 2018 17:31
10e30cb2-d65b-b9b7-dd53-a8ea5adb1f09@crazy-compilers.com
On Fri, 09 Mar 2018 23:59:26 +0100 ludo@gnu.org (Ludovic Courtès) wrote:
Toggle quote (4 lines)
>
>> Good catch! I implemented what you suggest above in commit
>> 464f5447396fcec9b43f7eab71d5d42b522a157f.

Thanks, this solved the issue only partially: On my "kde-plasma" branch:

$ ./pre-inst-env guix graph plasma-workspace | grep autoconf
  "133094208" [label = "autoconf-wrapper-2.69", shape = box, fontname =
Helvetica];
  "133094976" [label = "autoconf-2.69", shape = box, fontname = Helvetica];
  "152772352" [label = "autoconf-wrapper-2.69", shape = box, fontname =
Helvetica];
  "152420544" [label = "autoconf-2.69", shape = box, fontname = Helvetica];

There are other packages, which are duplicate but don't have a factory
AFAIK:

$ ./pre-inst-env guix graph plasma-workspace | grep kbus
  "130257472" [label = "kdbusaddons-5.42.0", shape = box, fontname =
Helvetica];
  "148171200" [label = "kdbusaddons-5.42.0", shape = box, fontname =
Helvetica];

--
Regards
Hartmut Goebel

| Hartmut Goebel | h.goebel@crazy-compilers.com |
| www.crazy-compilers.com | compilers which you thought are impossible |
L
L
Ludovic Courtès wrote on 12 Mar 2018 15:27
(name . Hartmut Goebel)(address . h.goebel@crazy-compilers.com)
87a7vdpdx4.fsf@gnu.org
Hello,

Hartmut Goebel <h.goebel@crazy-compilers.com> skribis:

Toggle quote (24 lines)
> On Fri, 09 Mar 2018 23:59:26 +0100 ludo@gnu.org (Ludovic Courtès) wrote:
>>
>>> Good catch! I implemented what you suggest above in commit
>>> 464f5447396fcec9b43f7eab71d5d42b522a157f.
>
> Thanks, this solved the issue only partially: On my "kde-plasma" branch:
>
> $ ./pre-inst-env guix graph plasma-workspace | grep autoconf
>   "133094208" [label = "autoconf-wrapper-2.69", shape = box, fontname =
> Helvetica];
>   "133094976" [label = "autoconf-2.69", shape = box, fontname = Helvetica];
>   "152772352" [label = "autoconf-wrapper-2.69", shape = box, fontname =
> Helvetica];
>   "152420544" [label = "autoconf-2.69", shape = box, fontname = Helvetica];
>
> There are other packages, which are duplicate but don't have a factory
> AFAIK:
>
> $ ./pre-inst-env guix graph plasma-workspace | grep kbus
>   "130257472" [label = "kdbusaddons-5.42.0", shape = box, fontname =
> Helvetica];
>   "148171200" [label = "kdbusaddons-5.42.0", shape = box, fontname =
> Helvetica];

That stems from the use of ‘package-mapping’ or
‘package-input-rewriting’ (maybe via ‘package-with-python2’), which can
create distinct package objects mapping to the same derivation.

I don’t have a good solution here.

Thanks,
Ludo’.
?