[PATCH] Specify native search path for all ruby packages

Without specifying this explicitly in each definition, the GEM_PATH is

inherited and the version is that of the inherited package.

search path, such that the version included in the path is 1.8, rather than

the inherited version.

---

gnu/packages/ruby.scm | 8 +++++++-

1 file changed, 7 insertions(+), 1 deletion(-)

search path, such that the version included in the path is 2.2, rather than

the inherited version.

---

gnu/packages/ruby.scm | 8 +++++++-

1 file changed, 7 insertions(+), 1 deletion(-)

search path, such that the version included in the path is 2.3, rather than

the inherited version.

---

gnu/packages/ruby.scm | 8 +++++++-

1 file changed, 7 insertions(+), 1 deletion(-)

Hi Chris,

On 21/06/17 16:36, Christopher Baines wrote:

I'm not sure if this is by design, but the version of the gems folder is

embedded in the build of each rubygem e.g. 'ruby-hoe' includes

/gnu/store/d867l5i2dqd5qnq4qlsrcwwb0x3443fl-ruby-hoe-3.16.0/lib/ruby/gems/2.4.0

So, in this environment, "require 'hoe'" fails after applying your patch

./pre-inst-env guix environment -C --ad-hoc ruby@2.2 ruby-hoe

Having said that, the gems bundled with each ruby are in a directory

with the actual version e.g. ruby@2.2 has this path

/gnu/store/f43dirk254lfhmgjs3sf515cpfl8d46q-ruby-2.2.7/lib/ruby/gems/2.2.0

This isn't ideal, any ideas on what we should do? Specify both paths maybe?

Thanks,

ben

Christopher Baines <mail@cbaines.net> skribis:

All 3 patches LGTM, thank you!

Ludo’.

Ben Woodcroft <b.woodcroft@uq.edu.au> skribis:

Or should the search path spec include both lib/ruby/gems/2.2.0 and

lib/ruby/gems/2.4.0, in this order?

Ludo’, who has no experience with Ruby.

On 21/06/17 23:12, Ludovic Courtès wrote:

Exactly.

Chris, what is your experience? Did you propose the patch because you

ran into a particular issue?

Ta, ben.

On 22/06/17 06:27, Ben Woodcroft wrote:

Yep, I ran in to problems trying to use the guix ruby-2.3 package with

the guix bundler package, when I build bundler with ruby-2.3.

Ben's email got me thinking about how this works in Debian, and it looks

like Debian uses a different location /usr/lib/ruby/vendor_ruby/ .

I think there might be benefits from doing similarly, but this needs a

bit of thought and testing, as I'm unsure how this might work,

especially in cases where libraries include native code that links

against ruby.

I've got a patch for the ruby-build-system to make a change roughly like

this, and I'll send that up soon. Relating this back to the issue at

hand, moving to a version independent directory would mean that the

GEM_PATH wouldn't be version specific.

Thanks,

Chris

On Thu, 22 Jun 2017 06:40:17 +0100

Christopher Baines <mail@cbaines.net> wrote:

So... I kind of forgot about this, sorry about that. But I'll send up

the patch shortly.

As an example of what this change means, here is a example of loading

ruby gems with different versions of ruby, with the current and

alternative situations.

I think arel doesn't use native extensions, whereas nokogiri does.

Also, ruby@2.1 is used, as the search path for the guix package

reflects the version of ruby (whereas many others don't).

** With the version of ruby used to build the gem

→ guix environment --pure --ad-hoc ruby-arel ruby -- ruby -e "puts require 'arel'"

true

(I'm pretty sure the "true" means that it worked)

→ guix environment --pure --ad-hoc ruby-nokogiri ruby -- ruby -e "puts require 'nokogiri'"

true

** With a different version of ruby

→ guix environment --pure --ad-hoc ruby-arel ruby@2.1 -- ruby -e "puts require 'arel'"

/gnu/store/gdg5m75axmrciwh3zhfnsc038q349ymg-ruby-2.1.10/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require': cannot load such file -- arel (LoadError)

from /gnu/store/gdg5m75axmrciwh3zhfnsc038q349ymg-ruby-2.1.10/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'

from -e:1:in `<main>'

→ guix environment --pure --ad-hoc ruby-arel ruby@2.1 -- ruby -e "puts require 'nokogiri'"

/gnu/store/gdg5m75axmrciwh3zhfnsc038q349ymg-ruby-2.1.10/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require': cannot load such file -- nokogiri (LoadError)

from /gnu/store/gdg5m75axmrciwh3zhfnsc038q349ymg-ruby-2.1.10/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'

from -e:1:in `<main>'

** With the version of ruby used to build the gem

→ guix environment --pure --ad-hoc ruby-arel ruby -- ruby -e "puts require 'arel'"

true

→ guix environment --pure --ad-hoc ruby-nokogiri ruby -- ruby -e "puts require 'nokogiri'"

true

** With a different version of ruby

→ guix environment --pure --ad-hoc ruby-arel ruby@2.1 -- ruby -e "puts require 'arel'"

true

→ guix environment --pure --ad-hoc ruby-nokogiri ruby@2.1 -- ruby -e "puts require 'nokogiri'"

/gnu/store/gdg5m75axmrciwh3zhfnsc038q349ymg-ruby-2.1.10/lib/ruby/2.1.0/fileutils.rb:250:in `mkdir': Read-only file system @ dir_s_mkdir - /gnu/store/jxmi5lr8dbll3pzlzwkafpd9mj1wwxma-profile/lib/ruby/vendor_ruby/extensions/x86_64-linux/2.1.0-static (Errno::EROFS)

from /gnu/store/gdg5m75axmrciwh3zhfnsc038q349ymg-ruby-2.1.10/lib/ruby/2.1.0/fileutils.rb:250:in `fu_mkdir'

...

So, putting the gems in a single location regardless of the version of

ruby they were built with means that a different version of ruby will

at least see them, however, it may still fail to load them.

I think this is an improvement, but I'm very uncertain about ruby. Does

anyone else have opinions on this?

directory, rather than the GEM_HOME. The vendor directory does not include

the version of ruby used to install the gem in the path, which makes it

easier to add it to the GEM_PATH for all versions of ruby to use.

lib/ruby/vendor_ruby.

---

gnu/packages/ruby.scm | 12 ++----------

guix/build/ruby-build-system.scm | 38 +++++++++++++++++++++++---------------

2 files changed, 25 insertions(+), 25 deletions(-)

Hi Chris,

On 17/07/17 03:37, Christopher Baines wrote:

Thanks for working on this. A few thoughts:

What happens to the default gems that come bundled with ruby itself? I'm

interpreting from your patch that these will not be available?

In general, except for some special circumstances, we don't support old

versions of software. To fix the issue that you are encountering

properly with nokogiri probably requires new package definitions using

"package-with-ruby-2.3" or similar to be made, I suppose. Ludo did some

nice work making this easier (see

but I worry in general about the resources required to support older

Ruby versions. WDYT?

The gem-home procedure in ruby-build-system will now return an incorrect

path, I think.

Perhaps I'm slow, but what are the advantages of the "vendor_ruby"

method over exporting multiple GEM_PATHs as Ludo and I suggested?

Changing the directory seems like a heavier touch and so more likely to

misbehave. WDYT?

Apologies if this comes across a little negative, I'm just trying to

consider all the issues. Thanks again for your efforts on this.

ben

On Thu, 20 Jul 2017 09:39:13 +1000

Ben Woodcroft <b.woodcroft@uq.edu.au> wrote:

They seem to be:

→ guix environment --container --ad-hoc ruby -- irb

...

for 1 packages... done in 0.028 s

irb(main):001:0> ENV

=> {"PS1"=>"\\u@\\h \\w [env]\\$ ", "TMPDIR"=>"/tmp",

"TEMPDIR"=>"/tmp", "TMP"=>"/tmp", "TEMP"=>"/tmp",

"HOME"=>"/home/chris",

"PATH"=>"/gnu/store/2dh34pi9yjq17qf1bw3dmgcdpri20fjs-profile/bin",

"GEM_PATH"=>"/gnu/store/2dh34pi9yjq17qf1bw3dmgcdpri20fjs-profile/lib/ruby/vendor_ruby",

"GUIX_ENVIRONMENT"=>"/gnu/store/2dh34pi9yjq17qf1bw3dmgcdpri20fjs-profile",

"LINES"=>"76", "COLUMNS"=>"190"}

irb(main):003:0> require 'rake'

=> true

irb(main):004:0> require 'minitest'

=> true

I'm not aware of any particular problems if you are working with the

package definitions in Guile, as it should be possible to make them use

the single ruby version that you want.

With the guix environment command I posted:

guix environment --pure --ad-hoc ruby-nokogiri ruby@2.1 -- ruby -e

"puts require 'nokogiri'"

It would be ideal if there was some way of telling guix environment to

rewrite the package definitions of all packages to use ruby@2.1 in

place of whatever ruby they might be using.

Good spot. I'll have a look at that.

I agree that it is heavier in some sense, but I like the simplicity of

getting rid of the version from the path.

The best documentation I've found for this is the NEWS of the release

where it was added [1]. While Guix blurs the lines between the "package

system" and the "user", using this vendor directory might come in

useful.

1: http://svn.ruby-lang.org/repos/ruby/tags/v1_8_7/NEWS

No need to apologise, I too think its important to consider the

details of the support for ruby, which is not something I'm

particularly knowledgeable on.

Hi Chris, sorry for the delay on this.

On 22/07/17 20:06, Christopher Baines wrote:

OK, excellent.

Is "package-mapping" sufficient?

[..]

Ah, OK. I hadn't realised there was support for this baked into Ruby

itself. Seems obvious in hindsight.

If all Ruby dependencies build with this change, then the change seems

reasonable to me, details aside.

ben

On Sat, 5 Aug 2017 13:59:56 +1000

Ben Woodcroft <b.woodcroft@uq.edu.au> wrote:

No problem :)

I don't think so. The ruby used is in the case of the ruby-build-system

is an argument to the build system, so you need to traverse part of the

dependency graph, altering the arguments of packages using the

ruby-build-system.

Or, perhaps do the transformation at a lower level abstraction than the

package record...

Ok, does anyone know a good process for testing if lots of packages

build? I think I've heard of Hydra building branches?

Hi,

[..]

For most compiled gems, I think it should be possible to write some

procedure so that this would provide you with what you want:

guix environment --ad-hoc ruby@2.1

--environment='(package-with-explicit-ruby ruby-2.1 ruby-nokogiri)'

For this specific case though, ruby-2.1 might not be the best example,

since it is explicitly not supported any more.

I believe we can go by the old standby "guix refresh -l ruby", so 659

packages are rebuilt. Given this it would be appropriate for the staging

branch, due to be merged after core-updates.

ben

Hey,

It's been a while since I looked at this, but I think it's still

relevant.

As a reminder, currently the search paths for the Ruby packages in Guix

are a bit off [1], which makes it difficult to use Ruby gems with

different versions of Ruby.

Rubygems by default installs gems to directories specific to the version

of Ruby, and this can be helpful when managing multiple versions of Ruby

and different sets of gem packages in a shared directory, but with Guix,

this isn't necessary, as the store can be used.

Ruby/Rubygems supports installing gems in to a vendor directory, which

is for gems installed by the package system [2]. While Guix blurs the

lines between the "package system" and the "user", using this vendor

directory might come in useful.

I've attached an updated patch, and successfully built every Ruby

package [3], as well as all of the packages that were changed in the

patch. The only failure was ruby-pg, which failed to build before this

patch.

Ideally when using specific versions of ruby, any gems that need to be

build with that version of ruby, would be. While this change doesn't do

that, it does simplify the search paths of ruby packages and the

packages for ruby gems.

Reading the bug report again, I think there was nearly consensus about

merging this, but I wanted to check if anyone has any comments or

experience with Ruby/Rubygems before merging this?

Thanks,

Chris

1:

scheme@(guile-user)> (use-modules (gnu packages) (guix packages) (guix search-paths) (ice-9 match))

scheme@(guile-user)> (map (lambda (ruby) (cons (package-version ruby) (match (package-native-search-paths ruby) (() #f) ((($ <search-path-specification> _ files)) files)))) (find-packages-by-name "ruby"))

$1 = (("2.4.3" "lib/ruby/gems/2.4.0") ("2.4.2" "lib/ruby/gems/2.4.0") ("2.3.6" "lib/ruby/gems/2.4.0") ("2.2.9" "lib/ruby/gems/2.4.0") ("2.1.10" "lib/ruby/gems/2.1.0") ("1.8.7-p374" . #f))

2: http://svn.ruby-lang.org/repos/ruby/tags/v1_8_7/NEWS

3: guix build --keep-going $(guix package -s ruby- | recsel -P name | tr '\n' ' ')

Hi Chris,

On 18/12/17 00:19, Christopher Baines wrote:

I think go ahead with this one. It will be useful particularly when we

update ruby to 2.5.

ben

Ben Woodcroft <b.woodcroft@uq.edu.au> writes:

Great, I've pushed this along with the other ruby build system change.

Thanks for reviewing,

Chris