[PATCH 0/2] Add Elasticsearch package and service.

  • Done
  • quality assurance status badge
Details
4 participants
  • Ludovic Courtès
  • Christopher Baines
  • Maxim Cournoyer
  • zimoun
Owner
unassigned
Submitted by
Christopher Baines
Severity
normal
C
C
Christopher Baines wrote on 13 Mar 2018 20:12
(address . guix-patches@gnu.org)
871sgn23ip.fsf@cbaines.net
Tags: moreinfo

I'm trying to reduce the number of patches I have sitting around in git
branches, so here is a bug about Elasticsearch.

I've had a really awful package definition for this for a while, which
just makes the built things in the release tarball run. While this isn't
suitable for inclusion in to Guix, it does still work.

Also included is a service, along with a really simple test.


Christopher Baines (2):
gnu: Add elasticsearch.
services: Add elasticsearch.

gnu/packages/databases.scm | 64 +++++++++++++++++++++++++++
gnu/services/databases.scm | 107 +++++++++++++++++++++++++++++++++++++++++++++
gnu/tests/databases.scm | 57 +++++++++++++++++++++++-
3 files changed, 227 insertions(+), 1 deletion(-)
-----BEGIN PGP SIGNATURE-----

iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAlqoIq5fFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF
ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE
9XdmbA//bxqx1YPnqNWxUURtKQawZoDzhG0DuNHrzH66PZOkLkiSrU53v8TBMQVH
o80IdFqfyAs7/BYEytgIGEprRW7SCwdpS2og8PZuDXTg07hDchdO1KP+7keOz12O
4Gvzr6s36ZfXqFXozLmDYe+Tlms/4WP22+DcYBNMPm4W/HcXhX3gxl7vlUoApkEU
B8Zb8Ap3YZD3IEgDhWkykoHuP0gR6Q0ArRXUCPwoFE+0gds5/TZMkKDB0WrDl4lk
HirSTp2r491Y+pVEO63hjpnuB43LRgwLGOleHeZ61ncCztrEqogUz8zDqQ3STPAN
VRLzK6kByGaRU1fDRh0/GzcDfEcNRsR2efsA5mYPNCY9dBxyNTv4a7a9tn+NjhEY
XnHULCNyGw+C4+sW8VuEt/OjCXGouHe9C8/akqqeNMhmeTw6Pl3jQGShZfZpRKEe
GjOY00WHaOCmiHa0i2R3EE+9zrP4yiDRZOf5K3sbAlSknFuMMl+KeGr8rCq+MX1z
ZSzxCi2phq5PYIxAwsVRyR93Dy5k8FjE6/8WPnX10PsZS6NszSslhewJatMDUhzb
4mGWsAyNKbMMPHQJEqcrkFIbGSn3Se/SLiHfWqN/tGyt8sZ2gTNg+vJy4KLK80t6
NB3jIgncYRSrybs+Zn6eCs+I5u3BOQSsTTs6R2HGnk/eYlOI4t8=
=2KjD
-----END PGP SIGNATURE-----

C
C
Christopher Baines wrote on 13 Mar 2018 20:17
[PATCH 2/2] services: Add elasticsearch.
(address . 30803@debbugs.gnu.org)
20180313191727.24508-2-mail@cbaines.net
---
gnu/services/databases.scm | 107 +++++++++++++++++++++++++++++++++++++++++++++
gnu/tests/databases.scm | 57 +++++++++++++++++++++++-
2 files changed, 163 insertions(+), 1 deletion(-)

Toggle diff (199 lines)
diff --git a/gnu/services/databases.scm b/gnu/services/databases.scm
index 72927c453..cbb9e1699 100644
--- a/gnu/services/databases.scm
+++ b/gnu/services/databases.scm
@@ -51,6 +51,18 @@
postgresql-service
postgresql-service-type
+ <elasticsearch-configuration>
+ elasticsearch-configuration
+ elasticsearch-configuration?
+ elasticsearch-configuration-elasticsearch
+ elasticsearch-configuration-data-path
+ elasticsearch-configuration-logs-path
+ elasticsearch-configuration-port
+ elasticsearch-configuration-transport-port
+
+ elasticsearch-service
+ elasticsearch-service-type
+
memcached-service-type
<memcached-configuration>
memcached-configuration
@@ -260,6 +272,101 @@ and stores the database cluster in @var{data-directory}."
(data-directory data-directory))))
+;;;
+;;; Elasticsearch
+;;;
+
+(define-record-type* <elasticsearch-configuration>
+ elasticsearch-configuration make-elasticsearch-configuration
+ elasticsearch-configuration?
+ (elasticsearch elasticsearch-configuration-elasticsearch
+ (default elasticsearch))
+ (data-path elasticsearch-configuration-data-path
+ (default "/var/lib/"))
+ (logs-path elasticsearch-configuration-logs-path
+ (default "/var/log/elasticsearch"))
+ (http-port elasticsearch-configuration-port
+ (default 9200))
+ (transport-port elasticsearch-configuration-transport-port
+ (default 9300)))
+
+(define (elasticsearch-configuration-directory
+ data-path logs-path http-port transport-port)
+ (computed-file
+ "elasticsearch-config"
+ #~(begin
+ (mkdir #$output)
+ (mkdir (string-append #$output "/scripts"))
+ (call-with-output-file (string-append #$output "/elasticsearch.yml")
+ (lambda (port)
+ (display
+ (string-append
+ "path.data: " #$data-path "\n"
+ "path.logs: " #$logs-path "\n"
+ "http.port: " #$(number->string http-port) "\n"
+ "transport.tcp.port: " #$(number->string transport-port) "\n")
+ port))))))
+
+(define %elasticsearch-accounts
+ (list (user-group (name "elasticsearch") (system? #t))
+ (user-account
+ (name "elasticsearch")
+ (group "elasticsearch")
+ (system? #t)
+ (comment "Elasticsearch server user")
+ (home-directory "/var/empty")
+ (shell (file-append shadow "/sbin/nologin")))))
+
+(define elasticsearch-activation
+ (match-lambda
+ (($ <elasticsearch-configuration> elasticsearch data-path logs-path
+ http-port transport-port)
+ #~(begin
+ (use-modules (guix build utils)
+ (ice-9 match))
+
+ (let ((user (getpwnam "elasticsearch")))
+ ;; Create db state directory.
+ (for-each
+ (lambda (path)
+ (mkdir-p path)
+ (chown path (passwd:uid user) (passwd:gid user)))
+ '(#$data-path #$logs-path "/var/run/elasticsearch")))))))
+
+(define elasticsearch-shepherd-service
+ (match-lambda
+ (($ <elasticsearch-configuration> elasticsearch data-path logs-path
+ http-port transport-port)
+ (list (shepherd-service
+ (provision '(elasticsearch))
+ (documentation "Run the Elasticsearch daemon.")
+ (requirement '(user-processes syslogd))
+ (start #~(make-forkexec-constructor
+ (list
+ (string-append #$elasticsearch "/bin/elasticsearch")
+ "-d"
+ "-p" "/var/run/elasticsearch/pid"
+ (string-append
+ "-Dpath.conf="
+ #$(elasticsearch-configuration-directory
+ data-path logs-path http-port transport-port)))
+ #:user "elasticsearch"
+ #:pid-file "/var/run/elasticsearch/pid"
+ #:log-file "/var/log/elasticsearch.log"))
+ (stop #~(make-kill-destructor)))))))
+
+(define elasticsearch-service-type
+ (service-type (name 'elasticsearch)
+ (extensions
+ (list (service-extension shepherd-root-service-type
+ elasticsearch-shepherd-service)
+ (service-extension activation-service-type
+ elasticsearch-activation)
+ (service-extension account-service-type
+ (const %elasticsearch-accounts))))
+ (default-value (elasticsearch-configuration))))
+
+
;;;
;;; Memcached
;;;
diff --git a/gnu/tests/databases.scm b/gnu/tests/databases.scm
index 5c8ca85c1..e07a1f9dd 100644
--- a/gnu/tests/databases.scm
+++ b/gnu/tests/databases.scm
@@ -31,7 +31,8 @@
#:export (%test-memcached
%test-mongodb
%test-postgresql
- %test-mysql))
+ %test-mysql
+ %test-elasticsearch))
(define %memcached-os
(simple-operating-system
@@ -319,3 +320,57 @@
(name "mysql")
(description "Start the MySQL service.")
(value (run-mysql-test))))
+
+
+;;;
+;;; The Elasticsearch service.
+;;;
+
+(define %elasticsearch-os
+ (simple-operating-system
+ (service elasticsearch-service-type)))
+
+(define (run-elasticsearch-test)
+ "Run tests in %ELASTICSEARCH-OS."
+ (define os
+ (marionette-operating-system
+ %elasticsearch-os
+ #:imported-modules '((gnu services herd)
+ (guix combinators))))
+
+ (define vm
+ (virtual-machine
+ (operating-system os)
+ (memory-size 512)))
+
+ (define test
+ (with-imported-modules '((gnu build marionette))
+ #~(begin
+ (use-modules (srfi srfi-64)
+ (gnu build marionette))
+
+ (define marionette
+ (make-marionette (list #$vm)))
+
+ (mkdir #$output)
+ (chdir #$output)
+
+ (test-begin "elasticsearch")
+
+ (test-assert "service running"
+ (marionette-eval
+ '(begin
+ (use-modules (gnu services herd))
+ (start-service 'elasticsearch))
+ marionette))
+
+ (test-end)
+ (exit (= (test-runner-fail-count (test-runner-current)) 0)))))
+
+ (gexp->derivation "elasticsearch-test" test))
+
+(define %test-elasticsearch
+ (system-test
+ (name "elasticsearch")
+ (description "Start the Elasticsearch service.")
+ (value (run-elasticsearch-test))))
--
2.16.2
C
C
Christopher Baines wrote on 13 Mar 2018 20:17
[PATCH 1/2] gnu: Add elasticsearch.
(address . 30803@debbugs.gnu.org)
20180313191727.24508-1-mail@cbaines.net
* gnu/packages/databases.scm (elasticsearch-2.4.6, elasticsearch): New
variables.
---
gnu/packages/databases.scm | 64 ++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 64 insertions(+)

Toggle diff (84 lines)
diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index 87f65404f..661ee3a3e 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -69,6 +69,7 @@
#:use-module (gnu packages guile)
#:use-module (gnu packages time)
#:use-module (gnu packages golang)
+ #:use-module (gnu packages java)
#:use-module (gnu packages jemalloc)
#:use-module (gnu packages language)
#:use-module (gnu packages libevent)
@@ -355,6 +356,69 @@ ElasticSearch server")
(home-page "https://github.com/patientslikeme/es_dump_restore")
(license license:expat)))
+(define-public elasticsearch-2.4.6
+ (package
+ (name "elasticsearch")
+ (version "2.4.6")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (string-append
+ "https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-"
+ version ".tar.gz"))
+ (sha256
+ (base32 "0vzw9kpyyyv3f1m5sy9zara6shc7xkgi5xm5qbzvfywijavlnzjz"))))
+ (build-system gnu-build-system)
+ (inputs
+ `(("jre" ,icedtea)
+ ("coreutils" ,coreutils)
+ ("inetutils" ,inetutils)
+ ("util-linux" ,util-linux)
+ ("grep" ,grep)))
+ (arguments
+ `(#:phases
+ (modify-phases %standard-phases
+ (delete 'check)
+ (delete 'configure)
+ (delete 'build)
+ (replace 'install
+ (lambda* (#:key inputs outputs #:allow-other-keys)
+ (let ((out (assoc-ref outputs "out")))
+ (for-each
+ (lambda (dir)
+ (copy-recursively dir (string-append out "/" dir)
+ #:log (%make-void-port "w")))
+ '("bin" "config" "lib" "modules"))
+ (for-each
+ (lambda (dir)
+ (mkdir (string-append out "/" dir)))
+ '("plugins"))
+ (for-each
+ delete-file
+ (find-files
+ (string-append out "/lib")
+ (lambda (name stat)
+ (or (string-contains name "freebsd")
+ (string-contains name "solaris")))))
+ (wrap-program
+ (string-append out "/bin/elasticsearch")
+ `("PATH" = (,(string-append (assoc-ref inputs "util-linux")
+ "/bin")
+ ,(string-append (assoc-ref inputs "coreutils")
+ "/bin")
+ ,(string-append (assoc-ref inputs "inetutils")
+ "/bin")
+ ,(string-append (assoc-ref inputs "grep")
+ "/bin")))
+ `("JAVA_HOME" = (,(assoc-ref inputs "jre"))))
+ #t))))))
+ (home-page "")
+ (synopsis "")
+ (description "")
+ (license "")))
+
+(define-public elasticsearch elasticsearch-2.4.6)
+
(define-public leveldb
(package
(name "leveldb")
--
2.16.2
L
L
Ludovic Courtès wrote on 17 Mar 2018 22:06
Re: [bug#30803] [PATCH 0/2] Add Elasticsearch package and service.
(name . Christopher Baines)(address . mail@cbaines.net)(address . 30803@debbugs.gnu.org)
874lleo1j1.fsf@gnu.org
Hello Christopher,

Christopher Baines <mail@cbaines.net> skribis:

Toggle quote (3 lines)
> I'm trying to reduce the number of patches I have sitting around in git
> branches, so here is a bug about Elasticsearch.

Cool, thanks for sharing!

Toggle quote (4 lines)
> I've had a really awful package definition for this for a while, which
> just makes the built things in the release tarball run. While this isn't
> suitable for inclusion in to Guix, it does still work.

Do you know if we’re missing a lot of the Java dependencies to build it
from source?

Ludo’.
C
C
Christopher Baines wrote on 17 Mar 2018 23:36
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 30803@debbugs.gnu.org)
87woyaxrcc.fsf@cbaines.net
Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (16 lines)
> Hello Christopher,
>
> Christopher Baines <mail@cbaines.net> skribis:
>
>> I'm trying to reduce the number of patches I have sitting around in git
>> branches, so here is a bug about Elasticsearch.
>
> Cool, thanks for sharing!
>
>> I've had a really awful package definition for this for a while, which
>> just makes the built things in the release tarball run. While this isn't
>> suitable for inclusion in to Guix, it does still work.
>
> Do you know if we’re missing a lot of the Java dependencies to build it
> from source?

Unfortunately not, I haven't attempted to build it from source.
-----BEGIN PGP SIGNATURE-----

iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAlqtmFNfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF
ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE
9Xdejg//ZoQiD5q0FjYUcpAWcGRaxh4YnkGwgnxOpm+HICgoNApdP/ZPN2baHYTE
7TpJGF+4IYGYoYdhPujxFHkzXxrwQzpvxPJbsb+5jutbThqfoQuO5FQCXUDP866d
IHiBKD3Kx+I0+FUwdvYElNI3ELxj/e/Sghgu1WESEsgmCUlWq2F+3tGAIBjGzOYJ
4rCaz9jzt0a4i3NxckPFFaTyhP22EpT4oWf5Q34RNelr8jaGwqgKbHxYWEVvrAiQ
c9P+zBz0FfDa7KVu9pkNljw321b6iJeOsK8lmpyUpIv3EfcH8eFAp34X11gzZtNG
M+jfcBfAq4GAFwCAqeOtGpeBYM9PC9KuVI8aOXxAWjabkDwORc5hYUdNeoN0+/z3
nWYYUj/yfc16Y9OpYRBx2nhmiM4AvJkZydCTMVU5Ie3cKQfK9TdDghyyKuFeHssb
P5+5M3zDWr7Mv8TXcpPoV9fBhMoE+1lzu7DMHEpuifgyd1kp0g7aDnMrp/8sYQDN
8qu6/L54jCL7GZOXJs9ZACwT/2BvGzDyl5UD/ZHr/cAj1DkZlWz5E2VRx0Mmj9x7
td2bwcC5zx4iNIoBf4acN/Ibwjx0wEp+33+NXkRw8MjWIFgg94cwM9owxnUeHzyL
A/lNenD+zp6Eu3+iTgCkmhLHOjtZXh4FFksHtQ7rfy1mK9xCs58=
=NGxi
-----END PGP SIGNATURE-----

Z
Z
zimoun wrote on 17 Sep 2022 12:41
Re: bug#30803: [PATCH 0/2] Add Elasticsearch package and service.
(name . Christopher Baines)(address . mail@cbaines.net)(address . 30803@debbugs.gnu.org)
86pmfu9tbl.fsf@gmail.com
Hi,

On Tue, 13 Mar 2018 at 19:12, Christopher Baines <mail@cbaines.net> wrote:

Toggle quote (2 lines)
> Also included is a service, along with a really simple test.

What about these patches?

Thanks,
simon

PS: From 10 Years event.
M
M
Maxim Cournoyer wrote on 21 Jul 2023 18:55
(name . zimoun)(address . zimon.toutoune@gmail.com)
87bkg5i4qt.fsf_-_@gmail.com
Hello,

zimoun <zimon.toutoune@gmail.com> writes:

Toggle quote (13 lines)
> Hi,
>
> On Tue, 13 Mar 2018 at 19:12, Christopher Baines <mail@cbaines.net> wrote:
>
>> Also included is a service, along with a really simple test.
>
> What about these patches?
>
> Thanks,
> simon
>
> PS: From 10 Years event.

I'm closing this stale patch issue (there was an issue about Java
binaries being required IIUC).

--
Thanks,
Maxim
Closed
?