Skip to content

Commit 8c73cbe

Browse files
pirespires
committed
Upgraded to Elasticsearch 2.1.0.
1 parent b2e8f03 commit 8c73cbe

File tree

4 files changed

+67
-66
lines changed

4 files changed

+67
-66
lines changed

‎README.md‎

Lines changed: 64 additions & 63 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
# kubernetes-elasticsearch-cluster
2-
Elasticsearch (2.0.0) cluster on top of Kubernetes made easy.
2+
Elasticsearch (2.1.0) cluster on top of Kubernetes made easy.
33

44
Elasticsearch best-practices recommend to separate nodes in three roles:
55
* `Master` nodes - intended for clustering management only, no data, no HTTP API
@@ -12,7 +12,7 @@ Given this, I'm hereby making possible for you to scale as needed. For instance,
1212

1313
## Pre-requisites
1414

15-
* Kubernetes cluster (tested with v1.1.1 on top of [Vagrant + CoreOS](https://github.com/pires/kubernetes-vagrant-coreos-cluster))
15+
* Kubernetes cluster (tested with v1.1.2 on top of [Vagrant + CoreOS](https://github.com/pires/kubernetes-vagrant-coreos-cluster))
1616
* `kubectl` configured to access your cluster master API Server
1717

1818
## Build images (optional)
@@ -46,43 +46,43 @@ Now, I leave up to you how to validate the cluster, but a first step is to wait
4646

4747
```
4848
$ kubectl get svc,rc,pods
49-
NAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGE
50-
elasticsearch 10.100.13.117 9200/TCP component=elasticsearch,role=client 29m
51-
elasticsearch-discovery 10.100.232.72 <none> 9300/TCP component=elasticsearch,role=master 29m
52-
kube-dns 10.100.0.10 <none> 53/UDP,53/TCP k8s-app=kube-dns 50m
53-
kubernetes 10.100.0.1 <none> 443/TCP <none> 53m
49+
NAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGE
50+
elasticsearch 10.100.116.17 9200/TCP component=elasticsearch,role=client 11m
51+
elasticsearch-discovery 10.100.152.227 <none> 9300/TCP component=elasticsearch,role=master 11m
52+
kube-dns 10.100.0.10 <none> 53/UDP,53/TCP k8s-app=kube-dns 46m
53+
kubernetes 10.100.0.1 <none> 443/TCP <none> 46m
5454
CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS AGE
55-
es-client es-client quay.io/pires/docker-elasticsearch-kubernetes:2.0.0 component=elasticsearch,role=client 1 7m
56-
es-data es-data quay.io/pires/docker-elasticsearch-kubernetes:2.0.0 component=elasticsearch,role=data 1 7m
57-
es-master es-master quay.io/pires/docker-elasticsearch-kubernetes:2.0.0 component=elasticsearch,role=master 1 29m
58-
kube-dns etcd gcr.io/google_containers/etcd:2.0.9 k8s-app=kube-dns 1 50m
55+
es-client es-client quay.io/pires/docker-elasticsearch-kubernetes:2.1.0 component=elasticsearch,role=client 1 8m
56+
es-data es-data quay.io/pires/docker-elasticsearch-kubernetes:2.1.0 component=elasticsearch,role=data 1 3m
57+
es-master es-master quay.io/pires/docker-elasticsearch-kubernetes:2.1.0 component=elasticsearch,role=master 1 11m
58+
kube-dns etcd gcr.io/google_containers/etcd:2.0.9 k8s-app=kube-dns 1 46m
5959
kube2sky gcr.io/google_containers/kube2sky:1.11
6060
skydns gcr.io/google_containers/skydns:2015-03-11-001
6161
NAME READY STATUS RESTARTS AGE
62-
es-client-5lb50 1/1 Running 0 7m
63-
es-data-0x7e0 1/1 Running 0 7m
64-
es-master-qsovc 1/1 Running 0 29m
65-
kube-dns-vkv57 3/3 Running 0 50m
62+
es-client-suigi 1/1 Running 0 8m
63+
es-data-ygq6o 1/1 Running 0 3m
64+
es-master-n4ykt 1/1 Running 0 11m
65+
kube-dns-ac0jp 3/3 Running 0 46m
6666
```
6767

6868
```
69-
$ kubectl logs es-master-qsovc
69+
$ kubectl logs es-master-n4ykt
7070
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.
7171
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.
7272
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.
73-
[2015-11-19 20:26:13,193][INFO ][node ] [Doctor Spectrum] version[2.0.0], pid[10], build[de54438/2015-10-22T08:09:48Z]
74-
[2015-11-19 20:26:13,199][INFO ][node ] [Doctor Spectrum] initializing ...
75-
[2015-11-19 20:26:13,548][INFO ][plugins ] [Doctor Spectrum] loaded [cloud-kubernetes], sites []
76-
[2015-11-19 20:26:13,590][INFO ][env ] [Doctor Spectrum] using [1] data paths, mounts [[/data (/dev/sda9)]], net usable_space [14.2gb], net total_space [15.5gb], spins? [possibly], types [ext4]
77-
[2015-11-19 20:26:16,413][INFO ][node ] [Doctor Spectrum] initialized
78-
[2015-11-19 20:26:16,413][INFO ][node ] [Doctor Spectrum] starting ...
79-
[2015-11-19 20:26:16,602][INFO ][transport ] [Doctor Spectrum] publish_address {10.244.81.2:9300}, bound_addresses {10.244.81.2:9300}
80-
[2015-11-19 20:26:16,641][INFO ][discovery ] [Doctor Spectrum] myesdb/nm9NRSuXT7mfJ0YGy-CFFA
81-
[2015-11-19 20:26:21,391][INFO ][cluster.service ] [Doctor Spectrum] new_master {Doctor Spectrum}{nm9NRSuXT7mfJ0YGy-CFFA}{10.244.81.2}{10.244.81.2:9300}{data=false, master=true}, reason: zen-disco-join(elected_as_master, [0] joins received)
82-
[2015-11-19 20:26:21,408][INFO ][node ] [Doctor Spectrum] started
83-
[2015-11-19 20:26:21,443][INFO ][gateway ] [Doctor Spectrum] recovered [0] indices into cluster_state
84-
[2015-11-19 20:48:27,303][INFO ][cluster.service ] [Doctor Spectrum] added {{Termagaira}{rSv0h7NXRg2_Z1MY7fTvRg}{10.244.63.2}{10.244.63.2:9300}{master=false},}, reason: zen-disco-join(join from node[{Termagaira}{rSv0h7NXRg2_Z1MY7fTvRg}{10.244.63.2}{10.244.63.2:9300}{master=false}])
85-
[2015-11-19 20:48:31,288][INFO ][cluster.service ] [Doctor Spectrum] added {{War V}{Rq6n51RlRFyt1VIgG54byQ}{10.244.13.2}{10.244.13.2:9300}{data=false, master=false},}, reason: zen-disco-join(join from node[{War V}{Rq6n51RlRFyt1VIgG54byQ}{10.244.13.2}{10.244.13.2:9300}{data=false, master=false}])
73+
[2015-11-25 14:20:21,538][INFO ][node ] [Beta Ray Bill] version[2.1.0], pid[13], build[72cd1f1/2015-11-18T22:40:03Z]
74+
[2015-11-25 14:20:21,539][INFO ][node ] [Beta Ray Bill] initializing ...
75+
[2015-11-25 14:20:21,915][INFO ][plugins ] [Beta Ray Bill] loaded [cloud-kubernetes], sites []
76+
[2015-11-25 14:20:21,968][INFO ][env ] [Beta Ray Bill] using [1] data paths, mounts [[/data (/dev/sda9)]], net usable_space [14.2gb], net total_space [15.5gb], spins? [possibly], types [ext4]
77+
[2015-11-25 14:20:26,201][INFO ][node ] [Beta Ray Bill] initialized
78+
[2015-11-25 14:20:26,202][INFO ][node ] [Beta Ray Bill] starting ...
79+
[2015-11-25 14:20:26,645][INFO ][transport ] [Beta Ray Bill] publish_address {10.244.85.2:9300}, bound_addresses {10.244.85.2:9300}
80+
[2015-11-25 14:20:26,671][INFO ][discovery ] [Beta Ray Bill] myesdb/3exFpLDzRpWKpaI9Vhafyg
81+
[2015-11-25 14:20:31,714][INFO ][cluster.service ] [Beta Ray Bill] new_master {Beta Ray Bill}{3exFpLDzRpWKpaI9Vhafyg}{10.244.85.2}{10.244.85.2:9300}{data=false, master=true}, reason: zen-disco-join(elected_as_master, [0] joins received)
82+
[2015-11-25 14:20:31,717][INFO ][node ] [Beta Ray Bill] started
83+
[2015-11-25 14:20:31,774][INFO ][gateway ] [Beta Ray Bill] recovered [0] indices into cluster_state
84+
[2015-11-25 14:23:44,437][INFO ][cluster.service ] [Beta Ray Bill] added {{Demogoblin}{j5RheK6uTJ2kDXOs9QC9Tg}{10.244.66.2}{10.244.66.2:9300}{data=false, master=false},}, reason: zen-disco-join(join from node[{Demogoblin}{j5RheK6uTJ2kDXOs9QC9Tg}{10.244.66.2}{10.244.66.2:9300}{data=false, master=false}])
85+
[2015-11-25 14:28:34,418][INFO ][cluster.service ] [Beta Ray Bill] added {{Tethlam}{manY6pckTZeDQmeLyifD8Q}{10.244.101.2}{10.244.101.2:9300}{master=false},}, reason: zen-disco-join(join from node[{Tethlam}{manY6pckTZeDQmeLyifD8Q}{10.244.101.2}{10.244.101.2:9300}{master=false}])
8686
```
8787

8888
As you can assert, the cluster is up and running. Easy, wasn't it?
@@ -102,39 +102,39 @@ Did it work?
102102
```
103103
$ kubectl get pods
104104
NAME READY STATUS RESTARTS AGE
105-
es-client-44h9j 1/1 Running 0 6m
106-
es-client-5lb50 1/1 Running 0 40m
107-
es-data-0x7e0 1/1 Running 0 40m
108-
es-data-jidcs 1/1 Running 0 1m
109-
es-master-ic3fd 1/1 Running 0 8m
110-
es-master-qsovc 1/1 Running 0 1h
111-
es-master-vvuba 1/1 Running 0 8m
112-
kube-dns-vkv57 3/3 Running 0 1h
105+
es-client-jnnnv 1/1 Running 0 1m
106+
es-client-suigi 1/1 Running 0 11m
107+
es-data-2g2iv 1/1 Running 0 1m
108+
es-data-ygq6o 1/1 Running 0 6m
109+
es-master-n4ykt 1/1 Running 0 14m
110+
es-master-svkq1 1/1 Running 0 2m
111+
es-master-yre86 1/1 Running 0 2m
112+
kube-dns-ac0jp 3/3 Running 0 49m
113113
```
114114

115115
Let's take another look of the Elasticsearch master logs:
116116
```
117-
$ kubectl logs es-master-qsovc
117+
$ kubectl logs es-master-n4ykt
118118
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.
119119
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.
120120
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.
121-
[2015-11-19 20:26:13,193][INFO ][node ] [Doctor Spectrum] version[2.0.0], pid[10], build[de54438/2015-10-22T08:09:48Z]
122-
[2015-11-19 20:26:13,199][INFO ][node ] [Doctor Spectrum] initializing ...
123-
[2015-11-19 20:26:13,548][INFO ][plugins ] [Doctor Spectrum] loaded [cloud-kubernetes], sites []
124-
[2015-11-19 20:26:13,590][INFO ][env ] [Doctor Spectrum] using [1] data paths, mounts [[/data (/dev/sda9)]], net usable_space [14.2gb], net total_space [15.5gb], spins? [possibly], types [ext4]
125-
[2015-11-19 20:26:16,413][INFO ][node ] [Doctor Spectrum] initialized
126-
[2015-11-19 20:26:16,413][INFO ][node ] [Doctor Spectrum] starting ...
127-
[2015-11-19 20:26:16,602][INFO ][transport ] [Doctor Spectrum] publish_address {10.244.81.2:9300}, bound_addresses {10.244.81.2:9300}
128-
[2015-11-19 20:26:16,641][INFO ][discovery ] [Doctor Spectrum] myesdb/nm9NRSuXT7mfJ0YGy-CFFA
129-
[2015-11-19 20:26:21,391][INFO ][cluster.service ] [Doctor Spectrum] new_master {Doctor Spectrum}{nm9NRSuXT7mfJ0YGy-CFFA}{10.244.81.2}{10.244.81.2:9300}{data=false, master=true}, reason: zen-disco-join(elected_as_master, [0] joins received)
130-
[2015-11-19 20:26:21,408][INFO ][node ] [Doctor Spectrum] started
131-
[2015-11-19 20:26:21,443][INFO ][gateway ] [Doctor Spectrum] recovered [0] indices into cluster_state
132-
[2015-11-19 20:48:27,303][INFO ][cluster.service ] [Doctor Spectrum] added {{Termagaira}{rSv0h7NXRg2_Z1MY7fTvRg}{10.244.63.2}{10.244.63.2:9300}{master=false},}, reason: zen-disco-join(join from node[{Termagaira}{rSv0h7NXRg2_Z1MY7fTvRg}{10.244.63.2}{10.244.63.2:9300}{master=false}])
133-
[2015-11-19 20:48:31,288][INFO ][cluster.service ] [Doctor Spectrum] added {{War V}{Rq6n51RlRFyt1VIgG54byQ}{10.244.13.2}{10.244.13.2:9300}{data=false, master=false},}, reason: zen-disco-join(join from node[{War V}{Rq6n51RlRFyt1VIgG54byQ}{10.244.13.2}{10.244.13.2:9300}{data=false, master=false}])
134-
[2015-11-19 21:17:54,140][INFO ][cluster.service ] [Doctor Spectrum] added {{Crossbones}{IVfzPHf8QkS8EX68pEv9oQ}{10.244.63.3}{10.244.63.3:9300}{data=false, master=true},}, reason: zen-disco-join(join from node[{Crossbones}{IVfzPHf8QkS8EX68pEv9oQ}{10.244.63.3}{10.244.63.3:9300}{data=false, master=true}])
135-
[2015-11-19 21:17:54,281][INFO ][cluster.service ] [Doctor Spectrum] added {{Lady Lark}{TSoI37adQPaNpEDJ2c9bsw}{10.244.13.3}{10.244.13.3:9300}{data=false, master=true},}, reason: zen-disco-join(join from node[{Lady Lark}{TSoI37adQPaNpEDJ2c9bsw}{10.244.13.3}{10.244.13.3:9300}{data=false, master=true}])
136-
[2015-11-19 21:20:33,233][INFO ][cluster.service ] [Doctor Spectrum] added {{Ben Urich}{P0EqCsVGRhOI5IRAWH34yg}{10.244.81.3}{10.244.81.3:9300}{data=false, master=false},}, reason: zen-disco-join(join from node[{Ben Urich}{P0EqCsVGRhOI5IRAWH34yg}{10.244.81.3}{10.244.81.3:9300}{data=false, master=false}])
137-
[2015-11-19 21:25:41,480][INFO ][cluster.service ] [Doctor Spectrum] added {{Cassandra Nova}{DeElUqr2QdyifWwJU68hcw}{10.244.13.4}{10.244.13.4:9300}{master=false},}, reason: zen-disco-join(join from node[{Cassandra Nova}{DeElUqr2QdyifWwJU68hcw}{10.244.13.4}{10.244.13.4:9300}{master=false}])
121+
[2015-11-25 14:20:21,538][INFO ][node ] [Beta Ray Bill] version[2.1.0], pid[13], build[72cd1f1/2015-11-18T22:40:03Z]
122+
[2015-11-25 14:20:21,539][INFO ][node ] [Beta Ray Bill] initializing ...
123+
[2015-11-25 14:20:21,915][INFO ][plugins ] [Beta Ray Bill] loaded [cloud-kubernetes], sites []
124+
[2015-11-25 14:20:21,968][INFO ][env ] [Beta Ray Bill] using [1] data paths, mounts [[/data (/dev/sda9)]], net usable_space [14.2gb], net total_space [15.5gb], spins? [possibly], types [ext4]
125+
[2015-11-25 14:20:26,201][INFO ][node ] [Beta Ray Bill] initialized
126+
[2015-11-25 14:20:26,202][INFO ][node ] [Beta Ray Bill] starting ...
127+
[2015-11-25 14:20:26,645][INFO ][transport ] [Beta Ray Bill] publish_address {10.244.85.2:9300}, bound_addresses {10.244.85.2:9300}
128+
[2015-11-25 14:20:26,671][INFO ][discovery ] [Beta Ray Bill] myesdb/3exFpLDzRpWKpaI9Vhafyg
129+
[2015-11-25 14:20:31,714][INFO ][cluster.service ] [Beta Ray Bill] new_master {Beta Ray Bill}{3exFpLDzRpWKpaI9Vhafyg}{10.244.85.2}{10.244.85.2:9300}{data=false, master=true}, reason: zen-disco-join(elected_as_master, [0] joins received)
130+
[2015-11-25 14:20:31,717][INFO ][node ] [Beta Ray Bill] started
131+
[2015-11-25 14:20:31,774][INFO ][gateway ] [Beta Ray Bill] recovered [0] indices into cluster_state
132+
[2015-11-25 14:23:44,437][INFO ][cluster.service ] [Beta Ray Bill] added {{Demogoblin}{j5RheK6uTJ2kDXOs9QC9Tg}{10.244.66.2}{10.244.66.2:9300}{data=false, master=false},}, reason: zen-disco-join(join from node[{Demogoblin}{j5RheK6uTJ2kDXOs9QC9Tg}{10.244.66.2}{10.244.66.2:9300}{data=false, master=false}])
133+
[2015-11-25 14:28:34,418][INFO ][cluster.service ] [Beta Ray Bill] added {{Tethlam}{manY6pckTZeDQmeLyifD8Q}{10.244.101.2}{10.244.101.2:9300}{master=false},}, reason: zen-disco-join(join from node[{Tethlam}{manY6pckTZeDQmeLyifD8Q}{10.244.101.2}{10.244.101.2:9300}{master=false}])
134+
[2015-11-25 14:31:10,670][INFO ][cluster.service ] [Beta Ray Bill] added {{Anomalito}{H90Ozq1CTYO60oMOWd2VfA}{10.244.101.3}{10.244.101.3:9300}{data=false, master=true},}, reason: zen-disco-join(join from node[{Anomalito}{H90Ozq1CTYO60oMOWd2VfA}{10.244.101.3}{10.244.101.3:9300}{data=false, master=true}])
135+
[2015-11-25 14:31:10,939][INFO ][cluster.service ] [Beta Ray Bill] added {{Viper}{WtZLz13oRfSEDnweayWABw}{10.244.66.3}{10.244.66.3:9300}{data=false, master=true},}, reason: zen-disco-join(join from node[{Viper}{WtZLz13oRfSEDnweayWABw}{10.244.66.3}{10.244.66.3:9300}{data=false, master=true}])
136+
[2015-11-25 14:32:08,522][INFO ][cluster.service ] [Beta Ray Bill] added {{Necromantra}{z1BrCL1SRW23i8aID0l2rw}{10.244.85.3}{10.244.85.3:9300}{data=false, master=false},}, reason: zen-disco-join(join from node[{Necromantra}{z1BrCL1SRW23i8aID0l2rw}{10.244.85.3}{10.244.85.3:9300}{data=false, master=false}])
137+
[2015-11-25 14:32:10,712][INFO ][cluster.service ] [Beta Ray Bill] added {{Protector}{-PqouJYzTJK_AJj4uJJaQg}{10.244.85.4}{10.244.85.4:9300}{master=false},}, reason: zen-disco-join(join from node[{Protector}{-PqouJYzTJK_AJj4uJJaQg}{10.244.85.4}{10.244.85.4:9300}{master=false}])
138138
```
139139

140140
### Access the service
@@ -144,27 +144,28 @@ log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFil
144144
```
145145
$ kubectl get service elasticsearch
146146
NAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGE
147-
elasticsearch 10.100.13.117 9200/TCP component=elasticsearch,role=client 1h
147+
elasticsearch 10.100.116.17 9200/TCP component=elasticsearch,role=client 15m
148148
```
149149

150150
From any host on your cluster (that's running `kube-proxy`), run:
151151

152152
```
153-
curl http://10.100.13.117:9200
153+
curl http://10.100.116.17:9200
154154
```
155155

156156
You should see something similar to the following:
157157

158158
```json
159+
$ curl http://10.100.116.17:9200
159160
{
160-
"name" : "War V",
161+
"name" : "Demogoblin",
161162
"cluster_name" : "myesdb",
162163
"version" : {
163-
"number" : "2.0.0",
164-
"build_hash" : "de54438d6af8f9340d50c5c786151783ce7d6be5",
165-
"build_timestamp" : "2015-10-22T08:09:48Z",
164+
"number" : "2.1.0",
165+
"build_hash" : "72cd1f1a3eee09505e036106146dc1949dc5dc87",
166+
"build_timestamp" : "2015-11-18T22:40:03Z",
166167
"build_snapshot" : false,
167-
"lucene_version" : "5.2.1"
168+
"lucene_version" : "5.3.1"
168169
},
169170
"tagline" : "You Know, for Search"
170171
}
@@ -173,7 +174,7 @@ You should see something similar to the following:
173174
Or if you want to see cluster information:
174175

175176
```
176-
curl http://10.100.13.117:9200/_cluster/health?pretty
177+
curl http://10.100.116.17:9200/_cluster/health?pretty
177178
```
178179

179180
You should see something similar to the following:

‎es-client-rc.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ spec:
2424
capabilities:
2525
add:
2626
- IPC_LOCK
27-
image: quay.io/pires/docker-elasticsearch-kubernetes:2.0.0
27+
image: quay.io/pires/docker-elasticsearch-kubernetes:2.1.0
2828
env:
2929
- name: KUBERNETES_CA_CERTIFICATE_FILE
3030
value: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt

‎es-data-rc.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ spec:
2424
capabilities:
2525
add:
2626
- IPC_LOCK
27-
image: quay.io/pires/docker-elasticsearch-kubernetes:2.0.0
27+
image: quay.io/pires/docker-elasticsearch-kubernetes:2.1.0
2828
env:
2929
- name: KUBERNETES_CA_CERTIFICATE_FILE
3030
value: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt

‎es-master-rc.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ spec:
2424
capabilities:
2525
add:
2626
- IPC_LOCK
27-
image: quay.io/pires/docker-elasticsearch-kubernetes:2.0.0
27+
image: quay.io/pires/docker-elasticsearch-kubernetes:2.1.0
2828
env:
2929
- name: KUBERNETES_CA_CERTIFICATE_FILE
3030
value: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt

0 commit comments

Comments
 (0)