欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页  >  IT编程

linux下ElasticSearch.6.2.2集群安装与head、Kibana、X-Pack..插件的配置安装

程序员文章站 2024-01-25 19:26:17
简介:主要在三台linux服务器上,集群安装ElasticSearch.6.2.1,及其es的插件,各种管理软件 1.集群安装es 1.1 环境 域名...

简介:主要在三台linux服务器上,集群安装ElasticSearch.6.2.1,及其es的插件,各种管理软件

1.集群安装es

1.1 环境

域名          ip
biluos.com      192.168.10.173
biluos1.com     192.168.10.174
biluos2.com     192.168.10.175

1.2 三台机器都安装jdk最新版本

[root@biluos es]# java -version
openjdk version "1.8.0_161"
OpenJDK Runtime Environment (build 1.8.0_161-b14)
OpenJDK 64-Bit Server VM (build 25.161-b14, mixed mode)
[root@biluos es]# 

[root@biluos1 ~]# java -version
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
[root@biluos1 ~]# 

[root@biluos2 ~]# java -version
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
[root@biluos2 ~]# 

1.3 三台机器都统一用户为es

[root@biluos ~]# useradd es
You have new mail in /var/spool/mail/root
[root@biluos ~]# passwd es                                 # 密码为 qwaszx
Changing password for user es.
New password: 
BAD PASSWORD: it is based on a dictionary word
BAD PASSWORD: is too simple
Retype new password: 
passwd: all authentication tokens updated successfully.
[root@biluos ~]#  mkdir /home/es
mkdir: cannot create directory `/home/es': File exists
[root@biluos ~]#  ll /home/                                    # 注意是不是es用户和用户组 
total 4
drwx------ 3 es es 4096 Feb 25 03:51 es
[root@biluos ~]# 

[root@biluos1 ~]# useradd es
[root@biluos1 ~]# passwd es                                    # 密码为 qwaszx
[root@biluos1 ~]#  mkdir /home/es
[root@biluos1 ~]#  ll /home/                               # 注意是不是es用户和用户组 


[root@biluos2 ~]# useradd es
[root@biluos2 ~]# passwd es                                    # 密码为 qwaszx
[root@biluos2 ~]#  mkdir /home/es
[root@biluos2 ~]#  ll /home/                               # 注意是不是es用户和用户组 

1.4 三台机器都建立/home/es/elasticsearch目录,用来存放es软件包和数据存储,使用es用户

[root@biluos1 ~]# su es
[es@biluos1 root]$ cd /home/es/
[es@biluos1 ~]$ mkdir elasticsearch
[es@biluos1 ~]$ ll
drwxrwxr-x 2 es es 4096 Feb 25 03:59 elasticsearch
[es@biluos1 ~]$ 

其余两台此处省略

1.5 三台机器都解压安装包到/home/es/elasticsearch

下载:https://www.elastic.co/cn/downloads/elasticsearch
包:elasticsearch-6.2.2.tar.gz
解压:tar -zxvf /home/es/elasticsearch/elasticsearch-6.2.2.tar.gz -C /home/es/elasticsearch

[root@biluos elasticsearch-6.2.0]# pwd
/home/es/elasticsearch/elasticsearch-6.2.0

[es@biluos1 elasticsearch-6.2.0]$ pwd
/home/es/elasticsearch/elasticsearch-6.2.0

[es@biluos2 elasticsearch-6.2.0]$ pwd
/home/es/elasticsearch/elasticsearch-6.2.0

1.6 三台机器都修改es软件包的权限为es用户

因为我是root解压的

[es@biluos2 elasticsearch]$ ll
total 4
drwxr-xr-x 9 root root 4096 Feb 25 04:04 elasticsearch-6.2.0
[es@biluos2 elasticsearch]$ 

使用root用户修改权限

[es@biluos2 ~]$ su root
Password: 
[root@biluos2 es]# chown -R es:es /home/es/

其余两台此处省略

1.7 三台机器都创建data数据目录和日志目录,使用es用户

[root@biluos2 es]# su es
[es@biluos2 ~]$ mkdir -p /home/es/elasticsearch/elasticsearch-6.2.0_data/data/
[es@biluos2 ~]$ mkdir -p /home/es/elasticsearch/elasticsearch-6.2.0_data/logs/

其余两台此处省略

1.8 三台机器都修改配置

191.168.10.173机器配置
vim /home/es/elasticsearch/elasticsearch-6.2.2/config/elasticsearch.yml

cluster.name: lcc-application                                                           # 必须一样
node.name: node-191-168-10-173                                                          # 必须不一样
path.data: /home/es/elasticsearch/elasticsearch-6.2.0_data/data/
path.logs: /home/es/elasticsearch/elasticsearch-6.2.0_data/logs/

bootstrap.memory_lock: false
bootstrap.system_call_filter: false


network.host: 0.0.0.0  
http.port: 9200

# 集群发现
#集群节点ip或者主机
discovery.zen.ping.unicast.hosts: ["192.168.10.173", "192.168.10.174","192.168.10.175"]     
#设置这个参数来保证集群中的节点可以知道其它N个有master资格的节点。默认为1,对于大的集群来说,可以设置大一点的值(2-4)                                                                         
discovery.zen.minimum_master_nodes: 3  

# 下面两行配置为haad插件配置,三台服务器一致。      
http.cors.enabled: true                                                                                                                                                                                                   
http.cors.allow-origin: "*"      

191.168.10.174机器配置
vim /home/es/elasticsearch/elasticsearch-6.2.2/config/elasticsearch.yml

cluster.name: lcc-application                                                           # 必须一样
node.name: node-191-168-10-174                                                          # 必须不一样
path.data: /home/es/elasticsearch/elasticsearch-6.2.0_data/data/
path.logs: /home/es/elasticsearch/elasticsearch-6.2.0_data/logs/

bootstrap.memory_lock: false
bootstrap.system_call_filter: false


network.host: 0.0.0.0  
http.port: 9200

# 集群发现
#集群节点ip或者主机
discovery.zen.ping.unicast.hosts: ["192.168.10.173", "192.168.10.174","192.168.10.175"]     
#设置这个参数来保证集群中的节点可以知道其它N个有master资格的节点。默认为1,对于大的集群来说,可以设置大一点的值(2-4)                                                                         
discovery.zen.minimum_master_nodes: 3  

# 下面两行配置为haad插件配置,三台服务器一致。      
http.cors.enabled: true                                                                                                                                                                                                   
http.cors.allow-origin: "*"      

191.168.10.175机器配置
vim /home/es/elasticsearch/elasticsearch-6.2.2/config/elasticsearch.yml

cluster.name: lcc-application                                                           # 必须一样
node.name: node-191-168-10-175                                                          # 必须不一样
path.data: /home/es/elasticsearch/elasticsearch-6.2.0_data/data/
path.logs: /home/es/elasticsearch/elasticsearch-6.2.0_data/logs/

bootstrap.memory_lock: false
bootstrap.system_call_filter: false


network.host: 0.0.0.0  
http.port: 9200

# 集群发现
#集群节点ip或者主机
discovery.zen.ping.unicast.hosts: ["192.168.10.173", "192.168.10.174","192.168.10.175"]     
#设置这个参数来保证集群中的节点可以知道其它N个有master资格的节点。默认为1,对于大的集群来说,可以设置大一点的值(2-4)                                                                         
discovery.zen.minimum_master_nodes: 3  

# 下面两行配置为haad插件配置,三台服务器一致。      
http.cors.enabled: true                                                                                                                                                                                                   
http.cors.allow-origin: "*"      

1.9 三台机器都修改 Linux下/etc/security/limits.conf文件设置

更改linux的最大文件描述限制要求
添加或修改如下:
* soft nofile 262144
* hard nofile 262144 更改linux的锁内存限制要求
添加或修改如下:
用户(elasticsearch) soft memlock unlimited
用户(elasticsearch) hard memlock unlimited

最后配置如下

vim /etc/security/limits.conf

* soft nofile 262144                                                                                                                                              
* hard nofile 262144                                                                                                                                              
es soft memlock unlimited                                                                                                                                         
es hard memlock unlimited 

这里es是es用户

1.10 三台机器都修改配置 Linux下/etc/security/limits.d/90-nproc.conf文件设置

更改linux的的最大线程数,添加或修改如下:
* soft nproc unlimited

vim /etc/security/limits.d/90-nproc.conf
*          soft    nproc     unlimited
root       soft    nproc     unlimited

1.11 三台机器都修改配置 Linux下/etc/sysctl.conf文件设置

更改linux一个进行能拥有的最多的内存区域要求,添加或修改如下:
vm.max_map_count = 262144 更改linux禁用swapping,添加或修改如下:
vm.swappiness = 1
vim /etc/sysctl.conf

vm.max_map_count = 262144
vm.swappiness = 1

1.12 三台机器都启动

[es@biluos elasticsearch-6.2.2]$  bin/elasticsearch

[es@biluos1 elasticsearch-6.2.2]$  bin/elasticsearch

[es@biluos2 elasticsearch-6.2.2]$  bin/elasticsearch

刚刚报错,

[2018-02-25T13:49:33,967][WARN ][o.e.d.z.ZenDiscovery     ] [node-191-168-10-173] not enough master nodes discovered during pinging (found [[Candidate{node={node-191-168-10-173}{Wquj4gYuTteo7M5uXzSBiQ}{19L-SAvFRvqvYyjsKtBP_g}{192.168.1.194}{192.168.1.194:9300}, clusterStateVersion=-1}]], but needed [3]), pinging again

我以为是因为没配置ssh,最后发现上面ip写错了,然后验证不配置ssh也没事。

这上面第一个启动的打印日志最多,其他的相同,日志比较少

现在有个问题,我访问
https://192.168.10.173:9200/
https://192.168.10.174:9200/
https://192.168.10.175:9200/
都能打印出版本号,证明都能正确运行,我尝试关闭掉一个174,发现https://192.168.10.174:9200/打不开了,然后,我觉得应该有一个是主master,但是谁是主呢?不好看到,下面我们安装图形化界面head试试

2 安装elasticsearch-head

三台机器只需要一台安装head就可以了

2.1 采用nodejs安装

6.X中,elasticsearch-head

不能放在elasticsearch的 plugins、modules 目录下 不能使用 elasticsearch-plugin install

直接启动elasticsearch即可

2.2 安装 elasticsearch-head

1.修改 elasticsearch/config/elasticsearch.yml
2.添加

http.cors.enabled: true
http.cors.allow-origin: "*"

3.下载 elasticsearch-head 或者 git clone 到随便一个文件夹

https://github.com/mobz/elasticsearch-head

我是直接下载到

[es@biluos elasticsearch-head-master]$ pwd
/home/es/elasticsearch-head-master

目录的

4.安装nodejs
https://blog.csdn.net/qq_21383435/article/details/79367366

5.cd /home/es/elasticsearch-head-master
6.npm install -g grunt-cli
grunt是一个很方便的构建工具,可以进行打包压缩、测试、执行等等的工作

[es@biluos elasticsearch-head-master]$ npm install -g grunt-cli 
npm WARN checkPermissions Missing write access to /opt/moudles/node-v8.9.4-linux-x64/lib/node_modules
npm ERR! path /opt/moudles/node-v8.9.4-linux-x64/lib/node_modules
npm ERR! code EACCES
npm ERR! errno -13
npm ERR! syscall access
npm ERR! Error: EACCES: permission denied, access '/opt/moudles/node-v8.9.4-linux-x64/lib/node_modules'
npm ERR!  { Error: EACCES: permission denied, access '/opt/moudles/node-v8.9.4-linux-x64/lib/node_modules'
npm ERR!   stack: 'Error: EACCES: permission denied, access \'/opt/moudles/node-v8.9.4-linux-x64/lib/node_modules\'',
npm ERR!   errno: -13,
npm ERR!   code: 'EACCES',
npm ERR!   syscall: 'access',
npm ERR!   path: '/opt/moudles/node-v8.9.4-linux-x64/lib/node_modules' }
npm ERR! 
npm ERR! Please try running this command again as root/Administrator.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/es/.npm/_logs/2018-02-25T02_49_37_372Z-debug.log

一看就知道是权限问题,因为我的nodejs是root安装的,这里我是es用户

[es@biluos elasticsearch-head-master]$ su root
Password: 
[root@biluos elasticsearch-head-master]#  npm install -g grunt-cli 
/opt/moudles/node-v8.9.4-linux-x64/bin/grunt -> /opt/moudles/node-v8.9.4-linux-x64/lib/node_modules/grunt-cli/bin/grunt
+ grunt-cli@1.2.0
added 16 packages in 21.663s

7.npm install

[root@biluos elasticsearch-head-master]# npm install 
npm WARN deprecated http2@3.3.7: Use the built-in module in node 9.0.0 or newer, instead
npm WARN deprecated coffee-script@1.10.0: CoffeeScript on NPM has moved to "coffeescript" (no hyphen)

> phantomjs-prebuilt@2.1.16 install /home/es/elasticsearch-head-master/node_modules/phantomjs-prebuilt
> node install.js

module.js:540
    throw err;
    ^

Error: Cannot find module '/home/es/elasticsearch-head-master/node_modules/phantomjs-prebuilt/install.js'
    at Function.Module._resolveFilename (module.js:538:15)
    at Function.Module._load (module.js:468:25)
    at Function.Module.runMain (module.js:684:10)
    at startup (bootstrap_node.js:187:16)
    at bootstrap_node.js:608:3
npm WARN elasticsearch-head@0.0.0 license should be a valid SPDX license expression
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.1.3 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.1.3: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! phantomjs-prebuilt@2.1.16 install: `node install.js`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the phantomjs-prebuilt@2.1.16 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2018-02-25T03_01_11_887Z-debug.log

这一点是因为我的elasticsearch是es用户,而elasticsearch-head-master文件是root用户

[root@biluos es]# ll
total 12
drwxr-xr-x 4 es   es   4096 Feb  9 14:55 elasticsearch

[root@biluos elasticsearch-head-master]# ll /home/es/elasticsearch-head-master/node_modules
total 1028
drwxr-xr-x  2 root root 4096 Feb 25 11:01 abbrev
drwxr-xr-x  2 root root 4096 Feb 25 11:01 accepts
drwxr-xr-x  3 root root 4096 Feb 25 11:01 after
drwxr-xr-x  2 root root 4096 Feb 25 11:01 ansi-regex
drwxr-xr-x  2 root root 4096 Feb 25 11:01 ansi-styles

这里我统一用户为es

[root@biluos es]# chown -R es:es /home/es/elasticsearch-head-master/

然后使用es用户,安装成功

[root@biluos elasticsearch-head-master]# su es 
[es@biluos elasticsearch-head-master]$ npm install 

> phantomjs-prebuilt@2.1.16 install /home/es/elasticsearch-head-master/node_modules/phantomjs-prebuilt
> node install.js

PhantomJS not found on PATH
Downloading https://github.com/Medium/phantomjs/releases/download/v2.1.1/phantomjs-2.1.1-linux-x86_64.tar.bz2
Saving to /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2
Receiving...
                                                                                                                                                                                               [=============---------------------------] 33                                                                                                                                                   [===================================-----] 87%
Received 22866K total.
Extracting tar contents (via spawned process)
Removing /home/es/elasticsearch-head-master/node_modules/phantomjs-prebuilt/lib/phantom
Copying extracted folder /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2-extract-1519527928056/phantomjs-2.1.1-linux-x86_64 -> /home/es/elasticsearch-head-master/node_modules/phantomjs-prebuilt/lib/phantom
Writing location.js file
Done. Phantomjs binary available at /home/es/elasticsearch-head-master/node_modules/phantomjs-prebuilt/lib/phantom/bin/phantomjs
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN elasticsearch-head@0.0.0 license should be a valid SPDX license expression
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.1.3 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.1.3: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

added 70 packages in 84.93s
[es@biluos elasticsearch-head-master]$ 

8.grunt server

[es@biluos elasticsearch-head-master]$ grunt server
(node:6612) ExperimentalWarning: The http2 module is an experimental API.
Running "connect:server" (connect) task
Waiting forever...
Started connect web server on https://localhost:9100

9.https://192.168.10.173:9100/ 成功

10.启动我们的elasticsearch

[es@biluos elasticsearch-6.2.2]$  bin/elasticsearch

[es@biluos1 elasticsearch-6.2.2]$  bin/elasticsearch

[es@biluos2 elasticsearch-6.2.2]$  bin/elasticsearch

11 .https://192.168.10.173:9100/ 查看

linux下ElasticSearch.6.2.2集群安装与head、Kibana、X-Pack..插件的配置安装

2.3 测试ctrl+c掉一个

我直接ctrl+c一个174会是什么结果呢?,发现173和175都报错

[2018-02-25T14:54:08,997][WARN ][o.e.d.z.ZenDiscovery     ] [node-191-168-10-173] not enough master nodes discovered during pinging (found [[Candidate{node={node-191-168-10-175}{dCejCwUFTnqOrYyrmfD4AQ}{y2SixREcR5STrzmlswaxcA}{192.168.10.175}{192.168.10.175:9300}, clusterStateVersion=8}, Candidate{node={node-191-168-10-173}{Wquj4gYuTteo7M5uXzSBiQ}{xzN6saH5Rp2CPQH5iunHZg}{192.168.1.194}{192.168.1.194:9300}, clusterStateVersion=8}]], but needed [3]), pinging again

而且https://192.168.10.173:9100/无法连接,连接也是红色的

感觉:应该和这个参数有关

## 设置这个参数来保证集群中的节点可以知道其它N个有master资格的节点。默认为1,对于大的集群来说,可以设置大一点的值(2-4)                                                                         
discovery.zen.minimum_master_nodes: 2

这个的含义是不是 我假设集群有3个节点 ,如果死掉了一个我的集群还能用?如果在死掉一个,节点还剩下一个了,小于2 是不是就不能用了?

discovery.zen.minimum_master_nodes: 5
是不是我集群可用的最少节点数限制


参数是最小可用的候选master节点数
一般设置为2,集群里留3个节点作为候选master

3 安装kibana(不需要安装x-pack)

集群的时候,我们只需要在一台机器上安装就可以了,
我开始安装kibana-6.2.1,但是报错如下,后来安装同样版本的kibana-6.2.2,就不报错了

log   [07:15:33.047] [warning] You're running Kibana 6.2.1 with some different versions of Elasticsearch. Update Kibana or Elasticsearch to the same version to prevent compatibility issues: v6.2.2 @ 192.168.10.174:9200 (192.168.10.174), v6.2.2 @ 192.168.1.194:9200 (192.168.1.194), v6.2.2 @ 192.168.10.175:9200 (192.168.10.175)                                                                                                                                                                                            

1.下载https://www.elastic.co/cn/downloads/kibana 版本要和es版本相同
2.直接解压到 /home/es/kibana/kibana-6.2.2-linux-x86_64
3.配置

# vim config/kibana.yml
elasticsearch.url: "https://192.168.10.173:9200"      # kibana监控哪台es机器
server.host: "192.168.10.173"                # kibana运行在哪台机器

4.运行

[root@biluos kibana-6.2.2-linux-x86_64]# bin/kibana            
  log   [07:54:52.807] [info][status][plugin:kibana@6.2.2] Status changed from uninitialized to green - Ready
  log   [07:54:52.882] [info][status][plugin:elasticsearch@6.2.2] Status changed from uninitialized to yellow - Waiting for Elasticsearch
  log   [07:54:53.091] [info][status][plugin:timelion@6.2.2] Status changed from uninitialized to green - Ready
  log   [07:54:53.107] [info][status][plugin:console@6.2.2] Status changed from uninitialized to green - Ready
  log   [07:54:53.116] [info][status][plugin:metrics@6.2.2] Status changed from uninitialized to green - Ready
  log   [07:54:53.152] [info][listening] Server running at https://192.168.10.173:5601
  log   [07:54:53.197] [info][status][plugin:elasticsearch@6.2.2] Status changed from yellow to green - Ready

5.查看界面https://192.168.10.173:5601 可以直接访问