服务端安装
Elasticsearch和Kibana（需要安装openjdk1.8以上）
安装方法：
以Ubuntu为例：wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -sudo apt-get install apt-transport-httpsecho "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-6.x.listapt-get updateapt-get install elasticsearchapt-get install kibana
elasticsearch配置
cat /etc/elasticsearch/elasticsearch.yml
path.data: /var/lib/elasticsearchpath.logs: /var/log/elasticsearchnetwork.host: 172.16.80.28http.port: 9200discovery.zen.ping.unicast.hosts: ["172.16.80.28","172.16.80.155"]
附件有文件elasticsearch.map
Kibana配置
cat /etc/kibana/kibana.yml
server.port: 5601server.host: "172.16.80.28"elasticsearch.url: "http://172.16.80.28:9200"
Elastalert（需要Python2.7）
下载
进入elastalert目录
pip install -r requirements.txtpython setup.py installelastalert-create-index
前台运行elastalert
python -m elastalert.elastalert --verbose --rule rules/my_rule.yaml
安装参考:
elastalert配置
cat /etc/elastalert/rules/my_rule.yaml
es_host: 172.16.80.28es_port: 9200name: filebeat rulestype: frequencyindex: filebeat*num_events: 5timeframe: hours: 1filter:- query: query_string: query: "message: *error*"alert:- "email"email:- "aaa@qq.com"- "bbb@qq.com"alert_text: "Ref Log http://172.16.80.28:5601/app/kibana"smtp_host: smtp.exmail.qq.comsmtp_port: 25smtp_ssl: falsesmtp_auth_file: /etc/elastalert/rules/smtp_auth_file.yamlfrom_addr: aaa@qq.com
cat /etc/elastalert/rules/smtp_auth_file.yaml
user: "aaa@qq.com"password: "邮箱密码"
cat /etc/elastalert/config.yaml
rules_folder: /etc/elastalert/rulesrun_every: minutes: 1buffer_time: minutes: 15es_host: 172.16.80.28es_port: 9200s_url_prefix: elasticsearchwriteback_index: elastalert_statusalert_time_limit: days: 2
elastalert配置注解
rules_folder：用来加载下一阶段rule的设置，默认是example_rulesrun_every：用来设置定时向elasticsearch发送请求buffer_time：用来设置请求里时间字段的范围，默认是45分钟es_host：elasticsearch的host地址es_port：elasticsearch 对应的端口号use_ssl：可选的，选择是否用SSL连接es，true或者falsees_username：es认证的usernamees_password：es认证的passwordwriteback_index：elastalert产生的日志在elasticsearch中的创建的索引alert_time_limit：失败重试的时间限制
告警参考
客户端安装
Filebeat安装
以Ubuntu为例：
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -sudo apt-get install apt-transport-httpsecho "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-6.x.listapt-get updateapt-get install filebeat
filebeat配置
cat /etc/filebeat/filebeat.yml
filebeat.prospectors:- type: log enabled: true paths: - /var/log/nginx/*.log path: ${path.config}/modules.d/*.yml reload.enabled: falsesetup.template.settings: index.number_of_shards: 3setup.kibana: host: "172.16.80.28:5601"output.elasticsearch: hosts: ["172.16.80.28:9200"] EFK备注
日志位置 /var/lib/elasticsearch/nodes/
日志大小 暂时为345Mb日志轮循 暂时不知道在哪设置elasticsearch.map内容：
# uint mapping{ "settings" : { "index" : { "number_of_replicas" : 1, "number_of_shards" : 5 } }, "mappings" : { "values" : { "properties" : { "itemid" : { "type" : "long" }, "clock" : { "format" : "epoch_second", "type" : "date" }, "value" : { "type" : "long" } } } }} # dbl mapping{ "settings" : { "index" : { "number_of_replicas" : 1, "number_of_shards" : 5 } }, "mappings" : { "values" : { "properties" : { "itemid" : { "type" : "long" }, "clock" : { "format" : "epoch_second", "type" : "date" }, "value" : { "type" : "double" } } } }} # str mapping{ "settings" : { "index" : { "number_of_replicas" : 1, "number_of_shards" : 5 } }, "mappings" : { "values" : { "properties" : { "itemid" : { "type" : "long" }, "clock" : { "format" : "epoch_second", "type" : "date" }, "value" : { "fields" : { "analyzed" : { "index" : true, "type" : "text", "analyzer" : "standard" } }, "index" : false, "type" : "text" } } } }} # text mapping{ "settings" : { "index" : { "number_of_replicas" : 1, "number_of_shards" : 5 } }, "mappings" : { "values" : { "properties" : { "itemid" : { "type" : "long" }, "clock" : { "format" : "epoch_second", "type" : "date" }, "value" : { "fields" : { "analyzed" : { "index" : true, "type" : "text", "analyzer" : "standard" } }, "index" : false, "type" : "text" } } } }} # log mapping{ "settings" : { "index" : { "number_of_replicas" : 1, "number_of_shards" : 5 } }, "mappings" : { "values" : { "properties" : { "itemid" : { "type" : "long" }, "clock" : { "format" : "epoch_second", "type" : "date" }, "value" : { "fields" : { "analyzed" : { "index" : true, "type" : "text", "analyzer" : "standard" } }, "index" : false, "type" : "text" } } } }}