monitor - percona.com $ ssh vagrant@127.0.0.1 -p 2222 -o preferredauthentications=password windows: ...
Post on 07-Mar-2018
236 Views
Preview:
TRANSCRIPT
Monitor with the Stack
Philipp Krenn�����@xeraa
1
Infrastructure | Developer Advocate
2
Who Is UsingElasticsearch
Logstash and KibanaBeats
4
5
6
7
Starting Pointhttps://github.com/xeraa/mongodb-monitoring
8
USB Sticks
9
BoxVagrant Ansible Provisioner
10
Credentialsvagrant & vagrant
11
SSH$ ssh vagrant@127.0.0.1 -p 2222 -o PreferredAuthentications=password
Windows: http://www.putty.org
12
Ansible
$ cd /elastic-stack/$ ls
13
14
REST
$ curl -XGET -u "elastic:changeme" http://localhost:9200/
15
16
18
Filebeat
19
Filebeat Modules
20
System Dashboards
21
22
MongoDB Logs/var/log/mongodb/mongod.log
23
/etc/filebeat/filebeat.yml
filebeat.prospectors:- input_type: log paths: - /var/log/mongodb/mongod.log document_type: mongodb
24
PS: Multiline Logs
25
/etc/filebeat/filebeat.yml
filebeat.prospectors:
- input_type: log paths: - /var/log/java-app/*.log document_type: java multiline.pattern: '^[[:space:]]' multiline.negate: false multiline.match: after
26
Test
$ java -jar /opt/pocdriver/bin/POCDriver.jar -k 20 -i 10 -u 10 -b 20
27
Kibana DiscoverLimit Kibana view to the mongodb type
28
29
Grok Patternshttps://github.com/logstash-plugins/logstash-patterns-
core/blob/master/patterns/grok-patterns
30
MongoDB Patternshttps://github.com/logstash-plugins/logstash-patterns-
core/blob/master/patterns/mongodb
31
Building PatternsGrokdebug in Kibana 5.5+ (X-Pack Basic)
https://grokdebug.herokuapp.com
32
33
/etc/filebeat/filebeat.yml
output.elasticsearch: hosts: ["localhost:9200"] username: "{{ elastic_user }}" password: "{{ elastic_password }}" pipeline: "mongodb_log"
34
ConsolePattern in a single line
PUT _ingest/pipeline/mongodb_log{ "description": "Ingest pipeline for MongoDB logs", "processors": [ { "grok": { "field": "message", "patterns": [ "%{TIMESTAMP_ISO8601:timestamp}%{SPACE}%{MONGO3_SEVERITY:mongodb.severity} %{SPACE}%{MONGO3_COMPONENT:mongodb.component}%{SPACE} (?:[%{DATA:mongodb.context}])?%{SPACE}%{GREEDYDATA:message}" ] } } ]}
35
Filebeat Restart$ sudo service filebeat restart
36
Test
$ java -jar /opt/pocdriver/bin/POCDriver.jar -k 20 -i 10 -u 10 -b 20
37
Refresh Field ListManagement → Index Patterns → filebeat-* → Refresh
field list
38
Visualizemongodb.component of log events
39
40
41
Fetch the Slow Log
42
/etc/logstash/conf.d/00-mongodb-input.conf
input { mongodb { uri => 'mongodb://127.0.0.1:27017/POCDB' placeholder_db_dir => '/var/local/logstash-mongodb/' collection => 'system.profile' batch_size => 500 generateId => true }}
43
/etc/logstash/conf.d/20-elasticsearch-output.conf
output { elasticsearch { hosts => ["localhost:9200"] manage_template => false index => "mongodb-%{+YYYY.MM.dd}" document_type => "slowlog" user => "{{ elastic_user }}" password => "{{ elastic_password }}" }}
44
Logstash Restart$ sudo service logstash restart
45
Debug Logstash
$ less /var/log/logstash/logstash-plain.log
46
Doesn't WorkGetting data from system collections
https://github.com/phutchins/logstash-input-mongodb/issues/8
47
Extension IdeaBuild an exec filter to run slow queries against
MongoDB's .explain()
48
Metricbeat
49
Metricbeat System
50
Metricbeat Service
51
/etc/metricbeat/metricbeat.yml
- module: mongodb metricsets: ["dbstats", "status"] hosts: ["localhost:27017"]
52
Metricbeat Restart$ sudo service metricbeat restart
53
Test
$ java -jar /opt/pocdriver/bin/POCDriver.jar -k 20 -i 10 -u 10 -b 20
54
55
Visual Buildermongodb.status.network.in.bytes vs mongodb.status.network.out.bytes
56
57
Packetbeat
58
Protocols
59
FlowsApplication layer: Unsupported / encrypted (TLS) protocols
IP / TCP / UDPNumber of packets & bytes
RetransmissionsTemporal flow
60
/etc/packetbeat/packetbeat.yml
packetbeat.protocols.mongodb: ports: [27017]
61
Packetbeat Restart$ sudo service packetbeat restart
62
Test
$ java -jar /opt/pocdriver/bin/POCDriver.jar -k 20 -i 10 -u 10 -b 20
63
64
Heartbeat
65
HeartbeatICMP, TCP, HTTP, HTTPS
66
/etc/heartbeat/heartbeat.yml
heartbeat.monitors:- type: tcp hosts: ["127.0.0.1:27017"] schedule: '@every 10s'
67
Heartbeat Restart$ sudo service heartbeat restart
68
Test
$ sudo service mongod stop$ sudo service mongod start
69
VisualizeUp or down and TCP response times
70
71
72
DashboardCombining visualizations
73
74
Winlogbeat
75
libbeathttps://github.com/elastic/beats/tree/master/generate/beat
76
77
X-PackSecurityMonitoringGraph
ReportingAlerting
Machine Learning
78
X-Pack Basic
79
Conclusion
80
81
82
83
Thanks!
Questions?Philipp Krenn�����@xeraa
84
top related