beautiful monitoring with grafana and influxdb

19
Beautiful Monitoring With Grafana and InfluxDB Lee Jensen ([email protected])

Upload: leesjensen

Post on 15-Jul-2015

1.410 views

Category:

Software


7 download

TRANSCRIPT

Page 1: Beautiful Monitoring With Grafana and InfluxDB

Beautiful Monitoring With Grafana and InfluxDB

Lee Jensen ([email protected])

Page 2: Beautiful Monitoring With Grafana and InfluxDB

What We Want

● Historical storage of metrics● Interactive exploration of metrics● Scalable● Performant● Something you can send to the CEO

Page 3: Beautiful Monitoring With Grafana and InfluxDB

Something Like This

Page 4: Beautiful Monitoring With Grafana and InfluxDB

Current - 0.8.8● Started: 2013ish● Paul Dix● ErrPlane● MIT license● Written in GO● Uses LevelDB, RocksDB, HyperLevelDB, or LMDB as database

Future - 0.9.0● Tags and measurements - API more coherent with domain● BoltDB

Page 5: Beautiful Monitoring With Grafana and InfluxDB

InfluxDB Design Goals

● Simple to install and manage. Shouldn’t require setting up external dependencies like Zookeeper and Hadoop.

● HTTP(S) interface for reading and writing data.● Horizontally scalable.● On disk and in memory. Most data is cold.● Compute percentiles and other functions on the fly.● Downsample data on different windows of time.

Page 6: Beautiful Monitoring With Grafana and InfluxDB

Installing InfluxDB$ brew install influxdb

/usr/local/opt/influxdb/usr/local/etc/influxdb.conf

$ influxdb -config=/usr/local/etc/influxdb.conf

Ports8083 - UI8086 - API - Send and query data8090 - Cluster management raft8099 - Cluster management protobuf

Page 7: Beautiful Monitoring With Grafana and InfluxDB

InfluxDB Design

Database, shard space, series (table), column

InfluxDB

Demo

Grafana

GrafanaMetrics

queries

dashboards

data

Page 8: Beautiful Monitoring With Grafana and InfluxDB

Simple UsageCreate Databasecurl -X POST 'g:8086/db?u=root&p=root' -d '{"name": "demo"}'

Add Datacurl -X POST "g:8086/db/demo/series?u=root&p=root" -d '[{"name": "server1.cpu","columns": ["val"],"points": [[88]]}]'

Querycurl "g:8086/db/demo/series?u=root&p=root&q=select%20*%20from%20server1.cpu"

Delete Databasecurl -X DELETE 'g:8086/db/demo?u=root&p=root'

Page 9: Beautiful Monitoring With Grafana and InfluxDB

InfluxDB Interface (http://g:8083/)

Page 10: Beautiful Monitoring With Grafana and InfluxDB

InfluxDB Query Languageselect val from server1select val1, val2 from server1select * from server1select cpu from /server.*/select * from /.*/ where time > now() - 1hselect * from /.*/ where time > '2013-08-12 23:32:01'select * from /.*/ group by time(10m)select count(val) from /.*/ group by time(10m)select percentile(val, 95) from /.*/ group by time(10m)select count(distinct(val)) from /.*/

Page 11: Beautiful Monitoring With Grafana and InfluxDB

Generating Metrics (http://g:8086/)

● 22 client libraries● 31 Sources -> fluentd -> InfluxDB● Graphite -> InfluxDB● Use the REST API Random Server CPU$ while true; do for i in {1..5}; do y=$[$i*2+$[RANDOM%20]]; curl -X POST "g:8086/db/demo/series?u=root&p=root" -d '[{"name": "server'"$i"'.cpu","columns": ["val"],"points": [['"$y"']]}]'; done; sleep 1; done;

Page 12: Beautiful Monitoring With Grafana and InfluxDB

Continuous Queries

Automatically downsample a series with a longer retention period

Series A - Every second for a week (604k)

Series B - Every hour for 5 years (40k)

Downsample aggregates to an hour

Page 13: Beautiful Monitoring With Grafana and InfluxDB

Current - 1.9.1● Started: Dec 5 2013● Torkel Ödegaard● Apache 2.0 license● Pure JavaScript● Inspired by Kibana

Future - 2.0● GO backend● PNG rendering● Corporate support by Raintank - Continue as open source

Page 14: Beautiful Monitoring With Grafana and InfluxDB

Try It Yourself

http://play.grafana.org/

Page 15: Beautiful Monitoring With Grafana and InfluxDB

Installing GrafanaDownload zip - http://grafana.org/download/

Configure$ cp config.sample.js config.js

Host$ sudo python -m SimpleHTTPServer 80

datasources: { influxdb: { type: 'influxdb', url: "http://127.0.0.1:8086/db/demo", username: 'root', password: 'root', }, grafana: { type: 'influxdb', url: "http://127.0.0.1:8086/db/grafana", username: 'root', password: 'root', grafanaDB: true },}

Page 16: Beautiful Monitoring With Grafana and InfluxDB

Visualizing MetricsFinally! What we came to see! Less talk more demo …

Page 17: Beautiful Monitoring With Grafana and InfluxDB

Annotations

Add Annotationcurl -X POST "g/db/demo/series?u=root&p=root" -d '[{"name": "events","columns": ["type","tag","text"],"points": [["deploy", "prod","1.0.3"]]}]'

Page 18: Beautiful Monitoring With Grafana and InfluxDB

Scripted Dashboards~/grafana/app/dashboards/script_small.js

http://g/#/dashboard/script/script_small.js?panels=5&name=world

Page 19: Beautiful Monitoring With Grafana and InfluxDB

Happy Monitoring!