beautiful monitoring with grafana and influxdb
TRANSCRIPT
Beautiful Monitoring With Grafana and InfluxDB
Lee Jensen ([email protected])
What We Want
● Historical storage of metrics● Interactive exploration of metrics● Scalable● Performant● Something you can send to the CEO
Something Like This
Current - 0.8.8● Started: 2013ish● Paul Dix● ErrPlane● MIT license● Written in GO● Uses LevelDB, RocksDB, HyperLevelDB, or LMDB as database
Future - 0.9.0● Tags and measurements - API more coherent with domain● BoltDB
InfluxDB Design Goals
● Simple to install and manage. Shouldn’t require setting up external dependencies like Zookeeper and Hadoop.
● HTTP(S) interface for reading and writing data.● Horizontally scalable.● On disk and in memory. Most data is cold.● Compute percentiles and other functions on the fly.● Downsample data on different windows of time.
Installing InfluxDB$ brew install influxdb
/usr/local/opt/influxdb/usr/local/etc/influxdb.conf
$ influxdb -config=/usr/local/etc/influxdb.conf
Ports8083 - UI8086 - API - Send and query data8090 - Cluster management raft8099 - Cluster management protobuf
InfluxDB Design
Database, shard space, series (table), column
InfluxDB
Demo
Grafana
GrafanaMetrics
queries
dashboards
data
Simple UsageCreate Databasecurl -X POST 'g:8086/db?u=root&p=root' -d '{"name": "demo"}'
Add Datacurl -X POST "g:8086/db/demo/series?u=root&p=root" -d '[{"name": "server1.cpu","columns": ["val"],"points": [[88]]}]'
Querycurl "g:8086/db/demo/series?u=root&p=root&q=select%20*%20from%20server1.cpu"
Delete Databasecurl -X DELETE 'g:8086/db/demo?u=root&p=root'
InfluxDB Interface (http://g:8083/)
InfluxDB Query Languageselect val from server1select val1, val2 from server1select * from server1select cpu from /server.*/select * from /.*/ where time > now() - 1hselect * from /.*/ where time > '2013-08-12 23:32:01'select * from /.*/ group by time(10m)select count(val) from /.*/ group by time(10m)select percentile(val, 95) from /.*/ group by time(10m)select count(distinct(val)) from /.*/
Generating Metrics (http://g:8086/)
● 22 client libraries● 31 Sources -> fluentd -> InfluxDB● Graphite -> InfluxDB● Use the REST API Random Server CPU$ while true; do for i in {1..5}; do y=$[$i*2+$[RANDOM%20]]; curl -X POST "g:8086/db/demo/series?u=root&p=root" -d '[{"name": "server'"$i"'.cpu","columns": ["val"],"points": [['"$y"']]}]'; done; sleep 1; done;
Continuous Queries
Automatically downsample a series with a longer retention period
Series A - Every second for a week (604k)
Series B - Every hour for 5 years (40k)
Downsample aggregates to an hour
Current - 1.9.1● Started: Dec 5 2013● Torkel Ödegaard● Apache 2.0 license● Pure JavaScript● Inspired by Kibana
Future - 2.0● GO backend● PNG rendering● Corporate support by Raintank - Continue as open source
Try It Yourself
http://play.grafana.org/
Installing GrafanaDownload zip - http://grafana.org/download/
Configure$ cp config.sample.js config.js
Host$ sudo python -m SimpleHTTPServer 80
datasources: { influxdb: { type: 'influxdb', url: "http://127.0.0.1:8086/db/demo", username: 'root', password: 'root', }, grafana: { type: 'influxdb', url: "http://127.0.0.1:8086/db/grafana", username: 'root', password: 'root', grafanaDB: true },}
Visualizing MetricsFinally! What we came to see! Less talk more demo …
Annotations
Add Annotationcurl -X POST "g/db/demo/series?u=root&p=root" -d '[{"name": "events","columns": ["type","tag","text"],"points": [["deploy", "prod","1.0.3"]]}]'
Scripted Dashboards~/grafana/app/dashboards/script_small.js
http://g/#/dashboard/script/script_small.js?panels=5&name=world
Happy Monitoring!