and prometheus to play nicely the uninstrumentable; getting … · 2020-06-03 · dan rathbone...
TRANSCRIPT
DAN RATHBONE & JOE STRINGER PROMCON 2017, AUGUST 2017
The Uninstrumentable; Getting Apache Spark and Prometheus to Play Nicely
2
● Healthcare data processing system using Apache PySpark
● Failed attempts and the crazy ideas that followed
● Actually working with lots of pretty graphs
6
https://cwiki.apache.org/confluence/display/SPARK/PySpark+Internals7
8https://cwiki.apache.org/confluence/display/SPARK/PySpark+Internals
“Occasionally you will need to monitor components which cannot be scraped. They might live behind a firewall, or they might be too short-lived to expose data reliably via the pull model. The Prometheus Pushgateway allows you to push time series from these components to an intermediary job which Prometheus can scrape.”
9
10
“The Pushgateway is explicitly not an aggregator or distributed counter but rather a metrics cache”
11
12
14
15
16
● Realtime is worth the effort, visibility is key
● Nothing’s uninstrumentable
● The solution is often quite simple
● Prometheus is pretty flexible
JOE STRINGER
@joeds13
DAN RATHBONE
@thetrilemma
www.infinityworks.com