streaming anomaly detection for big data / internet of things · 2020. 12. 27. · carbon-cache: a...

12
Streaming Anomaly Detection for Big Data / Internet of Things Adam Drake FOSSAsia 20160319 @aadrake http://aadrake.com

Upload: others

Post on 29-May-2021

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Streaming Anomaly Detection for Big Data / Internet of Things · 2020. 12. 27. · carbon-cache: a writes Wh r metrics | 0.0.10:2003 | carbon.relay consisteN hash metrics Carbon Relay

Streaming Anomaly Detection for Big Data /

Internet of ThingsAdam Drake

FOSSAsia 20160319 @aadrake

http://aadrake.com

Page 2: Streaming Anomaly Detection for Big Data / Internet of Things · 2020. 12. 27. · carbon-cache: a writes Wh r metrics | 0.0.10:2003 | carbon.relay consisteN hash metrics Carbon Relay

@aadrake #hacker

or

@aadrake #ThoughtLeader

Page 3: Streaming Anomaly Detection for Big Data / Internet of Things · 2020. 12. 27. · carbon-cache: a writes Wh r metrics | 0.0.10:2003 | carbon.relay consisteN hash metrics Carbon Relay

Background

• Applied mathematics

• Startups

• Many industries

Page 4: Streaming Anomaly Detection for Big Data / Internet of Things · 2020. 12. 27. · carbon-cache: a writes Wh r metrics | 0.0.10:2003 | carbon.relay consisteN hash metrics Carbon Relay

Time Series is the New Big Data

https://www.mapr.com/blog/time-series-data-new-big-data

Page 5: Streaming Anomaly Detection for Big Data / Internet of Things · 2020. 12. 27. · carbon-cache: a writes Wh r metrics | 0.0.10:2003 | carbon.relay consisteN hash metrics Carbon Relay

https://blogs.technet.microsoft.com/machinelearning/2014/11/05/anomaly-detection-using-machine-learning-to-detect-abnormalities-in-time-series-data/

Page 6: Streaming Anomaly Detection for Big Data / Internet of Things · 2020. 12. 27. · carbon-cache: a writes Wh r metrics | 0.0.10:2003 | carbon.relay consisteN hash metrics Carbon Relay

Graphite Prometheus (SoundCloud)

Nagios BreakoutDetection (Twitter)

Surus (Netflix) Atlas (Netflix) Skyline (Etsy)

Bosun (Stack Exchange)

Page 7: Streaming Anomaly Detection for Big Data / Internet of Things · 2020. 12. 27. · carbon-cache: a writes Wh r metrics | 0.0.10:2003 | carbon.relay consisteN hash metrics Carbon Relay
Page 8: Streaming Anomaly Detection for Big Data / Internet of Things · 2020. 12. 27. · carbon-cache: a writes Wh r metrics | 0.0.10:2003 | carbon.relay consisteN hash metrics Carbon Relay

https://grey-boundary.io/the-architecture-of-clustering-graphite/

Page 9: Streaming Anomaly Detection for Big Data / Internet of Things · 2020. 12. 27. · carbon-cache: a writes Wh r metrics | 0.0.10:2003 | carbon.relay consisteN hash metrics Carbon Relay
Page 10: Streaming Anomaly Detection for Big Data / Internet of Things · 2020. 12. 27. · carbon-cache: a writes Wh r metrics | 0.0.10:2003 | carbon.relay consisteN hash metrics Carbon Relay

metrics

kaas

Page 11: Streaming Anomaly Detection for Big Data / Internet of Things · 2020. 12. 27. · carbon-cache: a writes Wh r metrics | 0.0.10:2003 | carbon.relay consisteN hash metrics Carbon Relay

DON'T DO THIS

Page 12: Streaming Anomaly Detection for Big Data / Internet of Things · 2020. 12. 27. · carbon-cache: a writes Wh r metrics | 0.0.10:2003 | carbon.relay consisteN hash metrics Carbon Relay

https://blogs.technet.microsoft.com/machinelearning/2014/11/05/anomaly-detection-using-machine-learning-to-detect-abnormalities-in-time-series-data/