predictive analytics & visualization - towards data driven insights for openstack clouds

Download Predictive Analytics & Visualization - Towards Data Driven Insights for Openstack Clouds

Post on 11-Aug-2014

758 views

Category:

Data & Analytics

1 download

Embed Size (px)

DESCRIPTION

Slightly edited deck presented by myself and my awesome team at the Openstack Summit in May 2014 at Atlanta.

TRANSCRIPT

  • 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Cisco Confidential 2013 Cisco and/or its affiliates. All rights reserved. Predictive Analytics & Visualization Towards Data Driven Insights for Openstack Clouds
  • 2013 Cisco and/or its affiliates. All rights reserved. Cisco ConfidentialCisco ConfidentialCisco Confidential 2013 Cisco and/or its affiliates. All rights reserved. Cloud/Big-data Winnovation Team Alex Holden Damian Van VuurenAiling Zhang Marc Solanas Matt Weeks Pengfei Zhang Kai Zhang Xinyuan Huang Debo Dutta Yathiraj Udupi
  • 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Visualization in OpenStack today
  • 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Tomorrow? Sources: 2001 A Space Oddesy, MGM. Minority Report, Twentieth Century Fox Film Corporation. Mad Men, AMC OpenStack, deploy me an app
  • 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Our Approach: Predictive Analytics & Awesome Visualisation Why Predictive? OpenStack has lots of moving parts Rule/Model/Policy based system hard without insight Why Predictive? Ops folks want to know right now! Why Visualisation? Easy to observe insights by humans East way to consume lots of information from loads oflogs/metrics Improve user experience
  • 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Cisco Confidential 2013 Cisco and/or its affiliates. All rights reserved. Digital Exhaust = Metrics/Logs What information can we get from our cloud? Where can we get it? Ceilometer OpenStack Syslog Project APIs (Nova, Neutron, Cinder etc) What can we do with it? Extract Insights Figure out what failed and why Move workloads around Detect attacks on the cloud Physical Servers CPU Hardware Faults OpenStack Logs OpenStack Messaging Queue Tenant/Project Data Quotas Virtual Machines Config CPU Disk Network Flows Performance Latency Application Level Application Logs Application Requirements Application Performance
  • 2013 Cisco and/or its affiliates. All rights reserved. Cisco ConfidentialCisco ConfidentialCisco Confidential 2013 Cisco and/or its affiliates. All rights reserved. AVOS Dashboard
  • 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential AVOS: Summary Analytics and Visualization on OpenStack Stateless: easy deployment Client side architecture: Polls some OpenStack APIs Listens to the messaging bus Minimum overhead Provides users and service providers full insight into cloud Real time info on all instances/networks etc by project and as admin Historical data on performance Stack-search: Search for all components in the cloud, quickly. Multiple select (See combined data)
  • 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential https://www.youtube.com/watch?v=vyb-Cd4VEDg AVOS Demo
  • 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Bottlenecks Use Cases: Why use AVOS? Attacks
  • 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Use Case: Bottlenecks
  • 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Use Case: Attack Detection Extract Metrics Analyse Patterns We can differentiate between: Hadoop Traffic Network Faults DDoS Attack
  • 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Future Plans Advanced Search Primitives Reducing the volume of logs Real-time error analytics Add to/edit the cluster Ability to preview changes Curvature: https://github.com/CiscoSystems/curvature
  • 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Notes: OpenStack changes Ceilometer Better API queries, optimizing return structure Custom meters: VM to VM traffic flows Other suggestions/Ideas Standardization of the Python clients (several inconsistencies between method calls) Integration into horizon
  • 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Conclusion Openstack Predictive Analytics/ Visibility = Reduce Ops pains We must 1) gather data 2) extract insights 3) present to ops folks with awesome UX
  • 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Thank you. Please take our survey: http://goo.gl/CMAZje Coming Talk: Storage Visibility and Optimization for OpenStack: Wednesday, May 14 3:30pm - 4:10pm

Recommended

View more >