network visibility for efficient openstack operations
Post on 12-Sep-2014
240 views
DESCRIPTION
The slidedeck presented by me and my team at the Atlanta Openstack Summit 2014.TRANSCRIPT
© 2013 Cisco and/or its affiliates. All rights reserved.Cisco
Confidential Cisco Confidential© 2013 Cisco and/or its affiliates. All rights reserved.
Network VisibilityFor Efficient OpenStack Operations
© 2013 Cisco and/or its affiliates. All rights reserved.Cisco
Confidential Cisco ConfidentialCisco Confidential© 2013 Cisco and/or its affiliates. All rights reserved.
The Cloud / Big-Data Winnovation Team
Alex [email protected]
Damian Van [email protected]
Ailing [email protected]
Marc [email protected]
Matt [email protected]
Pengfei [email protected]
Xinyuan [email protected]
Debo [email protected]
Yathiraj [email protected]
© 2013 Cisco and/or its affiliates. All rights reserved.Cisco
Confidential
Today...
© 2013 Cisco and/or its affiliates. All rights reserved.Cisco
Confidential Cisco ConfidentialCisco Confidential© 2013 Cisco and/or its affiliates. All rights reserved.
© 2013 Cisco and/or its affiliates. All rights reserved.Cisco
Confidential
Outline• Data Gathering Architecture
• Network Visibility
• Network Analysis
• Conclusion
© 2013 Cisco and/or its affiliates. All rights reserved.Cisco
Confidential
Virtual status
Performance
Why is my job slow?
Operators
Physical status
Hotspots
What are they doing in my cloud?
Users
© 2013 Cisco and/or its affiliates. All rights reserved.Cisco
Confidential
© 2013 Cisco and/or its affiliates. All rights reserved.Cisco
Confidential
Network flowsNew ceilometer agent
Collects and aggregates data from Neutron
VM-VM flow traffic insight
© 2013 Cisco and/or its affiliates. All rights reserved.Cisco
Confidential
Application logs
OpenStack Logs
Server Syslogs
VM Syslogs
Application (Hadoop, etc) Logs
...
© 2013 Cisco and/or its affiliates. All rights reserved.Cisco
Confidential
AVOS: Analytics and Visualization on OpenStack Demo
© 2013 Cisco and/or its affiliates. All rights reserved.Cisco
Confidential Cisco Confidential © 2013 Cisco and/or its affiliates. All rights reserved.
Data Insights
© 2013 Cisco and/or its affiliates. All rights reserved.Cisco
Confidential
Data insights - What is going on in a Hadoop cluster?
Net
wor
k flo
ws
(bit/
s)
Outgoing network flows from VMs
Time (s) Time (s)
Incoming network flows to VMs
Net
wor
k flo
ws
(bit/
s)
© 2013 Cisco and/or its affiliates. All rights reserved.Cisco
Confidential
Data insights - What is going on in a Hadoop cluster?
Time (s)
Net
wor
k flo
ws
(bit/
s)
Outgoing network flows
master <-> slave
© 2013 Cisco and/or its affiliates. All rights reserved.Cisco
Confidential
Data insights - What is going on in a Hadoop cluster?
Time (s)
Net
wor
k flo
ws
(bit/
s)
Outgoing network flows
slave <-> slave
© 2013 Cisco and/or its affiliates. All rights reserved.Cisco
Confidential
Data insights - How to troubleshoot?
Normal Server down
© 2013 Cisco and/or its affiliates. All rights reserved.Cisco
Confidential
Data insights - How to troubleshoot?
INFO org.apache.hadoop.hdfs.DFSClient: Exception in createBlockOutputStream … java.io.IOException: Bad connect ack with firstBadLink as 192.168.0.7:…
INFO org.apache.hadoop.hdfs.DFSClient: Abandoning blk_…
INFO org.apache.hadoop.hdfs.DFSClient: Excluding datanode 192.168.0.7:…
WARNING nova.compute.manager … [instance: 6a475088-...] Instance shutdown by itself. Calling the stop API." …
© 2013 Cisco and/or its affiliates. All rights reserved.Cisco
Confidential
● Collect Multiple Metrics● Analyze Patterns● Classify Cloud States
Data insights - How to detect abnormals?
© 2013 Cisco and/or its affiliates. All rights reserved.Cisco
Confidential
Future PlansNetwork Distance
Data-driven Diagnoses
More data sources & integrated analysis
© 2013 Cisco and/or its affiliates. All rights reserved.Cisco
Confidential
Thank you.Please take our survey:
http://goo.gl/CMAZje