Download - ThousandEyes at Network Field Day 12
![Page 1: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/1.jpg)
1
Network IntelligenceWithout BordersMohit LadCEO and Co-founder
![Page 2: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/2.jpg)
2
About ThousandEyes
Founded by network experts; strong investor
backing
Relied on for critical operations
by leading enterprises
Recognized as an innovative new approach
ThousandEyes delivers network intelligence into every network.
30 Fortune 5005 top 5 SaaS Companies
4 top 6 US Banks
![Page 3: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/3.jpg)
3
When You Think of Network Troubleshooting
![Page 4: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/4.jpg)
4
Legacy EnvironmentsNY Branch
HK Branch
Datacenter
• On-premises Apps• Users in branch offices
over wired connections• MPLS backboneMPLS
MPLS
![Page 5: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/5.jpg)
5
Internet Centric Environment• Adoption of Cloud
Applications• Split-tunnel from
branch offices• Direct Internet
Connectivity between branch offices
• Wireless becoming primary connectivity at branch offices
• Remote Users accessing cloud applications directly
NY Branch
HK Branch
Datacenter 0365
Internet
![Page 6: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/6.jpg)
6
ThousandEyes Cloud AgentsNY Branch
Datacenter 0365Internet
![Page 7: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/7.jpg)
7
ThousandEyes Enterprise AgentsNY Branch
Datacenter 0365Internet
![Page 8: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/8.jpg)
8
ThousandEyes Endpoint AgentsNY Branch
Datacenter 0365Internet
![Page 9: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/9.jpg)
9
Product Design Principles
Intuitive &Effective UI
Harness the Power of SaaS
Innovative Data Collection &
Analytics• Powerful visualizations to
model complex data• UI design that is re-
usable and scalable• Seamless support help
• Minimal deployment effort• Auto-updates• Centralized configuration• Cross-customer data
correlation and analysis• Easy data sharing
between different customers
• Measure black-box environments using active probing
• Measure with minimum instrumentation
![Page 10: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/10.jpg)
10
• Tackling Hybrid Network Environments with Enterprise Agents– Nick Kephart
• End to End Visibility with Endpoint Agent– Scott Cressman, Martin Dam
• Internet Outage Detection– Ricardo Oliveira
Rest of the Day
![Page 11: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/11.jpg)
11
Tackling Hybrid Network Environments with Enterprise AgentsNick Kephart
![Page 12: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/12.jpg)
12
Enterprise Agent: Internal Vantage PointKey Use Cases• Internet connectivity
of ISP ingress and egress
• WAN visibility between branches and data centers
• Performance of web, voice and FTP application traffic
NY Branch
HK Branch
Datacenter 0365
Internet
![Page 13: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/13.jpg)
13
Deploying Enterprise Agents
• Locations with containerized monitoring and operations tools
• For remote branches and stores with limited IT infrastructure
• Branch and WAN routers (IOS XE 3.17+ on ASR 1000 and ISR 4000)
New
New
New
Virtual Appliance
Docker Container
Linux Package
Intel NUC Installer
Cisco IOS Virtual Container
• Easily deployable across the enterprise WAN and data center
![Page 14: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/14.jpg)
14
Visualizing the Entire Network PathHighlights• Forward and reverse
path (helpful for asymmetric routing)
• Measure and locate changes in loss, latency and QoS in each direction
• Also test UDP in addition to TCP
![Page 15: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/15.jpg)
15
End-to-End Visibility with Endpoint AgentScott CressmanMartin Dam
![Page 16: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/16.jpg)
16
End User Visibility Challenges• Remote and traveling
workers• SaaS deployments• LAN and WAN issues
in satellite offices
NY Branch
HK Branch
Datacenter 0365
Internet
![Page 17: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/17.jpg)
17
Today’s “Solutions”
![Page 18: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/18.jpg)
18
Enter ThousandEyes Endpoint Agent
You can’t get this from any other monitoring solution, period.
• Extends visibility to the end-user, in the office, at home, on-the-go
• Troubleshoot individual user sessions with live performance data
• Analyze trends across user populations, applications, geographies
![Page 19: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/19.jpg)
19
How Endpoint Agent Works
Lightweight client softwareWindows 7+, Mac OS X 10.9+
Negligible resource consumptionTypically <1% CPU, <40MB mem, <50MB disk
Easy deployment via standard toolsmsi & pkg installers w/ auto-registration
End-user & background componentsBrowser plugin (Chrome & IE) & system service
Always up-to-dateUpdates automatically, runs in the background
WEB/APPLICATIONCompletion, availability, response time, page load waterfall
NETWORKLoss, latency, jitter, failures, path visualization, wireless topology, VPN, proxy, Wi-Fi quality
(live user sessions!)
Browser-based web applications• Only collects data for domains you choose to monitorData streamed instantly to ThousandEyes service
![Page 20: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/20.jpg)
20
Complete Visibility from End User to Application
![Page 21: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/21.jpg)
Internet Outage Detection
Ricardo OliveiraCTO and Co-founder
![Page 22: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/22.jpg)
22
The Problem Landscape• Lack of visibility to apps
relying on the Internet {UC,S,I,P}aaS
• Lack of visibility to wireless/remote/mobile users
• Traditional NPM solutions design for static clients and on-prem apps– Packet capture– SNMP polling
NY Branch
HK Branch
Datacenter 0365
Internet
![Page 23: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/23.jpg)
23
ThousandEyes AgentsNY Branch
Datacenter 0365Internet
![Page 24: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/24.jpg)
24
• Internet is a shared network – same event impacts multiple customers
• Harness data from multiple customers for more accurate inference of problem
• Drive more value to customers with knowledge of depth and breadth of problem
Drive for Internet Outage Detection
![Page 25: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/25.jpg)
25
• Detect outages in ISPs and understand their impact both globally and as it relates to a specific customer
Overview: Internet Outage Detection
• See the global and account scope, as well as likely root cause of BGP reachability outages
Traffic Outage Detection
Routing Outage Detection
![Page 26: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/26.jpg)
26
1. Anonymized (http) traffic data is aggregated from all tests across the entire user base
2. Algorithms then look for patterns in path traces terminating in the same ISP
3. Exclude: noisy interfaces and networks not belonging to ISPs
How Traffic Outage Detection Works
New YorkCloud Agent
BostonEnterprise Agent
Los AngelesCloud Agent
Level 3 in San Jose
Cogent in Denver
Salesforce
NY Times
Customer 2
Customer 1
![Page 27: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/27.jpg)
27
Traffic Outage DetectionAccount scope
Global scope
Severity and scope of the issue at this interface
![Page 28: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/28.jpg)
28
• ~ 170 affected interfaces / hour
Traffic Outages All the Time
![Page 29: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/29.jpg)
29
Routing Outage DetectionAggregates reachability issues in routing data from 350 routers
Global scope
Account scope
Root cause analysis
![Page 30: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/30.jpg)
30
• ~ 1.6k prefixes affected / hour
Routing Outages All the Time
![Page 31: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/31.jpg)
31
Hurricane Electric route leak affecting AWSTrans-Atlantic issues in Level 3– https://blog.thousandeyes.com/trans-atlantic-issues-level-3-network/Tata and TISparkle issues with submarine cable– https://blog.thousandeyes.com
/smw-4-cable-fault-ripple-effects-across-networks/Hurricane Electric removed >500 prefixesTata cable cut in Singapore affecting DropboxLevel 3, NTT routing issues affecting JIRA– https://blog.thousandeyes.com/identifying-root-cause-routing-outage-detecti
on/
Widespread issues in Telia’s network in Ashburn– https://blog.thousandeyes.com/analyzing-internet-issues-traffic-outage-dete
ction/
Recent Major Outages DetectedApril 23May 3
May 20
June 6June 24July 10
July 17
![Page 32: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/32.jpg)
32
Examples of Notable Outages
![Page 33: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/33.jpg)
33
1. Network Layer Issues in Telia in Ashburn
Detected outage coincides with packet loss spikes
Ashburn, VA is “ground zero” for this outage
https://fvqmu.share.thousandeyes.com/
![Page 34: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/34.jpg)
34
Specific Failure Points in Telia
High severity and wide scope (Outages affecting at least 20 tests for a NA/EU interface are likely to be wide in scope)
Terminal nodes in Telia
![Page 35: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/35.jpg)
35
2. Hurricane Electric Route Flap
Detected outage coincides with spike in AS path changes
Root cause analysis points to Hurricane Electric and Telx
https://njjgkif.share.thousandeyes.com/
![Page 36: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/36.jpg)
36
Route Flap by Hurricane Electric
Hurricane Electric
Routes flap from using HE to NTT, then back to HE
![Page 37: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/37.jpg)
37
Traffic Issues in Hurricane Electric
Hurricane Electric
![Page 38: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/38.jpg)
38
3. NTT and Level 3 Routing Issues Affect JIRA
JIRA saw 0% availability and 100% packet loss
Most affected interfaces are in Ashburn, VA
https://ncigwwph.share.thousandeyes.com/
![Page 39: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/39.jpg)
39
Traffic Terminating in NTT
Traffic paths originally traversed Level 3 and NTT
Traffic paths then change to traverse only NTT, terminating there
![Page 40: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/40.jpg)
40
JIRA’s /24 Prefix Becomes Unreachable
As the primary upstream ISP, Level 3 is associated with the most affected routes Routes through upstream
ISPs NTT and Level 3 all withdrawn
![Page 41: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/41.jpg)
41
Routers Begin Using Misconfigured /16 Prefix
The backup /16 prefix directs to NTT, not JIRA’s network. This is why the traffic path changed to traverse only NTT, terminating there when JIRA’s IP couldn’t be found in NTT’s network.
![Page 42: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/42.jpg)
42
Traffic Outages @ Cloud• IaaS/PaaS (CDNs, hosting, DNS providers)• SaaS (+ app context)
Routing Outages• Leaks and hijacks
Outage Event Stream• Outage geo + topology maps• Alerts based on outage impact/location/type/etc
What’s Next
![Page 43: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/43.jpg)
43
Outage Created by Level3 Flaphttps://btwzofam.share.thousandeyes.com
![Page 44: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/44.jpg)
44
• Look for purple indicators and the ‘Outage Detected’ dropdown when investigating issues—these indicate detected outages!
• Use quick links or select specific nodes/ASes to see how paths have changed over time
• Correlate data from the web, network and routing layers to analyze root cause
• See our blogs and Knowledge Base articles for more info:– Blog on Traffic Outage Detection
– https://blog.thousandeyes.com/analyzing-internet-issues-traffic-outage-detection/
– Blog on Routing Outage Detection– https
://blog.thousandeyes.com/identifying-root-cause-routing-outage-detection/ – Knowledge Base: https://support.thousandeyes.com/entries/110214366
Tips for Diagnosing Internet Outages
![Page 45: ThousandEyes at Network Field Day 12](https://reader034.vdocuments.mx/reader034/viewer/2022042907/586f83a41a28ab54768b4a47/html5/thumbnails/45.jpg)
45
Thank You@thousandeyes