Mind the Gap
Nick McKeownStanford University
Dean President
My problem
I get excited about improving the practice.
I am a networking researcher.
The Internet’s problem
Architecture: Seems impossible to change– Well-known shortcomings– Many great proposals– No way to demonstrate and test at scale– Huge installed base understandably reluctant to change
Industry: Strong vested interest in status quo– Very high margins as-is– Closed, proprietary solutions– Complex, brittle products– Innovator’s Dilemma
No path to change
My solution
I am done
I quit
Dean President
Why I love my job
I work with people much smarter than me.
I get to work on intellectually interesting ideas.
… that might positively change the practice.
Then, we try to actually change the practice.
Whatever it takes
• Prove a theorem
• Write a paper
• Build a demo
• Talk to lots of industry people
• Write a standard
• Give lots of talks
• Write a blog
• Start a company
• Build an open source tool
• …
The course of action
Is determined by the idea.
Is determined by what you want to change.
Some things that worked for me
Let me start with some motherhood and apple pie…
Look for blind spots
Question old school assumptions
Open your heart and mind to people who question assumptions
Question AssumptionsExample 1
“On the self-similar nature of Ethernet traffic”
Leland, Taqqu, Willinger, Wilson [Sigcomm ‘93+
Whether you agree with them or not – it got the whole community questioning assumptions.
Question AssumptionsExample 2
How big to make a backbone router buffer?
CB
Buffer Size
RTT ´C
RTT ´C
N
Th
rou
gh
pu
t
25,000
100%
2,500,00010Gb/s WAN
Number of packets
On-chip buffersSmaller designLower power
tW
ind
ow
Siz
e
Buffer
Buffer Size
RTT ´C
RTT ´C
N
Th
rou
gh
pu
t
log(W)
25,000~50
~ 90%
100%
2,500,00010Gb/s WAN
Number of packets
20 pkts
Integrated all-optical buffer [UCSB 2008]
On-chip buffersSmaller designLower power
People
Stanford: Guido Appenzeller, Neda Beheshti, Tim Roughgarden, Ashish Goel, Mihaela Enachescu
Toronto: Yashar Ganjali
Cambridge: Damon Wischik, Frank Kelly, Gaurav Raina
UMass: Don Towsley, Chris Hollot
UW Madison: Paul Barford, Joel Sommers
GA Tech: Amogh Dhamdhere, Constantine Dovrolis
UCSB: John Bowers, Emily Burmeister, Dan Blumenthal
Question AssumptionsExample 3
Myth 1: It is hard for switch/router hardware to maintain lots of queues.
– Since 1995, seen 10-15 ASICs do it easily
– Recently: 64x10Gb/s switch, 128k flows, 10% overhead.
Myth 2: You can’t build a large flow table– Next couple of years: Over 1Tb/s, 100,000s entries,
100s bits wide, several tables.
It doesn’t mean you have to; just says you can.
Some things that worked for me
Some surprising takeaways
Choosing research projects
• Pick a problem that is intellectually interesting.
• And improves the practice.
• And industry doesn’t like (yet).
PacketForwarding
PacketForwarding
PacketForwarding
PacketForwarding
PacketForwarding
Control
Control
Control
Control
Control
EthaneMartin Casado et al [Sigcomm ‘07]
Policy
“Laptops can’t accept incoming connections”
“A can’t talk to B”
Network Control Plane
Microsoft Security Group: “Come on in…”
Cisco Switching Group: “It will never work…”
Raw nerve.
We must be onto something.
Checklist
• Intellectually interesting.
• And improves the practice.
• And industry doesn’t like (yet).
Corollary: You can’t give stuff away
Example 1: Ethane (2007)
– Tried to give it away for free.
– Realized industry wasn’t ready to see it.
– Started Nicira.
Example 2: Network Memory (2001)
Example 3: Scalable Routers (1997)
Packet Buffer
10Gb/s10Gb/s
40 bytes every 32ns
Problem - Two memory operations every 32ns.- One DRAM operation about 50ns.- SRAM 75x price per bit, and 10x power.- 40Gb/s links coming; linecards with 16 x 10Gb/s links.- Thousands of queues.
DRAM
Network Memory
10Gb/s10Gb/s
Solution- SRAM cache size for zero misses: 2qblnq
- Practical for 160Gb/s linecard, 1024 queues.
SRAM SRAM
DRAM
Tails Heads
Blocks, bBlocks, b
1
q
ASIC
History
2001: Started on problem; industry discussion.
2003: Tried to give it away.
2004:
– 80% of world’s SRAM: Ethernet switches.
– Of those, 80% used by Cisco ($400M p.a.)
– Started Nemo.
2007:
– Savings estimate $150M p.a.
Checklist
• Intellectually interesting.
• And improves the practice.
• And industry doesn’t like (yet).
Corollary: You can’t give stuff away
Example 1: Ethane (2007)
Example 2: Network Memory (2001)
Example 3: Scalable Routers (1997)
1 2 3 4 5 6 7 8 9 10 111213141516
17 1819 20 2122 232425262728 29303132
2km
History
1997
– State of the art was 40Gb/s per rack.
– Tried to give it away.
– “Leave it to us”.
– Started Abrizio.
1999
– All outsourced solutions.
Checklist
• Intellectually interesting.
• And improves the practice.
• And industry doesn’t like (yet).
I put everything in public domain
Industry– Invests huge amounts to develop and sell products.
– Patents protect their ideas, giving them confidence to invest.
University research– Serves society at large.
– Stay ahead by running fast, not by protecting.
– Makes it easier to work with industry.
Everything in public domain since 1999.
Why I decided to come back
By 2005Many smart people calling for change
DaveClark
LarryPeterson
TomAnderson
Scott Shenker
JonTurner
PeterFreeman
GuruParulkar
Overcoming the Internet Impasse through Virtualization FIND, GENI, …
100x100, 4D
HuiZhang
SandyFraser
JenniferRexford
AlbertGreenberg
Vision
Software Defined Networks
Martin Casado
Scott ShenkerTeemu Koponen
Guru Parulkar+ many (brave) students
Vertically integrated
Closed, proprietary
Slow innovation
Small industry
Specialized
Operating
System
Specialized
Hardware
AppAppAppAppAppAppAppAppAppAppApp
Specialized
Applications
Horizontal
Open interfaces
Rapid innovation
Huge industry
Microprocessor
Open Interface
LinuxMacOS
Windows(OS)
or or
Open Interface
Vertically integrated
Closed, proprietary
Slow innovation
AppAppAppAppAppAppAppAppAppAppApp
Horizontal
Open interfaces
Rapid innovation
ControlPlane
ControlPlane
ControlPlane
or or
Open Interface
Specialized
Control
Plane
Specialized
Hardware
Specialized
Features
Merchant
Switching Chips
Open Interface
Checklist
• Intellectually interesting.
• And improves the practice.
• And industry doesn’t like (yet).
What’s next?
Making Networks Work
An intellectual framework for verifying, troubleshooting and
debugging SDNs
With SDN we can:1. Formally verify that our networks are
behaving correctly.
2. Identify bugs, then systematically track down their root cause.
• Ensuring correctness [Frenetic][HFT][Netcore]Nate Foster, Andrew Ferguson, Mike Freedman, Jen Rexford, Rob Harrison, Dave Walker, ++
• Software Fault Localization [W3]Scott Shenker, Colin Scott, Kyriakos Zarifis, Andreas Wundsam.
• Checking behavior [NICE]Marco Canini, Daniele Venzano, Peter Peresini, DejanKostic, Jen Rexford.
• Checking Invariants [VeriFlow]Ahmed Khurshid, Wenxuan Zhou, Matthew Caesar, P. Brighten Godfrey
• Consistent updatesMark Reitblatt, Rick McGeer, ++
• …
Scott Shenker at 1st ONS in 2011 “The Future of Networking and the Past of Protocols”
Software Defined Network (SDN)
Global Network View
Network Virtualization
PacketForwarding
PacketForwarding
PacketForwarding
PacketForwarding
PacketForwarding
Network OS
Abstract Network View
Control
Programs
f View( )Control
Programs
f View( )Control
Programs
f View( )
Software Defined Network (SDN)
Global Network View
Network Virtualization
PacketForwarding
PacketForwarding
PacketForwarding
PacketForwarding
Abstract Network View
Control
Programs
f View( )Control
Programs
f View( )Control
Programs
f View( )firewall.c
…
if( pkt->tcp->dport == 22)
dropPacket(pkt);
…
PacketForwarding
Network OS1. <Match, Action>2. <Match, Action>3. <Match, Action>4. <Match, Action>5. <Match, Action>6. …7. …
1. <Match, Action>2. <Match, Action>3. <Match, Action>4. <Match, Action>5. <Match, Action>6. …7. …
1. <Match, Action>2. <Match, Action>3. <Match, Action>4. <Match, Action>5. <Match, Action>6. …7. …
1. <Match, Action>2. <Match, Action>3. <Match, Action>4. <Match, Action>5. <Match, Action>6. …7. …1. <Match, Action>
2. <Match, Action>3. <Match, Action>4. <Match, Action>5. <Match, Action>6. …7. …
How do other industries do it?
Making ASICs Work
$10B tool businesssupports a
$250B chip industry
Specification
Functional Description (RTL)
Testbench & Vectors
Functional Verification
Logical Synthesis
Static Timing
Place & Route
Design Rule Checking (DRC)
Layout vsSchematic (LVS)
Layout Parasitic Extraction (LPE)
Manufacture& Validate
100s of Books
>10,000 Papers
10s of Classes
Making Software Work
Static Code Analysis
Invariant Checker
Interactive Debugger
Model Checking
Run-time Checker
Specification
TestbenchFunctional Description
(Code)
$10B tool businesssupports a
$300B S/W industry
100s of Books
>100,000 Papers
10s of Classes
Making Networks Work (Today)
traceroute, ping, tcpdump, SNMP, Netflow
…. er, that’s about it.
Why debugging networks is hard
Complex interaction – Between multiple protocols on a switch/router.
– Between state on different switches/routers.
Multiple uncoordinated writers of state.
Operators can’t…– Observe all state.
– Control all state.
Networks are kept working by
“Masters of Complexity” A handful of books
Almost no papers
No classes
Philosophy of Making Networks Work
YoYo“You’re On Your Own”
Yo-Yo Ma“You’re On Your Own, Mate”
With SDN we can:1. Formally verify that our networks are
behaving correctly.
2. Identify bugs, then systematically track down their root cause.
Software Defined Network (SDN)
Global Network View
Network Virtualization
PacketForwarding
PacketForwarding
PacketForwarding
PacketForwarding
Abstract Network View
Control
Programs
Control
Programs
Control
Programs
firewall.c
…
if( pkt->tcp->dport == 22)
dropPacket(pkt);
…
PacketForwarding
Network OS
1. <Match, Action>2. <Match, Action>3. <Match, Action>4. <Match, Action>5. <Match, Action> 6. …7. …
Three of our projects
1. Static Checking [HSA]“Independently checking correctness”
2. Automatic Testing [ATPG]“Is the datapath behaving correctly?”
3. Interactive Debugging [ndb]“Finding bugs, and their root cause, in an operational network”
Peyman Kazemian
Hongyi ‘James’
Zeng
GeorgeVarghese(UCSD)
NikhilHandigol
BrandonHeller
VimalJeyakumar
DavidMazières
1. Static checkingIndependently checking correctness
Motivations
In today’s networks, simple questions are hard to answer:
– Can host A talk to host B?
– What are all the packet headers from A that can reach B?
– Are there any loops in the network?
– Is Group X provably isolated from Group Y?
– What happens if I remove a line in the config file?
57
Software Defined Network (SDN)
Global Network View
Network Virtualization
PacketForwarding
PacketForwarding
PacketForwarding
PacketForwarding
Abstract Network View
Control
Programs
Control
Programs
Control
Programs
PacketForwarding
Network OS1. <Match, Action>2. <Match, Action>3. <Match, Action>4. <Match, Action>5. <Match, Action> 6. …7. …
1. <Match, Action>2. <Match, Action>3. <Match, Action>4. <Match, Action>5. <Match, Action> 6. …7. …
1. <Match, Action>2. <Match, Action>3. <Match, Action>4. <Match, Action>5. <Match, Action> 6. …7. …
1. <Match, Action>2. <Match, Action>3. <Match, Action>4. <Match, Action>5. <Match, Action> 6. …7. …
1. <Match, Action>2. <Match, Action>3. <Match, Action>4. <Match, Action>5. <Match, Action> 6. …7. …
1. <Match, Action>2. <Match, Action>3. <Match, Action>4. <Match, Action>5. <Match, Action> 6. …7. …
1. <Match, Action>2. <Match, Action>3. <Match, Action>4. <Match, Action>5. <Match, Action> 6. …7. …
1. <Match, Action>2. <Match, Action>3. <Match, Action>4. <Match, Action>5. <Match, Action> 6. …7. …
1. <Match, Action>2. <Match, Action>3. <Match, Action>4. <Match, Action>5. <Match, Action> 6. …7. …
Static Checker
1. <Match, Action>2. <Match, Action>3. <Match, Action>4. <Match, Action>5. <Match, Action> 6. …7. …
1. <Match, Action>2. <Match, Action>3. <Match, Action>4. <Match, Action>5. <Match, Action> 6. …7. …
1. <Match, Action>2. <Match, Action>3. <Match, Action>4. <Match, Action>5. <Match, Action> 6. …7. …
1. <Match, Action>2. <Match, Action>3. <Match, Action>4. <Match, Action>5. <Match, Action> 6. …7. …
1. <Match, Action>2. <Match, Action>3. <Match, Action>4. <Match, Action>5. <Match, Action> 6. …7. …
“A can talk to B”
“Guests can’t reach PatientRecords”
Policy
How it works
Header Space Analysis
Header Space Analysis
12
34
1
2
3
4
Header Space Analysis
12
34
1
2
3
4
Port ID
Can A talk to B?
12
34
1
2
3
4
Port ID
All packets from A that can reach B
Header Space Analysis*NSDI ‘12+
Consequences– Abstract forwarding model; protocol independent
– Finds all packets from A that can reach B
– Find loops, regardless of protocol or layer
– Can prove that two groups are isolated
Can verify if network adheres to policy
Software
Hassel tool
– Reads Cisco IOS Configuration
– Checks reachability, loops and isolation
– C: 60ms for Stanford Backbone
– Python: 10 mins for Stanford Backbone
Code– http://bitbucket.org/peymank/hassel-public
Checklist
• Intellectually interesting.
• And improves the practice.
• And industry doesn’t like (yet).
In Summary…
What has worked for me
• There doesn’t need to be conflict between– An intellectually interesting idea, and
– Improving the practice.
• Plenty of research problems can do both.
• Improving the practice means going beyond writing papers. Whatever it takes.
• Listen to industry, but not too much.
• Give ideas away.
Software Defined Networks
• Allows a stronger intellectual foundation to networking
• Allows us to define the right abstractions
• Will allow us to transfer technology much faster, in both directions
• Is already closing the gap with industry
Almost the end…
Something that worries me
Internet: biggest societal change in our time.
• Our field is growing in size and scope.
• More researchers, more topics, more developers.
• Yet the Sigcomm tent remains very small.
Sigcomm
• Too far removed from the practice.
• Too conservative.
• Rule of thumb:
– If the area is new, Sigcomm won’t accept it.
• Bad for students and young professors.
Sigcomm attendees
88% 87% 91% 86%
12% 13% 9% 14%
0%
50%
100%
2008 2009 2010 2011
Industry, not research
Research
( Attendees Industry% Papers )
Sigcomm( 500 15% 35 )
Siggraph( 13,000 60% 115 )
Sigcomm 2015 ? ( 2,000 40% 70 )
Sigcomm
Let’s make the tent bigger
– More topics
– More papers
– More industry participation
The End
Giving the network a wedgie
Nick McKeownStanford University
Specialized
Control
Plane
Specialized
Hardware
Specialized
Features
Apps
Specialized
Control
Plane
Specialized
Hardware
Apps
Network OS
Simpler
Hardware