networks and distributed systems

Networks and distributed systems 1

Networks and distributed systems

Jinyang Li

Jinyang@cs.nyu.edu

Lec 1:Evolution of the Internet

Networks and distributed systems 2

Know your staff

• Instructor: prof. Jinyang Li – jinyang@cs.nyu.edu– Office Hour: Wed 5-6pm (715 Broadway Rm 705)

• Class webpagehttp://www.news.cs.nyu.edu/classes/fa07

• Register for class mailing list

Networks and distributed systems 3

The course will teach you …

• to appreciate design principles of the Internet– How it works and why it works

• to address new networking challenges– How to do independent research

Networks and distributed systems 4

Who should take this class?

• Core grad-level class– Satisfy M.S. requirement of a “project” class– Satisfy Ph.D. breadth requirement

• Pre-requisite:– Basic knowledge on networks– Programming experience

• Useful books:– Computer networks (Peterson & Davie)– TCP/IP illustrated (Stevens)

Networks and distributed systems 5

Class material

• Lectures/readings– Read assigned research papers before class– Participate in class discussion

• Assignments– Solve concrete problems, get your hands dirty!

• Projects– Can you identify and tackle a challenge with guidance?

Networks and distributed systems 6

Grading

• Participation 20%– two in-class mini-quiz on “readings du jour”

• Two take home assignments 20%

• Project 60%– Teams of 2-3 people– Starting new week– Bi-weekly meetings with me

Networks and distributed systems 7

Questions?

• Sign up sheet

Networks and distributed systems 8

A brief history of communication

• Telephone networks– Dial to set up a path– Paths carry analog voice signals from one

phone to another– Networking means building paths

Networks and distributed systems 9

Building paths connecting wires

Switchboard Operators 1960

Networks and distributed systems 10

The quest for a survivable network

• Sputnik --> ARPA --> survivable networks

• Telephone network is not survivable– Destroy of a switching center is highly disruptive– Not possible to build reliable paths under attacks

Networks and distributed systems 11

Packet switching

• Baran & Davies (60s)

• Packets are digital, self-contained, of limited size• Decentralized store and forward

– Networking means delivering packets to endpoints

SrcAddr

Dstaddr

pktlen

header payload

Networks and distributed systems 12

An example of packet switchingH2

H1

H2 H1

H1:P4H2:P1H3:P2

1

234

H3

H1:P1H2:P2H3:P3

12

3

H2 H1

Networks and distributed systems 13

ARPANET

Networks and distributed systems 14

Internet: Connecting many networks

• Many packet switching networks– ARPANET, Packet radio, SATnet

• Goal: make networks work together!

• Solution: TCP/IP

Kahn &Cerf

Networks and distributed systems 15

Alternative #1: single technology, single network

• Render existing networks/apps useless• Does not accommodate new technology• Hard for decentralized control• (early phone network is like this)

Networks and distributed systems 16

Alternative #2: Translation Gateway

• Translation is hard– different features/headers, N^2 combinations!– How to translate addresses?

Translationgateway

H1: ABCD

H2: 计算机

Networks and distributed systems 17

3. Internet wins

• IP over everything – A uniform header / addressing format

IP router

H1: 18.26.4.9

H2: 128.122.108.71

H1,H2’s IP addrH2, GW’s low-level addrH1, GW’s

low-level addr

Networks and distributed systems 18

Internet design challenges

• How to address networks and hosts?– Address size? Resolve IP addr to subnet addr?

• How to compute route and forward packets? • How to reliably deliver packets?

– Error recovery– Flow control

• How to cope with different max packet size?

Networks and distributed systems 19

Addressing scheme

• Early 80s: – 32-bit globally unique IP address– 8 bit net number, 24 bit host number– Embed subnet address to low 24 bit

•Now: 32-bit– Variable length net number (CIDR)– Address resolution protocol (ARP) to obtain subnet addr (MAC addr) from IP

Networks and distributed systems 20

Routing• Early 80s:

– 256-entry routing table, indexed by top 8 bits of addr– Static default g/w

•Now: – Intra-domain routing: OSPF, RIP– Inter-domain routing: BGP– approx. 250,000 BPG entries now

Networks and distributed systems 21

Reliable delivery

• Early 80s– IP is best-effort only– TCP ran at end hosts for error/flow control

•Now: – IP is best effort only– TCP is separated from IP– TCP performs both error and congestion control

Networks and distributed systems 22

Packet size policy

• Early 80s:– Senders only know local net’s MTU– G/Ws fragment large packets into smaller MTUS– End hosts reassembles fragments

•Now: same. :-)

Networks and distributed systems 23

“Internet” demo 1977

ARPANET

PRnet

SateNET

Networks and distributed systems 24

Internet map 1987

Networks and distributed systems 25

Why TCP/IP wins?

• Universal– IP-over-everything– Best effort only– End-to-end design

• Robust– Soft-state only inside network– Fate sharing– Be liberal in what you accept; be conservative in

what you send

Networks and distributed systems 26

Internet’s growing stage

• 1978 TCP/IP split• 1984 Domain name system • 1986 Incorporating congestion control in TCP• 1990 ARPANET disappears, first ISP is born• Nodes double every year….

Networks and distributed systems 27

The revolution, good and bad

• Email 1971• Apple II 1977, IBM PC 1981• Web 1990• VoIP, File sharing, Video streaming, Web 2.0

• Worms 1988, viruses• DoS attacks• Spam

Networks and distributed systems 28

Internet design goals

1. Interconnect different networks– Packet switching– Uniform addressing and IP header

2. Robust– Route packets instead of building path– Network is state-less, forwards packets based on addr

3. Flexible– IP is best effort only– Separate TCP from IP

Networks and distributed systems 29

The more problematic goals

4. Decentralization– Routing across multiple admin domains is

still error-prone

5. Cheap and easy to attach new nodes– Cumbersome to attach new devices, move

existing ones around

6. Accountability

Networks and distributed systems 30

Internet weaknesses

• Assumes trusted participants• Assumes non-greedy sources• Security• Hard to incrementally deploy new protocols

Networks and distributed systems 31

New challenges

Networks and distributed systems 32

New types of networks: wireless

2007 MIT Cartel

Networks and distributed systems 33

New networks: wireless mesh

Networks and distributed systems 34

New networks: sensor

Networks and distributed systems 35

New services

• What’s the next killer app?

Networks and distributed systems 36

Battling existing woes

Networks and distributed systems 37

Battling existing woes

Networks and distributed systems 38

Course Syllabus

1. Core networking concepts– Naming and addressing– Routing– Managing shared resources

2. Wireless

3. Network services

4. Security

Networks and distributed systems 39

Part I: Core networking concepts

Reliable transport

Networks and distributed systems 40

Coping with best-effort

• Why don’t applications use IP directly?– IP is a host-to-host protocol– Many applications want reliable, in-order delivery

Networks and distributed systems 41

TCP software architecture

browser ssh

kernel

User-space

apache sshd

kernel

User-space

write read

Networks and distributed systems 42

Coping with best-effort

• Challenges for a reliable transport protocol– Loss– Variable delays– Packet reordering– Duplicate packets

Networks and distributed systems 43

TCP overview• Provides in-order, reliable, duplex byte-streams

• Uses cumulative ACKs

Src port

Dst port Seq # Ack #

flags

window

cksum

1461 1701 1701 1701Ack:

1:1460 1461:1700 2000:2500 2501:2800Data: 1701:1999

Networks and distributed systems 44

Reliability via retransmission

• How does TCP know when to re-transmit?

• Timer driven– No ACKs for a while…

• Data driven– Many duplicate ACKs

Networks and distributed systems 45

Timer-driven retransmission

• What is the ideal time to retransmit?

• What if we literally use RTT as timeout?

Networks and distributed systems 46

Timer-driven retransmission

• Calculate running average of RTT– EWMA: srtt = * r + (1 - ) * srtt

• Set timeout (RTO)– Used to use RTO = 2 * srtt– Now: RTO = srtt + 4 * rttdev

rttdev = * |r-srtt| + (1- ) * rttdev

Networks and distributed systems 47

An example RTT distribution

Avg: 99.2msStd: 1.4ms

Networks and distributed systems 48

TCP timers

• What if a retransmission times out?– Exponential back off

• TCP timeouts are extremely conservative– Granularity of 500ms or 200ms

Networks and distributed systems 49

Fast retransmit

• If a segment is lost, duplicate ACKs result

• TCP retransmit upon seeing 3 duplicate ACKs

1:1460 1461:1700 2000:2500 2501:2800Data: 1701:1999

1461 1701 1701 1701Ack:

Networks and distributed systems 50

Fast retransmit

• What would trick fast retransmit into spurious retransmission?

• When would fast retransmit fail to avoid timeout?– Loss of a re-sent packet – Multiple losses in a window

Networks and distributed systems 51

Fast Recovery

• How should sender change its congestion window due to loss?– Unchanged?– Set to 1?– Half ?

Networks and distributed systems 52

Is TCP good for all applications?

• TCP imposes high delay for retransmitted packets

• TCP enforces in-order delivery