Networks and distributed systems 1
Networks and distributed systems
Jinyang Li
Lec 1:Evolution of the Internet
Networks and distributed systems 2
Know your staff
• Instructor: prof. Jinyang Li – [email protected]– Office Hour: Wed 5-6pm (715 Broadway Rm 705)
• Class webpagehttp://www.news.cs.nyu.edu/classes/fa07
• Register for class mailing list
Networks and distributed systems 3
The course will teach you …
• to appreciate design principles of the Internet– How it works and why it works
• to address new networking challenges– How to do independent research
Networks and distributed systems 4
Who should take this class?
• Core grad-level class– Satisfy M.S. requirement of a “project” class– Satisfy Ph.D. breadth requirement
• Pre-requisite:– Basic knowledge on networks– Programming experience
• Useful books:– Computer networks (Peterson & Davie)– TCP/IP illustrated (Stevens)
Networks and distributed systems 5
Class material
• Lectures/readings– Read assigned research papers before class– Participate in class discussion
• Assignments– Solve concrete problems, get your hands dirty!
• Projects– Can you identify and tackle a challenge with guidance?
Networks and distributed systems 6
Grading
• Participation 20%– two in-class mini-quiz on “readings du jour”
• Two take home assignments 20%
• Project 60%– Teams of 2-3 people– Starting new week– Bi-weekly meetings with me
Networks and distributed systems 7
Questions?
• Sign up sheet
Networks and distributed systems 8
A brief history of communication
• Telephone networks– Dial to set up a path– Paths carry analog voice signals from one
phone to another– Networking means building paths
Networks and distributed systems 9
Building paths connecting wires
Switchboard Operators 1960
Networks and distributed systems 10
The quest for a survivable network
• Sputnik --> ARPA --> survivable networks
• Telephone network is not survivable– Destroy of a switching center is highly disruptive– Not possible to build reliable paths under attacks
Networks and distributed systems 11
Packet switching
• Baran & Davies (60s)
• Packets are digital, self-contained, of limited size• Decentralized store and forward
– Networking means delivering packets to endpoints
SrcAddr
Dstaddr
pktlen
header payload
Networks and distributed systems 12
An example of packet switchingH2
H1
H2 H1
H1:P4H2:P1H3:P2
1
234
H3
H1:P1H2:P2H3:P3
12
3
H2 H1
Networks and distributed systems 13
ARPANET
Networks and distributed systems 14
Internet: Connecting many networks
• Many packet switching networks– ARPANET, Packet radio, SATnet
• Goal: make networks work together!
• Solution: TCP/IP
Kahn &Cerf
Networks and distributed systems 15
Alternative #1: single technology, single network
• Render existing networks/apps useless• Does not accommodate new technology• Hard for decentralized control• (early phone network is like this)
Networks and distributed systems 16
Alternative #2: Translation Gateway
• Translation is hard– different features/headers, N^2 combinations!– How to translate addresses?
Translationgateway
H1: ABCD
H2: 计算机
Networks and distributed systems 17
3. Internet wins
• IP over everything – A uniform header / addressing format
IP router
H1: 18.26.4.9
H2: 128.122.108.71
H1,H2’s IP addrH2, GW’s low-level addrH1, GW’s
low-level addr
Networks and distributed systems 18
Internet design challenges
• How to address networks and hosts?– Address size? Resolve IP addr to subnet addr?
• How to compute route and forward packets? • How to reliably deliver packets?
– Error recovery– Flow control
• How to cope with different max packet size?
Networks and distributed systems 19
Addressing scheme
• Early 80s: – 32-bit globally unique IP address– 8 bit net number, 24 bit host number– Embed subnet address to low 24 bit
•Now: 32-bit– Variable length net number (CIDR)– Address resolution protocol (ARP) to obtain subnet addr (MAC addr) from IP
Networks and distributed systems 20
Routing• Early 80s:
– 256-entry routing table, indexed by top 8 bits of addr– Static default g/w
•Now: – Intra-domain routing: OSPF, RIP– Inter-domain routing: BGP– approx. 250,000 BPG entries now
Networks and distributed systems 21
Reliable delivery
• Early 80s– IP is best-effort only– TCP ran at end hosts for error/flow control
•Now: – IP is best effort only– TCP is separated from IP– TCP performs both error and congestion control
Networks and distributed systems 22
Packet size policy
• Early 80s:– Senders only know local net’s MTU– G/Ws fragment large packets into smaller MTUS– End hosts reassembles fragments
•Now: same. :-)
Networks and distributed systems 23
“Internet” demo 1977
ARPANET
PRnet
SateNET
Networks and distributed systems 24
Internet map 1987
Networks and distributed systems 25
Why TCP/IP wins?
• Universal– IP-over-everything– Best effort only– End-to-end design
• Robust– Soft-state only inside network– Fate sharing– Be liberal in what you accept; be conservative in
what you send
Networks and distributed systems 26
Internet’s growing stage
• 1978 TCP/IP split• 1984 Domain name system • 1986 Incorporating congestion control in TCP• 1990 ARPANET disappears, first ISP is born• Nodes double every year….
Networks and distributed systems 27
The revolution, good and bad
• Email 1971• Apple II 1977, IBM PC 1981• Web 1990• VoIP, File sharing, Video streaming, Web 2.0
• Worms 1988, viruses• DoS attacks• Spam
Networks and distributed systems 28
Internet design goals
1. Interconnect different networks– Packet switching– Uniform addressing and IP header
2. Robust– Route packets instead of building path– Network is state-less, forwards packets based on addr
3. Flexible– IP is best effort only– Separate TCP from IP
Networks and distributed systems 29
The more problematic goals
4. Decentralization– Routing across multiple admin domains is
still error-prone
5. Cheap and easy to attach new nodes– Cumbersome to attach new devices, move
existing ones around
6. Accountability
Networks and distributed systems 30
Internet weaknesses
• Assumes trusted participants• Assumes non-greedy sources• Security• Hard to incrementally deploy new protocols
Networks and distributed systems 31
New challenges
Networks and distributed systems 32
New types of networks: wireless
2007 MIT Cartel
Networks and distributed systems 33
New networks: wireless mesh
Networks and distributed systems 34
New networks: sensor
Networks and distributed systems 35
New services
• What’s the next killer app?
Networks and distributed systems 36
Battling existing woes
Networks and distributed systems 37
Battling existing woes
Networks and distributed systems 38
Course Syllabus
1. Core networking concepts– Naming and addressing– Routing– Managing shared resources
2. Wireless
3. Network services
4. Security
Networks and distributed systems 39
Part I: Core networking concepts
Reliable transport
Networks and distributed systems 40
Coping with best-effort
• Why don’t applications use IP directly?– IP is a host-to-host protocol– Many applications want reliable, in-order delivery
Networks and distributed systems 41
TCP software architecture
browser ssh
kernel
User-space
apache sshd
kernel
User-space
write read
Networks and distributed systems 42
Coping with best-effort
• Challenges for a reliable transport protocol– Loss– Variable delays– Packet reordering– Duplicate packets
Networks and distributed systems 43
TCP overview• Provides in-order, reliable, duplex byte-streams
• Uses cumulative ACKs
Src port
Dst port Seq # Ack #
flags
window
cksum
1461 1701 1701 1701Ack:
1:1460 1461:1700 2000:2500 2501:2800Data: 1701:1999
Networks and distributed systems 44
Reliability via retransmission
• How does TCP know when to re-transmit?
• Timer driven– No ACKs for a while…
• Data driven– Many duplicate ACKs
Networks and distributed systems 45
Timer-driven retransmission
• What is the ideal time to retransmit?
• What if we literally use RTT as timeout?
Networks and distributed systems 46
Timer-driven retransmission
• Calculate running average of RTT– EWMA: srtt = * r + (1 - ) * srtt
• Set timeout (RTO)– Used to use RTO = 2 * srtt– Now: RTO = srtt + 4 * rttdev
rttdev = * |r-srtt| + (1- ) * rttdev
Networks and distributed systems 47
An example RTT distribution
Avg: 99.2msStd: 1.4ms
Networks and distributed systems 48
TCP timers
• What if a retransmission times out?– Exponential back off
• TCP timeouts are extremely conservative– Granularity of 500ms or 200ms
Networks and distributed systems 49
Fast retransmit
• If a segment is lost, duplicate ACKs result
• TCP retransmit upon seeing 3 duplicate ACKs
1:1460 1461:1700 2000:2500 2501:2800Data: 1701:1999
1461 1701 1701 1701Ack:
Networks and distributed systems 50
Fast retransmit
• What would trick fast retransmit into spurious retransmission?
• When would fast retransmit fail to avoid timeout?– Loss of a re-sent packet – Multiple losses in a window
Networks and distributed systems 51
Fast Recovery
• How should sender change its congestion window due to loss?– Unchanged?– Set to 1?– Half ?
Networks and distributed systems 52
Is TCP good for all applications?
• TCP imposes high delay for retransmitted packets
• TCP enforces in-order delivery