Masaki Hirabaru<masaki@crl.go.jp>

Internet Architecture Group

GL MeetingMarch 19, 2004

High Performance Data transfer onHigh Bandwidth-Delay Product Networks

VLBI (Very Long Baseline Interferometry)

(CRL鹿島宇宙電波応用Ｇホームページから )

Motivations• MIT Haystack – CRL Kashima e-VLBI Experiment

on August 27, 2003 to measure UT1-UTC in 24 hours– 41.54 GB CRL => MIT 107 Mbps (~50 mins)

41.54 GB MIT => CRL 44.6 Mbps (~120 mins)

– RTT ~220 ms, UDP throughput 300-400 MbpsHowever TCP ~6-8 Mbps (per session, tuned)

– BBFTP with 5 x 10 TCP sessions to gain performance

• HUT – CRL Kashima Gigabit VLBI Experiment

- RTT ~325 ms, UDP throughput ~70 MbpsHowever TCP ~2 Mbps (as is), ~10 Mbps (tuned)

- Netants (5 TCP sessions with ftp stream restart extension)

They need high-speed / real-time / reliable / long-haul high-performance, huge data transfer.

Purpose

• Measure, analyze and improve end-to-end performance in high bandwidth-delay product networks– to support for networked science applications– to help operations in finding a bottleneck– to evaluate advanced transport protocols

(e.g. Tsunami, SABUL, HSTCP, FAST, XCP, ikob)

• Improve TCP under easier conditions

Assumptions• Packet Switching Network

– Shared, Best-Effort

• End-to-End Principle– No hard state

Contents• Advanced TCP evaluation on TransPAC / Internet2• Advanced TCP evaluation in laboratory• Research Topics in 2004

　

KwangjuBusan

2.5G

Fukuoka

Korea

                                       

2.5G SONET

KORENTaegu

Daejon

10G

0.6G1Gx2

1Gx2

QGPOP

Seoul XP

Genkai XP

Kitakyushu

Tokyo XP

Kashima

0.1G

Fukuoka Japan

250km

1,000km2.5G

TransPAC

9,000km

4,000km

Los Angeles

Cicago

New York

MIT Haystack

HUT

10G

1G

APII/JGN

Abilene

0.1GHelsinki

2.4G

Stockholm

0.6G

2.4G

2.4G

GEANT

Nordunet

funetKoganei

1G

7,000km

Indianapolis

I2 Venue1G

10G

100km

server (general)

server (e-VLBI)

Abilene Observatory: servers at each NOC

CMM: common measurement

machines

Network Diagram for TransPAC/I2 Measurement(Oct. 2003)

1G x2

sender

receiver

Mark5Linux 2.4.7 (RH 7.1)P3 1.3GHzMemory 256MBGbE SK-9843

PE1650Linux 2.4.22 (RH 9)Xeon 1.4GHzMemory 1GBGbE Intel Pro/1000 XT

Iperf UDP ~900Mbps (no loss)

TransPAC/I2 #2: High Speed (60 mins)

Evaluating Advanced TCPs

• new Reno (Linux TCP, web100 version)– Ack: w=w+1/w, Loss: w=w-1/2*w

• HighSpeed TCP (included in web100)– Ack: w=w+a(w)/w, Loss: w=w-1/b(w)*w

• FAST TCP (binary, provided from Caltech)– w=1/2*(w_old*baseRTT/avgRTT+α+w_current)

• Limited Slow-Start (included in web 100)

Note: Differences in sender side only

Path

ReceiverSender

Backbone

B1 <= B2 & B1 <= B3

Access Access

B1B2

B3

a) w/o bottleneckqueue

ReceiverSender

Backbone

B1 > B2 || B1 > B3

Access Access

B1 B2 B3

b) w/ bottleneck (congestion)queue

bottleneck

Test in a laboratory – with bottleneck

PacketSphere

ReceiverSender

L2SW(12GCF)

Bandwidth 800Mbps Buffer 256KBDelay 88 msLoss 0

GbE/SX

GbE/TGbE/T

PE 2650 PE 1650

• #4: Reno => Reno

• #5: High Speed TCP => Reno

• #6: FAST TCP => Reno

2*BDP = 16MB

#4, 5: Data obtained on sender #6: Data obtained on receiver

Laboratory #4,#5,#6: 800M bottleneck

Reno

HighSpeed

FAST

(default setting)

Laboratory #5: High Speed (Limiting)

Window Size(16MB)

Rate Control

Cwnd Clamp

270 us every 10 packetsWith limited slow-start (1000)

(95%)With limited slow-start (100)

With limited slow-start (1000)

Laboratory #6: FAST

(alpha=17, beta=18,

gamma=2)

Research topics in 2004

Internet

Bottleneck

ReceiverSender Router

Bottleneck

ACK (100 ms delay)

100 x 1KB = 100KB ~ 1 ms at Gbps

100M1G

1G

100M

Throughput

Window size

•Parameter Auto-tuning (w/ packet pair/train)

•TCP Analysis WEB Tool with web100

•Router queue length

•RED on the bottleneck

•Signal from the bottleneck

•Mobile TCP (Mobile router support)

•Multi-Homing TCP (ID+Locator)

•I2 / piPEs collaboration (bwctl)

•APII / CMM collaboration with Korea

•Real-time Gbps e-VLBI experiment between CRL Kashima and MIT Haystack