고려대 컴퓨터구조 / 병렬컴퓨팅 연구실 1 LAN(ethernet switch) 기반 클러스터 시스템 김 동 승 컴퓨터구조 / 병렬 컴퓨팅 연구실 고려대학교

고려대 컴퓨터구조 / 병렬컴퓨팅 연구실

1

LAN(ethernet switch) LAN(ethernet switch) 기반기반클러스터 시스템클러스터 시스템

김 동 승

컴퓨터구조 / 병렬 컴퓨팅 연구실 고려대학교

1999.5.29


2


3

IBM SP2IBM SP2

•• Distributed memory multiprocessor

• • RS/6000 processors, routing processors

• • Multistage interconneciton network

with wormhole routing


4

PC clusterPC cluster (Beowulf)

High-performance, low-cost microprocessors WITH

– Low cost network (Sw

itchied ethernet, Fast ethernet, Myrinet)

– Linux, Windows NT, ...

– MPI, PVM, BSP


5

장점 단점

저비용 고성능 실현 설치가 용이한

운영체제 활용 Networking 의

편리성 각 노드가 하나의

온전한 컴퓨터 대규모 시스템으로

확장가능

통신 속도가 느림 관리 책임이

사용자에게 클러스터용 전문

프로그램의 미비 이기종간에는 설치 /

운영이 힘듦


6

PC clusters

Avalon - Los Alamos N. Lab.#315/500 super computers

70 Alpha 164 LX(533Mhz) - 19 TFlops

Hyglac -JPL16 Pentium Pro’s - 3.2 GFlops

Naegling - JPL120 Pentium Pro’s - 24 GFlops


7

소요비용

총 제작비 (16 PCs): 약 3,800 만원 - 1998.10 월

ethernet switch (24 ports) - 400 만원 PC 9 대 x 260 만원 ( 완벽 ) PC 7 대 x 150 만원

- No monitor, keyboard, CD ROM drive


8

Our system configuration

16 Pentium II PCs– CPU : Pentium II - 300 MHz– RAM : 64M Bytes – Fixed Hard Disk 4.3G– VGA : PCI S3 Virdge/DX– Motherboard : ABIT BX6– LAN card : Intel Ethernet Express+(100Mbps)


9

Ethernet Switch

Bay Networks Baystack 450-24T High speed forwarding rate: max. 3M pa

ckets per second Autosensing and autonegotiation: 10/1

00 Mbps Frame length: 64 to 1518 bytes Segment switch application


10

공유 HUB vs. 스위치


11

프로세서간 통신성능 / 한계

준비시간 (setup time - ts) 점대점 통신 대역폭 (bandwidth - B) 통합 대역폭


12


13

Modeling of message passing communication

T(m) = ts + tb m (microsecond)

SP2: ts = 46 tb = 0.035 (28.6MB/s)

T3E: ts = 17 tb = 0.006 (167MB/s)

Cluster: ts = 177 tb = 0.105 (9.52MB/s)


14

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

1024

62464

1E

+05

2E

+05

2E

+05

3E

+05

4E

+05

4E

+05

5E

+05

6E

+05

6E

+05

7E

+05

7E

+05

8E

+05

9E

+05

9E

+05

1E

+06

1E

+06

message s ize[byte]

tim

e[s

ec

]

점대점 통신 성능


15

0.E+00

2.E+06

4.E+06

6.E+06

8.E+06

1.E+07

1.E+07

1024

66560

1E

+05

2E

+05

3E

+05

3E

+05

4E

+05

5E

+05

5E

+05

6E

+05

7E

+05

7E

+05

8E

+05

9E

+05

9E

+05

1E

+06

message s ize[byte]

Ba

ndw

idth

[Mb

yte

/s]

Network Bandwidth


16

1e+02

1e+03

1e+04

1e+05

1e+06

2 4 8 16 32 64 128 256 512 1024

time[

usec

]

message size[KBytes]

generic pipe2

Kseg nEBST

1e+02

1e+03

1e+04

1e+05

1e+06

2 4 8 16 32 64 128 256 512 1024

time[

usec

]


generic pipe2 Kseg

nEBST

broadcast on SP2 (P=8) broadcast on SP2 (P=16)


17

1e+03

1e+04

1e+05

1e+06

2 4 8 16 32 64 128 256 512 1024

time[

usec

]


generic pipe2

Kseg nEBST

1e+03

1e+04

1e+05

1e+06

2 4 8 16 32 64 128 256 512 1024

time[

usec

]


generic pipe2 Kseg

nEBST

broadcast on cluster (P=8) broadcast on cluster (P=16)


18

1e+03

1e+04

1e+05

1e+06

2 4 8 16 32 64 128 256 512 1024

time[

usec

]


generic rootp brent pipe

1e+03

1e+04

1e+05

1e+06

2 4 8 16 32 64 128 256 512 1024

time[

usec

]


generic rootp brent pipe

scan on SP2 (P=8) scan on SP2 (P=16)


19

1e+03

1e+04

1e+05

1e+06

1e+07

2 4 8 16 32 64 128 256 512 1024

time[

usec

]


generic rootp brent

pipe

1e+03

1e+04

1e+05

1e+06

1e+07

2 4 8 16 32 64 128 256 512 1024

time[

usec

]


generic rootp brent

pipe

scan on cluster (P=8) scan on cluster (P=16)


20

Matrix multiplicatoinMatrix multiplicatoin

PC Cluster (8 processors)SP2 (16 porcessors)

0.01

0.1

1

10

256 384 512 640 768 896 1024

N

sec

0.01

0.1

1

10

100

256 512 1024N

sec

E_comm

E_comp

E_total

P_comm

P_comp

P_total


21

Analytic prediction & experimental resutltsAnalytic prediction & experimental resutlts

(bitonic sort)(bitonic sort)

PC cluster - 8 processorsSP2 - 8 processors

0.1

1.0

10.0

100.0

0.5M 1M 2M 4M

key

sec

0.1

1

10

0.25M 0.5M 1M 2Mpoint

sec

E_comm

E_comp

E_total

P_comm

P_comp

P_total


22

FFTFFT

PC Cluster (8 processors)SP2 (8 processors)

0.01

0.10

1.00

10.00

0.25M 0.5M 1M 2M

point

sec

0.1

1

10

0.25M 0.5M 1M 2M

point

sec

E_comm

E_comp

E_total

P_comm

P_comp

P_total


23

Parallel radix sortParallel radix sort

1. Count : bucket sort, and get key counts.

2. Plan : gather key count information, then determine:

Who will be in charge of each bucket of keys?

3. Exchange: move keys to selected owner processors

고려대학교 전기전자전파공학부 24

Load-balanced radix sort [Sohn98]

versus

Partitioned radix sort [Lee99]


25

Uniform (SP2)분포의 통신 시간 비교

0

1

2

3

4

5

6

7

8

9

10

2 4 8 16 32

프로세서 개수

(

)통

신시

간초

1M(LB)1M(SJ )2M(LB)2M(SJ )4M(LB)4M(SJ )8M(LB)8M(SJ )16M(LB)16M(SJ )32M(LB)32M(SJ )64M(LB)64M(SJ )

Times for communication (SP-2)

The number of porcessors

Time


26

Gaussian (SP2)분포

0.1

1

10

100

2 4 8 16 32

프로세서 개수

(

)실

행시

간초


Execution times: (SP-2)


27

Staggered (Cluster)분포

0.1

1

10

100

2 4 8 16

프로세서 개수

(

)실

행시

간초


Execution times: (PC cluster)


28

결 론

PC cluster - 저비용 , 고성능 Slow interprocessor communication

(Gigabit ethernet, 광통신 )

소용처 - Compute-bound 응용분야


29

클러스터형 WWW 서버


30

참고문헌 / 정보제공처

Beowulf 시스템 제작 방법- Beowulf Project at CESDIS, http://beowulf.gsfc.nasa.gov/- Beowulf HOWTO , http://www.sci.usq.edu.au/staff/jacek/beowulf/HOWTO/- How to Build a Beowulf: a Tutorial, http://www.cacr.caltech.edu/beowulf/tutorial/ 리눅스 관련- Linux Homepage, http://www.linux.org- Red Hat, http://www.redhat.com/- Beowulf Ethernet Channel Bonding, http://beowulf.gsfc.nasa.gov/software/ bonding.html MPI& PVM 관련- The Message Passing Interface (MPI) , http://www.erc.msstate.edu/mpi/, http://www-unix.mcs.anl.gov/mpi/index.html- MPICH(MPI Chameleon), http://www-unix.mcs.anl.gov/mpi/mpich/- PVM(Parallel Virtual Machine), http://www.epm.ornl.gov/pvm/pvm_home.html

Documents

고려대 컴퓨터구조 / 병렬컴퓨팅 연구실 1 LAN(ethernet switch) 기반 클러스터 시스템 김 동 승 컴퓨터구조 / 병렬 컴퓨팅 연구실 고려대학교