1

(1) Cluster computing(2) Grid computing)

　

Part 4 Current trend of parallel processing

2

Traditional parallel processing•　 Vector type or shared memory type or distributed memory type •　 Special technology/architecture • Very expensive (millions 〜 100 millions dollars), long developing period•　 One computer used by a number of users•　 Special ＯＳ and programming languages•　 Limited users

3

New trend of parallel processing

　 Cluster computing　 Grid computing

Features•　 Cheap to construct processing environments. •　 Easy to popularize because of the use of ordinary computers.

4

Cluster processing

What is cluster processing• Use a number of work stations or PCs to construct a virtual parallel computer.

• Comparing with parallel computers, cluster processing is cheap. • Performance is worse than parallel computers, but it can be largely improved by using high performance network （ 100BASE-TX, ATM, etc.).

5

Outline of global computing

What is global (GRID) computing • Use all the resources on internet to construct a super distributed parallel virtual computer

Intranet +

Internet

Super parallelcomputers

PC clusterUser

High performance PC

6

Implement of cluster processing (1)

Well known software for cluster processing•　 PVM (Parallel Virtual Machine)•　 MPI (Message Passing Interface)

History of PVM1989 　 PVM1.0 released by Oak Rige National Laboratory.1991 　 PVM2.0 released by University of Tennessee.1993 　 PVM3.0 released.1994 　 Published with free manual 2001 　 PVM 3.4.3 released with C and Fortran.

7

Implement of cluster processing (2)

History of MPI1993-1994 　 Specification (MP-I) was decided by the experts from 40 companies.1995 　 Specification of MPI-2 was decided.2001 　 MPI-1.1 based implementation was released.

•　 It is used as default environment in many distributed parallel processing systems.

8

Common points of PVM and MPI

•　 Communication is based on 1 to 1 message passing.

•　 They are designed for not only work stations and PCs but also parallel computers. •　 Corresponding to many OS and Languages.

（ OS: UNIX, Windows95/NT, Languages: C, Fortran, JAVA)•　 Free•　 Standard software in distributed and parallel computing.

send(5, message） receive(3)

Processor ID: ３ Processor ID: ５

9

Construction of PVM

Software of PVM•　 Daemon pvmd used for the communication between processors. •　 Console pvm used for constructing virtual parallel computer （ xpvm is the console combined with GUI) •　 Library of functions such as pvm_send and pvm_receive for message sending and receiving.

On any computer, one can use pvm or xpvm to construct a virtual parallel computer, and then make and execute programs.

A virtual parallel computer

pvmd pvmd

pvmdpvmd

(opty1) (optima)

(opty2) (opty3)

（ Constructed by connected 4 computers ）

10

Start of PVM (1)

(1) On any computer, use command pvm or xpvm to start pvmd.

pvmd

(opty1) (optima)

(opty2) (opty3)

On optima, use pvm or xpvm to start pvmd .

11

Start of PVM (2)

(2) 　 On console pvm add computers to construct a virtual parallel computer. (At the this time, pvmd is started in these computers.)

pvmd

(opty1) (optima)

(opty2) (opty3)

pvmd

pvmdpvmd

On optimaadd opty1, opty2, opty3 to virtual parallel computer by pvmd.

12

Execution of PVM (1)Program execution on PVM

(1) 　 When the program which contains communication functions of pvm is executed, the specified programs will be executed by pvmd

pvmd pvmd

pvmdpvmd

(opty1) (optima)

(opty2) (opty3)

prog1

prog1 prog1

prog1

When program prog1 is executed on optimawhich contains communication functions, the process (in this case prog 1) will be started in all processors by pvmd.

13

Execution of PVM (2)

(2) Once the execution of a program started, communication and processing inside the program will be done automatically until the end.

pvmd pvmd

pvmdpvmd

(opty1) (optima)

(opty2) (opty3)

prog1

prog1 prog1

prog1

Processing is held with the communication between processors.

14

Using PVM (1)Using command pvm

chen[11]% pvmpvm> add opty1 opty2 opty3 （ add opty1, opty2, opty3 to the virtual computer ）3 successful HOST DTID opty1 80000 opty2 c0000 opty3 100000pvm> conf 　 (Show the configuration of the virtual parallel computer)4 hosts, 1 data format HOST DTID ARCH SPEED DSIG optima 40000 X86SOL2 1000 0x00408841 opty1 80000 X86SOL2 1000 0x00408841 opty2 c0000 X86SOL2 1000 0x00408841 opty3 100000 X86SOL2 1000 0x00408841pvm> spawn edm4 　（ Execute program edm4 on the virtual computer)1 successfult80001pvm> halt 　　 （ Halt of the virtual computer ）Terminated

chen[12]%

15

Using PVM (2)

• 　 Using xpvm (X application)

16

PVM programming (1)Example (hello.c, hello_other.c)

(hello.c) #include "pvm3.h" main() { int tid, answer; pvm_spawn("hello_other",

(char**)0,0,"",1, &tid); printf("Hello, Dr.t%x\n",tid); pvm_recv(tid,1); pvm_upkint(&answer,1,1); printf("Fine, thank you,

Dr. t%x", answer); pvm_exit(); }

(hello_other.c) #include "pvm3.h" main() { int mytid, ptid; mytid = pvm_mytid(); ptid = pvm_parent(); printf("Good morning, Prof. t%x.\n",ptid); printf("How are you?\n"); pvm_initsend(PvmDataDefault); pvm_pkint(&mytid,1,1); pvm_send(ptid, 1); pvm_exit(); }

17

PVM Programming (2)

Start the process hello Hello, Dr. t80004

（ Receive the message by pvm_recv ） Fine, thank you, Dr. t80004

（ Process hello_other is started ） 　Hello, Prof. t4000a.

　How are you?

（ Send message by pvm_send ）

pvm_spawn

　 (Process hello_other is started on other computers by pvm_spawn. 　 Started process has its task ID.)

pvm_send

pvm_recv（ Finish the process by pvm_exit ）

（ Finish the process by pvm_exit ）

18

Grid computing (1)

•　 Distributed and parallel processing techniques-　 TCP/IP protocol such as telnet, rlogin-　 Cluster processing such as PVM, MPI-　 WWW related technique such as HTTP/HTML/CGI-　 Agent related technique such as Plangent, Aglets-　 ...

　 They can not be used to Grid computing without changing.　（ Some of them can be used. ）

19

Grid computing (2)Abilities necessary for grid computing•　 Generalization: easy to add computers•　 Security: 　 unknown users ， mobile users•　 Tolerance of fault •　 Heterogeneity: Language ， OS ， hardware, security level 　• High performance: super high performance, super high speed network•　 Scalability ： Very large scale

LAN

東京 大阪 福岡

LANLANInternetInternet

20

Grid computing (3)

History of Grid computing•　 1980’s ： Distributed computing•　 1990’s ： Gigabit network•　 I-way, 1995: Application•　 Alliance (NCSA) Virtual Machine Room•　 PACIs (NCSA/SDSC NSF National Technology Grid), 1998 〜•　 NASA Information Power Grid, 1999 〜•　 eGrid (European Grid), 2000 〜•　 ApGrid (Asia-Pacific Grid) 2000 〜•　 ...

21

Implement of grid computing

Software for grid computing•　 Toolkit: Globus, Legion, AppLes•　 Message passing: MPICH-G2•　 GridRPC: Ninf, Netsolve, Nimrod•　 Business software:United Devices, Entropia, Parabon,...

Ninf is developed in JapanNinf is developed in Japan

22

Achievement of Ninf•　 Solving 大規模非凸２次計画問題 by SCRM (kojima-Tuncel-Takeda) method:

　 The world record is achieved by using large scale cluster (128 processors)

•　 Solving Cyclic Polynomial Equation by Homotopy method: The world record is achieved by using large scale cluster (128 processors)

23

Summary (1)

Changing in parallel processing •　 From parallel computers to cluster processing ＆ grid computing•　 Combined with high performance network•　 Using popular technologies

Using internet which has huge processing ability•　 Parallel processing becomes possible for ordinary users•　 Advanced technologies are necessary.

24

Summary (2)

Cluster processing•　 wide used technologies are used for parallel processing.•　 Parallel processing becomes possible for ordinary users.

Grid computing• 　最先端の広帯域ネットワークを用いたインフラ技術•　 Combination of network technology and computing technology•　 Great progress in computer related fields.

Necessity of the comprehensive research •　 Parallel processing ， Language ， OS•　 Network•　 Application