Введение в параллельные вычисления. Основы программирования на языке СИ с использованием интерфейса МРI

  • View
    228

  • Download
    8

Embed Size (px)

Transcript

  • . ..

    .. , .. , .. , ..

    .

    MPI

    2009

    681.3.06

    .., .., .., -

    .. .

    -

    MPI. .: , 2009. 123 .

    -

    .

    . -

    -

    MPI. - -

    Linux. - .

    ,

    , -

    .

    : ... . . ..-.. . . .

    .

    ,

    ISBN 978-5-91450-031-0

  • 3

    1. ................................................................................................ 4 1.1. .................................................................................... 4 1.2. ....... 7 1.3. ............................................................................................... 9 1.4. - ........................................... 11 1.5. (SMP NUMA) ............................................................................... 13 1.6. (MPP)........................ 16 1.7. ............................................................... 19 2. ............................................................. 22 2.1. ............... 22 2.2. 24 2.3. ......................................... 25 2.4. ....... 31 2.5. ....................................... 32 3. ............................................... 42 3.1. GNU/Linux....................................... 43 3.2. .............................................. 50 4. MPI.............................................. 61 4.1. MPI ............................................................... 63 4.2. / ......................... 68 4.3. / ....................... 75 4.4. / ...... 85 4.5. ...................................................... 90 4.6. ................................................................... 92 4.7. ........................................................... 99 4.8. .......................... 104 4.9. ......................................................................... 106 ................................................................................... 123

    4

    1.

    1.1. -

    () .

    ,

    -

    .

    , -

    ,

    , .. , . -

    , , -

    , , -

    .

    , (. supercomputer) (George Michael) (Sidney Fernbach), -

    . . . (. Lawrence Livermore National Laboratory, LLNL) .

    -

    (. The University of California, UC) - - (. Los Alamos National Laboratory, LANL) - - -

    , .. ,

    .

    ,

  • 5

    ,

    , ( ).

    , IBM Blue Gene/L 2004-2008 .

    -

    (Seymour Cray), Cray-1, Cray-2 . -

    , ,

    ,

    .

    ,

    ,

    , (Steve Chen), - -

    Cray X-MP, - .

    Cray Inc., - .

    -

    , -

    . -

    , IBM Hewlett-Packard.

    - ,

    , -

    -. , 1989 -

    VAX (Gordon Bell) - ,

    .

    - ,

    6

    , .. -

    , .

    -

    , - , -

    .

    .

    -

    . , -

    ( , ..), (, - ..), . - ,

    ; - .

    -

    . , -

    , -

    ( ) . -

    , . -

    ABC, 1942 (. Iowa State University of Science and Technology, ISU) 30 , 2008 IBM Roadrunner - 1 - (1015) .

    , 65 - 30 . ,

  • 7

    .

    , -

    . ,

    -

    .

    .

    . ,

    , -

    , .

    -

    :

    ; , -

    ;

    .

    1.2.

    - .

    , , , .

    -

    . , - , . -

    60- .

    8

    .

    :

    SISD (. Single Instruction Single Data) . -

    .

    .

    SIMD (Single Instruction Multiple Data) . - ,

    , ( ) . - -

    , - .

    MISD (Multiple Instruction Single Date) . .

    -

    , . - -

    - ,

    -.

    MIMD (Multiple Instruction Multiple Date) - .

    -

    .

    ,

    - . ,

    -

    .

    , . ,

  • 9

    . , , -

    .

    ,

    -

    . -

    . , -

    , -

    , .

    1.3.

    .

    , .

    -

    () .

    1976 - (. University of Illinois at Urbana-Champaign, UIUC) ILLIAC IV, 64 ( 256) - (), , . -

    .

    -

    .

    -

    -

    , MPP- (. Massive Parallel

    10

    Processing, MPP). , , MPP- , ( ).

    , -

    . , ICL DAP (. Distributed Array Processor), 8192 , , -

    ( , ). -

    -

    .

    . -

    . ,

    2 16 , - () . -

    (. Symmetric Multiprocessing, SMP).

    -

    -

    -. , -

    SMP - - .

    -

    ,

    .

    -

    ,

    -

    ,

    (. Message Passing). -

  • 11

    -

    MPP-. ( ) .

    SMP MPP NUMA- (. Non-Uniform Memory Access), , -.

    . NUMA- Cray T3D, 1993 , ,

    .

    -

    :

    - ; SMP- NUMA-; MPP-; .

    1.4. - - Cray-1

    1976 . , . ,

    :

    ; ,

    -

    .

    - , , 128 256 . ,

    .

    , -

    . -

    12

    , -

    . ,

    -

    . - -

    ,

    . , - ,

    .

    ,

    .

    , - ( ) (SMP), , -

    , NUMA- MPP-. -

    Cray J90 (1994 ), Cray T90 (1995 ), NEC SX-4 (1995 ), Cray SV1 (1998 ), NEC SX-5 (1999 ).

    ,

    . , 90- ,

    ,

    .

    - . -

    2002 . NEC Earth Simulator (ES) 5120 - , 5 , MPP- ASCI White (2000 ), 8192 . - -

    . NEC ES - 2004 IBM Blue Gene/L MPP-.

  • 13

    1.5. (SMP NUMA)

    , - . SMP-

    ,

    . -

    , -

    .

    -

    .

    ,

    . -

    32 .

    SMP , -, - ,

    ,

    .

    . 1. SMP-

    CPU

    CPU

    CPU

    14

    -

    , -

    . - , .

    , , . ,

    ,

    .

    ,

    (-), , -

    .

    -

    - -

    1999 Pentium III. - :

    L1 ( ) 32 , 9976 /;

    L2 ( ) 256 , 4446 /.

    -

    255 /. , - -

    40 . , 10 / ( CPU) Intel Core i7 2009 .

    ,

    .

    ,

    . , -

    -,

    . , - - ,

  • 15

    . , -

    , -

    - ,

    , - .

    ccNUMA (. Cache coherent Non-Uniform Memory Access). , -

    .

    ,

    .

    . 2. NUMA- -

    , , ,

    . SMP- :

    - ; ; .

    CPU

    CPU

    CPU

    CPU

    16

    ,

    .

    ccNUMA .

    , , 256 200 . . -

    -

    2 128 -

    . -

    -

    AMD Opteron Intel Itanium 2, , HP Integrity Superdome (2002 ).

    2008 ccNUMA - Nehalem, Core 2, - Intel Core i7.

    -

    , ,

    . , -

    .

    1.6. (MPP) , -

    ,

    .

    - , -

    .

    -

    , ,

    , /, ..

  • 17

    .

    , , -

    , .

    . 3. MPP-

    . -

    .

    - ,

    -.

    , . -

    MPP- -,

    .

    MPP- . -

    .

    , , . MPP-

    CPU

    CPU

    CPU

    18

    . -

    .

    , N , - N 1 , -. MPP- - . Intel Paragon (1992 ) .

    . Cray T3D (1993 ) Cray T3E (1995 ) . , -

    .

    nCUBE - n- .

    -

    . ,

    , , - . -

    , - -

    ,

    . -

    ,

    IBM SP2 (1994 ). -

    , .

    ,

    - .

    , SMP-, . MPP- :

    - ; ; .

  • 19

    -

    , .

    , -

    . . ,

    ,

    .

    , SMP-.

    1.7.

    , MPP. - MPP- ,

    : .

    , ,

    , MPI, -

    , -

    .

    ,

    .

    , -

    - , -

    -.

    -

    , -

    . -

    1998 - (. Pennsylvania State University) COCOA (. The Cost Effective

    20

    Computing Array), 25 - 100 . . , -

    48- Cray T3D . . .

    , -