О.В.Сухорослов "Распределенное программирование. Разбор ДЗ №2"

  • View
    1.789

  • Download
    3

Embed Size (px)

DESCRIPTION

О.В.Сухорослов "Распределенное программирование. Разбор ДЗ №2", 27.04.2012, место показа: МФТИ, Школа анализа данных (ШАД)

Text of О.В.Сухорослов "Распределенное программирование....

  • 1. 10 .. oleg.sukhoroslov@gmail.com 26.04.2012.. 10 () 26.04.2012 1 / 64

2. .. 10 () 26.04.2012 2 / 64 3. , message passing, RPC, 4.. 10 () 26.04.2012 3 / 64 4. Distributed Computing A eld of computer science that studies distributed systems Use of distributed systems to solve computational problems.. 10 () 26.04.2012 4 / 64 5. Distributed SystemA collection of autonomous entities (processes, computers, nodes) Each node has its own local memory Nodes communicate with each other through a computer network.. 10 () 26.04.2012 5 / 64 6. Distributed SystemA collection of autonomous entities (processes, computers, nodes) Each node has its own local memory Nodes communicate with each other through a computer networkAll nodes have a common goal Solving a large computational problem.. 10 () 26.04.2012 5 / 64 7. Distributed SystemA collection of autonomous entities (processes, computers, nodes) Each node has its own local memory Nodes communicate with each other through a computer networkAll nodes have a common goal Solving a large computational problemEach node has its own goals/needs Coordination of the use of shared resources Communication services.. 10 () 26.04.2012 5 / 64 8. Distributed SystemA collection of autonomous entities (processes, computers, nodes) Each node has its own local memory Nodes communicate with each other through a computer networkAll nodes have a common goal Solving a large computational problemEach node has its own goals/needs Coordination of the use of shared resources Communication servicesA distributed system is one in which the failure of a computer youdidnt even know existed can render your own computer unusable.(Leslie Lamport).. 10 () 26.04.2012 5 / 64 9. Distributed Systems.. 10 () 26.04.2012 6 / 64 10. Related FieldsConcurrency Simultaneous execution of several computations interacting with each other.. 10 ()26.04.2012 7 / 64 11. Related FieldsConcurrency Simultaneous execution of several computations interacting with each otherParallel Computing Dividing large problems into smaller ones which can be solved concurrently (in parallel).. 10 () 26.04.2012 7 / 64 12. Features of Distributed SystemsLoosely coupled Each computer is autonomous and has only a limited, incomplete view of the system The system doesnt have global state or clock The structure of the system (network topology, network latency, number of nodes) may be not known in advance.. 10 ()26.04.2012 8 / 64 13. Features of Distributed SystemsLoosely coupled Each computer is autonomous and has only a limited, incomplete view of the system The system doesnt have global state or clock The structure of the system (network topology, network latency, number of nodes) may be not known in advanceHeterogeneous The system may consist of dierent kinds of computers and network links.. 10 ()26.04.2012 8 / 64 14. Features of Distributed SystemsLoosely coupled Each computer is autonomous and has only a limited, incomplete view of the system The system doesnt have global state or clock The structure of the system (network topology, network latency, number of nodes) may be not known in advanceHeterogeneous The system may consist of dierent kinds of computers and network linksDynamic The system may change during the execution.. 10 ()26.04.2012 8 / 64 15. Features of Distributed SystemsLoosely coupled Each computer is autonomous and has only a limited, incomplete view of the system The system doesnt have global state or clock The structure of the system (network topology, network latency, number of nodes) may be not known in advanceHeterogeneous The system may consist of dierent kinds of computers and network linksDynamic The system may change during the executionUnreliable Nodes and communication links may fail (partial failure).. 10 () 26.04.2012 8 / 64 16. Features of Distributed SystemsLoosely coupled Each computer is autonomous and has only a limited, incomplete view of the system The system doesnt have global state or clock The structure of the system (network topology, network latency, number of nodes) may be not known in advanceHeterogeneous The system may consist of dierent kinds of computers and network linksDynamic The system may change during the executionUnreliable Nodes and communication links may fail (partial failure)Slow communication High latency, limited bandwidth.. 10 () 26.04.2012 8 / 64 17. Why Distributed Systems? Performance Solve big problems Store large data Serve huge number of clients.. 10 () 26.04.2012 9 / 64 18. Why Distributed Systems? Performance Solve big problems Store large data Serve huge number of clients Communication and coordination Users and resources are geographically distributed.. 10 () 26.04.2012 9 / 64 19. Why Distributed Systems? Performance Solve big problems Store large data Serve huge number of clients Communication and coordination Users and resources are geographically distributed Availability Avoid single point of failure.. 10 () 26.04.2012 9 / 64 20. Why Distributed Systems? Performance Solve big problems Store large data Serve huge number of clients Communication and coordination Users and resources are geographically distributed Availability Avoid single point of failure Scalability Easier to expand and manage.. 10 () 26.04.2012 9 / 64 21. Network of workstations Desktop grids (SETI@Home) Service grids (European Grid Infrastructure) (Lustre, GFS, HDFS) (Oracle, Vertica, VoltDB) NoSQL (Cassandra, HBase, MongoDB) Peer-to-peer (BitTorrent) DNS, World Wide Web Messaging Enterprise Application Integration ..... 10 () 26.04.2012 10 / 64 22. (message passing) (RPC) Streams/pipes Publish/subscribe () .. 10 () 26.04.2012 11 / 64 23. - , , , ... 1 2 ( ) , ( ) () 2 (), (callback), MPI, Erlang (actors), Web Services, XMPP .. 10 () 26.04.2012 12 / 64 24. (shared nothing) locking , "" race condition .. 10 () 26.04.2012 13 / 64 25. (sync/async, unicast/multicast).. 10 () 26.04.2012 14 / 64 26. (TCP, UDP) IP- IP- (TCP) (TCP)0.0.0.0 - listening socket255.255.255.255 broadcasting socket .. 10 () 26.04.2012 15 / 64 27. TCP / - (, 80) ..... 10 () 26.04.2012 16 / 64 28. .. 10 () 26.04.2012 17 / 64 29. 1 ServerSocket serverSocket = new ServerSocket (4444); 2 Socket clientSocket = null ; 3 4 boolean listening = true ; 5 while ( listening ) { 6 clientSocket = serverSocket . accept (); 7 8 PrintWriter out = new PrintWriter ( 9 clientSocket . getOutputStream () , true );10 BufferedReader in = new BufferedReader (11 new InputStreamReader ( clientSocket . getInputStream ()));12 String request , response ;1314 Protocol protocol = new Protocol ();15 response = protocol . processRequest ( null );16 out . println ( response );1718 while (( request = in . readLine ()) != null ) {19 response = protocol . processRequest ( request );20 out . println ( request );21 if ( protocol . isTerminated ()) break ;22 }2324 out . close (); in . close (); clientSocket . close ();25 }2627 serverSocket . close (); .. 10 () 26.04.2012 18 / 64 30. 1 public class Protocol { 2 3 private boolean isTerminated = false ; 4 5 public String processRequest ( String request ) { 6 String response = null ; 7 8if ( request == null ) { 9 response = " Connection accepted " ;10} else if ( request . equals ( " Hello Server ! " )) {11 response = " Hello Client ! " ;12} else if ( request . equals ( " TIME " )) {13 response = String . valueOf ( System . currentTimeMillis ());14} else if ( request . equals ( " Bye Server ! " )) {15 response = " Bye Client ! " ;16 isTerminated = true ;17} else {18 response = " What ? " ;19}2021return response ;22 }2324 public boolean isTerminated () {25 return isTerminated ;26 }27 } .. 10 ()26.04.2012 19 / 64 31. 1 String host = args [0]; 2 Socket socket = new Socket ( host , 4444); 3 PrintWriter out = new PrintWriter ( socket . getOutputStream () , true ); 4 BufferedReader in = new BufferedReader ( 5 new Inp utStreamReader ( socket . getInputStream ())); 6 7 BufferedReader stdIn = new BufferedReader ( 8 new Inp utStreamReader ( System . in )); 9 String fromServer , fromUser ;1011 while (( fromServer = in . readLine ()) != null ) {12 System . out . println ( " Server : " + fromServer );13 if ( fromServer . equals ( " Bye Client ! " ))14break ;15 fromUser = stdIn . readLine ();16 if ( fromUser != null ) {17out . println ( fromUser );18 }19 }2021 out . close (); in . close (); stdIn . close (); socket . close (); .. 10 () 26.04.2012 20 / 64 32. 1 Server : Connection accepted2 Hello Server !3 Server : Hello Client !4 TIME5 Server : 12065455366566 MONEY7 Server : What ?8 Bye Server !9 Server : Bye Client !.. 10 () 26.04.2012 21 / 64 33. ?.. 10 () 26.04.2012 22 / 64 34. 1 while ( listening ) {2 Socket clientSocket = serverSocket . accept ();3 new Mul tiServerThread ( clientSocket ). start ();4 } ?.. 10 () 26.04.2012 23 / 64 35. ? IP- IP- ? DNS (naming service) (directory).. 10 () 26.04.2012 24 / 64 36. () IP- 224.0.0.0 - 239.255.255.255 , , UDP.. 10 () 26.04.2012 25 / 64 37. 1 DatagramSocket socket = new DatagramSocket (4444); 2 InetAddress group = InetAddress . getByName ( "230.0.0.1" ); 3 4 String request = " TIME " ; 5 int requestSize = request . getBytes (). length ; 6 DatagramPacket packet = new DatagramPacket ( 7 new byte [ requestSize ] , requestSize , group, 4445); 8 packet . setData ( request . getBytes ()); 9 socket . send ( packet );1011 byte [] buf = new byte [256];12 packet = new DatagramPacket ( buf , buf . length );13 socket . receive ( packet );1415 String response = new String (16 packet . getData () , 0 , packet . getLength ());17 System . out . println ( response );1819 socket . close (); .. 10 () 26.04.2012 26