1. NIPS More Effective Distributed ML via a Stale Synchronous Parallel Parameter Server Qirong Ho et. al. CMU & Intel Labs Preferred Infrastructure

2. 5 l l l l SSPMIX l PetuumNIPSOSS 2

3. http://www.cs.cmu.edu/~qho/ssp_nips2013.pdf 3

4. l l l l SVM l l 4

5. l l l l l MPIMapReduce 5

6. 2 l 1. l l l l l 2. l l l HWVM l l l l l l l 1 6

7. l l l l l l l l Hadoop (Mahout) l Nave-parallel SGD l Spark (MLbase) l l Single-machine asynchronous Partitioned SGD l GraphLab, Pregel l l l l l l 7

8. (1) BSP: Bulk Synchronous Parallel l l l l l l 8

9. (2) Asynchronous l l l l l l 9

10. SSP: Stale Synchronous Parallel l l s l l s l l l 10

11. Stale Synchronous Parallel s=0BSPs=Asynchronous l s l s l s 11

12. SSP l l SSP l (2s-1)x l 12

13. SSPTableParameter Server l Parameter ServerSSPTable l l SSPTable 13

14. (1): s 14

15. (2): BSPAsynchronous l sBSPAsynchronous 15

16. (3): l l l l SSPBSP BSP SSP s 16

17. Petuum: OSS l l l http://petuum.org/ https://github.com/sailinglab/petuum Ubuntu 12.0412.10 17

18. l "More Effective Distributed ML via a Stale Synchronous Parallel Parameter Server", Qirong Ho, James Cipar, Henggang Cui, Jin Kyu Kim, Seunghak Lee, Phillip. B. Gibbons, Garth A. Gibson, Greg R. Ganger, Eric P Xing. Neural . Information Processing Systems, 2013 (NIPS 2013) l "Structure-Aware Dynamic Scheduler for Parallel Machine Learning", Seunghak Lee, Jin Kyu Kim, Qirong Ho, Garth A. Gibson, Eric P Xing. arXiv:1312.5766 . (2013) l "Petuum: A Framework for Iterative-Convergent Distributed ML", Wei Dai, Jinliang Wei, Xun Zheng, Jin Kyu Kim, Seunghak Lee, Junming Yin, Qirong Ho, Eric P Xing. arXiv:1312.7651 (2013) . l "Consistency Models for Distributed ML with Theoretical Guarantees", Jinliang Wei, Wei Dai, Abhimanu Kumar, Xun Zheng, Qirong Ho, Eric P Xing. arXiv: . 1312.7869 (2013) 18