Upload
varun-goyal
View
41
Download
0
Embed Size (px)
Citation preview
Implementa)on Step1: • Master performs handshake with all
the nearby peers (iDevices) which offer the service of MapReduce using MulAPeer ConnecAvity.
• Job is to count the occurrences of alphabet (A, E, W, Z) in a data file.
• The data file is a dicAonary of words
with more than 5 million characters.
• Avg Speedup: 2.56
Introduc)on • A single iDevice like iPhone or iPad does not
have enough processing power to process computaAonally complex funcAons on large data sets.
• This project envisions a mesh of inter-‐connected iDevices to accomplish a task like that.
• How? By working together on a single data set.
Background
Conclusions • It can be noAced that the Ame required to perform
the job is inversely proporAonal to the number of peers used.
• Using this methodology many client-‐server
systems can be transformed into clustered client systems, such as face-‐recogniAon.
Implemen'ng MapReduce framework on iOS devices using Mul'Peer Connec'vity Student: Varun Goyal Advisor: Dr. Peizhao Hu
Rochester InsAtute Of Technology, One Lomb Memorial Drive, Rochester, NY 14623 -‐ 5603
0
10
20
30
40
50
60
1 Peer 2 Peers 3 Peers
Count of A
Count of E
Count of W
Count of Z
Results
• This was recently made possible when Apple released Mul'Peer Connec'vity(MPC) as a part of iOS 7 release.
• MPC makes it possible for iDevices to discover services provided by other iDevices near by.
• Uses basic infrastructure like WiFi or bluetooth.
• MapReduce with word count example: 1. Master splits the input data into chunks and
sends it to each worker node 2. Map: counts the number of occurances for
each key 3. Shuffle: data is shuffled based on key 4. Reduce: adds up the count for each key
Step 2: • User selects the job to be performed • Master divides the data according to
number of peers. • Master sends data, job, and
command to run Map funcAon to peers in a JSON.
Step 3: • Peers perform Map funcAon on their
data set • Peers return the map funcAon’s
output to Master to be shuffled.
Step 4: • Depending the number of unique keys
and the number of peers shuffled data from step 3 is divided in different parts and sent back to the peers to run reduce funcAon on it.
Step 5: • Peers perform reduce funcAon on
their data set. • Peers return the reduced data back to
master. • Master displays the final result.
References
• MulA-‐peer framework by Apple: hgps://developer.apple.com/library/prerelease/ios/documentaAon/MulApeerConnecAvity/Reference/MulApeerConnecAvityFramework/index.html
• MulAPeer ConnecAvity, Apple Docs. hgps://developer.apple.com/library/
prerelease/ios/documentaAon/MulApeerConnecAvity/Reference/MulApeerConnecAvityFramework/index.html
• Jeffrey Dean and Sanjay Ghemawat. 2008. MapReduce: simplified data processing on large clusters. Commun. ACM 51, 1 (January 2008), 107-‐113. DOI=10.1145/1327452.1327492 hgp://doi.acm.org/10.1145/1327452.1327492
Contact Info Name: Varun Goyal Email: [email protected] Phone: +1 585 743 8102
Future Work
Technical Challenges • Cannot inject code yet hence cannot supply user defined job. • Need individual device’s permission to join the network every Ame a peer
needs to join the system which provides added security from Apple but reduces the technical enhancements and new possibiliAes.
• Face RecogniAon job was implemented using this system, however due to
shortage of Ame the actual task of performing face-‐recogniAon is performed using Kairos API.
• Once a soluAon is found to inject code into the system, which Apple can approve, the framework could be enhanced to perform user defined jobs.
• Face-‐RecogniAon algorithm could be implemented on iOS and provided as a job to this framework to determine the effecAveness of the framework and viability of the algorithm to perform face recogniAon in fixed amount of Ame.
• If Apple updates the MulAPeer ConnecAvity framework to be able to use system resources in the background (like an Ideal Mode) this framework can be used with minimum user interacAon.
0
20
40
60
80
100
120
140
160
180
200
1 Peer 2 Peers 3 Peers
Dear
Bear
Car
Tree
• Job is to count the occurrences of words (Dear, Bear, Car, Tree) in a data file.
• The data file is a dicAonary of words
with more than 83 million characters.
• Avg Speedup: 2.97