HadoopEdit by Cassell Hsu
2013.04.19
Hadoop
Master
Slave
Slave
Slave
Hadoop
• Master
• NameNode
• JobTracker
• SecondaryNameNode
NameNode• Where is NameNode?
• Master
• HDFS(Hadoop Distributed File System)
• What is NameNode?
• 資料之位置資訊
• 資料之屬性
??
NameNode
• 位置資訊 ?
• 所有資料皆存放在 – DataNode
DataNode• What is DataNode
• 存放資料• Where is DataNode
• HDFS
• Slaves (and Master)
UserUser
DataNodeDataNode
NameNodeNameNode
DataNodeDataNode
DataNodeDataNode
128Mb B
64Mb
A64Mb
Check hdfs-site.xml
B64Mb
A64Mb
Hadoop
DataNodeDataNodeNameNodeNameNode
MasterDataNodeDataNode
DataNodeDataNode
DataNodeDataNode
Slaves
Hadoop
• Master
• NameNode
• JobTracker
• SecondaryNameNode
JobTracker
• What is JobTracker?
• 排程工作• Where is JobTracker?
• Master
誰來工作?
JobTracker & TaskTrackerJobTracker TaskTracker
Where Master Slaves
What 排程工作 執行工作
Hadoop
DataNodeDataNodeNameNodeNameNode
MasterDataNodeDataNode
Slaves
JobTrackerJobTracker
TaskTrackerTaskTracker
Hadoop
• Master
• NameNode
• JobTracker
• SecondaryNameNode
SecondaryNameNode
• What is SecondaryNameNode?
• NameNode發生錯誤時補救
• Where is SecondaryNameNode?
• Master
Hadoop
DataNodeDataNodeNameNodeNameNode
MasterDataNodeDataNode
Slaves
JobTrackerJobTracker
TaskTrackerTaskTracker
SecondaryNodeSecondaryNode
MapReduce
User Master
Slave2
Slave1
A
A1
A2Task
NameNode
Task
Result1
Result2
Reduce
Final ResultHDFS
MapReduce
• 案切割檔
• Hadoop上區塊切割
• 程式指定