pptx present
TRANSCRIPT
![Page 1: Pptx present](https://reader035.vdocuments.mx/reader035/viewer/2022070313/554bb783b4c9053a298b4a2d/html5/thumbnails/1.jpg)
HadoopEdit by Cassell Hsu
2013.04.19
![Page 2: Pptx present](https://reader035.vdocuments.mx/reader035/viewer/2022070313/554bb783b4c9053a298b4a2d/html5/thumbnails/2.jpg)
Hadoop
Master
Slave
Slave
Slave
![Page 3: Pptx present](https://reader035.vdocuments.mx/reader035/viewer/2022070313/554bb783b4c9053a298b4a2d/html5/thumbnails/3.jpg)
Hadoop
•Master
•NameNode
•JobTracker
•SecondaryNameNode
![Page 4: Pptx present](https://reader035.vdocuments.mx/reader035/viewer/2022070313/554bb783b4c9053a298b4a2d/html5/thumbnails/4.jpg)
NameNode•Where is NameNode?
•Master
•HDFS(Hadoop Distributed File System)
•What is NameNode?
•資料之位置資訊
•資料之屬性
??
![Page 5: Pptx present](https://reader035.vdocuments.mx/reader035/viewer/2022070313/554bb783b4c9053a298b4a2d/html5/thumbnails/5.jpg)
NameNode
•位置資訊 ?
•所有資料皆存放在 – DataNode
![Page 6: Pptx present](https://reader035.vdocuments.mx/reader035/viewer/2022070313/554bb783b4c9053a298b4a2d/html5/thumbnails/6.jpg)
DataNode
•What is DataNode
•存放資料
•Where is DataNode
•HDFS
•Slaves (and Master)
![Page 7: Pptx present](https://reader035.vdocuments.mx/reader035/viewer/2022070313/554bb783b4c9053a298b4a2d/html5/thumbnails/7.jpg)
User
DataNode
NameNode
DataNode
DataNode
128Mb
B64Mb
A64Mb
Check hdfs-site.xml
B64Mb
A64Mb
![Page 8: Pptx present](https://reader035.vdocuments.mx/reader035/viewer/2022070313/554bb783b4c9053a298b4a2d/html5/thumbnails/8.jpg)
Hadoop
DataNodeNameNode
MasterDataNode
DataNode
DataNode
Slaves
![Page 9: Pptx present](https://reader035.vdocuments.mx/reader035/viewer/2022070313/554bb783b4c9053a298b4a2d/html5/thumbnails/9.jpg)
Hadoop
•Master
•NameNode
•JobTracker
•SecondaryNameNode
![Page 10: Pptx present](https://reader035.vdocuments.mx/reader035/viewer/2022070313/554bb783b4c9053a298b4a2d/html5/thumbnails/10.jpg)
JobTracker
•What is JobTracker?
•排程工作
•Where is JobTracker?
•Master
誰來工作?
![Page 11: Pptx present](https://reader035.vdocuments.mx/reader035/viewer/2022070313/554bb783b4c9053a298b4a2d/html5/thumbnails/11.jpg)
JobTracker & TaskTrackerJobTracker TaskTracker
Where Master Slaves
What 排程工作 執行工作
![Page 12: Pptx present](https://reader035.vdocuments.mx/reader035/viewer/2022070313/554bb783b4c9053a298b4a2d/html5/thumbnails/12.jpg)
Hadoop
DataNodeNameNode
MasterDataNode
Slaves
JobTracker
TaskTracker
![Page 13: Pptx present](https://reader035.vdocuments.mx/reader035/viewer/2022070313/554bb783b4c9053a298b4a2d/html5/thumbnails/13.jpg)
Hadoop
•Master
•NameNode
•JobTracker
•SecondaryNameNode
![Page 14: Pptx present](https://reader035.vdocuments.mx/reader035/viewer/2022070313/554bb783b4c9053a298b4a2d/html5/thumbnails/14.jpg)
SecondaryNameNode
•What is SecondaryNameNode?
•NameNode 發生錯誤時補救
•Where is SecondaryNameNode?
•Master
![Page 15: Pptx present](https://reader035.vdocuments.mx/reader035/viewer/2022070313/554bb783b4c9053a298b4a2d/html5/thumbnails/15.jpg)
Hadoop
DataNodeNameNode
MasterDataNode
Slaves
JobTracker
TaskTracker
SecondaryNode
![Page 16: Pptx present](https://reader035.vdocuments.mx/reader035/viewer/2022070313/554bb783b4c9053a298b4a2d/html5/thumbnails/16.jpg)
MapReduce
User Master
Slave2
Slave1
A
A1
A2Task
NameNode
Task
Result1
Result2
Reduce
Final Result
HDFS
![Page 17: Pptx present](https://reader035.vdocuments.mx/reader035/viewer/2022070313/554bb783b4c9053a298b4a2d/html5/thumbnails/17.jpg)
MapReduce
•檔案切割
•Hadoop 上區塊切割
•程式指定