Download - Installing HortonWork's Hadoop on AWS EC2
Installing Hortonwork’s Hadoop on AWS
Rohit GhatolDirector of Engineering @ Synerzip
http://www.linkedin.com/in/rohitghatol @rohitghatol http://rohitghatol.com
BY
Software Stack
EC2
Hadoop
Hadoop
Hadoop
Hadoop
Hadoop
Hadoop
Controls &
Monitors
Hadoop Services• HDFS• YARN• Hbase• Hive• Pig• Sqoop• Oozie• Zookeeper• Nagios• Ganglia
Ambari
Apache Ambari
Step 1 – Create Base AMI
Community Red hat AMI Base AMI
6
SSHDownload PEM Key
Step 2 – Password Less SSH AccessLaptop accessing EC2 Instance EC2 Instance accessing other
EC2 Instances
AccessUsingSSH
Upload PEM Key
EC2(Ambari Server)
EC2
EC2
EC2
EC2
AMI
EC2
EC2
EC2
EC2
AMI
EC2
EC2
EC2
Step 3 – Launch 6 Instances using Base Image
Launch
6
Note the Private DNS names of all these machines• ip-10-23-12-33• ip-10-23-11-23• ip-10-23-32-54• ip-10-23-44-14• ip-10-23-65-73• ip-10-23-37-47
Install Ambari Server
EC2
EC2EC2
Step 4 – Install Ambari Server on EC2
EC2
Supply the earlier recorded private DNS to Ambari• ip-10-23-12-33• ip-10-23-11-23• ip-10-23-32-54• ip-10-23-44-14• ip-10-23-65-73• ip-10-23-37-47
Supply list of private DNS
EC2 EC2 EC2
EC2 EC2 EC2
Ambari Server Installs Ambari Agent
EC2
EC2
EC2
EC2
EC2
EC2
EC2
Installsservices
Hadoop Services• HDFS• YARN• Hbase• Hive• Pig• Sqoop• Oozie• Zookeeper• Nagios• Ganglia
Step 5 – Install X service on Y machine
EC2 withAmbari
Step 6 – Launch Ambari Web Consolehttp://<<ambari-server>>:8080