cloudera amazon-ec2

Download Cloudera amazon-ec2

Post on 25-Jun-2015

4.127 views

Category:

Documents

3 download

Embed Size (px)

TRANSCRIPT

  • 1. Building a Hadoop Cluster onAmazon EC2 using ClouderaApril 2013http://randyzwitch.com/big-data-hadoop-amazon-ec2-cloudera-part-2

2. Create Name Node: m1.largeUbuntu Server 12.04.1 LTS 64-bithttp://randyzwitch.com/big-data-hadoop-amazon-ec2-cloudera-part-2 3. Create Name Node: m1.largeCan use defaults for most Wizard screens except Firewallhttp://randyzwitch.com/big-data-hadoop-amazon-ec2-cloudera-part-2 4. Launch Instance, Connect Via SSHhttp://randyzwitch.com/big-data-hadoop-amazon-ec2-cloudera-part-2 5. Download & Run Cloudera ManagerDepending on settings, might need to run as sudoMight take 5 or sominutes to go throughlicensing andinstallation menushttp://randyzwitch.com/big-data-hadoop-amazon-ec2-cloudera-part-2 6. Open browser And Go To EC2 Instance PublicDNS At Port 7180Ex: http://ec2-50-17-162-58.compute-1.amazonaws.com:7180On this first login, you set your username and password credentialshttp://randyzwitch.com/big-data-hadoop-amazon-ec2-cloudera-part-2 7. Use Defaults,18 for instanceshttp://randyzwitch.com/big-data-hadoop-amazon-ec2-cloudera-part-2 8. Type In AWS Access Key ID & Secret Access KeyCredentials can be found under Security Credentials in EC2 dashboardhttp://randyzwitch.com/big-data-hadoop-amazon-ec2-cloudera-part-2 9. Review Settings Then Install!Provisioning Instances will takea few minuteshttp://randyzwitch.com/big-data-hadoop-amazon-ec2-cloudera-part-2 10. If Any Installations Fail, Retry UntilSuccesshttp://randyzwitch.com/big-data-hadoop-amazon-ec2-cloudera-part-2 11. Make Sure Consistency Check Passeshttp://randyzwitch.com/big-data-hadoop-amazon-ec2-cloudera-part-2 12. Cluster Services Will Start, Then Success!http://randyzwitch.com/big-data-hadoop-amazon-ec2-cloudera-part-2 13. Finding Hue Public DNShttp://randyzwitch.com/big-data-hadoop-amazon-ec2-cloudera-part-2Hue (Hadoop User Experience) is the more user-friendly wayto interact with Hadoop 14. Finding Hue Public DNShttp://randyzwitch.com/big-data-hadoop-amazon-ec2-cloudera-part-2Clicking on the Hue Web UI button doesnt work, because it references theInternal Address for Amazon EC2Clicking this link button wont work!Need to find the Public DNS for this Internal Address in Amazon Dashboard 15. Finding Hue Public DNShttp://randyzwitch.com/big-data-hadoop-amazon-ec2-cloudera-part-2Type in Internal Address in Search Box to find the Instance having HueThis is the public DNSAddress to access Hue 16. Finding Hue Public DNShttp://randyzwitch.com/big-data-hadoop-amazon-ec2-cloudera-part-2Hue is accessed via Port 8888Ex: http://ec2-54-224-118-78.compute-1.amazonaws.com:8888Pick your username/password carefully, this is the superuser 17. If You See Hue, Youre Ready For Analysis!http://randyzwitch.com/big-data-hadoop-amazon-ec2-cloudera-part-2This is the Hive editor, which allows for SQL-Like Syntax tocreate MapReduce jobs 18. Referencehttp://blog.cloudera.com/blog/2013/03/how-to-create-a-cdh-cluster-on-amazon-ec2-via-cloudera-manager/