apache ambari - what's new in 1.6.1
DESCRIPTION
Learn about the new features of Apache Ambari 1.6.1. http://ambari.apache.org/TRANSCRIPT
![Page 1: Apache Ambari - What's New in 1.6.1](https://reader033.vdocuments.mx/reader033/viewer/2022061103/540d76b58d7f72747e8b49ce/html5/thumbnails/1.jpg)
© Hortonworks Inc. 2014
Apache Ambari1.6.1
July 2014
![Page 2: Apache Ambari - What's New in 1.6.1](https://reader033.vdocuments.mx/reader033/viewer/2022061103/540d76b58d7f72747e8b49ce/html5/thumbnails/2.jpg)
© Hortonworks Inc. 2014
What’s New in 1.6.1
• New Host Checks• External Database Connection Testing• Stack Repository Mgmt Enhancements• Performance Improvements
• For a complete list of changeshttps://issues.apache.org/jira/browse/AMBARI/fixforversion/12326545
Page 2
![Page 3: Apache Ambari - What's New in 1.6.1](https://reader033.vdocuments.mx/reader033/viewer/2022061103/540d76b58d7f72747e8b49ce/html5/thumbnails/3.jpg)
© Hortonworks Inc. 2014
New Host ChecksCustom JDKHostname resolutionReverse hostname
Page 3
![Page 4: Apache Ambari - What's New in 1.6.1](https://reader033.vdocuments.mx/reader033/viewer/2022061103/540d76b58d7f72747e8b49ce/html5/thumbnails/4.jpg)
© Hortonworks Inc. 2014
Background: Current JDK Options
• Ambari Server setup provides three JDK options
• Use options [1] or [2] for Ambari to automatically download + install the JDK on all hosts–Requires Internet access
• Use option [3] to specify an already installed JDK–This is a must when performing a no-Internet / local install–Ambari expects this JDK to already be installed on ALL hosts
Page 4
![Page 5: Apache Ambari - What's New in 1.6.1](https://reader033.vdocuments.mx/reader033/viewer/2022061103/540d76b58d7f72747e8b49ce/html5/thumbnails/5.jpg)
© Hortonworks Inc. 2014
Custom JDK Host Check
• Problem: when selecting option [3], if a host did not already have the JDK installed, service and component starts would fail late in the Cluster Install Wizard process– Difficult to troubleshoot
• Solution: New “Custom JDK” Host Check performed during Host Registration– Confirms the Custom JDK is installed on each host
Page 5
![Page 6: Apache Ambari - What's New in 1.6.1](https://reader033.vdocuments.mx/reader033/viewer/2022061103/540d76b58d7f72747e8b49ce/html5/thumbnails/6.jpg)
© Hortonworks Inc. 2014
Hostname Resolution Check
• Checks that each host can resolve hostname for other hosts in the cluster
Page 6
![Page 7: Apache Ambari - What's New in 1.6.1](https://reader033.vdocuments.mx/reader033/viewer/2022061103/540d76b58d7f72747e8b49ce/html5/thumbnails/7.jpg)
© Hortonworks Inc. 2014
Reverse Hostname Check
• On each host, check hostname matches reverse lookup
Page 7
![Page 8: Apache Ambari - What's New in 1.6.1](https://reader033.vdocuments.mx/reader033/viewer/2022061103/540d76b58d7f72747e8b49ce/html5/thumbnails/8.jpg)
© Hortonworks Inc. 2014
External Database Connections
• Simplified JDBC Driver Setup• Ability to “test” Database Connections
Page 8
![Page 9: Apache Ambari - What's New in 1.6.1](https://reader033.vdocuments.mx/reader033/viewer/2022061103/540d76b58d7f72747e8b49ce/html5/thumbnails/9.jpg)
© Hortonworks Inc. 2014
Background: External Database Drivers
Page 9
• Ambari supports using an external database for Hive Metastore and Oozie
• Ambari 1.6.0 (or earlier) experience required user to put the JDBC Driver ON the PLANNED host for Hive Metastore or Oozie Server PRIOR to starting cluster install
• Potential chances for error– Wrong host– JDBC Driver in wrong location– Miss this step completely
![Page 10: Apache Ambari - What's New in 1.6.1](https://reader033.vdocuments.mx/reader033/viewer/2022061103/540d76b58d7f72747e8b49ce/html5/thumbnails/10.jpg)
© Hortonworks Inc. 2014
Simplified JDBC Driver Setup
• User provides driver during ambari-server setup…ambari-server setup --jdbc-db=oracle --jdbc-driver=/path/to/odjcb6.jar
• During Cluster Install Wizard…–Ambari checks that user provided the driver during setup–Ambari copies the driver to the correct destination host
Page 10
AMBARISERVER
HiveMetastore
OozieJDBC Driver JAR
![Page 11: Apache Ambari - What's New in 1.6.1](https://reader033.vdocuments.mx/reader033/viewer/2022061103/540d76b58d7f72747e8b49ce/html5/thumbnails/11.jpg)
© Hortonworks Inc. 2014
JDBC Driver Setup Reminder
Page 11
![Page 12: Apache Ambari - What's New in 1.6.1](https://reader033.vdocuments.mx/reader033/viewer/2022061103/540d76b58d7f72747e8b49ce/html5/thumbnails/12.jpg)
© Hortonworks Inc. 2014
Test Database Connection
• Get JDBC driver from Ambari Server• Distribute to target machine (Oozie or Hive Metastore)• Test the database connection (show success/fail)• Popup warning if user proceeds without testing
Page 12
1
2
3
![Page 13: Apache Ambari - What's New in 1.6.1](https://reader033.vdocuments.mx/reader033/viewer/2022061103/540d76b58d7f72747e8b49ce/html5/thumbnails/13.jpg)
© Hortonworks Inc. 2014
Stack Repository MgmtHDP and HDP-UTILS managed per StackTemplate-driven .repo filesManage Base URLs from Ambari Web
Page 13
![Page 14: Apache Ambari - What's New in 1.6.1](https://reader033.vdocuments.mx/reader033/viewer/2022061103/540d76b58d7f72747e8b49ce/html5/thumbnails/14.jpg)
© Hortonworks Inc. 2014
Background: Repositories
Page 14
AMBARI Repo HDP-UTILS Repo
HDP Repo
AMBARI HDP STACK
Ambari Server
and Agent
Packages
Hadoop Stack
Packages
Stack Utils
LZO, ExtJS,
Snappy,
Nagios, Ganglia
![Page 15: Apache Ambari - What's New in 1.6.1](https://reader033.vdocuments.mx/reader033/viewer/2022061103/540d76b58d7f72747e8b49ce/html5/thumbnails/15.jpg)
© Hortonworks Inc. 2014
Background: .repo Files
Page 15
AMBARI Repo HDP-UTILS Repo
HDP Repo
AMBARI HDP STACK
ambari.repo
HDP.repo
BUT
HDP-UTILS is
Stack specific
![Page 16: Apache Ambari - What's New in 1.6.1](https://reader033.vdocuments.mx/reader033/viewer/2022061103/540d76b58d7f72747e8b49ce/html5/thumbnails/16.jpg)
© Hortonworks Inc. 2014
ambari.repo Changes
Page 16
The ambari.repo file no longer includes HDP-UTILS, only AMBARI
[root@c6402 ~]# more /etc/yum.repos.d/ambari.repo
[AMBARI.1.6.1-1.x]name=Ambari 1.xbaseurl=http://public-repo-1.hortonworks.com/ambari/centos6/1.x/updates/1.6.1/gpgcheck=1gpgkey=http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkinsenabled=1priority=1
1
![Page 17: Apache Ambari - What's New in 1.6.1](https://reader033.vdocuments.mx/reader033/viewer/2022061103/540d76b58d7f72747e8b49ce/html5/thumbnails/17.jpg)
© Hortonworks Inc. 2014
HDP + HDP-UTILS Stack Repositories
Page 17
Prompted to provide HDP and HDP-UTILS Stack repository Base URLs2
![Page 18: Apache Ambari - What's New in 1.6.1](https://reader033.vdocuments.mx/reader033/viewer/2022061103/540d76b58d7f72747e8b49ce/html5/thumbnails/18.jpg)
© Hortonworks Inc. 2014
HDP.repo and HDP-UTILS.repo
Page 18
Ambari Server Ambari Agent
4) Server sends HDP.repo & HDP-UTILS.repo
2) Reads Stack Base URL values
6) Agent uses HDP.repo HDP-UTILS.repo when installing packages
1) Agent reports
3) Creates HDP.repo & HDP-UTILS.repo **
5) Agent puts HDP.repo & HDP-UTILS.repo in place
** Generated from Stack template file
![Page 19: Apache Ambari - What's New in 1.6.1](https://reader033.vdocuments.mx/reader033/viewer/2022061103/540d76b58d7f72747e8b49ce/html5/thumbnails/19.jpg)
© Hortonworks Inc. 2014
Stack .repo Template File
• HDP.repo and HDP-UTILS.repo template file
/var/lib/ambari-server/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/templates/repo_suse_rhel.j2
https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/templates/repo_suse_rhel.j2
Page 19
Ambari generates the Stack .repo files from a template3
![Page 20: Apache Ambari - What's New in 1.6.1](https://reader033.vdocuments.mx/reader033/viewer/2022061103/540d76b58d7f72747e8b49ce/html5/thumbnails/20.jpg)
© Hortonworks Inc. 2014
Managing Repositories
Page 20
Manage repository Base URLs from Ambari Web4
![Page 21: Apache Ambari - What's New in 1.6.1](https://reader033.vdocuments.mx/reader033/viewer/2022061103/540d76b58d7f72747e8b49ce/html5/thumbnails/21.jpg)
© Hortonworks Inc. 2014
Performance + Scale
Page 21
![Page 22: Apache Ambari - What's New in 1.6.1](https://reader033.vdocuments.mx/reader033/viewer/2022061103/540d76b58d7f72747e8b49ce/html5/thumbnails/22.jpg)
© Hortonworks Inc. 2014
2,000 Node Clusters
• Testing Ambari with 2,000 node clusters• Continued to move Web UI logic to Server• Big focus on Hosts page (filtering, sorting, paging)
• Ambari Session at Hadoop Summit 2014http://www.slideshare.net/Hadoop_Summit/managing-2000-node-cluster-with-ambari
http://www.youtube.com/watch?v=sVsHfmZ4s2M
Page 22
![Page 23: Apache Ambari - What's New in 1.6.1](https://reader033.vdocuments.mx/reader033/viewer/2022061103/540d76b58d7f72747e8b49ce/html5/thumbnails/23.jpg)
© Hortonworks Inc. 2014
Learn More
Page 23
Resource Location
Apache Ambari Project Page http://ambari.apache.org
Ambari Project Wiki https://cwiki.apache.org/confluence/display/AMBARI
Ambari Project JIRA https://issues.apache.org/jira/browse/AMBARI