apache ambari - what's new in 1.6.0
DESCRIPTION
TRANSCRIPT
© Hortonworks Inc. 2014
Apache Ambari1.6.0
May 2014
© Hortonworks Inc. 2014
What’s New in 1.6.0
• PostgreSQL Support• Ambari Blueprints (and /stacks API)
• For a complete list of changeshttps://issues.apache.org/jira/browse/AMBARI/fixforversion/12326544
Page 2
© Hortonworks Inc. 2014
Database Support
Page 3
Hive Metastore Oozie Ambari DB
MySQL 5.x
Oracle 11g
PostgreSQL 8, 9
Default
New in Ambari 1.6.0
PostgreSQLMySQL Derby
© Hortonworks Inc. 2014
Ambari DB
• Ambari Server Setup
ambari-server setup
Page 4
© Hortonworks Inc. 2014
Hive Metastore and Oozie DB
• Cluster Install Wizard > Customize Services• Hive and Oozie
Page 5
© Hortonworks Inc. 2014
/stacks API
Page 6
© Hortonworks Inc. 2014
Ambari Stacks
Page 7
AMBARISERVER
Stack
Command Scripts
Service Definitions
AMBARIAGENT/SAMBARI
AGENT/SAMBARIAGENT/S
pythonxml
Repos
© Hortonworks Inc. 2014
/stacks API
• REST API for reading Stacks information
• Including Services and Repositories
• Stack -> Versions -> Services -> Components
• Stack -> Versions -> Operating Systems -> Repositories
Page 8
© Hortonworks Inc. 2014
OS Families + Repositories
Page 9
HDP 2.1
redhat5*
redhat6*
suse11* HDP-2.1
HDP-2.1
HDP-2.1
* Represents the “OS Family” (for example: “redhat6 = centos6 = oraclelinux6”)
OS REPOSITORYSTACK
© Hortonworks Inc. 2014
HDP.repo
Page 10
Ambari Server Ambari Agent
4) Server sends HDP.repo
2) Reads Stacks for HDP-2.1 base url*
6) Agent uses HDP.repo when instructed to install packages
1) Agent reports OS
3) Creates HDP.repo
5) Agent puts HDP.repo in place
© Hortonworks Inc. 2014
HDP.repo
Page 11
Ambari Server Ambari Agent
4) Server sends HDP.repo
2) Reads Stacks for HDP-2.1 base url*
6) Agent uses HDP.repo when instructed to install packages
1) Agent reports OS
3) Creates HDP.repo
5) Agent puts HDP.repo in place
© Hortonworks Inc. 2014
Set Base URL: During Install
Page 12
Ambari will set Base URL using input from user.
Ambari gets the default values from Stack definition (and Latest Base URL, if internet available).
© Hortonworks Inc. 2014
Set Base URL: During Upgrade
Page 13
ambari-server upgradestack HDP-2.1 http://your.local.repository redhat6
Stack Base URL OS Family
© Hortonworks Inc. 2014
Ambari Blueprints
Page 14
© Hortonworks Inc. 2014
Ambari Blueprints
Page 15
STACKDEFINITION
COMPONENTLAYOUT BLUEPRINT
BLUEPRINT INSTANTIATE CLUSTER
© Hortonworks Inc. 2014
Using Blueprints
Page 16
Install, Setup and Start Ambari Server1
Install, Setup and Start Ambari Agents2
Define Blueprint (stack, layout and configurations)3
Define cluster instance4
Instantiate cluster instance and “watch” progress5
© Hortonworks Inc. 2014
Blueprint API Calls
Page 17
BLUEPRINTPOST /blueprints/my-blueprint
CLUSTERINSTANCE POST /clusters/MyCluster
1
2
© Hortonworks Inc. 2014
Example: Single-Node Definitions
Page 18
{ "configurations" : [ { ”hdfs-site" : {
"dfs.namenode.name.dir" : ”/hadoop/nn" } } ], "host_groups" : [ { "name" : ”uber-host", "components" : [ { "name" : "NAMENODE” }, { "name" : "SECONDARY_NAMENODE” }, { "name" : "DATANODE” }, { "name" : "HDFS_CLIENT” }, { "name" : "RESOURCEMANAGER” }, { "name" : "NODEMANAGER” }, { "name" : "YARN_CLIENT” }, { "name" : "HISTORYSERVER” }, { "name" : "MAPREDUCE2_CLIENT” } ], "cardinality" : "1" } ], "Blueprints" : { "blueprint_name" : "single-node-hdfs-yarn", "stack_name" : "HDP", "stack_version" : "2.1" }}
{ "blueprint" : "single-node-hdfs-yarn", "host_groups" :[ { "name" : ”uber-host", "hosts" : [ { "fqdn" : "c6401.ambari.apache.org”
} ] } ]}
BLUEPRINT
CLUSTER INSTANCE
Description• Single-node cluster• Use HDP 2.0 Stack• HDFS + YARN + MR2• Everything on c6401
© Hortonworks Inc. 2014
New Blueprint API Parameters
• Configurations• Password Checking
Page 19
© Hortonworks Inc. 2014
Configurations
Page 20
{ "configurations" : [ { ”hdfs-site" : {
"dfs.namenode.name.dir" : ”/hadoop/nn" } } ], "host_groups" : [ { ... } ], "Blueprints" : { "blueprint_name" : "single-node-hdfs-yarn", "stack_name" : "HDP", "stack_version" : "2.1" }}
SET AT BLUEPRINT{ "blueprint" : "single-node-hdfs-yarn", "configurations" : [ { "hdfs-site" : { "dfs.namenode.name.dir" : "/h/d/f/s/nn" } } ], "host_groups" :[ { "name" : "host_group_1", "hosts" : [ { "fqdn" : "c6402.ambari.apache.org" } ] } ]}
OVERRIDE AT CLUSTER
© Hortonworks Inc. 2014
Password Checking
• Certain Stack components require passwords–Nagios – Nagios Web Password–Hive Metastore “MySQL” – Database connection password–Oozie Server “Derby” – Database connection password
• Set each individually (with blueprint or cluster)• OR set with a “default password”
Page 21
{ "blueprint" : "single-node-hdfs-yarn", ”default_password" : ”mypass", "configurations" : [ ...
© Hortonworks Inc. 2014
Learn More
Page 22
Resource Location
Apache Ambari Project Page http://ambari.apache.org
Ambari Project Wiki https://cwiki.apache.org/confluence/display/AMBARI
Ambari Project JIRA https://issues.apache.org/jira/browse/AMBARI