building a mongodb cluster using ansible - percona · building a mongodb cluster using ansible...

Building a MongoDB Cluster using AnsibleOverview and demoPatrick Galbraith, ATG

March 2016

© Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.2

2

About the speaker● Patrick Galbraith

● HP Advanced Technology Group

● Has worked at Blue Gecko, MySQL AB, Classmates, Slashdot, Cobalt

Group, US Navy, K-mart

● MySQL projects: memcached UDFs, DBD::mysql, federated storage engine

● Family

● Outdoors


Overview

3

• What this talk is

• What this talk is not


Ansible

Automation Engine

Application deployment, configuration management, provisioning, orchestration

Agentless / SSH connections

Push model – programs (modules) pushed to nodes and executed over SSH

Copies files to remote location being configured, executes, wipes

Inventory described and managed in a text file

Inventory can be static or dynamic

Playbooks: the Ansible orchestration language

YAML file, designed to be human readable

4


Ansible: Inventory file

List of hosts being

managed

Grouped into categories

(master/minion, regions,

type)

Hierarchical

5

[web]

web1

web2

web3

[mongo-rs1]

mongo1

mongo2

mongo3

[ns]

ns1

ns2


Ansible: Inventory file

List of hosts being

managed

Grouped into categories

(master/minion, regions,

type)

Hierarchical

6

[southwest:children]

arizona

new-mexico

[arizona]

phoenix

tuscon

[new-mexico]

albuquerque

santa-fe


Ansible: Playbook, example top level

Language of Configuration,

deployment and orchestration

Describe configuration you want

to enforce

Contains "plays" (steps of

process being executed)

Map to specific groups of hosts

Include roles (pre-packaged units of

work)

7

---

- hosts: "{{ target|default(’web') }}”

roles:

- common

- nginx

- hosts: "{{ target|default('mongo-

rs1') }}”

roles:

- common

- mongodb-sharded-cluster


Ansible: Playbook

8

- name: Install the apt key for ubuntu

apt_key: id=7F0CEB10 keyserver="keyserver.ubuntu.com" state=present

- name: Install the repository for Ubuntu mongodb apt_repository: repo="deb

http://repo.mongodb.org/apt/{{ ansible_os_family|lower }} {{ debian_version.stdout

}}/mongodb-org/3.0 main" state=present

- name: pre-create ntp group, system group: name=ntp system=yes state=present

- name: pre-create ntp user, system user: name=ntp group=ntp system=yes

state=present

- name: install NTP

apt: name=ntp state=present update_cache=yes

notify: restart ntp

- name: install various packages

apt: name={{ item }} state=present update_cache=yes

with_items: common_packages when: ansible_os_family == "Debian"


Ansible roles

9

• Organized means of grouping together common tasks,

variables, handlers, templates, etc.

• Abstract complex playbooks

• Have a common directory structure

• Re-usable

• Ansible Galaxy


Ansible modules

1

0

• Module library

• Usually Written in Python

• Use a common API for returning json to Ansible to

indicate failure or success

• For just about everything you would need!

• Divided into core and extras


Ansible: Role directory layout

1

1

./defaults/main.yml

./files/10gen.repo

./files/epel.repo

./meta/main.yml

./tasks/configure.yml

./tasks/main.yml

./tasks/mongod.yml

./tasks/shards.yml

./templates

./templates/mongod.conf.j2

./templates/shard_init.j2

./templates/user.j2

./vars

./vars/Debian.yml

./vars/main.yml

./vars/RedHat.yml


ansible-mongodb-cluster automation steps

1

2

• Set up AWS key and security groups

• Launch instances (6)

• Install mongo base packages (no-server, shell,

tools, pymongo)

• Install mongodb server

• Configure and set up replicated mongod

• Configure and set up config servers

• Configure and set up Mongos routing service

• Add shards


Mongod configuration (6 - 2 repsets of 3)

1

3

• Install server

• Shut down

• Create /var/lib/mongo/mongod data dir

• Replace default init and config file, no auth

• Generate user json and repset json

• Start up mongod

• Add users

• Restart with auth

• Initialize replica set


Mongo config server configuration (3)

1

4

• Shut down

• Create /var/lib/mongodb/mongo_cs

datadir

• Replace default init and config file, no auth

• Generate user json

• Start up config server

• Add users

• Restart with auth


Mongos routing server configuration (1 .. 2)

1

5

• Shut down

• Create /var/lib/mongodb/mongos dir

• Replace default init and config file (list of config

servers)

• Start up mongos

• Add shards


Role layout for mongodb-sharded-cluster

1

6

• defaults/

• Generic settings for role

• files/

• Static files, repo files

• tasks/

• The various playbooks for each stage

• templates/

• Jinja templates for mongo init and config as well as

JSON for replication and sharding

• vars/

• Role variables, per Linux dist


mongodb-sharded-cluster top-level playbook

1

7

- name: install mongo base packages

include: mongod.yml

tags: - mongod

- name: configure config server

include: configure_cs.yml

when: inventory_hostname in groups['config-servers']

tags:

- cs

- name: configure mongos server

include: configure_mongos.yml

when: inventory_hostname in groups['mongos-server']

tags:

- mongos

- name: add shards

include: shards.yml

when: inventory_hostname in groups['mongos-servers']

tags:

- mongos

- shards


Inventory file structure

1

8

[mongo-cluster:children]

rs1

rs2

rsN

[rs1]

mongo1-rs1

mongo2-rs1

mongo3-rs1

[rs2]

mongo1-rs2

mongo2-rs2

mongo3-rs2

[rsN]

…

[config-servers]

cfg1

cfg2

cfg3

[mongos-servers]

mongos1

mongos2


Diagram representation of inventory

1

9


group_vars

2

0

ls group_vars

all all.example rs1 rs2 rsN

mongod_datadir_prefix: /var/lib/mongodb

mongod_replication: true

mongod_repl_master: mongo1-rs1

mongod_repset_members:

mongo2-rs1.example.com: rs1

mongo2-rs2.example.com: rs2

mongo2-rsN.example.com: rsN

mongod_replset_name: rs1

mongod_repl_servers:

- mongo1-rs1.example.com



mongod_config_servers:

- cfg1.example.com:27019



mongos_servers:

- mongos1.example.com

- mongos2.example.com


Basic provisioning process

2

1

us-west-1 (west1) us-west-2 (west2)

mongodmongod mongod mongod

mongo

dmongod

mongo_c

s

mongo_c

s

mongo_c

s

mongos


Automate the automation!

2

2

• Use Ansible with “facts” modules or dynamic inventory plugins. In this example, ec2_facts is used

• Use Ansible to render inventory using ec2_facts and jinja template

• Use Ansible to modify local files such as add entries to /etc/hosts

• Use Ansible to render playbooks and group_vars/host_vars

• https://github.com/CaptTofu/mongo-cluster-ansible.git


Demo

2

3


Repos and info

2

4

• https://github.com/CaptTofu/mongodb-sharded-cluster.git

• https://github.com/CaptTofu/mongo-cluster-ansible.git

• [email protected]

https://github.com/CaptTofu/mongodb-sharded-cluster.git

https://github.com/CaptTofu/mongo-cluster-ansible.git

building a mongodb cluster using ansible - percona · building a mongodb cluster using ansible...

Documents