ansible, mongodb ops manager and aws v1.1

MongoDB Pain Relief

Michael LynnSr. Solutions Architect, MongoDB

@mlynn, @[email protected]

OpsManager

OpsManager

Relieving the pain associated with manually managing your large-scale MongoDB estate.

mailto:[email protected]

mailto:[email protected]

Goals• Ops Manager, Cloud Manager, Atlas

– What are they?– How do they differ?– Why do I need them?

• Ansible– What is it and how can I use these tools to relieve some

pain in my life?

Show of Hands

• Who’s Here?– DevOps/Ops?– Site Reliability?– Engineer?– DBA?– Developer?

Admin Evolution

3. Puppet, Salt, Ansible

Leveraging automated solution for config mgt, some cloud deployment.

1. Manual

Download by hand, build server from CD, Manually Start and Config Services.

2. Scripting

Write scripts to maintain configurations, copy those to servers and run them.

4. Cloud/Hybrid Automation

Full Cloud-based provisioning and management

Managing databases manually can be...

• Painful• Risky• Complex• Susceptible to

human error*• Unnecessary...

* 80% of all downtime due to people/process error (Gartner)

What’s involved in managing MongoDB?

● Provision / Deploy

● Install/Upgrade Binaries

● Configure Database

● Administer Database

● Maintain OS - Mongod

Deploying 42

42

Upgrading 35

70

Maintaining 23

100

Play Top 3 Things DBA’s Hate to Do Manually?

https://open.spotify.com/track/2GIBwWkni4lD8e2uZ0LPQu

https://open.spotify.com/track/2GIBwWkni4lD8e2uZ0LPQu

So… what does it take (at scale)?It can take a lot of manual effort to care for a MongoDB

system in production

● Install + Configure○ 150+ steps

● Scale out, move servers, resize oplog, etc○ 10 - 180+ steps

● Upgrades, downgrades○ 100+ stepsUpgrade

Deploy

Maintain

Up to 95% Reduction in Operational Overhead

• Single-click provisioning, scaling & upgrades, admin tasks!

• Monitoring, with charts, dashboards and alerts on 100+ metrics

• Backup and restore, with point-in-time recovery, support for sharded clusters

MongoDB Ops ManagerThe Best Way to Manage MongoDB In Your Data Center


• Single-click provisioning, scaling & upgrades, admin tasks!

• Monitoring, with charts, dashboards and alerts on 100+ metrics

• Backup and restore, with point-in-time recovery, support for sharded clusters

MongoDB Cloud ManagerThe Best Way to Manage MongoDB In The Cloud


• Scalable back-end for your application on-demand

• Secure by default• Highly available, even while scaling• Patch maintenance performed for

you• Your own MongoDB cluster in the

cloud

MongoDB AtlasMongoDB’s Database as a Service

OpsManager Demo

http://ec2-54-162-176-239.compute-1.amazonaws.com:8080

This finds and leverages servers

where you’ve already installed the

automation agent



This provisions hosts AND deploys the

automation agents.

OpsManager Demo

1

2

Group/RBAC

Logical/Physical

3 Replica Set Members

3 MongoDB Version

1Add Replica Set

1Repset Name

2Version of MongoDB

3Server Pool Hostnames

4Port to Run MongoDB

1Member Roles

2DB Path on Server

3Advanced Options Settings

4CLICK APPLY

Activity

Deploy Server Instance

Upgrade (Done For You)

Configure

Administer Database

Maintain OS - Mongod

Maintain OS - OpsMgr

Activity


Upgrade MongoDB

Configure MongoDB

Administer Database



Activity


Upgrade MongoDB

Configure MongoDB

Administer Database



Ops Manager

Cloud Manager Atlas

CONTROL CONVENIENCE

MongoDB Enterprise Server

MongoDB Enterprise Advanced2

4 x

7 S

uppo

rtC

omm

ercial License

Platform Certifications

Emergency Patches

On-Demand Online Training

Customer Success Program

MongoDB CompassMongoDB Ops Manager

Monitoring & Alerting

Query Optimization

Backup & Recovery

Automation & Configuration

Schema Visualization

Data Exploration

Ad-Hoc Queries

MongoDB Connector for BI

Visualization

Analysis

Reporting

LDAP & Kerberos Auditing FIPS 140-2Encryption at Rest

REST API

MongoDB Deployment - Ops Manager

Configure Host

Provision Host Configure Agent

1. AWS 2. New Host 3. Ops Manager

Install OM Agent

Restart Agent

Install MongoDB

Configure Monitoring

Configure Backups

Configure Host


Install OM Agent

Restart Agent

Install MongoDB


Configure Backups

2. New Host 3. Ops Manager 1. AWS

More?

More?

MongoDB Deployment - Cloud Manager & Atlas

Configure Host


Install OM Agent

Restart Agent

Provision New Hosts


Configure Backups

AWS New Host Ops Manager

Provision New Hosts


Configure Backups

1. Cloud / Atlas

This provisions hosts AND deploys the

automation agents.

MongoDB Deployment - Ops Manager

Configure Host


1. AWS 2. New Host 3. Ops Manager

Install OW Agent

Restart Agent

Install MongoDB


Configure Backups

Configure Host


Install OW Agent

Restart Agent

Install MongoDB


Configure Backups

2. New Host 3. Ops Manager 1. AWS

Candidate for

Automation?

26

WHAT IS ANSIBLE?

Confidential

It’s a simple automation language that can perfectly describe an IT application infrastructure in Ansible Playbooks.

It’s an automation engine that runsAnsible Playbooks.

Ansible Tower is an enterprise framework for controlling, securing and managing your Ansible automation with a UI and RESTful API.

27

SIMPLE POWERFUL AGENTLESS

App deployment

Configuration management

Workflow orchestration

Orchestrate the app lifecycle

Human readable automation

No special coding skills needed

Tasks executed in order

Get productive quickly

Agentless architecture

Uses OpenSSH & WinRM

No agents to exploit or update

More efficient & more secure

Confidential

28

ANSIBLE TOWER

CONTROL

SIMPLE POWERFUL AGENTLESS

KNOWLEDGE DELEGATION

TOWER EMPOWERS TEAMS TO AUTOMATE

AT ANSIBLE’S CORE IS AN OPEN-SOURCE AUTOMATION ENGINE

Scheduled andcentralized jobs

Visibility and compliance

Role-based access and self-service

Everyone speaks thesame language

Designed for multi-tier deployments

Predictable, reliable,and secure

Confidential

Getting Started with Ansible - Lexicon• Commands

– ansible, ansible-playbook

• Configuration– ansible.cfg

• Modules– Hundreds of stand-alone scripts to solve common requirements

• Plays, Playbooks– The execution tools to carry out your management tasks

• Tasks– The components that connect ansible to the servers

Why Ansible?• Simple• Human Readable• Agentless (uses SSH)• Popular - Lots of Modules• Used by Twitter, NASA, Spotify

Demo of Ansible

Follow Along• Git (or download)

– git clone http://github.com/mrlynn/ansible-demo– Thank you Torsten Spindler

• Ansible– brew install ansible

• Pipe Viewer (optional)– Simulates typing

• AWS->EC2– Keys Set Up

http://github.com/mrlynn/ansible-demo

Post-Demo Check-in• Atlas

– DBaaS– Deploys new MongoDB Clusters from zero to cluster in minutes.

• Ops Manager – Automation for all the things you do with MongoDB - except deploying the agent.

• Ansible– Automation for configs, deployment and more - especially deploying O/M Agents

I hope you learned some ways you can relieve the pain associated with manually configuring, deploying and maintaining your database environment.

Questions?

Get These Scripts and Slides:http://github.com/mrlynn/ansible-demo



1 - Ansible Configuration Files

2 - Ansible Hosts File

3 - Ansible Modules - Ping

4 - Ansible Environment Variables

5 - Ansible Modules - Script

6 - Ansible Playbooks

7 - Deploying O/M in Playbooks

BONUS SLIDES!

• Slides probably not used in the presentation.

Replica Set Architecture (w/ Ops Manager)

43

APP SERVER

APPLICATION

DRIVER

OPS MANAGER

OM APP

APP BACKUP

REPLICA SET

PRIMARY

SECONDARY SECONDARY

OM AGENT

OM AGENT

OM AGENT

Demonstration SetupCommand Description

ansible -m ping -i ansible-hosts all Using the ping module and the hosts file ansible-hosts in the current directory, ping all hosts

export ANSIBLE_HOSTS=./ansible_hosts Save some time - ansible uses environment variables extensivel

ansible -m ping all Same as before - but now ansible leverages the env var to find the ansible hosts file.

Demonstration Command Description

ansible -m script test.sh all Here, we take a local script and execute it across all of our hosts. Ansible takes that script and delivers it via scp to the hosts, executes it and captures the output.

export ANSIBLE_HOSTS=./ansible_hosts Save some time - ansible uses environment variables extensivel

ansible -m ping all Same as before - but now ansible leverages the env var to find the ansible hosts file.

Appendix A - /etc/ansible/hosts sample[opsManager]ec2-54-93-114-205.eu-central-1.compute.amazonaws.com ansible_user=ec2-user[amlReplicaSet]ec2-54-93-79-122.eu-central-1.compute.amazonaws.com ansible_user=ec2-userec2-54-93-176-246.eu-central-1.compute.amazonaws.com ansible_user=ec2-userec2-54-93-207-148.eu-central-1.compute.amazonaws.com ansible_user=ec2-user[amlReplicaSet:vars]opsmanagerurl=http://ec2-54-93-114-205.eu-central-1.compute.amazonaws.com:8080opsmanager=ec2-54-93-114-205.eu-central-1.compute.amazonaws.com

Appendix B - ansible.cfg[defaults]host_key_checking = Falseprivate_key_file = PATH/TO/AWS/KEY.FILE

[ssh_connection]control_path = %(directory)s/%%C

ansible, mongodb ops manager and aws v1.1

Software