Best Practices for Upgrading to MongoDB 3.2

Achille Brighton

This talk

• What's cool in MongoDB version 3.2• Upgrading MongoDB In Outline• The Fine Details• Upgrading Revisited• One weird trick

What’s cool in 3.2

More use cases. Pluggable storage engines enables you to use MongoDB in

more projects with the same core database.

Mission-critical apps. MongoDB delivers major advances in the critical areas of

governance, high availability, and disaster recovery.

New tools for new users. Now MongoDB is an integral part of the tooling and

workflows of Data Analysts, DBAs, and Operations teams.

3.2 Overview

Encrypted Storage

What you get • Encryption of sensitive data for regulated industries• ~ 15% overhead

– Better than many 3rd party encryption

Note• Based on the WiredTiger Storage Engine • Available in MongoDB Enterprise Advanced

Document Validation

What you get • Implement data governance • Enforce data quality across• Use familiar MongoDB queries to specify rules

More power to the DBAs • The DBA can specify which documents in a collection

should be validated • Failed validations can be configured

– Hard error– Just a warning

Partial Indexes

$lookup Join

What you get • Equivalent to left join• Use with aggregation

$lookup: { localField : "stock", foreignField: "stockQuote", from : "other_collection", as : "joined_stock" }

CSRS – Config Server Replica Set

CSRS

Overview• Config server replica sets can span more than 3 data

centers with up to 50 replica set members supported

What you get • Simplified sharded deployments

– Config servers are deployed as replica sets• Improved metadata consistency • Easily scale to many data centers

Fast Failover

• Enhanced algorithm detects failure and isolation of primary.• Clusters more resilient to overloaded or unreliable networks

What you get • 2 seconds failover

– Exact time is dependent on system configuration – Tune the timing with the electionTimeoutMillis

Storage Engines

In-Memory Storage

• Predictable throughput and latency• In-memory computing without trading away guarantees

of disk-based databases like – Rich query flexibility – Real-time analytics– Scalable capacity – Durability

• Durability via replica set secondaries (using mmap/wt)

WiredTiger is the Default

What you get • Best general-purpose storage engine• 7-10x better throughput• Up to 80% compression

“Out of the box” • No special configuration needed

7x-10x Performance, 50%-80% Less StorageHow: WiredTiger Storage Engine• Same data model, same query

language, same ops• Write performance gains driven

by document-level concurrency control

• Storage savings driven by native compression

• 100% backwards compatible• Non-disruptive upgrade

MongoDB 3.0MongoDB 2.6

Performance

50%-80% Less Storage via Compression

• Better storage utilization• Higher I/O scalability• Multiple compression options

– Snappy– zlib– None

• Data and journal compressed on disk• Indexes compressed on disk and in memory

Compression in action

New Tools for New Users

Data Analysts• BI Connector• $lookup

• Aggregation Enhancements: Faster & simpler real-time analytics

DBAs

MongoDB Compass

• Visualize schema• View Documents• Graphically build queries• Authenticated access

Operations

Integration with standard operational workflow reduces overhead• Start from global view of infrastructure: APM tools

integration, e.g. New Relic, AppDynamics.• Then, drill down: Profiler visualization in Ops Manager • Then, deploy: Automated index builds• Next, refine: Partial Indexes improve resource utilization

Ops Manager & MMS

Single-click provisioning, scaling & upgrades, admin tasks

Monitoring, with charts, dashboards and alerts on 100+ metrics

Backup and restore, with point-in-time recovery, support for sharded clusters

MongoDB Ops Manager / MMS

The Best Way to Manage MongoDB In Your Data CenterUp to 95% Reduction in Operational Overhead

How Ops Manager / MMS Helps You

Scale EasilyMeet SLAs

Best Practices, Automated

Cut Management Overhead

Upgrading MongoDB in Outline

• Manual upgrades – lots of steps, but doable• Automatic – Ops Manager / MMS

Manual Upgrade

Upgrading a cluster manually

• Follow the steps in the fine documentation:– http://docs.mongodb.org/manual/release-notes/3.0-upgrade/– http://docs.mongodb.org/manual/release-notes/3.2-upgrade/

• (coming soon)– First result if Googling for “MongoDB Upgrade”

• It's critical to note that upgrade order matters:– For replica sets, first upgrade the secondaries, then

the primaries.– For a sharded cluster, follow the 7 steps in the

documentation.

Automatic

Upgrading a cluster with Automation

• Ops Manager / MMS is the easiest, fastest, and best way to upgrade a MongoDB deployment– One-click upgrades.– Rolling upgrades, so zero downtime.

Upgrades

• Select Desired Version from Dropdown

The fine details

• Backward compatibility & your application• Backward compatibility & your operations• Database behavior & performance• Enabling 3.2 Features

Backward Compatibility

Backward Compatibility: App

• Comprehensive documentation about incompatible changes– https://docs.mongodb.org/manual/release-notes/3.2-compatibility

• Highlights– You may need to upgrade your applications' drivers

before upgrading MongoDB!– Some commands/methods deprecated, removed,

changed.

• Audit your codebase for uses of deprecated functionality.

Backward Compatibility: Ops

• Comprehensive documentation about incompatible changes– https://docs.mongodb.org/manual/release-notes/3.2-compatibility

• Highlights– Some settings deprecated, others mmapv1-only.– Stricter replica set configuration validation– mongo* tools' options changed– Legacy (pre-2.6) user account model removed

• Audit scripts & other infrastructure you've got around.

Database Behavior & Performance

Database Behavior & Performance

• Always test your application against new version before upgrading production– Apps sometimes rely on unspecified behaviors (e.g.

stability of results from unsorted queries)– Query optimizer improvements may affect index

selection (check important queries with explain)

Good practice: Test against Staging

• Run all your performance correctness tests against a staging environment for some time before upgrading production.

• Best practice: run a real application workload against a 3.0 staging environment, too.– Replicate the workload manually– Use Facebook’s Flashback tool

Upgrading to use 3.2 Features

Protocol Version Upgrade

• Protocol Version 0 (Current)• Protocol Version 1 (New)

– Conceptually:• cfg=rs.conf()• cfg.protocolVersion=1• rs.reconfig(cfg)

CSRS – Config Replicaset

• Allows up to 50 config servers• New clusters only, or brief downtime

Encryption at Rest

• Rolling upgrade

Document Validation

• Validation Level– Off– Strict– Moderate (allows existing invalid documents)

• Validation Action– Warn– Error

Recap

Upgrading Revisited

• Plan• Practice• Productionize

Plan Your Upgrade

Plan Your Upgrade

• Review the compatibility notes– Audit apps, ops for deprecated details

• Upgrade drivers, scripts, if necessary– Might require some recoding, in edge cases

• Make sure you've got good tests– App behavior, performance, etc.

• Write a checklist of upgrade steps for your environments

Practice Your Upgrade

Practice Your Upgrade

• Upgrade apps & database in staging environment• Test, test, test.

– If no problems appear within (say) a week of continuous testing, probably a decent candidate.

Productionize Your Upgrade

We Can Help!

• MongoDB's Professional Services group offers consulting engagements.– Ensuring upward compatibility for your apps, tools– Assisting with performance & behavior testing around

database upgrades– Preparing & executing upgrade plans with your team

in your environments.

– Let us know: consulting@mongodb.com

consulting@mongodb.com