let the tiger roar - mongodb 3.0
TRANSCRIPT
Agenda
• MongoDB 3.0 • Pluggable Storage Engine API • Storage Engines
– MMAPv1 – WiredTiger – WT vs MMAPv1
• Improvements
MongoDB 3.0
h"p://www.pixelresort.com/wp-‐content/uploads/2013/06/3point0.jpg
A lot of good things come with 3
• USB
• 3G
• Tricycle
MongoDB 3.0
• Pluggable Storage Engine API • Storage Engines • Large Replica Sets • Big Polygon • Security Enhancements – SCRAM • Audit Trail • Simplified Operations – Ops Manager • Tools Rewrite
MongoDB 3.0 is a bag full of goodies!
Storage Engine
h"p://[email protected]/wp-‐content/uploads/2012/08/PowerHaul-‐Engine-‐Green_844x680.jpg
How does MongoDB persist data?
• <= MongoDB 2.6 – One unique mechanism using Memory Mapped Files – "mmapv1" Storage Engine
• MongoDB 3.0 has a few more options – mmapv1 – default – wiredTiger – (in_memory – experimental only)
Pluggable Storage Engine API
h"p://www.livingincebuforums.com/ipb/uploads/monthly_10_2011/post-‐198-‐0-‐67871200-‐1318223706.jpg
Storage Engine API
• Allows to "plug-in" different storage engines – Different work sets require different performance
characteristics – mmapv1 is not ideal for all workloads – More flexibility
• Can mix storage engines on same replica set/sharded cluster
• Opportunity to integrate further ( HDFS, native encrypted, hardware optimized …)
MMAPv1
h"ps://angrytechnician.files.wordpress.com/2009/05/memory.jpg
MMAPv1
MMAPv1
• Improved concurrency control • Great performance on read-heavy workloads
• Data & Indexes memory mapped into virtual address space
• Data access is paged into RAM
• OS evicts using LRU
• More frequently used pages stay in RAM
WiredTiger
What is WiredTiger?
• Storage engine company founded by BerkeleyDB alums • Recently acquired by MongoDB • Available as a storage engine option in MongoDB 3.0
Why is WiredTiger Awesome
• Document-level concurrency • Disk Compression • Consistency without journaling • Better performance on certain workloads
– write heavy
Improving Concurrency
• 2.2 – Global Lock • 2.4 – Database-level Locking • 3.0 MMAPv1 – Collection-level Locking • 3.0 WT – Document-level
– Writes no longer block all other writes – Higher level of concurrency leads to more
CPU usage
Compression
• WT uses snappy compression by default • Data is compressed on disk • 2 supported compression algorithms
– snappy: default. Good compression, relatively low overhead
– zlib: Better
• Indexes are compressed using prefix compression – Allows compression in memory
Consistency without Journaling
• MMAPv1 uses write-ahead log (journal) to guarantee consistency
• WT doesn't have this need: no in-place updates – Write-ahead log committed at checkpoints
• 2GB or 60sec by default – configurable! – No journal commit interval: writes are written to
journal as they come in – Better for insert-heavy workloads
• Replication guarantees the durability
MMAPv1 vs. WT
How to run MMAPv1
• MMAPv1 is the default
How to run WT
• mongod now has --storageEngine option
MMAPv1 Database Files
• MMAPv1 persists data to files per databases – Indexes – Data
WT Database Files
• Each collection & indexes stored in own file
Playing nice together
• Can not – Can't copy database files – Can't just restart w/ same dbpath
• Yes we can! – Initial sync from replica set works perfectly! – mongodump/restore
• Rolling upgrade of replica set to WT: – Shutdown secondary – Delete dbpath – Relaunch w/ --storageEngine=wiredTiger – Rollover
Other WT configuration options
• Compression: --wiredTigerCollectionBlockCompressor
• YAML format for configuration
Gotcha's!!!
• No 32-bit Support – WT is 64bit only
• system.indexes & system.namespaces deprecated – Explicit commands: db.getIndexes() db.getCollectionNames()
h"ps://@ngbudongchine.files.wordpress.com/2012/08/lemonde1.jpeg
Small Demo
Improvements
Benefits
Wider Range of Use Cases
How: Flexible Storage Architecture • Fundamental rearchitecture, with new pluggable storage engine API • Same data model, same query language, same ops • But under the hood, many storage engines optimized for many use
cases
Single View Content Management
Real-Time Analytics Catalog
Internet of Things (IoT) Messaging
Log Data Tick Data
Up to 95% Lower Operational Overhead
How: MongoDB Ops Manager • The best way to run MongoDB • Automates core management
tasks • Single-click provisioning, scaling,
upgrades, administration • Monitoring, with charts,
dashboards & alerts on 100+ metrics
• Backup and restore, with point-in-time recovery
7x-10x Performance, 50%-80% Less Storage
How: WiredTiger Storage Engine • Same data model, same query
language, same ops • Write performance gains driven
by document-level concurrency control
• Storage savings driven by native compression
• 100% backwards compatible • Non-disruptive upgrade
MongoDB 3.0 MongoDB 2.6
Performance
h"p://www.humanandnatural.com/data/media/178/badan_jaran_desert_oasis_china.jpg
Please go and test it!
3.0.0-RC7 https://www.mongodb.org/downloads http://www.mongodb.com/lp/misc/norberto-leite https://jira.mongodb.org/secure/Dashboard.jspa
h"p://[email protected]/blog/wp-‐content/uploads/2015/01/hammer.jpg
h"p://www.humanandnatural.com/data/media/178/badan_jaran_desert_oasis_china.jpg
Questions?
@nleite [email protected]
