MongoDB EnginesDemystified

April, 26, 2017Sveta Smirnova

• MySQL Support engineer• Author of

• MySQL Troubleshooting• JSON UDF functions• FILTER clause for MySQL

• Speaker• Percona Live, OOW, Fosdem,

DevConf, HighLoad...

Sveta Smirnova

2

• First version had only one engine: MMAP

• MMAP was not ideal for all workloads• TokuMX• Percona

• Acquired Tokutek in 2015• Percona Server for MongoDB

History

3

• First version had only one engine: MMAP• MMAP was not ideal for all workloads

• Collection-level locking• No ACID• No MVCC• Performance limited

• TokuMX• Percona

• Acquired Tokutek in 2015• Percona Server for MongoDB

History

3

• First version had only one engine: MMAP• MMAP was not ideal for all workloads• TokuMX

• Drop-in replacement• Document-level locking• ACID• MVCC• Scalability

• Percona• Acquired Tokutek in 2015• Percona Server for MongoDB

History

3

• First version had only one engine: MMAP• MMAP was not ideal for all workloads• TokuMX• Percona

• Acquired Tokutek in 2015• Percona Server for MongoDB

History

3

• Not everyone was happy with MMAP

• Other solutions on market• Version 3.0• Percona Server for MongoDB

• Released TokuMX as PerconaFT engine• Later discontinued

Limitations of SE API

• Added MongoRocks• Added Memory

Why Other Engines?

4

• Not everyone was happy with MMAP• Other solutions on market

• TokuMX• WiredTiger• RocksDB

• Version 3.0• Percona Server for MongoDB

• Released TokuMX as PerconaFT engine• Later discontinued

Limitations of SE API

• Added MongoRocks• Added Memory

Why Other Engines?

4

• Not everyone was happy with MMAP• Other solutions on market• Version 3.0

• MongoDB acquires WiredTiger in 2014• New Storage Engine API• WiredTiger is MongoDB storage engine

• Percona Server for MongoDB• Released TokuMX as PerconaFT engine• Later discontinued

Limitations of SE API

• Added MongoRocks• Added Memory

Why Other Engines?

4

• Not everyone was happy with MMAP• Other solutions on market• Version 3.0• Percona Server for MongoDB

• Released TokuMX as PerconaFT engine• Later discontinued

Limitations of SE API

• Added MongoRocks• Added Memory

Why Other Engines?

4

• Only one per mongod instance

• Many in multiple-instance setup• Replica Set• Sharding

Engines in MongoDB

5

• Only one per mongod instance• Many in multiple-instance setup

• Replica Set• Sharding

Engines in MongoDB

5

Memory

WiredTiger

Memory

Multiple Engines in Replica Set

6

MongoRocks

WiredTiger

MongoRocks

Multiple Engines in Replica Set

7

Shard 1 Shard 2

MR

MR

MR WT

WT

WT

Multiple Engines In Sharded Cluster

8

• Default since 3.2

• Document-level locking• MVCC/ACID• Own Journal• Compression• Own memory management• Online binary backups in Percona Server• Suites for any workload

WiredTiger

9

• Default since 3.2• Document-level locking

• MVCC/ACID• Own Journal• Compression• Own memory management• Online binary backups in Percona Server• Suites for any workload

WiredTiger

9

• Default since 3.2• Document-level locking• MVCC/ACID

• Own Journal• Compression• Own memory management• Online binary backups in Percona Server• Suites for any workload

WiredTiger

9

• Default since 3.2• Document-level locking• MVCC/ACID• Own Journal

• Compression• Own memory management• Online binary backups in Percona Server• Suites for any workload

WiredTiger

9

• Default since 3.2• Document-level locking• MVCC/ACID• Own Journal• Compression

• Own memory management• Online binary backups in Percona Server• Suites for any workload

WiredTiger

9

• Default since 3.2• Document-level locking• MVCC/ACID• Own Journal• Compression• Own memory management

• Online binary backups in Percona Server• Suites for any workload

WiredTiger

9

• Default since 3.2• Document-level locking• MVCC/ACID• Own Journal• Compression• Own memory management• Online binary backups in Percona Server

• Suites for any workload

WiredTiger

9

• Default since 3.2• Document-level locking• MVCC/ACID• Own Journal• Compression• Own memory management• Online binary backups in Percona Server• Suites for any workload

WiredTiger

9

• 3.2+

• MongoDB Enterprise only• Encryption at Rest• Based on WiredTiger• Percona suggests to use FS encryption

Encrypted

10

• 3.2+• MongoDB Enterprise only

• Encryption at Rest• Based on WiredTiger• Percona suggests to use FS encryption

Encrypted

10

• 3.2+• MongoDB Enterprise only• Encryption at Rest

• Based on WiredTiger• Percona suggests to use FS encryption

Encrypted

10

• 3.2+• MongoDB Enterprise only• Encryption at Rest• Based on WiredTiger

• Percona suggests to use FS encryption

Encrypted

10

• 3.2+• MongoDB Enterprise only• Encryption at Rest• Based on WiredTiger• Percona suggests to use FS encryption

Encrypted

10

• All data stored in memory

• No persistence after restart• Can be combined with persistent storage• Based on WiredTiger• In-Memory• Memory

• 3.2.8-2.0+• Percona Server for MongoDB• Open Source

In-Memory and Memory

11

• All data stored in memory• No persistence after restart

• Can be combined with persistent storage• Based on WiredTiger• In-Memory• Memory

• 3.2.8-2.0+• Percona Server for MongoDB• Open Source

In-Memory and Memory

11

• All data stored in memory• No persistence after restart• Can be combined with persistent storage

• Based on WiredTiger• In-Memory• Memory

• 3.2.8-2.0+• Percona Server for MongoDB• Open Source

In-Memory and Memory

11

• All data stored in memory• No persistence after restart• Can be combined with persistent storage• Based on WiredTiger

• In-Memory• Memory

• 3.2.8-2.0+• Percona Server for MongoDB• Open Source

In-Memory and Memory

11

• All data stored in memory• No persistence after restart• Can be combined with persistent storage• Based on WiredTiger• In-Memory

• 3.2.6+• MongoDB Enterprise only

• Memory• 3.2.8-2.0+• Percona Server for MongoDB• Open Source

In-Memory and Memory

11

• All data stored in memory• No persistence after restart• Can be combined with persistent storage• Based on WiredTiger• In-Memory• Memory

• 3.2.8-2.0+• Percona Server for MongoDB• Open Source

In-Memory and Memory

11

• 3.0+

• Percona Server for MongoDB• Compaction• Compression• Online binary backups• MVCC/ACID• Best suited for

• Large data• Write-intensive workloads

MongoRocks

12

• 3.0+• Percona Server for MongoDB

• Compaction• Compression• Online binary backups• MVCC/ACID• Best suited for

• Large data• Write-intensive workloads

MongoRocks

12

• 3.0+• Percona Server for MongoDB• Compaction

• Compression• Online binary backups• MVCC/ACID• Best suited for

• Large data• Write-intensive workloads

MongoRocks

12

• 3.0+• Percona Server for MongoDB• Compaction• Compression

• Online binary backups• MVCC/ACID• Best suited for

• Large data• Write-intensive workloads

MongoRocks

12

• 3.0+• Percona Server for MongoDB• Compaction• Compression• Online binary backups

• MVCC/ACID• Best suited for

• Large data• Write-intensive workloads

MongoRocks

12

• 3.0+• Percona Server for MongoDB• Compaction• Compression• Online binary backups• MVCC/ACID

• Best suited for• Large data• Write-intensive workloads

MongoRocks

12

• 3.0+• Percona Server for MongoDB• Compaction• Compression• Online binary backups• MVCC/ACID• Best suited for

• Large data• Write-intensive workloads

MongoRocks

12

• Based on original MMAP

• Default < 3.2• Memory-mapped files• Collection-level locking• Memory management delegated to system• No compression• No MVCC/ACID

MMAPv1

13

• Based on original MMAP• Default < 3.2

• Memory-mapped files• Collection-level locking• Memory management delegated to system• No compression• No MVCC/ACID

MMAPv1

13

• Based on original MMAP• Default < 3.2• Memory-mapped files

• Collection-level locking• Memory management delegated to system• No compression• No MVCC/ACID

MMAPv1

13

• Based on original MMAP• Default < 3.2• Memory-mapped files• Collection-level locking

• Memory management delegated to system• No compression• No MVCC/ACID

MMAPv1

13

• Based on original MMAP• Default < 3.2• Memory-mapped files• Collection-level locking• Memory management delegated to system

• No compression• No MVCC/ACID

MMAPv1

13

• Based on original MMAP• Default < 3.2• Memory-mapped files• Collection-level locking• Memory management delegated to system• No compression

• No MVCC/ACID

MMAPv1

13

• Based on original MMAP• Default < 3.2• Memory-mapped files• Collection-level locking• Memory management delegated to system• No compression• No MVCC/ACID

MMAPv1

13

• MMAPv1

Monitoring: PMM

14

• WiredTiger

Monitoring: PMM

14

• MongoRocks

Monitoring: PMM

14

• Storage engines provide great flexibility• They can be used in combinations• MongoDB architecture allows easily to test

if engine suits your needs• Percona Server for MongoDB

• 100 % Open Source• Includes Enterprise-grade features• Has reach monitoring

Summary

15

• MMAPv1• WiredTiger• MongoRocks• Percona Memory Engine• PMM Demo

More informaiton

16

???

Time For Questions

17

http://www.slideshare.net/SvetaSmirnova

https://twitter.com/svetsmirnova

Thank you!

18