using mongodb to build a fast and scalable content repository

H Y P E R S C A L E D I G I T A L A S S E T P L A T F O R M

U S I N G M O N G O D B T O B U I L D

A F A S T A N D S C A L A B L E

C O N T E N T R E P O S I T O R Y

N U X E O P L A T F O R M

We provide a platform that organisations can use to

drive digital transformation by unlocking the full value of digital assets.

https://www.nuxeo.com

http://www.nuxeo.com

LEADER ON DIGITAL LOCAL INFORMATION MANAGEMENT: Addresses, Phone Numbers, Activities of local businesses

USING NUXEO AS A CATALOG FOR INFORMATION ASSETS

HUNDREDS OF MILLIONS DOCUMENTS AND UP TO DAILY 1 M BULK UPDATE

ADAPT THE BUSINESS / INCREASE MARKET POSITION

INFORMATION UPDATE DELIVERY REDUCED FROM 1 YEAR TO 1H

N U X E O P L A T F O R M B E N E F I T S

SIMPLE SOFTWARE ARCHITECTURE

EASY SCALABILITY

HYPERSCALE PERFORMANCE

TRANSACTION MANAGEMENT

The Nuxeo + MongoDB Benefits

Large stream - Large storageattached blobs

Flexible Schema - Write Once/Read ManyAudit Log, activity log

Complex structures - R/W synchronousDocument properties and hierarchy

Flexible Schema - SearchSearch index

S I M P L E A R C H I T E C T U R E


Large stream - Large storageattached blobs

Flexible Schema - Write Once/Read ManyAudit Log, activity log

Complex structures - R/W synchronousDocument properties and hierarchy

Flexible Schema - SearchSearch index

S C A L A B I L I T Y

H Y P E R S C A L E P E R F O R M A N C E

The One Billion Benchmark

• 1 billion documents import in 8.5 hours

• 32,680 docs/s with peak at 40,400 docs/s

N U X E O P L A T F O R M

DOCUMENT REPOSITORY

No ORM Hell

S I M P L I C I T Y ?

Single consolidated storage

Out of the box robust deployment

I M P E D A N C E M I S M A T C H

I S S U E

I M P E D A N C E M I S M A T C H

I S S U E

NO LAZY LOADING

NO CACHE

NO INVALIDATION

A LOT OF COMPLEXITY AND PROBLEMS AVOIDED !

I M P A C T O N D E P L O Y M E N T

C O N S O L I D A T E D S T O R A G E

Structures

AuditBlobs

Indexes

SINGLE CONSOLIDATED STORAGE Structure, Blobs, Audit & Index

FEWER BUILDING BLOCKS TO PROVISION & CONFIGURE Easier to deploy

E A S Y D E P L O Y M E N T

"BUILT-IN" - DATA REDUNDANCY & FAULT TOLERANCE

Active

Active


WILL I SCALE BETTER WITH MONGODB ?


S C A L E O U T R E A D S

S C A L E O U T W R I T E S•Leverage sharding

•Spread Writes

•Leverages replicasets

•Read from secondaries


S C A L E O U T T E S T

Use massive read operations and queries.

2 Nuxeo nodes + 1 MongoDB node

1850 docs/sMongoDB CPU is the

bottleneck (800%)

S C A L E O U T T E S T

Use massive read operations and queries.

2 Nuxeo nodes + 2 MongoDB nodes 3400 docs/s

(using read preferences) S C A L A B I L I T Y


S H A R D I N G T E S T

2 Nuxeo nodes +

1 MongoDB ReplicaSet

11,000 docs/s


S H A R D I N G T E S T

2 Nuxeo nodes +

3 MongoDB Sharded ReplicaSet 27,400 docs/s

H Y P E R S C A L E P E R F O R M A N C E

WILL I BE FASTER WITH MONGODB ?

B U I LT F O R S P E E D

N O I M P E D A N C E I S S U E

D O C U M E N T L E V E L L O C K I N G

• No table level concurrency

• Fewer backend calls

• No invalidation costs

N A T I V E D I S T R I B U T E D A R C H I T E C T U R E

• Easy scale out

S P E E D

https://benchmarks.nuxeo.com/continuous/index.html

Significant RAW Speed improvements for all use cases

More importantly: some use cases are much better handled

M O R E T H A N R A W

P E R F O R M A N C E

• No cache

• Less Memory per connection

• Can handle more connection

• Can handle more concurrent users

Handle more concurrent connections



With SQL, Read and Write operation are competing



Writes are not blocked by Reads

With MongoDB writes operations are not blocked



Large object processing improved

lazy loadingcache trashing

750 documents/s 11,500 documents/s

T R A N S A C T I O N M A N A G E M E N T

MEANS• Different transaction paradigms

• Provide shared mitigation policies for critical use case

NEW STORAGE MODEL• Document Level transaction

• No MVCC isolation

T R A N S A C T I O N M A N A G E M E N T

C O N S I S T E N C Y

I N O U R C O N T E X T

Transactions can not span across multiple documents

• Atomic Document Operations are safe • Large batch updates can not be Atomic

Multi-documents transactions can be problematic Workflows or custom event handlers

FIND A WAY TO MITIGATE APPLICATION LEVEL IMPACT


TRANSIENT STATE MANAGER Run all operations in Memory

Populate an Undo Log

Recover Application level Transaction Management• Commit / Rollback model

"Read uncommited" isolation• Need to flush transient state for queries• "uncommited" changes are visible to others

T A K E A W A Y S Nuxeo +

MongoDB

Simple Architecture

Scalable & Performant

Transaction Management

Content Management + MongoDB You should try Nuxeo !

A N Y Q U E S T I O N S ?T H A N K Y O U !

https://github.com/nuxeo nuxeo.com/careers/ @damienmetzler

using mongodb to build a fast and scalable content repository

Software