application design per mongodb

Except where otherwise noted, this work is licensed under: http://creativecommons.org/licenses/by/3.0/

Application Design

FOR MongoDB

Alessandro Palumboapalumbo@byte-code.com

http://it.linkedin.com/in/alessandropalumbo/http://www.byte-code.com

Saturday, November 30,

Alessandro Palumbo - apalumbo@byte-code.com - http://www.byte-code.com

MongoDB

OPEN-source

MongoDB

OPEN-source

Document-OrientedJSON-style documents

MongoDB

from humongous “huge; enormous”

OPEN-source

Document-OrientedJSON-style documents

JSON-style documents

{ "_id" : "6c85fa4c-fa64-44e2-89c9-e5eb7f306ed7", "code" : "CRS0001", "name" : "Test", "description" : "Test description", "active" : true, "scheduledDate" : { "from" : ISODate("2013-09-12T00:00:00.000Z"), "to" : ISODate("2013-10-31T00:00:00.000Z") }, "version" : NumberLong(1)}

don’t be relationaL

no joins

NO FULL transactions

no joins

no SCHEMA

no joins

no SCHEMA

WE CAN EMBED

no joins

no SCHEMA

WE CAN EMBED

DOCUMENT LEVELTRANSACTIONS

no joins

no SCHEMA

WE CAN EMBED

IS IT REALLY AN ISSUE?

DOCUMENT LEVELTRANSACTIONS

friendly fire(aka RTFM)

Concern

PREFERENCE

ATOMIC

DOCUMENT

OPERATIONS

NATURAL

KEYS ASIDENTIFIERS

DESIGN

QUERYEMBEDDED

vsReferences

DBREFS

MANUALREFERENCE

DYNAMIC

SCHEMA

VSstatic

languages

PURE DRIVER

MAPPINGFRAMEWORKS

CAREFUL

WITH DATES

SPLIT DATA

MULTIPLECOLLECTIONS

Concern

PREFERENCE

ATOMIC

DOCUMENT

OPERATIONS

NATURAL

KEYS ASIDENTIFIERS

DESIGN

QUERYEMBEDDED

vsReferences

DBREFS

MANUALREFERENCE

DYNAMIC

SCHEMA

VSstatic

languages

PURE DRIVER

MAPPINGFRAMEWORKS

CAREFUL

WITH DATES

SPLIT DATA

MULTIPLECOLLECTIONS

Concern

PREFERENCE

ATOMIC

DOCUMENT

OPERATIONS

NATURAL

KEYS ASIDENTIFIERS

PERFORMANCE

PREALLOCATE

FIELDS?

be aware

the trees

PREPROCESS

RESOLUTIONDATA

TUNING

UPDATES

ANDINSERTS

DOCUMENT

MOVING

SLOWSYOU

FRIENDLY FIRE

ATOMIC

DOCUMENT

OPERATIONS

ATOMIC

DOCUMENT

OPERATIONS

OPERATIONS ON MULTIPLE DOCUMENTS ARE NOT ATOMIC

ATOMIC

DOCUMENT

OPERATIONS

NO “ALL OR NOTHING”

ATOMIC

DOCUMENT

OPERATIONS

EMBEDding OR APPLIcaTION TRANSACTIONS CAN be used to handle the issue

ATOMIC

DOCUMENT

OPERATIONS

EMBEDding OR APPLIcaTION TRANSACTIONS CAN be used to handle the issue

RELATIONAL TRANSACTIONS ARE NOT TOTALLY SAFE

“Describes the guarantee that

MongoDB provides when reporting on the success of a write

operation”

Concern

operation”

Concern

IT IS SET BY THE CLIENT AND CAN BE SET FOR EACH OPERATION

operation”

Concern

Errors Ignored

operation”

Concern

Errors Ignored Unacknowledged

operation”

Concern

Acknowledged (*)

operation”

Concern

Acknowledged (*) Journaled

operation”

Concern

Acknowledged (*) Journaled

Replica Acknowledged> 1 , majority , custom using tags

“IT describes how MongoDB clients

route read operations to members

of a replica set”

Preference

of a replica set”

Preference

of a replica set”

Preference

primary (*)

of a replica set”

Preference

primary (*) primary Preferred

of a replica set”

Preference

secondary

of a replica set”

Preference

secondary secondary PREFERRED

of a replica set”

Preference

primary (*)

nearest

primary Preferred

secondary secondary PREFERRED

All collections have an index on the id field that exists by default. If ID IS NOT PROVIDED the driver or the mongod will create an _id field with an ObjectID value.

NATURAL

KEYS ASIDENTIFIERS

NATURAL

KEYS ASIDENTIFIERS

ADD AN UNIQUE INDEX ON THE NATURAL KEY, SOMETIMES THE APPLICATION REALM CAN EVOLVE IN AN UNEXPECTED WAY

NATURAL

KEYS ASIDENTIFIERS

ADD AN UNIQUE INDEX ON THE NATURAL KEY, SOMETIMES THE APPLICATION REALM CAN EVOLVE IN AN UNEXPECTED WAY

REMEMBER THAT UNIQUE INDEXES FIELDS MUST BE PART OF THE SHARD KEY IF SHARDING IS ENABLED

DESIGN

DOCUMENT DESIGN IS FUNCTIONAL TO THE QUERIES THAT WILL EXISTS IN THE APPLICATION

DESIGN

REFERENCE OR EMBED DOCUMENTS,

“denormalized” is not always

a bad word

DESIGN

REFERENCE OR EMBED DOCUMENTS,

“denormalized” is not always

a bad word

your document design will affect what kind of OPERATIONS will be safe or not

Embedded data models allow applications to store related pieces of information in the same database record

EMBEDDED

vsReferences

EMBEDDED

vsReferences

USUALLY there is a “contains” relation

between the embedding and the embedded object

EMBEDDED

vsReferences

The maximum BSON document size is 16 megabytes and embedding may lead to performance issues if not correctly used

USUALLY there is a “contains” relation

between the embedding and the embedded object

Normalized data models describe relationships using references between documents

EMBEDDED

vsReferences

EMBEDDED

vsReferences

References provides more flexibility than embedding but remember that client-side applications will have to lookup for referenced objects with multiple queries

EMBEDDED

vsReferences

NO Referential integrity is supported, references could point to a not existing object

References provides more flexibility than embedding but remember that client-side applications will have to lookup for referenced objects with multiple queries

DBRefs are a convention for representing a document, it will hold the collection name, the id, and optionally the db name

DBREFS

MANUALREFERENCE

DBREFS

MANUALREFERENCE

MANUAL REFERENCES are just fields that will hold the id of the related document, without the collection name or the db name

DBREFS

MANUALREFERENCE

MANUAL REFERENCES are just fields that will hold the id of the related document, without the collection name or the db name

MANUAL REFERENCES are suitable for most of the use cases

CAREFUL

WITH DATES

ALWAYS Use bson date when is related to an instant of time or you will never be able to use operators on that fields

BSON Date is a 64-bit signed integer that represents the number of milliseconds since the Unix epoch (Jan 1, 1970), Negative values represent dates before 1970.The official BSON specification refers to the BSON Date type as the UTC datetime.

CAREFUL

WITH DATES

ALWAYS Use bson date when is related to an instant of time or you will never be able to use operators on that fields

split data on multiple collections to easily partition your data (a.k.a. Multitenancy)

SPLIT DATA

MULTIPLECOLLECTIONS

SPLIT DATA

MULTIPLECOLLECTIONS

use collections as namespaces for your data

SPLIT DATA

MULTIPLECOLLECTIONS

use collections as namespaces for your data

remember once data is partioned it will be more hard to aggregate if needed

why use dynamic schema if we are not using a dynamic programming language?

DYNAMIC

SCHEMA

VSstatic

languages

DYNAMIC

SCHEMA

VSstatic

languages

inheritance is not only a matter of hierarchy, it could be also a matter of composition

DYNAMIC

SCHEMA

VSstatic

languages

inheritance is not only a matter of hierarchy, it could be also a matter of composition

composition is the key to introduce dynamic schema in a static programming language

using the mongo driver directly will give you great powers, but will force you to write a lot of boilerplate code

PURE DRIVER

MAPPINGFRAMEWORKS

PURE DRIVER

MAPPINGFRAMEWORKS

MAPPING FRAMEWORKS WILL HELP TO WRITE LESS CODE, but you will sacrifice the control on all the aspects of the persistence

PURE DRIVER

MAPPINGFRAMEWORKS

MAPPING FRAMEWORKS WILL HELP TO WRITE LESS CODE, but you will sacrifice the control on all the aspects of the persistence

why not take the most from both?

PERFORMANCE

Indexes in MongoDB are defined at the collection level and can be on any field or sub-field of the document

be aware

the trees

be aware

the trees

Indexes are created using a b-tree and can be of different types

be aware

the trees

Single Field

be aware

the trees

Single Field Compound

be aware

the trees

Multikey

be aware

the trees

Multikey Geospatial

be aware

the trees

Multikey Geospatial

TEXT (BETA)

be aware

the trees

Multikey Geospatial

TEXT (BETA) Hashed

be aware

the trees

Multikey Geospatial

TEXT (BETA) Hashed

THEY COULD BE UNIQUE and sparse

MONGODB handle the space allocation of a RECORD considering also a PADDING FACTOR

DOCUMENT

MOVING

SLOWSYOU

DOCUMENT

MOVING

SLOWSYOU

WHEN AN UPDATED DOCUMENT DOES NOT FIT IN THE RECORD SPACE IT WILL BE MOVED

DOCUMENT

MOVING

SLOWSYOU

WHEN AN UPDATED DOCUMENT DOES NOT FIT IN THE RECORD SPACE IT WILL BE MOVED

DYNAMIC SCHEMA IS THE FIRST CAUSE OF DOCUMENT MOVING

FIELDS PREALLOCATION CAN FIX THE DOCUMENT MOVING ISSUES IN SOME USE CASES

PREALLOCATE

FIELDS?

PREALLOCATE

FIELDS?

Default values must be used to preallocate, this MUST BE HANDLEDin the application

PREALLOCATE

FIELDS?

Default values must be used to preallocate, this MUST BE HANDLEDin the application

NULL is not a default value :-) as it has its own type

MONGODB let you store the maximum resolution of your data

PREPROCESS

RESOLUTIONDATA

PREPROCESS

RESOLUTIONDATA

MAP REDUCE and aggregation ARE okbut you could also preprocess and have aggregated data that you can use for your queries

PREPROCESS

RESOLUTIONDATA

MAP REDUCE and aggregation ARE okbut you could also preprocess and have aggregated data that you can use for your queries

MONGODB rocks for business intelligence

MongoDB stores BSON documents as a sequence of fields and values, not as aN hash table

TUNING

UPDATES

ANDINSERTS

TUNING

UPDATES

ANDINSERTS

WRITING THE FIRST FIELD OF A DOCUMENT (OR A NESTED DOCUMENT) is considerably faster than writing THE LAST

TUNING

UPDATES

ANDINSERTS

WRITING THE FIRST FIELD OF A DOCUMENT (OR A NESTED DOCUMENT) is considerably faster than writing THE LAST

Intra-Document Hierarchy could help to handle the issue

Any questions?

application design per mongodb

cominalessandropalumbo

alessandro palumbo

mongodb alessandro palumbo

datessplit data

nosql opensourceexcept

test description

treesbe careful

datespreallocate fields

Technology

mongodb europe 2016 - mongodb 3.4 preview and introduction...

mosc2012 - building high-performance web-application with...

replacing oracle with mongodb for a templating application...

building your first mongodb application (mongo austin)

logging application behavior to mongodb

tuning linux for mongodb - percona · mongodb...

advanced troubleshooting techniques for your application...

squire: a polyglot application combining neo4j, mongodb,...

business track: metlife leapfrogs insurance industry with...

insights into mongodb for modern application development

mongodb training | mongodb online training | mongodb...

mongodb with rdbms for a portal application

back to basics: my first mongodb application

sql versus mongodb from an application development … ·...

mongodb: intro & application for big data

chris (i’m not a ghost) woods. what went right (with...

web application development · mongodb overview §...

application design for mongodb

transforming your business with scale-out flash: how mongodb...

scaling to 30,000 requests per second and beyond with...