MongoDB Drivers

& High Availability

A. Jesse Jiryu Davis Staff Engineer, MongoDB

@jessejiryudavis

ClientSide

• It’s a MongoDB driver specification.

• (It’s not MMS.)

• Written by me. Advisors: David Golden, Craig Wilson, Jeff Yemin.Overseen by Bernie Hackett.

Server Discovery

& Monitoring Spec

Good News

& Bad News

Agenda•Startup

• Steady State

•Crisis

•Resolution

•Practical Applications

• Further Study

Setup

•3-node replica set

•PyMongo 3

client = MongoClient(! 'mongodb://host1,host2/?replicaSet=my-rs')!

primaryhost1

secondaryhost2

secondaryhost3

Python Program

monitorthread

monitorthread

MongoClient

{secondary: true, hosts: ["host1", "host2", "host3"]}

"mongodb://host1,host2/?replicaSet=my-rs"

ismaster(from secondary)

{"setName": "rs-name",! "ismaster": false,! "secondary": true,! "hosts": ["host1", "host2", "host3"]}!

{"setName": "rs-name",! "ismaster": true,! "secondary": false,! "hosts": ["host1", "host2", "host3"]}!

ismaster(from primary)

Updating Topology Description

Current description +

ismaster response→ new description

Updating Topology Description

primaryhost1

secondaryhost2

secondaryhost3

Python Program

monitorthread

monitorthread

monitorthread

MongoClient

{secondary: true, hosts: ["host1", "host2", "host3"]}

"new info!"Your

Code?

✓

client.db.collection.insert({'my': 'document'})!

client = MongoClient(! 'mongodb://host1,host2/?replicaSet=my-rs')!

primaryhost1

secondaryhost2

secondaryhost3

Python Program

monitorthread

monitorthread

monitorthread

MongoClient

{ismaster: true, hosts: ["host1", "host2", "host3"]}

{secondary: true, hosts: ["host1", "host2", "host3"]}

YourCode

insert

ok

✓

✓

"new info!"

insert

ok

✓

Steady State

primaryhost1

secondaryhost2

secondaryhost3

monitorthread

monitorthread

monitorthread

MongoClient

YourCode

insert

ok

insert

ok

✓

✓

✓

Steady State

Crisis

primaryhost1

secondaryhost2

secondaryhost3

monitorthread

monitorthread

monitorthread

MongoClient

YourCode

insert

ok

insert

ok

✓

✓

✓

primaryhost1

secondaryhost2

secondaryhost3

monitorthread

monitorthread

monitorthread

YourCode

insert

ConnectionFailure!

insert

✓

✓

✓

MongoClient

?

Crisis

primaryhost1

secondaryhost2

secondaryhost3

monitorthread

monitorthread

monitorthread

YourCode

insert

✓ ✓

MongoClient

"Wake up!"

Crisis

?

Resolution

primaryhost1

secondaryhost2

secondaryhost3

monitorthread

monitorthread

monitorthread

YourCode

✓

insert

"Wake up!" (every half second)

MongoClient

primaryhost2

✓

insert

ok

ok

Resolution

?

So What?

client = MongoClient(! 'mongodb://host1,host2/?replicaSet=my-rs')!!try:! client.admin.command('ismaster')!except pymongo.errors.ConnectionFailure as e:! print("Can't connect: %s" % e)!

Non-Blocking Client Construction

#1

Error Handling

#2

def find_my_document():! try:! return collection.find_one(query)! except ConnectionFailure:! logging.exception("finding document")! return collection.find_one(query)!

Error Handling

#2

def insert_doc():! doc = {'_id': ObjectId(), 'a': 1}! try:! collection.insert(doc)! except ConnectionFailure:! logging.exception("inserting document")! # Try again.! collection.insert(doc)! except DuplicateKeyError:! # Previous try actually succeeded.! pass!

#2

Error Handling

collection.update(! {'_id': 1},! {'$inc': {'counter': 1}})!

#2

Only idempotent operations can

be retried

Error Handling

Knobs

#3

•connectTimeoutMS

• serverSelectionTimeoutMS

ConnectionFailure!

monitorthread

monitorthread

monitorthread

primaryhost1

secondaryhost2

secondaryhost3

YourCode

insert insertMongoClient

"Wake up!"

wait for connectTimeoutMS

wait for serverSelectionTimeoutMS

(30 seconds)

bit.ly/server-discovery

Further Study: Server Discovery

& Monitoring Spec