Implementing Async Networking in MongoDB
Samantha Ritter MongoDB Engineer
Why?
mongosapp
shards
connect auth send recv done
mongosapp
shards
connect auth send recv done
Execution engine
Standalone ASIOhttp://think-async.com/
connect auth send recv done
{work queue
B: send
B: recv
D: auth
A: send
F: done
C++11 lambdas
Constructs a closure: an unnamed function object capable of capturing variables in scope.
auto lambda = [capture list](params) { // body };
…
lambda(); // runs body
send
// the “send” task void send_task(NetworkOp* op) { // pass a lambda to async_send async_send(op->socket, op->command, [op](error_code err) { if (err) { return done(op); } receive_task(op); }); }
connect
auth
recv
send
done
mongos
Network Errors
connect
auth
recv
send
done
mongos
XNetworkError!
Network errors are fine: they are on the primary path of execution
The primary path controlsoperation lifetime
// the “send” task void send_task(NetworkOp* op) { // pass a lambda to async_send async_send(op->socket, op->command, [op](error_code err) { if (err) { return done(op); } receive_task(op); }); }
{work queue
B: send
B: recv
D: auth
A: send
F: done
B: recv
XNetworkError!
clean up B
Cancellations
connect
auth
recv
send
done
mongos
recv !Warning!
cancel job
// the “send” task void send_task(NetworkOp* op) { // pass a lambda to async_send async_send(op->socket, op->command, [op](error_code err) { if (err) { return done(op); } receive_task(op); }); }
Cancellations are NOT fine: they are on the secondary path of execution
On the secondary path we can’t make assumptions about lifetime
Only the primary path can end an operation
Rule of ownership:
// Basic “network operation” class class NetworkOp { bool cancelled; };
// Primary path if (op->cancelled) { done(op); }
// Secondary path cancel(NetworkOp *op) { op->cancelled = true; }
// the “send” task void send_task(NetworkOp* op) { // pass a lambda to async_send async_send(op->socket, op->command, [op](error_code err) { if (err || op->cancelled) return done(op); receive_task(op); }); }
connect
auth
recv
send
done
mongos
Please cancel yourself
Ok!
connect
auth
send
mongos
recvdonePoof!
Please cancel your…
?!@!&?
// Secondary path cancel(NetworkOp *op) { // op could be a null pointer! op->cancelled = true; }
Operation access is protected
Rule of cooperation:
// “network operation” class class NetworkOp { bool cancelled; };
// "access control" object class SafeOp { mutex lock; NetworkOp* op; };
shared_ptr
// Primary path done(shared_ptr<SafeOp> safe) { // lock before cleanup safe->lock.lock(); // cleanup safe->op = nullptr;
safe->lock.unlock(); }
// Secondary path cancel(shared_ptr<SafeOp> safe) { // once we lock, can’t change under us safe->lock.lock(); if (safe->op) { safe->op->cancelled = true; } safe->lock.unlock(); }
connect
auth
send
mongos
recvdonePoof!
Please cancel your…
JK!!
Why?
Threading is better
Engineering Process
1. Iterate!
2. Use language features where possible
3. Use external libraries where appropriate
@SamWhoCodes
mongodb.com/careers
Thanks!