automated testing with offscale and mongodb
TRANSCRIPT
Automated testing with
MongoDB and OffScale
Get your database under control
Omer GertelCTO @ OffScale
We Help Companies…
Iterate Faster &Release Better Products
By…
Shortening product cycles with DB management
automation
Production
Dev
QA
Integration
Copy dataWrite codeWrite unit tests
Clone SchemaCopy dataTest the application
Clone schemaCopy dataRun integration
Copy dataWrite codeWrite unit tests
Clone SchemaCopy dataTest the application
Clone schemaCopy dataRun integration
DataGroveGit for databases
Production
Dev
QA
Staging
For developers, by developers
$ datagrove tag -n=new_tag
$ datagrove list
$ datagrove load -n=old_tag
$ datagrove unload
• Works automagically• Easy to integrate with existing tools• Your personal database time machine for FREE
Automated Testing
• Integration tests
• Build servers and continuous integration
• Migrations
–Some MongoDB specific issues
What happens if a test breaks?
• Does it affect other tests?
• Does it stop the build cycle?
• How do you discover what went wrong?
• Is the bug related to the data?
• How can you reproduce the bug?
• What happens when you push a fix?
Automated Tests
• For each test:
– Load dataset
–Run test
– If test fails:
• Tag dataset at end of run
• Migrate ALL the objects
– Test before you run
• Support older structures in code
– Test code against old objects
– Have datasets ready tagged by version
Schema Migrations
Object Structure & Index Changes
Existing MongoDB SupportFor single node environments
$ datagrove load -n=tag_name
Create virtual mount point: /media/offscale(1-10 sec)
mongod --dpath= /media/offscale/media/offscale
Mongod
path-to-old-datapath-to-store-
changes
1
2
(optional)
Existing MongoDB SupportFor single node environments
$ datagrove tag -n=tag_name
Tag point in time in virtual drive(1-10 sec)
{fsync:1,lock:1} (using Java driver)
/media/offscale
Mongod
path-to-old-datapath-to-store-
changes
2
1
(optional)
db.$cmd.sys.unlock.findOne()3
Replica Set
Primary
Slave
Slave
Replica Set
Primary
Slave
Slave
Upcoming MongoDB Cluster SupportConceptual architecture for taking a snapshot
Replica Set
Primary
Slave
Slave
One node managed under OffScale, per replica set, allows for backup, recovery and creating new nodes
Configuration cluster
Cfg 1
Cfg 2
Cfg 3
One or more nodes managed by OffScale for backup and recovery of configuration servers
$ datagrove tag-n=tag_name
2Issue datagrove tag for configuration server
1Pause shard balancing
4Resume shard balancing
3 Issue datagrove tag, one shard at a time
• Data is a first class citizen in your environment
• Testing is important to keep your code working
• Iterate faster & release better products
Automated testing with OffScale & MongoDB