mongodb as a data warehouse: time series and device history data (medtronic)
DESCRIPTION
TRANSCRIPT
MongoDB at Medtronic
Jeff LemmermanMatt Chimento
1
Medtronic Energy and Component Center
*Disclaimer: We are not currently managing any Device Patient Data in MongoDB
| MDT Confidential2
Agenda
• Intro• Challenge• Approach• Opportunities• Takeaways
3
Medtronic Energy and Component Center• MECC est. 1976
• Manufacture components for devices
• Census – 1200 Employees
• Plant Size – 190,000 Square Feet40,000 Manufacturing15,000 R&D Labs38,000 Office97,000 Common, Support, Warehouse
| MDT Confidential4
Every 3 Seconds…
Where does our data come from?
6
Where stored?
Big Data: Volume, Variety, Velocity
| MDT Confidential7
Without MongoDB
| MDT Confidential8
Data Management Vision
| MDT Confidential9
Reporting and Analytics VarietyOperations Dashboards SPC Charts
Component Summary Time Series
| MDT Confidential10
Component Dataset In Excel
11
How Is Data Retrieved?
12
Loading Data Into Central Repository
13
Component Dataset in RDBMS
14
Ideal Future State
Medtronic Confidential
15
Loading Components Into MongoDB
16
Ideal Future State
Medtronic Confidential
| MDT Confidential17
| MDT Confidential18
Simple, Fast Queries – Complete History
19
Medtronic Confidential
Component Time Series Data
20
Collect and Store Raw Time Series Data
Medtronic Confidential
Our experience with MongoDB
• Consulting/Training has been excellent• Support agreement has been under-utilized
– Emails for security updates etc. are prompt– Release cycle is frequent
• MongoDB Monitoring Service– Potential concerns storing db stats externally– MMS can now be hosted locally
• MongoDB Certification now available– Udacity course, “Data Wrangling with MongoDB”
Gaps
• Enterprise acceptance of “new” approach• Integration with off-the-shelf reporting and
analytics tools• User interface for managing MongoDB cluster• Developer familiarity with JSON and MongoDB• 21 CFR Part 11 Compliance – Audit Tracing
Thank You
Questions?
23
| MDT Confidential24
Key Features• Data stored as documents (JSON-like BSON)
– Flexible-schema– In schema design, think about optimizing for read vs. storage
• Full CRUD support (Create, Read, Update, Delete)– Atomic in-place updates– Ad-hoc queries: Equality, RegEx, Ranges, Geospatial
• Secondary indexes• Replication – redundancy, failover• Sharding – partitioning for read/write scalability• Terminology
– Collection = Table– Index = Index– Document = Row– Column = Field– Joining = Embedding & Linking
26
Creating Components
• .insert() will always try to create new document• .save() if _id already exists will update • If document doesn’t have _id field it is added
27
Reading Components
28
Reading ComponentsReturns Null
29
Updating Components
• $set keyword used for partial updates• Without $set keyword entire document is replaced• {multi : true} to update multiple documents
30
Deleting ComponentsWorks like .find()
Drops collection
Drops database
31
Creating Components – Add()
32
Reading Components
33
Updating Components – Save()
Save sends entire document back to server
34
Updating Components – Update()
Update only sends changes
35
Deleting Components
Needed to add reference to Repo class
36
Automapping
37