taken some of the hype out of big data again - medtech pharma, nürnberg july 2014
DESCRIPTION
I was invitted to redo the talk about Big Data i did in Berlin earlier this year - slides also here. Slides are similar but updated to reflect my new company and some slides are new. EnjoyTRANSCRIPT
![Page 1: Taken some of the hype out of Big Data again - Medtech Pharma, Nürnberg july 2014](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54c675404a79593d1c8b459d/html5/thumbnails/1.jpg)
MedTech PharmaNürnberg 2014
Taking (some of) the mystery out of Big Data
![Page 3: Taken some of the hype out of Big Data again - Medtech Pharma, Nürnberg july 2014](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54c675404a79593d1c8b459d/html5/thumbnails/3.jpg)
Contact
Claus Stie Kallesøe
Founder, CEO
+45 30 14 15 36
![Page 4: Taken some of the hype out of Big Data again - Medtech Pharma, Nürnberg july 2014](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54c675404a79593d1c8b459d/html5/thumbnails/4.jpg)
Introduction
![Page 5: Taken some of the hype out of Big Data again - Medtech Pharma, Nürnberg july 2014](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54c675404a79593d1c8b459d/html5/thumbnails/5.jpg)
Big Data –Either VERY large datasets AND/OR other complexities
Characteristics of big data
Source: IBM methodology
![Page 6: Taken some of the hype out of Big Data again - Medtech Pharma, Nürnberg july 2014](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54c675404a79593d1c8b459d/html5/thumbnails/6.jpg)
A couple of words about scale• 100’s of Megabytes
• This should not be a problem. Can be handled with Matlab, R, Ruby
• 100/500 Gigabytes – 1Terabyte• 2 Terabyte harddrives can be bought in the local shop for €100
• Connect it to your laptop and install postgresql or a no-sql database on it
• > 5 Terabytes• Now you might have a size issue
Inspired by: http://www.chrisstucchio.com/blog/2013/hadoop_hatred.html
![Page 7: Taken some of the hype out of Big Data again - Medtech Pharma, Nürnberg july 2014](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54c675404a79593d1c8b459d/html5/thumbnails/7.jpg)
Big Data - “Definition”
"Big Data is high volume, high velocity, and/or high variety information assets that require new forms of processing to enable enhanced decision making, insight discovery and process optimization."
![Page 8: Taken some of the hype out of Big Data again - Medtech Pharma, Nürnberg july 2014](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54c675404a79593d1c8b459d/html5/thumbnails/8.jpg)
Cool, but remember where we are!Gartner Hype Cycle 2013
![Page 9: Taken some of the hype out of Big Data again - Medtech Pharma, Nürnberg july 2014](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54c675404a79593d1c8b459d/html5/thumbnails/9.jpg)
Big Data in Pharma R&D
![Page 10: Taken some of the hype out of Big Data again - Medtech Pharma, Nürnberg july 2014](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54c675404a79593d1c8b459d/html5/thumbnails/10.jpg)
What is Big Data in Pharma R&D?• Many ideas/possibilities across Pharma R&D and market
access• But many of them are likley NOT “real” Big Data problems!
• Are they relevant and can they bring insights?• Yes, very much so
• Should we than find a way to handle them?• Absolutely
![Page 11: Taken some of the hype out of Big Data again - Medtech Pharma, Nürnberg july 2014](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54c675404a79593d1c8b459d/html5/thumbnails/11.jpg)
Disclaimer
• I am a (web) tech geek• I have nothing against new technologies
• Like many other geeks I like it
• But do try to use the right tool for the right job
![Page 12: Taken some of the hype out of Big Data again - Medtech Pharma, Nürnberg july 2014](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54c675404a79593d1c8b459d/html5/thumbnails/12.jpg)
http://blog.mongohq.com/you-dont-have-big-data/
![Page 13: Taken some of the hype out of Big Data again - Medtech Pharma, Nürnberg july 2014](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54c675404a79593d1c8b459d/html5/thumbnails/13.jpg)
Another great tool - for some
Q: “Could you help me get to Nürnberg, pls?”A: “Yes, absolutely. Not a problem”
Q: “Ok, btw I want to try the Endeavour A: “...ahh why?”
Q: “Because I have read it’s great”A: “Yes, but the ICE….”
![Page 14: Taken some of the hype out of Big Data again - Medtech Pharma, Nürnberg july 2014](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54c675404a79593d1c8b459d/html5/thumbnails/14.jpg)
MapReduce explained in 41 wordsGoal: Count the number of books in the library.
Map: You count up shelf #1, I count up shelf #2.
(The more people we get, the faster this part goes. )
Reduce: We all get together and add up our individual counts.
http://www.chrisstucchio.com/blog/2011/mapreduce_explained.html
![Page 15: Taken some of the hype out of Big Data again - Medtech Pharma, Nürnberg july 2014](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54c675404a79593d1c8b459d/html5/thumbnails/15.jpg)
What is it then? Linked data?
![Page 16: Taken some of the hype out of Big Data again - Medtech Pharma, Nürnberg july 2014](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54c675404a79593d1c8b459d/html5/thumbnails/16.jpg)
Does it matter what it is?
No!
It’s data - and potential analytics (business) opportunities.
Size and complexity should drive the technology
![Page 17: Taken some of the hype out of Big Data again - Medtech Pharma, Nürnberg july 2014](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54c675404a79593d1c8b459d/html5/thumbnails/17.jpg)
TechnologiesCan we do anything on our own
![Page 18: Taken some of the hype out of Big Data again - Medtech Pharma, Nürnberg july 2014](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54c675404a79593d1c8b459d/html5/thumbnails/18.jpg)
For many people/companies”Big data technology” is a black box
”A lot of stuff”
And then the vendors go:If
{ box = magic or money}then
{ box = expensive}
![Page 19: Taken some of the hype out of Big Data again - Medtech Pharma, Nürnberg july 2014](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54c675404a79593d1c8b459d/html5/thumbnails/19.jpg)
Working within a communityA lot of tools available
From: ttp://people10.com/blog/ruby-on-rails-the-popular-platform-for-web-development/
![Page 20: Taken some of the hype out of Big Data again - Medtech Pharma, Nürnberg july 2014](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54c675404a79593d1c8b459d/html5/thumbnails/20.jpg)
New visualisations – easy and free
http://philogb.github.io/jit/demos.html
![Page 21: Taken some of the hype out of Big Data again - Medtech Pharma, Nürnberg july 2014](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54c675404a79593d1c8b459d/html5/thumbnails/21.jpg)
Automated calculations - can bring you far
Job submitted to asynccalculation server
![Page 22: Taken some of the hype out of Big Data again - Medtech Pharma, Nürnberg july 2014](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54c675404a79593d1c8b459d/html5/thumbnails/22.jpg)
https://circleci.com/
Also a lot of great tools to handle data
![Page 23: Taken some of the hype out of Big Data again - Medtech Pharma, Nürnberg july 2014](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54c675404a79593d1c8b459d/html5/thumbnails/23.jpg)
Elasticsearch text indexes
• Indexed research assay metadata=> Google like search to find the relevant assay
• Indexed sharepoint project workspaces=> Enable easy, fast cross project queries to find trends
![Page 24: Taken some of the hype out of Big Data again - Medtech Pharma, Nürnberg july 2014](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54c675404a79593d1c8b459d/html5/thumbnails/24.jpg)
Conclusion – Big data in Pharma R&D• Many opportunities across R&D and market access
• More data linking and data analytics than Big Data
• You can use freely available tools on ”normal” hardware
• No magic ”Under the hood” – it’s just data
![Page 25: Taken some of the hype out of Big Data again - Medtech Pharma, Nürnberg july 2014](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54c675404a79593d1c8b459d/html5/thumbnails/25.jpg)
BUT you still need to define the questions you
want to answer – before diving into technology!