introduction to big data - github pages...big data – business perspective it is a new business...
TRANSCRIPT
![Page 1: Introduction to Big Data - GitHub Pages...Big Data – Business perspective It is a new business model • People pay with data, e.g. Facebook, Google, Twitter: • use service, give](https://reader035.vdocuments.mx/reader035/viewer/2022062414/5ec6ea8f89b9675d89519171/html5/thumbnails/1.jpg)
IntroductiontoBigData
1
![Page 2: Introduction to Big Data - GitHub Pages...Big Data – Business perspective It is a new business model • People pay with data, e.g. Facebook, Google, Twitter: • use service, give](https://reader035.vdocuments.mx/reader035/viewer/2022062414/5ec6ea8f89b9675d89519171/html5/thumbnails/2.jpg)
BigData– Philosophicalperspective
Whatismorevaluable,ifyouhadtopickone?• experienceorintelligence?
• Traditional(computer)science:logic![intelligence]• understand theproblem, buildmodel/algorithm• answerquestion from implementationofmodel
• Newscience:statistics![experience]• collectdata• answerquestion fromdata(whatdidothersdo?)
2
![Page 3: Introduction to Big Data - GitHub Pages...Big Data – Business perspective It is a new business model • People pay with data, e.g. Facebook, Google, Twitter: • use service, give](https://reader035.vdocuments.mx/reader035/viewer/2022062414/5ec6ea8f89b9675d89519171/html5/thumbnails/3.jpg)
Questionsand(some)answers
• Findaspouse?• ShouldAdambiteintotheapple?• 1+1?• Cureforcancer?• Howtotreatacough?• ShouldIgiveDonaldaloan?• Premiumforfireinsurance?• Whenshouldmysoncomehome?• WhichbookshouldIreadnext?• TranslatefromGermantoEnglish.
3
![Page 4: Introduction to Big Data - GitHub Pages...Big Data – Business perspective It is a new business model • People pay with data, e.g. Facebook, Google, Twitter: • use service, give](https://reader035.vdocuments.mx/reader035/viewer/2022062414/5ec6ea8f89b9675d89519171/html5/thumbnails/4.jpg)
Questionsand(some)answers
• Findaspouse?Idonotwanttoknow!• ShouldAdambiteintotheapple?Ifyoubelieve...• 1+1?Definition• Cureforcancer?Idonotknow.Maybe.• Howtotreatacough?Yes.(GoogleInsight)• ShouldIgiveDonaldaloan?Yes.(e.g.,Schufa)• Premiumforfireinsurance?Yes.(e.g., … )• Whenshouldmysoncomehome?No!But...• WhichbookshouldIreadnext?Yes.(Amazon)• TranslatefromGermantoEnglish.Yes.(GoogleTransl.)
4
![Page 5: Introduction to Big Data - GitHub Pages...Big Data – Business perspective It is a new business model • People pay with data, e.g. Facebook, Google, Twitter: • use service, give](https://reader035.vdocuments.mx/reader035/viewer/2022062414/5ec6ea8f89b9675d89519171/html5/thumbnails/5.jpg)
DataScience
• Newapproachtodoscience• Step1:Collectdata• Step2:GenerateHypotheses• Step3:ValidateHypotheses• Step4:(Goto Step1or2)
• Whyisthisagoodapproach?• Automated:no thinking, lesserror
• Whyisthisabadapproach?• Howtodebugwithoutaground truth?
• Moregenerally,interdisciplinaryemergingfield(seeimages)
5
![Page 6: Introduction to Big Data - GitHub Pages...Big Data – Business perspective It is a new business model • People pay with data, e.g. Facebook, Google, Twitter: • use service, give](https://reader035.vdocuments.mx/reader035/viewer/2022062414/5ec6ea8f89b9675d89519171/html5/thumbnails/6.jpg)
“Big”data- Pros&Cons
• Pros• tolerateerrors• discoverthelongtailandcornercases– machinelearningworksmuchbetter
• Cons• Moredata,moreerror(e.g.,semanticheterogeneity)• Withenoughdatayoucanproveanything• stillneedhumans toaskrightquestions
6
![Page 7: Introduction to Big Data - GitHub Pages...Big Data – Business perspective It is a new business model • People pay with data, e.g. Facebook, Google, Twitter: • use service, give](https://reader035.vdocuments.mx/reader035/viewer/2022062414/5ec6ea8f89b9675d89519171/html5/thumbnails/7.jpg)
BigDataSuccessStory
• GoogleTranslate• Youcollectsnippetsoftranslations• Youmatchsentencestosnippets• Youcontinuouslydebugyour system
• Whydoesitwork?• TherearetonsofsnippetsontheWeb• Thereisaground truththathelps todebugsystem
7
GoogleTranslateisbasedonsomethingcalled"statisticalmachinetranslation".Thismeansthattheygatherasmuchtextastheycanfind thatseemstobeparallelbetweentwolanguages, andthentheycrunchtheirdatatofind thelikelihood thatsomethinginLanguageAcorrespondstosomethinginLanguageB.Thismethodworkstosomeextentforlanguage pairswherealotofmore-or-lessparallel dataisavailable, forexampleEnglish-Spanish. […](quora.com)
![Page 8: Introduction to Big Data - GitHub Pages...Big Data – Business perspective It is a new business model • People pay with data, e.g. Facebook, Google, Twitter: • use service, give](https://reader035.vdocuments.mx/reader035/viewer/2022062414/5ec6ea8f89b9675d89519171/html5/thumbnails/8.jpg)
BigData– Businessperspective
Itisanewbusinessmodel
• Peoplepaywithdata,e.g.Facebook,Google,Twitter:• useservice,givedata• Googlesellsyourdatatoadvertisers• youpayadvertisersindirectly
• 23andMe,Amazon:• payservice+givedata• sellsdataand• usesdatatoimproveservice
8
Bigdata:Thenextfrontierforinnovation, competition,andproductivity,McKinseyGlobalInstitute,June2011
![Page 9: Introduction to Big Data - GitHub Pages...Big Data – Business perspective It is a new business model • People pay with data, e.g. Facebook, Google, Twitter: • use service, give](https://reader035.vdocuments.mx/reader035/viewer/2022062414/5ec6ea8f89b9675d89519171/html5/thumbnails/9.jpg)
BigData– Technicalperspective
• Youcollectalldata• themorethebetter->statisticalrelevance,• keepingallischeaperthandecidingwhattokeep
• Youdecideindependentlywhattodowithdata• runexperimentsondatawhenquestionarises
• Hugedifference totraditionalinformationsystems• Designupfront whatdatatokeepandwhy!!!(e.g.,waterfallmodelofsoftwareengineering!)
9
![Page 10: Introduction to Big Data - GitHub Pages...Big Data – Business perspective It is a new business model • People pay with data, e.g. Facebook, Google, Twitter: • use service, give](https://reader035.vdocuments.mx/reader035/viewer/2022062414/5ec6ea8f89b9675d89519171/html5/thumbnails/10.jpg)
Consequences
• Volume:dataatrest• itisgoingtobealotofdata
• Velocity(Speed):datainmotion• itisgoingtoarrivefast
• Variety(Diversity):datainmanyformats• Differentshapes(e.g.,differentversions,differentsources)
• Veracity:dataindoubt• doyouknowwhatyouhave?
10
![Page 11: Introduction to Big Data - GitHub Pages...Big Data – Business perspective It is a new business model • People pay with data, e.g. Facebook, Google, Twitter: • use service, give](https://reader035.vdocuments.mx/reader035/viewer/2022062414/5ec6ea8f89b9675d89519171/html5/thumbnails/11.jpg)
11
![Page 12: Introduction to Big Data - GitHub Pages...Big Data – Business perspective It is a new business model • People pay with data, e.g. Facebook, Google, Twitter: • use service, give](https://reader035.vdocuments.mx/reader035/viewer/2022062414/5ec6ea8f89b9675d89519171/html5/thumbnails/12.jpg)
12
![Page 13: Introduction to Big Data - GitHub Pages...Big Data – Business perspective It is a new business model • People pay with data, e.g. Facebook, Google, Twitter: • use service, give](https://reader035.vdocuments.mx/reader035/viewer/2022062414/5ec6ea8f89b9675d89519171/html5/thumbnails/13.jpg)
13
![Page 14: Introduction to Big Data - GitHub Pages...Big Data – Business perspective It is a new business model • People pay with data, e.g. Facebook, Google, Twitter: • use service, give](https://reader035.vdocuments.mx/reader035/viewer/2022062414/5ec6ea8f89b9675d89519171/html5/thumbnails/14.jpg)
14