big data: people, technologies, jobs and hong kong (15 oct 2013)
DESCRIPTION
This talk is an expanded version of my previous upload from August of this year. I delivered this presentation to the Chinese University of Hong Kong (CUHK)'s Department of Statistics. In it I provided an overview of big data, the techs and skills used in the space, and some engaging stories in the space. I also focused heavily on Hong Kong with details on Open Data projects ongoing in the city right now.TRANSCRIPT
definitions
technologies
timing and hype
peopleHong Kong
future
the common definition
velocityvolume
varietyveracity
the subjective definition
the easy definition
the real challenge:
what to do with all these data?
components: creating value from data
hardware software
people
servers
storage
networks
appliances
platforms
traditional DBMS
columnar DBs
NoSQL
Hadoop
platform architects
stats. people
comp. scientists
traditional IT
visualization
hot components
hardware software
people
servers
storage
networks
appliances
platforms
traditional DBMS
columnar DBs
NoSQL
Hadoop
platform architects
stats. people
comp. scientists
traditional IT
visualization
applications
a few examples
privacy
• what is your expectation of your data’s lifespan?
• what is the relationship between privacy and intellectual property protection?
• do you know your digital exhaust?• should you be compensated for helping
Google earn another billion dollars?
why now?
• cheap storage• unbounded compute• data accessibility and world datafication• internet scale: Yahoo! and Google– offspring of Hadoop
visi
bilit
y/ex
pect
ation
s
time
trigger inflated expectations
disillusionment
enlightenment
productivity
adapted from Gartner hype cycle
visi
bilit
y/ex
pect
ation
s
time
trigger inflated expectations
disillusionment
enlightenment
productivity
adapted from Gartner hype cyclebig data: where are we
today?
this will be caused bya lack of statistics knowledge
Asia
busi
ness
inte
rest
time
Global
USA
Asia
Hong Kong
http://www.infoincog.com/big-data-in-asia-pacific/
enterprise IT adoption
Hong Kong
want to get involved?
• decision tree:– individual?• learn: join G+ group, ask Scott for reading
recommendations• work: Scott knows some recruiters and hiring
businesses
– research ideas?• meet with CS students about HK data research platform
– social or civic engagement?• come to regular ODHK meetings
changing future
• borderless big data will increasingly become invasive. how will regional laws keep up?
• “free” services will shift money from many small contributors to a few large businesses.
• data must be properly valued which requires a market.
• those with computer science and statistics skills will be very well paid for may years.
want more?
Google+: Hong Kong Big Datahttp://www.infoincog.com/
all content by Scott Brady Drummonds – [email protected]