how can we take advantage of cloud computing & big data?
DESCRIPTION
How can we take advantage of Cloud computing & big data?. Jordi Torres. Mobile Business & Apps Design master degree at UPC Talent School of Professional and Executive Development. Barcelona, February 2013TRANSCRIPT
HOW CAN WE TAKE ADVANTAGE OF CLOUD COMPUTING & BIG DATA?
Jordi TorresUPC-BSC, February - 2013
MOBILE BUSINESS & APPSDESIGN MASTER DEGREE
CONTENT
1. New Business models2. How do we get here?3. What is Cloud Computing?4. What is Big Data?5. An opportunity for you
YESTERDAY’S BUSINESSFo
to: D
esig
nedI
nBar
celo
na.c
om
TODAY’S BUSINESS
OR NOT?
IS MUSIC A BUSINESS?
BON JOVIEx
empl
ede
Jos
ep V
alor
(IE
SE)
, 20
11
The most downloadesThe best-selling
20101986
BON JOVIEx
empl
ede
Jos
ep V
alor
(IE
SE)
, 20
11
Sold: 28.000.000
BON JOVIEx
empl
ede
Jos
ep V
alor
(IE
SE)
, 20
11
Sold: 28.000.000 Sold: ?
BON JOVIEx
empl
ede
Jos
ep V
alor
(IE
SE)
, 20
11
Revenues: 30.000.000$Concerts tour 2010Revenues: 70.000.000$
BON JOVIEx
empl
ede
Jos
ep V
alor
(IE
SE)
, 20
11
Revenues: 30.000.000$Concerts tour 2010Revenues: 70.000.000$
The business model changed!
THE BUSINESS MODEL
from physical movement to the movement of information
Courtesy: D
ani Urgell –
CIS
CODesintermediation
music: Itunes, spotifybooks: Amazoneducation: e-learning UOC, MITtravels: eDreamsRetail: Zara. . .
The businessmodel changed!
WHERE DATA ARE STORED?
WHERE DATA ARE STORED?Sou
rce:
App
le.c
om.
201
1
INTO CLOUD COMPUTING!
CLOUD COMPUTING
It’s here to stay
CLOUD FORECAST
Source: http://softwarestrategiesblog.com/2012/01/17/roundup-of-cloud-computing-forecasts-and-market-estimates-2012/
Gartner is predicting that the IaaS market to grow by 47.8% through 2015.
IaaSAccording to IDC, by 2015, about 24% of all new business software purchases will be of service‐enabled software with SaaS delivery being 13.1% of worldwide software spending.
SaaS
CLOUD FORECAST
Workload processed in the Cloud
Source: CISCO
http://www.cisco.com/en/US/solutions/collateral/ns341/ns525/ns537/ns705/ns1175/Cloud_Index_White_Paper.pdf
CONTENT
1. New Business models2. How do we get here?3. What is Cloud Computing?4. What is Big Data?5. An opportunity for you
HOW DO WE GET HERE?
Pilot ACE, UK.1950
EVERY 15 YEARS …
1950 2012
That has redefined IT and industry in general
TIC
A major technological jump in IT
MAINFRAMES
1950 1980 201019951965 2025
Mainframes
PERSONAL COMPUTERS
1950 1980 201019951965 2025
Mainframes
Personal
Computers
INTERNET
1950 1980 201019951965 2025
Mainframes
Personal
Computers
Internet
INDUSTRIALIZATION OF IT
1950 1980 201019951965 2025
Mainframes
Personal
Computers
Internet
Huge Data Centers
HUGE DATA CENTERS
28.000 m2
Source: http://www.fa
cebo
ok.com
/med
ia/
set/?set=a.190
8426
2096
5185
.470
08.140
3752
8934
5252
40 Mw
HUGE DATA CENTERSFoto: G
oogle
HUGE DATA CENTERSFoto: G
oogle
> football pitch x 4
HUGE DATA CENTERSFo
to: c
orte
sía
BSC
Industrializaton of IT: has arrived!
Source: http://www.google.com/about/datacenters/gallery/images
Source: http://www.google.com/about/datacenters/gallery/images
Source: http://www.google.com/about/datacenters/gallery/images
CONTENT
1. New Business models2. How do we get here?3. What is Cloud Computing?4. What is Big Data?5. An opportunity for you
IT AS A SERVICE
Cloud Computing represents a majorchange in IT servicearchitecture, deliveryand consumption
Foto
: J.T
.
IT AS A SERVICE
Amazon Data Center
Foto: Tutorial SC2011 - Robert Grossman
(*) IaaS
IT AS A SERVICE
More flexible IT, which can adapt to businesses and where costs can be predicted
On-demand self-service
Rapid elasticity
Ubiquitous access
. . . .
Pay per use
EXAMPLE: WORKLOAD
SYSTEM CAPACITY
Demand
Time
Res
ourc
es
Capacity
RESOURCES NOT USED
Demand
Time
Res
ourc
es
CapacityResourcesnot used
WASTE OF RESOURCES
Demand
Time
Res
ourc
es
Capacity€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€ €€€€€€€€€€€€ €€€€€€€€€ €€€€€€€€€€ €€€€€€€ €€€€€€€€ €€€€€ €€€€€€ €€€€ €€€€ €€
€€€ €
CUSTOMER BEHAVIOR
Res
ourc
es
Demand
Capacity
Time (days)
1 2 3
CUSTOMER BEHAVIOR
Res
ourc
es
Demand
Capacity
Time (days)
1 2 3
CLOUD COMPUTING ELASTICITY
Demand
Capacity
Time
Res
ourc
es
AGILITY FOR SUPPLYINGvs in‐house data center
Foto
s: c
orte
sía
BSC
TYPES OF CLOUDS
Software as a Service (SaaS)
Plataforma as a Service (PaaS)
Infraestructura as a Service (IaaS)
TYPES OF CLOUDS
models of service delivery
Server
Apps
VM
Frameworks
PaaS SaaS
Server
Apps
VM
Frameworks
IaaS
Server
Apps
VM
Frameworks
Tradicional
Server
Apps
VM
Frameworks
INFRAESTRUCTURE AS A SERVICE
Idea : Tutorial SC2011 - Robert Grossman
1 computer in a rack for 120 hours 120 computers in three
racks for 1 hour
Ex IaaS: Amazon Web Services
To create an AWS account go to http://aws.amazon.comTo launch an instance go to
GET STARTED WITH EC2
Follow the Wizard
CONNECT TO INSTANCE
e.g. SSH
TERMINATE INSTANCE
As soon as your instance starts to boot, you're billed for each hour or partial hour that you keep the instance running (even if the instance is idle). When you've decided that you no longer need the instance, you can terminate it.
CLOUD COMPUTING INHIBITORS
Security Privacy Lack of Standards
Data Integrity Regulatory Data
Recovery
Control VendorMaturity ...
CLOUD COMPUTING TODAY
CLOUD
Enterprise resourcesData+ Programs + Hardware
Sep 2011User
electronic devices with identification/computing/storage/communication capabilities
CLOUD COMPUTING TODAY
PRIVATE CLOUDPUBLIC CLOUD
Enterprise resourcesData+ Programs + Hardware
User
electronic devices with identification/computing/storage/communication capabilities
HYBRID CLOUD
Workload
HYBRID CLOUD
capacity
Workload
HYBRID CLOUD
capacity
HYBRID CLOUD
External Provider
EVERYTHING AS A SERVICE
IaaSPaaSSaaS
XaaSDRaaS
SUaaSBDaaS…
CLOUD CAN DRIVE BUSINESS INNOVATIONFo
to: D
esig
nedI
nBar
celo
na.c
om
Taking cost out of company Improve business processes and operational effectiveness Using the cloud to enable new business models
SUMMARY
Sou
rce:
Sec
urity
Gui
danc
e fo
r Critic
al A
reas
of
Focu
s in
Clo
ud
Com
putin
g ,
Apr
il 20
09.
Clo
ud S
ecur
ity A
llian
ce.
Hundreds ofcompanieson stage
CONTENT
1. New Business models2. How do we get here?3. What is Cloud Computing?4. What is Big Data?5. An opportunity for you
TODAY
EVERYBODY CONNECTED
4th year ESO secondary school:100% of students on Facbook
http
://w
ww.
sear
chen
gine
jour
nal.c
om/th
e-gr
owth
-of-s
ocia
l-med
ia-a
n-in
fogr
aphi
c/32
788
A MOBILE FUTURE
http://money.cnn.com/2011/02/09/technology/smartphones_eclipse_pcs/index.htm
INTERNET OF THINGS
Not only will we have mobile devices, but also
http://www.urbiotica.com
INTERNET OF THINGShttp://dow
nload.intel.com/new
sroom/kits/em
bedded/pdfs/ECG
_WhitePaper.pdf
OPEN DATA
BIG DATA?Do you need a definition? is data that becomes large
enough that it cannot be processed using conventional methods.
enough for you? :-)
Deluge of data created dailysocial networks, mobile phones, sensors, science, …
Source:http://www.datacenterknowledge.com/archives/2011/06/28/digital-universe-to-add-1-8-zettabytes-in-2011/?utm-source=feedburner&utm-medium=feed&utm-campaign=Feed:+DataCenterKnowledge+%28Data
“CHULETA”
1 Gigabyte (GB) = 1.000.000.000 byte1 Terabyte (TB) = 1.000 Gigabyte (GB)1 Petabyte (PB) = 1.000.000 Gigabyte (GB)1 Exabyte (EB) = 1.000.000.000 Gigabyte (GB)1 Zettabyte (ZB) = 1.000.000.000.000 (GB)
DEFINING BIG DATA
A more complete definition of Big Data is one that combines three attributes of data:
Source: IBM Book –
Understanding
Big Data
DEFINING BIG DATA
Volume: Large Volumes of data Terabytes, Petabytes, … Data that cannot be stored in conventional RDBMS
Variety Source data is diverse – Web Logs, Application Logs,
Machine generated data, Social network data, etc. Doesn't fall into neat relational structures – Unstructured,
Semi-structured Velocity Streaming data, Complex Event Processing data Velocity of incoming data and Speed of responding to it
WHY IS BIG DATA IMPORTANT
60%Potential increase in retailers
operating margins possible with big data
(*) Source: Big Data: The next frontier for innovation, competition and productivity – Mckinsey Global Institute, July 2011
40% projected growth in global data generated per year vs
5% projected growth in global IT spending
WHY IS BIG DATA IMPORTANT
Data is more important than ever, but the exponential growth of data has overwhelmed most company's ability to manage (and monetize it).
Big Data Challenges Processing such large data sets Storing and Managing Analyzing …
assume 100MB/sec
Storage affordable
assume 100MB/sec
But scanning disks…
assume 100MB/sec more than 5 hours
But scanning disks…
assume 20.000 disks: scanning 2 TB takes 1 second
Source: http://www.google.com/about/datacenters/gallery/images/_2000/IDI_018.jpg
SOLUTION: MASSIVE PARALLELISM
assume 20.000 disks: scanning 2 TB takes 1 second
Source: http://www.google.com/about/datacenters/gallery/images/_2000/IDI_018.jpg
Rethinking data processing is required: MapReduce, …
DATA PROCESSING CHALLENGES
DATA STORAGE CHALLENGES
New Storage technologies are required
RAM vs HHD
Present solutions:
Research:
HHD 100 cheaper than RAMBut 1000 times slower
Solid- state drive (SSD)Not volatile
80
Storage Class Memory (SCM)
Source: gigaom.com/cloud/big‐data‐and‐nosql‐march‐to‐the‐enterprise/
Relational DB can’t support everything:
NoSQL, …
DATA MANAGEMENT CHALLENGES
Solution: the so called “NoSQL systems”
BASE property is sometimes attributed to them(Basically Available, Soft state, Eventual consistency)
Example eventual consistency: Facebook!
DATA MANAGEMENT
82
Data deluge, is it enough?
“Data = Information”?
prediction using data models
The information is non actionable
knowledge
OBTAINING VALUE FROM DATA
86
4
Knowledge
Information
Data
+Volum
e
-+
-
Valu
e
For this we have Data Miningtechniques
• Association• Classification• Clustering• Prediction• ...
The majority of algorithms functionwell in thousands of registers, however at the moment they are impractical for thousands of milions.
CONTENT
1. New Business models2. How do we get here?3. What is Cloud Computing?4. What is Big Data?5. An opportunity for you
AN OPPORTUNITY
AN OPPORTUNITY
1950 1980 201019951965 2025
Ap
plica
tio
ns
Thousandsof applications
Tens of thousandsof applications
Hundredsof applications
Hundreds of thousands
of applications
IT applications and services over the years
AN OPPORTUNITY
1950 1980 201019951965 2025
Ap
plica
tio
ns
Thousandsof applications
Tens of thousandsof applications
Hundredsof applications
Hundreds of thousands
of applications
we'll see millions of new applications and services built on top of these disruptive technologies
Millions ofapplications
AN OPPORTUNITY
1950 1980 201019951965
Millionsof users
Hundreds of Millionsof users
?
Thousandsof users
2025
Use
rs IT users growth
AN OPPORTUNITY
1950 1980 201019951965
Millionsof users
Hundreds of Millionsof users
Billions of users
Trillions of “things”
Thousandsof users
2025
Use
rs IT users growth
thanks to these millions of applications we'll see billions of users and trillions of “things” connected
BIG DATA TALENT
BIG DATA TALENT
CONCLUSION
We have toprepare our businesses for
the explosion of the cloudin a world of mobile technology
and social networks, dominated by the
boom of available information and all in a sustainable way.
www.smartcityexpo.com www.bsc.es/eBusiness
Autonomic Systems and e‐Business Platforms research line at BSC/UPC
Thank you for yourattention
Questions?
www.JordiTorres.eu
@JordiTorresBCNBarcelona13-15 November 2012
www.smartcityexpo.com