© 2013 VMware Inc. All rights reserved

Interesting Times: Paradigm Shifts in IT

Teodor Tonchev, Staff Engineer, VMware

2

Definition

Paradigm Shift:

A change from one way of thinking to another

3

Virtualization

4

Create Virtual Machines

Run different Operating

Systems and Applications

within these VMs

Clone/Move/Backup VMs

Snapshot/Restore VMs

VMware Workstation™

Virtual Machines

Chang

ed

script

and

slide

OS

APP

OS

APP

OS

APP

OS

5

Hypervisor – an Operating

System for running VMs

Many servers now come

with ROM-based hypervisor

VMware ESX™

Server Virtualization

Chang

ed

script

and

slide

OS

APP

OS

APP

OS

APP

6

OS

APP

vMotion allows running VMs

to be moved among servers

seamlessly

Clients connected to the

VMs do not observe any

changes VMware vSphere™

VMware vMotion

Chang

ed

script

and

slide

7

OS

APP

OS

APP

OS

APP

DPM consolidates

workloads onto fewer

servers when the cluster

needs fewer resources

Places unneeded

servers in standby

mode

Brings servers back

online as workload

needs increase

OS

APP

OS

APP

OS

APP

OS

APP

OS

APP

OS

APP

VMware vSphere™

VMware DPM (Distributed Power Mgmt)

DPM powers off

server when

requirements

are lower

DPM brings

servers back

online when

load increases

Chang

ed

script

and

slide

8

Distributed Resource Management (DRS)

Shrink and grow of

applications based on

demand and priority

Dynamic and responsive

load balancing VMware vSphere™

OS

APP

OS

APP

OS

APP

OS

APP

OS

APP

OS

APP

9

Current Virtualization Use

2012 Survey: 51% of x86 Servers Now Virtualized

Nearly 92 percent of all enterprises use at least some form of

virtualization.

Majority of servers on the market support integrated hypervisor

10

Cloud

11

What is Cloud Computing?

Where at home do you keep your Electric Power Station?

Cloud: Computing resources provided on demand, as a service,

remotely (Internet or Intranet)

• Utility-Based Computing is another term for cloud computing

12

Cloud Categories

13

Starting a software service is much easier now

Application Server

OS

App Libs

DB

With IaaS there is no need to buy hardware.

With PaaS one can focus only on developing the App.

14

Application Development

15

Pets vs. Cattle

16

Multicore Future

Chips are reaching the physical limits

Moore’s law still holds, but due to increasing the number of cores

Now: tens, hundreds of cores

Soon: 1000s of cores

2020s: Millions of cores

17

Parallel programming

Approach 1: Shared Memory

• Threads

• Locks/Mutexes

DOES NOT SCALE. COMPLEX (e.g. error handling).

Approach 2: Message Passing

• Actors

• Messages

SCALES WELL. RELATIVELY SIMPLE (once you learn it).

18

Actor Model

19

Actor Model

20

Actor Model

Languages

• Erlang

• Scala

Frameworks

• Akka (Java/Scala)

• Kilim (Java)

• Pulsar (Clojure, Python)

• …

Erlang Hall of Fame:

• Availability: 99.9999999%

21

Databases

22

Background

Limited data

Localized data

Homogeneous requirements

RDBMS have matched these requirements

23

The Rise of NoSQL

ACID

• Atomic

• Consistent

• Isolated

• Durable

vs

BASE

• Basic Availability

• Soft-state

• Eventual Consistency

24

Requirements Have Changed

Mass scale (Big Data)

Distributed Data Storage

Different varieties of data

Incompatible requirements

RDBMS are no longer suitable to handle all use cases

25

Many Databases

26

NewSQL strikes back

Eventual Consistency complicates life a lot

"We also have a lot of experience with eventual consistency systems at

Google. In all such systems, we find developers spend a significant

fraction of their time building extremely complex and error-prone

mechanisms to cope with eventual consistency and handle data that

may be out of date.“

MIT Prof. Michael Stonebraker: “The Traditional RDBMS Wisdom is

All Wrong”

• Most OLTP systems can afford to buy the amount of memory needed to keep

data off the disk.

• Single-threading gets rid of the latching overhead.

• OLAP: Column Stores are 50-100 times faster

• …

27

Memory/Network/Disk speed

28

Separate CPU/Memory/Storage Racks

140 TB

29

So Who is Right? NoSQL or NewSQL?

Well… Both!

Polyglot Persistence

30

“Software Defined Datacenter”

20.11.2013 г. 30

31

Storage

32

Interesting Times

Application/OS Deployment

• Deploy directly on the hardware

• New business -- buy hardware

Application Development

• Pet Apps (Fixed Scale)

• Sequential Programming + Parallel

Programming with Shared Memory

Databases

• Local fast disks

• RDBMS or NoSQL

Data centers

• Specialized hardware

• Virtualize the hardware

• Use a service

• Cattle Apps (Dynamic Scale)

• Parallel Programming with

Message Passing

• In-memory, distributed

• Polyglot Persistence

(NoSQL,NewSQL,etc)

• Cheap hardware + software