decomposing applications for deployability and scalability #gluecon

Decomposing applications for deployability and scalability

Chris Richardson

Author of POJOs in ActionFounder of the original CloudFoundry.com

@[email protected]

1

Presentation goal

2

How decomposing applications improves

deployability and scalability and

How Cloud Foundry helps

About Chris

3

(About Chris)

4

About Chris()

5

About Chris

6

About Chris

http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/

7

vmc push About-Chris

8

Developer Advocate for CloudFoundry.com

Signup at http://cloudfoundry.com

Promo code: Gluecon12

9

Agenda The (sometimes evil) monolith Decomposing applications into services How do services communicate? Presentation layer design How Cloud Foundry helps

Let’s imagine you are building an e-commerce application

10

Tomcat

Traditional web application architecture

11

Browser

WAR

MySQL Database

developtestdeployscale

ShippingService

AccountingService

InventoryService

StoreFrontUI

Simple to

Apache

But there are problems

12

Users expect a rich, dynamic and interactive experience on mobile devices and desktop

13

Java Web Application

Browser

HTTP Request

HTML/Javascript

Old style UI architecture isn’t good enough

Real-time web ≅ NodeJS

Obstacle to frequent deployments Need to redeploy everything to change one component Interrupts long running background (e.g. Quartz) jobs Increases risk of failure

Fear of change

Updates will happen less often e.g. Makes A/B testing UI really difficult 14

Large application

Slow IDESlow start times for web container

15

Scaling development

16

!=Scalable development

WAR

Shipping

Accounting

InventoryService

StoreFrontUI

Forces teams to synchronize development efforts Teams need to coordinate updates

Long-term commitment to a single technology stack Let’s suppose you go with the JVM•Some polyglot support•Many (but not all) JVM languages interoperate easily• e.g. Scala, Groovy modules within a Java application•But you can’t use non-JVM languages

Application depends on a particular:•Framework - e.g. Spring or Java EE•Container - e.g. Tomcat

Switching technology stack ⇒ touches entire application•Painful•Rarely done

17

18


3 dimensions to scaling

19

X axis - horizontal duplicationZ ax

is - d

ata pa

rtitio

ning

Y axis - functionaldecomposition

Scale by cloning

Scale

by sp

litting

simila

r thin

gsScale by splitting different things

Y axis scaling - functional partitioning Splits monolithic application into a set of services Each service implements a set of related functionality Partitioning schemes:•Partition functionality by noun or by verb•Single Responsibility Principle•Unix utilities - do one focussed thing well

20

ShippingService

AccountingService

billing web application

shipping web application

InventoryService

inventory web application

21

Y axis scaling - application level

Store Front

Store front web application

Apply X axis cloning and/or Z axis partitioning to each service

Real world examples

22

http://highscalability.com/amazon-architecture

Between 100-150 services are accessed to build a page.

http://techblog.netflix.com/

http://www.addsimplicity.com/downloads/eBaySDForum2006-11-29.pdf

http://queue.acm.org/detail.cfm?id=1394128

There are drawbacks Complexity: •Architectural•Development•Deployment•Application management

Deciding when to use it• In the beginning: you don’t need it and it will slow you down•When you do need it: refactoring existing code is painful

23

See Steve Yegge’s Google Platforms Rant re Amazon.com

But there are many benefits Scales development: focussed two pizza devops teams Deploy services independently Scale services independently Improves fault isolation Enforces well defined interfaces between components Eliminates long-term commitment to a single technology

stack

24

Modular, polyglot applications

Two levels of architecture System level•Defines the inter-service glue: interfaces and

communication mechanisms•Slow changing

Service level•Defines the internal architecture of each service•Far fewer constraints on technology•Each service could use a different technology stack•Rapid evolving

25

Versus

If services are small... Regularly rewrite using a better technology stack Pick the best developers rather than best <pick a

language> developers ⇒ polyglot culture Adapt system to changing requirements and better

technology without a total rewrite

26

Fred George “Developer Anarchy”

Moreover: you are not the same you ... Cell lifetimes:• hours - some white blood cells• days - stomach lining cells• years - bone cells• lifetime - brain cells

50 to 70 billion of your cells die each day Yet you (the system) remains intact

27

http://dreamsongs.com/Files/WhitherSoftware.pdf

http://dreamsongs.com/Files/DesignBeyondHumanAbilitiesSimp.pdf

Can we build software systems with these

characteristics?

Too much technical debt ⇒ component death?

28


Inter-service communication options Multiple collaborating services ⇒ need a communication

mechanism Many choices:•Synchronous ⇔ asynchronous•Transports: HTTP, AMQP, ...•Formats: JSON, XML, Protocol Buffers, Thrift, ...•Even via the database

Distributed application ⇒ error handling strategies

29

Asynchronous is preferred

JSON is fashionable but binary format is more efficient

StoreFrontUI

wgrus-store.war

AccountingService

wgrus-billing.war

WidgetInventoryService

wgrus-inventory.war

InventoryService

wgrus-inventory.war

MySQL

30

RabbitMQ(Message Broker)

Asynchronous message-based communication

Benefits and drawbacks Benefits•Decouples caller from server•Caller unaware of server’s coordinates (URL)•Message broker buffers message when server is down/slow

Drawbacks•Additional complexity of message broker •RPC using messaging is more complex

31

Writing code that calls services

32

Composable futures Problem:•Service A needs to call services B and C and then D•Makes sense to call B and C parallel•Yet most concurrency APIs are low-level, error-prone etc

Solution:•Use Akka composable futures = really nice abstraction

33

val futureB = callB()val futureC = callC()

val futureD = for { b <- futureB.mapTo[SomeType] c <- futureC.mapTo[SomeType] d <- callD(b, c) } yield d

val result = Await.result(futureD, 1 second). asInstanceOf[SomeOtherType]

http://doc.akka.io/docs/akka/2.0.1/scala/futures.htmlhttp://en.wikipedia.org/wiki/Futures_and_promises

Two calls execute in parallel

And then invokes D

Get the result of D

Spring Integration

Builds on Spring framework

Implements EAI patterns = high-level of abstraction for building message based applications

Provides the building blocks for a pipes and filters architecture

• Pipes = message channels

• Filters = endpoints that filter, transform, route messages and integrate with external messaging infrastructure

Enables development of components that are

• loosely coupled

• insulated from underlying messaging infrastructure

34

Handling failure Errors happen (especially in distributed systems) Use timeouts and retries•Never wait forever•Some errors are transient so retry

Use per-dependency bounded thread pool with bounded queue• Limits number of outstanding requests•Fails fast if service is slow or down

Use circuit breaker•High rate of errors ⇒ stop calling temporarily •Avoids calling service that has issues

On failure•Returned cached or default data• Invoke custom error handler

35http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html

36


Modular application

Choice of presentation layer technology+

Redeploy UI frequently/independently

37

NodeJS is the fashionable technology

38

Many JavaScript client frameworks have a NodeJS counterparte.g. socket.io

NodeJS isn’t the only game in town

39

JVM-based http://vertx.io/

A modern web application

40

Browser

Service 1

Service 2

...

HTML 5Application

Socket.ioclient

Events

RESTful WS

Server Application

Socket.ioserver

Node JS

NodeJS - using RESTful WS and AMQP

41

Node JS

Service

RabbitMQ Service

REST

AMQP AMQP

RESTRequests

Eventssocket.io

Updating the UI is easy

42

Update the UI independently of rest of system Easily run A/B tests Enables fast iteration of the UI

http://theleanstartup.com/principles

But coordination with backend changes required Let’s imagine that you are deploying an advanced search

feature:•Enhancements to search service•Enhancements to UI

Before•Deploy new war

Now:•Some coordination required•Deploy updated backend service•Deploy updated NodeJS and browser code•Enable feature using feature switch

43

44


Traditional tools: monolithic applications

45

Developing modular apps is more difficult Many more moving parts to manage•Platform services: SQL, NoSQL, RabbitMQ•Application services: your code

Who is going to setup the environments:• the developer sandbox?• ...•QA environments?

46

But Cloud Foundry helps...

Applica'on Service Interface

Data Services

Other Services

Msg Services

Easy polyglot application deployment and service provisioning

vFabric Postgres

vFabric RabbitMQTM

Additional partners services …

OSS community

Private Clouds

PublicClouds

MicroClouds

Multi-application manifest - part 1--- applications: inventory/target: name: inventory url: cer-inventory.chrisr.cloudfoundry.me framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 services: si-rabbit: type: :rabbitmq si-mongo: type: :mongodb si-redis: type: :redis

49

Path to application

Required platform services

Multi-application manifest - part 2 store/target: name: store url: cer-store.chrisr.cloudfoundry.me framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 services: si-mongo: type: :mongodb si-rabbit: type: :rabbitmq

50

Path to application

Required platform services

One command to create platform services and deploy application

$ vmc push Would you like to deploy from the current directory? [Yn]: Pushing application 'inventory'...Creating Application: OKCreating Service [si-rabbit]: OKBinding Service [si-rabbit]: OKCreating Service [si-mongo]: OKBinding Service [si-mongo]: OKCreating Service [si-redis]: OKBinding Service [si-redis]: OKUploading Application: Checking for available resources: OK Processing resources: OK Packing application: OK Uploading (12K): OK Push Status: OKStaging Application 'inventory': OK Starting Application 'inventory': OK Pushing application 'store'...Creating Application: OKBinding Service [si-mongo]: OKBinding Service [si-rabbit]: OKUploading Application: Checking for available resources: OK Processing resources: OK Packing application: OK Uploading (5K): OK Push Status: OKStaging Application 'store': OK Starting Application 'store': ... 51

vmc push:•Reads the manifest file•Creates the required platform services•Deploys all the applications

Micro Cloud Foundry: new developer sandbox

52

Open source Platform as a Service project

App Instances Services

10.04

A PaaS packaged as a VMware Virtual Machine

Use as a developer sandbox

• Use the services from Junit integration tests

• Deploy your application for functional testing

• Remote debugging from STS

Using Caldecott…$ vmc tunnel1: mysql-135e02: mysql1Which service to tunnel to?: 2Password: ********Stopping Application: OKRedeploying tunnel application 'caldecott'.Uploading Application: Checking for available resources: OK Packing application: OK Uploading (1K): OK Push Status: OKBinding Service [mysql1]: OKStaging Application: OK Starting Application: OK Getting tunnel connection info: OK

Service connection info: username : uMe6Apgw00AhS password : pKcD76PcZR7GZ name : d7cb8afb52f084f3d9bdc269e7d99ab50

Starting tunnel to mysql1 on port 10000.1: none2: mysqlWhich client would you like to start?: 2

…Using CaldecottLaunching 'mysql --protocol=TCP --host=localhost --port=10000 --

user=uMe6Apgw00AhS --password=pKcD76PcZR7GZ d7cb8afb52f084f3d9bdc269e7d99ab50'

Welcome to the MySQL monitor. Commands end with ; or \g.Your MySQL connection id is 10944342Server version: 5.1.54-rel12.5 Percona Server with XtraDB (GPL),

Release 12.5, Revision 188

Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

Running JUnit test with Caldecott

55

Configure your test code to use port + connection info

Summary Monolithic applications are simple to develop and deploy

BUT applying the scale cube

Decomposes your application into services Enables scaling for transactions and data volumes Tackles application complexity Enables scaling for development Enables frequent, independent deployments Make it easy to leverage other technologies

AND

Cloud Foundry simplifies the development and deployment of modular, service-based applications

56

Questions?

@crichardson [email protected]

www.cloudfoundry.com promo code Gluecon12

decomposing applications for deployability and scalability #gluecon

Technology