DEVELOPING WITH CLOUD SERVICES

Chris Richardson

Author of POJOs in ActionFounder of the original CloudFoundry.com

@crichardsonchris.richardson@springsource.comhttp://plainoldobjects.com

@crichardson

Presentation goal

How to build robust, scalable applications with

Cloud Services

@crichardson

About Chris

@crichardson

(About Chris)

@crichardson

About Chris()

@crichardson

About Chris

@crichardson

About Chris

http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/

@crichardson

vmc push About-Chris

Developer Advocate for CloudFoundry.com

Signup at http://cloudfoundry.com

@crichardson

Agenda

• Why use cloud services?

• Developing location-based applications

• Building SMS and telephony enabled applications

• Developing robust, fault tolerant applications

@crichardson

Three phases of every galactic civilization

Survival

Inquiry

Sophistication

@crichardson

Three phases of every galactic civilization

How

Why

Where

can we eat?

do we eat?

Where shall we have lunch?

@crichardson

Where shall we have lunch?

@crichardson

Solved by VoteMeetEat.com

•What restaurants are nearby?•Which friends are close by? •Where do your friends prefer to eat?

To sign up text "register" to 510-

XXX-YYYY

@crichardson

VoteMeetEat.com

Friend and restaurant location databases+

SMS +

Voice callsTo sign up text

"register" to 510-XXX-YYYY

@crichardson

Key story: registration

5551212

@crichardson

Key story: registration

+5105551212

@crichardson

Key story: voting

555 1212

@crichardson

Key story: announce location

VOTEMEETEAT.COM

To sign up text "register" to 510-

XXX-YYYY

@crichardson

High-level architecture

VoteMeetEat

TelephonyIntegration

Friend GeoDatabase

Restaurant Database

MobilePhone

Do we really want to build all this?

DIY = DIFFICULT

@crichardson

Use cloud-based services

• Highly scalable services

• Someone else’s headache to develop and maintain

• Provided by IaaS/PaaS

• Provided by 3rd party

@crichardson

Cloud Foundry services

@crichardson

Thousands of 3rd party services

http://www.programmableweb.com/apis/directory

http://www.slideshare.net/jmusser/j-musser-apishotnotgluecon2012

@crichardson

• Predominantly REST

• Predominantly JSON

• > billion API calls/day: Twitter, Google, Facebook, Netflix, Accuweather, ...

• Increasing number of API-only companies

http://www.slideshare.net/jmusser/j-musser-apishotnotgluecon2012

Cloud service trends

@crichardson

Diverse

@crichardson

Benefits of cloud services

• Someone else’s headache to develop and operate

• Focus on your core business problem

• Get up and running quickly

• Elasticity

• Capex ⇒ Opex

@crichardson

Drawbacks of cloud services

• Complexity and drawbacks of a distributed system

• You are dependent on service provider

@crichardson

Risks of cloud services

Urban Airship’s Strategic Partnership With SimpleGeo Turns Into An Acquisition

@crichardson

Cloud Services-based architecture

VoteMeetEat

TwilioMongoDB

Factual.Com

MobilePhone

@crichardson

DEMO

@crichardson

Agenda

• Why use cloud services?

• Developing location-based applications

• Building SMS and telephony enabled applications

• Developing robust, fault tolerant applications

@crichardson

Location-based services are hot!

@crichardson

Client-side APIs for finding location

W3C Geolocation API

@crichardson

BUT what about the server-side?

@crichardson

Lots of really difficult problems

•Scalable, spatial database – CRUD records, find nearby•Data management – database of places, street information•Forward geo-coding: address ⇒ lat/lon

•Reverse geo-coding: lat/lon ⇒ address

•Maps•Directions

Easier to use Geo-aaS

@crichardson

Examples of Geo-aaS

Beware the terms of service

• Maps• Forward and reverse geocoding• Directions• Elevation• Places

• Freely available geographic database

• Various APIs including reverse geocoding

• Business+review database• Neighborhood database

• Places database• Reverse geocoding

@crichardson

VoteMeetEat.com & Geo

trait FriendService { def addOrUpdate(request : AddOrUpdateUserRequest) def findNearbyFriends(request : NearbyFriendsRequest) :

FindNearbyFriendsResponse}

trait RestaurantService { def findNearbyRestaurants(location: Location) :

FindNearbyRestaurantResponse}

@crichardson

trait FriendService { def addOrUpdate(request : AddOrUpdateUserRequest) def findNearbyFriends(request : NearbyFriendsRequest) :

FindNearbyFriendsResponse}

Implementing the friends database

@crichardson

MongoDB

• Document-oriented database

• Very fast, highly scalable and available

• Rich query language that supports location-based queries

• Provided by CloudFoundry.com

@crichardson

MongoDB server

Database: VoteMeetEat

Collection: friendRecord

Storing friends in MongoDB

{ "_id": "+15105551212", "name": "Chris R.", "location": { "x": -122.25206103187264, "y": 37.847427441773796 }}

@crichardson

Spring Data for MongoDB

• Provides MongoTemplate

• Analogous to JdbcTemplate

• Hides boilerplate code

• Domain object ↔ Document mapping

@crichardson

Using Spring data: creating an index on location attribute

@Componentclass MongoFriendService extends FriendService {

@Autowired var mongoTemplate: MongoTemplate = _

@PostConstruct def createGeoIndex { val dbo = new BasicDBObject dbo.put("location", "2d") mongoTemplate.getCollection("friendRecord").ensureIndex(dbo) }

Create geospatial 2d index

Collection name

@crichardson

Using Spring Data: adding record@Componentclass MongoFriendService extends FriendService {

override def addOrUpdate(request: AddOrUpdateUserRequest) = { val name = request.name val phoneNumber = request.phoneNumber val fr = new FriendRecord(phoneNumber, name, new Point(request.longitude, request.latitude)) mongoTemplate.save(fr) }

case class FriendRecord(id : String, name : String, location : Point)

@crichardson

Using Spring Data: finding nearby friends

@Componentclass MongoFriendService extends FriendService {

override def findNearbyFriends(request: NearbyFriendsRequest) = { val location = new Point(request.longitude, request.latitude) val distance = new Distance(3, Metrics.MILES) val query = NearQuery.near(location).maxDistance(distance) val result = mongoTemplate.geoNear(query, classOf[FriendRecord])

val nearby = result.getContent.map(_.getContent) FindNearbyFriendsResponse(nearby.map(f => FriendInfo(f.name, f.id))) }

@crichardson

MongoDB and Cloud Foundry

$ vmc create-service mongodb vme-mongo

@crichardson

$ vmc push vme-user --path web/target/Application Deployed URL [cer-spring.cloudfoundry.com]: Detected a Java SpringSource Spring Application, is this correct? [Yn]: Memory Reservation (64M, 128M, 256M, 512M, 1G) [512M]:

Creating Application: OKWould you like to bind any services to 'vme-user'? [yN]: y

Would you like to use an existing provisioned service? [yN]: yThe following provisioned services are available

1: vme-mongo2: mysql-135e0Please select one you wish to use: 1

Binding Service [vme-mongo]: OKUploading Application:

Checking for available resources: OK Processing resources: OK

Packing application: OK Uploading (12K): OK

Push Status: OK

Binding a service to an application

Would you like to bind any services to 'vme-user'? [yN]: yWould you like to use an existing provisioned service? [yN]: yThe following provisioned services are available1: vme-mongo2: mysql-135e0Please select one you wish to use: 1Binding Service [vme-mongo]: OK

@crichardson

Connecting to MongoDB <bean id="mongoTemplate"

class="org.springframework.data.mongodb.core.MongoTemplate"> <constructor-arg ref="mongoFactory" /> </bean>

<beans profile="default"> <mongo:db-factory id="mongoFactory" dbname="surveygeo" /> </beans>

<beans profile="cloud"> <cloud:mongo-db-factory id="mongoFactory" /> </beans>

Outside of Cloud Foundry

Inside Cloud Foundry

@crichardson

Implementing the restaurant database

trait RestaurantService { def findNearbyRestaurants(location: Location) : FindNearbyRestaurantResponse}

@crichardson

Using Factual

• Geographic database as a Service

• Including 1.2M restaurants in the US

• Pricing: 10K calls day free, pay per use

@crichardson

Factual API

• RESTful/JSON interface

• Uses 2-legged OAuth 1.0.

• Geo and text filters

• Pagination

• Libraries for various languages

@crichardson

Restaurant Service@Serviceclass FactualRestaurantService extends RestaurantService {

@Value("${factual_consumer_key}") var consumerKey: String = _ @Value("${factual_consumer_secret}") var consumerSecret: String = _

var factual: Factual = _

@PostConstruct def initialize { factual = new Factual(consumerKey, consumerSecret, true) }

override def findNearbyRestaurants(location: Location) = { ... val restaurants = factual.get.fetch("restaurants-us", new Query().within(new Circle(location.lat, location.lon, 1000)).limit(5))

val rs = restaurants.getData.map { map => RestaurantInfo(map.get("name").asInstanceOf[String])

}

FindNearbyRestaurantResponse(rs.toList)

}...

5 restaurants within 1km

@crichardson

Agenda

• Why use cloud services?

• Developing location-based applications

• Building SMS and telephony enabled applications

• Developing robust, fault tolerant applications

@crichardson

The telephony and SMS are important

http://blog.nielsen.com/nielsenwire/online_mobile/new-mobile-obsession-u-s-teens-triple-data-usage/

7/ waking hr !

Nielsen

@crichardson

Reporting traffic light problems in London

@crichardson

Google 2-Factor authentication

@crichardson

VoteMeetEat.com & Telephony

• Handling registration SMS

• Sending SMS notifying users to vote

• Handling incoming voice call from voters:

• Text-to-speech of restaurants options

• Collecting digits entered via keypad

• Sending SMS/Voice notifications of voting results

@crichardson

DIY telephony = Difficult

• Difficult to setup and operate

• Expensive

• Complex SMS protocols

• …Bette

r to use S

MS/Telepho

ny-aaS:

@crichardson

Telephony/SMS - aaS

• SMS• Inbound and outgoing calls• Recording and transcription

• SMS• Inbound and outgoing calls• Recording and transcription• Twitter• IM

@crichardson

Twilio - Telephony and SMS as a service

• REST API• Allocate phone numbers• Make and receive phone calls• Send and receive SMS messages

• Pay per use:• Phone calls - per-minute • SMS – per SMS sent or received• Phone number – per month

• Examples• OpenVBX is a web-based, open source phone system• StubHub – notifies sellers of a pending sale via phone• SurveyMonkey – interactive polling• Salesforce – SMS-based voting for 19,000 conference attendees

@crichardson

Using Twilio

Twilio Your Application

TwiML doc

HTTP GET/POST

REST API

Manage resourcesSend SMS

Initiate voice calls

Handle incoming SMS and voice callsRespond to user input

VoiceSMS

Phone number ⇒

SMS URL + VOICE URL

@crichardson

Handling SMS registration

5551212

User texts

‘register’

System replies

@crichardson

Handling SMS registration

TwilioSMS

REGISTRATION

HTTP POST http://≪smsUrl≫?From=≪PhoneNumber≫

<Response> <Sms>To complete registration please go to http://... </Sms></Response>

SMS

@crichardson

Handling SMS registration

TwiML document describing the

response

@crichardson

Inviting users to vote

5551212555121255512125551212

System sends text

@crichardson

Inviting users to vote

POST /2010-04-01/Accounts/≪AccountSID≫/SMS/Messages From=+15105551212&To=+14155551212&Body=≪MESSAGE≫Authorization: Basic ....

Basic auth using Twilio AccountSid+AuthToken

@crichardson

Sending SMS using the Spring RestTemplate

@Componentclass TwilioService {

def sendSms(recipient : String, message : String) = { val response = postToTwilio("SMS/Messages", Map("From" -> twilioPhoneNumber, "To" -> recipient, "Body" -> message)) (response \ "SMSMessage" \ "Sid").text }

@crichardson

Sending SMS using the Spring RestTemplate

TODO

@Componentclass TwilioService {

def postToTwilio(resourcePath : String, requestParams : Map[String, String]) = { val entity = makeEntity(requestParams)

try { val response = restTemplate.postForObject(twilioUrl +

"/Accounts/{accountSid}/{resource}", entity, classOf[String],

accountSid, resourcePath) XML.loadString(response) } catch { case e : HttpClientErrorException if e.getStatusCode == HttpStatus.BAD_REQUEST => val body = e.getResponseBodyAsString() val xmlBody = XML.loadString(body) val code = Integer.parseInt((xmlBody \\ "Code").text) val message = (xmlBody \\ "Message").text throw new TwilioRestException(message, code) }}

@crichardson

Voting: user calls number

555 1212

@crichardson

Voting

Twilio

Survey Management<Response> <Say> Chris would like to meet and eat. </Say> <Gather action="handleresponse.html"

method="POST" numDigits="1"> <Say>Press 1 for ....</Say> <Say>Press 2 for ....</Say> </Gather></Response>

HTTP POST http://≪voiceUrl≫?From=≪PhoneNumber≫

Call

@crichardson

Voting

Twilio

Survey Management

<Response> <Say>Thank you for choosing. The most popular place so far is ... </Say> <Pause/> <Say>You will hear from us soon. Good bye</Say> <Hangup/></Response>

HTTP POST http://....handleresponse.html?From=≪PhoneNumber≫&Digits=≪...≫

Digits

@crichardson

Voting code 1@Controllerclass TwilioController { @Autowired var surveyManagementService: SurveyManagementService = _

@RequestMapping(value = Array("/begincall.html")) @ResponseBody def beginCall(@RequestParam("From") callerId: String) = { surveyManagementService.findSurveyByCallerId(callerId) match { case None => <Response> <Say>Sorry don't recognize your number</Say> <Hangup/> </Response> case Some(survey) => <Response> <Say>{ survey.prompt }</Say> <Gather action="handleresponse.html" method="POST" numDigits="1"> { for ((choice, index) <- survey.choices zipWithIndex) yield <Say>Press { index } for { choice }</Say> } </Gather> <Say>We are sorry you could not decide</Say> <Hangup/> </Response> } }

@crichardson

Voting code 2class TwilioController { ... @RequestMapping(value = Array("/handleresponse.html")) @ResponseBody def handleUserResponse(@RequestParam("From") callerId: String,

@RequestParam("Digits") digits: Int) = { val survey = surveyManagementService.recordVote(callerId, digits) <Response> <Say>Thank you for choosing. The most popular place so far is

{ survey.map(_.mostPopularChoice) getOrElse "oops" } </Say> <Pause/> <Say>You will hear from us soon. Good bye</Say> <Hangup/> </Response> }}

@crichardson

Agenda

• Why use cloud services?

• Developing location-based applications

• Building SMS and telephony enabled applications

• Developing robust, fault tolerant applications

@crichardson

The need for parallelism

Service A

Service B

Service C

Service D

b = serviceB()

c = serviceC()

d = serviceD(b, c)

Call in parallel

@crichardson

Futures are a great concurrency abstraction

• Object that will contain the result of a concurrent computation - http://en.wikipedia.org/wiki/Futures_and_promises

• Various implementations

• Java 7 Futures = ok

• Guava ListenableFutures = better

• Scala’s composable Futures = really good

• Java 8 CompletableFuture = great

Future<Integer> result = executorService.submit(new Callable<Integer>() {... });

@crichardson

Using futures to parallelize requeststrait FriendService { def findNearbyFriends(request : NearbyFriendsRequest) :

Future[FindNearbyFriendsResponse]}

trait RestaurantService { def findNearbyRestaurants(location: Location) :

Future[FindNearbyRestaurantResponse]}

val f1 = friendsService.findNearbyFriends(NearbyFriendsRequest.fromLocation(vmeRecord.location))val f2 = restaurantService.findNearbyRestaurants(vmeRecord.location)

val nearbyFriends = f1.get(2, TimeUnit.SECONDS)val nearbyRestaurants = f2.get(2, TimeUnit.SECONDS)

Two calls execute concurrently

Client Side

Proxies

@crichardson

Using external web services = Distributed system

VoteMeetEat

TwilioMongoDB

Factual.Com

MobilePhone

@crichardson

Internally = Distributed System

Survey management

VMEmanagement

Registration SMS

Registrationweb app

VME web app

Usermanagement

RabbitMQ

@crichardson

Handling failure

Service A Service B

Errors happen in distributed systems

@crichardson

About Netflix

> 1B API calls/day

1 API call ⇒ average 6 service calls

Fault tolerance is essential

http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html

@crichardson

Use network timeouts and retries

Never wait forever

Network errors can be transient ⇒ retry

http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html

@crichardson

Service A

Service B

Use per-dependency bounded thread pool

http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html

Runnable 1

Runnable 2

Runnable ...

bounded queue

Task 1

Task 2

Task ...

bounded thread pool

Limits number of outstanding requests

Fails fast if service is slow or down

@crichardson

Use a circuit breaker

High error rate ⇒ stop calling temporarily

Down ⇒ wait for it to come back up

Slow ⇒ gives it a chance to recover

http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html

Closed Open

Half open

errors

successtimeout

fail

@crichardson

On failure

http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html

AvoidFailing

Return cached data

Return default data

Fail fast

@crichardson

About Netflix Hystrix

• Open-source library from Netflix

• Implements

• Circuit Breaker pattern

• Bounded thread-pool

• Fallback logic

• ...

• https://github.com/Netflix/Hystrix

class MyCommand extends HystrixCommand[ResultType](...)

override def run() = { ... Invoke remote service ... }}

val future = new MyCommand().queue()...

@crichardson

@Serviceclass FactualRestaurantService extends RestaurantService {

@Autowired @Qualifier("factualHystrixConfig") var factualHystrixConfig: HystrixCommand.Setter = _

override def findNearbyRestaurants(location: Location) = {

class FindRestaurantsCommand extends HystrixCommand[FindNearbyRestaurantResponse] (factualHystrixConfig .andCommandKey(HystrixCommandKey.Factory.asKey("FindRestaurantsCommand"))) {

override def run() = { val restaurants = factual.fetch("restaurants", new Query().within(new Circle(location.lat, location.lon, 1000)).limit(5)) val rs = for (map <- restaurants.getData) yield { RestaurantInfo(map.get("name").asInstanceOf[String]) } FindNearbyRestaurantResponse(rs.toList) } }

new FindRestaurantsCommand().queue() }}

Using Hystrix

@crichardson

Summary

Cloud services are highly scalable services developed and operated by a 3rd party

Let’s you focus on your core business problem

Risk: provider is acquired and stops offering service

Developing an application that reliably consumes cloud services requires careful design

@crichardson

Questions?

@crichardson chris.richardson@springsource.comhttp://plainoldobjects.com - code and slides

Sign up for CloudFoundry.com