joost＠de-vries․namejoost-de-vries@jouke

REACTIVESTREAMSINPRACTICE

ReactiveStreamsmeetupUtrecht29-03-2017

THREEACTS1. Alittletaste:somepracticalcode

2. whatisstreamprocessingandwhatcanwedowithit

3. reactivestreamsLet'stalkaboutourexperi

enceswithstreams

LET'SSTARTWITHSOMECODEWewanttopollsomemarketplaceevery15minutesforthe

pricesofourproducts.Andwedon'twanttobesuspended,soweonlydo4

requestsatatime.'

LET'SSTARTWITHSOMECODEvaljobParallellism:Int=4

valtickStream=Spout.tick((),jobInterval)

defgetProductIds():List[String]=???//getthecurrentproductids

defpollPrice(productId:String):Future[Long]=???//invokethemarketplaceapi

valprices=tickStream//tickevery15mins.flatMap(tick=>getProductIds())//productidsstartstreamingevery15mins.mapAsync(jobParallellism)(productId=>pollPrice(productId))//pollmax4productidsinparallel.foreach(price=>println(s"priceis$price"))//printthepricesastheycomein

ThishappenstobeScala

②WHATISSTREAMPROCESSING?

Andwhatcanwedowithit?

LIKEACROSSBETWEENITERATORANDCOMPLETABLEFUTURE

synchronous asynchronous

one AgetA() ⇒ CompletableFuture[A]getA()

many Iterator[A] ⇒ Observable[A]

UsingJavaandRxJava

LIKEACROSSBETWEENITERATORANDCOMPLETABLEFUTURE

synchronous asynchronous

one AgetA() ⇒ CompletableFuture[A]getA()

many Iterator[A] ⇒ Observable[A]

Wecallfordatavswegetcalledwithdata

pullvspush

pollingvsreactive

Sohowisastreamdifferentfromacollection?

HMM..

Astreamispotentiallyinfinite

Astreamisspreadoutintimeinsteadofmemory

Astreamcanbenonrepeatable

..alsoSpouts,Pipes,Drains

Rx:Observable,Observer

CREATEASTREAM

constsource=Rx.Observable.create((observer)=>{observer.onNext(3)observer.onNext(2)observer.onNext(1)//0ormorevaluesobserver.onCompleted()//potentiallyendorerror})

3,2,1.

(element)*(complete|error)

AFAILINGSTREAM

constsource=Rx.Observable.create((observer)=>{observer.onNext(3)observer.onNext(2)observer.onError(newError("boom"))})

3,2,1x

STREAMSINDIAGRAMS

Marblediagrams

Streamsareabouttimeandchange

Seereactivex.io

③WHATCANWEDOWITHSTREAMS?

WHATCANWEDOWITHSTREAMS?

IoT:dealingwithsensormeasurements

Systemintegration:pumpingdatabetweenservers

Userinterfacecode

Serverpush

(Near)realtime

Fastdata

Immediateresultsastheinputsarecomingin

DETERMINEHIGHWATERMARKFORSENSORMEASUREMENTS

caseclassMeasurement(rotorSpeed:Int,windSpeed:Int,timestamp:Instant)

defmeasureTurbine():Measurement=Measurement(rotorSpeed=Random.nextInt(),windSpeed=tickStream.map(tick=>measureTurbine()).scan(0){case(currentMaxRotorspeed,measurement)=>if(measurement.rotorSpeed>currentMaxRotorspeed)measurement.rotorSpeedelsecurrentMaxRotorspeed}.deduplicate//oftencalled'distinct'.onElement(max=>println(s"maxrotorspeedis$max")).drainToBlackHole()

USERINTERFACELOGIC:NETFLIXfunctionplay(movieId,cancelButton,callback){varmovieTicket,playError,tryFinish=()=>{if(playError){callback(null,playError);}elseif(movieTicket&&player.initialized){callback(null,ticket);}};cancelButton.addEventListener(“click”,()=>{playError=“cancel”;});if(!player.initialized){player.init((error)=>{playError=error;tryFinish();}}authorizeMovie(movieId,(error,ticket)=>{playError=error;movieTicket=ticket;tryFinish();});});

SeeJafarHusainstalk

USERINTERFACELOGIC:NETFLIXvarauthorizations=player.init().map(()=>playAttempts.map(movieId=>player.authorize(movieId).catch(e=>Observable.empty).takeUntil(cancels)).concatAll())).concatAll();

authorizations.forEach(license=>player.play(license),error=>showDialog(“Sorry,can’tplayrightnow.”));

SeeJafarHusainstalk

CLUSTEREDSTREAMSdefrankLangsReduceByKey(langs:List[String],articles:DStream[WikipediaArticle]):List[(Stringarticles.flatMap{art=>langs.map{lang=>if(art.text.split("").contains(lang))(lang,1)else(lang,0)}}.reduceByKey{case(acc,i)=>acc+i}.collect().toList.sortBy{case(lang,i)=>i}.reverse}

CLUSTEREDSTREAMSdefrankLangsReduceByKey(langs:List[String],articles:DStream[WikipediaArticle]):List[(Stringarticles.flatMap{art=>langs.map{lang=>if(art.text.split("").contains(lang))(lang,1)else(lang,0)}}.reduceByKey{case(acc,i)=>acc+i}.collect().toList//RDDcode!collectisn'tavailableonDStream.sortBy{case(lang,i)=>i}.reverse}

Typesofstreamprocessing

NotonlyJVM:JS,.Net,Swift,..

④WHYSTREAMS?

STARTWORKINGWHENDATAAVAILABLE

Pushinsteadofpull

Leastlatency

Processstuffwithoutloadingitallinmemory

Reverseofquery/polling

NONBLOCKINGBOUNDEDUSEOFRESOURCES

limitedthreads

boundedmemoryusage(seebackpressure)

importantformicroservices/IoT-mobilelongrunningreqs

CLEANPROGRAMMINGMODEL

Concurrencymodel

(flat)map:collection,parallelcollection,clusteredcollection,stream

Functionalprogramming

Pathofexploringstreams

Symptoms:sideeffects,

assignments,subjects

UILOGICINTYPESCRIPTRXJSconstincassos:Observable<Pagination<IncassoListResult>>=this.queryObservable.filter((incassoQuery)=>{if(incassoQuery.search)returnincassoQuery.search.length>=3elsereturntrue}).switchMap((incassoQuery)=>{returnthis.incassoService.getIncassos(incassoQuery,PaginationComponent.calculatePaginationRange(incassoQuery.pageNr,incassoQuery.pageSize))}).share()

Moveassignmentsandsubjects

totheoutsideofyourcode

Observablein,Observableout

Usepuretransformations

⑤REACTIVESTREAMS

Whatmakesthemdifferent?

REACTIVESTREAMSInteroperabilitystandard

Oracle

Lightbend

Pivotal

Netflix

Redhat

Supportsbackpressure

STREAMANYTHING

ExamplefromAkka

SpringCloudwillsupportReactor(?)

RSINTERFACESINJAVA9

SPIvsAPI

BACKPRESSURE

Whatifupstreamisfaster?

Buffer

Dropelements

Slowdown

BACKPRESSURE

Doesn'tmakesenseforsensordata,mouseclicks,.

..

Butifyoucan'tloseevents...

Downstreamcommunicatesdemand

⑥RESOURCES

AsyncJavaScriptatNetflix,"JafarHusainhttps://www.infoq.com/presentations/netflix-rx-extensions

ErikMeijeronobservables

http://reactivex.io/

TalksbyStephaneMaldini

etc-

SOURCESOverviewoftypesofstreams:MathiasDoenitztalkonSwave

ErikMeijeronobservables-

Fin