automated scaling of microservice stacks for javaee applications - jeeconf 2017
TRANSCRIPT
![Page 1: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/1.jpg)
Automated Scaling of Microservice Stacks for
JavaEE Applications
by Ihor Kolodyuk
![Page 2: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/2.jpg)
Few words about me …
Driving technology direction at Jelastic
Like to solve unsolvable problems
Checking technology concepts on practice
![Page 3: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/3.jpg)
Why I am here?
I want to talk to you about:
General autoscaling concept for JavaEE applications
Real use cases. Real issues. Tricky things
My goals are:
Save your time if you decide you need to scale your app
Share our own experience
![Page 4: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/4.jpg)
I just deploy my application to Docker, set a number of replicas and … have it running … bla bla bla … super easy! Done!
The myth
![Page 5: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/5.jpg)
Yes, pretty easy …
If you need a super scalable and clustered Hello World application …
![Page 6: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/6.jpg)
In real life with JavaEE application we have architecture similar to this
and its autoscaling becomes not a very trivial task…
![Page 7: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/7.jpg)
The truth
Legacy Application
Scalable Cluster
Autoscalable Cluster
![Page 8: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/8.jpg)
Theoretical part
![Page 9: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/9.jpg)
At least 3 key things to make your applications scalable
![Page 10: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/10.jpg)
First:Adapted runtime
(Container + JVM + JavaEE Server)
micro
![Page 11: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/11.jpg)
Preferably, thin versions of application servers
YES!NO!
![Page 12: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/12.jpg)
microglassfish
There are various versions adapted for lightweight containers
![Page 13: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/13.jpg)
Any specific reason to use VMs for microservices?
NO!
![Page 14: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/14.jpg)
Well…it might be, but …
![Page 15: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/15.jpg)
What should we use instead?
YES!
![Page 16: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/16.jpg)
One container - one microservice instance
YES! NO!
![Page 17: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/17.jpg)
Second:Application
prepared for microservices
APPMS
MS MS
MS
![Page 18: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/18.jpg)
SORRY, Application
prepared for SCALABLE microservices
![Page 19: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/19.jpg)
Next few slides can be a bit boring for those who know…
BUT
Still important!!!!!!
![Page 20: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/20.jpg)
Ideal scaling and decomposition
horizontal
verti
cal
topolo
gy
![Page 21: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/21.jpg)
Decomposition into microservice
At least 2 key approaches
![Page 22: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/22.jpg)
Logical division
![Page 23: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/23.jpg)
Splitting bigger parts into smaller
1. Analyse metrics (NewRelic / JavaVisualVM)
2. Find a weak point (!)
3. Move this part to microservice
4. Repeat
![Page 24: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/24.jpg)
Third:Orchestration software
![Page 25: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/25.jpg)
What it typically does?
CONTAINERSPROVISIONING
HEALTHCHECKS
METRICSGATHERING
APPLICATION LIFECYCLEMANAGEMENT
SCALING
AUTOMATIC LOADBALANCING
SERVICE DISCOVERY&CONNECTIVITY
CUSTOM SCRIPTING*
![Page 26: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/26.jpg)
Typical triggers that are used for scaling
CPU RAM
HDD(disk I/O) NETWORK
SERVICEFAILURE SCHEDULER
![Page 27: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/27.jpg)
Practical part
Tricky things
Typical issues
Examples
![Page 28: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/28.jpg)
Tricky thing #1
In which direction to scale?
horizontal
verti
cal
topolo
gy
![Page 29: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/29.jpg)
Example #1Message Queue Service
Triggers configuration
CPURAMHDD (disk I/O)NETWORKSCHEDULER YES
NOPROBABLY NO
GENERAL SETGENERAL SET
Scaling type HORIZONTAL
Tricky place
![Page 30: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/30.jpg)
Example #2Databases
CPURAMHDD (disk I/O)NETWORKSCHEDULER
YES
VERTICAL
NO
NONO
Scaling type
Triggers configuration
YES
![Page 31: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/31.jpg)
Typical issues while scaling of database instances
Takes much time to sync data
Temporary locks
Temporary performance degradation
Hard to use auto horizontal scaling Automatic vertical scaling can be used just fine It’s better to keep spare instances underloaded
(dedicated storage per instance)
RESUME
![Page 32: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/32.jpg)
Example #3Legacy JavaEE application
TOPOLOGY
Java EE Server
Business Tier
Web Tier
JSP App1 JSP App2 JSP App2
EJB App1 EJB App2
Java EE Micro Instance
JSP App1
Java EE Micro Instance
JSP App1
Java EE Micro Instance
JSP App1Java EE
Micro Instance
JSP App1
Java EE Micro Instance
JSP App1
Java EE Micro Instance
JSP App2
Java EE Micro Instance
JSP App1
Java EE Micro Instance
JSP App1
Java EE Micro Instance
EJB App2Java EE
Micro Instance
JSP App1
Java EE Micro Instance
JSP App1
Java EE Micro Instance
EJB App2
Scaling type
![Page 33: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/33.jpg)
Tricky thing #2
Events subscription
Usually, just spinning up an extra instance is NOT enough
![Page 34: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/34.jpg)
Example #1Typical ScaleIn/ScaleOut events and cases
onAfterScaleOut[nodeGroup:ejb-app1] { “call”: “registerNodeInMonitoringSystem”, “call”: “addMemberToHazelcast”,
“call”: “call3rdPartyApiService”
}
onAfterScaleIn[nodeGroup:jsp-app2] { “call”: “removeNodeFromMonitoringSystem”, “call”: “removeMemberFromHazelcast” }
THE FULL WORKING AND MORE COMPLEX CODE EXAMPLE CAN BE FOUND HERE
![Page 35: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/35.jpg)
Example #2
2. Get auth keys by instance before joining DASonBeforeStartService[nodeGroup:instance] { “cmd”: “/root/scripts/pullAccessKeys.sh ${token}” }
THE FULL WORKING AND MORE COMPLEX CODE EXAMPLE CAN BE FOUND HERE
1. Decrypt encrypted volume, that was attached on-fly
onAfterVolumeAttached[nodeGroup:instance] { “call”: “decryptVolume” }
3. Deprovision node from DAS if cluster was shrinked
onAfterScaleOut[nodeGroup:das] { “cmd”: “./asadmin_proxy remove ${instance[@last].ip}” }
![Page 36: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/36.jpg)
Tricky thing #3Proper triggers configuration
This tuning is a long process, so be patient
![Page 37: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/37.jpg)
Typical problem #1
Changing the value in one place can be reflected in other places
![Page 38: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/38.jpg)
How to deal with this?
Load Testing
Gathering
Metrics
Detect deviations in a testing loop
Analyz
ing
Metrics
Edit
Trigg
ersCompare previous
results
![Page 39: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/39.jpg)
Typical problem #2The triggers can become outdated with new code
![Page 40: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/40.jpg)
The fix is easy here
Inject metrics collection as a part of your CI/CD step
COMMIT BUILD TESTING PERFORMANCEANALYSIS DELIVERY
![Page 41: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/41.jpg)
Tricky thing #4Getting rid of spare things
![Page 42: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/42.jpg)
Typical problem #1Network is full of useless traffic
Total bandwidth Multicast | HeartbeatsUseful traffic
![Page 43: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/43.jpg)
What we can do?
Avoid multicast (for members detection) Use events and static members lists
Static members list Static members list Static members list
Orchestrator
![Page 44: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/44.jpg)
Typical problem #2Some of JavaEE Servers are designed for VMs, not for containers
On example of Weblogic
VM VM VM VM
![Page 45: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/45.jpg)
What is the right way to go?Just remove everything spare
![Page 46: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/46.jpg)
LIVE demo comes here
![Page 47: Automated scaling of microservice stacks for JavaEE applications - JEEConf 2017](https://reader031.vdocuments.mx/reader031/viewer/2022013013/5a6d276b7f8b9ae5418b4e25/html5/thumbnails/47.jpg)
QUESTIONS