Highway to heaven - Microservices Meetup Munich

Download Highway to heaven - Microservices Meetup Munich

Post on 09-Aug-2015




2 download


<ol><li> 1. Highway to heaven Building microservices in the cloud Munich microservices meetup, June 2015 </li><li> 2. Agenda Meet AutoScout24 Shifting gears How we build our services How we organize ourselves </li><li> 3. AutoScout24 - 10.000ft C </li><li> 4. Baseline AutoScout24 IT Highly optimized, but of last decade IT platform supported growth for &gt;6 years Microsoft oriented stack Enterprise IT setup - MTBF over MTTR Proven agile and lean principles C </li><li> 5. From monolith to microservices... ...via swimlanes C </li><li> 6. New CEO C </li><li> 7. Do you attract talent? C </li><li> 8. C </li><li> 9. Project Tatsu W </li><li> 10. Microservices AWS JVM / Linux DevOps culture Product Five challenges W </li><li> 11. Shiny new cut Cut to the core Add new business value New user experience W </li><li> 12. Objectives Mobile first, user centric Sophisticated user experience Reduce time to market Attract talents Reduce costs in IT production W </li><li> 13. Technical transformation W </li><li> 14. Change the wheels while driving W </li><li> 15. Strategic Goals Goals of the business side Architectural Principles High-Level Principles Design and Delivery Principles Tactical measures Reduce Time to Market Speed, Fast Feedback Cost Transparency Collect metrics to allow decisions cost vs. value. Support Data-Driven Decisions Listen to users and validate hypothesis. Provide as many relevant metrics &amp; data as possible. You build it, you run it Responsibility to run and maintain a product stays with the building team. Fast feedback from live and customers helps us to continuously improve. Organized around Business Capabilities Build teams around products not projects. Follow the domain and respect bounded contexts. Inverse Conway Maneuver Shared Nothing Avoid shared infrastructure and tight coupling as much as possible. Dont create the next monolith. Macro and Micro Architecture Clear separation. Autonomous micro services within the rules and constraints of the macro architecture. AWS First Favor AWS service over managed service, over self-hosted OSS, over self-rolled solutions. Data-Driven/ Metric-Driven Collect metrics from processes and applications. Analyze, alert and act on them. Eliminate Accidental Complexity Strive to keep it simple. Focus on essential complexity. No silver bullet. Event Sourcing and Publishing Keep history of state changes and publish application events. Autonomous Teams Make fast local decisions. Be responsible. Know your boundaries. Share findings. Continuous Delivery Deliver changes reliable, often and fast. Infrastructure As Code Automate everything: Reproducible, traceable and tested. Immutable servers over snowflake servers. DevOps Culture Developers and Ops work together in collaborative teams as engineers. No silos. Be Bold Go into production early. Value monitoring over tests. Recover and learn. Optimize for MTTR not MTBF. Security &amp; Data Privacy Security must be first class citizen and everybodys concern. Keep data-privacy in mind. C </li><li> 16. Principles Organized around business capabilities You build it, you run it Be bold Macro and micro architecture Event sourcing and publishing C </li><li> 17. Event Sourcing and data pumps One way data highway Event Sourcing - store history of all changes C </li><li> 18. SQS + S3 Kinesis + S3 Kinesis + DynamoDB SQS + DynamoDB Proxy + DynamoDB DynamoDB Evolution W </li><li> 19. Classified Events - Overview C </li><li> 20. Classified Events - Boundaries C </li><li> 21. DynamoDB as the new Atom Feed Learn to be come a cloud native DynamoDB as service contract The return of the integration database? The price is right? W </li><li> 22. DynamoDB as the new Atom Feed Learn to be come a cloud native DynamoDB as service contract The return of the integration database? The price is right? Event Sourcing rocks C </li><li> 23. Watchlist evolution DynamoDB as JSON document store Scheduled user journey test Capacity exceeded Requests throttled Latency spikes for real users C </li><li> 24. From documents to events Refactoring toward deeper insight From CRUD to sync Offline first Writes are expensive USD 20 over USD 500 C </li><li> 25. How many layers to estimate a price Evolving architecture R backend and Play application Single Play application Play backend and Play web server Long feedback cycles Frequency of change W </li><li> 26. Shared infrastructure Shared nothing Availability over shared nothing Convenience offerings No side effects Fast local decisions over committees W </li><li> 27. Shared nothing? How many environments? Use over re-use Re-use only after hardening How to share Copy npaste, OSS, library W </li><li> 28. Unified log processing AWS EC2AWS Kinesis Event Streams Unstructured Log Stream Service A Service B System Log ELB Log Stream ELB Access Logs S3 Collector Collector Collector ElasticSearch Kibana JSON JSON JSON Collector in progress System Metrics Stream Collectd ??? AWS CloudWatch C </li><li> 29. Unified log processing C Optimised for imaginary use cases? JVM metrics ... err not used Application events over log files What are the options? </li><li> 30. How to build autonomous teams Do not fall back into old behaviours Beware of Mandelbrot teams Pager duty so that you run it Part-time ops not working Not all T-shapes are the same WolfCW </li><li> 31. Infrastructure guild Agree on things to do Share learnings Delegate implementation to teams Empty backlog should be normal How about infrastructure product teams? Mind the Shirky Principle C </li><li> 32. Team over infrastructure guild C </li><li> 33. Focus sliders Product over platform platform product time W </li><li> 34. Focus sliders (cont.) Cash stack meets shiny new stack One company Lights on in cash stack Feature freeze Where to build new features? Ease of integration helps business people C </li><li> 35. cdeger@autoscout24.com @cdeger wschlegel@thoughtworks.com @wolfwolf </li><li> 36. Attributions Blue sky, white-gray clouds by nature protector Natubico, www.vivism.info [CC BY-SA 3.0] http://commons.wikimedia.org/wiki/File%3ABlue_sky%2C_white-gray_clouds.JPG A Danish Perspective by NASA [Public domain] http://commons.wikimedia.org/wiki/File%3AA_Danish_Perspective.jpg http://commons.wikimedia.org/wiki/File%3ANASAComputerRoom7090.NARA.jpg GREG EINRAD Amazon16 by Neil Palmer/CIAT [CC BY-SA 2.0] https://www.flickr.com/photos/ciat/5641594952 BERGSTEIGER Barber in Cameroon by James Emery from Douglasville, United States (Daddy Joe_1355) [CC BY 2.0] http://commons.wikimedia.org/wiki/File%3ABarber_in_Cameroon.jpg Wide objectives by Kivela (Own work) [Public domain] href="http://commons.wikimedia.org/wiki/File%3AWide_objectives.jpg Transformer Fire Barrier by GerryS1 (Own work) [CC BY-SA 3.0 or GFDL] http://commons.wikimedia.org/wiki/File%3ATransformer_Fire_Barrier.jpg </li><li> 37. Attributions (cont) Alonso Renault Pitstop Chinese GP 2008 by Bert van Dijk (Pitstop F1 ING Renault) [CC BY-SA 2.0] http://commons.wikimedia.org/wiki/File%3AAlonso_Renault_Pitstop_Chinese_GP_2008.jpg Principle of Panchasheel by Prakash Adhikary (Own work) [CC BY 3.0] http://commons.wikimedia.org/wiki/File%3APrinciple_of_Panchasheel.JPG Traffic Jam by Doo Ho Kim [CC BY-SA 2.0] https://www.flickr.com/photos/titicat/3049591547 Pellets by The original uploader was Richard Mayer at German Wikipedia [GFDL or CC-BY-SA-3.0] http://commons.wikimedia.org/wiki/File%3APellets.jpg Pipes and Valves by Uwe Hermann [CC BY-SA 2.0] https://www.flickr.com/photos/73628542@N00/6272975359 Size variation in Coccinella undecimpunctata (2127991716) by Gilles San Martin from Namur, Belgium [CC BY-SA 2.0] http://commons.wikimedia.org/wiki/File%3ASize_variation_in_Coccinella_undecimpunctata_(2127991716).jpg Mille crpe by Laitr Keiows (Own work) [CC BY-SA 3.0 or GFDL] http://commons.wikimedia.org/wiki/File%3AMille_cr%C3%AApe.jpg Country Energy power line replacement 01 by Bidgee (Own work) [CC BY-SA 3.0] http://commons.wikimedia.org/wiki/File%3ACountry_Energy_power_line_replacement_01.jpg </li><li> 38. Attributions (cont) Sharing Sucks (4536747557) by eyeliam from Portland, United States [CC BY 2.0] http://commons.wikimedia.org/wiki/File%3ASharing_Sucks_(4536747557).jpg 7Line 9184 (8263568241) by Metropolitan Transportation Authority of the State of New York (7Line_9184 Uploaded by tm) [CC BY 2.0] http://commons.wikimedia.org/wiki/File%3A7Line_9184_(8263568241).jpg England rugby team 1905 by Russell &amp; Sons (The Graphic) [Public domain or Public domain] http://commons.wikimedia.org/wiki/File%3AEngland_rugby_team_1905.jpg Wandergeselle by Sigismund von Dobschtz [CC BY-SA 3.0] http://commons.wikimedia.org/wiki/File%3AWandergeselle_02.JPG Faber-Rechenschieber 5304 by User:Karl Gruber (Own work) [CC BY-SA 4.0] http://commons.wikimedia.org/wiki/File%3AFaber-Rechenschieber_5304.JPG Wheel clamps Texas by Richard Anderson from Denton, United States (Boots.) [CC BY-SA 2.0] http://commons.wikimedia.org/wiki/File%3AWheel_clamps_Texas.jpg GuadalupeNOLA15Oct07Thanks by Infrogmation of New Orleans (Photo by Infrogmation) [GFDL or CC BY-SA 3.0] http://commons.wikimedia.org/wiki/File%3AGuadalupeNOLA15Oct07Thanks.jpg AtariBasic by Calin99 (Own work) [GPL] http://commons.wikimedia.org/wiki/File%3AAtariBasic.png </li></ol>