evolutionary architectures and fitness functions: a pragmatic ......(asp.net webapi) http/rest...
TRANSCRIPT
Evolutionary architectures and fitness functions: A pragmatic approach
Software Architecture
#OReillySACon
Rene Weiss Berlin, 2019
AgendaEvolution!What are Fitness Functions?Examples! (#Code)How to start?
Rene [email protected]
xing.to/rweiss
@renebiancoSoftware Architect, Consultant, Trainer @ embarc
Focus areasq Software architecture (review, design, …)q iSAQB trainer for agile software architecture
(AGILA)q Agile methodologiesq (.NET technologies)
Founded 2013 in Hamburg
Since 2015 also in Vienna
IntroductionEvolutionary architectures
Fitness Functions
Nature has solutions for complex problems…
Evolutionary approaches….
… by using small, but steady adaptions
“Software architecture is about the important stuff, whatever that is.”
(Martin Fowler)
… what about architecture
Evolutionary architecture
Evolutionary architecture supports small, incremental changes spanning multiple dimensions of the software architecture.
q small changesq broad involvement with a lot of participantsq geared toward common goals
à Fitness Functions!
Fitness Functions!DefinitionCategorization (why categorize?)Tool overview
Fitness FunctionsBorrowed from evolutionary computing, a fitnessfunction is an objective function, used to summarize how close a given design solution is to achieving the set aims.
q On the search for improvements, the Fitness Function defines what is „better“.
q Equally „good“, but simpler approaches are superiour
Neal Ford, Rebecca Parsons, Patrick Kua
First Fitness Functions examplesProblem – Maintainability (Testability)� Is our test coverage sufficient?
Target audience:� Developers
Implementation idea:� Periodic review of test
coverage
Fitness Function(s):
§ Test coverage (Unit) > 0.9
§ Test coverage (Integration) > 0.6Timing:
� Each CI build� Each nightly integration test build
First Fitness Functions examplesProblem – Reliability (Availability, Resilience)� Is our service up and running if latency increases?
Fitness Function(s):
§ Integration test errors = 0% (set network latency for 3rd party connections = 5s)
Target audience:� Developers, Operations
Implementation idea:� Periodic tests of our
service with increased latency
Timing:� Each nightly integration test build
CategorizationExecution type§ Triggered § Continuous
Breadth of feedback§ Atomic § Holistic
Execution location§ CI/CD § Test environment§ Production
Quality attribute§ Maintainability§ Reliability§ Performance§ …
Monitoring type§ 0/1§ Value/Threshold§ Trend
Test applicability§ Temporary§ Permanent
Software quality attributes (-ilities)
Functional Suitability
Performance Compatibility Usability
Realiability Security Maintainability Portability
(ISO 25010)
Static code analysis (coupling, cyclomatic complexity, ..)
Functional tests + test coverage…
Autonomous runtime tests Immediate quality inspection
“Chaos Engineering”…
Consumer Driven ContractsExploratory testingPerformance testsQualitative analysis
…
LoggingMonitoring (transactions, queue
length, ressource usage …)...
Fitness Function Quadrantatomic
holistic
continuoustriggered
Fitness Function Quadrant - Toolingatomic
holistic
triggered continuous
Examples!A / B code testingMini-Chaos Engineering
< / >
Scientist (A / B Testing)Main Component Calculation Lib
Problems- Legacy (code not maintainable, „no“ test
cases)- Developers not available anymore- Errors are raised by various data constellations- Very hard to test – extremly large number of real world
test cases
Business-logic
Scientist (A / B Testing)
Solution idea
- Develop new library- Run experiments with new library in production- Continuous improvement through real world use cases
Fitness Function(s):
- Library V2 Result == Library V1 Result- Average: Lib. V2 Performance (processing time) -30% (vs. V1) - Worst Case (Single Result): Lib. V2 Performance (processing time) +10% (vs. V1)
Scientist (A / B Testing)
Main Component Calculation Lib. V1
Calculation Lib. V2
Scientist Experiment
Business-logic
Scientist Demo!
Scientist (A / B Testing)
Execution type: Continuous
Quality attribute: Funct. Suitability Performance
Monitoring type: 0/1Trend
Breadth of feedback: Atomic
Timing + Location: Production
Test applicability: Temporary
Fitness Function(s):
- Lib. V2 Result == Lib. V1 Result- Average: Lib. V2 Performance (processing time) -30% (vs. V1) - Worst Case (Single Result): Lib. V2 Performance (processing time) +10% (vs. V1)
Mini-Chaos Engineering
OpenWeather-Map
Client
Server(ASP.NET WebAPI)
HTTP/REST
HTTP/REST
WeatherApp
Problems- Dependency on 3rd party service- Resilience measures not testable
Mini-Chaos EngineeringSolution idea
- Circuit Breaker using Polly.NET- „Chaos“ (HTTP errors, latency, …) using Simmy- Tests running in integration test environment
Fitness Function(s):
- Integration-Test Error = 0% (network latency for 3rd party service = 5s)
Mini-Chaos Engineering
OpenWeather-Map
Client
Server(ASP.NET WebAPI)
HTTP/REST
HTTP/REST
WeatherApp
“CircuitBreaker” & fallback logic
Test for failure:
Chaos Demo!
Mini-Chaos Engineering
Execution type: Triggered(Continuous)
Quality attribute: Reliability
Monitoring type: 0/1
Breadth of feedback: Holistic
Timing + Location: Test env.(Production)
Test applicability: Permanent
Fitness Function(s):
- Integration-Test Error = 0% (network latency for 3rd party service = 5s)
How to start?
Initially
5. Automate testing and evaluation
1. Identify central quality goals (use ISO 25010)
2. Develop quality scenarios
3. Select top priority for evaluation / testing
4. Develop + formulate Fitness Functions
6. Visualize results
Continuouslye.g. every sprint
Add new, change or decommission unnecessary Fitness Functions(4., 5., 6.).
Step
Add quality goals à risk driven approach (1.)
Add scenarios(2., 3.)
Summaryq Evolutionary architectures: Small incremental architectural
changesq Direction of change guidance by Fitness Functionsq Fitness Functions can help in environments not using evolutionary
architecture approach
q Fitness Function categorization helps to generate ideas in solutionspace
ÞA lot of possibilities to test architecture!q Examples:
q A / B Testing on code level (libraries, …) incl. performance testing (Scientist.net)q Mini-Chaos engineering (Circuit-Breaker: Polly, Chaos-Testing Tool: Simmy)
Don’t automate (architecture) tests because you can…
… but because it makesmost sense in yourbusiness environment!
Rate today’s session
Session page on conference website O’Reilly Events App
Thank you very much!
Slide download: https://embarc.de/blogDemo source code: https://git.io/JegFX
xing.to/rweiss
www.embarc.de
@renebiancoAny Questions?
Ressources
q Sample source codehttps://git.io/JegFX
q Pollyhttps://github.com/App-vNext/Polly
q Simmyhttps://github.com/Polly-Contrib/Simmy
q Scientist.nethttps://github.com/scientistproject/Scientist.net
q Scientist Case Study (GitHub)https://github.blog/2015-12-15-move-fast/
q Building Evolutionary Architectures. Support Constant Change N. Ford, R. Parsons, P. Kua, Patrick | O'Reilly 2017
q Software Architecture in PracticeChapter 4 (Understanding Quality Attributes, Quality Scenarios)Len Bass, Rick Kazman, Paul Clements | SEI Series in Software Engineering 2012
q Thoughtworks / Fitness Functionhttps://www.thoughtworks.com/search?q=fitness+functions&c=sitewide