Move fast and consumer driven contract test things

Download Move fast and consumer driven contract test things

Post on 21-Jan-2018

60 views

Category:

Software

3 download

TRANSCRIPT

1. Making SoundClouds -services safe(r) to deploy Move Fast and Consumer-Driven- Contract-Test Things XP Days Ukraine 2017 2. 2.5 years @ SoundCloud Monetization Team Backend Engineer @alonpeer 3. About SoundCloud a cloud full of sounds 135M tracks, 175M listeners, 12 hours/minute 300+ services / 50+ teams 4. Agenda Testing, monolith style My first steps @ SoundCloud Introducing contract tests Pactifying our services Caveats QA Q&A 5. Testing, monolith style 6. One service, one client API changes are easy One client team to sync with for migrations API deprecation is easy Testing, monolith style The good ol days 7. Different requirements per client => Code complexity increases Harder to deploy without breaking at least one client Testing, monolith style Mo clients, mo problems 8. More manual QA expensive, slow, prone to human error, doesnt scale More end-to-end tests Maintenance nightmare, flaky, slow, creates bottlenecks Testing, monolith style Mo clients, mo problems 9. My first steps @ SoundCloud 10. My first steps @ SoundCloud Whoa, micro-services 11. My first steps @ SoundCloud Testing strategy E2E Unit Integration 12. My first steps @ SoundCloud Digging into post-mortems ofsmallthings 13. My first steps @ SoundCloud A lack of trust in the acceptance tests caused us to largely ignore the warnings they generated. We couldnt figure out the impact of the broken acceptance tests, and assumed the only problem was the tests themselves, rather than the underlying code. The commit went through as there weren't any tests for the serialisation from the client to the backend. Digging into post-mortems ofsmallthings 14. Introducing contract tests 15. Introducing contract tests My daily routine LockeSteerpike 16. Introducing contract tests My daily routine rickandmorty.com 17. Unit tests are not enough Introducing contract tests #cake >> { result = call(cake) assert(result == lie) } GET /cake: return lie 18. Unit tests are not enough Introducing contract tests GET /cake: return truth #cake >> { result = call(cake) assert(result == lie) } 19. Unit tests are not enough Introducing contract tests import JsonLibFoo GET /cake: return toJson(current_state) // { current_state : lie } import JsonLibFoo #cake >> { result = fromJson(call(cake)) assert(result == lie) } 20. Unit tests are not enough Introducing contract tests import JsonLibBar GET /cake: return toJson(current_state) // { currentState : lie } import JsonLibBar #cake >> { result = fromJson(call(cake)) assert(result == lie) } 21. Introducing contract tests How this works? Consumer Provider End-to-End Test 22. Introducing contract tests How this works? Consumer Provider Unit Test Unit Test 23. Introducing contract tests How this works? Consumer Provider Unit Test Unit Test 24. Introducing contract tests How this works? Requesting /cake. Expecting a JSON response with a key current_state. Asserting deserialization of the response succeeds. import JsonLibFoo GET /cake: return toJson(current_state) // { current_state : lie } Consumer Provider 25. Pactifying our services http://pact.io 26. Whats Pact? Pactifying our services A family of frameworks designed for consumer driven contract testing. Supports JSON over HTTP. Impl. for Java, Scala, Go, Ruby, .Net, Swift, JS etc. Tests run locally, no external dependencies. 27. Whats Pact? Pactifying our services Step 1: Define consumer expectations pact.io 28. Whats Pact? Pactifying our services Step 2: Verify expectations on provider pact.io 29. Consumer: client code (LikesClient) 30. Consumer: client code (LikesClient) 31. Consumer: client code (LikesClient) 32. Consumer: client code (LikesClient) 33. Consumer: client code (LikesClient) 34. Consumer: client code (LikesClient) 35. Consumer: client unit test 36. Consumer: client unit test 37. Consumer: client unit test 38. Consumer: client unit test 39. Consumer: client contract unit test 40. Consumer: client contract unit test 41. Consumer: client contract unit test 42. Consumer: client contract unit test 43. Consumer: client contract unit test 44. Consumer: client contract unit test 45. Consumer: client contract unit test 46. Consumer: client contract unit test 47. Consumer: my-consumer-likes.json 48. Consumer: my-consumer-likes.json 49. Consumer: my-consumer-likes.json 50. Consumer: my-consumer-likes.json 51. Consumer: my-consumer-likes.json 52. Consumer: my-consumer-likes.json 53. Consumer: my-consumer-likes.json 54. Pactifying our services Provider side verification Collect pact files from consumers and verify them all. Keep your provider environment isolated. Dockerized databases. Test doubles for upstream services. 55. Pactifying our services Provider states 56. Pactifying our services Provider states Likes App Likes DB 57. Pactifying our services Provider states Set state: User 1000 has liked 2 tracks Likes App Likes DB 58. Pactifying our services Provider states INSERT likes Set state: User 1000 has liked 2 tracks Likes App Likes DB 59. Pactifying our services Provider states INSERT likes Set state: User 1000 has liked 2 tracks HTTP Request: GET /likes/1000 Likes App Likes DB 60. Pactifying our services Provider states HTTP Request: GET /likes/1000 HTTP Response: 200 OK INSERT likes Set state: User 1000 has liked 2 tracks Likes App Likes DB 61. Pactifying our services Provider states Likes App Likes DB HTTP Response: 200 OK INSERT likes Set state: User 1000 has liked 2 tracks HTTP Request: GET /likes/2000 HTTP Response: 404 Not Found Set state: User 2000 has liked no tracks HTTP Request: GET /likes/1000 DELETE likes 62. Pactifying our services Pact broker Share pact files between projects. API for pact frameworks to fetch all pacts by provider. Support for versioning and tagging. Useful UI and visualization of the network. 63. Pactifying our services Pact broker pact broker 64. Pactifying our services Pact broker pact broker 65. Pactifying our services Pact broker pact broker 66. Pactifying our services Our CI pipeline Push new change Generate consumer contracts Deploy Upload pacts & tag with prod Consumer pipeline 67. Pactifying our services Our CI pipeline Push new change Deploy Upload pacts & tag with prod Consumer pipeline Push new change Verify all prod-tagged pacts Deploy Provider pipeline Generate consumer contracts 68. Communication is key. Caveats 69. Communication is key. New moving parts. Caveats 70. Communication is key. New moving parts. Learning curve per pact framework. Caveats 71. Communication is key. New moving parts. Learning curve per pact framework. Frameworks are WIP. Caveats 72. Communication is key. New moving parts. Learning curve per pact framework. Frameworks are WIP. Provider side setup is time consuming. Caveats 73. Communication is key. New moving parts. Learning curve per pact framework. Frameworks are WIP. Provider side setup is time consuming. Automating consumer-triggered provider verification. Caveats 74. Communication is key. New moving parts. Learning curve per pact framework. Frameworks are WIP. Provider side setup is time consuming. Automating consumer-triggered provider verification. Adoption is essential. Caveats 75. Recap 76. Recap Write contract tests Consum ers 77. Recap Write contract tests Generate pact files Consum ers 78. Recap Write contract tests Generate pact files Publish to broker Consum ers 79. Recap Write contract tests Generate pact files Publish to brokerVerify pacts Consum ers Providers 80. Recap Write contract tests Generate pact files Publish to brokerVerify pacts Consum ers Providers 81. QA Q&A @alonpeer