![Page 1: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/1.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
From C to Q, one Event at a time
1
Lorenzo Nicora OpenCredo
Event Sourcing illustrated
![Page 2: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/2.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
Lorenzo Nicora @nicusX
[email protected] https://opencredo.com/author/lorenzo/
Senior Consultant @ OpenCredo • Microservices • Cloud • Event-driven, Event Sourcing, Reactive • Java, Spring, Akka…
2
![Page 3: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/3.jpg)
3
Concepts from DDD
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
Aggregate
Event Sourcing,CQRS => DDD/
State: f(t)
Domain Event
![Page 4: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/4.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 4
Once upon a time…
Everything was Synchronous
Request <-> Response
![Page 5: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/5.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 5
Scaling out…
Updates —> Locks —> Contention!<— Block! <—
-> Distributed still Synchronous
![Page 6: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/6.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 6
Let’s go Asynchronous
-> Distributed & Message-drivenRequest/Response ACID Transactions
• Distributed: Propagation takes time • Async: No global order strictly guaranteed
Updates applied in the wrong order => Inconsistent state!
![Page 7: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/7.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 7
Command Sourcing💡
• Append Only —> No Contention, No Update • Build State from Commands history
K/V Store Distributed
![Page 8: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/8.jpg)
Command “Submit this Order! (Please)” ->A request (imperative sentence) ->May fail, be rejected ->May affect multiple Aggregates
8
Commands
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
✉
Rebuild Aggregate State from Commands
![Page 9: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/9.jpg)
9
Events
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
Event “Order submitted” -> Statement of facts (past tense) ->Never fails ->Can’t be changed ->May be designed to affect a single Aggregate
✉
Rebuild Aggregate State from Events
![Page 10: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/10.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 10
Command > Event Sourcing💡
![Page 11: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/11.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 11
Commands to Events
X
Y
Z?
![Page 12: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/12.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 12
(Business) Consistency
(ACID) Transactions
Eventual (Business) Consistency
Guess —> Compensate —> Apologies
![Page 13: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/13.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 13
e.g. Saga
StatefulOut of band
Corrective Action(Command / Event)
Saga
![Page 14: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/14.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
–Greg Young
“Accountants don’t use pencils.
They use pens”
Corrective Actions
14
![Page 15: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/15.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 15
Benefits of Event Sourcing
History (for free)
Rebuild State at a point in Time
![Page 16: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/16.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 16
Benefits of Event Sourcing
Easier Eventual Business Consistency
—> Corrective Events
Robust to data corruption (bugs, malicious, fat fingers…)
![Page 17: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/17.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 17
Benefits of Event Sourcing
Horizontal Scalability &
Low Latency writes
-> Distributed systems & data store
—> Append-only Log
—> Asynchronous processing
![Page 18: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/18.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 18
Commands to Events
Stateful: Events depends 0n current State
Stateless: Validate, split by Aggregate…
Depends on Business Domain
x Point of synchronisation? Out-of-order commands (IoT, Mobile)
![Page 19: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/19.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 19
Thinking fast or slow
Stateless: Think fast,
Write fast, More thinking later
Stateful: Think slow…(rebuild state),
Write fast, Less thinking later
![Page 20: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/20.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
What about reads?
20
![Page 21: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/21.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 21
Retrieving the State
How do I retrieve the State? “Get details of Order ‘AB123’”
❔
not very efficient, but… …may work
![Page 22: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/22.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 22
Querying (Searching) the State❓How do query the State?
“Get all Orders delivered to ‘SE1 0NZ’”
![Page 23: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/23.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 23
CQRS
Command Query Responsibility Segregation
💡
Separate • Code • muService • Datastore
-> Update -—> Retrieve
![Page 24: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/24.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 24
Not a new idea
Specialised
Downstream
![Page 25: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/25.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 25
CQRS and Event Sourcing
for low-latenc
y writes
Event Sourcing => CQRS
![Page 26: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/26.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 26
Materialised Views
• In MemoryK/V Store Graph DB RDBMS
• Rebuildable from Events
💡a.k.a. Read-optimised Views Read Views Projections…
![Page 27: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/27.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 27
Materialised Views
* Views are optimised for specific query use cases
—> multiple Views from same Events
* May be updated asynchronously + low latency writes, scalability - delayed + may reorder Events
![Page 28: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/28.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 28
Materialised Views
The Event Log is your Source of Truth
* Easy to evolve or fix —> change or fix logic; rebuild view from events
![Page 29: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/29.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 29
Indexes
• Search EnginesK/V Stores Graph DB
+ Optimised for querying (less for retrieving) + Reduced delay of State
💡
![Page 30: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/30.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 30
Hybrid solutions: e.g. Snapshots
+ Speed up rebuilding the State
+ Use recent Events to rebuild up-to-date
💡Long delayed
![Page 31: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/31.jpg)
Distributed + Asynchronous system Distributed ACID Transactions (2PC)
31
No Global Current State
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
AS
SA
B
Information propagates at a finite speed
![Page 32: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/32.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 32
Lessonfrom the Trenches
![Page 33: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/33.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 33
Lesson Learned #1
If you put data in…
…you will eventuallyhave to get them out!
The “Query” side is not secondary
![Page 34: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/34.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 34
Lessons Learned #2
In old days:normalising one DB
to support as many queries as possible
With CQRS (also No SQL in general)
multiple denormalised “data stores” optimised for different queries
![Page 35: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/35.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 35
Lessons Learned #3
Centralised Event Store= Event-sourced monolith
Each microservice owns its state (Event Store)
Microservices:
![Page 36: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/36.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
+++ Summing up +++
36
![Page 37: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/37.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 37
ES/CQRS Optimal Use Cases
• High Volume, Low Latency writes
• Out-of-order Commands (IoT, Mobile)
Event Sourcing + CQRS😋
Domain: stateless Command-> Event
![Page 38: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/38.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 38
ES/CQRS Drawbacks
x No “One-Size-Fits-All”
—> Multiple “Q” implementations
x Delayed reads
x No ACID Transactions
x Additional complexity (!!!)
🙁
![Page 39: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/39.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 39
ES/CQRS Benefits
+ No “One-Size-Fits-All” —> “Q” optimised per use cases
+ Distributed systems (Microservices)
+ Eventual (Business) Consistency
+ History, Temporal queries
+ Robust to data corruption
😀
![Page 40: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/40.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
That’s all, Folks!
40
![Page 41: From C to Q one event at a time: Event Sourcing illustrated](https://reader031.vdocuments.mx/reader031/viewer/2022030313/58cefcff1a28abab738b5e11/html5/thumbnails/41.jpg)
Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX
??? Questions ???
41
Thanks.
⏳