microservices manchester: concursus - event sourcing evolved by domonic fox
TRANSCRIPT
![Page 1: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/1.jpg)
ConcursusEvent Sourcing Evolved
![Page 2: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/2.jpg)
Introductions
Dominic Fox
Twitter: @dynamic_proxy
Email: [email protected]
Github: http://github.com/poetix
Concursus Github: http://github.com/opencredo/concursus
![Page 3: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/3.jpg)
Agenda
1. From Presence to Presents
2. “Write First, Reason Later”
3. Three processing schedules
4. Concursus
5. Future Directions
![Page 4: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/4.jpg)
Agenda
1. From Presence to Presents
2. “Write First, Reason Later”
3. Three processing schedules
4. Concursus
5. Future Directions
![Page 5: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/5.jpg)
Agenda
1. From Presence to Presents
2. “Write First, Reason Later”
3. Three processing schedules
4. Concursus
5. Future Directions
![Page 6: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/6.jpg)
Agenda
1. From Presence to Presents
2. “Write First, Reason Later”
3. Three processing schedules
4. Concursus
5. Future Directions
![Page 7: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/7.jpg)
1. From Presence to Presents
2. “Write First, Reason Later”
3. Three processing schedules
4. Concursus
5. Future Directions
Agenda
![Page 8: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/8.jpg)
1. From Presence to Presents
2. “Write First, Reason Later”
3. Three processing schedules
4. Concursus
5. Future Directions
Agenda
![Page 9: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/9.jpg)
"I have told people over and over and over again, don't write a
CQRS framework…I can basically guarantee you that it will be
abandonware within one year, like every other one has become.”
- Greg Young, inventor of the term “CQRS”
Why Concursus?
![Page 10: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/10.jpg)
Agenda
1. From Presence to Presents
2. “Write First, Reason Later”
3. Three processing schedules
4. Concursus
5. Future Directions
![Page 11: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/11.jpg)
From Presence to Presents
![Page 12: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/12.jpg)
From Presence to Presents
Aggregate services together into a single container or integrate
them through a single enterprise service bus...
Manage all of our data through a single relational database
schema with global constraints...
Use distributed locks and transactions to make a distributed
system behave as if it were one single system with a global
transactional semantics...
![Page 13: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/13.jpg)
From Presence to Presents
![Page 14: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/14.jpg)
From Presence to Presents
![Page 15: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/15.jpg)
From Presence to Presents
![Page 16: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/16.jpg)
From Presence to Presents“Here the Microservice can become an escape route from reality. Within each Microservice, we can
live on a safe island of determinism and strong consistency — an island where we can live happily
under the illusion that time and the present is absolute.
However, as soon as we exit the boundary of the Microservice we enter a wild ocean of non-
determinism—the world of distributed systems, which is a very different world. You have probably
heard that building distributed systems is hard. It is. That being said it is also the world that gives
us solutions for resilience, elasticity, isolation amongst others. At this point, what we need to do is
not to run back to the monolith, but instead learn how to apply and use the right set of principles,
abstractions and tools in order to manage it.”
- Jonas Bonér, Reactive Microservices Architecture, p. 28-29
![Page 17: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/17.jpg)
From Presence to Presents
![Page 18: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/18.jpg)
Agenda
1. From Presence to Presents
2. “Write First, Reason Later”
3. Three processing schedules
4. Concursus
5. Future Directions
![Page 19: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/19.jpg)
“Write First, Reason Later”
![Page 20: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/20.jpg)
“Write First, Reason Later”
![Page 21: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/21.jpg)
“Write First, Reason Later”
Scene from Endgame, by Samuel Beckett
![Page 22: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/22.jpg)
“Write First, Reason Later”
![Page 23: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/23.jpg)
“Write First, Reason Later”
![Page 24: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/24.jpg)
Agenda
1. From Presence to Presents
2. “Write First, Reason Later”
3. Three processing schedules
4. Concursus
5. Future Directions
![Page 25: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/25.jpg)
Three Processing Schedules
1.Transient
2.Durable
3.Persistent
![Page 26: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/26.jpg)
Three Processing Schedules
1.Transient
2.Durable
3.Persistent
![Page 27: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/27.jpg)
Three Processing Schedules
1.Transient
2.Durable
3.Persistent
![Page 28: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/28.jpg)
Three Processing Schedules
1.Transient - what happens
2.Durable - what’s happening
3.Persistent - what happened
![Page 29: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/29.jpg)
Agenda
1. From Presence to Presents
2. “Write First, Reason Later”
3. Three processing schedules
4. Concursus
5. Future Directions
![Page 30: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/30.jpg)
Domain Model: Events
![Page 31: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/31.jpg)
aggregateType: lightbulbaggregateId: 69016fb5-1d69-4a34-910b-f8ff5c702ad9
eventTimestamp: 2016-03-31T10:31:17.981Zparameters: { “wattage”: 60 }
Domain Model: Events
![Page 32: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/32.jpg)
aggregateType: lightbulbaggregateId: 69016fb5-1d69-4a34-910b-f8ff5c702ad9
eventTimestamp: 2016-03-31T10:36:42.171Zparameters: { “location”: “hallway”}
Domain Model: Events
![Page 33: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/33.jpg)
aggregateType: lightbulbaggregateId: 69016fb5-1d69-4a34-910b-f8ff5c702ad9
eventTimestamp: 2016-03-31T10:36:42.171ZprocessingTimestamp: 2016-03-31T10:36:48.3904Zparameters: { “location”: “hallway”}
Domain Model: Events
![Page 34: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/34.jpg)
Domain Model: Summary
Every Event occurs to an Aggregate, identified by its type and id.
Every Event has an eventTimestamp, generated by the source of the event.
An Event History is a log of Events, ordered by eventTimestamp, with an additional processingTimestampwhich records when the Event was captured.
![Page 35: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/35.jpg)
Network
Event sources
Event processors
Events arrive:• Partitioned• Interleaved• Out-of-order
Processing Model: Ordering
![Page 36: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/36.jpg)
Log is:• Partitioned by aggregate id• Ordered by event timestamp
Processing Model: Ordering
![Page 37: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/37.jpg)
CREATE TABLE IF NOT EXISTS concursus.Event (aggregateType text,aggregateId text,eventTimestamp timestamp,streamId text,processingId timeuuid,name text,version text,parameters map<text, text>,characteristics int,PRIMARY KEY((aggregateType, aggregateId), eventTimestamp,
streamId)) WITH CLUSTERING ORDER BY (eventTimestamp DESC);
Cassandra Schema
![Page 38: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/38.jpg)
CassandraEvent Store
RabbitMQ Topic
DownstreamprocessingLog
events
Publish events
Cassandra & AMQP
![Page 39: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/39.jpg)
CassandraEvent Store
RabbitMQ Topic
Downstreamprocessing
out-of-order events
ordered query results
Cassandra & AMQP
![Page 40: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/40.jpg)
CassandraEvent Store
Kafka Topic
Downstreamprocessing
Event store listener
Publish events
Log events
Cassandra & Kafka
![Page 41: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/41.jpg)
Processing Model: Summary
Events arrive partitioned, interleaved and out-of-order.
Events are sorted into event histories by aggregate type and id.
Events are sorted within event histories by event timestamp, not processing timestamp.
Event consumers need to take into account the possibility that an event history may be incomplete at the time it is read – consider using a watermark to give incoming events time to “settle”.
![Page 42: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/42.jpg)
Programming Model: Core Metaphor
![Page 43: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/43.jpg)
Consumer<Event>
Programming Model: Core Metaphor
![Page 44: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/44.jpg)
You give me a Consumer<Event>, and I send Events to it one at a time:
Emitting Events
![Page 45: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/45.jpg)
I implement Consumer<Event>, and handle Events that are sent to me.
Handling Events
![Page 46: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/46.jpg)
Event-handling middleware is a chain of Consumer<Event>s that transforms, routes, persists and dispatches events. A single event submitted to this chain may be:
■ Serialised to JSON
■ Written to a message queue topic
■ Retrieved from the topic and deserialised
■ Persisted to an event store (e.g. Cassandra)
■ Published to an event handler which maintains a query-optimised view of part of the system
■ Published to an event handler which maintains an index of aggregates by event property values (e.g. lightbulbs by wattage)
Event-Handling Middleware
![Page 47: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/47.jpg)
Java 8 Mapping
![Page 48: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/48.jpg)
Java 8 Mapping
![Page 49: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/49.jpg)
Java 8 Mapping
![Page 50: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/50.jpg)
Querying and Replaying
![Page 51: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/51.jpg)
Querying and Collecting
![Page 52: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/52.jpg)
Kotlin Mapping
![Page 53: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/53.jpg)
Kotlin Mapping
![Page 54: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/54.jpg)
Kotlin Mapping
![Page 55: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/55.jpg)
■ Event type and middleware implementing Consumer<Event>provide basic mechanics for creating, processing and handling events.
■ Java 8 mappings provide a convenient, type-safe method-mapping wrapper around these mechanisms.
■ Kotlin mappings provide an alternative wrapper based on immutable classes.
■ Other mappings are possible – Scala, Clojure, Java with POJOs…
Programming Model: Summary
![Page 56: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/56.jpg)
Agenda
1. From Presence to Presents
2. “Write First, Reason Later”
3. Three processing schedules
4. Concursus
5. Future Directions
![Page 57: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/57.jpg)
Future Directions
1. Stable Kafka integration
![Page 58: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/58.jpg)
Future Directions
1. Stable Kafka integration
2. Avro integration
![Page 59: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/59.jpg)
Future Directions
1. Stable Kafka integration
2. Avro integration
3. End-to-end async programming model
![Page 60: Microservices Manchester: Concursus - Event Sourcing Evolved By Domonic Fox](https://reader034.vdocuments.mx/reader034/viewer/2022051710/5aab55e57f8b9a43728b4c4f/html5/thumbnails/60.jpg)
Questions?