Download - Microservices .NET Core
Microservices .NET Core
About us
Piotr Gankiewicz Dariusz Pawlukiewicz
Microsoft MVP, Bottega IT Minds trainer, Software Engineer, IT Consultant, Open Source contributor.
Microsoft MVP, Bottega IT Minds trainer, Software Engineer, IT Consultant, Open Source contributor.
Distributed systemsIntroduction
Monolith vs microservicesusers, orders,
deliveries, discounts, etc.
API Gateway
Domain
DAL
DB
APIDomain
DAL
DB
APIDomain
DAL
DB
APIDomain
DAL
DB
APIDomain
DAL
DB
Users
Orders
Discounts
Deliveries
Monolith vs microservices
RepositorySingle
Monolith Microservices
Multiple
TechnologySingle Multiple
DomainWhole Partial
DeploymentSingle Multiple
SPOFYES NO
MaintenanceEasy Difficult
CommunicationProcess
Monolith Microservices
Network
IntegrationSynchronous Asynchronous
ConsistencyImmediate Eventual
TestabilityEasy Difficult
ScalabilityVertical Horizontal
Applicability~90% ~10%
Systems division
Monolith
Modular monolith
Distributed monolith
Microservices
Distribution
Modularity
API
Services
Domain
Users
API
CRUD
Orders
API
Port
Adapter
Deliveries
API
Domain
DAL
Discounts
Modular monolith
DB
DAL
DB
Domain
DB DB
DTO DTO DTO
Summary● There are no precise measures of microservices (e.g. 2 weeks to rewrite it)
● Services should be autonomous (repository, code, contracts, CI & CD)
● Independence and scalability are one of the core advantages of microservices
● Polyglot programming - beware of that freedom due to maintainability
● Network partitioning is one of the key challenges in distributed systems
● Data consistency is mostly the business decision
● Distributed transactions, testing - a set of whole new problems/challenges
● Infrastructure and DevOps are critical when working with microservices
● Modular monolith - good starting point before separating microservices
What is Pacco?
● The exclusive package delivery system
● Users might send parcels containing guns, money, jewelry etc.
● Delivery is handled by the selected courier at a given date
● There’s a limited number of couriers (availability archetype)
● Customers with the higher priority (e.g. VIP) may expropriate other ones
● The availability service will be responsible for the resource reservations
Clean architecture
Clean architecture
Core
● Focuses on modelling (proper OOP) of the business entities and their behaviours.
● Free from all of the infrastructural implementation details (DB, mailing, authentication etc.) - instead, defines the abstractions to be implemented in the other layers.
● An area of the “pure functions”, where behaviour of particular objects is fully deterministic and depends only on the input parameters.
Aggregates (Entities + VO)
Exceptions
Repositories (abstractions)
Domain events
Clean architecture
App
● Modelling the particular use cases (e.g. with the usage of CQRS).
● Makes use of e.g. implemented repositories and domain models in order to orchestrate application logic.
● May define its own abstractions (ports).
● Does not contain business logic - it does belong to the domain layer.
Exceptions
Integration events
DTO
CQRS
Clean architecture
Infra
DB client
Mailing integration
Message broker
etc...
● Contains adapters (implementation of the ports).
● Cross-cutting concerns - persistence, logging, security etc.
● Might be highly modularized (e.g. separate project per infrastructural concern)
Summary
● Adjust the architecture to the application, not the other way around● Avoid over-engineering (DDD + CQRS + ES for everything)● Central layer (Core) should not have any dependencies● Aggregate defines transactional boundaries and object consistency● Keep the business logic in the domain layer● Do not allow the domain leakage (beyond the application layer boundaries)● Violation of the business logic or invariants should throw an exception● Domain events might help with decoupling the responsibilities● Business use cases should be kept in an application layer● Use the proper abstractions (ports) to define the possible dependencies● Persistence, communication etc. - adapters are part of the infrastructure
CQRSCommand Query Responsibility Segregation
Command Query Separation
Method
Query Command
Returns the data
Doesn’t mutate state
Doesn’t return data
Mutates the state
IdempotentMicro-level of
code separation
For example IEnumerable<T>
Micro-level of code separation
Macro-level of code separation (architecture)
MethodObject
Command Query Responsibility Segregation
Query Command
Returns the data
Doesn’t mutate state
Doesn’t return data
Mutates the state
Idempotent
WRITE SIDE
API GATEWAY
READ SIDE
WRITE DB READ DB
Events
Command Query
Event Sourcing
ID AccountID Balance LastUpdate
1 1287654 1000.00 12-12-2012 21:38
2 0986778 23.89 12-12-2012 12:00
AccountCreated
Balance Changed
Balance Changed
CQRS
ES
Required Not required
Summary
● CQS - divide the methods to the ones writing and fetching the data
● Ensure the idempotency of the operations retrieving the data (HTTP GET)
● Changing the system state (side-effects), should not return the data
● CQRS - handling the commands and queries (separate objects, SRP)
● Model might be splitted into 2 separate ones: write (domain), read (DTO)
● CQRS does not require Event Sourcing
● Unique ID generation might be accomplished with GUID, Snowflake ID etc.
Message exchange (direct)
PaymentsService
DeliveriesService
OrdersService
DeliveryStarted
OrderCreated
OrderCreated
PaymentCompleted
Integration via events (EDA)
Message broker
PaymentsService
OrderCreated
DeliveriesService
OrdersService
OrderCreated OrderCreated
Message brokerIntroduction to RabbitMQ.
RabbitMQ
● An open message broker, supporting e.g. AMQP● Started in 2007, implemented in Erlang language● Supported by most of the technologies (.NET, Java, Python, PHP, JS etc.)● Messages can be exchanged using the binary serialization● Exchange is a set of routing rules, that redirects the messages to queue● Routing is based on the so-called routing key, depending on the exchange● There are 4 network topologies: direct, topic, fanout, headers● Messages might contain additional metadata and headers (like in HTTP)● Queues might be durable, and the broker can persist the data e.g. in DB
Exchange types in RabbitMQ
Exchange
Polish_Queue
General_Queue
routing_key: queue.1.pl
binding: queue.*.pl
binding: queue.#
Message delivery mode
Message broker
OrderCreated
DeliveriesService
OrdersService
OrderCreated
ACK
● At most once delivery - message might get lost.● At least once delivery - message might be sent again.● MessageId - the unique message identifier, sent along with each message.
Exactly-once processing
Message broker
CreateOrder
DeliveriesService
MessageHandler
CreateOrder
UniqueMessages
CreateOrder
Entities Messages
EXISTS messageId
ACK
OrderCreated
Outbox pattern
Message broker
OrderCreated
DeliveriesService
MessageHandler
OrderCreated
ACK
Entities Outbox
INSERT order
OutboxProcessor
OrderCreated
OrderCreated
Summary
● Message exchange is being handled by the infrastructure component● Message broker allows to isolate the microservices from each other● Regardless of microservices being offline - messages will be delivered● All of the instances of particular microservices do share a common queue● Flexible serialization schema allow sending any type of data● Message count * size might slow down the queue● There are 4 network topologies (“topic” is the default one)● Message persistence reflects on the overall performance due to I/O● “At least once delivery” requires the proper handling of messages● Outbox pattern gives us a certainty of messages being always sent
API GatewaySynchronous vs. asynchronous gateway.
API Gateway (synchronous)
API
RabbitMQ
OrdersService
HTTPCreateOrder
DeliveriesService
OrderCreated
HTTP 201
OrderCreated
API Gateway (asynchronous)
APIHTTP 202
RabbitMQ
OrdersService
DeliveriesServiceOrderCreated
OrderCreated
CreateOrder
CreateOrder
CorrelationContext as message metadata
routing_key: queue.1.pl,correlationContext: { “id”: “d037e1e4-127e-41ae-a75c-7deeea266e2b”, “userId” : “0b9f0485-33fc-4d0a-871e-2e2b83a7ee6c”, “resource” : “orders” ….}
● CorrelationId - common ID binding the messages
● Created in API Gateway as an immutable object
● Contains the data that might be useful in particular services (cross-cutting concerns), yet we don’t want to put them directly into contracts definitions
● Is part of the flow, meaning that it’s never changed during the whole messages chain
Informing end-user about operation state
API RabbitMQ
OrdersService
Request-ID: 234e1da7781d421282733eebd671c84a
CreateOrder
OperationsService
OrderCreated
WS: Operation Updated
Request-ID: 234e1da7781d421282733eebd671c84aStatus: PendingRequest-ID: 234e1da7781d421282733eebd671c84aStatus: Completed
Common CorrelationId
HTTP 202
Summary
● API Gateway acts as a facade and an entrypoint to the overall system● Gateway is usually aware about the internal microservices ● API is responsible for the things like:
○ Establishing a common communication protocol○ Proper traffic redirection (e.g. typical HTTP request-response)○ Retry and other patterns making our solution resilient○ Load balancing○ Data transformation and aggregation
● Redirecting an incoming HTTP request to the queue implies asynchronicity● Asynchronous requests should have a reply channel (e.g. WebSockets)
An issue with services communication via HTTP
Service A Service B
{“host” : “locahost”,“port” : 5001,“scheme” : “http”}
appsettings.json
GET http://localhost:5001/values
dynamic* const
{“host” : “app.my”,“port” : 443,“scheme” : “https”}
var response = _client.GetAsync<Response>(“http://localhost:5001/values”)
appsettings.prod.json
Service DiscoveryWith the usage of Consul and Fabio.
Service Discovery with Consul
Service A Service B{ “name” : “service-b”, “ID” : “uniqueId”, “address” : “http://localhost”, “port”: 5001}
service-b
GET http://localhost:5001/values
Service Discovery with Consul and Fabio
Service A
Service B
{ “name” : “service-b”, “ID” : “uniqueId”, “address” : “http://localhost”, “port”: 5001, “tags”: [“url-prefix-values”]}
GET http://localhost:5001/values
values | http://localhost:5001/values
GET http://localhost:9999/values
Routing table
Summary
● HTTP communication results in the temporal coupling● The data is always up to date (which is not always true for async events)● Microservices needs to be aware of their existence (and endpoints)● Retry, circuit breaker - such patterns might make our apps more resilient● In a dynamic environment, we want to use the service names / DNS● Service registry keeps track of the existing microservices instances● Service discovery allows to find available instances of microservices● Load balancing might work either in server side or client side mode● Orchestration engines have their own registries and load balancers.
Distributed business transactionsIn the world of microservices.
Distributed transaction
API
Vehicles service
Hotels service
Flights serviceConfirmation, only if all of the steps succeed.
Otherwise, drop the reservation
and release resources.
Two phase commit (2PC)
Coordinator
Vehicles service
Hotels service
Flights service
Lock the resources at a given time.
2PC - unhappy path
Coordinator
Vehicles service
Hotels service
Flights service
At least one service didn’t send the confirmation.
Release the resources.
2PC - happy path
Coordinator
Vehicles service
Hotels service
Flights service
All of the services sent confirmations. Commit
the transaction.
Two phase commit (2PC)
● Easy to understand● Centralized business process manager● Synchronous communication
● Requires the additional code● Overall processing time depends on the slowest node● Does lock the resources
Event choreography
API
Vehiclesservice
Hotels service
Flights service
Send compensating messages
BookFlight (C)
FlightBooked (E)
HotelBooked (E)CancelHotel (C)
CancelFlight (C)
Vehicles service
Event choreography
● Easy to understand● Does not require an additional code● Fits into the Event-Driven Architecture
● No centralized place to manage the distributed process● Might result in “spaghetti” dependencies● Difficult to test
StateNo state
Saga Process Manager
Has current state
DataNo data Has the data
ScopeCompensation mechanism
Process management
Saga pattern
API
Vehiclesservice
Hotels service
Flights service
BookTrip (C)
FlightBooked (E)
HotelBooked (E)
Vehicles service
Saga
BookHotel (C)
BookVehicle (C) BookVehicleRejected (E)
Saga
Send compensating messages
CancelHotel (C)
BookFlight (C)CancelFlight (C)
Saga pattern
● Easy to understand● Centralized business process manager● Fits into the Event-Driven Architecture● “Tangible” transaction (logs)
TestsTesting the distributed applications.
Contract Testing (CDC) with PACT
Service A(consumer)
Service B(provider)
Service B(MOCK)
{ id: 1, Name: “Product”, Size: “huge”}
{ consumer: “serviceA”, provider: “serviceB”, interactions:[{ description: “GET to retrieve product”, request : { method: “GET”, path: “/parcels/1” }, response: { status: 200, body: {
id: 1, Name: “Product”, Size: “huge” } }}] }
http:localhost:9222/parcels/1
~pacts/serviceA-serviceB.jsonOn the consumer’s side the tests always
pass.
http:localhost:5003/parcels/1
On the provider’s side the tests will pass only if the
response has 100% match with the
expectations defined in the pact
ObservabilityLogging, monitoring and distributed tracing.
MetricsMonitoring the services with Prometheus and Grafana.
Logs vs. Metrics
LOGS
REACTIVE
TEXT
“DRILL DOWN”
METRICS
PROACTIVE
NUMERIC
WHOLE APP/ENV
“Processed a message with id: 1” { "value": 51683328, "name": "Memory", "unit": "bytes" }
How to efficiently monitor an app?
DEFINE THE METRICS
QUERY THE METRICS
VISUALIZE THE QUERIES
How to efficiently monitor an app?
APP
http://host:port/metricshttp_requests_total{method="post",code="400"} 3
Summary
● Centralized logs are a good starting point for system diagnostics. They give precise information about the processes taking place in a particular service.
● Metrics are countable information. They can help monitor the entire application and prevent developers from crashing the system (e.g. by informing about leaking memory in advance).
● Prometheus enables simple structuring of metrics using labels.
● PromQL is the language used to create metric projections.
Distributed tracingWith OpenTracing and Jaeger.
Fundamentals of distributed tracing
SPAN - UNIT OF WORK
START STOP
CHILD
START STOP
FOLLOWS
START STOP
Structure of Span object
CONTEXT
TRACE ID
SPAN ID
BAGGAGE
PROCESS NAME
TAG
LOG
TAG TAG
LOG
LOG
How to efficiently trace an app?
DEFINE THE SPANS
QUERY THE SPANS
VISUALIZE THE QUERIES
How does Jaeger work?
https://www.jaegertracing.io/docs/1.9/architecture/
Summary
● Span is a logical unit used for distributed tracing. It is a kind of Unit of Work, which has time-defined boundaries and additional metadata (logs, tags, baggage).
● Spans can be in one of two relationships: CHILD and FOLLOWS FROM.
● Open Tracing provides a full mechanism of span instrumentation within our application. For span reporting, a tracer is used, which consists of a sampler (with sampling heuristics selected) and a reporter (by default, reporting after UDP).
● Jaeger provides the complete infrastructure needed for effective span analysis.
SecurityHardening the microservices.
Secure configuration
Configuration Store
Service B
Service C
Service A
Decrypt service X appsettings on the fly.
ConfigureApp()
ConfigureApp()
ConfigureApp() Override appsettings
Override appsettings
Override appsettings
Dynamic credentials
Credentials Store
Service AGenerateCredentials()
[connectionString]
Database
OpenConnection(connectionString)
ExtendLease()
Lease the dynamic credentials with expiry.
Allow opening the database connection.
PKI certificates rotation
Certificate Authority
Service AIssueCertificate()
[X.509 certificate]
GetData(header: X.509 RawData())
IssueCertificate()
Issue the revokable X.509 certificate.
Service B
Validate the certificate contents and expiry.
Authentication JWT
IdentityService
Service A
Issue the expirable JSON Web Token.
SignIn()
[JWT]
GetData(header: Authorization: Bearer [JWT])
Validate JWT (issuer, expiry, signature etc.)
DevOpsBuilding, deploying and orchestrating applications.
What is Service Mesh?
k8s cluster
Ordersservice
PaymentsserviceSidecar Sidecar
pod pod