the hardest part of microservices: your data - christian posta, red hat
TRANSCRIPT
Full slide deck here: h"p://bit.ly/ceposta-hardest-part
Twitter: @christianposta
Blog: http://blog.christianposta.com
Email: [email protected]
Christian Posta Principal Architect – Red Hat
• Author “Microservices for Java Developers”
• Committer/contributor Apache Camel, Apache ActiveMQ,
Fabric8.io, Apache Kafka, Debezium.io, et. al.
• Worked with large Microservices, web-scale, unicorn company
• Blogger, speaker about DevOps, integration, and microservices
PeopletrytocopyNet,lix,buttheycanonlycopywhattheysee.Theycopytheresults,nottheprocess.
AdrianCockcro,,formerChiefCloudArchitect,Ne6lix
“Microservices” is about optimizing … for speed.
Howdoesyourcompanygofast?
Managedependencies.
Wait.Whatisdata?
Whatisone“thing”?
Bookcheckout/purchase TitleSearch
RecommendaBons
WeeklyreporBng
Focusondomainmodels,notdatamodels
• Break things into smaller,
understandable models
• Surround a model and its
“context” with an explicit
boundary
• Implement the model in code
or get a new model
• Explicitly map between
different contexts
• Model transactional
boundaries as aggregates
SBckwiththeseconveniencesaslongasyoucan.Seriously.
But...
• Load/sizeistoogreattofitononebox
• Modules/usecaseshavedifferentread/writecharacterisBcs
• Queries/joinsaregeOngtoocomplex
• Securityissues
• LotsofconflicBngchangestothemodel/schema
• Needdenormalized,opBmizedindexingengines
• Wewanttoexplicitlyreducedependenciesondatabetween
ourservices
Fromhereonout,whatwe’resayingis“thankyouoldwork-horsedatabase,we’vegotitfromhere”…
“Amicroservicehasitsowndatabase”
We’renowbuildingafull-fledgeddata-centricdistributedsystem.Somethingstoremember…
Planforfailures.
BuildconceptsofBme,delay,
network,andfailuresintothe
designasafirst-classciBzen.
h"ps://secure.phabricator.com/book/phabcontrib/arBcle/n_plus_one/
h"ps://secure.phabricator.com/book/phabcontrib/arBcle/n_plus_one/
getBulkHats()
getBulkHatsForCatsExcept()
wellReallyIJustWantCertainHats()
justExecuteThisSqlForMe()
Weneed“consistency”.Butweexpectfailures.ThisisstarBngtosoundlikeCAP…
Consistencymodels…
h"ps://en.wikipedia.org/wiki/Consistency_model
• Strict consistency (Linearizability) • Sequential consistency • Causal consistency • Processor consistency • PRAM consistency (FIFO) • Bounded staleness consistency • Monotonic read consistency • Monotonic write consistency • Read your writes consistency • Eventual consistency
ReplicatedDataConsistencyExplainedthroughBaseball(DougTerry)
h"ps://www.microsoa.com/en-us/research/publicaBon/replicated-data-consistency-explained-through-baseball/
• What consistency model do you need, depending on what role you’re playing?
• What consistency model are you willing to pay for? • Official score keeper? (Linearizability or RMW) • Umpire? (Linearizability) • Sports writer? (Bounded staleness, Eventual
consistency) • Radio updates? (Monotonic read, Bounded staleness) • Statistician (Bounded staleness) • Friends in the pub (Eventual consistency)
MaybewecanusearelaxedconsistencymodelforsomeofthosepreviouslymenBonedusecases…
...andsolvefordata-sharingissueswhiletakinginto
accountthenetworkandfailures.
ExampleusingsequenBalconsistency…
Whatwe’vedoneisgoneoffandbuiltadatasystemattheapplicaBonlayer.
Andthisiswhattheinternetcompaniesdidalso.(someevenopensource!!)
• Yelp – MySQL Streamer
https://github.com/Yelp/mysql_streamer
• LinkedIn – Databus
https://github.com/linkedin/databus
• Zendesk – Maxwell
https://github.com/zendesk/maxwell
Meetdebezium.io
Meetdebezium.io
Twitter: @christianposta
Blog: http://blog.christianposta.com
Email: [email protected]
Thanks for listening! Time for demo?
Full slide deck here: h"p://bit.ly/ceposta-hardest-part