Distributed Transaction in Microservice

Download Distributed Transaction in Microservice

Post on 15-Apr-2017

2.435 views

Category:

Engineering

0 download

Embed Size (px)

TRANSCRIPT

<p>Distributed Transaction in MicroserviceL Minh NghaSolution Architect at Tiki.Vn</p> <p>ContentsDistributed Transaction Problem.CAP TheoremReplicate ModelConsistency and Eventually ConsistencyImplementationCase Study</p> <p>Phn 1. Cc kin thc cn bn</p> <p>Distributed Transaction ProblemL vn i hi m bo tnh nht qun ca d liu trn mi trng phn tnC s tham gia nhiu hn mt data source vo mt qu trnh x l</p> <p>Data SourcesDatabaseServiceWrite</p> <p>CachingMessage QueueExternal ServiceCc data source a dngPhi m bo d liu gia cc ngun nht qun</p> <p>TransactionTransaction phi m bo tnh ACID:Atomic: phi m bo thc thi tt c hoc khng gConsistency: m bo nht qun d liuIsolation: m bo vn tranh chpDuration: d liu phi c lu tr n nh</p> <p>Disitributed Transaction in MicroserviceH thng chia nh thnh cc Microservice rt nhPhi m bo d liu tt c cc service phi nht qunCc service hu ht l cc restful serviceMicroservice l mt h thng phn tn mc caoV c bn: khng c Transaction trong kin trc Microservice!</p> <p>CAP TheoremConsistencyAvalaibilityPartition TolerantMt h thng phn tn ch c th t c 2 trn ba tnh cht trn</p> <p>CAP TheoremConsistency: m bo d liu gia cc node l nht qunAvailabiilty: m bo tnh available ca h thng khi mt s node khng availablePartition Tolerance: m bo kh nng hot ng khi mng gia cc node khng n nh</p> <p>CAP Theorem in MicroserviceNi chung Microservice ch c th m bo hai tnh cht:AvaialabilityPartition ToleranceMicroservice khng m bo consistency gia cc node trn ton b h thng!</p> <p>Consistency vs Eventually ConsistencyConsistency: m bo tnh nht qun ca d liu. Ti mi thi im trn h thng d liu phi nhn nht qun gia tt c cc node.Eventually Consistency: m bo d liu gia cc h thng s nht qun sau mt khong thi gian nht nh.Microservice ch c th m bo d liu l Eventually Consistency</p> <p>Microservicenh i Consistency bng tnh cht Avalaibility v Partition ToleranceKhng consistency nhng phi m bo l eventually consistency gia cc service</p> <p>Phn 2:Gii php</p> <p>Replicate ModelPrimary Data BackupState Machine Model (Active Active Model)</p> <p>Replicate ModelVD: thay i d liu tn kho khi mua bn v xut nhp. Ban u c 10. Bn i 2, nhp vo 5, bn i 7. Cn replicate d liu i cc ngun:Primary Backup: 8, 13, 6State Machine: -2, + 5, -7</p> <p>Replicate ModelLun m bo ghi log cc qu trnh thay iGi cc thay i ng th t v n nh</p> <p>Problems in MicroserviceDatabaseServiceWrite</p> <p>CachingMessage QueueExternal ServiceKhng c bt c cch no m bo vic ghi d liu ng thi nhiu ngun d liu tho mn tnh cht ca mt transaction</p> <p>Problems in MicroserviceCc kch bn li:Ghi d liu nhng khng gi c service khcGi c serivce khc nhng khng ghi c d liu</p> <p>Solutionsm bo ch thay i d liu bt u mt ngun. Khng ng thi thay i d liu ca tt c cc ngun.Tin hnh replicate cc thay i mt cch n nh ti cc ngun chTrin khai h thng log m bo qu trnh replicate l n nh.Mc tiu: m bo d liu gia cc h thng l Eventually Consistency</p> <p>Phn 3: Implementations</p> <p>1. LogLog l phng thc chnh v quan trng nht gii quyt vn replicate d liu trong h thng phn tnXy dng cu trc log, lu tr cc thay i mun gi i v gi bt ng b ti ngun chm bo transation gia business data v log dataXo b log sau khi gi i</p> <p>1. LogServiceBusiness DataLog Data</p> <p>m bo transactionBackGround Worker</p> <p>Other ServiceCachingMessage QueueOther Data Sources</p> <p>2. Event Sourcing Pattern</p> <p>Lu tr cc thay i di dng log event</p> <p>Gi cc event ti cc ngun ch</p> <p>3. Saga PatternXy dng c ch log theo di cc bc thc thiThc hin roll back khi cc bc b li</p> <p>4. CDC Capture Data ChangeBt trc tip cc thay i Database replicate thay i ti cc data source khc nhau. VD:MySQL: decode file binlogMongo: s dng c ch oplog</p> <p>5. Message Queue</p> <p>5. Message QueueS dng message queue l nn tng tch hp bt ng b chnh cho ton b cc serviceLoi b vic gi api cho gia cc serviceTng kh nng availability ca h thngGim thiu chi ph maitain khi ch cn m bo message queue available. Cc Service khng cn available ti cng mt thi im</p> <p>6. Integration Model</p> <p>6. Integration ModelLog:m bo cc d liu cn gi i c lu tr n nh. m bo business data v log data lun c lu tr n nh.Background Worker:m bo qu trnh gi d liu t log ti message queue n nh: chc chn gi c v gi ng th t.Message Queue:m bo gi message ti cc ngun d liu chm bo cc message c delivery ng th t gi vom bo cc message c persistent khng b mtKafka v Windows Service Bus l cc gii php tt nhtCc Data Source khc:Trc tip nhn message t queueCh vic chng trng lp message (Idempotemcy filtering)</p> <p>7. Consistency RequirementsTrong ba yu t CAP, liu AP c th tho mn mi trng hp?Trong trng hp bt buc i hi Consistency cn phi nh gi li m hnh:Liu vic chia service hp l hay cha? Thng thng s khng chia nu d liu bt buc phi strong consistency.C th iu chnh nghip v c khng? C trade off c gia i hi tnh nht qun v kh nng m rng khng?C th trin khai SOAP handle distributed transaction khng?C th ci t two phase commit handle distributed transaction khng?</p> <p>Phn 4 Mt s case study</p> <p>1. t hngKch bn: c hai serivce, catalog service qun l d liu sn phm, order service qun l n hng. Khi t hng th tr i mt lng sn phm nht nh ri to n hng.Risk: c th tr s lng sn phm m khng to c n hng</p> <p>SolutionOrder ServiceCatalog Service</p> <p>1. Gi request gi tn (Sync)Database</p> <p>2. Place OrderMessage Queue</p> <p>3. Gi mt lnh tr tn4. Nu khng nhn c lnh tr tn th sau mt khong thi gian nht nh th rollback li s lng tn.</p> <p>2. Bi ton tr tinKch bn: c hai service l order service v payment service. Payment service cha thng tin tin o ca ngi dng. Tin o tr c th n hng c t.Risk: tin tr m n hng khng t c</p> <p>SolutionsOrder ServiceCatalog Service</p> <p>1. Gi request gi tin (Sync)Database</p> <p>2. Place OrderRequest Queue</p> <p>3. Gi mt lnh tr tin5. Nu khng nhn c lnh tr tin th sau mt khong thi gian nht nh th rollback li tin cho ngi dngResponse Queue</p> <p>4. Gi response</p> <p>3. Bi ton tnh tin gi hngKch bn: gi s mt khch hng mua 2 sn phm tr gi 200k, mi mn 100k. tr trc 150k, hng c chia lm hai gi, giao hai ln. Cn thu khch hng thm 50k. Ln 1 i giao mn th nht, khng phi thu g. tr 100k vo s tin tr. S tin khch hng cn l 50k.Ln 2 i giao nt mn cn li v thu 50k.iu g s xy ra nu ngay mn hng 2 xut giao, th mn hng th nht b giao li v phi cp nht li s tin ca khch hng? Kch bn hp l nht: khch hng khng phi tr thm tin.Nu h thng c chia lm hai service: service qun l hng ti trm v service qun l hng i giao. Trong trng hp ny handle consistency kiu g?</p> <p>AnalysisKh khn:Mun consistency phi gp hai service lm mt &gt; nh hng ti thit k tng th.H thng khng th m bo khi quy trnh khng consistency. VD: nhn vin giao nhn th nht va v th nhn vin th hai i, c hai cn cha kp cp nht qua my tnh.</p> <p>SolutionDng nghip v gii quyt: ghi nhn li v hon tin cho khch hng qua ti khon ngn hng -&gt; v thng thit gi xin li khch hng :(((</p> <p>ConclusionsKhi phn chia h thng phi m bo d liu l eventually consistencyC bn khng c transaction gia cc data source khc nhauKhng ng thi ghi d liu vo nhiu ngun khc nhau.Lu tr log y cc thay i mt cch n nh sang cc data source khcNn s dng message queue l nn tng chnh cho vic tch hp bt ng b gia cc service. Hn ch s dng gi cho api.nh gi vic phn chia service ph hp vi i hi v consistency d liu.</p> <p>Thank you!</p>

Recommended

View more >