couchbase @ paypal: couchbase connect 2014

Download Couchbase @ Paypal: Couchbase Connect 2014

Post on 24-Jun-2015

920 views

Category:

Data & Analytics

6 download

Embed Size (px)

DESCRIPTION

The talk will illustrate PayPal’s usage of Couchbase to light up two distinct infrastructures of Personalization and Server Side Cookies respectively.PayPal’s ecosystem of customers is vast & diverse. It connects with its customer through Omni-channel – across both Online & Retail – touch-points to provide tailored, adaptive and predictive experiences to customer’s needs . The goal is to deliver right experience to the right user at the right time on the right device through Personalization. The talk will showcase how we have architected our Real time Personalization system with key principals of linear scalability, availability, flexibility and extensibility using Couchbase, which provides it both a distributed cache & persistence to manage identities & profiles. This use case will illustrate on how we separate read and write workloads. On the other hand the Cookie Infrastructure will provide an overview of how to deploy Couchbase in a multi data center scenario. Along the way it will share certain scale, workload patterns and practical tips which might be useful as you plan to adopt it for your enterprise.

TRANSCRIPT

  • 1. Couchbase@PayPalAnil MadanSr. Director of Engineeringamadan@paypal.com 2014 PayPal Inc. All rights reserved.1

2. 100CURRENCIES SUPPORTED155MACTIVE REGISTEREDACCOUNTS203MARKETS OFFER PAYPALEUROPEAN UNIONEUROAUSTRALIANDOLLARCANADIANDOLLARNEW ZEALANDDOLLARHUNGARIANFORINTMALAYSIANRINGGITUNITED KINGDOMPOUNDS STERLINGHONG KONGDOLLARUNITED STATESDOLLARTAIWANNEW DOLLARCHINESERMBSWEDISHKRONASINGAPOREDOLLARPHILIPPINEPESOBRAZILIANREALRUSSIANRUBLENORWEGIANKRONEJAPANESEYENMEXICANPESOTURKISHLIRASWISSFRANCCZECHKORUNAISRAELINEW SHEKELDANISHKRONETHAIBAHTPOLISHZLOTY2 3. $1 in every $6Spent on e-commerce isspent through PayPal.*3 4. Why Couchbase?Data volume Online system ; 300M 1B documents 4-10k size ; 3-10TB total storageData Access Distributed caching PersistenceData Structure Flexible & Schemaless; document basedWorkload characteristics 50% read/50% write; Low latency < 5-10 msecAvailability and scalability Multi data center DR/BCP Linearly Scalable 2014 PayPal Inc. All rights reserved..4 5. Cookie StorePersonalizationUse Cases 2014 PayPal Inc. All rights reserved..5 6. BackgroundProblem Statement 15 years of ungoverned growth and abuse Customer experience impact Performance impact SecurityGoals Reduce cookie payload by 70% Central repository for site cookies 100% Cookie approval < 1 day Availability >= 99.99% 2014 PayPal Inc. All rights reserved..6 7. Problem Statement - Cookie Puzzle 2014 PayPal Inc. All rights reserved..Known Cookies ~3507NodeOthersC++JavaPersistent & PlainTextSession & Plain TextPersistent &EncryptedSession &Encrypted 8. Migration Strategy 2014 PayPal Inc. All rights reserved..AnalyzeRegressionDual-ModeMonitorMigrateDual-Mode Plain Text Encrypted8 9. Functional ViewCustomersFront Tier 2014 PayPal Inc. All rights reserved..ApplicationCookie LibrariesCookieServiceMid TierData TierCouchbase ClientXDCRCouchbase DC A Couchbase DC B9 10. Deployment Model 2014 PayPal Inc. All rights reserved..Active PassiveCookieServiceCookieServiceCookieServiceXDCRActiveReadWriteBirdirectional Unidirectional10 11. Cookie Analyticshttps://github.com/paypal/couchbasekafka 2014 PayPal Inc. All rights reserved..11 12. Cookie Schema 2014 PayPal Inc. All rights reserved..12 13. Cookie Documents - AnalyticsCookieAuditCookieMetrics 2014 PayPal Inc. All rights reserved..KEY : cookieauditValue: {"name": "usersession", //Name of Cookie"encryptedName": "TVKbnJIgoopqWu",//Encrypted name of cookie"lastModified": 1412275511226 //Last modified timestamp}KEY : cookiename_Value: {"cookieName": "ca_-CBr2MVP2, //Cookie nameAppGetMap": {App1": 5, App2: 9, }, //apps reading the cookie.AppPutMap": {App1": 2, App2": 3, }, //apps writing to the cookie"totalGet": 7, //# of reads across all apps"totalPut": 12,//# of writes across all apps"totalKilled": 0, //# killed from browser .}13 14. Cookie Documents - AnalyticsCookieUserSessionMap 2014 PayPal Inc. All rights reserved..KEY : Value: {totalsessions": 2, //# sessions"metadata": {"USERSESSIONKEYS": [ //Map of Server Sessions{"cookieProfileVal": "pp_8d86d0c53d30a39ac3b546f4ffffffbc},{ "cookieProfileVal": "pp_b14fea6c1470a39ac33dwf4ffffff3d}CookieUserMigrationStatusKEY : Cookie Key (UUID)Value: {{"h": {"clen": 105, //Cookie Length"updated": 1388606400000, //Last Updated date/time"brcount": 5, //# of cookies in Browser Side"sercount": 10, //# of cookies migrated to server"brsize": 1000, // Total size of browser cookies"sersize": 3000, //Total size of Server side cookies},} 14 15. Cluster Overview 2014 PayPal Inc. All rights reserved..15 16. Data Buckets 2014 PayPal Inc. All rights reserved..16 17. Cookie StorePersonalizationUse Cases 2014 PayPal Inc. All rights reserved..17 18. PersonalizationPersonalized ExperiencesOnline Retail BeaconMerchants PayPal 3rd Parties 2014 PayPal Inc. All rights reserved..18Touch pointsDataSegmentsScienceModels HyperlocalMobileCustomer JourneysPayPal Merchant Inc 3rd Party 19. Business ProblemConnect with 90MM US Shoppers Connect the right offer to the rightReach Relevancy Redemption 2014 PayPal Inc. All rights reserved..consumerDrive customers to purchase.Connect with over 155MM users viamobile and onlineConnect the right message to theright customerDrive shoppers to purchase andclose the loop19 20. Real Time Personalization Service (RPS)3rd Party User IDSocial Platform IDeBay User IDPayPal User ID 2014 PayPal Inc. All rights reserved..Email AddressHome AddressPhone NumberProfile ID IDFA20 21. Identity & Profile SchemaProfile Record 2014 PayPal Inc. All rights reserved..Match Key ValueMaster ID 123ABC456DEFEmail JSmith@paypal.com,john.smith@gmail.com,Ebay ID 120AS09812DNE0983PayPal ID 03824AD814912NMD1ID Gender HHI Age PayPalStatusCreditStandingAccountLockedPayPalBalance123ABC456DEF Male $75K+25 35Active Approved No 10Identity Record21 22. Document id=550e8400-e29b-41d4-a716-446655440000{"matchKeyData" : [{pguid_1234" : "45564757"}, {eguid_5678" : "45657556"}],"segmentProviders" : [{"name" : paypal,"attributes" : {key : eguid_1234"value: "550e8400-e29b-41d4-a716-446655440000"key : pguid_5678"value: "550e8400-e29b-41d4-a716-446655440000"key : idfa_90"value: "550e8400-e29b-41d4-a716-446655440000""created" : 698465466,"updated" : 698465466,},"segments" : {pp.signup.recency":6579696",pp.bml.standing":Approved","pp.account.locked":4","pp.account.balance":"10"}}, 2014 PayPal Inc. All rights reserved..{"name" : ebay","attributes" : {"created" : 698465466,"updated" : 759669696,},"segments" : {ebay.gender":"1 ,ebay.married:"0,ebay.age_range:2,ebay_hhi:75}}]}Identity & Profile Documents22 23. Cluster Overview 2014 PayPal Inc. All rights reserved..23 24. Data Buckets 2014 PayPal Inc. All rights reserved..24