Download - More than Data
![Page 1: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/1.jpg)
MORE THAN DATAUsing Riak Core to Manage Distributed
Services
![Page 2: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/2.jpg)
Who Am I?
Miles O’Connell
@milesoc (twitter/github)
Software Engineer at StackMob
![Page 3: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/3.jpg)
Backend-As-A-Service
• Data API
• Push
• Third Party Marketplace
• etc.
![Page 4: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/4.jpg)
Backend-As-A-Service
• Data API
• Push
• Third Party Marketplace
• etc.
Service Oriented Architecture (SOA)
![Page 5: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/5.jpg)
Service Oriented Architecture
Creative Commons License – Flickr/Jared Tarbell
![Page 6: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/6.jpg)
CAP Theorem
• Consistency
• Availability
• Partition Tolerance
![Page 7: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/7.jpg)
Service Oriented Architecture
• Versioning (“Consistency”)
• Availability
• Partition Tolerance
![Page 8: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/8.jpg)
Service Oriented Architecture
Service A Service B
Service C Service D
![Page 9: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/9.jpg)
Service Oriented Architecture
Service A Service B
Service C Service D
Service B
(v2)
Service D
(v2)
![Page 10: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/10.jpg)
Service Oriented Architecture
Service A Service B
Service C Service D
Service B
(v2)
Service D
(v2)
![Page 11: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/11.jpg)
Service Oriented Architecture
Service A
Service C
Service B
(v2)
Service D
(v2)
![Page 12: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/12.jpg)
Service Oriented Architecture
• Versioning (“Consistency”)
• Availability
• Partition Tolerance
![Page 13: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/13.jpg)
Custom Code
Data API Push API Versioning
3rd Partyetc.
etc.etc.
![Page 14: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/14.jpg)
Custom Code
Data API Push API Versioning
3rd Partyetc.
etc.etc.
Custom Code
![Page 15: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/15.jpg)
Custom Code
Data API Push API Versioning
3rd Partyetc.
etc.etc.
![Page 16: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/16.jpg)
Service Oriented Architecture
• Versioning (“Consistency”)
• Availability
• Partition Tolerance
![Page 17: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/17.jpg)
Service Oriented Architecture
• Versioning (“Consistency”)
• Availability
• Partition Tolerance
• Automated
![Page 18: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/18.jpg)
Riak Core
• https://github.com/basho/riak_core
• Erlang library for building distributed systems
![Page 19: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/19.jpg)
Riak Core2160 0
![Page 20: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/20.jpg)
Riak Core2160 0
Key
![Page 21: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/21.jpg)
Riak Core
vnode
![Page 22: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/22.jpg)
Riak Core
Physical Server 1
Physical Server 2
Physical Server 3
![Page 23: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/23.jpg)
Riak Core
Physical Server 1
Physical Server 2
Physical Server 3
![Page 24: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/24.jpg)
Riak Core
K, V
Physical Server 1
Physical Server 2
Physical Server 3
![Page 25: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/25.jpg)
Riak Core
K, V
Physical Server 1
Physical Server 2
Physical Server 3
![Page 26: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/26.jpg)
Riak Core
K, V
Physical Server 1
Physical Server 2
Physical Server 3
![Page 27: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/27.jpg)
Riak Core
K, V
Physical Server 1
Physical Server 2
Physical Server 3
![Page 28: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/28.jpg)
Riak Core
K, V
Physical Server 1
Physical Server 2Physical Server 3
![Page 29: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/29.jpg)
Riak Core
K, V
Physical Server 1
Physical Server 2
Physical Server 3
![Page 30: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/30.jpg)
Riak Core
K, V
Physical Server 1
Physical Server 2
Physical Server 3
![Page 31: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/31.jpg)
SOA on Riak Core
RequestService
Instance
Physical Server 1
Physical Server 2
Physical Server 3
![Page 32: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/32.jpg)
SOA on Riak Core
RequestCC
Instance
Physical Server 1
Physical Server 2
Physical Server 3
![Page 33: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/33.jpg)
SOA on Riak Core
RequestCC
Instance
Physical Server 1
Physical Server 2
![Page 34: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/34.jpg)
SOA on Riak Core
Request
CC
Instance
Physical Server 1
Physical Server 2
![Page 35: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/35.jpg)
SOA on Riak Core
Request
Webmachine
Physical Server 1
Physical Server 2
CC
Instance
![Page 36: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/36.jpg)
SOA on Riak Core
Request
Webmachine
proxy_coord
Physical Server 1
Physical Server 2
CC
Instance
![Page 37: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/37.jpg)
SOA on Riak Core
Request
Webmachine
proxy_coord
vnode
Physical Server 1
Physical Server 2
CC
Instance
![Page 38: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/38.jpg)
SOA on Riak Core
Request
Webmachine
proxy_coord
vnode
instance_mgr
Physical Server 1
Physical Server 2
CC
Instance
![Page 39: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/39.jpg)
SOA on Riak Core
Request
Webmachine
proxy_coord
vnode
Physical Server 1
Physical Server 2
proxy_op
instance_mgr
CC
Instance
![Page 40: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/40.jpg)
SOA on Riak Core
Request
Webmachine
proxy_coord
vnode
Physical Server 1
Physical Server 2
proxy_op
instance_mgr
CC
Instance
![Page 41: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/41.jpg)
SOA on Riak Core
Request
Webmachine
proxy_coord
vnode
Physical Server 1
Physical Server 2
proxy_op
instance_mgr
Deploy
deploy_coord
deploy_op
instance_mgr
CC
Instance
CC
Instance
![Page 42: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/42.jpg)
SOA on Riak Core
RequestCC
Instance
Physical Server 1
Physical Server 2
Physical Server 3
![Page 43: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/43.jpg)
Service Oriented Architecture
• Versioning (“Consistency”)
• Availability
• Partition Tolerance
• Automated
![Page 44: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/44.jpg)
SOA on Riak Core - Replication
RequestService
Instance
Service
Instance
Service
Instance
![Page 45: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/45.jpg)
SOA on Riak Core - Replication
• R
• How many replicas must we read
• W
• How many replicas must we write
• N
• How many replicas should exist
![Page 46: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/46.jpg)
Riak Core - Replication
K, V
![Page 47: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/47.jpg)
Riak Core - Replication
K, V
Physical Server 1
Physical Server 2
Physical Server 3
![Page 48: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/48.jpg)
SOA on Riak Core - Replication
Request
Physical Server 1
Physical Server 2
Physical Server 3
Service
Instance
Service
Instance
Service
Instance
![Page 49: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/49.jpg)
SOA on Riak Core - Replication
RequestService
Instance
Service
Instance
Physical Server 1
Physical Server 2
Physical Server 3
Service
Instance
![Page 50: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/50.jpg)
SOA on Riak Core - Replication
RequestService
Instance
Service
Instance
Service
Instance
Physical Server 1
Physical Server 2
Physical Server 3
![Page 51: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/51.jpg)
SOA on Riak Core - Replication
Request
Service
Instance
Service
Instance
Service
Instance
Physical Server 1
Physical Server 2
Physical Server 3
![Page 52: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/52.jpg)
SOA on Riak Core - Replication
Request
Service
Instance
Service
Instance
Service
Instance
Physical Server 1
Physical Server 2
Physical Server 3
![Page 53: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/53.jpg)
SOA on Riak Core - Replication
Request
Service
Instance
Service
Instance
Service
Instance
Physical Server 1
Physical Server 2
Physical Server 3
![Page 54: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/54.jpg)
SOA on Riak Core - Replication
Request
Service
Instance
Service
Instance
Service
Instance
Physical Server 1
Physical Server 2
Physical Server 3
![Page 55: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/55.jpg)
SOA on Riak Core - Replication
Deploy
Physical Server 1
Physical Server 2
Physical Server 3
![Page 56: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/56.jpg)
SOA on Riak Core - Replication
Deploy
Service
Instance
Service
Instance
Service
Instance
Physical Server 1
Physical Server 2
Physical Server 3
![Page 57: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/57.jpg)
SOA on Riak Core
• Versioning (“Consistency”)
• Availability✔
• Partition Tolerance
![Page 58: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/58.jpg)
SOA on Riak Core
• Versioning (“Consistency”)
• Availability✔
• Partition Tolerance
![Page 59: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/59.jpg)
SOA on Riak Core - Handoff
Service
Instance
Service
Instance
Physical Server 1
Physical Server 2
Physical Server 3
![Page 60: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/60.jpg)
SOA on Riak Core - Handoff
Service
Instance
Physical Server 1
Physical Server 2
Physical Server 3
![Page 61: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/61.jpg)
SOA on Riak Core - Handoff
Service
Instance
Physical Server 1
Physical Server 2
Physical Server 3
![Page 62: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/62.jpg)
SOA on Riak Core - Handoff
Service
Instance
Physical Server 1
Physical Server 2
Physical Server 3
![Page 63: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/63.jpg)
SOA on Riak Core - Handoff
Service
Instance
Physical Server 1
Physical Server 2
Physical Server 3
![Page 64: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/64.jpg)
SOA on Riak Core - Handoff
Service
Instance
Physical Server 1
Physical Server 2
Physical Server 3
![Page 65: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/65.jpg)
SOA on Riak Core - Handoff
Service
Instance
Physical Server 1
Physical Server 2
Physical Server 3
![Page 66: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/66.jpg)
SOA on Riak Core - Handoff
Service
Instance
Physical Server 1
Physical Server 2
Physical Server 3
Service
Instance
![Page 67: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/67.jpg)
SOA on Riak Core - Handoff
Service
Instance
Service
Instance
Physical Server 1
Physical Server 2
Physical Server 3
![Page 68: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/68.jpg)
SOA on Riak Core
• Versioning (“Consistency”)
• Availability✔
• Partition Tolerance✔
![Page 69: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/69.jpg)
SOA on Riak Core
• Versioning (“Consistency”)
• Availability✔
• Partition Tolerance✔
![Page 70: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/70.jpg)
SOA on Riak Core - Versioning
Deploy
(v2)
Service
(v1)
Service
(v1)
Service
(v1)
![Page 71: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/71.jpg)
SOA on Riak Core - Versioning
Deploy
(v2)
Service
(v1)
Service
(v1)
Service
(v1)
![Page 72: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/72.jpg)
SOA on Riak Core - Versioning
/{service}/{version}/api/{endpoint}
![Page 73: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/73.jpg)
SOA on Riak Core - Versioning
Request
Service
(v1)
Service
(v1)
Service
(v1)
Bastion
![Page 74: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/74.jpg)
SOA on Riak Core - Versioning
Service B
(v1)
Service B
(v1)
Service A
(v1)
Service A
(v1)
![Page 75: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/75.jpg)
SOA on Riak Core - Versioning
Service B
(v1)
Service B
(v1)
Service A
(v1)
Service A
(v1)
![Page 76: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/76.jpg)
SOA on Riak Core - Versioning
Service B
(v1)
Service B
(v1)
Service A
(v1)
Service A
(v1)
Service A
(V2)
Service A
(V2)
Service B
(V2)
Service B
(V2)
![Page 77: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/77.jpg)
SOA on Riak Core - Versioning
Service B
(v1)
Service B
(v1)
Service A
(v1)
Service A
(v1)
Service A
(V2)
Service A
(V2)
Service B
(V2)
Service B
(V2)
![Page 78: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/78.jpg)
SOA on Riak Core - Versioning
Service B
(v1)
Service B
(v1)
Service A
(v1)
Service A
(v1)
Service A
(V2)
Service A
(V2)
Service B
(V2)
Service B
(V2)
![Page 79: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/79.jpg)
SOA on Riak Core
• Versioning (“Consistency”) ✔?
• Availability✔
• Partition Tolerance✔
![Page 80: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/80.jpg)
SOA on Riak Core - Integration
Three key features
![Page 81: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/81.jpg)
SOA on Riak Core - Integration
Three key features
• Management API (Startup/Shutdown)
![Page 82: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/82.jpg)
SOA on Riak Core - Integration
Three key features
• Management API (Startup/Shutdown)
• Consistent Service Responses
![Page 83: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/83.jpg)
SOA on Riak Core - Integration
Three key features
• Management API (Startup/Shutdown)
• Consistent Service Responses
• Status Check Endpoint
![Page 84: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/84.jpg)
SOA on Riak Core
• Riak Core can work for any distributed system
![Page 85: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/85.jpg)
SOA on Riak Core
• Riak Core can work for any distributed system
• Riak Core provides:
• Availability
• Partition Tolerance
![Page 86: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/86.jpg)
SOA on Riak Core
• Riak Core can work for any distributed system
• Riak Core provides:
• Availability
• Partition Tolerance
• You need external coordination (Consistency)
![Page 87: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/87.jpg)
SOA on Riak Core
• Riak Core can work for any distributed system
• Riak Core provides:
• Availability
• Partition Tolerance
• You need external coordination (Consistency)
• You provide:
• Management API
• Consistent Responses
• Health Check
![Page 88: More than Data](https://reader033.vdocuments.mx/reader033/viewer/2022052508/559c1b211a28ab18598b4839/html5/thumbnails/88.jpg)
Acknowledgements
• Jordan West (GitHub: jrwest)
• Everyone at StackMob