bbc's graphdb (formerly owlim) aws cloud migration
DESCRIPTION
BBC's presentation for the GraphDB user groupTRANSCRIPT
![Page 1: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/1.jpg)
Moving OWLIMto Amazon Web Services
![Page 2: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/2.jpg)
Moving OWLIMto Amazon Web Services
GraphDB
![Page 3: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/3.jpg)
Introduction to LDP
● Stands for Linked Data Platform● Building APIs and Tools for semantic
publishing at the BBC● Team of 8-12 people (5-8 developers) ● Based in London (moving to Salford)
![Page 4: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/4.jpg)
Introduction to LDP
● Tagging and surfacing content
![Page 5: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/5.jpg)
Why move to AWS?
● Started with taking ownership of Owlim● BBC shift to move to AWS● Loss of key people● Freedom with responsibility
![Page 6: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/6.jpg)
Current Architecture
![Page 7: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/7.jpg)
Cloud Architecture
![Page 8: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/8.jpg)
Keeping in sync
![Page 9: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/9.jpg)
![Page 10: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/10.jpg)
![Page 11: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/11.jpg)
OpsWorks
● Part of AWS● Easily deploy things to EC2
![Page 12: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/12.jpg)
OpsWorks
● Define infrastructure in a stack● The stack is comprised of layers● Each layer has a role (eg. master, slave)● Layers contain a number of instances
![Page 13: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/13.jpg)
![Page 14: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/14.jpg)
![Page 15: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/15.jpg)
![Page 16: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/16.jpg)
![Page 17: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/17.jpg)
OpsWorks – Advantages
● Conceptually, fairly simple.● Entire stack defined as JSON.
○ Can be (and is) version controlled.
● Very easy to clone an entire stack.○ We use this to spin up ‘experimental’ environments.○ Next step: blue/green deployments?
![Page 18: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/18.jpg)
OpsWorks – Disadvantages
● It can be a little slow.● Chef support can be tricky to debug.
![Page 19: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/19.jpg)
![Page 20: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/20.jpg)
● “Infrastructure as code”● Cookbook, made up of recipes.
○ Each recipe installs/configures a specific thing.
● Each layer has a set of recipes that define it.
Chef
![Page 21: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/21.jpg)
include_recipe 'tomcat::default'
cookbook_file 'owlim-production.license' do path File.join(node[:owlim][:basedir], 'owlim.license') mode 0644 owner 'tomcat' group 'tomcat'end
execute 'create_default_cluster_repository' do command SystemRepository.create_repo 'default-cluster' not_if SystemRepository.repo_exists? 'default-cluster' action :nothingend
![Page 22: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/22.jpg)
![Page 23: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/23.jpg)
Lifecycleevents
![Page 24: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/24.jpg)
Recipes for each event type
![Page 25: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/25.jpg)
● A common and well-documented tool.○ And it’s Ruby, a skill we possess.
● Can re-use existing cookbooks.○ eg. Tomcat, Yum.
● Should be platform-agnostic.○ Can theoretically use the same Chef recipes against
any OS, if written well.
Chef – Advantages
![Page 26: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/26.jpg)
● OpsWorks support is a little lacking.○ Using Berkshelf for dependencies is tricky.○ Many of the error messages are a bit cryptic.
Chef – Disadvantages
![Page 27: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/27.jpg)
What next?
![Page 28: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/28.jpg)
What next?
● Autoscaling○ Not supported by OpsWorks.○ For now: predict spikes, use time-based instances.
![Page 29: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/29.jpg)
What next?
● Autoscaling○ Not supported by OpsWorks.○ For now: predict spikes, use time-based instances.
● Backups○ We still do these from our DCs.
![Page 30: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/30.jpg)
What next?
● Autoscaling○ Not supported by OpsWorks.○ For now: predict spikes, use time-based instances.
● Backups○ We still do these from our DCs.
● Chef improvements○ The recipes do too much at the moment.
![Page 31: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/31.jpg)
What next?
● Improve Chef recipes○ The recipes do too much at the moment.○ Example: cluster management.
![Page 32: BBC's GraphDB (formerly Owlim) AWS Cloud Migration](https://reader034.vdocuments.mx/reader034/viewer/2022051323/5481205db37959b55d8b4627/html5/thumbnails/32.jpg)
Questions?