Download - Unlocked London - Technical Track
No 11 cavendish
square
11 july 2013
london
Thursday, July 11, 13
developer VOCAB
Alex brandtsr developer
Thursday, July 11, 13
MODULES
3
Thursday, July 11, 13
LIBRARIES
4
Thursday, July 11, 13
FRAMEWORKS
•The building blocks of an application - a set of common software functions that take the work out writing all code from scratch.
5
Thursday, July 11, 13
6
Thursday, July 11, 13
7
Thursday, July 11, 13
8
Thursday, July 11, 13
LANGUAGES
9
Thursday, July 11, 13
10
Thursday, July 11, 13
APIs
11
Thursday, July 11, 13
APIs
12
Thursday, July 11, 13
What is a data structure?
•images• url
• stars
• *title
• caption
• variants
•small
•medium
•big
•tags
•*_id
•rolestars•*image_id
•*role_id
•roles• *name
• privileges
• login
• description
• location
• child_roles
• _id
13
Thursday, July 11, 13
API tier/Application
STORAGE
APPLICATION
PRESENTATION
14
Thursday, July 11, 13
API tier/Application
STORAGE
APPLICATION
PRESENTATION
APPLICATION/API
14
Thursday, July 11, 13
Bindings
15
Thursday, July 11, 13
Bindings
16
Thursday, July 11, 13
Bindings
17
Thursday, July 11, 13
EVERYTHING IS CODE
wayne wallscloud evangelist
alex brandtsr developer
Thursday, July 11, 13
Infrastructure as code
19
Thursday, July 11, 13
20
Thursday, July 11, 13
21
Thursday, July 11, 13
Chef Server
Chef Workstation
Chef Client
knife
node
Chef Client
node
environment
22
Thursday, July 11, 13
•Nodes: Servers
•Environments: Groups of nodes (Dev/QA/Prod)
•Recipes: Describes the desired state of a node
•Cookbooks: Group of Recipes
•Roles: Group of Cookbooks (Web)
•Attribute: information about the current state of a node
23
Thursday, July 11, 13
24
Thursday, July 11, 13
PuppetCA
PuppetMaster
PuppetAgent
https
PuppetAgent
PuppetAgent
25
Thursday, July 11, 13
•Nodes: Servers
•Environments: Groups of nodes (Dev/QA/Prod)
•Manifests: Describes the desired state of a node
•Catalog: Group of Manifests
•Facts: information about the current state of a node
26
Thursday, July 11, 13
27
Thursday, July 11, 13
SaltMaster
SaltMinion
SaltMinion
SaltMinion
28
Thursday, July 11, 13
•Nodes: Minions
•Environments: Groups of nodes (Dev/QA/Prod)
•States: Describes the desired state of a node
•Grains: information about the current state of a node
29
Thursday, July 11, 13
Thinking like a developer30
Thursday, July 11, 13
scrum Development
Product Backlog
Sprint Backlog Sprint Working increment
of the software
31
1. Design
5. QA 2. Build
4. Release 3. Configure
Thursday, July 11, 13
scrum Development
32
1. Design
5. QA 2. Build
4. Release 3. Configure
Thursday, July 11, 13
TYPES OF DEVELOPMENT
33
Thursday, July 11, 13
TYPES OF DEVELOPMENT
33
Test Driven Development
Thursday, July 11, 13
TYPES OF DEVELOPMENT
33
Test Driven Development
Behavior Driven Development
Thursday, July 11, 13
TYPES OF DEVELOPMENT
33
Test Driven Development
Behavior Driven Development Domain Driven Development
Thursday, July 11, 13
TYPES OF DEVELOPMENT
33
Test Driven Development
Behavior Driven Development
Application Bliss
Domain Driven Development
Thursday, July 11, 13
Model View Controller (MVC)
UPDATES MANIPULATES
SEES USES
USER
MODEL
VIEW CONTROLLER
34
Thursday, July 11, 13
DEv tools
Thursday, July 11, 13
36
Thursday, July 11, 13
37
Thursday, July 11, 13
Gerrit
38
Thursday, July 11, 13
39
Thursday, July 11, 13
CODE REVIEW PROCESS
Thursday, July 11, 13
Our hero, the developer, submits code for review.
The code is reviewed in Gerrit by other developers, who approve the change.
Jenkins runs automated tests against the code change.
Jenkins pushes the code to GitHub so others can build on the change.
41
Thursday, July 11, 13
POLYGLOT PERSISTENCE
wayne wallscloud evangelist
technical track
alex brandtsr developer
Thursday, July 11, 13
A database, in the most general sense, is an organized collection of data.
Database Defined
http://glenn-packer.net
43
Thursday, July 11, 13
ACID
•Atomicity: refers to a database’s ability to either fully process or fully roll back a transaction
•Consistency: database should ensure that all data written therein follows all rules and constraints specified in the database
•Isolation: transactions must be processed securely and independently, without interfering with each other
•Durability: the database must ensure that all committed transactions are saved permanently and cannot be accidentally erased, even in a database crash
44
Thursday, July 11, 13
•A relational database is essentially a group of tables or, to use the technical name, entities.
•Each table is made up of rows (tuples) and columns (attributes).
relationaldatabases
45
Thursday, July 11, 13
Some companies needed something more to accommodate their growing data
rdbms may not be the best fit
46
Thursday, July 11, 13
Enter Non-Relational database
management systems
47
Thursday, July 11, 13
INTRODUCTION TO NOSQL
• NoSQL Database (1998)• NoREL• “Not only SQL”• NoSQL (2009)
48
Thursday, July 11, 13
NoSQL is difficult to define but most NoSQL databases will share common characteristics
1. Non-relational
2. Distributed
3. Open-source
4. Horizontally Scalable
5. Schema-less
NoSQL
49
Thursday, July 11, 13
An “aggregate” is a collection of related objects that we wish to treat as a unit
orders
customers
order lines
credit cards
ID: 1001
line items:
customer: Ann
0321293533
0321601912
0131495054
2
1
1
$48 $48
$39
$51
$39
$51
payment details:
Card: AmexCC #: 12345expires: 04/2001
aggregate
50
Thursday, July 11, 13
There are several different types of data stores that are
considered to be NoSQL
•Key/Value Databases
•Document Databases
•Column-Family Stores
•Graph Databases
51
Thursday, July 11, 13
Key/Value Database•Primarily constructed of aggregates that have a Key or ID that’s used to access the data
•The database cannot see structure within the aggregate
52
Thursday, July 11, 13
Key/Value Database
•Good Use Cases:
• Storing Session Information
• User Profiles & Preferences
• Shopping Cart Data
• Bad Use Cases:
• Data Relationships
•Multi-operation Transactions
•Query by Data
53
Thursday, July 11, 13
•Like a Key-Value database, document databases primarily store and access data using aggregates
•In contrast to the Key-Value database, document data bases can see structure within the aggregate.
•You are able to retrieve the whole aggregate or specific parts of it
document Database
54
Thursday, July 11, 13
document Databases
•Good Use Cases:
• Event Logging
• Blogging Platforms
• Real-time Analytics
• Bad Use Cases:
• Complex Transactions
•Queries against Varying Aggregate Structures
55
Thursday, July 11, 13
• Can be looked at as a two-level aggregate structure
• The first key is often described as a row ID picking up the aggregate of interest
• The second level values of the aggregates are referred to as columns
• You are able to access a row as a whole or operations will allow you to pick out a particular column to get specific data
COLUMN FAMILY
56
Thursday, July 11, 13
COLUMN-FAMILY datastores
•Good Use Cases:
• Event Logging
• Blogging Platforms
• Counters (page visits)
• Expiring Usage
• Bad Use Cases:
• ACID transactions required
• Early prototype applications
57
Thursday, July 11, 13
•Unique compared to most NOSQL types because it was not developed with the intention of scaling horizontally into a cluster
•Data is stored as “nodes” that are connected by edges
•Edges are directional in nature and must be specified
•Queries need a starting point and them will traverse the edges to gather the data requested
graph database
58
Thursday, July 11, 13
Amanda
KNOWS
KNOWS
KNOWS
Robert
Gerry
Lamar
KNOWS
App 1
BUILTBUILT
KNOWS
App 2
USES
BUILT
Christie
KNOWS
Destiny
Jessica
KNOWS
App 3
BUILT
USES
USES
USES
KNOWS
graph database
59
Thursday, July 11, 13
graph databases
•Good Use Cases:
• Connected Data
• Location-based services
•Recommendation engines
• Bad Use Cases:
• Ability to update specific aggregates
60
Thursday, July 11, 13
Companies using nosql
61
DATABASE COMPANIES
Thursday, July 11, 13
THE FUTURE IS POLYGLOT
62
Neo4Jdatabase
Cassandradatabase
Couchdatabase
MongoDBdatabase
RELATIONALDATABASE
Graph-structureddomain rules
Columnar dataaccess with
decentralization
Documentstructures
Document structureswith offlineprocessing
(Actors) (Actors)
(Actors)(Actors)
Asynchronous message passing
MODULE 1
MODULE 2
MODULE 3
MODULE 4
IEEE Software Sept/October 2010 - Debasish Ghosh / Twitter @debasishg
Thursday, July 11, 13
63
POLYGLOT PATTERN: cache
ORDERTAKING
RESTAURANTMANAGEMENT
REDISCACHE
MYSQL DATABASE SECONDFIRST
CONSUMERRESTAURANT
OWNER
Thursday, July 11, 13
64
POLYGLOT PATTERN: write to sql and NOSQL
NEWTWEET
INSERT
LPUSHXLTRIM
MySQL
TWEETS
FOLLOWS
FOLLOWERS
Redis: Timeline
FOLLOWER 1
FOLLOWER 2
FOLLOWER 3
Avoidexpensive
MySQL joins
Thursday, July 11, 13
65
POLYGLOT PATTERN: REPLICATE FROM MYsql to NOSQL
QUERYSERVICE
UPDATESERVICE
REDISMYSQL
DATABASE
SYSTEMOF RECORD
MATERIALIZED VIEW
READER WRITER
replicate and
denormalize
query() update()
Thursday, July 11, 13
66
Atomic
Consistent
Isolated
Durable
ACID to base
Basically Available
Soft state
Eventually consistent
Thursday, July 11, 13
5 PILLARS OF CLOUDINESS
wayne wallscloud evangelist
alex brandtsr developer
technical track
Thursday, July 11, 13
FIVE PILLARS OF CLOUDINESS
1 2 3 4 5
Parallel Modular Horizontal Agile Secure
68
Thursday, July 11, 13
Parallel
Thursday, July 11, 13
Parallel - what
1MRecords
Answer
Node
70
Thursday, July 11, 13
Parallel - how
1MRecords
Answer
Node1-250K
Node251K-500K
Node501K-750K
Node751K-1M
71
Thursday, July 11, 13
Parallel - how
72
1MRecords
Answer
Node1-250K
Node251K-500K
Node501K-750K
Node751K-1M
Thursday, July 11, 13
Parallel - how
72
1MRecords
Answer
Node1-250K
Node251K-500K
Node501K-750K
Node751K-1M
Thursday, July 11, 13
Parallel - how
73
Thursday, July 11, 13
Parallel - how
DATA
CALCULATIONMAP
CALCULATIONREDUCE
DATA
CALC
D
C
R
D
D
C
R
D
D
C
R
D
SHUFFLE
DATA
73
Thursday, July 11, 13
Avoid Deadlock
74
Thursday, July 11, 13
Thread 1
Deadlock
75
Thread 2
Thursday, July 11, 13
Thread 1
Deadlock
75
Thread 2
Thursday, July 11, 13
File A
Thread 1
Deadlock
75
Thread 2
Thursday, July 11, 13
File A
Thread 1
Deadlock
75
Thread 2
Thursday, July 11, 13
File A File B
Thread 1
Deadlock
75
Thread 2
Thursday, July 11, 13
File A File B
Thread 1
Deadlock
75
Thread 2
Thursday, July 11, 13
File A File B
Thread 1
Deadlock
75
File B
Thread 2
Thursday, July 11, 13
File A File B
Thread 1
Deadlock
75
File B
Thread 2
Thursday, July 11, 13
File A File B
File A
Thread 1
Deadlock
75
File B
Thread 2
Thursday, July 11, 13
File A File B
File A
Thread 1
Deadlock
75
File B
Thread 2
Thursday, July 11, 13
File A File B
File A
Thread 1
Deadlock
75
X
File B
Thread 2
Thursday, July 11, 13
File A File B
File A
Thread 1
Deadlock
75
X
File B
Thread 2
Thursday, July 11, 13
File A File B
File A
Thread 1
Deadlock
75
X X
File B
Thread 2
Thursday, July 11, 13
Solution? Scheduling
Y
XA
B
A
B
Y
X
1
2
3
4
76
Thursday, July 11, 13
Parallel - why
1M records
Answer
Node
1M records
Answer
Node1-250K
Node251K-500K
Node501K-750K
Node750K-1M
77
Thursday, July 11, 13
Parallel - why
1M records
Answer
Node
1M records
Answer
Node1-250K
Node251K-500K
Node501K-750K
Node750K-1M
77
Thursday, July 11, 13
Parallel - why
78
Thursday, July 11, 13
Parallel - why
79
Thursday, July 11, 13
modular
Thursday, July 11, 13
ONE PERSON BUILDS A CAR
Modular - whatExample: Assembly Line
81
Thursday, July 11, 13
CHASSIS ENGINE BODY INTERIOR
DecoupledSelf-contained unitsData Communication
Modular - whatExample: Assembly Line
82
Thursday, July 11, 13
Modular - what
Modules within Modules
CHASSIS ENGINE BODY INTERIOR
83
Nested Modularity
Thursday, July 11, 13
CHASSIS ENGINE BODY INTERIOR
Modular - what
84
Modules within ModulesNested Modularity
Thursday, July 11, 13
CHASSIS ENGINE BODY INTERIOR
Modular - what
SEATS FLOOR
DASHBOARD
GASAND
BRAKE PEDALS
84
Modules within ModulesNested Modularity
Thursday, July 11, 13
Small units are more maintainableAllows us to tolerate some partitions (but not all)
Allows us to be parallel!
CHASSIS ENGINE BODY INTERIOR
modular - Why
85
Thursday, July 11, 13
Direct communication (1:1)Topic communication (1:N)
Fanout communication (1:ALL)
modular - Why
86
Thursday, July 11, 13
•Decouple requests
•Message Queue
•Make small units of work
•Define tiers by function
modular - how
87
Thursday, July 11, 13
Potential for Upstream Complexity
CHASSIS ENGINE BODY INTERIOR
88
Thursday, July 11, 13
defect
Potential for Upstream Complexity
CHASSIS ENGINE BODY INTERIOR
88
Thursday, July 11, 13
defect defect
Potential for Upstream Complexity
CHASSIS ENGINE BODY INTERIOR
88
Thursday, July 11, 13
defect defect defect
Potential for Upstream Complexity
CHASSIS ENGINE BODY INTERIOR
88
Thursday, July 11, 13
defect defect defectdefect
Potential for Upstream Complexity
CHASSIS ENGINE BODY INTERIOR
88
Thursday, July 11, 13
defect defect defectdefect FAIL
Potential for Upstream Complexity
CHASSIS ENGINE BODY INTERIOR
88
Thursday, July 11, 13
defect defect defectdefect FAIL
Mandelbugs
Potential for Upstream Complexity
CHASSIS ENGINE BODY INTERIOR
88
Thursday, July 11, 13
horizontal
Thursday, July 11, 13
2 GB2 CORES
Horizontal - What
90
8 GB4 CORES
Thursday, July 11, 13
2 GB2 CORES
VERTICAL
Horizontal - What
90
8 GB4 CORES
Thursday, July 11, 13
2 GB2 CORES
VERTICAL
Horizontal - What
90
8 GB4 CORES
Thursday, July 11, 13
2 GB2 CORES
VERTICAL
HORIZONTAL
Horizontal - What
90
8 GB4 CORES
Thursday, July 11, 13
2 GB2 CORES
VERTICAL
HORIZONTAL
Horizontal - What
90
8 GB4 CORES
Thursday, July 11, 13
2 GB2 CORES
2 GB2 CORES
VERTICAL
HORIZONTAL
Horizontal - What
90
8 GB4 CORES
Thursday, July 11, 13
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
VERTICAL
HORIZONTAL
Horizontal - What
90
8 GB4 CORES
Thursday, July 11, 13
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
VERTICAL
HORIZONTAL
Horizontal - What
90
8 GB4 CORES
Thursday, July 11, 13
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
VERTICAL
HORIZONTAL
Horizontal - What
90
8 GB4 CORES
Thursday, July 11, 13
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
VERTICAL
HORIZONTAL
Horizontal - What
90
8 GB4 CORES
Thursday, July 11, 13
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
VERTICAL
HORIZONTAL
Horizontal - What
90
8 GB4 CORES
Thursday, July 11, 13
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
VERTICAL
HORIZONTAL
SHARED NOTHING THROUGH STATELESSNESS
Horizontal - What
90
8 GB4 CORES
Thursday, July 11, 13
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
VERTICAL
HORIZONTAL
SHARED NOTHING THROUGH STATELESSNESS
Horizontal - What
90
8 GB4 CORES
Thursday, July 11, 13
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
2 GB2 CORES
VERTICAL
HORIZONTAL
SHARED NOTHING THROUGH STATELESSNESS
Horizontal - What
2 GB2 CORES
90
8 GB4 CORES
Thursday, July 11, 13
Horizontal - why
91
Thursday, July 11, 13
Horizontal - why
92
Thursday, July 11, 13
ONE SERVER NEAR
CAPACITY
Horizontal - why
92
Thursday, July 11, 13
ONE SERVER NEAR
CAPACITY
ADD ANOTHER SERVER
Horizontal - why
92
Thursday, July 11, 13
ONE SERVER NEAR
CAPACITY
ADD ANOTHER SERVER
Horizontal - why
93
Thursday, July 11, 13
1234 MESSAGE QUEUE
94
Horizontal - HOW
Thursday, July 11, 13
1234MESSAGE QUEUE
95
Horizontal - HOW
Thursday, July 11, 13
1
2
3
4
96
MESSAGE QUEUE
Horizontal - HOW
Thursday, July 11, 13
1
2
3
4
96
56789101112 MESSAGE QUEUE
Horizontal - HOW
Thursday, July 11, 13
97
5678 9101112MESSAGE QUEUE
1
2
3
4
Horizontal - HOW
Thursday, July 11, 13
97
5678 9101112
ADD MORE SERVERSBASED ON
MESSAGE QUEUE
MESSAGE QUEUE
1
2
3
4
Horizontal - HOW
Thursday, July 11, 13
MESSAGE QUEUE
98
5
6
7
8
9
10
11
12
13141516
ADD MORE SERVERSBASED ON
MESSAGE QUEUE
1
2
3
4
Horizontal - HOW
Thursday, July 11, 13
99
5678 9101112
13141516
ADD MORE SERVERSBASED ON
MESSAGE QUEUE
1234
MESSAGE QUEUE
Horizontal - HOW
Thursday, July 11, 13
100
5678 9101112
13141516
ADD MORE SERVERSBASED ON
MESSAGE QUEUE
1234
MESSAGE QUEUE
Horizontal - HOW
Thursday, July 11, 13
101
5678 9101112
13141516
ADD MORE SERVERSBASED ON
MESSAGE QUEUE1234
MESSAGE QUEUE
Horizontal - HOW
Thursday, July 11, 13
101
5678 9101112
13141516
ADD MORE SERVERSBASED ON
MESSAGE QUEUE1234
MESSAGE QUEUE
Horizontal - HOW
DECREASE SERVERSBASED ON
MESSAGE QUEUE
Thursday, July 11, 13
MESSAGE QUEUE
102
5678 9101112
13
14
15
16
ADD MORE SERVERSBASED ON
MESSAGE QUEUE1234
Horizontal - HOW
DECREASE SERVERSBASED ON
MESSAGE QUEUE
Thursday, July 11, 13
agile
Thursday, July 11, 13
Agile - WHAT
the power of moving quickly and easily; nimbleness
104
Thursday, July 11, 13
Features In Progress Blocked Tabled Testing Complete
Feature A
Feature B
Feature C
105
Agile - why Monday
Thursday, July 11, 13
Agile - why
Features In Progress Blocked Tabled Testing Complete
Feature A
Feature B
Feature C
106
Tuesday
Thursday, July 11, 13
Agile - why
Features In Progress Blocked Tabled Testing Complete
Feature A
Feature B
Feature C
Feature E
Feature D
106
Tuesday
Thursday, July 11, 13
Features In Progress Blocked Tabled Testing Complete
Feature AFeature BFeature C
Feature E
Feature G
Agile - why
107
Wednesday
Thursday, July 11, 13
Features In Progress Blocked Tabled Testing Complete
Feature AFeature BFeature C
Feature E
Feature D
Feature F
Feature G
Agile - why
107
Wednesday
Thursday, July 11, 13
Features In Progress Blocked Tabled Testing Complete
Feature A
Feature B
Feature C
Feature E
Feature D
Feature F
Feature G
Agile - why
108
Friday
Thursday, July 11, 13
Features In Progress Blocked Tabled Testing Complete
Feature A
Feature B
Feature C
Feature E
Feature D
Feature F
Feature G
Feature H
Feature I
Agile - why
108
Friday
Thursday, July 11, 13
Features In Progress Blocked Tabled Testing Complete
Feature A
Feature B
Feature C
Feature EFeature D
Feature F
Feature G
Feature H
Feature I
Agile - why
109
Friday - WK 2
Thursday, July 11, 13
Agile - why
57%
14%
29%
Failed Successful Challenged
49%
42%
9%
Waterfall Agile
Source: The CHAOS Manifesto, The Standish Group, 2012
110
Thursday, July 11, 13
• CI/CD when appropriate
• Quality is in the eye of the beholder
• Release fast, release slow
Agile - How
Features In Progress Blocked Tabled Testing Complete
111
Thursday, July 11, 13
IMAGE: blogs.bu.edu
Agile - How
• Increase quality by reusing code from partners and libraries
112
Thursday, July 11, 13
Agile - How
113
Lint Testing
Test Driven Development
Functional Testing
Black Box
Integration Testing
Unit Testing
Behavior Driven Development
White Box
Smoke Testing
Acceptance Testing
Thursday, July 11, 13
ENSURES QUALITY
Agile - How
113
Lint Testing
Test Driven Development
Functional Testing
Black Box
Integration Testing
Unit Testing
Behavior Driven Development
White Box
Smoke Testing
Acceptance Testing
Thursday, July 11, 13
ENSURES CONFIDENCE
Agile - How
114
Lint Testing
Test Driven Development
Functional Testing
Black Box
Integration Testing
Unit Testing
Behavior Driven Development
White Box
Smoke Testing
Acceptance Testing
Thursday, July 11, 13
secure
Thursday, July 11, 13
Security - what
116
•All about Trust•Provider•Customer•User
Thursday, July 11, 13
117
SECURITY - HOW•Security at every layer
PRESENTATION
APPLICATION
STORAGE
Thursday, July 11, 13
117
SECURITY - HOW•Security at every layer
PRESENTATION
APPLICATION
STORAGE
INTERFACESECURITYACROSS ALL LAYERS
Thursday, July 11, 13
118
SECURITY - HOW
•Use industry standards
•SSL
•Encryption
•At rest
•In-transit
Thursday, July 11, 13
119
SECURITY - HOW•Authentication
•User authentication
•Manage yourself
•Offload to 3rd party
•Host-based Authentication
•Security Groups
•Firewalls
•Configuration Management Systems
Thursday, July 11, 13
120
SECURITY - HOW•Hybrid Cloud
•Store different types of data in the right place
•Sensitive data in on-premise cloud
•Non-sensitive on public cloud
Thursday, July 11, 13
121
SECURITY - WHY
•Trustworthiness as an organization
•Cloud security is too important to leave up to the end-user alone
•Ownership gaps that leads to process and governance issues
•Compliance
•Logging and auditing
•Regulatory requirements
Thursday, July 11, 13
BREAK
3:00 - 3:15
Thursday, July 11, 13
APPLICATION dissection
alex brandtsr developer
technical track
wayne wallscloud evangelist
Thursday, July 11, 13
Data Schema
•images• url
• stars
• *title
• caption
• variants
•small
•medium
•big
•tags
•*_id
•rolestars•*image_id
•*role_id
•roles• *name
• privileges
• login
• description
• location
• child_roles
• _id
124
Thursday, July 11, 13
Requirements•List stars
•List roles’ images
•CRUD
•Tag searching
•Who liked what
•Social media
•Image resizing
•Added by tag
•Hot tagged images
125
Thursday, July 11, 13
application decisions
API TIER
UT UR RSWORKERTIER
LB
AUTH
MQ DB
OBJECTSTORAGE
CDN
SOCIALMEDIA
126
INTERNET
•Load Balancer•Infrastructure•Authentication•Message Queue•Database•Content Delivery•Social Media
Thursday, July 11, 13
application decisions
API TIER
UT UR RSWORKERTIER
AUTH
MQ DB
OBJECTSTORAGE
CDN
SOCIALMEDIA
126
INTERNET
•Load Balancer•Infrastructure•Authentication•Message Queue•Database•Content Delivery•Social Media
Thursday, July 11, 13
application decisions
API TIER
UT UR RSWORKERTIER
AUTH
MQ DB
OBJECTSTORAGE
CDN
SOCIALMEDIA
126
INTERNET
•Load Balancer•Infrastructure•Authentication•Message Queue•Database•Content Delivery•Social Media
Thursday, July 11, 13
application decisions
API TIER
UT UR RSWORKERTIER
MQ DB
OBJECTSTORAGE
CDN
SOCIALMEDIA
126
INTERNET
•Load Balancer•Infrastructure•Authentication•Message Queue•Database•Content Delivery•Social Media
Thursday, July 11, 13
application decisions
API TIER
UT UR RSWORKERTIER
DB
OBJECTSTORAGE
CDN
SOCIALMEDIA
126
INTERNET
•Load Balancer•Infrastructure•Authentication•Message Queue•Database•Content Delivery•Social Media
Thursday, July 11, 13
application decisions
API TIER
UT UR RSWORKERTIER
OBJECTSTORAGE
CDN
SOCIALMEDIA
126
INTERNET
•Load Balancer•Infrastructure•Authentication•Message Queue•Database•Content Delivery•Social Media
Thursday, July 11, 13
application decisions
API TIER
UT UR RSWORKERTIER
CDN
SOCIALMEDIA
126
INTERNET
•Load Balancer•Infrastructure•Authentication•Message Queue•Database•Content Delivery•Social Media
Thursday, July 11, 13
application decisions
API TIER
UT UR RSWORKERTIER
CDN
126
INTERNET
•Load Balancer•Infrastructure•Authentication•Message Queue•Database•Content Delivery•Social Media
Thursday, July 11, 13
application decisions
API TIER
UT UR RSWORKERTIER
126
INTERNET
•Load Balancer•Infrastructure•Authentication•Message Queue•Database•Content Delivery•Social Media
Thursday, July 11, 13
PARALLEL
API TIER
UT UR RS
INTERNET
WORKERTIER
127
Thursday, July 11, 13
•API Tier
PARALLEL
API TIER
UT UR RS
INTERNET
WORKERTIER
API TIER
127
Thursday, July 11, 13
•API Tier
PARALLEL
API TIER
UT UR RS
INTERNET
WORKERTIER
API TIER
•Works in tandem
127
Thursday, July 11, 13
•API Tier
PARALLEL
API TIER
UT UR RS
INTERNET
WORKERTIER
API TIER
•Works in tandem•Doesn’t wait for workers
127
Thursday, July 11, 13
•API Tier
PARALLEL
API TIER
UT UR RS
INTERNET
•Update Tags, Update Relations, Resize Workers operate in tandem
WORKERTIER UT UR RS
•Works in tandem•Doesn’t wait for workers
127
Thursday, July 11, 13
•API Tier
PARALLEL
API TIER
UT UR RS
INTERNET
•Update Tags, Update Relations, Resize Workers operate in tandem
•Non-blocking actions
WORKERTIER UT UR RS
API TIER
•Works in tandem•Doesn’t wait for workers
127
Thursday, July 11, 13
MODULAR
API TIER
UT UR RSWORKERTIER
INTERNET
128
Thursday, July 11, 13
•Separate tiers by functionality
MODULAR
API TIER
UT UR RSWORKERTIER
API TIER
UT UR RS
INTERNET
128
Thursday, July 11, 13
•Separate tiers by functionality
MODULAR
API TIER
UT UR RS
•Use MQ to decouple API from worker
WORKERTIER
INTERNET
128
Thursday, July 11, 13
•Separate tiers by functionality
MODULAR
API TIER
UT UR RS
•Use MQ to decouple API from worker
•Separate workers to simplify units of work
WORKERTIER UT UR RS
INTERNET
128
Thursday, July 11, 13
Horizontal
API TIER
UT UR RSWORKERTIER
INTERNET
129
Thursday, July 11, 13
•Restful API for shared nothing scale
Horizontal
API TIER
UT UR RSWORKERTIER
API TIER
INTERNET
129
Thursday, July 11, 13
•Restful API for shared nothing scale
Horizontal
API TIER
UT UR RS
•Workers share no state, simply work on input
WORKERTIER
INTERNET
129
Thursday, July 11, 13
•Restful API for shared nothing scale
Horizontal
API TIER
UT UR RS
•Workers share no state, simply work on input
•Scale data across nodes (even DCs)
WORKERTIER UT UR RS
API TIER
INTERNET
129
Thursday, July 11, 13
AGILE
API TIER
UT UR RSWORKERTIER
INTERNET
130
Thursday, July 11, 13
•Don’t re-invent ability or hosting
AGILE
API TIER
UT UR RSWORKERTIER
INTERNET
130
Thursday, July 11, 13
•Don’t re-invent ability or hosting
AGILE
API TIER
UT UR RS
•Split processes that are individually testable
WORKERTIER
API TIER
UT UR RS
INTERNET
130
Thursday, July 11, 13
•Don’t re-invent ability or hosting
AGILE
API TIER
UT UR RS
•Split processes that are individually testable
•CI/CD is achievable by component
WORKERTIER
INTERNET
130
Thursday, July 11, 13
•Don’t re-invent ability or hosting
AGILE
API TIER
UT UR RS
•Split processes that are individually testable
•CI/CD is achievable by component
WORKERTIER
INTERNET
130
Thursday, July 11, 13
•Don’t re-invent ability or hosting
AGILE
API TIER
UT UR RS
•Split processes that are individually testable
•CI/CD is achievable by component
WORKERTIER
INTERNET
130
Thursday, July 11, 13
•Don’t re-invent ability or hosting
AGILE
API TIER
UT UR RS
•Split processes that are individually testable
•CI/CD is achievable by component
WORKERTIER
INTERNET
130
Thursday, July 11, 13
•Don’t re-invent ability or hosting
AGILE
API TIER
UT UR RS
•Split processes that are individually testable
•CI/CD is achievable by component
WORKERTIER
INTERNET
130
Thursday, July 11, 13
•Don’t re-invent ability or hosting
AGILE
API TIER
UT UR RS
•Split processes that are individually testable
•CI/CD is achievable by component
WORKERTIER
API TIER
INTERNET
130
Thursday, July 11, 13
SECURE
API TIER
UT UR RSWORKERTIER
INTERNET
131
Thursday, July 11, 13
•Use HTTPs for secure data transmission
SECURE
API TIER
UT UR RSWORKERTIER
API TIER
INTERNET
131
Thursday, July 11, 13
•Use HTTPs for secure data transmission
SECURE
API TIER
UT UR RS
•No host logins; managed by Chef
WORKERTIER
API TIER
UT UR RS
INTERNET
131
Thursday, July 11, 13
•Use HTTPs for secure data transmission
SECURE
API TIER
UT UR RS
•No host logins; managed by Chef
•StormPath for user authentication
WORKERTIER
INTERNET
131
Thursday, July 11, 13
thank you!@rackspaceUK@Racker_beighton@waynewalls@alunduil@jennbolestweets
Thursday, July 11, 13