Download - DynamoDB Deep Dive

Transcript
Page 1: DynamoDB Deep Dive

Deep dive on DynamoDB to create scalable app Eduardo Horai

AWS Solutions Architect

Page 2: DynamoDB Deep Dive

What is DynamoDB?

1

Page 3: DynamoDB Deep Dive

DynamoDB is a managed NoSQL database service.

Store and retrieve any amount of data.

Serve any level of request traffic.

Page 4: DynamoDB Deep Dive

Without the operational burden.

Page 5: DynamoDB Deep Dive

Consistent, predictable performance.

Single digit millisecond latency.

Backed on solid-state drives.

Page 6: DynamoDB Deep Dive

Key/attribute pairs. No schema required.

Easy to create. Easy to adjust.

Flexible data model.

Page 7: DynamoDB Deep Dive

No table size limits. Unlimited storage.

No downtime.

Seamless scalability.

Page 8: DynamoDB Deep Dive

Consistent, disk only writes.

Replication across data centers and availability zones.

Durable.

Page 9: DynamoDB Deep Dive

Focus on your app.

Page 10: DynamoDB Deep Dive

Reserve IOPS for reads and writes.

Scale up for down at any time.

Provisioned throughput.

Page 11: DynamoDB Deep Dive

Pay per capacity unit.

Priced per hour of provisioned throughput.

Page 12: DynamoDB Deep Dive

Size of item x writes per second

Write throughput.

>= 1KB

Page 13: DynamoDB Deep Dive

Atomic increment and decrement.

Optimistic concurrency control: conditional writes.

Consistent writes.

Page 14: DynamoDB Deep Dive

Item level transactions only.

Puts, updates and deletes are ACID.

Transactions.

Page 15: DynamoDB Deep Dive

Read throughput.

Strong or eventual consistency

Page 16: DynamoDB Deep Dive

Read throughput.

Strong or eventual consistency

Provisioned units = size of item x reads per second >= 4KB

Page 17: DynamoDB Deep Dive

Read throughput.

Strong or eventual consistency

Provisioned units = size of item x reads per second 2

Page 18: DynamoDB Deep Dive

Read throughput.

Strong or eventual consistency

Same latency expectations.

Mix and match at ‘read time’.

Page 19: DynamoDB Deep Dive

Provisioned throughput is managed by DynamoDB.

Page 20: DynamoDB Deep Dive

Data is partitioned and managed by DynamoDB.

Page 21: DynamoDB Deep Dive

•  DynamoDB automatically partitions data by the hash key –  Hash key spreads data & workload across partitions

•  Auto-Partitioning driven by: –  Data set size –  Provisioned Throughput

•  Tip: large number of unique hash keys and uniform distribution of workload across hash keys lends well to massive scale!

Partitioning

Page 22: DynamoDB Deep Dive

Tiered bandwidth pricing: aws.amazon.com/dynamodb/pricing

Indexed data storage.

Page 23: DynamoDB Deep Dive

Up to 53% for 1 year reservation.

Up to 76% for 3 year reservation.

Reserved capacity.

Page 24: DynamoDB Deep Dive

Session based to minimize latency. Uses the Amazon Security Token Service.

Handled by AWS SDKs. Integrates with IAM.

Authentication.

Page 25: DynamoDB Deep Dive

CloudWatch metrics: latency, consumed read and write throughput,

errors and throttling.

Monitoring.

Page 26: DynamoDB Deep Dive

Libraries, mappers and mocks.

ColdFusion, Django, Erlang, Java, .Net, Node.js, Perl, PHP, Python, Ruby

http://j.mp/dynamodb-libs

Page 27: DynamoDB Deep Dive

NoSQL Data Modeling

2

Page 28: DynamoDB Deep Dive

id = 100

date = 2012-05-16-12-00-10 id = 101 total = 100.00

total = 25.00

id = 101 date = 2012-05-15-15-00-11 total = 35.00

date = 2012-05-16-09-00-10

Page 29: DynamoDB Deep Dive

id = 100 date = 2012-05-16-09-00-10 total = 25.00

id = 101 date = 2012-05-15-15-00-11 total = 35.00

Table

date = 2012-05-16-12-00-10 id = 101 total = 100.00

Page 30: DynamoDB Deep Dive

id = 100 total = 25.00

id = 101 date = 2012-05-15-15-00-11 total = 35.00

Item

date = 2012-05-16-12-00-10 id = 101 total = 100.00

date = 2012-05-16-09-00-10

Page 31: DynamoDB Deep Dive

id = 100

2012-05-15-15-00-11

total = 25.00 Attribute

date = 2012-05-16-12-00-10 id = 101 total = 100.00

id = 101 date = total = 35.00

date = 2012-05-16-09-00-10

Page 32: DynamoDB Deep Dive

Tables do not require a formal schema.

Items are an arbitrarily sized hash.

Where is the schema?

Page 33: DynamoDB Deep Dive

Items are indexed by primary and secondary keys.

Primary keys can be composite.

Secondary keys are local to the table.

Indexing.

Page 34: DynamoDB Deep Dive

ID Date Total

Page 35: DynamoDB Deep Dive

ID Date Total

Hash key

Page 36: DynamoDB Deep Dive

ID Date Total

Hash key Range key

Composite primary key

Page 37: DynamoDB Deep Dive

ID Date Total

Hash key Range key Secondary range key

Page 38: DynamoDB Deep Dive

Programming DynamoDB.

Small but perfectly formed API.

Page 39: DynamoDB Deep Dive

CreateTable

Scan

UpdateTable

DeleteTable

DescribeTable

ListTables

Query

PutItem

GetItem

UpdateItem

DeleteItem

BatchGetItem

BatchWriteItem

Page 40: DynamoDB Deep Dive

CreateTable

Scan

UpdateTable

DeleteTable

DescribeTable

ListTables

Query

PutItem

GetItem

UpdateItem

DeleteItem

BatchGetItem

BatchWriteItem

Page 41: DynamoDB Deep Dive

CreateTable

Scan

UpdateTable

DeleteTable

DescribeTable

ListTables

Query

PutItem

GetItem

UpdateItem

DeleteItem

BatchGetItem

BatchWriteItem

Page 42: DynamoDB Deep Dive

PutItem, UpdateItem, DeleteItem can take optional conditions for operation.

UpdateItem performs atomic increments.

Conditional updates.

Page 43: DynamoDB Deep Dive

One API call, multiple items

BatchGet returns multiple items by key.

BatchWrite performs up to 25 put or delete operations.

Throughput is measured by IO, not API calls.

Page 44: DynamoDB Deep Dive

CreateTable

UpdateTable

DeleteTable

DescribeTable

ListTables

Query

Scan

PutItem

GetItem

UpdateItem

DeleteItem

BatchGetItem

BatchWriteItem

Page 45: DynamoDB Deep Dive

Query returns items by key.

Scan reads the whole table sequentially.

Query vs Scan

Page 46: DynamoDB Deep Dive

Query patterns

Retrieve all items by hash key.

Range key conditions: ==, <, >, >=, <=, begins with, between.

Counts. Top and bottom n values. Paged responses.

Page 47: DynamoDB Deep Dive

Example

3

Page 48: DynamoDB Deep Dive

Page 49: DynamoDB Deep Dive

AmazonDynamoDBClient dynamoDB; = new AmazonDynamoDBClient( new ClasspathPropertiesFileCredentialsProvider()); dynamoDB.setRegion(Region.getRegion(Regions. SA_EAST_1));

Page 50: DynamoDB Deep Dive

Players user_id =

mza location = Cambridge

joined = 2011-07-04

user_id = jeffbarr

location = Seattle

joined = 2012-01-20

user_id = werner

location = Worldwide

joined = 2011-05-15

Page 51: DynamoDB Deep Dive

CreateTableRequest createPlayersTable = new CreateTableRequest().withTableName("Players") .withKeySchema(new KeySchemaElement().withAttributeName("user_id") .withKeyType(KeyType.HASH)) .withAttributeDefinitions(newAttributeDefinition() .withAttributeName("user_id").withAttributeType(ScalarAttributeType.S)) .withProvisionedThroughput(new ProvisionedThroughput() .withReadCapacityUnits(10L) .withWriteCapacityUnits(10L)); dynamoDB.createTable(createPlayersTable);

Page 52: DynamoDB Deep Dive

Players

Scores

user_id = mza

location = Cambridge

joined = 2011-07-04

user_id = jeffbarr

location = Seattle

joined = 2012-01-20

user_id = werner

location = Worldwide

joined = 2011-05-15

user_id = mza

game = angry-birds

score = 11,000

user_id = mza

game = tetris

score = 1,223,000

user_id = werner

game = bejewelled

score = 55,000

Page 53: DynamoDB Deep Dive

CreateTableRequest createScoresTable = new CreateTableRequest().withTableName(”Scores") .withKeySchema(new KeySchemaElement().withAttributeName("user_id") .withKeyType(KeyType.HASH)) .withAttributeDefinitions(newAttributeDefinition() .withAttributeName("user_id").withAttributeType(ScalarAttributeType.S)) .withKeySchema(new KeySchemaElement().withAttributeName(”game") .withKeyType(KeyType.RANGE)) .withAttributeDefinitions(newAttributeDefinition() .withAttributeName(”game").withAttributeType(ScalarAttributeType.S)) .withProvisionedThroughput(new ProvisionedThroughput() .withReadCapacityUnits(100L) .withWriteCapacityUnits(100L));

Page 54: DynamoDB Deep Dive

Players

Scores Leader boards

user_id = mza

location = Cambridge

joined = 2011-07-04

user_id = jeffbarr

location = Seattle

joined = 2012-01-20

user_id = werner

location = Worldwide

joined = 2011-05-15

user_id = mza

game = angry-birds

score = 11,000

user_id = mza

game = tetris

score = 1,223,000

user_id = werner

game = bejewelled

score = 55,000

game = angry-birds

score = 11,000

user_id = mza

game = tetris

score = 1,223,000

user_id = mza

game = tetris

score = 9,000,000

user_id = jeffbarr

Page 55: DynamoDB Deep Dive

CreateTableRequest createLeaderBoardsTable = new CreateTableRequest().withTableName(”LeaderBoards") .withKeySchema(new KeySchemaElement().withAttributeName(”game") .withKeyType(KeyType.HASH)) .withAttributeDefinitions(newAttributeDefinition() .withAttributeName(”game").withAttributeType(ScalarAttributeType.S)) .withKeySchema(new KeySchemaElement().withAttributeName(”score") .withKeyType(KeyType.RANGE)) .withAttributeDefinitions(newAttributeDefinition() .withAttributeName(”score").withAttributeType(ScalarAttributeType.N)) .withProvisionedThroughput(new ProvisionedThroughput() .withReadCapacityUnits(50L) .withWriteCapacityUnits(50L));

Page 56: DynamoDB Deep Dive

Players

Scores Leader boards

Query for user user_id =

mza location = Cambridge

joined = 2011-07-04

user_id = jeffbarr

location = Seattle

joined = 2012-01-20

user_id = werner

location = Worldwide

joined = 2011-05-15

game = angry-birds

score = 11,000

user_id = mza

game = tetris

score = 1,223,000

user_id = mza

game = tetris

score = 9,000,000

user_id = jeffbarr

user_id = mza

game = angry-birds

score = 11,000

user_id = mza

game = tetris

score = 1,223,000

user_id = werner

game = bejewelled

score = 55,000

Page 57: DynamoDB Deep Dive

Map<String, Condition> keyConditions = new HashMap<String, Condition>(); keyConditions.put("user_id", new Condition() .withComparisonOperator(ComparisonOperator.EQ.toString()) .withAttributeValueList(new AttributeValue().withS("mza"))); QueryRequest queryRequest = new QueryRequest() .withTableName("Players") .withKeyConditions(keyConditions);

QueryResult result = dynamoDB.query(queryRequest); for (Map<String, AttributeValue> item : result.getItems()) { printItem(item); }

Page 58: DynamoDB Deep Dive

Players

Scores Leader boards

Query for scores by user

user_id = mza

location = Cambridge

joined = 2011-07-04

user_id = jeffbarr

location = Seattle

joined = 2012-01-20

user_id = werner

location = Worldwide

joined = 2011-05-15

game = angry-birds

score = 11,000

user_id = mza

game = tetris

score = 1,223,000

user_id = mza

game = tetris

score = 9,000,000

user_id = jeffbarr

user_id = mza

game = angry-birds

score = 11,000

user_id = mza

game = tetris

score = 1,223,000

user_id = werner

game = bejewelled

score = 55,000

Page 59: DynamoDB Deep Dive

Map<String, Condition> keyConditions = new HashMap<String, Condition>(); keyConditions.put("user_id", new Condition() .withComparisonOperator(ComparisonOperator.EQ.toString()) .withAttributeValueList(new AttributeValue().withS("mza"))); QueryRequest queryRequest = new QueryRequest() .withTableName(”Scores") .withAttributesToGet(”score”, “game”) .withKeyConditions(keyConditions);

QueryResult result = dynamoDB.query(queryRequest); for (Map<String, AttributeValue> item : result.getItems()) { printItem(item); }

Page 60: DynamoDB Deep Dive

Players

Scores Leader boards

Query for scores by user, game

user_id = mza

location = Cambridge

joined = 2011-07-04

user_id = jeffbarr

location = Seattle

joined = 2012-01-20

user_id = werner

location = Worldwide

joined = 2011-05-15

game = angry-birds

score = 11,000

user_id = mza

game = tetris

score = 1,223,000

user_id = mza

game = tetris

score = 9,000,000

user_id = jeffbarr

user_id = mza

game = angry-birds

score = 11,000

user_id = mza

game = tetris

score = 1,223,000

user_id = werner

game = bejewelled

score = 55,000

Page 61: DynamoDB Deep Dive

Map<String, Condition> keyConditions = new HashMap<String, Condition>(); keyConditions.put("user_id", new Condition() .withComparisonOperator(ComparisonOperator.EQ.toString()) .withAttributeValueList(new AttributeValue().withS("mza"))); keyConditions.put(”game", new Condition() .withComparisonOperator(ComparisonOperator.EQ.toString()) .withAttributeValueList(new AttributeValue().withS(”tetris"))); QueryRequest queryRequest = new QueryRequest() .withTableName(”Scores") .withKeyConditions(keyConditions);

QueryResult result = dynamoDB.query(queryRequest); for (Map<String, AttributeValue> item : result.getItems()) { printItem(item); }

Page 62: DynamoDB Deep Dive

Players

Scores Leader boards

High scores by game

user_id = mza

location = Cambridge

joined = 2011-07-04

user_id = jeffbarr

location = Seattle

joined = 2012-01-20

user_id = werner

location = Worldwide

joined = 2011-05-15

user_id = mza

game = angry-birds

score = 11,000

user_id = mza

game = tetris

score = 1,223,000

user_id = werner

game = bejewelled

score = 55,000

game = angry-birds

score = 11,000

user_id = mza

game = tetris

score = 1,223,000

user_id = mza

game = tetris

score = 9,000,000

user_id = jeffbarr

Page 63: DynamoDB Deep Dive

Map<String, Condition> keyConditions = new HashMap<String, Condition>(); keyConditions.put(”game", new Condition() .withComparisonOperator(ComparisonOperator.EQ.toString()) .withAttributeValueList(new AttributeValue().withS(”tetris"))); QueryRequest queryRequest = new QueryRequest() .withTableName(”LeaderBoards") .withKeyConditions(keyConditions) . withScanIndexForward(false);

QueryResult result = dynamoDB.query(queryRequest); for (Map<String, AttributeValue> item : result.getItems()) { printItem(item); }

Page 64: DynamoDB Deep Dive

Players

Scores Leader boards

Insert Players user_id =

mza location = Cambridge

joined = 2011-07-04

user_id = jeffbarr

location = Seattle

joined = 2012-01-20

user_id = werner

location = Worldwide

joined = 2011-05-15

game = angry-birds

score = 11,000

user_id = mza

game = tetris

score = 1,223,000

user_id = mza

game = tetris

score = 9,000,000

user_id = jeffbarr

user_id = mza

game = angry-birds

score = 11,000

user_id = mza

game = tetris

score = 1,223,000

user_id = werner

game = bejewelled

score = 55,000

Page 65: DynamoDB Deep Dive

Map<String, AttributeValue> itemPlayer = new HashMap<String, AttributeValue>(); itemPlayer.put("user_id", new AttributeValue("eduardohorai")); itemPlayer.put("location", new AttributeValue("Sao Paulo")); itemPlayer.put("joined", new AttributeValue("27/01/2013")); PutItemRequest putItemRequest = new PutItemRequest("Players", itemPlayer); PutItemResult putItemResult = dynamoDB.putItem(putItemRequest);

Page 66: DynamoDB Deep Dive

Players

Scores Leader boards

Increase writes/reads on Scores!!!!!

user_id = mza

location = Cambridge

joined = 2011-07-04

user_id = jeffbarr

location = Seattle

joined = 2012-01-20

user_id = werner

location = Worldwide

joined = 2011-05-15

game = angry-birds

score = 11,000

user_id = mza

game = tetris

score = 1,223,000

user_id = mza

game = tetris

score = 9,000,000

user_id = jeffbarr

user_id = mza

game = angry-birds

score = 11,000

user_id = mza

game = tetris

score = 1,223,000

user_id = werner

game = bejewelled

score = 55,000

Page 67: DynamoDB Deep Dive

UpdateTableRequest updateTableRequest = new UpdateTableRequest() .withTableName("Scores") .withProvisionedThroughput(new ProvisionedThroughput() .withReadCapacityUnits(200L) .withWriteCapacityUnits(200L)); UpdateTableResult result = dynamoDB.updateTable(updateTableRequest);

Page 68: DynamoDB Deep Dive

Using AWS Console

4

Page 69: DynamoDB Deep Dive

§  aws.amazon.com/dynamodb  

§  aws.typepad.com/brasil/  §  aws.typepad.com  

§  awshub.com.br  

§  [email protected]  

Links

Page 70: DynamoDB Deep Dive

Questions?

Learn More: aws.amazon.com/dynamodb

Page 71: DynamoDB Deep Dive

Obrigado!

Learn More: aws.amazon.com/dynamodb


Top Related