November 13, 2014 | Las Vegas, NV
“Everything fails, all the time.”
500
Black Friday balance: Submarino down, Americanas down, Walmart
down, Magazine Luiza with problems, FNAC down. #BlackFridayFail
Magazine Luiza is doing a clearance sale for New Year. It’s a nice sale,
but so good that their website is down
–120 orders / minute
XA
Going beyond the traditional high availability concept
eXtreme availability
XA
global infrastructure
beginning
processes
developers
architects
any failure
Graceful
Micro services m-SOA
availability needs
AWS
Amazon EC2 Amazon EC2
Elastic Load Balancing
Availability
Zone A
Availability
Zone B
Amazon S3
Static
Website
Amazon
CloudFront
User
Amazon
Route 53
Secondary
Sta
tic
Assets
Primary + Health Checks
Auto
Scaling
Group
us-east
Amazon S3
Static
Website
Amazon
CloudFront
User
Amazon
Route 53
SecondaryS
tatic
Assets
Active/Active with Latency Based Routing + Health Checks
EC2 EC2
Elastic Load Balancing
Availability
Zone A
Availability
Zone B
us-east-1
EC2 EC2
Elastic Load Balancing
Availability
Zone A
Availability
Zone B
us-west-1
Amazon S3
Static
Website
Amazon
CloudFront
User
Amazon
Route 53
SecondaryS
tatic
Assets
Active/Active with Latency Based Routing + Health Checks
EC2 EC2
Elastic Load Balancing
Availability
Zone A
Availability
Zone B
us-east-1
EC2 EC2
Elastic Load Balancing
Availability
Zone A
Availability
Zone B
us-west-1
EC2
us-east-2
EC2
us-west-2
Amazon
CloudFront
User
Amazon
Route 53
Sta
tic
Assets
Active/Active with Latency Based Routing + Health Checks
EC2 EC2
Elastic Load Balancing
Amazon
S3
Static
Content
EC2 EC2
Elastic Load Balancing
CA
CH
EW
EB
SE
RV
ER
Availability
Zone A
Availability
Zone B
Availability
Zone A
Availability
Zone B
sa-east-1 us-east-1
CA
CH
EW
EB
SE
RV
ER
CA
CH
EW
EB
SE
RV
ER
CA
CH
EW
EB
SE
RV
ER
Amazon RDS Multi-AZ
(with read replicas)
Amazon DynamoDB and
Amazon S3 have built-in
HA
Availability Zone A
Availability Zone B
Amazon RDS read replicas rock!
Availability Zone A
Availability Zone B
Amazon RDS
Read Replica
Amazon RDS
Read Replica
Availability Zone A
Availability Zone B
Amazon RDS Master
Amazon RDS
Read Replica Amazon RDS
Read Replica
Amazon RDS Standby
Availability Zone C
Amazon RDS
Read Replica
us-east-1 us-west-1
Amazon RDS read replicas rock!
Availability Zone A
Availability Zone B
Amazon RDS
Read Replica
Amazon RDS
Read Replica
Availability Zone A
Availability Zone B
Amazon RDS Master
Amazon RDS
Read Replica Amazon RDS
Read Replica
Amazon RDS Standby
Availability Zone C
Amazon RDS
Read Replica
us-east-1 us-west-1
Amazon RDS read replicas rock!
Availability Zone A
Availability Zone B
Amazon RDS
Read Replica
Amazon RDS
Read Replica
Availability Zone A
Availability Zone B
Amazon RDS Master
Amazon RDS
Read Replica Amazon RDS
Read Replica
Amazon RDS Standby
Availability Zone C
Amazon RDS
Read Replica
us-east-1 us-west-1
Amazon RDS Master
(promoted)
Availability Zone
A
Amazon RDS
us-east-1
Amazon
EC2 as
GoldenGate
Hub for
Source DB
Availability Zone
A
Amazon RDS
Amazon
EC2 as
GoldenGate
Hub for
Source DB
us-west-1
GET
us-east-1 us-west-1
GET
us-east-1 us-west-1
Let’s look
inside here!
Consuming AWSDynamoDBstreamsAdapterClient adapterClient =new AWSDynamoDBstreamsAdapterClient(updateStreamsCredentials, .. );
..
AmazonDynamoDBClient dynamoDBClient =new AmazonDynamoDBClient(dynamoDBCredentials, ..);
..
KinesisClientLibConfiguration workerConfig =new KinesisClientLibConfiguration (.., streamId,
updateStreamsCredentials, ..)
.withMaxRecords(100)
.withInitialPositionInStream(InitialPositionInStream.TRIM_HORIZON);
Worker worker =new Worker(recordProcessorFactory, workerConfig, adapterClient,
dynamoDBClient, ..);
Thread t = new Thread(worker);t.start();
Consuming AWSDynamoDBstreamsAdapterClient adapterClient =new AWSDynamoDBstreamsAdapterClient(updateStreamsCredentials, .. );
..
AmazonDynamoDBClient dynamoDBClient =new AmazonDynamoDBClient(dynamoDBCredentials, ..);
..
KinesisClientLibConfiguration workerConfig =new KinesisClientLibConfiguration (.., streamId,
updateStreamsCredentials, ..)
.withMaxRecords(100)
.withInitialPositionInStream(InitialPositionInStream.TRIM_HORIZON);
Worker worker =new Worker(recordProcessorFactory, workerConfig, adapterClient,
dynamoDBClient, ..);
Thread t = new Thread(worker);t.start();
Consuming AWSDynamoDBstreamsAdapterClient adapterClient =new AWSDynamoDBstreamsAdapterClient(updateStreamsCredentials, .. );
..
AmazonDynamoDBClient dynamoDBClient =new AmazonDynamoDBClient(dynamoDBCredentials, ..);
..
KinesisClientLibConfiguration workerConfig =new KinesisClientLibConfiguration (.., streamId,
updateStreamsCredentials, ..)
.withMaxRecords(100)
.withInitialPositionInStream(InitialPositionInStream.TRIM_HORIZON);
Worker worker =new Worker(recordProcessorFactory, workerConfig, adapterClient,
dynamoDBClient, ..);
Thread t = new Thread(worker);t.start();
Consuming AWSDynamoDBstreamsAdapterClient adapterClient =new AWSDynamoDBstreamsAdapterClient(updateStreamsCredentials, .. );
..
AmazonDynamoDBClient dynamoDBClient =new AmazonDynamoDBClient(dynamoDBCredentials, ..);
..
KinesisClientLibConfiguration workerConfig =new KinesisClientLibConfiguration (.., streamId,
updateStreamsCredentials, ..)
.withMaxRecords(100)
.withInitialPositionInStream(InitialPositionInStream.TRIM_HORIZON);
Worker worker =new Worker(recordProcessorFactory, workerConfig, adapterClient,
dynamoDBClient, ..);
Thread t = new Thread(worker);t.start();
Processingpublic class StreamsRecordProcessor implements IRecordProcessor {
..
@Override
public void processRecords(List<Record> records,.. ) {
for(Record record : records) {
if (record instanceof RecordAdapter) {
Record ddbStreamRecord = ((RecordAdapter)
record).getInternalObject();
switch(ddbStreamRecord.getEventName()) {
case "INSERT" : case "MODIFY" :
DemoHelper.putItem(dynamoDBClient, tableName,
ddbStreamRecord.getDynamodb().getNewImage());
break;
...
Processingpublic class StreamsRecordProcessor implements IRecordProcessor {
..
@Override
public void processRecords(List<Record> records,.. ) {
for(Record record : records) {
if (record instanceof RecordAdapter) {
Record ddbStreamRecord = ((RecordAdapter)
record).getInternalObject();
switch(ddbStreamRecord.getEventName()) {
case "INSERT" : case "MODIFY" :
DemoHelper.putItem(dynamoDBClient, tableName,
ddbStreamRecord.getDynamodb().getNewImage());
break;
...
Processingpublic class StreamsRecordProcessor implements IRecordProcessor {
..
@Override
public void processRecords(List<Record> records,.. ) {
for(Record record : records) {
if (record instanceof RecordAdapter) {
Record ddbStreamRecord = ((RecordAdapter)
record).getInternalObject();
switch(ddbStreamRecord.getEventName()) {
case "INSERT" : case "MODIFY" :
DemoHelper.putItem(dynamoDBClient, tableName,
ddbStreamRecord.getDynamodb().getNewImage());
break;
...
Processingpublic class StreamsRecordProcessor implements IRecordProcessor {
..
@Override
public void processRecords(List<Record> records,.. ) {
for(Record record : records) {
if (record instanceof RecordAdapter) {
Record ddbStreamRecord = ((RecordAdapter)
record).getInternalObject();
switch(ddbStreamRecord.getEventName()) {
case "INSERT" : case "MODIFY" :
DemoHelper.putItem(dynamoDBClient, tableName,
ddbStreamRecord.getDynamodb().getNewImage());
break;
...
ERP
Database
Replica
ERP
Database
Replica
Availability
Zone AAvailability
Zone B
sa-east-1
EC2 EC2
WORKERS
COLO
ERP
Database
AWS Direct
Connect
NoSQL NoSQL
WORKERSAmazon Simple
Queue Service
us-east-1 Internet
Same infrastructure with constant
data replication
Please give us your feedback on this session.
Complete session evaluations and earn re:Invent swag.
http://bit.ly/awsevals