Cassandra on Azure - "Tel-Aviv-Cassandra-Users" meetup 2015

Download Cassandra on Azure -

Post on 09-Apr-2017

292 views

Category:

Software

0 download

TRANSCRIPT

TITLE ALL CAPS

CASSANDRA ON AZURE? REALLY?!Oded Peer, RSA Security

# Copyright 2015 EMC Corporation. All rights reserved.

THE ELEPHANT IN THE ROOM

# Copyright 2015 EMC Corporation. All rights reserved.https://centerstagecma.corp.emc.com/CenterStagePro/#CenterStagePro/Brand_+_Creative_Development;v=content/0b0278cd80623579https://upload.wikimedia.org/wikipedia/commons/thumb/5/5c/AWS_Simple_Icons_AWS_Cloud.svg/2000px-AWS_Simple_Icons_AWS_Cloud.svg.png

#TITLE

Only two leaders in Gartner Magic Quadrant for Cloud IaaS (2015)Only two leaders in Gartner Magic Quadrant for Public Cloud Storage Services (2014)Computing capacity: AWS has X10 the combined competition, Microsoft has X2 the non-Amazon combined competitionCloud platform revenues: AWS 6bn revenue, Azure 1.5bnAZURE AND AWS

# Copyright 2015 EMC Corporation. All rights reserved.Microsoft has invested heavily in Azure infrastructure and its ability to accommodate hybrid computing tasksjobs that run partly on the Microsoft Azure public cloud and partly on customers data centers. Microsoft alone has more than twice as much cloud computing capacity in use than all the non-Amazon contenders combined.

3,4,5 have less than half of MS in revenueHapoel Jerusalem and Maccabi Tel Aviv

#TITLE

AZURE IAAS

# Copyright 2015 EMC Corporation. All rights reserved.TypeCPURAM (GB)Temp Storage (GB)IOPS$/moA785660516*500655A885638216*5001091

VM TYPES

# Copyright 2015 EMC Corporation. All rights reserved.Cost East US 2, LinuxA8 and A9 virtual machines featureIntel Xeon E5 processors. Adds a 40Gbit/s InfiniBand network with remote direct memory access (RDMA) technologyWhen configured with the necessary network drivers, the A8 and A9 instances can communicate with other A8 and A9 instances over a low-latency, high-throughput network in Azure that is based on remote direct memory access (RDMA) technology. A8 provides 10 Gb ethernet#TITLE

TypeCPURAM (GB)Temp Storage (GB)IOPS$/moA785660516*500655A885638216*5001091D13856400 (SSD)16*500521D1416112800 (SSD)32*5001032D13_v2856400 (SSD)16*500599

VM TYPES

# Copyright 2015 EMC Corporation. All rights reserved.D-series VMs are designed to run applications that demand higher compute power and temporary disk performance.D-series VMs provide faster processors, a higher memory-to-core ratio, and a solid-state drive (SSD) for the temporary disk. For details, see the announcement on the Azure blog,New D-Series Virtual Machine Sizes.

Dv2-series, a follow-on to the original D-series, features a more powerful CPU.The Dv2-series CPU is about 35% faster than the D-series CPU. It is based on the latest generation 2.4 GHz Intel Xeon E5-2673 v3 (Haswell) processor, and with the Intel Turbo Boost Technology 2.0, can go up to 3.2 GHz.The Dv2-series has the same memory and disk configurations as the D-series.

#TITLE

TypeCPURAM (GB)Temp Storage (GB)IOPS$/moA785660516*500655A885638216*5001091D13856400 (SSD)16*500521D1416112800 (SSD)32*5001032D13_v2856400 (SSD)16*500599DS13856112 (SSD)25,600256 MB/SD13++

VM TYPES

# Copyright 2015 EMC Corporation. All rights reserved.DS-series and GS-series VMs can use Premium Storage, which provides high-performance, low-latency storage for I/O intensive workloads. These VMs use solid-state drives (SSDs) to host a virtual machines disks and also provide a local SSD disk cache. Premium Storage is available in certain regions. For details, seePremium Storage: High-performance storage for Azure virtual machine workloads.

#TITLE

TypeCPURAM (GB)Temp Storage (GB)IOPS$/moA785660516*500655A885638216*5001091D13856400 (SSD)16*500521D1416112800 (SSD)32*5001032D13_v2856400 (SSD)16*500599DS13856112 (SSD)25,600256 MB/SD13++G4162243072 (SSD)32*5003274GS416224448 (SSD)40,0001000 MB/SG4++

VM TYPES

# Copyright 2015 EMC Corporation. All rights reserved.G-series VMs offer the biggest size and best performance and run on hosts that have Intel Xeon E5 V3 family processors.G5 has double the spec values of G4#TITLE

Ephemeral Fast, data loss

Standard 16k IOPS limit (single VM)

Premium (SSD) Per-disk IOPS limitSTORAGE OPTIONS

# Copyright 2015 EMC Corporation. All rights reserved.https://upload.wikimedia.org/wikipedia/commons/a/aa/Floppy_disk_2009_G1.jpghttp://www.datastax.com/2015/04/getting-started-with-azure-premium-storage-and-datastax-enterprise-dse

Durability based on LRS (Locally Redundant system) 3 replicas in the same region.

#TITLE

TypeSize (GB)IOPSThroughputMB/S$/moP1012850010018P20512230015066P3010245000200122

PREMIUM STORAGE PRICING

# Copyright 2015 EMC Corporation. All rights reserved.Data and logs on ephemeral SSD100k IOPS and 1.5 GB/sec on G5Data and logs on premium storageStriped volumes (better than JBOD)Network latency 1-3 msMax 64 TB, 80k IOPS and 1 GB/sec disk throughputUse standard storage for backup snapshotsData on premium logs on ephemeralCASSANDRA STORAGE OPTIONS

# Copyright 2015 EMC Corporation. All rights reserved.https://upload.wikimedia.org/wikipedia/commons/a/aa/Floppy_disk_2009_G1.jpg

http://www.datastax.com/2015/04/getting-started-with-azure-premium-storage-and-datastax-enterprise-dse

Commitlog on ephemeral storage

#TITLE

Logically isolated networkWithin a VNET bandwidth is a function of VM typeUp to 20 Gbps for G5VNET to VNET VPN gatewayCross region supportStandard (100 Mbps)High performance (200 Mbps)Latency according to distanceVNET AND MULTI-REGION DEPLOYMENT

# Copyright 2015 EMC Corporation. All rights reserved.The Azure equivalent of AWS VPC~20ms (West US, California to South-Central US, Texas)No-crypto option, get additional bandwidth

AA has 2 VNETs, one for the application and another for managementYou can control access by using Network Security Groups (NSGs).VNets can be connected to each other, and even to your on-premises datacenter, by using a site-to-site VPN connection

#TITLE

Availability ZonesPLACING REPLICAS

# Copyright 2015 EMC Corporation. All rights reserved.Every VM is associated with an upgrade and a fault domain.

A fault domain is a rack of computers. the allocation of a fault domain is determined by Azure at VM deployment time. A service owner can not control the allocation of a fault domain.

An upgrade domain is a strategy to ensure an application stays up and running, while undergoing an update of the application. Azure will distribute instances evenly into multiple upgrade domains with each upgrade domain as a logical unit of a deployment. When upgrading a deployment, it is then carried out one upgrade domain at a time. The steps are: stopping the instances running in the first upgrade domain, upgrading the application, bringing the instances back online followed by repeating the steps in the next upgrade domain.

#TITLE

Availability ZonesUpgrade Domain, Fault DomainPLACING REPLICAS

# Copyright 2015 EMC Corporation. All rights reserved.Every VM is associated with an upgrade and a fault domain.

A fault domain is a rack of computers. the allocation of a fault domain is determined by Azure at VM deployment time. A service owner can not control the allocation of a fault domain.

An upgrade domain is a strategy to ensure an application stays up and running, while undergoing an update of the application. Azure will distribute instances evenly into multiple upgrade domains with each upgrade domain as a logical unit of a deployment. When upgrading a deployment, it is then carried out one upgrade domain at a time. The steps are: stopping the instances running in the first upgrade domain, upgrading the application, bringing the instances back online followed by repeating the steps in the next upgrade domain.

#TITLE

Availability ZonesUpgrade Domain, Fault Domain assigned at deploymentPLACING REPLICAS

# Copyright 2015 EMC Corporation. All rights reserved.Every VM is associated with an upgrade and a fault domain.

A fault domain is a rack of computers. the allocation of a fault domain is determined by Azure at VM deployment time. A service owner can not control the allocation of a fault domain.

An upgrade domain is a strategy to ensure an application stays up and running, while undergoing an update of the application. Azure will distribute instances evenly into multiple upgrade domains with each upgrade domain as a logical unit of a deployment. When upgrading a deployment, it is then carried out one upgrade domain at a time. The steps are: stopping the instances running in the first upgrade domain, upgrading the application, bringing the instances back online followed by repeating the steps in the next upgrade domain.

#TITLE

AUTOMATIC DEPLOYMENT

# Copyright 2015 EMC Corporation. All rights reserved.Single node sandboxDSE cluster (BYOL)Single region4, 12, 36 or 90 nodesChoose your VM type (D12-D14, G2-G5)Ephemeral storageIncludes OpsCenterAZURE MARKETPLACESIMPLIFIED DEPLOYMENT

# Copyright 2015 EMC Corporation. All rights reserved.

# Copyright 2015 EMC Corporation. All rights reserved.

Fully managed Cassandra-as-a-Service hosted on Microsoft AzureApache Cassandra and DSEMultiple Cassandra versionsOpsCenterUses D-series VMsMulti regionMANAGED CASSANDRA

# Copyright 2015 EMC Corporation. All rights reserved.https://www.youtube.com/watch?v=rhqSmc9meMw#TITLE

REAL WORLD USE CASES

# Copyright 2015 EMC Corporation. All rights reserved.DSE on UbuntuSingle Azure region, 2 vDCG4 nodes, considering D14_v2Data on ephemeral disks12 nodes: ingest 20k - 50k events/sec30 nodes: ingest 200k events/sec

OFFICE 365

# Copyright 2015 EMC Corporation. All rights reserved.https://vimeopro.com/user35188327/cassandra-summit-2015/video/140949184

G1 GC reduced pause times quite a bit, RF=3 in every DC

#TITLE

DS13 VMs (8 CPUs, 56 GB RAM)16 p10 premium storage disksSingle Azure regionCombination of Upgrade Domain and Fault Domain for replica placement

ADAPTIVE AUTHENTICATION

# Copyright 2015 EMC Corporation. All rights reserved.Started with A7, tested G3

#TITLE

Your mileage may varyMicrosoft eager to helpVPN throughput lower than expectedCommitlog flush IO spikes change in flush intervalLCS STCS to reduce compaction IO

TIPS

# Copyright 2015 EMC Corporation. All rights reserved.THANK YOU

# Copyright 2015 EMC Corporation. All rights reserved.