![Page 1: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/1.jpg)
BECOMING FRIENDS WITH CASSANDRA & SPARK
DANI TRAPHAGEN & JON HADDAD
YOU
SPARK
C*
![Page 2: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/2.jpg)
BECOMING FRIENDS WITH CASSANDRA & SPARK
DANI TRAPHAGEN & JON HADDAD
YOU
SPARKC*
![Page 3: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/3.jpg)
HOUSEKEEPING
![Page 4: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/4.jpg)
RAISE YOUR HAND IF YOU DON’T HAVE THE VM OSCON2016.ZIP
![Page 5: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/5.jpg)
1.copy the vm files to a place of your choosing
2.open virtual ovf
VM INSTRUCTIONS
![Page 6: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/6.jpg)
3.import the .ovf as prompted
![Page 7: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/7.jpg)
3.open the packer ovf in VirtualBox
![Page 8: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/8.jpg)
4.check out the vm
![Page 9: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/9.jpg)
LET’S GET STARTED
![Page 10: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/10.jpg)
WHAT ARE WE GOING TO COVER?1. CASSANDRA ARCHITECTURE,
CQL, DATA MODELING 2. SPARK DATAFRAMES
![Page 11: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/11.jpg)
RDBMS & YOU
![Page 12: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/12.jpg)
SQLITE, PYTHON SCRIPTS, LOG FILES
SUCH AS?
SMALL DATA
![Page 13: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/13.jpg)
MOST WEB SITES
RDBMS
MEDIUM DATA
![Page 14: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/14.jpg)
CAN RDBMS WORK FOR BIG DATA?
YOU BIG DATA
![Page 15: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/15.jpg)
VERTICAL SCALE
![Page 16: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/16.jpg)
VERTICAL SCALESTARTING
MY BUSINESS
YAY!
![Page 17: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/17.jpg)
VERTICAL SCALESTARTING
MY BUSINESS
YAY!
![Page 18: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/18.jpg)
VERTICAL SCALESTARTING
MY BUSINESS
YAY!
OH, WHOA, THINGS ARE KICKING UP
![Page 19: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/19.jpg)
VERTICAL SCALESTARTING
MY BUSINESS
YAY!
OH, WHOA, THINGS ARE KICKING UP
![Page 20: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/20.jpg)
VERTICAL SCALESTARTING
MY BUSINESS
YAY!
OH, WHOA, THINGS ARE KICKING UP
![Page 21: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/21.jpg)
ACID IS A LIE
![Page 22: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/22.jpg)
ACID IS A LIEATOMICITY
![Page 23: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/23.jpg)
ACID IS A LIEATOMICITYCONSISTENCY
![Page 24: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/24.jpg)
ACID IS A LIEATOMICITYCONSISTENCYISOLATION
![Page 25: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/25.jpg)
ACID IS A LIEATOMICITYCONSISTENCYISOLATIONDURABILITY
![Page 26: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/26.jpg)
ACID IS A LIEATOMICITYCONSISTENCYISOLATIONDURABILITY
![Page 27: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/27.jpg)
ASYNC REPLICATION != CONSISTENCY
![Page 28: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/28.jpg)
ASYNC REPLICATION != CONSISTENCY
CLIENT
![Page 29: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/29.jpg)
ASYNC REPLICATION != CONSISTENCY
CLIENT
![Page 30: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/30.jpg)
ASYNC REPLICATION != CONSISTENCY
CLIENTMASTER
![Page 31: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/31.jpg)
ASYNC REPLICATION != CONSISTENCY
CLIENTMASTER SLAVE
![Page 32: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/32.jpg)
ASYNC REPLICATION != CONSISTENCY
CLIENTMASTER SLAVE
![Page 33: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/33.jpg)
ASYNC REPLICATION != CONSISTENCY
CLIENTMASTER SLAVE
REPLICATION LAG
![Page 34: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/34.jpg)
CONSISTENT?
ASYNC REPLICATION != CONSISTENCY
CLIENTMASTER SLAVE
REPLICATION LAG
![Page 35: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/35.jpg)
CONSISTENT?
ASYNC REPLICATION != CONSISTENCY
CLIENTMASTER SLAVE
REPLICATION LAG
IDK?
![Page 36: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/36.jpg)
CONSISTENT?
ASYNC REPLICATION != CONSISTENCY
CLIENTMASTER SLAVE
REPLICATION LAG
LOL NO! IDK?
![Page 37: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/37.jpg)
THIRD NORMAL FORM DOESN’T SCALE
▸ UNPREDICTABLE
▸ DATA > MEMORY?
▸ DISK SEEKS ALL DAY
▸ USERS = ANGRY
![Page 38: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/38.jpg)
THIRD NORMAL FORM DOESN’T SCALE
AWFUL▸ UNPREDICTABLE
▸ DATA > MEMORY?
▸ DISK SEEKS ALL DAY
▸ USERS = ANGRY
![Page 39: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/39.jpg)
SHARDING
![Page 40: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/40.jpg)
SHARDING
CLIE
NT
![Page 41: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/41.jpg)
SHARDING
CLIE
NT
![Page 42: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/42.jpg)
SHARDING
CLIE
NTNIGHTMARE
![Page 43: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/43.jpg)
AVAILABILITY?
![Page 44: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/44.jpg)
AVAILABILITY?NOT WITH
THESE KNUCKLEHEADS
![Page 45: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/45.jpg)
CONCLUSION: SCALING IS HARD
![Page 46: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/46.jpg)
FRIEND #1: CASSANDRA
![Page 47: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/47.jpg)
FRIEND #1: CASSANDRA
![Page 48: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/48.jpg)
ARCHITECTURE
![Page 49: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/49.jpg)
ARCHITECTURE
PEER TO PEER
▸ With Cassandra there is no Master Slave Hierarchy
▸ Every node is the captain of it’s own ship
▸ Processes within Cassandra make this possible
▸ Replication
▸ Consistency Level
NODE1
NODE2
NODE3
NODE4
![Page 50: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/50.jpg)
ARCHITECTURE
PEER TO PEER
▸ With Cassandra there is no Master Slave Hierarchy
▸ Every node is the captain of it’s own ship
▸ Processes within Cassandra make this possible
▸ Replication
▸ Consistency Level
NODE1
NODE2
NODE3
NODE4
![Page 51: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/51.jpg)
WHAT DOES THIS GET US?
![Page 52: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/52.jpg)
WHAT DOES THIS GET US?
LINEAR SCALABILITY
![Page 53: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/53.jpg)
WHAT DOES THIS GET US?
LINEAR SCALABILITY
HIGH AVAILABILITY
![Page 54: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/54.jpg)
TOPOLOGY
![Page 55: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/55.jpg)
CLIENT
TOPOLOGY
![Page 56: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/56.jpg)
CLIENT
TOPOLOGY
OPERATION
![Page 57: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/57.jpg)
CLIENT
TOPOLOGY
OPERATION
![Page 58: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/58.jpg)
CLIENT
TOPOLOGY
OPERATION
![Page 59: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/59.jpg)
CLIENT
TOPOLOGY
OPERATION
![Page 60: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/60.jpg)
NODE3
NODE4
▸ Replication factor is the number of replicas/puppies
ARCHITECTURE
REPLICATION IS HOW CASSANDRA DISTRIBUTES DATA
NODE1
NODE2
![Page 61: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/61.jpg)
NODE3
NODE4
▸ Replication factor is the number of replicas/puppies
ARCHITECTURE
REPLICATION IS HOW CASSANDRA DISTRIBUTES DATA
NODE1
NODE2
![Page 62: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/62.jpg)
NODE3
NODE4
▸ Replication factor is the number of replicas/puppies
ARCHITECTURE
REPLICATION IS HOW CASSANDRA DISTRIBUTES DATA
NODE1
NODE2
![Page 63: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/63.jpg)
NODE3
NODE4
▸ The coordinator talks to the client, sending an ack for the write
ARCHITECTURE
HOW DO WE ACKNOWLEDGE REPLICATION?
NODE1
NODE2
COORDINATOR
![Page 64: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/64.jpg)
NODE3
NODE4
▸ The coordinator talks to the client, sending an ack for the write
ARCHITECTURE
HOW DO WE ACKNOWLEDGE REPLICATION?
NODE1
NODE2
COORDINATOR
![Page 65: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/65.jpg)
NODE3
NODE4
▸ The coordinator talks to the client, sending an ack for the write
ARCHITECTURE
HOW DO WE ACKNOWLEDGE REPLICATION?
NODE1
NODE2
COORDINATOR
ack
![Page 66: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/66.jpg)
ARCHITECTURE
TUNABLE CONSISTENCY LEVELS
NODE1
NODE2
NODE3
NODE4
▸ One
▸ Quorum
▸ All
![Page 67: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/67.jpg)
ONE
ARCHITECTURE
NODE1
NODE2
NODE3
NODE4
▸ One replica acks adorable puppy data
![Page 68: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/68.jpg)
ONE
ARCHITECTURE
NODE1
NODE2
NODE3
NODE4
▸ One replica acks adorable puppy data
![Page 69: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/69.jpg)
▸ All replicas ack adorable puppy data
NODE3
NODE4
ARCHITECTURE
ALL
NODE1
NODE2
![Page 70: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/70.jpg)
▸ All replicas ack adorable puppy data
NODE3
NODE4
ARCHITECTURE
ALL
NODE1
NODE2
![Page 71: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/71.jpg)
▸ All replicas ack adorable puppy data
NODE3
NODE4
ARCHITECTURE
ALL
NODE1
NODE2
![Page 72: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/72.jpg)
ARCHITECTURE
QUORUM
NODE1
NODE2
NODE3
▸ Quorum = (sum_of_replication_factors / 2) + 1
▸ How many nodes get puppies if our replication factor is 3, & we want quorum?
NODE4
![Page 73: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/73.jpg)
ARCHITECTURE
QUORUM
NODE1
NODE2
NODE3
▸ Quorum = (sum_of_replication_factors / 2) + 1
▸ How many nodes get puppies if our replication factor is 3, & we want quorum?
NODE4
![Page 74: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/74.jpg)
MULTI-DC PARAMETERS▸Quorum vs. Local_Quorum
▸One vs. Local_One
US-EAST US-WEST
![Page 75: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/75.jpg)
PARTITIONER
CONSISTENT HASHINGJust how is data actually distributed around the cluster?
![Page 76: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/76.jpg)
PARTITIONER
CONSISTENT HASHINGJust how is data actually distributed around the cluster?
![Page 77: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/77.jpg)
PARTITIONER
CONSISTENT HASHINGJust how is data actually distributed around the cluster?
![Page 78: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/78.jpg)
PARTITIONER
CONSISTENT HASHINGJust how is data actually distributed around the cluster?
![Page 79: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/79.jpg)
PARTITIONER
CONSISTENT HASHINGJust how is data actually distributed around the cluster?
![Page 80: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/80.jpg)
CASSANDRA DATA MODELING SOUNDS HARD
![Page 81: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/81.jpg)
CASSANDRA DATA MODELING SOUNDS HARDNOT REALLY
![Page 82: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/82.jpg)
GAIN QUERY POWERSWITH CQL
![Page 83: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/83.jpg)
GAIN QUERY POWERSWITH CQL
![Page 84: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/84.jpg)
DATA STRUCTURES IN CASSANDRA
![Page 85: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/85.jpg)
KEYSPACE
DATA STRUCTURES IN CASSANDRA
![Page 86: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/86.jpg)
KEYSPACE
DATA STRUCTURES IN CASSANDRA
TABLE
![Page 87: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/87.jpg)
KEYSPACE
DATA STRUCTURES IN CASSANDRA
ROWS TABLE
![Page 88: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/88.jpg)
KEYSPACE
DATA STRUCTURES IN CASSANDRA
ROWS
TABLE
![Page 89: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/89.jpg)
KEYSPACE
PARTITIONS
DATA STRUCTURES IN CASSANDRA
ROWS
TABLE
![Page 90: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/90.jpg)
KEYSPACE
PARTITIONS
DATA STRUCTURES IN CASSANDRA
ROWS
TABLE
![Page 91: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/91.jpg)
KEYSPACE
PARTITIONS
DATA STRUCTURES IN CASSANDRA
ROWS
TABLE
![Page 92: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/92.jpg)
PRIMARY KEY = PARTITION KEY + CLUSTERING COLUMNS
![Page 93: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/93.jpg)
PARTITION KEY
![Page 94: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/94.jpg)
PARTITION KEYTHIS IS HOW YOU RETRIEVE A PARTITION
![Page 95: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/95.jpg)
CLUSTERING COLUMNS
![Page 96: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/96.jpg)
CLUSTERING COLUMNSTHIS IS HOW YOU GET SORTING, ORDER AND UNIQUE IDENTIFICATION
![Page 97: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/97.jpg)
WHY ARE CLUSTERING COLUMNS SO COOL?
![Page 98: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/98.jpg)
HOW DO I USE CQL?
![Page 99: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/99.jpg)
CQLSH
HOW DO I USE CQL?
![Page 100: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/100.jpg)
SOME EXAMPLES FROM A MOVIE DB
![Page 101: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/101.jpg)
CREATE A KEYSPACECREATE KEYSPACE movielens_small WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '1'};
![Page 102: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/102.jpg)
CREATE A TABLECREATE TABLE movies ( id uuid PRIMARY KEY, avg_rating float, genres set<text>, name text, release_date date, url text, video_release_date date)
PRIMARY KEY IN WHITE
![Page 103: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/103.jpg)
CREATE A TABLECREATE TABLE ratings_by_movie ( movie_id uuid, user_id uuid, rating int, ts int, PRIMARY KEY (movie_id, user_id))
PRIMARY KEY IN WHITE
![Page 104: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/104.jpg)
INSERT STATEMENT EXAMPLEinsert into movies (id, name, genres) values (976de5da-93ae-4bf0-b127-d19eea1c8ea4, 'My Awesome Movie (2016)', {'Comedy'});
![Page 105: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/105.jpg)
THIS ALL LOOKS TOO FAMILIAR, DOESN’T IT?
![Page 106: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/106.jpg)
BUT REMEMBER…
![Page 107: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/107.jpg)
THIRD NORMAL FORM DOESN’T SCALE
▸ UNPREDICTABLE
▸ DATA > MEMORY?
▸ DISK SEEKS ALL DAY
▸ USERS = ANGRY
![Page 108: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/108.jpg)
THIRD NORMAL FORM DOESN’T SCALE
AWFUL▸ UNPREDICTABLE
▸ DATA > MEMORY?
▸ DISK SEEKS ALL DAY
▸ USERS = ANGRY
![Page 109: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/109.jpg)
DATA MODELING PRO TIPS
![Page 110: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/110.jpg)
DATA MODELING PRO TIPS▸no joins
![Page 111: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/111.jpg)
DATA MODELING PRO TIPS▸no joins
▸query driven methodology, instead
![Page 112: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/112.jpg)
DATA MODELING PRO TIPS▸no joins
▸query driven methodology, instead
▸denormalize
![Page 113: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/113.jpg)
DATA MODELING PRO TIPS▸no joins
▸query driven methodology, instead
▸denormalize
▸disks are cheap
![Page 114: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/114.jpg)
JON & DANI, I’M STARTING TO GET COLD FEET!
![Page 115: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/115.jpg)
I MISS THE WARM EMBRACE OF RDBMS
I DIDN’T HAVE TO DENORMALIZE
BACK THEN
![Page 116: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/116.jpg)
CHILL OUT
![Page 117: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/117.jpg)
& PREPARE TO BE WOWED
![Page 118: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/118.jpg)
& PREPARE TO BE WOWED
![Page 119: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/119.jpg)
CDM
![Page 120: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/120.jpg)
ROLL UP YOUR SLEEVES
TYPE STUFF
![Page 121: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/121.jpg)
REMEMBER THAT VM?
![Page 122: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/122.jpg)
1.use movielens_small;2.desc tables;3.desc movies;4.select * from movies limit 10;
TRY IT OUT
![Page 123: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/123.jpg)
YOU SHOULD GET…
![Page 124: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/124.jpg)
YOUR 10 MOVIES
![Page 125: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/125.jpg)
ADDING ON5. select * id, name from movies limit 100;6. PICK YOUR FAVORITE MOVIE
BONUS: CAN YOU FIND THE AVERAGE
RATINGS FOR YOUR FAVORITE MOVIE?
![Page 126: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/126.jpg)
MOVIE ID LIST
![Page 127: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/127.jpg)
SELECT A MOVIE
![Page 128: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/128.jpg)
TOP GUN EXAMPLE
![Page 129: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/129.jpg)
TOP GUN EXAMPLE
![Page 130: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/130.jpg)
FIFTH ELEMENT BECAUSE OBVIOUSLY
![Page 131: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/131.jpg)
FIFTH ELEMENT BECAUSE OBVIOUSLY
![Page 132: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/132.jpg)
NICE WORK YOU!
![Page 133: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/133.jpg)
FRIEND #2: SPARK
![Page 134: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/134.jpg)
FRIEND #2: SPARK
![Page 135: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/135.jpg)
BATCH PROCESSING
LOTS OF DATA?
![Page 136: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/136.jpg)
STREAMING & REAL TIME AGGREGATION
![Page 137: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/137.jpg)
MACHINE LEARNING FOR THE INEVITABLE END OF TIMES
![Page 138: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/138.jpg)
GRAPH ANALYTICS
![Page 139: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/139.jpg)
2 WAYS OF WORKING
![Page 140: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/140.jpg)
1. RDDBASED ON FUNCTIONAL PROGRAMMING
![Page 141: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/141.jpg)
blah.map( lambda x : x * 2 )
![Page 142: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/142.jpg)
COOL BUT NOT EASY
![Page 143: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/143.jpg)
COOL BUT NOT EASY
![Page 144: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/144.jpg)
2. DATAFRAMES
![Page 145: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/145.jpg)
PRETTY EASY
![Page 147: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/147.jpg)
TODAY WE TALK BATCH WITH DATAFRAMES AND PYTHON
![Page 148: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/148.jpg)
ROLL UP YOUR SLEEVESOPEN THE OSCON TUTORIAL ON YOUR DESKTOP
![Page 149: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/149.jpg)
FRIENDSHIP LEVELS
![Page 150: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/150.jpg)
OTHER RESOURCES TO LEARN:1. free courses -
www.academy.datatax.com 2. our blogs -
www.rustyrazorblade.com & www.dtrapezoid.com
3. our friend’s blog - https://lostechies.com/ryansvihla/
4. datastax blog - http://www.datastax.com/dev/blog
![Page 151: OSCON TALK: Becoming Friends with Cassandra and Spark](https://reader034.vdocuments.mx/reader034/viewer/2022052117/588a803d1a28abad628b557f/html5/thumbnails/151.jpg)
THANK YOU, MAGICAL HUMANS
@DTRAPEZOID @RUSTYRAZORBLADE