bad habits die hard

45
Bad Habits Die Hard Staying on the Right Cassandra Path

Upload: datastax-academy

Post on 20-Feb-2017

421 views

Category:

Technology


0 download

TRANSCRIPT

Page 1: Bad Habits Die Hard

Bad Habits Die HardStaying on the Right Cassandra Path

Page 2: Bad Habits Die Hard

1 Sliding back into a relational mindset

2 Benchmarking, how does this go again?

3 My client is slower than molasses in January

4 Bad operations habits

5 Q & A

© 2015 DataStax, All Rights Reserved. 2

Page 3: Bad Habits Die Hard

Your

© 2015 DataStax, All Rights Reserved. 3

Background is Showing

Page 4: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 4

BadHabitsDieHard

Page 5: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 5

Sliding back into the

relationalmindset

Page 6: Bad Habits Die Hard

Sliding Back into the Relational Mindset

CQL is almost too reminiscent of SQL

© 2015 DataStax, All Rights Reserved. 6

I’m lazy, let’s just copy the schema

Page 7: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 7

Page 8: Bad Habits Die Hard

Sliding Back into the Relational Mindset

CQL is almost too reminiscent of SQL

© 2015 DataStax, All Rights Reserved. 8

Query-Driven Methodology, say it with me

Page 9: Bad Habits Die Hard

Sliding Back into the Relational Mindset

Oh, wait, I forgot there’s no joins in Cassandra!

© 2015 DataStax, All Rights Reserved. 9

That’s OK, we’ll just do those client side

Page 10: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 10

Page 11: Bad Habits Die Hard

Sliding Back into the Relational Mindset

Oh, wait, I forgot there’s no joins in Cassandra!

© 2015 DataStax, All Rights Reserved. 11

Remember, denormalization is cheaper than joins

Page 12: Bad Habits Die Hard

Sliding Back into the Relational Mindset

© 2015 DataStax, All Rights Reserved. 12

I’ve played with C* and it has indexes

Great! We’ll use them then.

Page 13: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 13

Nope

Page 14: Bad Habits Die Hard

Sliding Back into the Relational Mindset

© 2015 DataStax, All Rights Reserved. 14

Querying one partition is cheaper.

I’ve played with C* and it has indexes

Page 15: Bad Habits Die Hard

Sweet! DataStax Enterprise has SparkSQL!

© 2015 DataStax, All Rights Reserved. 15

Great…we can use the ODBC driver just like an RDBMS!

Sliding Back into the Relational Mindset

Page 16: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 16

Page 17: Bad Habits Die Hard

Sliding Back into the Relational Mindset

DataStax Enterprise has SparkSQL!

© 2015 DataStax, All Rights Reserved. 17

Spark is OLAP not OLTP

Page 18: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 18

!=

Remember:

Page 19: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 19

Benchmarking

How does this go

again?

Page 20: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 20

Or perhaps a

SAN?

Benchmarking, how does this go again?

Let us test on a VM!

Page 21: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 21

No Cigar

Page 22: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 22

Benchmarking, how does this go again?Why not select suitable hardware instead?

Test on 5 nodes

with RF=3For search or analytics use more

RAM (128GB+)

32GB RAM 8–12 CPU cores 2TB SSD

Page 23: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 23

Let’s try and break it with cassandra-stress!

With only one client running, the cluster won’t do more than 1,000 tps? What gives?

Benchmarking, how does this go again?

Page 24: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 24

Page 25: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 25

Let’s try and break it with cassandra-stress!

Distributed Database = Distributed Clients

Benchmarking, how does this go again?

Page 26: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 26

How long should we run this for?

No longer than a

sneeze.

Benchmarking, how does this go again?

Page 27: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 27

Page 28: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 28

How long do we run this for?

Aim for days or weeks.

Benchmarking, how does this go again?

Page 29: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 29

Testing it with 100

GBs sounds great!

Benchmarking, how does this go again?

Page 30: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 30

NO

MAKE IT STOP

Page 31: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 31

Use real data volumes, not pretend ones.

Benchmarking, how does this go again?

Page 32: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 32

My client

is slower thanMolasses in

January

Page 33: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 33

Why is my Client slower than molasses in January?

1. Download DataStax Driver2. Used unlogged BATCH

statements3. Connected Client to Load

Balancer

Page 34: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 34

Page 35: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 35

1. Download DataStax Driver2. Use Prepared Statements

and Async Requests3. Use the DataStax Driver to

Load Balance

Why is my Client slower than molasses in January?

Page 36: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 36

BadOPERATIONS

Habits

Page 37: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 37

Bad Operations Habits

1. Meh Backups2. Meh Repairs3. Meh Latency on 1% of our

Ops4. Let’s add a couple nodes to

scale

Page 38: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 38

Page 39: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 39

Use OpsCenter

1. Backup service2. Repair service3. Capacity service4. Proactive Alerts, External

Notifications and Best Practices

Page 40: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 40

Bad Operations Habits

1. Clock skew, you got it covered on that one node right?

2. Configured your Topology Strategy?

3. Set the right snitch?4. Did you just use C* as a

queue?5. Checked your versions are

the same?

Page 41: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 41

NO

Page 42: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 42

Good Operations Habits

1. Install & configure NTP on every node

2. NetworkTopology Strategy3. GossipingPropertyFileSnitch4. Queue? Stop misusing

Cassandra5. Upgrade strategy: rolling

upgrades, node by node

Page 43: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 43

YesFeeling better?

Page 44: Bad Habits Die Hard

• Free Online Training to Up Your Game• www.academy.datastax.com

• More Advanced Courses as well as Beginner• Introduction to Apache Cassandra• Cassandra Core Concepts• Operations and Performance Tuning• Data Modeling• DataStax Enterprise Search with Apache Solr• DataStax Enterprise Analytics with Apache Spark

• Seek Advice• Talk to a friendly Solutions Engineer from DataStax

© 2015 DataStax, All Rights Reserved. 44

Where to learn more:

Page 45: Bad Habits Die Hard

© 2015 DataStax, All Rights Reserved. 45

Questions?