bad habits die hard
TRANSCRIPT
Bad Habits Die HardStaying on the Right Cassandra Path
1 Sliding back into a relational mindset
2 Benchmarking, how does this go again?
3 My client is slower than molasses in January
4 Bad operations habits
5 Q & A
© 2015 DataStax, All Rights Reserved. 2
Your
© 2015 DataStax, All Rights Reserved. 3
Background is Showing
© 2015 DataStax, All Rights Reserved. 4
BadHabitsDieHard
© 2015 DataStax, All Rights Reserved. 5
Sliding back into the
relationalmindset
Sliding Back into the Relational Mindset
CQL is almost too reminiscent of SQL
© 2015 DataStax, All Rights Reserved. 6
I’m lazy, let’s just copy the schema
© 2015 DataStax, All Rights Reserved. 7
Sliding Back into the Relational Mindset
CQL is almost too reminiscent of SQL
© 2015 DataStax, All Rights Reserved. 8
Query-Driven Methodology, say it with me
Sliding Back into the Relational Mindset
Oh, wait, I forgot there’s no joins in Cassandra!
© 2015 DataStax, All Rights Reserved. 9
That’s OK, we’ll just do those client side
© 2015 DataStax, All Rights Reserved. 10
Sliding Back into the Relational Mindset
Oh, wait, I forgot there’s no joins in Cassandra!
© 2015 DataStax, All Rights Reserved. 11
Remember, denormalization is cheaper than joins
Sliding Back into the Relational Mindset
© 2015 DataStax, All Rights Reserved. 12
I’ve played with C* and it has indexes
Great! We’ll use them then.
© 2015 DataStax, All Rights Reserved. 13
Nope
Sliding Back into the Relational Mindset
© 2015 DataStax, All Rights Reserved. 14
Querying one partition is cheaper.
I’ve played with C* and it has indexes
Sweet! DataStax Enterprise has SparkSQL!
© 2015 DataStax, All Rights Reserved. 15
Great…we can use the ODBC driver just like an RDBMS!
Sliding Back into the Relational Mindset
© 2015 DataStax, All Rights Reserved. 16
Sliding Back into the Relational Mindset
DataStax Enterprise has SparkSQL!
© 2015 DataStax, All Rights Reserved. 17
Spark is OLAP not OLTP
© 2015 DataStax, All Rights Reserved. 18
!=
Remember:
© 2015 DataStax, All Rights Reserved. 19
Benchmarking
How does this go
again?
© 2015 DataStax, All Rights Reserved. 20
Or perhaps a
SAN?
Benchmarking, how does this go again?
Let us test on a VM!
© 2015 DataStax, All Rights Reserved. 21
No Cigar
© 2015 DataStax, All Rights Reserved. 22
Benchmarking, how does this go again?Why not select suitable hardware instead?
Test on 5 nodes
with RF=3For search or analytics use more
RAM (128GB+)
32GB RAM 8–12 CPU cores 2TB SSD
© 2015 DataStax, All Rights Reserved. 23
Let’s try and break it with cassandra-stress!
With only one client running, the cluster won’t do more than 1,000 tps? What gives?
Benchmarking, how does this go again?
© 2015 DataStax, All Rights Reserved. 24
© 2015 DataStax, All Rights Reserved. 25
Let’s try and break it with cassandra-stress!
Distributed Database = Distributed Clients
Benchmarking, how does this go again?
© 2015 DataStax, All Rights Reserved. 26
How long should we run this for?
No longer than a
sneeze.
Benchmarking, how does this go again?
© 2015 DataStax, All Rights Reserved. 27
© 2015 DataStax, All Rights Reserved. 28
How long do we run this for?
Aim for days or weeks.
Benchmarking, how does this go again?
© 2015 DataStax, All Rights Reserved. 29
Testing it with 100
GBs sounds great!
Benchmarking, how does this go again?
© 2015 DataStax, All Rights Reserved. 30
NO
MAKE IT STOP
© 2015 DataStax, All Rights Reserved. 31
Use real data volumes, not pretend ones.
Benchmarking, how does this go again?
© 2015 DataStax, All Rights Reserved. 32
My client
is slower thanMolasses in
January
© 2015 DataStax, All Rights Reserved. 33
Why is my Client slower than molasses in January?
1. Download DataStax Driver2. Used unlogged BATCH
statements3. Connected Client to Load
Balancer
© 2015 DataStax, All Rights Reserved. 34
© 2015 DataStax, All Rights Reserved. 35
1. Download DataStax Driver2. Use Prepared Statements
and Async Requests3. Use the DataStax Driver to
Load Balance
Why is my Client slower than molasses in January?
© 2015 DataStax, All Rights Reserved. 36
BadOPERATIONS
Habits
© 2015 DataStax, All Rights Reserved. 37
Bad Operations Habits
1. Meh Backups2. Meh Repairs3. Meh Latency on 1% of our
Ops4. Let’s add a couple nodes to
scale
© 2015 DataStax, All Rights Reserved. 38
© 2015 DataStax, All Rights Reserved. 39
Use OpsCenter
1. Backup service2. Repair service3. Capacity service4. Proactive Alerts, External
Notifications and Best Practices
© 2015 DataStax, All Rights Reserved. 40
Bad Operations Habits
1. Clock skew, you got it covered on that one node right?
2. Configured your Topology Strategy?
3. Set the right snitch?4. Did you just use C* as a
queue?5. Checked your versions are
the same?
© 2015 DataStax, All Rights Reserved. 41
NO
© 2015 DataStax, All Rights Reserved. 42
Good Operations Habits
1. Install & configure NTP on every node
2. NetworkTopology Strategy3. GossipingPropertyFileSnitch4. Queue? Stop misusing
Cassandra5. Upgrade strategy: rolling
upgrades, node by node
© 2015 DataStax, All Rights Reserved. 43
YesFeeling better?
• Free Online Training to Up Your Game• www.academy.datastax.com
• More Advanced Courses as well as Beginner• Introduction to Apache Cassandra• Cassandra Core Concepts• Operations and Performance Tuning• Data Modeling• DataStax Enterprise Search with Apache Solr• DataStax Enterprise Analytics with Apache Spark
• Seek Advice• Talk to a friendly Solutions Engineer from DataStax
© 2015 DataStax, All Rights Reserved. 44
Where to learn more:
© 2015 DataStax, All Rights Reserved. 45
Questions?