[db tech showcase tokyo 2016] d27: next generation apache cassandra by ヤフー株式会社 星井...
TRANSCRIPT
![Page 1: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/1.jpg)
Next Generation
Apache Cassandra
データ&サイエンスソリューション統括本部データプラットフォーム本部開発3部KVS
星井祥吾
![Page 2: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/2.jpg)
今日のアジェンダ
• 自己紹介
• Cassandra利用状況
• Cassandra利用事例
• NGCCとは?
• NGCC発表内容
• NGCC重要トピック
• 最後に
2
![Page 3: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/3.jpg)
今日のアジェンダ
• 自己紹介
• Cassandra利用状況
• Cassandra利用事例
• NGCCとは?
• NGCC発表内容
• NGCC重要トピック
• 最後に
3
![Page 4: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/4.jpg)
• Cassandraオペレータ– 広告部署から全社KVSチームに異動 -> サービス側にKVSプラット
フォームを提供、運用
• Cassandra自動デプロイシステムの刷新を担当– Chef
– Jenkins
• Cassandraコントリビュータ– https://github.com/shoshii
• Video gamer– PSN ID: kobuchizawa0102
自己紹介
4
![Page 5: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/5.jpg)
今日のアジェンダ
• 自己紹介
• Cassandra利用状況
• Cassandra利用事例
• NGCCとは?
• NGCC発表内容
• NGCC重要トピック
• 最後に
5
![Page 6: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/6.jpg)
提供サービスMedia
US
Search Video Answer Mail
JP
US
JP
Membership C2C Payment C2C EC B2C EC Local
Search Knowledge search MailNews
YAHUOKU!Premium Wallet Loco
![Page 7: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/7.jpg)
利用データベース
7
300
Systems
Yahoo! JAPAN
Cassandra利用システムは約100
![Page 8: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/8.jpg)
Cassandraクラスタ状況
8
50clusters
30TBusages
1000nodes
300,000
read/sec
100,000
write/sec
2016
10nodes
160nodes
…1
Common
clusters
50Special
clusters
50systems
50systems
3DCs
![Page 9: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/9.jpg)
今日のアジェンダ
• 自己紹介
• Cassandra利用状況
• Cassandra利用事例
• NGCCとは?
• NGCC発表内容
• NGCC重要トピック
• 最後に
9
![Page 10: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/10.jpg)
利用事例 #1 : パーソナルデータ
• 月間3000万のアクティブユーザーID属性
• 弊社プライバシーポリシー[1]に基づき、各サービスに応じて様々な属性を格納
10
個人属性
[1] Yahoo! JAPAN利用規約 第2章 プライバシーポリシー
エンドユーザの趣味嗜好
アプリ通知メッセージ
![Page 11: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/11.jpg)
利用事例 #2 : 行動履歴
• 弊社プライバシーポリシー[1]に基づき、各サービスの履歴データを格納
11
[1] Yahoo! JAPAN利用規約 第2章 プライバシーポリシー
行動履歴
広告クリック履歴・CV履歴
リアルタイム検索ツイート閲覧ログ
![Page 12: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/12.jpg)
利用事例 #3 : コンテンツキャッシュ
• RDBのコンテンツ情報や、集計情報などを格納し、各サービスのスケールアウトに活用
12
メタデータ集計情報
電子書籍:商品ID、レコメ情報
ニュース:テーマID、記事ID、記事内容
![Page 13: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/13.jpg)
今日のアジェンダ
• 自己紹介
• Cassandra利用状況
• Cassandra利用事例
• NGCCとは?
• NGCC発表内容
• NGCC重要トピック
• 最後に
13
![Page 14: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/14.jpg)
Next Generation Cassandra Conference
• Cassandraトップコントリビュータが集まる国際会議
• ほぼDataStax社の主要メンバで構成されるが、最近は外部メンバの取り込みに積極的
• 各自が議題を持ち寄って議論する
– 専任のファシリテータがいないので、待ちの姿勢だと議論に参加できないことも
• 今年で3年目、3回目の開催
• youtube公開動画:資料末尾の付録参照
14
![Page 15: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/15.jpg)
弊社注目ポイント
• (弊社発表分)コネクション過多時の問題の解決
• 巨大パーティション対応
15
![Page 16: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/16.jpg)
今日のアジェンダ
• 自己紹介
• Cassandra利用状況
• Cassandra利用事例
• NGCCとは?
• NGCC発表内容
• NGCC重要トピック
• 最後に
16
![Page 17: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/17.jpg)
How to manage 30,000+ connections
17
![Page 18: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/18.jpg)
What’s the problem?
18
![Page 19: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/19.jpg)
Cassandra process goes down when too many clients connect it
19
Client machine
Apache child process
Client machine
Client machine
![Page 20: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/20.jpg)
• 200 client machines
Cassandra process goes down when too many clients connect it
20
![Page 21: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/21.jpg)
• 200 client machines * 128 apache child processes
Cassandra process goes down when too many clients connect it
21
![Page 22: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/22.jpg)
• 200 client machines * 128 apache child processes * 2 (request + heart beat) =
Cassandra process goes down when too many clients connect it
22
![Page 23: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/23.jpg)
• 200 client machines * 128 apache child processes * 2 (request + heart beat) =
51,200 connections / node
Cassandra process goes down when too many clients connect it
23
![Page 24: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/24.jpg)
Cassandra process goes down when too many clients connect it
24
Client machine
Apache child process
51,200 > 32,768 ( max open file num )
Client machine
Client machine
写真:アフロ
![Page 25: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/25.jpg)
Cassandra process goes down when too many clients connect it
25
Process down
写真:アフロ
![Page 26: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/26.jpg)
Cassandra process goes down when too many clients connect it
26
Process down
写真:アフロ
![Page 27: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/27.jpg)
Our solution
27
![Page 28: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/28.jpg)
Cassandra Proxy
28
写真:アフロ
![Page 29: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/29.jpg)
Our solution ~ Cassandra Proxy
29
Client machine
Apache child process
Cassandra Proxy
Client machine
Client machine
写真:アフロ
![Page 30: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/30.jpg)
• 200 client machines
Our solution ~ Cassandra Proxy
30
![Page 31: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/31.jpg)
• 200 client machines * 128 apache child processes
Our solution ~ Cassandra Proxy
31
![Page 32: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/32.jpg)
• 200 client machines * 128 apache child processes 1 proxy
Our solution ~ Cassandra Proxy
32写真:アフロ
![Page 33: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/33.jpg)
• 200 client machines * 128 apache child processes 1 proxy * 2 (request + heart beat) =
Our solution ~ Cassandra Proxy
33写真:アフロ
![Page 34: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/34.jpg)
• 200 client machines * 128 apache child processes 1 proxy * 2 (request + heart beat)
= 400 connections / node
Our solution ~ Cassandra Proxy
34写真:アフロ
![Page 35: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/35.jpg)
• 51,200 -> 400 connections / node
Our solution ~ Cassandra Proxy
35写真:アフロ
![Page 36: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/36.jpg)
Consequences
36
![Page 37: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/37.jpg)
• Stress test
– 1 msec / request (95%)
– 1 Cassandra Proxy can send 300 requests/sec to C*
Consequences
37写真:アフロ
![Page 38: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/38.jpg)
• Production– 370 client machines – 33 Cassandra nodes
– 3,000 read requests / sec (cluster)• To be 100,000 read requests / sec
– Advertisement platform
– Available on datastax cpp driver
Consequences
38
![Page 39: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/39.jpg)
• In the future?
– Cassandra Proxy could become one of the functions of datastax cpp driver?
Consequences
39
![Page 40: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/40.jpg)
今日のアジェンダ
• 自己紹介
• Cassandra利用状況
• Cassandra利用事例
• NGCCとは?
• NGCC発表内容
• NGCC重要トピック
• 最後に
40
![Page 41: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/41.jpg)
Large partition in Cassandra
41
![Page 42: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/42.jpg)
remark
• This is a summary of following tickets:– https://issues.apache.org/jira/browse/CASSANDRA-11206
– https://issues.apache.org/jira/browse/CASSANDRA-9738
![Page 43: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/43.jpg)
Agenda
• Recap the read path
• What’s the problem?
• Solutions
![Page 44: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/44.jpg)
High level: read path
Row Cache
Key Cache
SSTables Mem Table
1. Check row cache before going to key cache
2. Check the key cache to get the offsets to data
3. Find the offsets to data and retrieve data
4. Merge data from sstables and memtable
5. Populate row cache with new row returned
http://docs.datastax.com/en/cassandra/3.x/cassandra/dml/dmlAboutReads.html
![Page 45: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/45.jpg)
Pattern 1. The row is in row cache
PartitionSummary
Disk
Mem Table
CompressionOffsets
Bloom Filter
Row Cache
Heap Off Heap
Key Cache
PartitionIndex
Data
1. read request
2. return row when that is in row cache
![Page 46: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/46.jpg)
Pattern 2. The key is in key cache
PartitionSummary
Disk
Mem Table
CompressionOffsets
Bloom Filter
Row Cache
Heap Off Heap
Key Cache
PartitionIndex
Data
1. read request
2. Check bloom filters 3. Check the partition key is in key cache
4. Find the offset to the result set
5. Access the result set
![Page 47: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/47.jpg)
Pattern 3. The key is not cached
PartitionSummary
Disk
Mem Table
CompressionOffsets
Bloom Filter
Row Cache
Heap Off Heap
Key Cache
PartitionIndex
Data
1. read request
2. Miss -> Check bloom filters
3. Check the partition key is in key cache
4. Miss -> Bsearch the close location of index
5. Disk scan to find the offsets 6. Find the offset into the result set
7. Access the result set
8. Update key cache
![Page 48: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/48.jpg)
What’s the problem?
• GC pressure by key cache when a large partition is read
![Page 49: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/49.jpg)
Partition Index Recap
• http://distributeddatastore.blogspot.jp/2013/08/cassandra-sstable-storage-format.html
![Page 50: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/50.jpg)
RowIndexEntry• Partition size < 64 kb
– RowIndexEntry• Position• Seriarized size of data
• Partition size > 64 kb– IndexedEntry
• Position• Seriarized size of data• IndexInfo[]
– Seriarize method– Offset– width– Etc.
Approximation on 16 byte value
1mb : 3kb / > 200 objects4mb : 11kb / > 800 objects64mb : 180kb / > 13k objects512mb : 1.4mb / > 106k objects
![Page 51: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/51.jpg)
3. The key is not cached
PartitionSummary
Disk
Mem Table
CompressionOffsets
Bloom Filter
Row Cache
Heap Off Heap
Key Cache
PartitionIndex
Data
1. read request
2. Miss -> Check bloom filters
3. Check the partition key is in key cache
4. Miss -> Bsearch the close location of index
5. Disk scan to find the offsets 6. Find the offsets into the result set
7. Access the result set
8. Update key cache
9. GC, GC, GC…
![Page 52: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/52.jpg)
Current solution
• If partition size < column_index_cache_size_in_kb(configurable)– IndexedEntry is kept on heap
• Otherwise– Always read from disk when needed
• https://issues.apache.org/jira/browse/CASSANDRA-11206
• https://www.youtube.com/watch?v=qa84vABqftM
![Page 53: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/53.jpg)
Other possible solutions
• IndexInfo never be kept on heap– Read from disk when needed
– degrades performance when small partition is read
![Page 54: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/54.jpg)
Other possible solutions
• Migrate key cache to be fully off heap– https://issues.apache.org/jira/browse/CASSANDRA-9738
– Serialization & deserialization cost so much when large partition is read
• Will Birch help us to solve this problem?
– https://issues.apache.org/jira/browse/CASSANDRA-9754
![Page 55: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/55.jpg)
What we go for
• 来年もNGCCに呼んでもらえるように頑張ろう!
そのためには?: Cassandraコミュニティに貢献する
1. 日本で一番Apache Cassandraを使っている会社になる。
2. Cassandraのコード改善や問題提起の活動を継続する。
3. Cassandraコミュニティの人と仲良くなる。
55
![Page 56: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/56.jpg)
DAY3 ?
• 3日目(おまけ)
56
![Page 57: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/57.jpg)
今日のアジェンダ
• 自己紹介
• Cassandra利用状況
• Cassandra利用事例
• NGCCとは?
• NGCC発表内容
• NGCC重要トピック
• 最後に
57
![Page 58: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/58.jpg)
最後に
58
D I S T R I B U T E D
大規模分散処理
H A R D W A R E
ハードウエア分野
N E T W O R K
ネットワーク分野
mailto:[email protected]
![Page 59: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/59.jpg)
ご清聴ありがとうございました
59
![Page 60: [db tech showcase Tokyo 2016] D27: Next Generation Apache Cassandra by ヤフー株式会社 星井 祥吾](https://reader034.vdocuments.mx/reader034/viewer/2022042611/5876b9b71a28abad1a8b68d5/html5/thumbnails/60.jpg)
付録:NGCC動画集• Next-Gen Schema
– https://www.youtube.com/watch?v=eAWRj0kqpvU
• Change Data Capture– https://www.youtube.com/watch?v=Y0fOxa3tC98
• Explicit support for time series data– https://www.youtube.com/watch?v=CmsQNNdDuSA
• Automated Repair– https://www.youtube.com/watch?v=8sGUn6Q2bUU
• Storage format and key cache changes to support large partitions– https://www.youtube.com/watch?v=qa84vABqftM
• SASI update– https://www.youtube.com/watch?v=yUFoSAg6rA4
• Instagram’s use cases– https://www.youtube.com/watch?v=VwhovoqavT4
• Lightning Talks– https://www.youtube.com/watch?v=6y5UV4OTawg
60