tachyon: memory-speed data sharing - rise sics · pdf filetachyon: memory-speed data sharing...
TRANSCRIPT
![Page 1: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/1.jpg)
Tachyon:
memory-speed data sharing
Ali Ghodsi, Haoyuan (HY) Li,
Matei Zaharia, Scott Shenker, Ion Stoica
UC Berkeley
![Page 2: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/2.jpg)
Memory trumps everything else
• RAM throughput increasing exponentially
• Disk throughput increasing slowly
Memory-locality key to interactive response time
![Page 3: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/3.jpg)
Realized by many…
• Frameworks already leverage memory
– e.g. Spark, Piccolo, GraphX…
![Page 4: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/4.jpg)
Example: -
• Fast in-memory data processing within a job
– Keep only one copy in-memory copy JVM
– Track lineage of operations used to derive data
– Upon failure, use lineage to re-compute data
map
filter map
join reduce
Lineage Tracking
![Page 5: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/5.jpg)
Challenge 1
Spark Task
Spark memory block manager
block 1
block 3
HDFS disk
block 1
block 3
block 2
block 4
execution engine & storage engine same JVM process
![Page 6: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/6.jpg)
Challenge 1
crash
Spark memory block manager
block 1
block 3
HDFS disk
block 1
block 3
block 2
block 4
execution engine & storage engine same JVM process
![Page 7: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/7.jpg)
Challenge 1
JVM crash: lose all cache
HDFS disk
block 1
block 3
block 2
block 4
execution engine & storage engine same JVM process
crash
![Page 8: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/8.jpg)
Challenge 2
JVM heap overhead: GC & duplicate memory per job
Spark Task
Spark mem block manager
block 1
block 3
Spark Task
Spark mem block manager
block 3
Block 1
HDFS disk
block 1
block 3
block 2
block 4
execution engine & storage engine same JVM process (GC & duplication)
![Page 9: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/9.jpg)
Challenge 3
Different jobs share data: Slow writes to disk
Spark Task
Spark mem block manager
block 1
block 3
Spark Task
Spark mem block manager
block 3
block 1
HDFS disk
block 1
block 3
block 2
block 4
storage engine & execution engine same JVM process (slow writes)
![Page 10: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/10.jpg)
Challenge 3
Different frameworks share data: Slow writes to disk
Spark Task
Spark mem block manager
block 1
block 3
Hadoop MR
YARN
HDFS disk
block 1
block 3
block 2
Block 4
storage engine & execution engine same JVM process (slow writes)
![Page 11: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/11.jpg)
Tachyon
Reliable data sharing at memory-speed within and across cluster frameworks/jobs
![Page 12: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/12.jpg)
Challenge 1 revisited
Spark Task
Spark memory block manager
block 1
HDFS disk
block 1
block 3
block 2
block 4
execution engine & storage engine same JVM process
Tachyon in-memory
block 1
block 3 block 4
![Page 13: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/13.jpg)
Challenge 1 revisited
Spark memory block manager
block 1
HDFS disk
block 1
block 3
block 2
block 4
execution engine & storage engine same JVM process
Tachyon in-memory
block 1
block 3 block 4
crash
HDFS disk
block 1
block 3
block 2
block 4
![Page 14: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/14.jpg)
Challenge 1 revisited
HDFS disk
block 1
block 3
block 2
block 4
execution engine & storage engine same JVM process
Tachyon in-memory
block 1
block 3 block 4
crash
HDFS disk
block 1
block 3
block 2
block 4
JVM crash: keep memory-cache
![Page 15: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/15.jpg)
Challenge 2 revisited
Off-heap memory storage No GC & one memory copy
Spark Task
Spark mem block 1
Spark Task
Spark mem block 4
HDFS disk
block 1
Block 3
block 2
Block 4
execution engine & storage engine same JVM process (no GC & duplication)
HDFS disk
block 1
block 3
block 2
block 4
Tachyon in-memory
block 1
block 3 block 4
![Page 16: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/16.jpg)
Challenge 3 revisited
Different frameworks share at memory-speed
execution engine & storage engine same JVM process (fast writes)
Spark Task
Spark mem block 1
Hadoop MR
YARN
HDFS disk
block 1
Block 3
block 2
Block 4
HDFS disk
block 1
block 3
block 2
block 4
Tachyon in-memory
block 1
block 3 block 4
![Page 17: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/17.jpg)
Tachyon and Spark
Spark’s of off-JVM-heap RDD-store
• In-memory RDDs (serialized)
• Fault-tolerant cache
Enables
• avoiding GC overhead
• fine-grained executors
• fast RDD sharing
![Page 18: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/18.jpg)
Tachyon research vision
Vision
• Push lineage down to storage layer
• Use memory aggressively
Approach
• One in-memory copy
• Rely on recomputation for fault-tolerance
![Page 19: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/19.jpg)
Architecture
![Page 20: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/20.jpg)
Comparison with in Memory HDFS
![Page 21: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/21.jpg)
Further Improve Spark’s Performance
Grep
![Page 22: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/22.jpg)
Master Faster Recovery
![Page 23: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/23.jpg)
Open Source Status
• New release
– V0.4.0 (July 2014)
– 20 Developers (7 from Berkeley, 13 from outside)
– 11 Companies
– Writes go synchronously to under filesystem
(No lineage information in Developer Preview release)
– MapReduce and Spark can run without any code change
(ser/de becomes the new bottleneck)
![Page 24: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/24.jpg)
Using HDFS vs Tachyon
• Spark val file = sc.textFile(“hdfs://ip:port/path”)
• Shark CREATE TABLE orders_cached AS SELECT * FROM
orders;
• Hadoop MapReduce hadoop jar examples.jar wordcount
hdfs://localhost/input hdfs://localhost/output
![Page 25: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/25.jpg)
Using HDFS vs Tachyon
• Spark val file = sc.textFile(“tachyon://ip:port/path”)
• Shark CREATE TABLE orders_tachyon AS SELECT * FROM
orders;
• Hadoop MapReduce hadoop jar examples.jar wordcount
tachyon://localhost/input
tachyon://localhost/output
![Page 26: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/26.jpg)
Thanks to Redhat!
![Page 27: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/27.jpg)
Future Research Focus
• Integration with HDFS caching
• Memory Fair Sharing
• Random Access Abstraction
• Mutable Data Support
![Page 28: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/28.jpg)
Acknowledgments Calvin Jia, Nick Lanham, Grace Huang, Mark Hamstra,
Bill Zhao, Rong Gu, Hobin Yoon, Vamsi Chitters,
Joseph Jin-Chuan Tang, Xi Liu, Qifan Pu, Aslan Bekirov,
Reynold Xin, Xiaomin Zhang, Achal Soni, Xiang Zhong,
Dilip Joseph, Srinivas Parayya, Tim St. Clair,
Shivaram Venkataraman, Andrew Ash
![Page 29: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/29.jpg)
Tachyon Summary
• As more workloads move into memory, big data
data sharing across frameworks will become a
bottleneck
– Tachyon provides in-memory, fault-tolerant data
sharing across frameworks
![Page 30: Tachyon: memory-speed data sharing - RISE SICS · PDF fileTachyon: memory-speed data sharing Ali Ghodsi, Haoyuan (HY) Li, Matei Zaharia, Scott Shenker, Ion Stoica UC Berkeley](https://reader031.vdocuments.mx/reader031/viewer/2022022003/5a9e53527f8b9a2e688d90d5/html5/thumbnails/30.jpg)
Thanks!
• More: https://github.com/amplab/tachyon