![Page 1: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/1.jpg)
Early Experience with the Distributed Nebula Cloud
Pradeep Sundarrajan, Abhishek Gupta, Mathew Ryden,
Abhishek Chandra, Jon Weissman
Department of CS&E University of Minnesota
![Page 2: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/2.jpg)
Outline
• ConvenIonal cloud • LimitaIons and opportuniIes • Nebula project
![Page 3: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/3.jpg)
The “Standard” Cloud
Results out
Data in
“No limits” § Storage § Computing
Computation
![Page 4: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/4.jpg)
Current Cloud Model • Largely centralized • Pay-‐as-‐you-‐go • Strong guarantees • 3rd party
![Page 5: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/5.jpg)
Appealing Features • Scale/consolidaIon
– elasIcity, lower TCO
• Strong locality – data and compuIng => great for analyIcs
• Novel sharing plaVorm – data/state and applicaIons => gaming, Web 2.0
![Page 6: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/6.jpg)
Fraying at the Edges • Privacy
– don’t want everything going to the cloud but some things
• Social/community networks – limited sharing
• Locality – largely centralized cloud => bo[lenecks
• to users … • to/from data sources … (think: Big Data)
![Page 7: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/7.jpg)
Big Data Trend
• Big data is distributed – earth science: weather data, seismic data – life science: GenBank, NCI BLAST, PubMed – health science: GoogleEarth + CDC pandemic data – web 2.0: user mulImedia blogs – “everyone is a sensor”
![Page 8: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/8.jpg)
Privacy/Locality Trend
• Privacy – restrict/filter data (think: paIent records)
• Locality – mobile users: latency sensiIve applicaIon access – criIcality: “deliver go-‐signal to my insulin pump”
![Page 9: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/9.jpg)
Need New Features • Process data in-‐situ or close by
– save Ime and money – privacy-‐aware
• Organize plaVorm based different noIons of “closeness” – network distance – trusted nodes – social groups – communiIes of interest
![Page 10: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/10.jpg)
Idea
• Make the cloud more “distributed” – “move” it closer to data – “move” it closer to end-‐users – “move” it closer to other clouds
![Page 11: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/11.jpg)
Example: Dispersed-‐Data-‐Intensive Services
n Data is geographically distributed n Costly, inefficient to move to central locaIon
![Page 12: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/12.jpg)
Example: Blog Analysis
blog1 blog2
blog3
![Page 13: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/13.jpg)
Nebula: A New Cloud Model
• Stretch the cloud – exploit the rich collecIon of edge computers – volunteers (P2P, @home), commercial (CDNs)
Nebula Central
![Page 14: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/14.jpg)
Nebula • Decentralized, less-‐managed cloud
– dispersed storage/compute resources – low user cost
Users
![Page 15: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/15.jpg)
Example: Blog Analysis
blog1 blog2
blog3
![Page 16: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/16.jpg)
Blog Results
0
20000
40000
60000
80000
100000
120000
140000
40 80 120 160 240 320
Time taken (sec)
Amazon emulator Nebula testbed
# Blogs
![Page 17: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/17.jpg)
Failure Resistant
0
50000
100000
150000
200000
250000
300000
350000
400000
450000
500000
400 480 560 640 720 800
Time Taken (m
s)
Totan Number of Blogs
CCE -‐ 0 Failures
Nebula -‐ 0 Failures
Nebula -‐ 1 Failure
Nebula -‐ 2 Failure
Nebula -‐ 3 Failure
![Page 18: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/18.jpg)
Another Example: Latency-‐SensiIve
• Mobile service
Tour of
Paris
![Page 19: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/19.jpg)
How is Nebula different from @home?
Requirement Nebula @home
CollecIve performance
High None
Locality/Context-‐awareness
High Low
Statefulness High/medium Low
![Page 20: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/20.jpg)
Common Service CharacterisIcs
• ElasIc resource consumpIon – scale up/down based on demand
• Geographical data/user distribuIon – execuIon dependent on locaIon of data/user
• Weak performance/robustness requirements – some failures may be tolerable
![Page 21: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/21.jpg)
Inside Nebula
• Nebula central • Chrome • Dashboard • Datastore
Nebula Central
![Page 22: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/22.jpg)
Nebula Central
• Manager • Volunteers check-‐in • Tracks global state of other services • Distributes code and nebula soqware • Run at UMn • Central point of trust
Nebula Central
![Page 23: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/23.jpg)
DataStore
• Data service that runs on subset of nodes • Provides basic store/retrieval • Policy-‐based management for a specific DS
– capacity, latency, fault tolerance, durability
![Page 24: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/24.jpg)
SecureNode
• Nebula nodes run a Chrome Browser – secure sandbox (NaCL) naIve client inside – all naIve code executes inside it
![Page 25: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/25.jpg)
Network Dashboard
• Soqware tool netstat.cs.umn.edu • Runs on all nebula nodes • Provides point-‐to-‐point latency, ji[er, bandwidth
• Used by DataStore service, NodeGroup service (future)
![Page 26: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/26.jpg)
Dashboard Output
![Page 27: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/27.jpg)
Summary • Nebula: new cloud architecture
– Preserves cloud behavior: APIs, elasIcity, transparency – Stronger noIon of external locality – Weaker noIon of internal locality
• Future work – End-‐to-‐end system operaIonal – Connect to the commercial cloud
“use the edge opportunisIcally”
![Page 28: Early&Experience&with&the&Distributed& NebulaCloud&& · blog3& Blog&Results& 0 20000 40000 60000 80000 100000 120000 140000 40 80 120 160 240 320! Amazon&& emulator& & Nebula testbed&](https://reader033.vdocuments.mx/reader033/viewer/2022052805/5f0c9d077e708231d4364662/html5/thumbnails/28.jpg)
Thank you! QuesIons?