Early  Experience  with  the  Distributed  Nebula  Cloud    

 Pradeep  Sundarrajan,  Abhishek  Gupta,  Mathew  Ryden,  

Abhishek  Chandra,  Jon  Weissman    

Department  of  CS&E  University  of  Minnesota    

Outline  

•  ConvenIonal  cloud  •  LimitaIons  and  opportuniIes  •  Nebula  project    

The  “Standard”  Cloud  

Results out

Data in

“No limits” §  Storage §  Computing

Computation

Current  Cloud  Model  •  Largely  centralized  •  Pay-­‐as-­‐you-­‐go  •  Strong  guarantees  •  3rd  party  

Appealing  Features  •  Scale/consolidaIon  

– elasIcity,  lower  TCO  

•  Strong  locality  – data  and  compuIng  =>  great  for  analyIcs  

•  Novel  sharing  plaVorm  – data/state  and  applicaIons  =>  gaming,  Web  2.0  

 

 

   

Fraying  at  the  Edges  •  Privacy  

– don’t  want  everything  going  to  the  cloud  but  some  things  

•  Social/community  networks  –  limited  sharing  

•  Locality    –  largely  centralized  cloud  =>  bo[lenecks  

•  to  users  …    •  to/from  data  sources  …  (think:  Big  Data)  

Big  Data  Trend  

•  Big  data  is  distributed  – earth  science:  weather  data,  seismic  data  –  life  science:  GenBank,  NCI  BLAST,  PubMed  – health  science:  GoogleEarth  +  CDC  pandemic  data  – web  2.0:  user  mulImedia  blogs  – “everyone  is  a  sensor”  

Privacy/Locality  Trend  

•  Privacy  –  restrict/filter  data  (think:  paIent  records)  

•  Locality    – mobile  users:  latency  sensiIve  applicaIon  access  – criIcality:    “deliver  go-­‐signal  to  my  insulin  pump”  

Need  New  Features  •  Process  data  in-­‐situ  or  close  by    

– save  Ime  and  money  – privacy-­‐aware  

•  Organize  plaVorm  based  different  noIons  of  “closeness”  – network  distance  –  trusted  nodes  – social  groups  – communiIes  of  interest    

Idea  

•  Make  the  cloud  more  “distributed”  – “move”  it  closer  to  data  – “move”  it  closer  to  end-­‐users  – “move”  it  closer  to  other  clouds      

Example:  Dispersed-­‐Data-­‐Intensive  Services  

n  Data  is  geographically  distributed  n  Costly,  inefficient  to  move  to  central  locaIon  

Example:  Blog  Analysis  

blog1   blog2  

blog3  

Nebula:  A  New  Cloud  Model  

•  Stretch  the  cloud  – exploit  the  rich  collecIon  of  edge  computers    – volunteers  (P2P,  @home),  commercial  (CDNs)    

Nebula  Central  

Nebula  •  Decentralized,  less-­‐managed  cloud  

– dispersed  storage/compute  resources  –  low  user  cost  

Users

Example:  Blog  Analysis  

blog1   blog2  

blog3  

Blog  Results  

0  

20000  

40000  

60000  

80000  

100000  

120000  

140000  

40   80   120   160   240   320  

Time  taken  (sec)  

Amazon    emulator    Nebula  testbed  

# Blogs

Failure  Resistant  

0  

50000  

100000  

150000  

200000  

250000  

300000  

350000  

400000  

450000  

500000  

400   480   560   640   720   800  

Time  Taken  (m

s)  

Totan  Number  of  Blogs  

CCE  -­‐  0  Failures  

Nebula  -­‐  0  Failures  

Nebula  -­‐  1  Failure  

Nebula  -­‐  2  Failure  

Nebula  -­‐  3  Failure  

Another  Example:  Latency-­‐SensiIve  

•  Mobile  service  

Tour of

Paris

How  is  Nebula  different  from  @home?  

Requirement   Nebula   @home  

CollecIve  performance  

High   None  

Locality/Context-­‐awareness  

High   Low  

Statefulness   High/medium   Low  

Common  Service  CharacterisIcs  

•  ElasIc  resource  consumpIon  – scale  up/down  based  on  demand  

•  Geographical  data/user  distribuIon  – execuIon  dependent  on  locaIon  of  data/user    

•  Weak  performance/robustness  requirements  – some  failures  may  be  tolerable  

Inside  Nebula  

•  Nebula  central  •  Chrome  •  Dashboard  •  Datastore    

Nebula  Central  

Nebula  Central  

•  Manager  •  Volunteers  check-­‐in  •  Tracks  global  state  of  other  services  •  Distributes  code  and  nebula  soqware  •  Run  at  UMn  •  Central  point  of  trust  

Nebula  Central  

DataStore  

•  Data  service  that  runs  on  subset  of  nodes  •  Provides  basic  store/retrieval  •  Policy-­‐based  management  for  a  specific  DS  

– capacity,  latency,  fault  tolerance,  durability  

SecureNode  

•  Nebula  nodes  run  a  Chrome  Browser  – secure  sandbox  (NaCL)  naIve  client  inside  – all  naIve  code  executes  inside  it  

Network  Dashboard  

•  Soqware  tool  netstat.cs.umn.edu •  Runs  on  all  nebula  nodes  •  Provides  point-­‐to-­‐point  latency,  ji[er,  bandwidth  

•  Used  by  DataStore  service,  NodeGroup  service  (future)  

Dashboard  Output  

Summary  •  Nebula:  new  cloud  architecture  

– Preserves  cloud  behavior:  APIs,  elasIcity,    transparency  – Stronger  noIon  of  external  locality  – Weaker  noIon  of  internal  locality  

•  Future  work  – End-­‐to-­‐end  system  operaIonal  – Connect  to  the  commercial  cloud  

   “use  the  edge  opportunisIcally”  

                     

       Thank  you!    QuesIons?