rick mcgeer chief scientist, us ignite march 17, 2014
TRANSCRIPT
Rick McGeerChief Scientist, US Ignite
March 17, 2014
The GENI Experiment Engine
A Fast, Simple On-Ramp for GENI
3
“Hello, World” Across GENI in FIVE MINUTES
4
5
Objectives
• “Hello, World” Across GENI in < 5 Minutes– Allocate GEE “Slicelet”– Upload/Deploy Program (one-click)– Execute– Tear Down
• Non-Objective: be all things to all people– Making things fast & easy means giving up some options– Easy to do something simple, go to “full GENI” for complex stuff
• Use as a starter for a “full GENI” slice
6
GENI Experiment Engine Architecture
7
2 Principles, 3 Services
• Design Principles– Don’t develop new GENI facilities: expose existing facilities– Pre-allocate to bring stuff up quickly
• Services– GEE Compute Engine: PaaS “Slicelet” + Deployment
service– GEE Storage Engine: Global filesystem across the GENI
Infrastructure– GEE Message Service: Distributed Messaging System
across the GENI Infrastructure
8
GEE Compute Engine
• GEE Slicelet– Network of PlanetLab slivers– Private network between slivers (full port space)– Python VM installed and running in slivers
• Pre-allocated– Access through use-once key pair– Fixed topology, IP addresses, etc.
• Automated Deployment (optional)– Fabric-as-a-Service through GEE Portal
9
GEE Network
• Two interfaces/sliver– Private network (full access)– Public internet (outbound only)
• Problem: how to expose the private network– IP addresses are specific to pre-allocated Slicelet– Programmer doesn’t know what addresses she’ll get per sliver– But she does know where slivers are (all Slicelets have slivers in the same
places)
• Symbolic addresses (Python constants) for Slicelet nodes on private network– Resolve to IP addresses in deployed sliver– E.g. GEENetwork.NORTHWESTERN binds to 10.24.128.7 – Resolution in simple binding in downloaded internal file
10
GEE Network Implementation
• Tunneling using GRE• Use of VLANs
– Enough now, more coming– Performance/broadcast isolation within a
VLAN (TBD: experience)
• Overlay on Nick Bastin’s VTS…– Very attractive…
11
GEE Filesystem
• Overall concept: POSIX Filesystem over the GENI substrate
• Will use standard Filesystem calls• Volumes to indicate file location• Implementation
– Swift repositories as block store– Syndicate as meta data repository
12
GEE Message System
• Still in design phase• Global message queues• Current design: RabbitMQ-as-a-Service
– Multiple clients– RESTful API
• Still TBD: centralized vs. distributed• Plan on initial implementation GEC 20
13
How Do I Use This Stuff?
• GEE Compute Engine (today! Well, almost…)– Login at site, ask for slicelet (4 clicks)– Unpack downloaded zip file
• (Easy) add experirment task to downloaded fabfile.py and run $ fab <task>• (Harder) ssh into slivers, yum, scp, run, etc…
• GEE Filesystem (GEC 20+)– Download Python library– Open, read, write, etc….
• GEE Message System (GEC 20, for sure)– Download Python library– Send, receive,….
14
Current Status
• GEE Compute Engine– Slicelets allocated and demonstrated here(!)– Production Use by GEC 20
• GEE Network– Design alternatives being evaluated
• GEE Filesystem Engine– Swift Adapter for Syndicate written– Demonstratable for GEC 20
• GEE Message Engine– Preliminary version under test– Demonstratable for GEC 20
15
GEE Team• Compute Engine:
– Andy Bavier, PW
• Networking– Joe Mambretti and Jim Chen, NW/StarLight
• Filesystem– Stephen Tredger, U Vic
• Message System– Rick McGeer, Ignite
• Overall Architecture– Rick, Andy, Joe, Jack Brassil (HP), Glenn Ricart (US Ignite), Marshall Brinn
• Thanks to our collaborators and friends– Jude Nelson, Nick Bastin, Rob Ricci, Niky Riga, Mark Berman
Thanks!