are you ready for the tutorial?

Sponsored by the National Science Foundation 1GEC16: March 20, 2013

Are you ready for the tutorial?1. Did you do the pre-work?

A. Are you able to login to the GENI Portal?• Using your home institution username and password, or• Using a GPO issued username and password, or• A temporary account

B. Have the GEC16 Tutorial VM including both:• VirtualBox• GEC16 Tutorial VM image

2. Sign In to GENI Portal

3. Join Project: GEC16AdvNetw4. Grab a Worksheet and Instructions

Sponsored by the National Science Foundation

Tutorial: Advanced Network Experiments

Niky RigaGENI Project Office

Jay AikatUniversity, North Carolina

GEC16, 20 March2013

$ omni.py createsliver aliceslice myRSpec.xml INFO:omni:Loading config file omni_config INFO:omni:Using control framework pgeni INFO:omni:Slice urn:publicid:IDN+pgeni.gpolab. expires within 1 day on 2011-07-07 INFO:omni:Creating sliver(s) from rspec fileINFO:omni:Writing result of createsliver for INFO:omni:Writing to ‘aliceslice-manifest-rspeINFO:omni: -----------------------------------INFO:omni: Completed createsliver:

Options as run: aggregate: https://www.emulab. framework: pgeni native: True

Args: createsliver aliceslice myRSpec.xml

Result Summary: Slice urn:publicid:IDN+pgeniReserved resources on https://www.emulab.net/p Saved createsliver results to aliceslice-manINFO:omni: ===================================

omni


Tutorial Outline• Part I: Learn how to use Omni Presentation and Hands-on

• Part II: Presentation of Advanced Experiments Tmix traffic generation Click Software Routers

• Part III: Hands-on Experiments Split in two groups:

– Tmix– Non-IP routing using Click


Part I: Introduction to Omni

Niky Riga, Sarah Edwards GENI Project Office






Hands On Exercise

Do a Simple Client-Server Experiment

Reserve two hosts using Omni and exchange traffic between them


New GENI Terminology

AM API

slivercredentials

certificate


Experimenter

GENI Credentials, GENI Certificates, GENI Keys

GENI Certificate : identifies user; created by the Portal and you can download it through the webUI, it is an ssl certificateGENI Credential : identifies user's permissions; you almost never have to handle; tools query if from Portal Services

GENI Keys : Access to compute resources are through ssh-keys, you can use your own or use the one GENI created for you Passphrase : is the password that protects your private key


Omni: Resource Reservation tool

• A command line experimenter tool

• Create slices and slivers using the GENI AM API

• Written in and scriptable from Python

• Use existing accounts– GENI Portal– ProtoGENI– PlanetLab

• Works with aggregates that implement the GENI AM API – ProtoGENI, PlanetLab,

OpenFlow, InstaGENI, ExoGENI





http://trac.gpolab.bbn.com/gcf/wiki/Omni

http://trac.gpolab.bbn.com/gcf/wiki/Omni


Omni Command Workflow

createslice

listmyslices

getversion

renewslice

createsliver

listresources

sliverstatus

deletesliver

Repeat for each aggregate

Repeat for each aggregate

CreateSlice

CreateSliver Cleanup

Legend: AM API command


omni_configure.py[omni]default_cf = portalusers = gpousr21# ---------- Users ----------[gpousr21]urn = urn:publicid:IDN+pgeni.gpolab.bbn.com+user+gpousr21 keys = ~/.ssh/geni_key

...

You only need the Omni bundle from the portal Standard AM nicknames

Run this script for standard configuration:1. Download GENI bundle2. Run the script


omni_config[omni]default_cf = pgeni # Tutorial accounts are on GPO’s PGusers = gpousr21 # gpousr21’s keys loaded on the VM to allow login

# ---------- Users ----------[gpousr21]urn = urn:publicid:IDN+pgeni.gpolab.bbn.com+user+gpousr21 # Really important to get the keys correct!!!#key to load on VMkeys = ~/Tutorials/Omni/gpousr20/ssh/gpousr20_key.pub

# default aggregates to run omni commands onaggregates = http://emulab.net/protogeni/xmlrpc/am, https://pgeni.gpolab.bbn.com/protogeni/xmlrpc/am, ...


Getting Help

• omni.py –hLists all commands and their argumentsLists all command line optionsLists Omni versionLists url to find out more information about Omni

• Omni Troubleshooting page: http://trac.gpolab.bbn.com/gcf/wiki/OmniTroubleShoot

http://trac.gpolab.bbn.com/gcf/wiki/OmniTroubleShoot

http://trac.gpolab.bbn.com/gcf/wiki/OmniTroubleShoot


Before we start

• Many people will be accessing the resources, so some calls might fail. Wait a bit and try again!

• Omni is a command line tool, copy-paste is your friend

• You can copy-paste between your computer and the VM.


• Part I: Design/Setup– Step 1: Configure Omni– Step 2: Obtain Resources

• Part II: Execute– Execute Experiment– View results

• Part III: Finish– Teardown Experiment


Configure omni

geni@NSDI13-Tutorials:~$> omni-configure.py –f portal


• Part I: Design/Setup– Step 1: Configure Omni– Step 2: Obtain Resources

• Part II: Execute– Execute Experiment– View results



Experimenter

For the Tutorial: Keep it simple

Slice

Sliver

AM

ProtoGENI Aggregate

You Once you understand how to run this simple experiment, scaling up

is easy!

GENI Portal


Request Specifications

Resource Specification Document (RSpec)• XML document that describes resources

– hosts, links, switches, etc

GENI v3• AM extensions• Client extensions• Tool extensions


The Three Types of RSpecs

AggregateManager

Client

Advertisement

Request

Manifest

Advertisement RSpec : What does the AM have?Request RSpec : What does the Experimenter want? Manifest RSpec: What does the Experimenter have?

omni.py list

resources sl

icename

omni.py list

resources

omni.py createsliver slicename reqRSpec


Omni Commands

• omni.py getversion• omni.py createslice slicename• omni.py renewslice slicename date• omni.py listmyslices username• omni.py createsliver slicename requestRSpec

• omni.py sliverstatus slicename• omni.py listresources [slicename]• omni.py deletesliver slicename


Experiment Workflow

• Part I: Design/Setup– Step 1: Establish Management Environment– Step 2: Obtain Resources

• Part II: Execute– Run Experiment



Scripting Omni

• You can write custom Python scripts – Call existing Omni functions– Parse the Output

• Example: readyToLogin.py– Calls getversion, sliverstatus, listresources– Parses output – Determines ssh command to log into node– Print ssh configuration blocks to place in your ssh

config file

• More examples distributed with Omni– remote-execute.py


Run Experimentgeni@NSDI13-Tutorials:~$> readyToLogin.py omni99 –a pg-utah

xterm -e ssh -i /home/geni/.ssh/ geni_key [email protected] &


Experiment Workflow

• Part I: Design/Setup– Step 1: Establish Management Environment– Step 2: Obtain Resources

• Part II: Execute– Execute Experiment



Part III: Finish Experiment

When your experiment is done, you should always release your resources.

– Normally this is when you would archive your data– Delete your slivers at each aggregate

slice

projectaggregate

RSpecuserresourcesliv

er

AM API

slivercredentials

certificate


Omni: Command Line Tool

Written in PythonEasy to automate experimentsHelps with simple orchestration

through omni scriptingTalks to all AMs that speak the

GENI AM API

✗Very manual✗Needs RSpecs






Part II: TMix Traffic Generator

Jay AikatUniversity of North Carolina,

Chapel Hill


Part II: Click Software Routers

Niky RigaGENI Project Office


Software Routers

• Not wire-speed• Easy to deploy• Provide ability to implement new routing

protocols over a L2 network fast–MobilityFirst, Packet Cache, XIA

• An easy way to bandwidth limit your slice


Access of multiple users in resources

• Omni allows multiple users to access the same resources–Modification of omni_config file

–Different ways for each compute aggregate

Alice Bob

Alice’s SliceProtoGENI

MyPLC MyPLC


Install scripts

• Specify in the rspec–Where to download software from

• Usually comes with an install-script.sh–What commands to run at boot time

• It runs in the context of a the “geni” user– It is not your user account– It does have sudo priviledges


Install scripts

• Automatic installation of software


Click Example Experiment

• You will run your own non-IP multipath router– Odd checksums left– Even checksums right

• 6 VMs in Utah– 4 routers– 2 end hosts


Part III: Hands-on Experiments


Part III: Click Example Experiment

Niky Riga, Vic ThomasGENI Project Office


Hands On Exercise

Run a non-IP router over L2

Reserve 6 hosts:4 will be click routers, 2 will be end

hosts.

Odd checksums left Even checksums right


• Part I: Design/Setup– Setup: Omni with Multiple Users– Obtain Resources: Create a slice and reserve

resources

• Part II: Execute– Configure and Initialize Services: Configure the

Click Routers– Execute Experiment: Use custom routing to

forward traffic over multi-path topology• Part III: Finish

– Teardown Experiment


Multiple Users

[omni]default_cf = portalusers = alice, bob# ---------- Users ----------[alice]urn = urn:publicid:IDN+pgeni.gpolab.bbn.com+user+alicekeys = ~/.ssh/alice_key[bob]urn = urn:publicid:IDN+pgeni.gpolab.bbn.com+user+bobkeys = ~/.ssh/bob_key

ProtoGENI AMs[omni]default_cf = portalusers = alice, bob# ---------- Users ----------[alice]urn = urn:publicid:IDN+pgeni.gpolab.bbn.com+user+alicekeys = ~/.ssh/alice_key, ~/.ssh/bob_key

SFA (PlanetLab) AMs

Multiple accounts, each one with each own key

One account accessible by many keys


• Part I: Design/Setup– Setup: Omni with Multiple Users– Obtain Resources: Create a slice and

reserve resources



forward traffic over multi-path topology



Experimenter

For the Tutorial: Portal Account with Omni

Slice : click<initials>

Sliver

AM

ProtoGENI Aggregate

You

GENI Portal





omni

http://www.gpolab.bbn.com/experiment-support/ ClickExampleExperiment/rspecs/click-XX.rspec

http://www.gpolab.bbn.com/experiment-support/ClickExampleExperiment/rspecs/click-01.rspec






… in the meantime … Install Scripts

• specified in the rspec– It needs to be available over http

• does not run as “you”,but has sudo• it runs at every boot • $PATH might not be set• OS-dependent

1. Write script in language of choice2. Upload script3. Test script4. Add to rspec


Install Scripts: Look inside# 1. Change to dir. with installed softwarecd /local

# 2. Check if first timeif [ ! -f "./installed.txt" ]then #### Create the file #### sudo touch "./installed.txt” #### Run one-time commands #### #2a. Install necessary packages #2b. Install custom software #2c. Reboot if neededfi#3. Run Boot-time commands#3a. Start common serviceshn=`echo $HOSTNAME | cut -d'.' -f 1`if [ $hn == "server" ]then #4. Host specific commandsfi

1. Download install script

2. Identify the 4 sections


Experiment Workflow


resources

• Part II: Execute– Configure and Initialize Services: Configure

the Click Routers– Execute Experiment: Use custom routing to




Scripting Omni: remote-execute.py

• Script that helps you execute commands in your nodes from your computer

• Makes it easy to orchestrate experiments– Write scripts to orchestrate experiments

• Uses readyToLogin.py to get login information

remote-execute.py –m <command> [-h host]


Remote configuration of Click

• Remotely invoke configuration script: extractClickConfig.py– parse manifest rspec– extract information about topology– create click.config

• Remotely turn off IP in the routers

IP


Experiment Workflow


resources



forward traffic over multi-path topology



Send Traffic

Start routersSend packet using netcat

HostA Top Bottom

Left

Right

HostBIP


Click Config Fileroute :: Classifier(27/01%01,-);modify :: Unstrip(2) -> StoreData(0, "AliceWasHere3546") -> route;FromDevice(eth3, PROMISC true) -> Classifier(12/0800) -> modify;route[0] -> left :: EtherEncap(0x7744, 00:04:23:b7:14:76, 00:04:23:b7:18:fa) -> SimpleQueue -> Print(outL) -> ToDevice(eth2);route[1] -> right :: EtherEncap(0x7744, 00:04:23:b7:1c:e0, 00:04:23:b7:19:2e) -> SimpleQueue -> Print(outR) -> ToDevice(eth4);

restore :: SimpleQueue -> Strip(30) -> EtherEncap(0x800, 00:04:23:b7:14:77, 00:04:23:b7:20:00) -> ToDevice(eth3);


Experiment Workflow


resources






Part III: Finish Experiment

When your experiment is done, you should always release your resources.

– Normally this is when you would archive your data– Delete your slivers at each aggregate

slice

projectaggregate

RSpecuserresourcesliv

er

AM API

slivercredentials

certificate

are you ready for the tutorial?

Documents

omni presentation

config file omni

createsliver aliceslice

config info

writing result of createsliver

createsliver results

control framework pgeni

slice urn