FIBRE-BR Camp, 28-29 April 2012

Ouro Preto (MG), Brazil

FIBRE-BR

Monitoring Activities

Joberto Martins

José Augusto Suruagy Monteiro

• Prof. Joberto Sérgio Barbosa Martins (UNIFACS)

• Prof. José Augusto Suruagy Monteiro (UFPE)

• Prof. Leobino Sampaio (UFBA)

• Adriano Spínola (UNIFACS)

• Herbert Monteiro (IFBA)

• Igor Leonardo (UNIFACS)

• Igor Luiz (UNIFACS)

• Marcelo Pinheiro (UNIFACS)

• Paulo Ricardo (UNIFACS)

• Raphael Dourado (UFPE)

• Thiago Hohlenweger (UNIFACS)

Monitoring Team

Where are we located?

Salvador

Recife

• Measurements:

– perfSONAR

– MonIPÊ Service

– RedCLARA’s Measurement WG

Experience for FIBRE

PERFormance Service

Oriented Network monitoring

ARchitecture

Analysis &

Visualization

Measurement

Infrastructure

Data

Collection Performance

Tools

Analysis &

Visualization

Measurement

Infrastructure

API

API

perfSONAR as a Middleware

Measurement

Points

Data Services

Measurement

Archives

Transformations

Service

Configuration

Auth(n/z)

Services

Infrastructure

Information Services

Topology

Service

Lookup

Analysis/Visualization

User GUIs

Web Pages

NOC

Alarms

perfSONAR Architecture Overview

• Base network measurement schema

– OGF Network Measurement Working Group (NM-WG)

• Topology Schema

– OGF Network Markup Language (NML-) WG

– Includes Topology Network ID

• perfSONAR Protocol Documents

– OGF Network Measurement and Control (NMC-) WG

perfSONAR

MonIPÊ Infrastructure

FIBRE-BR I&M

Initial Ideas

• Ideally we would monitor:

– Experiments (slices)

– Infrastructure (Testbed)

• User groups:

– Experimenters

– Central Operators

– Aggregate (Island) Managers

• Initial proposals:

– Leverage on existing monitoring

– Federation through perfSONAR

Initial Ideas @ FIBRE-KOM

FIBRE-BR Camp, 28-29 April 2012

Ouro Preto (MG), Brazil

FIBRE-BR Monitoring

Architecture

UNIFACS and UFPE teams

Marcelo Pinheiro (UNIFACS)

• FIBRE-BR will possibly use three different control and

monitoring frameworks in its nine islands

– OFELIA Control Framework

– cOntrol and Management Framework (OMF) and

– ProtoGENI

• Each one takes a different approach in addressing I&M

requirements and demands

• Each CMF has its monitoring capabilities

• How to put all these together?

Motivation

• An Instrumentation and Measurement Architecture

Supporting Multiple Control Monitoring Frameworks

• Our target is:

– to provide, possibly, with a maximum reuse of the

available CMFs I&M services over a new integrated

and federated network structure;

– To provide instrumentation and monitoring

considering different I&M Services through FIBRE-

BR (Monitoring Orchestration);

– Multiple CMFs I&M data integration.

Goal

FIBRE-BR I&M

ARCHITECTURE

FIBRE-BRI&M PersistentData Repository

OFELIAMonitoring

Data

MDIPMeasurement

Data Integration

Point

Aggr03

Aggr02

Aggr01

OFELIA CMF

OFELIA

Control

Commands

ProtoGeni Monitoring

Data

MDIPMeasurement

Data Integration

Point

Aggr03

Aggr02

Aggr01

ProtoGENI CMF

ProtoGeni Monitoring Facilities

ProtoGeni

Control

Commands

OMF Monitoring

Data

MDIPMeasurement

Data Integration

Point

Aggr03

Aggr02

Aggr01

OMF CMF

OMF Monitoring Facilities

OMF

Control

Commands

Experimenter

Researcher

NetworkManager

Visualization/ Portal Service

Security Services

Orchestration & Configuration Service

FIBRE-BRPolicy

Repository

OFELIA Monitoring Facilities

FIBRE-BRI&M

SERVICES

FIBRE-BR I&MSERVICES

FIBRE-BR I&MSERVICES

FIBRE-BR I&MSERVICES

1

2

3

5

4

Measurement Data Integration Point (MDIP) - conforms the collected data

from the available CMFs to FIBRE-BR I&M standard format (NM-WG), representation and distribution (including visualization).

- This service includes all measurement data processing related aspects such as, message format, message transport protocol and/or service, access privileges and common data storage or on-the-fly data distribution.

- Hold your questions for now! This will be further discussed

1

The security and police service will use the global definitions implemented and controlled by the ClearingHouse (CH definition - is both an entity and a

system consisting of software, operations, and policy to broker trust between federation partners.)

Implementation issues either necessary or considered by I&M solution: • Trust relationship (CA, SASL, etc) • Identity credentials • Integrated authentication/authorization (XMPP to help

on that???) • Federation level policies • Slice behavior • Data access policy • Policy enforcement • FIBRE-BR policy document

2

The Orchestration and Configuration Services act on behalf of the users allowing them to configure, to define measurement points, and to orchestrate these measurement data collection facilities according to each individual CMF.

Implementation initial ideas: - Use XML pub/sub messaging service, based on

XMPP server - Currently supported by OMF - IMF @ GENI - Does OCF support it?

3

The I&M Portal main functionality is to provide a user friendly interface to control and access the measured data, according to a defined policy.

Implementation issues: Data visualization: - From real-time experiments - From data stored Persistent Data

repository in each individual CMF (I&M perspective)

- Verify privileges access - Available only to authorized users

4

The architecture has a storage strategy that allows users to retrieve data from their own or from others previous experiments, according to their access privileges. The persistent storage option is an experimenter decision that must comply with FIBRE-BR retention policy.

Implementation steps: • MDIP will be in charge of saving it

persistently • Data retention policy • MySQL/RRD database (access from

I&M solution) • I&M Standard storage • Each CMF will keep its storage

mechanism. I&M will, eventually, store it centrally or access it based on users demand and/or privilege (being discussed)

• Logs storage is being discussed

5

FIBRE-BR Camp, 28-29 April 2012

Ouro Preto (MG), Brazil

ProtoGENI I&M

Architecture and Tools

Raphael Dourado (UFPE)

• There are several GENI I&M related projects compatible

with ProtoGENI. The most important ones are:

– INSTOOLS

– LAMP

– OneTimeMeasure

– S3Monitor

• Some of them are complementary and will later be

integrated to compose a GENI Integrated I&M

Framework

Introduction

Instrumentation Tools

(INSTOOLS)

• INSTOOLS’ high-level goal:

– Make it easy for users to see what is going on in their

experiment – i.e., make it trivial to monitor a slice

• What can INSTOOLS

measure?

– Note that INSTOOLS is

concerned only about

passive measurements

• INSTOOLS’ philosophy

– Don’t reinvent the wheel

Overview

Architecture (overview)

Experimenter

Measurement Plane connections

Data Plane connections

MC Portal: single interface for all MCs in a slice

Slice’s nodes (instrumentized to act also as MPs)

Measurement Controller (MC) = GENI MAP+MC Automatically deployed by INSTOOLS (at least one per aggregate)

INSTOOLS Toolset Architectural Components

INSTOOLS Web Portal

INSTOOLS Web Portal

LAMP

• LAMP stands for “Leveraging and Abstracting Measurements

with perfSONAR”

• The main goal is to “create an instrumentation and

measurement system, based on perfSONAR, for use by

experimenters on ProtoGENI”

• Which tools does LAMP support?

– OWAMP, BWCTL, Ganglia, PingER, NTP; ps-BUOY MA; etc

• So... what’s the difference between LAMP and pS-PSToolkit?

– LAMP adapted perfSONAR-PS software suite to recognize GENI’s

Authentication and Authorization model and infrastructure

– Added Ganglia as a host monitoring solution

– Added distributed configuration through annotations in the topology

stored in UNIS

– These annotations make it easy for users to save their slice

configuration and load it at a different slice

Overview

• Lamp Web Portal

– The “goto” resource for

experimenters to manage and

visualize their I&M services and data

• UNIS

– Unified Network Information Service

– It’s basically the combination of the

Lookup Service and the Topology

Service of the perfSONAR framework

– Provides information to the slice’s

nodes

• MPs

– Nodes with perfSONAR tools

installed (OWAMP, BWCT, etc...)

LAMP’s Basic Architecture

UNIS

Slice LAMP Web Portal

MP MP

1. On the Rspec, the user chooses which nodes will

be “instrumentized” with LAMP

– The user also chooses one (or more) node to host the

LAMP Web Portal

2. Using this modified Rspec, the slice is created as

usual by the CMF

3. The slice manifest (returned by the CMF) is

converted and sent to UNIS

4. Through the LAMP Web Portal, one can enable and

configure measurement services on all nodes that

comprise the slice

– The “Portal node” knows the slice’s topology by querying

UNIS…

5. All changes made on the Portal are sent to UNIS

6. All nodes pulls the configuration from UNIS (every

5 minutes) and applies the new configurations on

themselves.

How does it work?

Architecture (a broader view)

ProtoGENI UNIS

Slice 1 Slice 2 Slice n

Node with LAMP tools

Node with LAMP tools + LAMP Portal enabled UNIS keeps

information about ALL slices Note that it is possible to

have more than one node running the Web Portal

Not every node has to be

“instrumentized”

This node has only “Measurement Plane

interfaces”

LAMP Portal

Each node has as set of (possible

different) measurement tools enabled

OneTimeMeasure and

S3Monitor

• OnTimeMeasure

– Is “an on-demand measurement service used in forecasting,

anomaly detection, and fault-location diagnosis in GENI

experiments and GENI operations.”

– Can be integrated with INSTOOLS

• S3Monitor

– Has a flexible design that allows easy “plug in” of new network

measurement tools

– Based on 𝑆3 (Scalable Sensing Service for PlanetLab)

– Focused on measurement of large networked systems

– Already integrated with INSTOOLS

OneTimeMeasure and S3Monitor

Integrating with

FIBRE-BR I&M architecture

INSTOOLS MDIP

ProtoGENI Fibre I&M Architecture

Slice (INSTOOLS)

INSTOOLS Meas Controller

RRD

Pro

toG

ENI

MD

IP

RRD Collector

FIBRE I&M Data Repository

1 2

SQL DB

SQL Collector

1. MC collects measurement data from the MPs

2. MDIP (through his collectors) collects measurement data, makes any

necessary format adjustments and

3. Stores the data in the permanent repository (if demanded)

4. Measurement data can be accessed through the Visualization Portal

3

Visualization Portal

4

LAMP MDIP ProtoGENI

Fibre I&M Architecture

Slice ProtoGENI MDIP

FIBRE I&M Data Repository

NMWG Generic

Collector

UNIS

MP

MP

LAMP Web Portal

SNMP MA

ps-BUOY MA

Ganglia MA

PingER MA

MA-specific defs

1. Fetch experiment description from UNIS

2. Start copying measurement data from MAs

3. Stores the data in the permanent repository (if demanded)

1

2

3

41

• INSTOOLS-MDIP

• How to handle AA?

• How to discover active slices? [Clearing House?]

• LAMP-MDIP

• How to handle AA?

INSTOOLS-MDIP AND LAMP-MDIP OPEN ISSUES

FIBRE-BR Camp, 28-29 April 2012

Ouro Preto (MG), Brazil

OML Overview and OML-

MDIP Proposal

Igor Leonardo (UNIFACS)

43

• OML Resources (Overview)

• OML Introduction

• OML Measurement Points – MP

• OML’s Filtering Mechanisms

• OML Server

• OML MDIP Proposal

Agenda

44

• OML is the OMF measurement tool

• It was first developed as a component of it

• Today is a stand-alone project (independent)

• Shortly, it is a framework (set of libraries) to collect and store

measurements

OML Introduction

45

• “It is the input port for recording measurements”

• A MP is a tuple that indicates a measurement

• A measurement is not just a number

• Can be a group of itens

• In today’s version, data can be represented as a string |

integer | double.

• To define/register an OML MP:

• Create a OmlMPDef array. This array defines the data to be stored in

a relational database (sqlite3), like a table;

• Use the function omlc_add_mp, to register the “table” in the database

• To insert the data into the database, use the function oml_inject

OML Resources - MP

46

• It is the result of some pre-processing on a measurement

stream

• The big advantage of this is to reduce the amount of data

collected

• Definided by the OML Client Library

• It is possible to customize, but there are some default

filters (Average, First, Last, Sum, etc):

• http://mytestbed.net/projects/oml/repository/revisions/master/show/l

ib/client/filter

• How to create your own filter:

• http://omf.mytestbed.net/projects/oml/wiki/Developing_Filters

OML Filters

47

• Allows users to record their measurements inside a

remote database

• It works like a daemon program for OML architecture

• Receives the collected measurements from clients

• Creates a database named “oml-exp-id” (a parameter for

configuration, required by the omlc_init function)

• “The oml2-server proposes an abstraction for developers

to change the back-end database. Currently, only the

sqlite3 backend is fully supported”

• Export database to a txt file (on NICTA testbed)

• wget

"http://ServerAddress:5053/result/dumpDatabase?expID=Experi

ment_ID" –O DumpFile

OML Server

48

OML Use - Example

#include <stdio.h>

#include <stdlib.h>

#include <oml2/omlc.h>

int main (int argc, const char **argv)

{

int result = omlc_init ("Simple", &argc, argv,

NULL);

if (result == -1) {

fprintf (stderr, "Could not initialize OML\n");

exit (1);

}

OmlMPDef mp_def [] = {

{ "count", OML_UINT32_VALUE },

{ "count_str", OML_STRING_VALUE },

{ "count_real", OML_DOUBLE_VALUE },

{ NULL, (OmlValueT)0 }

};

OmlMP *mp = omlc_add_mp ("counter", mp_def);

if (mp == NULL) {

fprintf (stderr, "Error: could not register

Measurement Point 'counter'");

exit (1);

}

omlc_start();

int i = 0;

for (i = 0; i < 1000; i++) {

uint32_t count = (uint32_t)i;

char count_str[16];

double count_real = (double)i;

OmlValueU values[3];

snprintf(count_str, sizeof(count_str),"%d", i);

omlc_set_uint32 (values[0], count);

omlc_set_string (values[1], count_str);

omlc_set_double (values[2], count_real);

omlc_inject (mp, values);

}

omlc_close();

return 0;

}

49

• OML MDIP consists basically of one service: OML MA

(OML Measurement Archive)

− It’s responsible for receiving and sending the requests and

responses

− When it receives the requests, the MA queries OML Server DB

and then uses standard perfSONAR messages to communicate

with FIBRE-BR I&M Services

− The response is sent to the MDIP, in order to be stored

persistently in FIBRE-BR I&M repository (if demanded)

OML MDIP Proposal(1)

50

OML MDIP

Proposal(2)

51

- How to handle Authentication and Authorization?

- How to access the measured data in real-time?

OML MDIP OPEN ISSUES

FIBRE-BR Camp, 28-29 April 2012

Ouro Preto (MG), Brazil

OFELIA I&M Architecture

Igor Luiz (UNIFACS)

53

OFELIA I&M

Working with the OFELIA Control Framework (https://alpha.fp7-ofelia.eu/doc/index.php/Working_with_the_OFELIA_Control_Framework)

Introduction

OFELIA's Control Framework Web interface is called Expedient and is

one of the components of the OFELIA's Control Framework. It enables

experimenters to create and run experiments within the OFELIA autonomous

and federated facilities.

Through this user interface, one can instantiate and deploy

experiments, which may include virtual machines, switch configurations and

other resources. The control framework handles the separation of the

experiments and the monitoring. Hence the user can fully concentrate

on his/her research.

54

(FIBRE ↔ OFELIA) I&M

FIBRE-BRI&M PersistentData Repository

OFELIAMonitoring

Data

MDIPMeasurement

Data Integration

Point

Aggr03

Aggr02

Aggr01

OFELIA CMF

OFELIA

Control

Commands

ProtoGeni Monitoring

Data

MDIPMeasurement

Data Integration

Point

Aggr03

Aggr02

Aggr01

ProtoGENI CMF

ProtoGeni Monitoring Facilities

ProtoGeni

Control

Commands

OMF Monitoring

Data

MDIPMeasurement

Data Integration

Point

Aggr03

Aggr02

Aggr01

OMF CMF

OMF Monitoring Facilities

OMF

Control

Commands

Experimenter

Researcher

NetworkManager

Visualization/ Portal Service

Security/ Police Services

Orchestration & Configuration Service

FIBRE-BRPolicy

Repository

OFELIA Monitoring Facilities

FIBRE-BRI&M

SERVICES

FIBRE-BR I&MSERVICES

FIBRE-BR I&MSERVICES

FIBRE-BR I&MSERVICES

?

55

OFELIA I&M STATUS

NOWADAYS

Backend Infrastructure Monitoring (servers, switches, links)

- ZENOSS (not integrated to OCF)

OFELIA´s experiment monitoring:

- Not clearly defined or found (to our knowledge)

56

OFELIA I&M STATUS

Second OFELIA Open Call

(http://www.fp7-ofelia.eu/open-calls/2nd-open-call/)

Proposals for specific measurements and evaluations on

the OFELIA experimental facility that need extensions of

the already available infrastructure. Measurement

frameworks, measurement equipment, inclusion of this

equipment into OFELIA Control Framework, integration of

OMF (ORBIT Management Framework) for wireless

OpenFlow experiments

*The Second OFELIA Open Call is Open for Proposals Until April 18th, 2012, 17:00 Brussels time.

57

- FIBRE I&M possible alternatives for OFELIA CMF are (in

discussion):

- Focus on FlowVisor and OF-related basic

measurements parameters (pragmatic approach)

- Incorporate current OFELIA monitoring developments

(need to identify them)

- Align with OFELIA current developments

OFELIA & FIBRE I&M

58

OFELIA I&M PROPOSALS

OFELIA I&M technical possibilities/discussions:

- OML

- sFlow

- perfSONAR

- LAMP

- ... !?

Thank you / Obrigado