module 1: perfsonar technical overview

Connect. Communicate. Collaborate

Click to edit Master title styleMODULE 1: perfSONAR TECHNICAL

OVERVIEW

2

WHAT IS PERFSONAR?

perfSONAR is:

• A project consisting of a variety of organisations and individuals

• A set of protocols that:

• Assume a set of services based on defined roles.

• Define their communication syntax and semantics.

• Allow anyone to develop an implementation of a service.

• A set of code

• Service implementations

3

WHAT ARE THE PERFSONAR SERVICES?

The perfSONAR services form an interoperable, distributed performance measurement middleware framework.

perfSONAR stands for PERformance Service Oriented Network monitoring Architecture.

4

DESIGN OBJECTIVES

perfSONAR is designed to be:

• Decentralised and Scaleable

• Large number of networks and services, large volume of data

• Each domain can set its own security policy

• Dynamic and ‘Self-Discovering’

• Add and remove components during operation

• Components ‘automatically’ become aware of one another

• Secure

• Will not put participating networks at risk of attack or congest them

• Modular

• Allows discrete module development

5

THREE-TIER ARCHITECTURE

The perfSONAR framework:

• Is middleware.

• Is distributed between domains.

• Facilitates inter-domain performance information sharing.

perfSONAR services ‘wrap’ existing measurement tools.

perfSONAR Visualization Tools

Domain B

MeasurementTool

perfSONARservice

MeasurementTool

perfSONARservice

MeasurementStore

perfSONARservice

Domain CDomain D

Domain A

MeasurementTool

perfSONARservice

MeasurementTool

perfSONARservice

MeasurementStore

perfSONARservice

Key

= perfSONAR protocols

6

WHAT IS A SERVICE?

A Service is a tightly defined, independent entity that has a well defined interface and can be accessed directly.

7

ARCHITECTURAL PRINCIPLES

Postulate: all measurement systems contain a combination of:

• Measurement tools• Data Storage• Security and Policy implementation • Topology information• Visualization

Services have been identified that are:• Based on these ‘roles’• Based on requirement for other

functionality such as• Service discovery, resource protection,

data formatting

MeasurementDatabase

Measurementsin file system

Router

RouterCommand linetools

BWCTL

8

THE PERFSONAR SERVICES FRAMEWORK (1)

perfSONAR divides measurement system tools into generic ‘families’ or ‘services’. Each service has a protocol.

= Registration

Key

Performance Data Services

MeasurementPoints

MeasurementTools

MeasurementArchives

MeasurementStores

TransformationServices

Enabling Services

LookupServices

AuthenticationServices

Resourceprotectors

Domain

= perfSONARservice

= existingmeasurement

tool ormeasurement

data store

9


AuthenticationService


LookupService

LookupService

Resource Protector Service


Measurement Archive Service


MeasurementPoint Service


Transformation Service


ClientClient

10

THE CLIENT



LookupService

LookupService









ClientClient

For example the perfSONAR Visualisation User Interface

11

THE LOOKUP SERVICE



LookupService

LookupService









ClientClient

Registers services including their capabilities.

Facilitates complex searches.

Participates in network of Lookup Services

12

THE LOOKUP SERVICE (1)

Purpose: all other services must register with the lookup service in order to participate in the framework.

13

THE LOOKUP SERVICE (2)

Other services (including measurement points) register their existence with a lookup service, by delivering ‘lookup information’:

• Location

• Type of Service

Each domain has an instance of the lookup service

• These instances (will) communicate with one another

Clients find other services by querying the lookup service.

• All the client needs to know is the URL of a Lookup Service

14

THE AUTHENTICATION SERVICE



LookupService

LookupService









ClientClient

Provides authentication for clients and protects privacy.

Can be federated.

15

THE AUTHENTICATION SERVICE

Purpose: ensures client-privacy and domain security by using role-based authentication and authorisation.

16

THE MEASUREMENT POINT SERVICE



LookupService

LookupService









ClientClient

Exposes measurement tools and publishes their data.

17

MEASUREMENT POINTS

Measurement Points:

• Belong to domains (domain = a network)

• Each measurement point implementation maps to a tool that provides one or several metrics

• Examples:

– One-way-loss

– Jitter

– TCP throughput

– Show commands on routers

18

THE MEASUREMENT ARCHIVE SERVICE



LookupService

LookupService









ClientClient

Exposes measurement databases and file stores.

Publishes measurement data

Avoids queries to multiple Measurement Point Services

19

MEASUREMENT ARCHIVES

Purpose: expose measurement data held in databases or file systems.

• Wrapper for any type of storage mechanism (SQL Databases, RRD files, etc)

• Provides access to recent and stored data

• Collects information from Measurement Points, Transformation Services or other Measurement Archives.

– i.e. it ‘subscribes’ to these other services

• Can also write to databases and file systems

20



LookupService

LookupService









ClientClientTransforms data in a variety of ways (e.g., aggregation, filtering, correlation).

For future development. Precise role needs to be defined.

THE TRANSFORMATION SERVICE

21

THE RESOURCE PROTECTOR SERVICE



LookupService

LookupService









ClientClientControls the comsumption of limited resources (e.g. network bandwidth).

22


Each service has a specific function.

Each instance of a service belongs to an administrative domain.

23

PROTOCOLS

perfSONAR has developed a set of protocols for sharing performance data. These:

• Assume the services set out in the framework.

• Define their communication syntax (schema) and semantics (business logic).

• Allow anyone to develop an implementation of a defined service.

• Are compliant with the Global Grid Forum’s Network Measurement Working Group (NM-WG) schema specification.

• Are based on XML over SOAP.

You can see the protocols as ‘rules and tools’ for participating in the perfSONAR framework.

24

GENERIC SERVICES AND SERVICE IMPLEMENTATIONS

Implementation: tool-specific functionality

added

RRDMeasurement

Archive serviceimplementation

Round RobinDatabase files

SQLMeasurement


SQL database

Implementation: tool-specific functionalityadded

MeasurementArchive Service

Common functionality,common protocol

Implementation: tool-specific functionality

added

BWCTLMeasurementPoint service

implementationBWCTL

measurementtool

SSH / TelnetMeasurementPoint service

implementationSSH / Telnetmeasurement

tool

Implementation: tool-specific functionalityadded

MeasurementPoint ServiceCommon functionality,

common protocol

25

SERVICE IMPLENTATIONS

The perfSONAR project has also developed a set of service implementations that use the defined protocols.

• Some of these have been developed by JRA1, within the GEANT2 project for the perfSONAR pilot:

• The Lookup Service

• The BWCTL Measurement Point

• The SSH / Telnet Measurement Point

• The RRD Measurement Archive

• The SQL Measurement Archive

26

DATA COLLECTION, NORMALISATION AND SHARING VIA THE FRAMEWORK

OtherperfSONAR

Services (e.g.measurement

archive)

ExistingMeasurement

Tools

Standardised Syntax

perfSONARMeasurement

Framework

StandardisedSchema

perfSONARSSH / TelnetMeasurementPoint (Web)

Service

SSH / Telnet-specificCode

Dat

a no

rmal

izat

ion

/co

nver

sion

to X

ML

Dat

a no

rmal

izat

ion

/co

nver

sion

to X

ML

SSH / TELNET

Too

l-spe

cific

com

man

ds

Mea

sure

men

t Dat

a

Stand

ardis

ed S

ynta

x

StandardisedSchema

perfSONARBWCTL

MeasurementPoint (Web)

Service

BWCTL Tool-specificCode

Dat

a no

rmal

izat

ion

/co

nver

sion

to X

ML

Dat

a no

rmal

izat

ion

/co

nver

sion

to X

ML

BWCTL Tool

Too

l-spe

cific

com

man

ds

Mea

sure

men

t Dat

a

27

FRAMEWORK FACILITATES NORMALISED END-TO-END PERFORMANCE DATA (SIMPLIFIED DEPICTION)

Client (e.g. Visualisations Tool)

Network 1


LookupService

RRDMeasurement

ArchiveSSH / TelnetMeasurement

Point

RouterNetwork 2

Data

SQLMeasurement

Archive

LookupService


SSH / TelnetMeasurement

Point

RouterData

Network 3

Network 4

28

THE PERFSONAR PILOT: MEASUREMENT AND ARCHIVING SERVICES

Network 5

Network 4

Network 1

Network 2

RRDMeasurement


Round RobinDatabase files -link utilisation

data

SQLMeasurement


SQL database -utilisation dataand path status

BWCTLMeasurementPoint service

implementation

BWCTLmeasurementtool - available

bandwidth

SSH / TelnetMeasurement


SSH / Telnetmeasurementtool - routercommands

Network 3

29

THE PERFSONAR PILOT

Key

Performance Data Services

BWCTL & SSH/ Telnet

MeasurementPoints

MeasurementTools

RRD & SQLMeasurement

Archives

MeasurementStores

TransformationServices

Enabling Services

LookupServices

AuthenticationServices

Resourceprotectors

Domain

Serviceimplementation not

included in pilot

Partial serviceimplementationincluded in pilot

Serviceimplementationsincluded in pilot

Existingmeasurement tools

/ data stores

30

THE ROADMAP

Enhanced Lookup Service:

• Lookup Services (ideally one per domain) peer with one another for increased ease of data discovery.

Authentication Service:

• Collaborating with JRA5 and Edugain to produce an implementation of the service.

• A data subscriber (client) and a data producer (server) can communicate directly across domains provided that they are authorised to do so.

Further service implementations using Netflow information or packet capture features

31

TECHNICAL OVERVIEW: SUMMARY

perfSONAR will provide an infrastructure to:

• Locate data sources

• Authenticate and authorise clients

• Protect resources and ration their usage

• Retrieve, normalise, transform and share data

• Only a partial infrastructure is implemented in the pilot

perfSONAR is a flexible and open framework:

• perfSONAR services can ‘wrap around’ existing data collection tools

32

THE PERFSONAR SERVICES FRAMEWORK - RECAP

How do we expose measurement tools?

• Measurement point services

How do we expose measurement databases and file stores?

• Measurement archive services

How do we transform data (aggregate, correlate, filter etc.)?

• Transformation services

How do we locate all these services and their capabilities?

• Lookup Services

How do we protect resources?

• Resource Protection Services

How do we ensure a client is allowed to access a service?

• Authorisation and Authentication Services

module 1: perfsonar technical overview

Documents