bonfire 2012 infocall technical overview of bonfires facility

8/3/2019 BonFIRE 2012 InfoCall Technical Overview of BonFIREs Facility

1/53

Building service testbeds on FIRE

Technical Overview of BonFIRE FacilityArchitecture entry points, Available infrastructure capacity,

Setting up experiments

Florian Schreiner, Fraunhofer Institute FOKUSKostas Kavvousanakis, EPCC

W: www.bonfire-project.euE: [email protected]

BonFIRE Open Calls Information Telcon, 7 February 2012


2/53

BonFIRE2

2 Fraunhofer Institute FOKUS, ATOS Origin and other members of the BonFIRE consortium 2010

BonFIREs Experimental Facility Architecture

BonFIREs Infrastructure, available Resources

BonFIREs Experiment Support

Examples of current BonFIRE Experiments

Demo of BonFIRE Experiment Setup and Monitoring Procedure

Agenda


3/53

BonFIRE3

3 Fraunhofer Institute FOKUS, ATOS Origin and other members of the BonFIRE consortium 2010

High Level View

Controlled deployment Monitoring

Support

Networks: Std. Int.

Emultd.

Contrld.

Permanent and on-request

infrastructure facilities


4/53

BonFIRE4 4 Fraunhofer Institute FOKUS, ATOS Origin and other members of the BonFIRE consortium 2010

2. Cloud scenario with emulated network (IBBTs Virtual Wall)

1. Extended multi-site clouds connected through standard internet

3. Extended Cloud scenario with controlled network (implies federation)

Three Scenarios Service Experiments on top ofthree different Network Infrastructures


5/53


Architecture

Message

QueueRead/

Write

Monitoring

VM

(Monitoring

Aggregator)

Monitoring

GUI

Monitoring

API

Monitoring

dashboard

SSH

Gateway

SSH

Gateway

SSH

IdentityServer

(Used by Portal,

Experiment

Manager, Broker

and Testbeds)

LDAP


6/53


BonFIRE Offering (1/2)

Support experiments over multiple heterogeneous cloudtestbeds using a single declarative experiment descriptor.

Support geographically distributed experiments.

Support experiment monitoring at both resource level (e.g.CPU usage, temperature, packet delay etc.) and applicationlevel.

Support the deployment of different software stacks over avariety of differently configured resources (compute, storage,network etc.) in multiple heterogeneous cloud testbeds.

Support elasticity within an experiment, i.e. dynamically create,update and destroy resources from a running node of theexperiment, including. cross-testbed elasticity.


7/53


BonFIRE Offering (2/2)

Support experiment management including experimentsharing, repeating and result collation and storage.

Support the definition of an entire infrastructure in a singleuniform experiment description.

Study the possible federation of the BonFIRE testbeds with avariety of external cloud facilities, such as those provided byFederica or OpenCirrus.

Support advanced network emulation via the Virtual Wall,including: Dynamic modifications of running experiments (at the moment

the network topology and node images have to be fully configuredat the start of the experiment.)

Additional generic network (e.g. overlay routing) and applicationlayer functionality


8/53


Architecture Principles

It must always be possible to include testbeds over whichBonFIRE has no control in a BonFIRE federation if theappropriate adapters are written and deployed into the

BonFIRE system. It should not be necessary to force atestbed to change its functionality in order to be included inBonFIRE.

Always provide APIs to BonFIRE functionality, in addition toany BonFIRE graphical user interfaces (GUIs).

Allow experimenters full access to the specific functionalityof particular testbeds; i.e. functionality that belongs only toone of the clouds and is not available through the BonFIREAPI.

Allow federated functionality to exclude specificfunctionality of a testbed if this makes common tasks easier toachieve. Support incremental adoption of the BonFIREsystem by experimenters.

Support declarative specification of experiments as far as

possible.


9/53







Agenda


10/53


BONFIRESINFRASTRUCTURE & RESOURCES


11/53


BonFIREs Operations Support

To ensure the operation of the experimental facility

Appropriate planning and control of the evolution of thepermanently provided infrastructure

Provision of on-request resources for large scaleexperiments in a well defined environment

Support monitoring of experiments under well definedconditions

Allowing repetition of experiments under similar conditions


12/53


The BonFIRE Service OrientedInfrastructure

Component 1 The Core Infrastructure

Provided permanently by the BonFIRE consortiummembers

Used as general purpose platform for medium scaledexperiments

Architecture and software provided by BonFIRE


13/53



Component 2 Infrastructure/Resources on request

On-request provisioning

Not exclusively for BonFIRE

Availability for limited times for large scale testing


14/53



Component 3 Experiment-owned infrastructure

Experiments may have own small scale testbeds

BonFIRE as extension to these testbeds or vice versa

Effort needed to allow the connection of experimentsinfrastructure and the BonFIRE infrastructure


15/53


BonFIREs Infrastructures andResources

EPCC: 1 frontend + 2 worker nodes + 1 storage node, up-to-date OpenNebula v3.0 w/ OCCI server

HLRS: 1 frontend + 10 worker nodes, up-to-dateOpenNebula v2.0.1 w/ OCCI server

HP: Using the Cells infrastructure

IBBT: Using the VirtualWall infrastructure

INRIA: OpenNebula, using parts of Grid5k infrastructure

PSNC: OpenNebula, exclusively for controlled networkexperimentation


16/53







Agenda


17/53


BONFIRESEXPERIMENT SUPPORT


18/53



To Support the experiments carried out by BonFIREcustomers

Support Tools

Tutorials, Guidelines

Individual support via email

Help experimenters to install software/applications andtheir own tools into the BonFIRE facility

Support migration of an experiment from theexperimenter's in-house testbed into BonFIREs facility

Help experimenters to find appropriate third party toolsand install these tools into the BonFIREs facility

Clearly define the services a customer may ask for

Enable customers to understand the capabilities of BonFIRE


19/53







Agenda


20/53


Examples for current embedded andOpen Call Experiments

BonFIRE is currently running 3 embedded experiments and4experiments from the 1st Open Call

Virtual Clusters on federated sites(1st

Open Call Experiment)

QoS-Oriented Service Engineering forFederated Clouds(EmbeddedExperiment)

Elasticity in Cloud-based webapplications(Embedded Experiment)


21/53


Example use case: VCOCVirtual Clusters on federated sites

To get a betterunderstanding aboutthe deployment and

usage of a multi-site,Cloud virtual cluster

Three main topics:

I. The time to deploy a virtual cluster on

a federated infrastructure;

II. The performance penalties associatedto a distributed cluster;

III. And the benefit from the distributednature of the federation to tackle sitefailures

Centre of Supercomputing of Galicia (CESGA)


22/53


Example use case: VCOCVirtual Clusters on federated sites

Experiment Hypothesis:

H1. For each application and dataset it is

possible to deploy a distributed virtual clusterwhere the total time-to-solution (includingtime to deploy, execution of the applicationand to retire of the virtual cluster) is morefavourable in a federated Cloud than in asingle provider.

H2. On a heterogeneous Cloudinfrastructure, it is possible to adaptdynamically a virtual distributed cluster toguarantee the Quality of Service (i.e., thetime-to-solution) using the performance of

the application as information to trigger thechange.

H3. A federated Cloud infrastructure can beused to reduce the lost time due to failuresfor a virtual cluster without a significantovercost.



23/53


VCOC requirements on the testbed

Controlled deployment

Controlled network

Very fine-grained monitoring

Elasticity and cross-site elasticity

Steering triggered by events

A Cloud facility for experimentation



24/53


Examples for current embedded andOpen Call Experiments

BonFIRE is currently running 3 embedded experiments and4experiments from the 1st Open Call

Virtual Clusters on federated sites(1st

Open Call Experiment)

QoS-Oriented Service Engineering forFederated Clouds(EmbeddedExperiment)

Elasticity in Cloud-based webapplications(Embedded Experiment)


25/53


Problem Statement

How to predict performance of applications

deployed on different IaaS clouds?

Models

Resource

description

Why? Portability and federation between providers

Make better QoS provisioning decisions

Hardware or benchmark scores


26/53


Experiment Hypothesis

Hypothesis:

If IaaS resources are described in

terms of application benchmarkscores then PaaS performancemodels will be faster, cheaper andmore accurate

Approach:

replace low-level resource metrics(e.g. CPU architecture, clock-speed, RAM, EC2 compute

units) with benchmark scores based on defined patterns of

communication and computation(e.g. dwarfs)

?

IaaS Virtualisation

IaaS SLA

IaaSProvisioning


27/53


Role in BonFIRE

BonFIRE Capability Why the experiment needs it

Heterogeneous hardware The models must be predictive on a widerange or target platforms

Detailed monitoring information Monitoring info on several levels is required to

create the modelsUniform request and access methods To make access to all the resources at different

sites easy

Repeatability of hardware allocation So that benchmark scores may be comparedwith test application data

Cross-site federation For complex test applicationsControl of the network To simulate different network conditions

Access to long-term storage and experimentaldata

Essential for off-line analysis of experiments

Authentication and authorisation Ensuring that experiments are not interfered

with

The vast majority of these capabilities are of use to all experiments. Theexperiment will help BonFIRE by providing concrete requirements for thesefeatures and by testing and validating the infrastructure, the documentation

and the support systems.


28/53


29/53


Technical overview

Develop a web-app for test purposes, and generate an image inside a VM to be instantiated at anytime.

Create a dummy load generator. Define load scenarios.

An application must be developed in order to:

Collect /store/process KPI (on demand or periodically).

Check, in all scenarios, the SLAs accomplishment.

Manage rules of SLAs.

Generate changes over the testbed scheme.

Define a testbed in the cloud with some minimum initial requirements eg:

Number of VM

Geographical location of resources. Network topology

Identify the KPIs for each load scenario

Set the upper and lower thresholds of the initial testbed scheme by load tests.

Define the set of KPIs values that trigger the release or the enlargement of the resourcesdeployed.

Using a time diagram of a load model, resize the testbed resources in advance.

With the information collected about KPI in a defined period of time generate a load model inferringthe ideal deployment.


30/53


Example of Embedded Experimentsto prepare the Infrastructure

Practical example Elasticity in Cloud-based web applications:The experiment tests in a comprehensive manner the wholefunctionality offered by BonFIRE:

CRUD of VMs

Complex network topologies

Geographical dispersed allocation ofresources, DNS, storage, etc.

Measurement of the latency of BonFIRE asinfrastructure provider in a wide variety ofload scenarios.

Resize or modify resources dynamically.

Dynamic configuration of load balancers.

The use of resources through federation withother cloud providers in some scenarios.

Monitor API

Improvements to


31/53


Experiment Objectives

To optimize the distribution of web-apps containers.

To investigate new scalability policies.

To be able to predict the ideal deployment under theupcoming load.

Learn how to resize dynamically the number of resourcesdeployed, from a load model of a web application, inside a

cloud federation infrastructure.

Develop algorithms and procedures to avoid over-provisioning resources and to satisfy SLAs with web-app

users. To study the concept of infinite elasticity in the cloud.


32/53


CLOUD-MANAGER

LB

WS

AS

DB

WS

AS

WS

AS

KPI-collectorChanges-generator

DLG

WEB-APP

KPI/ SLAsPOLICIES/

LOAD TESTS

VM-REPOSITORY

KPI

TESTBED

MANAGEMENT

OPERATIONS

MONITORING

EXPERIMENT -MANAGER

http/https

http://www.web-app.com

Experiment: in more detail


33/53


34/53


Benefits for BonFIRE

Comprehensive test of BonFIRE.

Better understanding of BonFIRE elasticity capabilities. Monitoring process.

Users interface (OVF, portal)

Dynamic testbed configuration. Creation of scalability policies.

Simulation of realistic scenarios with geogr. sep. sites.

Improvement of some features of the BonFIRE facility: Load balancing. Evolution of web traffic generators.


35/53






Demo of BonFIRE Experiment Setup and Monitoring

Procedure

Agenda


36/53


Demo


37/53


This Demo

Client-server application

Server VM on EPCC BonFIRE site

Monitoring

VM on EPCC BonFIRE site

Display CPU load

Configure and monitor application metric

Interactions with the BonFIRE Portal

Cross-site elasticity

Controlled network


38/53


The user application

The experiment will use the iperfclient-server application

Scenario: assess the impact ofmultiple iperf clients accessing thesame iperf server

Impact on iperf server VM

performance

Here is how BonFIRE supportsthis application

Iperf Server

2

43

5

1

Monitoring


39/53


Set up experiment


40/53


What is an experiment?

Collection of Compute, Storage and Network resources,each associated with a BonFIRE site

Software implementing the experiment logic


41/53


Create VMs


42/53


43/53


Monitoring

S t M it i


44/53


Set up Monitoring

BonFIRE base images include Zabbix monitoring client

Zabbix monitoring server deployed as a separate VM


45/53


Cross-site Elasticity

Monitoring for (Cross Site) Elasticit


46/53


Monitoring for (Cross-Site) Elasticity

162 # Check metric values:163 values = experiment.zabbix.metric('system.cpu.util[,system,avg1]', :type

=> :numeric, :hosts => server).values164 avg3, avg5 = [values[0..3].avg, values[0..5].avg]169 # Here if the CPU usage of the iperf server is too low, we'll spawn a

170 # new client. If it's too high, we'll shut a client down.171 if clients_count = 3 && avg3 aggregator_ip,

175 'iperf_server' => server_ip176 })177 sleep(10) until vm.ssh.accessible?178 elsif clients_count > 1 && values.length >= 5 && avg5 >= 22179 session.logger.warn "Scaling DOWN (avg=#{avg5})!"

180 # Delete the first client of the location which has the most clients:181 locations.sort{|loc1,loc2|182 loc2[1].size loc1[1].size183 }.first[1].shift.delete186 end


47/53


Controlled network

Emulated Control


48/53


Emulated Control

Virtual Wall: Large-scale testbed in IBBT

Based on Emulab software

Add a private network with configured parameters:

> private_network = experiment.networks.submit(

> :location => ibbt,

> :name => "network-experiment#{experiment['id']}",

> :bandwidth => 1000,

> :latency => 0,

> :size => 24,

> :lossrate => 0,

> :address => "192.168.0.0"

> )

Controlled network


49/53


Controlled network

To interconnect with FEDERICA duringthe summer

Advanced network experimentation,supported by GEANT3

Ability to control routing on FEDERICAslices

Possibility to route EPCC-PSNC trafficthrough FEDERICA

To provide access to AutoBAHN

GEANT3 experimental Bandwidth on Demand service

Involving traffic between EPCC and PSNC

Option to route traffic through Virtual Wall

Emulated, highly configurable.


50/53


Summary

Facility for services experimentation


51/53


Facility for services experimentation

6 sites; 350 virtualised cores; 700GB of RAM; 30TBstorage On-request access to 2,000 additional, multi-core nodes

Real and emulated networks Emulab-based Virtual Wall

Controlled networks

Experiment Descriptors Portal

Restfully

JSON DSL (OVF on the way)

Advanced monitoring Zabbix on all VMs

Infrastructure monitoring

What will you add to it?


52/53


Thank you for your attention


53/53


+ +

=

W: www.bonfire-project.euE: [email protected]

bonfire 2012 infocall technical overview of bonfires facility

Documents