ieee intercloud interoperability and federation …...access public access intercloud (ccs)...

49
IEEE Intercloud Interoperability and Federation Framework: Architecture, Infrastructure Components, Interoperability David Bernstein Chair, IEEE P2302 Standard for Intercloud Interoperability and Federation Chief Architect, IEEE Global Intercloud Testbed Project [email protected] IEEE Cloud Course IEEE Intercloud Framework 1

Upload: others

Post on 11-Jul-2020

15 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

IEEE Intercloud Interoperability and Federation Framework:

Architecture, Infrastructure Components, Interoperability

David Bernstein

Chair, IEEE P2302 Standard for Intercloud Interoperability and Federation

Chief Architect, IEEE Global Intercloud Testbed Project

[email protected]

IEEE Cloud Course IEEE Intercloud Framework 1

Page 2: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Outline

• Motivation, Background

• IEEE P2302/D0.2 Draft Standard for Intercloud

Interoperability and Federation (SIIF)

• IEEE Global Intercloud Testbed

• Main Architecture components

IEEE Cloud Course IEEE Intercloud Framework 2

Page 3: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

IEEE P2302 Working Group

Standard for Intercloud Interoperability & Federation

IEEE Intercloud Testbed Project

IEEE Cloud Course IEEE Intercloud Framework 3

• An Open Source project

• Architecture Similar to

Internet Routing protocols,

with Autonomous System

and distributed capabilities

concepts

• Cloud Implementation

Independent – Openstack,

VMware, C12G, MS, ..

• Identity and Trust scheme

• Uses Semantic Resource

definitions to federate any

imaginable IaaS or PaaS

resource, and to allow for

dynamic SDN based

federation network

transport

Page 4: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Motivation – Evolve to Interoperability, like the

Internet Did

• Interoperability killed the silo’d BBS’s and caused the Internet to Explode

IEEE Cloud Course IEEE Intercloud Framework 4

1989

1980

1984

1997Earthlink

hits 1M users,

has IPO

Page 5: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

It Really is a Déjà Vu!

IEEE Cloud Course IEEE Intercloud Framework 5

Does it really take a visionary to see what will happen next?

"I'm seeing a possibility of inter-cloud problems mirroring

the Internet problems we had thirty or forty years ago,“, Vint

Cerf, Vice President and Chief Internet Evangelist for Google

Page 6: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Public Network Federation Trends

IEEE Cloud Course Future Cloudscape 6

Took 100 Years Took 15-20 Years Taking 5-10 Years

Formal Standard (ITU) for

Protocols

Informal Standard

(IETF) for Protocols

De Facto Standards

for User Protocols

(AWS, GCE)

No Open Source for any Protocols Open Source for User

Protocols (TCP/IP)

No Open Source for

Federation Protocols

(Routing)

Open Source for

Everything

Peer to Peer

Federation model

Peer to Peer

Federation model

Peer to Peer

Federation model

Internet FederationCloud

FederationTelephony Federation

Page 7: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Interoperability Requires Deeper Mechanism

IEEE Cloud Course Future Cloudscape 7

Source: GICTF

Page 8: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Architectural Classification of Interoperable

Clouds

IEEE Cloud Course Future Cloudscape 8

Cla

ssific

ation

Inter-Clouds

Volunteer

FederationIndependent/

Multi-Cloud

CentralizedPeer-to-Peer Service Libraries

IEEE P2302

IntercloudCloudSwitch

OGF OCCI or

Helix Nebula

Exam

ple

Pro

jects

U of Melbourne

Inter-Cloud

Inter-Cloud architectures and application brokering:

taxonomy and survey; Nikolay Grozev and Rajkumar Buyya

Page 9: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Topologies - different cloud interoperability

IEEE Cloud Course Future Cloudscape 9

Mu

lti-

Clo

ud

sF

ed

era

tio

ns Cloud BCloud A

Cloud C

Central

Entity

Centralized Inter-Cloud Federation:

Clouds use a central entity to facilitate

resource sharing

Multi-Cloud

ServiceMulti-Cloud

ServiceMulti-Cloud

Library

Cloud A

Cloud B

Cloud C

Cloud A

Cloud B

Cloud C

Multi-Cloud Service: Clients access

Multiple clouds through a serviceMulti-Cloud Library: Clients develop their own

Brokers by using a unified cloud API as a library

Cloud B

Cloud A

Cloud C

Peer-to-Peer Inter-Cloud Federation:

Clouds collaborate directly with each other

but may use distributed entities for

directories or brokering

Distributed

Entity

Distributed

Entity

Page 10: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

The “Multicloud” Approach

IEEE Cloud Course IEEE Intercloud Framework 10

Internet

Cloud

Provider A

Cloud

Provider B

Cloud

Provider C

Provider A API Provider B APIProv

C API

Cloud

User 1

Cloud

User 3

Cloud

Mediator

Z

Prov

A API

Prov

B API

Prov

A API

Prov

C API

Mediator Z

API

Internet

Cloud

User 2

Prov

B APIProv

C API

Prov

B API

Land of User to Network

Interfaces (UNI)

Page 11: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

The Intercloud Approach

IEEE Cloud Course IEEE Intercloud Framework 11

Internet

Cloud Provider A

Intercloud GW

Cloud Provider B

Intercloud GW

Cloud Provider C

Intercloud GW

Cloud

User 1

Intercloud GW

Cloud

User 2

Intercloud GW Intercloud GW Intercloud GW

Intercloud

Root

Intercloud

Exchange

Land of Network to

Network Interfaces (NNI)

Page 12: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

IEEE Intercloud Background

• 2007 – Kevin Kelly, founding executive editor of Wired magazine, and a former editor/publisher of the Whole Earth Catalog, blogs about Cloud Computing and theorizes “Eventually we'll have the intercloud, the cloud of clouds. This intercloud will have the dimensions of one machine comprised of all servers on the planet”

• 2009 – Cisco team writes paper “Blueprint for the Intercloud”

• 2009 – Industry group “Global Intercloud Technology Forum” (GICTF) forms in Japan

• 2010 – Intercloud Research explodes. First IEEE International Workshop on Cloud Computing Interoperability and Services (InterCloud2010) held in France

• 2011 – IEEE launches technical standards effort called P2302 - Standard for Intercloud Interoperability and Federation (SIIF)

• 2012 – “Intercloud” made the Wired Magazine Jargon Watch list

• 2013 – IEEE announces the IEEE Global Intercloud Testbed initiative. Two dozen cloud and network service providers, cloud-enabling companies, and academic and industry research institutions from the United States, the Asia-Pacific region, and Europe.

IEEE Cloud Course IEEE Intercloud Framework 12

Page 13: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Intercloud Use Case (Part 1)

• Wholesale Computing/Storage with MPLS

IEEE Cloud Course IEEE Intercloud Framework 13

US Carrier provides VPN to multi-

location Corporation via MPLS using

it’s own network infrastructure

US Carrier provides “US VPN” to multi-

location Corporation via MPLS via Wholesale

of partner network

Page 14: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Intercloud Use Case (Part 2)

• Cloud Services such as Compute and Storage can ALSO be Wholesaled

by US Carrier through the MPLS VPN in area where they don’t operate

infrastructure

IEEE Cloud Course IEEE Intercloud Framework 14

Page 15: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Multiple VPC Federation

• Two VPCs isolate resources within the cloud sites and securely link them

to enterprise networks

IEEE Cloud Course IEEE Intercloud Framework 15

VM

VM VM

VM

Clo

ud

Site

s

VPC 1

VPC 2

Internet

En

terp

rise

S

ites

Page 16: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Multiple VPC Federation Mechanism

IEEE Cloud Course IEEE Intercloud Framework 16

VM

VM

VPNs

En

terp

rise

S

ites

Logical Customer

Edge

Customer Edge

Customer Edge

Provider Edge

SDN Software

SDN Software

Page 17: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Multi Carrier MPLS/VPN Federation

IEEE Cloud Course IEEE Intercloud Framework 17

Intercloud, Federating Carrier A and C Clouds

through backend Intercloud-over- MPLS

MPLS

Cloud

Cloud

Cloud

MPLS

Cloud

Cloud

Clouds from

Carrier C

Clouds from

Carrier A

MPLS from

Carrier B

MPLS from

Carrier A

MPLS

Page 18: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

IEEE P2302 WG Cloud Interoperability Federation

• IEEE P2302/D0.2 Draft Standard for Intercloud

Interoperability and Federation (SIIF), IEEE Global

Intercloud Testbed

• Main component of the architecture

– Intercloud root

– Intercloud Exchange

– Intercloud gateway

IEEE Cloud Course IEEE Intercloud Framework 18

Page 19: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

IEEE Intercloud Elements

IEEE Cloud Course IEEE Intercloud Framework 19

protocols

formats

processes

practices

governance

Clouds which are

Intercloud Enabled

Gateways which are

Intercloud Enabled

Intercloud Root

Intercloud

Exchanges

Standards,

Industry

Associations

University

Funded work

and

Partnerships

Public

Testbed

Page 20: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Intercloud Protocols Taxonomy

IEEE Cloud Course IEEE Intercloud Framework 20

IP

TCP

XMPP

Web Sockets

HTTP

UDT

Presence and Conversational Protocols

Generic Services and TransportInfrastructure

Specialized Storage TransportInfrastructure

UDP

DNSBitTorrent

Directory Replication

Internet Routing and Transport

IP Routing

Mgt. API’s

Page 21: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Intercloud Gateway

IEEE Cloud Course IEEE Intercloud Framework 21

• Software or Appliance– Open Source and Adapted to Each Cloud Platform

• Supports “Common Channel Signaling” profile of Intercloud

protocols and standards– Naming

– Identity and Trust

– Conversation Substrate

– Services Transport

• Supports Cloud OS specific Federation API’s and Bearer

Network “Drivers”– Federation APIs

• Remote Compute – Simple Remote VM Lifecycle Protocol (SRVM)

• Storage Remoting – Simple Remote Object (SROB)

• Storage Replication – Simple Storage Replications Protocol (SSRP)

– Bearer Network Drivers• IPSEC VPN/VPC, MPLS VPN/VPC, 802.1q VLAN/VPC, RDMA/OFA

Infiniband

• TCP, UDT, and others (eg, MPI, SDP/rsockets)

Page 22: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Intercloud Exchanges

IEEE Cloud Course IEEE Intercloud Framework 22

• Software on a Cloud Platform– Open Source

• Supports CCS profile of Intercloud protocols and standards via

Gateway– Naming

– Identity and Trust

– Conversation Substrate

– Services Transport

• Transient Copy of Semantic Resources Directory

• Exchange Extents– Trust Levels for Remote Exchange Use via Exchange Extent Policies

Mechanisms

– Replication of Semantic Resources Directories via Exchange Extent

Policies Mechanisms

• Solver / Optimized Matching System – Supply / Demand Algorithms

• Audit Records

Page 23: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Intercloud Roots

IEEE Cloud Course IEEE Intercloud Framework 23

• Software on a Cloud Platform

– Open Source

• Supports CCS profile of Intercloud protocols and standards via

Gateway

– Naming

– Identity and Trust

– Conversation Substrate

– Services Transport

• Stable Copy of Semantic Resources Directory

• Root Extents

– Trust Levels for Peer Roots via Root Extent Policies Mechanisms

– Replication of Semantic Resources Directories via Root Extent

Policies Mechanisms

Page 24: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Namespace Details

IEEE Cloud Course IEEE Intercloud Framework 24

• Representation of CSP Numbers– Conceptually similar to ASN scheme as defined in RFC 5396

– Textual representation of “cspplain” (simple integer form)

• URN CSP Number Designation– <nnnn>.csp.intercloud.net

– Eg, 4.csp.intercloud.net is CSP 4

• Possible XMPP Mapping using DNS– XMPP naming is very flexible, depending on the services at the target end

to figure out what exactly is being asked for

– XMPP supports names in inbox form such as [email protected], but also <foo>@example.com, ::foo::@example.com,

[email protected]/service, or service@[email protected]

– Utilize Name Authority Pointer (“NAPTR”) DNS Resource Record 35 (RFC

3403) to map CSP URN to CSP XMPP Service Endpoint

– Using an example of Megacloud Inc. with CSP 4, a NAPTR query of 4.csP.intercloud.net might be mapped to

xmpp://intercloud.megacloud.net which resolves to the IP address

of the Conversational Service of Intercloud Gateway of Megacloud, which

can then be actually used

Page 25: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

PKI Certificates

IEEE Cloud Course IEEE Intercloud Framework 25

Intercloud Roots as

the Root CAs Issue PKI

Certificates to Affiliated

Intercloud Exchanges Intercloud Exchanges

as the Subordinate CAs Issue Temporary PKI

Certificates to Affiliated

Cloud Providers

Cloud Providers use

Temporary PKI Certificates as

part of the Delegation Process –

Acting on behalf of Originating

Cloud Provider

Intercloud PKI Certificates

Topology

Page 26: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Trust Management – Extended PKI

• From Intercloud topology perspectives, Intercloud Roots will provide static PKI CA

root like functionality and Intercloud exchanges will be responsible for the

dynamic “Trust Level” model layered on top of the PKI certificate based trust

model

• Exchanges are the custodians/brokers of “Domain based Trust” systems

environment for their affiliated cloud providers

• Cloud providers rely on the Intercloud exchanges to manage trust

IEEE Cloud Course IEEE Intercloud Framework 26

Intercloud Trust Management Model

Page 27: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Reference Intercloud Components

IEEE Cloud Course IEEE Intercloud Framework 27

CSP Namespace Federated Identity

Conversational Substrate

(XMPP)

Transport/Services (Web

Sockets)

Replication (BitTorrent)

Semantic Directory (OWL Ontology)

Certificate Authority

XMPP Servers

CSP/DNS NAPTR

IntercloudGateway

Intercloud RootIntercloudExchanges

CSP Namespace Federated Identity

Conversational Substrate

(XMPP)

Transport/Services (Web

Sockets)

Replication (BitTorrent)

Semantic Directory (OWL Ontology)

Solver (Hadoop/Sparql)

Auditing

SDN Controller

CSP Namespace Federated Identity

Conversational Substrate

(XMPP)

Transport/Services (Web

Sockets)

Federating API

Federating Transport via SDN VPC

Federating Implementation

Page 28: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Reference Intercloud Topology

IEEE Cloud Course IEEE Intercloud Framework 28

PublicCloud

PublicCloud

PublicCloud

Private Cloud

Internal User Access

Public Access

Intercloud Root

IntercloudExchanges

Public Access

Public Access

Intercloud (CCS) Protocols

Federation (Bearer) Network

Intercloud Gateway

Page 29: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Topology Initialization and Scale Out

IEEE Cloud Course IEEE Intercloud Framework 29

Intercloud Root “Zero”

operated by intercloud.net

xmpp://root.intercloud.net and

http://root.intercloud.net for

Gateway XMPP and Web Sockets

Carrier Neutral Internet Connected with Autonomous

System (AS) no.

Is a Cloud itself of course

CSP no. 0

Gateway address is actually 0.root.intercloud.net,

root0.intercloud.net, or root.intercloud.net

Root CSP Numbering Database

Root Certificate Authority

Root XMPP server (for Root to Root and Root to Exchange

conversations)

Root BitTorrent node

Root Semantic Resource Directory

Audit server

(optional) DNS server

Intercloud Exchange “Zero”

operated by intercloud.net

xmpp://exchange.intercloud.net and

http://exchange.intercloud.net for

Gateway XMPP and Web Sockets

Well connected on the Internet

Is a Cloud itself of course

CSP no. 0

Gateway address is xmpp://0.exchange.intercloud.net or

xmpp://exchange0.intercloud.net or just

xmpp://exchange.intercloud.net

XMPP server

BitTorrent “super node”

Semantic Resource Directory

Network Management server (SDN Controller)

Audit server

Page 30: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Roots and Exchanges Topology

IEEE Cloud Course IEEE Intercloud Framework 30

root.intercloud.net

intercloud.megacloud.net

Root of Trust

System of Record CSP Numbering feeds

NAPTR DNS function

XMPP directory and Root BitTorrent node

for Roots & Exchanges

Aged Semantic Resource Directory

exchange.intercloud.net

Trust Proxy

XMPP directory and BitTorrent node for

Exchanges & Clouds

Working Set Resource Directory

Resource Solver/Matching Function

Uses Gateway to Join Intercloud

Publishes Resource Availability

Makes Resource Requests

Allows network management by

Exchange to provision Federations

Page 31: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Exchange & Supplier Clouds Initialization

Scenario Flow

IEEE Cloud Course IEEE Intercloud Framework 31

intercloud.megacloud.net

exchange.intercloud.net

root.intercloud.net

1

Assume that Certificate Issuance is already done.

1. Exchange contacts hard coded Root address, Authenticates, and

registers presence

2. Root updates internal Exchange Databases. Switches to Services

Framework

3. Root updates Exchange with Exchange-Vendor subset, Aged

Resource Directory using BitTorrent

4. Supplier Cloud contacts hard coded Root address, Authenticates, files

a Request for Exchange Pointer based on functional Criteria

5. Root finds appropriate Exchange based on Supplier Cloud request,

switches to Services Framework

6. Root points Supplier Cloud to that Exchange

3

2

4

6

5

7

7. Supplier Cloud contacts Exchange using Pointer,

Authenticates, and registers presence

8. Exchange updates internal Cloud database

9. Exchange acknowledges Supplier Cloud, switches to

Services Framework

10. Supplier Cloud uploads Resources Inventory to

Exchange

11. Exchange updates Root with new Resources inventory

12. Root chooses what to update based on ageing or

Exchange-vendor specific algorithm

13. Exchange and Supplier Cloud wait for Request of

Resources

8

9

10

11

12

13

Page 32: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Additional roots and exchanges operated by

intercloud.net - Trust Hierarchy

IEEE Cloud Course IEEE Intercloud Framework 32

exchange.intercloud.net also

known as

exchange0.intercloud.net

root.intercloud.net

also known as

root0.intercloud.net

root1.intercloud.net

root2.intercloud.net

exchange1.intercloud.net

exchange4.intercloud.net

exchange2.intercloud.net

exchange3.intercloud.net

Page 33: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Additional roots and exchanges operated by

intercloud.net – BitTorrent Groupings

IEEE Cloud Course IEEE Intercloud Framework 33

exchange.intercloud.net also

known as

exchange0.intercloud.net

root1.intercloud.net

root2.intercloud.net

exchange1.intercloud.net

exchange4.intercloud.net

exchange2.intercloud.net

exchange3.intercloud.net

This will attach

to other vendor

worlds

Page 34: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Semantic Resource Directory

IEEE Cloud Course IEEE Intercloud Framework 34

Page 35: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Properties describing Concepts in the Ontology

IEEE Cloud Course IEEE Intercloud Framework 35

Name Domain Range Description

summary String Short Description of the Resource

time-to-live CloudResource floatTime expressed in mms before the resource becomes

unavailable

update-time CloudResource dateTime Last time the resource was updated

hostname Compute String Name of the hosting compute resource

state Compute Status of the compute resource

architecture Compute Architecture of the Compute resource

cores Compute int Number of CPUs owned by the Compute resource

memory Compute String Size of the memory owned by the Compute resource

speed Compute StringSpeed in MHz of each core owned by the Compute

Resource

gpu Compute StringDescription of the GPU owned by the Compute

Resource (if any)

price Compute String Price details

persistent Compute StringDetermine if the resource is persistent or it will be lost

after a certain time-to-live

label Network String general label attached to the Network

gateway Network String Address of the Network’s gateway

allocation Network String ???

address Network String Broadcast address of the network

vlan Network String ???

status Storage String status of the Storage resource

size Storage String Dimension of the Storage resource

hasURI Interface anyURI URI associated to an Interface

hasInterface Group Interface Interface associated to a Group

part of Group Participation of a Group of resources

Page 36: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Description/Match of A Compute Resource

IEEE Cloud Course IEEE Intercloud Framework 36

SELECT ?resources ?vendor ?CPU ?Memory

WHERE { ?resource rdf:type pf:Compute.

?resource pf:hasVendor ?vendor.

?resource pf:cores ?CPU.

?resource pf:memory ?Memory.

FILTER (?cores >=16)}

SPARQL query

SPARQL result

Resource Configuration Vendor vCPUs Memory

IBMPlatinum64Compute IBM 16 16

AmazonClusterQuadrupleXLCompute Amazon 33 23

AmazonHiCPUXLCompute Amazon 20 7

AmazonHiMemoryQuadrupleXLCompute Amazon 26 68.4

AmmazonClusterEightXLCompute Amazon 88 60.5

AmazonClusterGpuQuadrupleXLCompute Amazon 33 22

IBMReservationUnitCompute IBM 64 96

Semantic Description

Page 37: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Semantic Resource Definitions also have

Resource and Bearer Network Declarations

IEEE Cloud Course IEEE Intercloud Framework 37

• Semantic Resource Definition– Uses OWL Specification against a common “Ontology Schema” for Resources (Incl

SLA, Pricing)

– Uses OWL Specification against a common “Ontology Schema” for Bearer Network

• Suppliers of Resources declare the Offered Resources and the Bearer

Network within which they can be Supplied

• Requesters of Resources declare the Desired Resources and the

Bearer Network within which they can be Consumed

SUPPLIER

Resource {

----------

};

SLA {

----------

};

Pricing {

----------

};

Bearer Network {

----------

}

REQUESTER

Resource {

----------

};

SLA {

----------

};

Pricing {

----------

};

Bearer Network {

----------

}

Supplier meets / exceeds requester specification for Resource and SLA

Supplier meets / exceeds requester specification for Bearer Network

Page 38: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Examples of Bearer Network Declarations

IEEE Cloud Course IEEE Intercloud Framework 38

• Network Performance, specifications like – Average and Maximum end to end latency specification

– Average maximum sustained throughput, peak measured throughput

– Bandwidth Delay Product, etc

• Type of Network, specifications like– Public Internet

– NREN

– Private Interconnection

• Security of Network, specifications like– Encryption type

– Key Lengths

– Route Path Restrictions

• Special Network Connection– Private IP Peering (a la Direct Connect)

– Co-located shared switch fabric (a la Powered by Peak)

– Network Type (a la Infiniband using RDMA/OFA)

• Metric– Just like in an IP Router, a Bearer Network Ranking

Page 39: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Bearer Network Consideration

IEEE Cloud Course IEEE Intercloud Framework 39

• Clouds may have several

options for Bearer Network

• Bearer Network Drivers are

implemented in the Intercloud

Gateway

– IPSEC VPN/VPC

– MPLS VPN/VPC

– 802.1q VLAN/VPC

– RDMA/OFA Infiniband

– TCP

– UDT

– MPI

– SDP/rsockets

• Exchange finds best match of

not only the resource, but also

of the Bearer Network

Page 40: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Federation Steps

IEEE Cloud Course IEEE Intercloud Framework 40

User Cloud Request

Network Management

HomeCloud

Gateway(1)

(2) (4)

Supplier Cloud

Gateway

(6)

(7)

(5)

(3)

(4)

(7)

1(a). User Requests Cloud Resources by issuing existing cloud specific UNI

API call (EC2/S3, Nova/Swift, OCCI, MS SC 2012) to a Home Cloud.

1(b). If the Home Cloud is able/configured to fulfil the entire User Request

locally, it does so and returns the User Request API. The flow stops here as

there is no need for rest of the Intercloud federation steps.

Page 41: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Federation Steps

IEEE Cloud Course IEEE Intercloud Framework 41

2(a). Where the Home Cloud wishes to use Intercloud federation to fulfil certain of

the User Request, the Home Cloud uses a Cloud OS specific interface to the it’s

associated Gateway communicating the User Request. User Request API has not

returned yet.

2(b) The Gateway construct the canonical Resource Description Declaration

Including Resource, SLA, and Bearer Network sections.

2(c). The Gateway serializes these into the Federation API format including and

invokes them onto the implicit Gateway at the Exchange.

User Cloud Request

Network Management

HomeCloud

Gateway(1)

(2) (4)

Supplier Cloud

Gateway

(6)

(7)

(5)

(3)

(4)

(7)

Page 42: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Federation Steps

IEEE Cloud Course IEEE Intercloud Framework 42

3(a). The implicit Gateway at the Exchange and forms appropriate Exchange-Internal

API’s for the Solver/Matching process.

3(b). The Resource Description Declaration General Form is used to create a

SPARQL/SWRL query to the Solver/Matcher; the timestamp/deadline requirements

3(c). The Solver/Matcher seeks qualifying inventory (here this example from

Supplier Cloud) solving the constraints/quantifiers and if found, constructs a

canonical Resource Description Declaration Including the specific Resource, SLA,

and Bearer Network which the Supplier Cloud would deliver

User Cloud Request

Network Management

HomeCloud

Gateway(1)

(2) (4)

Supplier Cloud

Gateway

(6)

(7)

(5)

(3)

(4)

(7)

Page 43: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Federation Steps

IEEE Cloud Course IEEE Intercloud Framework 43

4(a). If the Exchange did not solve for inventory it returns an error to the Home

Cloud Gateway, which unwinds this to return the User Request API with an error.

4(b). If the Exchange found inventory it produces a Resource Description

Declaration Supplier Cloud Form onto the Gateway at the Home Cloud.

4(c). In this case the Exchange will also construct a Federation API for the Supplier

Cloud, The Supplier cloud may or may not choose to optimistically reserve or begin

to provision those resources. The User Request API has not returned yet.

User Cloud Request

Network Management

HomeCloud

Gateway(1)

(2) (4)

Supplier Cloud

Gateway

(6)

(7)

(5)

(3)

(4)

(7)

Page 44: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Federation Steps

IEEE Cloud Course IEEE Intercloud Framework 44

5(a). The Network Management (SDN Controllers) in the Exchange reach out to the

Network Management points in the Home Cloud and Supplier Cloud Gateways.

5(b). The Home Cloud and the Supplier Cloud Gateways may have self-provisioned

based on the bearer network information in the Federation API supplied in Step 4.

5(c). If not, the Network Management system (SDN Controller) provisions the

connectivity to the bearer network specified in the Resource Description

Declaration

5(d). If the bearer network cannot be provisioned, the Exchange returns an error to

the Home Cloud Gateway, which returns to the User Request API

User Cloud Request

Network Management

HomeCloud

Gateway(1)

(2) (4)

Supplier Cloud

Gateway

(6)

(7)

(5)

(3)

(4)

(7)

Page 45: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Federation Steps

IEEE Cloud Course IEEE Intercloud Framework 45

6(a). The Home Cloud Gateway serializes the Resource Description Declaration

Supplier Cloud Form onto the Supplier Cloud Gateway.

6(b). The Supplier Cloud provisions the resources, if not done optimistically earlier.

6(c). If the Supplier Cloud resources cannot be provisioned, it returns an error to the

Home Cloud Gateway, which unwinds this to return the User Request API

7(a). The Resources are provided over the Bearer network via VPN based VPC, to

the User as if they were supplied directly by the Home Cloud

User Cloud Request

Network Management

HomeCloud

Gateway(1)

(2) (4)

Supplier Cloud

Gateway

(6)

(7)

(5)

(3)

(4)

(7)

Page 46: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Federation of Workloads

• Looks like Dynamic/SDN VPC (Virtual Private Cloud)

IEEE Cloud Course IEEE Intercloud Framework 46

IPSEC VPN Tunnel, orMPLS VPN

Availability Zone 1

Availability Zone 2

Workloads in Network Space of CSP 2, AZ 1

Federated (Actual) Workloads in Network Space of CSP 1, AZ 2

CSP 2

Availability Zone 1

Availability Zone 2

Federated (Phantom) Workloads in Network Space of CSP 1, AZ 2

Workloads in Network Space of CSP 1, AZ 2

CSP 1

Requesting Cloud Supplying Cloud

Page 47: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Federation of Storage

• Storage System Replicate Extension

IEEE Cloud Course IEEE Intercloud Framework 47

Native Storage CSP 1 Replicates

Availability Zone 1

Availability Zone 2

Federated (Actual) Storage CSP 2 Replicates

CSP 2

Availability Zone 1

Availability Zone 2

Federated (Phantom requester) Storage CSP 2 Replicate

CSP 1

Requesting Cloud Fulfilling Cloud

UDT or other UDP/IP protocol

Federated (Phantom Supplier) Storage CSP 2 Replicate

Page 48: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

Wrap up and Take away

• Think about the great ICT systems of the World– Phone System

– Internet

– MPLS

• There are many architectural approaches for Interoperability– Portability is for applications

– Interoperability is for networks

– Federation is for whole systems

• For the Intercloud, key architectural concepts– Voluntary

• Cloud Decide to Join

• Obtain membership through a numbering authority

• Must Adapt their cloud operating system software

– Transparent• Clouds act as proxies to users

• Users do not know the machinery which is federating for them

– Generalized• Independent of any cloud operating system

• Semantic Resources based

– Signaling Network Based• Like SS7 for the phone network, routing protocols for the Internet

• Internet Protocols over XMPP/Web Sockets to find and set up matching resources

• SDN to organize Bearer Network

– Multi-Domain Capable• Technically and Governance wise

• This project is well along but has just begun– There is a place for you to make your mark on the world, please join us!

IEEE Cloud Course Tutorial 6. Cloud Standardisation 48

Page 49: IEEE Intercloud Interoperability and Federation …...Access Public Access Intercloud (CCS) Protocols Federation (Bearer) Network Intercloud Gateway Topology Initialization and Scale

References

• IEEE Global Testbed

http://intercloudtestbed.org

• IEEE ICWG/2302 WG - Intercloud Working Group

http://standards.ieee.org/develop/wg/ICWG-2302_WG.html

IEEE Cloud Course Tutorial 6. Cloud Standardisation 49