apache libcloud
DESCRIPTION
An introduction to Apache libcloud, a python module to abstract the differences in cloud providers API.TRANSCRIPT
Sebastien Goasguen,
@sebgoa
Apache libcloud
Drag picture to placeholder or click icon to add
Background
The Ten areas covered by the 60 core WS-* Specifications
WS-* Specification Area Typical Grid/Web Service Examples
1: Core Service Model XML, WSDL, SOAP
2: Service Internet WS-Addressing, WS-MessageDelivery; Reliable Messaging WSRM; Efficient Messaging MOTM
3: Notification WS-Notification, WS-Eventing (Publish-Subscribe)
4: Workflow and Transactions BPEL, WS-Choreography, WS-Coordination
5: Security WS-Security, WS-Trust, WS-Federation, SAML, WS-SecureConversation
6: Service Discovery UDDI, WS-Discovery
7: System Metadata and State WSRF, WS-MetadataExchange, WS-Context
8: Management WSDM, WS-Management, WS-Transfer
9: Policy and Agreements WS-Policy, WS-Agreement
10: Portals and User Interfaces WSRP (Remote Portlets)
Web 2.0 Grids and Parallel Computing, Geoffrey Fox
WS-* Areas and Web 2.0 WS-* Specification Area Web 2.0 Approach
1: Core Service Model XML becomes optional but still usefulSOAP becomes JSON RSS ATOM WSDL becomes REST with API as GET PUT etc.Axis becomes XmlHttpRequest
2: Service Internet No special QoS. Use JMS or equivalent?
3: Notification Hard with HTTP without polling– JMS perhaps?
4: Workflow and Transactions (no Transactions in Web 2.0)
Mashups, Google MapReduceScripting with PHP JavaScript ….
5: Security SSL, HTTP Authentication/Authorization, OpenID is Web 2.0 Single Sign on
6: Service Discovery http://www.programmableweb.com
7: System Metadata and State Processed by application – no system state – Microformats are a universal metadata approach
8: Management==Interaction WS-Transfer style Protocols GET PUT etc.
9: Policy and Agreements Service dependent. Processed by application
10: Portals and User Interfaces Start Pages, AJAX and Widgets(Netvibes) Gadgets
No more EC2 SOAP
GCE is RESTfull
Standards and de-facto standards
Adapters
Client side
Interfaces
e.g EC2 interface to CloudStack: ec2stack
e.g GCE interface to CloudStack: gstack
Server side
Drag picture to placeholder or click icon to add
Libcloud API
libcloud
Python module that provides a Cloud Provider API abstractionAbility to write apps using multiple providersNow with an improved CloudStack driver
FunctionalityManages:
• Compute nodes• Data volumes• DNS• Load balancers
DNS providers
6 providers:• Route53• Rackspace• Linode• Zerigo• Gandi• hostvirtual
LB providers
7 providers:• EC2 ELB• CloudStack• Rackspace• Ninefold• gce• Gogrid• brightbox
Compute Providers
40 providers including:• CloudStack• OpenStack• Opennebula• Rackspace• EC2• GCE
Installationpip install apache-libcloud
Or from source:
git clone https://git-wip-us.apache.org/repos/asf/libcloud.git python ./setup.py install
Test installation
$ pythonPython 2.7.5 (default, May 19 2013, 13:26:46) [GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))] on darwinType "help", "copyright", "credits" or "license" for more information.>>> import libcloud
Base APIBase Classes
• Node()• NodeImage()• NodeSize()• NodeLocation()• StorageVolume()• NodeDriver()
Base APIBase Methods
• list_nodes()• list_images()• list_sizes()• list_locations()• create_node()• destroy_node()• create_volume()• …..
New API
• list_key_pairs()• create_key_pair()• get_key_pair()• delete_key_pair()
• Possible to promote new API as soon as there is support in several providers.
Deploying nodes
Create a node and execute a script to configure it
deploy_node()
script=ScriptDeployment(wordpress)msd = MultiStepDeployment([script])deploy_node(name='apachecon',image=image,size=size,ex_keyname=mykey,pub_key_identity=</path/to/key>,deploy=msd)
Contributing to libcloud
http://docs.libcloud.apache.org
Github integration
http://github.com/apache/libcloud
Drag picture to placeholder or click icon to add
CloudStack Driver
Apache Libcloud
Ecosystem
Contributing to Apache CloudStack
API
Extension methods
For specific drivers, e.g:
def ex_list_networks()
def ex_create_security_group(self, name, description)
CloudStack + libcloud
from libcloud.compute.types import Providerfrom libcloud.compute.providers import get_driver
Driver = get_driver(Provider.CLOUDSTACK)
conn=Driver(key=apikey,secret=secretkey,secure=True,host=host,path=path)
CloudStack/libcloud basics
conn.list_locations()
images=conn.list_images()
offerings=conn.list_sizes()
conn.list_nodes()
conn.create_node(name='yoyo’, image=images[0], size=offerings[0], extra_args={'keypair':’testkey'})
CloudStack basic zones
Instances isolated at Layer 3 through security groups
Usual access to instances through ssh key pairs.
Similar to AWS EC2
CloudStack Basic Zone
conn.ex_list_security_groups()
conn.ex_create_security_group(name=’libcloud')
conn.ex_authorize_security_group_ingress(securitygroupname=’libcloud',protocol='TCP',startport=22,cidrlist='0.0.0.0/0')
conn.ex_delete_security_group(’libcloud’)
CloudStack Advanced Zone
Creates isolated guest networks (L2 isolation).
Need to manage IP and port forwarding rules, possibly NAT and firewalls.
Advanced zone
ex_list_networks()ex_list_public_ips()ex_create_port_forwarding_rule()ex_create_ip_forwarding_rule()ex_allocate_public_ip()ex_release_public_ip()
Drag picture to placeholder or click icon to add
CloudStack “inherited” Drivers
Apache Libcloud
KTUCloud
iKoula
http://www.ikoula.com
Exoscale
http://exoscale.ch
from libcloud.compute.types import Providerfrom libcloud.compute.providers import get_driver
Driver = get_driver(Provider.IKOULA)
conn = Driver(key=apikey, secret=secretkey)
from libcloud.compute.types import Providerfrom libcloud.compute.providers import get_driver
apikey = os.getenv('EXOSCALE_API_KEY')secretkey = os.getenv('EXOSCALE_SECRET_KEY')
Driver = get_driver(Provider.EXOSCALE)
conn = Driver(key=apikey, secret=secretkey)
Drag picture to placeholder or click icon to add
Time for Demos ?
Apache Libcloud
Conclusions
• Use libcloud to abstract API differences.• Useful if you are using multiple cloud
providers• Actively developed and easy to contribute
to• @libcloud• [email protected]