cloudstack for java user group
Post on 08-May-2015
2.232 Views
Preview:
DESCRIPTION
TRANSCRIPT
Apache CloudStack
Sebastien GoasguenGenevaJUG, Oct 30th
Geneve, Suisse
Info• Apache incubator project• http://www.cloudstack.org• http://incubator.apache.org/cloudstack/• #cloudstack on irc.freenode.net• @cloudstack on Twitter• http://www.slideshare.net/cloudstack• http://cloudstack.org/discuss/mailing-lists.html
Welcoming contributions and feedback, Join the fun !
A bit of History• Original company VMOPs (2008)– Founded by Sheng Liang former lead dev on JVM
• Open source (GPLv3) as CloudStack• Acquired by Citrix (July 2011)• Relicensed under ASL v2 April 3, 2012• Accepted as Apache Incubating Project April 16,
2012• First Apache (ACS 4.0) release expected Sept 26th
Apache Software Foundation
Apache Process
• 100% community driven• New ideas, decisions only taken on mailing lists.
Votes taken by community• Project led by Project Management Committee
(PMC):– http://www.apache.org/dev/pmc.html– http://incubator.apache.org/guides/committer.html
• Non committers get invited as committers:– http://community.apache.org/newcommitter.html
Apache Processes
First Release almost out
• Apache CloudStack 4.0• Check out the testing procedure:– https://cwiki.apache.org/CLOUDSTACK/
cloudstack-40-test-procedure.html
• Or watch my amazing screencast:– http://vimeo.com/52150218
Contributions from outside CitrixExamples:
Sungard: Announced that 6 developers were joining the Apache projectSchuberg Philis: Big contribution in building/packaging and Nicira supportGo Daddy: Early proto of Maven buildingCaringo: Support for own object store
CloudStack and Citrix• CloudPlatform powered
by CloudStack• Feature parity, moving
to CloudStack being upstream
• Enterprise support• Enterprise training• Main contributor to
Apache CloudStack, up to now
Why make it Open Source ?
• Large partner ecosystem needed a way to integrate quickly
• Users drive adoption in early markets – providing shortest path to adoption is open source
• Open source communities are driving integrations and standards
• Faster time to market, short feedback loop from user community
• Enterprise ready and wanted an open source solution for customers to develop on.
What Does it do ?
• Open source Infrastructure as a Service (IaaS) platform.
• Data Center orchestrator• Hypervisor agnostic (with addition of bare
metal provisioning)• Support complex enterprise networking (e.g
Firewall, load balancer, VPN, VPC…)• Multi-tenant
Other OSS IaaS Players
A Very Flexible IaaS PlatformCompute Hypervisor
Storage Block & Object
Network Network & Network Services
Primary Storage Secondary Storage
http://www.slideshare.net/cloudstack/cloudstack-architecture
Architecture / Language
• Java application• Tomcat6, Axis2, Maven build + ant– Ant going away in 4.1
• Moving towards a plugin architecture– Cocoon ? Spring ?
• Collaboration Conference, Nov 30th -Dec 2nd – http://collab12.cloudstack.org/
Build and Run in 4.1
• git clone https://git-wip-us.apache.org/repos/asf/incubator-cloudstack.git
• mvn clean• mvn install• mvn –P developer –pl developer –Ddeploydb
• mvn –pl :cloud-client-ui jetty:run
Cloud Interactions
CloudStackCloudStack
Cloud user{API client (Fog/etc)}
End User UI
End User UI
Admin UI
Admin UI
MySQLMySQL
CloudStackCloudStackClustered
CloudStackManagement
Server
ClusteredCloudStack
ManagementServer
Domain Admin
UI
Domain Admin
UI
CS Admin & End-user API
Cloud user{ec2 API client }
ec2 API
Monitoring CS API vSphere ClusterPrimaryStorage
vcentervcenter
Cluster Mgmt
XS ClusterPrimaryStorage
vCenter API
XAPI
KVM ClusterPrimaryStorageJSON
OVM Cluster PrimaryStorage
XenApi
NetConf
Nitro APIJuniper SRX
Netscaler
Console Proxy VMConsole
Proxy VMConsole Proxy VMConsole
Proxy VM
JSON
Cloud user
HTTPSAjax Console
Ajax Console
VNC
Sec. StorageVM
Sec. StorageVM
NFS Server
NFSSec. Storage
VMSec. Storage
VM
HTTP (Template Download)
HTTP (Template Copy)
HTTP (Swift)
NFS
Router VMRouter VMRouter VMRouter VM
Router VMRouter VM
JSON
{Proxied} SSH
http://www.slideshare.net/cloudstack/cloudstack-architecture
TerminologyZone: Availability zone, aka Regions. Could be worldwide. Different data centersPods: Racks or aisles in a data centerClusters: Group of machines with a common type of HypervisorHost: A Single serverPrimary Storage: Shared storage across a clusterSecondary Storage: Shared storage in a single Zone
Storage• Primary Storage:
– Anything that can be mounted on the node of a cluster. – Cluster LVM…iSCSI…– Holds disk images of running VMs– Support for CEPH with KVM hypervisors
• Secondary Storage:– Available across the zone– Holds snapshots and templates (image repo)– Can use Openstack swift or any object store (Gluster FS…)– New support for Caringo
• Can use NFS for both to start• Storage Abstraction refactoring underway
Networking
• Extremely flexible to:– Provide isolation with VLANs– Provide isolation at L3 with shared L2 (scalability)– Support hardware devices that exposes API– Deployed on existing networking infrastructure– Support new networking paradigm (SDN)• Support for Nicira Virtual P• Extensive use of Open VSwitch
Router
L3 Core Switch
Access Layer
Switches
………… …
Availability Zone
Servers
CloudStack Mgmt Server Cluster
Secondary Storage
Pod 1 Pod 2 Pod 3 Pod N
MySQL
Load Balancer
Operations Admin and Cloud API
Users
Physical Network
Slide from Chiradeep Vittal, http://www.slideshare.net/cloudstack/cloudstack-networking
Layer-2 Guest Virtual Network
Public Network/Internet
Guest Virtual Network 10.1.1.1/8VLAN 100
Gateway address 10.1.1.1
DHCP, DNSNATLoad BalancingVPN
Public IP 65.37.141.11
10.1.1.1Guest VM 1Guest VM 1
10.1.1.3Guest VM 2Guest VM 2
10.1.1.4Guest VM 3Guest VM 3
10.1.1.5Guest VM 4Guest VM 4
CSVirtual Router
CSVirtual Router
Public Network/Internet
Guest Virtual Network 10.1.1.1/8VLAN 100
Private IP10.1.1.112
DHCP, DNS
Public IP 65.37.141.112
10.1.1.1Guest VM 1Guest VM 1
10.1.1.3Guest VM 2Guest VM 2
10.1.1.4Guest VM 3Guest VM 3
10.1.1.5Guest VM 4Guest VM 4
NetScalerLoad
Blancer
NetScalerLoad
Blancer
Private IP10.1.1.111
Public IP 65.37.141.111
Juniper SRX
Firewall
Juniper SRX
Firewall
CS Virtual Router provides Network Services External Devices provide Network ServicesNetwork Hardware exposing API can be controlled
CSVirtual Router
CSVirtual Router
Slide from Chiradeep Vittal, http://www.slideshare.net/cloudstack/cloudstack-networking
L3 isolation
• To go beyond the limitation of VLANs (4096 vlans per switch)
• Move to L3 isolation– Shared L2 – Manage one firewall per VM
• Potential use of GRE tunnels to create single L2 overlay
• GRE in tech preview for site to site VPN.
L3 isolation with distributed firewallsTenant 1 VM 1
10.1.0.2
Tenant 2 VM 1
10.1.0.3
Tenant 1 VM 2
10.1.0.4
Tenant 2 VM 2
10.1.16.12
Tenant 2 VM 3 10.1.16.21
Tenant 1 VM 3 10.1.16.47
Tenant 1 VM 4 10.1.16.85
Public Internet
10.1.0.1
Public IP address 65.37.141.1165.37.141.2465.37.141.3665.37.141.80
Load Balancer
L3 Core
Pod 1 L2 Switch
Pod 3 L2 Switch
10.1.16.1
…
…10.1.8.1Pod 2 L2 Switch
Slide from Chiradeep Vittal
A customizable GUIAJAX + API
A very extensive API
API• Not really REST• A set of methods available over http(s)• Unauthenticated on integration port• Authenticated on 8080 using Access and Secret Key• Python/Ruby clients available• Internal Marvin client• CLI under development• Other clouds client support the API
Making API calls
def make_request(requests, secretKey): request = zip(requests.keys(), requests.values()) request.sort(key=lambda x: str.lower(x[0]))
requestUrl = "&".join(["=".join([r[0], urllib.quote_plus(str(r[1]))]) for r in request]) hashStr = "&".join(["=".join([str.lower(r[0]), str.lower(urllib.quote_plus(str(r[1]))).replace("+", "%20")]) for r in request]) sig = urllib.quote_plus(base64.encodestring(hmac.new(secretKey, hashStr, hashlib.sha1).digest()).strip()) print "Signature: %s"%sig requestUrl += "&signature=%s"%sig print requestUrl
if __name__ == '__main__': requests = { "apiKey": "BRZ5j4E8O4di2MZWnQsYBLThCrTGO-LGeZaMjsnvelkHuY5P8FdTnluNZTDQhCUy-wqeJzk8EAc_NbcZxTF_FA", "response" : "json", "command" : "listZones" } secretKey = "bFlx2llt3OmM4AiHzfwV1ZbuJ5tsv6hAx6IeM32CkM-obCA77BRwBr3_yQ0bO1-kdZyfD3-lY6khsXCx18n3Mw" make_request(requests, secretKey)
Can be authenticated or not.HTTP call.User Keys can be generated via the GUIBase url: http://<manager-host>:8080/client/api?….
EC2 /S3 compatibility
• Significant development work happening to make Cloudstack highly compatible with EC2 /S3 API.
• http://wiki.cloudstack.org/display/RelOps/EC2+API+support+in+CloudStack
• http://www.slideshare.net/sebastiengoasguen/cloudstack-ec2-configuration
• Euca tools, boto etc…should work with cloudstack.
Enabling EC2 and S3
• Via the GUI
• Via API call on integration API port 8096http://localhost:8096/client/api?command=updateConfiguration&name=enable.s3.api&value=true
http://localhost:8096/client/api?command=updateConfiguration&name=enable.ec2.api&value=true
Highly Scalable• See:– http://www.slideshare.net/cloudstack/scalability-
12819428– From Alex Huang. 10k “resources” managed per
Mgt server. 30k resources with 30k VM in simulation.
• Management server can be setup in a multi-node configuration with a load-balancer and replicated MySQL.
SandBox: DevCloud
• A Virtual box appliance packaged to provide a working CloudStack environment.
• Aimed at developers but has other use cases:– Xen PV hosts gives nested virtualization– Local EC2/S3 Cloud on your laptop– Networking experiments ?
• http://wiki.cloudstack.org/display/comm/DevCloud
DevCloud: self-containedCloudStack runs in the appliance
Testing “4.0” code in DevCloud
• Deploy new CloudStack code in self-contained DevCloud:
• mvn –P deps• ant rdeploy • ant rdeploydb – Wipes database of mgt server, you will need to
reconfigure the “data center”
• ant rdebug
DevCloud: as HostRun CloudStack on local machineUse DevCloud to setup hosts
DevCloud to test 4.1 branch
• mvn -P developer clean• mvn -P developer install• mvn -P developer -pl developer –Ddeploydb• mvn -P developer -pl tools/devcloud –Ddeploydb
• mvn -pl :cloud-client-ui jetty:run
Configure infrastructure:• mvn -P developer,deploysvr -pl tools/devcloud -Ddeploysvr
Testing Framework –for the PyUG • Marvin is a Python
based framework to run tests against a CloudStack install
• Could be used as a simulator of a datacenter
• Used to configure an infrastructure on a mgt server
[environment]
dns=10.147.28.6
mshost=10.147.39.69
mysql.host=10.147.39.69
[cloudstack]
private.gateway=10.147.40.1
private.pod.startip=10.147.41.121
private.pod.endip=10.147.41.160
private.netmask=255.255.254.0
public.gateway=10.147.40.1
public.vlan.startip=10.147.41.162
public.vlan.endip=10.147.41.200
public.netmask=255.255.254.0
hypervisor=XenServer
host=10.147.40.10
host.password=password
#storage pools
primary.pool=nfs://10.147.28.7:/export/home/automation/sadhu/primary
secondary.pool=nfs://10.147.28.6:/export/home/automation/sadhu/secondary
Opportunities for Java developers
• Improve Maven build• Move to Spring framework• True REST API• Plugin framework with better
componentization – Cocoon ?• Junit tests• Support for Hadoop as storage backend• …
Info
• Apache incubator project• http://www.cloudstack.org• #cloudstack on irc.freenode.net• @cloudstack on Twitter• http://www.slideshare.net/cloudstack• http://cloudstack.org/discuss/mailing-lists.html
Welcoming contributions and feedback, Join the fun !
top related