evaluating cloud application development and runtime platforms project team: supervisors - a/prof....

Evaluating Cloud Application Development and Runtime Platforms

Project Team:Supervisors - A/Prof. Anna Liu (annaliu@cse) and Dr. Helen Paik (hpaik@cse)

Students - Fei Teng (ften303@cse), Liang Zhao (lzha077@cse) and Xiaomin Wu (xmwu432@cse)

Summary

• Architecture overview of Cloud Platforms• Test set up description• Various tests and results• Qualitative evaluation

Architecture Overview - Azure

• 1.1

Architecture Overview - Amazon

• 1.2

Architecture Overview - AppEngine

• 1.3

Test Set up Description

• Test application architecture• Test design rationale• Sample testing code snippets

Test Set up Description• 2.1 Test application architecture

Interface : public Result InstanceResponse(String value){ // Echo the receiving value back to client // Test instance connection time}public Result Read(String value){ // Retrieve data from DB based on the given value // Test DB read performance}public Result Create(String content){ // Persist given content into DB // Test DB write performance}

Test Set up Description

• 2.2 Test design rationale2.2.1 Terminology

Test Set up Description2.2.2 RationaleStrategy one : Concurrent request based (Thread based)

We try to fire fix amount of concurrent requests to evaluate server performance, to see how efficient the platform is.

Strategy two : Duration basedWe continually and concurrently fire request to server,

keep server always busy with a fixed amount of request, to see how much stress the server can stand.

2.2.3 Sample testing code snippets// init resourcefor (int i = 0; i < TOTAL_THREADS; i++) { threadGroup.add(new Thread( ThreadFactory.coloneThreadInstance(evaThread))); } try { // fire threads all together for (Thread thread : threadGroup) { thread.start(); } // waiting till all threads finished for (Thread thread : threadGroup) { while (thread.isAlive()) { try { Thread.sleep(5); continue; } catch (InterruptedException ex) { log.error("Failed to sleep thread in App.", ex); } } } } catch (Exception e) { log.error("Failed to run thread.", e.getMessage()); }

3.Various tests and results

• Time consumed under different amount of concurrent connection

Expected Result

Current Observa-tionPredict Trend

0 200 400 600 800 1000 1200 1400 1600 18000

2000

4000

6000

8000

10000

12000

14000

16000

18000

Azure StorageAzureApp EngineAmazon

Concurrent Requests

Tota

l run

ning

Tim

e (M

ilise

cond

s)

Various tests and results• Host ability to handle concurrent connection, how

long does it take to process one request

0 200 400 600 800 1000 1200 1400 1600 18000

100

200

300

400

500

600

700

800

Azure with StorageLogarithmic (Azure with Storage)Azure with SDSLogarithmic (Azure with SDS)App EngineLogarithmic (App Engine)AmazonLogarithmic (Amazon)

Concurrent Requests

Aver

age

Conn

ectio

n Ti

me

(Mili

Seco

nds)

Various tests and results

• Database read performance

0 100 200 300 400 500 600 700 800 9000

200

400

600

800

1000

1200

1400

1600

1800

2000

Azure StorageSDSApp EngineAmazon SimpleDB

Concurrent Requests

Aver

age

Proc

essin

g Ti

me/

Requ

est


• Database write performance

0 100 200 300 400 500 600 700 800 9000

100

200

300

400

500

600

700

800

Azure StorageApp EngineAmazon SimpleDBSDSAmazon Local DBApp Engine Batch

Concurrent Requests

Aver

age

Proc

essin

g Ti

me/

Requ

est


• Database write performance error occurred rate

0 100 200 300 400 500 600 700 800 9000.00%

5.00%

10.00%

15.00%

20.00%

25.00%

30.00%

35.00%

Azure StorageApp EngineAmazon SimpleDBSDSAmazon Local DBApp Engine Batch

Concurrent Requests

Erro

r Pre

cent

ages

4. Quantitative Evaluation

• 4.1 Programming model

Azure AppEngine Amazon EC2

.Net technology only: ASP.Net, ASMX, WCF …

After install Azure SDK,There will be a Cloud Project template in visual studio

Python, Java(April 8th) Granted total control to a virtual machine

4. Quantitative Evaluation• 4.2 Development, deployment and testing tool

supportAzure AppEngine Amazon

Development Visual Studio,Azure SDK

Eclipse plug-in (For java only),AppEngine SDK

Deployment Via web control panel

Command line come with SDK

Amazon EC2 API ToolsCommand line via SSH

Testing tool support

Visual studio,Command line terminal for SDS,SDS explorer

Online log for per request

Debug mode

SSH to server


• 4.3 Management, Monitoring and metering support


Simple web control panel,a) Deploy, undeploy, stop and run host.b) See up time and down timec) Generate access key for Azure storage

Rich web control panel,Detail information of host and database usage.a) Cost reportb) Quota (request,

bandwidth, CPU …)c) Request log.d) Cron schedulee) Database viewer

Database import

Rich web control panel,Detail information of host and database usage.a) Cost reportb) Generate access keyc) Manage Instance(Stop,

start, change location, generate IP …)


• 4.4 costAzure AppEngine Amazon EC2

Under beta, free to use for 2000 VM hours

Limitation: Cloud storage capacity: 50GBTotal storage bandwidth: 20GB/day

Free to use, but with limit CPU, storage and request ….

Want more quota, pay more…

Charge by a) Different size of

instance(Small, large, extra large)

b) Different kind of OSc) Different kind of CPU

performance(Medium, extra large)

d) Per request, among of storage

e) Different cloud location

4. Quantitative Evaluation• Security modelsAzure AppEngine Amazon EC2

a) Azure host : Username/password, and application ID

b) Azure storage : Share key together with account name (Share key generated from web control panel)

c) SDS and .Net service1>Username/Password2> X.509 Certificates3>Windows CardSpace ™ Information Card

Username/Password

Firewalled DB access

a) Public-private key pairb) X.509 Certificatec) SSH

Others


a) Azure Storage Queue, task put in the queue cannot larger than 8KB

b) Can Specify multi WebRole and WorkerRole

c) So far, SDS and Azure storage none batch and Tx support

d) Maximum entries retrieve from DB 1000

e) Azure storage table, create, delete table don`t work sometimes

a) Cannot do long processing job. E.g cannot delete all data all at once

b) Limitation on batch DB operation 500 entries in maximum

a) No Tx supportb) No querying across

tables

More detail :http://docs.amazonwebservices.com/AmazonSimpleDB/2007-11-07/DeveloperGuide/index.html?SDBLimits.html

http://docs.amazonwebservices.com/AmazonSimpleDB/2007-11-07/DeveloperGuide/index.html?SDBLimits.html





Future work areas

• Classify enterprise by different amount of request. Give solution to different kind enterprise.

End! Thanks!

evaluating cloud application development and runtime platforms project team: supervisors - a/prof....

Documents