Exploiting Dynamic Resource Allocation for Efficient Parallel Data Processing in the Cloud

OVERVIEW IntroductionAbstractExisting systemDisadvantageProposed systemAdvantageModulesModule DescriptionModule Diagram

INTRODUCTION“Cloud" is used as a metaphor for the Internet Cloud computing is a technology that uses the internet

and central remote servers to maintain data and applications.

It is a paradigm in which virtualized resource are provided as a service over the internet to the cloud user.

Cloud computing is broken down into three segments: "application" "storage" and "connectivity."

Only pay for what we use

ABSTRACT

Parallel data processing is to make easy access of service

Perform different task using different resources as the infrastructure needed

Goal is to decrease the overload and increase the performance of the cloud

EXISTING SYSTEMVast amount data handled by traditional database

solutions

Regenerating web index

Process large amount of data occasionally

DISADVANTAGE Increase processing time and cost

Processing crawled document

Increase traffic

PROPOSED SYSTEMResource allocation by parallel data processing in cloud

Segregating all jobs of the cloud by job manager and task manager

Allocate ,access and control a set of virtual machine , run inside the data center

Nephele’s basic architecture used

ADVANTAGETime and cost efficient

Charge only for period of time allocated

Reduce traffic

SYSTEM REQUIREMENTS

HARDWARE: PROCESSOR : PENTIUM IV 2.6GHz, Intel Core2Duo RAM : 512 MB DD RAM MONITOR : 15” COLOR

SOFTWARE:

Front End : J2EE (JSP, SERVLETS, EJB)

Back End : MS SQL 2005

Operating System : Windows XP/07

IDE : Net Beans, Eclipse

Web Server : Tomcat 5.5.

MODULES

User interface

Login

Resource storage

Client request

Job manager

Task manager

Process the task

MODULE DESCRIPTION User InterfaceCreate a user page using GUI Establish the communication between client and cloudUser able to know about the overview of the whole

application LoginCheck the user credential by login pageCan add new user through user registration Resource StorageDisplay resources and services provided by cloud in

one home page

MODULE DESCRIPTION Client request Receive client request and submit to the Job manager Check the type of the taskAllocate to the right task manager Job manager Media between the cloud and resources Allocate the job of the client to the proper resources Task manager Process the real task from Job manager After process send to the client directly Process the task Process the client’s task by appropriate task manager

MODULE DIAGRAM

REFERENCE[1]Daniel Warneke and Odej Kao , “Exploiting Dynamic Resource

Allocation for Parallel Data Processing in Cloud”/vol.2011

[2] Amazon Web Services LLC, “Amazon Elastic Compute Cloud(Amazon EC2),” http://aws.amazon.com/ec2/, 2009.

[3] Amazon Web Services LLC, “Amazon Elastic MapReduce,”

http://aws.amazon.com/elasticmapreduce/, 2009.

[4] D. Battre´, S. Ewen, F. Hueske, O R. Chaiken, B. Jenkins, P.-A. Larson, B. Ramsey, D. Shakib, S. Weaver, and J. Zhou,

“SCOPE: Easy and Efficient Parallel

Processing of Massive Data Sets,” Proc. Very Large Database

Endowment, vol. 1, no. 2, pp. 1265-1276, 2008.

[5] Kao, V. Markl, and D. Warneke,“Nephele/PACTs: A Programming Model and Execution Framework for Web-Scale Analytical Processing,” Proc. ACM Symp.

Cloud Computing (SoCC ’10), pp. 119-130, 2010.

THANK YOU

QUERIES??