Dynamic Allocation of Cloud Resources for Parallel Data Processing

Abstract

Cloud computing is the access to computers and their
functionality via the Internet. Cloud computing paradigm makes the computing be assigned in a great number of distributed computers, rather than local computer or remote server. The character of cloud
computing is in the virtualization, distribution and dynamic
extendibility. Infrastructure as a Service (IaaS) cloud computing focuses on providing a computing infrastructure that leverages system virtualization to allow multiple Virtual Machines (VM) to be
consolidated on one Physical Machine (PM) where VMs often
represent components of Application Environments (AE).Ad-hoc parallel data processing has emerged to be one of the killer applications for Infrastructure-as-a-Service (IaaS) clouds. Major Cloud computing companies have started to integrate frameworks for parallel data processing, making it easy for customers to access these
services and to deploy their programs. The processing frameworks which are currently used have been designed for static, homogeneous cluster setups and disregard the particular nature of a cloud. Consequently, the allocated compute resources may be inadequate for
big parts of the submitted job and unnecessarily increase processing time and cost. The objective of this paper is to explicitly exploit the dynamic resource allocation offered by today’s IaaS clouds for both,
task scheduling and execution. Particular tasks of a processing job can be assigned to different types of virtual machines which are automatically instantiated and terminated during the job execution.