Generic task distribution system on Microsoft Azure platform

OData support
Dr. Kővári Bence András
Department of Automation and Applied Informatics

These days, cloud computing based services are steadily gaining significance in the IT sector, making access to various computing resources more akin to public utilities, like electricity or drinking water. It is possible for researchers and companies, who up till now could not afford to invest in the kind of computing capacity necessary to process large amounts of data, to profit from this, because now they have the option to rent the capacity from a public cloud, in an economically sound fashion.

My task was to create an application, which can be deployed into the Microsoft Azure public cloud, capable to run an arbitrary parallel algorithm on a large number of data, using the user-given number of machine instances, and to create user interfaces to help interacting with the application. The goal is to save the user of the application from having to do any cloud specific programming, so that he can focus on the creation of the algorithm, and the analysis of the its results.

In the first part of this paper, I'll review the technologies necessary to create this application. Cloud technologies in general and the Microsoft Azure cloud in particular will be discussed. In the part about Azure, I'll focus on the specific services I used to create the application. Reflection will also be discussed, since it's necessary for the extraction of the algorithm from the Assembly uploaded by the user.

In the second part of this paper, I'll review the architecture of the application, the more important data structures, the flow of an entire user session, as well as that of the processing of a single unit of data.

In the last part of this paper, I'll examine the behavior of the application while it's running an algorithm. Specifically, I'll run the Sobel operator on a number of pictures, therefore confirming that the application do indeed works, and it is capable to run an arbitrary algorithm in the Microsoft Azure cloud.


Please sign in to download the files of this thesis.