Research and optimization of efficient storage solutions for cloud

OData support
Supervisor:
Németh Felicián Bálint
Department of Telecommunications and Media Informatics

Infrastructure-as-a-Service (IaaS) capacities are offered as a matured product (amazon.com, cloud.hu). There is but a category of tasks, which the cloud services are not adequate for: these are the tasks, which need quick disk access. This usually means 1-2 Gb of data, that has to be written and read fast. Typically, this is the case if the data is generated continuously, like network trafficking logs or real-time processing of web-contents. Disk access in a cloud environment passes through multiple abstraction layers, causing serious performance-losses.

During the private cloud project of Hungarian Academy of Sciences, Institute for Computer Science and Control the requirements and feasibility for such systems were investigated. The chosen implementation for the system is Kernel-based Virtual Machine (KVM). This thesis contains the documentation of the optimization and research of storage related issues.

The first part of the thesis presents the technologies related to the cloud environment and KVM, particularly, those that are related to our implementation. Libvirt, Qemu, Intel VT-d and AMD-V are introduced shortly. To compare the possible choices within the implementation different configurations were measured. Bonnie++ was used for the measurements of disk performance, and cpustat was used to monitor the cpu performance.

In the hardware layer, I compared traditional HDDs to SSDs and RAM-based disks, with various drivers, such as VirtIO and IDE. I also measured the performance overhead for image-based and partition-based volumesó. I compared the ext4, XFS, btrfs and zfs filesystems alongside with some Linux disk I/O schedulers: cfq, noop and deadline. I also investigated the impact of the different Qemu strategies on performance.

I explored options for backing up running virtual machines: LVM snapshot and read-only iSCSI mount, and also using software RAID within the virtual machine, consisting of iSCSI- and RAM-based disks.

Finally, based on the experiences of the measurements, I made recommendations on the configurations of the virtual machines in the private cloud project.

Downloads

Please sign in to download the files of this thesis.