Development of a Human-Computer Interface Based on Object Recognition and Tracking

OData support
Dr. Vajda Ferenc
Department of Control Engineering and Information Technology

The goal of virtual and augmented reality systems is to provide new and efficient methods for the interaction between man and machine. A group of these systems, called Tangible Augmented Reality (TAR) achieves this goal by using real objects as an input device. These systems provide an intuitive, natural user interface by allowing users to manipulate virtual objects interacting with the aforementioned real objects.

The central subject of my master’s thesis is the design and implementation of a TAR system that is able to insert virtual objects intelligently into a real scene of which the system has no a priori knowledge. The system is able to create an intelligent pairing of the real and virtual objects based on their similarity of shape. Using this method of pairing the system allows users to interact with virtual object physically without neural conflict.

The main part of my work was the creation of a framework responsible for handling the algorithms of the TAR system, and the implementation of the algorithms themselves. The most important algorithms are a shape matching method used to pair virtual and real objects, and the real-time tracking of real objects used to enable interaction. I use a 3D scene constructed from the images of a stereo camera pair, and a graph-kernel based SVM learning algorithm to implement the shape matching. The real-time tracking algorithm is based on natural image features.


Please sign in to download the files of this thesis.