Development of a Gesture Based Manipulation System for Virtual Environments

OData support
Dr. Vajda Ferenc
Department of Control Engineering and Information Technology

Virtual reality systems and augmented reality environments have become the target of more and more scientific research. The increasing processing capabilities of hardware continuously open new possibilities for implementation of more complex real-time processing algorithms. The debut of several low cost cameras with the ability to capture the depth characteristics of a working scene has inspired hundreds of projects. The application possibilities of the combination of a well-working gesture detection system and a responsive virtual environment cover almost all aspects of everyday life from medicine to home entertainment, as well as marketing, tourism, industrial design, robotics and the modelling for oil and gas exploration. Surface based multi-touch devices are so common that a user interface cannot become successful without a natural user interface. The development of freehand multi-touch interaction is the next task in this area.

In this thesis the base research towards the finding of the best implementation of such a system is targeted. Contemporary research is extensively reviewed for most commonly used hardware for depth recording and to collect a knowledge base of applicable algorithms and techniques that has proven successful in processing the measured input. Frameworks with the highest level of services have been selected and modern software architecture patterns have been taken into account during system planning.

The reproduction of the results reported on in three of the reviewed works was attempted. Complemented by own methods and combined with ideas in other publications two fingertip detection solutions were neglected to focus on the improvement of the best performing convexity defects based fingertip detection algorithm. Primitive gestures and statistics based gesture recognition approaches were designed. A small virtual environment was developed to test the capabilities of the alignment matching and displacement statistics based gesture recognition.

Performance testing revealed that the development of new heuristics for more robust fingertip detection and tracking is needed. The small computational complexity of the tip detection method leaves room for rich extension possibilities of the gesture recognition apparatus.


Please sign in to download the files of this thesis.