Analysis of movies by multimedia mining tools

OData support
Dr. Szűcs Gábor
Department of Telecommunications and Media Informatics

My thesis presents the system I developed for automatically creating annotations for movies in certain aspects. These aspects include analyzing the shots through their field view and camera movements, while also interpreting the dynamic and crowdedness of the pictures. In the first part of my thesis I present the details on the film industry nomenclature regarding the aforementioned shot properties. Then I introduce the essential computer vision algorithms I used during my work, such as optical flow, SURF descriptors, Viola-Jones detector, Canny edge detector.

In the second half of my thesis I describe the designed system. I explain the details of the subtasks involved, for example shot detection, feature extraction, training and classification. Classification is performed by neural networks and linear regression, depending on the type of film property analyzed. The finished system includes the use of different frameworks and platforms. For shot detection and video manipulation during development I used ffmpeg, for feature extraction and outputting data I wrote an application in C++ using OpenCV library, for data aggregation and merging it with manual annotations I wrote an Excel VBA script, and for classification and validation I used RapidMiner. I tested the whole system on a complete movie, evaluated the results, and considered the possible improvements that can be implemented in order to better the performance of the movie annotation system.


Please sign in to download the files of this thesis.