Industrial application development for mixed reality device

OData support
Dr. Forstner Bertalan
Department of Automation and Applied Informatics

During industrial processes, the operatives have to execute complicated tasks consisting of several steps, each of them carrying potential costs upon misconduct. The training of these complex tasks costs time, since an experienced specialist is needed to tutor the new workforce, who otherwise could focus on productive activities. Furthermore, the breakdown of a machine may cause much trouble. In case the local repairman can not carry out the task, the firm may lose a substantial portion of its income until the mechanic's arrival.

These problems can be solved by a monitoring, process supporting, instructor system. Present solutions are suffering from several drawbacks: the processes are stationary, training requires the attention of a professional, and the slow reaction. Another challenge is the binding of the physical objects in the supporting system. Several researches in the field proposed such a system, which required the pre-upload of relevant, process-specific data and 3D models. Filling the system with these 3D models require skills, time, and money.

With the help of the new mixed reality headsets, there is an opportunity for creating a system, which can display and explain the steps of the tasks. The system can learn the environment of the user’s workstation, the exact task to be performed, the cabinet of the parts to be used, and where to find all necessary parts. The user can watch a 3d animation or film snippet to find out the next step of the process, and the headset also marks the physical place, like a drawer, where the part to be used can be found. The headset also identifies the part and can even warn if the wrong part was chosen. To the written problems there are no elaborated solutions yet.

Reflecting on the issue above, the paper presents the suitable prototype with the solution. Moreover, it shows how can we link the process and the steps to the existing manufacturing process without significant IT knowledge, and from a remote place.

In the paper I investigate existing technologies for the implementation of the system, I pan out the inconvenience of the solutions, and I suggest a new solution with the new technologies. To validate the results, I designed a prototype system, and describe the difficulty of the implementation. Finally, I summarize the functioning process of the ready-made demonstrative system, and introduce opportunities for future improvements.


Please sign in to download the files of this thesis.