Object tracking and grasping with robotic arm using image processing

OData support
Dr. Kiss Bálint
Department of Control Engineering and Information Technology

The topic of my thesis is the development of a system which catches a rolling object by a robotic arm using digital image processing. I have worked out the maneuver so that the robotic arm is restricted to move along a line and in two dimensions as well. The case of one dimension led to more reliable solution.

I have solved the image acquisition with a web camera and a with the LabVIEW suite. The image processing is based on pattern recognition where the object pattern is recorded prior to the maneuver. Since the image resolution of the camera is 1600x896 pixels, which would results in long processing time, the area of interest is reduced using image snipping. I divided the working space into segments since the movement of the robot arm was restricted.

In the examined space the measured position was further refined with the help of Kalman filtering, which I implemented in a SubVI. Every 50 ms, the LabVIEW sends the coordinates of the point which is in midway between found rolling object and the robot arm using a TCP/IP based Modbus protocol. The PLC than forwards the data to the CPU of robot arm.

In addition to the coordinates the LabVIEW communicates to the robot arm, an additional item is also sent. From this data part of the space where the ball is situated can be identified. The data defines the captured area to the robot arm. Due to the exact measurements the ball is successfully captured.


Please sign in to download the files of this thesis.