In industrial environment reliability and stability are significant factors, the qualities of which are often ensured through solutions based on testing. Validation can play a key role in maintaining and even increasing the quality of the software product. Detecting errors in an early stage is also cost effective: the earlier we find a problem the easier to fix it.
This is the reason why evosoft uses daily tests, running a bunch of verifications every day. It ensures the integrity of the already implemented features and the fast discovery of incompatibilities. The regularly ran performance and load tests make it possible to trace the efficiency of the system. The number of tests needed to ensure quality in a continuously developed, long lifecycle software development project is so high, that it does not worth to run each one manually. That is why most of the developer companies deploy automated test frameworks.
The maintenance of the scenarios and the analysis of the generated results are essential parts of the automated testing. My thesis discusses these two topics in an environment, where thousands of tests are run on a daily basis. The application used to maintain the test scenarios needs much data about the test code and the syntax of the test files. In this paper I describe an alternative way to provide this information to the application. The analysis of the results raises another problem. As the tests run on more than one computer simultaneously and distributed, different errors can occur and all these aspects have to be taken into account during the analysis. The manual analysis of the results would be a huge work. In this paper I describe a tool developed and implemented by me that helps to perform these actions effectively and mostly automatic.
Both solutions resulted in a drastic increase of speed in the maintenance and analysis procedures. The implemented test code attribute hierarchy made it possible to change already existing test code or to add new methods to the codebase faster. With the new test analyzer tool is it possible to analyze up to 12500 steps in a day, compared to the former several thousands, and the results can be continuously published to the server. Without these innovations it would be very hard to handle a project of this size.