Implementing a Monitoring System on Log Analysis in Large Business Environment

OData support
Supervisor:
Dr. Martinek Péter
Department of Electronics Technology

This master’s thesis is about the implementation of a monitoring system on log analysis. I have received this task from the DBT branch of itelligence Hungary Kft.

My study has three structural parts. The first is a kind of theoretical discussion in which I have analyzed the basic and the more advanced concepts of this topic, like industry-specific solutions, technologies and systems. Of these, I specifically dealt with application monitoring, log parsers and monitoring of ERP systems.

In the following part I have defined the process to be monitored, the affected systems, the sources of the log files and the requirements related to the to-be created log-parser system. In the Design phase I defined the various use cases of the system, the architectural plan, the interaction of the system’s elements. Lastly the utilized technologies, support systems, and programming languages to be used.

The last part of my thesis covers system implementation. I explained in great detail what solutions I chose for my system to accomplish the previously set goals in both the log file analysis program and the monitoring interface. I described the operational process of my system, then proceeded on to testing its functionality, integrity and performance. Finally, I have also illustrated the potential for further expanding and evolving the system to current generation mainstream devices.

In summary, I successfully made a proactive Logparser system, which processes different type of logs from numerous sources (SAP, PPM, XML import, Microsoft SQL Server, Batch program, Excel macro, Oracle stored procedure, FTP import MashZone, Talend). Log messages are categorized based on their content, and whether they contain any type of predefined errors, in which case it sends a mail to the corresponding person. Furthermore, I created several dashboards that display the current status of a given system, their log messages in detail, moreover they show relevant data aggregated by time and state, in both raw numbers and charts.

Downloads

Please sign in to download the files of this thesis.