Optimizing the parameters of stochastic models: methods and challenges

OData support
Molnár Vince
Department of Measurement and Information Systems

We can see models on any fields of science. As engineers we always make the real life simpler in a way to solve our tasks. In those frequent cases, when we can't omit the probability of the occurrence of an event, we have to work with stochastic models.

In case of a stochastic model, we calculate probability distibutions, expected values and firing probabilities. If our model is parametric, these transition probabilites are unknown variables. We face these problems in the design process of various systems, as IT engineer for example at designing safety systems or service performance.

A process is Markov chain, if its future and past values are independet, the future depends on only the present state of the model. Besides, the state changes can occur at any time.

With concrete parameters as inputs, the output of the system is observable. In this way, although the function that represents the system is unknown, we can use these results to solve our problem -- in our case, to optimize the parameters. We use parameter synthesis to search for the parameters with which the model meets the requirements given in the specification.

The main step at cheking probability models is the calculation of the accessible probabilites: how likely is to get to a state? In case of nondeterministic models, just like the Markov decision process, we can solve the nondeterminism with these probabilites. With their assistance, using iterative techniques we can evaluate the model.

However, in lots of cases the evaluation of a model is not the primary goal. We search for optimal solutions on all fields of life, as well as at designing a system.

In this thesis I'm looking for the answer of the question, with a certain model which parameters will lead to the optimal behavior. For this purpose I'm going to try several different optimization algorithms, using my own and also existing implementations to find an optimal solution.

I work with models written in the PetriDotNet application, which was a project at the department. The SPDN solver evaluates the qualitiative properties of these models. SPDN was also developed at the department. I use these tools with many common optimalization algorithm to find the optimal parameters which approaches our objective function the best. This objective function is defined as the squared sum of the difference of the measured and expected values. Our task is to reduce this error rate to as small as we can.


Please sign in to download the files of this thesis.