Stochastic Dynamic Optimisation


  • To understand the foundations of probability theory.
  • To obtain knowledge of the behaviour of martingales.
  • To understand the theory of stochastic integration.
  • To obtain knowledge of the behaviour of Brownian motion and Itô processes.
  • To obtain knowledge of the HJB equation and its solution.


The scheme below is under proviso.


Part I: Measure and probability

In essence, this course studies stochastic processes, and thus requires knowledge of the modern approach to probability. The modern approach uses measure theory to define the probability space. Measure theory does not distinguish between discrete and continuous distributions, and also covers distributions which cannot be represented as (mixtures of) discrete and continuous distributions.


Lecture 1: Spaces and σ-algebras

metric spaces, completeness, Banach space, measurable space, Borel σ-algebra.


Lecture 2: Measurable functions

induced space, indicator and simple functions, limits, Lebesgue integral.


Lecture 3: Probability spaces

random elements, expectation, L1 space, L2 space, completeness of Lp space, independent events, independent random elements, zero-one laws.




Part II: Martingales

In recent decades, martingale theory has been booming in probability theory. Martingales arise as the innovative parts of stochastic processes. Martingale theory has had a unifying influence: where previously different sorts of stochastic processes, such as Markov processes, were considered independently of each other, now these can be treated from the point of view of martingale theory as well. This makes martingale theory a good basis for further techniques.


Lecture 4: Conditional expectation

types of conditional expectation, conditional expectation in L2, conditional expectation in L1, properties of conditional expectation.


Lecture 5: Martingales, optional stopping, martingale inequalities

definition, stopping times, stopped processes, optional stopping, Doob’s inequality, upcrossing inequality.


Lecture 6: Martingale convergence, uniform integrability

forward convergence, reversed martingale, uniform integrability, UI martingales, UI reversed martingales.


Part III: Stochastic integration

A semimartingale can be decomposed as the sum of an innovation part (a local martingale) and a predictable part (a well-behaved stochastic process). Semimartingales form the largest class of stochastic processes with respect to which Itô integrals can be defined. The theory of stochastic integration is specialized to Brownian motion and related processes.


Lecture 7: Decomposition

local martingales, Doob-Meyer decomposition, predictable (co)variation, semimartingales.


Lecture 8: The Itô integral

quadratic (co)variation, one-dimensional Itô formula, Doléans-Dade exponential, multi-dimensional itô formula.


Lecture 9: Brownian motion and Itô processes

existence, properties, paths, martingale representation, stochastic differentials, Itô diffusions, Itô processes.





Part IV: Stochastic dynamic programming

Stochastic integration allows the extension of the deterministic Hamilton-Jacobi-Bellman (HJB) equation so as to include semimartingales. Optimal control of a semimartingale boils down to solving the stochastic HJB equation. Some applications are discussed.


Lecture 10: HJB equation

partial differential equation approach: deterministic and stochastic; maximum principle: deterministic case;


Lecture 11: Viscosity solutions

equivalence of HJB equation and maximum principle: deterministic and stochastic; martingale and convex duality methods;


Lecture 12: Applications

production planning and inventory; optimal exploitation of renewable sources


Oral examination, in which the participant is asked to discuss a relevant paper in his/her own field. The participant selects the paper, its relevancy needs to be approved by the lecturers before the examination.

The timetable for this course can be found in the EUR course guide.

ERIM PhD candidates and Research Master students can register for this course via Osiris Student.

External (non-ERIM) participants are welcome to this course. To register, please fill in the registration form and e-mail it to the ERIM Doctoral Office by four weeks prior to the start of the course. For external participants, the course fee is 260 euro per ECTS credit.


D.Williams, Probability with Martingales, Cambridge University Press, 2007. ISBN 0-521-40605-6.

- this text is used in parts I and II.


Morimoto, Hiroaki. Stochastic control and mathematical modeling. Applications in economics. Encyclopedia of Mathematics and its Applications 131. Cambridge University Press, Cambridge, 2010. ISBN 978-0-521-19503-4.

- this text is used in parts III and IV, and in particular contains the applications discussed in Lecture 12.


Pham, Huyên. Continuous-time stochastic control and optimization with financial applications. Stochastic Modelling and Applied Probability 61. Springer-Verlag, Berlin, 2009. ISBN 978-3-540-89499-5.

- this text is a less expensive alternative to Morimoto (2010), but does not cover the applications of Lecture 12.

Additional info

Prerequisites include some familiarity with basic concepts from calculus and probability theory such as set theory, limits, differentiation and integration, expectation/mean and probability distributions ( in particular the normal/Gaussian distribution).