Stochastic Dynamic Optimisation

Information

The scheme below is under proviso.

Part I: Measure and probability

In essence, this course studies stochastic processes, and thus requires knowledge of the modern approach to probability. The modern approach uses measure theory to define the probability space. Measure theory does not distinguish between discrete and continuous distributions, and also covers distributions which cannot be represented as (mixtures of) discrete and continuous distributions.

Lecture 1: Spaces and σ-algebras

metric spaces, completeness, Banach space, measurable space, Borel σ-algebra.

Lecture 2: Measurable functions

induced space, indicator and simple functions, limits, Lebesgue integral.

Lecture 3: Probability spaces

random elements, expectation, L1 space, L2 space, completeness of Lp space, independent events, independent random elements, zero-one laws.

Part II: Martingales

In recent decades, martingale theory has been booming in probability theory. Martingales arise as the innovative parts of stochastic processes. Martingale theory has had a unifying influence: where previously different sorts of stochastic processes, such as Markov processes, were considered independently of each other, now these can be treated from the point of view of martingale theory as well. This makes martingale theory a good basis for further techniques.

Lecture 4: Conditional expectation

types of conditional expectation, conditional expectation in L2, conditional expectation in L1, properties of conditional expectation.

Lecture 5: Martingales, optional stopping, martingale inequalities

definition, stopping times, stopped processes, optional stopping, Doob’s inequality, upcrossing inequality.

Lecture 6: Martingale convergence, uniform integrability

forward convergence, reversed martingale, uniform integrability, UI martingales, UI reversed martingales.

Part III: Stochastic integration

A semimartingale can be decomposed as the sum of an innovation part (a local martingale) and a predictable part (a well-behaved stochastic process). Semimartingales form the largest class of stochastic processes with respect to which Itô integrals can be defined. The theory of stochastic integration is specialized to Brownian motion and related processes.

Lecture 7: Decomposition

local martingales, Doob-Meyer decomposition, predictable (co)variation, semimartingales.

Lecture 8: The Itô integral

quadratic (co)variation, one-dimensional Itô formula, Doléans-Dade exponential, multi-dimensional itô formula.

Lecture 9: Brownian motion and Itô processes

existence, properties, paths, martingale representation, stochastic differentials, Itô diffusions, Itô processes.

Part IV: Stochastic dynamic programming

Stochastic integration allows the extension of the deterministic Hamilton-Jacobi-Bellman (HJB) equation so as to include semimartingales. Optimal control of a semimartingale boils down to solving the stochastic HJB equation. Some applications are discussed.

Lecture 10: HJB equation

partial differential equation approach: deterministic and stochastic; maximum principle: deterministic case;

Lecture 11: Viscosity solutions

equivalence of HJB equation and maximum principle: deterministic and stochastic; martingale and convex duality methods;

Lecture 12: Applications

production planning and inventory; optimal exploitation of renewable sources