Stochastic Dynamic Optimization

Information

The scheme below is under proviso.

Part I : Measure and probability

In essence, this course studies stochastic processes, and thus requires knowledge of the modern approach to probability. The modern approach uses measure theory to define the probability space. Measure theory does not distinguish between discrete and continuous distributions, and also covers distributions which cannot be represented as (mixtures of) discrete and continuous distributions.

Lecture 1: Spaces and σ-algebras

•  metric spaces, completeness, Banach space, measurable space, Borel σ-algebra.

Lecture 2: Measurable functions

•  induced space, indicator and simple functions, limits, Lebesgue integral.

Lecture 3: Probability spaces

•  random elements, expectation, L^1 space, L^2 space, completeness of L^p space, independent events, independent random elemens, zero-one laws.

Part II : Martingales

In recent decades, martingale theory has been booming in probability theory. Martingales arise as the innovative parts of stochastic processes. Martingale theory has had a unifying influence: where previously different sorts of stochastic processes, such as Markov processes, were considered independently of each other, now these can be treated from the point of view of martingale theory as well. This makes martingale theory a good basis for further techniques.

Lecture 4: Conditional expectation

•  types of conditional expectation, conditional expectation in L^2, conditional expectation in L^1, properties of conditional expectation.

Lecture 5: Martingales, optional stopping, martingale inequalities

•  definition, stopping times, stopped processes, optional stopping, Doob’s inequality, upcrossing inequality.

Lecture 6: Martingale convergence, uniform integrability

•  forward convergence, reversed martingale, uniform integrability, UI martingales, UI reversed martingales.

Part III : Stochastic integration

A semimartingale can be decomposed as the sum of an innovation part (a local martingale) and a predictable part (a well-behaved stochastic process). Semimartingales form the largest class of stochastic processes with respect to which Itô integrals can be defined. The theory of stochastic integration is specialized to Brownian motion and related processes.

Lecture 7: Decomposition

•  local martingales, Doob-Meyer decomposition, predictable (co)variation, semimartingales.

Lecture 8: The Itô integral

•  quadratic (co)variation, one-dimensional Itô formula, Doléans-Dade exponential, multi-dimensional itô formula.

Lecture 9: Brownian motion and Itô processes

•  existence, properties, paths, martingale representation, stochastic differentials, Itô diffusions, Itô processes.

Part IV : Stochastic dynamic programming

Stochastic integration allows the extension of the deterministic Hamilton-Jacobi-Bellman (HJB) equation so as to include semimartingales. Optimal control of a semimartingale boils down to solving the stochastic HJB equation. Some applications are discussed.

Lecture 10: HJB equation

•  partial differential equation approach: deterministic and stochastic; maximum principle: deterministic case;

Lecture 11: Viscosity solutions

•  equivalence of HJB equation and maximum principle: deterministic and stochastic; martingale and convex duality methods;

Lecture 12: Applications

•  production planning and inventory; optimal exploitation of renewable sources