Invariant Measures for Controlled Diffusions - Adam Oberman Math

Invariant Measures for Controlled Diffusions

Contents

Background on Diffusions with no Control

Referring to Stochastic Differential Equations we see that there is a forward equation (for value functions) and a backward equation (for the evolution of measures). Kolmogorov backward equation.

-\frac{\partial}{\partial t}u(x,t)=L u \qquad \text{ for } \qquad t\le T,

subject to the final condition

u(x,T) = uT(x).

and the Kolmogorov forward equation, or Fokker–Planck equation which describes the time evolution of the probability density function of the position of a particle,

\frac{\partial}{\partial t}p(x,t)=L^* u \qquad \text{ for } \qquad t \ge 0,

with initial condition

p(x,0) = p0(x).

These equations are Adjoint_Processes, in the sense that if we define

\langle u, p \rangle = \int u dp

we have

\langle L u, p \rangle = \langle u,L^* p \rangle

which follows from integration by parts, ignoring boundary terms, by assuming, for example, that we are in the periodic setting. In particular, we have for any u,p satisfying the forward and backwards equations

\frac{d}{dt} \langle u, p \rangle = \langle u_t, p \rangle + \langle u, p_t \rangle = \langle -Lu, p \rangle + \langle u, L^* p \rangle = 0

since the operators are adjoints.

Background on Controlled Diffusions

Referring to Stochastic Control we see that the HJB is the analogue of the backward Kolmogorov equation. However there is no forward equation, for how a measure evolves under the optimal choice of constraints.

Actually, some attempted, but no satisfying equations have been proposed. See the references in Adjoint_Processes. The goal would be to have an equation which doesn't require advanced knowledge of the value function. This way the (adjoint) symmetry of the linear case is preserved.

Idea for the Forward HJB equation

Try to use the adjoint equation, and give a variational formulation of the equation to read off the adjoint. Use duality in convex analysis. In particular, we know how to compute the dual/adjoint of a the HJB equation Refer to Dual cone of the HJB equation


References

The main paper, which predates viscosity solutions is by Wendall Fleming, author of Controlled Markov processes and viscosity solutions along with H. Mete Soner.

See also the papers by Diogo Gomes

Also http://www.unisanet.unisa.edu.au/Staff/VladimirGaitsgory/

Retrieved from "http://www.divbyzero.ca/oberman/wiki/Invariant_Measures_for_Controlled_Diffusions"