Hamiltonian (control theory)

From Wikipedia, the free encyclopedia
Jump to: navigation, search

The Hamiltonian of optimal control theory was developed by Lev Pontryagin as part of his maximum principle.[1] It was inspired by, but is distinct from, the Hamiltonian of classical mechanics. Pontryagin proved that a necessary condition for solving the optimal control problem is that the control should be chosen so as to minimize the Hamiltonian. For details see Pontryagin's maximum principle.

Notation and Problem statement[edit]

A control is to be chosen so as to minimize the objective function

where is the system state, which evolves according to the state equations

and the control must satisfy the constraints

Definition of the Hamiltonian[edit]

where is a vector of costate variables of the same dimension as the state variables .

For information on the properties of the Hamiltonian, see Pontryagin's maximum principle.

The Hamiltonian in discrete time[edit]

When the problem is formulated in discrete time, the Hamiltonian is defined as:

and the costate equations are

(Note that the discrete time Hamiltonian at time involves the costate variable at time [2] This small detail is essential so that when we differentiate with respect to we get a term involving on the right hand side of the costate equations. Using a wrong convention here can lead to incorrect results, i.e. a costate equation which is not a backwards difference equation).

The Hamiltonian of control compared to the Hamiltonian of mechanics[edit]

William Rowan Hamilton defined the Hamiltonian as a function of three variables:

where is defined implicitly by

Hamilton then formulated his equations as

Similarly the Hamiltonian of control theory (as normally defined) is a function of 4 variables

and the associated conditions for a maximum are

This definition agrees with that given by the article by Sussmann and Willems.[3] (see p. 39, equation 14). Sussmann-Willems show how the control Hamiltonian can be used in dynamics e.g. for the brachystochrone problem, but do not mention the prior work of Carathéodory on this approach.[4]


  1. ^ Dixit, Avinash K. (1990). Optimization in Economic Theory. New York: Oxford University Press. pp. 145–161. ISBN 0-19-877210-6. 
  2. ^ Varaiya, Chapter 6
  3. ^ Sussmann; Willems (June 1997). "300 Years of Optimal Control" (PDF). IEEE Control Systems. 
  4. ^ See Pesch, H. J.; Bulirsch, R. (1994). "The maximum principle, Bellman's equation, and Carathéodory's work". Journal of Optimization Theory and Applications. 80 (2): 199–225. doi:10.1007/BF02192933. 

External links[edit]