User:Yutiannie/sandbox

In mathematics, the theory of optimal stopping is concerned with the problem of choosing a time to take a particular action, in order to maximise an expected reward or minimise an expected cost. Optimal stopping problems can be found in areas of statistics, economics, and mathematical finance (related to the pricing of American options). A key example of an optimal stopping problem is the secretary problem. Optimal stopping problems can often be written in the form of a Bellman equation, and are therefore often solved using dynamic programming.

Definition

Discrete time case

Stopping rule problems are associated with two objects:

1. A sequence of random variables ${\displaystyle X_{1},X_{2},\ldots }$, whose joint distribution is something assumed to be known
2. A sequence of 'reward' functions ${\displaystyle (y_{i})_{i\geq 1}}$ which depend on the observed values of the random variables in 1.:
${\displaystyle y_{i}=y_{i}(x_{1},\ldots ,x_{i})}$

Given those objects, the problem is as follows:

• You are observing the sequence of random variables, and at each step ${\displaystyle i}$, you can choose to either stop observing or continue
• If you stop observing at step ${\displaystyle i}$, you will receive reward ${\displaystyle y_{i}}$
• You want to choose a stopping rule to maximise your expected reward (or equivalently, minimise your expected loss)

Continuous time case

Given a gain processes ${\displaystyle G=(G_{t})_{t\geq 0}}$ defined on a filtered probability space ${\displaystyle (\Omega ,{\mathcal {F}},({\mathcal {F}}_{t})_{t\geq 0},\mathbb {P} )}$ and assume that ${\displaystyle G}$ is adapted to the filtration. The optimal stopping problem is to find the stopping time ${\displaystyle \tau ^{*}}$ which maximizes the expected gain

${\displaystyle V_{t}^{T}=\mathbb {E} G_{\tau ^{*}}=\sup _{t\leq \tau \leq T}\mathbb {E} G_{\tau }}$

where ${\displaystyle V_{t}^{T}}$ is called the value function. Here ${\displaystyle T}$ can take value ${\displaystyle \infty }$.

A more specific formulation is as follows. We consider an adapted strong Markov process ${\displaystyle X=(X_{t})_{t\geq 0}}$ defined on a filtered probability space ${\displaystyle (\Omega ,{\mathcal {F}},({\mathcal {F}}_{t})_{t\geq 0},\mathbb {P} _{x})}$ where ${\displaystyle \mathbb {P} _{x}}$ denotes the probability measure where the stochastic process starts from ${\displaystyle x}$. Given continuous functions ${\displaystyle M,L,K}$, the optimal stopping problem is

${\displaystyle V(x)=\sup _{0\leq \tau \leq T}\mathbb {E} _{x}\left(M(X_{\tau })+\int _{0}^{\tau }L(X_{t})dt+\sup _{0\leq t\leq \tau }K(X_{t})\right).}$

This is sometimes called the MLS (which stand for Mayer, Lagrange, and supremum, respectively) formulation. [1]

Solution methods

There are generally two approaches of solving optimal stopping problems. [1] When the underlying process (or the gain process) is described by its unconditional finite dimensional distributions, the appropriate techniques of solution are based on concepts and methods from martingale theories, and is therefore called the martingale approach, the most important concept of which being Snell envelope. In the discrete time case, if the planning horizon ${\displaystyle T}$ is finite, the problem can also be easily solved by dynamic programming.

When the underlying process is determined by a family of (conditional) transition functions leading to a Markovian family of transition probabilities, very powerful analytical tools provided by the theory of Markov processes can often be utilized and this approach is referred to as the Markovian method. The solution is usually obtained by solving the associated free-boundary problems (Stefan problems).

A jump diffusion result

Let ${\displaystyle Y_{t}}$ be a Lévy diffusion in ${\displaystyle \mathbb {R} ^{k}}$ given by the SDE

${\displaystyle dY_{t}=b(Y_{t})dt+\sigma (Y_{t})dB_{t}+\int _{\mathbb {R} ^{k}}\gamma (Y_{t-},z){\bar {N}}(dt,dz),\quad Y_{0}=y}$

where ${\displaystyle B}$ is an ${\displaystyle m}$-dimensional Brownian motion, ${\displaystyle {\bar {N}}}$ is an ${\displaystyle l}$-dimensional compensated Poisson random measure, ${\displaystyle b:\mathbb {R} ^{k}\to \mathbb {R} ^{k}}$, ${\displaystyle \sigma :\mathbb {R} ^{k}\to \mathbb {R} ^{k\times m}}$, ${\displaystyle \gamma :\mathbb {R} ^{k}\times \mathbb {R} ^{k}\to \mathbb {R} ^{k\times l}}$ are given functions such that a unique solution ${\displaystyle (Y_{t})}$ exists. Let ${\displaystyle {\mathcal {S}}\subset \mathbb {R} ^{k}}$ be an open set (the solvency region) and

${\displaystyle \tau _{\mathcal {S}}=\inf\{t>0:Y_{t}\notin {\mathcal {S}}\}}$

be the bankruptcy time. The optimal stopping problem is:

${\displaystyle V(y)=\sup _{\tau \leq \tau _{\mathcal {S}}}J^{\tau }(y)=\sup _{\tau \leq \tau _{\mathcal {S}}}\mathbb {E} _{y}\left[M(Y_{\tau })+\int _{0}^{\tau }L(Y_{t})dt\right]}$

It turns out that under some regularity conditions, [2] the following verification theorem holds:

If function ${\displaystyle \phi :{\bar {\mathcal {S}}}\to \mathbb {R} }$ satisfies

• ${\displaystyle \phi \in C({\bar {\mathcal {S}}})\cap C^{1}({\mathcal {S}})\cap C^{2}({\mathcal {S}}\setminus \partial D)}$ where the continuation region ${\displaystyle D=\{y\in {\mathcal {S}}:\phi (y)>M(y)\}}$
• ${\displaystyle \phi \geq M}$ on ${\displaystyle {\mathcal {S}}}$
• ${\displaystyle {\mathcal {A}}\phi +L\leq 0}$ on ${\displaystyle {\mathcal {S}}\setminus \partial D}$, where ${\displaystyle {\mathcal {A}}}$ is the infinitesimal generator of ${\displaystyle (Y_{t})}$

Then ${\displaystyle \phi (y)\geq V(y)}$ for all ${\displaystyle y\in {\bar {\mathcal {S}}}}$. Moreover, if

• ${\displaystyle {\mathcal {A}}\phi +L=0}$ on ${\displaystyle D}$

Then ${\displaystyle \phi (y)=V(y)}$ for all ${\displaystyle y\in {\bar {\mathcal {S}}}}$ and ${\displaystyle \tau ^{*}=\inf\{t>0:Y_{t}\notin D\}}$ is an optimal stopping time.

These conditions can also be written is a more compact form (integro-variational inequality):

• ${\displaystyle \max \left\{{\mathcal {A}}\phi +L,M-\phi \right\}=0}$ on ${\displaystyle {\mathcal {S}}\setminus \partial D.}$

Examples

Coin tossing (${\displaystyle \mathbb {E} (y_{i})}$ converges)

You have a fair coin and are repeatedly tossing it. Each time, before it is tossed, you can choose to stop tossing it and get paid (in dollars, say) the average number of heads observed.

You wish to maximise the amount you get paid by choosing a stopping rule. If Xi (for i ≥ 1) forms a sequence of independent, identically distributed random variables with Bernoulli distribution

${\displaystyle {\text{Bern}}\left({\frac {1}{2}}\right),}$

and if

${\displaystyle y_{i}={\frac {1}{i}}\sum _{k=1}^{i}X_{k}}$

then the sequences ${\displaystyle (X_{i})_{i\geq 1}}$, and ${\displaystyle (y_{i})_{i\geq 1}}$ are the objects associated with this problem.

House selling (${\displaystyle \mathbb {E} (y_{i})}$ does not necessarily converge)
You have a house and wish to sell it. Each day you are offered ${\displaystyle X_{n}}$ for your house, and pay ${\displaystyle k}$ to continue advertising it. If you sell your house on day ${\displaystyle n}$, you will earn ${\displaystyle y_{n}}$, where ${\displaystyle y_{n}=(X_{n}-nk)}$.
You wish to maximise the amount you earn by choosing a stopping rule.
In this example, the sequence (${\displaystyle X_{i}}$) is the sequence of offers for your house, and the sequence of reward functions is how much you will earn.

Secretary problem (${\displaystyle (X_{i})}$ is a finite sequence)

You are observing a sequence of objects which can be ranked from best to worst. You wish to choose a stopping rule which maximises your chance of picking the best object.
Here, if ${\displaystyle R_{1},\ldots ,R_{n}}$ (n is some large number, perhaps) are the ranks of the objects, and ${\displaystyle y_{i}}$ is the chance you pick the best object if you stop intentionally rejecting objects at step i, then ${\displaystyle (R_{i})}$ and ${\displaystyle (y_{i})}$ are the sequences associated with this problem. This problem was solved in the early 1960s by several people. An elegant solution to the secretary problem and several modifications of this problem is provided by the more recent odds algorithm of optimal stopping (Bruss algorithm).

Search theory

Economists have studied a number of optimal stopping problems similar to the 'secretary problem', and typically call this type of analysis 'search theory'. Search theory has especially focused on a worker's search for a high-wage job, or a consumer's search for a low-priced good.

American option

The holder of an American option is allowed to exercise the right to buy (or sell) the underlying asset at any time before or at the expiry date. Therefore the valuation of American options is essentially an optimal stopping problem. In a classical Black-Scholes set-up, let ${\displaystyle r}$ be the risk-free interest rate, ${\displaystyle \delta }$ and ${\displaystyle \sigma }$ be the dividend rate and volatility of the stock. The stock price ${\displaystyle S}$ follows geometric Brownian motion

${\displaystyle S_{t}=S_{0}\exp \left\{\left(r-\delta -{\frac {\sigma ^{2}}{2}}\right)t+\sigma B_{t}\right\}}$

under the risk-neutral measure. When the option is perpetual, the optimal stopping problem writes

${\displaystyle V(x)=\sup _{\tau }\mathbb {E} _{x}\left[e^{-r\tau }g(S_{\tau })\right]}$

where the payoff function ${\displaystyle g(x)=(x-K)^{+}}$ for call option and ${\displaystyle g(x)=(K-x)^{+}}$ for put option. The variational inequality is

${\displaystyle \max \left\{{\frac {1}{2}}\sigma ^{2}x^{2}V''(x)+(r-\delta )xV'(x)-rV(x),g(x)-V(x)\right\}=0,\quad x\in (0,\infty )\setminus \{b\}}$

where ${\displaystyle b}$ is the exercise boundary. The solution of it is known to be [3]

• (Perpetual call) ${\displaystyle V(x)={\begin{cases}(b-K)(x/b)^{\gamma }&x\in (0,b)\\x-K&x\in [b,\infty )\end{cases}}}$ where ${\displaystyle \gamma =({\sqrt {\nu ^{2}+2r}}-\nu )/\sigma ,\quad \nu =(r-\delta )/\sigma -\sigma /2,\quad b=\gamma K/(\gamma -1).}$
• (Perpetual put) ${\displaystyle V(x)={\begin{cases}K-x&x\in (0,c]\\(K-c)(x/c)^{\tilde {\gamma }}&x\in (c,\infty )\end{cases}}}$ where ${\displaystyle {\tilde {\gamma }}=-({\sqrt {\nu ^{2}+2r}}+\nu )/\sigma ,\quad \nu =(r-\delta )/\sigma -\sigma /2,\quad c={\tilde {\gamma }}K/({\tilde {\gamma }}-1).}$

If the expiry date is finite, the problem is associated with a 2-dimensional free-boundary problem with no known closed-form solution.

References

{{expand article}}

1. ^ a b Peskir, Goran; Shiryaev, Albert (2006). Optimal Stopping and Free-Boundary Problems. Berlin: Birkhäuser Basel. ISBN 3764324198.
2. ^ Øksendal, Bernt; Sulem, Agnès (2007). Applied Stochastic Control of Jump Diffusions. Berlin: Springer. ISBN 3540698256.
3. ^ Karatzas, Ioannis; Shreve, Steven (1998). Methods of Mathematical Finance. New York: Springer. ISBN 0387948392.
• T. P. Hill. "Knowing When to Stop". American Scientist, Vol. 97, 126-133 (2009). (For French translation, see cover story in the July issue of Pour la Science (2009))
• Optimal Stopping and Applications, retrieved on 21 June 2007
• Thomas S. Ferguson. "Who solved the secretary problem?" Statistical Science, Vol. 4.,282-296, (1989)
• F. Thomas Bruss. "Sum the odds to one and stop." Annals of Probability, Vol. 28, 1384–1391,(2000)
• F. Thomas Bruss. "The art of a right decision: Why decision makers want to know the odds-algorithm." Newsletter of the European Mathematical Society, Issue 62, 14-20, (2006)
• R. Rogerson, R. Shimer, and R. Wright (2005), 'Search-theoretic models of the labor market: a survey'. Journal of Economic Literature 43, pp. 959–88.