{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", " \"Transposition\n", "

\n", " Stochastic calculus and control - tutorial

\n", "

Stochastic Control

\n", "

Eduardo Abi Jaber

\n", "
\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We start by extending the definition of the Brownian motion\n", "to the vector case.\n", "\n", "
Definition. Let $(W_t)_{t \\geq 0}$ be a stochastic process on the probability space $(\\Omega, \\mathcal{F}, \\mathbb{P})$, and $\\mathbb{F}$ its canonical filtration. $W$ is an $\\mathbb{F}-$Brownian motion if\n", "
    \n", "
  1. (i) $W_0 = 0$,
    2. \n", "
  2. (ii) For all $0 \\leq s \\leq t$, $W_{t} - W_{s}$ is independent of $\\mathcal{F}_{s}$,
    3. \n", "
  3. (iii) For all $0 \\leq s \\leq t$, $W_{t} - W_{s} \\sim \\mathcal{N}(0, t-s)$.
    4. \n", "
\n", "
\n", "\n", "
Definition. Let $(W_t)_{t \\geq 0}$ be an $\\mathbb{R}^{d}-$valued stochastic process on the probability space $(\\Omega, \\mathcal{F}, \\mathbb{P})$. $W$ is a Brownian motion if the components $W^i, i \\in \\{1 \\ldots d\\}$ are independent Brownian motions, i.e. (i), (ii) hold and for all $0 \\leq s \\leq t$, $W_t - W_s \\sim \\mathcal{N}(0, (t-s)I_d)$ where $I_d$ is the indentity matrix of dimension $d$.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "\n", "

1. Portfolio allocation with multiple assets\n", "

\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Consider a market with $d$ risky assets $S=(S^1,\\ldots, S^d)$ with (multi-dimensional) Black-Scholes dynamics of the form \n", "\n", "$$ dS_t = \\mbox{diag}{(S_t)} \\left(\\mu dt + A dW_t\\right) $$\n", "\n", "with $W$ is a $d$-dimensional Brownian motion, a vector $\\mu \\in \\mathbb R^d$, a volatility matrix $A\\in \\mathbb R^{d\\times d}$ and $\\mbox{diag}{(S_t)}$ is a $d\\times d$ diagonal matrix with diagonal components $(S^1_t,\\ldots, S^d_t)$. \n", "\n", "Consider an investor with initial wealth $X_0=1$ fully invested in the risky assets using an $\\mathbb R^d$-valued self-financing strategy $(\\pi_t)_{t\\geq 0}$, where $\\pi^i_t$ corresponds to the amount of shares held in asset $S^i$ at time $t$. \n", "\n", "\n", "**General Questions**\n", "1. What can you say when $AA^\\top = \\mbox{diag}(\\sigma_1^2, \\ldots, \\sigma^2_d)$?\n", "2. What is meant by self-financing strategy?\n", "3. Justify that the dynamics of the wealth process of the investor $X$ are given by \n", "$$dX_t = X_t \\alpha_t^\\top (\\mu dt + A dW_t) $$\n", "with $\\alpha_t = \\mbox{diag}(S_t)\\pi_t/X_t $\n", "4. Interpret the quantity $\\alpha^i_t$. \n", "5. Can you solve the equation for $X$ explicitily? Denote by $\\Sigma = AA^\\top$ the covariance matrix which we assume to be invertible. \n", "\n", "In what follows, we denote $X$ by $X^{\\alpha}$ to make explicit the dependence of the wealth on the investement strategy $\\alpha$. \n", "We assume that the investor has a logarithm utility and seeks to maximize $\\mathbb E[\\log X^{\\alpha}_T]$ over all $\\mathbb R^d$-processes $\\alpha$ , where $T$ corresponds to the final time horizon. \n", "\n", "**Constant case**\n", "We assume first that the investors looks for investements where $\\alpha$ is kept constant trhought the horizon $[0,T]$. \n", "\n", "6. For a constant control $\\alpha$ what is the distribution of $X^{\\alpha}$?\n", "\n", "7. Show that the optimal deterministic solution $\\alpha^{*,det}$ is given by \n", "$$ \\alpha^{*,det} = \\Sigma^{-1} \\mu. $$\n", "and compute the optimal value of the problem $J^{*,\\det} = \\max_{\\alpha \\in \\mathbb R^d} \\mathbb E[\\log X^{\\alpha}_T]$\n", "8. Comment. \n", "\n", "\n", "**General case**\n", "Now we consider general square-integrable processes $(\\alpha_t)_{t\\geq 0}$.\n", "\n", "9. Write the HJB equation for the problem. \n", "10. Making a suitable Ansatz on the solution, solve the problem. \n", "11. Comment. \n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "\n", "

2. Linear Quadratic control

\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let $T > 0$ a finite time horizon and $\\left( \\Omega, \\mathcal{F}, (\\mathcal{F}_t)_{0\\leq t \\leq T}, \\mathbb{P} \\right)$ a filtered probability space. \n", "\n", "For a given admissible control in \n", "$$ \\mathcal A = \\{\\alpha \\text{ progressively measurable: } \\mathbb E[\\int_0^T \\alpha_s^2]ds<\\infty \\},$$\n", "\n", "we consider the following controlled variable:\n", "\n", "$$ dX^{\\alpha}_s = \\left( BX^{\\alpha}_s + C\\alpha_s \\right)ds + \\left( DX^{\\alpha}_s + F\\alpha_s \\right)dW_s$$ \n", "\n", "with $B,C,D,F \\in \\mathbb R$ and $X_0 \\in \\mathbb R$.\n", "\n", "Our aim is to minimize over all controls in $\\mathcal A$ the following cost functional: \n", "$$ J(\\alpha) = \\mathbb E\\left[ \\int_0^T \\left(Q(X_s^{\\alpha})^2 + N \\alpha_s^2 \\right) ds \\right] $$\n", "with \n", "$ Q \\geq 0$ and $N>0$. \n", "\n", "\n", "# Generalities\n", "\n", "1. For $\\alpha \\in \\mathcal A$, argue that there exists a unique solution $X^{\\alpha}$ to the Stochastic Differential equation. \n", "2. Can we find $X^{\\alpha}$ explicitely in terms of $\\alpha$?\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Probablistic approach \n", "We make the following Ansatz for the value function:\n", "$$V_t^{\\alpha}= \\Gamma_t (X^{\\alpha})^2_t $$\n", "for some deterministic function $t\\to \\Gamma_t$ to be determined that we assume to be continuously differentiable such that $\\Gamma_T=0$.\n", "\n", "Define \n", "$$ S_t^{\\alpha}:= V_t^{\\alpha} + \\int_0^t \\left(Q X_s^2+ N \\alpha_s^2 \\right) ds $$\n", "\n", "\n", "\n", "3. For $\\alpha \\in \\mathcal A$, show that \n", "\n", "\\begin{align*}\n", "d S_t^{\\alpha} &= X_t^2 \\left( \\dot \\Gamma_t + Q + 2B \\Gamma_t + D^2 \\Gamma_t - \\left( N + F^2\\Gamma_t \\right)^{-1} \\left( C\\Gamma_t + DF\\Gamma_t \\right)^2 \\right) dt \\\\\n", "&\\quad \\quad \\quad + \\left( N + F^2\\Gamma_t \\right) \\left( {\\alpha_t} - {\\alpha^*_t} \\right)^2 dt \\\\\n", "&\\quad \\quad \\quad + {2 \\left( D\\Gamma_t X_t^2 + F \\alpha_t X_t \\right)dW_t}\n", "\\end{align*}\n", "\n", "for some explicit $\\alpha^*$ to be determined. \n", "4. Deduce a necessary and sufficient condition on $\\Gamma$ such that the process $M^{\\alpha} := S^{\\alpha}- \\int_0^{\\cdot} \\left( N + F^2\\Gamma_s \\right) (\\alpha_s - \\alpha_s^*)^2 ds $ is a {local martingale} for any $\\alpha \\in \\mathcal A$. \n", "\n", "5. Assuming such condition satisfied and assuming that the local martingale $M^{\\alpha}$ is a true martingale show that \n", "\n", "\\begin{align*}\n", "J_t(\\alpha) - V_t^\\alpha &\\geq 0,\n", "\\end{align*}\n", "where $$ J_t(\\alpha):=\\mathbb E\\left[\\int_{t}^{T} \\left( Q X_s^2 + \\alpha_s^2 \\right)ds | \\mathcal{F}_t \\right],$$\n", "for $\\alpha \\in \\mathcal A$. \n", "\n", "6. Deduce that $\\alpha^*$ is an optimal control and $V^{\\alpha^*}_t$ is the value function of the problem:\n", "$$ V_t^{\\alpha^*}= \\inf_{\\alpha \\in \\mathcal A_t(\\alpha^*)} J_t(\\alpha) $$\n", "where \n", "$$ \\mathcal A_t(\\alpha'):=\\{ \\alpha \\in \\mathcal A: \\alpha_s = \\alpha_s', \\quad s\\leq t \\}.$$\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# PDE approach \n", "\n", "7. Solve the problem using the PDE approach. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "\n", "

3. Optimal execution/trading with Market Impact

\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let $T > 0$ a finite time horizon and $\\left( \\Omega, \\mathcal{F}, (\\mathcal{F}_t)_{0\\leq t \\leq T}, \\mathbb{P} \\right)$ a filtered probability space. \n", "\n", "Consider a trader holding an initial inventory $q$ in some given asset $S$ and is looking to liquidate/trade the asset on interval $[0,T]$. Its inventory at times $t$ is given by:\n", " \\begin{align*}\n", " Q_t^u = q -\\int_0^t u_s ds \n", " \\end{align*}\n", " with $(u_s)_{0 \\leq s \\leq t}$ the speed of rate, $t \\in [0,T]$ and $q > 0$ the initial amount of shares in the risky asset.\n", " \n", "In this model, we suppose that the visible price is composed of three components: the unaffected price and two impacts, one temporary and one transient. \n", " \\begin{align*}\n", " S_t = \\underbrace{{P_t}}_{\\mbox{unaffected price}} - \\underbrace{\\lambda u_t}_{\\mbox{lin. temp}} - \\underbrace{\\kappa Z_t^u}_{\\mbox{transient}}\n", " \\end{align*}\n", " \n", "The transient impact is defined as follows:\n", " \\begin{align*}\n", " Z_t^u= \\int_0^t G(t-s)u_s ds,\n", " \\end{align*}\n", " for some square integrable kernel $G$. \n", " \n", "In the literature, the transient impact considered are : \n", " - $G(t) = 1$: permanent impact case \n", " - $G(t) = c e^{-\\lambda t}$: exponential decay \n", " - $G(t) = c(\\epsilon + t)^{-\\alpha}$, $\\alpha>0$: power law decay \n", " \n", "We introduce the price predictive signal in the unaffected price:\n", " \\begin{align*}\n", " P_t = M_t + A_t, \n", " \\end{align*}\n", " $(P_t)_{t \\in [0, T]}$ is a semimartingale decomposed in a local martingale $(M_t)_{t \\in [0, T]}$ and a finite-variation process $(A_t)_{t \\in [0, T]} $. The signal $(A_t)_{t \\in [0, T]}$ is defined by:\n", " \\begin{align*}\n", " A_t = \\int_0^t I_s ds\n", " \\end{align*}\n", " with $(I_t)_{t \\in [0, T]}$ a Markovian signal.\n", "\n", "\n", " \n", "# Stochastic control problem \n", " \n", " \n", "The trading cost, affected by the temporary and transient impact is defined by: \n", " \\begin{align*}\n", " -\\int_0^T S_t dQ^u_t = \\int_0^T \\left(\\left(P_t -\\kappa Z^u_t \\right)u_t -\\lambda u_t^2\\right)dt\n", " \\end{align*}\n", " \n", "The trader's terminal wealth is defined as the price of his terminal inventory deducted by the trading costs:\n", " \\begin{align*}\n", " Q_T^u P_T -\\int_0^T S_t dQ_t^u\n", " \\end{align*}\n", "As we have set terminal constraint on the inventory, we want the terminal inventory to be as close to zero as possible. Thus we introduce a penalty on the terminal inventory given by:\n", "$$ -\\varrho (Q_T^u)^2$$\n", "\n", "We assume that the trader's objective is to liquidate his portfolio by minimizing the trading costs and maximizing the following performance functional:\n", "\\begin{align*}\n", " J(u)= \\mathbb E\\left[ \\int_0^T \\left(\\left(P_t -\\kappa Z_t^u \\right)u_t -\\lambda u_t^2\\right)dt + Q_T^u P_T -\\varrho (Q_T^u)^2 \\right]\n", "\\end{align*}\n", "\n", "\n", "The goal of this research is to find the optimal strategy $u^*$ solution of our optimal liquidation problem such that:\n", "\\begin{align}\n", " J(u^*)=\\sup_{u \\in \\mathcal U} J(u)\n", "\\end{align}\n", "where $\\mathcal U$ is a suitable set of admissible trading strategies (controls). \n", "\n", "\n", "1. Show that the performance functional can be re-written as \n", "\\begin{align*}\n", " J(u)= \\mathbb E\\left[ \\int_0^T \\left(\\left(A_t -\\kappa Z_t^u \\right)u_t -\\lambda u_t^2\\right)dt + A_T Q_T^u -\\varrho (Q_T^u)^2 \\right] + qP_0\n", "\\end{align*}\n", "\n", "To simplify the following steps, we drop the term $A_T Q^u_T$ as, at time $T$, we assume that the trader has liquidated all his portfolio and that the signal has a value close to zero. We decide to drop the term as it is not significant.\n", "\n", "From now on, we normalize by 2 the cross term, and consider the functional: \n", "\\begin{align}\\label{eq:costinitial}\n", " J(u)= \\mathbb E\\left[ \\int_0^T 2\\left(\\left(A_t -\\kappa Z_t^u \\right)u_t -\\lambda u_t^2\\right)dt -\\varrho (Q_T^u)^2 \\right] + qP_0\n", "\\end{align}\n", "\n", "\n", "2. Show that the problem is equivalent to the following control problem in one dimension on $X$ defined by\n", "\\begin{align}\n", " X_t^{\\alpha} &= g_0(t)+\\int_0^t K(t-s)\\left(- \\frac 1 \\lambda X_s^{\\alpha} -\\alpha_s \\right)ds, \\quad g_0(t)= A_t + \\varrho q; \\\\\n", " \\end{align}\n", " with the final performance functional defined by:\n", " \\begin{align}\n", " J(\\alpha)&= \\mathbb E\\left[ \\int_0^T \\left(-\\lambda {\\alpha}_t^2 + \\frac 1 \\lambda (X_t^{\\alpha})^2 \\right)dt \\right] + qP_0 -\\varrho q^2,\n", "\\end{align}\n", " where we set that the kernel $K(t)=G(t)+ \\varrho$ and after a suitable change of variables for the control $u$ to $\\alpha$. \n", "3. Assuming that market impact is permanent, i.e. $G(t) = \\text{cst}$ and that $A_t=0$, can you solve the problem?\n", "\n", "# Bonus: More general kernels and signals \n", "4. What is the difficulty if the market impact kernel $G$ is a general function in $L^2$, think of power law decay for instance?\n", "5. Can you figue out a way to still solve the problem?\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "\n", "

References 📖

\n", "\n", "
\n", "\n", "
\n", "\n", "$\\bullet$ Yong, J., & Zhou, X. Y. (1999). Stochastic controls: Hamiltonian systems and HJB equations (Vol. 43). Springer Science & Business Media.\n", "\n", "$\\bullet $ Almgren, R., & Chriss, N. (2001). Optimal execution of portfolio transactions. Journal of Risk, 3, 5-40.\n", "\n", "$\\bullet $ Abi Jaber, E., & Neuman, E. (2022). Optimal liquidation with signals: the general propagator case. Available at SSRN." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.13" } }, "nbformat": 4, "nbformat_minor": 2 }