Linearisation bounds for smooth mutlivalued functions

How far is a function $f$ from its linearisation? Typically, one would assume that $f$ is twice continuously differentiable and use the following second-order version of the mean value theorem:

$f(y) = f(x) + \langle J f(x),y-x\rangle + \int_0^1 (1-\tau) (y-x)'\nabla^2 f(x+\tau (y-x))(y-x)\mathrm{d}\tau$

This is typically used in the context of linearisation of nonlinear dynamical systems as in [Sec. 2.5.1.3, 1]. The requirement that $f$ is twice continuously differentiable, can, however, be reduced to $f$ being continuously differentiable with Lipschitz gradient.For a function $f:\mathbb{R}^n\to\mathbb{R}^m$ we denote its Jacobian as $J f:\mathbb{R}^n\to\mathbb{R}^{m\times n}$. The inner product of two vectors is denoted as $\langle x,y \rangle = x'y$. The product of matrix $A$ with a vector $x$ will be also denoted as $\langle A,x \rangle = A\cdot x$.

Let $f:\mathbb{R}^n\to\mathbb{R}^m$ be a continuously differentiable function with L-Lipschitz Jacobian, that is, for all $x,y\in\mathbb{R}^n$ it is

$\|J f(x) - J f(y)\| \leq L \|x-y\|$

for some $L>0$. We will show that

$\|f(y) - f(x) - \langle J f(x), y-x\rangle \| \leq \frac{L}{2}\|x-y\|^2.$

Indeed, by the fundamental theorem of calculus

\begin{aligned} f(y) &= f(x) + \langle\int_0^1 J f(x+\tau(y-x) \mathrm{d} \tau , y-x\rangle\\ &= f(x) + \int_0^1 \langle J f(x+\tau(y-x), y-x\rangle \mathrm{d} \tau\\ &= f(x) + \int_0^1 \langle J f(x+\tau(y-x) -J f(x) + J f(x), y-x\rangle \mathrm{d} \tau\\ &= f(x) + \langle J f(x), y-x \rangle + \int_0^1 \langle J f(x+\tau(y-x) - J f(x), y-x \rangle \mathrm{d} \tau \end{aligned}

Note that in the above the integral is meant component-wise, so it is a matrix itself.
For convenience, define $x_\tau = x + \tau (y-x)$. Then

\begin{aligned} \|f(y)-f(x)-\langle J f(x), y-x \rangle \| &= \left\|\int_0^1 \langle J f(x_\tau) - J f(x), y-x\rangle \mathrm{d} \tau\right\|\\ &\leq \int_0^1 \|-\langle J f(x_\tau) - J f(x), y-x\rangle \| \mathrm{d} \tau\\ &\leq \int_0^1 \|-\langle J f(x_\tau) - J f(x)\| \ \|y-x \rangle \| \mathrm{d} \tau\\ &\leq \|x-y\| \int_0^1 L\|x-y\| \tau \mathrm{d} \tau\\ &=L\|x-y\|^2. \end{aligned}

Such bounds can also be derived – for real-valued functions – when second order information is available. Read this post for details. In that case, the right-hand side involves a cubic term $\|x-y\|^3$.

mathbabe

Exploring and venting about quantitative issues

Look at the corners!

The math blog of Dmitry Ostrovsky

The Unapologetic Mathematician

Mathematics for the interested outsider

Almost Sure

A random mathematical blog

Mathematix

Mathematix is the mathematician of the village of Asterix