Linearisation bounds for smooth mutlivalued functions

How far is a function f from its linearisation? Typically, one would assume that f is twice continuously differentiable and use the following second-order version of the mean value theorem:

f(y) = f(x) + \langle J f(x),y-x\rangle + \int_0^1 (1-\tau) (y-x)'\nabla^2 f(x+\tau (y-x))(y-x)\mathrm{d}\tau

This is typically used in the context of linearisation of nonlinear dynamical systems as in [Sec. 2.5.1.3, 1]. The requirement that f is twice continuously differentiable, can, however, be reduced to f being continuously differentiable with Lipschitz gradient.For a function f:\mathbb{R}^n\to\mathbb{R}^m we denote its Jacobian as J f:\mathbb{R}^n\to\mathbb{R}^{m\times n}. The inner product of two vectors is denoted as \langle x,y \rangle = x'y. The product of matrix A with a vector x will be also denoted as \langle A,x \rangle = A\cdot x.

Let f:\mathbb{R}^n\to\mathbb{R}^m be a continuously differentiable function with L-Lipschitz Jacobian, that is, for all x,y\in\mathbb{R}^n it is

\|J f(x) - J f(y)\| \leq L \|x-y\|

for some L>0. We will show that

\|f(y) - f(x) - \langle J f(x), y-x\rangle \| \leq \frac{L}{2}\|x-y\|^2.

Indeed, by the fundamental theorem of calculus

\begin{aligned} f(y) &= f(x) + \langle\int_0^1 J f(x+\tau(y-x) \mathrm{d} \tau , y-x\rangle\\ &= f(x) + \int_0^1 \langle J f(x+\tau(y-x), y-x\rangle \mathrm{d} \tau\\ &= f(x) + \int_0^1 \langle J f(x+\tau(y-x) -J f(x) + J f(x), y-x\rangle \mathrm{d} \tau\\ &= f(x) + \langle  J f(x), y-x \rangle + \int_0^1 \langle J f(x+\tau(y-x) - J f(x), y-x \rangle \mathrm{d} \tau \end{aligned}

Note that in the above the integral is meant component-wise, so it is a matrix itself.
For convenience, define x_\tau = x + \tau (y-x). Then

\begin{aligned} \|f(y)-f(x)-\langle  J f(x), y-x \rangle \| &= \left\|\int_0^1 \langle J f(x_\tau) - J f(x), y-x\rangle \mathrm{d} \tau\right\|\\ &\leq \int_0^1 \|-\langle J f(x_\tau) - J f(x), y-x\rangle \| \mathrm{d} \tau\\ &\leq \int_0^1 \|-\langle J f(x_\tau) - J f(x)\| \ \|y-x \rangle \| \mathrm{d} \tau\\ &\leq \|x-y\| \int_0^1 L\|x-y\| \tau \mathrm{d} \tau\\ &=L\|x-y\|^2. \end{aligned}

Such bounds can also be derived – for real-valued functions – when second order information is available. Read this post for details. In that case, the right-hand side involves a cubic term \|x-y\|^3.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

mathbabe

Exploring and venting about quantitative issues

Look at the corners!

The math blog of Dmitry Ostrovsky

The Unapologetic Mathematician

Mathematics for the interested outsider

Almost Sure

A random mathematical blog

Mathematix

Mathematix is the mathematician of the village of Asterix

%d bloggers like this: