Tag Archives: Lipschitz

Convergence of the iterates of the gradient method with constant stepsize

By psopasakis on November 6, 2016 | Leave a comment

The gradient method with constant step length is the simplest method for solving unconstrained optimisation problems involving a continuously differentiable function with Lipschitz-continuous gradient. The motivation for this post came after reading this Wikipedia article where it is stated that under certain assumptions the sequence $\{x_k\}$ converges to a local optimum, but it is no further discussion is provided. Continue reading →

Posted in: Optimisation | Tagged: convergence, convexity, Fejér, gradient method, Lipschitz, quasi-Fejér

Error bounds for second order approximation

By psopasakis on August 3, 2016 | Leave a comment

Where here we prove an approximation bound for twice continuously differentiable functions $f:\mathbb{R}^n \to \mathbb{R}$ with M-Lipschitzian Hessian, that is $\|\nabla^2 f(x) - \nabla^2 f(y)\| \leq M \|x-y\|$ for all $x,y$ . In particular, we show that for all $x,y$

$\begin{aligned} |f(y) - f(x) - \langle \nabla f(x), y-x\rangle - \frac{1}{2}\langle \nabla f^2(x)(y-x), y-x\rangle| \leq \frac{M}{6}\|x-y\|^3. \end{aligned}$

This is stated as Lemma 1.2.4 in: Y. Nesterov, Introductory Lectures on Convex Optimization – A basic course, Kluwer Ac. Publishers, 2004. Continue reading →

Posted in: Variational analysis | Tagged: Analysis, approximation, Hessian, Lipschitz, Lipschitzian, quadratic approximation

Mathematix

Tag Archives: Lipschitz

Convergence of the iterates of the gradient method with constant stepsize

Error bounds for second order approximation

Recent Posts

Archives

Categories

Mathematix

Blogs I Follow