J 2016

Steepest descent method with random step lengths

KALOUSEK, Zdeněk

Basic information

Original name

Steepest descent method with random step lengths

Authors

KALOUSEK, Zdeněk

Edition

Foundations of Computational Mathematics, 2016, 1615-3383

Other information

Language

English

Type of outcome

Article in a journal

Field of Study

General mathematics

Country of publisher

United States of America

Confidentiality degree

is not subject to a state or trade secret

References:

Marked to be transferred to RIV

Yes

Organization

Faculty of Science, Humanities and Education – Technical University of Liberec – Repository

Keywords in English

Optimization;gradient methods;stochastic process;algorithms;convergence;computational experiments
Changed: 22/4/2016 15:34, Mgr. Jiří Šmída, Ph.D.

Abstract

In the original language

: The paper studies the steepest descent method applied to the minimization of a twice continuously differentiable function. Under certain conditions, the random choice of the step length parameter, independent of the actual iteration, generates a process that is almost surely R-convergent for quadratic functions. The convergence properties of this random procedure are characterized based on the mean value function related to the distribution of the step length parameter. The distribution of the random step length, which guarantees the maximum asymptotic convergence rate independent of the detailed properties of the Hessian matrix of the minimized function, is found and its uniqueness is proved. The asymptotic convergence rate of this optimally created random procedure is equal to the convergence rate of the Chebyshev polynomials method. Under practical conditions, the efficiency of the suggested random steepest descent method is degraded by numeric noise, particularly for ill-conditioned problems; furthermore, the asymptotic convergence rate is not achieved due to the finiteness of the realized calculations. The suggested random procedure is also applied to the minimization of a general non-quadratic function. An algorithm needed to estimate relevant bounds for the Hessian matrix spectrum is created. In certain cases, the random procedure may surpass the conjugate gradient method. Interesting results are achieved when minimizing functions having a large number of local minima. Preliminary results of numerical experiments show that some modifications of the presented basic method may significantly improve its properties.