Přehled o publikaci
2016
Steepest descent method with random step lengths
KALOUSEK, ZdeněkBasic information
Original name
Steepest descent method with random step lengths
Authors
KALOUSEK, Zdeněk
Edition
Foundations of Computational Mathematics, 2016, 1615-3383
Other information
Language
English
Type of outcome
Article in a journal
Field of Study
General mathematics
Country of publisher
United States of America
Confidentiality degree
is not subject to a state or trade secret
References:
Marked to be transferred to RIV
Yes
Organization
Faculty of Science, Humanities and Education – Technical University of Liberec – Repository
Keywords in English
Optimization;gradient methods;stochastic process;algorithms;convergence;computational experiments
Changed: 22/4/2016 15:34, Mgr. Jiří Šmída, Ph.D.
Abstract
In the original language
: The paper studies the steepest descent method applied to the minimization of a twice continuously differentiable function. Under certain conditions, the random choice of the step length parameter, independent of the actual iteration, generates a process that is almost surely R-convergent for quadratic functions. The convergence properties of this random procedure are characterized based on the mean value function related to the distribution of the step length parameter. The distribution of the random step length, which guarantees the maximum asymptotic convergence rate independent of the detailed properties of the Hessian matrix of the minimized function, is found and its uniqueness is proved. The asymptotic convergence rate of this optimally created random procedure is equal to the convergence rate of the Chebyshev polynomials method. Under practical conditions, the efficiency of the suggested random steepest descent method is degraded by numeric noise, particularly for ill-conditioned problems; furthermore, the asymptotic convergence rate is not achieved due to the finiteness of the realized calculations. The suggested random procedure is also applied to the minimization of a general non-quadratic function. An algorithm needed to estimate relevant bounds for the Hessian matrix spectrum is created. In certain cases, the random procedure may surpass the conjugate gradient method. Interesting results are achieved when minimizing functions having a large number of local minima. Preliminary results of numerical experiments show that some modifications of the presented basic method may significantly improve its properties.