Acceleration of stochastic approximation by averaging

We consider the Kiefer-Wolfowitz (KW) stochastic approximation algorithm and derive general upper bounds on its mean-squared error. The bounds are established using an elementary induction argument and phrased directly in the terms of tuning sequences ...

Stopping rules are developed for stochastic optimization algorithms, which minimize an unknown objective function using noise corrupted measurements. In particular, the finite-difference stochastic approximation and the simultaneous perturbation ...

We study constrained nested stochastic optimization problems in which the objective function is a composition of two smooth functions whose exact values and derivatives are not available. We propose a single timescale stochastic approximation algorithm, ...