article

General Bounds and Finite-Time Improvement for the Kiefer-Wolfowitz Stochastic Approximation Algorithm

Authors:

Assaf ZeeviAuthors Info & Claims

Operations Research, Volume 59, Issue 5

Pages 1211 - 1224

https://doi.org/10.1287/opre.1110.0970

Published: 01 September 2011 Publication History

Abstract

We consider the Kiefer-Wolfowitz (KW) stochastic approximation algorithm and derive general upper bounds on its mean-squared error. The bounds are established using an elementary induction argument and phrased directly in the terms of tuning sequences of the algorithm. From this we deduce the nonnecessity of one of the main assumptions imposed on the tuning sequences by Kiefer and Wolfowitz [Kiefer, J., J. Wolfowitz. 1952. Stochastic estimation of the maximum of a regression function. Ann. Math. Statist.23(3) 462--466] and essentially all subsequent literature. The optimal choice of sequences is derived for various cases of interest, and an adaptive version of the KW algorithm, scaled-and-shifted KW (or SSKW), is proposed with the aim of improving its finite-time behavior. The key idea is to dynamically scale and shift the tuning sequences to better match them with characteristics of the unknown function and noise level, and thus improve algorithm performance. Numerical results are provided that illustrate that the proposed algorithm retains the convergence properties of the original KW algorithm while dramatically improving its performance in some cases.

References

[1]

Andrad�ttir, S., "A stochastic approximation algorithm with varying bounds," Oper. Res., v43, pp. 1037-1048, 1995.

Digital Library

[2]

Andrad�ttir, S., "A scaled stochastic approximation algorithm," Management Sci., v42, pp. 475-498, 1996.

Digital Library

[3]

Benveniste, A., M�tivier, M. and Priouret, P., Adaptive Algorithms and Stochastic Approximations, Springer-Verlag, Berlin, 1990.

Digital Library

[4]

Blum, J. R., "Approximation methods which converge with probability one," Ann. Math. Statist., v25, pp. 382-386, 1954a.

[5]

Blum, J. R., "Multidimensional stochastic approximation methods," Ann. Math. Statist., v25, pp. 737-744, 1954b.

[6]

Broadie, M., Cicek, D. M. and Zeevi, A., "Multidimensional stochastic approximation: Adaptive algorithms and applications," ACM Trans. Model. Comput. Simulation, 2011.

[7]

Burkholder, D. L., "On a class of stochastic approximation processes," Ann. Math. Statist., v27, pp. 1044-1059, 1956.

[8]

Chen, H. F., Duncan, T. E. and Pasik-Duncan, B., "A Kiefer-Wolfowitz algorithm with randomized differences," IEEE Trans. Automatic Control, v44, pp. 442-453, 1999.

[9]

Chung, K. L., "On a stochastic approximation method," Ann. Math. Statist., v25, pp. 463-483, 1954.

[10]

Derman, C., "An application of Chung's lemma to the Kiefer-Wolfowitz stochastic approximation procedure," Ann. Math. Statist., v27, pp. 532-536, 1956.

[11]

Dippon, J., "Accelerated randomized stochastic optimization," Ann. Statist., v31, pp. 1260-1281, 2003.

[12]

Dippon, J. and Renz, J., "Weighted means in stochastic approximation of minima," SIAM J. Control Optim., v35, pp. 1811-1827, 1997.

Digital Library

[13]

Dupac, V., "On the Kiefer-Wolfowitz approximation method," Časopis pro Pěstování Matematiky, v82, pp. 47-75, 1957.

[14]

Fabian, V., "Stochastic approximation of minima with improved asymptotic speed," Ann. Math. Statist., v38, pp. 191-200, 1967.

[15]

Kesten, H., "Accelerated stochastic approximation," Ann. Math. Statist., v29, pp. 41-59, 1958.

[16]

Kiefer, J. and Wolfowitz, J., "Stochastic estimation of the maximum of a regression function," Ann. Math. Statist., v23, pp. 462-466, 1952.

[17]

Kushner, H. J. and Yin, G. G., Stochastic Approximation and Recursive Algorithms and Applications, Springer-Verlag, New York, 2003.

[18]

Lai, T. L., "Stochastic approximation," Ann. Statist., v31, pp. 391-406, 2003.

[19]

Mokkadem, A. and Pelletier, M., "A companion for the Kiefer-Wolfowitz-Blum stochastic approximation algorithm," Ann. Statist., v35, pp. 1749-1772, 2007.

[20]

Nemirovski, A., Juditsky, A., Lan, G. and Shapiro, A., "Robust stochastic approximation approach to stochastic programming," SIAM J. Optim., v19, pp. 1574-1609, 2009.

Digital Library

[21]

Polyak, B. T., "New stochastic approximation type procedures," Automat. i Telemekh, v7, pp. 98-107, 1990.

[22]

Polyak, B. T. and Juditsky, A. B., "Acceleration of stochastic approximation by averaging," SIAM J. Control Optim., v30, pp. 838-855, 1992.

Digital Library

[23]

Robbins, H. and Monro, S., "A stochastic approximation method," Ann. Math. Statist., v22, pp. 400-407, 1951.

[24]

Ruppert, D., "Efficient estimators from a slowly convergent Robbins-Monro process," 1988.

[25]

Sacks, J., "Asymptotic distribution of stochastic approximation procedures," Ann. Math. Statist., v29, pp. 373-405, 1958.

[26]

Spall, J. C., "Multivariate stochastic approximation using a simultaneous perturbation gradient approximation," IEEE Trans. Automatic Control, v37, pp. 332-341, 1992.

[27]

Tsybakov, A. and Polyak, B. T., "Optimal order of accuracy of search algorithms in stochastic optimization," Problemy Peredachi Informatsii, v26, pp. 45-53, 1990.

[28]

Vaidya, R. and Bhatnagar, S., "Robust optimization of random early detection," Telecomm. Systems, v33, pp. 291-316, 2006.

Digital Library

[29]

Wasan, M. T., Stochastic Approximation, Cambridge University Press, London, 1969.

Cited By

Shashaani SEckman DSanchez S(2024)Data Farming the Parameters of Simulation-Optimization SolversACM Transactions on Modeling and Computer Simulation10.1145/368028234:4(1-29)Online publication date: 13-Aug-2024
https://dl.acm.org/doi/10.1145/3680282
Meylahn JV. den Boer A(2022)Learning to Collude in a Pricing DuopolyManufacturing & Service Operations Management10.1287/msom.2021.107424:5(2577-2594)Online publication date: 1-Sep-2022
https://dl.acm.org/doi/10.1287/msom.2021.1074
Chen BChao XShi C(2021)Nonparametric Learning Algorithms for Joint Pricing and Inventory Control with Lost Sales and Censored DemandMathematics of Operations Research10.1287/moor.2020.108446:2(726-756)Online publication date: 1-May-2021
https://dl.acm.org/doi/10.1287/moor.2020.1084
Show More Cited By

Recommendations

A Scaled Stochastic Approximation Algorithm

Consider a stochastic system of such complexity that its performance can only be evaluated by using simulation or direct experimentation. To optimize the expected performance of such systems as a function of several continuous input parameters decision ...
An adaptive multidimensional version of the Kiefer-Wolfowitz stochastic approximation algorithm
WSC '09: Winter Simulation Conference

We extend the scaled-and-shifted Kiefer-Wolfowitz (SSKW) algorithm developed by Broadie, Cicek, and Zeevi (2009) to multiple dimensions. The salient feature of this algorithm is that it makes adjustments of the tuning parameters that adapt to the ...
Stopping Rules for Optimization Algorithms Based on Stochastic Approximation

Stopping rules are developed for stochastic optimization algorithms, which minimize an unknown objective function using noise corrupted measurements. In particular, the finite-difference stochastic approximation and the simultaneous perturbation ...

Comments

Information & Contributors

Information

Published In

cover image Operations Research

Operations Research Volume 59, Issue 5

09-10 2011

258 pages

ISSN:0030-364X

Issue’s Table of Contents

Publisher

INFORMS

Linthicum, MD, United States

Publication History

Published: 01 September 2011

Accepted: 01 August 2010

Received: 01 February 2009

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Shashaani SEckman DSanchez S(2024)Data Farming the Parameters of Simulation-Optimization SolversACM Transactions on Modeling and Computer Simulation10.1145/368028234:4(1-29)Online publication date: 13-Aug-2024
https://dl.acm.org/doi/10.1145/3680282
Meylahn JV. den Boer A(2022)Learning to Collude in a Pricing DuopolyManufacturing & Service Operations Management10.1287/msom.2021.107424:5(2577-2594)Online publication date: 1-Sep-2022
https://dl.acm.org/doi/10.1287/msom.2021.1074
Chen BChao XShi C(2021)Nonparametric Learning Algorithms for Joint Pricing and Inventory Control with Lost Sales and Censored DemandMathematics of Operations Research10.1287/moor.2020.108446:2(726-756)Online publication date: 1-May-2021
https://dl.acm.org/doi/10.1287/moor.2020.1084
Chen YRyzhov I(2020)Technical Note—Consistency Analysis of Sequential Learning Under Approximate Bayesian InferenceOperations Research10.1287/opre.2019.185068:1(295-307)Online publication date: 2-Jan-2020
https://dl.acm.org/doi/10.1287/opre.2019.1850
Goeva ALam HQian HZhang B(2019)Optimization-Based Calibration of Simulation Input ModelsOperations Research10.1287/opre.2018.180167:5(1362-1382)Online publication date: 1-Sep-2019
https://dl.acm.org/doi/10.1287/opre.2018.1801
Lam HJiang GFu MJohansson BJain S(2018)On efficiencies of stochastic optimization procedures under importance samplingProceedings of the 2018 Winter Simulation Conference10.5555/3320516.3320741(1862-1873)Online publication date: 9-Dec-2018
https://dl.acm.org/doi/10.5555/3320516.3320741
Singamasetty VNair NBhashyam SPachai Kannu A(2017)Change detection with unknown post-change parameter using Kiefer-Wolfowitz method2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP.2017.7952891(3919-3923)Online publication date: 5-Mar-2017
https://dl.acm.org/doi/10.1109/ICASSP.2017.7952891
Yousefian FNedić AShanbhag U(2017)On smoothing, regularization, and averaging in stochastic approximation methods for stochastic variational inequality problemsMathematical Programming: Series A and B10.1007/s10107-017-1175-y165:1(391-431)Online publication date: 1-Sep-2017
https://dl.acm.org/doi/10.1007/s10107-017-1175-y
Wada TFujisaki Y(2016)Stopping Rules for Optimization Algorithms Based on Stochastic ApproximationJournal of Optimization Theory and Applications10.1007/s10957-015-0808-7169:2(568-586)Online publication date: 1-May-2016
https://dl.acm.org/doi/10.1007/s10957-015-0808-7
Razaviyayn MSanjabi MLuo Z(2016)A Stochastic Successive Minimization Method for Nonsmooth Nonconvex Optimization with Applications to Transceiver Design in Wireless Communication NetworksMathematical Programming: Series A and B10.1007/s10107-016-1021-7157:2(515-545)Online publication date: 1-Jun-2016
https://dl.acm.org/doi/10.1007/s10107-016-1021-7
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents