On robust streaming for learning with experts: algorithms and lower bounds
Article No.: 3480, Pages 79518 - 79539
Abstract
In the online learning with experts problem, an algorithm makes predictions about an outcome on each of T days, given a set of n experts who make predictions on each day. The algorithm is given feedback on the outcomes of each day, including the cost of its prediction and the cost of the expert predictions, and the goal is to make a prediction with the minimum cost, compared to the best expert in hindsight. However, often the predictions made by experts or algorithms at some time influence future outcomes, so that the input is adaptively generated.
In this paper, we study robust algorithms for the experts problem under memory constraints. We first give a randomized algorithm that is robust to adaptive inputs that uses $\widetilde{O}\left(\frac{n}{R\sqrt{T}}\right)$ space for regret R when the best expert makes $M=O\left(\frac{R^2 T}{\log^2 n}\right)$ mistakes, thereby showing a smooth space-regret trade-off. We then show a space lower bound of $\widetilde{\Omega}\left(\frac{nM}{RT}\right)$ for any randomized algorithm that achieves regret R with probability 1 - 2-Ω(T). Such an algorithm is useful for adaptive inputs, as the failure probability is low enough to union bound over all computation paths. Our result implies that the natural deterministic algorithm, which iterates through pools of experts until each expert in the pool has erred, is optimal up to polylogarithmic factors. Finally, we empirically demonstrate the benefit of using robust procedures against a white-box adversary that has access to the internal state of the algorithm.
References
[1]
Mikl�s Ajtai, Vladimir Braverman, T.S. Jayram, Sandeep Silwal, Alec Sun, David P. Woodruff, and Samson Zhou. The white-box adversarial data stream model. In Proceedings of the 41st ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems (PODS), 2022. A
[2]
Raman Arora, Teodor Vanislavov Marinov, and Mehryar Mohri. Bandits with feedback graphs and switching costs. In Advances in Neural Information Processing Systems (NeurIPS), 2019. A
[3]
Sepehr Assadi, Amit Chakrabarti, Prantar Ghosh, and Manuel Stoeckl. Coloring in graph streams via deterministic and adversarially robust algorithms. CoRR, abs/2212.10641, 2022. A
[4]
Idan Attias, Edith Cohen, Moshe Shechner, and Uri Stemmer. A framework for adversarial streaming via differential privacy and difference estimators. In 14th Innovations in Theoretical Computer Science Conference, ITCS, pages 8:1-8:19, 2023. 1.2, A
[5]
Peter Auer, Nicol� Cesa-Bianchi, Yoav Freund, and Robert E. Schapire. The nonstochastic multiarmed bandit problem. SIAM J. Comput., 32(1):48-77, 2002. A
[6]
Dmitrii Avdiukhin, Slobodan Mitrovic, Grigory Yaroslavtsev, and Samson Zhou. Adversarially robust submodular maximization under knapsack constraints. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD, pages 148-156, 2019. A
[7]
Baruch Awerbuch and Robert D. Kleinberg. Adaptive routing with end-to-end feedback: distributed learning and geometric approaches. In Proceedings of the 36th Annual ACM Symposium on Theory of Computing (STOC), 2004. A
[8]
Ziv Bar-Yossef, T. S. Jayram, Ravi Kumar, and D. Sivakumar. An information statistics approach to data stream and communication complexity. J. Comput. Syst. Sci., 68(4):702-732, 2004. 4, B.5, B.6
[9]
Raef Bassily, Kobbi Nissim, Adam D. Smith, Thomas Steinke, Uri Stemmer, and Jonathan R. Ullman. Algorithmic stability for adaptive data analysis. SIAM J. Comput., 50(3), 2021. 2.4
[10]
Amos Beimel, Haim Kaplan, Yishay Mansour, Kobbi Nissim, Thatchaphol Saranurak, and Uri Stemmer. Dynamic algorithms against an adaptive adversary: generic constructions and lower bounds. In 54th Annual ACM SIGACT Symposium on Theory of Computing (STOC), 2022. 1.2, A
[11]
Omri Ben-Eliezer, Talya Eden, and Krzysztof Onak. Adversarially robust streaming via dense-sparse trade-offs. In 5th Symposium on Simplicity in Algorithms (SOSA@SODA), 2022. A
[12]
Omri Ben-Eliezer, Rajesh Jayaram, David P. Woodruff, and Eylon Yogev. A framework for adversarially robust streaming algorithms. SIGMOD Rec., 50(1):6-13, 2021. 1.1, A
[13]
Allan Borodin and Ran El-Yaniv. Online computation and competitive analysis. Cambridge University Press, 1998. A
[14]
Mark Braverman, Ankit Garg, Tengyu Ma, Huy L Nguyen, and David P Woodruff. Communication lower bounds for statistical estimation problems via a distributed data processing inequality. In Proceedings of the Forty-eighth Annual ACM Symposium on Theory of Computing (STOC), 2016. 4
[15]
Vladimir Braverman, Avinatan Hassidim, Yossi Matias, Mariano Schain, Sandeep Silwal, and Samson Zhou. Adversarial robustness of streaming algorithms through importance sampling. In Advances in Neural Information Processing Systems (NeurIPS), 2021. A
[16]
Nicol� Cesa-Bianchi, Ofer Dekel, and Ohad Shamir. Online learning with switching costs and other adaptive adversaries. In Advances in Neural Information Processing Systems (NIPS), 2013. A
[17]
Nicol� Cesa-Bianchi and Gabor Lugosi. Prediction, learning, and games. Cambridge university press, 2006. 1.2, A
[18]
Amit Chakrabarti, Prantar Ghosh, and Manuel Stoeckl. Adversarially robust coloring for graph streams. In 13th Innovations in Theoretical Computer Science Conference (ITCS), 2022. A
[19]
Yeshwanth Cherapanamjeri and Jelani Nelson. On adaptive distance estimation. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems, NeurIPS, 2020. A
[20]
Yeshwanth Cherapanamjeri and Jelani Nelson. Uniform approximations for randomized hadamard transforms with applications. In STOC '22: 54th Annual ACM SIGACT Symposium on Theory of Computing, pages 659-671, 2022. A
[21]
Yeshwanth Cherapanamjeri, Sandeep Silwal, David P. Woodruff, Fred Zhang, Qiuyi Zhang, and Samson Zhou. Robust algorithms on adaptive inputs from bounded adversaries. In 11th International Conference on Learning Representations, ICLR, Conference Track Proceedings, 2023. A
[22]
Edith Cohen, Xin Lyu, Jelani Nelson, Tam�s Sarl�s, Moshe Shechner, and Uri Stemmer. On the robustness of countsketch to adaptive inputs. In International Conference on Machine Learning, ICML, pages 4112-4140, 2022. A
[23]
Thomas M Cover. Universal portfolios. Mathematical finance, 1(1):1-29, 1991. 1.2
[24]
Thomas M. Cover and Erik Ordentlich. Universal portfolios with side information. IEEE Trans. Inf. Theory, 42(2):348-363, 1996. 1.2
[25]
Daniela Pucci de Farias and Nimrod Megiddo. Combining expert advice in reactive environments. J. ACM, 53(5):762-799, 2006. A
[26]
Ofer Dekel, Jian Ding, Tomer Koren, and Yuval Peres. Bandits with switching costs: T2/3 regret. In Annual ACM Symposium on Theory of Computing (STOC), 2014. A
[27]
Ofer Dekel, Ambuj Tewari, and Raman Arora. Online bandit learning against an adaptive adversary: from regret to policy regret. In Proceedings of the 29th International Conference on Machine Learning (ICML), 2012. A
[28]
Itai Dinur, Uri Stemmer, David P. Woodruff, and Samson Zhou. On differential privacy and adaptive data analysis with bounded space. CoRR, abs/2302.05707, 2023. A
[29]
Cynthia Dwork, Vitaly Feldman, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Aaron Leon Roth. Preserving statistical validity in adaptive data analysis. In Proceedings of the Forty-Seventh Annual ACM on Symposium on Theory of Computing (STOC), 2015. 2.4
[30]
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam D. Smith. Calibrating noise to sensitivity in private data analysis. In Third Theory of Cryptography Conference (TCC), 2006. 2.1
[31]
Cynthia Dwork, Guy N. Rothblum, and Salil P. Vadhan. Boosting and differential privacy. In 51th Annual IEEE Symposium on Foundations of Computer Science (FOCS), 2010. 2.3
[32]
Yoav Freund and Robert E Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Cciences, 55(1):119-139, 1997. 1.2
[33]
Nick Gravin, Yuval Peres, and Balasubramanian Sivan. Tight lower bounds for multiplicative weights algorithmic families. In 44th International Colloquium on Automata, Languages, and Programming (ICALP), 2017. 1, 1
[34]
Avinatan Hassidim, Haim Kaplan, Yishay Mansour, Yossi Matias, and Uri Stemmer. Adversarially robust streaming algorithms via differential privacy. In Advances in Neural Information Processing Systems (NeurIPS), 2020. 1.2, 2.2, A
[35]
David Haussler, Jyrki Kivinen, and Manfred K Warmuth. Tight worst-case loss bounds for predicting with expert advice. In European Conference on Computational Learning Theory (EuroCOLT), 1995. 1.2
[36]
David P Helmbold and Robert E Schapire. Predicting nearly as well as the best pruning of a decision tree. Machine Learning, 27(1):51-68, 1997. 1.2
[37]
Daniel A Herrmann. Prediction with expert advice applied to the problem of prediction with expert advice. Synthese, 200(4):1-24, 2022. 1.2
[38]
Adam Tauman Kalai and Santosh S. Vempala. Efficient algorithms for online decision problems. J. Comput. Syst. Sci., 71(3):291-307, 2005. 1, 1.2
[39]
Akshay Kamath, Eric Price, and David P Woodruff. A simple proof of a new set disjointness with applications to data streams. In 36th Computational Complexity Conference (CCC), 2021. 1.1, E.4, E.5
[40]
Robert D. Kleinberg. Nearly tight bounds for the continuum-armed bandit problem. In Advances in Neural Information Processing (NIPS), 2004. A
[41]
Aryeh Kontorovich, Menachem Sadigurschi, and Uri Stemmer. Adaptive data analysis with correlated observations. In International Conference on Machine Learning, ICML, pages 11483-11498, 2022. A
[42]
Nick Littlestone and Manfred K. Warmuth. The weighted majority algorithm. Inf. Comput., 108(2):212-261, 1994. 1, 1, 1.2
[43]
Wolfgang Maass and Manfred K Warmuth. Efficient learning with virtual threshold gates. Information and Computation, 141(1):66-83, 1998. 1.2
[44]
H. Brendan McMahan and Avrim Blum. Online geometric optimization in the bandit setting against an adaptive adversary. In 17th Annual Conference on Learning Theory, (COLT), 2004. A
[45]
Neri Merhav, Erik Ordentlich, Gadiel Seroussi, and Marcelo J. Weinberger. On sequential strategies for loss functions with memory. IEEE Trans. Inf. Theory, 48(7):1947-1958, 2002. A
[46]
Binghui Peng and Aviad Rubinstein. Near optimal memory-regret tradeoff for online learning, 2023. 1.2
[47]
Binghui Peng and Fred Zhang. Online prediction in sub-linear space. In ACM-SIAM Symposium on Discrete Algorithms (SODA), 2023. 1, 1, 1.2, A
[48]
Chlo� Rouyer, Yevgeny Seldin, and Nicol� Cesa-Bianchi. An algorithm for stochastic and adversarial bandits with switching costs. In Proceedings of the 38th International Conference on Machine Learning (ICML), 2021. A
[49]
Vaidehi Srinivas, David P. Woodruff, Ziyu Xu, and Samson Zhou. Memory bounds for the experts problem. In 54th Annual ACM SIGACT Symposium on Theory of Computing (STOC), 2022. 1, 1, 1.1, 1.2, 3.2, 4, 6, D.1, D.1, D.1, D.3, D.1
[50]
Eiji Takimoto, Akira Maruoka, and Vladimir Vovk. Predicting nearly as well as the best pruning of a decision tree through dynamic programming scheme. Theor. Comput. Sci., 261(1):179-209, 2001. 1.2
[51]
Flemming Topsøe. Bounds for entropy and divergence for distributions over a two-element set. J. Ineq. Pure & Appl. Math, 2(2):300, 2001. B.1
[52]
Vladimir Vovk. Aggregating strategies. In Proceedings of the Third Annual Workshop on Computational Learning Theory (COLT), 1990. 1.2
[53]
Vladimir Vovk. A game of prediction with expert advice. J. Comput. Syst. Sci., 56(2):153—173, 1998.
[54]
Vladimir Vovk. Derandomizing stochastic prediction strategies. Machine Learning, 35(3):247-282, 1999.
[55]
Vladimir Vovk. Defensive prediction with expert advice. In 16th International Conference on Algorithmic Learning Theory (ALT), 2005. 1.2
[56]
David Wajc. Rounding dynamic matchings against an adaptive adversary. In 52nd Annual ACM SIGACT Symposium on Theory of Computing (STOC), 2020. A
[57]
David P. Woodruff and Samson Zhou. Tight bounds for adversarially robust streams and sliding windows via difference estimators. In 62nd IEEE Annual Symposium on Foundations of Computer Science (FOCS), 2021. A
Recommendations
A Framework for Adversarially Robust Streaming Algorithms
PODS'20: Proceedings of the 39th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database SystemsWe investigate the adversarial robustness of streaming algorithms. In this context, an algorithm is considered robust if its performance guarantees hold even if the stream is chosen adaptively by an adversary that observes the outputs of the algorithm ...
Integrating machine learning with knowledge acquisition through direct interaction with domain experts
Knowledge elicitation from experts and empirical machine learning are two distinct approaches to knowledge acquisition with differing and mutually complementary capabilities. Learning apprentices have provided environments in which a knowledge engineer ...
Bandits with Stochastic Experts: Constant Regret, Empirical Experts and Episodes
We study a variant of the contextual bandit problem where an agent can intervene through a set of stochastic expert policies. Given a fixed context, each expert samples actions from a fixed conditional distribution. The agent seeks to remain competitive ...
Comments
Information & Contributors
Information
Published In
Copyright � 2023 Neural Information Processing Systems Foundation, Inc.
Publisher
Curran Associates Inc.
Red Hook, NY, United States
Publication History
Published: 30 May 2024
Qualifiers
- Research-article
- Research
- Refereed limited
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 17 Oct 2024