TY - GEN
T1 - Super-Teaching in Machine Learning
AU - Barak-Pelleg, Dina
AU - Berend, Daniel
AU - Kontorovich, Aryeh
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.
PY - 2025/1/1
Y1 - 2025/1/1
N2 - In machine learning, efficiently leveraging large datasets is essential, particularly when computational resources are limited. Sample compression techniques, which involve using carefully chosen subsets of data to train models, provide significant reductions in runtime and memory requirements while preserving generalization performance. This paper investigates a concept known as “super-teaching,” where a knowledgeable teacher selectively provides an optimal subset of training data to a learner, thereby enhancing learning outcomes. Building on the work of Ma et al. (2018), who demonstrated that a teacher with complete knowledge of the data distribution could significantly improve a learner’s performance, our study extends this idea beyond the scenario in their paper. We present a more robust and generalized approach, offering insights into how super-teaching can be effectively applied to a broader range of machine learning problems, potentially leading to better training efficiency and improved generalization.
AB - In machine learning, efficiently leveraging large datasets is essential, particularly when computational resources are limited. Sample compression techniques, which involve using carefully chosen subsets of data to train models, provide significant reductions in runtime and memory requirements while preserving generalization performance. This paper investigates a concept known as “super-teaching,” where a knowledgeable teacher selectively provides an optimal subset of training data to a learner, thereby enhancing learning outcomes. Building on the work of Ma et al. (2018), who demonstrated that a teacher with complete knowledge of the data distribution could significantly improve a learner’s performance, our study extends this idea beyond the scenario in their paper. We present a more robust and generalized approach, offering insights into how super-teaching can be effectively applied to a broader range of machine learning problems, potentially leading to better training efficiency and improved generalization.
KW - concentration
KW - Machine learning
KW - super-teacher
KW - teacher
KW - unimodal distribution
UR - http://www.scopus.com/inward/record.url?scp=85214239413&partnerID=8YFLogxK
U2 - 10.1007/978-3-031-76934-4_24
DO - 10.1007/978-3-031-76934-4_24
M3 - Conference contribution
AN - SCOPUS:85214239413
SN - 9783031769337
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 335
EP - 342
BT - Cyber Security, Cryptology, and Machine Learning - 8th International Symposium, CSCML 2024, Proceedings
A2 - Dolev, Shlomi
A2 - Elhadad, Michael
A2 - Kutyłowski, Mirosław
A2 - Persiano, Giuseppe
PB - Springer Science and Business Media Deutschland GmbH
T2 - 8th International Symposium on Cyber Security, Cryptology, and Machine Learning, CSCML 2024
Y2 - 19 December 2024 through 20 December 2024
ER -