## Abstract

We obtain the first positive results for bounded sample compression in the agnostic regression setting with the ℓ_{p} loss, where p ∈ [1, ∞]. We construct a generic approximate sample compression scheme for real-valued function classes exhibiting exponential size in the fat-shattering dimension but independent of the sample size. Notably, for linear regression, an approximate compression of size linear in the dimension is constructed. Moreover, for ℓ_{1} and ℓ_{∞} losses, we can even exhibit an efficient exact sample compression scheme of size linear in the dimension. We further show that for every other ℓ_{p} loss, p ∈ (1, ∞), there does not exist an exact agnostic compression scheme of bounded size. This refines and generalizes a negative result of David, Moran, and Yehudayoff (2016) for the ℓ_{2} loss. We close by posing general open questions: for agnostic regression with ℓ_{1} loss, does every function class admit an exact compression scheme of polynomial size in the pseudo-dimension? For the ℓ_{2} loss, does every function class admit an approximate compression scheme of polynomial size in the fat-shattering dimension? These questions generalize Warmuth's classic sample compression conjecture for realizable-case classification (Warmuth, 2003).

Original language | English |
---|---|

Pages (from-to) | 2069-2085 |

Number of pages | 17 |

Journal | Proceedings of Machine Learning Research |

Volume | 235 |

State | Published - 1 Jan 2024 |

Event | 41st International Conference on Machine Learning, ICML 2024 - Vienna, Austria Duration: 21 Jul 2024 → 27 Jul 2024 |

## ASJC Scopus subject areas

- Artificial Intelligence
- Software
- Control and Systems Engineering
- Statistics and Probability