TafBERTa:  Learning Grammatical Rules from Small-Scale Language Acquisition Data in Hebrew

Anita Gelboim; Elior Sulem

doi:10.18653/v1/2025.babylm-main.6

TafBERTa: Learning Grammatical Rules from Small-Scale Language Acquisition Data in Hebrew

Anita Gelboim
, Elior Sulem

Institute for Applied AI Research

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

We present TafBERTa, a compact RoBERTa based language model tailored for Hebrew child-directed speech (CDS). This work builds upon the BabyBERTa framework to address data scarcity and morphological complexity in Hebrew. Focusing on determiner-noun grammatical agreement phenomena, we show that TafBERTa achieves competitive performance compared to large-scale Hebrew language models while requiring significantly less data and computational resources. As part of this work, we also introduce a new corpus of Hebrew CDS, HTBerman, aligned with morphological metadata and our new grammatical evaluation benchmark for Hebrew, HeCLiMP, based on minimal pairs. Our results demonstrate the effectiveness of TafBERTa in grammaticality judgments and its potential for efficient NLP in low-resource settings.

Original language	English
Title of host publication	Proceedings of the First BabyLM Workshop
Editors	Lucas Charpentier, Leshem Choshen, Ryan Cotterell, Mustafa Omer Gul, Michael Y. Hu, Jing Liu, Jaap Jumelet, Tal Linzen, Aaron Mueller, Candace Ross, Raj Sanjay Shah, Alex Warstadt, Ethan Gotlieb Wilcox, Adina Williams
Place of Publication	Suzhou, China
Publisher	Association for Computational Linguistics
Pages	76-90
Number of pages	15
DOIs	https://doi.org/10.18653/v1/2025.babylm-main.6
State	Published - 1 Nov 2025

Access to Document

10.18653/v1/2025.babylm-main.6

Cite this

Gelboim, A., & Sulem, E. (2025). TafBERTa: Learning Grammatical Rules from Small-Scale Language Acquisition Data in Hebrew. In L. Charpentier, L. Choshen, R. Cotterell, M. O. Gul, M. Y. Hu, J. Liu, J. Jumelet, T. Linzen, A. Mueller, C. Ross, R. S. Shah, A. Warstadt, E. G. Wilcox, & A. Williams (Eds.), Proceedings of the First BabyLM Workshop (pp. 76-90). Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.babylm-main.6

Gelboim, Anita ; Sulem, Elior. / TafBERTa : Learning Grammatical Rules from Small-Scale Language Acquisition Data in Hebrew. Proceedings of the First BabyLM Workshop. editor / Lucas Charpentier ; Leshem Choshen ; Ryan Cotterell ; Mustafa Omer Gul ; Michael Y. Hu ; Jing Liu ; Jaap Jumelet ; Tal Linzen ; Aaron Mueller ; Candace Ross ; Raj Sanjay Shah ; Alex Warstadt ; Ethan Gotlieb Wilcox ; Adina Williams. Suzhou, China : Association for Computational Linguistics, 2025. pp. 76-90

@inproceedings{044c3d7507c249d9b8395cdf06397267,

title = "TafBERTa: Learning Grammatical Rules from Small-Scale Language Acquisition Data in Hebrew",

abstract = "We present TafBERTa, a compact RoBERTa based language model tailored for Hebrew child-directed speech (CDS). This work builds upon the BabyBERTa framework to address data scarcity and morphological complexity in Hebrew. Focusing on determiner-noun grammatical agreement phenomena, we show that TafBERTa achieves competitive performance compared to large-scale Hebrew language models while requiring significantly less data and computational resources. As part of this work, we also introduce a new corpus of Hebrew CDS, HTBerman, aligned with morphological metadata and our new grammatical evaluation benchmark for Hebrew, HeCLiMP, based on minimal pairs. Our results demonstrate the effectiveness of TafBERTa in grammaticality judgments and its potential for efficient NLP in low-resource settings.",

author = "Anita Gelboim and Elior Sulem",

year = "2025",

month = nov,

day = "1",

doi = "10.18653/v1/2025.babylm-main.6",

language = "English",

pages = "76--90",

editor = "Lucas Charpentier and Leshem Choshen and Ryan Cotterell and Gul, \{Mustafa Omer\} and Hu, \{Michael Y.\} and Jing Liu and Jaap Jumelet and Tal Linzen and Aaron Mueller and Candace Ross and Shah, \{Raj Sanjay\} and Alex Warstadt and Wilcox, \{Ethan Gotlieb\} and Adina Williams",

booktitle = "Proceedings of the First BabyLM Workshop",

publisher = "Association for Computational Linguistics",

}

Gelboim, A & Sulem, E 2025, TafBERTa: Learning Grammatical Rules from Small-Scale Language Acquisition Data in Hebrew. in L Charpentier, L Choshen, R Cotterell, MO Gul, MY Hu, J Liu, J Jumelet, T Linzen, A Mueller, C Ross, RS Shah, A Warstadt, EG Wilcox & A Williams (eds), Proceedings of the First BabyLM Workshop. Association for Computational Linguistics, Suzhou, China, pp. 76-90. https://doi.org/10.18653/v1/2025.babylm-main.6

TafBERTa: Learning Grammatical Rules from Small-Scale Language Acquisition Data in Hebrew. / Gelboim, Anita; Sulem, Elior.
Proceedings of the First BabyLM Workshop. ed. / Lucas Charpentier; Leshem Choshen; Ryan Cotterell; Mustafa Omer Gul; Michael Y. Hu; Jing Liu; Jaap Jumelet; Tal Linzen; Aaron Mueller; Candace Ross; Raj Sanjay Shah; Alex Warstadt; Ethan Gotlieb Wilcox; Adina Williams. Suzhou, China: Association for Computational Linguistics, 2025. p. 76-90.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - TafBERTa

T2 - Learning Grammatical Rules from Small-Scale Language Acquisition Data in Hebrew

AU - Gelboim, Anita

AU - Sulem, Elior

PY - 2025/11/1

Y1 - 2025/11/1

N2 - We present TafBERTa, a compact RoBERTa based language model tailored for Hebrew child-directed speech (CDS). This work builds upon the BabyBERTa framework to address data scarcity and morphological complexity in Hebrew. Focusing on determiner-noun grammatical agreement phenomena, we show that TafBERTa achieves competitive performance compared to large-scale Hebrew language models while requiring significantly less data and computational resources. As part of this work, we also introduce a new corpus of Hebrew CDS, HTBerman, aligned with morphological metadata and our new grammatical evaluation benchmark for Hebrew, HeCLiMP, based on minimal pairs. Our results demonstrate the effectiveness of TafBERTa in grammaticality judgments and its potential for efficient NLP in low-resource settings.

AB - We present TafBERTa, a compact RoBERTa based language model tailored for Hebrew child-directed speech (CDS). This work builds upon the BabyBERTa framework to address data scarcity and morphological complexity in Hebrew. Focusing on determiner-noun grammatical agreement phenomena, we show that TafBERTa achieves competitive performance compared to large-scale Hebrew language models while requiring significantly less data and computational resources. As part of this work, we also introduce a new corpus of Hebrew CDS, HTBerman, aligned with morphological metadata and our new grammatical evaluation benchmark for Hebrew, HeCLiMP, based on minimal pairs. Our results demonstrate the effectiveness of TafBERTa in grammaticality judgments and its potential for efficient NLP in low-resource settings.

U2 - 10.18653/v1/2025.babylm-main.6

DO - 10.18653/v1/2025.babylm-main.6

M3 - Conference contribution

SP - 76

EP - 90

BT - Proceedings of the First BabyLM Workshop

A2 - Charpentier, Lucas

A2 - Choshen, Leshem

A2 - Cotterell, Ryan

A2 - Gul, Mustafa Omer

A2 - Hu, Michael Y.

A2 - Liu, Jing

A2 - Jumelet, Jaap

A2 - Linzen, Tal

A2 - Mueller, Aaron

A2 - Ross, Candace

A2 - Shah, Raj Sanjay

A2 - Warstadt, Alex

A2 - Wilcox, Ethan Gotlieb

A2 - Williams, Adina

PB - Association for Computational Linguistics

CY - Suzhou, China

ER -

Gelboim A, Sulem E. TafBERTa: Learning Grammatical Rules from Small-Scale Language Acquisition Data in Hebrew. In Charpentier L, Choshen L, Cotterell R, Gul MO, Hu MY, Liu J, Jumelet J, Linzen T, Mueller A, Ross C, Shah RS, Warstadt A, Wilcox EG, Williams A, editors, Proceedings of the First BabyLM Workshop. Suzhou, China: Association for Computational Linguistics. 2025. p. 76-90 doi: 10.18653/v1/2025.babylm-main.6

TafBERTa: Learning Grammatical Rules from Small-Scale Language Acquisition Data in Hebrew

Abstract

Access to Document

Fingerprint

Cite this