Overview of the SPMRL 2013 shared task: Cross-framework evaluation of parsing morphologically rich languages

Djamé Seddah; Reut Tsarfaty; Sandra Kübler; Marie Candito; Jinho D. Choi; Richárd Farkas; Jennifer Foster; Iakes Goenaga; Koldo Gojenola; Yoav Goldberg; Spence Green; Nizar Habash; Marco Kuhlmann; Wolfgang Maier; Joakim Nivre; Adam Przepiórkowski; Ryan Roth; Wolfgang Seeker; Yannick Versley; Veronika Vincze; Marcin Woliński; Alina Wróblewska; Eric Villemonte De La Clérgerie

Overview of the SPMRL 2013 shared task: Cross-framework evaluation of parsing morphologically rich languages

Djamé Seddah, Reut Tsarfaty, Sandra Kübler, Marie Candito, Jinho D. Choi, Richárd Farkas, Jennifer Foster, Iakes Goenaga, Koldo Gojenola, Yoav Goldberg, Spence Green, Nizar Habash, Marco Kuhlmann, Wolfgang Maier, Joakim Nivre, Adam Przepiórkowski, Ryan Roth, Wolfgang Seeker, Yannick Versley, Veronika VinczeMarcin Woliński, Alina Wróblewska, Eric Villemonte De La Clérgerie

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

143 Scopus citations

Abstract

This paper reports on the first shared task on statistical parsing of morphologically rich languages (MRLs). The task features data sets from nine languages, each available both in constituency and dependency annotation. We report on the preparation of the data sets, on the proposed parsing scenarios, and on the evaluation metrics for parsing MRLs given different representation types. We present and analyze parsing results obtained by the task participants, and then provide an analysis and comparison of the parsers across languages and frameworks, reported for gold input as well as more realistic parsing scenarios.

Original language	English
Title of host publication	SPMRL 2013 - 4th Workshop on Statistical Parsing of Morphologically Rich Languages, Proceedings of the Workshop
Publisher	Association for Computational Linguistics (ACL)
Pages	146-182
Number of pages	37
ISBN (Electronic)	9781937284978
State	Published - 1 Jan 2013
Externally published	Yes
Event	4th Workshop on Statistical Parsing of Morphologically Rich Languages, SPMRL 2013 - Seattle, United States Duration: 18 Oct 2013 → …

Publication series

Name	SPMRL 2013 - 4th Workshop on Statistical Parsing of Morphologically Rich Languages, Proceedings of the Workshop

Conference

Conference	4th Workshop on Statistical Parsing of Morphologically Rich Languages, SPMRL 2013
Country/Territory	United States
City	Seattle
Period	18/10/13 → …

ASJC Scopus subject areas

Language and Linguistics
Computer Science Applications
Linguistics and Language

Cite this

Seddah, D., Tsarfaty, R., Kübler, S., Candito, M., Choi, J. D., Farkas, R., Foster, J., Goenaga, I., Gojenola, K., Goldberg, Y., Green, S., Habash, N., Kuhlmann, M., Maier, W., Nivre, J., Przepiórkowski, A., Roth, R., Seeker, W., Versley, Y., ... De La Clérgerie, E. V. (2013). Overview of the SPMRL 2013 shared task: Cross-framework evaluation of parsing morphologically rich languages. In SPMRL 2013 - 4th Workshop on Statistical Parsing of Morphologically Rich Languages, Proceedings of the Workshop (pp. 146-182). (SPMRL 2013 - 4th Workshop on Statistical Parsing of Morphologically Rich Languages, Proceedings of the Workshop). Association for Computational Linguistics (ACL).

Seddah, Djamé ; Tsarfaty, Reut ; Kübler, Sandra et al. / Overview of the SPMRL 2013 shared task : Cross-framework evaluation of parsing morphologically rich languages. SPMRL 2013 - 4th Workshop on Statistical Parsing of Morphologically Rich Languages, Proceedings of the Workshop. Association for Computational Linguistics (ACL), 2013. pp. 146-182 (SPMRL 2013 - 4th Workshop on Statistical Parsing of Morphologically Rich Languages, Proceedings of the Workshop).

@inproceedings{5aac5c9f6b084c32814ed4c1fad37506,

title = "Overview of the SPMRL 2013 shared task: Cross-framework evaluation of parsing morphologically rich languages",

abstract = "This paper reports on the first shared task on statistical parsing of morphologically rich languages (MRLs). The task features data sets from nine languages, each available both in constituency and dependency annotation. We report on the preparation of the data sets, on the proposed parsing scenarios, and on the evaluation metrics for parsing MRLs given different representation types. We present and analyze parsing results obtained by the task participants, and then provide an analysis and comparison of the parsers across languages and frameworks, reported for gold input as well as more realistic parsing scenarios.",

author = "Djam{\'e} Seddah and Reut Tsarfaty and Sandra K{\"u}bler and Marie Candito and Choi, {Jinho D.} and Rich{\'a}rd Farkas and Jennifer Foster and Iakes Goenaga and Koldo Gojenola and Yoav Goldberg and Spence Green and Nizar Habash and Marco Kuhlmann and Wolfgang Maier and Joakim Nivre and Adam Przepi{\'o}rkowski and Ryan Roth and Wolfgang Seeker and Yannick Versley and Veronika Vincze and Marcin Woli{\'n}ski and Alina Wr{\'o}blewska and {De La Cl{\'e}rgerie}, {Eric Villemonte}",

note = "Publisher Copyright: {\textcopyright} 2013 Association for Computational Linguistics.; 4th Workshop on Statistical Parsing of Morphologically Rich Languages, SPMRL 2013 ; Conference date: 18-10-2013",

year = "2013",

month = jan,

day = "1",

language = "English",

series = "SPMRL 2013 - 4th Workshop on Statistical Parsing of Morphologically Rich Languages, Proceedings of the Workshop",

publisher = "Association for Computational Linguistics (ACL)",

pages = "146--182",

booktitle = "SPMRL 2013 - 4th Workshop on Statistical Parsing of Morphologically Rich Languages, Proceedings of the Workshop",

address = "United States",

}

Seddah, D, Tsarfaty, R, Kübler, S, Candito, M, Choi, JD, Farkas, R, Foster, J, Goenaga, I, Gojenola, K, Goldberg, Y, Green, S, Habash, N, Kuhlmann, M, Maier, W, Nivre, J, Przepiórkowski, A, Roth, R, Seeker, W, Versley, Y, Vincze, V, Woliński, M, Wróblewska, A & De La Clérgerie, EV 2013, Overview of the SPMRL 2013 shared task: Cross-framework evaluation of parsing morphologically rich languages. in SPMRL 2013 - 4th Workshop on Statistical Parsing of Morphologically Rich Languages, Proceedings of the Workshop. SPMRL 2013 - 4th Workshop on Statistical Parsing of Morphologically Rich Languages, Proceedings of the Workshop, Association for Computational Linguistics (ACL), pp. 146-182, 4th Workshop on Statistical Parsing of Morphologically Rich Languages, SPMRL 2013, Seattle, United States, 18/10/13.

Overview of the SPMRL 2013 shared task: Cross-framework evaluation of parsing morphologically rich languages. / Seddah, Djamé; Tsarfaty, Reut; Kübler, Sandra et al.
SPMRL 2013 - 4th Workshop on Statistical Parsing of Morphologically Rich Languages, Proceedings of the Workshop. Association for Computational Linguistics (ACL), 2013. p. 146-182 (SPMRL 2013 - 4th Workshop on Statistical Parsing of Morphologically Rich Languages, Proceedings of the Workshop).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Overview of the SPMRL 2013 shared task

T2 - 4th Workshop on Statistical Parsing of Morphologically Rich Languages, SPMRL 2013

AU - Seddah, Djamé

AU - Tsarfaty, Reut

AU - Kübler, Sandra

AU - Candito, Marie

AU - Choi, Jinho D.

AU - Farkas, Richárd

AU - Foster, Jennifer

AU - Goenaga, Iakes

AU - Gojenola, Koldo

AU - Goldberg, Yoav

AU - Green, Spence

AU - Habash, Nizar

AU - Kuhlmann, Marco

AU - Maier, Wolfgang

AU - Nivre, Joakim

AU - Przepiórkowski, Adam

AU - Roth, Ryan

AU - Seeker, Wolfgang

AU - Versley, Yannick

AU - Vincze, Veronika

AU - Woliński, Marcin

AU - Wróblewska, Alina

AU - De La Clérgerie, Eric Villemonte

PY - 2013/1/1

Y1 - 2013/1/1

N2 - This paper reports on the first shared task on statistical parsing of morphologically rich languages (MRLs). The task features data sets from nine languages, each available both in constituency and dependency annotation. We report on the preparation of the data sets, on the proposed parsing scenarios, and on the evaluation metrics for parsing MRLs given different representation types. We present and analyze parsing results obtained by the task participants, and then provide an analysis and comparison of the parsers across languages and frameworks, reported for gold input as well as more realistic parsing scenarios.

AB - This paper reports on the first shared task on statistical parsing of morphologically rich languages (MRLs). The task features data sets from nine languages, each available both in constituency and dependency annotation. We report on the preparation of the data sets, on the proposed parsing scenarios, and on the evaluation metrics for parsing MRLs given different representation types. We present and analyze parsing results obtained by the task participants, and then provide an analysis and comparison of the parsers across languages and frameworks, reported for gold input as well as more realistic parsing scenarios.

UR - http://www.scopus.com/inward/record.url?scp=84926138689&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84926138689

T3 - SPMRL 2013 - 4th Workshop on Statistical Parsing of Morphologically Rich Languages, Proceedings of the Workshop

SP - 146

EP - 182

BT - SPMRL 2013 - 4th Workshop on Statistical Parsing of Morphologically Rich Languages, Proceedings of the Workshop

PB - Association for Computational Linguistics (ACL)

Y2 - 18 October 2013

ER -

Seddah D, Tsarfaty R, Kübler S, Candito M, Choi JD, Farkas R et al. Overview of the SPMRL 2013 shared task: Cross-framework evaluation of parsing morphologically rich languages. In SPMRL 2013 - 4th Workshop on Statistical Parsing of Morphologically Rich Languages, Proceedings of the Workshop. Association for Computational Linguistics (ACL). 2013. p. 146-182. (SPMRL 2013 - 4th Workshop on Statistical Parsing of Morphologically Rich Languages, Proceedings of the Workshop).

Overview of the SPMRL 2013 shared task: Cross-framework evaluation of parsing morphologically rich languages

Abstract

Publication series

Conference

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this