Multilingual Text Analysis: History, Tasks, and Challenges

Text analytics (TA) is a very broad research area that deals with knowledge discovery in written text. Almost all techniques of machine learning, data mining and information retrieval are applied to TA tasks which include text categorization, summarization, question answering and many more. Among a very large variety of TA methods, multilingual techniques hold a special place. In order to be deemed as multilingual, a system or an algorithm must be able to handle texts in several languages equally well; a very good method should be able to produce good results for languages from different language families. Multilingual techniques and algorithms need to apply analysis that is not related to a linguistic structure of text in one specific language but rather relies on general statistical and mathematical properties common to many languages. In this chapter we provide an overview of the field of multilingual text analysis, starting with description of various TA tasks and the history of TA. We then survey TA challenges related to the multilingual domain.