The effect of bilingual term list size on dictionary-based cross-language information retrieval

TitleThe effect of bilingual term list size on dictionary-based cross-language information retrieval
Publication TypeConference Papers
Year of Publication2003
AuthorsDemner-Fushman D, Oard D
Conference NameProceedings of the 36th Annual Hawaii International Conference on System Sciences, 2003
Date Published2003/01/06/9
ISBN Number0-7695-1874-5
Keywordsbilingual term list, Chinese language, Computer science, Control systems, Cross-language information retrieval, data mining, Dictionaries, dictionary-based information retrieval, Educational institutions, English language, Frequency, Information retrieval, language translation, named-entity translation, natural languages, Surface morphology, Terminology

Bilingual term lists are extensively used as a resource for dictionary-based cross-language information retrieval (CLIR), in which the goal is to find documents written in one natural language based on queries that are expressed in another. This paper identifies eight types of terms that affect retrieval effectiveness in CLIR applications through their coverage by general-purpose bilingual term lists, and reports results from an experimental evaluation of the coverage of 35 bilingual term lists in news retrieval application. Retrieval effectiveness was found to be strongly influenced by term list size for lists that contain between 3,000 and 30,000 unique terms per language. Supplemental techniques for named entity translation were found to be useful with even the largest lexicons. The contribution of named-entity translation was evaluated in a cross-language experiment involving English and Chinese. Smaller effects were observed from deficiencies in the coverage of domain-specific terminology when searching news stories.