Jump to content

User:Dengyuliang/Cross-language information retrieval

From Wikipedia, the free encyclopedia

Cross-language information retrieval (CLIR) is a subfield of information retrieval dealing with retrieving information written in a language different from the language of the user's query.[1] The term "cross-language information retrieval" has many synonyms, of which the following are perhaps the most frequent: cross-lingual information retrieval, translingual information retrieval, multilingual information retrieval. The term "multilingual information retrieval" refers more generally both to technology for retrieval of multilingual collections and to technology which has been moved to handle material in one language to another. The term Multilingual Information Retrieval (MLIR) involves the study of systems that accept queries for information in various languages and return objects (text, and other media) of various languages, translated into the user's language. Cross-language information retrieval refers more specifically to the use case where users formulate their information need in one language and the system retrieves relevant documents in another. Cross-lingual information retrieval needs to solve the following main problems:[2]

  • Correspondence between languages
  • Word ambiguity and polysemy
  • Word segmentation in question
  • Multilingualism of target document
  • How to sort the output results
A schematic diagram of CLIR

CLIR is based on different methods of querying and different processing methods, but this processing is before the IR system query. And according to the taxonomy developed by Oard & Dorr, It is divided into three Main Approaches to CLIR: [3]

  • Machine Translation-based approaches.
  • Thesaurus-based approaches.
  • Corpus-based approaches.


For the MT（machine translation) Approach, MT Approach has two steps: (1) Search query translation and (2) Target document translation. the query sentence will be automatically translated from the source language to the target language. The second step, Target document translation, will perform offline translation before searching for the target, and then search for the cached translation. The advantage of the MT Approach is that it is simple and direct and usually has a faster retrieval speed.[4] The disadvantage are that (1) it is relatively easy to make mistakes in shorter queries, especially for specific nouns, (2)Offline translation can be very time-consuming and consume a lot of storage space, (3) Inherited most of the weaknesses of the MT and MT system.

For the Thesaurus-Based Approach, Thesaurus is a resource that organizes the terminology of a domain of knowledge, in Thesaurus-Based Approach we will use a multilingual thesauri to process the CLIF, Each term in the thesaurus uniquely specifies a concept, and the target document is marked with the concept in the thesaurus and then execute the query. The query can be directly queried, or also can execute Concept Retrieval. The advantage of the Thesaurus-Based Approach is high efficiency and clear unambiguous mapping. But its disadvantages are that: expensive thesaurus, the target document needs to be marked, the scalability is insufficient, and it is limited to the specific field of the predefined thesaurus.

For the Corpus-Based Approach, it uses the statistical information in parallel corpora to query, which is usually based on two retrieval principles: target documents that frequently use the query terms are more relevant than those use query terms infrequently, and rare query terms are more useful than common query terms. The advantage of using parallel corpora is that it can include updated terminology, but it is also limited by the domains of the corpora.[5]

CLIR systems have improved so much that the most accurate multi-lingual and cross-lingual adhoc information retrieval systems today are nearly as effective as monolingual systems.[6] For languages with a small number of users, it is even more accurate than monolingual systems.[7] Other related information access tasks, such as media monitoring, information filtering and routing, sentiment analysis, and information extraction require more sophisticated models and typically more processing and analysis of the information items of interest. Much of that processing needs to be aware of the specifics of the target languages it is deployed in.

Mostly, the various mechanisms of variation in human language pose coverage challenges for information retrieval systems: texts in a collection may treat a topic of interest but use terms or expressions which do not match the expression of information need given by the user. This can be true even in a mono-lingual case, but this is especially true in cross-lingual information retrieval, where users may know the target language only to some extent. The benefits of CLIR technology for users with poor to moderate competence in the target language has been found to be greater than for those who are fluent.[8] Specific technologies in place for CLIR services include morphological analysis to handle inflection, decompounding or compound splitting to handle compound terms, and translations mechanisms to translate a query from one language to another.

The first workshop on CLIR was held in Zürich during the SIGIR-96 conference.[9] Workshops have been held yearly since 2000 at the meetings of the Cross Language Evaluation Forum (CLEF). Researchers also convene at the annual Text Retrieval Conference (TREC) to discuss their findings regarding different systems and methods of information retrieval, and the conference has served as a point of reference for the CLIR subfield.[10]

Google Search had a cross-language search feature that was removed in 2013.[11]

See also

[edit]
  • EXCLAIM (EXtensible Cross-Linguistic Automatic Information Machine)
  • CLEF (Conference and Labs of the Evaluation Forum, formerly known as Cross-Language Evaluation Forum)

References

[edit]
  1. ^ Wang, Jianqiang, and Douglas W. Oard. "Matching meaning for cross-language information retrieval." Information Processing & Management48.4 (2012): 631-53.
  2. ^ Zhu Qunyan (2015): "Research on Cross-Language Information Retrieval Based on Comparable Corpora." Web. gb.oversea.cnki.net/
  3. ^ Oard, D. W. and Dorr, B. J. (1996): "A Survey of Multilingual Text Retrieval." terpconnect.umd.edu
  4. ^ Oard, D. W. and Dorr, B. J. (1996): "Evaluating Cross-Language Text Filtering Effectiveness." Web. terpconnect.umd.edu
  5. ^ Fluhr, C. (1995): Multilingual information retrieval. In: Cole, R. A.; Mariani, J; Uszkoreit, H.; Zaenen, A. and Zue, V. (eds.): Survey of the State of the Art in Human Language Technology. pp. 391–305. Center for Spoken Language Understanding, Oregon Graduate Institute. http://www.lt-world.org/HLT_Survey/ltw-chapter8-5.pdf
  6. ^ Oard, Douglas. "Multilingual Information Access." Understanding Information Retrieval Systems(2011): 373-80. Web.
  7. ^ Dwivedi, Sanjay & Chandra, Ganesh. (2016). A Survey on Cross Language Information Retrieval. International Journal on Cybernetics & Informatics. 5. 127-142. 10.5121/ijci.2016.5113.
  8. ^ Airio, Eija (2008). "Who benefits from CLIR in web retrieval?". Journal of Documentation. 64 (5): 760–778. doi:10.1108/00220410810899754.
  9. ^ The proceedings of this workshop can be found in the book Cross-Language Information Retrieval (Grefenstette, ed; Kluwer, 1998) ISBN 0-7923-8122-X.
  10. ^ Olvera-Lobo, María-Dolores. "Cross-Language Information Retrieval on the Web." Handbook of Research on Social Dimensions of Semantic Technologies and Web Services(n.d.): 704-19. Web.
  11. ^ "Google Drops "Translated Foreign Pages" Search Option Due To Lack Of Use". 20 May 2013.
[edit]

Category:Information retrieval genres Category:Natural language processing

Stub icon

This computational linguistics-related article is a stub. You can help Wikipedia by expanding it.

Retrieved from "https://en.wikipedia.org/w/index.php?title=User:Dengyuliang/Cross-language_information_retrieval&oldid=1017231853"