Text Retrieval Conference

From Wikipedia, the free encyclopedia

Jump to: navigation, search
For other uses of "TREC", see TREC.

The Text REtrieval Conference (TREC) is an on-going series of workshops focusing on a list of different information retrieval (IR) research areas, or tracks. It is co-sponsored by the National Institute of Standards and Technology (NIST) and the Disruptive Technology Office of the U.S. Department of Defense, and began in 1992 as part of the TIPSTER Text program. Its purpose is to support and encourage research within the information retrieval community by providing the infrastructure necessary for large-scale evaluation of text retrieval methodologies and to increase the speed of lab-to-product transfer of technology.

Each track has a challenge wherein NIST provides participating groups with data sets and test problems. Depending on track, test problems might be questions, topics, or target extractable features. Uniform scoring is performed so the systems can be fairly evaluated. After evaluation of the results, a workshop provides a place for participants to collect together thoughts and ideas and present current and future research work.

Contents

[edit] Tracks

New tracks are added as new research needs are identified, this list is current for TREC 2009.

  • Blog Track - Goal: to explore information seeking behavior in the blogosphere.
  • Chemical Track - Goal: to develop and evaluate technology for large scale search in chemistry-related documents, including academic papers and patents, to better meet the needs of professional searchers, and specifically patent searchers and chemists. New for 2009.
  • Entity Track - Goal: to perform entity-related search on Web data. These search tasks (such as finding entities and properties of entities) address common information needs that are not that well modeled as ad hoc document search. New for 2009.
  • Legal Track - Goal: to develop search technology that meets the needs of lawyers to engage in effective discovery in digital document collections.
  • Million Query Track - Goal: to test the hypothesis that a test collection built from many very incompletely judged topics is a better tool than one built using traditional TREC collection pooling.
  • Relevance Feedback Track - Goal: provide a framework for exploring the effects of different factors on the success of relevance feedback.
  • Web Track - Goal: to explore information seeking behaviors common in general web search.

Past tracks

  • Genomics Track - Goal: to study the retrieval of genomic data, not just gene sequences but also supporting documentation such as research papers, lab reports, etc. Last ran on TREC 2008.
  • Enterprise Track - Goal: to study search over the data of an organization to complete some task. Last ran on TREC 2008.
  • Cross-Language Track - Goal: to investigate the ability of retrieval systems to find documents topically regardless of source language.
  • Filtering Track - Goal: to binarily decide retrieval of new incoming documents given a stable information need.
  • HARD Track - Goal: to achieve High Accuracy Retrieval from Documents by leveraging additional information about the searcher and/or the search context.
  • Interactive Track - Goal: to study user interaction with text retrieval systems.
  • Novelty Track - Goal: to investigate systems' abilities to locate new (i.e., non-redundant) information.
  • Question Answering Track - Goal: to achieve more information retrieval than just document retrieval by answering factoid, list and definition-style questions.
  • Robust Retrieval Track - Goal: to focus on individual topic effectiveness.
  • Spam Track - Goal: to provide a standard evaluation of current and proposed spam filtering approaches.
  • Terabyte Track - Goal: to investigate whether/how the IR community can scale traditional IR test-collection-based evaluation to significantly large collections.
  • Video Track - Goal: to research in automatic segmentation, indexing, and content-based retrieval of digital video.
In 2003, this track became its own independent evaluation named TRECVID.

In 1997, a Japanese counterpart of TREC was launched, called NTCIR (NII Test Collection for IR Systems), and in 2001, a European counterpart was launched, called CLEF (Cross Language Evaluation Forum).

[edit] Conference Contributions

TREC claims that within the first six years of the workshops, the effectiveness of retrieval systems approximately doubled[1]. The conference was also the first to hold large-scale evaluations of non-English documents, speech, video and retrieval across languages. Additionally, the challenges have inspired a large body of publications. Technology first developed in TREC is now included in many of the world's commercial search engines.

[edit] Participation

The conference is made up of a varied, international group of researchers and developers. In 2003, there were 93 groups from both academia and industry from 22 countries participating.

[edit] TREC 2009 Chemistry Track

Thirteen teams[2] are participating in the TREC Chemistry Track for 2009:

[edit] TREC 2008 Legal Track

The TREC 2008 Legal Track included fifteen research teams[3], each participating in one or more out of three possible tasks: the interactive task, the relevance feedback test, and the ad-hoc task.

The four teams participating in the interactive task were:

The five teams participating in the relevance feedback task were:

The ten teams participating in the ad-hoc task were:

[edit] TREC 2008 Million Query Track

In 2008, seven teams[4] participated in the TREC Million Query Track:

[edit] References

  1. ^ From TREC homepage: "... effectiveness approximately doubled in the first six years of TREC"
  2. ^ https://wiki.ir-facility.org/index.php/Participants
  3. ^ http://trec.nist.gov/pubs/trec17/papers/LEGAL.OVERVIEW08.pdf
  4. ^ http://trec.nist.gov/pubs/trec17/appendices/million.query.results.html

[edit] External links

Retrieved from "http://en.wikipedia.org/wiki/Text_Retrieval_Conference"
Personal tools