Talk:SemEval/Proposed Revision

From Wikipedia, the free encyclopedia
Jump to: navigation, search
Disciplines: Natural Language Processing
Computational Linguistics
Workshop Overview
1998 (Senseval)
Latest: Semseval 2
Summer 2010 (Ended)
ACL @ Uppsala, Sweden
Upcoming: Semseval 3
Summer 2012(tentative)
ACL @ Jeju Island, Korea
Senseval-1 1998 @ Sussex
Senseval-2 2001 @ Toulouse
Senseval-3 2004 @ Barcelona
SemEval-1 /
2007 @ Prague
SemEval-2 2010 @ Uppsala

SemEval (Semantic Evaluation) is an ongoing series of evaluations, tasks and events used to evaluate computational semantic analysis systems. The evaluations are intended to explore the nature of meaning in language. While meaning is intuitive to humans, transferring those intuitions to computational analysis has proved elusive.

SemEval evolved from the Senseval Word sense evaluation series. The first three evaluations, Senseval-1 through Senseval-3, were focused on word sense disambiguation, each time growing in the number of languages offered in the tasks and in the number of participating teams. Beginning with the fourth workshop, SemEval-2007 (SemEval-1), the nature of the tasks evolved to include semantic analysis tasks outside of word sense disambiguation.

List of Senseval and Semeval Workshops[edit]

  • Senseval-1 took place in the summer of 1998 for English, French, and Italian, culminating in a workshop held at Herstmonceux Castle, Sussex, England on September 2–4.
  • Senseval-2 took place in the summer of 2001, and was followed by a workshop held in July 2001 in Toulouse, in conjunction with ACL 2001. Senseval-2 included tasks for Basque, Chinese, Czech, Danish, Dutch, English, Estonian, Italian, Japanese, Korean, Spanish, Swedish.
  • Senseval-3 took place in March–April 2004, followed by a workshop held in July 2004 in Barcelona, in conjunction with ACL 2004. Senseval-3 included 14 different tasks for core word sense disambiguation, as well as identification of semantic roles, multilingual annotations, logic forms, subcategorization acquisition.
  • Semeval-1/Senseval-4 took place in 2007, followed by a workshop held in conjunction with ACL in Prague. Semeval-1 included 18 different tasks targeting the evaluation of systems for the semantic analysis of text. A special issue of Language Resources and Evaluation is devoted to the result[1].
  • Semeval-2 took place in 2010, followed by a workshop held in conjunction with ACL in Uppsala. Semeval-2 included 18 different tasks targeting the evaluation of semantic analysis systems.

Areas of evaluation[edit]

This list is expected to grow as the field progresses[2]. The following tables shows the areas of studies that are involved from Senseval-1 to SemEval-2:

Areas of Study Senseval-1 Senseval-2 Senseval-3 SemEval-1 SemEval-2
Keyphrase Extraction
Noun Compounds
Semantic Relations .
Semantic Role Labeling
Sentimental Analysis
Time Expression
Textual Entailment
Word Sense Disambiguation

See also[edit]


  1. ^ Language Resources and Evaluation Volume 43, Number 2
  2. ^ SemEval Portal (n.d.). In ACLwiki. Retrieved August 12, 2010 from

External links[edit]