Entity Recognition and Disambiguation Challenge

Call for Participation

We are excited to announce the 2014 Entity Recognition and Disambiguation (ERD) Challenge! The participating teams will have the opportunity to not only win cash prizes in the total amount of US$1,500 but also be invited to publish and present their results at a SIGIR 2014 workshop in Gold Coast, Australia, co-sponsored by Google and Microsoft.

The objective of an ERD system is to recognize mentions of entities in a given text, disambiguate them, and map them to the known entities in a given collection or knowledge base. Building a good ERD system is challenging because

* Entities may appear in different surface forms.
* The context in which a surface form appears often constrains valid entity interpretations.
* An ambiguous surface form may match multiple entity interpretations, especially in short text.

The Challenge will have two tracks, with one focusing on ERD for long texts (i.e., web documents) and the other on short texts (i.e., web search queries), respectively. Each team can elect to participate either in one or both tracks.
The Challenge is open to the general public and participants are asked to build their systems as publicly accessible web services using whatever resources at their disposal. The entries to the Challenge are submitted in the form of URLs to the participants’ web services. Participants will have a period of 3 months to test run their systems using development datasets hosted by the Challenge website. The final
evaluations and the determination of winners will be performed on the held-out datasets that have similar properties of the development sets. Please visit the Challenge website listed below for more details.

Important Dates
February 5:
Challenge Website Live at http://web-ngram.research.microsoft.com/erd2014
Discussion Forum Live at https://groups.google.com/d/forum/erd-2014

March 10: Leaderboard and trial submission system online (tentative)
June 10: Trial runs end at 11:59AM PDT; Test begins at noon PDT
June 20: Team results announced
June 27: Workshop paper due
July 11: Workshop at SIGIR-2014, Gold Coast, Australia

Organizers
David Carmel, Yahoo Lab
Ming-Wei Chang, Microsoft Research
Evgeniy Gabrilovich, Google
Bo-June (Paul) Hsu, Microsoft Research
Kuansan Wang, Microsoft Research