Analysis of named entity recognition and linking for tweets by Leon Derczynski, et al.
Abstract:
Applying natural language processing for mining and intelligent information access to tweets (a form of microblog) is a challenging, emerging research area. Unlike carefully authored news text and other longer content, tweets pose a number of new challenges, due to their short, noisy, context-dependent, and dynamic nature. Information extraction from tweets is typically performed in a pipeline, comprising consecutive stages of language identification, tokenisation, part-of-speech tagging, named entity recognition and entity disambiguation (e.g. with respect to DBpedia). In this work, we describe a new Twitter entity disambiguation dataset, and conduct an empirical analysis of named entity recognition and disambiguation, investigating how robust a number of state-of-the-art systems are on such noisy texts, what the main sources of error are, and which problems should be further investigated to improve the state of the art.
The questions addressed by the paper are:
…
RQ1 How robust are state-of-the-art named entity recognition and linking methods on short and noisy microblog texts?RQ2 What problem areas are there in recognising named entities in microblog posts, and what are the major causes of false negatives and false positives?
RQ3 Which problems need to be solved in order to further the state-of-the-art in NER and NEL on this difficult text genre?
…
The ultimate conclusion is that entity recognition in microblog posts falls short of what has been achieved for newswire text but if you need results now or at least by tomorrow, this is a good guide to what is possible and where improvements can be made.