Enhancing navigation in biomedical databases by community voting and database-driven text classification demonstrates improvement of automatic classification of literature by harnessing community knowledge.
From the authors:
Using PepBank as a model database, we show how to build a classification-aided retrieval system that gathers training data from the community, is completely controlled by the database, scales well with concurrent change events, and can be adapted to add text classification capability to other biomedical databases.
The system can be seen at: PepBank.
You need to read the article in full to appreciate what the authors have done but a couple of quick points to notice:
1) The use of heat maps to assist users in determining the relevance of a given abstract. (Domain specific facts.)
2) The user interface allows yes/no voting on the same facts as appear in the heat map.
Voting results in reclassification of the entries.
Equally important is a user interface that enables immediate evaluation of relevance and, quick user feedback on relevance.
The user is not asked a series of questions, given complex rating choices, etc., it is yes or no. That may seem coarse but the project demonstrates with proper design, that can be very useful.