Recall vs. Precision by Gene Golovchinsky.
From the post:
Stephen Robertson’s talk at the CIKM 2011 Industry event caused me to think about recall and precision again. Over the last decade precision-oriented searches have become synonymous with web searches, while recall has been relegated to narrow verticals. But is precision@5 or NCDG@1 really the right way to measure the effectiveness of interactive search? If you’re doing a known-item search, looking up a common factoid, etc., then perhaps it is. But for most searches, even ones that might be classified as precision-oriented ones, the searcher might wind up with several attempts to get at the answer. Dan Russell’s a Google a day lists exactly those kinds of challenges: find a fact that’s hard to find.
So how should we think about evaluating the kinds of searches that take more than one query, ones we might term session-based searches?
Read the post and the comments more than once!
Then think about how you would answer the questions raised, in or out of a topic map context.
Much food for thought here.