Relevancy Driven Development with Solr by Robin Bramley.
From the post:
The relevancy of search engine results is very subjective so therefore testing the relevancy of queries is also subjective. One technique that exists in the information retrieval field is the use of judgement lists; an alternative approach discussed here is to follow the Behaviour Driven Development methodology employing user story acceptance criteria – I’ve been calling this Relevancy Driven Development or RDD for short.
I’d like to thank Eric Pugh for a great discussion on search engine testing and for giving me a guest slot in his ‘Better Search Engine Testing‘ talk* at Lucene EuroCon Barcelona 2011 to mention RDD. The first iteration of Solr-RDD combines my passion for automated testing with my passion for Groovy by leveraging EasyB (a Groovy BDD testing framework).
The Solr-RDD GitHub site comes closer to the expectations of the project:
The aim of RDD is to allow the business users to gain confidence in the relevancy of the search query results.
…
The trick is that the business users can use a constrained data set, define a query and the results they expect in the order that they expect.
Well…, maybe. Two things of concern:
First, a user would have to “know” the data extremely well to formulate queries in that sort of detail, and
Second, it does not appear to leave any room for unexpected information that might also be useful to the user.
Perhaps this is a technique that works well with very well known data sets with few if any unexpected results.