Summary

Relevance feedback has been shown to be very effective at improving relevance of results. Its successful use requires queries for which the set of relevant documents is medium to large. Full relevance feedback is often onerous for the user, and its implementation is not very efficient in most IR systems. In many cases, other types of interactive retrieval may improve relevance by about as much with less work.

Beyond the core ad hoc retrieval scenario, other uses of relevance feedback include:

• Following a changing information need (e.g., names of car models of interest change over time)
• Maintaining an information filter (e.g., for a news feed). Such filters are discussed further in Chapter 13 .
• Active learning (deciding which examples it is most useful to know the class of to reduce annotation costs).

Exercises.

• Under what conditions would the modified query $q_m$ in Equation 49 be the same as the original query $q_0$? In all other cases, is $q_m$ closer than $q_0$ to the centroid of the relevant documents?

• Why is positive feedback likely to be more useful than negative feedback to an IR system? Why might only using one nonrelevant document be more effective than using several?

• Suppose that a user's initial query is cheap CDs cheap DVDs extremely cheap CDs. The user examines two documents, $d_1$ and $d_2$. She judges $d_1$, with the content CDs cheap software cheap CDs relevant and $d_2$ with content cheap thrills DVDs nonrelevant. Assume that we are using direct term frequency (with no scaling and no document frequency). There is no need to length-normalize vectors. Using Rocchio relevance feedback as in Equation 49 what would the revised query vector be after relevance feedback? Assume $\alpha =1,\beta=0.75,\gamma=0.25$.

• Omar has implemented a relevance feedback web search system, where he is going to do relevance feedback based only on words in the title text returned for a page (for efficiency). The user is going to rank 3 results. The first user, Jinxing, queries for:

  banana slug

  and the top three titles returned are:

  banana slug Ariolimax columbianus
  Santa Cruz mountains banana slug
  Santa Cruz Campus Mascot

  Jinxing judges the first two documents relevant, and the third nonrelevant. Assume that Omar's search engine uses term frequency but no length normalization nor IDF. Assume that he is using the Rocchio relevance feedback mechanism, with $\alpha = \beta = \gamma = 1$. Show the final revised query that would be run. (Please list the vector elements in alphabetical order.)