Article,

Evaluating Generative Ad Hoc Information Retrieval

, , , , , , , , , , , and .
(2023)

Abstract

Recent advances in large language models have enabled the development of viable generative retrieval systems. Instead of a traditional document ranking, generative retrieval systems often directly return a grounded generated text as a response to a query. Quantifying the utility of the textual responses is essential for appropriately evaluating such generative ad hoc retrieval. Yet, the established evaluation methodology for ranking-based ad hoc retrieval is not suited for the reliable and reproducible evaluation of generated responses. To lay a foundation for developing new evaluation methods for generative retrieval systems, we survey the r…(more)

Tags

Users

  • @scadsfct

Comments and Reviews