Evaluating Summaries and Answers: Two Sides of the Same Coin?

TitleEvaluating Summaries and Answers: Two Sides of the Same Coin?
Publication TypeJournal Articles
Year of Publication2005
AuthorsJimmy Lin, Demner-Fushman D
JournalIntrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization
Pagination41 - 41
Date Published2005///

This paper discusses the convergencebetween question answering and multi-
document summarization, pointing out
implications and opportunities for knowl-
edge transfer in both directions. As a
case study in one direction, we discuss
the recent development of an automatic
method for evaluating definition questions
based on n-gram overlap, a commonly-
used technique in summarization evalua-
tion. In the other direction, the move to-
wards topic-oriented summaries requires
an understanding of relevance and topi-
cality, issues which have received atten-
tion in the question answering literature.
It is our opinion that question answering
and multi-document summarization repre-
sent two complementary approaches to the
same problem of satisfying complex user
information needs. Although this points
to many exciting opportunities for system-
building, here we primarily focus on im-
plications for system evaluation.