Non-Compositional Term Dependence for Information Retrieval

Christina Lioma, Jakob Grue Simonsen, Birger Larsen, Niels Dalum Hansen

Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

14 Citations (Scopus)

Abstract

We present two novel models of document coherence and their application to information retrieval (IR). Both models approximate document coherence using discourse entities, e.g. the subject or object of a sentence. Our first model views text as a Markov process generating sequences of discourse entities (entity n-grams); we use the entropy of these entity n-grams to approximate the rate at which new information appears in text, reasoning that as more new words appear, the topic increasingly drifts and text coherence decreases. Our second model extends the work of Guinaudeau & Strube [28] that represents text as a graph of discourse entities, linked by different relations, such as their distance or adjacency in text. We use several graph topology metrics to approximate different aspects of the discourse flow that can indicate coherence, such as the average clustering or betweenness of discourse entities in text. Experiments with several instantiations of these models show that: (i) our models perform on a par with two other well-known models of text coherence even without any parameter tuning, and (ii) reranking retrieval results according to their coherence scores gives notable performance gains, confirming a relation between document coherence and relevance. This work contributes two novel models of document coherence, the application of which to IR complements recent work in the integration of document cohesiveness or comprehensibility to ranking [5, 56].
Original languageEnglish
Title of host publicationSIGIR '15 Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval
Number of pages10
PublisherAssociation for Computing Machinery (ACM)
Publication date2015
Pages595-604
ISBN (Print)978-1-4503-3621-5
DOIs
Publication statusPublished - 2015
EventSIGIR '15 - Santiago de Chile, Denmark
Duration: 9 Aug 201513 Aug 2015

Conference

ConferenceSIGIR '15
Country/TerritoryDenmark
CitySantiago de Chile
Period09/08/201513/08/2015

Cite this