Non-Compositional Term Dependence for Information Retrieval

Christina Lioma, Jakob Grue Simonsen, Birger Larsen, Niels Dalum Hansen

Publikation: Bidrag til bog/antologi/rapport/konference proceedingKonferenceartikel i proceedingForskningpeer review

14 Citationer (Scopus)

Abstract

We present two novel models of document coherence and their application to information retrieval (IR). Both models approximate document coherence using discourse entities, e.g. the subject or object of a sentence. Our first model views text as a Markov process generating sequences of discourse entities (entity n-grams); we use the entropy of these entity n-grams to approximate the rate at which new information appears in text, reasoning that as more new words appear, the topic increasingly drifts and text coherence decreases. Our second model extends the work of Guinaudeau & Strube [28] that represents text as a graph of discourse entities, linked by different relations, such as their distance or adjacency in text. We use several graph topology metrics to approximate different aspects of the discourse flow that can indicate coherence, such as the average clustering or betweenness of discourse entities in text. Experiments with several instantiations of these models show that: (i) our models perform on a par with two other well-known models of text coherence even without any parameter tuning, and (ii) reranking retrieval results according to their coherence scores gives notable performance gains, confirming a relation between document coherence and relevance. This work contributes two novel models of document coherence, the application of which to IR complements recent work in the integration of document cohesiveness or comprehensibility to ranking [5, 56].
OriginalsprogEngelsk
TitelSIGIR '15 Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval
Antal sider10
ForlagAssociation for Computing Machinery
Publikationsdato2015
Sider595-604
ISBN (Trykt)978-1-4503-3621-5
DOI
StatusUdgivet - 2015
BegivenhedSIGIR '15 - Santiago de Chile, Danmark
Varighed: 9 aug. 201513 aug. 2015

Konference

KonferenceSIGIR '15
Land/OmrådeDanmark
BySantiago de Chile
Periode09/08/201513/08/2015

Emneord

  • non-compositionality,
  • information retrieval
  • term dependence

Citationsformater