The Turn: Integration of Information Seeking and Retrieval in Context.

The Laboratory Model of Information Retrieval (IR) evaluation has its origins in the Cranfield II project (Cleverdon 1967). It is the paradigm of the Computer Science oriented IR research, seeking to develop ever better IR algorithms and systems. In recent years, the TREC conferences (Voorhees and Harman 2000) have been the major forum for research based on the Laboratory Model – Fig. 1.1. An essential component in evaluation based on the Model is a test collection consisting of a document database, a set of fairly well defined topical requests, and a set of (typically binary) relevance assessments identifying the documents that are topically relevant to each request. IR algorithms are evaluated for their ability of finding the relevant documents. The test results are typically expressed in terms of average recall and precision, and recallprecision curves for each algorithm. The Laboratory Model has recently been challenged by progress in research related to relevance and information seeking as well as by the growing need for accounting for interaction or human involvement in evaluation. Recent work in analyzing the concept of relevance has resulted in identifying higher order relevances, such as cognitive relevance and situational relevance, in addition to algorithmic and topical relevance (Borlund 2000b; Cosijn and Ingwersen 2000; Saracevic 1996; Schamber et al. 1990). Real human users of IR systems introduce non-binary, subjective and dynamic relevance judgments into IR processes, which affect the processes directly. Recent theoretical and empirical work in Information Seeking and Retrieval (IS&R for short; Belkin 1993; Byström and Järvelin 1995; Ellis and Haugan 1997; Ingwersen 1996; Kuhlthau 1993a; Schamber 1994; Vakkari 2001a; Wilson 1999) suggests that IR is but one means of information seeking which takes place in a context determined by, e.g., a person’s task, its phase, and situation. For larger tasks one may identify multiple stages, strategies, tactics or modes of nformation access, and dynamic relevance. IR strategies, tactics and relevance assessments are affected by the stages of task performance. Also some user-oriented research in IR, e.g., by Bates (1989; 1990), points out the variety of strategies people might use in information access, topical retrieval being only one. Because of these empirical findings and theoretical arguments, the traditional Laboratory Model of IR evaluation has been challenged for its (lack of) realism. There are proposals (Borlund 2000a) concerning how IR evaluation should be done validly under these circumstances – realistically and at the same time retaining as much control as possible. There also is empirical work (e.g., Vakkari 2001a) tracing interactive information seeking and IR processes and providing models and methods for the analysis of IS&R. Developers of IR algorithms should therefore consider how the algorithms are to be evaluated – in which frameworks and how guaranteeing validity. This book reviews the literature in IS&R, discusses the developments of the respective research areas and proposes an extended cognitive viewpoint to integrate the results and further work in these areas. We believe that research in IS&R needs new models to formulate its research programs and methodology. Our proposal is based on understanding the situational nature of information and on assuming persons' work tasks or cultural interests, and information needs based on them, as the basis for IS&R.