Abstract
Information retrieval and recommender systems are deployed in real world environments. Therefore, to get a real feeling for the system, we should study their characteristics in “real world studies”. This raises the question: What does it mean for a study to be realistic? Does it mean the user has to be a real user of the system or can anyone participate in a study of the system? Does it mean the system needs to be perceived as realistic by the user? Does it mean the manipulations need to be perceived as realistic by the user?
| Original language | English |
|---|---|
| Title of host publication | Frontiers of Information Access Experimentation for Research and Education : Report from Dagstuhl Seminar 23031 |
| Editors | Christine Bauer, Ben Carterette, Nicola Ferro, Norbert Fuhr |
| Number of pages | 21 |
| Volume | 13 |
| Publisher | Schloss Dagstuhl- Leibniz-Zentrum fur Informatik GmbH, Dagstuhl Publishing |
| Publication date | 2023 |
| Edition | 1 |
| Pages | 20-40 |
| Chapter | 4.1 |
| Publication status | Published - 2023 |
| Event | Frontiers of Information Access Experimentation for Research and Education - Schloss Dagstuhl – Leibniz-Zentrum für Informatik GmbH, Wadern, Germany Duration: 15 Jan 2020 → 20 Jan 2023 Conference number: 23031 https://www.dagstuhl.de/en/seminars/seminar-calendar/seminar-details/23031 |
Seminar
| Seminar | Frontiers of Information Access Experimentation for Research and Education |
|---|---|
| Number | 23031 |
| Location | Schloss Dagstuhl – Leibniz-Zentrum für Informatik GmbH |
| Country/Territory | Germany |
| City | Wadern |
| Period | 15/01/2020 → 20/01/2023 |
| Internet address |
| Series | Dagstuhl Reports |
|---|---|
| Number | 1 |
| Volume | 13 |
| ISSN | 2192-5283 |
Bibliographical note
This report documents the program and the outcomes of Dagstuhl Seminar 23031 “Frontiers of Information Access Experimentation for Research and Education”, which brought together 37 participants from 12 countries.The seminar addressed technology-enhanced information access (information retrieval, recommender systems, natural language processing) and specifically focused on developing more responsible experimental practices leading to more valid results, both for research as well as for scientific education.
The seminar brought together experts from various sub-fields of information access, namely Information Retrieval (IR), Recommender Systems (RS), Natural Language Processing (NLP), information science, and human-computer interaction to create a joint understanding of the problems and challenges presented by next generation information access systems, from both the research and the experimentation point of views, to discuss existing solutions and impediments, and to propose next steps to be pursued in the area in order to improve not also our research methods and findings but also the education of the new generation of researchers and developers.
The seminar featured a series of long and short talks delivered by participants, who helped in setting a common ground and in letting emerge topics of interest to be explored as the main output of the seminar. This led to the definition of five groups which investigated challenges, opportunities, and next steps in the following areas: reality check, i.e. conducting real-world studies, human–machine-collaborative relevance judgment frameworks, overcoming methodological challenges in information retrieval and recommender systems through awareness and education, results-blind reviewing, and guidance for authors.
Keywords
- Information Retrieval
- Recommender Systems
- Natural Language Processing
- Research
- Education
