System and User Centered Evaluation Approaches in Interactive Information Retrieval (SAUCE 2016)

External website Proposal in ACM proceedings

Abstract

The purpose of this workshop is to bring together academic and industry interactive information retrieval (IIR) researchers with an interest in evaluation methodologies. The workshop articulates contemporary challenges in the investigation of IIR and invites user- and system-oriented researchers to work collaboratively to address these challenges by combining user- and system-centered methodologies in meaningful ways. We anticipate that this workshop will initiate productive knowledge exchange and partnerships that can respond to the increasing user, task, system, and contextual complexity of the IIR field.

Organizers

Heather O'Brien is Assistant Professor at the iSchool, University of British Columbia in Vancouver, Canada. Her research focuses on the measurement of subjective user experience, namely the concept of user engagement. To this end, she has developed a self-report instrument, the User Engagement Scale (UES), and has been concentrating on its utility, reliability, and validity in various settings, including online search and news.

Nicola Ferro is Associate Professor at the Department of Information Engineering of the University of Padua. His research interests include information retrieval, its experimental evaluation, multilingual information access and digital libraries. He is the coordinator of the CLEF evaluation initiative, which involves more than 200 research groups world-wide in large-scale IR evaluation activities. He is the general chair of ECIR 2016, the European Conference on Information Retrieval. He has been the coordinator of PROMISE (2010-2013), the European network of excellence on IR experimental evaluation.

Hideo Joho is Associate Professor at the Research Center for Knowledge Communities, Faculty of Library, Information and Media Science, University of Tsukuba, Japan. His research interests include cognitive and affective interactions between search engines and users. He has also been involved in the development of several test collections such as GeoCLEF, NTCIR VisEx, NTCIR Temporalia, and NTCIR Lifelog. Dr. Joho was a Program Co-Chair of NTCIR-9, 10, and 11 (2010-14).

Dirk Lewandowski is Professor of Information Research and Information Retrieval at the Hamburg University of Applied Sciences, Germany. His research areas are web information retrieval, user behavior in web search and the impact of web search on knowledge acquisition in society.

Paul Thomas is Senior Research Scientist at CSIRO, Australia. His research includes evaluation techniques for information retrieval systems, especially models of user behavior and how to build offline methods that predict user performance or preference.

Keith van Rijsbergen is Professor Emeritus in the School of Computing Science, University of Glasgow and Honorary Member of the Computer Laboratory, University of Cambridge. His research spans theoretical and experimental aspects of information retrieval, including the specification and implementation of several theoretical models for IR. His current research is concerned with the design of appropriate logics to model the flow of information.