When the Test of Time Award guidelines were created, eligible papers were identified as those that were published in a window of time 10 to 12 years prior to the award. This meant that in the first year this award was given, 2014, eligible papers came from the years 2002-2004.
To identify pre-2002 papers that should receive the award, a ten-person committee was formed and led by Keith van Rijsbergen. This committee used citation counts and other techniques to identify papers that should be considered for the award. Nominations were also solicited from the community. In addition, a sub-committee was formed of people active in the 1980s to identify papers from the 70s and 80s that should be considered for the award.
As a result of this process, a nomination pool of 59 papers was created. Each paper in the pool was reviewed by a team of three committee members. These teams assigned a grade to the paper and papers were scored according to grade. From this graded list, 30 papers were selected to receive a SIGIR Test of Time Award.
| Year | Title | Author(s) |
|---|---|---|
| 1980 | Probabilistic Models of Indexing and Searching | S. E. Robertson, C. J. van Rijsbergen & M. F. Porter |
| 1982 | The Unified Probabilistic Model for IR | S. E. Robertson, M. E. Maron & W. S. Cooper |
| 1983 | A Study of the Overlap among Document Representations | P. Das-Gupta & J. Katzer |
| 1985 | The Cluster Hypothesis Revisited | E. M. Voorhees |
| 1986 | A New Theoretical Framework for Information Retrieval | C. J. van Rijsbergen |
| 1987 | Automatic Phrase Indexing for Document Retrieval | J. Fagan |
| 1988 | A Look Back and a Look Forward | K. Spärck Jones |
| Information Retrieval using a Singular Value Decomposition Model of Latent Semantic Structure | G. W. Furnas, S. Deerwester, S. T. Dumais, T. K. Landauer, R. A. Harshman, L. S. Streeter & K. E. Lochbaum | |
| Towards Interactive Query Expansion | D. Harman | |
| 1989 | A Library System for Information Retrieval based on a Cognitive Task Analysis and Supported by an Icon-based Interface | A. M. Pejtersen |
| Adaptive Information Retrieval: Using a Connectionist Representation to Retrieve and Learn about Documents | R. K. Belew | |
| 1990 | Inference Networks for Document Retrieval | H. Turtle & W. B. Croft |
| 1992 | Scatter/Gather: A Cluster-based approach to Browsing Large Document Collections | D. R. Cutting, D. R. Karger, J. O. Pedersen & J. W. Tukey |
| 1994 | A Sequential Algorithm for Training Text Classifiers | D. D. Lewis & W. A. Gale |
| Some Simple Effective Approximation to the 2-Poisson Model for Probabilistic Weighted Retrieval | S. E. Robertson & S. Walker | |
| 1995 | Searching Distributed Collections with Inference Networks | J. Callan, Z. Lu & W. B. Croft |
| 1996 | Pivoted Document Length Normalization | A. Singhal, C. Buckley & M. Mitra |
| Query Expansion using Local and Global Document Analysis | J. Xu & W. B. Croft | |
| 1998 | A Language Modeling Approach to Information Retrieval | J. M. Ponte & W. B. Croft |
| Improved Algorithms for Topic Distillation in a Hyperlinked Environment | K. Bharat & M. R. Henzinger | |
| On-line New Event Detection and Tracking | J. Allan, R. Papka & V. Lavrenko | |
| The use of MMR, Diversity-based Reranking for Reordering Documents and Producing Summaries | J. Carbonell & J. Goldstein | |
| 1999 | An Algorithmic Framework for Performing Collaborative Filtering | J. L. Herlocker, J. A. Konstan, A. Borchers & J. Riedl |
| Information Retrieval as Statistical Translation | A. Berger & J. Lafferty | |
| Probabilistic Latent Semantic Indexing | T. Hofman | |
| 2000 | Evaluating Evaluation Measure Stability | C. Buckley & E. M. Voorhees |
| IR Evaluation Methods for Retrieving Highly Relevant Documents | K. Järvelin & J. Kekäläinen | |
| 2001 | A Study of Smoothing Methods for Language Models applied to Ad Hoc Information Retrieval | C. Zhai & J. Lafferty |
| Document Language Models, Query Models and Risk Minimization for Information Retrieval | J. Lafferty & C. Zhai | |
| Relevance based Language Models | V. Lavrenko & W. B. Croft |