Tutorials

The controlled laboratory experiment is one of the central methods used in interactive information retrieval (IIR) to study users. Controlled laboratory experiments are useful in a number of different situations. They help us examine search behavior (e.g., How does task complexity impact search behavior?), the relevance judgment process (e.g., How does document order impact relevance assessments?), evaluate system and interface designs (e.g., Is my new interface any good? Is my query expansion technique any good?) and test and develop theory (e.g., To what extent does Information Foraging Theory explain people’s stopping behaviors in search?). However, designing an experiment with strong internal validity that allows one to demonstrate causality presents many challenges.

This full day tutorial will (1) provide general instruction about how to design laboratory IIR experiments with human participants, with an emphasis on control; (2) describe different data collection methods and procedures, with an emphasis on self-report measures and scales and techniques for establishing validity and reliability of these measures; (3) introduce the use of statistical power analysis for sample size estimation in experiments with users and (4) introduce and demonstrate two data analysis techniques, multilevel modeling and structural equation modeling, to estimate the effects of variables present in IIR experiments.

The audience for this tutorial includes students and researchers who are interested in conducting laboratory experiments with human participants. We will assume participants have had little formal experience learning about experimental design in the context of laboratory user studies, but have a basic understanding of some terminology such as variables and hypotheses. With respect to the data analysis section, we will assume participants have a basic understanding of descriptive and inferential statistics, and hypotheses testing.

Diane Kelly is a Professor at the School of Information and Library Science at the University of North Carolina at Chapel Hill. Her research and teaching interests are in interactive information search and retrieval, information search behavior, and research methods. Kelly is the recipient of a Francis Carroll McColl Term Professorship at UNC, the 2014 ASIST Research Award and the 2013 British Computer Society’s IRSG Karen Spärck Jones Award. She is the recipient of two teaching awards: the 2009 ASIST/Thomson Reuters Outstanding Information Science Teacher Award and the 2007 SILS Outstanding Teacher of the Year Award. Kelly received a Ph.D. in information science and a Graduate Certificate in cognitive science from Rutgers University and an undergraduate degree in psychology from the University of Alabama.

Anita Crescenzi is a PhD student at the School of Information and Library Science at the University of North Carolina at Chapel Hill. Her research interests are in interactive information retrieval, information-seeking and information behavior. Crescenzi also has a special interest in research methods and statistical analysis methods including multilevel modeling and structural equation modeling. Crescenzi teaches graduate-level courses in systems analysis and information behavior. Crescenzi also has eight years experience conducting applied user research including lab-based usability evaluation and three years of teaching experience. Crescenzi has a master’s degree in information science from UNC and a bachelor’s degree in education from the University of Illinois.

Succinct structures are used today in many information retrieval applications, e.g., posting lists representation, language model representation, indexing (social) graphs, query auto-completion, document retrieval and indexing dictionary of strings, just to mention the most recent ones. These new kind of data structures mimic the operations of their classical counterparts within a comparable time complexity but requiring much less space. This is achieved by carefully combining ideas from data compression and algorithmics. With the availability of several libraries for basic succinct structures – like SDSL, Succinct, Facebook’s Folly, and Sux – it is relatively easy to directly profit from advances in this field. In this tutorial we will introduce this field of research by presenting the most important succinct data structures to represent set of integers, set of points, trees, graphs and strings together with their most important applications to Information Retrieval problems. The introduction of the succinct data structures will be sustained with a practical session with programming handouts to solve. This will allow the attendees to directly experiment with implementations of these solutions on real datasets and understand the potential benefits they can bring on their own projects.The theoretical part of the tutorial requires a basic knowledge of algorithms and data structures. The practical part and its handouts require a basic knowledge of C++, even if the knowledge of any modern programming language suffice for most of the tutorial.

Tutorial page here.

Simon Gog is a researcher at Karlsruhe Institute of Technology (KIT), Germany. His main research area is the design of practical index data structures with focus on compact and succinct solutions. He received his Ph.D. in 2011 from Ulm University. In his thesis titled “Compressed Suffix Trees: Design, Construction, and Applications” he improved compressed suffix trees for applications in Bioinformatics. A practical byproduct of his thesis is the Succinct Data Structure Library (SDSL), which was further developed and adapted to Information Retrieval applications at the University of Melbourne. SDSL is available on GitHub and successfully used in many projects in Bioinformatics, Natural Language Processing and Information Retrieval. Simon is teaching graduate courses at KIT on a regular basis. In the last year he taught the courses Advanced Data Structures and Text Indexing and also gave several invited tutorials about SDSL at different venues.

Rossano Venturini is a researcher at Computer Science Department, University of Pisa. He received his Ph.D. from the Computer Science Department of the University of Pisa in 2010 discussing his thesis titled “On Searching and Extracting Strings from Compressed Textual Data”. His research interests are mainly focused on the design and the analysis of algorithms and data structures with special attention to problems of indexing and searching large textual collections. He received two Best Paper Awards at ACM SIGIR in 2014 and 2015. Rossano is teaching both undergraduate and graduate courses at the Department of Computer Science at the University of Pisa on a regular basis. In the last year, he taught a undergraduate course on Algorithms and Data Structures, a graduate course on Algorithms for Big Data, two graduate courses on introduction to Computer Science with Python, and a graduate course on Information Retrieval.

Search engines have developed into complex systems that combines hundreds of ranking criteria with the aim of producing the optimal result list in response to users’ queries. Traditionally, learning to rank algorithms are trained in batch mode, on a dataset of query and document pairs with their associated manually created relevance labels. Creation of such datasets is expensive and therefore infeasible for smaller search engines, such as small web-store search engines. It may be impossible for experts to annotate documents, as in the case of personalized search. Also, the relevance of documents to queries can change over time, like in a news search engine. Online learning to rank addresses all of these issues by incrementally learning from user feedback in real time. Online learning is closely related to active learning, incremental learning, and counterfactual learning. However, online learning is more difficult because one has to balance exploration and exploitation: actions with unknown performance have to be explored to learn better solutions. There is a growing body of established methods for online learning to rank for information retrieval and the time is right to organize and present this material to a broad audience of interested information retrieval researchers. Prerequisite knowledge: Basic familiarity with machine learning. Basic knowledge in statistics and probability theory is also required. Understanding basics of statistical estimation. Basic programming skills to follow examples.

Artem Grotov is a PhD candidate at the Informatics Institute of the University of Amsterdam. He works on online Learning to Rank as well as other topics that deal with interpreting data obtained from user interactions with interactive systems. His work on click models and evaluating search engines based on logged user interaction have been published at SIGIR 2015 and CLEF 2015. Grotov has helped to teach multiple courses on Information Retrieval and has been a thesis supervisor for Bachelor and Master students working on algorithmic information retrieval topics.

Maarten de Rijke is a Professor of Computer Science at the Informatics Institute of the University of Amsterdam. Together with a team of PhD students and postdocs he works on problems on semantic search and on- and offline learning to rank for information retrieval. Some of their recent work on on- and offline learning to rank has been (or will be) published at ICML 2014, WSDM 2014, CIKM 2014, WSDM 2015, SIGIR 2015, NIPS 2015, WSDM 2016, ECIR 2016, WWW 2016. De Rijke has taught extensively at all levels, from public lectures on search engine technology to advanced tutorials aimed at PhD students and researchers. Recent tutorials include SIGIR 2015 and WSDM 2016.

Recent years have observed a significant progress in information retrieval and natural language processing with deep learning technologies successfully applied into almost all of their major problems. The key to the success of deep learning is its capability of accurately learning distributed representations (vector representations) of natural language expressions such as sentences, and effectively utilizing the representations in various tasks. This tutorial aims at summarizing and introducing the recent results of research on deep learning for information retrieval, in order to stimulate and foster more significant research and development work on the topic in the future.

The tutorial consists of three parts. In the first part, we introduce the fundamental techniques of deep learning for natural language processing and information retrieval, such as word embedding, recurrent neural networks, and convolutional neural networks. In the second part, we explain how deep learning, particularly representation learning techniques, can be utilized in fundamental NLP and IR problems, including classification, structured prediction, matching, and translation. In the third part, we describe how deep learning can be used in specific application tasks in details. The tasks are search, question answering (from either documents or knowledge base), and image retrieval.

The tutorial is set at an intermediate level. It is assumed that the attendees have certain knowledge on machine learning and information retrieval. It is not a requirement, however, that the attendees know much about deep learning.

Hang Li is director of the Noah’s Ark Lab of Huawei Technologies, adjunct professors of Peking University and Nanjing University. He is ACM Distinguished Scientist. His research areas include information retrieval, natural language processing, statistical machine learning, and data mining. Hang graduated from Kyoto University in 1988 and earned his PhD from the University of Tokyo in 1998. He worked at the NEC lab as researcher during 1991 and 2001, and Microsoft Research Asia as senior researcher and research manager during 2001 and 2012. He joined Huawei Technologies in 2012. Hang has published three technical books, and more than 100 technical papers at top international conferences including SIGIR, WWW, WSDM, ACL, EMNLP, ICML, NIPS, SIGKDD and top international journals including CL, NLE, JMLR, TOIS, IRJ, IPM, TKDE, TWEB, TIST.

Zhengdong Lu is a senior researcher at Noah’s Ark Lab, Huawei Technologies. His research interests are neural network-based methods for natural language processing, including dialogue, machine translation, semantic parsing, and reasoning. Previously he was an associate researcher at Microsoft Research Asia and a postdoctoral researcher at University of Texas at Austin, after receiving his Ph.D. degree from Oregon Health and Science University in 2008 in computer science. He has published over 30 papers in prestigious journals and conferences, including NIPS, ICML, ACL, KDD, IJCAI and AAAI, including over 10 recent papers on deep learning methods for NLP and AI.

Recent years have witnessed a proliferation of large-scale knowledge graphs, such as Freebase, YAGO, Google’s Knowledge Graph, and Microsoft’s Satori. Whereas there is a large body of research on mining homogeneous graphs, this new generation of information networks are highly heterogeneous, with thousands of entity and relation types and billions of instances of vertices and edges. In this tutorial, we will present the state of the art in constructing, mining, and growing knowledge graphs. The purpose of the tutorial is to equip newcomers to this exciting field with an understanding of the basic concepts, tools and methodologies, available datasets, and open research challenges. A publicly available knowledge base (Freebase) will be used throughout the tutorial to exemplify the different techniques. An undergraduate degree in CS or statistics should be sufficient to understand most of the content in the tutorial.

Evgeniy Gabrilovich is a senior staff research scientist at Google, where he works on improving healthcare. Prior to joining Google in 2012, he was a director of research and head of the natural language processing and information retrieval group at Yahoo! Research. Evgeniy is an ACM Distinguished Scientist, and is a recipient of the 2014 IJCAI-JAIR Best Paper Prize. He is also a recipient of the 2010 Karen Sparck Jones Award for his contributions to natural language processing and information retrieval. He earned his PhD in computer science from the Technion — Israel Institute of Technology.

Nicolas Usunier is a research scientist at Facebook AI Research. Before joining Facebook in 2015, he was associate professor at Universite de Technologie de Compiegne (UTC, France) with a Higher-education chair from the CNRS, the French National Research Center. Prior to his position at UTC, he was associate professor at Universite Pierre et Marie Curie in Paris, from where he received his PhD in computer science in 2006. His main areas of research are large scale learning of embeddings and learning to rank, in particular with applications to knowledge bases. He served as an area chair for NIPS 2014 and is a reviewer for the major conference in machine learning and journals such as NIPS, ICML, JMLR and the Machine Learning journal.

This tutorial will provide a comprehensive overview of temporal IR approaches, essentially regarding processing dynamic content, temporal information extraction, temporal query analysis, and time-aware retrieval and ranking. We will explain the general and wide aspects associated to temporal dynamics by focusing on the web domain, from content and structural changes to variations of user behavior and interactions. We will begin with the pre-processing step of a temporal document collection, such as, dynamic crawling and temporal in order to support temporal queries. Later, we will discuss particular aspects of temporal information extraction, for instance, identifying and extracting of temporal information useful for leveraging in the retrieval.In the latter session, we will describe research issues centered on determining the temporal intent of queries, and time-aware query enhancement, namely, temporal relevance feedback, and time-aware query reformulation. Next, we will explain current approaches to time-aware retrieval and ranking, which can be classified into different types based on two main notions of relevance with respect to time, namely, recency-based ranking, and time-dependent ranking. In addition, we present applications in related research areas, e.g., exploration, summarization, clustering of search results, future event retrieval and prediction. Finally, we conclude our tutorial and outline future directions. This tutorial targets graduate students, junior researchers and practitioners in the field of information retrieval. Our prospective participants should have a basic knowledge of search processes, essentially regarding web crawling, document indexing, query analysis, and retrieval and ranking. A prerequisite skill about other research areas, e.g., natural language processing, is required but not mandatory.

Nattiya Kanhabua is an assistant professor at the Department of Computer Science, Aalborg University, Denmark, with several years of research experience in information retrieval, data mining, machine learning, statistical and predictive analytics, temporal analysis. Her research interests are information retrieval, Web and social media mining, and web archiving. She worked in several EU-funded projects, e.g., (1) ForgetIT – Concise Preservation by Combining Managed Forgetting and Contextualized Remembering, (2) ALEXANDRIA, an ERC Advanced Grant Project on Foundations for Temporal Retrieval, Exploration and Analytics in Web Archives, and (3) Medical Ecosystem: Personalized Event-based Surveillance. She has published her research work in top-tier conferences, such as, SIGIR, WSDM, CIKM, JCDL and ECIR.

Avishek Anand is a postdoctoral researcher at the L3S Research Center in Hannover, Germany. His research interests lie in the intersection of retrieval, mining, and data management aspects of temporal Web collections like Web archives, Wikipedia and news collections. He did his PhD at the Department of Databases and Information Systems, Max Planck Institute for Informatics, Saarbruecken where he worked on indexing and query processing approaches for supporting temporal text workloads. Currently, he is working on retrieval models for historical intents, tag-based search over Archives and mining methods for enriching Wikipedia using news collections. He has published his research in several top-tier conferences, such as, SIGIR, WSDM, CIKM, ICDE and EDBT.

Traditional IR techniques, systems, and methods that assume an individual searcher are often shown to be inadequate for addressing search problems that are multi-faceted and/or too complex or difficult for individuals. The next big leap in information seeking/retrieval could happen by considering social and collaborative aspects of search. In this half-day tutorial, this concept, along with some of the foundational works and latest developments in the field of collaborative information seeking (CIS) will be presented. Specifically, the course will introduce the student to theories, methodologies, and tools that focus on information retrieval/seeking in collaboration. The student will have an opportunity to learn about the social aspect of IR with a focus on collaborative search or CIS situations, systems, and evaluation techniques. The three hours will be divided as: (1) introduction to group-based IR models, approaches, and systems; (2) back-end of CIS systems with system-focused mediation and front-end with user-focused mediation; and (3) evaluation of CIS systems/approaches, prediction and recommendations with collaborative aspects of IR, and future directions. The attendees will be given a course-pack that will include a reference list, an annotated bibliography of seminal works in the field, and depictions of relevant models/frameworks.

Tutorial page here.

Chirag Shah is an assistant professor in both the School of Communication & Information (SC&I) and the Department of Computer Science at Rutgers University, USA. Shah received a PhD in Information Science from the University of North Carolina (UNC) at Chapel Hill. He holds an MTech, Computer Science & Engineering from IIT Madras, India and an MS, Computer Science from UMass Amherst. His research interests include various aspects of interactive information retrieval/seeking, especially in the context of online social networks and collaborations, resulting in two streams of works – Collaborative Information Seeking (CIS) and Social Information Seeking (SIS). This research is supported by grants from NSF, IMLS, Google, and Yahoo! At Rutgers, he directs the InfoSeeking Lab. He has published two books on CIS – one as a solo author (2012) and the other as a co-editor (2015). He was a guest editor for the IEEE Computer Special Issue on CIS published in March 2014.

How many clicks will a new ad-placement system get? Will a different news-ranking algorithm increase the dwell times of my users? What ranking function will minimize abandonment? Answering such evaluation and learning questions based on data recorded in the past is the topic of this tutorial, providing an overview of the rapidly growing body of research on offline evaluation and learning using online metrics. Unlike deploying new systems in weeks-long A/B tests to get unbiased performance estimates, counterfactual estimators can provide unbiased estimates from existing logs without delay. Analogously, counterfactual learning techniques can use existing log files for learning improved systems, without any need for online learning methods that require interactive control. A confluence of developments in machine learning, causal inference, economics, and information retrieval has recently pushed these methods to be well-founded and practical. This tutorial will provide a unifying view that will enable newcomers to enter the field, enabling broader research and further adoption of these methods in practice.

Prerequisites:
– This tutorial is aimed at an audience with intermediate experience with information retrieval.
– Familiarity with standard IR methods, applications and evaluation metrics is assumed and only briefly reviewed.
– Basic understanding of probability theory and introductory statistics is sufficient for understanding most of the tutorial.
– Some topics require basic understanding of machine learning.
– All code samples demonstrating counterfactual analysis for IR will be in Python3.
– Participants who wish to run these demos locally must bring a device capable of running Python3 scripts.

Thorsten Joachims is a Professor in the Department of Computer Science and in the Department of Information Science at Cornell University. His research interests center on a synthesis of theory and system building in machine learning, with applications in information retrieval and recommendation. His past research focused on learning to rank, learning with preferences, learning from implicit feedback, text classification, and structured output prediction. He is an ACM Fellow, AAAI Fellow and Humboldt Fellow.

Adith Swaminathan is a PhD candidate in the Department of Computer Science at Cornell University, advised by Prof. Thorsten Joachims. His research interests are at the core of this tutorial, focusing on principles and algorithms for off-policy evaluation and learning for retrieval and recommendation systems. He received a BTech degree in Computer Science and Engineering from IIT Bombay in 2010 and a MSc in Computer Science from Cornell University in 2014.

Search is an inherently interactive, non-deterministic and user-dependent process. This means that there are many different possible sequences of interactions which could be taken (some ending in success and others ending in failure). Simulation provides a low cost, repeatable and reproducible way to explore a large range of different possibilities. This makes simulation very appealing, but it also requires care and consideration in developing, implementing and instantiating models of user behaviour for the purposes of experimentation.

In this tutorial, we aim to provide researchers with an overview of simulation, detailing the various types of simulation, models of search behavior used to simulate interaction, along with the models of querying, stopping, selecting documents and marking documents. Through the course of the tutorial we will describe various studies and how they have used to simulation to explore different behaviours and aspects of the search process. The final section of the tutorial will be dedicated to “best practice” and how to build, ground and validate simulations. The tutorial will conclude with a demonstration of an open source simulation framework that can be used develop various kinds of simulations. An optional follow on session will be offered to participants wanting to learn more about the toolkit and how to build simulations.

Dr. Leif Azzopardi is a Senior Lecturer within the School of Computing Science at the University of Glasgow, within the Glasgow Information Retrieval Group. His research focuses on building formal models for Information Retrieval – usually drawing upon different disciplines for inspiration, such as Quantum Mechanics, Operations Research, Microeconomics, Transportation Planning and Gamification. In 2010, he co-organized an ACM SIGIR Workshop on the Simulation of Interaction, delivered a keynote on the `Assimilation of Users’ at the ACM SIGIR 2013 Workshop Modeling User Behaviour for Evaluation and gave a keynote on `Usor Economicus’ at CORIA 2015. He has also given numerous lectures and invited talks on simulation at various universities and at the Information Foraging Summer School (2010-2012). He has also given a series of tutorials on Retrievability (ECIR 2015, ACM SIGIR 2014, ACM ICTIR 2015) and on Formal Models of Information Seeking, Search and Retrieval (ACM SIGIR 2015, ACM CIKM 2015).

Developing a Question Answering (QA) system to automatically answer natural-language questions has been a long-standing research problem. In this tutorial, we aim to give the audience a coherent overview of the QA research, covering three categories. The first category, QA with knowledge base aims to answer natural language questions using real-world facts stored in an existing, large-scale database. The representative approach for this task is to develop a semantic parser (of questions). Other approaches based on text matching and information extraction will also be discussed. The second category, QA with the Web, targets answering questions using mainly from the facts extracted from general text corpora derived from the Web. In addition to the common components and techniques used in this setting, we will also introduce latest work on how to leverage and incorporate additional structured and semi-structured data to improve the performance. The third category of the QA problems that we will highlight is the non-factoid questions. Due to the broad coverage, we will briefly cover three exemplary topics: story comprehension, reasoning questions and paragraph QA. The tutorial will conclude by summarizing a whole area of research that is worthy of more detailed investigation for many years to come.

Scott Wen-tau Yih is a Senior Researcher at Microsoft Research Redmond. His research interests include natural language processing, machine learning and information retrieval. Yih received his Ph.D. in computer science at the University of Illinois at Urbana-Champaign. His work on joint inference using integer linear programming (ILP) helped the UIUC team win the CoNLL-05 shared task on semantic role labeling, and the approach has been widely adopted in the NLP community since then. After joining MSR in 2005, he has worked on email spam filtering, keyword extraction and search & ad relevance. His recent work focuses on continuous semantic representations, with applications in lexical semantics, knowledge base embedding and question answering. Yih received the best paper award from CoNLL-2011, an outstanding paper award from ACL-2015 and has served as program co-chairs (CEAS-09, CoNLL-14), area chairs and co-chairs (HLT-NAACL-12, ACL-14, EMNLP-16) and action editor (Transactions of ACL).

Hao Ma is a Researcher at Microsoft Research, Redmond, WA, USA. He obtained his Ph.D. in Computer Science at The Chinese University of Hong Kong. His research interests include Information Retrieval, Natural Language Processing, Machine Learning, Recommender Systems and Social Network Analysis. Most recently, Dr. Ma has been working on entity related research problems and applications. He designed the core learning algorithms that powered both Bing’s and Microsoft’s entity experience, including question answering, entity recommendation, attributes ranking, interpretation, exploration, carousel ranking, etc. He has published more than 40 research papers in prestigious conferences and journals, including WWW, SIGIR, WSDM, AAAI, TOIS, TKDE, TMM, TIST, etc. Some of his research work has been widely reported by popular news media, like MIT Technology Review, Search Engine Land, etc. Dr. Ma is also in the winning team that won the Microposts Entity Linking Challenge in WWW 2014.

Instant search has become a common part of the search experience in most popular search engines and social networking websites. The goal is to provide instant feedback to the user in terms of query completions (instant suggestions) or directly provide search results (instant results) as the user is typing their query. Examples of instant results include web queries like “weather new york” (which directly provides the current temperature), social network queries like searching for someone’s name on Facebook or LinkedIn (which directly provide the people matching the query). We consider instant search experience to be a combination of instant results and instant suggestions with the goal of satisfying the user’s information need as quickly as possible with minimal effort on the part of the user. The tutorial consists of two parts. The first part presents a survey of IR and NLP techniques used for instant search. The second part consists of a hands-on session where we use open source tools to put together an instant search solution using a publicly available dataset. This tutorial is aimed at both researchers interested in knowing about retrieval techniques used for instant search as well as practitioners interested in deploying an instant search system at scale. Knowledge expected from attendees: Basic programming in python.

Ganesh Venkataraman currently leads several search quality efforts at LinkedIn. His contributions at LinkedIn include, leading end to end re-architecture of job search, machine learned ranking for people search typeahead (system that allows members to search for 400MM+ users via instant results), introducing machine learned ranking towards skills search at LinkedIn (ex: searching for people skilled at ‘information retrieval’). He co-authored a paper on personalized ranking which won the best paper award at the IEEE Big Data Conference 2015. Prior to LinkedIn he was the founding engineer of a payments startup where he developed algorithms to detect/prevent eCommerce fraud. He holds a Ph.D. from Texas A&M in Electrical & Computer Engineering where he was the recipient of the Dean’s graduate merit scholarship. He has several publications with 200+ citations including one fundamental contribution to graph theory.

Viet Ha-Thuc leads machine learning efforts for improving search quality at LinkedIn. He has played a key role in designing and implementing machine learned ranking for personalized search and federation across several verticals at LinkedIn. His work on LinkedIn search has been published at conferences such as CIKM, Big Data and WWW. One of the publications received the Best Application Paper Award at 2015 IEEE Big Data. Prior to LinkedIn, he was a scientist in the Content Understanding group at Yahoo! Labs, where he developed a machine learning system for extracting relevant entities and concepts in text documents. The system was deployed to annotate every email and news article in the Yahoo! ecosystem. He received a Ph.D. in Computer Science from the University of Iowa in 2011.

Dhruv Arya currently focusses on improving job search quality at LinkedIn. His goal is to apply machine learning and data mining approaches to build talent matching algorithms that connect job seekers to the most relevant jobs. Apart from this, he has made key contributions to query understanding and rewriting, whole page optimization, as well as personalized federated search, which was presented at CIKM 2015. He received a Master’s degree in Computer Science from University of Pennsylvania in 2013.

Commercial web search engines need to process thousands of queries every second and provide responses to user queries within a few hundred milliseconds. As a consequence of these tight performance constraints, search engines construct and maintain very large compute infrastructures for crawling the Web, indexing discovered pages, and processing user queries. The scalability and efficiency of these infrastructures require careful performance optimizations in every major component of the search engine. This tutorial aims to provide a fairly comprehensive overview of the scalability and efficiency challenges in large-scale web search engines. In particular, the tutorial provides an in-depth architectural overview of a web search engine, mainly focusing on the web crawling, indexing, and query processing components. The scalability and efficiency issues encountered in the above-mentioned components are presented at four different granularities: at the level of a single computer, a cluster of computers, a single data center, and a multi-center search engine. The tutorial also points at open research problems and provides recommendations to researchers who are new to the field. Required knowledge level: Intermediate/advanced (familiarity with web search engines; some knowledge of indexing and ranking techniques).

Berkant Barla Cambazoglu (Independent Researcher) received his BS, MS, and PhD degrees, all in computer engineering, from the Computer Engineering Department of Bilkent University in 1997, 2000, and 2006, respectively. After getting his PhD degree, he worked as a postdoctoral researcher in Bilkent University for a short period of time. In 2006, he joined the Biomedical Informatics Department of the Ohio State University as a postdoctoral researcher. In 2008, he joined Yahoo Labs as a postdoctoral researcher. He got research scientist and senior research scientist positions at the same institution, in 2010 and 2012, respectively. Between 2013 and 2015, he was previously a senior manager, heading the web retrieval group in Yahoo Labs Barcelona. His main research interests are distributed information retrieval and web search efficiency. In 2010, 2011, 2014, and 2015, he co-organized the LSDS-IR workshop. He was the proceedings chair for WSDM’09 and the poster and proceedings chairs for ECIR’12. He served as an area chair in SIGIR’13 and SIGIR’14. He regularly serves in the program committees of SIGIR, WWW, and KDD conferences. He currently serves in the editorial board of IP&M. He has many papers published in prestigious journals including IEEE TPDS, JPDC, JASIST, Inf. Syst., ACM TWEB, and IP&M, as well as papers and tutorials presented at top-tier conferences, such as SIGIR, CIKM, WSDM, WWW, and KDD.

Ricardo Baeza-Yates (Universitat Pompeu Fabra, Barcelona, Spain & Universidad de Chile, Santiago, Chile) areas of expertise are information retrieval, web search and data mining, data science and algorithms. He was VP of Research at Yahoo Labs, based in Sunnyvale, California, from August 2014 to February 2016. Before he founded and lead from 2006 to 2015 the Yahoo labs in Barcelona and Santiago de Chile. Between 2008 and 2012 he also oversaw the Haifa lab, and started the London lab in 2012. He is part time Professor at the Dept. of Information and Communication Technologies (DTIC) of the Universitat Pompeu Fabra (UPF), in Barcelona, Spain, as well as in the Dept. of Computing Science (DCC) of Universidad de Chile at Santiago. During 2005 he was an ICREA research professor at UPF. Until 2004 he was Professor and founding director of the Center for Web Research at Universidad de Chile. He obtained a Ph.D. in CS from the University of Waterloo, Canada, in 1989. Before he obtained two masters (M.Sc. CS & M.Eng. EE) and the electronics engineer degree from the University of Chile in Santiago. He is co-author of the best-seller Modern Information Retrieval textbook, published in 1999 by Addison-Wesley with a second enlarged edition in 2011, that won the ASIST 2012 Book of the Year award. He is also co-author of the 2nd edition of the Handbook of Algorithms and Data Structures, Addison-Wesley, 1991; and co-editor of Information Retrieval: Algorithms and Data Structures, Prentice-Hall, 1992, among more than 500 other publications. From 2002 to 2004 he was elected to the board of governors of the IEEE Computer Society and in 2012 he was elected for the ACM Council. He has received the Organization of American States award for young researchers in exact sciences (1993), the Graham Medal for innovation in computing given by the University of Waterloo to distinguished ex-alumni (2007), the CLEI Latin American distinction for contributions to CS in the region (2009), and the National Award of the Chilean Association of Engineers (2010), among other distinctions. In 2003 he was the first computer scientist to be elected to the Chilean Academy of Sciences and since 2010 is a founding member of the Chilean Academy of Engineering. In 2009 he was named ACM Fellow and in 2011 IEEE Fellow.