{"id":1167,"date":"2022-11-30T01:58:41","date_gmt":"2022-11-30T01:58:41","guid":{"rendered":"https:\/\/sigir.org\/sigir2023\/?page_id=1167"},"modified":"2022-12-26T07:21:40","modified_gmt":"2022-12-26T07:21:40","slug":"call-for-reproducibility-track-papers","status":"publish","type":"page","link":"https:\/\/sigir.org\/sigir2023\/submit\/call-for-reproducibility-track-papers\/","title":{"rendered":"Call for Reproducibility Track Papers"},"content":{"rendered":"<div id='full_slider_1'  class='avia-fullwidth-slider main_color avia-shadow   avia-builder-el-0  el_before_av_section  avia-builder-el-first   container_wrap sidebar_right'  >\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-l96qjn6c-024c0a94805b0240543d019e12b25ee4\">\n#top #wrap_all .avia-slideshow .av-slideshow-caption.av-l96qjn6c-024c0a94805b0240543d019e12b25ee4__0 .avia-caption-title{\nfont-size:45px;\n}\n#top .avia-slideshow .av-slideshow-caption.av-l96qjn6c-024c0a94805b0240543d019e12b25ee4__0 .avia-caption-content{\nfont-size:25px;\n}\n#top .avia-slideshow .av-slideshow-caption.av-l96qjn6c-024c0a94805b0240543d019e12b25ee4__0 .avia-caption-content p{\nfont-size:25px;\n}\n\n@media only screen and (max-width: 479px){ \n#top #wrap_all .avia-slideshow .av-slideshow-caption.av-l96qjn6c-024c0a94805b0240543d019e12b25ee4__0 .avia-caption-title{\nfont-size:30px;\n}\n}\n<\/style>\n<div  class='avia-slideshow av-l96qjn6c-024c0a94805b0240543d019e12b25ee4 av-control-default avia-slideshow-featured av_slideshow_full avia-slide-slider av-default-height-applied   avia-slideshow-1'  data-size='featured'  data-lightbox_size='large'  data-animation='slide'  data-conditional_play=''  data-ids='834'  data-video_counter='0'  data-autoplay='false'  data-bg_slider='false'  data-slide_height=''  data-handle='av_slideshow_full'  data-interval='5'  data-class=''  data-extra_class=' '  data-el_id=''  data-css_id=''  data-scroll_down=''  data-control_layout='av-control-default'  data-custom_markup=''  data-perma_caption=''  data-autoplay_stopper=''  data-image_attachment=''  data-min_height='200px'  data-lazy_loading='disabled'  data-default-height='28.666666666667'  data-stretch=''  data-src=''  data-position='top left'  data-repeat='no-repeat'  data-attach='scroll'  data-img_scrset=''  data-av-desktop-hide=''  data-av-medium-hide=''  data-av-small-hide=''  data-av-mini-hide=''  data-id=''  data-custom_class=''  data-template_class=''  data-av_uid='av-l96qjn6c'  data-sc_version='1.0'  data-heading_tag=''  data-heading_class=''  data-min_width='698px'   itemprop=\"image\" itemscope=\"itemscope\" itemtype=\"https:\/\/schema.org\/ImageObject\" ><ul class='avia-slideshow-inner ' style='padding-bottom: 28.666666666667%;'><li  class='avia-slideshow-slide av-l96qjn6c-024c0a94805b0240543d019e12b25ee4__0  av-single-slide slide-1 ' ><div data-rel='slideshow-1' class='avia-slide-wrap '   ><div class='av-slideshow-caption av-l96qjn6c-024c0a94805b0240543d019e12b25ee4__0 caption_fullwidth caption_center'><div class=\"container caption_container\"><div class=\"slideshow_caption\"><div class=\"slideshow_inner_caption\"><div class=\"slideshow_align_caption\"><h2 class='avia-caption-title '  itemprop=\"name\" >Call for Reproducibility Track Papers<\/h2><\/div><\/div><\/div><\/div><\/div><img decoding=\"async\" class=\"wp-image-834 avia-img-lazy-loading-not-834\"  src=\"https:\/\/sigir.org\/sigir2023\/wp-content\/uploads\/2022\/10\/banner_05-1500x430.jpg\" width=\"1500\" height=\"430\" title='banner_05' alt=''  itemprop=\"thumbnailUrl\"  style='min-height:200px; min-width:698px; ' \/><div class='av-section-color-overlay' style='opacity: 0.3; background-color: #000000; '><\/div><\/div><\/li><\/ul><\/div><\/div>\n<div id='av_section_1'  class='avia-section av-5lyajc-3134daf831fbcef5bc794ceffe33e473 main_color avia-section-default avia-no-border-styling  avia-builder-el-1  el_after_av_slideshow_full  avia-builder-el-last  avia-bg-style-scroll container_wrap sidebar_right'  ><div class='container av-section-cont-open' ><div class='template-page content  av-content-small alpha units'><div class='post-entry post-entry-type-page post-entry-1167'><div class='entry-content-wrapper clearfix'>\n\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-5e722g-8fcf7aadf67cb36756a282f5a7cea639\">\n.flex_column.av-5e722g-8fcf7aadf67cb36756a282f5a7cea639{\npadding:0 40px 0 40px;\n}\n<\/style>\n<div class='flex_column av-5e722g-8fcf7aadf67cb36756a282f5a7cea639 av_one_full  avia-builder-el-2  el_before_av_one_full  avia-builder-el-first  first flex_column_div '   ><section  class='av_textblock_section av-5565b4-a218a8632e359e2159feaddf3a754aef'  itemscope=\"itemscope\" itemtype=\"https:\/\/schema.org\/CreativeWork\" ><div class='avia_textblock text_justify'  itemprop=\"text\" ><p>The annual SIGIR conference is the major international forum for the presentation of new research results and the demonstration of new systems and techniques in the broad field of information retrieval (IR). The 46th ACM SIGIR conference, will be run as a hybrid conference \u2013 in person in Taiwan, with support for remote participation, from July 23rd to 27th, 2023. We cordially invite all those working in areas related to IR to submit reproducibility papers.<\/p>\n<p>The Reproducibility Papers Track solicits papers that repeat, reproduce, generalize, and analyze prior work with a strong impact on information retrieval. The focus of this track is on generating new findings of established approaches akin to a test of time. Submitted papers should analyze to which extent assumptions of the original work held up, and elaborate error modes and unexpected conclusions.<\/p>\n<p>We are particularly interested in reproducibility papers (different team, different experimental setup) rather than replicability papers (different team, same experimental setup). The emphasis is not on reproducibility badging, but on generating new research insights with existing approaches.<\/p>\n<p>Reproducibility papers are considered equal to regular SIGIR papers. Accepted reproducibility papers will appear in the SIGIR 2023 main conference proceedings.<\/p>\n<\/div><\/section><\/div>\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-i8i34-db37dbf6681b9fa7bd58878e4937ab00\">\n.flex_column.av-i8i34-db37dbf6681b9fa7bd58878e4937ab00{\npadding:0 40px 0 40px;\n}\n<\/style>\n<div class='flex_column av-i8i34-db37dbf6681b9fa7bd58878e4937ab00 av_one_full  avia-builder-el-4  el_after_av_one_full  el_before_av_one_full  first flex_column_div column-top-margin'   ><p>\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-4vb7j4-71b3ff69c05390a61eb23cd208733c05\">\n#top .av-special-heading.av-4vb7j4-71b3ff69c05390a61eb23cd208733c05{\nmargin:0px 0px 0px 0px;\npadding-bottom:10px;\n}\nbody .av-special-heading.av-4vb7j4-71b3ff69c05390a61eb23cd208733c05 .av-special-heading-tag .heading-char{\nfont-size:25px;\n}\n.av-special-heading.av-4vb7j4-71b3ff69c05390a61eb23cd208733c05 .av-subheading{\nfont-size:15px;\n}\n<\/style>\n<div  class='av-special-heading av-4vb7j4-71b3ff69c05390a61eb23cd208733c05 av-special-heading-h3 blockquote modern-quote  avia-builder-el-5  el_before_av_hr  avia-builder-el-first  av-linked-heading'><h3 class='av-special-heading-tag'  itemprop=\"headline\"  >Important Dates<\/h3><div class=\"special-heading-border\"><div class=\"special-heading-inner-border\"><\/div><\/div><\/div><br \/>\n\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-4oil7c-66a007c3864951ce006119e2f6214644\">\n#top .hr.av-4oil7c-66a007c3864951ce006119e2f6214644{\nmargin-top:0px;\nmargin-bottom:10px;\n}\n.hr.av-4oil7c-66a007c3864951ce006119e2f6214644 .hr-inner{\nwidth:100%;\nborder-color:#e1e1e1;\n}\n<\/style>\n<div  class='hr av-4oil7c-66a007c3864951ce006119e2f6214644 hr-custom  avia-builder-el-6  el_after_av_heading  el_before_av_textblock  hr-left hr-icon-no'><span class='hr-inner inner-border-av-border-fat'><span class=\"hr-inner-style\"><\/span><\/span><\/div><br \/>\n<section  class='av_textblock_section av-4iyog0-678ed56e523f8005e4d7e76d069c1e25'  itemscope=\"itemscope\" itemtype=\"https:\/\/schema.org\/CreativeWork\" ><div class='avia_textblock text_justify'  itemprop=\"text\" ><p>Time zone: <a href=\"https:\/\/www.timeanddate.com\/time\/zones\/aoe\"><em>Anywhere on Earth(AoE)<\/em><\/a><br \/>\nReproducibility track papers due: February 21, 2023<br \/>\nReproducibility track papers notifications: March 31, 2023<br \/>\nCamera Ready for accepted reproducibility papers: April 26, 2023<\/p>\n<\/div><\/section><\/p><\/div>\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-49iyi8-85aa6f14bec53db94858612378de3974\">\n.flex_column.av-49iyi8-85aa6f14bec53db94858612378de3974{\npadding:0 40px 0 40px;\n}\n<\/style>\n<div class='flex_column av-49iyi8-85aa6f14bec53db94858612378de3974 av_one_full  avia-builder-el-8  el_after_av_one_full  el_before_av_one_full  first flex_column_div column-top-margin'   ><p>\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-4048u0-3578c5f5bd826f6bcc2804eb57e1076f\">\n#top .av-special-heading.av-4048u0-3578c5f5bd826f6bcc2804eb57e1076f{\nmargin:0px 0px 0px 0px;\npadding-bottom:10px;\n}\nbody .av-special-heading.av-4048u0-3578c5f5bd826f6bcc2804eb57e1076f .av-special-heading-tag .heading-char{\nfont-size:25px;\n}\n.av-special-heading.av-4048u0-3578c5f5bd826f6bcc2804eb57e1076f .av-subheading{\nfont-size:15px;\n}\n<\/style>\n<div  class='av-special-heading av-4048u0-3578c5f5bd826f6bcc2804eb57e1076f av-special-heading-h3 blockquote modern-quote  avia-builder-el-9  el_before_av_hr  avia-builder-el-first  av-linked-heading'><h3 class='av-special-heading-tag'  itemprop=\"headline\"  >Submission Guidelines<\/h3><div class=\"special-heading-border\"><div class=\"special-heading-inner-border\"><\/div><\/div><\/div><br \/>\n\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-xhfns-bf3527757bf7571d5e7f2c1f7336d3b4\">\n#top .hr.av-xhfns-bf3527757bf7571d5e7f2c1f7336d3b4{\nmargin-top:0px;\nmargin-bottom:10px;\n}\n.hr.av-xhfns-bf3527757bf7571d5e7f2c1f7336d3b4 .hr-inner{\nwidth:100%;\nborder-color:#e1e1e1;\n}\n<\/style>\n<div  class='hr av-xhfns-bf3527757bf7571d5e7f2c1f7336d3b4 hr-custom  avia-builder-el-10  el_after_av_heading  el_before_av_textblock  hr-left hr-icon-no'><span class='hr-inner inner-border-av-border-fat'><span class=\"hr-inner-style\"><\/span><\/span><\/div><br \/>\n<section  class='av_textblock_section av-laj3t6pe-39310ba2c81ac317f5c7b83c0b6e4611'  itemscope=\"itemscope\" itemtype=\"https:\/\/schema.org\/CreativeWork\" ><div class='avia_textblock text_justify av_inherit_color'  itemprop=\"text\" ><p>Reproducibility track papers are expected to help establish whether prior research in IR is generalizable beyond the theoretical or experimental settings that the paper(s) being reproduced assume(s). Submissions are welcome on reproducibility in any area in IR. Papers submitted to the Reproducibility paper track must explain:<\/p>\n<ul>\n<li style=\"list-style-type: none;\">\n<ul>\n<li>Their motivation for selecting the methods that are replicated or reproduced and the impact of these methods on the IR community;<\/li>\n<li>The directions in which they try to generalize, chose different angles from the original work that they reproduce, and the experimental setup(s) they select to support their research in these new directions;<\/li>\n<li>The assumptions of the original work that they found to hold up, and the ones that could not be confirmed. For papers in the reproducibility track the key is to share knowledge about what lessons from prior work held up.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>We expect authors to provide a comprehensive online appendix, with the code they developed, the data, and clear instructions for reproduction. The review process is\u00a0<del><span style=\"color: #00ccff;\">~single~<\/span><\/del><span style=\"color: #ff0000;\">*double*<\/span>-blind so that personal or institutional repositories<del> <span style=\"color: #00ccff;\">~can~<\/span><\/del><span style=\"color: #ff0000;\">*should not*<\/span> be used for the submission.<\/p>\n<\/div><\/section><\/p><\/div>\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-c7gtc-ce83e56adc1cc3da84d756d8b8cf62fc\">\n.flex_column.av-c7gtc-ce83e56adc1cc3da84d756d8b8cf62fc{\npadding:0 40px 0 40px;\n}\n<\/style>\n<div class='flex_column av-c7gtc-ce83e56adc1cc3da84d756d8b8cf62fc av_one_full  avia-builder-el-12  el_after_av_one_full  el_before_av_one_full  first flex_column_div column-top-margin'   ><p>\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-36h1uo-5da0e4a2e9461333c4df88f1bf480ad4\">\n#top .av-special-heading.av-36h1uo-5da0e4a2e9461333c4df88f1bf480ad4{\nmargin:0px 0px 0px 0px;\npadding-bottom:10px;\n}\nbody .av-special-heading.av-36h1uo-5da0e4a2e9461333c4df88f1bf480ad4 .av-special-heading-tag .heading-char{\nfont-size:25px;\n}\n.av-special-heading.av-36h1uo-5da0e4a2e9461333c4df88f1bf480ad4 .av-subheading{\nfont-size:15px;\n}\n<\/style>\n<div  class='av-special-heading av-36h1uo-5da0e4a2e9461333c4df88f1bf480ad4 av-special-heading-h3 blockquote modern-quote  avia-builder-el-13  el_before_av_hr  avia-builder-el-first  av-linked-heading'><h3 class='av-special-heading-tag'  itemprop=\"headline\"  >Reviewing Criteria<\/h3><div class=\"special-heading-border\"><div class=\"special-heading-inner-border\"><\/div><\/div><\/div><br \/>\n\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-rn01k-7bd47f4f949091988c6c9e611938b675\">\n#top .hr.av-rn01k-7bd47f4f949091988c6c9e611938b675{\nmargin-top:0px;\nmargin-bottom:10px;\n}\n.hr.av-rn01k-7bd47f4f949091988c6c9e611938b675 .hr-inner{\nwidth:100%;\nborder-color:#e1e1e1;\n}\n<\/style>\n<div  class='hr av-rn01k-7bd47f4f949091988c6c9e611938b675 hr-custom  avia-builder-el-14  el_after_av_heading  el_before_av_textblock  hr-left hr-icon-no'><span class='hr-inner inner-border-av-border-fat'><span class=\"hr-inner-style\"><\/span><\/span><\/div><br \/>\n<section  class='av_textblock_section av-2uoxc0-e07286fdf07a3f71fa4f5bd571bc9cd9'  itemscope=\"itemscope\" itemtype=\"https:\/\/schema.org\/CreativeWork\" ><div class='avia_textblock text_justify'  itemprop=\"text\" ><p>A dedicated reviewing committee will review submitted reproducibility track papers. The reviewing will be as rigorous as for regular SIGIR papers. However, the reviewing criteria will be different from the full paper track, with a strong emphasis on lessons learned, limitations, and generalizability.<\/p>\n<p>Submissions will be evaluated using the following criteria<\/p>\n<ul>\n<li style=\"list-style-type: none;\">\n<ul style=\"list-style-type: upper-roman;\">\n<li>Contribution: Does this work provide a novel angle on existing approaches and thereby lead to novel insights for the IR community?<br \/>\n1. To a small extent<br \/>\n2. To a moderate extent<br \/>\n3. To a good extent<br \/>\n4. To a great extent<\/li>\n<li>Motivation: How relevant is the replicated or reproduced work for the IR community, and how impactful are the achieved conclusions?<br \/>\n1. To a small extent<br \/>\n2. To a moderate extent<br \/>\n3. To a good extent<br \/>\n4. To a great extent<\/li>\n<li>Soundness: Is the replicated or reproduced paper sufficiently solid in terms of methodology and evaluation?<br \/>\n1. To a small extent<br \/>\n2. To a moderate extent<br \/>\n3. To a good extent<br \/>\n4. To a great extent<\/li>\n<li>Quality of reproduction artifacts: Do the supplementary materials for this submission support ease of reproducibility?<br \/>\n1. To a small extent<br \/>\n2. To a moderate extent<br \/>\n3. To a good extent<br \/>\n4. To a great extent<\/li>\n<li>Paper length: Is the contribution appropriate for the given length?<br \/>\n1. The paper is too long for the contribution that it makes<br \/>\n2. The length is just about right<br \/>\n3. The paper is too dense<\/li>\n<li>Overall evaluation (Summary of the final decision)<br \/>\n1. Main strengths<br \/>\n2. Main weaknesses<\/li>\n<li>Summary: Does it hold up? In which way does this paper confirm the reviewer\u2019s prior beliefs or provide new insights into what works and what doesn\u2019t? What new insight did you gain?<\/li>\n<li>Overall score:<br \/>\n1. Reject [-2]<br \/>\n2. Weak reject [-1]<br \/>\n3. Borderline [0]<br \/>\n4. Weak accept [1]<br \/>\n5. Accept [2]<\/li>\n<li>Reviewer confidence:<br \/>\n1. None<br \/>\n2. Low<br \/>\n3. Medium<br \/>\n4. High<br \/>\n5. Expert<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/div><\/section><\/p><\/div>\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-tep94-0d97a7c704b934622ea41358460355c2\">\n.flex_column.av-tep94-0d97a7c704b934622ea41358460355c2{\npadding:0 40px 0 40px;\n}\n<\/style>\n<div class='flex_column av-tep94-0d97a7c704b934622ea41358460355c2 av_one_full  avia-builder-el-16  el_after_av_one_full  el_before_av_one_full  first flex_column_div column-top-margin'   ><p>\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-juha0-412880eb061b123e7429573d0c6f5954\">\n#top .av-special-heading.av-juha0-412880eb061b123e7429573d0c6f5954{\nmargin:0px 0px 0px 0px;\npadding-bottom:10px;\n}\nbody .av-special-heading.av-juha0-412880eb061b123e7429573d0c6f5954 .av-special-heading-tag .heading-char{\nfont-size:25px;\n}\n.av-special-heading.av-juha0-412880eb061b123e7429573d0c6f5954 .av-subheading{\nfont-size:15px;\n}\n<\/style>\n<div  class='av-special-heading av-juha0-412880eb061b123e7429573d0c6f5954 av-special-heading-h3 blockquote modern-quote  avia-builder-el-17  el_before_av_hr  avia-builder-el-first  av-linked-heading'><h3 class='av-special-heading-tag'  itemprop=\"headline\"  >Format<\/h3><div class=\"special-heading-border\"><div class=\"special-heading-inner-border\"><\/div><\/div><\/div><br \/>\n\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-9coyw-97c81271ea76f06c45f9c85645bd826c\">\n#top .hr.av-9coyw-97c81271ea76f06c45f9c85645bd826c{\nmargin-top:0px;\nmargin-bottom:10px;\n}\n.hr.av-9coyw-97c81271ea76f06c45f9c85645bd826c .hr-inner{\nwidth:100%;\nborder-color:#e1e1e1;\n}\n<\/style>\n<div  class='hr av-9coyw-97c81271ea76f06c45f9c85645bd826c hr-custom  avia-builder-el-18  el_after_av_heading  el_before_av_textblock  hr-left hr-icon-no'><span class='hr-inner inner-border-av-border-fat'><span class=\"hr-inner-style\"><\/span><\/span><\/div><br \/>\n<section  class='av_textblock_section av-81jfs-db1b6374861a1a11eaa0c527ac9eb82c'  itemscope=\"itemscope\" itemtype=\"https:\/\/schema.org\/CreativeWork\" ><div class='avia_textblock text_justify'  itemprop=\"text\" ><p>Submissions of reproducibility track papers must be written in English. The page length limit is flexible with up to 9 pages (including figures, tables, and proofs), plus unrestricted space for references. Paper length should be commensurate with contribution size and authors should submit a paper whose length reflects what is needed for the content of the work. We invite reproducibility studies that were conducted within research for a full paper submission; in that case, please respect the anonymity requirement of the regular submission and note the necessity to submit a complete paper with its own distinct message and contribution.<\/p>\n<p>The submission should be in PDF, using the current ACM two-column conference format. Suitable LaTeX, Word, and Overleaf templates are available from the <a href=\"https:\/\/www.acm.org\/publications\/proceedings-template\">ACM website<\/a> (use the acmart.cls style file and set \u201csigconf=true,review=true\u201d as options)<\/p>\n<p><span style=\"font-weight: 400;\">Submissions should be submitted electronically via EasyChair. Note that the reproducibility papers track must be selected in the submission form. The submission site is:<\/span><\/p>\n<p><a href=\"https:\/\/easychair.org\/conferences?conf=sigir23\">https:\/\/easychair.org\/conferences?conf=sigir23<\/a><\/p>\n<p><span style=\"font-weight: 400;\">At least one author of an accepted paper is required to register for and present the work at the conference.<\/span><\/p>\n<\/div><\/section><\/p><\/div>\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-70hj4f-b2bbc238170118abeb80fa58f837474d\">\n.flex_column.av-70hj4f-b2bbc238170118abeb80fa58f837474d{\npadding:0 40px 0 40px;\n}\n<\/style>\n<div class='flex_column av-70hj4f-b2bbc238170118abeb80fa58f837474d av_one_full  avia-builder-el-20  el_after_av_one_full  el_before_av_one_full  first flex_column_div column-top-margin'   id=\"anonymity_and_authorship\"  ><p>\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-lb2zq7s1-d29a441a5e9b9dd81644df04c008e13d\">\n#top .av-special-heading.av-lb2zq7s1-d29a441a5e9b9dd81644df04c008e13d{\nmargin:0px 0px 0px 0px;\npadding-bottom:10px;\n}\nbody .av-special-heading.av-lb2zq7s1-d29a441a5e9b9dd81644df04c008e13d .av-special-heading-tag .heading-char{\nfont-size:25px;\n}\n.av-special-heading.av-lb2zq7s1-d29a441a5e9b9dd81644df04c008e13d .av-subheading{\nfont-size:15px;\n}\n<\/style>\n<div  class='av-special-heading av-lb2zq7s1-d29a441a5e9b9dd81644df04c008e13d av-special-heading-h3 blockquote modern-quote  avia-builder-el-21  el_before_av_hr  avia-builder-el-first  av-linked-heading'><h3 class='av-special-heading-tag'  itemprop=\"headline\"  >Anonymity and Authorship<\/h3><div class=\"special-heading-border\"><div class=\"special-heading-inner-border\"><\/div><\/div><\/div><br \/>\n\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-6hlvj3-489af6c3e0a71d988bf3ba100b9b2171\">\n#top .hr.av-6hlvj3-489af6c3e0a71d988bf3ba100b9b2171{\nmargin-top:0px;\nmargin-bottom:10px;\n}\n.hr.av-6hlvj3-489af6c3e0a71d988bf3ba100b9b2171 .hr-inner{\nwidth:100%;\nborder-color:#e1e1e1;\n}\n<\/style>\n<div  class='hr av-6hlvj3-489af6c3e0a71d988bf3ba100b9b2171 hr-custom  avia-builder-el-22  el_after_av_heading  el_before_av_textblock  hr-left hr-icon-no'><span class='hr-inner inner-border-av-border-fat'><span class=\"hr-inner-style\"><\/span><\/span><\/div><br \/>\n<section  class='av_textblock_section av-lb2zrjk7-663baee56f64a64fce945d175f09e595'  itemscope=\"itemscope\" itemtype=\"https:\/\/schema.org\/CreativeWork\" ><div class='avia_textblock text_justify'  itemprop=\"text\" ><p>The reproducibility paper review process is double-blind. Authors are required to take all reasonable steps to preserve the anonymity of their submission. The submission must not include author information and must not include citations or discussion of related work that would make the authorship apparent. However, it is acceptable to refer to companies or organizations that provided datasets, hosted experiments or deployed solutions if there is no implication that the authors are currently affiliated with these organizations. While authors can upload to institutional or other preprint repositories such as arXiv.org before reviewing is complete, we generally discourage this since it places anonymity at risk. If the paper is already on arXiv, please change the title and abstract so that it is not immediately obvious they are the same. Do not upload the paper to a preprint site after submission to SIGIR\u2014wait until a review decision to avoid reviewers seeing the paper in daily digests or other places. Breaking anonymity puts the submission at risk of being desk rejected.<\/p>\n<p>Link to codes is encouraged in reproducibility paper. However, simply sharing a Github link may have an anonymity violation risk, as it can contain committer information. The authors are suggested to take one of the following actions when sharing codes in their paper:<\/p>\n<p>&#8211; Use an anonymization tool for Github.<\/p>\n<p>&#8211; Use a cloud storage you can use, upload codes removing author information, and put a link to it.<\/p>\n<p>Authors should carefully go through <a href=\"https:\/\/www.acm.org\/publications\/policies\/authorship\">ACM\u2019s authorship policy<\/a> before submitting a paper. Please ensure that all authors are clearly identified in EasyChair before the submission deadline. To support the identification of reviewers with conflicts of interest, the full author list must be specified at submission time. No changes to authorship will be permitted for the camera-ready submission under any circumstance or after submissions close. So please, make sure you have them listed correctly when submissions close.<\/p>\n<\/div><\/section><\/p><\/div>\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-3s4cxr-00192eae2673cad2f34bbefb9eb72ad4\">\n.flex_column.av-3s4cxr-00192eae2673cad2f34bbefb9eb72ad4{\npadding:0 40px 0 40px;\n}\n<\/style>\n<div class='flex_column av-3s4cxr-00192eae2673cad2f34bbefb9eb72ad4 av_one_full  avia-builder-el-24  el_after_av_one_full  el_before_av_one_full  first flex_column_div column-top-margin'   ><p>\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-lb2zrssh-b7fac5d05c0d9383cf94580e90a41179\">\n#top .av-special-heading.av-lb2zrssh-b7fac5d05c0d9383cf94580e90a41179{\nmargin:0px 0px 0px 0px;\npadding-bottom:10px;\n}\nbody .av-special-heading.av-lb2zrssh-b7fac5d05c0d9383cf94580e90a41179 .av-special-heading-tag .heading-char{\nfont-size:25px;\n}\n.av-special-heading.av-lb2zrssh-b7fac5d05c0d9383cf94580e90a41179 .av-subheading{\nfont-size:15px;\n}\n<\/style>\n<div  class='av-special-heading av-lb2zrssh-b7fac5d05c0d9383cf94580e90a41179 av-special-heading-h3 blockquote modern-quote  avia-builder-el-25  el_before_av_hr  avia-builder-el-first  av-linked-heading'><h3 class='av-special-heading-tag'  itemprop=\"headline\"  >Reproducibility Paper Chairs<\/h3><div class=\"special-heading-border\"><div class=\"special-heading-inner-border\"><\/div><\/div><\/div><br \/>\n\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-2y0gvj-130f5494fb13c544132f039c51e8dc81\">\n#top .hr.av-2y0gvj-130f5494fb13c544132f039c51e8dc81{\nmargin-top:0px;\nmargin-bottom:10px;\n}\n.hr.av-2y0gvj-130f5494fb13c544132f039c51e8dc81 .hr-inner{\nwidth:100%;\nborder-color:#e1e1e1;\n}\n<\/style>\n<div  class='hr av-2y0gvj-130f5494fb13c544132f039c51e8dc81 hr-custom  avia-builder-el-26  el_after_av_heading  el_before_av_textblock  hr-left hr-icon-no'><span class='hr-inner inner-border-av-border-fat'><span class=\"hr-inner-style\"><\/span><\/span><\/div><br \/>\n<section  class='av_textblock_section av-lb2zs3iw-62e25fdc8d83affd655935eca6b5f5c5'  itemscope=\"itemscope\" itemtype=\"https:\/\/schema.org\/CreativeWork\" ><div class='avia_textblock text_justify'  itemprop=\"text\" ><ul>\n<li style=\"list-style-type: none;\">\n<ul>\n<li>Gabriella Pasi, University of Milano-Bicocca<\/li>\n<li>Jiafeng Guo, Institute of Computing Technology, CAS<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/div><\/section><\/p><\/div>\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-hkpre-b50610b1369cbcc26a16a6bd04f6e3c3\">\n.flex_column.av-hkpre-b50610b1369cbcc26a16a6bd04f6e3c3{\npadding:0 40px 0 40px;\n}\n<\/style>\n<div class='flex_column av-hkpre-b50610b1369cbcc26a16a6bd04f6e3c3 av_one_full  avia-builder-el-28  el_after_av_one_full  avia-builder-el-last  first flex_column_div column-top-margin'   ><p>\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-laj6psuz-199187eaeec7f13aa7b7dae42cb2990e\">\n#top .av-special-heading.av-laj6psuz-199187eaeec7f13aa7b7dae42cb2990e{\nmargin:0px 0px 0px 0px;\npadding-bottom:10px;\n}\nbody .av-special-heading.av-laj6psuz-199187eaeec7f13aa7b7dae42cb2990e .av-special-heading-tag .heading-char{\nfont-size:25px;\n}\n.av-special-heading.av-laj6psuz-199187eaeec7f13aa7b7dae42cb2990e .av-subheading{\nfont-size:15px;\n}\n<\/style>\n<div  class='av-special-heading av-laj6psuz-199187eaeec7f13aa7b7dae42cb2990e av-special-heading-h3 blockquote modern-quote  avia-builder-el-29  el_before_av_hr  avia-builder-el-first  av-linked-heading'><h3 class='av-special-heading-tag'  itemprop=\"headline\"  >Contact<\/h3><div class=\"special-heading-border\"><div class=\"special-heading-inner-border\"><\/div><\/div><\/div><br \/>\n\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-9ja0a-5ebe9e0698aa66e358721a2aa6da8484\">\n#top .hr.av-9ja0a-5ebe9e0698aa66e358721a2aa6da8484{\nmargin-top:0px;\nmargin-bottom:10px;\n}\n.hr.av-9ja0a-5ebe9e0698aa66e358721a2aa6da8484 .hr-inner{\nwidth:100%;\nborder-color:#e1e1e1;\n}\n<\/style>\n<div  class='hr av-9ja0a-5ebe9e0698aa66e358721a2aa6da8484 hr-custom  avia-builder-el-30  el_after_av_heading  el_before_av_textblock  hr-left hr-icon-no'><span class='hr-inner inner-border-av-border-fat'><span class=\"hr-inner-style\"><\/span><\/span><\/div><br \/>\n<section  class='av_textblock_section av-laj6reh1-71dabd486c9f5dc0452879a383e03d93'  itemscope=\"itemscope\" itemtype=\"https:\/\/schema.org\/CreativeWork\" ><div class='avia_textblock text_justify'  itemprop=\"text\" ><p>For further information, please contact the SIGIR 2023 Reproducibility Co-chairs by email to <a href=\"mailto:sigir23-reproducibility@easychair.org\">sigir23-reproducibility@easychair.org<\/a><\/p>\n<\/div><\/section><\/p><\/div><\/div><\/div><\/div><!-- close content main div --><\/div><\/div><div id='after_section_1'  class='main_color av_default_container_wrap container_wrap sidebar_right'  ><div class='container av-section-cont-open' ><div class='template-page content  av-content-small alpha units'><div class='post-entry post-entry-type-page post-entry-1167'><div class='entry-content-wrapper clearfix'>\n","protected":false},"excerpt":{"rendered":"","protected":false},"author":4,"featured_media":0,"parent":27,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-1167","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/sigir.org\/sigir2023\/wp-json\/wp\/v2\/pages\/1167","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sigir.org\/sigir2023\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/sigir.org\/sigir2023\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/sigir.org\/sigir2023\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/sigir.org\/sigir2023\/wp-json\/wp\/v2\/comments?post=1167"}],"version-history":[{"count":15,"href":"https:\/\/sigir.org\/sigir2023\/wp-json\/wp\/v2\/pages\/1167\/revisions"}],"predecessor-version":[{"id":1307,"href":"https:\/\/sigir.org\/sigir2023\/wp-json\/wp\/v2\/pages\/1167\/revisions\/1307"}],"up":[{"embeddable":true,"href":"https:\/\/sigir.org\/sigir2023\/wp-json\/wp\/v2\/pages\/27"}],"wp:attachment":[{"href":"https:\/\/sigir.org\/sigir2023\/wp-json\/wp\/v2\/media?parent=1167"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}