MS MARCO Web Search: Powering Next-Gen Information Access & Neural Indexers

MS MARCO Web Search dataset provides real-world web data to mitigate LLM hallucination and update challenges, fostering research in neural indexers, embedding models, and LLM-based IR systems.

Table of Links

Abstract and 1 Introduction

2 Background and Related work

2.1 Web Scale Information Retrieval

2.2 Existing Datasets

3 MS Marco Web Search Dataset and 3.1 Document Preparation

3.2 Query Selection and Labeling

3.3 Dataset Analysis

3.4 New Challenges Raised by MS MARCO Web Search

4 Benchmark Results and 4.1 Environment Setup

4.2 Baseline Methods

4.3 Evaluation Metrics

4.4 Evaluation of Embedding Models and 4.5 Evaluation of ANN Algorithms

4.6 Evaluation of End-to-end Performance

5 Potential Biases and Limitations

6 Future Work and Conclusions, and References

ABSTRACT

Recent breakthroughs in large models have highlighted the critical significance of data scale, labels and modals. In ...