Tasks of the Implementation Challenge and Demo Track

Test Data and Queries: Approximately 100 million CLIP descriptors extracted from the LAION database Similarity between two objects is measured by their dot product. The similarity function is a dot product The goal is to evaluate 30 nearest neighbours queries using 10K query objects described below

Task 1: Unrestricted Indexing

In this task, system solutions will have access to all resources on our testing computer to build their indexing solutions. The goal is to achieve the highest search performance within the given constraints.

Task 2: Memory-Constrained Indexing with Reranking

This task challenges participants to develop memory-efficient indexing solutions with reranking capabilities. Each solution will be run in a Linux container with limited memory and storage resources.

Task 3: Memory-Constrained Indexing without Reranking

In this task, participants are asked to develop memory-efficient indexing solutions that will be used without reranking the search results. The container provided will have higher memory capacity compared to Task 2. Participants have to build an index in the first phase. In the search phase, the original vectors cannot be used.

Datasets and queries

We will reproduce your results post-challenge and produce rankings on quality and search time using a different query set. So, it is essential that your solution can be replicated.

Please visit examples section for working examples. Note that the available examples can be used as starting points.

CC BY-SA 4.0 sisap challenge committee. Last modified: April 08, 2024. Website built with Franklin.jl and the Julia programming language.