Skip to content

feat: add SINDI inverted index with SIMD-accelerated window-based sparse vector search #1555

@sparknack

Description

@sparknack

Summary

Add the SINDI (SIMD-accelerated Inverted Index) for sparse vector search, based on the SINDI algorithm. SINDI partitions posting lists into fixed-size windows, storing only 16-bit local offsets instead of full 32-bit doc IDs. This significantly reduces memory footprint and enables efficient SIMD-based scoring.

Key Features

  • Window-based TAAT search: For each window, scores are scatter-accumulated into a dense buffer via SIMD (AVX-512/AVX2/SVE), then batch-inserted into a top-k heap
  • Platform-specific SIMD: Separate compilation units for AVX-512, AVX2, and SVE with runtime dispatch

Reference

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions