Skip to content

Support a vanilla faiss IndexNode that exposes faiss index_factory DSL #1583

@foxspy

Description

@foxspy

Description

Knowhere currently wraps each faiss index variant (INDEX_FAISS_IDMAP, INDEX_FAISS_IVFFLAT, INDEX_HNSW, INDEX_FAISS_IVFPQ, ...) with a dedicated IndexNode class, a typed config, and registration boilerplate. Adding a new faiss index variant, or evaluating a composition faiss itself already supports (e.g. OPQ16,IVF1024,PQ16x8, IVF1024,PQ16x8,RFlat, SVSVamana128,LVQ4x4), requires writing a new IndexNode wrapper and enum entry even when the only thing we want is to call faiss::index_factory and forward parameters verbatim.

This is a blocker for:

  • Rapidly prototyping new faiss indexes (e.g. Refine wrappers, OPQ + IVF + PQ stacks, Intel SVS)
  • Evaluating faiss composition strings without touching Knowhere source
  • Giving advanced users direct access to faiss's DSL for research purposes

Expected Behavior

A new abstract INDEX_FAISS IndexNode (index_type = "FAISS") that takes a faiss factory string via a faiss_index_name config field and forwards all other JSON keys to faiss. The interface should match Milvus's existing parameter style — top-level keys, no nested param object:

{
  "index_type": "FAISS",
  "faiss_index_name": "IVF1024,PQ16x8",
  "metric_type": "L2",
  "nprobe": 16
}

The adapter should:

  • Build the index via faiss::index_factory / faiss::index_binary_factory
  • Forward build-time params via faiss::ParameterSpace
  • Build per-request SearchParametersXxx (IVF / HNSW / PQ / Refine / PreTransform / SVS Vamana) for search, so concurrent queries with different knobs are isolated
  • Honor BitsetView filtering via SearchParameters::sel
  • Surface errors for unknown faiss parameters rather than silently dropping them
  • Support both fp32 (dense) and bin1 (binary) data types

Actual Behavior

No such IndexNode exists today. Adding support for a new faiss index requires a full per-index wrapper each time.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions