Summary
When a SentenceTransformer model uses a Router for asymmetric encoding (separate query and document encoders), there's currently no built-in way to export a single route as a standalone model artifact. I'm wondering whether introducing such functionality would would be useful.
Motivation
In deployment scenarios, the query and document encoders are often deployed independently. For example, the document encoder runs offline during indexing, and the query encoder runs online in a retrieval service (e.g. Elasticsearch). There would be redundancy if loading the full model (both routes) in each service.
The model.save() method already writes each route's modules to separate subdirectories, so the weights are physically separated on disk. However, there's no public API to reconstruct a single route as a standalone SentenceTransformer that can be loaded without any knowledge of Router.
Possible Solution
A method on SentenceTransformer (since that's the natural entry point for saving):
model.save_route("./query_encoder", route="query")
model.save_route("./document_encoder", route="document")Which would produce standard, fully loadable SentenceTransformer artifacts with no Router wrapper — just the route's modules plus any surrounding modules from the parent model.
Happy to try to implement this if the approach sounds reasonable.
Summary
When a SentenceTransformer model uses a
Routerfor asymmetric encoding (separate query and document encoders), there's currently no built-in way to export a single route as a standalone model artifact. I'm wondering whether introducing such functionality would would be useful.Motivation
In deployment scenarios, the query and document encoders are often deployed independently. For example, the document encoder runs offline during indexing, and the query encoder runs online in a retrieval service (e.g. Elasticsearch). There would be redundancy if loading the full model (both routes) in each service.
The
model.save()method already writes each route's modules to separate subdirectories, so the weights are physically separated on disk. However, there's no public API to reconstruct a single route as a standalone SentenceTransformer that can be loaded without any knowledge of Router.Possible Solution
A method on
SentenceTransformer(since that's the natural entry point for saving):Which would produce standard, fully loadable SentenceTransformer artifacts with no Router wrapper — just the route's modules plus any surrounding modules from the parent model.
Happy to try to implement this if the approach sounds reasonable.