Skip to content

[FEAT]: Model router #5313

@shatfield4

Description

@shatfield4

What would you like to see?

  • Build an intelligent prompt routing layer that intercepts user prompts before they reach LLM to dynamically select the model that would provide the best result for the costs
  • Main idea is to use calculated (deterministic) rules such as: keyword triggers, token count, message counts, date/time periods (some models have peak pricing times) etc
  • See if we can get semantic routing working by allowing users to configure a specific model used for routing chats inside admin settings for model router and using that model to do a tool call to route to the correct model each time
  • Follow backend design per internal spec on Notion

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions