-
-
Notifications
You must be signed in to change notification settings - Fork 6.2k
[FEAT]: Model router #5313
Copy link
Copy link
Open
Labels
Description
What would you like to see?
- Build an intelligent prompt routing layer that intercepts user prompts before they reach LLM to dynamically select the model that would provide the best result for the costs
- Main idea is to use calculated (deterministic) rules such as: keyword triggers, token count, message counts, date/time periods (some models have peak pricing times) etc
- See if we can get semantic routing working by allowing users to configure a specific model used for routing chats inside admin settings for model router and using that model to do a tool call to route to the correct model each time
- Follow backend design per internal spec on Notion
Reactions are currently unavailable