Skip to content

fix: validate routing models against discovery#1206

Open
guillaumegay13 wants to merge 3 commits intomnfst:mainfrom
guillaumegay13:fix/routing-available-model-validation
Open

fix: validate routing models against discovery#1206
guillaumegay13 wants to merge 3 commits intomnfst:mainfrom
guillaumegay13:fix/routing-available-model-validation

Conversation

@guillaumegay13
Copy link
Copy Markdown
Collaborator

@guillaumegay13 guillaumegay13 commented Mar 20, 2026


Summary by cubic

Enforces that routing selections only use models discovered for the agent and removes pricing/prefix-based fallbacks. Improves cleanup on provider removal while preserving native-only picks when an API key remains. Related to #1203.

  • Bug Fixes
    • Validate setOverride/setFallbacks against discovery; reject missing models with BadRequestException.
    • getEffectiveModel: use the override only if it’s discovered; otherwise fall back to auto; remove pricing/provider-prefix heuristics.
    • On provider removal, clear overrides/fallbacks tied to the removed provider or its removed cached_models, and notify; keep native-only models if another active auth (e.g., API key) for the same provider remains.
    • Discovery: do not synthesize api-key models from pricing/OpenRouter when native discovery returns empty; subscription providers are still supplemented with known subscription models.
    • setFallbacks: de-duplicate inputs, exclude the current override, and return the cleaned list.

Written for commit 7d0f40e. Summary will update on new commits.

Related to #1203

@codecov
Copy link
Copy Markdown

codecov bot commented Mar 20, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.48%. Comparing base (0786a68) to head (7d0f40e).

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #1206   +/-   ##
=======================================
  Coverage   98.48%   98.48%           
=======================================
  Files          94       94           
  Lines        7605     7605           
  Branches     2901     2901           
=======================================
  Hits         7490     7490           
  Misses         88       88           
  Partials       27       27           
Flag Coverage Δ
frontend 98.73% <ø> (ø)
plugin 95.89% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 4 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/backend/src/routing/routing.service.ts">

<violation number="1" location="packages/backend/src/routing/routing.service.ts:182">
P2: `removedModels` is collected from all `unsupported` records, but should only include models from providers in `removedProviders`. If a provider is still active via a different auth type, its models from the unsupported subscription record will be incorrectly flagged for tier-assignment removal.</violation>
</file>

<file name="packages/backend/src/routing/model-discovery/model-discovery.service.spec.ts">

<violation number="1" location="packages/backend/src/routing/model-discovery/model-discovery.service.spec.ts:884">
P2: The comment says subscription models "are still supplemented" but the assertion `toBeGreaterThanOrEqual(0)` is a tautology — it passes even when the result is empty (i.e., no supplementing occurred). If the intent is to verify fallback models are returned, use `toBeGreaterThan(0)` or assert on specific model IDs.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

Comment thread packages/backend/src/routing/routing.service.ts Outdated
@guillaumegay13
Copy link
Copy Markdown
Collaborator Author

@brunobuddy merge conflicts resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant