Skip to content

Specific-tier fallbacks are persisted but ignored by the proxy fallback path #1586

@Nicolas-Formenton

Description

@Nicolas-Formenton

Title: Specific-tier fallbacks are persisted but ignored by the proxy fallback path

Summary

Specific-tier fallback chains can be configured and persisted in specificity_assignments.fallback_models, but they are not used when a specificity-routed request fails upstream.

As a result, requests matched to a specific tier (for example coding or trading) use the specific tier's primary model, but fallback behavior is taken only from the general tier assignment in the proxy path.

Why this looks like a bug

The data model, service layer, controller, and UI all support specific-tier fallbacks:

  • specificity_assignments.fallback_models
  • SpecificityService.setFallbacks(...)
  • SpecificityController.setFallbacks(...)
  • UI for adding fallbacks on specific tiers

But in the request path:

  • ResolveService.resolveSpecificity(...) returns the selected model/provider/auth/specificity category, but not the specific tier's fallback_models
  • ProxyService.proxyRequest(...) reads fallback models only from tierService.getTiers(...) using resolved.tier

That means specific-tier fallback chains are silently ignored.

Expected behavior

If a request matched a specific tier and that route fails with a fallback-triggering status, the proxy should use:

  1. the specific tier's fallback_models, if present
  2. otherwise the general tier fallback list

Observed behavior

Only the general tier fallback list is considered.

Minimal reproduction

  1. Configure a specific tier such as coding with:
    • primary model A
    • fallback model B
  2. Leave the general tier for the resolved scorer tier with no fallback, or with a different fallback list
  3. Send a request that matches the coding specific tier
  4. Force primary model A to fail upstream with a fallback-triggering status (for example 429)
  5. Observe that the proxy does not use the specific-tier fallback chain

Root cause hypothesis

ResolveService.resolveSpecificity(...) should propagate assignment.fallback_models, and ProxyService.proxyRequest(...) should prefer those fallbacks when present.

Candidate fix areas

  • packages/backend/src/routing/resolve/resolve.service.ts
  • packages/backend/src/routing/proxy/proxy.service.ts
  • packages/backend/src/routing/dto/resolve-response.ts

Suggested fix

  • extend ResolveResponse with fallback_models?: string[] | null
  • in resolveSpecificity(...), include fallback_models: assignment.fallback_models ?? null
  • in proxyRequest(...), use:
    const fallbackModels = resolved.fallback_models ?? assignment?.fallback_models;

Suggested regression coverage

  1. resolve.service.spec.ts
  • assert that specificity resolution propagates fallback_models
  1. proxy.service.spec.ts
  • assert that when resolveService.resolve(...) returns a specificity-routed response with fallback_models, those models are used even if the general tier fallback list is null

Notes

This report intentionally omits local environment details, tokens, hostnames, and user-specific configuration.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions