Skip to content

feat: add routing and model discovery for more API-key providers#1283

Open
guillaumegay13 wants to merge 1 commit intomnfst:mainfrom
guillaumegay13:main
Open

feat: add routing and model discovery for more API-key providers#1283
guillaumegay13 wants to merge 1 commit intomnfst:mainfrom
guillaumegay13:main

Conversation

@guillaumegay13
Copy link
Copy Markdown
Collaborator

@guillaumegay13 guillaumegay13 commented Mar 26, 2026

Summary

  • add routing and dynamic model discovery support for Cohere, Cerebras, Cloudflare Workers AI, GitHub Models, Groq, Hugging Face, LLM7.io, and Ollama Cloud
  • use provider-native catalogs where needed, including Cloudflare account-scoped discovery and Hugging Face router model listing, while preserving provider-specific model IDs and handles in proxy routing
  • add frontend provider metadata, validation, API-key help links, and logos for the new providers so discovered models are grouped under the connected host provider in the routing UI

Scope Notes

  • Kluster was intentionally removed before submission because its product fit as a first-class inference provider was unclear and we could not validate a current general-purpose key flow
  • the temporary provider-wide is_free tagging was removed; free filtering stays on the existing safe 0/0 pricing behavior rather than trying to infer provider free-tier access heuristically
  • no database migrations required

Validation

  • fork PR CI passed fully, including backend, frontend, shared, plugin, lint, changeset, and Cubic review
  • local targeted backend tests: npm test --workspace=packages/backend -- --runInBand src/routing/model-discovery/provider-model-fetcher.service.spec.ts src/routing/routing.controller.spec.ts
  • local targeted frontend tests: npm test --workspace=packages/frontend -- ModelPickerModal.test.tsx Routing.test.tsx
  • local builds: npm run build --workspace=packages/shared, npm run build --workspace=packages/backend, npm run build --workspace=packages/frontend
  • live smoke validation with real credentials for Cloudflare, GitHub Models, Cohere, Cerebras, Groq, Hugging Face, LLM7.io, and Ollama Cloud

Notes

  • Cloudflare Workers AI currently uses ACCOUNT_ID:API_TOKEN in the existing API key field so the backend can derive the account-scoped base URL without adding a new settings surface
  • plugin asset hashes were regenerated from a fresh build during merge resolution rather than hand-picked from either side of the conflict

@codecov
Copy link
Copy Markdown

codecov bot commented Mar 26, 2026

Codecov Report

❌ Patch coverage is 94.73684% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 98.65%. Comparing base (c7b396f) to head (79ffb5d).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1283      +/-   ##
==========================================
- Coverage   98.76%   98.65%   -0.11%     
==========================================
  Files         103      103              
  Lines        7280     7456     +176     
  Branches     2822     2856      +34     
==========================================
+ Hits         7190     7356     +166     
- Misses         81       88       +7     
- Partials        9       12       +3     
Flag Coverage Δ
frontend 98.71% <94.70%> (-0.12%) ⬇️
manifest 100.00% <ø> (ø)
plugin 96.01% <ø> (ø)
shared 100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@codecov
Copy link
Copy Markdown

codecov bot commented Mar 26, 2026

Bundle Report

Changes will increase total bundle size by 17.18kB (3.3%) ⬆️. This is within the configured threshold ✅

Detailed changes
Bundle name Size Change
manifest-frontend-esm 537.63kB 17.18kB (3.3%) ⬆️

Affected Assets, Files, and Routes:

view changes for bundle: manifest-frontend-esm

Assets Changed:

Asset Name Size Change Total Size Change (%)
assets/index-*.css 94 bytes 115.63kB 0.08%
assets/Routing-*.js 428 bytes 83.98kB 0.51%
assets/ProviderIcon-*.js 16.28kB 43.8kB 59.16% ⚠️
assets/provider-*.js 379 bytes 1.16kB 48.4% ⚠️

Files in assets/Routing-*.js:

  • ./src/pages/RoutingTierCard.tsx → Total Size: 12.65kB

  • ./src/components/ModelPickerModal.tsx → Total Size: 15.23kB

Files in assets/ProviderIcon-*.js:

  • ./src/services/providers.ts → Total Size: 7.69kB

  • ./src/services/provider-utils.ts → Total Size: 3.46kB

  • ./src/components/ProviderIcon.tsx → Total Size: 36.69kB

  • ./src/services/routing-utils.ts → Total Size: 1.24kB

Files in assets/provider-*.js:

  • ./src/services/provider-api-key-urls.ts → Total Size: 1.47kB

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 33 files

@guillaumegay13 guillaumegay13 changed the title feat: add dynamic discovery for more inference providers feat: add routing and model discovery for more API-key providers Mar 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant