Skip to content

mlx_lm.server --adapter-path silently ignored at startup #1248

@odysa

Description

@odysa

CLI accepts --adapter-path and stores it on ModelProvider, but the value never reaches the model loader. No error or warning is raised. This affects anyone serving an adapter via the CLI.

Repro

Pass a non-existent adapter path:

mlx_lm.server \
  --model mlx-community/Qwen2.5-0.5B-Instruct-4bit \
  --adapter-path /tmp/this-does-not-exist

The server starts successfully and serves requests.

Expected

The server should fail loudly:

      raise FileNotFoundError(f"The adapter path does not exist: {adapter_path}")
  FileNotFoundError: The adapter path does not exist: /tmp/this-does-not-exist

Behavior should match the per-request HTTP API, where invalid adapter paths correctly raise FileNotFoundError.

Root cause

ModelProvider.__init__ stores both model and adapter under the alias "default_model":

self._model_map["default_model"] = self.cli_args.model
self._adapter_map["default_model"] = self.cli_args.adapter_path

load_default() calls:

self.load("default_model", None, "default_model")

Inside load():

def load(self, model_path, adapter_path=None, draft_model_path=None):
    model_path = self._model_map.get(model_path, model_path)
    adapter_path = self._adapter_map.get(model_path, adapter_path)

model_path is resolved before _adapter_map is queried. After resolution, the lookup key becomes the real model name instead of "default_model".

Since _adapter_map only contains "default_model", the lookup misses and adapter_path falls back to None.

As a result, the CLI-provided adapter path is silently dropped.

Notes

Per-request loads are unaffected because they pass the real model name directly, so the alias lookup path is bypassed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions