rope argument inconsistent

[yfcAzul]

In Qwen3Attention, the code appears to support RoPE scaling by forwarding a rope_scaling argument into get_rope:
self.rotary_emb = get_rope( self.head_dim, rotary_dim=self.head_dim, max_position=max_position, base=rope_theta, rope_scaling=rope_scaling, )

However, in layers/rotary_embedding.py, get_rope immediately enforces:

assert rope_scaling is None

It seems like the two pieces of code encode contradictory assumptions

[GeeeekExplorer]

You're right, the Qwen3 model doesn't implement RoPE scaling, so we asserts rope_scaling is None. The parameter was included in Qwen3Attention for potential future use.

For YaRN or other scaling methods, the key change is in cos_sin_cache initialization, but optimization techniques stay similar.
We keep it simple for now.

[CZH-THU]

@GeeeekExplorer Hi I also meet same problem,but when I use Qwen3 model.it's rope_scaling is {'rope_theta': 1000000, 'rope_type': 'default'} not None which leads TypeError: unhashable type: 'dict'

[Nouzen-86]

@CZH-THU You can just comment out the rope_scaling=rope_scaling line, and the parts in get_rope.

[yfcAzul]

@GeeeekExplorer Hi I also meet same problem,but when I use Qwen3 model.it's rope_scaling is {'rope_theta': 1000000, 'rope_type': 'default'} not None which leads TypeError: unhashable type: 'dict'

You can disable this argument to run the test, though I’m not sure what the potential consequences might be.

[guojia99]

i also encountered a similar bug , i don't know how solve

[lingkerio]

@guojia99
nanovllm/models/qwen3.py
line 59: change to rope_scaling=None,

[banfeb]

Root Cause:

This issue stems from a change in configuration parsing behavior across transformers library versions. In version 4.57.3, loading the configuration via AutoConfig.from_pretrained(self.model) (at line 24 of config.py) sets self.hf_config.rope_scaling to None, which executes without error. However, in version 5.1.0, the same call initializes rope_scaling as a dictionary (e.g., {'rope_theta': 1000000, 'rope_type': 'default'}). When this dictionary is passed to the get_rope function in rotary_embedding.py, it triggers a TypeError because the function is decorated with @lru_cache(1), which requires all input arguments to be hashable.

Solution:

To resolve this, you can explicitly set rope_scaling = None at any point before calling the get_rope function. For instance, you can add self.hf_config.rope_scaling = None immediately after the AutoConfig loading line in config.py, or modify line 59 in qwen3.py to pass rope_scaling=None directly into the function call.

[wln20]

I also encountered the same issue of getting "TypeError: unhashable type: 'dict'" when calling the get_rope() method with Qwen3. My solution is to simply comment out @lru_cache(1) and assert rope_scaling is None, and that resolves the problem:

# @lru_cache(1)
def get_rope(
    head_size: int,
    rotary_dim: int,
    max_position: int,
    base: float,
    rope_scaling: dict | None = None,
):
    # assert rope_scaling is None
    rotary_emb = RotaryEmbedding(head_size, rotary_dim, max_position, base)
    return rotary_emb

[Seas0]

Just pin transformers<5 is enough for a temporary fix