Skip to content

Add Qwen Adapter unit tests#1388

Open
has9800 wants to merge 4 commits into
TransformerLensOrg:devfrom
has9800:test_qwen_adapter
Open

Add Qwen Adapter unit tests#1388
has9800 wants to merge 4 commits into
TransformerLensOrg:devfrom
has9800:test_qwen_adapter

Conversation

@has9800

@has9800 has9800 commented Jun 14, 2026

Copy link
Copy Markdown

Description

Adds unit tests for the Qwen (Qwen-1) architecture adapter.

New file covers:

  • Config flags the adapter sets (RMS norm, rotary embeddings, gated MLP, final
    RMS, attn_only)
  • Component mapping for the TransformerLens canonical names to Qwen HF module paths,
    and bridge types for each entry
  • Weight-processing conversion keys (q/k/v/o) and the HF source weights they
    read from
  • _split_qkv_matrix splitting Qwen's fused c_attn projection back into
    separate q/k/v projections, covering the standard [3*d_model, d_model] layout,
    the Conv1D style [d_model, 3*d_model] layout, no-bias path, and an
    invalid-shape error. All work, all tests pass
  • Factory registration for QwenForCausalLM resolves to QwenArchitectureAdapter

Contributes to #1302

Type of change

  • Test addition

Checklist:

  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have not rewritten tests relating to key interfaces which would affect backward compatibility

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant