Skip to content

[OpenVINO] Support Qwen3.5, Qwen3.5-MoE and Qwen3.6#1689

Open
rkazants wants to merge 203 commits intohuggingface:mainfrom
rkazants:support_qwen3_5
Open

[OpenVINO] Support Qwen3.5, Qwen3.5-MoE and Qwen3.6#1689
rkazants wants to merge 203 commits intohuggingface:mainfrom
rkazants:support_qwen3_5

Conversation

@rkazants
Copy link
Copy Markdown
Collaborator

@rkazants rkazants commented Apr 15, 2026

What does this PR do?

Re-created PR #1634

Fixes 181271, 181280, 182003

Installation instructions:

pip install git+https://github.com/rkazants/optimum-intel.git@support_qwen3_5
pip install --pre -U openvino openvino-tokenizers nncf --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly
pip install transformers==5.2.0
pip install requests torchvision opencv-python

Exporting cmd-line:

optimum-cli export openvino -m Qwen/Qwen3.5-0.8B Qwen3.5-0.8B

Inference script:

from transformers import AutoProcessor
from transformers.video_utils import load_video
from huggingface_hub import hf_hub_download
from optimum.intel.openvino import OVModelForVisualCausalLM

model_dir = "Qwen/Qwen3.5-0.8B"

processor = AutoProcessor.from_pretrained(model_dir)
model = OVModelForVisualCausalLM.from_pretrained(model_dir)

# Prepare video input
video_path = hf_hub_download(
                repo_id="raushan-testing-hf/videos-test",
                filename="sample_demo_1.mp4",
                repo_type="dataset",
            )
input_video, _ = load_video(video_path, num_frames=10, backend="opencv")

messages = [
    {"role": "user", "content": [
        {"type": "video"},
        {"type": "text", "text": "Why is this video funny?"},
    ]}
]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], videos=[input_video], return_tensors="pt")

# Run inference
output_ids = model.generate(**inputs, max_new_tokens=100)
output_text = processor.decode(output_ids[0], skip_special_tokens=True)

print(output_text)

Before submitting

  • [N/A] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you make sure to update the documentation with your changes?
  • [] Did you write any new necessary tests?

Comment thread .github/workflows/test_offline.yaml Outdated
Comment thread optimum/exporters/openvino/model_configs.py Outdated
Comment thread optimum/exporters/openvino/model_patcher.py Outdated
@HuggingFaceDocBuilderDev
Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Comment thread tests/openvino/test_decoder.py Outdated
Comment thread tests/openvino/test_decoder.py Outdated
Comment thread tests/openvino/test_decoder.py Outdated
Comment thread tests/openvino/test_decoder.py Outdated
Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
@rkazants rkazants changed the title Support qwen3 5 [OpenVINO] Support Qwen3.5 and Qwen3.5-MoE Apr 15, 2026
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
@rkazants rkazants changed the title [OpenVINO] Support Qwen3.5 and Qwen3.5-MoE [OpenVINO] Support Qwen3.5, Qwen3.5-MoE and Qwen3.6 Apr 16, 2026
@savvadesogle
Copy link
Copy Markdown

Qwen3.6
Hooray!🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants