[Feature] Support Non-Diffusers Version

[KasenYoung]

Motivation

Thanks for your excellent work. I wonder whether this repo can support non-diffusers models, or is there a simply way to convert a model to diffusers version? Looking forward to your reply.

Related resources

None

[SolitaryThinker]

Is there a particular model you want to see FastVideo support?

We mainly follow diffusers as a standardized way to organize weights on HF repos. We usually convert non-diffusers model to diffusers format manually and upload to FastVideo's HF Repo.

Here's some example scripts we use: https://github.com/hao-ai-lab/FastVideo/tree/main/scripts/checkpoint_conversion

[KasenYoung]

Wan-2.1, both T2V and I2V, 1.3B and 14B. If I trained a Lora model using non-diffusers version, can I efficiently convert it to diffusers version?

[MihirJagtap]

yes in most cases you can.

There are two steps usually - 1) Map the Lora state dict keys from the non-diffusers naming to the names expected by the diffusers pipeline, and 2) save the remapped weights in the format diffusers expects.