feat: add comfyUI generate video workflow (HunyuanVideo1.5)

[weedge]

HunyuanVideo1.5:

comfyUI with hunyuanVideo1.5 720p text2video workflow config

content-3.mp4

Surreal dreamscape with floating islands, upside-down waterfalls, and impossible geometric structures, all bathed in a soft, ethereal light

content-2.mp4

A paper airplane released from the top of a skyscraper, gliding through urban canyons, crossing traffic, flying over streets, spiraling upward between buildings. The camera follows the paper airplane's perspective, shooting cityscape in first-person POV, finally flying toward the sunset, disappearing in golden light. Creative camera movement, free perspective, dreamlike colors.

---

AI podcast:

• 混元视频1.5：开源视频生成的新标杆 https://podcast-997.pages.dev/podcast/2205908f219c4c44bf723481b527faf2

---

feat:

• add openCV video stream source to image frames

# run http server
# Run the Flask app for streaming
python demo/cv_video2Images.py

# To test, ensure 'videos/cv/capture4_2024-09-15_20-10-29.mp4' exists relative to this script.
# Then open your browser access this url:
http://127.0.0.1:8001/stream_video_frames?video_source=videos/cv/capture4_2024-09-15_20-10-29.mp4
# To stream from an HTTP URL (e.g., a public MP4 file):
http://127.0.0.1:8001/stream_video_frames?video_source=https://www.learningcontainer.com/wp-content/uploads/2020/05/sample-mp4-file.mp4

• add comfyui video huanyuanVideo1.5

Download Models(ckpt)

• Download models from huggingface

# download flux2.0 dev ckpt (text_encoders, clip_vision, latent_upsampler, diffusion_models, vae, and LoRA model)
modal run src/download_models.py --repo-ids "Comfy-Org/HunyuanVideo_1.5_repackaged" --allow-patterns "*.safetensors"

• Download workflow config

# download video_hunyuan_video_1_5_720p_t2v workflow config for comfyUI
wget https://raw.githubusercontent.com/Comfy-Org/workflow_templates/refs/heads/main/templates/video_hunyuan_video_1.5_720p_t2v.json -O src/comfyui/workflow_config/video_hunyuan_video_1_5_720p_t2v.json

DIY Workflow

• DIY workflow, import workflow config and export workflow api json config (need Nvidia GPU / AMD / MAC ARM Silicon)

# run  confyui with hunyuanVideo1.5 720p (text2video) workflow json config
MODEL_NAME=video_hunyuan_video_1_5_720p_t2v IMAGE_GPU=L40S modal serve src/comfyui/ui.py

Deploy API Server (wrap comfy launch server and comfy run --workflow api workflow)

• deploy comfyui api server (need Nvidia GPU / AMD / MAC ARM Silicon)

# run  confyui server with hunyuanVideo1.5 720p (text2video) workflow api json config
MODEL_NAME=video_hunyuan_video_1_5_720p_t2v IMAGE_GPU=L40S modal serve src/comfyui/server.py

hunyuanVideo1.5 720p workflow api json run log (text2Video):

Executing : 加载VAE - VAELoader (10)
Executing : Empty HunyuanVideo 1.5 Latent - EmptyHunyuanVideo15Latent (124)
Executing : UNet加载器 - UNETLoader (12)
Executing : 基本调度器 - BasicScheduler (128)
Executing : K采样器选择 - KSamplerSelect (130)
Executing : 双CLIP加载器 - DualCLIPLoader (11)
Executing : CLIP Text Encode (Negative Prompt) - CLIPTextEncode (93)
Executing : CLIP Text Encode (Positive Prompt) - CLIPTextEncode (44)
Executing : 采样算法（SD3） - ModelSamplingSD3 (132)
Executing : CFG引导器 - CFGGuider (131)
Executing : 随机噪波 - RandomNoise (129)
Executing : 自定义采样器（高级） - SamplerCustomAdvanced (127)
Executing : VAE解码 - VAEDecode (8)
Executing : 创建视频 - CreateVideo (101)
Executing : 保存视频 - SaveVideo (102)

Client

• generate video

# hunyuanVideo1.5 720p (text2Video)
python deploy/modal/src/comfyui/client.py --modal-workspace $(modal profile current) --prompt "一只熊猫在吃火锅，写实风格。" --size 1280x720 --steps 20 --dev --type video --model video_hunyuan_video_1_5_720p_t2v

---

achatbot integrate ComfyUI API Video Gen Processor

• add ComfyUIAPIVideoGenProcessor and test

VIDEO_GEN_PROCESSOR=ComfyUIAPIVideoGenProcessor \
    API_URL=https://weedge--server-comfyui-api-dev.modal.run/video \
    MODEL=video_hunyuan_video_1_5_720p_t2v \
    python -m unittest test.integration.processors.test_video_gen_processor.TestProcessor

reference

• 2025.11 HunyuanVideo 1.5 Technical Report
• https://github.com/Tencent-Hunyuan/HunyuanVideo-1.5
• https://huggingface.co/tencent/HunyuanVideo-1.5
• https://docs.comfy.org/tutorials/video/hunyuan/hunyuan-video-1-5
• https://github.com/Tencent-Hunyuan/HunyuanVideo-1.5/blob/main/assets/HunyuanVideo_1_5_Prompt_Handbook_EN.md

[gemini-code-assist]

Summary of Changes

Hello @weedge, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly expands the project's capabilities by integrating video generation features. It introduces the HunyuanVideo1.5 model into the ComfyUI ecosystem, provides a new utility for streaming video frames, and refactors the API to clearly separate image and video generation functionalities. These changes enable users to generate videos from text prompts and stream them efficiently, enhancing the overall multimedia generation pipeline.

Highlights

• ComfyUI Video Generation Workflow: Introduced a new ComfyUI workflow for generating videos using the HunyuanVideo1.5 model, complete with model loading and configuration.
• OpenCV Video Stream Source: Added a Flask application that streams video frames from a local file or HTTP URL using OpenCV, converting them to JPEG bytes for live viewing.
• API Endpoint Refactoring: Refactored the ComfyUI API server and client to support distinct /image and /video endpoints, improving modularity and clarity for different generation tasks.
• New Video Generation Processor: Implemented a ComfyUIAPIVideoGenProcessor to handle video generation requests, along with a base class and integration tests.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a new video generation workflow using ComfyUI and HunyuanVideo1.5, along with a Flask-based video frame streaming demo. The changes are extensive, adding new processors, server endpoints, and configuration files. While the core functionality is a great addition, I've identified several critical security vulnerabilities and bugs that need to be addressed. Specifically, the demo script has path traversal and remote code execution vulnerabilities. The new video processor has critical bugs that will cause it to fail at runtime. I've also left some comments on maintainability and potential runtime issues. Please review the comments carefully to ensure the code is secure and robust.