Skip to content

[JS] Implement Text2ImagePipeline#3740

Open
Retribution98 wants to merge 3 commits intoopenvinotoolkit:masterfrom
Retribution98:text2image
Open

[JS] Implement Text2ImagePipeline#3740
Retribution98 wants to merge 3 commits intoopenvinotoolkit:masterfrom
Retribution98:text2image

Conversation

@Retribution98
Copy link
Copy Markdown
Contributor

Signed-off-by: Kirill Suvorov kirill.suvorov@intel.com

Description

  • Implemented Node.js binding for Text2ImagePipeline
  • Added Performance metrics for image generation
  • Step callback support (except intermediate tensors)
  • The simplest sample text2image.js
  • Updated hugging_face.py to correct download flux model

CVS-184436

Checklist:

  • This PR follows GenAI Contributing guidelines.
  • Tests have been updated or added to cover the new code.
  • This PR fully addresses the ticket.
  • I have made corresponding changes to the documentation.

Signed-off-by: Kirill Suvorov <kirill.suvorov@intel.com>
Copilot AI review requested due to automatic review settings April 21, 2026 16:29
@github-actions github-actions Bot added no-match-files category: GH Pages Docs Github Pages documentation category: GGUF GGUF file reader category: JS API GenAI JS API category: Image generation samples GenAI Image generation samples labels Apr 21, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds Node.js bindings for OpenVINO GenAI Text2ImagePipeline, including perf metrics exposure, step-callback support, and end-to-end coverage through JS samples/tests and documentation updates. It also adjusts Python test utilities to support exporting/downloading a tiny FLUX model for image-generation testing.

Changes:

  • Implemented Text2ImagePipeline Node.js addon wrapper + TypeScript API surface (generate/config/perf metrics + step callback).
  • Added JS tests and a minimal JS sample for text-to-image generation, and wired the sample into Python sample tests.
  • Updated docs to include JavaScript usage snippets for image generation; updated HF test utility export logic for FLUX.

Reviewed changes

Copilot reviewed 28 out of 29 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
tests/python_tests/utils/hugging_face.py Adds optional-tokenizer flow + task-specific optimum-cli export override (incl. tiny FLUX).
tests/python_tests/samples/test_text2image.py Extends sample test to execute the JS text2image sample.
src/js/tests/text2ImagePipeline.test.js Adds Node.js test coverage for Text2ImagePipeline creation/generate/config/perf/callback behavior.
src/js/tests/setup.py Adds download/convert setup for an image-generation test model (tiny FLUX).
src/js/src/text2image_pipeline/pipeline_wrapper.cpp C++ N-API wrapper for init/generate/config/perf metrics + step callback wiring.
src/js/src/text2image_pipeline/perf_metrics.cpp N-API wrapper for image-generation perf metrics and raw metrics access.
src/js/src/text2image_pipeline/init_worker.cpp Async init worker for constructing the C++ Text2ImagePipeline off-thread.
src/js/src/helper.cpp Adds JS↔C++ conversions for ImageGenerationConfig and image-gen perf metrics.
src/js/src/addon.cpp Registers Text2ImagePipeline and Text2ImagePerfMetrics classes in the addon.
src/js/lib/utils.ts Adds TS types for ImageGenerationConfig, pipeline properties, and per-step callback.
src/js/lib/pipelines/text2ImagePipeline.ts Implements the JS/TS Text2ImagePipeline wrapper (promisified init/generate).
src/js/lib/perfMetrics.ts Adds TS interface/types for Text2Image perf metrics + raw metrics.
src/js/lib/index.ts Exposes Text2ImagePipeline via the factory/export surface.
src/js/lib/addon.ts Adds addon-side TS typings for Text2ImagePipeline and Text2ImagePerfMetrics.
src/js/include/text2image_pipeline/pipeline_wrapper.hpp Declares the N-API wrapper class for Text2ImagePipeline.
src/js/include/text2image_pipeline/perf_metrics.hpp Declares the N-API wrapper class for Text2Image perf metrics.
src/js/include/text2image_pipeline/init_worker.hpp Declares the async init worker for Text2ImagePipeline.
src/js/include/helper.hpp Declares helper template specializations for new image-gen bindings.
src/js/include/addon.hpp Adds stored prototypes for Text2ImagePipeline and Text2ImagePerfMetrics.
src/js/eslint.config.cjs Allows snake_case config keys used by image generation options.
site/src/pages/_sections/UseCasesSection/components/image-generation.tsx Adds JavaScript tab to the image generation use-case section.
site/docs/use-cases/image-generation/_sections/_usage_options/index.mdx Adds JS snippet for configuring image generation parameters.
site/docs/use-cases/image-generation/_sections/_run_model/index.mdx Adds JS tab/device examples for running text-to-image.
site/docs/use-cases/image-generation/_sections/_run_model/_text2image_js.mdx New JS code example (save result via Jimp).
site/docs/bindings/node-js.md Updates Node.js bindings coverage list to include Text2ImagePipeline.
samples/js/package.json Adds jimp for the image-generation JS sample.
samples/js/package-lock.json Lockfile update to include Jimp and transitive deps.
samples/js/image_generation/text2image.js New minimal JS Text2Image sample with per-step callback + BMP output.
samples/js/image_generation/README.md New README documenting the JS text-to-image sample usage.
Files not reviewed (1)
  • samples/js/package-lock.json: Language not supported

Comment thread tests/python_tests/utils/hugging_face.py
Comment thread src/js/src/text2image_pipeline/pipeline_wrapper.cpp
Comment thread samples/js/image_generation/text2image.js Outdated
Comment thread tests/python_tests/utils/hugging_face.py Outdated
Comment thread tests/python_tests/utils/hugging_face.py
Comment thread tests/python_tests/utils/hugging_face.py Outdated
@Retribution98 Retribution98 requested a review from Copilot April 22, 2026 08:32
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 29 out of 30 changed files in this pull request and generated 7 comments.

Files not reviewed (1)
  • samples/js/package-lock.json: Language not supported
Comments suppressed due to low confidence (1)

tests/python_tests/utils/hugging_face.py:1

  • get_huggingface_models() can now return hf_tokenizer=None, but the provided context shows OVConvertedModelSchema.hf_tokenizer is typed as AutoTokenizer (non-optional). This mismatch can lead to downstream code assuming a tokenizer exists and failing at runtime or during type checking; update the schema annotation (and any consumers) to accept AutoTokenizer | None, or ensure the schema is only constructed when a tokenizer is present.
# Copyright (C) 2018-2026 Intel Corporation

Comment thread src/js/src/text2image_pipeline/pipeline_wrapper.cpp
Comment thread src/js/src/text2image_pipeline/pipeline_wrapper.cpp
Comment thread src/js/src/text2image_pipeline/pipeline_wrapper.cpp
Comment thread src/js/src/text2image_pipeline/pipeline_wrapper.cpp
Comment thread src/js/src/text2image_pipeline/pipeline_wrapper.cpp
Comment thread src/js/tests/text2ImagePipeline.test.js
Comment thread samples/js/image_generation/text2image.js
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category: GGUF GGUF file reader category: GH Pages Docs Github Pages documentation category: Image generation samples GenAI Image generation samples category: JS API GenAI JS API no-match-files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants