[Bug] Flux FP4 doesn't handle multiple LoRA

[Matriv-org]

Checklist

• 1. I have searched for related issues and FAQs (Nunchaku Frequently Asked Questions nunchaku#262) but was unable to find a solution.
• 2. The issue persists in the latest version.
• 3. Please note that without environment information and a minimal reproducible example, it will be difficult for us to reproduce and address the issue, which may delay our response.
• 4. If your report is a question rather than a bug, please submit it as a discussion at https://github.com/mit-han-lab/ComfyUI-nunchaku/discussions/new/choose. Otherwise, this issue will be closed.
• 5. I will do my best to describe the issue in English.

Describe the Bug

Hello,

On ComfyUI when I use in series two LoRA models, I've always one of the two LoRA who doesn't work.
I've tested it

• Each of LoRA one by one it works, if I use both of them it looks like it bypass one of them, lora model weight doesn't change nothing, I used Nunchaku Flux.1 LoRA Loader.
• I tested it with GGUF Q4 (Unet Loader), FP8 and CR LoRA stack on the exact same pipeline and it works.

Environment

Window 11, Pytorch2.8Cu128, RTX 5090, ComfyUI 0.3.27, sageattention 2

Reproduction Steps

FP4 vs FP8 vs GGUF.json

[lmxyy]

Do you have the LoRA weights for debugging?

[Matriv-org]

I tested it with every Flux LoRA combination possible it doesn't work when I use them in series.

The first LoRA is based on my image so I unfortunately don't share him.
The arcane one: https://civitai.com/models/846744/arcane-style

[zappazack]

Same issue here I have uses the nunchaku lora loader with 1 lora and its works fine:

..Converting LoRAs to nunchaku format: 100%|█████████████████████████████████████████████| 57/57 [00:00<00:00, 76.80it/s]...

But 5 Loras are resulting into the following error, no conversion into nunchaku format:
Requested to load FluxClipModel_
loaded partially 8443.533070373534 8443.5302734375 0
Requested to load SDXLClipModel
loaded completely 2432.5330703735353 1560.802734375 True
loaded partially 8443.533070373534 8443.5302734375 0
Requested to load Flux
0 models unloaded.
0%| | 0/20 [00:01<?, ?it/s]
!!! Exception during processing !!! "mul_cpu_reduced_float" not implemented for 'Float8_e4m3fn'
Traceback (most recent call last):
File "C:\Users\Intersurf\Documents\ComfyUI\execution.py", line 327, in execute
output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
File "C:\Users\Intersurf\Documents\ComfyUI\execution.py", line 202, in get_output_data
return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
File "C:\Users\Intersurf\Documents\ComfyUI\execution.py", line 174, in _map_node_over_list
process_inputs(input_dict, i)
File "C:\Users\Intersurf\Documents\ComfyUI\execution.py", line 163, in process_inputs
results.append(getattr(obj, func)(**inputs))
File "C:\Users\Intersurf\Documents\ComfyUI\nodes.py", line 1543, in sample
return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise)
File "C:\Users\Intersurf\Documents\ComfyUI\nodes.py", line 1510, in common_ksampler
samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
File "C:\Users\Intersurf\Documents\ComfyUI\custom_nodes\comfyui-stablesr\nodes.py", line 69, in hook_sample
return original_sample(*args, **kwargs)
File "C:\Users\Intersurf\Documents\ComfyUI\comfy\sample.py", line 45, in sample
samples = sampler.sample(noise, positive, negative, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed)
File "C:\Users\Intersurf\Documents\ComfyUI\comfy\samplers.py", line 1133, in sample
return sample(self.model, noise, positive, negative, cfg, self.device, sampler, sigmas, self.model_options, latent_image=latent_image, denoise_mask=denoise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed)
File "C:\Users\Intersurf\Documents\ComfyUI\comfy\samplers.py", line 1023, in sample
return cfg_guider.sample(noise, latent_image, sampler, sigmas, denoise_mask, callback, disable_pbar, seed)
File "C:\Users\Intersurf\Documents\ComfyUI\comfy\samplers.py", line 1008, in sample
output = executor.execute(noise, latent_image, sampler, sigmas, denoise_mask, callback, disable_pbar, seed)
File "C:\Users\Intersurf\Documents\ComfyUI\comfy\patcher_extension.py", line 110, in execute
return self.original(*args, **kwargs)
File "C:\Users\Intersurf\Documents\ComfyUI\comfy\samplers.py", line 976, in outer_sample
output = self.inner_sample(noise, latent_image, device, sampler, sigmas, denoise_mask, callback, disable_pbar, seed)
File "C:\Users\Intersurf\Documents\ComfyUI\comfy\samplers.py", line 959, in inner_sample
samples = executor.execute(self, sigmas, extra_args, callback, noise, latent_image, denoise_mask, disable_pbar)
File "C:\Users\Intersurf\Documents\ComfyUI\comfy\patcher_extension.py", line 110, in execute
return self.original(*args, **kwargs)
File "C:\Users\Intersurf\Documents\ComfyUI\comfy\samplers.py", line 738, in sample
samples = self.sampler_function(model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar, **self.extra_options)
File "C:\Users\Intersurf\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\utils_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "C:\Users\Intersurf\Documents\ComfyUI\comfy\k_diffusion\sampling.py", line 161, in sample_euler
denoised = model(x, sigma_hat * s_in, **extra_args)
File "C:\Users\Intersurf\Documents\ComfyUI\comfy\samplers.py", line 390, in call
out = self.inner_model(x, sigma, model_options=model_options, seed=seed)
File "C:\Users\Intersurf\Documents\ComfyUI\comfy\samplers.py", line 939, in call
return self.predict_noise(*args, **kwargs)
File "C:\Users\Intersurf\Documents\ComfyUI\comfy\samplers.py", line 942, in predict_noise
return sampling_function(self.inner_model, x, timestep, self.conds.get("negative", None), self.conds.get("positive", None), self.cfg, model_options=model_options, seed=seed)
File "C:\Users\Intersurf\Documents\ComfyUI\comfy\samplers.py", line 370, in sampling_function
out = calc_cond_batch(model, conds, x, timestep, model_options)
File "C:\Users\Intersurf\Documents\ComfyUI\comfy\samplers.py", line 206, in calc_cond_batch
return executor.execute(model, conds, x_in, timestep, model_options)
File "C:\Users\Intersurf\Documents\ComfyUI\comfy\patcher_extension.py", line 110, in execute
return self.original(*args, **kwargs)
File "C:\Users\Intersurf\Documents\ComfyUI\comfy\samplers.py", line 319, in calc_cond_batch
output = model.apply_model(input_x, timestep, **c).chunk(batch_chunks)
File "C:\Users\Intersurf\Documents\ComfyUI\comfy\model_base.py", line 137, in apply_model
return comfy.patcher_extension.WrapperExecutor.new_class_executor(
File "C:\Users\Intersurf\Documents\ComfyUI\comfy\patcher_extension.py", line 110, in execute
return self.original(*args, **kwargs)
File "C:\Users\Intersurf\Documents\ComfyUI\comfy\model_base.py", line 170, in _apply_model
model_output = self.diffusion_model(xc, t, context=context, control=control, transformer_options=transformer_options, **extra_conds).float()
File "C:\Users\Intersurf\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1739, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "C:\Users\Intersurf\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1750, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Users\Intersurf\Documents\ComfyUI\custom_nodes\ComfyUI-nunchaku\nodes\models\flux.py", line 84, in forward
composed_lora = compose_lora(lora_to_be_composed)
File "C:\Users\Intersurf\AppData\Local\Programs\Python\Python310\lib\site-packages\nunchaku\lora\flux\compose.py", line 88, in compose_lora
v = v * strength
RuntimeError: "mul_cpu_reduced_float" not implemented for 'Float8_e4m3fn'

Prompt executed in 9.02 seconds

[LiteSoul]

You need to use the Nunchaku Lora Loader, not a regular Lora loader. And chain it.
Nunchaku makes a Lora conversion on the fly to it's format I think.

[zappazack]

You need to use the Nunchaku Lora Loader, not a regular Lora loader. And chain it. Nunchaku makes a Lora conversion on the fly to it's format I think.

Hi thx but that's exactly what I did as u can see at the beginning of my post 1 or 2 Loras are beeing converted into nunchaku format but the error message U see occurred from a chain of 5. Also I have figured out while testing regular Flux that the output is much better if the Loras are embedded directly in Clip text. Fe hands, fingers are perfect.

[LiteSoul]

@zappazack Yeah I was commenting to OP which uses a custom Lora stacker.
By the way I was left curious... how can we embed Loras directly in Clip text? Can you explain or screenshot that section of the workflow or attach it? Thanks.

[Matriv-org]

@zappazack Yeah I was commenting to OP which uses a custom Lora stacker. By the way I was left curious... how can we embed Loras directly in Clip text? Can you explain or screenshot that section of the workflow or attach it? Thanks.

On the nunchaku loader it uses the nunchaku lora loader

[zappazack]

@zappazack Yeah I was commenting to OP which uses a custom Lora stacker. By the way I was left curious... how can we embed Loras directly in Clip text? Can you explain or screenshot that section of the workflow or attach it? Thanks.

Its not possible doing this the nunchaku repo as mentioned its a Clip node for regular Flux which I am running in gguf multigpu mode with enabled sageattn. About the node u can check this discussion: https://civitai.com/models/686814?dialog=commentThread&commentId=773061&highlight=777028 I have noticed that any extra lora loader influences the output of hands and fingers if u use fe more than 4 loras. No clue why.

[lmxyy]

You need to use the Nunchaku Lora Loader, not a regular Lora loader. And chain it. Nunchaku makes a Lora conversion on the fly to it's format I think.

Hi thx but that's exactly what I did as u can see at the beginning of my post 1 or 2 Loras are beeing converted into nunchaku format but the error message U see occurred from a chain of 5. Also I have figured out while testing regular Flux that the output is much better if the Loras are embedded directly in Clip text. Fe hands, fingers are perfect.

You are using FP8 LoRAs I think. See #58

[lmxyy]

Checklist

• 1. I have searched for related issues and FAQs (Nunchaku Frequently Asked Questions nunchaku#262) but was unable to find a solution.[x] 2. The issue persists in the latest version.[x] 3. Please note that without environment information and a minimal reproducible example, it will be difficult for us to reproduce and address the issue, which may delay our response.[x] 4. If your report is a question rather than a bug, please submit it as a discussion at https://github.com/mit-han-lab/ComfyUI-nunchaku/discussions/new/choose. Otherwise, this issue will be closed.[x] 5. I will do my best to describe the issue in English.

Describe the Bug

Hello,

On ComfyUI when I use in series two LoRA models, I've always one of the two LoRA who doesn't work. I've tested it

• Each of LoRA one by one it works, if I use both of them it looks like it bypass one of them, lora model weight doesn't change nothing, I used Nunchaku Flux.1 LoRA Loader.
• I tested it with GGUF Q4 (Unet Loader), FP8 and CR LoRA stack on the exact same pipeline and it works.

Environment

Window 11, Pytorch2.8Cu128, RTX 5090, ComfyUI 0.3.27, sageattention 2

Reproduction Steps

FP4 vs FP8 vs GGUF.json

Please use our Nunchaku FLUX LoRA Loader.