Skip to content

Adafactor fused backward pass and optimizer step, lowers SDXL (@ 1024 resolution) VRAM usage to BF16(10GB)/FP32(16.4GB)#1259

Merged
kohya-ss merged 1 commit intokohya-ss:fused-backward-passfrom
2kpr:fused_backward_pass
May 6, 2024
Merged

Adafactor fused backward pass and optimizer step, lowers SDXL (@ 1024 resolution) VRAM usage to BF16(10GB)/FP32(16.4GB)#1259
kohya-ss merged 1 commit intokohya-ss:fused-backward-passfrom
2kpr:fused_backward_pass

Commits

Commits on Apr 14, 2024
⚡