[Bug]: Cannot use LARS or LAMB

[KewkLW]

What happened?

Any time I attempt to test LARS or LAMB the error happens.

What did you expect would happen?

No errors

Relevant log output

step:   0%|          | 0/213 [00:13<?, ?it/s]
epoch:   0%|          | 0/100 [00:15<?, ?it/s]
Traceback (most recent call last):
  File "F:\StabilityMatrix-win-x64\Data\Packages\OneTrainer\modules\ui\TrainUI.py", line 538, in __training_thread_function
    trainer.train()
  File "F:\StabilityMatrix-win-x64\Data\Packages\OneTrainer\modules\trainer\GenericTrainer.py", line 594, in train
    scaler.step(self.model.optimizer)
  File "F:\StabilityMatrix-win-x64\Data\Packages\OneTrainer\venv\lib\site-packages\torch\amp\grad_scaler.py", line 453, in step
    retval = self._maybe_opt_step(optimizer, optimizer_state, *args, **kwargs)
  File "F:\StabilityMatrix-win-x64\Data\Packages\OneTrainer\venv\lib\site-packages\torch\amp\grad_scaler.py", line 351, in _maybe_opt_step
    retval = optimizer.step(*args, **kwargs)
  File "F:\StabilityMatrix-win-x64\Data\Packages\OneTrainer\venv\lib\site-packages\torch\optim\lr_scheduler.py", line 75, in wrapper
    return wrapped(*args, **kwargs)
  File "F:\StabilityMatrix-win-x64\Data\Packages\OneTrainer\venv\lib\site-packages\torch\optim\optimizer.py", line 391, in wrapper
    out = func(*args, **kwargs)
  File "F:\StabilityMatrix-win-x64\Data\Packages\OneTrainer\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "F:\StabilityMatrix-win-x64\Data\Packages\OneTrainer\venv\lib\site-packages\bitsandbytes\optim\optimizer.py", line 287, in step
    self.update_step(group, p, gindex, pindex)
  File "F:\StabilityMatrix-win-x64\Data\Packages\OneTrainer\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "F:\StabilityMatrix-win-x64\Data\Packages\OneTrainer\venv\lib\site-packages\bitsandbytes\optim\optimizer.py", line 707, in update_step
    F.optimizer_update_32bit(
  File "F:\StabilityMatrix-win-x64\Data\Packages\OneTrainer\venv\lib\site-packages\bitsandbytes\functional.py", line 1584, in optimizer_update_32bit
    optim_func = str2optimizer32bit[optimizer_name][0]
KeyError: 'lars'

Output of pip freeze

absl-py==2.1.0
accelerate==0.30.1
aiohttp==3.9.5
aiosignal==1.3.1
antlr4-python3-runtime==4.9.3
async-timeout==4.0.3
attrs==23.2.0
bitsandbytes==0.43.1
certifi==2024.2.2
charset-normalizer==3.3.2
cloudpickle==3.0.0
colorama==0.4.6
coloredlogs==15.0.1
contourpy==1.2.1
customtkinter==5.2.2
cycler==0.12.1
dadaptation==3.2
darkdetect==0.8.0
-e git+https://github.com/huggingface/diffusers.git@dd4b731e68f88f58dfabfb68f28e00ede2bb90ae#egg=diffusers
einops==0.8.0
exceptiongroup==1.2.2
filelock==3.14.0
flatbuffers==24.3.25
fonttools==4.53.1
frozenlist==1.4.1
fsspec==2024.5.0
ftfy==6.2.0
grpcio==1.64.0
huggingface-hub==0.23.3
humanfriendly==10.0
idna==3.7
importlib_metadata==7.1.0
iniconfig==2.0.0
intel-openmp==2021.4.0
invisible-watermark==0.2.0
Jinja2==3.1.4
kiwisolver==1.4.5
lightning-utilities==0.11.2
lion-pytorch==0.2.2
Markdown==3.6
markdown-it-py==3.0.0
MarkupSafe==2.1.5
matplotlib==3.9.1
mdurl==0.1.2
-e git+https://github.com/Nerogar/mgds.git@5ea389a62408cff79a8f3a11b2f25c185c7c1c2e#egg=mgds
mkl==2021.4.0
mpmath==1.3.0
multidict==6.0.5
networkx==3.3
numpy==1.26.4
omegaconf==2.3.0
onnxruntime-gpu==1.18.0
open-clip-torch==2.24.0
opencv-python==4.9.0.80
packaging==24.0
pandas==2.2.2
pillow==10.3.0
platformdirs==4.2.2
pluggy==1.5.0
pooch==1.8.1
prodigyopt==1.0
protobuf==4.25.3
psutil==5.9.8
Pygments==2.18.0
pynvml==11.5.0
pyparsing==3.1.2
pyreadline3==3.4.1
pytest==8.2.2
python-dateutil==2.9.0.post0
pytorch-lightning==2.2.5
pytz==2024.1
PyWavelets==1.6.0
PyYAML==6.0.1
regex==2024.5.15
requests==2.32.3
rich==13.7.1
safetensors==0.4.3
scalene==1.5.41
schedulefree==1.2.5
scipy==1.14.0
sentencepiece==0.2.0
six==1.16.0
sympy==1.12.1
tbb==2021.12.0
tensorboard==2.17.0
tensorboard-data-server==0.7.2
timm==1.0.3
tokenizers==0.19.1
tomli==2.0.1
torch==2.3.0+cu118
torchmetrics==1.4.0.post0
torchvision==0.18.0+cu118
tqdm==4.66.4
transformers==4.42.3
typing_extensions==4.12.1
tzdata==2024.1
urllib3==2.2.1
wcwidth==0.2.13
Werkzeug==3.0.3
xformers==0.0.26.post1+cu118
yarl==1.9.4
zipp==3.19.1

[Nerogar]

This is a bug in bitsandbytes that should be fixed with the next release. Once it's fixed, we can update the dependency
bitsandbytes-foundation/bitsandbytes#1118

[O-J1]

Im farily confident this has been fixed. Please reopen if not.