Skip to content

Commit d7c179c

Browse files
TimDettmersclaude
andcommitted
docs: Add docstrings for NF4/FP4 quantization types and Linear8bitLt threshold
Document create_normal_map (NF4 lookup table construction, offset parameter), create_fp8_map (configurable floating-point format), get_4bit_type NF4/FP4 inline comments, and Linear8bitLt threshold parameter (operates on activations, not weights). Also add issue maintenance workflow and common issue patterns guide for triage. Addresses recurring questions: #938, #968, #988, #1265, #1609, #1772. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 66a8e93 commit d7c179c

4 files changed

Lines changed: 332 additions & 19 deletions

File tree

agents/issue_maintenance_guide.md

Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
# Issue Maintenance Guide
2+
3+
You are an issue maintenance agent. Your job is to review open GitHub issues for bitsandbytes, identify issues that should be closed, and close them with helpful comments. You are **not** fixing bugs — you are triaging.
4+
5+
## Prerequisites
6+
7+
Refresh the issue data:
8+
9+
```bash
10+
python3 ~/git/lab_tools/github/fetch_issues.py
11+
```
12+
13+
Read `agents/github_tools_guide.md` for the full reference on query tools, and `agents/issue_patterns.md` for known closeable patterns.
14+
15+
## Step 1: Get the Landscape
16+
17+
```bash
18+
# All open issues
19+
python3 ~/git/lab_tools/github/query_issues.py list
20+
21+
# Low-hanging fruit
22+
python3 ~/git/lab_tools/github/query_issues.py list --label "Duplicate"
23+
python3 ~/git/lab_tools/github/query_issues.py list --label "Proposing to Close"
24+
python3 ~/git/lab_tools/github/query_issues.py list --label "Waiting for Info"
25+
python3 ~/git/lab_tools/github/query_issues.py list --label "Question"
26+
python3 ~/git/lab_tools/github/query_issues.py list --label "Likely Not a BNB Issue"
27+
python3 ~/git/lab_tools/github/query_issues.py list --unlabeled
28+
```
29+
30+
## Step 2: Identify Closeable Issues
31+
32+
For each issue, determine if it matches a known pattern from `agents/issue_patterns.md`. The most common categories are:
33+
34+
### Already triaged by maintainers
35+
36+
- **Labeled `Duplicate`** but still open — close with a comment pointing to the canonical issue.
37+
- **Labeled `Proposing to Close`** — review the reason, close if appropriate.
38+
- **Labeled `Waiting for Info`** with no response for 2+ months — close as stale.
39+
- **Labeled `Likely Not a BNB Issue`** — close with a redirect to the correct project.
40+
41+
### Old version issues
42+
43+
Check the bitsandbytes version in the report. Key version boundaries:
44+
- **< 0.43.0**: Old `cuda_setup/main.py` system (replaced). No official Windows support. Fragile CUDA detection.
45+
- **< 0.45.0**: Before improved C library error messaging (PR #1615).
46+
47+
If the issue was clearly caused by old-version behavior that's been fixed, close it.
48+
49+
### Pattern matching
50+
51+
Read the issue body and tracebacks. Compare against the patterns in `agents/issue_patterns.md`:
52+
- Legacy CUDA setup errors
53+
- Windows pre-support issues
54+
- Missing shared library mismatches
55+
- C library load failures showing as `NameError`/`NoneType`
56+
- Third-party app issues
57+
- Transformers version mismatches
58+
- Questions filed as bugs
59+
- FSDP optimizer issues (duplicate of #1633)
60+
61+
### Stale issues
62+
63+
Issues with no activity for 6+ months, no maintainer engagement, and insufficient information to reproduce. Especially:
64+
- No bitsandbytes version specified
65+
- No traceback or only screenshots
66+
- Reporter never responded to requests for info
67+
- Zero comments, zero reactions
68+
69+
## Step 3: Deep-Dive Suspected Duplicates
70+
71+
When two or more issues look related:
72+
73+
```bash
74+
# Full context on both
75+
python3 ~/git/lab_tools/github/query_issues.py show <NUMBER1> <NUMBER2>
76+
77+
# Find more related issues
78+
python3 ~/git/lab_tools/github/query_issues.py related <NUMBER> -v
79+
80+
# Check if already resolved
81+
python3 ~/git/lab_tools/github/query_issues.py related <NUMBER> --state closed -v
82+
```
83+
84+
Before closing a duplicate, verify:
85+
1. The canonical issue is still open (or was resolved with a fix that covers this too).
86+
2. The duplicate doesn't contain unique information that should be preserved — if it does, add a comment on the canonical issue referencing the useful info before closing.
87+
88+
## Step 4: Close with Helpful Comments
89+
90+
Every closed issue should get a comment that:
91+
1. **Explains why** it's being closed (not just "closing as stale").
92+
2. **Points to the fix or canonical issue** if applicable.
93+
3. **Invites reopening** if the problem persists on the latest version.
94+
95+
Use the closing templates from `agents/issue_patterns.md` as a starting point, but tailor them to the specific issue. Mention the actual version the user was on if known, reference the specific fix if one exists.
96+
97+
Close issues using the `gh` CLI:
98+
99+
```bash
100+
gh issue close <NUMBER> --comment "Your comment here"
101+
```
102+
103+
For duplicates, use the `--reason "not planned"` flag:
104+
105+
```bash
106+
gh issue close <NUMBER> --comment "Closing as duplicate of #XXXX." --reason "not planned"
107+
```
108+
109+
## Step 5: Report Results
110+
111+
After a triage session, output a summary:
112+
- How many issues were closed
113+
- Breakdown by category/pattern
114+
- Any issues that are borderline and need human review
115+
- Any new patterns discovered that should be added to `issue_patterns.md`
116+
117+
## Guidelines
118+
119+
- **Be respectful.** People filed these issues because they were stuck. A clear explanation of what went wrong and how to fix it is valuable even on a closed issue.
120+
- **Don't close genuine bugs.** If there's any chance the issue represents a real bug in current code, leave it open. When in doubt, leave it open.
121+
- **Don't close feature requests** unless they're exact duplicates of another open feature request. Feature requests reflect community demand.
122+
- **Preserve information.** If a duplicate issue contains useful reproduction steps, error details, or workarounds not present in the canonical issue, add a comment on the canonical issue with that info before closing the duplicate.
123+
- **Check the version.** The single most important piece of information is the bitsandbytes version. If it's old and the issue area has been reworked, it's likely closeable.

agents/issue_patterns.md

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
# Common Issue Patterns in bitsandbytes
2+
3+
This document catalogs recurring issue patterns across the bitsandbytes issue tracker. Use it during issue triage to quickly identify duplicates, stale reports, and issues that can be closed.
4+
5+
## CUDA Setup / Library Loading
6+
7+
These are the single largest category of issues. Most are environment problems on old bitsandbytes versions, not code bugs.
8+
9+
### Legacy `cuda_setup/main.py` (versions 0.41.x–0.42.x)
10+
11+
**How to identify:** Tracebacks reference `bitsandbytes/cuda_setup/main.py` (line 166 or 167). Error output includes `UserWarning: Welcome to bitsandbytes. For bug reports, please run python -m bitsandbytes` in the old format. The import chain goes through `bitsandbytes/research/__init__.py``modules.py``GlobalOptimManager``cextension.py` line 20.
12+
13+
**What happened:** Versions 0.41.x–0.42.x used a fragile CUDA detection system in `cuda_setup/main.py` that searched for `libcudart.so` in environment paths. It had bugs:
14+
- It re-initialized `cuda_runtime_libs = set()` after already populating it from `CONDA_PREFIX` and `LD_LIBRARY_PATH`, discarding valid search results.
15+
- It failed in conda environments, Docker containers, and systems with multiple CUDA versions.
16+
- It searched for Linux `.so` files on Windows.
17+
- Error messages gave Linux-specific advice (`sudo ldconfig`, `export LD_LIBRARY_PATH`) regardless of platform.
18+
19+
**Resolution:** The entire `cuda_setup/main.py` module was replaced in v0.43.0 with a new library loading mechanism in `cextension.py`. Users should upgrade to the latest version.
20+
21+
**Closing template:**
22+
> Closing this issue. The CUDA detection system (`cuda_setup/main.py`) used in bitsandbytes 0.41.x–0.42.x was fragile and had known bugs — it could fail to find CUDA libraries even when they were correctly installed, particularly in conda environments, Docker containers, and systems with multiple CUDA versions. That entire module was replaced starting in v0.43.0 with a more robust library loading mechanism.
23+
>
24+
> If you're still hitting CUDA setup problems on the **latest** bitsandbytes (v0.45+), please open a new issue with the output of `python -m bitsandbytes` and your environment details (OS, Python version, PyTorch version, GPU).
25+
26+
### Windows pre-support (before v0.43.0)
27+
28+
**How to identify:** Windows paths in tracebacks, but error messages reference `libcudart.so` (Linux) or `libbitsandbytes_cpu.so` (Linux extension on Windows). The `cuda_setup/` subdirectory path appears. May reference the unofficial `jllllll/bitsandbytes` Windows fork. Error advice includes `sudo ldconfig` or `find / -name libcuda.so` — Linux commands on Windows. The `argument of type 'WindowsPath' is not iterable` error is a strong signal.
29+
30+
**What happened:** Official Windows support was added in v0.43.0. Before that, users relied on unofficial forks or got the Linux-only `.so` builds that don't work on Windows.
31+
32+
**Closing template:**
33+
> Closing this issue. This was reported before official Windows support was added in bitsandbytes v0.43.0. The old CUDA detection system also gave Linux-specific guidance on Windows. Both Windows support and the library loading system have been overhauled in recent releases.
34+
>
35+
> If you're still hitting problems on the **latest** bitsandbytes (v0.45+), please open a new issue with the output of `python -m bitsandbytes` and your environment details.
36+
37+
### Missing `libcusparse.so.11` / shared library mismatch
38+
39+
**How to identify:** `OSError: libcusparse.so.11: cannot open shared object file: No such file or directory`. Or similar errors for `libcusparse.so.12`, `libcublasLt.so.11`, etc.
40+
41+
**What happened:** The bnb binary was compiled against one CUDA version (e.g., 11.x) but the system only has another (e.g., 12.x). The shared library dependencies don't exist. Modern releases ship platform-specific wheels with better CUDA version detection and multiple binary variants.
42+
43+
**Closing template:**
44+
> Closing this issue. The error indicates a mismatch between the CUDA version bitsandbytes was compiled against and the system CUDA libraries. Modern bitsandbytes releases (v0.43.0+) ship platform-specific wheels that handle CUDA version detection more reliably.
45+
>
46+
> If you're still hitting this on the **latest** bitsandbytes (v0.45+), please open a new issue with the output of `python -m bitsandbytes` and your environment details.
47+
48+
### C library load failure → `NameError: str2optimizer32bit` / `NoneType` errors
49+
50+
**How to identify:** `NameError: name 'str2optimizer32bit' is not defined`, `AttributeError: 'NoneType' object has no attribute 'cquantize_blockwise_...'`, or `AttributeError: 'NoneType' object has no attribute 'split'` in `cuda_specs.py`. May also show `module 'bitsandbytes' has no attribute 'nn'`.
51+
52+
**What happened:** When the C/CUDA binary fails to load (for any reason — wrong platform, missing deps, version mismatch), the `lib` object is `None` and Python-level dispatch dictionaries are never populated. The resulting errors are confusing symptoms of the real problem. PR #1615 (merged, tracked by #1548) improved error messaging to surface the actual load failure.
53+
54+
**Closing template:**
55+
> Closing this issue. This error is a symptom of the C/CUDA library failing to load — the confusing `NameError`/`AttributeError` was a downstream effect. Error messaging for this case was improved in PR #1615. Please upgrade to the latest bitsandbytes, which will show a clearer error if the library fails to load.
56+
>
57+
> If you're still hitting this on the **latest** bitsandbytes (v0.45+), please open a new issue with the output of `python -m bitsandbytes` and your environment details.
58+
59+
### Unsupported platform / architecture
60+
61+
**How to identify:** Platform is aarch64 (Jetson), ppc64le, or uses a very old GPU (Kepler/compute 3.5). Binary file is missing for the architecture. Error like `libbitsandbytes_cuda122.so: cannot open shared object file`.
62+
63+
**What happened:** Pre-built binaries only cover x86-64 + certain CUDA versions. aarch64 support has improved in recent releases. Kepler (compute 3.5) and ppc64le are not officially supported.
64+
65+
**Closing template:**
66+
> Closing this issue. Pre-built binaries were not available for this platform at the time of reporting. Please check the latest release notes for current platform support. For source builds, see the [installation docs](https://huggingface.co/docs/bitsandbytes/main/en/installation).
67+
68+
## Not bitsandbytes Issues
69+
70+
### Third-party application issues
71+
72+
**How to identify:** User is running Automatic1111, Forge UI, ComfyUI, kohya_ss, or similar Stable Diffusion tools. The error occurs inside bitsandbytes but is caused by the app pinning old bnb versions or misconfiguring the environment. Minimal or no diagnostic info. Often no bnb version specified.
73+
74+
**Resolution:** These are dependency management issues in third-party apps. Close with a note to report to the app's issue tracker and upgrade bitsandbytes.
75+
76+
**Closing template:**
77+
> Closing this issue. This appears to be a dependency/environment issue in the application you're using rather than a bitsandbytes bug. Please ensure the application is using the latest bitsandbytes version (v0.45+). If the issue persists, reporting it to the application's own issue tracker may be more effective.
78+
79+
### Transformers version mismatch
80+
81+
**How to identify:** `ImportError: Using 'bitsandbytes' 8-bit quantization requires Accelerate: pip install accelerate and the latest version of bitsandbytes`. This error message comes from the `transformers` library, not from bitsandbytes.
82+
83+
**What happened:** Older `transformers` versions had a version check that could emit this misleading error even when both accelerate and bitsandbytes were installed. Upgrading `transformers` resolves it.
84+
85+
**Closing template:**
86+
> Closing this issue. This error message originates from the `transformers` library, not from bitsandbytes. Upgrading `transformers` to the latest version resolves it.
87+
88+
### Unrelated errors filed against bitsandbytes
89+
90+
**How to identify:** The traceback's root cause is in another library (sentencepiece, diffusers, ONNX, etc.) but the user filed it here because bitsandbytes appeared somewhere in their stack. Look at the actual exception — if it's about tokenizer parsing, model loading from a different library, or API changes in diffusers/transformers, it's not a bnb issue.
91+
92+
**Closing template:**
93+
> Closing this issue. The error originates in [library name], not in bitsandbytes. Please report it to the appropriate issue tracker.
94+
95+
## Other Recurring Patterns
96+
97+
### Undefined symbol errors (old builds)
98+
99+
**How to identify:** `undefined symbol: cadam32bit_grad_fp32` or `undefined symbol: cdequantize_blockwise_fp32`. Occurs with old builds or version mismatches between the compiled C library and the Python package.
100+
101+
**Resolution:** Upgrade to the latest version. If building from source, do a clean build.
102+
103+
### Questions filed as bugs
104+
105+
**How to identify:** The issue asks about NF4 internals (offset value, data format, quantile bins), how quantization works, or how to use a feature. Often has the `Question` label. No actual error or bug report.
106+
107+
**Resolution:** If answered in comments or by the reporter themselves, close. If useful, convert to a discussion. Consider whether the question reveals a documentation gap worth addressing.
108+
109+
### FSDP + bitsandbytes optimizers
110+
111+
**How to identify:** Errors when using bnb optimizers (Adam8bit, PagedAdamW, etc.) with FSDP or FSDP2. Common errors include `AssertionError` in `_convert_all_state_info`, `AttributeError: 'int' object has no attribute 'cpu'`, or `illegal memory access`.
112+
113+
**Current status:** FSDP support for bnb optimizers is a known gap. The maintainer has stated this repeatedly. Track via #1633 (open, Contributions Welcome). Historical context in #89 (closed).
114+
115+
**Resolution:** Close as duplicate of #1633, noting that FSDP optimizer support is not yet available.

0 commit comments

Comments
 (0)