Skip to content

Commit 3e2a67b

Browse files
committed
fix: skip _fix_blob_seq_ids sqlite open on already-migrated palaces (#1090)
Opening chroma.sqlite3 via Python's sqlite3.connect() against a live ChromaDB 1.5.x WAL-mode database leaves state that segfaults the next PersistentClient call — the same failure mode tracked at #1090. _fix_blob_seq_ids runs unconditionally on every make_client() call, so every fresh process (MCP server, stop hook, CLI) re-triggers the sqlite open → corrupt → segfault cycle on palaces that have already completed the 0.6.x → 1.5.x seq_id migration. Guard with a .blob_seq_ids_migrated marker file in the palace directory: - If marker exists, return immediately — skip sqlite entirely - After successful migration (or confirmation that no BLOBs remain), write the marker so subsequent opens take the fast path - Palaces that never had BLOB seq_ids also get the marker on first open, so they too avoid the redundant sqlite open after that - Already-migrated palaces can touch the marker manually to opt in Test plan: Direct test — run _fix_blob_seq_ids twice against a fresh palace; second call returns immediately because marker exists. 1094 existing tests pass.
1 parent 8ac98f0 commit 3e2a67b

1 file changed

Lines changed: 22 additions & 0 deletions

File tree

mempalace/backends/chroma.py

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -131,6 +131,9 @@ def quarantine_stale_hnsw(palace_path: str, stale_seconds: float = 3600.0) -> li
131131
return moved
132132

133133

134+
_BLOB_FIX_MARKER = ".blob_seq_ids_migrated"
135+
136+
134137
def _fix_blob_seq_ids(palace_path: str) -> None:
135138
"""Fix ChromaDB 0.6.x -> 1.5.x migration bug: BLOB seq_ids -> INTEGER.
136139
@@ -140,10 +143,20 @@ def _fix_blob_seq_ids(palace_path: str) -> None:
140143
type INTEGER) is not compatible with SQL type BLOB".
141144
142145
Must run BEFORE PersistentClient is created (the compactor fires on init).
146+
147+
Opening a Python sqlite3 connection against a ChromaDB 1.5.x WAL-mode
148+
database leaves state that segfaults the next PersistentClient call. After
149+
the migration has run once successfully, a marker file is written so
150+
subsequent opens skip the sqlite connection entirely. Already-migrated
151+
palaces can touch the marker manually to opt into the fast path.
143152
"""
144153
db_path = os.path.join(palace_path, "chroma.sqlite3")
145154
if not os.path.isfile(db_path):
146155
return
156+
marker = os.path.join(palace_path, _BLOB_FIX_MARKER)
157+
if os.path.isfile(marker):
158+
return
159+
migrated = False
147160
try:
148161
with sqlite3.connect(db_path) as conn:
149162
for table in ("embeddings", "max_seq_id"):
@@ -158,9 +171,18 @@ def _fix_blob_seq_ids(palace_path: str) -> None:
158171
updates = [(int.from_bytes(blob, byteorder="big"), rowid) for rowid, blob in rows]
159172
conn.executemany(f"UPDATE {table} SET seq_id = ? WHERE rowid = ?", updates)
160173
logger.info("Fixed %d BLOB seq_ids in %s", len(updates), table)
174+
migrated = True
161175
conn.commit()
162176
except Exception:
163177
logger.exception("Could not fix BLOB seq_ids in %s", db_path)
178+
return
179+
# Write marker whether or not rows needed migration — the palace is now
180+
# confirmed to be in the INTEGER-seq_id state and future opens can skip the
181+
# sqlite3.connect() entirely.
182+
try:
183+
open(marker, "a").close()
184+
except OSError:
185+
logger.exception("Could not write migration marker %s", marker)
164186

165187

166188
# ---------------------------------------------------------------------------

0 commit comments

Comments
 (0)