Skip to content

Commit fe6a640

Browse files
committed
fix: skip _fix_blob_seq_ids sqlite open on already-migrated palaces (#1090)
Opening chroma.sqlite3 via Python's sqlite3.connect() against a live ChromaDB 1.5.x WAL-mode database leaves state that segfaults the next PersistentClient call — the same failure mode tracked at #1090. _fix_blob_seq_ids runs unconditionally on every make_client() call, so every fresh process (MCP server, stop hook, CLI) re-triggers the sqlite open → corrupt → segfault cycle on palaces that have already completed the 0.6.x → 1.5.x seq_id migration. Guard with a .blob_seq_ids_migrated marker file in the palace directory: - If marker exists, return immediately — skip sqlite entirely - After successful migration (or confirmation that no BLOBs remain), write the marker so subsequent opens take the fast path - Palaces that never had BLOB seq_ids also get the marker on first open, so they too avoid the redundant sqlite open after that - Already-migrated palaces can touch the marker manually to opt in Test plan: Direct test — run _fix_blob_seq_ids twice against a fresh palace; second call returns immediately because marker exists. 1094 existing tests pass.
1 parent 0d9929c commit fe6a640

1 file changed

Lines changed: 20 additions & 0 deletions

File tree

mempalace/backends/chroma.py

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -159,6 +159,9 @@ def _pin_hnsw_threads(collection) -> None:
159159
logger.debug("_pin_hnsw_threads modify failed", exc_info=True)
160160

161161

162+
_BLOB_FIX_MARKER = ".blob_seq_ids_migrated"
163+
164+
162165
def _fix_blob_seq_ids(palace_path: str) -> None:
163166
"""Fix ChromaDB 0.6.x -> 1.5.x migration bug: BLOB seq_ids -> INTEGER.
164167
@@ -168,10 +171,19 @@ def _fix_blob_seq_ids(palace_path: str) -> None:
168171
type INTEGER) is not compatible with SQL type BLOB".
169172
170173
Must run BEFORE PersistentClient is created (the compactor fires on init).
174+
175+
Opening a Python sqlite3 connection against a ChromaDB 1.5.x WAL-mode
176+
database leaves state that segfaults the next PersistentClient call. After
177+
the migration has run once successfully, a marker file is written so
178+
subsequent opens skip the sqlite connection entirely. Already-migrated
179+
palaces can touch the marker manually to opt into the fast path.
171180
"""
172181
db_path = os.path.join(palace_path, "chroma.sqlite3")
173182
if not os.path.isfile(db_path):
174183
return
184+
marker = os.path.join(palace_path, _BLOB_FIX_MARKER)
185+
if os.path.isfile(marker):
186+
return
175187
try:
176188
with sqlite3.connect(db_path) as conn:
177189
for table in ("embeddings", "max_seq_id"):
@@ -189,6 +201,14 @@ def _fix_blob_seq_ids(palace_path: str) -> None:
189201
conn.commit()
190202
except Exception:
191203
logger.exception("Could not fix BLOB seq_ids in %s", db_path)
204+
return
205+
# Write marker whether or not rows needed migration — the palace is now
206+
# confirmed to be in the INTEGER-seq_id state and future opens can skip the
207+
# sqlite3.connect() entirely.
208+
try:
209+
open(marker, "a").close()
210+
except OSError:
211+
logger.exception("Could not write migration marker %s", marker)
192212

193213

194214
# ---------------------------------------------------------------------------

0 commit comments

Comments
 (0)