Skip to content

Commit cf8a6a5

Browse files
jpheinlealvona
authored andcommitted
fix: skip _fix_blob_seq_ids sqlite open on already-migrated palaces (MemPalace#1090)
Opening chroma.sqlite3 via Python's sqlite3.connect() against a live ChromaDB 1.5.x WAL-mode database leaves state that segfaults the next PersistentClient call — the same failure mode tracked at MemPalace#1090. _fix_blob_seq_ids runs unconditionally on every make_client() call, so every fresh process (MCP server, stop hook, CLI) re-triggers the sqlite open → corrupt → segfault cycle on palaces that have already completed the 0.6.x → 1.5.x seq_id migration. Guard with a .blob_seq_ids_migrated marker file in the palace directory: - If marker exists, return immediately — skip sqlite entirely - After successful migration (or confirmation that no BLOBs remain), write the marker so subsequent opens take the fast path - Palaces that never had BLOB seq_ids also get the marker on first open, so they too avoid the redundant sqlite open after that - Already-migrated palaces can touch the marker manually to opt in Test plan: Direct test — run _fix_blob_seq_ids twice against a fresh palace; second call returns immediately because marker exists. 1094 existing tests pass.
1 parent ef01a69 commit cf8a6a5

1 file changed

Lines changed: 20 additions & 0 deletions

File tree

mempalace/backends/chroma.py

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -239,6 +239,9 @@ def _pin_hnsw_threads(collection) -> None:
239239
logger.debug("_pin_hnsw_threads modify failed", exc_info=True)
240240

241241

242+
_BLOB_FIX_MARKER = ".blob_seq_ids_migrated"
243+
244+
242245
def _fix_blob_seq_ids(palace_path: str) -> None:
243246
"""Fix ChromaDB 0.6.x -> 1.5.x migration bug: BLOB seq_ids -> INTEGER.
244247
@@ -248,10 +251,19 @@ def _fix_blob_seq_ids(palace_path: str) -> None:
248251
type INTEGER) is not compatible with SQL type BLOB".
249252
250253
Must run BEFORE PersistentClient is created (the compactor fires on init).
254+
255+
Opening a Python sqlite3 connection against a ChromaDB 1.5.x WAL-mode
256+
database leaves state that segfaults the next PersistentClient call. After
257+
the migration has run once successfully, a marker file is written so
258+
subsequent opens skip the sqlite connection entirely. Already-migrated
259+
palaces can touch the marker manually to opt into the fast path.
251260
"""
252261
db_path = os.path.join(palace_path, "chroma.sqlite3")
253262
if not os.path.isfile(db_path):
254263
return
264+
marker = os.path.join(palace_path, _BLOB_FIX_MARKER)
265+
if os.path.isfile(marker):
266+
return
255267
try:
256268
with sqlite3.connect(db_path) as conn:
257269
for table in ("embeddings", "max_seq_id"):
@@ -269,6 +281,14 @@ def _fix_blob_seq_ids(palace_path: str) -> None:
269281
conn.commit()
270282
except Exception:
271283
logger.exception("Could not fix BLOB seq_ids in %s", db_path)
284+
return
285+
# Write marker whether or not rows needed migration — the palace is now
286+
# confirmed to be in the INTEGER-seq_id state and future opens can skip the
287+
# sqlite3.connect() entirely.
288+
try:
289+
open(marker, "a").close()
290+
except OSError:
291+
logger.exception("Could not write migration marker %s", marker)
272292

273293

274294
# ---------------------------------------------------------------------------

0 commit comments

Comments
 (0)