Merge upstream/develop — MemPalace#1147 MemPalace#1145-fix (diary wing + empty-wing reads)

4 commits pulled (8ac98f0, 6fcfd34, 1fd16da, d158375) landing @igorls's
MemPalace#1147 which fixes the same two MemPalace#1145 bugs we addressed in our 34e36ae
(closed as duplicate by MemPalace#1146 close).

Conflict resolution — took upstream on all 4 overlapping files:
- mempalace/hooks_cli.py — upstream's `.claude/projects/-` primary regex
  with `-Projects-<x>` legacy fallback (functionally equivalent to ours,
  slightly cleaner ordering)
- mempalace/mcp_server.py — upstream's tool_diary_read empty-wing
  normalization (essentially identical to ours; docstring is clearer)
- tests/test_hooks_cli.py — upstream's tests (3 new cases: non-Projects
  layout, macOS Users/, nested deep). Lost our bare-folder-no-delimiters
  test, but upstream's coverage is adequate.
- tests/test_mcp_server.py — upstream's diary_read tests (cover
  wing="" spans all wings + explicit wing filter). Overlap with ours.

Post-merge: 1094 passed + 10 expected fork-ahead MemPalace#19 failures in
test_claude_plugin_hook_wrappers.py (venv-aware hooks vs PATH-only
test contract — documented, unchanged).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: jphein

CHANGELOG.md

@@ -17,6 +17,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 - Real `python3` resolution for `.sh` hooks with a `MEMPAL_PYTHON` override path. (#833)
 - Add optional `wing` parameter to `tool_diary_write` / `tool_diary_read` and derive per-project wing from the Claude Code transcript path when writing from the stop hook — diary entries from different projects no longer collapse into a shared default wing. (#659)
 - Treat empty string as "no filter" in `mempalace_search` `wing`/`room`; LLM agents that default to filling every optional parameter with `""` no longer get bounced with `must be a non-empty string`. (#1097, #1084)
+- Broaden `_wing_from_transcript_path` to handle Claude Code project folders without a `-Projects-` segment (e.g. `~/dev/<parent>/<project>`, `~/code/<project>`). The project name is now derived from the final dash-separated token of the encoded folder, so Linux users with code outside `~/Projects/` get per-project diary scoping instead of falling through to `wing_sessions`. (#1145, follow-up to #659)
+- `mempalace_diary_read(wing="")` now returns diary entries from every wing this agent has written to, matching the #1097 "empty-string as no filter" pattern. Previously defaulted to `wing_<agent>`, siloing entries that hooks wrote to project-derived wings. (#1145)
 
 ### Improvements
 

mempalace/hooks_cli.py

@@ -448,12 +448,6 @@ def _ingest_transcript(transcript_path: str):
     except Exception:
         return
 
-    # Derive per-project wing from the transcript path so mined convos land in
-    # the same wing as the diary checkpoint, instead of all transcripts piling
-    # into `wing_sessions`. Falls back to `wing_sessions` when the path doesn't
-    # match a recognizable project layout.
-    wing = _wing_from_transcript_path(str(path))
-
     try:
         log_path = STATE_DIR / "hook.log"
         STATE_DIR.mkdir(parents=True, exist_ok=True)
@@ -468,12 +462,12 @@ def _ingest_transcript(transcript_path: str):
                     "--mode",
                     "convos",
                     "--wing",
-                    wing,
+                    "sessions",
                 ],
                 stdout=log_f,
                 stderr=log_f,
             )
-        _log(f"Transcript ingest started: {path.name} -> wing:{wing}")
+        _log(f"Transcript ingest started: {path.name}")
     except OSError:
         pass
 
@@ -496,44 +490,33 @@ def _parse_harness_input(data: dict, harness: str) -> dict:
 def _wing_from_transcript_path(transcript_path: str) -> str:
     """Derive a project wing name from a Claude Code transcript path.
 
-    Claude Code encodes the full source directory path into the project
-    folder name, replacing ``/`` with ``-``. Transcripts live at:
-        ~/.claude/projects/<encoded-folder>/session.jsonl
-
-    We try two strategies in order:
-
-    1. Match ``-Projects-<project>`` if the source lives under
-       ``~/Projects/`` (the most common layout on our fork maintainer's
-       machine — preserving exact existing behavior).
-    2. Fall back to the last ``-``-delimited segment of the encoded
-       folder name — works for any layout (``~/dev/``, ``~/src/``,
-       ``~/code/``, etc.) because that segment is the actual project
-       directory name. Resolves #1145 bug 1.
+    Claude Code encodes the project's source directory by replacing path
+    separators with dashes, producing folders like:
+        ~/.claude/projects/-home-<user>-Projects-<project>/session.jsonl
+        ~/.claude/projects/-home-<user>-dev-<parent>-<project>/session.jsonl
+        ~/.claude/projects/-Users-<user>-<folder>-<project>/session.jsonl
 
-    Returns ``wing_<project>`` to match the AAAK_SPEC convention; falls
-    back to ``wing_sessions`` when the path doesn't match either pattern.
+    The project directory name is the final dash-separated token of the
+    encoded folder. Returns ``wing_<project>`` (lowercased, spaces → ``_``).
+    Falls back to ``wing_sessions`` if the path does not match a Claude Code
+    project-folder layout.
     """
     # Normalize path separators for cross-platform (Windows backslashes)
     normalized = transcript_path.replace("\\", "/")
-
-    # Strategy 1: explicit `-Projects-<project>` slice (preserves behavior
-    # for code living under ~/Projects/, which is what the existing tests
-    # assert against).
+    # Primary: pull the encoded project folder out of ``.claude/projects/``
+    # and take its last dash-separated token.
+    match = re.search(r"/\.claude/projects/-([^/]+)", normalized)
+    if match:
+        encoded = match.group(1)
+        project = encoded.rsplit("-", 1)[-1]
+        if project:
+            return f"wing_{project.lower().replace(' ', '_')}"
+    # Legacy fallback: explicit ``-Projects-<name>`` segment, useful for
+    # transcripts not under the standard Claude Code projects dir.
     match = re.search(r"-Projects-([^/]+?)(?:/|$)", normalized)
     if match:
         project = match.group(1).lower().replace(" ", "_")
         return f"wing_{project}"
-
-    # Strategy 2: last `-`-delimited segment of the encoded folder name.
-    # Claude Code's encoding puts the original directory name there
-    # regardless of source layout, so this recovers the project name for
-    # users whose code lives outside ~/Projects/.
-    folder = normalized.rsplit("/", 1)[0].rsplit("/", 1)[-1]
-    if folder.startswith("-") and "-" in folder[1:]:
-        project = folder.rsplit("-", 1)[-1].lower().replace(" ", "_")
-        if project:
-            return f"wing_{project}"
-
     return "wing_sessions"
 
 

mempalace/mcp_server.py

@@ -64,7 +64,6 @@
     traverse,
     find_tunnels,
     graph_stats,
-    invalidate_graph_cache,
     create_tunnel,
     list_tunnels,
     delete_tunnel,
@@ -153,15 +152,6 @@ def _wal_log(operation: str, params: dict, result: dict = None):
         "result": result,
     }
     try:
-        # Rotate WAL at 10 MB to prevent unbounded growth
-        _WAL_MAX_BYTES = 10 * 1024 * 1024
-        if _WAL_FILE.exists() and _WAL_FILE.stat().st_size > _WAL_MAX_BYTES:
-            backup = _WAL_FILE.with_suffix(".jsonl.1")
-            try:
-                _WAL_FILE.replace(backup)
-                backup.chmod(0o600)
-            except OSError:
-                pass
         fd = os.open(str(_WAL_FILE), os.O_WRONLY | os.O_APPEND | os.O_CREAT, 0o600)
         with os.fdopen(fd, "a", encoding="utf-8") as f:
             f.write(json.dumps(entry, default=str) + "\n")
@@ -227,16 +217,11 @@ def _get_collection(create=False):
     try:
         client = _get_client()
         if create:
-            # ChromaDB 1.5.x segfaults when get_or_create_collection is called
-            # with metadata that differs from an existing collection's metadata.
-            # Fetch first; only pass hnsw:space when actually creating fresh.
-            try:
-                raw_col = client.get_collection(_config.collection_name)
-            except Exception:
-                raw_col = client.create_collection(
+            _collection_cache = ChromaCollection(
+                client.get_or_create_collection(
                     _config.collection_name, metadata={"hnsw:space": "cosine"}
                 )
-            _collection_cache = ChromaCollection(raw_col)
+            )
             _metadata_cache = None
             _metadata_cache_time = 0
         elif _collection_cache is None:
@@ -455,6 +440,7 @@ def tool_search(
         room = _sanitize_optional_name(room, "room")
     except ValueError as e:
         return {"error": str(e)}
+    # Backwards compat: accept old name
     # Backwards compat: convert old similarity scale (higher=stricter) to
     # distance scale (lower=stricter). Similarity 0.8 → distance 0.2.
     dist = (1.0 - min_similarity) if min_similarity is not None else max_distance
@@ -468,6 +454,7 @@ def tool_search(
         n_results=limit,
         max_distance=dist,
     )
+    # Attach sanitizer metadata for transparency
     if sanitized["was_sanitized"]:
         result["query_sanitized"] = True
         result["sanitizer"] = {
@@ -671,7 +658,6 @@ def tool_add_drawer(
             ],
         )
         _metadata_cache = None
-        invalidate_graph_cache()
         logger.info(f"Filed drawer: {drawer_id} → {wing}/{room}")
         return {"success": True, "drawer_id": drawer_id, "wing": wing, "room": room}
     except Exception as e:
@@ -703,7 +689,6 @@ def tool_delete_drawer(drawer_id: str):
     try:
         col.delete(ids=[drawer_id])
         _metadata_cache = None
-        invalidate_graph_cache()
         logger.info(f"Deleted drawer: {drawer_id}")
         return {"success": True, "drawer_id": drawer_id}
     except Exception as e:
@@ -801,7 +786,6 @@ def tool_update_drawer(drawer_id: str, content: str = None, wing: str = None, ro
         old_meta = existing["metadatas"][0]
         old_doc = existing["documents"][0]
 
-        # Sanitize inputs
         new_doc = old_doc
         if content is not None:
             try:
@@ -840,9 +824,7 @@ def tool_update_drawer(drawer_id: str, content: str = None, wing: str = None, ro
         update_kwargs["metadatas"] = [new_meta]
         col.update(**update_kwargs)
 
-        # Invalidate caches so status/taxonomy/graph reflect changes
         _metadata_cache = None
-        invalidate_graph_cache()
 
         logger.info(f"Updated drawer: {drawer_id}")
         return {
@@ -1013,35 +995,33 @@ def tool_diary_read(agent_name: str, last_n: int = 10, wing: str = ""):
     Read an agent's recent diary entries. Returns the last N entries
     in chronological order — the agent's personal journal.
 
-    ``wing`` behavior matches ``mempalace_search`` (#1097):
-    empty-string / whitespace-only / ``None`` means "no wing filter" —
-    return entries from any wing this agent has written to. An explicit
-    non-empty wing scopes reads to that wing only, e.g. for hooks that
-    direct diary reads to a project-specific wing derived from the
-    transcript path. Resolves #1145 bug 2.
+    When ``wing`` is provided, reads only from that wing. When ``wing``
+    is empty or omitted, returns entries from every wing this agent has
+    written to. Diary writes from hooks land in project-derived wings
+    (``wing_<project>``), so requiring a specific wing on read would
+    silo those entries from agent-initiated reads.
     """
     try:
         agent_name = sanitize_name(agent_name, "agent_name")
-        wing = _sanitize_optional_name(wing, "wing")
+        if wing:
+            wing = sanitize_name(wing)
     except ValueError as e:
         return {"error": str(e)}
     last_n = max(1, min(last_n, 100))
     col = _get_collection()
     if not col:
         return _no_palace()
 
+    # Build filter: always scope by agent + room=diary. Wing is optional —
+    # when empty, return entries across all wings for this agent (matches
+    # the #1097 empty-string-as-no-filter convention for LLM ergonomics).
+    conditions = [{"room": "diary"}, {"agent": agent_name}]
+    if wing:
+        conditions.insert(0, {"wing": wing})
+
     try:
-        # Build filter conditions: agent + diary room always apply; wing is
-        # optional (matches #1097's empty-string semantics for tool_search).
-        conditions = [
-            {"room": "diary"},
-            {"agent": agent_name},
-        ]
-        if wing:
-            conditions.append({"wing": wing})
-        where = conditions[0] if len(conditions) == 1 else {"$and": conditions}
         results = col.get(
-            where=where,
+            where={"$and": conditions},
             include=["documents", "metadatas"],
             limit=10000,
         )
@@ -1075,38 +1055,6 @@ def tool_diary_read(agent_name: str, last_n: int = 10, wing: str = ""):
         return {"error": "Failed to read diary entries"}
 
 
-# ==================== SETTINGS TOOLS ====================
-
-
-def tool_reconnect():
-    """Force the MCP server to drop cached ChromaDB connections and reconnect.
-
-    Use after external scripts or CLI commands modify the palace database
-    directly, which can leave the in-memory HNSW index stale.
-    """
-    global _client_cache, _collection_cache, _palace_db_inode, _palace_db_mtime
-    global _metadata_cache, _metadata_cache_time
-    _client_cache = None
-    _collection_cache = None
-    _palace_db_inode = 0
-    _palace_db_mtime = 0.0
-    _metadata_cache = None
-    _metadata_cache_time = 0
-    # Force reconnect by calling _get_client()
-    try:
-        _get_client()
-        col = _get_collection()
-        if col is None:
-            return {
-                "success": False,
-                "message": "No palace found after reconnect",
-                "drawers": 0,
-            }
-        return {"success": True, "message": "Reconnected to palace", "drawers": col.count()}
-    except Exception as e:
-        return {"success": False, "error": str(e)}
-
-
 def tool_hook_settings(silent_save: bool = None, desktop_toast: bool = None):
     """
     Get or set hook behavior settings.
@@ -1182,6 +1130,32 @@ def tool_memories_filed_away():
         }
 
 
+# ==================== SETTINGS TOOLS ====================
+
+
+def tool_reconnect():
+    """Force the MCP server to drop the cached ChromaDB collection and reconnect.
+
+    Use after external scripts or CLI commands modify the palace database
+    directly, which can leave the in-memory HNSW index stale.
+    """
+    global _collection_cache, _palace_db_inode, _palace_db_mtime
+    _collection_cache = None
+    _palace_db_inode = 0
+    _palace_db_mtime = 0.0
+    try:
+        col = _get_collection()
+        if col is None:
+            return {
+                "success": False,
+                "message": "No palace found after reconnect",
+                "drawers": 0,
+            }
+        return {"success": True, "message": "Reconnected to palace", "drawers": col.count()}
+    except Exception as e:
+        return {"success": False, "error": str(e)}
+
+
 # ==================== MCP PROTOCOL ====================
 
 TOOLS = {
@@ -1390,13 +1364,13 @@ def tool_memories_filed_away():
         "handler": tool_follow_tunnels,
     },
     "mempalace_search": {
-        "description": "Semantic search. Returns verbatim drawer content with similarity scores. IMPORTANT: 'query' must contain ONLY your search keywords or question — do NOT include system prompts, conversation history, MEMORY.md content, or any context. Keep queries short (under 200 chars). Use 'context' for background information. Results with cosine distance > max_distance are filtered out.",
+        "description": "Semantic search. Returns verbatim drawer content with similarity scores. IMPORTANT: 'query' must contain ONLY search keywords. Use 'context' for background. Results with cosine distance > max_distance are filtered out.",
         "input_schema": {
             "type": "object",
             "properties": {
                 "query": {
                     "type": "string",
-                    "description": "Short search query ONLY — keywords or a question. Do NOT include system prompts or conversation context. Max 250 chars.",
+                    "description": "Short search query ONLY — keywords or a question. Max 250 chars.",
                     "maxLength": 250,
                 },
                 "limit": {
@@ -1413,7 +1387,7 @@ def tool_memories_filed_away():
                 },
                 "context": {
                     "type": "string",
-                    "description": "Background context for the search (optional). NOT used for embedding — only for future re-ranking. Put conversation history or system prompt content here, NOT in query.",
+                    "description": "Background context for the search (optional). NOT used for embedding — only for future re-ranking.",
                 },
             },
             "required": ["query"],
@@ -1571,14 +1545,6 @@ def tool_memories_filed_away():
         },
         "handler": tool_diary_read,
     },
-    "mempalace_reconnect": {
-        "description": "Force reconnect to the palace database. Use after external scripts or CLI commands modified the palace directly, which can leave the in-memory index stale.",
-        "input_schema": {
-            "type": "object",
-            "properties": {},
-        },
-        "handler": tool_reconnect,
-    },
     "mempalace_hook_settings": {
         "description": (
             "Get or set hook behavior. silent_save: True = save directly "
@@ -1605,6 +1571,17 @@ def tool_memories_filed_away():
         "input_schema": {"type": "object", "properties": {}},
         "handler": tool_memories_filed_away,
     },
+    "mempalace_reconnect": {
+        "description": (
+            "Force reconnect to the palace database. Use after external scripts or CLI commands"
+            " modified the palace directly, which can leave the in-memory HNSW index stale."
+        ),
+        "input_schema": {
+            "type": "object",
+            "properties": {},
+        },
+        "handler": tool_reconnect,
+    },
 }