Skip to content

Commit 559e43b

Browse files
bensigbensig
authored
Merge pull request MemPalace#502 from milla-jovovich/fix/chromadb-version-migration
feat: mempalace migrate — recover palaces from different ChromaDB versions
2 parents a036b43 + 2d7d7e0 commit 559e43b

9 files changed

Lines changed: 408 additions & 4 deletions

File tree

‎.github/CODEOWNERS‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Default owners for everything
2+
* @milla-jovovich @bensig @igorls
3+
4+
# Core library
5+
mempalace/ @milla-jovovich @bensig
6+
7+
# CI and workflows
8+
.github/ @bensig
9+
10+
# Plugins and integrations
11+
.claude-plugin/ @bensig
12+
.codex-plugin/ @bensig
13+
integrations/ @bensig

‎.github/dependabot.yml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
version: 2
2+
updates:
3+
- package-ecosystem: "pip"
4+
directory: "/"
5+
schedule:
6+
interval: "weekly"
7+
open-pull-requests-limit: 5
8+
- package-ecosystem: "github-actions"
9+
directory: "/"
10+
schedule:
11+
interval: "weekly"
12+
open-pull-requests-limit: 3

‎.github/workflows/ci.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ jobs:
1818
with:
1919
python-version: ${{ matrix.python-version }}
2020
- run: pip install -e ".[dev]"
21-
- run: python -m pytest tests/ -v --ignore=tests/benchmarks --cov=mempalace --cov-report=term-missing --cov-fail-under=80
21+
- run: python -m pytest tests/ -v --ignore=tests/benchmarks --cov=mempalace --cov-report=term-missing --cov-fail-under=80 --durations=10
2222

2323
test-windows:
2424
runs-on: windows-latest
@@ -28,7 +28,7 @@ jobs:
2828
with:
2929
python-version: "3.9"
3030
- run: pip install -e ".[dev]"
31-
- run: python -m pytest tests/ -v --ignore=tests/benchmarks --cov=mempalace --cov-report=term-missing --cov-fail-under=80
31+
- run: python -m pytest tests/ -v --ignore=tests/benchmarks --cov=mempalace --cov-report=term-missing --cov-fail-under=80 --durations=10
3232

3333
test-macos:
3434
runs-on: macos-latest
@@ -38,7 +38,7 @@ jobs:
3838
with:
3939
python-version: "3.9"
4040
- run: pip install -e ".[dev]"
41-
- run: python -m pytest tests/ -v --ignore=tests/benchmarks --cov=mempalace --cov-report=term-missing --cov-fail-under=80
41+
- run: python -m pytest tests/ -v --ignore=tests/benchmarks --cov=mempalace --cov-report=term-missing --cov-fail-under=80 --durations=10
4242
lint:
4343
runs-on: ubuntu-latest
4444
steps:

‎.gitignore‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,3 +6,30 @@ __pycache__/
66
.pytest_cache/
77
mempal.yaml
88
.a5c/
9+
10+
# Environment
11+
.env
12+
.env.*
13+
14+
# OS
15+
.DS_Store
16+
Thumbs.db
17+
18+
# IDEs
19+
.idea/
20+
.vscode/
21+
*.swp
22+
*.swo
23+
*~
24+
25+
# Coverage
26+
htmlcov/
27+
.coverage
28+
coverage.xml
29+
30+
# Virtual environments
31+
.venv/
32+
venv/
33+
34+
# ChromaDB local data
35+
*.sqlite3-journal

‎AGENTS.md‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
# AGENTS.md
2+
3+
> How to build, test, and contribute to MemPalace.
4+
5+
## Setup
6+
7+
```bash
8+
pip install -e ".[dev]"
9+
```
10+
11+
## Commands
12+
13+
```bash
14+
# Run tests
15+
python -m pytest tests/ -v --ignore=tests/benchmarks
16+
17+
# Run tests with coverage
18+
python -m pytest tests/ -v --ignore=tests/benchmarks --cov=mempalace --cov-report=term-missing
19+
20+
# Lint
21+
ruff check .
22+
23+
# Format
24+
ruff format .
25+
26+
# Format check (CI mode)
27+
ruff format --check .
28+
```
29+
30+
## Project structure
31+
32+
```
33+
mempalace/
34+
├── mcp_server.py # MCP server — all read/write tools
35+
├── miner.py # Project file miner
36+
├── convo_miner.py # Conversation transcript miner
37+
├── searcher.py # Semantic search
38+
├── knowledge_graph.py # Temporal entity-relationship graph (SQLite)
39+
├── palace.py # Shared palace operations (ChromaDB access)
40+
├── config.py # Configuration + input validation
41+
├── normalize.py # Transcript format detection + normalization
42+
├── cli.py # CLI dispatcher
43+
├── dialect.py # AAAK compression dialect
44+
├── palace_graph.py # Room traversal + cross-wing tunnels
45+
├── hooks_cli.py # Hook system for auto-save
46+
└── version.py # Single source of truth for version
47+
```
48+
49+
## Conventions
50+
51+
- **Python style**: snake_case for functions/variables, PascalCase for classes
52+
- **Linter**: ruff with E/F/W rules
53+
- **Formatter**: ruff format, double quotes
54+
- **Commits**: conventional commits (`fix:`, `feat:`, `test:`, `docs:`, `ci:`)
55+
- **Tests**: `tests/test_*.py`, fixtures in `tests/conftest.py`
56+
- **Coverage**: 85% threshold (80% on Windows due to ChromaDB file lock cleanup)
57+
58+
## Architecture
59+
60+
```
61+
User → CLI / MCP Server → ChromaDB (vector store) + SQLite (knowledge graph)
62+
63+
Palace structure:
64+
WING (person/project)
65+
└── ROOM (topic)
66+
└── DRAWER (verbatim text chunk)
67+
68+
Knowledge Graph:
69+
ENTITY → PREDICATE → ENTITY (with valid_from / valid_to dates)
70+
```
71+
72+
## Key files for common tasks
73+
74+
- **Adding an MCP tool**: `mempalace/mcp_server.py` — add handler function + TOOLS dict entry
75+
- **Changing search**: `mempalace/searcher.py`
76+
- **Modifying mining**: `mempalace/miner.py` (project files) or `mempalace/convo_miner.py` (transcripts)
77+
- **Input validation**: `mempalace/config.py` — `sanitize_name()` / `sanitize_content()`
78+
- **Tests**: mirror source structure in `tests/test_<module>.py`

‎docs/schema.sql‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
-- MemPalace Knowledge Graph Schema
2+
-- SQLite database at ~/.mempalace/knowledge_graph.db
3+
4+
CREATE TABLE IF NOT EXISTS entities (
5+
id TEXT PRIMARY KEY,
6+
name TEXT NOT NULL,
7+
type TEXT DEFAULT 'unknown',
8+
properties TEXT DEFAULT '{}'
9+
);
10+
11+
CREATE TABLE IF NOT EXISTS triples (
12+
id TEXT PRIMARY KEY,
13+
subject TEXT NOT NULL,
14+
predicate TEXT NOT NULL,
15+
object TEXT NOT NULL,
16+
valid_from TEXT,
17+
valid_to TEXT,
18+
confidence REAL DEFAULT 1.0,
19+
source_closet TEXT,
20+
source_file TEXT
21+
);
22+
23+
CREATE TABLE IF NOT EXISTS attributes (
24+
entity_id TEXT NOT NULL,
25+
key TEXT NOT NULL,
26+
value TEXT,
27+
valid_from TEXT,
28+
valid_to TEXT,
29+
PRIMARY KEY (entity_id, key, valid_from)
30+
);
31+
32+
-- Indexes
33+
CREATE INDEX IF NOT EXISTS idx_triples_subject ON triples(subject);
34+
CREATE INDEX IF NOT EXISTS idx_triples_object ON triples(object);
35+
CREATE INDEX IF NOT EXISTS idx_triples_predicate ON triples(predicate);
36+
CREATE INDEX IF NOT EXISTS idx_triples_valid ON triples(valid_from, valid_to);

‎mempalace/cli.py‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -150,6 +150,14 @@ def cmd_split(args):
150150
sys.argv = old_argv
151151

152152

153+
def cmd_migrate(args):
154+
"""Migrate palace from a different ChromaDB version."""
155+
from .migrate import migrate
156+
157+
palace_path = os.path.expanduser(args.palace) if args.palace else MempalaceConfig().palace_path
158+
migrate(palace_path=palace_path, dry_run=args.dry_run)
159+
160+
153161
def cmd_status(args):
154162
from .miner import status
155163

@@ -531,6 +539,17 @@ def main():
531539
)
532540

533541
# status
542+
# migrate
543+
p_migrate = sub.add_parser(
544+
"migrate",
545+
help="Migrate palace from a different ChromaDB version (fixes 3.0.0 → 3.1.0 upgrade)",
546+
)
547+
p_migrate.add_argument(
548+
"--dry-run",
549+
action="store_true",
550+
help="Show what would be migrated without changing anything",
551+
)
552+
534553
sub.add_parser("status", help="Show what's been filed")
535554

536555
args = parser.parse_args()
@@ -565,6 +584,7 @@ def main():
565584
"compress": cmd_compress,
566585
"wake-up": cmd_wakeup,
567586
"repair": cmd_repair,
587+
"migrate": cmd_migrate,
568588
"status": cmd_status,
569589
}
570590
dispatch[args.command](args)

0 commit comments

Comments
 (0)
⚡