Skip to content

Commit 1b2b582

Browse files
committed
feat(agents): implement CORTEX cognitive architecture
CORTEX (Cognitive Operations for Reasoning and Task EXecution) is a fractal cognitive architecture where every cognitive operation follows the same pattern. Core components: - cortex-cell: Main workflow supporting 7 modes (categorize, understand, formulate, gather, analyze, synthesize, act) or full cycle - Phase workflows: Individual cognitive phases in phases/ - Atomic capabilities: Block wrappers (gather-files, action-edit, etc.) - Cognitive capabilities: cortex-cell invocations with specific modes Key features: - Embedded capabilities registry in main.yaml for LLM-driven selection - Recursion control with depth limiting and cycle detection - SQLite state management via agent-state-management workflow - Evidence-based analysis with citation requirements
1 parent 299ad91 commit 1b2b582

34 files changed

Lines changed: 5721 additions & 557 deletions

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -229,3 +229,4 @@ src/workflows_mcp/templates/agents/*
229229
!src/workflows_mcp/templates/agents/memory/
230230
!src/workflows_mcp/templates/agents/iteration/
231231
!src/workflows_mcp/templates/agents/investigation/
232+
!src/workflows_mcp/templates/agents/cortex/

docs/guides/recursive-workflows.md

Lines changed: 124 additions & 85 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ Recursive workflows enable complex, multi-level task execution with built-in sta
2525

2626
**Hierarchical State Tracking**: Build multi-level task trees with parent-child relationships, enabling complex workflow decomposition and progress tracking.
2727

28-
**Persistent State**: Task state persists across workflow calls in JSON files, enabling pause/resume, progress monitoring, and audit trails.
28+
**Persistent State**: Task state persists across workflow calls in SQLite databases, enabling pause/resume, progress monitoring, and audit trails with concurrent access support.
2929

3030
**Progress Aggregation**: Automatic rollup of child task status, including completion percentages, pending tasks, and failure detection.
3131

@@ -153,7 +153,7 @@ result = await execute_workflow(
153153
}
154154
)
155155

156-
# State file created at: ~/.workflows/tasks/<trace_id>.json
156+
# State database created at: ~/.workflows/tasks/<trace_id>.db
157157
print(f"State: {result['state']}")
158158
print(f"Processed: {result['processed_count']} items")
159159
```
@@ -180,42 +180,71 @@ await execute_workflow(
180180

181181
## State Management Basics
182182

183-
### State File Structure
183+
### State Database Structure
184+
185+
The state management system persists task hierarchies in SQLite databases at `~/.workflows/tasks/<trace_id>.db`.
186+
187+
**Database Schema:**
188+
189+
```sql
190+
-- Task hierarchy with parent-child relationships
191+
CREATE TABLE tasks (
192+
task_id TEXT PRIMARY KEY,
193+
parent_id TEXT REFERENCES tasks(task_id),
194+
task TEXT NOT NULL,
195+
task_type TEXT DEFAULT '',
196+
status TEXT DEFAULT 'pending',
197+
data JSON DEFAULT '{}',
198+
created_at TEXT NOT NULL,
199+
updated_at TEXT NOT NULL
200+
);
201+
202+
-- Key-value memory storage for workflow state
203+
CREATE TABLE memory (
204+
key TEXT PRIMARY KEY,
205+
value JSON,
206+
updated_at TEXT NOT NULL
207+
);
208+
209+
-- Audit trail for all state operations
210+
CREATE TABLE audit (
211+
id INTEGER PRIMARY KEY AUTOINCREMENT,
212+
timestamp TEXT NOT NULL,
213+
task_id TEXT,
214+
caller TEXT NOT NULL,
215+
action TEXT NOT NULL,
216+
description TEXT,
217+
changes JSON,
218+
parent_id TEXT
219+
);
220+
221+
-- Iteration tracking for loops
222+
CREATE TABLE iterations (
223+
task_id TEXT PRIMARY KEY,
224+
current INTEGER DEFAULT 0,
225+
total INTEGER DEFAULT 0,
226+
cap INTEGER DEFAULT 100,
227+
started_at TEXT,
228+
completed_at TEXT,
229+
checkpoints JSON DEFAULT '[]'
230+
);
231+
```
184232

185-
The state management system persists task hierarchies in JSON files at `~/.workflows/tasks/<trace_id>.json`:
233+
**Example task data (logical structure):**
186234

187235
```json
188236
{
189-
"root_task_id": "task-abc123",
237+
"task_id": "task-abc123",
238+
"task": "PR Review",
239+
"task_type": "pr-review",
240+
"status": "in-progress",
241+
"parent_id": null,
242+
"data": {
243+
"platform": "github",
244+
"files_changed": 19
245+
},
190246
"created_at": "2025-12-12T10:30:00.123Z",
191-
"updated_at": "2025-12-12T10:35:45.456Z",
192-
"tasks": {
193-
"task-abc123": {
194-
"task": "PR Review",
195-
"task_type": "pr-review",
196-
"status": "in-progress",
197-
"parent_id": null,
198-
"children": ["task-def456", "task-ghi789"],
199-
"data": {
200-
"platform": "github",
201-
"files_changed": 19
202-
},
203-
"created_at": "2025-12-12T10:30:00.123Z",
204-
"updated_at": "2025-12-12T10:35:45.456Z"
205-
},
206-
"task-def456": {
207-
"task": "Phase: Context Gathering",
208-
"task_type": "phase",
209-
"status": "done",
210-
"parent_id": "task-abc123",
211-
"children": [],
212-
"data": {
213-
"files_found": 19
214-
},
215-
"created_at": "2025-12-12T10:30:01.234Z",
216-
"updated_at": "2025-12-12T10:30:11.234Z"
217-
}
218-
}
247+
"updated_at": "2025-12-12T10:35:45.456Z"
219248
}
220249
```
221250

@@ -234,7 +263,7 @@ The state management system persists task hierarchies in JSON files at `~/.workf
234263
The `agent-state-management` workflow provides comprehensive outputs for building recursive patterns:
235264

236265
#### Core Identifiers
237-
- **`state`**: Path to state file (`~/.workflows/tasks/<trace_id>.json`)
266+
- **`state`**: Path to SQLite database (`~/.workflows/tasks/<trace_id>.db`)
238267
- **`task_id`**: ID of created/updated task
239268
- **`root_task_id`**: ID of the root task in the tree
240269
- **`trace_id`**: Execution trace ID (filename without extension)
@@ -265,38 +294,42 @@ The `agent-state-management` workflow provides comprehensive outputs for buildin
265294

266295
### Audit Trail
267296

268-
Each state operation is logged to a separate audit file (`<trace_id>.audit.json`):
297+
Each state operation is logged to the `audit` table in the same SQLite database. Query with SQL:
298+
299+
```sql
300+
SELECT timestamp, task_id, caller, action, description, changes
301+
FROM audit
302+
ORDER BY timestamp;
303+
```
304+
305+
**Example audit entries (logical structure):**
269306

270307
```json
271-
{
272-
"entries": [
273-
{
274-
"timestamp": "2025-12-12T10:30:00.123Z",
275-
"task_id": "task-abc123",
276-
"caller": "pr-review",
277-
"action": "create_root",
278-
"description": "Created root task: PR Review"
279-
},
280-
{
281-
"timestamp": "2025-12-12T10:30:01.234Z",
282-
"task_id": "task-def456",
283-
"parent_id": "task-abc123",
284-
"caller": "pr-review:context-gathering",
285-
"action": "create_subtask",
286-
"description": "Created sub-task: Phase: Context Gathering"
287-
},
288-
{
289-
"timestamp": "2025-12-12T10:30:11.234Z",
290-
"task_id": "task-def456",
291-
"caller": "pr-review:context-gathering",
292-
"action": "task_completed",
293-
"description": "Updated: ['status']",
294-
"changes": {
295-
"status": ["in-progress", "done"]
296-
}
297-
}
298-
]
299-
}
308+
[
309+
{
310+
"timestamp": "2025-12-12T10:30:00.123Z",
311+
"task_id": "task-abc123",
312+
"caller": "pr-review",
313+
"action": "create_root",
314+
"description": "Created root task: PR Review"
315+
},
316+
{
317+
"timestamp": "2025-12-12T10:30:01.234Z",
318+
"task_id": "task-def456",
319+
"parent_id": "task-abc123",
320+
"caller": "pr-review:context-gathering",
321+
"action": "create_subtask",
322+
"description": "Created sub-task: Phase: Context Gathering"
323+
},
324+
{
325+
"timestamp": "2025-12-12T10:30:11.234Z",
326+
"task_id": "task-def456",
327+
"caller": "pr-review:context-gathering",
328+
"action": "task_completed",
329+
"description": "Updated: ['status']",
330+
"changes": {"status": ["in-progress", "done"]}
331+
}
332+
]
300333
```
301334

302335
---
@@ -1039,7 +1072,7 @@ Use the `agent-state-visualize` workflow to inspect task hierarchies:
10391072
result = await execute_workflow(
10401073
workflow="agent-state-visualize",
10411074
inputs={
1042-
"state": "/Users/user/.workflows/tasks/exec-abc123.json",
1075+
"state": "/Users/user/.workflows/tasks/exec-abc123.db",
10431076
"show_data": True,
10441077
"max_depth": 0 # 0 = unlimited
10451078
}
@@ -1092,40 +1125,45 @@ inputs:
10921125
description: Visualize subtree starting from this task
10931126
```
10941127

1095-
### Reading State Files Directly
1128+
### Querying State Directly
10961129

1097-
State files are human-readable JSON:
1130+
State is stored in SQLite with queryable tables:
10981131

10991132
```bash
1100-
# View state file
1101-
cat ~/.workflows/tasks/exec-abc123.json | jq .
1133+
# Open database
1134+
sqlite3 ~/.workflows/tasks/exec-abc123.db
11021135
11031136
# List all tasks
1104-
cat ~/.workflows/tasks/exec-abc123.json | jq '.tasks | keys'
1137+
SELECT task_id, status, task FROM tasks;
11051138
11061139
# Get task details
1107-
cat ~/.workflows/tasks/exec-abc123.json | jq '.tasks["task-abc123"]'
1140+
SELECT * FROM tasks WHERE task_id = 'task-abc123';
11081141
11091142
# Check children status
1110-
cat ~/.workflows/tasks/exec-abc123.json | \
1111-
jq '.tasks["task-abc123"].children | map(.status)'
1143+
SELECT task_id, status FROM tasks WHERE parent_id = 'task-abc123';
1144+
1145+
# Task tree with hierarchy
1146+
SELECT t.task_id, t.status, t.task, p.task_id as parent
1147+
FROM tasks t
1148+
LEFT JOIN tasks p ON t.parent_id = p.task_id;
11121149
```
11131150

11141151
### Audit Trail Analysis
11151152

11161153
Review audit logs to understand workflow execution:
11171154

11181155
```bash
1156+
# Open database
1157+
sqlite3 ~/.workflows/tasks/exec-abc123.db
1158+
11191159
# View audit log
1120-
cat ~/.workflows/tasks/exec-abc123.audit.json | jq .
1160+
SELECT * FROM audit ORDER BY timestamp;
11211161
11221162
# Filter by caller
1123-
cat ~/.workflows/tasks/exec-abc123.audit.json | \
1124-
jq '.entries[] | select(.caller == "pr-review:investigation-loop")'
1163+
SELECT * FROM audit WHERE caller LIKE 'pr-review:investigation-loop%';
11251164
11261165
# Timeline of events
1127-
cat ~/.workflows/tasks/exec-abc123.audit.json | \
1128-
jq '.entries[] | {timestamp, task_id, action, description}'
1166+
SELECT timestamp, task_id, action, description FROM audit ORDER BY timestamp;
11291167
```
11301168

11311169
---
@@ -1843,7 +1881,8 @@ execute_workflow(
18431881

18441882
```bash
18451883
# Check parent-child relationships
1846-
cat ~/.workflows/tasks/exec-abc123.json | jq '.tasks | to_entries | .[] | {id: .key, parent: .value.parent_id, children: .value.children}'
1884+
sqlite3 ~/.workflows/tasks/exec-abc123.db \
1885+
"SELECT task_id, parent_id FROM tasks"
18471886
```
18481887

18491888
**Solution**: Ensure `parent_id` matches the actual parent's `task_id`.
@@ -1854,7 +1893,7 @@ cat ~/.workflows/tasks/exec-abc123.json | jq '.tasks | to_entries | .[] | {id: .
18541893

18551894
### Q: Can I pause and resume recursive workflows?
18561895

1857-
**A**: Yes, the state file persists between calls. You can stop execution and resume later by passing the same `state` path:
1896+
**A**: Yes, the SQLite database persists between calls. You can stop execution and resume later by passing the same `state` path:
18581897

18591898
```yaml
18601899
# First execution
@@ -1879,13 +1918,13 @@ result2 = execute_workflow(workflow="my-workflow", inputs={"items": [...], "stat
18791918
for_each_mode: parallel # Process all files concurrently
18801919
```
18811920

1882-
### Q: How do I delete old state files?
1921+
### Q: How do I delete old state databases?
18831922

1884-
**A**: State files accumulate in `~/.workflows/tasks/`. Clean up manually:
1923+
**A**: State databases accumulate in `~/.workflows/tasks/`. Clean up manually:
18851924

18861925
```bash
1887-
# Delete state files older than 7 days
1888-
find ~/.workflows/tasks -name "*.json" -mtime +7 -delete
1926+
# Delete state databases older than 7 days
1927+
find ~/.workflows/tasks -name "*.db" -mtime +7 -delete
18891928
```
18901929

18911930
### Q: Can I use state management without recursion?
@@ -1894,7 +1933,7 @@ find ~/.workflows/tasks -name "*.json" -mtime +7 -delete
18941933

18951934
### Q: What's the performance impact of state tracking?
18961935

1897-
**A**: Minimal - state operations are fast file writes. Typical overhead: < 100ms per task.
1936+
**A**: Minimal - SQLite operations are fast with WAL mode. Typical overhead: < 100ms per task. Concurrent writes are handled automatically.
18981937

18991938
### Q: Can I share state between different workflows?
19001939

0 commit comments

Comments
 (0)