Indexing: Fix missing dir sizes after replay

- Dirs created by reconciler/live events via `UpsertEntryV2` never got a `dir_stats` row, so they showed no sizes in the UI
- `UpsertEntryV2` now initializes a zero-valued `dir_stats` row when inserting a new directory
- Added `backfill_missing_dir_stats()` aggregator function that finds dirs without stats and computes them bottom-up
- `BackfillMissingDirStats` writer message triggers after post-scan reconciler replay and cold-start FSEvents replay

Author: vdavid

apps/desktop/src-tauri/src/indexing/CLAUDE.md

@@ -12,10 +12,10 @@ Full design: `docs/specs/drive-indexing/plan.md`
 - **store.rs** -- SQLite schema v2 (integer-keyed entries, dir_stats by entry_id, meta), platform_case collation, read queries, DB open/migrate. Schema version check: mismatch triggers drop+rebuild. Both path-keyed (backward compat) and integer-keyed APIs.
 - **path_resolver.rs** -- `PathResolver`: resolves filesystem paths to integer entry IDs via component-by-component walk with full-path LRU cache (50K entries). Case-aware `CacheKey` on macOS (NFD + case fold). Prefix-based invalidation for deletes/renames.
 - **memory_watchdog.rs** -- Background task monitoring resident memory via `mach_task_info` (macOS). Warns at 8 GB, stops indexing at 16 GB, emits `index-memory-warning` event to frontend. No-op stub on non-macOS. Started from `start_indexing()`.
-- **writer.rs** -- Single writer thread, owns the write connection, processes `WriteMessage` channel (bounded `sync_channel`, 20K capacity, backpressure via blocking). Priority: `UpdateDirStats` before `InsertEntries`. `Flush` variant + async `flush()` method let callers wait for all prior writes to commit. Has both integer-keyed variants (`InsertEntriesV2`, `UpsertEntryV2`, `DeleteEntryById`, `DeleteSubtreeById`, `PropagateDeltaById`) and path-keyed backward-compat variants. The integer-keyed delete/subtree-delete handlers auto-propagate negative deltas via the `parent_id` chain (same pattern as the path-keyed variants). `propagate_delta_by_id` walks the parent chain using `get_parent_id` lookups. Maintains `AccumulatorMaps` during `InsertEntriesV2` processing (two HashMaps: direct children stats and child dir relationships + an `entries_inserted` counter), cleared on `TruncateData`. On `ComputeAllAggregates`, passes accumulated maps to `aggregator::compute_all_aggregates_with_maps()` to skip expensive full-table-scan SQL queries. Accepts an optional `AppHandle` at spawn time to emit `index-aggregation-progress` events during aggregation (phase, current, total). Also emits `saving_entries` phase progress during `InsertEntriesV2` processing when the expected total is set via `set_expected_total_entries()` (an `Arc<AtomicU64>` shared between the writer thread and the `IndexWriter` handle).
+- **writer.rs** -- Single writer thread, owns the write connection, processes `WriteMessage` channel (bounded `sync_channel`, 20K capacity, backpressure via blocking). Priority: `UpdateDirStats` before `InsertEntries`. `Flush` variant + async `flush()` method let callers wait for all prior writes to commit. Has both integer-keyed variants (`InsertEntriesV2`, `UpsertEntryV2`, `DeleteEntryById`, `DeleteSubtreeById`, `PropagateDeltaById`) and path-keyed backward-compat variants. The integer-keyed delete/subtree-delete handlers auto-propagate negative deltas via the `parent_id` chain (same pattern as the path-keyed variants). `propagate_delta_by_id` walks the parent chain using `get_parent_id` lookups. `UpsertEntryV2` initializes a zero-valued `dir_stats` row when inserting a NEW directory, so enrichment always has a row (subsequent `PropagateDeltaById` calls update it incrementally). Maintains `AccumulatorMaps` during `InsertEntriesV2` processing (two HashMaps: direct children stats and child dir relationships + an `entries_inserted` counter), cleared on `TruncateData`. On `ComputeAllAggregates`, passes accumulated maps to `aggregator::compute_all_aggregates_with_maps()` to skip expensive full-table-scan SQL queries. Accepts an optional `AppHandle` at spawn time to emit `index-aggregation-progress` events during aggregation (phase, current, total). Also emits `saving_entries` phase progress during `InsertEntriesV2` processing when the expected total is set via `set_expected_total_entries()` (an `Arc<AtomicU64>` shared between the writer thread and the `IndexWriter` handle).
 - **scanner.rs** -- jwalk-based parallel directory walker. `scan_volume()` for full scan, `scan_subtree()` for micro-scans. Uses `ScanContext` (from store.rs) to assign integer IDs and parent IDs during the walk: maintains a `HashMap<PathBuf, i64>` mapping directory paths to assigned IDs. The scan root is mapped to `ROOT_ID` (1). Sends `InsertEntriesV2(Vec<EntryRow>)` batches to the writer. Platform-specific exclusion filters (macOS system paths, Linux virtual filesystems). Physical sizes (`st_blocks * 512`).
 - **micro_scan.rs** -- `MicroScanManager`: bounded task pool (default 3 concurrent), priority queue (`UserSelected` > `CurrentDir`), deduplication, cancellation. Skips after full scan completes.
-- **aggregator.rs** -- Dir stats computation. Bottom-up after full scan (O(N) single pass), per-subtree after micro-scan, incremental delta propagation up ancestor chain for watcher events. Two entry points for full aggregation: `compute_all_aggregates_reported` (loads maps from SQL) and `compute_all_aggregates_with_maps` (accepts pre-built maps from the writer). Both accept an `on_progress: &mut dyn FnMut(AggregationProgress)` callback and delegate to `compute_and_write()` for the shared topological sort + bottom-up computation + batch write. Progress is reported at phase transitions and every ~1% during compute/write loops. `AggregationPhase` enum: `SavingEntries` (flushing writer channel), `LoadingDirectories`, `Sorting`, `Computing`, `Writing`.
+- **aggregator.rs** -- Dir stats computation. Bottom-up after full scan (O(N) single pass), per-subtree after micro-scan, incremental delta propagation up ancestor chain for watcher events. Two entry points for full aggregation: `compute_all_aggregates_reported` (loads maps from SQL) and `compute_all_aggregates_with_maps` (accepts pre-built maps from the writer). Both accept an `on_progress: &mut dyn FnMut(AggregationProgress)` callback and delegate to `compute_and_write()` for the shared topological sort + bottom-up computation + batch write. Progress is reported at phase transitions and every ~1% during compute/write loops. `AggregationPhase` enum: `SavingEntries` (flushing writer channel), `LoadingDirectories`, `Sorting`, `Computing`, `Writing`. `backfill_missing_dir_stats` is a catch-up pass that finds directories without `dir_stats` rows and computes their stats bottom-up; triggered after reconciler replay and cold-start replay via `BackfillMissingDirStats` writer message.
 - **watcher.rs** -- Drive-level filesystem watcher. macOS: FSEvents via `cmdr-fsevent-stream` with event IDs and `sinceWhen` replay. Linux: `notify` crate (inotify backend) with recursive watching and synthetic event counter. Other platforms: stub. `supports_event_replay()` lets callers branch on whether journal replay is available.
 - **reconciler.rs** -- Buffers FSEvents during scan (capped at 500K events; overflow sets `buffer_overflow` flag forcing full rescan), replays after scan completes using event IDs to skip stale events. Processes live events for file creates/removes/modifies using integer-keyed write messages (`UpsertEntryV2`, `DeleteEntryById`, `DeleteSubtreeById`, `PropagateDeltaById`). Resolves filesystem paths to entry IDs via `store::resolve_path()` using a read connection passed by callers. Key functions (`process_fs_event`, `emit_dir_updated`) are `pub(super)` so `mod.rs` can call them directly during cold-start replay.
 - **firmlinks.rs** -- Parses `/usr/share/firmlinks`, builds prefix map, normalizes paths. Converts `/System/Volumes/Data/Users/foo` to `/Users/foo`.
@@ -146,6 +146,8 @@ Key test files are alongside each module (test functions within `#[cfg(test)]` b
 
 **`verifier.rs` is a placeholder**: Per-navigation readdir diff is a future milestone. Currently just a TODO comment.
 
+**Dirs created by reconciler/live events must get `dir_stats` immediately**: `UpsertEntryV2` inserts a zero-valued `dir_stats` row when creating a new directory. Without this, directories created after the last full aggregation have no `dir_stats` and show no sizes in the UI. `BackfillMissingDirStats` runs after reconciler replay and cold-start replay as a catch-up pass for any dirs that slipped through (for example, from older code before this fix). The zero-init + backfill combination guarantees every directory always has a `dir_stats` row.
+
 **Scan cancellation leaves partial data**: By design. `scan_completed_at` not set in meta, so next startup detects incomplete scan and runs fresh. No cleanup needed.
 
 **`ReadPool` replaces `INDEXING` lock for all read-only DB access**: Enrichment (`enrich_entries_with_index`), verification Phase 1 (`verify_affected_dirs`), and background verification dir-stat reads all use `get_read_pool()` + `pool.with_conn()` — thread-local SQLite connections with no lock contention. The `INDEXING` mutex now guards only lifecycle transitions and IPC commands that need `PathResolver`. `with_conn` uses `thread_local!` storage, so callers must not have `.await` points between obtaining the pool and completing the closure (async task migration would break thread affinity).

apps/desktop/src-tauri/src/indexing/aggregator.rs

@@ -301,6 +301,107 @@ pub fn compute_subtree_aggregates(conn: &Connection, root: &str) -> Result<u64,
     Ok(count)
 }
 
+/// Backfill `dir_stats` for directories that have entries but no stats row.
+///
+/// Finds all directories missing a `dir_stats` row and computes their stats
+/// bottom-up. This catches directories created by reconciler/live events
+/// after the last full aggregation. Returns the number of dirs backfilled.
+pub fn backfill_missing_dir_stats(conn: &Connection) -> Result<u64, IndexStoreError> {
+    // Find directories without dir_stats
+    let missing_ids = load_dirs_missing_stats(conn)?;
+    if missing_ids.is_empty() {
+        return Ok(0);
+    }
+
+    let start = std::time::Instant::now();
+    let count = missing_ids.len();
+    log::debug!("Backfill: {count} directories missing dir_stats, computing...");
+
+    // Load ALL directory entries for the topological sort (we need the full
+    // tree structure to compute bottom-up correctly, since a missing dir's
+    // children may also be missing).
+    let all_dir_entries = load_all_directory_ids(conn)?;
+
+    // Build direct_stats and child_dirs maps scoped to the missing dirs
+    // and their descendants. We use the full-table bulk queries since the
+    // missing dirs can be scattered across the tree.
+    let direct_stats = bulk_get_children_stats_by_id(conn)?;
+    let child_dirs_map = bulk_get_child_dir_ids(conn)?;
+
+    // Topological sort all dirs (we need correct ordering)
+    let sorted = topological_sort_bottom_up(&all_dir_entries);
+
+    // Build set of missing IDs for fast lookup
+    let missing_set: std::collections::HashSet<i64> = missing_ids.into_iter().collect();
+
+    // Compute stats bottom-up for ALL dirs, but only write the missing ones.
+    // We need to compute all because a missing dir's stats depend on its
+    // children (which might have existing stats in the DB or might also be
+    // missing).
+    let mut computed: HashMap<i64, DirStatsById> = HashMap::with_capacity(sorted.len());
+    let mut to_write: Vec<DirStatsById> = Vec::with_capacity(count);
+
+    for &dir_id in &sorted {
+        let (file_size_sum, file_count, child_dir_count) =
+            direct_stats.get(&dir_id).copied().unwrap_or((0, 0, 0));
+
+        let mut recursive_size = file_size_sum;
+        let mut recursive_file_count = file_count;
+        let mut recursive_dir_count = child_dir_count;
+
+        if let Some(children) = child_dirs_map.get(&dir_id) {
+            for &child_id in children {
+                // Prefer freshly computed stats, fall back to existing DB stats
+                if let Some(child_stats) = computed.get(&child_id) {
+                    recursive_size += child_stats.recursive_size;
+                    recursive_file_count += child_stats.recursive_file_count;
+                    recursive_dir_count += child_stats.recursive_dir_count;
+                } else if let Ok(Some(db_stats)) = IndexStore::get_dir_stats_by_id(conn, child_id) {
+                    recursive_size += db_stats.recursive_size;
+                    recursive_file_count += db_stats.recursive_file_count;
+                    recursive_dir_count += db_stats.recursive_dir_count;
+                }
+            }
+        }
+
+        let stats = DirStatsById {
+            entry_id: dir_id,
+            recursive_size,
+            recursive_file_count,
+            recursive_dir_count,
+        };
+
+        if missing_set.contains(&dir_id) {
+            to_write.push(stats.clone());
+        }
+        computed.insert(dir_id, stats);
+    }
+
+    // Batch-write only the missing stats
+    for chunk in to_write.chunks(1000) {
+        IndexStore::upsert_dir_stats_by_id(conn, chunk)?;
+    }
+
+    log::debug!(
+        "Backfill: wrote {} dir_stats rows in {:.1}s",
+        to_write.len(),
+        start.elapsed().as_secs_f64(),
+    );
+
+    Ok(to_write.len() as u64)
+}
+
+/// Load directory IDs that have entries but no `dir_stats` row.
+fn load_dirs_missing_stats(conn: &Connection) -> Result<Vec<i64>, IndexStoreError> {
+    let mut stmt = conn.prepare(
+        "SELECT e.id FROM entries e
+         LEFT JOIN dir_stats ds ON ds.entry_id = e.id
+         WHERE e.is_directory = 1 AND ds.entry_id IS NULL",
+    )?;
+    let rows = stmt.query_map([], |row| row.get::<_, i64>(0))?;
+    rows.collect::<Result<Vec<_>, _>>().map_err(Into::into)
+}
+
 // ── Internal helpers ─────────────────────────────────────────────────
 
 /// Load all directory `(id, parent_id)` pairs from the entries table.
@@ -690,6 +791,67 @@ mod tests {
         assert_eq!(count, 0);
     }
 
+    // ── backfill_missing_dir_stats tests ─────────────────────────────
+
+    #[test]
+    fn backfill_fills_missing_stats() {
+        let (conn, _dir) = open_temp_conn();
+
+        // Tree: /a (id=2) with /a/f.txt (id=3, 100 bytes), /a/sub (id=4), /a/sub/g.txt (id=5, 200)
+        insert_entries(
+            &conn,
+            &[
+                make_dir(2, ROOT_ID, "a"),
+                make_file(3, 2, "f.txt", 100),
+                make_dir(4, 2, "sub"),
+                make_file(5, 4, "g.txt", 200),
+            ],
+        );
+
+        // Only compute stats for /a/sub (id=4) — leave /a (id=2) and root (id=1) missing
+        IndexStore::upsert_dir_stats_by_id(
+            &conn,
+            &[DirStatsById {
+                entry_id: 4,
+                recursive_size: 200,
+                recursive_file_count: 1,
+                recursive_dir_count: 0,
+            }],
+        )
+        .unwrap();
+
+        // Backfill should fill in root sentinel (id=1) and /a (id=2)
+        let count = backfill_missing_dir_stats(&conn).unwrap();
+        assert_eq!(count, 2); // root sentinel + /a
+
+        // /a should now have correct recursive stats
+        let a_stats = get_stats(&conn, 2).unwrap();
+        assert_eq!(a_stats.recursive_size, 300); // 100 + 200
+        assert_eq!(a_stats.recursive_file_count, 2);
+        assert_eq!(a_stats.recursive_dir_count, 1);
+
+        // Root sentinel should also be correct
+        let root_stats = get_stats(&conn, ROOT_ID).unwrap();
+        assert_eq!(root_stats.recursive_size, 300);
+    }
+
+    #[test]
+    fn backfill_noop_when_all_stats_present() {
+        let (conn, _dir) = open_temp_conn();
+
+        insert_entries(
+            &conn,
+            &[make_dir(2, ROOT_ID, "a"), make_file(3, 2, "f.txt", 100)],
+        );
+
+        // Compute all stats first
+        compute_all_aggregates(&conn).unwrap();
+
+        // Backfill should find nothing to do
+        let count = backfill_missing_dir_stats(&conn).unwrap();
+        assert_eq!(count, 0);
+    }
+
     // ── topological sort test ────────────────────────────────────────
 
     #[test]

apps/desktop/src-tauri/src/indexing/mod.rs

@@ -861,6 +861,10 @@ impl IndexManager {
                         }
                     }
 
+                    // Backfill dir_stats for any directories created by the replay
+                    // that didn't go through the full aggregation pass.
+                    let _ = writer.send(WriteMessage::BackfillMissingDirStats);
+
                     // Switch to live mode
                     reconciler.switch_to_live();
 
@@ -1602,6 +1606,10 @@ async fn run_replay_event_loop(
         reconciler::emit_dir_updated(&app, affected_paths.iter().cloned().collect());
     }
 
+    // Backfill dir_stats for any directories created by the replay
+    // that didn't go through a full aggregation pass.
+    let _ = writer.send(WriteMessage::BackfillMissingDirStats);
+
     // ── Switch to live mode immediately (before verification) ────────
 
     log::debug!("Replay: switching to live mode");