Skip to content

Commit d9994bc

Browse files
committed
Metadata: phase 1
- Add decisions - Enhance file structure: Add new fields to the `FileEntry` struct in `operations.rs`. Update `types.ts` with new fields. - Add `uzers` crate for uid→name resolution (used instead of users due to RUSTSEC-2023-0059) - Create owner name cache - Add metadata transmission - Handle symlinks and permission errors gracefully - Update frontend mock service (created `test-helpers.ts`) - Test: All checks pass
1 parent 8834507 commit d9994bc

13 files changed

Lines changed: 463 additions & 46 deletions
Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
# ADR 006: File metadata scope and cost tiers
2+
3+
## Status
4+
5+
Accepted
6+
7+
## Context
8+
9+
When displaying files in the explorer, we can retrieve various metadata. macOS provides extensive file information, but
10+
each piece has different performance characteristics. With lists of 50k+ files, we must be deliberate about what to
11+
fetch eagerly vs. on-demand.
12+
13+
## Decision
14+
15+
We will categorize metadata into tiers by cost and load accordingly:
16+
17+
### Tier 1: Free (from single `stat()` call, already performed)
18+
19+
| Field | Source | Notes |
20+
| ------------- | ------------------------------------ | -------------- |
21+
| Name | `DirEntry::file_name()` | Already have |
22+
| Size | `metadata.len()` | Already have |
23+
| Is directory | `metadata.is_dir()` | Already have |
24+
| Modified date | `metadata.modified()` | Already have |
25+
| Created date | `MetadataExt::st_birthtime()` | Same syscall |
26+
| Permissions | `metadata.permissions().mode()` | Unix mode bits |
27+
| Owner uid/gid | `MetadataExt::st_uid()` / `st_gid()` | Same syscall |
28+
| Is symlink | `metadata.is_symlink()` | Same syscall |
29+
30+
### Tier 2: Cheap (extra syscall, cacheable)
31+
32+
| Field | How to get | Cost |
33+
| -------------- | --------------------------------- | --------------- |
34+
| Owner name | `users` crate to resolve uid→name | ~1μs, cacheable |
35+
| Symlink target | `std::fs::read_link()` | ~1μs if symlink |
36+
37+
### Tier 3: macOS-specific (requires Objective-C APIs)
38+
39+
| Field | API | Cost |
40+
| ------------------- | ------------------------------------------------- | ------------------- |
41+
| Added date | Spotlight / `NSURL resourceValuesForKeys:` | ~50-100μs/file |
42+
| Last opened date | Spotlight / NSURL | Same |
43+
| Locked flag | `NSURL` with `NSURLIsUserImmutableKey` | ~50μs |
44+
| Stationery pad flag | `NSURL` with `NSURLStationeryKey` | Same |
45+
| Kind (localized) | `NSURL.localizedTypeDescription` | Requires macOS APIs |
46+
| Cloud sync status | xattrs like `com.apple.icloud.itemDownloadStatus` | ~10μs |
47+
48+
### Tier 4: Extended/content-based
49+
50+
| Category | How to get | Cost |
51+
| -------------------- | ------------------------------ | -------------------- |
52+
| EXIF/media metadata | `kamadak-exif`, `image` crates | 1-50ms+ (reads file) |
53+
| PDF metadata | `lopdf` crate | 10-100ms+ |
54+
| Audio/video metadata | `lofty` crate | 10-100ms+ |
55+
56+
## Chosen scope for initial implementation
57+
58+
**Include in list view (Tier 1-2)**:
59+
60+
- All Tier 1 fields (zero extra cost)
61+
- Owner name (cached uid→name resolution)
62+
63+
**Defer (Tier 3-4)**:
64+
65+
- Added/opened dates (Spotlight-dependent, unreliable)
66+
- Locked/Stationery flags (rarely used)
67+
- Kind (can derive from extension on frontend)
68+
- EXIF and media metadata (on-demand only)
69+
70+
**Future work**:
71+
72+
- Cloud sync status (iCloud, Dropbox, GDrive) - valuable, requires xattr reads
73+
74+
## Consequences
75+
76+
### Positive
77+
78+
- Zero performance regression for current functionality
79+
- Created/permissions/owner cost nothing extra
80+
- Clear path for adding macOS-specific metadata later
81+
82+
### Negative
83+
84+
- Some Finder-equivalent fields not immediately available
85+
- macOS-specific features require platform-gated code

docs/adr/007-json-for-ipc.md

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# ADR 007: Use JSON for Tauri IPC, optimize with chunking
2+
3+
## Status
4+
5+
Accepted
6+
7+
## Context
8+
9+
Tauri 2.0 supports both JSON (default) and binary formats via Raw Payloads (MessagePack, Protobuf, etc.). For lists of
10+
50k files, we need to choose the right serialization approach.
11+
12+
### Options considered
13+
14+
1. **JSON (default)** - Simple, debuggable, no extra dependencies
15+
2. **MessagePack** - ~37% smaller, ~4x faster serialization
16+
3. **Protobuf** - Schema-based, very compact, complex setup
17+
18+
### Benchmarks (from research)
19+
20+
For 50k file entries (~200 bytes/entry):
21+
22+
- JSON: ~10MB payload, ~50ms serialization
23+
- MessagePack: ~6.3MB payload, ~12ms serialization
24+
25+
## Decision
26+
27+
Use **JSON** for IPC, combined with **chunking** (1000 entries per response).
28+
29+
Rationale:
30+
31+
1. The chunking strategy reduces per-request payload to ~200KB regardless of format
32+
2. JSON is simpler to debug (readable in browser devtools)
33+
3. No additional dependencies (`rmp-serde`, `@msgpack/msgpack`)
34+
4. Can always switch to MessagePack later if profiling shows bottleneck
35+
36+
## Consequences
37+
38+
### Positive
39+
40+
- Simpler implementation, no binary serialization libraries
41+
- Easier debugging with browser devtools
42+
- Tauri's default path, best documented
43+
44+
### Negative
45+
46+
- Slightly larger payloads (~37% overhead vs MessagePack)
47+
- Slightly slower serialization (negligible with chunking)
48+
49+
### Notes
50+
51+
If profiling reveals IPC as a bottleneck with 50k+ files after virtual scrolling and chunking are implemented, we can
52+
revisit this decision and switch to MessagePack using Tauri 2.0's `Response::new(bytes)` API.
Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
# ADR 008: Icon registry pattern for efficient icon transmission
2+
3+
## Status
4+
5+
Accepted
6+
7+
## Context
8+
9+
File icons need to be displayed for every file in the list. Without optimization, 50k JPEG files would transmit 50k
10+
identical JPEG icons (~2–4KB each = 100–200MB of redundant data).
11+
12+
### Options considered
13+
14+
1. **Inline icons**: Send icon data with each file entry
15+
2. **Icon IDs**: Send icon reference, fetch icons separately
16+
3. **CSS sprites**: Prebuilt sprite sheet of common icons
17+
4. **No icons**: Use emoji or text-only display
18+
19+
## Decision
20+
21+
Use an **icon registry pattern**:
22+
23+
1. Backend generates stable `iconId` for each file:
24+
- Extension-based: `"ext:jpg"`, `"ext:pdf"` (99% of files)
25+
- Custom icons: `"custom:<hash>"` (apps, special folders)
26+
27+
2. `FileEntry` includes only `iconId`, not icon data
28+
29+
3. Separate `get_icons(icon_ids: Vec<String>)` command returns icon data
30+
31+
4. Frontend caches icons in localStorage/IndexedDB
32+
33+
### Icon format
34+
35+
- Size: 32×32 pixels (good for retina at 2x)
36+
- Format: WebP (~50% smaller than PNG)
37+
- Fallback: Extension-based generic icons
38+
39+
### Data flow
40+
41+
```
42+
┌─────────────┐ ┌───────────────────┐ ┌─────────────┐
43+
│ File list │────▶│ iconId refs │────▶│ Frontend │
44+
│ response │ │ ("ext:jpg", ...) │ │ checks │
45+
└─────────────┘ └───────────────────┘ │ cache │
46+
└──────┬──────┘
47+
48+
┌───────────────────────┘
49+
50+
┌─────────────────────┐
51+
│ get_icons() for │
52+
│ uncached IDs only │
53+
└─────────────────────┘
54+
```
55+
56+
## Consequences
57+
58+
### Positive
59+
60+
- 50k files transmit ~50 unique icon IDs (not 50k icon blobs)
61+
- Icons cached persistently across sessions
62+
- Lazy loading — only fetch icons as needed
63+
- Extension-based IDs are stable and predictable
64+
65+
### Negative
66+
67+
- Two-phase loading (file list, then icons)
68+
- Requires frontend cache management
69+
- Custom icons (apps, branded folders) need hash-based invalidation
70+
71+
### Implementation notes
72+
73+
- Use `file_icon_provider` crate for cross-platform icon retrieval
74+
- Icon generation is async (~50–100 μs per unique icon via NSWorkspace)
75+
- Backend should cache generated icons in memory during a session

src-tauri/Cargo.lock

Lines changed: 11 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

src-tauri/Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ serde = { version = "1", features = ["derive"] }
2626
serde_json = "1"
2727
notify = "8"
2828
dirs = "6"
29+
uzers = "0.12.2"
2930

3031
[target.'cfg(not(any(target_os = "android", target_os = "ios")))'.dependencies]
3132
tauri-plugin-window-state = "2"

src-tauri/src/file_system/mock_provider.rs

Lines changed: 20 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -19,12 +19,26 @@ impl MockFileSystemProvider {
1919
/// Useful for stress testing with large file counts.
2020
pub fn with_file_count(count: usize) -> Self {
2121
let entries = (0..count)
22-
.map(|i| FileEntry {
23-
name: format!("file_{:06}.txt", i),
24-
path: format!("/mock/file_{:06}.txt", i),
25-
is_directory: i % 10 == 0, // Every 10th entry is a directory
26-
size: Some(1024 * (i as u64)),
27-
modified_at: Some(1640000000 + i as u64),
22+
.map(|i| {
23+
let is_dir = i % 10 == 0;
24+
let name = format!("file_{:06}.txt", i);
25+
FileEntry {
26+
name: name.clone(),
27+
path: format!("/mock/file_{:06}.txt", i),
28+
is_directory: is_dir,
29+
is_symlink: i % 50 == 0, // Every 50th is a symlink for testing
30+
size: Some(1024 * (i as u64)),
31+
modified_at: Some(1640000000 + i as u64),
32+
created_at: Some(1639000000 + i as u64),
33+
permissions: 0o644,
34+
owner: "testuser".to_string(),
35+
group: "staff".to_string(),
36+
icon_id: if is_dir {
37+
"dir".to_string()
38+
} else {
39+
"ext:txt".to_string()
40+
},
41+
}
2842
})
2943
.collect();
3044
Self::new(entries)

src-tauri/src/file_system/mock_provider_test.rs

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,15 +10,27 @@ fn test_mock_provider_returns_entries() {
1010
name: "test.txt".to_string(),
1111
path: "/test/test.txt".to_string(),
1212
is_directory: false,
13+
is_symlink: false,
1314
size: Some(1024),
1415
modified_at: Some(1640000000),
16+
created_at: Some(1639000000),
17+
permissions: 0o644,
18+
owner: "testuser".to_string(),
19+
group: "staff".to_string(),
20+
icon_id: "ext:txt".to_string(),
1521
},
1622
FileEntry {
1723
name: "folder".to_string(),
1824
path: "/test/folder".to_string(),
1925
is_directory: true,
26+
is_symlink: false,
2027
size: None,
2128
modified_at: Some(1640000000),
29+
created_at: Some(1639000000),
30+
permissions: 0o755,
31+
owner: "testuser".to_string(),
32+
group: "staff".to_string(),
33+
icon_id: "dir".to_string(),
2234
},
2335
];
2436

0 commit comments

Comments
 (0)