Skip to content

Commit 54f43ff

Browse files
comp615ampcode-com
authored andcommitted
fix(core): use recursive FSEvents on macOS instead of non-recursive kqueue (#34523)
## Current Behavior Since Nx 22.5.0, the daemon's native file watcher silently drops all file change events on macOS in large monorepos (~5,250+ watched directories). `nx watch`, `nx serve`, and any daemon-dependent file watching is broken. The root cause is that #34329 switched all watched paths to `WatchedPath::non_recursive()`. On macOS, the `notify` crate uses **kqueue** for non-recursive watches instead of **FSEvents**. kqueue silently fails at scale due to vnode table pressure (`kern.num_vnodes == kern.maxvnodes`), causing the daemon to never detect file changes. This is a **scale-dependent** bug: it works fine in small workspaces (~30 directories) but breaks silently in large ones. | | **Nx 22.4.5** | **Nx 22.5.0+** | |---|---|---| | **Small repo (~30 dirs)** | Works (FSEvents) | Works (~30 kqueue watches) | | **Large repo (~5,250+ dirs)** | Works (FSEvents) | **Broken** (kqueue silently drops all events) | ## Expected Behavior The macOS file watcher should detect file creates, modifications, and deletions at any scale, matching the behavior of Nx 22.4.x. ## Fix Use platform-conditional watch modes: - **macOS:** Single recursive watch on the workspace root (uses FSEvents natively) - **Linux/Windows:** Non-recursive per-directory watches (preserves the #33781 inotify fix) On macOS, FSEvents handles recursive watching from a single root path, so directory enumeration and dynamic registration are skipped entirely. This also improves daemon startup time on macOS from ~10 minutes to <1 second in a 354-project monorepo. ### What changed in `watcher.rs` 1. **Initial pathset:** On macOS, watch only the root directory recursively via FSEvents instead of enumerating all directories for non-recursive kqueue watches. 2. **Dynamic directory registration (`on_action`):** Wrapped in `#[cfg(not(target_os = "macos"))]` since FSEvents already watches the full tree. Linux/Windows behavior is completely unchanged. ### Why the event filter is fine as-is We verified that with recursive FSEvents watches, macOS emits specific `FileEventKind` variants (`Create(File)`, `Modify(Data(Content))`, `Remove(File)`, `Modify(Name(Any))`) that the current `watch_filterer.rs` already handles correctly. Zero events were rejected by the catch-all. The `Modify(Any)` / `Create(Any)` variants are kqueue artifacts that are not needed with FSEvents. ### Why kqueue fails silently Apple's [File System Events Programming Guide](https://developer.apple.com/library/archive/documentation/Darwin/Conceptual/FSEvents_ProgGuide/KernelQueues/KernelQueues.html) explicitly recommends FSEvents over kqueue for large hierarchies: *"If you are monitoring a large hierarchy of content, you should use file system events instead."* kqueue requires `open(path, O_EVTONLY)` per watched directory. Under vnode table pressure, the kernel recycles vnodes with kqueue watches attached without notifying the watcher. There is no error, no partial delivery, and no diagnostic signal. ## Tested on - macOS 26.3 (Tahoe), Apple Silicon (arm64), APFS - 354-project pnpm monorepo (~19,865 non-ignored directories) - Verified: file modifications, file creates, and file deletes all detected - Daemon init time: ~10 min (with enumeration) -> <1s (with root-only FSEvents watch) ## Related Issue(s) Fixes #34522 Co-authored-by: Amp <amp@ampcode.com> (cherry picked from commit d5cd6a1)
1 parent 15a6856 commit 54f43ff

1 file changed

Lines changed: 49 additions & 24 deletions

File tree

packages/nx/src/native/watch/watcher.rs

Lines changed: 49 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -201,20 +201,26 @@ impl Watcher {
201201
return action;
202202
}
203203

204-
// Check for new directory creation events and dynamically register watches.
205-
// Without this, non-recursive watches would miss changes in newly created dirs.
206-
let new_directories = extract_new_directories(&action.events, &ignore_globs_for_action);
207-
208-
if !new_directories.is_empty() {
209-
let mut path_set = watched_path_set_for_action.lock();
210-
path_set.extend(new_directories);
211-
trace!(
212-
count = path_set.len(),
213-
"updating pathset with new directories"
214-
);
215-
watch_exec_for_action
216-
.config
217-
.pathset(path_set.iter().map(|p| WatchedPath::non_recursive(p)));
204+
// On macOS, FSEvents watches the entire tree recursively from the root,
205+
// so we don't need to dynamically register new directories.
206+
#[cfg(not(target_os = "macos"))]
207+
{
208+
// Check for new directory creation events and dynamically register watches.
209+
// Without this, non-recursive watches would miss changes in newly created dirs.
210+
let new_directories =
211+
extract_new_directories(&action.events, &ignore_globs_for_action);
212+
213+
if !new_directories.is_empty() {
214+
let mut path_set = watched_path_set_for_action.lock();
215+
path_set.extend(new_directories);
216+
trace!(
217+
count = path_set.len(),
218+
"updating pathset with new directories"
219+
);
220+
watch_exec_for_action
221+
.config
222+
.pathset(path_set.iter().map(|p| WatchedPath::non_recursive(p)));
223+
}
218224
}
219225

220226
let mut origin_path = origin.clone();
@@ -284,16 +290,35 @@ impl Watcher {
284290
let start = async move {
285291
trace!("configuring watch exec");
286292

287-
// Enumerate non-ignored directories and watch each one non-recursively.
288-
// This prevents inotify watches on node_modules and other ignored trees.
289-
let path_set = enumerate_watch_paths(&origin, use_ignore);
290-
trace!(count = path_set.len(), "setting watched paths");
291-
292-
// Store the initial path set so the on_action handler can append to it
293-
watch_exec
294-
.config
295-
.pathset(path_set.iter().map(|p| WatchedPath::non_recursive(p)));
296-
*watched_path_set.lock() = path_set;
293+
// On macOS, FSEvents handles recursive watching natively from a single
294+
// root path. No need to enumerate individual directories — this avoids
295+
// kqueue (used for non-recursive watches) which silently fails at scale
296+
// due to vnode table pressure. See #34522.
297+
#[cfg(target_os = "macos")]
298+
{
299+
let mut path_set = HashSet::new();
300+
path_set.insert(PathBuf::from(&origin));
301+
trace!(
302+
count = path_set.len(),
303+
"macOS: watching root recursively via FSEvents"
304+
);
305+
watch_exec
306+
.config
307+
.pathset(path_set.iter().map(|p| WatchedPath::recursive(p)));
308+
*watched_path_set.lock() = path_set;
309+
}
310+
// On Linux/Windows, enumerate non-ignored directories and watch each one
311+
// non-recursively. This prevents inotify watches on node_modules and other
312+
// ignored trees (fixing #33781).
313+
#[cfg(not(target_os = "macos"))]
314+
{
315+
let path_set = enumerate_watch_paths(&origin, use_ignore);
316+
trace!(count = path_set.len(), "setting watched paths");
317+
watch_exec
318+
.config
319+
.pathset(path_set.iter().map(|p| WatchedPath::non_recursive(p)));
320+
*watched_path_set.lock() = path_set;
321+
}
297322

298323
watch_exec.config.filterer(watch_filterer::create_filter(
299324
&origin,

0 commit comments

Comments
 (0)