Skip to content

Implement SunOS FileSystemWatcher using portfs event ports#124716

Open
gwr wants to merge 1 commit intodotnet:mainfrom
gwr:illumos-fswatch
Open

Implement SunOS FileSystemWatcher using portfs event ports#124716
gwr wants to merge 1 commit intodotnet:mainfrom
gwr:illumos-fswatch

Conversation

@gwr
Copy link
Copy Markdown
Contributor

@gwr gwr commented Feb 22, 2026

This implementation uses SunOS portfs (event ports) to watch for directory changes. Unlike Linux inotify, portfs can only detect WHEN a directory changes, not WHAT changed. Therefore, this implementation:

  1. Maintains a snapshot of each watched directory's contents
  2. When portfs signals a change, re-reads the directory
  3. Compares new vs old snapshots to detect creates/deletes/modifications
  4. Generates appropriate FileSystemWatcher events

Key implementation details:

  • Uses pinned GC.AllocateArray for file_obj structures required by portfs
  • When watching attributes or timestamps watches directory contents too
  • Each watched objct gets a unique cookie for identification
  • port_get returns the cookie to indicate which object changed
  • Optimized snapshot comparison with sorted single-pass algorithm
  • Supports IncludeSubdirectories by recursively watching child directories
  • Track mtime of the watched directory to avoid missing changes
  • Implement graceful cancellation using PortSend

Native changes (pal_io.c):

  • SystemNative_PortCreate: Opens event port file descriptor
  • SystemNative_PortAssociate: Associates directory with port using file_obj
  • SystemNative_PortGet: Waits for events, returns cookie identifying directory
  • SystemNative_PortSend: Send an event (used in cancellation)
  • SystemNative_PortDissociate: Removes directory association

Performance notes:

  • Less efficient than inotify (must re-read directories on each change)
  • Better than polling (blocks until change occurs)
  • Memory overhead for directory snapshots

Passes all System.IO.FileSystem.Watcher.Tests


Copilot AI review requested due to automatic review settings February 22, 2026 01:54
@dotnet-policy-service dotnet-policy-service bot added the community-contribution Indicates that the PR has been added by a community member label Feb 22, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements FileSystemWatcher support for SunOS (Solaris/illumos) using the portfs (event ports) API. Unlike Linux's inotify which reports specific file changes, portfs only signals that a directory has changed, requiring the implementation to maintain snapshots and perform comparisons to determine what actually changed.

Changes:

  • Native interop layer for portfs operations (port_create, port_associate, port_get, port_send, port_dissociate)
  • SunOS-specific FileSystemWatcher implementation using snapshot-based change detection
  • Hybrid monitoring mode that watches both directories (for name changes) and individual files (for attribute changes)
  • Support for recursive subdirectory monitoring with resource limits (50 subdirs, 1000 files per directory)

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 18 comments.

Show a summary per file
File Description
src/native/libs/configure.cmake CMake configuration for portfs feature detection (has critical bug)
src/native/libs/System.Native/pal_io.h Header declarations for portfs native functions (has spelling errors)
src/native/libs/System.Native/pal_io.c Native implementation of portfs wrapper functions (has critical string lifetime bug)
src/native/libs/System.Native/entrypoints.c Export table entries for new portfs functions
src/native/libs/Common/pal_config.h.in Configuration flag for HAVE_SUNOS_PORTFS
src/libraries/System.IO.FileSystem.Watcher/tests/System.IO.FileSystem.Watcher.Tests.csproj Added illumos and solaris to test target frameworks
src/libraries/System.IO.FileSystem.Watcher/src/System/IO/FileSystemWatcher.SunOS.cs Main SunOS implementation with snapshot comparison and event generation (multiple issues)
src/libraries/System.IO.FileSystem.Watcher/src/System.IO.FileSystem.Watcher.csproj Added SunOS-specific source files and target frameworks
src/libraries/Common/src/Interop/SunOS/portfs/Interop.portfs.cs Managed P/Invoke declarations for portfs (has hardcoded struct size issue)

@gwr gwr force-pushed the illumos-fswatch branch 2 times, most recently from cbe385d to 04c54a8 Compare February 22, 2026 02:19
Copilot AI review requested due to automatic review settings February 22, 2026 02:19
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 13 comments.

Copilot AI review requested due to automatic review settings February 22, 2026 02:54
@gwr gwr force-pushed the illumos-fswatch branch 2 times, most recently from 8b4f6e2 to 3c5ee5f Compare February 22, 2026 03:00
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 10 comments.

Copilot AI review requested due to automatic review settings February 22, 2026 03:15
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.

@jkotas jkotas added the os-SunOS SunOS, currently not officially supported label Feb 22, 2026
Copilot AI review requested due to automatic review settings February 22, 2026 05:40
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.

Copilot AI review requested due to automatic review settings February 22, 2026 06:02
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

@gwr
Copy link
Copy Markdown
Contributor Author

gwr commented Mar 19, 2026

It seems like I'm caught up with Copilot feedback, and this is up to 33 commits, so I'll squash it now to make this a bit easier to deal with locally (want to rebase and re-test).

@gwr
Copy link
Copy Markdown
Contributor Author

gwr commented Mar 24, 2026

Rebased and re-tested. One fix coming.

@gwr
Copy link
Copy Markdown
Contributor Author

gwr commented Mar 24, 2026

This passes all tests now, though I need a work-around (main...gwr:dotnet-runtime:work-around-103584) for a limitation of this watcher. I'm not sure what I should do about that. Suggestions?

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

@gwr
Copy link
Copy Markdown
Contributor Author

gwr commented Apr 5, 2026

Added a (platform-specific) workaround to the FileSystemWatcher test: Short delay between create and destroy.
This works around an inherent limitation of directory snapshot comparison, which is that when a create and destroy of some directory entry happens in quick succession, the snapshot comparison might not see any change.

With the work-around, all tests pass:
System.IO.FileSystem.Watcher.Tests Total: 305, Errors: 0, Failed: 0, Skipped: 0, Time: 10.426s

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

@gwr
Copy link
Copy Markdown
Contributor Author

gwr commented Apr 5, 2026

Caught up with Copilot feedback. I'm going to squash and then rebase to pick up recent changes in FileSystemWatcher etc. No code changes other than the squash and rebase.

This implementation uses SunOS portfs (event ports) to watch for directory
changes. Unlike Linux inotify, portfs can only detect WHEN a directory
changes, not WHAT changed. Therefore, this implementation:

1. Maintains a snapshot of each watched directory's contents
2. When portfs signals a change, re-reads the directory
3. Compares new vs old snapshots to detect creates/deletes/modifications
4. Generates appropriate FileSystemWatcher events

Key implementation details:

- Uses a single RunningInstance (event port and thread) per watcher
- Uses pinned GC.AllocateArray for file_obj structures required by portfs
- When watching attributes or timestamps watches directory contents too
- Each watched objct gets a unique cookie for identification
- port_get returns the cookie to indicate which object changed
- Optimized snapshot comparison with sorted single-pass algorithm
- Supports IncludeSubdirectories by recursively watching child directories
- Track mtime of the watched directory to avoid missing changes
- Implement graceful cancellation using PortSend

Native changes (pal_io.c):
- SystemNative_PortCreate: Opens event port file descriptor
- SystemNative_PortAssociate: Associates directory with port using file_obj
- SystemNative_PortGet: Waits for events, returns cookie identifying directory
- SystemNative_PortSend: Send an event (used in cancellation)
- SystemNative_PortDissociate: Removes directory association

Performance notes:
- Less efficient than inotify (must re-read directories on each change)
- Better than polling (blocks until change occurs)
- Memory overhead for directory snapshots

---------

Co-authored-by: Copilot <[email protected]>
@gwr
Copy link
Copy Markdown
Contributor Author

gwr commented Apr 5, 2026

Rebase and re-test complete.
System.IO.FileSystem.Watcher.Tests Total: 305, Errors: 0, Failed: 0, Skipped: 0, Time: 10.192s

Parent = parent;
Cookie = cookie;
Metadata = metadata;
FileObjBuffer = GC.AllocateArray<byte>(Interop.PortFs.FileObjSize, pinned: true);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIR the general guidance is to preferrably only use POH allocations for stuff that is never/very rarely freed to avoid fragmenting the heap, would a native alloc maybe make more sense here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been using Copilot to help me with this project. Here's what it tells me about this suggestion:

FileNode Buffer Allocation: GC Pinned Array vs Native Memory

Context

FileNode.FileObjBuffer holds a file_obj_t-sized byte buffer that must be pinned in
memory for use with port_associate(3C) on SunOS/illumos. The reviewer asked whether
Interop.MemAlloc.Malloc / Free would be better than the current
GC.AllocateArray<byte>(size, pinned: true).

Current Approach

FileObjBuffer = GC.AllocateArray<byte>(Interop.PortFs.FileObjSize, pinned: true);
  • Allocates on the Pinned Object Heap (POH), introduced in .NET 5.
  • POH objects do not cause GC fragmentation — that is its entire purpose.
  • The buffer lives as long as the FileNode object; no explicit deallocation needed.
  • No finalizer, no IDisposable, no risk of leaks.
  • Idiomatic .NET pattern for long-lived pinned buffers.

Alternative: Native Allocation

// Constructor
FileObjBuffer = Interop.MemAlloc.Malloc(Interop.PortFs.FileObjSize);

// Safety-net finalizer
~FileNode()
{
    Interop.MemAlloc.Free(FileObjBuffer);
}
  • Fully off-heap; zero GC involvement in the allocation itself.
  • Requires explicit Free at every removal/cleanup site.
  • Finalizer adds GC overhead and non-determinism (ironically reintroducing GC cost).
  • Ideally also IDisposable, which ripples through all dictionary cleanup,
    DissociateNode, and node-removal paths — significant added complexity.
  • Risk of double-free or leak if any removal path is missed.

Assessment

The reviewer's instinct — avoid fragmentation from long-lived pinned objects — was
well-founded before POH existed. POH was added to .NET 5 specifically to address that
concern, making GC.AllocateArray<byte>(..., pinned: true) the correct modern answer.

Additional factors favouring the current approach:

  • The buffer is small (one file_obj_t struct per watched entry).
  • FileNode objects are not high-churn; they are created when a file/directory is first
    observed and removed when it disappears.
  • No fragmentation risk with POH regardless of lifetime.

Recommendation

Keep GC.AllocateArray<byte>(Interop.PortFs.FileObjSize, pinned: true).

Adding Malloc/Free + finalizer/dispose would increase code complexity significantly
with no practical benefit, given that POH already handles the fragmentation concern that
motivated the suggestion.

A brief comment at the allocation site can make this rationale explicit:

// Pinned on the Pinned Object Heap (POH) to avoid GC fragmentation while keeping
// the buffer at a stable address for port_associate(3C).
FileObjBuffer = GC.AllocateArray<byte>(Interop.PortFs.FileObjSize, pinned: true);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-System.IO community-contribution Indicates that the PR has been added by a community member os-SunOS SunOS, currently not officially supported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants