Is your feature request related to a problem? Please describe.
When a shared NAS is used as HF_HUB_CACHE across multiple machines running different operating systems, a common pattern in studios, render farms, and ML pipelines, symlinks created by a Linux machine cannot be followed by Windows machines accessing the same share. When a Linux node downloads a model, are_symlinks_supported() correctly detects that Linux supports symlinks and creates them in the cache. When a Windows machine later tries to scan or load that model from the same network path, Windows raises WinError 1463 (ERROR_STOPPED_ON_SYMLINK).
This is frustrating because the failure is not apparent on the Linux side and only surfaces as a confusing error on the Windows side, against a model that appears to be fully downloaded.
Describe the solution you'd like
An environment variable such as HF_HUB_DISABLE_SYMLINKS=1 that causes are_symlinks_supported() to always return False.
Another implementation would be to add a public setter for _are_symlinks_supported_in_dir. Then users could set networked paths to False manually.
Either way, I am requesting a mechanism to force huggingface_hub to store real files in the cache instead of symlinks regardless of what the downloading machine's OS supports. This would make the cache portable across OSes at the cost of blob deduplication, which is an acceptable tradeoff for shared network deployments. A single NAS with no blob dedupe is more efficient than 2 different caches (one for Windows, one for Unix).
Describe alternatives you've considered
-
Windows Developer Mode: The huggingface_hub docs suggest enabling Developer Mode to allow symlink creation on Windows. However, symlink creation and symlink traversal on network shares are governed by entirely separate Windows mechanisms. Developer Mode grants the SeCreateSymbolicLink privilege (which are_symlinks_supported() tests), but following symlinks on remote paths is controlled by a separate evaluation policy (fsutil behavior set SymlinkEvaluation R2R:1). This is disabled by default as a security measure, is a system-wide change, and may not be permitted in managed enterprise environments. Even users who have correctly followed the documentation to enable Developer Mode will still hit WinError 1463 on network shares.
-
Manipulating the private _are_symlinks_supported_in_dir dict: It is possible to pre-seed this module-level dict in file_download.py to force the no-symlinks path, but this relies on a private internal API that could change without notice.
-
Using local_dir on snapshot_download: Passing local_dir copies real files to the target directory without the blobs/snapshots/cache structure. This requires callers to pass the local path directly to from_pretrained rather than the model I, a significant calling convention change that breaks the common from_pretrained(model_id, local_files_only=True) pattern used by most downstream libraries.
Additional context
The four Windows symlink evaluation policies (L2L, L2R, R2L, R2R) mean there is no single "enable symlinks on Windows" setting. A user can have full Developer Mode enabled and still be unable to follow symlinks on a network share because R2R (remote-to-remote) is independently disabled. This makes the current documentation guidance incomplete for network share scenarios.
Is your feature request related to a problem? Please describe.
When a shared NAS is used as HF_HUB_CACHE across multiple machines running different operating systems, a common pattern in studios, render farms, and ML pipelines, symlinks created by a Linux machine cannot be followed by Windows machines accessing the same share. When a Linux node downloads a model, are_symlinks_supported() correctly detects that Linux supports symlinks and creates them in the cache. When a Windows machine later tries to scan or load that model from the same network path, Windows raises
WinError 1463 (ERROR_STOPPED_ON_SYMLINK).This is frustrating because the failure is not apparent on the Linux side and only surfaces as a confusing error on the Windows side, against a model that appears to be fully downloaded.
Describe the solution you'd like
An environment variable such as HF_HUB_DISABLE_SYMLINKS=1 that causes are_symlinks_supported() to always return False.
Another implementation would be to add a public setter for _are_symlinks_supported_in_dir. Then users could set networked paths to False manually.
Either way, I am requesting a mechanism to force huggingface_hub to store real files in the cache instead of symlinks regardless of what the downloading machine's OS supports. This would make the cache portable across OSes at the cost of blob deduplication, which is an acceptable tradeoff for shared network deployments. A single NAS with no blob dedupe is more efficient than 2 different caches (one for Windows, one for Unix).
Describe alternatives you've considered
Windows Developer Mode: The huggingface_hub docs suggest enabling Developer Mode to allow symlink creation on Windows. However, symlink creation and symlink traversal on network shares are governed by entirely separate Windows mechanisms. Developer Mode grants the SeCreateSymbolicLink privilege (which are_symlinks_supported() tests), but following symlinks on remote paths is controlled by a separate evaluation policy (
fsutil behavior set SymlinkEvaluation R2R:1). This is disabled by default as a security measure, is a system-wide change, and may not be permitted in managed enterprise environments. Even users who have correctly followed the documentation to enable Developer Mode will still hit WinError 1463 on network shares.Manipulating the private _are_symlinks_supported_in_dir dict: It is possible to pre-seed this module-level dict in file_download.py to force the no-symlinks path, but this relies on a private internal API that could change without notice.
Using local_dir on snapshot_download: Passing local_dir copies real files to the target directory without the blobs/snapshots/cache structure. This requires callers to pass the local path directly to from_pretrained rather than the model I, a significant calling convention change that breaks the common from_pretrained(model_id, local_files_only=True) pattern used by most downstream libraries.
Additional context
The four Windows symlink evaluation policies (L2L, L2R, R2L, R2R) mean there is no single "enable symlinks on Windows" setting. A user can have full Developer Mode enabled and still be unable to follow symlinks on a network share because R2R (remote-to-remote) is independently disabled. This makes the current documentation guidance incomplete for network share scenarios.