Describe the bug
DAVTime fails when creating a datetime from WebHDFS file modification timestamps because the constructor assumes the timestamp is in seconds, while WebHDFS provides it in milliseconds. This causes a ValueError: year 57646 is out of range for recent timestamps.
Important context: In the latest versions, DAVTime was rewritten to use datetime.fromtimestamp() (seconds), but WebHDFS still returns timestamps in milliseconds. This mismatch causes the bug.
To Reproduce
Steps to reproduce the behavior:
-
Use asgi-webdav with a WebHDFS backend.
-
Fetch a file's status from HDFS (e.g., modificationTime).
-
Pass the timestamp directly to DAVTime:
from asgi_webdav.constants import DAVTime
file_status = {
"modificationTime": 1756981636806 # milliseconds from WebHDFS
}
dav_time = DAVTime(file_status.get("modificationTime"))
- Observe the error:
ValueError: year 57646 is out of range
Expected behavior
DAVTime should correctly handle WebHDFS timestamps in milliseconds and create a proper datetime object in UTC without errors. The behavior should be consistent with previous versions where arrow.get() handled milliseconds automatically.
Additional context
The issue only occurs for timestamps from WebHDFS or other sources that return milliseconds.
Previous versions of DAVTime using arrow.get(timestamp) worked correctly with milliseconds.
A fix would either normalize milliseconds to seconds in the constructor or automatically detect the unit.
This regression was introduced when DAVTime was rewritten in the latest release, while WebHDFS was not updated to provide seconds.
Two options to fix the timestamp mismatch:
-
In DAVTime constructor: detect if the timestamp is very large (milliseconds) and divide by 1000. Transparent and works for all providers, minimal changes, but adds logic in the constructor.
-
In the WebHDFS provider: convert modificationTime from ms to seconds before passing to DAVTime. Keeps DAVTime clean, but must remember to normalize every timestamp and risks inconsistencies.
Let me know which approach you prefer, and I can draft the fix.
Describe the bug
DAVTime fails when creating a datetime from WebHDFS file modification timestamps because the constructor assumes the timestamp is in seconds, while WebHDFS provides it in milliseconds. This causes a ValueError: year 57646 is out of range for recent timestamps.
Important context: In the latest versions, DAVTime was rewritten to use datetime.fromtimestamp() (seconds), but WebHDFS still returns timestamps in milliseconds. This mismatch causes the bug.
To Reproduce
Steps to reproduce the behavior:
Use asgi-webdav with a WebHDFS backend.
Fetch a file's status from HDFS (e.g., modificationTime).
Pass the timestamp directly to DAVTime:
Expected behavior
DAVTime should correctly handle WebHDFS timestamps in milliseconds and create a proper datetime object in UTC without errors. The behavior should be consistent with previous versions where arrow.get() handled milliseconds automatically.
Additional context
The issue only occurs for timestamps from WebHDFS or other sources that return milliseconds.
Previous versions of DAVTime using arrow.get(timestamp) worked correctly with milliseconds.
A fix would either normalize milliseconds to seconds in the constructor or automatically detect the unit.
This regression was introduced when DAVTime was rewritten in the latest release, while WebHDFS was not updated to provide seconds.
Two options to fix the timestamp mismatch:
In DAVTime constructor: detect if the timestamp is very large (milliseconds) and divide by 1000. Transparent and works for all providers, minimal changes, but adds logic in the constructor.
In the WebHDFS provider: convert modificationTime from ms to seconds before passing to DAVTime. Keeps DAVTime clean, but must remember to normalize every timestamp and risks inconsistencies.
Let me know which approach you prefer, and I can draft the fix.