Fix tensorstore empty array handling#326
Merged
Merged
Conversation
- Add validation for empty arrays in _save_transformed before tensorstore write - Skip write operations for empty arrays with warning messages - Add comprehensive error handling with detailed diagnostics for tensorstore failures - Improve error messages to include array shapes, sizes, and tensorstore details This resolves the ValueError: Error aligning dimensions issue when empty arrays are passed to tensorstore write operations.
Adds validation in apply_transform_to_tczyx_and_save() to check for empty results dictionary before calling _save_transformed(). When no valid time points are available, logs diagnostic message and skips write operation instead of attempting to write empty arrays to tensorstore, which causes alignment dimension mismatches.
This reverts commit 65c9ddb.
Contributor
Not sure if we want to include this in the git tree. |
ziw-liu
approved these changes
Aug 24, 2025
Contributor
ziw-liu
left a comment
ziw-liu
left a comment
There was a problem hiding this comment.
Thanks for catching this edge case! Please remove the LLM authorship from the commit message before merging.
ieivanov
force-pushed
the
fix-tensorstore-empty-array-handling
branch
from
7fb9827 to
1110367
Compare
ziw-liu
added a commit
that referenced
this pull request
Sep 4, 2025
* update signature type hints * clarify the type of position keys * expose sharding in plate creation * isort * fix default version * fix example code block format * fix type hints * utility to split indices by shards * separate apply and save * simplify random number generation * wip: batched writing in time * match storage keys with values * removed unused argument * fix import * fix string formatting * add more version testing * use tensorstore instead * control tensorstore concurrency * memory management * isort * format * remove platform check * warning if shards is specified for 0.4 * set shards to none for 0.4 * Update acquire-zarr OME v0.5 fixture / aqz test to reflect new config API and downsampling behavior * add notes about upstream issues * add example of sharded plate * explicitly add layout in open_ome_zarr * Fix tensorstore empty array handling (#326) * Fix tensorstore empty array handling - Add validation for empty arrays in _save_transformed before tensorstore write - Skip write operations for empty arrays with warning messages - Add comprehensive error handling with detailed diagnostics for tensorstore failures - Improve error messages to include array shapes, sizes, and tensorstore details This resolves the ValueError: Error aligning dimensions issue when empty arrays are passed to tensorstore write operations. * Add empty results check to prevent tensorstore alignment errors Adds validation in apply_transform_to_tczyx_and_save() to check for empty results dictionary before calling _save_transformed(). When no valid time points are available, logs diagnostic message and skips write operation instead of attempting to write empty arrays to tensorstore, which causes alignment dimension mismatches. * Revert "Fix tensorstore empty array handling" This reverts commit 65c9ddb. * better handling of output_time_indices * style * bugfix and better type hints * better messaging * style * raise error if attempting sharding along channel dimension --------- Co-authored-by: Alan Liddell <aliddell@chanzuckerberg.com> Co-authored-by: Ivan Ivanov <ivan.ivanov@czbiohub.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixing a bug where writing writing with
_save_transformedthrows an error when all chunks returned by_apply_transform_to_czyxareNone