Commit b211db5
authored
fix(delete): use Add metadata for partition only DELETE (delta-io#4150)
# Description
Fix regression where `DELETE` with partition only predicates failed to
remove empty files in matching partitions.
**Root Cause:**
When the delete predicate references only partition columns, file
removal should be decided from log metadata alone. Prior implementation
relied on scan derived results and empty files (zero rows) produced no
matches. They weren't removed even though their partition satisfied the
predicate.
**Changes:**
- Add partition only delete fast path: evaluates predicate against
`Add.partition_values` (metadata only)
- Refine partition only predicate detection, remove redundant validation
- Add `remove_from_add` helper for removing files via their `Add`
actions
- Add helper for partition predicate file finding
**Tests:**
Regression coverage for:
- Partition only deletes remove empty files in matching partitions
(repro for delta-io#4149)
- NULL partition values handled correctly
- Partition-only path avoids unnecessary scanning
```bash
cargo fmt --check
cargo test --workspace
cargo test -p deltalake-core --features datafusion
```
# Related Issue(s)
- Fixes delta-io#4149
<!---
For example:
- closes #106
--->
# Documentation
<!---
Share links to useful documentation
--->
Notes
- Scoped to partition only predicates; intended semantics preserved
- NULL partition handling explicitly tested
---------
Signed-off-by: Ethan Urbanski <ethan@urbanskitech.com>1 parent 4ac9f71 commit b211db5
2 files changed
Lines changed: 519 additions & 15 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
410 | 410 | | |
411 | 411 | | |
412 | 412 | | |
413 | | - | |
414 | | - | |
415 | | - | |
416 | | - | |
417 | 413 | | |
418 | 414 | | |
419 | 415 | | |
| |||
467 | 463 | | |
468 | 464 | | |
469 | 465 | | |
470 | | - | |
471 | 466 | | |
472 | | - | |
| 467 | + | |
473 | 468 | | |
474 | 469 | | |
475 | 470 | | |
| |||
554 | 549 | | |
555 | 550 | | |
556 | 551 | | |
557 | | - | |
558 | 552 | | |
559 | 553 | | |
560 | 554 | | |
| |||
0 commit comments