Commit ff7a650
fix: SanityCheckPlan error with window functions and NVL filter (#20231)
## Which issue does this PR close?
Closes #20194
## Rationale for this change
A query with `ROW_NUMBER() OVER (... ORDER BY CASE WHEN col='0' THEN 1
ELSE 0 END)` combined with a filter `nvl(t2.value_2_3,'0')='0'` fails
with a `SanityCheckPlan` error. This worked in 50.3.0 but broke in
52.1.0.
## What changes are included in this PR?
**Root cause**: `collect_columns_from_predicate_inner` was extracting
equality pairs where neither side was a `Column` (e.g. `nvl(col, '0') =
'0'`), creating equivalence classes between complex expressions and
literals. `normalize_expr`'s deep traversal would then replace the
literal `'0'` inside unrelated sort/window CASE WHEN expressions with
the complex NVL expression, corrupting the sort ordering and causing a
mismatch between `SortExec`'s reported output ordering and
`BoundedWindowAggExec`'s expected ordering.
**Fix** (two changes in `filter.rs`):
1. **`collect_columns_from_predicate_inner`**: Only extract equality
pairs where at least one side is a `Column` reference. This matches the
function's documented intent ("Column-Pairs") and prevents
complex-expression-to-literal equivalence classes from being created.
2. **`extend_constants`**: Recognize `Literal` expressions as inherently
constant (previously only checked `is_expr_constant` on the input's
equivalence properties, which doesn't know about literals). This ensures
constant propagation still works for `complex_expr = literal` predicates
— e.g. `nvl(col, '0')` is properly marked as constant after the filter.
## How was this tested?
- Unit test `test_collect_columns_skips_non_column_pairs` verifying the
filtering logic
- Sqllogictest reproducing the exact query from the issue
- Full test suites: equivalence tests (51 passed), physical-plan tests
(1255 passed), physical-optimizer tests (20 passed)
- Manual verification with datafusion-cli running the reproduction query
## Test plan
- [x] Unit test for `collect_columns_from_predicate_inner` column
filtering
- [x] Sqllogictest regression test for #20194
- [x] Existing test suites pass
- [x] Manual reproduction query succeeds
---------
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>1 parent 01437a2 commit ff7a650
2 files changed
Lines changed: 134 additions & 13 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
57 | 57 | | |
58 | 58 | | |
59 | 59 | | |
60 | | - | |
| 60 | + | |
61 | 61 | | |
62 | 62 | | |
63 | 63 | | |
64 | | - | |
65 | | - | |
| 64 | + | |
| 65 | + | |
66 | 66 | | |
67 | 67 | | |
68 | 68 | | |
| |||
348 | 348 | | |
349 | 349 | | |
350 | 350 | | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
351 | 365 | | |
352 | 366 | | |
353 | 367 | | |
| |||
360 | 374 | | |
361 | 375 | | |
362 | 376 | | |
363 | | - | |
364 | | - | |
365 | | - | |
366 | | - | |
367 | | - | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
368 | 389 | | |
369 | 390 | | |
370 | | - | |
371 | | - | |
372 | | - | |
373 | | - | |
374 | | - | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
375 | 395 | | |
376 | 396 | | |
377 | 397 | | |
| |||
1003 | 1023 | | |
1004 | 1024 | | |
1005 | 1025 | | |
| 1026 | + | |
| 1027 | + | |
| 1028 | + | |
| 1029 | + | |
| 1030 | + | |
| 1031 | + | |
| 1032 | + | |
| 1033 | + | |
| 1034 | + | |
| 1035 | + | |
| 1036 | + | |
| 1037 | + | |
| 1038 | + | |
1006 | 1039 | | |
1007 | 1040 | | |
1008 | 1041 | | |
| |||
2155 | 2188 | | |
2156 | 2189 | | |
2157 | 2190 | | |
| 2191 | + | |
| 2192 | + | |
| 2193 | + | |
| 2194 | + | |
| 2195 | + | |
| 2196 | + | |
| 2197 | + | |
| 2198 | + | |
| 2199 | + | |
| 2200 | + | |
| 2201 | + | |
| 2202 | + | |
| 2203 | + | |
| 2204 | + | |
| 2205 | + | |
| 2206 | + | |
| 2207 | + | |
| 2208 | + | |
| 2209 | + | |
| 2210 | + | |
| 2211 | + | |
| 2212 | + | |
| 2213 | + | |
| 2214 | + | |
| 2215 | + | |
| 2216 | + | |
| 2217 | + | |
| 2218 | + | |
| 2219 | + | |
| 2220 | + | |
| 2221 | + | |
| 2222 | + | |
| 2223 | + | |
| 2224 | + | |
| 2225 | + | |
| 2226 | + | |
| 2227 | + | |
| 2228 | + | |
| 2229 | + | |
| 2230 | + | |
| 2231 | + | |
| 2232 | + | |
2158 | 2233 | | |
2159 | 2234 | | |
2160 | 2235 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6081 | 6081 | | |
6082 | 6082 | | |
6083 | 6083 | | |
| 6084 | + | |
| 6085 | + | |
| 6086 | + | |
| 6087 | + | |
| 6088 | + | |
| 6089 | + | |
| 6090 | + | |
| 6091 | + | |
| 6092 | + | |
| 6093 | + | |
| 6094 | + | |
| 6095 | + | |
| 6096 | + | |
| 6097 | + | |
| 6098 | + | |
| 6099 | + | |
| 6100 | + | |
| 6101 | + | |
| 6102 | + | |
| 6103 | + | |
| 6104 | + | |
| 6105 | + | |
| 6106 | + | |
| 6107 | + | |
| 6108 | + | |
| 6109 | + | |
| 6110 | + | |
| 6111 | + | |
| 6112 | + | |
| 6113 | + | |
| 6114 | + | |
| 6115 | + | |
| 6116 | + | |
| 6117 | + | |
| 6118 | + | |
| 6119 | + | |
| 6120 | + | |
| 6121 | + | |
| 6122 | + | |
| 6123 | + | |
| 6124 | + | |
| 6125 | + | |
| 6126 | + | |
| 6127 | + | |
| 6128 | + | |
| 6129 | + | |
0 commit comments