Skip to content

Commit 28fc91a

Browse files
authored
[branch-53] Update Release Notes (apache#20952)
## Which issue does this PR close? - Part of apache#19692 ## Rationale for this change We have backported a bunch of bug fixes to branch-53, so let's make sure the release notes reflect that ## What changes are included in this PR? I ran ```shell uv run ./dev/release/generate-changelog.py 52.3.0 branch-53 53.0.0 > dev/changelog/53.0.0.md ``` And then had codex review via ``` › Please review dev/changelog/53.0.0.md to ensure it reflects all commits between where `apache/branch-53` and `main` diverged ``` Then I updated the change log to reflect the original authors not the backport authors ## Are these changes tested? By CI
1 parent 2c1ca2f commit 28fc91a

1 file changed

Lines changed: 67 additions & 29 deletions

File tree

dev/changelog/53.0.0.md

Lines changed: 67 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ under the License.
1919

2020
# Apache DataFusion 53.0.0 Changelog
2121

22-
This release consists of 447 commits from 105 contributors. See credits at the end of this changelog for more information.
22+
This release consists of 475 commits from 114 contributors. See credits at the end of this changelog for more information.
2323

2424
See the [upgrade guide](https://datafusion.apache.org/library-user-guide/upgrading.html) for information on how to upgrade from previous versions.
2525

@@ -37,6 +37,7 @@ See the [upgrade guide](https://datafusion.apache.org/library-user-guide/upgradi
3737
- Remove the statistics() api in execution plan [#20319](https://github.com/apache/datafusion/pull/20319) (xudong963)
3838
- Remove recursive const check in `simplify_const_expr` [#20234](https://github.com/apache/datafusion/pull/20234) (AdamGS)
3939
- Cache `PlanProperties`, add fast-path for `with_new_children` [#19792](https://github.com/apache/datafusion/pull/19792) (askalt)
40+
- [branch-53] feat: parse `JsonAccess` as a binary operator, add `Operator::Colon` [#20717](https://github.com/apache/datafusion/pull/20717) (Samyak2)
4041

4142
**Performance related:**
4243

@@ -96,6 +97,7 @@ See the [upgrade guide](https://datafusion.apache.org/library-user-guide/upgradi
9697
- perf: Use Arrow vectorized eq kernel for IN list with column references [#20528](https://github.com/apache/datafusion/pull/20528) (zhangxffff)
9798
- perf: Optimize `array_agg()` using `GroupsAccumulator` [#20504](https://github.com/apache/datafusion/pull/20504) (neilconway)
9899
- perf: Optimize `array_to_string()`, support more types [#20553](https://github.com/apache/datafusion/pull/20553) (neilconway)
100+
- [branch-53] perf: sort replace free()->try_grow() pattern with try_resize() to reduce memory pool interactions [#20733](https://github.com/apache/datafusion/pull/20733) (mbutrovich)
99101

100102
**Implemented enhancements:**
101103

@@ -244,9 +246,11 @@ See the [upgrade guide](https://datafusion.apache.org/library-user-guide/upgradi
244246
- add redirect for old upgrading.html URL to fix broken changelog links [#20582](https://github.com/apache/datafusion/pull/20582) (mishop-15)
245247
- Upgrade DataFusion to arrow-rs/parquet 58.0.0 / `object_store` 0.13.0 [#19728](https://github.com/apache/datafusion/pull/19728) (alamb)
246248
- Document guidance on how to evaluate breaking API changes [#20584](https://github.com/apache/datafusion/pull/20584) (alamb)
249+
- [branch-53] chore: prepare 53 release [#20649](https://github.com/apache/datafusion/pull/20649) (comphead)
247250

248251
**Other:**
249252

253+
- [branch-53] chore: Add branch protection (comphead)
250254
- Add a protection to release candidate branch 52 [#19660](https://github.com/apache/datafusion/pull/19660) (xudong963)
251255
- Downgrade aws-smithy-runtime, update `rust_decimal`, ignore RUSTSEC-2026-0001 to get clean CI [#19657](https://github.com/apache/datafusion/pull/19657) (alamb)
252256
- Update dependencies [#19667](https://github.com/apache/datafusion/pull/19667) (alamb)
@@ -486,117 +490,151 @@ See the [upgrade guide](https://datafusion.apache.org/library-user-guide/upgradi
486490
- Add deterministic per-file timing summary to sqllogictest runner [#20569](https://github.com/apache/datafusion/pull/20569) (kosiew)
487491
- chore: Enable workspace lint for all workspace members [#20577](https://github.com/apache/datafusion/pull/20577) (neilconway)
488492
- Fix serde of window lead/lag defaults [#20608](https://github.com/apache/datafusion/pull/20608) (avantgardnerio)
493+
- [branch-53] fix: make the `sql` feature truly optional (#20625) [#20680](https://github.com/apache/datafusion/pull/20680) (linhr)
494+
- [53] fix: Fix bug in `array_has` scalar path with sliced arrays (#20677) [#20700](https://github.com/apache/datafusion/pull/20700) (neilconway)
495+
- [branch-53] fix: Return `probe_side.len()` for RightMark/Anti count(\*) queries (#… [#20726](https://github.com/apache/datafusion/pull/20726) (jonathanc-n)
496+
- [branch-53] FFI_TableOptions are using default values only [#20722](https://github.com/apache/datafusion/pull/20722) (timsaucer)
497+
- chore(deps): pin substrait to `0.62.2` [#20827](https://github.com/apache/datafusion/pull/20827) (milenkovicm)
498+
- chore(deps): pin substrait version [#20848](https://github.com/apache/datafusion/pull/20848) (milenkovicm)
499+
- [branch-53] Fix repartition from dropping data when spilling (#20672) [#20792](https://github.com/apache/datafusion/pull/20792) (xanderbailey)
500+
- [branch-53] fix: `HashJoin` panic with String dictionary keys (don't flatten keys) (#20505) [#20791](https://github.com/apache/datafusion/pull/20791) (alamb)
501+
- [branch-53] cli: Fix datafusion-cli hint edge cases (#20609) [#20887](https://github.com/apache/datafusion/pull/20887) (comphead)
502+
- [branch-53] perf: Optimize `to_char` to allocate less, fix NULL handling (#20635) [#20885](https://github.com/apache/datafusion/pull/20885) (neilconway)
503+
- [branch-53] fix: interval analysis error when have two filterexec that inner filter proves zero selectivity (#20743) [#20882](https://github.com/apache/datafusion/pull/20882) (haohuaijin)
504+
- [branch-53] correct parquet leaf index mapping when schema contains struct cols (#20698) [#20884](https://github.com/apache/datafusion/pull/20884) (friendlymatthew)
505+
- [branch-53] ser/de fetch in FilterExec (#20738) [#20883](https://github.com/apache/datafusion/pull/20883) (haohuaijin)
506+
- [branch-53] fix: use try_shrink instead of shrink in try_resize (#20424) [#20890](https://github.com/apache/datafusion/pull/20890) (ariel-miculas)
507+
- [branch-53] Reattach parquet metadata cache after deserializing in datafusion-proto (#20574) [#20891](https://github.com/apache/datafusion/pull/20891) (nathanb9)
508+
- [branch-53] fix: do not recompute hash join exec properties if not required (#20900) [#20903](https://github.com/apache/datafusion/pull/20903) (askalt)
509+
- [branch-53] fix(spark): handle divide-by-zero in Spark `mod`/`pmod` with ANSI mode support (#20461) [#20896](https://github.com/apache/datafusion/pull/20896) (davidlghellin)
510+
- [branch-53] fix: Provide more generic API for the capacity limit parsing (#20372) [#20893](https://github.com/apache/datafusion/pull/20893) (erenavsarogullari)
511+
- [branch-53] fix: sqllogictest cannot convert <subquery> to Substrait (#19739) [#20897](https://github.com/apache/datafusion/pull/20897) (kumarUjjawal)
512+
- [branch-53] Fix DELETE/UPDATE filter extraction when predicates are pushed down into TableScan (#19884) [#20898](https://github.com/apache/datafusion/pull/20898) (kosiew)
513+
- [branch-53] fix: preserve None projection semantics across FFI boundary in ForeignTableProvider::scan (#20393) [#20895](https://github.com/apache/datafusion/pull/20895) (Kontinuation)
514+
- [branch-53] Fix FilterExec converting Absent column stats to Exact(NULL) (#20391) [#20892](https://github.com/apache/datafusion/pull/20892) (fwojciec)
515+
- [branch-53] backport: Support Spark `array_contains` builtin function (#20685) [#20914](https://github.com/apache/datafusion/pull/20914) (comphead)
516+
- [branch-53] Fix duplicate group keys after hash aggregation spill (#20724) (#20858) [#20918](https://github.com/apache/datafusion/pull/20918) (gboucher90)
517+
- [branch-53] fix: SanityCheckPlan error with window functions and NVL filter (#20231) [#20932](https://github.com/apache/datafusion/pull/20932) (EeshanBembi)
489518

490519
## Credits
491520

492521
Thank you to everyone who contributed to this release. Here is a breakdown of commits (PRs merged) per contributor.
493522

494523
```
495524
73 dependabot[bot]
496-
35 Neil Conway
497-
31 Kumar Ujjawal
498-
27 Andrew Lamb
525+
37 Neil Conway
526+
32 Kumar Ujjawal
527+
28 Andrew Lamb
499528
26 Adrian Garcia Badaracco
500529
21 Jeffrey Vo
501530
13 cht42
502-
10 Albert Skalt
503-
10 kosiew
531+
11 Albert Skalt
532+
11 kosiew
504533
10 lyne
505534
8 Nuno Faria
535+
8 Oleks V
506536
7 Sergey Zhukov
507537
7 xudong.w
508538
6 Daniël Heres
539+
6 Huaijin
509540
5 Adam Gutglick
510541
5 Gabriel
511-
5 Oleks V
542+
5 Jonathan Chen
512543
4 Andy Grove
513544
4 Dmitrii Blaginin
514-
4 Huaijin
545+
4 Eren Avsarogullari
515546
4 Jack Kleeman
516-
4 Jonathan Chen
517-
4 Yongting You
518547
4 notashes
519548
4 theirix
520-
3 Eren Avsarogullari
549+
4 Tim Saucer
550+
4 Yongting You
551+
3 dario curreri
552+
3 feniljain
521553
3 Kazantsev Maksim
522554
3 Kosta Tarasov
523555
3 Liang-Chi Hsieh
524556
3 Lía Adriana
525-
3 Tim Saucer
526-
3 Yu-Chuan Hung
527-
3 dario curreri
528-
3 feniljain
557+
3 Marko Milenković
529558
3 mishop-15
559+
3 Yu-Chuan Hung
530560
2 Acfboy
531561
2 Alan Tang
562+
2 David López
532563
2 Devanshu
533564
2 Frederic Branczyk
534565
2 Ganesh Patil
566+
2 Heran Lin
567+
2 jizezhang
535568
2 Miao
536569
2 Michael Kleen
570+
2 niebayes
537571
2 Pepijn Van Eeckhoudt
538572
2 Peter L
539573
2 Subham Singhal
540574
2 Tobias Schwarzinger
541575
2 UBarney
576+
2 Xander
542577
2 Yuvraj
543578
2 Zhang Xiaofeng
544-
2 jizezhang
545-
2 niebayes
546579
1 Andrea Bozzo
547580
1 Andrew Kane
548581
1 Anjali Choudhary
549582
1 Anna-Rose Lescure
583+
1 Ariel Miculas-Trif
550584
1 Aryan Anand
551585
1 Aviral Garg
552586
1 Bert Vermeiren
553587
1 Brent Gardner
554588
1 ChanTsune
555-
1 David López
589+
1 comphead
590+
1 danielhumanmod
556591
1 Dewey Dunnington
592+
1 discord9
557593
1 Divyansh Pratap Singh
558594
1 Eesh Sagar Singh
595+
1 EeshanBembi
559596
1 Emil Ernerfeldt
560597
1 Emily Matheys
561598
1 Eric Chang
562599
1 Evangeli Silva
600+
1 Filip Wojciechowski
563601
1 Filippo
564602
1 Gabriel Ferraté
565603
1 Gene Bordegaray
566604
1 Geoffrey Claude
567605
1 Goksel Kabadayi
606+
1 Guillaume Boucher
568607
1 Haresh Khanna
569-
1 Heran Lin
608+
1 hsiang-c
609+
1 iamthinh
570610
1 Josh Elkind
571-
1 Marko Milenković
611+
1 karuppuchamysuresh
612+
1 Kristin Cowalcijk
572613
1 Mason
614+
1 Matt Butrovich
615+
1 Matthew Kim
573616
1 Mikhail Zabaluev
574617
1 Mohit rao
618+
1 nathan
575619
1 Nathaniel J. Smith
576620
1 Nick
577621
1 Oleg V. Kozlyuk
578622
1 Paul J. Davis
579623
1 Pierre Lacave
624+
1 pmallex
580625
1 Qi Zhu
581626
1 Raz Luvaton
582627
1 Rosai
583628
1 Ruihang Xia
629+
1 Samyak Sarnayak
584630
1 Sergio Esteves
585631
1 Simon Vandel Sillesen
586632
1 Siyuan Huang
587633
1 Tim-53
588634
1 Tushar Das
589635
1 Vignesh
590-
1 XL Liang
591-
1 Xander
592636
1 Xiangpeng Hao
593-
1 comphead
594-
1 danielhumanmod
595-
1 discord9
596-
1 hsiang-c
597-
1 iamthinh
598-
1 karuppuchamysuresh
599-
1 pmallex
637+
1 XL Liang
600638
```
601639

602640
Thank you also to everyone who contributed in other ways such as filing issues, reviewing PRs, and providing feedback on this release.

0 commit comments

Comments
 (0)