You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
## Which issue does this PR close?
- Part of apache#19692
## Rationale for this change
We have backported a bunch of bug fixes to branch-53, so let's make sure
the release notes reflect that
## What changes are included in this PR?
I ran
```shell
uv run ./dev/release/generate-changelog.py 52.3.0 branch-53 53.0.0 > dev/changelog/53.0.0.md
```
And then had codex review via
```
› Please review dev/changelog/53.0.0.md to ensure it reflects all commits between where `apache/branch-53` and `main` diverged
```
Then I updated the change log to reflect the original authors not the
backport authors
## Are these changes tested?
By CI
Copy file name to clipboardExpand all lines: dev/changelog/53.0.0.md
+67-29Lines changed: 67 additions & 29 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -19,7 +19,7 @@ under the License.
19
19
20
20
# Apache DataFusion 53.0.0 Changelog
21
21
22
-
This release consists of 447 commits from 105 contributors. See credits at the end of this changelog for more information.
22
+
This release consists of 475 commits from 114 contributors. See credits at the end of this changelog for more information.
23
23
24
24
See the [upgrade guide](https://datafusion.apache.org/library-user-guide/upgrading.html) for information on how to upgrade from previous versions.
25
25
@@ -37,6 +37,7 @@ See the [upgrade guide](https://datafusion.apache.org/library-user-guide/upgradi
37
37
- Remove the statistics() api in execution plan [#20319](https://github.com/apache/datafusion/pull/20319) (xudong963)
38
38
- Remove recursive const check in `simplify_const_expr`[#20234](https://github.com/apache/datafusion/pull/20234) (AdamGS)
39
39
- Cache `PlanProperties`, add fast-path for `with_new_children`[#19792](https://github.com/apache/datafusion/pull/19792) (askalt)
40
+
-[branch-53] feat: parse `JsonAccess` as a binary operator, add `Operator::Colon`[#20717](https://github.com/apache/datafusion/pull/20717) (Samyak2)
40
41
41
42
**Performance related:**
42
43
@@ -96,6 +97,7 @@ See the [upgrade guide](https://datafusion.apache.org/library-user-guide/upgradi
96
97
- perf: Use Arrow vectorized eq kernel for IN list with column references [#20528](https://github.com/apache/datafusion/pull/20528) (zhangxffff)
97
98
- perf: Optimize `array_agg()` using `GroupsAccumulator`[#20504](https://github.com/apache/datafusion/pull/20504) (neilconway)
98
99
- perf: Optimize `array_to_string()`, support more types [#20553](https://github.com/apache/datafusion/pull/20553) (neilconway)
100
+
-[branch-53] perf: sort replace free()->try_grow() pattern with try_resize() to reduce memory pool interactions [#20733](https://github.com/apache/datafusion/pull/20733) (mbutrovich)
99
101
100
102
**Implemented enhancements:**
101
103
@@ -244,9 +246,11 @@ See the [upgrade guide](https://datafusion.apache.org/library-user-guide/upgradi
244
246
- add redirect for old upgrading.html URL to fix broken changelog links [#20582](https://github.com/apache/datafusion/pull/20582) (mishop-15)
- Add a protection to release candidate branch 52 [#19660](https://github.com/apache/datafusion/pull/19660) (xudong963)
251
255
- Downgrade aws-smithy-runtime, update `rust_decimal`, ignore RUSTSEC-2026-0001 to get clean CI [#19657](https://github.com/apache/datafusion/pull/19657) (alamb)
-[branch-53] FFI_TableOptions are using default values only [#20722](https://github.com/apache/datafusion/pull/20722) (timsaucer)
497
+
- chore(deps): pin substrait to `0.62.2`[#20827](https://github.com/apache/datafusion/pull/20827) (milenkovicm)
498
+
- chore(deps): pin substrait version [#20848](https://github.com/apache/datafusion/pull/20848) (milenkovicm)
499
+
-[branch-53] Fix repartition from dropping data when spilling (#20672) [#20792](https://github.com/apache/datafusion/pull/20792) (xanderbailey)
-[branch-53] fix: interval analysis error when have two filterexec that inner filter proves zero selectivity (#20743) [#20882](https://github.com/apache/datafusion/pull/20882) (haohuaijin)
504
+
-[branch-53] correct parquet leaf index mapping when schema contains struct cols (#20698) [#20884](https://github.com/apache/datafusion/pull/20884) (friendlymatthew)
505
+
-[branch-53] ser/de fetch in FilterExec (#20738) [#20883](https://github.com/apache/datafusion/pull/20883) (haohuaijin)
506
+
-[branch-53] fix: use try_shrink instead of shrink in try_resize (#20424) [#20890](https://github.com/apache/datafusion/pull/20890) (ariel-miculas)
507
+
-[branch-53] Reattach parquet metadata cache after deserializing in datafusion-proto (#20574) [#20891](https://github.com/apache/datafusion/pull/20891) (nathanb9)
508
+
-[branch-53] fix: do not recompute hash join exec properties if not required (#20900) [#20903](https://github.com/apache/datafusion/pull/20903) (askalt)
509
+
-[branch-53] fix(spark): handle divide-by-zero in Spark `mod`/`pmod` with ANSI mode support (#20461) [#20896](https://github.com/apache/datafusion/pull/20896) (davidlghellin)
510
+
-[branch-53] fix: Provide more generic API for the capacity limit parsing (#20372) [#20893](https://github.com/apache/datafusion/pull/20893) (erenavsarogullari)
-[branch-53] Fix DELETE/UPDATE filter extraction when predicates are pushed down into TableScan (#19884) [#20898](https://github.com/apache/datafusion/pull/20898) (kosiew)
513
+
-[branch-53] fix: preserve None projection semantics across FFI boundary in ForeignTableProvider::scan (#20393) [#20895](https://github.com/apache/datafusion/pull/20895) (Kontinuation)
-[branch-53] backport: Support Spark `array_contains` builtin function (#20685) [#20914](https://github.com/apache/datafusion/pull/20914) (comphead)
516
+
-[branch-53] Fix duplicate group keys after hash aggregation spill (#20724) (#20858) [#20918](https://github.com/apache/datafusion/pull/20918) (gboucher90)
517
+
-[branch-53] fix: SanityCheckPlan error with window functions and NVL filter (#20231) [#20932](https://github.com/apache/datafusion/pull/20932) (EeshanBembi)
489
518
490
519
## Credits
491
520
492
521
Thank you to everyone who contributed to this release. Here is a breakdown of commits (PRs merged) per contributor.
493
522
494
523
```
495
524
73 dependabot[bot]
496
-
35 Neil Conway
497
-
31 Kumar Ujjawal
498
-
27 Andrew Lamb
525
+
37 Neil Conway
526
+
32 Kumar Ujjawal
527
+
28 Andrew Lamb
499
528
26 Adrian Garcia Badaracco
500
529
21 Jeffrey Vo
501
530
13 cht42
502
-
10 Albert Skalt
503
-
10 kosiew
531
+
11 Albert Skalt
532
+
11 kosiew
504
533
10 lyne
505
534
8 Nuno Faria
535
+
8 Oleks V
506
536
7 Sergey Zhukov
507
537
7 xudong.w
508
538
6 Daniël Heres
539
+
6 Huaijin
509
540
5 Adam Gutglick
510
541
5 Gabriel
511
-
5 Oleks V
542
+
5 Jonathan Chen
512
543
4 Andy Grove
513
544
4 Dmitrii Blaginin
514
-
4 Huaijin
545
+
4 Eren Avsarogullari
515
546
4 Jack Kleeman
516
-
4 Jonathan Chen
517
-
4 Yongting You
518
547
4 notashes
519
548
4 theirix
520
-
3 Eren Avsarogullari
549
+
4 Tim Saucer
550
+
4 Yongting You
551
+
3 dario curreri
552
+
3 feniljain
521
553
3 Kazantsev Maksim
522
554
3 Kosta Tarasov
523
555
3 Liang-Chi Hsieh
524
556
3 Lía Adriana
525
-
3 Tim Saucer
526
-
3 Yu-Chuan Hung
527
-
3 dario curreri
528
-
3 feniljain
557
+
3 Marko Milenković
529
558
3 mishop-15
559
+
3 Yu-Chuan Hung
530
560
2 Acfboy
531
561
2 Alan Tang
562
+
2 David López
532
563
2 Devanshu
533
564
2 Frederic Branczyk
534
565
2 Ganesh Patil
566
+
2 Heran Lin
567
+
2 jizezhang
535
568
2 Miao
536
569
2 Michael Kleen
570
+
2 niebayes
537
571
2 Pepijn Van Eeckhoudt
538
572
2 Peter L
539
573
2 Subham Singhal
540
574
2 Tobias Schwarzinger
541
575
2 UBarney
576
+
2 Xander
542
577
2 Yuvraj
543
578
2 Zhang Xiaofeng
544
-
2 jizezhang
545
-
2 niebayes
546
579
1 Andrea Bozzo
547
580
1 Andrew Kane
548
581
1 Anjali Choudhary
549
582
1 Anna-Rose Lescure
583
+
1 Ariel Miculas-Trif
550
584
1 Aryan Anand
551
585
1 Aviral Garg
552
586
1 Bert Vermeiren
553
587
1 Brent Gardner
554
588
1 ChanTsune
555
-
1 David López
589
+
1 comphead
590
+
1 danielhumanmod
556
591
1 Dewey Dunnington
592
+
1 discord9
557
593
1 Divyansh Pratap Singh
558
594
1 Eesh Sagar Singh
595
+
1 EeshanBembi
559
596
1 Emil Ernerfeldt
560
597
1 Emily Matheys
561
598
1 Eric Chang
562
599
1 Evangeli Silva
600
+
1 Filip Wojciechowski
563
601
1 Filippo
564
602
1 Gabriel Ferraté
565
603
1 Gene Bordegaray
566
604
1 Geoffrey Claude
567
605
1 Goksel Kabadayi
606
+
1 Guillaume Boucher
568
607
1 Haresh Khanna
569
-
1 Heran Lin
608
+
1 hsiang-c
609
+
1 iamthinh
570
610
1 Josh Elkind
571
-
1 Marko Milenković
611
+
1 karuppuchamysuresh
612
+
1 Kristin Cowalcijk
572
613
1 Mason
614
+
1 Matt Butrovich
615
+
1 Matthew Kim
573
616
1 Mikhail Zabaluev
574
617
1 Mohit rao
618
+
1 nathan
575
619
1 Nathaniel J. Smith
576
620
1 Nick
577
621
1 Oleg V. Kozlyuk
578
622
1 Paul J. Davis
579
623
1 Pierre Lacave
624
+
1 pmallex
580
625
1 Qi Zhu
581
626
1 Raz Luvaton
582
627
1 Rosai
583
628
1 Ruihang Xia
629
+
1 Samyak Sarnayak
584
630
1 Sergio Esteves
585
631
1 Simon Vandel Sillesen
586
632
1 Siyuan Huang
587
633
1 Tim-53
588
634
1 Tushar Das
589
635
1 Vignesh
590
-
1 XL Liang
591
-
1 Xander
592
636
1 Xiangpeng Hao
593
-
1 comphead
594
-
1 danielhumanmod
595
-
1 discord9
596
-
1 hsiang-c
597
-
1 iamthinh
598
-
1 karuppuchamysuresh
599
-
1 pmallex
637
+
1 XL Liang
600
638
```
601
639
602
640
Thank you also to everyone who contributed in other ways such as filing issues, reviewing PRs, and providing feedback on this release.
0 commit comments