…STSEC-2026-0001` (#19666) (#19686)

## Which issue does this PR close?

- part of #18566

## Rationale for this change

Let's resolve the audit workflow on branch-52 by removing our dependency
on the `rust_decimal` library, per @xudong963 's request here
#19666 (review)

## What changes are included in this PR?

- Backport  #19666 to branch-52

## Are these changes tested?

By CI

## Are there any user-facing changes?
No, this is an internal dev tool change

… constructors (#19681)

Backport of #19668 to branch-52.

This PR cherry-picks commit 680ddcc from main.

Includes:
- Split of BatchPartitioner::try_new into hash and round-robin
constructors
- Documentation improvements
- No behavior changes


part of #18566

Co-authored-by: Your Name <[email protected]>

…sCache table scoped (#19704)

## Which issue does this PR close?

- part of #18566

## Rationale for this change

Backport the fix for this regression into 52 release branch:
-  #19573 

## What changes are included in this PR?

Backport these two commits to `branch-52` (cherry-pick was clean)
- 1037f0a / #19388
- e6049de / #19616

<details><summary>Commands</summary>
<p>

```shell
andrewlamb@Andrews-MacBook-Pro-3:~/Software/datafusion$ git cherry-pick 1037f0a
[branch-52 1fc70ac] feat: add list_files_cache table function for `datafusion-cli` (#19388)
 Author: jizezhang <[email protected]>
 Date: Tue Jan 6 05:23:39 2026 -0800
 5 files changed, 446 insertions(+), 31 deletions(-)
andrewlamb@Andrews-MacBook-Pro-3:~/Software/datafusion$ git cherry-pick  e6049de
Auto-merging datafusion/core/src/execution/context/mod.rs
[branch-52 aa3d413] Make default ListingFilesCache table scoped (#19616)
 Author: jizezhang <[email protected]>
 Date: Thu Jan 8 06:34:10 2026 -0800
 10 files changed, 474 insertions(+), 184 deletions(-)
```

</p>
</details> 

## Are these changes tested?

By CI and new tests

## Are there any user-facing changes?

A new datafusion-cli function and dropping a external table now clears
the listing cache

---------

Co-authored-by: jizezhang <[email protected]>

…19659) (#19705)

## Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->

- part of #18566

## Rationale for this change

I propose back porting the fix for
#19641 to 52 release

## What changes are included in this PR?

- Backport #19659

## Are these changes tested?
eYes

## Are there any user-facing changes?

bug fix

Co-authored-by: Kumar Ujjawal <[email protected]>

## Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->

- Part of #18566

## Rationale for this change

<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->

## What changes are included in this PR?

<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->

## Are these changes tested?

<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code

If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->

## Are there any user-facing changes?

<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->

<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->

## Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->

- part of #19784.

## Rationale for this change

<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->

This is a follow-up of #19573

## What changes are included in this PR?

<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->

Backport 
- #19804

## Are these changes tested?

<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code

If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->

Existing tests

## Are there any user-facing changes?

<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->
no
<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->

…itedBatchCoalescer (#19785) (#19836)

## Which issue does this PR close?

- part of #19784 
- related to #19785
- backport fix of #19781

## Rationale for this change

Backport a regression found by @bert-beyondloops into the 52 release
line

## What changes are included in this PR?
Backport fix for #19781,
#19785 to branch-52

## Are these changes tested?

Yes
## Are there any user-facing changes?

bug fix

Co-authored-by: Bert Vermeiren <[email protected]>
Co-authored-by: Bert Vermeiren <[email protected]>

Brings #19853 into `branch-52`

Co-authored-by: Pierre Lacave <[email protected]>

…19877)

- part of #19784

- Brings #19716 from @adriangb
into `branch-52`

Co-authored-by: Adrian Garcia Badaracco <[email protected]>

## Which issue does this PR close?

- part of #19784
## Rationale for this change

Prepare for release

## What changes are included in this PR?

Update release version to 52.1.0 and add changelog

## Are these changes tested?

By CI
## Are there any user-facing changes?

New version

## Which issue does this PR close?

- related to #20287

## Rationale for this change

see issue #20109

## What changes are included in this PR?

1. Remap parent filter expressions: When a FilterExec has a projection,
remap unsupported parent filter expressions from output schema
coordinates to input schema coordinates using `reassign_expr_columns()`
before combining them with the current filter's predicates.

2. Preserve projection: When creating the merged FilterExec, preserve
the original projection instead of discarding it .

## Are these changes tested?

yes, add some test case

## Are there any user-facing changes?

---------

## Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->

- Closes #.

## Rationale for this change

<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->

## What changes are included in this PR?

<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->

## Are these changes tested?

<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code

If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->

## Are there any user-facing changes?

<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->

<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->

---------

Co-authored-by: Adrian Garcia Badaracco <[email protected]>

…#20286) (#20347)

- part of #20287
- related to  #20285
- backports 0f74dbf /
#20286 from @jackkleeman to
branch-52

) (#20509)

- Part of #20287
- Closes #20508 on branch-52

This PR simply
- Backports #20329 from
@adriangb to the branch-52 line


I simply cherry-picked 53b0ffb to the branch-52 branch

```shell
andrewlamb@Andrews-MacBook-Pro-3:~/Software/datafusion2$ git cherry-pick 53b0ffb
Auto-merging datafusion/core/tests/physical_optimizer/partition_statistics.rs
Auto-merging datafusion/datasource/src/file_scan_config.rs
[alamb/backport_20329 9286563] fix: validate inter-file ordering in eq_properties() (#20329)
 Author: Adrian Garcia Badaracco <[email protected]>
 Date: Sat Feb 14 14:04:01 2026 -0500
 5 files changed, 660 insertions(+), 47 deletions(-)
```

Co-authored-by: Adrian Garcia Badaracco <[email protected]>
Co-authored-by: Claude Opus 4.6 <[email protected]>

- Closes #17508

The previous implementation used UUID-based aliasing as a workaround to
prevent duplicate names for literals in Substrait plans. This approach
had several drawbacks:
- Non-deterministic plan names that made testing difficult (requiring
UUID regex filters)
- Only addressed literal naming conflicts, not the broader issue of name
deduplication
- Added unnecessary dependency on the `uuid` crate
- Didn't properly handle cases where the same qualified name could
appear with different schema representations

  1. Enhanced NameTracker: Refactored to detect two types of conflicts:
- Duplicate schema names: Tracked via schema_name() to prevent
validate_unique_names failures (e.g., two Utf8(NULL) literals)
- Ambiguous references: Tracked via qualified_name() to prevent
DFSchema::check_names failures when a qualified field (e.g.,
left.Utf8(NULL)) and unqualified field (e.g., Utf8(NULL)) share the same
column name
2. **Removed UUID dependency**: Eliminated the `uuid` crate from
`datafusion/substrait`
3. **Removed literal-specific aliasing**: The UUID-based workaround in
`project_rel.rs` is no longer needed as the improved NameTracker handles
all naming conflicts consistently
4. **Deterministic naming**: Name conflicts now use predictable
`__temp__N` suffixes instead of random UUIDs

Note: This doesn't fully fix all the issues in #17508 which allow some
special casing of `CAST` which are not included here.

Yes:
- Updated snapshot tests to reflect the new deterministic naming (e.g.,
`Utf8("people")__temp__0` instead of UUID-based names)
- Modified some roundtrip tests to verify semantic equivalence (schema
matching and execution) rather than exact string matching, which is more
robust
- All existing integration tests pass with the new naming scheme

Minimal. The generated plan names are now deterministic and more
readable (using `__temp__N` suffixes instead of UUIDs), but this is
primarily an internal representation change. The functional behavior and
query results remain unchanged.

## Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->

- Closes #.

## Rationale for this change

<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->

## What changes are included in this PR?

<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->

## Are these changes tested?

<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code

If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->

## Are there any user-facing changes?

<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->

<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->

Co-authored-by: Xander <[email protected]>

…lti-key joins (#20441) (#20512)

- part of #20287
- Closes #20437 on branch-52
- Back port of #20441 from
@Tim-53 / @adriangb

Made using

```shell
git cherry-pick ffc5b55
git cherry-pick a18be6f
```

---------

Co-authored-by: Tim-53 <[email protected]>

…20247) (#20507)

## Which issue does this PR close?
- part of #20287
- Fixes #20287 on branch-52
## Rationale for this change
See issues

## What changes are included in this PR?

- backports #20247

## Are these changes tested?

By CI
## Are there any user-facing changes?

<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->

<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->

---------

Co-authored-by: Mustafa Akur <[email protected]>

…0546)

## Which issue does this PR close?

- Part of #20287

## Rationale for this change

The security audit CI check [failed
here](https://github.com/apache/datafusion/actions/runs/22381549301/job/64783156671?pr=20539)
on
- #20539

This is due to some dependencies being yanked (aws-smithy specifically)

## What changes are included in this PR?

Let's update the relevant dependencies with small security related fixes

## Are these changes tested?

<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code

If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->

## Are there any user-facing changes?

<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->

<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->

…ing partial group ordering (#20446) (#20558)

## Which issue does this PR close?
- part of #20287
- Fixes #20445 on branch-52
## Rationale for this change
See issues

## What changes are included in this PR?

- backports #20446

## Are these changes tested?

By CI

Co-authored-by: Jack Kleeman <[email protected]>

## Which issue does this PR close?

- part of #20287

## Rationale for this change

Prepare for release

## What changes are included in this PR?

1. Update version 
2. prepare CHANGELOG. See rendered version here:
https://github.com/alamb/datafusion/blob/alamb/prepare_52.2.0/dev/changelog/52.2.0.md

I'll have to update this at least once more once we merge the other
outstanding PRs:

```shell
./dev/release/generate-changelog.py  52.1.0 branch-52 52.2.0 > dev/changelog/52.2.0.md
npx [email protected]  -w dev/changelog/52.2.0.md
```

Note the security audit CI check will fail until we merge
- #20546


## Are these changes tested?

By CI
## Are there any user-facing changes?

<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->

<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->

---------

Co-authored-by: Oleks V <[email protected]>