Skip to content

refactor: consolidate extension planners#3804

Merged
roeap merged 3 commits intodelta-io:mainfrom
roeap:refactor/ext-planners
Oct 3, 2025
Merged

refactor: consolidate extension planners#3804
roeap merged 3 commits intodelta-io:mainfrom
roeap:refactor/ext-planners

Conversation

@roeap
Copy link
Copy Markdown
Collaborator

@roeap roeap commented Oct 3, 2025

Description

We currently create a dedicated session with a dedicated planner in some operations to collect metrics. This creates a situation where - even if we pass a session - we need to recreate the session on the fly, potentially replacing an external planner and thus potentially loosing support for UDFs or other custom nodes.

This PR consolidates the various extension planner we currently have into a single planner to eventually make it easy for us to centralise session creation or allow others to extend their own planner with the delta extension planners.

cc @abhi-airspace-intelligence

Related Issue(s)

part-of: #3799

@roeap roeap requested review from hntd187 and rtyler as code owners October 3, 2025 07:34
@github-actions github-actions bot added the binding/rust Issues for the Rust crate label Oct 3, 2025
@roeap roeap requested a review from ion-elgreco October 3, 2025 07:34
@codecov
Copy link
Copy Markdown

codecov bot commented Oct 3, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.07%. Comparing base (fb73c6f) to head (a776a97).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3804      +/-   ##
==========================================
+ Coverage   76.05%   76.07%   +0.02%     
==========================================
  Files         145      145              
  Lines       45274    45313      +39     
  Branches    45274    45313      +39     
==========================================
+ Hits        34433    34472      +39     
  Misses       9150     9150              
  Partials     1691     1691              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Signed-off-by: Robert Pack <robstar.pack@gmail.com>
@roeap roeap force-pushed the refactor/ext-planners branch from e87e09c to 48e5ef9 Compare October 3, 2025 07:46
@roeap roeap moved this to In review in delta-rust Oct 3, 2025
@ion-elgreco
Copy link
Copy Markdown
Collaborator

Correct me if I am wrong, but adding all extensions planner together will cause the plan become messy and do multiple metric counts

@roeap
Copy link
Copy Markdown
Collaborator Author

roeap commented Oct 3, 2025

Correct me if I am wrong, but adding all extensions planner together will cause the plan become messy and do multiple metric counts

The individual planners, while all downcasting to the same MetricObserver node, also match on unqiue IDs on the node for their respective metrics, as long as these don't overlap, only one planner should ever return a plan..

In case there are duplicate IDs we would potentially match wrong depending on the first planner that signifies it can plan the extension node (returning Some(...)). However we do always plan a node at most once .

@ion-elgreco
Copy link
Copy Markdown
Collaborator

Correct me if I am wrong, but adding all extensions planner together will cause the plan become messy and do multiple metric counts

The individual planners, while all downcasting to the same MetricObserver node, also match on unqiue IDs on the node for their respective metrics, as long as these don't overlap, only one planner should ever return a plan..

In case there are duplicate IDs we would potentially match wrong depending on the first planner that signifies it can plan the extension node (returning Some(...)). However we do always plan a node at most once .

Yeah, I thought our IDs werent unique but it's been a while that I've touched these extension planners

@roeap
Copy link
Copy Markdown
Collaborator Author

roeap commented Oct 3, 2025

it's been a while that I've touched these extension planners

same 😆

In a post kernelized world, I do believe we'll see much more of the likes though ...

@roeap roeap enabled auto-merge (squash) October 3, 2025 08:31
@roeap roeap merged commit c5eb4c2 into delta-io:main Oct 3, 2025
62 of 66 checks passed
@github-project-automation github-project-automation bot moved this from In review to Done in delta-rust Oct 3, 2025
@roeap roeap deleted the refactor/ext-planners branch October 3, 2025 09:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

binding/rust Issues for the Rust crate

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

2 participants