Move logical expression type-coercion code from physical-expr crate to expr crate#2257
Conversation
datafuision-exprdatafusion-expr
datafusion-exprphysical-expr crate to expr crate
physical-expr crate to expr cratephysical-expr crate to expr crate
|
@alamb @jimexist Apologies for the monster PR. I am happy to try and break this down into smaller changes if that will help with reviews. There are no functional changes, just moving code around to fix one of the dependency issues that is blocking me from making progress with subquery support. I'm not sure who else I should be @ mentioning here so feel free to tag others that you think should review this. |
|
|
||
| /// Return `true` if `arg_type` is of a [`DataType`] that the | ||
| /// [`ApproxPercentileCont`] aggregation can operate on. | ||
| pub fn is_approx_percentile_cont_supported_arg_type(arg_type: &DataType) -> bool { |
There was a problem hiding this comment.
I've thought of a trait with supported_arg_type and return_type methods before but seems not much difference.
alamb
left a comment
There was a problem hiding this comment.
I think moving more coercion / expr logic into datafusion-expr makes sense -- thank you @andygrove
FWIW I think there was some discussion of keeping the Expr -> PhysicalExpr mapping isolated (including the type coercion) to allow different implementations of PhysicalExprs for each Expr (I think @jorgecarleitao discussed this 1+ year ago).
I haven't heard this idea mentioned for a while and I think it would be quite challenging to implement given the current DataFusion codebase, and I think coercion logic close to the exprs is 👍
cc @liukun4515 who I think has also had some thoughts in this area
|
The background of my thoughts back then was: there are two types of "coercion rules":
imo the former is part of the step |
… to `expr` crate (apache#2257)
… to `expr` crate (apache#2257)
… to `expr` crate (apache#2257)
… to `expr` crate (apache#2257)
… to `expr` crate (apache#2257)
Which issue does this PR close?
Closes #2251.
Rationale for this change
We have logic to determine the return type of various logical expressions and this logic is called from
expr_schema.rsin thelogical_planmodule, but the code is located in thephysical-exprcrate and it does not need to be there.We could move this logic into the
datafusion-exprcrate instead and remove many logical plan dependencies on the datafusion-physical-exprcrate, as a step towards to moving the logical plan itself todatafusion-expr, which would allow us to have logical expressions that reference logical plans (needed for subquery support).What changes are included in this PR?
This PR moves logical type coercion code from
datafusion-physical-exprtodatafusion-expr. There are no functional changes.Note that physical type coercion logic remains in the
physical-exprcrate.Are there any user-facing changes?
Yes - code is moved between crates, so users will need to update their imports.