Skip to content

Commit b44e243

Browse files
Pajarajacloud-fan
authored andcommitted
[SQL][MINOR] Update comments and remove dead code in UnionLoop
### What changes were proposed in this pull request? Remove function assertNoRecursiveCTE which is no longer used (when recursive references from subqueries were enabled, this function became obsolete). Also edit comments and error message to reflect current state. ### Why are the changes needed? Code cleanliness. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Existing golden file tests in cte-recursion and postgreSQL/with.sql. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #51097 from Pajaraja/pavle-martinovic_data/rctecleanup. Authored-by: pavle-martinovic_data <[email protected]> Signed-off-by: Wenchen Fan <[email protected]>
1 parent 4554600 commit b44e243

File tree

3 files changed

+4
-26
lines changed

3 files changed

+4
-26
lines changed

common/utils/src/main/resources/error/error-conditions.json

Lines changed: 1 addition & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -3416,14 +3416,9 @@
34163416
"Invalid recursive reference found inside WITH RECURSIVE clause."
34173417
],
34183418
"subClass" : {
3419-
"NUMBER" : {
3420-
"message" : [
3421-
"Multiple self-references to one recursive CTE are not allowed."
3422-
]
3423-
},
34243419
"PLACE" : {
34253420
"message" : [
3426-
"Recursive references cannot be used on the right side of left outer/semi/anti joins, on the left side of right outer joins, in full outer joins, in aggregates, and in subquery expressions."
3421+
"Recursive references cannot be used on the right side of left outer/semi/anti joins, on the left side of right outer joins, in full outer joins and in aggregates"
34273422
]
34283423
}
34293424
},

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala

Lines changed: 2 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -947,30 +947,14 @@ trait CheckAnalysis extends LookupCatalog with QueryErrorsBase with PlanToString
947947
if (expr.plan.isStreaming) {
948948
plan.failAnalysis("INVALID_SUBQUERY_EXPRESSION.STREAMING_QUERY", Map.empty)
949949
}
950-
assertNoRecursiveCTE(expr.plan)
951950
checkAnalysis0(expr.plan)
952951
ValidateSubqueryExpression(plan, expr)
953952
}
954953

955-
private def assertNoRecursiveCTE(plan: LogicalPlan): Unit = {
956-
plan.foreach {
957-
case r: CTERelationRef if r.recursive =>
958-
throw new AnalysisException(
959-
errorClass = "INVALID_RECURSIVE_REFERENCE.PLACE",
960-
messageParameters = Map.empty)
961-
case p => p.expressions.filter(_.containsPattern(PLAN_EXPRESSION)).foreach {
962-
expr => expr.foreach {
963-
case s: SubqueryExpression => assertNoRecursiveCTE(s.plan)
964-
case _ =>
965-
}
966-
}
967-
}
968-
}
969-
970954
/**
971955
* Validate that collected metrics names are unique. The same name cannot be used for metrics
972-
* with different results. However multiple instances of metrics with with same result and name
973-
* are allowed (e.g. self-joins).
956+
* with different results. However, multiple instances of metrics with same result and name are
957+
* allowed (e.g. self-joins).
974958
*/
975959
private def checkCollectedMetrics(plan: LogicalPlan): Unit = {
976960
val metricsMap = mutable.Map.empty[String, CollectMetrics]

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1041,8 +1041,7 @@ object ColumnPruning extends Rule[LogicalPlan] {
10411041
p
10421042
}
10431043

1044-
// TODO: Pruning `UnionLoop`s needs to take into account both the outer `Project` and the inner
1045-
// `UnionLoopRef` nodes.
1044+
// Avoid pruning UnionLoop because of its recursive nature.
10461045
case p @ Project(_, _: UnionLoop) => p
10471046

10481047
// Prune unnecessary window expressions

0 commit comments

Comments
 (0)