Skip to content

Commit df8e507

Browse files
jkschneiderjkschneider
authored
Enable Java recipes for Python via ty LSP type attribution (#6593)
* Enable Java recipes for Python via ty LSP type attribution This change enables 7 Java building-block recipes to work on Python code: - FindMethods - FindTypes - ChangeMethodName - ChangeType - DeleteMethodArgument - ReorderMethodArguments - ChangePackage (via recipes.csv entry) Key changes: 1. Type attribution via ty LSP client - New TyLspClient that runs `ty server` as a long-lived subprocess - Communicates via LSP protocol over stdin/stdout - Provides hover-based type information for Python code 2. Enhanced PythonTypeMapping - Maps Python types to JavaType equivalents for MethodMatcher - Parses method signatures from ty hover responses to extract parameter names and types - Handles declaring type resolution for both builtins and modules 3. Python-specific import recipes - AddImport: Add Python imports with proper formatting - RemoveImport: Remove unused Python imports - ChangeImport: Migrate imports between modules 4. Dependency workspace support - DependencyWorkspace.java manages cached venvs for type resolution - LRU cache with automatic cleanup of evicted entries - Enables type attribution for external packages 5. Integration tests for all 7 recipes - Tests use builtin types (str, list) that work without dependencies - Validates both positive matches and negative cases Technical notes: - ty is required for full type attribution; without it, recipes fall back to placeholder parameter names - Parameter names are parsed from ty hover responses like: `def split(sep: str | None = ..., maxsplit: SupportsIndex = ...) -> list[str]`
1 parent e0d7e9a commit df8e507

26 files changed

Lines changed: 4508 additions & 160 deletions

‎rewrite-java/src/main/resources/META-INF/rewrite/recipes.csv‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -104,3 +104,10 @@ maven,org.openrewrite:rewrite-java,org.openrewrite.java.DeleteMethodArgument,Del
104104
maven,org.openrewrite:rewrite-java,org.openrewrite.java.ReorderMethodArguments,Reorder method arguments,Reorder method arguments into the specified order.,1,,JavaScript,,Basic building blocks for transforming JavaScript code.,"[{""name"":""methodPattern"",""type"":""String"",""displayName"":""Method pattern"",""description"":""A [method pattern](https://docs.openrewrite.org/reference/method-patterns) is used to find matching method invocations. For example, to find all method invocations in the Guava library, use the pattern: `com.google.common..*#*(..)`.<br/><br/>The pattern format is `<PACKAGE>#<METHOD_NAME>(<ARGS>)`. <br/><br/>`..*` includes all subpackages of `com.google.common`. <br/>`*(..)` matches any method name with any number of arguments. <br/><br/>For more specific queries, like Guava's `ImmutableMap`, use `com.google.common.collect.ImmutableMap#*(..)` to narrow down the results."",""example"":""com.yourorg.A foo(String, Integer, Integer)"",""required"":true},{""name"":""newParameterNames"",""type"":""String[]"",""displayName"":""New parameter names"",""description"":""An array of parameter names that indicates the new order in which those arguments should be arranged."",""example"":""[foo, bar, baz]"",""required"":true},{""name"":""oldParameterNames"",""type"":""String[]"",""displayName"":""Old parameter names"",""description"":""If the original method signature is not type-attributed, this is an optional list that indicates the original order in which the arguments were arranged."",""example"":""[baz, bar, foo]""},{""name"":""ignoreDefinition"",""type"":""Boolean"",""displayName"":""Ignore type definition"",""description"":""When set to `true` the definition of the old type will be left untouched. This is useful when you're replacing usage of a class but don't want to rename it.""},{""name"":""matchOverrides"",""type"":""Boolean"",""displayName"":""Match on overrides"",""description"":""When enabled, find methods that are overrides of the method pattern.""}]",
105105
maven,org.openrewrite:rewrite-java,org.openrewrite.java.search.FindMethods,Find method usages,Find method calls by pattern.,1,Search,JavaScript,,Basic building blocks for transforming JavaScript code.,"[{""name"":""methodPattern"",""type"":""String"",""displayName"":""Method pattern"",""description"":""A [method pattern](https://docs.openrewrite.org/reference/method-patterns) is used to find matching method invocations. For example, to find all method invocations in the Guava library, use the pattern: `com.google.common..*#*(..)`.<br/><br/>The pattern format is `<PACKAGE>#<METHOD_NAME>(<ARGS>)`. <br/><br/>`..*` includes all subpackages of `com.google.common`. <br/>`*(..)` matches any method name with any number of arguments. <br/><br/>For more specific queries, like Guava's `ImmutableMap`, use `com.google.common.collect.ImmutableMap#*(..)` to narrow down the results."",""example"":""java.util.List add(..)"",""required"":true},{""name"":""matchOverrides"",""type"":""Boolean"",""displayName"":""Match on overrides"",""description"":""When enabled, find methods that are overrides of the method pattern.""}]","[{""name"":""org.openrewrite.java.table.MethodCalls"",""displayName"":""Method calls"",""description"":""The text of matching method invocations."",""columns"":[{""name"":""sourceFile"",""type"":""String"",""displayName"":""Source file"",""description"":""The source file that the method call occurred in.""},{""name"":""method"",""type"":""String"",""displayName"":""Method call"",""description"":""The text of the method call.""},{""name"":""className"",""type"":""String"",""displayName"":""Class name"",""description"":""The class name of the method call.""},{""name"":""methodName"",""type"":""String"",""displayName"":""Method name"",""description"":""The method name of the method call.""},{""name"":""argumentTypes"",""type"":""String"",""displayName"":""Argument types"",""description"":""The argument types of the method call.""}]}]"
106106
maven,org.openrewrite:rewrite-java,org.openrewrite.java.search.FindTypes,Find types,Find type references by name.,1,Search,JavaScript,,Basic building blocks for transforming JavaScript code.,"[{""name"":""fullyQualifiedTypeName"",""type"":""String"",""displayName"":""Fully-qualified type name"",""description"":""A fully-qualified type name, that is used to find matching type references. Supports glob expressions. `java..*` finds every type from every subpackage of the `java` package."",""example"":""java.util.List"",""required"":true},{""name"":""checkAssignability"",""type"":""Boolean"",""displayName"":""Check for assignability"",""description"":""When enabled, find type references that are assignable to the provided type.""}]","[{""name"":""org.openrewrite.java.table.TypeUses"",""displayName"":""Type uses"",""description"":""The source code of matching type uses."",""columns"":[{""name"":""sourceFile"",""type"":""String"",""displayName"":""Source file"",""description"":""The source file that the method call occurred in.""},{""name"":""code"",""type"":""String"",""displayName"":""Source"",""description"":""The source code of the type use.""},{""name"":""concreteType"",""type"":""String"",""displayName"":""Concrete type"",""description"":""The concrete type in use, which may be a subtype of a searched type.""}]}]"
107+
maven,org.openrewrite:rewrite-java,org.openrewrite.java.ChangeMethodName,Change method name,Rename a method.,1,,Python,,Basic building blocks for transforming Python code.,"[{""name"":""methodPattern"",""type"":""String"",""displayName"":""Method pattern"",""description"":""A [method pattern](https://docs.openrewrite.org/reference/method-patterns) is used to find matching method invocations. For example, to find all method invocations in the Guava library, use the pattern: `com.google.common..*#*(..)`.<br/><br/>The pattern format is `<PACKAGE>#<METHOD_NAME>(<ARGS>)`. <br/><br/>`..*` includes all subpackages of `com.google.common`. <br/>`*(..)` matches any method name with any number of arguments. <br/><br/>For more specific queries, like Guava's `ImmutableMap`, use `com.google.common.collect.ImmutableMap#*(..)` to narrow down the results."",""example"":""org.mockito.Matchers anyVararg()"",""required"":true},{""name"":""newMethodName"",""type"":""String"",""displayName"":""New method name"",""description"":""The method name that will replace the existing name."",""example"":""any"",""required"":true},{""name"":""matchOverrides"",""type"":""Boolean"",""displayName"":""Match on overrides"",""description"":""When enabled, find methods that are overrides of the method pattern.""},{""name"":""ignoreDefinition"",""type"":""Boolean"",""displayName"":""Ignore type definition"",""description"":""When set to `true` the definition of the old type will be left untouched. This is useful when you're replacing usage of a class but don't want to rename it.""}]",
108+
maven,org.openrewrite:rewrite-java,org.openrewrite.java.ChangePackage,Rename package name,"A recipe that will rename a package name in package statements, imports, and fully-qualified types.",1,,Python,,Basic building blocks for transforming Python code.,"[{""name"":""oldPackageName"",""type"":""String"",""displayName"":""Old package name"",""description"":""The package name to replace."",""example"":""com.yourorg.foo"",""required"":true},{""name"":""newPackageName"",""type"":""String"",""displayName"":""New package name"",""description"":""New package name to replace the old package name with."",""example"":""com.yourorg.bar"",""required"":true},{""name"":""recursive"",""type"":""Boolean"",""displayName"":""Recursive"",""description"":""Recursively change subpackage names""}]",
109+
maven,org.openrewrite:rewrite-java,org.openrewrite.java.ChangeType,Change type,Change a given type to another.,1,,Python,,Basic building blocks for transforming Python code.,"[{""name"":""oldFullyQualifiedTypeName"",""type"":""String"",""displayName"":""Old fully-qualified type name"",""description"":""Fully-qualified class name of the original type."",""example"":""org.junit.Assume"",""required"":true},{""name"":""newFullyQualifiedTypeName"",""type"":""String"",""displayName"":""New fully-qualified type name"",""description"":""Fully-qualified class name of the replacement type, or the name of a primitive such as \""int\"". The `OuterClassName$NestedClassName` naming convention should be used for nested classes."",""example"":""org.junit.jupiter.api.Assumptions"",""required"":true},{""name"":""ignoreDefinition"",""type"":""Boolean"",""displayName"":""Ignore type definition"",""description"":""When set to `true` the definition of the old type will be left untouched. This is useful when you're replacing usage of a class but don't want to rename it.""}]",
110+
maven,org.openrewrite:rewrite-java,org.openrewrite.java.DeleteMethodArgument,Delete method argument,Delete an argument from method invocations.,1,,Python,,Basic building blocks for transforming Python code.,"[{""name"":""methodPattern"",""type"":""String"",""displayName"":""Method pattern"",""description"":""A [method pattern](https://docs.openrewrite.org/reference/method-patterns) is used to find matching method invocations. For example, to find all method invocations in the Guava library, use the pattern: `com.google.common..*#*(..)`.<br/><br/>The pattern format is `<PACKAGE>#<METHOD_NAME>(<ARGS>)`. <br/><br/>`..*` includes all subpackages of `com.google.common`. <br/>`*(..)` matches any method name with any number of arguments. <br/><br/>For more specific queries, like Guava's `ImmutableMap`, use `com.google.common.collect.ImmutableMap#*(..)` to narrow down the results."",""example"":""com.yourorg.A foo(int, int)"",""required"":true},{""name"":""argumentIndex"",""type"":""int"",""displayName"":""Argument index"",""description"":""A zero-based index that indicates which argument will be removed from the method invocation."",""example"":""0"",""required"":true,""value"":0}]",
111+
maven,org.openrewrite:rewrite-java,org.openrewrite.java.ReorderMethodArguments,Reorder method arguments,Reorder method arguments into the specified order.,1,,Python,,Basic building blocks for transforming Python code.,"[{""name"":""methodPattern"",""type"":""String"",""displayName"":""Method pattern"",""description"":""A [method pattern](https://docs.openrewrite.org/reference/method-patterns) is used to find matching method invocations. For example, to find all method invocations in the Guava library, use the pattern: `com.google.common..*#*(..)`.<br/><br/>The pattern format is `<PACKAGE>#<METHOD_NAME>(<ARGS>)`. <br/><br/>`..*` includes all subpackages of `com.google.common`. <br/>`*(..)` matches any method name with any number of arguments. <br/><br/>For more specific queries, like Guava's `ImmutableMap`, use `com.google.common.collect.ImmutableMap#*(..)` to narrow down the results."",""example"":""com.yourorg.A foo(String, Integer, Integer)"",""required"":true},{""name"":""newParameterNames"",""type"":""String[]"",""displayName"":""New parameter names"",""description"":""An array of parameter names that indicates the new order in which those arguments should be arranged."",""example"":""[foo, bar, baz]"",""required"":true},{""name"":""oldParameterNames"",""type"":""String[]"",""displayName"":""Old parameter names"",""description"":""If the original method signature is not type-attributed, this is an optional list that indicates the original order in which the arguments were arranged."",""example"":""[baz, bar, foo]""},{""name"":""ignoreDefinition"",""type"":""Boolean"",""displayName"":""Ignore type definition"",""description"":""When set to `true` the definition of the old type will be left untouched. This is useful when you're replacing usage of a class but don't want to rename it.""},{""name"":""matchOverrides"",""type"":""Boolean"",""displayName"":""Match on overrides"",""description"":""When enabled, find methods that are overrides of the method pattern.""}]",
112+
maven,org.openrewrite:rewrite-java,org.openrewrite.java.search.FindMethods,Find method usages,Find method calls by pattern.,1,Search,Python,,Basic building blocks for transforming Python code.,"[{""name"":""methodPattern"",""type"":""String"",""displayName"":""Method pattern"",""description"":""A [method pattern](https://docs.openrewrite.org/reference/method-patterns) is used to find matching method invocations. For example, to find all method invocations in the Guava library, use the pattern: `com.google.common..*#*(..)`.<br/><br/>The pattern format is `<PACKAGE>#<METHOD_NAME>(<ARGS>)`. <br/><br/>`..*` includes all subpackages of `com.google.common`. <br/>`*(..)` matches any method name with any number of arguments. <br/><br/>For more specific queries, like Guava's `ImmutableMap`, use `com.google.common.collect.ImmutableMap#*(..)` to narrow down the results."",""example"":""java.util.List add(..)"",""required"":true},{""name"":""matchOverrides"",""type"":""Boolean"",""displayName"":""Match on overrides"",""description"":""When enabled, find methods that are overrides of the method pattern.""}]","[{""name"":""org.openrewrite.java.table.MethodCalls"",""displayName"":""Method calls"",""description"":""The text of matching method invocations."",""columns"":[{""name"":""sourceFile"",""type"":""String"",""displayName"":""Source file"",""description"":""The source file that the method call occurred in.""},{""name"":""method"",""type"":""String"",""displayName"":""Method call"",""description"":""The text of the method call.""},{""name"":""className"",""type"":""String"",""displayName"":""Class name"",""description"":""The class name of the method call.""},{""name"":""methodName"",""type"":""String"",""displayName"":""Method name"",""description"":""The method name of the method call.""},{""name"":""argumentTypes"",""type"":""String"",""displayName"":""Argument types"",""description"":""The argument types of the method call.""}]}]"
113+
maven,org.openrewrite:rewrite-java,org.openrewrite.java.search.FindTypes,Find types,Find type references by name.,1,Search,Python,,Basic building blocks for transforming Python code.,"[{""name"":""fullyQualifiedTypeName"",""type"":""String"",""displayName"":""Fully-qualified type name"",""description"":""A fully-qualified type name, that is used to find matching type references. Supports glob expressions. `java..*` finds every type from every subpackage of the `java` package."",""example"":""java.util.List"",""required"":true},{""name"":""checkAssignability"",""type"":""Boolean"",""displayName"":""Check for assignability"",""description"":""When enabled, find type references that are assignable to the provided type.""}]","[{""name"":""org.openrewrite.java.table.TypeUses"",""displayName"":""Type uses"",""description"":""The source code of matching type uses."",""columns"":[{""name"":""sourceFile"",""type"":""String"",""displayName"":""Source file"",""description"":""The source file that the method call occurred in.""},{""name"":""code"",""type"":""String"",""displayName"":""Source"",""description"":""The source code of the type use.""},{""name"":""concreteType"",""type"":""String"",""displayName"":""Concrete type"",""description"":""The concrete type in use, which may be a subtype of a searched type.""}]}]"

‎rewrite-python/rewrite/pyproject.toml‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,15 +24,16 @@ requires-python = ">=3.10"
2424
dependencies = [
2525
"cbor2>=5.6.5",
2626
"more_itertools>=10.0.0",
27-
"pytype>=2024.1.1; python_version < '3.13'",
2827
]
2928

3029
[project.optional-dependencies]
3130
dev = [
3231
"pytest>=8.0.0",
3332
"black>=24.0.0",
3433
"ruff>=0.1.0",
35-
"ty>=0.0.12",
34+
]
35+
typing = [
36+
"ty>=0.0.12", # Required for type attribution with Java recipes
3637
]
3738
publish = [
3839
"build>=1.0.0",

‎rewrite-python/rewrite/src/rewrite/python/__init__.py‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,8 @@
5454
YieldFrom,
5555
)
5656
from rewrite.python.visitor import PythonVisitor
57+
from rewrite.python.add_import import AddImport, AddImportOptions, maybe_add_import
58+
from rewrite.python.remove_import import RemoveImport, RemoveImportOptions, maybe_remove_import
5759

5860
__all__ = [
5961
# Marker class
@@ -92,4 +94,11 @@
9294
"YieldFrom",
9395
# Visitor
9496
"PythonVisitor",
97+
# Import handling
98+
"AddImport",
99+
"AddImportOptions",
100+
"maybe_add_import",
101+
"RemoveImport",
102+
"RemoveImportOptions",
103+
"maybe_remove_import",
95104
]

‎rewrite-python/rewrite/src/rewrite/python/_parser_visitor.py‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,7 @@ class ParserVisitor(ast.NodeVisitor):
5353
# UTF-8 BOM character
5454
_BOM = '\ufeff'
5555

56-
def __init__(self, source: str):
56+
def __init__(self, source: str, file_path: Optional[str] = None):
5757
super().__init__()
5858
# Detect and strip UTF-8 BOM if present
5959
if source.startswith(self._BOM):
@@ -64,7 +64,7 @@ def __init__(self, source: str):
6464

6565
self._source = source
6666
self._parentheses_stack = []
67-
self._type_mapping = PythonTypeMapping(source)
67+
self._type_mapping = PythonTypeMapping(source, file_path)
6868

6969
# Pre-compute byte-to-char mappings for lines with multi-byte characters
7070
self._byte_to_char = self._build_byte_to_char_mapping(source)
@@ -1699,6 +1699,8 @@ def visit_Call(self, node):
16991699
Markers.EMPTY
17001700
)
17011701

1702+
method_type = self._type_mapping.method_invocation_type(node)
1703+
17021704
return j.MethodInvocation(
17031705
random_id(),
17041706
prefix,
@@ -1708,7 +1710,7 @@ def visit_Call(self, node):
17081710
name if isinstance(name, j.Identifier) else j.Identifier(random_id(), Space.EMPTY, Markers.EMPTY, [], "",
17091711
None, None),
17101712
args,
1711-
name.type if isinstance(name.type, JavaType.Method) else None, # ty: ignore[possibly-missing-attribute]
1713+
method_type,
17121714
)
17131715

17141716
def __sort_call_arguments(self, call: ast.Call) -> List[Union[ast.expr, ast.keyword]]:
@@ -2325,8 +2327,6 @@ def _map_comprehension_condition(self, i):
23252327
)
23262328

23272329
def visit_Module(self, node: ast.Module) -> py.CompilationUnit:
2328-
self._type_mapping.resolve_types(node)
2329-
23302330
cu = py.CompilationUnit(
23312331
random_id(),
23322332
Space.EMPTY,

0 commit comments

Comments
 (0)
⚡