Add Python language support#6508
Merged
jkschneider merged 10 commits intomainfrom Jan 12, 2026
Merged
Conversation
Initial implementation of Python language support for OpenRewrite including: - Python AST model (tree.py) extending Java AST elements - Python parser using Python's native AST module with type mapping - Python visitor pattern implementation - RPC serialization for Python LST elements (sender/receiver) - Python printer for source code generation - Integration with existing Java infrastructure
- Update PythonRpcResetExtension to shutdown RPC after each test (matching JavaScript pattern) - Fix Semicolon marker printing in PythonPrinter._visit_markers - Add missing STATEMENT_EXPRESSION_PREFIX to PySpace.Location - Fix StatementExpression to not print duplicate prefix spacing - Register receive codecs for Semicolon, TrailingComma, and OmitParentheses markers
- Create pytest-friendly test harness with RecipeSpec and python() helper - Consolidate duplicate Recipe classes (move run() from execution.py to recipe.py) - Add replace() method to Tree and JRightPadded for visitor compatibility - Implement visit_method_declaration and visit_block in JavaVisitor for traversal - Fix visit_compilation_unit to use padding accessors for JRightPadded lists - Add RemovePass recipe as end-to-end example with 3 tests
PushObject was added to work around a bidirectional RPC deadlock in the in-process Java-to-Java test setup. However, this is not the right solution because it modifies the RPC spec unnecessarily. The Print RPC handler correctly calls GetObject to fetch the tree, matching the JavaScript implementation. This works correctly when calling to a real subprocess (e.g., Java to Python/JS) because the subprocess can handle the GetObject callback while the client waits. The test is disabled because the in-process test setup with piped streams causes a deadlock when bidirectional RPC calls are needed.
- Return None for Python built-in types in to_java_type_name to prevent ClassNotFoundException for types like builtins.dict - Modify handle_print to fetch the tree from Java instead of using stale local cache, ensuring recipe modifications are properly reflected - Remove unused PushObject handler
The Python RPC server requires the rewrite module to be installed. This adds Gradle tasks to: - Create a Python virtual environment if not exists - Upgrade pip to avoid compatibility issues - Install the openrewrite package in development mode Tests now depend on pythonInstall, ensuring the venv is ready.
…C issues - Replace with_* methods with replace() throughout Python codebase - Update JsonRpc constructor calls to use new 2-argument API with MessageFormatter - Add @JsonCreator to VisitResponse for Jackson deserialization - Fix tuple parsing bug in _parser_visitor.py (use padding.elements) - Add StatementExpression.replace() to handle delegated prefix/markers
jkschneider
force-pushed
the
python
branch
2 times, most recently
from
f514b2c to
e34da59
Compare
- Add Gradle tasks for Python build/publish (pythonBuild, pythonPublish) - Generate PEP 440 compliant versions (.dev suffix for snapshots) - Add publish dependencies to pyproject.toml - Pass PYPI_OPENREWRITE_PUBLISH secret through workflows - Fix JsonRpc constructor calls for updated library API (single arg)
jkschneider
added a commit
that referenced
this pull request
Jan 12, 2026
knutwannheden
added a commit
that referenced
this pull request
Jan 14, 2026
* UUID generation fallback for Node version pre-14.17.0 (#6495) * UUID generation fallback for Node version pre-14.17.0 * Apply suggestions from code review * Use module-load-time selection for performance * Polish --------- Co-authored-by: Tim te Beek <tim@moderne.io> Co-authored-by: Knut Wannheden <knut@moderne.io> * Changing `AddDependencyVisitor` so it is able to broaden the scope of a direct dependency if needed (#6434) * Allowing `AddDependency` to broaden the scope of existing dependency, provided you're requesting an equal or higher version number. If requesting same scope but higher version number, the version number will be upgraded. * Dropping `import` scope validity in favour of `compile`, given `import` is only applicable for dependencies in `dependencyManagement` Co-authored-by: Tim te Beek <tim@moderne.io> --------- Co-authored-by: Tim te Beek <tim@moderne.io> * Add style parameter to OrderImports. (#6496) * Drop Lombok hint from RemoveUnusedImports class (#6500) * Drop Lombok hint from RemoveUnusedImports class Since we've supported Lombok for quite some time now. * Also update recipes.csv * In yaml `CopyValue`, invoke `UnfoldProperties` after `MergeYaml` (#6499) * unit test with poc for UnfoldProperties * use imperative CopyValue in unit test * Invoke UnfoldProperties after MergeYaml * Apply suggestions from code review Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Slight polish --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Tim te Beek <tim@moderne.io> * Allow adding comments to Maven plugins too Fixes #6502 * JavaScript: Make visitor base clases sync and add async alternative * Polish parser APIs * More async cleanups * JavaScript: Notify about parsed source files from `parseProject()` * Polish object property shorthand * Fix Prettier integration when calling it using sync API * Altering check for `AddDependency` to compare versions differently when direct vs indirect dependency. (#6505) * If transitive, actually just allow regardless of version, as per old behaviour. * Changing the parsing of the `relativePath` for `parent` in the `MavenPomDownloader` to prevent the situation where `/` was producing `/\pom.xml` on Windows rather than `\pom.xml` (#6506) * Claude/catalogue async apis u67 hy (#6507) * Convert package-manager and related APIs from async to sync - Convert runInstallInTempDir and runWorkspaceInstallInTempDir to use sync fs APIs (fs.mkdtempSync, fs.writeFileSync, fs.readFileSync, fs.rmSync) - Convert runInstallIfNeeded callback from async to sync - Convert updateNodeResolutionMarker to sync (was marked async with no awaits) - Convert createLockFileEditor to use sync JsonVisitor and TreeVisitor - Update add-dependency, upgrade-dependency-version, and upgrade-transitive-dependency-version recipes to use sync visitors - Convert Result.diff() to sync (createTwoFilesPatch is already sync) The package manager operations were unnecessarily async since the underlying process spawning (spawnSync) was already synchronous. Only the file I/O was async, which is negligible compared to the npm/yarn/pnpm install time. * Remove unnecessary async from IsSourceFile.preVisit() * Add async property to TreeVisitor and AsyncTreeVisitor Add a readonly `async` property to both visitor base classes: - TreeVisitor: async = false - AsyncTreeVisitor: async = true This allows runtime discrimination of sync vs async visitors via the RecipeVisitor union type. Useful for sync-to-sync recipe composition where a sync visitor wants to call another sync visitor without async overhead. * Lift async I/O out of visitors into editorWithData() Refactor package-manager recipes to do async I/O (npm install) in editorWithData() before returning the visitor, keeping visitors pure. Changes: - Add async runInstallInTempDirAsync() using spawn() and fs.promises - Add async runWorkspaceInstallInTempDirAsync() for workspace support - Refactor AddDependency, UpgradeDependencyVersion, and UpgradeTransitiveDependencyVersion to run package manager installs in editorWithData() before returning visitors - Remove unused runInstallIfNeeded() helper function - Visitors are now pure tree transformations with no I/O This provides a cleaner separation of concerns: - Async I/O happens in the recipe's async editor() method - Visitors are pure, synchronous tree transformations using pre-computed data * More sync visitors * Simplify receivers by extending visitors --------- Co-authored-by: Claude <noreply@anthropic.com> * Bugfix * Add back runInstallIfNeeded() * Also parse `jrxml` as XML Fixes #6512 * Update description for onlyIfUsing option Clarified description for 'onlyIfUsing' option to specify its importance in multi-module projects. Fixes #5795 * Retain nested class imports in `ChangePackage` (#6515) Fixes #6513 * Add Python language support (#6508) * Compact array RpcObjectData only for JS * Less async test code --------- Co-authored-by: Benjamin Muschko <benjamin.muschko@gmail.com> Co-authored-by: Tim te Beek <tim@moderne.io> Co-authored-by: Steve Elliott <steve@moderne.io> Co-authored-by: Sam Snyder <sam@moderne.io> Co-authored-by: David Grieve <david@moderne.io> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Jonathan Schnéider <jkschneider@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Initial implementation of Python language support for OpenRewrite