-
Notifications
You must be signed in to change notification settings - Fork 2k
[airflow] Add ruff rules to catch deprecated Airflow imports for Airflow 3.1 (AIR321)
#22376
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
ntBre
merged 18 commits into
astral-sh:main
from
sjyangkevin:catch-deprecated-imports-airflow-3_1
Feb 4, 2026
Merged
Changes from all commits
Commits
Show all changes
18 commits
Select commit
Hold shift + click to select a range
58f6732
add a new code for rules check for deprecated imports in airflow 3.1
sjyangkevin 1b0bd7c
update comment for airflow.utils.setup_teardown
sjyangkevin b5b9d99
group related imports and use SourceModuleMoved
sjyangkevin 5a5afbf
move BaseOperator to airflow.sdk
sjyangkevin bf5da1e
udpate test case
sjyangkevin 1521a97
refine rules and improve message for sdk change
sjyangkevin b40bbd9
refactor module moved task sdk to use SourceModuleMovedToSDK replacement
sjyangkevin 8112c30
update comment for utils.log.secrets_masker
sjyangkevin 417ed95
refactor to use Message for warning, be explicit on task sdk version,…
sjyangkevin 378e4bc
update preview_since version to 0.14.12
sjyangkevin cfa2407
add a new item in enum for raising internal module warning message
sjyangkevin 1dc05b0
add a blank line at end of file
sjyangkevin da80949
rename to which can be used to include custom message and optionall…
sjyangkevin 5f8363f
update comment for
sjyangkevin 250a248
update sdk version to 1.1.0 for AIR321, and version to 1.0.0 for AIR301
sjyangkevin 2a3f6e6
clean up redundant match and improve message
sjyangkevin 531cbca
rename the rule, and some updates to rule
sjyangkevin a513e55
refine message, refactor AIR311 sdk move to use SourceModuleMovedToSDK
sjyangkevin File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
102 changes: 102 additions & 0 deletions
102
crates/ruff_linter/resources/test/fixtures/airflow/AIR321_names.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,102 @@ | ||
| from __future__ import annotations | ||
|
|
||
| # airflow.utils.setup_teardown | ||
| from airflow.utils.setup_teardown import ( | ||
| BaseSetupTeardownContext, | ||
| SetupTeardownContext, | ||
| ) | ||
|
|
||
| BaseSetupTeardownContext() | ||
| SetupTeardownContext() | ||
|
|
||
| # airflow.utils.xcom | ||
| from airflow.utils.xcom import XCOM_RETURN_KEY | ||
|
|
||
| XCOM_RETURN_KEY | ||
|
|
||
| # airflow.utils.task_group | ||
| from airflow.utils.task_group import TaskGroup | ||
|
|
||
| TaskGroup() | ||
|
|
||
| # airflow.utils.timezone | ||
| from datetime import datetime as dt | ||
| from airflow.utils.timezone import ( | ||
| coerce_datetime, | ||
| convert_to_utc, | ||
| datetime, | ||
| make_naive, | ||
| parse, | ||
| utc, | ||
| utcnow, | ||
| ) | ||
|
|
||
| current_time = dt.now() | ||
|
|
||
| coerce_datetime(v=current_time, tz=None) | ||
| convert_to_utc(current_time) | ||
| datetime(2026, 1, 1, 21, 30, 0) | ||
| make_naive(current_time, tz=None) | ||
| parse("2026-01-01 21:30:00") | ||
| utc | ||
| utcnow() | ||
|
|
||
| # airflow.utils.decorators | ||
| from airflow.utils.decorators import ( | ||
| remove_task_decorator, | ||
| fixup_decorator_warning_stack, | ||
| ) | ||
|
|
||
| remove_task_decorator() | ||
| fixup_decorator_warning_stack() | ||
|
|
||
| # airflow.models.abstractoperator | ||
| from airflow.models.abstractoperator import ( | ||
| AbstractOperator, | ||
| NotMapped, | ||
| TaskStateChangeCallback, | ||
| ) | ||
|
|
||
| AbstractOperator() | ||
| NotMapped() | ||
| TaskStateChangeCallback() | ||
|
|
||
| # airflow.models.baseoperator | ||
| from airflow.models.baseoperator import BaseOperator | ||
|
|
||
| BaseOperator() | ||
|
|
||
| # airflow.macros | ||
| from airflow.macros import ( | ||
| ds_add, | ||
| ds_format, | ||
| datetime_diff_for_humans, | ||
| ds_format_locale, | ||
| ) | ||
|
|
||
| ds_add("2026-01-01", 5) | ||
| ds_format("2026-01-01", "%Y-%m-%d", "%m-%d-%y") | ||
| datetime_diff_for_humans( | ||
| dt(2026, 1, 1, 21, 30, 0), | ||
| since=dt(2026, 1, 2, 21, 30, 0) | ||
| ) | ||
| ds_format_locale("01/01/2026", "%d/%m/%Y", "EEEE dd MMMM yyyy", "en_US") | ||
|
|
||
| # airflow.io | ||
| from airflow.io import ( | ||
| get_fs, | ||
| has_fs, | ||
| Properties, | ||
| ) | ||
|
|
||
| get_fs() | ||
| has_fs() | ||
| Properties() | ||
|
|
||
| # airflow.secrets.cache | ||
| from airflow.secrets.cache import SecretCache | ||
| SecretCache() | ||
|
|
||
| # airflow.hooks | ||
| from airflow.hooks.base import BaseHook | ||
| BaseHook() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Create this new enum item which can be used as follow:
fix_titleformat is: "{name}has been moved to{module}since Airflow 3.1.{message}"; we can use themessageparameter to include a static string as additional message that we want to show to users.suggest_fixparameter is used to optionally report diagnostics. In some cases, we might want to just show a warning (e.g., internal module usage), but in some case, we might want to report diagnostic and suggest fix (e.g.,BaseHookis moved from one module to another)Why
messageis a static string. Use the following example to explain. MakingmessageaStringallow us to useformat!to templating the string and render it at runtime. However, onlyrestor variables defined in the accessible scope can be used to render the string. Asmoduleandnamewill be handled infix_title. It might not be necessary to use those values to render themessage. So, here use static string.