Status of testing of Apache Airflow 2.9.2rc1

[utkarsharma2]

Body

We are kindly requesting that contributors to Apache Airflow RC 2.9.2rc1 help test the RC.

Please let us know by commenting if the issue is addressed in the latest RC.

• Fix visible DagRun doesn't point to the same dag run id (#38365): @lotrias17 @nivdror
  Linked issues:
  • Visible DAG RUN doesn't point to the same dag run id (#35946)
• Fix TaskHandlerWithCustomFormatter now adds prefix only once (#38502): @TiDeane @Aakcht
  Linked issues:
  • Logging handler TaskHandlerWithCustomFormatter adds prefix three times (#35622)
• Fix inconsistency of cache-directive vs. docker-cache (#38580): @potiuk
• Add support for OrbStack in breeze (#38725): @jedcunningham
• Fix Mark Instance state buttons stay disabled if user lacks permission (#37451) (#38732): @RodrigoGanancia @sreenusuuda
  Linked issues:
  • Marking task state as Failed or Success without taskinstance edit role is resulting into partial empty screen (#37451)
• Add "use self-hosted runners" to CI setup (#38779): @potiuk
• Fix sql_alchemy_engine_args config example (#38971): @jedcunningham
• Fix alembic autogeneration and rename mismatching constraints (#39032): @ephraimbuddy
• Fix stacklevel for TaskContextLogger (#39142): @dstandish
• Remove unused index idx_last_scheduling_decision on dag_run table (#39275): @pankajkoti
• add missing conn_id to string representation of ObjectStoragePath (#39313): @Lee-W
• Do not provide deprecated execution_date in @apply_lineage (#39327): @Taragolis
  Linked issues:
  • Follow zero internal warning in unit tests (#38642)
• Fix static file caching is disabled in Airflow Webserver. (#39345): @VertexToEdge
  Linked issues:
  • webserver static file caching is disabled above 2.3.0 version. (#39344)
• Refactor SafeDogStatsdLogger to use get_validator to enable pattern matching (#39370): @rawwar
  Linked issues:
  • Add a pattern-matching support for allow and block list in Datadog metric (#39368)
• Add the existing_nullable to the downgrade side of the migration (#39374): @ephraimbuddy
• Fix HTTP 500 Internal Server Error if DAG is triggered with bad params (#39409): @jscheffl
• Resolve SA warnings in migrations scripts (#39418): @Taragolis
• Fix custom actions in security manager has_access (#39421): @jedcunningham @vincbeck
  Linked issues:
  • Custom FAB actions no longer work (#39144)
  • Update is_authorized_custom_view from auth manager to handle custom actions (#39167)
• add lost space to breeze help (#39458): @uzhastik
• Increase volume size for ARM instance (#39472): @potiuk
• Add hatch_build.py to k8s test venv cache calculation (#39473): @potiuk
• Update uv to 0.1.41 (#39476): @potiuk
• Add Cache-Control "no-store" to all dynamically generated content (#39550): @potiuk
• Update uv to 0.1.42 (#39561): @potiuk
• Remove mysql/postgres from extras before determining the installation_command_flags (#39610): @macchiang
• Change dataset URI validation to raise warning instead of error in Airflow 2.9 (#39670): @uranusjr @tatiana
  Linked issues:
  • Implement AIP-60 Dataset URI formats (#37005)
  • [Bug] Strict validation in Dataset URI in Airflow 2.9 breaks some DAGs (#39486)
• More typing in TimeSensor and TimeSensorAsync (#39696): @jedcunningham
• Reraise exception from strict dataset URI checks (#39719): @uranusjr @tatiana
  Linked issues:
  • Change dataset URI validation to raise warning instead of error in Airflow 2.9 (#39670)
• Better typing for BaseOperator defer (#39742): @jedcunningham
• Use SKIP LOCKED instead of NOWAIT in mini scheduler (#39745): @VladimirYushkevich
• Change type definition for provider_info_cache decorator (#39750): @Taragolis
• Warn on mini scheduler failures instead of debug (#39760): @RNHTTR
• Fix example_branch_operator failing in python 3.12 (#39783): @ephraimbuddy
• Limit concurrent downloads by UV (#39810): @potiuk
• Add max_consecutive_failed_dag_runs in API spec (#39830): @pankajastro
• Also limit concurrent uv downloads when building (#39852): @potiuk
  Linked issues:
  • Limit concurrent downloads by UV (#39810)
• Remove DAG Run Add option from FAB view (#39881): @jscheffl @fliaj
  Linked issues:
  • "Add Dag Run" /dagrun/add page issue (#39856)
• Fix bug that makes AirflowSecurityManagerV2 leave transactions in the idle in transaction state (#39935): @alejandro-rivera
• Limit yandex provider to avoid mypy errors (#39990): @potiuk
• Fix mocking in kerberos integration test (#40042): @potiuk

Thanks to all who contributed to the release (probably not a complete list!):
@jscheffl @nivdror @dstandish @ephraimbuddy @pankajastro @VladimirYushkevich @uzhastik @sreenusuuda @uranusjr @VertexToEdge @RodrigoGanancia @lotrias17 @rawwar @macchiang @Lee-W @potiuk @pankajkoti @Taragolis @Aakcht @alejandro-rivera @RNHTTR @fliaj @TiDeane @jedcunningham @tatiana @vincbeck

Committer

• I acknowledge that I am a maintainer/committer of the Apache Airflow project.

[potiuk]

Checked my changes. Unfortunately there is likely a breaking "celery" change that we should make sure to handle. See #40097

[potiuk]

UPDATE: so it seems (see #40097 ) we should also cherry-pick #40012 and #40035 in order to include fix for that (cc: @o-nikolas ). I marked them as 2.9.2 milestone.

[utkarsharma2]

Sure @potiuk, we will cherry-pick those commit and release rc2

[ephraimbuddy]

UPDATE: so it seems (see #40097 ) we should also cherry-pick #40012 and #40035 in order to include fix for that (cc: @o-nikolas ). I marked them as 2.9.2 milestone.

I'm confused here. #40012 is provider only change but #40035 is only test changes. I think the fix would be by releasing a new celery provider or am I missing something?

[pankajastro]

#39830 looking good

[potiuk]

UPDATE: so it seems (see #40097 ) we should also cherry-pick #40012 and #40035 in order to include fix for that (cc: @o-nikolas ). I marked them as 2.9.2 milestone.

I'm confused here. #40012 is provider only change but #40035 is only test changes. I think the fix would be by releasing a new celery provider or am I missing something?

I looked closer and yes - you are absolutely right @ephraimbuddy. This looks like celery provider bug for 3.7.1 and that we should wait with 2.9.2 until celery provider is 3.7.2 with the fix is released and we should - I think - yank celery provider 3.7.1

See: #40097 (comment)

[jscheffl]

Tested the two contributed fixes, works as expected.

[utkarsharma2]

Airflow 2.9.2 has been released. Thank you all for testing this release