Skip to content

Add fix and tests for handling Airflow dags with dots and task groups#2126

Merged
wslulciuc merged 2 commits into
mainfrom
fix/handle_airflow_dags_and_groups
Sep 19, 2022
Merged

Add fix and tests for handling Airflow dags with dots and task groups#2126
wslulciuc merged 2 commits into
mainfrom
fix/handle_airflow_dags_and_groups

Conversation

@collado-mike

Copy link
Copy Markdown
Collaborator

Problem

A recent change broke how marquez handles DAGs with dots and tasks within task groups. This change corrects that and adds a few test cases to validate

Note: All database schema changes require discussion. Please link the issue for context.

Checklist

  • You've signed-off your work
  • Your changes are accompanied by tests (if relevant)
  • Your change contains a small diff and is self-contained
  • You've updated any relevant documentation (if relevant)
  • You've updated the CHANGELOG.md with details about your change under the "Unreleased" section (if relevant, depending on the change, this may not be necessary)
  • You've versioned your .sql database schema migration according to Flyway's naming convention (if relevant)
  • You've included a header in any source code files (if relevant)

Signed-off-by: Michael Collado <collado.mike@gmail.com>
@boring-cyborg boring-cyborg Bot added the api API layer changes label Sep 16, 2022
@codecov

codecov Bot commented Sep 16, 2022

Copy link
Copy Markdown

Codecov Report

Merging #2126 (f82331e) into main (9bb877d) will increase coverage by 0.02%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##               main    #2126      +/-   ##
============================================
+ Coverage     75.28%   75.30%   +0.02%     
- Complexity     1036     1038       +2     
============================================
  Files           203      203              
  Lines          4879     4883       +4     
  Branches        398      399       +1     
============================================
+ Hits           3673     3677       +4     
  Misses          763      763              
  Partials        443      443              
Impacted Files Coverage Δ
api/src/main/java/marquez/db/OpenLineageDao.java 95.17% <100.00%> (+0.05%) ⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@wslulciuc wslulciuc left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@wslulciuc wslulciuc enabled auto-merge (squash) September 19, 2022 19:03
@wslulciuc wslulciuc merged commit e61fe48 into main Sep 19, 2022
@wslulciuc wslulciuc deleted the fix/handle_airflow_dags_and_groups branch September 19, 2022 19:09
jonathanpmoraes referenced this pull request in nubank/NuMarquez Feb 6, 2025
… (#2126)

Signed-off-by: Michael Collado <collado.mike@gmail.com>

Signed-off-by: Michael Collado <collado.mike@gmail.com>
Co-authored-by: Willy Lulciuc <willy@datakin.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api API layer changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants