Skip to content

[SPARK-56378] Add PR description guidelines to AGENTS.md#55242

Open
szehon-ho wants to merge 1 commit intoapache:masterfrom
szehon-ho:agents-md-pr-template
Open

[SPARK-56378] Add PR description guidelines to AGENTS.md#55242
szehon-ho wants to merge 1 commit intoapache:masterfrom
szehon-ho:agents-md-pr-template

Conversation

@szehon-ho
Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

Require PR descriptions to use the section headings from .github/PULL_REQUEST_TEMPLATE in AGENTS.md (AI agent instructions).

Why are the changes needed?

The previous instruction ("Follow the template") was too vague and resulted in AI-generated PRs that did not follow the template format.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

No tests needed — documentation-only change.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude (Cursor agent mode)

The previous instruction ("Follow the template") was too vague.
Require using the template section headings.
@szehon-ho szehon-ho changed the title [MINOR] Add PR description guidelines to AGENTS.md [SPARK-56378] Add PR description guidelines to AGENTS.md Apr 7, 2026
@szehon-ho
Copy link
Copy Markdown
Member Author

fyi @peter-toth @cloud-fan , i tried AGENTS.md to make the pr #55179

but it looked weird initially:

Summary

  1. Converts partition-column filters to PartitionPredicates (reusing SPARK-55596 infrastructure)
  2. Translates remaining data-column filters to standard V2 predicates
  3. Combines them (partition predicates first) and calls table.canDeleteWhere

Changes

  • OptimizeMetadataOnlyDeleteFromTable: Added tryDeleteWithPartitionPredicates fallback method and tryTranslateToV2 helper
  • PushDownUtils: Extracted createPartitionPredicates and made flattenNestedPartitionFilters package-private for reuse; getPartitionPredicateSchema now returns None for empty partition fields
  • InMemoryTableWithV2Filter: Extracted evalPredicate to companion object for reuse by test tables
  • InMemoryPartitionPredicateDeleteTable (new): Test table supporting PartitionPredicates and configurable data predicate acceptance
  • DataSourceV2EnhancedDeleteFilterSuite (new): 9 test cases covering first-pass accept, second-pass accept/reject, mixed partition+data filters, UDF on non-contiguous partition columns, multiple PartitionPredicates, and row-level fallback

Test plan

  • DataSourceV2EnhancedDeleteFilterSuite — 9/9 pass
  • DataSourceV2EnhancedPartitionFilterSuite — 19/19 pass (no regressions)
  • GroupBasedDeleteFromTableSuite — 32/32 pass (no regressions)
  • Scalastyle — 0 errors

So want to clarify to make AGENTS use the right template

@szehon-ho
Copy link
Copy Markdown
Member Author

actually im not 100% sure , it already mentioned the template, not sure why it didnt follow

@cloud-fan
Copy link
Copy Markdown
Contributor

actually I already have a fix here: #55139

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants