[AIROCMLIR-707] Add split_kv memory checks for attention

[bogdan-petkovic]

Motivation

Some attention configurations on gfx1201 with large split_kv values were consuming too much memory and failing late (OOM/timeout behavior).
This PR makes those cases fail early with a clear validation error, so bad configurations are rejected before expensive lowering and runtime execution.

Technical Details

Added support to read total GPU global memory from HIP for the target architecture.
In attention verification, added a split_kv extra-memory estimator for output and LSE buffers, using overflow-safe arithmetic.
The verifier now compares required extra bytes against a limit and rejects oversized configurations with a clear error message that prints both required and allowed bytes.

The limit is chosen in this order:

• ROCMLIR_ATTENTION_SPLITKV_MAX_EXTRA_BYTES environment override.

• A device-based limit computed from GPU memory (device memory divided by 8, with safety clamping).

• A fallback default when device memory cannot be queried.

Test Plan

Build MLIRRockOps and rocmlir-gen.
Run a large split_kv attention case and verify early rejection with an explicit memory-limit message.
Re-run the same case with ROCMLIR_ATTENTION_SPLITKV_MAX_EXTRA_BYTES override and verify generation succeeds.
Replay the 4 reported failing attention configurations through the same parameter sweep pipeline (gen -> driver -> runner).

Test Result

Build status: PASS.

Replay of the 4 reported configurations:

• 3 configurations are now INVALID at rocmlir-gen stage with explicit splitKV memory-limit errors.

• 1 configuration passes end-to-end.

Submission Checklist

• Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

[Copilot]

Pull request overview

Adds an early-validation guard for attention splitKV configurations to prevent late OOM/timeout failures by estimating extra temporary storage and rejecting oversized cases up front, with a user override via environment variable.

Changes:

• Add overflow-safe splitKV extra-memory estimation in rock.attention verification and emit a clear validation error when exceeding a limit.
• Add HIP-based device global-memory query helper to derive a dynamic default limit per target architecture (with clamping and fallback).
• Minor: adjust bf16 attention sweep default -RMS_threshold; treat 0-byte ROCm allocations as a no-op.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
mlir/utils/performance/parameterSweeps.py	Changes default bf16 attention sweep RMS threshold.
mlir/lib/Dialect/Rock/IR/RockDialect.cpp	Implements splitKV extra-memory estimation + verifier limit/diagnostic.
mlir/lib/Dialect/Rock/IR/AmdArchDb.cpp	Adds HIP query to retrieve total GPU global memory for an arch.
mlir/include/mlir/Dialect/Rock/IR/AmdArchDb.h	Exposes the new device global-memory query API.
external/llvm-project/mlir/lib/ExecutionEngine/RocmRuntimeWrappers.cpp	Makes 0-byte mgpuMemAlloc return nullptr early.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.