Skip to content

Ralph/eip7805 inclusion lists#120

Open
jflo wants to merge 51 commits intomainfrom
ralph/eip7805-inclusion-lists
Open

Ralph/eip7805 inclusion lists#120
jflo wants to merge 51 commits intomainfrom
ralph/eip7805-inclusion-lists

Conversation

@jflo
Copy link
Copy Markdown
Owner

@jflo jflo commented Mar 7, 2026

EIP-7805: Fork-Choice Enforced Inclusion Lists (FOCIL) — Engine API Implementation

This PR implements the execution-layer Engine API changes specified by EIP-7805 (FOCIL). The spec document used is eip7805.md in this repo, which mirrors the upstream Engine API spec for EIP-7805.


Suggested review order

1. Core domain model (ethereum/core) — start here

These are small, self-contained classes with no external dependencies. They establish the vocabulary used everywhere else.

File Spec reference
InclusionListConstants.java MAX_BYTES_PER_INCLUSION_LIST = 2^13 (spec: Constants)
InclusionListValidationStatus.java Three-state enum: VALID, INVALID, UNSATISFIED — maps to INCLUSION_LIST_UNSATISFIED status in the spec
InclusionListValidationResult.java Result wrapper with factory methods
InclusionListValidator.java Interface: validate(payloadTransactions, inclusionListTransactions)
StrictInclusionListValidator.java Enforces ordered-subsequence constraint per EIP-7805 inclusion list satisfaction
LenientInclusionListValidator.java Same checks, but logs warnings and returns VALID (for testnet use)
InclusionListValidationMode.java Enum (STRICT / LENIENT) with createValidator() factory
InclusionListTransactionSelector.java Interface for mempool → IL transaction selection
DefaultInclusionListSelector.java Priority-queue selector: highest gas price first, filters blob txs (spec: getInclusionListV1 §3), respects byte limit (spec: §2), validates sequential nonces
InclusionListConfiguration.java / InclusionListSelectorType.java Config record + selector type enum

2. Engine API method: engine_getInclusionListV1 (spec)

  • EngineGetInclusionListV1.java — New JSON-RPC handler. Accepts parentHash, returns Array of DATA transaction bytes from mempool.
    • Returns -38006: Unknown parent if parent block not found (spec: §4)
    • Filters blob transactions (spec: §3)
    • Respects MAX_BYTES_PER_INCLUSION_LIST (spec: §2)
  • RpcMethod.java — Added ENGINE_GET_INCLUSION_LIST_V1 enum value
  • RpcErrorType.java — Added UNKNOWN_PARENT (-38006) error code
  • Test: EngineGetInclusionListV1Test.java

3. Engine API method: engine_newPayloadV5 (spec)

  • EngineNewPayloadV5.java — Extended to accept 5th parameter inclusionListTransactions. After successful execution, validates payload against IL constraints.
    • Returns INCLUSION_LIST_UNSATISFIED in strict mode (spec: §1)
    • Returns VALID in lenient mode (Besu extension for testnets)
  • AbstractEngineNewPayload.java — Added handleSuccessfulExecution() template method hook for V5+ post-execution validation
  • ExecutionEngineJsonRpcMethod.java — Added INCLUSION_LIST_UNSATISFIED to EngineStatus enum (spec: Response)
  • Test: EngineNewPayloadV5Test.java

4. Engine API method: engine_forkchoiceUpdatedV4 (spec)

  • EnginePayloadAttributesParameter.java — Added inclusionListTransactions field to match PayloadAttributesV4 (spec: Structures)
  • AbstractEngineForkchoiceUpdated.java — Passes IL transactions through to payload building
  • Test: EngineForkchoiceUpdatedV4Test.java, EnginePayloadAttributesParameterTest.java

5. Payload building (spec: Routines > Payload building)

The spec says the built payload MUST satisfy IL constraints. This is implemented by:

  • BlockTransactionSelector.java — New buildTransactionListForBlock(List<Transaction>) overload that processes IL transactions first, then fills remaining gas with pool transactions
  • AbstractBlockCreator.java / MergeBlockCreator.java — New createBlock overloads accepting IL transactions
  • MergeCoordinator.java — Decodes IL bytes → Transaction objects, passes to block creator
  • MergeMiningCoordinator.java — Extended preparePayload() with 8th parameter for IL transactions (backwards-compatible default method)
  • Test: LondonFeeMarketBlockTransactionSelectorTest.java

6. CLI configuration & wiring

  • EngineRPCOptions.java — Two new CLI flags: --engine-inclusion-list-validation-mode (strict|lenient), --engine-inclusion-list-selector-type (default)
  • EngineRPCConfiguration.java — Record extended with new fields + toInclusionListConfiguration()
  • RunnerBuilder.java / BesuCommand.java — Threads config through to Engine API method registration
  • JsonRpcMethodsFactory.java / ExecutionEngineJsonRpcMethods.java — Overloaded constructors to wire IL validator and register new methods under AMSTERDAM milestone
  • Test: EngineRPCOptionsTest.java

7. Logging, metrics & integration tests

  • Metrics (counters under BesuMetricCategory.RPC): transactions_generated, bytes_generated, selector_duration_ms, validation_failures
  • InclusionListWorkflowIntegrationTest.java — End-to-end test: getInclusionListV1forkchoiceUpdatedV4newPayloadV5, covering both strict and lenient modes

8. Documentation

  • docs/eip7805_inclusion_lists.md — Full API reference with JSON-RPC examples

Spec compliance summary

Spec requirement Implementation
MAX_BYTES_PER_INCLUSION_LIST = 2^13 InclusionListConstants.MAX_BYTES_PER_INCLUSION_LIST
PayloadAttributesV4 with inclusionListTransactions EnginePayloadAttributesParameter
engine_newPayloadV5 with 5th IL param EngineNewPayloadV5
INCLUSION_LIST_UNSATISFIED status EngineStatus.INCLUSION_LIST_UNSATISFIED
INVALID_BLOCK_HASH supplanted by INVALID Handled in V5 response logic
engine_getInclusionListV1 with parentHash EngineGetInclusionListV1
-38006: Unknown parent error RpcErrorType.UNKNOWN_PARENT
No blob transactions in IL DefaultInclusionListSelector filters TransactionType.BLOB
IL within MAX_BYTES_PER_INCLUSION_LIST Enforced in selector and validator
engine_forkchoiceUpdatedV4 with PayloadAttributesV4 AbstractEngineForkchoiceUpdated + V4 test
Payload building respects IL constraints BlockTransactionSelector prioritizes IL txs
1s timeout for getInclusionList Documented (timeout enforcement is at CL level)
8s timeout for forkchoiceUpdated Unchanged from V3

What's NOT in this PR

  • Consensus-layer (beacon chain) changes — FOCIL committee selection, IL gossip, and attestation logic live in the CL client
  • Fork scheduling — the methods are gated behind AMSTERDAM milestone but no fork schedule is set
  • Network-layer IL propagation — out of scope for Engine API

🤖 Generated with Claude Code

jflo and others added 29 commits March 6, 2026 18:54
Signed-off-by: jflo <justin+github@florentine.us>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Signed-off-by: jflo <justin+github@florentine.us>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Signed-off-by: jflo <justin+github@florentine.us>
Signed-off-by: Ralph Agent <ralph@besu.dev>
- Remove --tool argument and tool selection logic
- Remove amp support
- Simplify to always use claude with --dangerously-skip-permissions
- Update usage and output messages
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Add inclusionListTransactions field (List<String> of hex-encoded transaction
bytes) to EnginePayloadAttributesParameter with JSON serialization/deserialization
and hex format validation. Fix test constructor calls to include new parameter
and fix test data (valid even-length hex strings, null-safe list construction).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Pass inclusionListTransactions from EnginePayloadAttributesParameter
through to the payload building process via preparePayload().
- Extended MergeMiningCoordinator.preparePayload() interface with
  inclusionListTransactions parameter (8th param), keeping 7-param
  default method for backwards compatibility
- Updated MergeCoordinator and TransitionCoordinator implementations
- Added getInclusionListBytes() helper in AbstractEngineForkchoiceUpdated
  to convert hex strings to Bytes for the preparePayload call
- Created EngineForkchoiceUpdatedV4Test with tests for:
  - Missing PBSR validation (INVALID_PAYLOAD_ATTRIBUTES)
  - Missing slot number validation (INVALID_SLOT_NUMBER_PARAMS)
  - Null payload attributes handling
  - IL transactions passed to preparePayload
  - Null IL transactions handling
  - Withdrawal validation with V4 attributes
  - Protocol schedule empty handling
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Copilot AI review requested due to automatic review settings March 7, 2026 05:35
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this constant warrant its own file or would it fit in naturally in an existing file containing constants?

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be inlined into InclusionListValidationResult

Ralph Agent added 3 commits March 9, 2026 18:53
The spec's "anywhere-in-block" property means IL transactions can appear
in any order within the payload. Replace ordered subsequence matching
with HashSet-based membership check in both Strict and Lenient validators.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Signed-off-by: Ralph Agent <ralph@besu.dev>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Signed-off-by: Ralph Agent <ralph@besu.dev>
EIP-7805 does not exclude blob transactions from inclusion lists.
Blob txs only contain versioned hashes (not blob data) so their
RLP-encoded size is comparable to regular transactions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Signed-off-by: Ralph Agent <ralph@besu.dev>

@Override
public boolean isEnabled() {
return true;
Copy link
Copy Markdown
Owner Author

@jflo jflo Mar 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this makes no sense, how would one disable this selector? suggest removing the method from the interface.

}

@Override
public List<Bytes> selectTransactions(
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method should also consider the amount of time the tx has been in the pool - older transactions that have been fairly priced while they have sat in pool are likely being censored.

* @param selectorType the transaction selector type
*/
public record InclusionListConfiguration(
InclusionListValidationMode validationMode, InclusionListSelectorType selectorType) {
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The type doesn't need its own class, just allow the configuration to take a generified reference to anything that implements the InclusionListSelector interface. Then InclusionListSelectorType can be removed.

Signed-off-by: jflo <justin+github@florentine.us>
@jflo
Copy link
Copy Markdown
Owner Author

jflo commented Mar 16, 2026

Note: The --engine-inclusion-list-selector-type CLI option is accepted and stored in InclusionListConfiguration, but it's never actually consulted when building the selector.

In EngineGetInclusionListV1.java:106, the selector is hard-coded:

final DefaultInclusionListSelector selector = new DefaultInclusionListSelector(baseFeePerGas);

This should be wired up to use the configured selectorType from InclusionListConfiguration to choose the appropriate selector implementation, otherwise the option has no effect.

final List<WithdrawalParameter> withdrawals;
private final Bytes32 parentBeaconBlockRoot;
private final Long slotNumber;
private final List<String> inclusionListTransactions;
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refactor this for stronger typing, the inclusionListTransactions field should be a List

Signed-off-by: jflo <justin+github@florentine.us>
}

@Override
protected JsonRpcResponse handleSuccessfulExecution(
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method is unnecessary, it would make more sense to do this during validateParameters(), see engine-api-eip7805.md for details on newPayloadV5, and see that it does not require validation to happen after execution.

return INVALID;
}

protected JsonRpcResponse handleSuccessfulExecution(
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove these, see comment re: EngineNewPayloadV5

* @return The {@code TransactionSelectionResults} containing the results of transaction
* evaluation.
*/
public TransactionSelectionResults buildTransactionListForBlock(
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fab-10 this is a naive approach, would be open to your advice on optimizing profitability while making sure inclusion is respected.

Ralph Agent and others added 5 commits March 18, 2026 17:09
Update StrictInclusionListValidator to apply the EIP-7805 conditional
validation algorithm instead of a simple presence check:

1. Skip if T is present in the block (set lookup)
2. Skip if T.gas > gas_left (block has no room)
3. Validate nonce and balance of T.origin against post-execution state

Returns INCLUSION_LIST_UNSATISFIED only when a transaction passes all
three checks (i.e., could have been included but wasn't).

Add InclusionListValidationContext record with gasLeft, accountNonces,
and accountBalances for post-execution state. Update interface with
3-arg validate method; keep 2-arg default for backward compatibility.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Signed-off-by: Ralph Agent <ralph@besu.dev>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Signed-off-by: Ralph Agent <ralph@besu.dev>
- Inline InclusionListValidationStatus as nested Status enum in InclusionListValidationResult
- Delete InclusionListSelectorType and InclusionListConstants (merged into InclusionListConfiguration)
- Move MAX_BYTES_PER_INCLUSION_LIST constant to InclusionListConfiguration
- Replace InclusionListConfiguration.selectorType field with concrete selector instance
- Remove isEnabled() from InclusionListTransactionSelector interface
- Move baseFeePerGas from DefaultInclusionListSelector constructor to selectTransactions() parameter
- Propagate InclusionListConfiguration through RunnerBuilder/JsonRpcMethodsFactory instead of just mode
- Override syncResponse() in EngineNewPayloadV5 for IL validation (replaces handleSuccessfulExecution)
- Delete scaffolding files: ralph.sh, progress.txt, AGENTS.md, CLAUDE.md, SPEC.md
- Update all affected tests for new signatures and removed classes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
…ation

US-010: Fix EngineGetInclusionListV1Test to pass DefaultInclusionListSelector
US-011: Fix InclusionListWorkflowIntegrationTest constructor call
US-013: Replace IL-first ordering with anywhere-in-block algorithm
  - Pool transactions are selected first
  - IL txs already in pool appear at their natural position
  - Remaining IL txs are appended at end (conditional inclusion)
  - Update tests to reflect no-ordering-requirement semantics

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Agent <ralph@besu.dev>
fab-10 and others added 8 commits April 3, 2026 12:30
# Conflicts:
#	ethereum/blockcreation/src/test/java/org/hyperledger/besu/ethereum/blockcreation/LondonFeeMarketBlockTransactionSelectorTest.java

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Signed-off-by: Justin Florentine <justin+github@florentine.us>
This leverage the already present order in the txpool

Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
…o ralph/eip7805-inclusion-lists

# Conflicts:
#	ethereum/blockcreation/src/test/java/org/hyperledger/besu/ethereum/blockcreation/LondonFeeMarketBlockTransactionSelectorTest.java
Signed-off-by: Fabio Di Fabio <fabio.difabio@consensys.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants