Inconsistent Read-After-Write Behavior in JSON-RPC Relay Due to Mirror Node Latency

[mshakeg]

Problem

The Hiero Local Node exhibits inconsistent read-after-write behavior that breaks standard Ethereum development workflows. When a transaction is executed and its receipt is returned, immediate balance queries via JSON-RPC often return stale data, causing integration tests to fail and creating unreliable dApp behavior.

Root Cause

• Write path: JSON-RPC Relay → Consensus Node → Transaction executed → Receipt returned immediately
• Read path: JSON-RPC Relay → Mirror Node → Queries database for current state
• Gap: Mirror node stream ingestion from consensus nodes has 2-3 block latency

This creates a race condition where eth_getTransactionReceipt shows transaction success, but subsequent eth_call operations (balance queries, state reads) return pre-transaction values.

Impact on Developers

• Flaky integration tests that require arbitrary delays or block waiting
• Unreliable dApp behavior when users expect immediate state updates after transactions
• Poor developer experience compared to standard Ethereum development
• Difficult to debug timing-dependent issues in complex smart contract interactions

Reproduction

A complete reproduction case is available in a currently private saucerswaplabs repo. See the integration tests in test/hardhat/ERC20Wrapper.integration.test.ts where we had to add explicit block waiting:

// Wait for Hedera state propagation (2 blocks)
console.log('⏳ Waiting for Hedera state propagation...');
const startBlock = await ethers.provider.getBlockNumber();
while ((await ethers.provider.getBlockNumber()) < startBlock + 2) {
  await new Promise((resolve) => setTimeout(resolve, 1000));
}

Without this workaround, tests that perform contract.withdraw() followed by token.balanceOf() fail because the balance query returns stale data.

Solution

Option 1: Configuration-Based Consistency Modes

Add configuration options to hiero-local-node for different consistency guarantees:

docker-compose.override.yml or CLI flags

services:
  json-rpc-relay:
    environment:
      CONSISTENCY_MODE: "strong" # strong|eventual|fast (default: eventual)
      READ_AFTER_WRITE_TIMEOUT: "5000" # ms to wait for mirror node sync

• strong: JSON-RPC relay waits for mirror node to reflect write before returning receipt
• eventual: Current behavior (fast receipts, eventual consistency)
• fast: Optimize for speed with no consistency guarantees

Option 2: Smart Read Routing

Enhance JSON-RPC relay to detect read-after-write scenarios:

• Track recent write transactions per account/contract
• For a configurable window (e.g., 10 seconds), route state queries to consensus node instead of mirror node
• Automatically fall back to mirror node after the window expires

Option 3: Block Node Integration

Leverage the new Block Node service for more consistent read behavior:

• Route state queries through Block Node when available
• Block Node's block-aligned data might provide better read-after-write consistency
• Graceful fallback to mirror node if Block Node is unavailable

Alternatives

Documentation-Only Approach

• Document the timing behavior in official Hedera/Hiero developer guides
• Provide code examples showing proper block waiting patterns
• Add warnings about read-after-write assumptions from Ethereum development

Pros: No architectural changes required
Cons: Pushes complexity to every developer(could be as simple as tx.wait(2/3) which just means local node interactions are even slower), much poorer DX compared to hardhat/foundry local nodes.

Mirror Node Performance Optimization

• Focus on reducing mirror node stream ingestion latency
• Optimize database writes and query performance
• Improve stream processing pipeline

Pros: Benefits all users, maintains current architecture
Cons: May not eliminate the race condition entirely, complex optimization work