Skip to content

Commit ea038a4

Browse files
fluffyponystringhandlerSWvheerdenHansie Odendaal
authored
feat: full PayRef implementation (#7154)
## Description --- Implements the Payment Reference (PayRef) system for Tari, providing globally unique identifiers for individual transaction outputs to enable payment verification in Mimblewimble-based blockchains. PayRefs solve the critical business problem of payment verification for exchanges and merchants without compromising privacy. **Key Features:** - Generate PayRefs using `PayRef = Blake2b_256(block_hash || output_hash)` for merkle proof compatibility - Transaction-based PayRef calculation ensuring sender and receiver generate identical references - Support for multiple PayRefs per transaction (multi-output payments) - Confirmation-based stability with reorg safety (default: 5 confirmations) - Privacy-preserving payment verification - Full FFI support for external integrations - Exchange documentation and integration examples ## Motivation and Context --- The primary use case is when users send payments to exchanges but the exchange claims they haven't received the payment. Currently, there's no reliable way to prove payments in Mimblewimble due to: - **Kernel obfuscation**: No direct link between kernels and specific outputs - **Receiver limitations**: Receivers never see kernels in one-sided payments - **Privacy by design**: Transaction relationships are intentionally hidden This creates significant business friction: - High customer support burden for exchanges - Users cannot prove payments were sent - Difficulty maintaining audit trails - Adoption friction due to payment verification problems PayRefs provide a practical solution that ensures both sender and receiver can generate the same payment reference for verification, while maintaining privacy and enabling future merkle proof verification against block headers. ## How Has This Been Tested? --- - **Unit tests**: PayRef generation with new formula, multiple output scenarios, change output identification - **Integration tests**: End-to-end payment flows ensuring sender-receiver PayRef matching - **Reorg tests**: Verification of PayRef cleanup during blockchain reorganizations - **Manual testing**: Console wallet transaction history display with multiple PayRefs support - **API testing**: gRPC endpoints for PayRef retrieval supporting multiple references per transaction - **Edge case validation**: Handling of spent outputs, burned outputs, and same-block spends ## What process can a PR reviewer use to test or verify this change? --- **Basic PayRef Generation:** 1. Send a transaction with multiple outputs using the console wallet 2. Wait for 5+ confirmations 3. Check transaction history - PayRefs should appear as "Available" 4. Verify each PayRef is a 64-character hex string 5. Confirm receiver generates identical PayRefs for the same outputs **Multi-Output Transactions:** 1. Create a transaction with multiple recipients 2. Verify multiple PayRefs are displayed (one per non-change output) 3. Confirm each recipient can verify their specific PayRef **PayRef Search:** 1. In console wallet, press `t` for transactions, then `s` to search 2. Enter a known PayRef (full or partial) 3. Verify it finds the matching transaction **Reorg Safety Testing:** 1. Force a reorg on a test network 2. Verify PayRefs are properly cleaned up for reorged transactions 3. Confirm no orphaned PayRef entries remain **FFI Interface:** 1. Build with `cargo build` 2. Check that new FFI functions compile without errors 3. Verify header file `wallet.h` includes new PayRef structures with array support **Documentation:** 1. Review exchange integration guide in `docs/src/09_adding_tari_to_your_exchange.md` 2. Check user guide in `docs/src/payref_userguide.md` 3. Verify code examples reflect the new multi-PayRef support **API Testing:** ```bash # Test PayRef retrieval for specific transaction grpcurl -plaintext -d '{"tx_id": "YOUR_TX_ID"}' \ localhost:18143 tari.rpc.Wallet/GetTransactionPayRefs # Test payment lookup by PayRef grpcurl -plaintext -d '{"payment_reference_hex": "YOUR_PAYREF_HERE"}' \ localhost:18143 tari.rpc.Wallet/GetPaymentByReference # Verify multiple PayRefs for multi-output transactions grpcurl -plaintext localhost:18143 tari.rpc.Wallet/GetCompletedTransactions ``` ## Breaking Changes --- - [x] None - [ ] Requires data directory on base node to be deleted - [ ] Requires hard fork - [ ] Other - Please specify <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Introduced Payment Reference (PayRef) support across Tari blockchain outputs, enabling unique, privacy-preserving identifiers. - Added CLI commands and UI enhancements in the console wallet for displaying, searching, and verifying PayRefs. - Enabled gRPC and FFI APIs for retrieving and verifying transactions by PayRef, facilitating integration with external services. - Implemented database schema migrations, indexing, and backend support for efficient PayRef lookups. - Added a BaseNode gRPC streaming method to search blockchain outputs by payment references. - Extended wallet and transaction services with PayRef generation, storage, and querying capabilities. - Added PayRef verification and display configuration options in the wallet UI. - **Documentation** - Added comprehensive user guides and integration documentation for PayRef usage and best practices. - Updated application and API overviews to include PayRef features. - Added detailed gRPC and FFI overviews covering PayRef functionality. - **Bug Fixes / Refactor** - Improved error handling and code clarity across multiple modules. - Streamlined output hash tracking and transaction model updates to support PayRef features. - Simplified error constructions and iterator usage in core and communication modules for cleaner code. - **Chores** - Upgraded Diesel dependency in multiple crates for improved compatibility and security. - Added new hash domain for Payment Reference generation. - Minor code cleanups and simplifications across core and comms crates. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: stringhandler <stringhandler@protonmail.com> Co-authored-by: SW van Heerden <swvheerden@gmail.com> Co-authored-by: Hansie Odendaal <pluto@tari.com>
1 parent cf4254c commit ea038a4

File tree

85 files changed

+6820
-352
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

85 files changed

+6820
-352
lines changed

‎Cargo.lock‎

Lines changed: 3 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎applications/minotari_app_grpc/Cargo.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ license.workspace = true
1010
[dependencies]
1111
tari_common_types = { workspace = true }
1212
tari_comms = { workspace = true }
13-
tari_core = { workspace = true }
13+
tari_core = { workspace = true, features = ["base_node"] }
1414
tari_crypto = { workspace = true }
1515
tari_script = { workspace = true }
1616
tari_max_size = { workspace = true }

‎applications/minotari_app_grpc/proto/base_node.proto‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -100,6 +100,8 @@ service BaseNode {
100100
rpc GetTemplateRegistrations(GetTemplateRegistrationsRequest) returns (stream GetTemplateRegistrationResponse);
101101
rpc GetSideChainUtxos(GetSideChainUtxosRequest) returns (stream GetSideChainUtxosResponse);
102102
rpc GetNetworkState(GetNetworkStateRequest) returns (GetNetworkStateResponse);
103+
// PayRef (Payment Reference) lookup for block explorers and external services
104+
rpc SearchPaymentReferences(SearchPaymentReferencesRequest) returns (stream PaymentReferenceResponse);
103105
}
104106

105107
message GetAssetMetadataRequest {
@@ -559,3 +561,35 @@ message LivenessResult{
559561
uint64 ping_latency = 3;
560562
}
561563

564+
// PayRef (Payment Reference) search and lookup messages
565+
566+
// Request to search for outputs by payment reference
567+
message SearchPaymentReferencesRequest {
568+
// Payment reference as hex string (64 characters)
569+
repeated string payment_reference_hex = 1;
570+
// Optional: include spent outputs in results
571+
bool include_spent = 2;
572+
}
573+
574+
// Response containing payment reference match
575+
message PaymentReferenceResponse {
576+
// The payment reference that was found
577+
string payment_reference_hex = 1;
578+
// Block height where the output was mined
579+
uint64 block_height = 2;
580+
// Block hash where the output was mined
581+
bytes block_hash = 3;
582+
// Timestamp when the output was mined
583+
uint64 mined_timestamp = 4;
584+
// Output commitment (32 bytes)
585+
bytes commitment = 5;
586+
// Whether this output has been spent
587+
bool is_spent = 6;
588+
// Height where output was spent (if spent)
589+
uint64 spent_height = 7;
590+
// Block hash where output was spent (if spent)
591+
bytes spent_block_hash = 8;
592+
// Transaction output amount will be 0 for non set a
593+
uint64 min_value_promise = 9;
594+
}
595+

‎applications/minotari_app_grpc/proto/transaction.proto‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,10 @@ message TransactionOutput {
115115
bytes encrypted_data = 10;
116116
// The minimum value of the commitment that is proven by the range proof (in MicroMinotari)
117117
uint64 minimum_value_promise = 11;
118+
// Payment reference (PayRef) - 32-byte Blake2b hash of (block_hash || output_hash)
119+
// This provides a unique, deterministic reference for the output that can be used
120+
// for payment verification without revealing wallet ownership
121+
bytes payment_reference = 12;
118122
}
119123

120124
// Options for UTXOs

‎applications/minotari_app_grpc/proto/wallet.proto‎

Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -434,6 +434,44 @@ service Wallet {
434434
// ```
435435
rpc GetBlockHeightTransactions (GetBlockHeightTransactionsRequest) returns (GetBlockHeightTransactionsResponse);
436436

437+
// Returns all PayRefs (payment references) for a specific transaction.
438+
//
439+
// The `GetTransactionPayRefs` call retrieves all PayRefs associated with the specified
440+
// transaction ID. PayRefs are cryptographic references generated from output hashes
441+
// that allow recipients to verify payments without revealing sensitive transaction details.
442+
//
443+
// ### Request Parameters:
444+
//
445+
// - `transaction_id` (required):
446+
// - **Type**: `uint64`
447+
// - **Description**: The transaction ID to retrieve PayRefs for.
448+
// - **Restrictions**:
449+
// - Must be a valid transaction ID that exists in the wallet.
450+
// - If the transaction ID is invalid or not found, an error will be returned.
451+
//
452+
// ### Example JavaScript gRPC client usage:
453+
//
454+
// ```javascript
455+
// const request = {
456+
// transaction_id: 12345
457+
// };
458+
// const response = await client.getTransactionPayRefs(request);
459+
// console.log("PayRefs:", response.payment_references.map(ref => Buffer.from(ref).toString('hex')));
460+
// ```
461+
//
462+
// ### Sample JSON Response:
463+
//
464+
// ```json
465+
// {
466+
// "payment_references": [
467+
// "0x1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef",
468+
// "0xfedcba0987654321fedcba0987654321fedcba0987654321fedcba0987654321"
469+
// ]
470+
// }
471+
// ```
472+
rpc GetTransactionPayRefs (GetTransactionPayRefsRequest) returns (GetTransactionPayRefsResponse);
473+
474+
437475
// Returns the wallet balance details.
438476
//
439477
// The `GetBalance` call retrieves the current balance status of the wallet,
@@ -1050,6 +1088,52 @@ service Wallet {
10501088

10511089
// Get all completed transactions including cancelled ones, sorted by timestamp and paginated
10521090
rpc GetAllCompletedTransactions(GetAllCompletedTransactionsRequest) returns (GetAllCompletedTransactionsResponse);
1091+
1092+
// Gets transaction information by payment reference (PayRef)
1093+
//
1094+
// The `GetPaymentByReference` call retrieves transaction information using a 32-byte payment reference hash.
1095+
// PayRefs are generated as Blake2b_256(block_hash || output_hash) and provide a stable way to look up
1096+
// transactions even after outputs are spent.
1097+
//
1098+
// ### Request Parameters:
1099+
//
1100+
// - `payment_reference` (required):
1101+
// - **Type**: `bytes` (32 bytes)
1102+
// - **Description**: The payment reference hash to look up
1103+
// - **Restrictions**: Must be exactly 32 bytes representing a valid PayRef
1104+
//
1105+
// ### Example JavaScript gRPC client usage:
1106+
//
1107+
// ```javascript
1108+
// const payref = Buffer.from('a1b2c3d4e5f6789012345678901234567890123456789012345678901234567890', 'hex');
1109+
// const request = { payment_reference: payref };
1110+
// client.getPaymentByReference(request, (err, response) => {
1111+
// if (err) console.error(err);
1112+
// else console.log('Transaction found:', response.transaction);
1113+
// });
1114+
// ```
1115+
//
1116+
// ### Sample JSON Response:
1117+
//
1118+
// ```json
1119+
// {
1120+
// "transaction": {
1121+
// "tx_id": 12345,
1122+
// "source_address": "0x1234abcd...",
1123+
// "dest_address": "0x5678efgh...",
1124+
// "status": "TRANSACTION_STATUS_MINED_CONFIRMED",
1125+
// "direction": "TRANSACTION_DIRECTION_INBOUND",
1126+
// "amount": 1000000,
1127+
// "fee": 20,
1128+
// "is_cancelled": false,
1129+
// "excess_sig": "0xabcdef...",
1130+
// "timestamp": 1681234567,
1131+
// "payment_id": "0xdeadbeef...",
1132+
// "mined_in_block_height": 150000
1133+
// }
1134+
// }
1135+
// ```
1136+
rpc GetPaymentByReference(GetPaymentByReferenceRequest) returns (GetPaymentByReferenceResponse);
10531137
}
10541138

10551139

@@ -1197,6 +1281,9 @@ message TransactionInfo {
11971281
bytes user_payment_id = 14;
11981282
repeated bytes input_commitments = 15;
11991283
repeated bytes output_commitments = 16;
1284+
repeated bytes payment_references_sent = 17;
1285+
repeated bytes payment_references_received = 18;
1286+
repeated bytes payment_references_change = 19;
12001287
}
12011288

12021289
enum TransactionDirection {
@@ -1252,6 +1339,7 @@ message GetCompletedTransactionsResponse {
12521339
TransactionInfo transaction = 1;
12531340
}
12541341

1342+
12551343
// Request message for GetBalance RPC.
12561344
message GetBalanceRequest {
12571345
// Optional: A user-defined payment ID to filter balance data.
@@ -1443,3 +1531,52 @@ message GetBlockHeightTransactionsResponse {
14431531
// List of transactions mined at the specified block height
14441532
repeated TransactionInfo transactions = 1;
14451533
}
1534+
1535+
// PayRef (Payment Reference) related messages and enums
1536+
1537+
// Request message for GetTransactionPayRefs RPC.
1538+
message GetTransactionPayRefsRequest {
1539+
// The transaction ID to retrieve PayRefs for.
1540+
uint64 transaction_id = 1;
1541+
}
1542+
1543+
// Response message for GetTransactionPayRefs RPC.
1544+
message GetTransactionPayRefsResponse {
1545+
// List of PayRefs (32-byte payment references) for the transaction.
1546+
repeated bytes payment_references = 1;
1547+
}
1548+
1549+
1550+
// Response message for GetTransactionsWithPayRefs RPC.
1551+
message GetTransactionsWithPayRefsResponse {
1552+
// The transaction information.
1553+
TransactionInfo transaction = 1;
1554+
// List of PayRefs associated with this transaction.
1555+
repeated bytes payment_references = 2;
1556+
// Number of unique recipients for this transaction.
1557+
uint64 recipient_count = 3;
1558+
}
1559+
1560+
// Request message for getting payment details by payment reference
1561+
message GetPaymentByReferenceRequest {
1562+
// The 32-byte payment reference hash to look up
1563+
bytes payment_reference = 1;
1564+
}
1565+
1566+
// Response message containing transaction information for a payment reference
1567+
message GetPaymentByReferenceResponse {
1568+
// The transaction information if PayRef is found (optional).
1569+
// Returns full transaction details
1570+
TransactionInfo transaction = 1;
1571+
}
1572+
1573+
1574+
// Enum for payment direction
1575+
enum PaymentDirection {
1576+
// Unknown or unspecified direction
1577+
PAYMENT_DIRECTION_UNKNOWN = 0;
1578+
// Payment received by this wallet
1579+
PAYMENT_DIRECTION_INBOUND = 1;
1580+
// Payment sent from this wallet
1581+
PAYMENT_DIRECTION_OUTBOUND = 2;
1582+
}

‎applications/minotari_app_grpc/src/conversions/transaction_output.rs‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -111,6 +111,9 @@ impl TryFrom<TransactionOutput> for grpc::TransactionOutput {
111111
version: output.version as u32,
112112
encrypted_data: output.encrypted_data.to_byte_vec(),
113113
minimum_value_promise: output.minimum_value_promise.into(),
114+
// Payment reference will be populated when the output is included in a block
115+
// and the block hash is available
116+
payment_reference: vec![],
114117
})
115118
}
116119
}

‎applications/minotari_console_wallet/src/automation/commands.rs‎

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2711,6 +2711,132 @@ pub async fn command_runner(
27112711
println!("removing temp wallet in: {:?}", temp_path);
27122712
fs::remove_dir_all(temp_path)?;
27132713
},
2714+
ShowPayRef(args) => {
2715+
// Show transaction details first
2716+
match transaction_service
2717+
.get_any_transaction(args.transaction_id.into())
2718+
.await
2719+
{
2720+
Ok(Some(tx)) => {
2721+
println!("Transaction ID: {}", args.transaction_id);
2722+
let _status = match &tx {
2723+
WalletTransaction::Completed(completed_tx) => {
2724+
println!("Transaction status: Completed");
2725+
println!("Amount: {}", completed_tx.amount);
2726+
println!("Fee: {}", completed_tx.fee);
2727+
println!("Direction: {:?}", completed_tx.direction);
2728+
if let Some(height) = completed_tx.mined_height {
2729+
println!("Mined at height: {}", height);
2730+
}
2731+
if let Some(timestamp) = completed_tx.mined_timestamp {
2732+
println!("Mined timestamp: {}", timestamp);
2733+
}
2734+
if completed_tx.mined_in_block.is_some() {
2735+
println!("\nReceived PayRefs for this transaction:");
2736+
for (i, pay_ref) in completed_tx.calculate_received_payment_references().iter().enumerate() {
2737+
println!("{}. PayRef: {}", i + 1, pay_ref);
2738+
}
2739+
println!("\nSent PayRefs for this transaction:");
2740+
for (i, pay_ref) in completed_tx.calculate_sent_payment_references().iter().enumerate() {
2741+
println!("{}. PayRef: {}", i + 1, pay_ref);
2742+
}
2743+
println!("\nChange PayRefs for this transaction:");
2744+
for (i, pay_ref) in completed_tx.calculate_change_payment_references().iter().enumerate() {
2745+
println!("{}. PayRef: {}", i + 1, pay_ref);
2746+
}
2747+
} else {
2748+
println!("Payrefs: Transaction not mined yet.");
2749+
}
2750+
"Completed"
2751+
},
2752+
minotari_wallet::transaction_service::storage::models::WalletTransaction::PendingInbound(_) => {
2753+
println!("Transaction status: PendingInbound");
2754+
"PendingInbound"
2755+
},
2756+
minotari_wallet::transaction_service::storage::models::WalletTransaction::PendingOutbound(_) => {
2757+
println!("Transaction status: PendingOutbound");
2758+
"PendingOutbound"
2759+
},
2760+
};
2761+
},
2762+
Ok(None) => {
2763+
println!("Transaction ID {} not found", args.transaction_id);
2764+
},
2765+
Err(e) => eprintln!("ShowPayRef error! {}", e),
2766+
}
2767+
},
2768+
FindPayRef(args) => match FixedHash::from_hex(&args.payment_reference_hex) {
2769+
Ok(payref) => match transaction_service.get_payment_by_reference(payref).await {
2770+
Ok(Some(payment_details)) => {
2771+
println!("Found PayRef: {}", args.payment_reference_hex);
2772+
println!("Transaction ID: {}", payment_details.tx_id);
2773+
println!("Amount: {}", payment_details.amount);
2774+
println!("Direction: {:?}", payment_details.direction);
2775+
println!("Block height: {}", payment_details.block_height);
2776+
println!("Confirmations: {}", payment_details.confirmations);
2777+
if let Some(timestamp) = payment_details.timestamp {
2778+
println!("Timestamp: {}", timestamp);
2779+
}
2780+
if let Some(payment_id) = &payment_details.payment_id {
2781+
println!("Payment ID: {}", String::from_utf8_lossy(payment_id));
2782+
}
2783+
},
2784+
Ok(None) => {
2785+
println!("No payment found for PayRef: {}", args.payment_reference_hex);
2786+
},
2787+
Err(e) => eprintln!("FindPayRef error! {}", e),
2788+
},
2789+
Err(e) => {
2790+
eprintln!("FindPayRef error! Invalid PayRef format: {}", e);
2791+
},
2792+
},
2793+
ListTx => {
2794+
debug!(target: LOG_TARGET, "payref_debug: List all transactions command starting execution");
2795+
2796+
match transaction_service.get_completed_transactions(None, None, None).await {
2797+
Ok(txs) => {
2798+
debug!(target: LOG_TARGET, "ListTxs command got {} transactions", txs.len());
2799+
2800+
if txs.is_empty() {
2801+
println!("No transactions.");
2802+
continue;
2803+
}
2804+
println!("Found {} transaction(s)", txs.len());
2805+
println!("{}", "=".repeat(80));
2806+
2807+
for (i, tx) in txs.iter().enumerate() {
2808+
println!("{}. Transaction ID: {}", i + 1, tx.tx_id);
2809+
println!(" Amount: {}", tx.amount);
2810+
println!(" Direction: {:?}", tx.direction);
2811+
println!(" Status: {:?}", tx.status);
2812+
if let Some(height) = tx.mined_height {
2813+
println!(" Mined at height: {}", height);
2814+
}
2815+
if let Some(timestamp) = tx.mined_timestamp {
2816+
println!(" Mined timestamp: {}", timestamp);
2817+
}
2818+
if tx.mined_in_block.is_some() {
2819+
println!("\nReceived PayRefs for this transaction:");
2820+
for (i, pay_ref) in tx.calculate_received_payment_references().iter().enumerate() {
2821+
println!("{}. PayRef: {}", i + 1, pay_ref);
2822+
}
2823+
println!("\nSent PayRefs for this transaction:");
2824+
for (i, pay_ref) in tx.calculate_sent_payment_references().iter().enumerate() {
2825+
println!("{}. PayRef: {}", i + 1, pay_ref);
2826+
}
2827+
println!("\nChange PayRefs for this transaction:");
2828+
for (i, pay_ref) in tx.calculate_change_payment_references().iter().enumerate() {
2829+
println!("{}. PayRef: {}", i + 1, pay_ref);
2830+
}
2831+
} else {
2832+
println!("Payrefs: Transaction not mined yet.");
2833+
}
2834+
println!();
2835+
}
2836+
},
2837+
Err(e) => eprintln!("ListTxs error! {}", e),
2838+
}
2839+
},
27142840
}
27152841
}
27162842
if unban_peer_manager_peers {

0 commit comments

Comments
 (0)
⚡