feat: add BuildHasher variants for hash_utils

[xudong963]

Which issue does this PR close?

• Closes Add BuildHasher variants for hash_utils #21428.

Rationale for this change

This PR adds BuildHasher-based variants for hash_utils so callers can compute row hashes with a caller-provided hash builder instead of always using DataFusion's default RandomState.

The main constraint is performance: with_hashes is a hot path, especially for string, dictionary, and nested array hashing. A previous version in #21429 caused measurable regressions in the default RandomState path, for example large_utf8: single, no nulls regressed from roughly 26.7us to 36.3us, and large_utf8: multiple, no nulls from roughly 112us to 127us.

This version keeps the default path performance-oriented by avoiding a fully generic BuildHasher rewrite of the existing hot loops.

What changes are included in this PR?

This PR adds:

• with_hashes_with_hasher
• create_hashes_with_hasher
• custom-hasher implementations for primitive, string, binary, byte-view, dictionary, and nested arrays
• tests covering custom hashers, multi-column hashing, and dictionary equivalence

The implementation intentionally uses a hybrid design:

• Default RandomState leaf hot paths remain specialized.
• Custom BuildHasher leaf paths live separately in hash_utils/build_hasher.rs.
• Nested/structural logic is shared through an internal child-hashing adapter, so struct/list/map/union/run/dictionary behavior does not need to be broadly duplicated.

The trade-off is that there is still some duplication for primitive/string/binary leaf loops. That duplication is intentional: those are the hottest loops, and keeping them separate prevents the existing RandomState path from becoming generic over BuildHasher or being perturbed by the custom-hasher implementation.

Are these changes tested?

Yes.

[xudong963]

run benchmark with_hashes

[adriangbot]

🤖 Criterion benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c4310621161-1803-2z8k9 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing xudong963/hash-buildhasher (0823c8d) to fd093fb (merge-base) diff
BENCH_NAME=with_hashes
BENCH_COMMAND=cargo bench --features=parquet --bench with_hashes
BENCH_FILTER=
Results will be posted here when complete

---

File an issue against this benchmark runner

[adriangbot]

🤖 Criterion benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

group                                        main                                   xudong963_hash-buildhasher
-----                                        ----                                   --------------------------
dense_union: multiple, no nulls              1.00    234.0±0.28µs        ? ?/sec    1.00    234.2±0.37µs        ? ?/sec
dense_union: single, no nulls                1.00     78.3±0.12µs        ? ?/sec    1.00     78.3±0.10µs        ? ?/sec
dictionary_utf8_int32: multiple, no nulls    1.09     16.1±1.75µs        ? ?/sec    1.00     14.8±0.13µs        ? ?/sec
dictionary_utf8_int32: multiple, nulls       1.00     32.2±0.07µs        ? ?/sec    1.00     32.2±0.04µs        ? ?/sec
dictionary_utf8_int32: single, no nulls      1.00      4.6±0.00µs        ? ?/sec    1.00      4.6±0.01µs        ? ?/sec
dictionary_utf8_int32: single, nulls         1.00     10.9±0.01µs        ? ?/sec    1.02     11.0±0.01µs        ? ?/sec
int64: multiple, no nulls                    1.00     16.9±0.01µs        ? ?/sec    1.00     17.0±0.08µs        ? ?/sec
int64: multiple, nulls                       1.00     26.8±0.02µs        ? ?/sec    1.07     28.7±0.03µs        ? ?/sec
int64: single, no nulls                      1.00      4.6±0.01µs        ? ?/sec    1.00      4.6±0.00µs        ? ?/sec
int64: single, nulls                         1.00      8.7±0.01µs        ? ?/sec    1.00      8.8±0.01µs        ? ?/sec
large_utf8: multiple, no nulls               1.03    111.2±1.46µs        ? ?/sec    1.00    108.2±0.49µs        ? ?/sec
large_utf8: multiple, nulls                  1.00    134.2±0.60µs        ? ?/sec    1.00    134.6±0.47µs        ? ?/sec
large_utf8: single, no nulls                 1.00     33.9±0.10µs        ? ?/sec    1.01     34.2±0.32µs        ? ?/sec
large_utf8: single, nulls                    1.04     46.1±0.13µs        ? ?/sec    1.00     44.5±0.71µs        ? ?/sec
list_array: multiple, no nulls               1.01    147.0±0.39µs        ? ?/sec    1.00    146.2±0.34µs        ? ?/sec
list_array: multiple, nulls                  1.01    177.4±0.46µs        ? ?/sec    1.00    176.1±0.63µs        ? ?/sec
list_array: single, no nulls                 1.00     49.3±0.15µs        ? ?/sec    1.00     49.2±0.14µs        ? ?/sec
list_array: single, nulls                    1.01     59.4±0.14µs        ? ?/sec    1.00     58.7±0.15µs        ? ?/sec
list_array_sliced: 1/10 of 81920 rows        1.00     49.2±0.21µs        ? ?/sec    1.00     49.2±0.40µs        ? ?/sec
list_array_sliced: 1/2 of 16384 rows         1.00     49.5±0.11µs        ? ?/sec    1.00     49.5±0.12µs        ? ?/sec
list_array_sliced: 1/5 of 40960 rows         1.00     49.6±0.20µs        ? ?/sec    1.00     49.7±0.08µs        ? ?/sec
map_array: multiple, no nulls                1.00    242.6±0.72µs        ? ?/sec    1.00    242.7±0.61µs        ? ?/sec
map_array: multiple, nulls                   1.00    247.8±1.20µs        ? ?/sec    1.00    247.9±0.77µs        ? ?/sec
map_array: single, no nulls                  1.00     81.1±0.23µs        ? ?/sec    1.01     81.7±0.20µs        ? ?/sec
map_array: single, nulls                     1.01     83.3±0.51µs        ? ?/sec    1.00     82.8±0.28µs        ? ?/sec
map_array_sliced: 1/10 of 81920 rows         1.00     81.4±0.25µs        ? ?/sec    1.00     81.7±0.33µs        ? ?/sec
map_array_sliced: 1/2 of 16384 rows          1.01     81.3±0.18µs        ? ?/sec    1.00     80.9±0.22µs        ? ?/sec
map_array_sliced: 1/5 of 40960 rows          1.00     81.0±0.28µs        ? ?/sec    1.01     81.7±0.44µs        ? ?/sec
run_array_int32: multiple, no nulls          1.00      5.0±0.01µs        ? ?/sec    1.00      5.0±0.01µs        ? ?/sec
run_array_int32: multiple, nulls             1.01      6.7±0.01µs        ? ?/sec    1.00      6.7±0.01µs        ? ?/sec
run_array_int32: single, no nulls            1.00   1831.9±4.47ns        ? ?/sec    1.00   1825.9±2.24ns        ? ?/sec
run_array_int32: single, nulls               1.00      2.4±0.01µs        ? ?/sec    1.01      2.4±0.00µs        ? ?/sec
sparse_union: multiple, no nulls             1.00    256.0±0.46µs        ? ?/sec    1.00    255.3±0.69µs        ? ?/sec
sparse_union: single, no nulls               1.00     85.4±0.23µs        ? ?/sec    1.00     85.4±0.22µs        ? ?/sec
sparse_union_sliced: 1/10 of 81920 rows      1.01     86.0±0.22µs        ? ?/sec    1.00     85.3±0.17µs        ? ?/sec
sparse_union_sliced: 1/2 of 16384 rows       1.00     85.5±0.26µs        ? ?/sec    1.00     85.5±0.23µs        ? ?/sec
sparse_union_sliced: 1/5 of 40960 rows       1.00     85.6±0.21µs        ? ?/sec    1.00     85.4±0.25µs        ? ?/sec
struct_array: multiple, no nulls             1.01    156.2±1.48µs        ? ?/sec    1.00    154.6±0.64µs        ? ?/sec
struct_array: multiple, nulls                1.00    170.1±0.87µs        ? ?/sec    1.00    170.7±1.04µs        ? ?/sec
struct_array: single, no nulls               1.01     52.8±0.15µs        ? ?/sec    1.00     52.2±0.32µs        ? ?/sec
struct_array: single, nulls                  1.00     57.8±0.15µs        ? ?/sec    1.00     57.7±0.17µs        ? ?/sec
utf8: multiple, no nulls                     1.01    103.3±0.41µs        ? ?/sec    1.00    102.3±0.38µs        ? ?/sec
utf8: multiple, nulls                        1.00    144.1±2.47µs        ? ?/sec    1.00    143.8±2.55µs        ? ?/sec
utf8: single, no nulls                       1.00     27.3±0.12µs        ? ?/sec    1.03     28.1±0.13µs        ? ?/sec
utf8: single, nulls                          1.00     46.6±0.55µs        ? ?/sec    1.01     47.2±0.19µs        ? ?/sec
utf8_view (small): multiple, no nulls        1.00     19.0±0.02µs        ? ?/sec    1.02     19.3±0.16µs        ? ?/sec
utf8_view (small): multiple, nulls           1.02     29.9±0.07µs        ? ?/sec    1.00     29.2±0.05µs        ? ?/sec
utf8_view (small): single, no nulls          1.00      5.1±0.01µs        ? ?/sec    1.00      5.1±0.03µs        ? ?/sec
utf8_view (small): single, nulls             1.01      9.7±0.01µs        ? ?/sec    1.00      9.7±0.01µs        ? ?/sec
utf8_view: multiple, no nulls                1.00    106.0±0.35µs        ? ?/sec    1.00    105.9±0.40µs        ? ?/sec
utf8_view: multiple, nulls                   1.01    116.8±0.19µs        ? ?/sec    1.00    115.1±0.44µs        ? ?/sec
utf8_view: single, no nulls                  1.00     34.8±0.10µs        ? ?/sec    1.00     34.7±0.12µs        ? ?/sec
utf8_view: single, nulls                     1.00     34.1±0.06µs        ? ?/sec    1.08     36.8±0.15µs        ? ?/sec

Resource Usage

base (merge-base)

Metric	Value
Wall time	525.1s
Peak memory	4.0 GiB
Avg memory	4.0 GiB
CPU user	662.1s
CPU sys	1.3s
Peak spill	0 B

branch

Metric	Value
Wall time	525.1s
Peak memory	4.0 GiB
Avg memory	4.0 GiB
CPU user	661.7s
CPU sys	0.7s
Peak spill	0 B

---

File an issue against this benchmark runner

[xudong963]

@Dandandan, this PR resolves the regression for the #21429. It would be sweet to get your thoughts!