Skip to content

add FixedSizeBinary support to create_hashes#3458

Merged
alamb merged 2 commits intoapache:masterfrom
urbanlogiq:fixedsizebinary-hasher
Sep 14, 2022
Merged

add FixedSizeBinary support to create_hashes#3458
alamb merged 2 commits intoapache:masterfrom
urbanlogiq:fixedsizebinary-hasher

Conversation

@mcassels
Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes #1516.

Rationale for this change

Allows aggregation queries on FixedSizeBinary columns.

What changes are included in this PR?

Treats FixedSizeBinary columns the same as Binary columns for creating a hash.

Are there any user-facing changes?

no

@github-actions github-actions bot added the core Core DataFusion crate label Sep 13, 2022
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Sep 13, 2022

Codecov Report

Merging #3458 (35d1e0d) into master (17f069d) will decrease coverage by 0.00%.
The diff coverage is 56.25%.

@@            Coverage Diff             @@
##           master    #3458      +/-   ##
==========================================
- Coverage   85.70%   85.69%   -0.01%     
==========================================
  Files         298      298              
  Lines       54961    54977      +16     
==========================================
+ Hits        47103    47113      +10     
- Misses       7858     7864       +6     
Impacted Files Coverage Δ
datafusion/core/src/physical_plan/hash_utils.rs 41.04% <56.25%> (+0.70%) ⬆️
datafusion/core/src/execution/context.rs 79.06% <0.00%> (ø)
datafusion/core/tests/parquet_pruning.rs 99.43% <0.00%> (ø)
datafusion/expr/src/logical_plan/plan.rs 77.52% <0.00%> (+0.16%) ⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

…s because feature force_hash_collisions changes hash values
@alamb
Copy link
Copy Markdown
Contributor

alamb commented Sep 14, 2022

Thanks @mcassels !

@alamb alamb merged commit 9d028b3 into apache:master Sep 14, 2022
@ursabot
Copy link
Copy Markdown

ursabot commented Sep 14, 2022

Benchmark runs are scheduled for baseline = 9166540 and contender = 9d028b3. 9d028b3 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on ec2-t3-xlarge-us-east-2] ec2-t3-xlarge-us-east-2
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on test-mac-arm] test-mac-arm
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on ursa-i9-9960x] ursa-i9-9960x
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on ursa-thinkcentre-m75q] ursa-thinkcentre-m75q
Buildkite builds:
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Internal error: Unsupported data type in hasher: FixedSizeBinary(16)

5 participants