Skip to content

Commit 99484e9

Browse files
pixelherodevNoam Preil
andauthored
optimization: comparison: when DataType is static, skip reflection (#542)
### Rationale for this change Reflection is slow. When working with primitive data types - e.g. PrimitiveType.Float64 - the DataType value is a constant consisting of the (constant) type indicator, and the constant pointer to the only instance of the Float64 type. For such cases, we can do a fast-path for TypeEqual and detect the interface equality without the current path of switching on the type of DataType, and using reflect.DeepEqual. reflect.DeepEqual makes several allocations and is not particularly fast. This patch cuts out 50-75% of the runtime of `schema.Equals` in our real-world scenario in my testing. ### What changes are included in this PR? A branch is added to the TypeEqual switch, which returns early when the two types compare equal through shallow comparison. In such a case, deeper comparison must necessarily show equality, as every property is definitionally identical. Detecting this early can make e.g. schema validations significantly faster. ### Are these changes tested? Yes, though only with our internal schemas. I noticed arrow showing up heavily in our profiles and started hacking on it to make it go away :) ### Are there any user-facing changes? It runs faster? That's technically user-facing ;) Co-authored-by: Noam Preil <noam@pixelhero.dev>
1 parent a489ef0 commit 99484e9

1 file changed

Lines changed: 3 additions & 1 deletion

File tree

arrow/compare.go

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,8 +45,10 @@ func TypeEqual(left, right DataType, opts ...TypeEqualOption) bool {
4545
}
4646

4747
switch {
48+
case left == right:
49+
return true
4850
case left == nil || right == nil:
49-
return left == nil && right == nil
51+
return false
5052
case left.ID() != right.ID():
5153
return false
5254
}

0 commit comments

Comments
 (0)