Commit d41e0b5
authored
fix(parquet): strip repetition_type from root SchemaElement during serialization (#723)
### Rationale
The Parquet spec says the root of the schema doesn't have a
`repetition_type`. arrow-go writes `REPEATED` for the root
`SchemaElement` in the Thrift footer, which breaks interop with
consumers like Snowflake. #722 has the full writeup and
cross-implementation comparison.
### What changes are included in this PR?
`ToThrift()` now nils out `RepetitionType` on the root element before
returning, stripping it from the serialized output. This matches how
parquet-java and arrow-rs handle the root.
The in-memory representation and `WithRootRepetition` API are
unaffected. `FromParquet` [already tolerates a nil root repetition
type](https://github.com/apache/arrow-go/blob/main/parquet/schema/schema.go#L78-L79),
so this is backwards-compatible for both readers and writers.
### Are these changes tested?
Updated the existing `TestNestedExample` and added
`TestToThriftRootRepetitionStripped` which checks that the root's
`repetition_type` is stripped for all three repetition variants and that
non-root elements keep theirs.
Closes #7221 parent b0287d7 commit d41e0b5
3 files changed
Lines changed: 23 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
172 | 172 | | |
173 | 173 | | |
174 | 174 | | |
175 | | - | |
| 175 | + | |
176 | 176 | | |
177 | 177 | | |
178 | 178 | | |
| |||
206 | 206 | | |
207 | 207 | | |
208 | 208 | | |
209 | | - | |
| 209 | + | |
210 | 210 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
272 | 272 | | |
273 | 273 | | |
274 | 274 | | |
| 275 | + | |
275 | 276 | | |
276 | 277 | | |
277 | 278 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
92 | 92 | | |
93 | 93 | | |
94 | 94 | | |
| 95 | + | |
| 96 | + | |
95 | 97 | | |
96 | | - | |
| 98 | + | |
97 | 99 | | |
98 | 100 | | |
99 | 101 | | |
| |||
120 | 122 | | |
121 | 123 | | |
122 | 124 | | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
123 | 142 | | |
124 | 143 | | |
125 | 144 | | |
| |||
0 commit comments