Commit c542dd6
authored
feat: expose Payload.WritePayload to allow serializing into IPC format (#421)
### Rationale for this change
The BigQuery Storage Write API now accepts Arrow data. But schema data
and record batches needs to send separately. Right now the
`ipc.NewWriter` writes the schema data to the output buffer, which makes
it fail on the BigQuery side, as the first message is the schema. So we
need a way to write just RecordBatches.
See related discussion on
googleapis/google-cloud-go#12478
### What changes are included in this PR?
Add `WritePayload ` method to `Payload`, which exposes the underlying
`writeIPCPayload` method, allowing to write IPC messages more easily.
### Are these changes tested?
Yes, tested with BigQuery Storage Write API. But also added local tests
to it too.
### Are there any user-facing changes?
Yes, `WritePayload` will be a public method available option to users.1 parent c88dd3e commit c542dd6
2 files changed
Lines changed: 31 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
203 | 203 | | |
204 | 204 | | |
205 | 205 | | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
206 | 212 | | |
207 | 213 | | |
208 | 214 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
332 | 332 | | |
333 | 333 | | |
334 | 334 | | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
0 commit comments