Commit a2964a8
Fix FlightSQL client to display all batches from multi-partition queries
Previously, the FlightSQL client would only display partial results when
queries returned multiple batches (common with GROUP BY without ORDER BY
due to partitioned execution). This happened because:
1. `run_flightsqls()` only fetched one batch using `match streams.next().await`
instead of iterating through all batches
2. `take_record_batches()` only used the first batch when multiple batches
existed, ignoring the rest
This fix:
- Changes `run_flightsqls()` to use `while let` loop to collect all batches
from the FlightSQL stream
- Updates `take_record_batches()` to concatenate all batches before applying
pagination indices, ensuring all rows are visible
Known limitations for future consideration:
- All batches are collected eagerly into memory before display, which could
be problematic for very large result sets
- `concat_batches()` creates a temporary copy when concatenating, briefly
doubling memory usage during pagination
- Unlike the SQL tab which uses lazy batch loading, FlightSQL now loads
all data upfront (this matches expected behavior for aggregation queries
but may warrant lazy loading for large streaming results in the future)
Fixes #3461 parent 2251b56 commit a2964a8
2 files changed
Lines changed: 42 additions & 33 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
279 | 279 | | |
280 | 280 | | |
281 | 281 | | |
282 | | - | |
283 | | - | |
284 | | - | |
285 | | - | |
286 | | - | |
287 | | - | |
288 | | - | |
289 | | - | |
290 | | - | |
291 | | - | |
292 | | - | |
293 | | - | |
294 | | - | |
295 | | - | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
296 | 317 | | |
297 | | - | |
298 | | - | |
299 | | - | |
300 | | - | |
301 | | - | |
302 | | - | |
303 | | - | |
304 | | - | |
305 | | - | |
306 | | - | |
307 | | - | |
308 | | - | |
309 | | - | |
310 | | - | |
311 | | - | |
312 | | - | |
313 | 318 | | |
314 | 319 | | |
315 | 320 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
24 | | - | |
| 24 | + | |
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
| |||
316 | 316 | | |
317 | 317 | | |
318 | 318 | | |
319 | | - | |
320 | | - | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
321 | 325 | | |
322 | 326 | | |
0 commit comments