error reading delta table to pandas: runtime dropped the dispatch task

[0xdarkman]

delta-rs v0.6.3
pyarrow==9.0.0
pandas==1.3.5
deltalake==0.6.3

import pandas as pd
import pyarrow.dataset as ds

from typing import List
from deltalake.table import DeltaTable

input_table_path = "abfss://CONTAINER@SA.dfs.core.windows.net/1d"

storage_options = {
        "AZURE_STORAGE_ACCOUNT_NAME": account_name,
        "AZURE_STORAGE_ACCOUNT_KEY": account_key,
    }

df: DeltaTable = DeltaTable(input_table_path, storage_options=storage_options).to_pandas()
print(df.head())

deltalake.PyDeltaTableError: Generic MicrosoftAzure error: Error performing get request 1d/date=2022-11-03/part-00013-50e50673-dbc1-481f-9103-0dd0d246a7f7.c000.snappy.parquet: response error "request error", after 0 retries: error sending request for url (https://raddeakarankstorage.blob.core.windows.net/output/1d/date=2022-11-03/part-00013-50e50673-dbc1-481f-9103-0dd0d246a7f7.c000.snappy.parquet): dispatch task is gone: runtime dropped the dispatch task

[0xdarkman]

lol

switching to delta-rs 0.6.4 solves the problem

[roeap]

i was hoping it would :)