Environment
Delta-rs version: 0.15.1
Binding: Python 3.8
Environment: Local
- Cloud provider: Azure
- OS: Linux
- Other: External table mounted on adls gen 2 mount
Bug
What happened:
I'm trying to read an external table containing an IDENTITY column. However I encounter the following error when running locally :
dt = DeltaTable(path)
File "/home/***/Documents/***/venv/lib/python3.8/site-packages/deltalake/table.py", line 396, in __init__
self._table = RawDeltaTable(
_internal.DeltaError: Delta protocol violation: Kernel: Invalid url: data did not match any variant of untagged enum MetadataValue at line 1 column 181
If I recreate the table without the Identity column, it works correctly
What you expected to happen:
The table is read correctly in python
How to reproduce it:
Create a table with an identity :
CREATE TABLE spark_catalog.data_model.d_dates (
ID_D_DATE BIGINT GENERATED ALWAYS AS IDENTITY (START WITH 1 INCREMENT BY 1),
TXT_DateKey STRING
) USING delta LOCATION 'dbfs:/mnt/adls_gen2_mount/d_dates' TBLPROPERTIES ('delta.minReaderVersion' = '1','delta.minWriterVersion' = '3')
Mount the adls endpoint locally or copy the folder to your local computer.
Read the folder using DeltaTable(local_path)
More details:
I tried changing the minWriterVersion / switching between GENERATED ALWAYS and GENERATED BY DEFAULT but none worked.
Except for local access, the table works just fine when working with SQL or spark.
Environment
Delta-rs version: 0.15.1
Binding: Python 3.8
Environment: Local
Bug
What happened:
I'm trying to read an external table containing an IDENTITY column. However I encounter the following error when running locally :
If I recreate the table without the Identity column, it works correctly
What you expected to happen:
The table is read correctly in python
How to reproduce it:
Create a table with an identity :
Mount the adls endpoint locally or copy the folder to your local computer.
Read the folder using DeltaTable(local_path)
More details:
I tried changing the minWriterVersion / switching between GENERATED ALWAYS and GENERATED BY DEFAULT but none worked.
Except for local access, the table works just fine when working with SQL or spark.