Skip to content

Merge on IS NULL condition doesn't work for empty table #2148

@echai58

Description

@echai58

Environment

Delta-rs version: 0.15.1

Binding: python


Bug

What happened:
When you try to merge on a predicate like " t.a IS NULL" and the length of the table is 0, you get a PanicException: index out of bounds: the len is 0 but the index is 0.

What you expected to happen:
This should not throw an error, like how s.a = t.a works correctly when the table is empty.

How to reproduce it:

from deltalake import DeltaTable, write_deltalake
import pyarrow as pa
import pandas as pd

data = pd.DataFrame.from_dict(
    {
        "a": [],
        "b": [],
        "c": [],
    }
)
schema = pa.schema(
    [
        ("a", pa.string()),
        ("b", pa.int32()),
        ("c", pa.int32()),
    ]
)

table = pa.Table.from_pandas(data, schema=schema)


write_deltalake(
    "test",
    table,
    mode="overwrite",
)


new_data = pd.DataFrame.from_dict(
    {
        "a": ["a", "a", "a"],
        "b": [None, 2, 4],
        "c": [5, 6, 7],
    }
)
new_table = pa.Table.from_pandas(new_data, schema=schema)

dt = DeltaTable("test")
dt.merge(
    source=new_table,
    predicate="t.a IS NULL",
    source_alias="s",
    target_alias="t",

).when_matched_update_all().when_not_matched_insert_all().execute()

More details:

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions