Skip to content

Add created_at indexed column to Lineage events table #2300

@prachim-collab

Description

@prachim-collab

For analytics use case we need to incrementally copy and export lineage_events to DWH. For this use case there is no good way to identify incrementally created events in database. The current event_time in lineage_events table is client generated and can be back dated.
Proposing to alter lineage_events table to add a new created_at timestamp column. Also index this new column.
More discussion to be had regarding backfill and default values of this column.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    Status
    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions