Skip to content

TrinoOfflineStore does not cleanup temporary tables #6306

@mattiamatrix

Description

@mattiamatrix

Expected Behavior

Temporary entity tables created during get_historical_features() should be cleaned up after the retrieval job completes, consistent with how BigQuery, Redshift, and Athena handle it.

Current Behavior

Each call to get_historical_features() with a pandas DataFrame entity_df creates a permanent table ({catalog}.{dataset}.feast_entity_df_<uuid>) that is never dropped, leaking tables in the Trino catalog indefinitely.

Steps to reproduce

  1. This is a known issue in TrinoOfflineStore https://github.com/feast-dev/feast/blob/master/sdk/python/feast/infra/offline_stores/contrib/trino_offline_store/trino.py#L486

Specifications

  • Version: 0.50+
  • Platform: Any
  • Subsystem: TrinoOfflineStore

Possible Solution

Refactor TrinoRetrievalJob to accept a context manager (query_generator) instead of a plain query string, following the pattern used by BigQuery/Redshift/Athena, and add a DROP TABLE IF EXISTS in the finally block.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions