Add facet tables to avoid querying `lineage_events` table

[wslulciuc]

We are seeing OpenLineage events that easily exceed > 10MBs resulting in out-of-memory (OOM) errors as facet queries require loading the raw event in memory, then filtering for relevant facets. To avoid querying the lineage_events table for facets, let's group facets in tables by how they will be accessed:

• dataset_facets
• job_facets
• run_facets

[pawel-big-lebowski]

Are we filtering for relevant facets in database or Marquez?
Perhaps we could add some json indexing in postgresql (like described here https://scalegrid.io/blog/using-jsonb-in-postgresql-how-to-effectively-store-index-json-data-in-postgresql/) to improve filtering.

[RNHTTR]

I'm interested in taking this on, but I'm not sure I have enough context to pick it up 😅 . Is the idea to pull facet data for dataset_version, job_version and run out of the lineage_events table and into their own tables? Or, to @pawel-big-lebowski 's point, by adding indexes to the lineage_events table to make queries more efficient?

[pawel-big-lebowski]

@RNHTTR please take a look at the proposal: #2076

[mobuchowski]

Has been done