Marquez should be able to provide the UI that displays the raw event API result (should also be paginated for ease of use and performance)
solution
Utilize the following API : https://marquezproject.github.io/marquez/openapi.html#operation/getLineageEvents which dumps the received openlineage events.
The icon is only for illustrative purpose and is not the final look
Devise a menu on left to move to a new page, which will contain the result of the latest events received, in paginated fashion. By default, the page should display the last 10 minutes of the events, but the page will have option to look back at 10, 30, 60, 3h, 6h, etc.
By default all the events should be sorted based on the time, and it should be sorted in descending order, such that the latest event is located at the top. The following information should be visible to better locate the event that you are looking for:
- ID
- state
- job/dataset name
- namespace
- date started, and completed
- duration
In order to make the page display more optimized, it might be better if the event JSON should all be 'collapsed' at first, with only a limited set of information visible, and can be expanded as necessary. When expanded, should display the JSON payload in full.
Marquez should be able to provide the UI that displays the raw event API result (should also be paginated for ease of use and performance)
solution
Utilize the following API : https://marquezproject.github.io/marquez/openapi.html#operation/getLineageEvents which dumps the received openlineage events.
Devise a menu on left to move to a new page, which will contain the result of the latest events received, in paginated fashion. By default, the page should display the last 10 minutes of the events, but the page will have option to look back at 10, 30, 60, 3h, 6h, etc.
By default all the events should be sorted based on the time, and it should be sorted in descending order, such that the latest event is located at the top. The following information should be visible to better locate the event that you are looking for:
In order to make the page display more optimized, it might be better if the event JSON should all be 'collapsed' at first, with only a limited set of information visible, and can be expanded as necessary. When expanded, should display the JSON payload in full.