display closest nodes not all the nodes in the tree

[dvirgiln]

When the datasets in the namespace are growing it is impossible to use marquez ui to follow dependencies.

I ve attached an image of a subset of the graph

You can see that is impossible to follow. This is in my company what everyone is complaining. If they want to see the lineage of one dataset, they see all the tree with things that they are not really interested. Ideally I would display a button or filter where you can decide how many levels you want to see, and by default you just see one level downstream nodes and one level upstream nodes.

Hope it is clear

[wslulciuc]

@dvirgiln: You're absolutely right, for very large graphs it's hard to navigate / follow dependencies (not a great experience). Usability is something we've been meaning to address, and there's plenty of room for improvement here. I can start opening issues that outline some of the changes we'd like to make in the UI (with mockups), but I also encourage you to do the same and open UI issues using the labels web and feature. I'll make sure to add the issues to our roadmap

To give you a sneak peek of what the UI changes we'd like to make to the lineage graph navigation, you can checkout demo.datakin.com(Datakin is powered by Marquez ❤️ ):

The UI above uses the Marquez LineageAPI, but applies client-side grouping logic. The team is currently discussing the effort to begin moving the dataset and job grouping logic to Marquez. What are your initial thoughts?

Note, specifying how many levels deep in the graph should be rendered is already supported by the LineageAPI, it's just a matter of providing that option to the user 😄