UI Hints in USD

[agb63]

Description of Proposal

Proposal for expanding presentation metadata (aka UI hints) by adding e.g., widget type, numeric limits, and conditional visibility/hiddenness.

Rendered view available here.
See updated version here.

Contributing

• I agree to and accept the Supplemental Terms.

[xeash]

Great work!

We actually explored very similar ideas a few years back, which we recently open-sourced here: OpenDCC. While we never turned it into a formal proposal due to concerns around maturity, our goals closely aligned with yours — particularly in trying to support args-style presentation metadata for complex authoring UIs.

We similarly followed the .args format for hint names and structure, and tried to leverage existing standard metadata wherever possible, extending it with fields like displayWidget and displayWidgetHints in our custom property editor framework.

One main difference is that, at the time, we viewed Sdr as shader-specific (before UsdLux adopted that pattern in usd-core), and we wanted UI hints to be applicable to all USD prims. Because of that, we chose separate metadata keys and avoided coupling with Sdr.

Instead, we introduced a dedicated mechanism called UsdPrimFallbackProxy:
https://github.com/shapefx/OpenDCC/tree/main/src/lib/usd/usd_fallback_proxy

This proxy was designed specifically for UI tooling and does extend the schema, but its scope goes beyond just metadata aggregation. It supports a variety of UI-driven behaviors, such as exposing fallback/default attributes, handling conditional visibility via logic or render-specific APIs, and pulling from multiple metadata sources including Sdr where applicable.

Looking forward to seeing how this proposal evolves — it’s great to see this area getting formal attention!

[asluk]

Thanks @agb63 ! Here's summary of the feedback from NV folks.

---

• Scoping and Namespacing: There's a desire for UI hints to be scoped or namespaced, drawing an analogy to assetInfo, with suggestions for names like uiHints or uiInfo.
• Tooltips for Generic Cases: A proposal was made to include tooltips within the hints. This would allow the same tooltip to travel with an asset across applications and be set by users without needing code changes.
• Widget Behavior and Application:
  • For material binding, the use of popup widgets was suggested for selecting from existing material lists. A question also arose regarding why radio options are used for ground types in the environment property instead of popups.
  • Specific queries were raised about how sliderCenter would operate with min and max values, and it was noted that sliderExponent isn't currently supported.
  • Concerns were voiced about handling misconfigurations, such as applying multiSelect to Sdf.AssetPath, which would necessitate an Sdf.AssetPathArray.
  • Clarification was sought on whether "popup" refers to a new window appearing or simply a list with a misleading name, and how this would integrate with property windows.
  • The implication of arrayIsDynamic=False was discussed, given that current pxr arrays are dynamic without blank entries, potentially requiring code to check for None values. It was also mentioned that popup widgets displaying an integer with a list of options are currently supported for attributes and could be expanded.
• Handling Time-Varying Properties: A suggestion was put forth to consider an editableIfTimeVarying boolean option for "number" widget types. This option, which could default to False (mirroring current Kit behavior), would allow or prevent editing of attributes with time samples. If set to True, it might enable adding a time sample of the authored value at the current USD timecode.
• Advanced Widget Functionality: Several considerations for more complex widget interactions were brought up:
  • The need for specifying numeric precision for float and double typed widgets.
  • Clarifying the distinction and appropriate use cases for checkboxes (multiple selections) versus radio boxes (exclusive selection).
  • Implementing dynamic linked data and conditional expressions, where certain data might be disabled or become visible/editable only based on a previous selection (e.g., a radio button determining which widgets become active).
  • Managing complex dictionary data (e.g., {variantA: [subvariantAA, subvariantAB]}). This might necessitate creating two linked comboboxes, where the second dynamically updates based on the selection in the first.
• Examples and Clarity: A request was made for more examples to better understand how elements like valueLabels and valueLabelsOrder are meant to indicate a popup, though the overall usefulness and similarity to existing Kit hinting were acknowledged. An example demonstrating a "Scope" prim with an integer "level" attribute using these labels was provided.

[spiffmon]

• Handling Time-Varying Properties: A suggestion was put forth to consider an editableIfTimeVarying boolean option for "number" widget types. This option, which could default to False (mirroring current Kit behavior), would allow or prevent editing of attributes with time samples. If set to True, it might enable adding a time sample of the authored value at the current USD timecode.

Should this not be driven instead by the attribute's variability (i.e. Uniform vs Varying)? Also unsure why this would be applicable only to "number" widget types? If there are asset-specific reasons for wanting to restrict a Varying attribute to only authoring defaults (is that the desired behavior?) then I'd think the default for editableIfTimeVarying would need to be True or else almost the whole existing world becomes unanimatable? Apologies if I'm misunderstanding, here!

[asluk]

• Handling Time-Varying Properties: A suggestion was put forth to consider an editableIfTimeVarying boolean option for "number" widget types. This option, which could default to False (mirroring current Kit behavior), would allow or prevent editing of attributes with time samples. If set to True, it might enable adding a time sample of the authored value at the current USD timecode.

Should this not be driven instead by the attribute's variability (i.e. Uniform vs Varying)? Also unsure why this would be applicable only to "number" widget types? If there are asset-specific reasons for wanting to restrict a Varying attribute to only authoring defaults (is that the desired behavior?) then I'd think the default for editableIfTimeVarying would need to be True or else almost the whole existing world becomes unanimatable? Apologies if I'm misunderstanding, here!

yes I think I agree-- I'll pass that on-- thanks @spiffmon !

[JamesPedFoundry]

Thank you very much for this proposal, we are very interested in this proposal.

I've attached a few comments as comments on lines (should allow for replies and ability to resolve them and close them up.

[agb63]

Responding to asluk above. My guess is most or all of this feedback came out of the more limited early proposal we circulated prior to posting the full proposal, so some of my responses below will reference stuff that wasn't available til this past week.

• Scoping and Namespacing: There's a desire for UI hints to be scoped or namespaced, drawing an analogy to assetInfo, with suggestions for names like uiHints or uiInfo.

In the full proposal we have exactly this -- hints are grouped within a uiHints dictionary field.

• Tooltips for Generic Cases: A proposal was made to include tooltips within the hints. This would allow the same tooltip to travel with an asset across applications and be set by users without needing code changes.

We've discussed this internally as well -- there's an existing plugin metadata field userDocBrief which I believe is intended to be usable as a tooltip. I'll chat with folks here about whether that should move into the uiHints dict and/or be made available by the same API classes that the rest of the hints will use.

• Widget Behavior and Application:

  • For material binding, the use of popup widgets was suggested for selecting from existing material lists. A question also arose regarding why radio options are used for ground types in the environment property instead of popups.
  • Specific queries were raised about how sliderCenter would operate with min and max values, and it was noted that sliderExponent isn't currently supported.

The various slider options I just carried forward from the Katana args format. I will try and confirm how they are expected to work there. sliderExponent in particular seems like it might not be super necessary to include.

• Concerns were voiced about handling misconfigurations, such as applying multiSelect to Sdf.AssetPath, which would necessitate an Sdf.AssetPathArray.

We hadn't planned on doing any validation on the USD side, except possibly adding a validator that checks the limits values against authored value, taking the strict flag into account. In any case though, I think DCCs will have to do some amount of sanity checking for misconfigured hints, since I don't think we can't prevent them from from existing in an asset. If that feels like a major deficiency though, I'm definitely open to hearing about it.

• Clarification was sought on whether "popup" refers to a new window appearing or simply a list with a misleading name, and how this would integrate with property windows.

Yeah this name could be clearer (it was also inherited from the args format). It's intended to be a dropdown list / combo box, not a small popup dialog. Maybe we should call it dropdown or combo ?

• The implication of arrayIsDynamic=False was discussed, given that current pxr arrays are dynamic without blank entries, potentially requiring code to check for None values. It was also mentioned that popup widgets displaying an integer with a list of options are currently supported for attributes and could be expanded.

The intention with arrayIsDynamic=False is to signal to the UI whether it should allow the user to add or remove values from an array, vs. just changing them in place.

• Advanced Widget Functionality: Several considerations for more complex widget interactions were brought up:

  • The need for specifying numeric precision for float and double typed widgets.

The full proposal added a widget option for this called digits.

• Clarifying the distinction and appropriate use cases for checkboxes (multiple selections) versus radio boxes (exclusive selection).

I'm not totally following this so I may be missing the mark here. The checkbox widget type as described in the proposal is intended to be a single boolean on/off switch for a single attribute. One could imagine placing multiple related boolean attributes together in a display group though, all with widgetType=checkbox specified, to get a "multi-select" type interface.

We have the popup widget type (perhaps to be renamed, per above) for use in exclusive selection situations. Is there a desire to have a separate widget type (say, radioGroup) that would present a group of radio buttons that select among an attribute's allowed values?

• Implementing dynamic linked data and conditional expressions, where certain data might be disabled or become visible/editable only based on a previous selection (e.g., a radio button determining which widgets become active).

The full proposal has a notion of conditional visibility that gets at this.

• Managing complex dictionary data (e.g., {variantA: [subvariantAA, subvariantAB]}). This might necessitate creating two linked comboboxes, where the second dynamically updates based on the selection in the first.

We hadn't considered anything this complex for allowed attribute values, but it's an interesting idea.

[asluk]

Thanks @agb63 ! I've passed your responses to the devs here. I suspect "combo" is a clearer name than "popup" . Cheers!

[asluk]

. Is there a desire to have a separate widget type (say, radioGroup) that would present a group of radio buttons that select among an attribute's allowed values?

@agb63 I discussed this internally-- a developer was thinking of using a series of checkboxes for e.g., allowing multiple RenderVars to be selected for a given RenderProduct, or for exposing individual X, Y, Z-axis options that can be multiply enabled for setting up xformOps.

I suggested to keep things simpler for now, as those kinds of use cases don't exactly map directly to the allowed values for a given property, and they agreed.

(I also pointed out that check boxes wouldn't be able to author the required ordering of the RenderVars for a RenderProduct)

[agb63]

I just updated the proposal with our latest thinking, for anyone who's following along:

* Removed reference to the locked hint, which will be addressed separately
* Added more detail to the limits discussion section
* Added arraySizeConstraint discussion section
* Noted deferral of widgetType and widgetOptions implementation

The big takeaway is the deferral on implementing widgetType and widgetOptions, for now. Some concerns have been raised in particular about them being open-ended and what that means for DCC vendors. We want to return to these fields in the future and make sure they're fully baked, but in the meantime we don't want to hold up implementation of the other fields.

The arraySizeConstraint core field, newly added to the proposal, will replace the array-related widget options that had been described in the proposal.

The updated rendered view is here.

[JamesPedFoundry]

It's been a while since I've caught up on this but I had a question as a follow up to the GitHub issue I logged a while ago which isn't quite covered by this, but feels related.

I'll start with the latter, in the fact currently, and this is a problem scoped to SdrRegistry more so than schemas.

There isn't yet a consistent catagorisation metadata for the whole shader itself. Such that it is currently difficult for DCC's to be able to create interfaces that only show, for example, one renderers set of shaders. This can even go a step further, with shading lists like MaterialX having a gigantic amount of shader identifiers, categorisation would be important to help users scope down to the shader they want to add. Ideally this comes from the plug-ins defining the shader.

My proposal for this was detailed out in PixarAnimationStudios/OpenUSD#3575, where we describe the category argument for this purpose with an update to the documentation to specify this, and to specify the ability to separate with : (similar to displayGroups).

Although not attribute UI hint related, I still believe this falls into the nature of tidying up the hints for user interfaces to utilise. From my investigations this field tends to not be specified.

Again, may well be a case for not holding up the other metadata, but would be nice to have utilization for display grouping shaders themselves, not just their properties. More than happy to spin off a separate proposal for some of those other thoughts from the GitHub issue linked if we feel it does expand the scope too far.

Cheers,
James

[spiffmon]

Hi @JamesPedFoundry - apologies we haven't provided any updates on the Sdr categorization work. We have been working at it and have a proposal we hope to share soon - a "final" round of internal review produced some suggestions we are doing some deeper diving into. Thanks for your patience - it is definitely a priority; I'm not sure if you were sugesting this, but we have not, thus far, considered applying the same set of categorizations we're talking about for Sdr nodes to prim schemas more generally. Given that schema domains are often so different from each other, we hadn't considered trying to categorize them all in a consistent way (beyond displayGroup).

[JamesPedFoundry]

Hi @JamesPedFoundry - apologies we haven't provided any updates on the Sdr categorization work. We have been working at it and have a proposal we hope to share soon - a "final" round of internal review produced some suggestions we are doing some deeper diving into. Thanks for your patience - it is definitely a priority; I'm not sure if you were sugesting this, but we have not, thus far, considered applying the same set of categorizations we're talking about for Sdr nodes to prim schemas more generally. Given that schema domains are often so different from each other, we hadn't considered trying to categorize them all in a consistent way (beyond displayGroup).

Thank you for the update! Just mentioned it here as the updates above and some internal work prompted it in my mind.

In terms of schema grouping, it was something that crossed my mind whilst typing up the questions above. I am not certain of it's necessity and doesn't sit as high as shaders in priority for us. Schemas have the nicety of having inheritance and auto-applying behaviours we can query, which is something we do utilise to good effect already. However, it could still be nice to have such that we could separate out Physics based schemas from material or light related schemas, even ways for studios to categorise their own schemas from the defaults to make it clearer what is custom. Very likely a different idea for a different day, although alignment between schemas and the SdrRegistry is always a nicety!

[spiffmon]

However, it could still be nice to have such that we could separate out Physics based schemas from material or light related schemas, even ways for studios to categorise their own schemas from the defaults to make it clearer what is custom.

Ah, yeah ; I think I filed an internal sug awhile back that the UsdSchemaRegistry needs to retain and return to you the domain of each schema. Other things you mentioned are interesting also, though the domain seems like an uncontrovsial first step.