question regarding color value range enforcement

[Sam-Izdat]

Hi. Quite excited about the work you're doing.

Reading over the white paper, I noticed that the acceptable range for color values is set to be [0,1]. Does this imply that nonconforming color spaces should be excluded?

For example, ACES 2065-1, CIE LAB/LUV and OKLAB can have negative color values. Now, perceptual color spaces might be a questionable choice here, but it seems conceivable that someone would want to define their materials with AP0 primaries.

[portsmouth]

This is a non-trivial question.

I would like to think of the color values in the model as defining the properties of the physical material. If the colors were specified in a fully spectral manner, I think they would correspond essentially to "spectral albedos", i.e. physically what fraction of each particular wavelength is absorbed or transmitted on passing through some layer, say. (Except for emission_color which would be the spectrum of emitted light).

Those albedos should physically all be in the range $[0,1]$ (i.e. it would be meaningless to absorb a negative amount of energy, or more than 100%).

However since we use RGB values, the spectral interpretation is ambiguous. The albedo per color channel represents some integral over (unknown) underlying physical albedos, with weights depending on the color space. I would expect that normally the channels should be in $[0,1]$, in standard color spaces and under some reasonable assumptions about the possible underlying spectral albedo curve. But I'm not sure, in the cases of the "non-conforming" color spaces you mentioned, perhaps it could make sense to allow that the RGB albedo components can be negative, or exceed 1? It would be strange to work with negative color channels in a renderer, but perhaps could be done consistently with care (obviously physical radiances must be positive, but RGB renderers don't work directly with physical radiance).

Generalizing this properly might also require re-expressing some of the math in the model (not to mention, in other similar models). For example, in the standard Schlick model for metallic Fresnel:

it is assumed that $\mathbf{F_0}=(1, 1, 1)$ corresponds to perfect Fresnel reflectivity. But in these exotic color spaces, is that the case?

[KelSolaar]

Hello,

Thanks to @portsmouth for pointing me to this thread!

This is a great question and I will try to address it by reinstating what I think the prime objective of physically-based rendering is: Simulating the imaging of the electro-magnetic spectrum radiation incident to an observer in a way that the simulation would be indistinguishable from reality to a similar observer.

I took great care not using the words human and colour (or even light) anywhere in the previous sentence because, often, and especially in the VFX industry, the observer is a motion picture or stills camera. A camera is typically not colorimetric, i.e. its sensitivities are not a linear combination of the human cone responses. Put another way, it does not capture incident light the same way than us, and we need to process the images it captures to make them colorimetric. We might also be required to simulate an infrared camera which images a portion of the electro-magnetic spectrum that is invisible to us.

You might be wondering where I am going with that? It is simple: We should strive for the modelling of the scene in the physically-based rendering process to be agnostic of the observer. To be very specific, defining the physical properties of the objects taking part in the simulation should be independent of the observer. The act of "observing" an object does not change its characteristics in the real word, why should there be correlation in our simulations?

With that in mind, describing colour information in a renderer using a perceptually uniform space correlates the simulation to the observer, thus, changing the space will change the simulation. This is true with RGB colourspaces and especially noticeable with indirect illumination. We have shown that different spaces produce different images. See https://www.colour-science.org/anders-langlands/, https://www.colour-science.org/posts/about-rendering-engines-colourspace-agnosticism/ and https://computergraphics.stackexchange.com/questions/8152/for-shader-math-why-should-linear-rgb-keep-the-gamut-of-srgb/8163#8163 for more information.

The act of using photometric quantities is problematic, especially when describing surface characteristics: They are weighted by our very own human sensitivities.

The ideal renderer would only use radiometric quantities as input, e.g. spectral reflectance, transmittance or absorptance, spectral irradiance, etc.., unfortunately, we have created a world of problems by adopting RGB as the space for most renderers to operate in. There are good reasons for that, e.g. "apparent" simplicity and cost effectiveness. Whilst it is correct that a RGB renderer is typically faster and has readily available material to feed on, the consequential complexity in modelling the real world is problematic.

When we take a photograph and use it as a texture in a renderer, the illuminant is "baked" into it: In a simplified way, it is the result of the integration by the camera sensitivities of the multiplication of the environment irradiance with the reflectance, transmittance, and absorptance of the objects in the scene. The mechanics are similar for us:

Let me continue a bit as I'm also hoping to see more definitions and terminology clarification in the whitepaper: What is albedo? The CIE does not have a definition for it and I could not find any standard one. A tentative one might be as follows:

Albedo refers to the measure of the reflectivity of a surface or object. It is the ratio of the reflected light from an object to the incident light upon it. Albedo is typically expressed as a percentage or a decimal value between 0 and 1, where 0 represents complete absorption of light (no reflection) and 1 represents complete reflection (no absorption). A high albedo means that an object reflects a large portion of the incident light, whereas a low albedo indicates that most of the light is absorbed.

Why are we then surfacing our virtual objects with photographs that have illumination baked in? What illuminant should we be using when creating new textures in Mari or Substance Painter? D60, D65? Why not using the equal energy illuminant E? How should a blackbody, e.g. fire, converted to RGB for rendering, what does this mean to the physical camera white balancing?

It should hopefully be clear why I wrote "apparent" simplicity: We introduce observer correlation and coupling everywhere in the virtual scene authoring process when working with RGB values. Everything becomes extremely complicated when you start deeply thinking about it. Fortunately, all those problems disappear under the elegance of spectral rendering :)

To finally reply to your question, and to echo what Jamie said, I would limit albedo to [0, 1] because it is by "definition" a measure of reflectivity.

Cheers,

Thomas

[Sam-Izdat]

Thank you both for the detailed responses.

So, if I'm understanding correctly, the design philosophy here is to limit parameters to "computation-ready" values, immediately suitable for rendering, without any additional pre-processing steps.

I'm working on an interchange file format, which will probably not see much use (if any) outside its associated projects, but I wanted make sure that I'm reading the white paper correctly. If this is the goal - I suppose I might make a version for "OpenPBR-ish" parameters, where color spaces can be arbitrary (and converted for the renderer as needed), and then a strictly-conforming one, where all color values must be in [0, 1] range, in a scene-linear color space.

[KelSolaar]

the design philosophy here is to limit parameters to "computation-ready" values, immediately suitable for rendering, without any additional pre-processing steps.

Yes, this is ideal because the renderer can focus only on rendering and does not have to deal with pre-transformations of all sort to bring the image resources into the RGB working space.

It is worth noting that your RGB working space is primarily driven by the RGB colourspace the image resources are encoded with. Some conversions require knowledge of the RGB working space, e.g. blackbody irradiance to RGB, but it is mostly an implicit condition.

Cheers,

Thomas

[Sam-Izdat]

That sounds reasonable. I suppose my confusion stems from the fact that a scene may include many different materials, potentially using different color spaces. I assumed that, at some point, this would necessitate choosing a rendering space anyway, and doing the necessary transformations.

e.g. a scene-linear Rec. 2020 and ACEScg are both fine according to the spec, but you can't exactly use them together side-by-side as-is.

[KelSolaar]

that a scene may include many different materials, potentially using different color spaces

If the renderer exposes a way to conform the textures to the same working space, e.g. OpenColorIO or internal color transformation engine, it should be fine. As you point out, you would want all the input to be in the same working space otherwise you would be mixing metrics, akin to mix meters with inches.

Worth noting that BT.2020 and ACEScg are for practical purposes the same space and a human observer has a hard time making a difference between the two, so it would not be catastrophic, in this particular case to mix them! :)

[portsmouth]

It seems like a good idea to try to formalize the reasoning a bit to clarify what it means exactly to say that the reflectance (i.e. albedo) of a material is described by some (R, G, B) triple.

There may be different possible reasonable interpretations, so I though it was worth writing down how I interpret the discussion above from a computational/mathematical standpoint, at least to verify that I have the right mental model. (I found this exercise helpful anyway, though this is probably obvious to others). I found the paper "Physically Meaningful Rendering using Tristimulus Colours" by Meng et. al to be quite helpful in understand the relationship between physical albedos and RGB colors.

If we assume the illumination is uniform with luminance $Y$ (in nits) and has the spectral power distribution (SPD) of (say) the standard illuminant D65, we can express the incident spectral radiance as:

$$L_e(\lambda) = \left(\frac{Y}{Y_\mathrm{D65}}\right) \; S_\mathrm{D65}(\lambda)$$

where

$$Y_\mathrm{D65} = \int \bar{y}(\lambda) \; S_\mathrm{D65}(\lambda) \; \mathrm{d}\lambda \ .$$

Say for simplicity that the surface has a Lambertian BRDF with spectral reflectance (albedo) $\rho(\lambda)$, where physically, $0 \le \rho(\lambda) \le 1$ for every wavelength $\lambda$, in order to conserve energy.

Given that, we can write down the rendering equation at the surface point, for the reflected radiance $L_o$:

$$L_o(\lambda) = \frac{1}{\pi} \int \rho(\lambda) \, L_e(\lambda) \; \mathrm{d}\omega^\perp = \rho(\lambda) \, L_e(\lambda) \ .$$

Thus the tristimulus values of the reflected light (as observed by the measuring/recording apparatus, i.e. camera or eye) are given by

$$X_o = \left(\frac{Y}{Y_\mathrm{D65}}\right) \; \int \bar{x}(\lambda) \; \rho(\lambda) \; S_\mathrm{D65}(\lambda) \; \mathrm{d}\lambda$$ $$Y_o = \left(\frac{Y}{Y_\mathrm{D65}}\right) \; \int \bar{y}(\lambda) \; \rho(\lambda) \; S_\mathrm{D65}(\lambda) \; \mathrm{d}\lambda$$ $$Z_o = \left(\frac{Y}{Y_\mathrm{D65}}\right) \; \int \bar{z}(\lambda) \; \rho(\lambda) \; S_\mathrm{D65}(\lambda) \; \mathrm{d}\lambda$$

using the CIE color matching functions $(\bar{x}(\lambda), \bar{y}(\lambda), \bar{z}(\lambda))$.

Then we can interpret the albedo colors in our model (and in input textures) as corresponding to the resulting measured tristimulus $(X_0, Y_0, Z_0)$ when illuminated by the D65 illuminant normalized to luminance $Y = 1$ nit. The equations above then guarantee that $Y_o \le 1$, since $\rho(\lambda) \le 1$. Presumably this constraint corresponds to the color theory version of the BRDF energy conservation requirement.

This tristimulus can be converted to/from the RGB values $(\rho_r, \rho_g, \rho_b)$ of reflectances in the model via the usual transformation depending on the specified color space (assumed ACEScg by default, and presumably generally always linear / "scene referred"). Note though, it is not guaranteed (I think) that these RGB albedos are necessarily in the range $[0,1]$.

Given the RGB reflectance values in the model defined in that fashion, we then have a well-defined transformation to RGB albedo in the renderer, leading to the usual operations in RGB, e.g. the calculation of the R channel of the reflected light:

$$R = \int \bar{r}(\lambda) \; \rho(\lambda) \; L_e(\lambda) \; \mathrm{d}\lambda = Y \; \rho_r \ .$$

RGB values in textures can be converted to the model color space as usual, although this strictly only makes sense if the standard illuminant of the color space matches the assumed SPD of all light sources.

As Meng notes, if the illuminant of the lights can vary, then it would make more sense to stipulate e.g. that the model albedo colors are defined assuming a flat SPD (standard illuminant E). But this only seems to make sense for a spectral renderer, which does "uplifting" from RGB albedos to spectral.

For a tristimulus renderer, it makes more sense to just assume that the same illuminant (e.g. D65) is used consistently, as @anderslanglands proposed in UsdLux. This is essentially baking the illumination into the definition of the albedo which is not ideal, but if doing tristimulus rendering there are limited options (and the renders are physically inaccurate as they do not account for the true colors generated by different metameric spectra, as discussed by Meng, but unless one is doing predictive rendering with measured spectra -- not the use case of OpenPBR most likely, this is probably not a practical problem).

For OpenPBR, I'm not sure if we need to elaborate on any of this, though perhaps to be complete we should include some kind of discussion. We do say that color values are enforced to be in $[0,1]$, which may technically be violated in certain color spaces, though (as pointed out in the original question by @Sam-Izdat) -- even if the underlying spectral albedos are physical in $[0,1]$.

[anderslanglands]

This is essentially baking the illumination into the definition of the albedo which is not ideal

the Illuminant is already baked into the definition of whatever RGB space you’re using to store your texture data in.

[portsmouth]

the Illuminant is already baked into the definition of whatever RGB space you’re using to store your texture data in.

Right yes, so given texture data in a given (linear) RGB space, the R value means (I assume)

$$R = \int \bar{r}(\lambda) \; \rho(\lambda) \; S(\lambda) \; \mathrm{d}\lambda$$

given the spectral albedo $\rho$ and (normalized) standard illuminant SPD $S$ (i.e. the color of the surface is operationally defined as what you see when illuminated with a known light..).

I meant this is problematic in the sense that once this is baked in the dependence on that particular SPD can't be disentangled. If the surface is assigned that RGB albedo, but illuminated with a different SPD, there is no way to compute the integral for that to get the reflected RGB. The only way to complete the integral is if you assume that the SPD of all lights in the scene is the same.

Which seems a workable, if artificial, assumption for an RGB renderer. That's all I meant by "not ideal". (For a spectral renderer you can do everything correctly/ideally though, in principle).

[KelSolaar]

It seems like a good idea to try to formalize the reasoning a bit to clarify what it means exactly to say that the reflectance (i.e. albedo) of a material is described by some (R, G, B) triple.

I never liked talking about reflectance in that case because it is integrated reflectance for a given observer and illuminant. I don't really like albedo either because there are multiple definitions for it and I haven't seen a standard one that can be pointed at, are we talking about spectral albedo, average albedo, etc... I suggested that OpenPBR clarifies the definition it uses.

When we talk about scene-referred rendering the RGB values are colour estimates of the scene and not its irradiance. The critical aspect is the linear relationship between the scene and the scene-referred image: We know that a doubling of scene exposure yields a doubling of image luminance. In that sense, and I understand it is mouthful, but maybe "referred-reflectance" is a better term? Or this could be given as an explanation example tying the logic that the scene is in the radiometric domain and the scene-referred image is necessarily in the photometric domain.

[anderslanglands]

Do you mean we're looping back around and actually "diffuse colour" was the best nomencalture after all? :P

[anderslanglands]

agreed that albedo is a horrible term for this. I'm fine with reflectivity though personally as the referred- or integrated- nature being implicit in this context is ok imo.

I do prefer reflectivity over reflectance as reflectance implies it's the output quantity rather than the factor, but anyway...

[portsmouth]

I think "color" works ... these are colors aren't they (RGB triples)? We just need to specify a color space (as we say the metadata should provide, otherwise default to ACEScg).

The only potentially non-obvious thing is how exactly to interpret the color as a reflectance. We may want to say explicitly that it's this relationship (if you agree that is right):

$$R = \int \bar{r}(\lambda) \; \rho(\lambda) \; S(\lambda) \; \mathrm{d}\lambda$$ $$G = \int \bar{g}(\lambda) \; \rho(\lambda) \; S(\lambda) \; \mathrm{d}\lambda$$ $$B = \int \bar{b}(\lambda) \; \rho(\lambda) \; S(\lambda) \; \mathrm{d}\lambda$$

and then some brief discussion about how this works in a traditional tristimulus renderer, versus a spectral renderer (along the lines of the discussion above).

What about this (the original question):

We do say that color values are enforced to be in $[0,1]$, which may technically be violated in certain color spaces, though (as pointed out in the original question by @Sam-Izdat) -- even if the underlying spectral albedos are physical in $[0,1]$.

That does seem to be the case actually -- i.e. it could be totally legit for a material to have a diffuse color of (-0.1, 0.5, 1.0) in some color space (ACES 2065-1 was mentioned) according to the definition above, and this be meaningful, though weird. If that's allowed, we should mention it in the spec, since right now we forbid that (color ranges are said to be $[0,1]$).

[KelSolaar]

The only potentially non-obvious thing is how exactly to interpret the color as a reflectance.

What about the others cases? Including all the .*_color for completeness:

• base_color : Integrated reflectance
• specular_color : ?
• subsurface_color : ?
• transmission_color : Integrated transmittance
• coat_color : ?
• fuzz_color : ?
• emission_color : Integrated irradiance

[anderslanglands]

ACES 2065-1 covers the entire spectral locus so you're not going to get negative RGB values there, but you could get negative values if your colours are stored in a wide gamut space (e.g. ACEScg) and your rendering space is narrower (e.g. sRGB). Negative reflectances don't make any sense, so you have to adapt the gamut somehow, and the easiest thing to do there is to clip to [0, 1).