KHR_gaussian_splatting_compression_spz_2

[weegeekps]

This is a draft extension proposal for using Niantic Spatial's SPZ compression library for compression 3D Gaussian splatting data using the base KHR_gaussian_splatting extension with glTF. This extension targets version 2 of SPZ.

As we get closer to done with the base extension for 3D Gaussian splatting, KHR_gaussian_splatting, it has made more and more sense to break out the KHR_gaussian_splatting_compression_spz_2 extension for SPZ compression of 3DGS data. This allows them each to independently go through the ratification process and neither will hold up the other.

For previous history and conversations regarding the development of this extension, please look at the PR for KHR_gaussian_splatting before this linked comment.

[javagl]

The table formatting is broken

[javagl]

Not sure whether it's on the radar (I know that there currently is some turmoils around this topic...), but the wording and description around COLOR_0 may have to be updated.

This extension casually mentions COLOR_0, "as-if" it was a required attribute in the base extension. But in the base extension, it is described as an optional fallback. (I think that this means that there will be "fewer conversions" necessary, so in some way, it may actually simplify things, but the details should be reviewed carefully)

[weegeekps]

@javagl I have updates in progress that bring this in line with the current base spec including the changes around color spaces. I plan on pushing those changes as soon as that work is fully settled and we have consensus.

[larsdal-hexagon]

Hi, I've been looking into how KHR_gaussian_splatting_compression_spz_2 is implemented in Cesium, and there the spz data is stored in LUF coordinates, rather than the standard RUB, with no conversion on load.
Is the intention here for data in the spz files to be stored in the standard RUB coordinates and converted to/from LUF on read/write, or for the spz data to be stored in LUF coordinates like in the current Cesium implementation?
My interpretation of the spec would indicate the former.

[javagl]

I think that this is one point that (has been brought up occasionally and) may have to be clarified. The comment at CesiumGS/cesium#12682 (comment) refers to an older state of the implementation, and it seems like the specific rotations that this comment is linking to are no longer applied. But it might be that some rotations are now done elsewhere.

(An aside: The rotation was originally done by the SPZ-loader library. And this is using the spz convertCoordinates function under the hood. And I'm about 95% sure that this implementation is wrong. But I haven't confirmed that yet...)

[javagl]

Coming back to the COLOR_0 handling mentioned above:

According to the current base extension specification, the COLOR_0 attribute is only a "fallback". It can not replace the SH_DEGREE_0_COEF_0+OPACITY attributes from the base extension. One task here is to align this requirement with the SPZ compression extension. The SPZ library/format itself always stores separate color and alpha, so that could be relatively easy to align with the glTF world. It mainly means that these values have to be converted according to their domain (e.g. the color into the SH coefficients).

[weegeekps]

Just so folks are aware: I have a lot of updates in queue for this extension, but I am waiting to see the final editorial draft of the base extension.

It mainly means that these values have to be converted according to their domain (e.g. the color into the SH coefficients).

Most all of the SPZ loaders immediately apply 0.282 and the 0.5 offset to the colors result and return an RGBA with opacity encoded in. As a clarifying point, I'm saying "most", but the truth is all of the loaders I've seen do this. There may be a loader out there that doesn't that I haven't seen, but the popular ones including the Babylon.js loader, spz-loader library, and others all perform this calculation immediately.

As far as I can tell, this is very much in the "spirit" of SPZ. I don't believe we should require language to the SPZ extension to require implementations to change. I believe this is also why it's called "colors" and not DC or SH0. (DC with spherical harmonics does not mean diffuse component although that is a common point of confusion.) This is one of the reasons we put this statement in the spec:

Extensions extending this extension may define alternative lighting methods, have specific requirements for handling compression, or define different spherical harmonics handling.

[javagl]

Most all of the SPZ loaders immediately apply 0.282

And I'm not even sure that it is "correct" to do this, because it does not anticipate the "colorScale" in SPZ. See the "Colors" point at the bottom of the first comment of nianticlabs/spz#42

But the main reason for "reviving" this PR by adding that comment:

We talked about some of that in the 3D Formats call. And one point that we talked about was that the COLOR_0 can not replace the SH_DEGREE_0_COEF_0+OPACITY. That wasn't self-evident for me. I thought that it could be reasonable to see that COLOR_0 as an ~"alternative way" of encoding that data. But the consensus was that content creators should not mix the "real" attributes and that "fallback" attribute.

That reduces ambiguities and simplifies the loaders that do support the extension. It has the drawback that any glTF asset that wants to offer the fallback path has to store this as additional (!) data. If it was non-exclusive, people could opt in for always representing SH0+OPACITY as COLOR_0. But I don't have a strong opinion.

(And I know one "creator" of glTF-SPZ splat file that does always write the COLOR_0 and omit the opacity. You probably know where this could be tracked)

[weegeekps]

And I'm not even sure that it is "correct" to do this, because it does not anticipate the "colorScale" in SPZ.

Under the covers, SPZ packs these as RGB for stability. While doing so and to support values greater than 1.0, they use a scale of 0.15. When unpacking, it reverses this calculation leaving you with the DC components with the intent of you applying the expected scale of 0.282 in your codebase.

And one point that we talked about was that the COLOR_0 can not replace the SH_DEGREE_0_COEF_0+OPACITY.

On the surface I think this is a reasonable take, but when I think about it more it feels like we've pushed past the fine line we've been treading. There is a separation of concerns matter here that I think we have to account for; we've made it upfront in the base extension that other extensions, such as compression extensions, may have slightly different ways of providing the data. This rule violates that policy and to me it seems like we're telling compression extensions that if you don't fit this mold correctly, you can't build off of this base extension. That seems very wrong to me.

There are some maths that we must specify for consistent rendering of 3DGS from glTF. The kernel property specifies those. We should not mandate that a particular compression format cannot provide the variables for those maths in a different manner than the base extension would. At that point we're mandating implementation details that should be left to the implementers.

[gugu23456789]

这是一个基于 glTF 的基础 KHR_gaussian_splatting 扩展，使用 Niantic Spatial 的 SPZ 压缩库进行压缩 3D 高斯散布数据的草稿扩展提案。该扩展面向SPZ第2版。

随着我们接近完成3D高斯喷溅的基础扩展，KHR_gaussian_splatting，拆分用于3DGS数据的SPZ压缩扩展越来越有意义。这使得双方各自独立完成批准程序，双方都不会拖延对方。KHR_gaussian_splatting_compression_spz_2

关于此扩展开发的历史和讨论，请查看本链接评论前的PR KHR_gaussian_splatting。

Hi @weegeekps and Khronos glTF community,

This is a fascinating and much-needed proposal. We’ve been closely following the development of the KHR_gaussian_splatting and KHR_gaussian_splatting_compression_spz_2 extensions.

As the draft approaches a critical stage, we wanted to share that a complete, production-ready, and community-vetted open-source toolchain implementing this specification already exists and is actively used.

We are the maintainers of the SPZ Open-Source Ecosystem (spz-ecosystem), and we have built two core, interoperable projects that form a full pipeline for SPZ v2:

spz_gatekeeper - The Format Compliance Engine

• L2-Only Legal Validator: A pure C++17/WASM tool that performs rigorous, non-destructive validation of SPZ headers, flags, and TLV trailer extensions. It defines a clear contract for what constitutes a compliant SPZ derivative.
• Governance & Registry: Features a split ExtensionSpecRegistry / ExtensionValidatorRegistry system, providing a framework for registering, documenting, and validating vendor extensions (starting with 0xADBE0002).
• Production-Grade: Includes a fully functional WebAssembly UI for in-browser validation, comprehensive CI/CD, and a dual-audit model (browser_lightweight_wasm_audit + local_cli_spz_artifact_audit).
• Repository: spz_gatekeeper

spz2glb - The High-Performance Conversion Hub

• Lossless, Spec-Compliant Packaging: A converter that embeds SPZ streams byte-for-byte into GLB containers using the KHR_gaussian_splatting_compression_spz_2 extension. It features a built-in 3-layer verification suite (structure, losslessness, decode-consistency).
• Web-Optimized & Performant: Offers a high-performance WebAssembly build, enabling sub-second, client-side conversion in the browser with no data upload. Our benchmarks show ~506ms for a 25MB SPZ file.
• Ecosystem Integration: Our output is already being used experimentally by community projects like Real3DViewer, validating the end-to-end workflow from SPZ to rendered content.
• Repository: spz2glb

Why This Matters for the Spec:

• Provides a Production Reference: We offer a neutral, community-driven implementation that can serve as a concrete, testable reference for spec discussions and compatibility testing.
• Solves Real Tooling Gaps: Our tools address the immediate needs for validation, conversion, and developer onboarding in the SPZ format.
• Independent & Safe Foundation: The projects include clear legal and political separation statements, ensuring they are a clean, vendor-neutral foundation for the ecosystem.

We are fully aligned with the goal of a robust, interoperable standard. If helpful, we are ready to:

• Run compatibility tests using our toolchain against various draft iterations.
• Provide detailed feedback from our implementation experience.
• Collaborate on fostering a broader open-source ecosystem around this standard.

The repositories are under active development (currently at v2.0.1), and we welcome any feedback or collaboration.

Best regards,
The spz-ecosystem maintainers.