Croissant validation issues

[pdurbin]

@amberleahey put on my radar a Croissant validation tool I didn't know about: https://huggingface.co/spaces/JoaquinVanschoren/croissant-checker

It looks like it was added in mlcommons/croissant#826 and people who are submitting to NeurIPS are asked (at https://neurips.cc/Conferences/2025/DataHostingGuidelines ) to use that "checker" tool to verify their Croissant file before submitting. I'm assuming the production version of the tool can be cloned from https://huggingface.co/spaces/JoaquinVanschoren/croissant-checker?clone=true

We should spend some time with this checker tool and see what it says. It wasn't on my radar.

From a quick test I can confirm that I see errors reported from this dataset, as reported by Amber: https://borealisdata.ca/dataset.xhtml?persistentId=doi%3A10.5683%2FSP3%2F3N6JVZ&version=1.0

Further, my own dataset doesn't validate with this tool either: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/TJCLKP&version=4.0

When developing the Croissant exporter ( https://github.com/gdcc/exporter-croissant and docs added in #10533), I used the only validation tool I was aware of at the time. Docs are here: https://github.com/mlcommons/croissant/tree/v1.0.17/python/mlcroissant#verifyload-a-croissant-dataset

When I run the "validate" command on my dataset, it looks fine:

mlcroissant validate --jsonld "https://dataverse.harvard.edu/api/datasets/export?exporter=croissant&persistentId=doi:10.7910/DVN/TJCLKP"

I0501 15:52:55.617974 8719305856 validate.py:53] Done.

When I run it against the Borealis dataset above, it shows warnings (but no errors):

mlcroissant validate --jsonld "https://borealisdata.ca/api/datasets/export?exporter=croissant&persistentId=doi%3A10.5683/SP3/3N6JVZ"

W0501 15:54:11.002909 8719305856 datasets.py:41] Found the following 3 warning(s) during the validation:
  -  `content_size` has cardinality `ONE`, but got a list.
  -  `content_url` has cardinality `ONE`, but got a list.
  -  `md5` has cardinality `ONE`, but got a list.
I0501 15:54:11.002985 8719305856 validate.py:53] Done.

I took a quick look at the code but I'm not sure why those warnings are happening. I can also reproduce them with a Kaggle dataset I found at mlcommons/croissant#837 (heads up @goeffthomas):

`mlcroissant validate --jsonld https://www.kaggle.com/datasets/heptapod/titanic/croissant/download

W0501 15:57:03.731079 8719305856 rdf.py:80] WARNING: The JSON-LD `@context` is not standard. Refer to the official @context (e.g., from the example datasets in https://github.com/mlcommons/croissant/tree/main/datasets/1.0). The different keys are: {'isLiveDataset', 'rai', 'examples'}
W0501 15:57:03.765449 8719305856 datasets.py:41] Found the following 2 warning(s) during the validation:
  -  [Metadata(Titanic)] Property "http://mlcommons.org/croissant/citeAs" is recommended, but does not exist.
  -  `source` has cardinality `ONE`, but got a list.
I0501 15:57:03.765532 8719305856 validate.py:53] Done.

While I think we should investigate the errors in from checker and the warnings from the validator, we should probably focus a bit more on the checker since NeurIPS is coming up and Harvard Dataverse is listed first in the submission guidelines at https://neurips.cc/Conferences/2025/DataHostingGuidelines (we worked with @mrisdal on this and because of this we working on support for exporting Croissant and other formats while the dataset is still in draft in #11398).

Finally, Amber alerted me to a git repo at https://github.com/kenlhlui/dv-croissant-testing where @kenlhlui has done some analysis of native JSON vs Croissant exports and mismatches between both @id values and contentUrl values. I haven't absorbed this yet but it may provide clues about the problems with the checker.

[goeffthomas]

Looking at the Croissant and the dataset (for the titanic one), I think the issue for this one likely stems from the fact that there are many columns with the same name: zero

Unfortunately, users can upload whatever they want on Kaggle, which includes some poorly formed data :(

For your example, I suspect the issue is somewhat related. Looking in your Croissant, there are multiple distributions with the same name of 40_MPa_Trial1.csv. Granted, the @id is different for them, so they shouldn't collide. But I think name used to be used as an identifier, so perhaps there's a bug in the mlcroissant validator. Take this with a grain of salt though, Pierre from the TFDS team would probably be best equipped to isolate the actual cause.

[pdurbin]

@goeffthomas thanks, much appreciated!

Via https://help.hmdc.harvard.edu/Ticket/Display.html?id=381301#txn-8710865 I learned of another dataset at https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/LBMXJY shows validations errors from https://huggingface.co/spaces/JoaquinVanschoren/croissant-checker

Here's the JSON I'm testing with from that dataset: lbmxjy.json

To back up, I'm going to start with the validate.sh script in our repo as of our latest release: https://github.com/gdcc/exporter-croissant/blob/croissant-0.1.4/validate.sh

This is what I'm running:

mlcroissant validate --jsonld lbmxjy.json

I get no errors and the expected warning about versions:

W0515 14:34:56.965307 8550172736 datasets.py:32] Found the following 1 warning(s) during the validation:
  -  [Metadata(QUT-DV25)] WarningException("Version doesn't follow MAJOR.MINOR.PATCH: 3.0. For more information refer to: https://semver.org/spec/v2.0.0.html")
I0515 14:34:56.965380 8550172736 validate.py:53] Done.

However, the validation script is using version 1.0.5 of mlcroissant.

Updating to 1.0.17 removes the warning about the version and shows no errors at all:

I0515 14:40:43.248532 8550172736 validate.py:53] Done.

I got https://huggingface.co/spaces/JoaquinVanschoren/croissant-checker running locally and was able to reproduce errors with the lbmxjy dataset.

I tried with the first dataset under https://github.com/mlcommons/croissant/tree/v1.0.17?tab=readme-ov-file#trying-it-out which is https://raw.githubusercontent.com/mlcommons/croissant/main/datasets/1.0/gpt-3/metadata.json and no errors were shown.

Another dataset to test with is https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/W7OUZM via https://help.hmdc.harvard.edu/Ticket/Display.html?id=382005

I used pyDataverse to generate W7OUZM-pyDataverse.json and then hacked on it until it passed. For contentSize I put quotes around the int to make it a String. For stuff like "column": "v36657055" (don't quote me on the exact number) I changed it to something like "column": "Age" to get rid of the error below. Also, I greatly reduced the number of columns and files to make troubleshooting easier.

Here's the "before" error (thrown here):

AssertionError: Column "v36656288" does not exist. Inspect the ancestors of the field Field(uuid="_:Nf6c4979ace0f496186ba9dee01082d15") to understand why. Possible fields: Index([ 'Age',

And now (again, with many fewer columns and files), it validates:

WARNING:absl:Found the following 3 warning(s) during the validation:
  -  [Metadata(DermaCon_IN_Dataset_Release_v1.0)] Property "https://schema.org/datePublished" is recommended, but does not exist.
  -  [Metadata(DermaCon_IN_Dataset_Release_v1.0)] WarningException("Version doesn't follow MAJOR.MINOR.PATCH: 6.6 build iqss-3. For more information refer to: https://semver.org/spec/v2.0.0.html")
  -  `extract` has cardinality `ONE`, but got a list
WARNING:absl:Found the following 3 warning(s) during the validation:
  -  [Metadata(DermaCon_IN_Dataset_Release_v1.0)] Property "https://schema.org/datePublished" is recommended, but does not exist.
  -  [Metadata(DermaCon_IN_Dataset_Release_v1.0)] WarningException("Version doesn't follow MAJOR.MINOR.PATCH: 6.6 build iqss-3. For more information refer to: https://semver.org/spec/v2.0.0.html")
  -  `extract` has cardinality `ONE`, but got a list
# CROISSANT VALIDATION REPORT
================================================================================
## VALIDATION RESULTS
--------------------------------------------------------------------------------
Starting validation for file: W7OUZM-pyDataverse.json
### JSON Validation
✓
Record set '_:N953623a6fce04161a09936425a17c4e1' passed validation.
Record set '_:N877c8eaa95bd4cf8bb836ebadd05ce50' passed validation.
### Croissant Validation
✓
Record set '_:N953623a6fce04161a09936425a17c4e1' passed validation.
Record set '_:N877c8eaa95bd4cf8bb836ebadd05ce50' passed validation.
### Records Generation
✓
Record set '_:N953623a6fce04161a09936425a17c4e1' passed validation.
Record set '_:N877c8eaa95bd4cf8bb836ebadd05ce50' passed validation.

[pdurbin]

I created these PRs:

• include "extract column" object per field per NeurIPS checker gdcc/exporter-croissant#16
• match on storageIdentifier (order in array can change) gdcc/exporter-croissant#17

[pdurbin]

I built croissant-0.1.5-SNAPSHOT.jar locally as of those two PRs above and put it on demo.

Then I increased the CSV ingest limit from 0.6 to 6 MB, uploaded QUT-DV25_FiletopTracesDataset.csv from https://dataverse.harvard.edu/file.xhtml?fileId=11210361&version=3.0 , published, and pasted https://demo.dataverse.org/api/datasets/export?exporter=croissant&persistentId=doi%3A10.70122/FK2/UDRXLG into the checker at https://huggingface.co/spaces/JoaquinVanschoren/croissant-checker

The results look good: