Mismatch between released ASE metadata split and public ASE dataset scene count

[seanmu]

I downloaded the official training metadata from the Pre-computed Training Metadata section in the data processing README, and I am trying to use it to finetune the released pretrained weights.

However, I noticed a mismatch for the ASE dataset:

the ASE dataset I downloaded contains 100,000 scenes
the released metadata contains:
98,696 scenes in ase_scene_list_train.npy
5,194 scenes in ase_scene_list_val.npy

This gives a total of 103,890 scenes. I also checked that the train and val scene lists do not overlap, so the combined count is indeed 103,890.

Could you clarify why these numbers do not match?
Specifically, I would like to understand whether:

the released metadata was generated from a different version of ASE,
it corresponds to an internal ASE split rather than the public 100K release,
or the metadata was built from a processed subset/superset whose scene definition differs from the public ASE dataset.

I want to make sure I am using the correct ASE data and metadata setup for finetuning the official pretrained weights.

Thanks.

[cr333]

This mismatch is expected and has been discussed in prior issues (#63, #153). MapAnything was trained using ASE-V2, an internal Meta-hosted version of the ASE dataset, which differs from the publicly available ASE-V1 (the 100K-scene release).

Key points from previous responses:

• From ASE dataset preprocess issue #63: "We use an internal version of the ASE dataset and its format/data might not necessarily align with the official public ASE."
• From WAI conversion about the ase dataset #153: "ASE-V2 is currently only an internal-only dataset [...] We are unsure if there are any open-source plans."
• The conversion script itself notes: "We use an internal version of the ASE dataset (ASE-V2) and its format/data might not necessarily align with the official public ASE-V1."

So to answer your specific questions:

1. Yes, the released metadata was generated from a different version of ASE (the internal V2).
2. The 103,890 scenes in the metadata correspond to ASE-V2's scene set, which is a superset of different from the public 100K release (and may also define scenes differently).
3. It is not possible to directly use the released metadata with the public ASE-V1 dataset for finetuning, since the scene lists won't align. You would likely need to regenerate metadata for the subset of scenes that overlap, or filter the metadata down to only scenes present in your local ASE-V1 download.

Unfortunately we don't currently have the bandwidth to bridge this gap, and ASE-V2 has no public release plans at this time.

[seanmu]

@cr333 Thanks for the clarification. I have one follow-up question about point 3.

If ASE-V1 and ASE-V2 are not fully aligned, and ASE-V2 is not accessible, how can we reliably identify the overlapping scenes between the two versions?

The public ASE-V1 release has 100,000 scenes with IDs from 0 to 99,999, while the released ASE-V2 metadata appears to contain 103,890 scenes with IDs ranging roughly from 0 to 199,749. This makes me concerned that scene IDs may not be directly comparable across versions. If so, filtering the released metadata or regenerating metadata based on assumed overlaps could produce train/val splits that no longer match the original pretraining setup.

To avoid possible leakage or bias from a mismatched ASE validation split, would it be reasonable to use all ASE-V1 scenes only for training during finetuning, and simply not define an ASE validation split?

Thank you again for your help.

[cr333]

Sorry for the confusion about our internal ASE V2 being a superset of the public ASE V1 dataset. As far as I know (but this might be wrong), these are actually independently generated datasets that don't seem to share any scenes.

If you want to use ASE, I would encourage you to write a conversion script, perhaps based on the existing ASE conversion script, and to open a pull request for it, so we can review it for inclusion in the MapAnything codebase.

I'm afraid I'm not familiar with the training/test splits on the public ASE dataset.