chore: 4.5x reduction in number of golden files

[andygrove]

Which issue does this PR close?

Closes #.

Rationale for this change

What changes are included in this PR?

• only generate golden files for the native_datafusion and native_iceberg_compat scans, dropping auto
• only generate extended explain plan - drop simplified and regular explain plan

How are these changes tested?

[kazuyukitanimura]

So auto == native_iceberg_compat IIUC
Will this cause to slip some bugs in the future?

[andygrove]

So auto == native_iceberg_compat IIUC Will this cause to slip some bugs in the future?

Correct, auto is currently the same as native_iceberg_compat. The main change with this PR is that we no longer test with native_datafusion, so it is possible that we could introduce changes that mean we fall back to Spark for some operator when native_datafusion is enabled and not notice that.

However, the plan is for auto mode to eventually choose between native_datafusion and native_iceberg_compat, so we would catch any regressions once this happens.

Note that the worst case scenario here is not missing any kind of correctness issue. This test suite does not run any queries. It just checks that the plan is expected.

[comphead]

The main change with this PR is that we no longer test with native_datafusion

I'm prob not getting this. native_datafusion is supported, why not longer test? 🤔

[andygrove]

Alright, let's try a different approach. Let's generate golden files for the two scan implementations native_datafusion and native_iceberg_compat, and not for auto since that will always pick from these two.

[mbutrovich]

Maintaining two sets, one for each parquet scan, makes sense to me. Thanks @andygrove!

[andygrove]

Thanks for the good feedback @kazuyukitanimura @comphead @mbutrovich