Trying to get AutoCompaction to work.

[atzannes]

I am trying experiment with delta.autoOptimize.autoCompact in Python. I am able to create a table with this property set to true or auto, but when I go a head and append 100 or even 1000 small commits to the table and then I go look at what happened, I see that the number of active files in the DeltaTable equals the number of commits that I just performed. In other words, auto-compaction did not get triggered. In case this is relevant, I am also setting delta.appendOnly to True, delta.checkpointInterval to 10, delta.targetFileSize to about 256MB, disabling deletion vectors and enabling change data feed, and I'm also setting the retention-duration for logs and deleted (tombstoned) files.

Is auto-compaction supported and I'm doing something wrong, or is it a shim ? I've tried v 1.2.1 and 1.1.4.

I spent a good amount of time reading the issues (open and closed), the PRs, and the discussions here, but haven't found anything that could help me. By the way great direction you're taking this project with delta-kernel-rs and thank you for this amazing tool!

[ion-elgreco]

Auto compaction is not a supported feature as of now