[Variant] Nested builder rollback is broken

[scovich]

Describe the bug

Rolling back variant builder insertions produces invalid variant bytes, e.g.

cd ~/arrow-rs/parquet-variant && cargo test --lib -- test_append_xx
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.12s
     Running unittests src/lib.rs (/Users/ryan.johnson/arrow-rs/target/debug/deps/parquet_variant-584b10a00525f9e0)

running 7 tests
test builder::tests::test_append_xx_list_list ... ok
test builder::tests::test_append_xx_list_object ... FAILED
test builder::tests::test_append_xx_object_object ... FAILED
test builder::tests::test_append_xx_object_list ... FAILED
test builder::tests::test_append_xx_list_object_list ... FAILED
test builder::tests::test_append_xx_object_list_object ... FAILED
test builder::tests::test_append_xx_list_object_list_object ... FAILED

failures:

---- builder::tests::test_append_xx_list_object stdout ----

thread 'builder::tests::test_append_xx_list_object' panicked at parquet-variant/src/builder.rs:3170:54:
called `Result::unwrap()` on an `Err` value: InvalidArgumentError("Tried to extract byte(s) 1..2 from 1-byte buffer")

---- builder::tests::test_append_xx_object_object stdout ----

thread 'builder::tests::test_append_xx_object_object' panicked at parquet-variant/src/builder.rs:490:26:
index out of bounds: the len is 0 but the index is 5

---- builder::tests::test_append_xx_object_list stdout ----

thread 'builder::tests::test_append_xx_object_list' panicked at parquet-variant/src/builder.rs:490:26:
index out of bounds: the len is 0 but the index is 1
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- builder::tests::test_append_xx_list_object_list stdout ----

thread 'builder::tests::test_append_xx_list_object_list' panicked at parquet-variant/src/builder.rs:490:26:
index out of bounds: the len is 0 but the index is 8

---- builder::tests::test_append_xx_object_list_object stdout ----

thread 'builder::tests::test_append_xx_object_list_object' panicked at parquet-variant/src/builder.rs:490:26:
index out of bounds: the len is 6 but the index is 7

---- builder::tests::test_append_xx_list_object_list_object stdout ----

thread 'builder::tests::test_append_xx_list_object_list_object' panicked at parquet-variant/src/builder.rs:490:26:
index out of bounds: the len is 11 but the index is 13


failures:
    builder::tests::test_append_xx_list_object
    builder::tests::test_append_xx_list_object_list
    builder::tests::test_append_xx_list_object_list_object
    builder::tests::test_append_xx_object_list
    builder::tests::test_append_xx_object_list_object
    builder::tests::test_append_xx_object_object

test result: FAILED. 1 passed; 6 failed; 0 ignored; 0 measured; 120 filtered out; finished in 0.00s

To Reproduce

#8135

Expected behavior

The tests should successfully build variant values even if some of the insertions are rolled back.

[scovich]

I was trying to explore whether ValueBuffer::new_object and ValueBuffer::new_list were actually correct (they seem to use the wrong parent state), but when I built these unit tests to explore that question I hit these errors instead.

[scovich]

Wow, an AI assistant actually found the bug by code inspection:

me> This test is broken, can you find the bug in the variant builder code that causes it?

llm> I’m going to update the rollback logic to correctly track and restore the number of field names, fixing the bug where dropping an unfinished builder truncates the dictionary using the metadata buffer length instead of the field-name count:

diff --git a/parquet-variant/src/builder.rs b/parquet-variant/src/builder.rs
index 2d82dc38a..352758560 100644
--- a/parquet-variant/src/builder.rs
+++ b/parquet-variant/src/builder.rs
@@ -689,5 +689,5 @@ impl ParentState<'_> {
             | ParentState::List {
                 metadata_builder, ..
-            } => metadata_builder.metadata_buffer.len(),
+            } => metadata_builder.field_names.len(),
         }

[alamb]

label_issue.py automatically added labels {'parquet-variant'} from #8140