Skip to content

Support for BEVE aligned typed arrays#2400

Merged
stephenberry merged 6 commits intomainfrom
beve-aligned-typed-arrays
Mar 27, 2026
Merged

Support for BEVE aligned typed arrays#2400
stephenberry merged 6 commits intomainfrom
beve-aligned-typed-arrays

Conversation

@stephenberry
Copy link
Copy Markdown
Owner

@stephenberry stephenberry commented Mar 24, 2026

Aligned Typed Arrays for BEVE

Adds aligned typed arrays to BEVE, enabling zero-copy access to numeric array data via std::span<const T> pointing directly into the message buffer.

Wire format

A new typed array sub-type (category 3, sub-type 2, tag 0x5C):

ALIGNED_HEADER | NUMERIC_HEADER | SIZE | PADDING_LENGTH | PADDING | DATA

The PADDING_LENGTH byte stores the number of padding bytes so decoders can skip them without tracking absolute byte offsets. The padding aligns DATA to the element type's natural alignment.

Writing

Enable with the aligned_arrays option:

struct my_opts : glz::opts { bool aligned_arrays = true; };
glz::write<my_opts{glz::opts{.format = glz::BEVE}}>(data, buffer);

Single-byte types (uint8_t, int8_t) use standard typed arrays since alignment provides no benefit.

Zero-copy reading

Read into std::span<const T> to get a pointer directly into the buffer — no copies:

std::span<const double> span;
glz::read<glz::opts{.format = glz::BEVE}>(span, buffer);
// span.data() points into buffer, properly aligned

Works as struct members too — write with std::vector, read back with std::span<const T>:

struct sensor_view {
   std::string name;
   std::span<const double> readings; // zero-copy into buffer
};

The span specialization rejects non-aligned typed arrays with a syntax error (alignment can't be guaranteed). On big-endian systems, reading into std::span<const T> returns feature_not_supported since BEVE is little-endian.

Changes

  • header.hpptag::aligned_typed_array constant (0x5C)
  • opts.hppaligned_arrays option with check_aligned_arrays()
  • write.hpp — Writes aligned header, numeric header, size, padding length byte, padding, then data
  • read.hpp — Decodes aligned typed arrays; from<BEVE, std::span<const T>> specialization for zero-copy
  • skip.hpp — Skips aligned typed arrays using the stored padding length
  • beve_to_json.hpp — Converts aligned typed arrays to JSON
  • peek_header.hpp — Peeks at aligned typed array headers
  • size.hpp — Exact size calculation with offset threading for alignment padding
  • docs/binary.md — Documentation for aligned arrays and zero-copy spans

Validation

All decode paths validate padding_length < alignment, rejecting values that violate the spec.

Tests

27 new tests covering roundtrips (float32/64, int16/32, uint64), alignment verification, empty arrays, std::array, nested in objects, beve-to-json conversion, skip paths, peek header, zero-copy spans (double/float/int32/uint64), struct members with spans, rejection of non-aligned buffers and type mismatches, large arrays crossing compressed int boundaries, allow_conversions, variants containing aligned arrays, and map values.

@stephenberry stephenberry merged commit 6746d58 into main Mar 27, 2026
48 of 49 checks passed
@stephenberry stephenberry deleted the beve-aligned-typed-arrays branch March 27, 2026 16:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant