glz::generic - ability to customize type and/or allocator

[meastp-nk]

I would like to be able to use PMR (polymorphic memory resources / allocators) with glz::generic, but unlike previous versions where I (think I) could achieve this with a glz::basic_json_t< ... >, I think this is no longer possible...?

I need to change the types for array_t and object_ t to something allocator aware, like std::pmr::vector and std::pmr::map (or, more likely, a pmr-enabled boost::unordered_flat_map).

I also need to add allocator-awareness to generic_json, I don't know the best way to achieve this. Perhaps inherit from generic_json in a custom type?

   // Generic json type.
   // Template parameter controls number storage mode for precise integer handling
   template <num_mode Mode = num_mode::f64>
   struct generic_json
   {
      virtual ~generic_json() {}

      using array_t = std::vector<generic_json<Mode>>;
      using object_t = std::map<std::string, generic_json<Mode>, std::less<>>;
      using null_t = std::nullptr_t;
   
      // ...

[stephenberry]

This is a great request, I'll look into adding a glz::pmr::generic

[stephenberry]

I would think that array_t and object_ t already work with pmr based types. Let me know if this isn't the case, because they are just general concepts. The generic request is a bit more complex and not currently supported, but I'm experimenting with a pmr solution here: #2280

[meastp-nk]

@stephenberry Wow, great!

As you propably already know, the std::variant in value_t needs to be handled specially for glz::pmr::generic because std::variant isn't allocator aware and does not (always) propagate the allocator correctly, if I'm reading the papers correctly: P3153 An allocator-aware optional type (github issue).

--

If you make glz::pmr::generic - which would be awesome! - I would like to change object_t to my preferred (more performant) type if possible as well. Is the following the right way to customize, or are there other (better?) alternatives? :)

struct my_pmr_generic : glz::pmr::generic
{
 // ...

using object_t = boost::unordered_flat_map<string_t, my_pmr_generic, boost::hash<string_t>, std::equal_to<string_t>,
														 std::pmr::polymorphic_allocator<std::pair<const string_t, my_pmr_generic>>>;

 // ...
};

[stephenberry]

I fully understand this issue now. Thanks for pointing out the issue is with std::variant not having access to the polymorphic allocator.

Right now the only way to get the allocator to the std::variant would be to pass it through Glaze's runtime context. This is doable, but fragile, because it requires the user to ensure the allocator matches, and it only works for a single allocator, and more complex structures could very easily have two or more allocators.

The only truly robust solution I see is creating a custom variant type that provides access to the allocator. Note that there is an open proposal to add this to the standard: https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2024/p3153r0.html

Adding a custom glz::variant is doable. However, it is a lot more code to test for a niche feature.

You might be best off looking to see if glz::lazy_json would satisfy your needs and be even higher performance than glz::generic with a more optimized allocator. Because glz::lazy_json only when you need data and can use already allocated memory.

Could you explain your motivation for this boost::unordered_flat_map optimization? I'm still open to making our own variant type until we get a polymorphic variant in the standard library, but I want to be cautious about managing two different variant implementations.

[meastp]

@stephenberry I've had a go at using glz::lazy_json and I think it works very well. I'm deserializing a GeoJSON structure, and I do a two-pass over the geometries to calculate the number of coordinates and reserve the target vector to avoid wasting allocations. This two-pass should be cheap with glz::lazy_json, right? :)

(I think what's below is sufficient, but tell me if you need more detailed info)

When it comes to properties, there are still issues, though:

template <typename LazyNode> void parse_properties(FeatureBase &feature, LazyNode &&properties_node)
{
	// Iterate keys directly on the properties_node object
	for (auto item : properties_node)
	{
		std::string_view key = item.key();

		// skip
		if (key == "internal_id")
			continue;

		// parse into struct field instead
		if (key == "uuid")
		{
			feature.uuid = parse_uuid(item.get<std::string_view>().value());
			continue;
		}

		// store attribute
		if (auto &&result = glz::read_json<glz::json_t>(item.raw_json()))
		{
			feature.attributes.emplace(key, std::move(result).value());
		}
	}
}

1. I need to move some of the properties, as they have dedicated class members in FeatureBase.
2. The property value can be a primitive (strings, integers, booleans..), object or array (with nested properties). After thinking quite a bit about this, I think just sticking with glz::generic is the best option. (You convinced me in this discussion [feature][json] use unkown keys for a partially known structure / known at runtime #1942)

Adding a custom glz::variant is doable. However, it is a lot more code to test for a niche feature.

When I use PMR now, I only have "regular" allocations when parsing each property into glz::generic. The only way I see to get rid of them is with a glz::pmr::generic-like type. I can then perhaps use an std::pmr::unsynchronized_pool_resource for these temporary allocations. So I definitely have use for this type even when I use glz::lazy_json, so I would of course really appreciate it if this was part of glaze :)

EDIT: I think I should replace feature.attributes (one glz::generic per key in a boost::unordered_flat_map) with a single glz::generic containing all of a feature's attributes in FeatureBase, to avoid wasting a lot of memory. The need for a glz::pmr::generic with this (better?) approach is even higher. The ability to replace the default std::map would then be appreciated as well... :) I guess I'm asking for a lot. Hopefully, if it is advertised and documented, it will be used by many.

Could you explain your motivation for this boost::unordered_flat_map optimization? I'm still open to making our own variant type until we get a polymorphic variant in the standard library, but I want to be cautious about managing two different variant implementations.

Well, it might not be needed for very small independent glz::generic objects, but in my experience it is much more performant than std::(unordered_)map due to it being flat (backed by continuous storage) and utilizing modern techniques for storage and lookup in hash-containers (std::(unordered_)maps cannot do this due to ABI breakage, as far as I understand). (I've had a function with (lots of) lookups in (very) large std::maps go from 30s to <100ms by using boost::unordered_flat_map, so it has become the default, pre-optimization choice.)

Note that there is an open proposal to add this to the standard: https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2024/p3153r0.html

Yes, I linked to the github issue for that exact paper in my post. ;)

[stephenberry]

I'm deserializing a GeoJSON structure, and I do a two-pass over the geometries to calculate the number of coordinates and reserve the target vector to avoid wasting allocations. This two-pass should be cheap with glz::lazy_json, right? :)

If you only ever parse your GeoJSON once, or you reallocate every time, or you care about the first time it is parsed and need absolute peak performance in that case, then a two-pass approach might be faster. However, if you can reuse your data structures and parse more than once, then the two pass approach will likely be slower.

---

Glaze also has a glz::flat_map, which we could also use for an optimized glz::generic. The reason why glz::generic uses std::map is so that the key ordering stays fixed. But, that's not a requirement for many use cases.

I do also want to be able to use pmr allocations and stack allocate reads. So, I'm totally in agreement with your requests. They'll just take some time because they aren't bugs to solve and we'll want to make sure they are implemented as good as (or better than) the standard template library.

---

So, I think the two primary features we want to add for glz::generic are:

• Allocator aware variant support, with handling for custom allocators like pmr
• Unordered map support for higher performance when key ordering doesn't matter

Thanks for the detailed thoughts and motivation!

[meastp]

@stephenberry Thanks. :)

Yes, I need to parse the entire JSON. The two pass does save all re-allocations every vector would do during lots of push_backs. BTW: Is coords.size() or if(auto coord = coords[2].get<double>(); coord.has_value()) likely to be faster (to determine if point is 2d or 3d - it unfortunately has to be done for every point)?

---

I didn't know about glz::flat_map.

That's great, thanks :) Is and will the WIP be kept in #2280 so I can start (very early) experimental testing?

---

I guess those are most important, yes.

Would you consider having the array_t and object_t types be configurable, or document how it can be achieved? (inheritance?)
Perhaps this is/will be achievable with reflection, somehow - someday.

No problem - thank you for the great library! I am also really impressed with your quick responses to various issues. :)

[meastp]

@stephenberry I'm experimenting a bit with #2280 and customizing it locally, and it is working pretty well. Have some feedback/issues/roadblocks I was hoping you could help me with:

1. boost::unordered_flat_map has an initial size that can not be made smaller, even with .reserve(). Since I'm using it for each feature, with typically 8-15 properties, the memory overhead is too large. Currently getting good results with boost::container::flat_map and PMR. I think glz::flat_map will be equally good alternative if/when the support for PMR arrives.

2. I have successfully assigned "primitive" types, but I'm stuck on nested array_t/object_t. I also (ideally) need a way to reserve the nested object_t with correct number of attributes.

Perhaps this could be done in a simpler/more efficient way, with read_json<glz::pmr::generic_f64>(...) or some other way ?

size_t attribute_count = 0;
for (auto item : properties_node) { ++attribute_count; }

// need to reserve number of attributes up front, 
// to avoid wasting allocations (the flat_map would resize/reallocate for every insert otherwise)
feature.attributes = glz::pmr::generic_f64{feature.get_allocator(), attribute_count};

// write lazy_json into feature.attributes / glz::pmr::generic_f64
for (auto item : properties_node)
{
	if (item.is_null())
	{
		feature.attributes[key] = nullptr;
	}
	else if (item.is_boolean())
	{
		feature.attributes[key] = item.get<bool>().value();
	}
	else if (item.is_number())
	{
		feature.attributes[key] = item.get<double>().value();
	}
	else if (item.is_string())
	{
		feature.attributes[key] = item.get<std::string_view>().value();
	}
	else // if (item.is_array() || item.is_object())
	{
		// What to do here?
	}
}

[stephenberry]

@meastp, so #2280 doesn't actually allocate most of the memory using the polymorphic allocator, because the variant fields get default constructed in the parsing logic without the allocator. So, it doesn't really function as it should currently.

I realized I want to support both flat maps and ordered flat maps, and allow these to have custom allocators. So, I'm working on a highly optimized ordered flat map implementation at the moment.

to avoid wasting allocations (the flat_map would resize/reallocate for every insert otherwise)

Remember that std::vector only needs to reallocate when the size has doubled, because then it doubles the allocation size. So, there is no new realloc for every insert. The allocation cost is amortized.

Getting an allocator aware variant will be a focus of mine soon and solve a critical piece. But, from past optimization I know that a "variant" that is built around a fixed set of types can be better optimized that a general purpose variant type. So, I'll probably look into making a specialized variant type just for glz::generic, since we know the fixed set of types included therein.