Dynamic Consistency Boundary Support just to join the herd

[jeremydmiller]

Other event stores cough Axon IQ and KurrentDb cough can only support consistency boundaries within operations for a single stream at a time and also have some weird limitations about only supporting a single "aggregate" projection per stream. To get around that, the "Dynamic Consistency Boundary" idea has taken off in the greater Event Sourcing world.

I 100% think that Marten's ability to handle consistency across multiple streams at one time mostly made DCB unnecessary for Marten, but there are some use cases where DCB could lead to simpler code than the current Marten approach, and we probably can't resist the hype train for DCB otherwise. So, all that aside, let's talk about adding DCB support to Marten (and Polecat immediately after)

Tagging Events

For DCB, we need a way to "tag" events and also to be able to efficiently query for events just by tag values. Tags can be captured either at:

1. Time of appending events
2. Later through a new "tag events" API of some sort to retrofit tags

We could utilize the existing Event.Headers storage, but that's absolutely not optimized for fast querying whatsoever. I instead propose that we keep the tag values in a separate table for each registered/known type of tag, and do INNER JOINs in event queries to get the right events.

Going to catch everybody by surprise here and say that I recommend "tags" be done through strong typed identifiers. So for the canonical
sample of student and course tags from the DCB examples:

[ValueObject<string>]
public partial record StudentId;

[ValueObject<string>]
public partial record CourseId;

And these would have to be registered w/ Marten (or discovered some other way) like:

StoreOptions.Events.RegisterTagType<StudentId>().RegisterTagType<CourseId>();

The reason they need to be registered is that I'm proposing completely separate tables for each tag with the shape:

mt_event_tag_student_id
--------------------------
seq_id (PK)
value

This will spill into the Event<T> and IEvent abstractions, so we'll add a Tags collection. Not sure it's really necessary to worry about tags when we're loading events other than a query item.

In Combination w/ Wolverine

Let's say you need to do a multi-stream operation with Wolverine, and you need consistency with both streams even if only one stream gets events appended in the operation. Something like this conceptual message handler in Wolverine:

public static class RegisterStudentHandler
{
    public static IEnumerable<object> Handle(
        RegisterStudent command,

        // This is a new attribute that would be a superset of the existing [Aggregate] attribute but would reject
        // with an optimistic version check on SaveChangesAsync even if there were no events appended to this 
        // stream during the operation, but there were events appended to *some* stream. This has come up before.
        [ConsistentAggregate] Student student,
        [ConsistentAggregate] Course course)
        {
             // Depending on the state of the Student and Course, you maybe decide to:
             // Assuming that Student and Course use strong typed identifiers
             yield return Event.For(new StudentRegistered{}).WithTag(student.Id, course.Id);
        }
}

In the case above, maybe Wolverine is smart enough to use the tags to connect the event and append to both the Student and Course stream. Or if some other event is tagged to only one Id, that event goes to that stream and Wolverine still has Marten enforce an optimistic concurrency check on the other stream.

We do know too or can derive the relationship between an aggregate type and strong typed identifier type if you use that.

I think the use case I've seen for a JasperFx client needed to:

• Load one related aggregate from FetchForWriting() just to see a state, but also to verify that that stream was not advanced after. And this was a repeated pattern, so they make a helper extension method for it. Great, but it made a database round trip every time.
• Append events to another stream that was also fetched by FetchForWriting().

In that case they had to call FetchForWriting() twice and make two round trips to the database. It'd be cool if you could do that all through one
round trip!

Some of the work about is in Wolverine and some in Marten, but the release will probably come out together. And the actual work as well.

The DCB Use Cases

The DCB web site lists a couple possible use cases for DCB, so let's run through them and how you can either deal w/ them in Marten today or we'll change Marten to handle:

1. "Constraints affecting multiple entities" -- you can already do this out of the box today with Marten by just calling FetchForWriting multiple times for different streams (this can be batched today as well!). The changes here are all about syntactical sugar to make the code for this cleaner with Wolverine and hopefully to opt into Marten batch queries under the covers.
2. "Enforcing global uniqueness" -- easy to do with Marten as is because of our consistency model. No real need for DCB here
3. "Replacing Read Models" -- this example I think is all built around Axon IQ. You can already query for events in a more loose way in Marten and aggregate to any model you want. Maybe there's some work here to create new constraints for the DCB consistency checks across streams here. It's also possible in Marten to use all the unique read models you might want for the same stream, so you can use cutdown models to your heart's content, and even teach Marten to only pull the events you care about by event type anyway
4. "Idempotency" -- again, easy to do today with Inline projections and unique constraints. Not something that requires DCB

Other Use Cases

• We'll need a new API to query for events the "DCB way" where you feed it OR conditions of Event Type and Tag values as explained here. I see that as low hanging fruit.
• The real DCB spec doesn't do this, but I think I'd opt for the ability to run an Aggregate over the events returned above just like our existing AggregateTo<T>() operator.
• That first bullet will need to work like FetchForWriting() in that it will reuse the query later as an IStorageOperation that can assert no new events were added that used that same criteria

MOAR Wolverine Integration

There will have to be some kind of equivalent "aggregate handler workflow", but for the DCB query + aggregate

[jeremydmiller]

Closed by #4162