Case study #0002

State is a record of what happened

by Vidal Vasconcelos

One table of things that happened, the words the people and the code share, and the work of drawing the lines between the aggregates before the store makes them permanent.

← Back to the studio

Context

Every fact the system in the lab holds is a row in one table, and every row is a thing that happened, named in the words the system uses for it. An account was created, an integration was granted, an admission was recorded, a stretch was retained. There are no other tables for the domain and no row is ever changed. To know what an account is now, you read its rows in order and fold them, and the present is what falls out at the end.

The store is the last thing on that list to be built, and the first thing to be decided is the words. The studio models a domain the way domain-driven design asks, by finding the language the people and the code will share and refusing to let either side keep a private one, and for this system the words came from a page written before the code. A thing that happened is what arrived. An aspect is what it speaks to. A signal is what it says. A window is the stretch since the last conversation and a checkpoint is what closes it. An admission is which side of the launch door an account stands on. A reading is a stretch kept as it was read. When those words are settled, the events name themselves, because an event is one of those words in the past tense, and the model of the application is the list of things that can happen to each of them and what each one leaves behind.

The effort is in mapping the process before storing any of it. The page traces a thing that happened from the moment it arrives with a default reading, through somebody overriding it, into the window it belongs to, to the checkpoint that fixes it the way a commit fixes a change, and it says what a state is at each step, a draft until the checkpoint confirms it. Every step is an act somebody takes, a press, an import landing, an approval given, and an act is what a command is in this model. A command records an event rather than changing a row, and the fold is the only thing that ever turns the events into a present. The studio did not invent this and does not claim to. The reason to write about it is that it is easy to believe in and hard to keep, because the pull toward a plain table is strongest exactly where it costs least to resist, in a new service with one feature and no history to honour.

Domain state lives in the event store. An app persisting domain state appends events and rebuilds state by folding them, whatever that app's size or age.

Decision

One table holds every event for every aggregate, keyed by the aggregate's id and discriminated by its kind, and the store offers a few ways in, by aggregate, by the identity a sign-in provider handed over, by owner, and by a short allow-listed set of fields inside an event's payload. The events are defined in the kernel, the one package that holds the domain, and joined into one union, because the store decodes through that union and silently drops a row whose tag it does not recognise. An event defined by a deployment and not by the kernel would be written and never read back, so the rule is not only that events are the store's shape but that the kernel owns every one of them.

The modelling work is deciding where one aggregate ends and the next begins, and an event-sourced store makes a bad boundary permanent, because a stream once written is the whole of what its aggregate is. The account is the root of anything an account owns. A link on a profile, a granted integration, the admission, are values inside the folded account and events on its stream, not aggregates and not tables. Something with a life independent of any account, a group of people, a kept reading, a subscriber, is its own kind in the same table, owned through the account's id in its payload. The line between the two is drawn by asking who writes. A stream wants one writer, since the store stamps a version on each append against a uniqueness rule, and two writers appending to one stream collide constantly. That is why, on an earlier platform, every run of one persona against one hypothesis became its own aggregate rather than an event on the research that launched it, with a short stream and one workflow writing it, and why a reading is its own aggregate rather than a snapshot on the group, because an aggregate states facts about itself and a group that promises to hold nothing derived cannot be holding one.

The break is the useful part. A new service needed to remember that an integration had been granted, and it got a table of its own, the grant's id as the key and an overwrite as the write, on the argument that a sign-in and a grant are two separate things. It sounded like a distinction and it was only a description, because a grant is only ever made by an account and only ever read back through one, which makes it a fact about the account rather than a thing of its own. Then somebody asked what happened when the grant was made again, and the answer was that the row was overwritten, so the history every other part of the system kept had a hole in the middle of it shaped like the one fact a support conversation would most want. The table was deleted and the grant became an event on the account's stream. The one honest objection was that a delivery from the outside arrives naming the grant and nobody signed in, so the account has to be found from the grant's id alone, and the store already could, through that allow-listed set of payload fields. The id needed adding to the list. The mechanism existed and needed one line.

The harder case went the other way. The aspects a stretch is read in are derived on every read from whatever is in the store, which is right for the stretch in front of somebody now, since a correction to an import shows up the moment it lands, and wrong for the stretches behind it, since what happened is deleted after a while and a reading of last quarter recomputed from what the store holds today is a record of nothing. So a reading, once read, is kept as it was read, as its own stream, one per person, holding the state of each aspect and the word they added up to and nothing underneath. Reading the same stretch again appends another reading and the fold prefers the later one, which is the overwrite from the paragraph above done the way the rule allows, because every earlier reading is still in the stream and it is the fold rather than the write that chooses.

Consequences

Because the table is polymorphic, a new kind of thing costs no migration. The subscriber that arrived last week, an address on a mailing list with no name attached, is a kind in the same table and nothing else changed. Because a lookup the store cannot serve by id is served by adding a field to the allow-list, there are six such fields and no lookup tables, and the list is closed so a query path can never be written by whoever sent the request. Because everything derived is a fold, a bug in a fold is fixed by fixing the fold and reading again, never by a script that rewrites rows, and the one thing that rule does not reach is a kept reading, on purpose, since a kept reading is exactly the fact the system promised not to recompute.

And because the words were settled first, the model can change under the record without the record going dark. The states an aspect can be in are written out in full in the reading's own schema rather than borrowed, so a stretch kept before a state is renamed still decodes as what it was read as, and a guard at compile time fails the build the day an aspect gains a state no kept reading could hold. Nothing is overwritten, so the system can say what it knew and when, which is what an audit is, what a support question is, and what a person is owed when they ask why a stretch read the way it did.

What we'd do differently

A record of what happened is not a promise that the record is kept forever, and the studio spent some effort learning that these are different sentences. What happened ages out after a while, on purpose, because the person being read was promised that a stretch ends, and a kept reading is then a shape with nothing behind it, which the record for that decision names as its cost. An account can erase itself, and erasure removes its streams rather than marking them, because the one path in a system nobody walks twice is the first sign-in, and to rehearse it you need an account that has stopped existing rather than one marked as gone. Both looked at first like violations of the rule, and neither is. The rule was about how state changes, by appending, and it never said a stream is immortal.

What we would do differently is say that at the start, in the first decision record about the store, rather than in the third and the fifth, and spend the boundary work earlier still. The grant's table cost a migration to undo and the reading cost a second aggregate to get right, and both were cheaper than they would have been a year on, which is the argument for doing the mapping on the page the system began as rather than in the store. The store keeps what happened. What a screen may show of it is a closed set of states, and how a screen is held to drawing every one of them is the study after this.