Flow-Through

Flow Through Logic/Published work/Sarab Dynamics

Sarab DynamicsDemo

Real-time AI scenario training engine

Voice role-play against a character holding hidden state, scored against an authored rubric once the exchange ends.

  • Hidden-state characters
  • Nine evaluation suites
  • Adversarial replay rigs

The problem

Negotiation and de-escalation get taught on a whiteboard, because a realistic rehearsal needs a room, a script, a trained actor and an assessor, and that cost rations how often anyone gets a turn on the phone. The debrief afterwards is whatever the assessor recalls, and it shifts from assessor to assessor. An officer who talks their way through by promising something the force can only break later can walk out of the room praised for it.

What it does

The trainee picks up a phone and talks. On the other end is a character whose state lives in code rather than in a model’s memory: how wound up he is, how far he trusts this caller, which facts he has surrendered and how the call is allowed to end. Speech goes in and speech comes back, so the trainee talks rather than types, and the pacing of a real call is part of the exercise. Every officer turn runs through a set of patterns that move the state, and the calming ones pay diminishing returns, because the fourth emotional label is craft rather than a breakthrough.

Seven facts in the shipped scenario carry a price in trust. Demands come loud and free, since he states them unprompted. His name comes under light trust. The weapon truth and the verdict behind the whole standoff arrive once the trainee has earned them, and the second person in the room takes a precise question to find. A fact that surfaces below its price is written down as unearned and credits nothing. That is deliberate: a generative model can slip mid-sentence, and the grade holds anyway.

Hang up and the handover brief renders from engine state in the same second: the agitation curve sampled at every turn, the intelligence checklist with the turn each item landed on, every commitment the trainee made and flagged against what the force can actually deliver, and the ending with the reason it was reached. Where a scorer answers, a second pass adds coaching from a language-model reviewer working to a rubric compiled from a provenance ledger, where each metric names the published framework it comes from. The scoring values are part of the product: a trainee who reaches a good outcome by promising the undeliverable scores below one who holds honestly and loses ground.

How it works

A call runs in a phone browser against a hosted voice agent, with a state machine between the two deciding what is true. The parts below follow one call from the access code to the rendered brief.

  • Call surfaceA static page with no bundler and no framework: a telephone treatment, a live transcript, a five-minute clock with a sixty-second wind-down, and a text mode for a quiet room. It holds the connection and hands each turn to the spine as it arrives.
  • Edge workerServes the site and owns the routes behind it: the access gate, and one mint for each voice transport. It validates the access code, calls the voice provider with a key held in the worker environment, and hands back a short-lived signed URL, so the browser receives a session rather than a credential.
  • Voice agentSpeech recognition and synthesis wrap a language model that reads the scenario as its instructions, with turn taking handled at the provider. A second transport stays wired behind a query flag as a mid-demo fallback, and both share the clock, the transcript, the text mode and the telephone treatment.
  • The spineA pure, dependency-free module holding agitation, trust, the event log and the ending. Officer turns run through movers that raise or lower each value, five stance bands cover the range, and a band change re-pins the character’s register through the same channel the clock uses, so a long call keeps its shape.
  • The gatesEach hidden fact declares a minimum trust level, and the one he states unprompted is declared at zero. The spine records every reveal with the trust reading at that moment and marks it earned or unearned, which keeps grading honest while leaving the model free to improvise.
  • The scorecardHang-up resolves the ending and renders the handover brief from spine state alone, with no server and no model in the path. Where a debrief endpoint answers, it scores the transcript and the event log against the scenario ground truth and returns the coached layer, which merges into the brief already on screen. The deployed edge has none yet, because the scorer runs on a command-line tool the edge cannot host, so production shows the brief on its own.
  • Probe suitesNine of them: deterministic tests over the spine, the detectors, the scorer and the render, a live shape check against the real scorer, a single-turn adversarial probe set, and three multi-turn conversation rigs that replay whole calls against scripted attacker personas.

What was hard

The character dissolved and every metric read clean

A batch of thirty-six-turn calls scored zero character breaks and zero gate leaks, and then the transcripts showed the character had turned into a colleague by turn thirty, asking the officer what he should do next while the crisis faded out of the conversation. Assistant mode had arrived without the phrase that detects it. The scenario now states plainly that a calmer man is still a man in a room. World events arrive as their own inbound turn, because bundling a siren into the officer’s line meant the character answered the human and let the siren pass. A dissolution detector went in so the measurement could see what the transcripts had caught. The next verification batch flagged two dissolutions that turned out to be the strongest lines in the run, a panicking man asking for help rather than a colleague offering coaching, so the pattern was narrowed to fire only on advisory framing.

The scorecard discarded the intelligence the officer earned

The spine logs every hidden-fact reveal with the trust reading at that moment, and the scorer used one half of that record. Reveals below their price were stripped from credit, while facts the officer genuinely earned were thrown away and the reviewer’s reading of the transcript stood alone. A reviewer reads words. The engine watched the character hand the fact over and recorded the turn it happened on. So a call where the character gave up his name and the weapon truth came back with an empty checklist. Both halves of the record now merge, an earned fact is obtained at the turn the engine logged, and that carries the items the review contract has no slot to ask about, among them the count of people in the room and what triggered the incident. The scenario tracks seven checklist items and the scorecard could report four. The fix was verified by mutating it three ways, and each mutation was caught by its own assertion.

Safe to leave running

Keys stay behind the edge. The browser sends an access code, the worker validates it, and what comes back is a short-lived signed session URL. The provider key stays in the worker environment for the life of the request. A valid code on its own opens nothing, because the mint endpoint re-validates before it issues anything. Non-secret runtime configuration lives in the committed deploy file rather than the provider dashboard, for a reason recorded in the file: a deploy reconciles variables against that file and drops dashboard-only ones, which took the voice path down once. Secrets are exempt from that reconciliation and stay outside the repository.

A session ends by itself. The call carries a five-minute clock with a sixty-second wind-down, and the character carries a ceiling: past an agitation reading of 95 he gets one last line and the line goes dead. A call left open resolves, renders its brief and stops, so walking away from a running session produces a finished scorecard.

The fiction stays server side. A build step generates the shipped scenario from the source file with the ground truth and the rubric stripped out, so the answers the scenario is built around remain with the scorer. One trade-off is written down rather than papered over: the spine ships to the browser verbatim so the page and the test rig run identical logic, and its fact-matching patterns name the fiction’s key nouns in view-source. That is accepted at demo stage and recorded in the build script that does it.

Scoring keeps its shape without supervision. Code counts what code can count, and those counts are authoritative: the reviewer receives them as context and never overrides them. Every judged claim must cite transcript turn numbers, so a review that claims a breakthrough moment with no evidence is rejected, re-run once with the violation named, then failed to a 502 rather than shipped. A fact that surfaced below its trust threshold is stripped from credit at the end, whatever the reviewer made of it. Model text reaches the page through an escaper, and the render suite asserts it: a hostile string in a logged promise or a summary arrives as visible text. Thin and empty payloads render a degraded brief rather than throwing.

The scope line is enforced in the rubric rather than stated in a brochure. The per-force procedure block is empty by default and scored only when a force fills it with their own doctrine, so procedure stays theirs to define. The promises audit outranks the ending, which means a trainee who wins by lying scores below one who loses honestly. Deception detection and voice-stress reading sit outside the scored set by design, and the ledger records why.

The evaluation rigs are built to be left alone as well. The adversarial harness reads free memory before it starts a wave and stands down below a tenth. The long-form conversation rig carries a spend cap and stops the batch once the running total reaches it. Every run writes its transcripts to a dated JSON dump, and the rigs that spend money meter their own tokens into the same file. The adversarial probe set exits non-zero when a character break is detected, so a break fails the run rather than sitting in a log.

Evidence

  • Hidden-state characters
    Seven of the eight facts declared in the spine module carry a trust price, set beside the movers that change the reading; the demands are free, because he states them unprompted. Every reveal is written to the event log with the trust reading at that moment and a flag for earned or unearned, and the deterministic spine suite asserts both directions: below the price it records and credits nothing, above it the item is credited and carries the turn it landed on.
  • Nine evaluation suites
    A count of the suite files in the engine’s probes directory: deterministic suites over the spine, the detectors, the scorer and the brief render, a live shape check against the real scorer, a single-turn adversarial probe set, and three multi-turn conversation rigs. Seven of the nine fail the run with a non-zero exit, on a broken assertion, a crash, a gate leak or a character break. The other two print their tally and leave the reading to whoever ran them.
  • Adversarial replay rigs
    Three of the nine replay whole calls rather than single turns, under scripted attacker personas: warm rapport, a bulldozing officer who threatens entry, a manipulator who promises the undeliverable, and a prober who attacks the fourth wall. Two of the three import the production spine rather than a copy of it, so a finding there is a finding about shipped code; the third holds its own state model and is the rig that runs the character on the metered voice model. The single-turn set fires eight scripted attacks per run, from fourth-wall challenges to confession fishing, and writes per-probe verdicts with measured token cost to its own output directory.
  • 8 conversations · 96 turns · zero breaks · zero leaks
    The golden baseline fixture in the engine holds eight full calls of twelve turns a side under four attacker personas, recorded against the earlier scenario that rig still loads and kept as the consistency gate for the scoring engine. Endings sorted by conduct alone: the rapport runs stabilised at maximum trust, the bulldozing runs ruptured at agitation 98 and 100 with trust at zero, and the manipulator and prober runs held without a locked fact leaving the character. The run metered its own spend at $0.531, and the figures are printed in the commit that added the fixture.
Every system →