BTPublic

The problem

Work described as artificial intelligence is usually a call to somebody else’s model, and from the outside that call looks the same whether the caller understands the machinery underneath or treats it as a sealed box. One way to settle the question was to build the machinery: a vocabulary, a transformer, a training loop, and the server that answers a stranger, from nothing, on hardware a person can own. The result had to survive somebody opening the code and reading it.

What it does

The shape is GPT-2 small: twelve layers at 768 dimensions, with a context window of 1,024 tokens. What sits inside that shape is current practice rather than the 2019 recipe. Normalisation is RMSNorm, computed in float32 whatever precision the model is running in, because the sum of squares over a wide vector is where bfloat16 costs accuracy that then propagates into every downstream activation. Position arrives by rotating each query and key rather than by adding a learned vector, so relative distance reaches attention directly and no parameters are spent on position at all. The feed-forward is a gated SwiGLU unit, and the table that turns tokens into vectors is the same table that turns vectors back into tokens, which at this vocabulary size saves 12.6 million parameters, and those buy more in the blocks than in a second copy of the vocabulary.

The vocabulary is the model’s own, trained on the corpus the model reads. A borrowed one was ruled out on arithmetic: Qwen’s runs to 151,000 tokens, and at the first rung’s width of 512 that embedding table alone would have come to 77 million parameters, more than twice the size of the whole 29.9M model. A byte-level BPE trained on this corpus instead spends its merges on the words that actually appear, so a sentence costs fewer tokens, and fewer tokens per sentence means more sentences seen per hour of training. Four special tokens exist and no more, one of which separates speaker turns so the conversational fine-tune has a structure to lean on.

The same code trained a ladder of three sizes on one card: 29.9M parameters on children’s stories, then 97.5M on 1.95 billion tokens of educational web text, then a third rung of 320.4M on 6.41 billion tokens. A short fine-tune on generated conversation turns a base model that continues text into one that answers a visitor, and it computes loss on the model’s own replies while every token of the question stays masked. What a visitor reaches is a streaming server where questions with one computable answer, sums and dates among them, are worked out in Python before the model sees them and labelled as computed. The model states its own size accurately and will state facts about the world with the same confidence and be wrong, which the page in front of it says plainly.

How it works

The pipeline runs in the order the repository lists it, and each stage writes an artifact the next stage reads, so any one of them can be rerun on its own.

• Vocabularytokenizer.py trains a byte-level BPE on the corpus itself, 16,384 tokens at the sizes that ship, so every input decomposes into bytes the model already holds and an unknown character has nowhere to occur. Its check prints characters per token, which is the number that says whether the vocabulary is earning its slots.
• Token streamdata.py encodes the corpus once into a flat memory-mapped array of 16-bit integers, joining documents with an end-of-text token between them. Training then reads that array without touching text again, which keeps the GPU as the bottleneck.
• Pretrainingtrain.py runs bfloat16 autocast over float32 master weights, reaches a large effective batch by gradient accumulation rather than by filling memory, and decays the learning rate on a cosine after a linear warmup. Weight decay applies to matrices alone, since decaying a normalisation gain pulls it toward zero, which deletes rather than regularises.
• Persona and fine-tunepersona.py holds every fact the model may state about itself in one file, make_chat.py generates conversations against those facts, and finetune.py teaches them with the label set to ignore on every token of the visitor’s question. Training on the question produces a model that answers and then invents your next message.
• Deterministic answerstools.py intercepts arithmetic, the date and counting, computes them in Python and hands the model the result to phrase. A language model samples a distribution and holds no internal difference between knowing and inventing, so the honest way to make a number right is to keep the question away from it.
• Servingserve.py is the standard library HTTP server and torch, with nothing else in the dependency list, streaming tokens over server-sent events as they are produced. Each piece of text is taken as the difference between two full decodes, because a byte-level BPE can split one character across two tokens and decoding a token alone would emit broken bytes.
• Watching a long runwatch.py reads temperature, memory, power and throughput off the card and answers the two questions a long run raises at 2am: is it too hot, and is it still moving. Its thresholds are the card’s own behaviour rather than invented ones.

What was hard

Safe to leave running

The deterministic layer parses an expression by walking a whitelist of syntax nodes, so a string that is not a sum is refused rather than run. Behind a public URL, handing that text to eval would be remote code execution, and the repository says so in the module docstring next to the parser. Exponents are bounded as well, since a short expression can ask for a number with a billion digits and computing it is a denial of service that needs no malice. A check that fails to recognise a question passes it to the model rather than answering it anyway, because a tool that answers what it misread is worse than no tool at all.

Requests are typed before they reach the GPU. The message must be a string rather than something coerced into one, since str() on a dictionary produces a plausible sentence that the model would then answer on a stranger’s say-so. The prompt is capped at 400 characters and the body at 16KB, and every entry of the conversation history is checked individually after a list of integers once reached inside the generator and ended a stream that had already begun, leaving the visitor an empty reply and the log nothing at all. Static paths are resolved and contained, so a path containing .. stays inside the directory being served.

One card serving a public URL is a queue, so generation runs behind a single lock, each address gets 20 requests a minute, and the address table prunes itself above 4,000 entries. Batching concurrent visitors would be faster and a great deal more code, and one person’s GPU answering one visitor at a time is the honest shape of the deployment.

Long runs are bounded rather than trusted. Training takes a hard ceiling on GPU memory, counted in gibibytes because that is the unit the allocator reports and the unit an out-of-memory message quotes: an earlier version divided by 1e9, turned a request for 14 into a 13.04 GiB ceiling, and the run died on a 128MB allocation. A run that dies of its own cap fails cleanly at startup, where a desktop starved by the run’s appetite becomes a mystery an hour later. Checkpoints save on improvement and resume from the last one, and the unattended overnight chain wraps every step so the multi-day run restarts whatever else failed, stops only process ids it recorded itself, and keeps the previous dataset and the previous checkpoint beside the new ones.

The facts the model states about itself are guarded the same way. Importing persona.py raises while the measured training time is unset, which is the last point at which a guessed number can be caught before it becomes a false statement in the model’s own mouth on a public domain. The generator refuses to write a set less than half the size of the one already on disk, after an exhausted API quota once failed every call in a run and wrote the empty result over a working set while reporting success. The panel on the demo page reads its specification from the running model rather than from the HTML, so the numbers stay true on the day a bigger checkpoint replaces the current one.

The last control is scope. The process reads a message and writes a reply, holds no state between requests, and calls nothing that writes anywhere. The intent is that it becomes the conversational front of the operations platform already running on the same machine; until that connection is built and reviewed, talking is the whole of what it does, and the repository record says exactly that.

Evidence

• 97.5M parameters
  The second rung’s count, measured by the model’s own parameter_count(), which counts tied weights once, and printed at the head of its training log: artifacts/train-124m.log opens with 97.5M params, 12 layers, 12 heads, 768 wide, vocabulary 16,384, context 1,024. The name 124M comes from GPT-2’s shape rather than from a count: a 12 layer, 768 wide model is conventionally called 124M, but that figure assumes a 50,257 word vocabulary. This one uses 16,384, which removes 26 million parameters from the embedding table and leaves 97.5M. Quoting the shape name as a quantity would inflate the figure by 27%, and the README states that directly under the table carrying the number. The /facts endpoint the demo panel reads reports whichever checkpoint is loaded, so the panel follows the ladder rather than repeating this figure.
• One RTX 5080
  A single 16GB consumer card, and the only hardware in the project. The 97.5M run is logged at 9,921 steps of 196,608 tokens each, finishing in 489 minutes, with every step line and the final validation loss printed to artifacts/train-124m.log. The third rung of 320.4M parameters trained on the same card, sharing it with the server that was answering visitors at the time.
• 1.95B tokens, held-out loss 3.041
  Token budgets follow the Chinchilla ratio of twenty tokens per parameter, so the budget is derived from the count rather than chosen. Loss is measured on a held-out split every 250 steps and again at the end over twice as many batches, and the final figure is written into the checkpoint and the run’s JSON summary. The README notes that this number sits above the 1.178 of the smaller model trained on children’s stories and that the two are not comparable, since educational web text is far less predictable than a children’s story.
• 69,000 tokens a second
  Sustained throughput during the 97.5M run, around 40 TFLOP/s in bfloat16, printed on every twentieth step of artifacts/train-124m.log and visible rising from 59k at step 0 to 69k across the second half of the run.

See it running

The model answers at bt.flow-through.com.au, where the panel behind the information icon reads its parameter count, layers, width, vocabulary and context window from the running checkpoint rather than from the page.