Talaria Scientific

Blog

A system of record for computational science

Design Notes

Talaria keeps a system of record underneath your research: every idea, run, result, and decision, kept with its provenance, whether or not it worked out. We have argued (Why I'm building Talaria Scientific) that validation, not ideas, bottlenecks computational science, and that a harness can take the schlep without taking the science. This piece explains the memory systems that make that safe: how the record is structured, and why each structural choice was made, failure mode by failure mode.

The cold restart problem

If you put a project down in May and pick it up in September the first week or more back goes to archaeology. What did I try last? Which variants actually ran? Which configuration produced the plot in the draft? Why was the second baseline abandoned? Did the "good" result predate or postdate the preprocessing fix? This is compounded if you hand the project to a collaborator, or a new student, or a skeptical reviewer.

The research often moves faster than the record keeping, every interruption resets the context, and the incentives all point at the next result, not the receipts of the last one. A lab notebook only works if the scientist keeps it up to date, and there are better uses for attention. So the record slowly diverges from the work, and everything downstream of the record (reproduction, collaboration, review) inherits the gaps.

The system of record we have built inverts the lab notebook ritual: a record the work keeps of itself, instead of one the scientist has to remember to keep. When the harness runs your validation, the bookkeeping is not a separate chore that competes with the science; it is a side effect of the machinery doing the work. We have built the shape that record should take in a very opinionated way, which we explain below.

The shape

At the top of the system of record sits a citation directed acyclic graph (DAG): the literature you build on, linked so every claim traces to its source chronologically and by reference. The graph is directed because each reference cites others and acyclic because, in the record, you can't cite the future. Below it grows a triad of hypothesis, results, and findings DAGs. A hypothesis graph holds what you're explicitly asking for each experiment/run. A results graph holds what actually ran, each result carrying the exact code, configuration, and environment it ran with; exact, because that is what makes a rerun a reproduction instead of a reenactment. A findings graph holds what you now believe, an updated posterior from the results, each finding pinned to what led to it. These graphs are directed because each new entry comes from what was learned before and acyclic because that is how the scientific process behaves by construction; you hypothesize, test, and then learn. Overshadowing all of these DAGs is a concept graph: the living map of definitions and connections the other graphs share, information about the field, baselines, definitions, statistics, and more. Time flows down the page, the references above the ideas, the work growing beneath it.

THE SYSTEM OF RECORD · FOUR DAGS AND A CONCEPT GRAPH the record the work keeps of itself time THE REFERENCES DAG the literature you build on, linked: every claim traces to its source ref the method paper ref the dataset paper ref the rival baseline ref the survey ref your own last paper now the literature seeds the questions THE HYPOTHESIS DAG what you're asking: ideas, refinements, follow-ups H1 superseded: marked, kept H2 H3 H4 H5 H6 open hypotheses wait, first-class: the parked research agenda you can hand off THE RESULTS DAG what actually ran, win or lose R1 R2 R3 (one question, several runs) answered by every run pinned: exact code + config + environment (a rerun is a reproduction) R? no hypothesis? not allowed to exist (orphan numbers are how spurious findings start) THE FINDINGS DAG what you now believe, pinned to the results that earned it F1 the claim carries its evidence F2 pins the evidence CONCEPTS (a directed graph, not a DAG) the living map of definitions all the other graphs share what the record buys you: pick up where you left off · collaborate on the real history · an audit trail you can rerun schematic, teaser altitude: the full anatomy gets its own Design Notes post
The system of record: references above, the hypothesis-results-findings triad below, concepts overshadowing all of it. Every result answers a hypothesis and findings carry their evidence with them.

The layers are separated intentionally; what you read, what you ask, what you run, and what you believe are different kinds of claims with different lifetimes and provenance. A system of record should not blur them (ie a notebook page where a citation, a hunch, a number, and a conclusion share a line) because it can't answer the questions that matter later: what evidence does this belief rest on, and what exactly produced that evidence? In the record, those questions are edges you follow on a graph, not memories you reconstruct.

Five design decisions, and the failure mode each one kills

This structure was built in an opinionated way, and each is a design choice aimed at a specific way research bookkeeping can go wrong.

An untested hypothesis is first-class. Parked ideas aren't clutter, they're your research agenda: the queue you, a student, or the harness can pick up next. The failure mode this kills is the one the manifesto (Why I'm building Talaria Scientific) opened with: the reviewer suggestion, the seminar question, and the shower thought all evaporating because nothing held them. In the record they wait, visibly unpaired with results, until someone (or the harness, within its budget) picks them up.

A result that answers no hypothesis is not allowed to exist. Orphan numbers are how spurious findings start: fifty configurations ran, one "worked," and nobody remembers what question it was answering. Forcing every result to name its hypothesis is the structural version of pre-registration: entries are timestamped and append-only, so whether a hypothesis preceded its results is a fact of the record, not a claim in the paper. That is what makes "we tried 50 things and report the best one" visible in the record instead of invisible in a results folder.

Superseded ideas get marked, never deleted. Research doesn't evolve cleanly, it morphs and branches, and a record that only keeps the current story is the polished paper's hindsight trail. Keeping the whole history means you can revisit abandoned paths when a new tool makes them viable, hand a collaborator what actually happened instead of a reconstruction, and even publish the roads not taken.

Overrides are journaled. The scientist can force past an integrity gate (sometimes you should; a hunch can lead to a breakthrough), but the override itself is recorded along with the exact checks it waived. The failure mode this kills is silent rule-bending: the gap between "we followed the protocol" and what happened, which no one intended and no one can find later. The record stays honest even where you overruled it.

The record self-heals before you publish. Mid-flight mess is tolerated on purpose, because rigor that blocks serendipity would be its own failure; science sometimes lurches forward by stumbling into things. Claims are gated by strength, never existence: a lucky observation enters the record labeled as what it is, and earns its confirmation later. When you are done, the harness walks the record with you and heals what's incomplete, and the heal itself is journaled like any override, so a repaired record never masquerades as an original one. Every result you plan to publish is one you can trust and reproduce exactly.

What standard techniques miss

Each standard tool holds a slice of this. The lab notebook (paper or electronic) holds the narrative, but it is only as complete as the scientist's discipline on their worst week, and it isn't linked to the artifacts: the notebook says what you meant to run, not what ran. The wiki or shared doc holds the write-up, prose about runs rather than the runs themselves. The experiment tracker holds the opposite slice: metrics and configurations logged automatically, with the questions nowhere, so orphan numbers are the default rather than the exception, sometimes lost in flat-file-hell (ie "results-final-v2-final-final.log"). None of them is built to hold the whole causal chain, from a claim in a draft back through the finding, the results, the hypothesis, and the references that motivated it. Pieces of this chain exist across provenance standards and workflow tools, and the deep dives in this series will name them and position relative to them; what none of them holds is the whole chain, attached at the moment of execution.

The system of record is that chain, kept as structure. It is not a better notebook next to the work; it is the work's own trail, with the provenance attached at the moment each step happens, by the machinery that ran the step. This not only allows you to research with confidence, it also means every result you publish carries what a rerun needs, now and in the future.

What the shape buys you

For you, the researcher: put the project down in May, pick it up in September, and start where you left off, because the record holds the state your memory dropped.

For your collaborators: they walk your actual search, dead ends included, instead of your recollection of it.

For your readers and reviewers: the trail from claim back to code, configuration, and environment is there by construction, kept automatically, and the full search is visible, not just the winners.

We believe the pace of science should be set by the ideas. The system of record is where the ideas, results, and findings all live. So you and others can stand on a strong foundation when continuing to push forward into the frontier (Surfing the Frontier Wave that rests between The Bitter Lesson and No Free Lunch).