Unstructured Material Turned Into Training-Grade Structure
Frontier models are thin exactly where a specialist buyer needs them: Indian legal procedure, a state's administrative practice, a regulated industry's actual paperwork. That material exists. It is almost never in a form anyone can train on.
What We Hand Over
Scans, filings, recordings, whatever the domain runs on. Originals kept byte-exact, so any later reprocessing starts from the source rather than from someone's cleaned copy.
Text extracted, structure recovered, entities resolved, and every record given a stable address. Ground facts and anything inferred are kept in separate fields and never merged.
Per record: where it came from, when it was fetched, and its licence position. A dataset without this is not usable by anyone with a legal department.
Typed records against a published schema, validated, with the gaps documented rather than quietly filled.
Step 3 is the one that decides whether a dataset can be used. Most scraped corpora fail not on quality but on the fact that nobody can say where a given row came from or whether it was permitted.
The Rule That Makes the Data Trustworthy
A model's output is never recorded as a fact. Anything normalised, classified or inferred is stored as a derived value carrying the method that produced it, the confidence, and what it was derived from. The ground fact stays exactly as printed in the source, in its own field.
That separation lets a buyer decide how much of the interpretation to accept. It also stops a pipeline turning a guess into a training label.
The verbatim value from the document, with source, exact location and the snippet that contains it. Never edited, never smoothed.
The normalised or classified form, with method, confidence and what it came from. Carries no source of its own, because it is not from a source.
They stay in two separate fields. Once an interpretation is written into the same field as the record itself, nobody downstream can tell which parts came from the document and which were inferred.
What One Delivered Record Carries
| Field | Holds | Comes from | Safe to train on as fact |
|---|---|---|---|
| Value as printed | The exact string in the document | The source | yes |
| Source and locator | Document, page, and the surrounding snippet | The source | yes |
| Fetched at | When the bytes were taken, and their licence position | The fetch | yes |
| Normalised value | The cleaned or standardised form | A model | as a label |
| Method and confidence | How it was derived, and how sure | A model | as a label |
Scroll the table sideways →
The last column is the one a buyer's legal team reads first. Rows drawn from a document can be defended. Rows produced by a model are labels, useful and clearly marked as interpretation, and a dataset that blurs the two cannot be used by anyone who has to answer for it later.
Evaluation Sets That Have Not Been Memorised
Public benchmarks leak into training data, and a model that has seen the answers cannot be measured by them. So we build evaluation sets from material that is genuinely held out. Each question is written against a specific passage in a controlled corpus. The answer is checked as an exact quotation from a document, rather than by fuzzy comparison to an expected string.
Because the corpus is structured, scoring can be deterministic. A citation either resolves to the right passage or it does not, which removes the judge model and the argument about whether it is fair.
Retrieval Without a Vector Database
Where a lab or a product team wants grounded answering over a domain, the structure usually does the work that embeddings are reached for. A typed catalogue, exact-match search and authored relations give a model the shape of the domain, which similarity search flattens away. Citations resolve to a paragraph rather than to a chunk, so an answer can be followed back.
Delivered as a defined set of operations any assistant can call, so the corpus is usable from whatever stack you already run. The same interface is described for archives, and is running in public on our open library.