Three MCP tools
semantics_record, semantics_list, semantics_retire — record, review and retire, with evidence run at record time and every record landing an audit row.
Learned by the agent, with evidence
A semantic layer that learns, instead of one you write
…for the AI agents that write your SQL. Every agent session reads your schema and probes your data; this one records what it established — a unit, a time zone, a grain, what a code means — with the query that proved it, and every agent after it starts from evidence instead of guessing. Open source, self-hosted, part of the server your agent already talks to.
Last updated 2026-09-11
What a semantic layer is, in the searcher's words
A semantic layer is the set of shared definitions that make two people's — or two agents' — numbers agree: what "revenue" means, which column is the timestamp and in what time zone, how the orders table joins to the customers table, what status code 4 means, what one row represents. dbt's Semantic Layer, Cube, AtScale and LookML all sell versions of the same idea, and behind the different names sits the same anatomy: measures, dimensions, grain, join paths, and the governance that keeps them honest. The honest sentence about the category: in every existing implementation, that shared definition is a file somebody writes by hand — and nobody updates.
Why written models rot
The person who knew what pickup_ts really is leaves. The schema changes under the model. The YAML
says the timestamp is UTC and nobody re-checks; the warehouse rename lands on Tuesday and the definition file
still points at a column that no longer exists. The failure is silent, because nothing checks a hand-written
model against the live database — and an LLM reading that stale file does so with total confidence. That is
the argument against the category as-is, and it is why we did not build a better editor for model files.
A semantic model you do not write
A semantic model is usually a thing you author: entities, measures, dimensions, joins. Here the model is a side effect of the work. The loop, in four steps:
-
The agent reads the schema and probes the rows
Introspection says a column is DECIMAL(6,2); it cannot say the value is already in degrees Celsius while a sibling column names its unit. So the agent probes — a bounded, read-only SELECT.
-
It establishes a fact introspection cannot state
A unit. A time zone. A sample rate. A grain — one row per what. What a coded value means. A join that holds. A trap. A business definition like "revenue excludes tips".
-
It records the fact with the query that proved it
semantics_recordstores the fact, its references and its evidence SQL; the evidence runs once at record time, and a fact without evidence is stored as merely asserted — never as observed. A business definition, an exclusion or a preference is a choice, not an observation: it is stored as asserted even with evidence, until a person confirms it. -
The next agent sees it inline, where it is looking
No separate "query the memory" step to forget: the facts arrive on
datasources_get_columns,datasources_get_tablesanddatasources_get, and on their REST twins.
One real response — the sample deployment's datasources_get_columns, captured verbatim from the
end-to-end wire test the learned-semantics round shipped — with a recorded unit fact riding on exactly one
column:
[
{
"name": "id",
"type": "INTEGER",
"nullable": false,
"source_type": "serial",
"warnings": []
},
{
"name": "station",
"type": "STRING",
"nullable": false,
"source_type": "text",
"warnings": []
},
{
"name": "value_c",
"type": "DECIMAL",
"precision": 6,
"scale": 2,
"nullable": false,
"source_type": "numeric",
"warnings": [],
"facts": [
{
"id": "08938150-f062-4dfb-a52e-b797c2786e15",
"scope": "DATASOURCE",
"kind": "unit",
"fact": "value_c is already in degrees Celsius — never tenths",
"trust": "observed",
"evidence_summary": "value_c=21.40, unit=c | value_c=19.05, unit=c",
"recorded_via": "mcp",
"recorded_at": "2026-09-11T16:43:04.080418Z",
"from_this_workspace": true,
"source_pipeline": {
"id": "5e5a0000-0000-0000-0000-0000000000d4",
"name": "finance/revenue"
}
}
]
},
{
"name": "unit",
"type": "STRING",
"nullable": false,
"source_type": "text",
"warnings": []
}
]Two scopes of fact: a business glossary that maintains itself
Facts come in two scopes, and the split is the difference between physics and policy. A datasource fact is true for everyone who reads the data — a unit, a time zone, a sample rate — and it travels with the datasource to every workspace the datasource is granted to. A workspace fact is an organisation's meaning — "revenue excludes tips", "regions means the five named ones" — and it stays private to the workspace that recorded it, because another workspace on the same warehouse may define it differently.
Nothing JDBC already knows is stored. Types, nullability, keys and comments stay where introspection serves them, because a second copy of a type is a lie by the second week — the store exists for what introspection cannot say, and the facts are checked against introspection, not duplicated by it.
Trust is earned, not assumed
Every fact carries its trust state, and the ladder is short: asserted — recorded without
evidence, or a choice (a definition, an exclusion, a preference) whatever it carries; observed — the evidence query ran at record time and its first rows became the
evidence summary, for a fact about the data; verified — a person confirmed it. A fact is never higher-trust than its
evidence: "the pipeline executed" is not evidence of semantic correctness. And when two live facts of the
same kind disagree — gross versus net revenue — they coexist and are flagged as a conflict, with provenance
on each. The store never picks a winner for you.
Schema drift is detected, never guessed
Every introspection response that carries facts also recomputes each fact's fingerprint from the columns it
just read — no extra queries. Rename or drop a column and its facts flip to stale at the next
read, shown beside the current columns with a message naming what no longer exists. Change a table's column
set around a fact that still resolves and the fact is marked needs_review. Three states, all
visible, none fatal:
- served as stored — every reference resolves and the fingerprint matches;
- needs_review — the refs still resolve but the table changed since the fact was recorded;
- stale — a referenced column or table no longer exists; the fact is shown only as history, never as current truth.
What a machine never does is decide that a dropped column and a new column are the same thing — a rename and
a "drop plus unrelated add" are indistinguishable to software, so nothing is re-mapped automatically. Someone
re-verifies and records the successor with supersedes, which links the history and retires the
predecessor as superseded. Facts are never deleted; retirement is a state with a reason.
Today
What ships now
Facts on MCP and REST introspection
The facts[] block rides on datasources_get, _get_tables and _get_columns — and on the same three REST endpoints, so a dashboard can read it too.
Drift at read
Fingerprints recomputed on every introspection read: stale and needs_review marked beside the live columns, with the drift message naming what changed.
The facts list, read-only
A read-only Facts dialog on every datasource row and the same list on the lake detail — trust badges, drift messages, conflicts and provenance. Recording stays with the agent (and REST reads) for now.
Next · decided, not scheduled
What is next
- Recording and verifying from the UI (round 1's screen is deliberately read-only).
- Harvesting candidate definitions from saved pipelines — proposed as
asserted, never auto-observed. - A question-level index: "what do we know about revenue?" across datasources.
- Promotion by agreement — two independent pipelines computing the same definition.
- Export as OSI, the interchange format the research brief left the door open for.
How this differs from the dbt semantic layer and Cube
The dbt semantic layer and Cube are compiled metrics layers: you author a model of measures, dimensions and joins — in YAML for dbt's MetricFlow, in a data model for Cube — and their engines compile that model into consistent metric SQL for the BI tools and applications that call them. That compilation is genuinely valuable, and it is the part we do not do: one measure definition generating the same SQL in Looker, Tableau and a notebook is their strength, and the honest comparison names it.
Ours is the opposite direction: not a model you author and an engine compiles, but facts the agent records — attached to your live schema, checked against it on every read, and served to the agent in the introspection response it was already making. There is no YAML to rot and no file to deploy; there is also, today, no compiled metric SQL for BI tools. If that direction matters to you, it is on the roadmap — a universal semantic layer for dashboards and metrics is the same round that brings dashboards, and the headless-BI integration it implies is named there, not here.
For the engineer: the tools, the kinds, the rules
Three tools, from tools/list: semantics_record — one fact, its structural
references, its optional evidence SQL; semantics_list — the facts on a datasource, with trust,
drift and provenance, filtered by table, scope or time; semantics_retire — one id, one reason,
no delete. Recording needs an author-scoped key; viewing needs none beyond the datasource's
visibility — authors record, viewers never. Every record lands a
semantics.recorded audit row with kind, scope, refs and whether evidence ran.
The facts live in learned_facts — a Postgres table in your deployment's metadata database.
Nothing leaves: the store is yours, the evidence queries run against your data through the same read-only
probe path every other query uses, and a datasource delete cascades its facts when the object itself is gone.
The kinds of fact
| Kind | Scope | The fact it records |
|---|---|---|
unit | DATASOURCE | The unit of a numeric column |
time_zone | DATASOURCE | What a timestamp's wall-clock means |
sampling | DATASOURCE | The population relation of a table |
grain | DATASOURCE | One row is one what |
window | DATASOURCE | The data's coverage in time |
enum_meaning | DATASOURCE | What a coded value means |
join | DATASOURCE | How two tables relate |
caveat | DATASOURCE | A trap |
format | DATASOURCE | Encoding of a text column |
definition | WORKSPACE | A business measure or entity |
exclusion | WORKSPACE | What a business question leaves out |
preference | WORKSPACE | How this organisation wants a thing computed |
Questions this page gets
The semantic layer, in questions
Is this a data catalog?
No. A catalog lists what exists — tables, columns, owners. This records what it means: a unit, a time zone, a grain, what a code means, how tables join, what revenue excludes — each with the query that proved it. The introspection surface it rides is docs/datasources.md §7.
Can a wrong fact poison the agent?
That is what the trust ladder and the conflict rule are for: a fact is never higher-trust than its evidence, two live facts of the same kind are shown as a conflict — never a silent winner — and any fact can be retired with a reason. Facts are specified in docs/mcp-server.md §6.2.
Do I have to write anything?
No. The agent records what it learns while it works; you verify when you want to. Recording needs an author-scoped key and a release is still a person's act — the same bar as authoring a pipeline (docs/auth.md §7.6).
What happens when my schema changes?
Drift is detected at read time: a renamed or dropped column flips its facts to stale, shown beside the current columns — nothing is silently re-mapped to a new column; someone re-verifies and records the successor. The drift rule is docs/metadata-db.md §4.18.
Does it work on my own warehouse?
Yes — the skill and the tools are demo-free, and the facts come from whatever your agent probes: Postgres, Oracle, SQL Server, MySQL, H2, DuckDB, SQLite and dp-lake (docs/datasources.md §4).
The engineering picture is how it works; the sample data it learns on is the demo data; what ships next is on the roadmap.