Reporting for many clients, without thirty spreadsheets
A client reporting API per client: one pipeline each, one key each, a release you can invoice
An agency's reporting is a map of other people's databases: a Postgres here, a client's SQL Server there, a lake for the third. The work is the same shape every time — join their sources, produce their numbers, put those numbers in front of them, prove where the number came from. datapipelines gives each client its own workspace, its own key, an agent that drafts the pipeline, and a release your team signs — which becomes the API the client's portal calls.
Isolation by construction
One workspace per client, one key per client
Every client gets a workspace, and a key is pinned to one workspace at issuance: the analyst (or the agent) working client A sees nothing of client B — the other workspace's pipelines, datasources and executions are absent, not merely hidden. Cross-client leakage by a mistyped id resolves as not-found, which is the property you want when the clients are each other's competitors.
The client-facing half is a second kind of key: bound to a path prefix under /api/x, it can
call that client's endpoints and nothing else. Handing client A their key never touches client B's, and
revoking one is a checkbox — no shared password to rotate across the roster.
Isolation is also the answer to the question agencies ask first, which is not technical: what happens when one client's data must never influence another's numbers? Here the answer is structural — the workspaces do not share datasources unless you deliberately bind a global one, the agent's key cannot see across them, and the endpoint keys cannot reach another prefix. There is no "careful not to touch the other tab" step in the runbook, because the isolation is not a convention the team follows; it is the shape of the system they are working in.
The first draft, automated
The agent does the first draft of every client's pipeline
The repetitive half of agency work is walking a new client's schema: what tables exist, what the columns hold, how their "orders" differs from the last client's. The agent does that walk with the same tools on every engagement — schemas, catalogue statistics, a bounded probe — and drafts the pipeline: one node per source, the join in the per-run scratch database, the client's definition of "active customer" as the filter your reviewer reads before anything ships. The draft is the deliverable your team reviews; the variance between clients becomes reading, not re-typing. Engagement number four costs a fraction of engagement number one, because the ritual is identical and only the schemas are new.
you › Which rideshare company carried the most trips in each borough last quarter?
- read the datasource facts, columns and stats — hvfhv_zone_day is a census at zone × day × company
- resolved "last quarter" from the data's last day — Q4 2024
- rendered 3 templates, ran the draft: 4 nodes · 763 ms
- draft demo/top_company_by_borough v1 — left for a human to release
| borough | top company | trips | borough trips | share |
|---|---|---|---|---|
| Manhattan | Uber | 17,660,839 | 23,669,163 | 74.62% |
| Brooklyn | Uber | 11,946,362 | 16,057,699 | 74.40% |
| Queens | Uber | 9,786,602 | 12,847,231 | 76.18% |
| Bronx | Uber | 6,078,750 | 7,572,899 | 80.27% |
| Staten Island | Uber | 715,593 | 918,360 | 77.92% |
GET /api/x/demo/top-company-by-borough?anchor_date=2025-01-01 · after release · key scoped to /demo
from the demo workspace — a real result, not a promise
The engagement, end to end
Onboarding a new client, as a ritual
-
Open the workspace
One workspace per client, created in minutes. Everything that follows — datasources, pipelines, releases, keys — lives inside it, invisible to every other workspace.
-
Register the client's sources, read-only
Each database the client's reporting reads becomes a datasource, marked read-only unless the engagement explicitly includes writing. Credentials are encrypted on the server; the agent never sees one.
-
Mint the working keys
One key per agent and per analyst, scoped to the smallest set that does the job, pinned to this workspace, revocable without touching any other client's access.
-
Let the agent draft, run, and compare
Describe the client's reporting in business terms; the agent drafts the pipeline, runs it against the client's sources, and you compare the numbers with the report it replaces.
-
Release, publish, hand over the key
A person releases the version the engagement agrees on; the endpoint is published; the client-facing key is bound to its path. The invoice can name the version.
Sign-off with a version number
A release per client, with a version you can name in the invoice itself
The report a client disputes is the report you can defend: the pipeline that produced it was released — by a person on your team, as an immutable version — and the endpoint serves the version you point it at. "The numbers changed" becomes "version 4 was the deliverable; here is its diff", which is a different conversation. Agents cannot release; the sign-off is structurally yours.
It also changes the retainer conversation. "Maintenance" stops being a euphemism for re-running things by hand: a schema change upstream is a new draft your agent produces, a diff your reviewer reads, and a release your client's invoice can reference. The engagement's artefacts are versioned all the way down — which is what "professional services" was always supposed to mean.
Today and planned
What you embed in the client's portal: the API today, the dashboard planned
Ships today The endpoint in their portal
- The client's white-label page calls
GET /api/x/…with their key; the query string binds the report's parameters — month, region, account. Cache-Control: no-store: every call is a fresh run of the released version, so the portal never shows a stale cached copy.- The key reaches that path and nothing else; the contract is enforced by the router, not by a promise.
Planned The dashboard beside it
- Embedded dashboards are the planned roadmap item: created by your agent, embedded in the client's product, fed by released pipelines, filtered per viewer.
- Until it ships, the endpoint is the embed — a portal table or chart that reads the client's JSON is an afternoon of their front-end.
- Either way, the isolation holds: the dashboard, like the key, belongs to one client's workspace and stops at its boundary.
- The Tableau roadmap page carries the same plans, labelled as unshipped.
What it looks like
The client's endpoint, the key, the version
| endpoint | serves | key reach |
|---|---|---|
GET /api/x/demo/top-company-by-borough?anchor_date=2025-01-01 |
demo/top_company_by_borough · v1 | /demo/** |
from the demo workspace — released by a human, served as this version forever
One more property matters at review season: the audit log. Every call a client's key made, every release a teammate signed, every agent action on every workspace — recorded with the actor and the surface, and never the data itself. When a client asks "who looked at our numbers and when", the answer is a query over your own deployment's log, not a search through inboxes. The same log answers the earlier question too — which version of the report was live on which date — because releases and calls land in one record your agency owns outright.
Asked before
Agency questions, answered plainly
Can each client be kept separate from the others?
Yes — one workspace per client. A key is pinned to one workspace at issuance, so an agent or a colleague working client A sees nothing of client B: the other workspace's pipelines, datasources and executions are absent, not hidden. Scopes and pinning are docs/auth.md §7.5.
Can a client's key reach only that client's numbers?
Yes — endpoint keys are bound to a path prefix under /api/x, so the key you hand a client calls its endpoints and nothing else, and you revoke it without touching anyone else's access. Key kinds and bindings are docs/auth.md §7.7.
Who does the work, and who signs it off?
Your agent drafts each client's pipeline over the same authoring surface every user gets, and a person on your team releases it: agents cannot release, and a released version is immutable — which is the version you can name in the invoice. The lifecycle is docs/versioning.md §3.
What do we put in the client's portal today?
The API: a released pipeline published as a GET endpoint returns JSON with the query string bound to its declared parameters — the thing a portal, a scheduled report or a white-label page calls. Embedded dashboards are the planned roadmap item. The contract is docs/rest-api.md §19.
Pilot it on one real client this week
One workspace, one datasource, one pipeline, one release, one endpoint: the smallest engagement-shaped slice of the demo proves the flow before you point it at a real client's database. The same ritual scales from the pilot client to the roster without changing shape — only the workspace names change.