How-to · about two hours the first time
Publish a governed dataset for Tableau from Postgres, MySQL and S3, without a warehouse
The goal is a dataset Tableau reads that a person released, an agent maintains, and no one hand-copies between systems. Six steps, each one a feature that ships today. The example is the demo's own data, taxi trips in Postgres, weather in MySQL, zones in SQLite and rideshare trips as Parquet on S3, so you can run every step on a laptop before you point it at your databases.
The steps
From four sources to one dataset Tableau trusts
-
Register the sources, read-only
Each database and the bucket becomes a datasource: a name, a dialect, a connection, credentials the server encrypts and never returns. Mark the ones the dataset only reads as
readonly; a pipeline that tries to write to them is refused at save time, before anything runs. The agent will see the names and the schemas; it will never see a password. -
Ask the agent for the dataset
With the MCP server added to Claude Code or Cursor, describe the dataset the way you would to an analyst: “rideshare share of trips per borough, split by rainy and dry days, for the last quarter.” The agent reads the schemas, the catalogue statistics and the indexes, drafts a pipeline with one node per source and a join in the scratch database, runs it, reads the failure when there is one, and fixes the shape. It leaves a draft.
claude mcp add --transport http datapipelines https://your-host/mcp \ --header "DP-API-Key: dpk_<id>.<secret>" -
Review the draft, then release it
Open the draft in the editor: the graph, each node's SQL, the result of the last run. Run it yourself. When the numbers are the numbers, release it. A released version is immutable, so the next change becomes a new draft, and an agent cannot release by design. The dataset Tableau will read is now a version with a number and a name on it.
-
Publish the endpoint, or write the table
Endpoint: publish the released pipeline under a path such as
/api/nyc/v1/rain-share. AGETruns it with the query string bound to its declared parameters and returns the rows as JSON; the response is versioned with the pipeline.Table: if Tableau already connects to one of your databases, add a write-back node so the pipeline's last step lands the result in a table there. Tableau live-connects to the table it already knows; each pipeline run refreshes it. No connector to learn, and the dataset still comes from a released version.
-
Scope a key to the path
Issue an API key and bind it to the endpoint's path prefix. That key can call
/api/nyc/v1/…and nothing else, not the product's own API and not another team's endpoints. Give it to the workbook, the partner, or the service that needs the dataset.curl -s "https://your-host/api/nyc/v1/rain-share?quarter=2024-Q3" \ -H "DP-API-Key: dpk_<id>.<secret>" -
Connect Tableau
For the endpoint, a web data connector that reads JSON over HTTP with a header is all Tableau needs; for the table, a live connection you already have. Either way the dataset behind the chart is a released pipeline version, traceable, restorable, and maintained by an agent that reads the same specs you do.
Why this shape
Governance is a release step, not a permission matrix
Most “governed dataset” stories end in a certification badge someone clicks. Here the governance is structural: the dataset is a version that a human released, that no one can edit in place, and that an endpoint serves by number. When a chart is questioned, the answer is a version, not a conversation about who changed the flow.
Access is structural too. Keys are scoped to endpoint paths, so the surface each consumer sees is exactly the API you published, the way you would design it if you were building the API by hand, without building the API by hand.
And there is still no warehouse. The join happens in a scratch database that exists for the run; nothing is copied anywhere permanent unless you chose a write-back. When scheduled extracts arrive, they are planned to land on your own bucket. The same principle, one step further.
What each step looks like
The screens you will actually see
| datasource | what it is | access |
|---|---|---|
| sample-lake | Parquet on S3, read in place | read-only ✓ |
| sample-reference | SQLite | read-only ✓ |
from the demo workspace. The agent sees the schema, never the credential
The release itself stays a person's step: the draft carries its pins until a human releases it. The home page's pipeline shows the same state, a DRAFT bar waiting on that one decision, and then the endpoint answers as that version forever.
Run it on the demo first
./app.sh --start --demo nyc,lake brings up the four sources and the pipeline that joins them. Then
the six steps above are a two-hour afternoon, and the third one is the only one that needs you.