Same job, different artefact

Tableau Prep alternative: a flow is a file, a pipeline is code you can release

Tableau Prep and a datapipelines pipeline do the same first job — pull several sources together, shape them, hand the result to the next step. The difference is the artefact that job produces. Prep produces a flow: a document one tool opens, edits and runs. A pipeline here is a JSON document of SQL nodes: diffable in a pull request, versioned, runnable by an AI agent, releasable only by a person, and publishable as an API. This page compares them honestly, including the part Prep does better.

The artefact

Same job, different artefact — and the artefact decides the second week

A Prep flow is a diagram you edit. A pipeline here is a JSON document of SQL nodes with explicit dependencies, which happens to render as a diagram. The distinction sounds academic until the second week, when the questions change from "can I build it?" to "can I review it, diff it, hand it to someone else, and say which version of it produced the number on the board?". The flow answers those in person and from memory. The pipeline answers them structurally: the diff is a pull request, the review is a draft a person can run before releasing, and the version is immutable once released.

Both join across sources without a warehouse. Prep joins in memory on the machine running the flow; a pipeline stages each source's rows into a scratch database created for the run and joins them there, with ordinary SQL any reviewer can read. When a source is slow, the difference gets loud: the pipeline's node carries a query timeout, and the SQL probe returns the plan, so the agent fixes the shape from evidence rather than from a spinner.

There is a quieter difference underneath, and it decides who can help you. The flow's knowledge lives in the person who built it and in the file only that tool opens; hand the work to a colleague, a contractor or an agent, and the first week is archaeology. The pipeline's knowledge lives in the document: the nodes are named for what they do, the SQL is in the file, the dependencies are declared — so the second person (or the second agent) starts from reading, not from reverse-engineering. Teams do not adopt pipelines because the first build is faster; often it is not. They adopt them because the second week, the second person and the second year are all cheaper.

The list

Six things a flow cannot do

Diff in a pull request

The pipeline is JSON referencing named SQL templates; a change is a line-level diff a reviewer reads before it merges. A flow file changes wholesale and invisibly.

Carry a draft a reviewer runs

The agent's work lands as a draft. A person runs it, reads the rows, and releases it or not. The released version is immutable; the next edit becomes a new draft.

Be written by an agent

The whole authoring surface is MCP tools: schemas in, catalogue statistics and indexes read, the draft created and iterated, the run inspected. A flow has no interface an agent can work through.

Show why a node is slow

Every statement carries a query timeout, and the SQL probe captures the plan before the query runs — so a slow node names itself, with a plan, instead of being a spinner in someone else's UI.

Write its result back

A node can land its result in a datasource — the Postgres table a BI tool live-connects to. The flow's output stays where the flow puts it.

Publish as an API

Release the pipeline, publish the endpoint: a versioned GET with bound parameters and a key scoped to its path. That is the step a flow has no answer for.

The honest column

What Prep does better

Visual profiling — the distributions, the nulls, the value counts a flow shows while you drag — is a real strength, and this server does not ship an equivalent: the agent reads catalogue statistics and the probe's rows instead, which is evidence, not a histogram. The click-to-pivot is faster than writing the SQL by hand if you do not have an agent to write it. And Tableau-native output — the flow that lands directly where a Tableau author expects — is friction this does not attempt to match; the honest path here is the write-back node or the endpoint.

If the preparation is a one-off reshape by a person who lives in Prep, none of the six above pays for the move. The pipeline earns its keep when the preparation is repeated, reviewed, handed between people and machines, or served — which is the moment a file one tool opens stops being enough.

The move

Migrating a flow, step by step

Pick the flow you would least like to rebuild from scratch next quarter — the weekly one, with the joins someone has memorised. That flow pays for the migration twice: once in the move, and once every week it no longer depends on the person who built it. The five steps below are each a shipped feature; the whole sequence is an afternoon for a flow of ordinary size.

  1. Inventory the flow's sources

    List every input the flow touches — databases, files, the join keys between them. This list is the migration's scope; most flows are smaller than they look once the inputs are named.

  2. Register them as datasources

    Each source becomes a datasource: name, dialect, connection, credentials the server encrypts and never returns. Mark read-only every source the pipeline must never write to — the refusal is enforced at save time and again at run time.

  3. Describe the transformation to the agent

    The flow's steps are a spec: describe the shape in business terms. The agent reads the real schemas and catalogue statistics, drafts one node per source plus a join in the scratch database, runs it, and reads the failure when there is one.

  4. Run it and compare against the flow's output

    Run the flow once more and the pipeline once, and compare row counts and spot totals. The pipeline's result is paged rows you can read in the results dock — the comparison is a morning, not a project.

  5. Release it

    When the numbers match, a person releases. The flow on the shared drive can retire; the pipeline is versioned, diffable, and — the step Prep never had — publishable as an endpoint.

What it looks like

The pipeline a flow became

The editor: the flow's reshape as a graph of SQL nodes, each one readable, the result dock open.
SOURCES STAGING · H2 IN MEMORY ANSWER sample-lake · Parquet on S3 hvfhv_zone_day · 377k rows sample-reference · SQLite zones · 265 rows calculator · trailing_periods anchor 2025-01-01 → Q4 2024 stage_company_zone 523 rows · 482 ms stage_zones 263 rows · 204 ms answer 6 rows · caller DRAFT demo/top_company_by_borough · v1 · awaiting a human release · then GET /api/x/demo/…
The agent's session: schemas read, the draft iterated, the run verified — the authoring a flow file cannot accept.
workspace: demo · agent session

you › Which rideshare company carried the most trips in each borough last quarter?

  • read the datasource facts, columns and stats — hvfhv_zone_day is a census at zone × day × company
  • resolved "last quarter" from the data's last day — Q4 2024
  • rendered 3 templates, ran the draft: 4 nodes · 763 ms
  • draft demo/top_company_by_borough v1 — left for a human to release
boroughtop companytripsborough tripsshare
Manhattan Uber 17,660,839 23,669,163
Brooklyn Uber 11,946,362 16,057,699
Queens Uber 9,786,602 12,847,231
Bronx Uber 6,078,750 7,572,899
Staten Island Uber 715,593 918,360

GET /api/x/demo/top-company-by-borough?anchor_date=2025-01-01 · after release · key scoped to /demo

from the demo workspace — a real result, not a promise

Both panels are the demo workspace's run of record — demo/top_company_by_borough, a recorded acceptance result over the published sample data (the reference zones and the lake's ride ledger on S3), reproduced from the bucket. The same sentence, the same steps and the same numbers the home page shows.

Asked before

Prep questions, answered plainly

Is a pipeline here really code?

Yes: a pipeline is a JSON document of named SQL nodes with explicit dependencies, so it diffs in a pull request, carries a version number, and is reviewed the way code is. A flow file is a diagram only one tool opens. The contract is docs/pipeline-contract.md §3.

What does Tableau Prep still do better?

Visual profiling and the click-to-pivot are real strengths, and a one-off reshape by a person who lives in Prep is not a problem this needs to solve. The pipeline earns its keep when the preparation must be repeatable, reviewable and agent-authored — the authoring surface is docs/mcp-server.md §6.2.

Can an agent build the pipeline for me?

That is the intended workflow: the agent reads your schemas, catalogue statistics and indexes, drafts the pipeline, runs it, and reads the failure when there is one — then leaves a draft that a person releases. The tool surface is docs/mcp-server.md §6.2.

How do I migrate an existing flow?

In five steps, each a shipped feature: inventory the flow's sources, register them as datasources (read-only where nothing should write), describe the transformation to your agent, run the pipeline and compare row counts with the flow's output, then release. Registering datasources is docs/datasources.md §3.

Bring one flow over and compare

The demo stack gives you the four sources to practise on. Migrate the smallest flow you run weekly — the comparison against its output is the whole argument, whichever way it lands.