The Transform workbench lets you reshape raw data into analysis-ready datasets without writing procedural code. You compose an ordered sequence of declarative steps on top of a connected datasource, see the result on live data as you build, and commit a versioned pipeline that downstream charts, reports, and natural-language queries can rely on. This reference walks through opening the workbench, building a pipeline step by step, every rule type available, and the practices that keep pipelines maintainable in production.
What a Pipeline Is
A pipeline is an ordered list of transform steps applied to a single datasource. Each step takes the output of the previous step as its input, so the order of steps matters: a filter that runs before an aggregate produces a different result than the same filter running after it. Because steps are declarative, the workbench can re-evaluate the entire chain instantly against a sample of your data, and it can store the whole definition as a portable, human-readable file.
Pipelines are non-destructive. They never mutate the source system. The original tables in your database, warehouse, or file connector remain untouched; the pipeline defines a transformation that is materialized only when you commit it as a transformed dataset.
Opening the Transform Workbench
- Select a connected datasource from your workspace. Any of the 24 supported connectors can be used as a pipeline source, including databases reached directly inside your network.
- Open the Transform workbench for that datasource. The workbench loads a sample of rows so you have immediate context for the columns, types, and value distributions you are working with.
- Choose the starting table. If the datasource exposes multiple tables, you pick the primary table the pipeline begins from; later join steps can pull in additional tables.
The workbench is divided into three regions: the step list on the left where you add and reorder rules, the configuration panel in the center where you set the parameters for the selected step, and the preview grid below where roughly 1,000 rows of the current result are shown.
Building a Pipeline Step by Step
Building a pipeline is iterative. A typical flow looks like this:
- Add a step. Click to add a rule and choose its type from the rule list.
- Configure it. Set the columns, conditions, or expressions the rule needs. The configuration panel adapts to the rule type you chose.
- Inspect the preview. The preview grid re-runs the full chain on the ~1,000-row sample and shows the result of every step up to and including the one you are editing.
- Reorder if needed. Drag steps to change their position. Because each step feeds the next, reordering changes the result, and the preview updates so you can confirm the new order does what you intend.
- Repeat until the shape of the data matches your target, then commit.
Keep steps small and single-purpose. A pipeline made of many focused steps is far easier to debug than a few steps that each try to do several things at once.
Rule Types
The workbench supports the following rule types. Most pipelines combine a handful of them.
| Rule | What it does | Typical use |
|---|---|---|
| filter | Keeps only rows that match a condition | Drop nulls, restrict to a date range, exclude test records |
| sort | Orders rows by one or more columns, ascending or descending | Prepare top-N views, make output deterministic |
| rename | Changes a column's name | Standardize naming, clarify ambiguous fields |
| aggregate | Collapses rows using SUM, AVG, COUNT, MAX, or MIN | Totals, averages, record counts per category |
| join | Combines columns from another table on matching keys | Enrich transactions with customer or product attributes |
| group | Groups rows by one or more columns for aggregation | Revenue by region, orders by month |
| pivot | Rotates row values into columns | Turn a month column into one column per month |
| unpivot | Rotates columns into rows | Normalize wide spreadsheet-style data into long form |
| flatten | Expands nested or structured fields into flat columns | Unpack JSON-like or hierarchical fields |
| type-cast | Converts a column's data type | Parse strings to dates, text to numeric |
| derive | Creates a new column from a custom expression | Margins, full names, bucketed values, flags |
filter
A filter narrows the row set by evaluating a condition per row. Place filters early when you want to reduce data volume before expensive steps like joins or aggregates, which also makes the preview more representative.
sort
Sorting orders the result by one or more keys. Sorts are commonly paired with downstream consumers that expect a stable order, or used to build top-N style outputs in combination with a filter.
rename
Renaming columns is cosmetic but important. Clear, consistent names make a transformed dataset self-documenting for analysts and improve the quality of natural-language query results, since the AI reasons about column names.
aggregate and group
Aggregation reduces many rows to summary values using SUM, AVG, COUNT, MAX, or MIN. In practice you pair group with aggregate: group by the dimensions you want to keep (such as region and month), then aggregate the measures (such as summing revenue). The result has one row per group.
join
A join brings in columns from a second table by matching keys. You choose the join keys and the join behavior. Joins are where relationship inference (below) saves the most effort, because the workbench can propose the correct keys for you.
pivot and unpivot
Pivot and unpivot are inverses. Pivot turns distinct values in a column into separate columns, useful for building cross-tab style outputs. Unpivot does the reverse, collapsing many columns into key/value row pairs, which is the right move when a source arrives in wide spreadsheet format and you need a normalized long shape for charting.
flatten
Flatten expands structured or nested fields into ordinary flat columns so the rest of the pipeline can treat them like any other field.
type-cast
Type casting converts a column from one type to another, for example parsing text into proper dates or converting numeric-looking strings into numbers so aggregates behave correctly.
derive
Derive adds a computed column from a custom expression. This is the most flexible rule: compute a profit margin from revenue and cost, concatenate first and last names, bucket a continuous value into bands, or set a boolean flag from a condition.
-- derive: margin_pct
(revenue - cost) / revenue * 100
Using Real-Time Preview
Every change you make re-evaluates the pipeline against a sample of roughly 1,000 rows and updates the preview grid before anything is committed. This tight feedback loop is the core of the no-code experience: you can confirm that a filter caught the right rows, that a join did not unexpectedly multiply rows, or that a derive expression produced sensible values, all without running the transform against the full dataset.
Treat the preview as a correctness check at each step, not just at the end. Catching a bad join or a type-cast failure when it happens is far cheaper than discovering it after the dataset has been published and consumed.
AI Step Suggestions
You can describe what you want in plain language and let the AI suggest the transform steps. For example, you might type "group sales by region and month, then keep only regions with total revenue above 100k." The AI proposes the corresponding sequence of steps, which you then review, adjust, and accept. The suggestion is a starting point you stay in control of: nothing is applied until you accept it, and you can tweak any proposed step in the configuration panel before committing.
This is most useful for getting a rough pipeline scaffolded quickly, or for discovering the right combination of group, aggregate, and filter steps when the desired shape is clear but the mechanics are not.
Relationship Inference for Smart Joins
When you add a join, relationship inference automatically detects likely foreign-key relationships across the tables in your datasource and proposes the matching keys. Instead of manually hunting for which column in the orders table maps to the customers table, the workbench suggests the join condition for you. You can accept the inferred keys or override them. This dramatically reduces the friction of multi-table pipelines and lowers the chance of joining on the wrong columns.
DAX for Power BI Datasources
When the connected datasource is a Power BI source, DAX evaluation is available. This lets you leverage DAX expressions where they are the natural fit for the underlying model, alongside the standard rule types, so pipelines built on Power BI datasources can use the modeling semantics that source already understands.
TOML Storage and Versioning
Every pipeline is stored as TOML — a readable, diff-friendly text format — and each pipeline is versioned. Because the definition is plain text, you can review exactly what a pipeline does, compare versions to see what changed, and trust that the same definition produces the same shape every time it runs. Versioning means you can evolve a pipeline with confidence, knowing prior versions are retained.
A simplified pipeline definition looks like this:
[pipeline]
name = "regional_revenue"
source = "sales_db"
[[step]]
type = "filter"
column = "order_status"
op = "equals"
value = "completed"
[[step]]
type = "join"
table = "customers"
left_key = "customer_id"
right_key = "id"
[[step]]
type = "group"
by = ["region", "order_month"]
[[step]]
type = "aggregate"
column = "amount"
func = "SUM"
as = "total_revenue"
[[step]]
type = "filter"
column = "total_revenue"
op = "greater_than"
value = 100000
[[step]]
type = "sort"
by = "total_revenue"
order = "desc"
The TOML mirrors the step list exactly: each [[step]] block is one rule, in execution order.
Committing and Publishing a Transformed Dataset
When the preview matches your target shape, commit the pipeline to materialize a transformed dataset. Once published, that dataset becomes a first-class source across the platform: you can build charts on it, schedule auto-reports against it, and ask natural-language questions of it. Because the pipeline is versioned, the published dataset is tied to a known definition, so you always know how it was produced.
Best Practices
- One concern per step. Many small steps beat a few overloaded ones. They preview faster and are far easier to debug.
- Filter early. Reduce row volume before joins and aggregates to keep previews representative and the committed run efficient.
- Cast before you aggregate. Make sure numeric and date columns are the right type before summing or grouping, or results will be wrong.
- Verify joins in the preview. Watch the row count after a join; an unexpected increase usually means a key is not unique.
- Let inference propose, then confirm. Accept inferred join keys only after checking they reflect the real relationship.
- Use AI suggestions as a draft. Review every proposed step before committing rather than accepting blindly.
- Name columns clearly. Good names make the dataset self-documenting and improve natural-language query accuracy.
- Review the TOML diff. Before publishing a new version, read the TOML to confirm the change is exactly what you intended.
With these practices, the Transform workbench turns raw, multi-table data into clean, versioned datasets that the rest of your analytics stack can build on with confidence.