Transformations
Deduplicate by Key
Keeps one row per key — or the first few — and drops the other rows that share that key, even when their other columns differ.
When to use it
Use it for "latest record per customer" or "first order per account". Remove Duplicates drops only rows that match in every chosen column without choosing which copy survives; this node lets you choose.
In a pipeline
File InputDeduplicate by KeyExport CSV
Settings
| Setting | Type | Required | Default or hint |
|---|---|---|---|
| Key columns | select | Yes | Select key columns… |
| Keep | select | — | first |
| Rows per key (N) | number | — | 1 |
| Order by column first (ascending, optional) | select | — | Arrival order |
Example
Latest address per customer
- Key columns=CustomerId
- Keep=Last row per key
- Order by column first (ascending, optional)=Updated
| CustomerId | City | Updated |
|---|---|---|
| C-7 | Leeds | 2026-03-01 |
| C-9 | York | 2026-01-15 |
| C-7 | Hull | 2026-08-20 |
| CustomerId | City | Updated |
|---|---|---|
| C-9 | York | 2026-01-15 |
| C-7 | Hull | 2026-08-20 |
The node reports how many rows it removed, e.g. "Removed 1 of 3 row(s) as duplicates of CustomerId."
Worth knowing
- Order by always sorts ascending, so pair it with Last to keep the newest or largest, and with First to keep the oldest or smallest.
- Without Order by, First and Last mean the order the rows arrived in.
- First N rows per key keeps up to N rows for each key — the three latest orders per customer, for example.
- Key cells are compared as text, so 7 and "7" are the same key and blank keys are grouped together.