Data Quality
Data Profile
Summarises every column — how many blanks, how many distinct values, the smallest and largest value and the commonest pattern — as a table with one row per column.
When to use it
Use it first on any new or unfamiliar data, and on a schedule to notice when a feed starts arriving with gaps or a changed format.
In a pipeline
File InputData ProfileExport Excel
Settings
| Setting | Type | Required | Default or hint |
|---|---|---|---|
| Columns to profile | select | — | None picked: every column is profiled |
| Rows to sample | number | — | 10000 |
| Detect value patterns | checkbox | — | true |
Example
Profile of a customer list
- Rows to sample=10000
| ID | Amount | |
|---|---|---|
| C-1 | ann@example.com | 120.5 |
| C-2 | — | 80 |
| C-3 | bob@example.com | 310 |
| Column | Rows profiled | Null / blank | Null % | Distinct | Min | Max | Sample | Pattern |
|---|---|---|---|---|---|---|---|---|
| ID | 3 | 0 | 0.0% | 3 | C-1 | C-3 | C-1 | text |
| 3 | 1 | 33.3% | 2 | ann@example.com | bob@example.com | ann@example.com | ||
| Amount | 3 | 0 | 0.0% | 3 | 80 | 310 | 120.5 | decimal |
Worth knowing
- The profile is ordinary rows, so you can filter it (Null % above 5%), sort it or export it like any other table.
- Rows to sample profiles the first N rows (10,000 by default) to keep huge tables quick; 0 profiles every row. The note says when a sample was used.
- Min and Max order values the way Sort Rows does — numbers as numbers, dates as dates, text alphabetically.
- Pattern is the commonest shape among the values (integer, decimal, date, email, url, phone, hex32 or text), with its share when not every value has it.