DataSet
DataSet manifests describe derived tables built using SQL or PRQL (Pipelined Relational Query Language).
Datasets typically select from DataSource views and may depend on other datasets.
Minimal definition
Section titled “Minimal definition”The smallest well-formed DataSet: the envelope, a name, a query and the datasources the query reads.
dependencies is not required by the schema, but without it the dataset cache does not notice when the underlying file changes, so set it whenever the query reads a datasource.
apiVersion: bino.bi/v1alpha1
kind: DataSet
metadata:
name: sales_by_region
spec:
query: |
SELECT
region AS category,
row_number() OVER (ORDER BY region) AS categoryIndex,
SUM(amount) AS ac1
FROM sales_csv
GROUP BY region
dependencies:
- sales_csvInstead of query you can write the transformation in PRQL (spec.prql) or skip it entirely with a direct source pass-through (spec.source).
All attributes are listed in the Attribute Reference below.
Fields:
spec.query– SQL query, either inline or loaded from an external file. Required ifspec.prqlandspec.sourceare not set.spec.prql– PRQL query, either inline or loaded from an external file. When set, takes precedence overspec.query. The PRQL is sent directly to the query engine which compiles it via the prql extension.spec.source– direct pass-through to a DataSource without transformation. Mutually exclusive withqueryandprql.spec.dependencies– optional list of datasource references. Can be string names or inline DataSource definitions. Inline definitions are referenced via@inline(N)syntax in queries.
External query files
Section titled “External query files”For complex queries, you can store them in separate .sql or .prql files and reference them using the $file syntax. This keeps your YAML manifests clean and enables syntax highlighting in your editor.
Loading SQL from an external file
Section titled “Loading SQL from an external file”apiVersion: bino.bi/v1alpha1
kind: DataSet
metadata:
name: sales_summary
spec:
query:
$file: ./queries/sales_summary.sql
dependencies:
- sales_csvLoading PRQL from an external file
Section titled “Loading PRQL from an external file”apiVersion: bino.bi/v1alpha1
kind: DataSet
metadata:
name: customer_orders
spec:
prql:
$file: ./queries/customer_orders.prql
dependencies:
- orders
- customersFile path resolution
Section titled “File path resolution”- Paths are resolved relative to the manifest file containing the
$filereference - Both
./relative/path.sqlandrelative/path.sqlformats are supported - Absolute paths are also supported but not recommended for portability
Auto-reload and caching
Section titled “Auto-reload and caching”External query files are fully integrated with bino's caching and hot-reload system:
- Cache invalidation: When an external SQL or PRQL file changes, the dataset cache is automatically invalidated
- Hot-reload in preview: Changes to external query files trigger automatic refresh in
bino previewmode - Dependency graph: External files appear in
bino graphoutput for visibility
Organizing query files
Section titled “Organizing query files”A recommended project structure for external queries:
my-report/
├── bino.toml
├── manifests/
│ ├── datasources.yaml
│ └── datasets.yaml
├── queries/
│ ├── sales_by_region.sql
│ ├── monthly_revenue.sql
│ └── customer_analysis.prql
└── data/
└── sales.csvSimple aggregation
Section titled “Simple aggregation”---
apiVersion: bino.bi/v1alpha1
kind: DataSource
metadata:
name: sales_csv
spec:
type: csv
path: ./data/sales.csv
---
apiVersion: bino.bi/v1alpha1
kind: DataSet
metadata:
name: sales_by_region
spec:
query: |
SELECT
region,
SUM(amount) AS total_amount
FROM sales_csv
GROUP BY region
dependencies:
- sales_csvDataset depending on multiple inputs
Section titled “Dataset depending on multiple inputs”---
apiVersion: bino.bi/v1alpha1
kind: DataSource
metadata:
name: sales_fact
spec:
type: parquet
path: ./warehouse/fact_sales/*.parquet
---
apiVersion: bino.bi/v1alpha1
kind: DataSource
metadata:
name: currency_rates
spec:
type: csv
path: ./data/currency_rates.csv
---
apiVersion: bino.bi/v1alpha1
kind: DataSet
metadata:
name: revenue_eur
spec:
query: |
SELECT
f.region,
f.booking_date,
f.amount * r.rate_to_eur AS revenue_eur
FROM sales_fact f
JOIN currency_rates r
ON r.currency = f.currency
AND r.valid_on = f.booking_date;
dependencies:
- sales_fact
- currency_ratesDatasets are referenced from layouts via their metadata.name using the dataset field of charts, tables, and text components.
Using PRQL
Section titled “Using PRQL”PRQL is a modern language for transforming data that compiles to SQL. It offers a more readable, pipeline-based syntax compared to traditional SQL.
When you specify spec.prql, bino automatically loads the PRQL extension and sends your PRQL query directly to the query engine for compilation and execution.
Simple PRQL example
Section titled “Simple PRQL example”---
apiVersion: bino.bi/v1alpha1
kind: DataSource
metadata:
name: sales_csv
spec:
type: csv
path: ./data/sales.csv
---
apiVersion: bino.bi/v1alpha1
kind: DataSet
metadata:
name: sales_by_region
spec:
prql: |
from sales_csv
filter amount > 0
group {region} (
aggregate {
total_amount = sum amount,
order_count = count this
}
)
sort {-total_amount}
dependencies:
- sales_csvPRQL with joins and transformations
Section titled “PRQL with joins and transformations”---
apiVersion: bino.bi/v1alpha1
kind: DataSource
metadata:
name: orders
spec:
type: csv
path: ./data/orders.csv
---
apiVersion: bino.bi/v1alpha1
kind: DataSource
metadata:
name: customers
spec:
type: csv
path: ./data/customers.csv
---
apiVersion: bino.bi/v1alpha1
kind: DataSet
metadata:
name: customer_orders
spec:
prql: |
from orders
join customers (==customer_id)
derive full_name = f"{customers.first_name} {customers.last_name}"
select {
order_id,
full_name,
order_date,
total
}
sort {-order_date}
take 100
dependencies:
- orders
- customersFor more PRQL syntax and examples, see the PRQL documentation.
Standard Dataset Schema
Section titled “Standard Dataset Schema”To work correctly with visualization components like Charts and Tables, your dataset queries should return rows that conform to the standard schema. While not all fields are required for every component, following this structure ensures proper aggregation, drill-down, and filtering behavior.
Measure Values (Scenarios)
Section titled “Measure Values (Scenarios)”The schema supports four parallel sets of scenario columns for comparative analysis (e.g., Actual vs Plan).
| Column | Description |
|---|---|
ac1 ... ac4 | Actual values (current measurements) |
pp1 ... pp4 | Previous Period values. What a slot compares against (prior month, prior year, …) is declared per dataset with derive/assert, see Deriving previous period |
fc1 ... fc4 | Forecast values (predictions) |
pl1 ... pl4 | Plan/Budget values (targets) |
Grouping Dimensions
Section titled “Grouping Dimensions”These fields determine how data is hierarchically organized and sorted.
| Dimension | Index Column | Description |
|---|---|---|
rowGroup | rowGroupIndex | Top-level row grouping (e.g., "Revenue", "Costs"). |
category | categoryIndex | Primary data dimension (e.g., "Product A", "Region North"). |
subCategory | subCategoryIndex | Detail dimension for drill-down (creates "thereOf" rows). |
columnGroup | columnGroupIndex | Column-level grouping for breaking down measures. |
columnSubGroup | columnSubGroupIndex | Second column level underneath columnGroup. |
Note: The two columns of a pair depend on each other in both directions. Provide the index (e.g., rowGroupIndex) whenever you provide the dimension string (e.g., rowGroup) to ensure consistent sort ordering — and likewise, an index column without its dimension string is reported by data validation.
Metadata & Control Fields
Section titled “Metadata & Control Fields”| Column | Type | Description |
|---|---|---|
date | string | Required for TimeCharts. ISO 8601 date (e.g., 2024-01-15) or datetime (e.g., 2024-01-15T08:30:00Z). Used for time-series axes and normalization. |
operation | string | Aggregation sign: '+' (add) or '-' (subtract). Defaults to '+'. Useful for P&L structures where costs subtract from totals. |
setname | string | Dataset identifier. Used by some charts to distinguish multiple query results. |
Example Query
Section titled “Example Query”Here is an example SQL query producing a compliant dataset:
SELECT
'Revenue' as rowGroup,
1 as rowGroupIndex,
product_name as category,
product_rank as categoryIndex,
'2024-01-15' as date,
sum(sales_amount) as ac1,
sum(budget_amount) as pl1
FROM sales_data
GROUP BY product_name, product_rankBuilt-in SQL functions
Section titled “Built-in SQL functions”bino registers a few helper functions on the DuckDB engine — two scalar functions and one table macro — so they are available in every DataSet (and DataSource) query without any import or extension.
op(value) — sign as an operator
Section titled “op(value) — sign as an operator”op reduces a numeric value to its sign as a string, the shape the operation control field (documented above) expects.
Note that operation only accepts '+' and '-', so the empty string returned for 0 is reported by data validation — wrap the call (for example nullif(op(x), '')) when zero values can occur.
| Input | Result |
|---|---|
value > 0 | '+' |
value < 0 | '-' |
value = 0 | '' (empty string) |
NULL | NULL |
It accepts any numeric type (INTEGER, BIGINT, DECIMAL, DOUBLE, …) and returns VARCHAR.
SELECT
account_name AS category,
sum(amount) AS ac1,
op(sum(amount)) AS operation -- '+' for revenue, '-' for costs
FROM ledger
GROUP BY account_nameiop(value) — inverse sign
Section titled “iop(value) — inverse sign”iop is the inverse of op (+ becomes - and vice versa; 0 stays ''). Use it for inverted measures, where an increase is unfavorable — costs, headcount, error rates, churn — so a rising value reads as a negative signal.
| Input | op | iop |
|---|---|---|
value > 0 | '+' | '-' |
value < 0 | '-' | '+' |
value = 0 | '' | '' |
Absolute value
Section titled “Absolute value”There is no bino-specific helper for absolute values — DuckDB's built-in abs() already does it (abs(-500) → 500). Combine it with op to render a signed magnitude as text:
SELECT op(variance) || abs(variance)::VARCHAR AS signed_variance
-- variance = -500 -> '-500'
-- variance = 500 -> '+500'bino_shift(src, source, shift, grain) — previous period as a table macro
Section titled “bino_shift(src, source, shift, grain) — previous period as a table macro”bino_shift is the table macro behind derive and assert. It returns every row and column of src plus one column named shifted: the value of source on the row with the same identity one shift earlier, or NULL when there is none.
| Parameter | Value |
|---|---|
src | Name of a view or table in the session (a DataSource or DataSet name). A macro parameter cannot be a subquery. |
source | The slot to read from the earlier row: one of ac1…ac4, pp1…pp4, fc1…fc4, pl1…pl4 |
shift | '<n> <unit>', e.g. '1 year', '1 month', '2 week', '3 day', '1 quarter' |
grain | The period one row stands for: 'day', 'week', 'month', 'quarter' or 'year' |
The identity of a row is every column except date and the sixteen slot columns; two rows match when their identities are equal and the period of the earlier one (date truncated to grain) equals the period of the later one minus shift. The extra column is always called shifted — a macro cannot name an output column from a parameter — so alias it to the slot you want:
SELECT * EXCLUDE (shifted), shifted AS pp2
FROM bino_shift('sales_view', 'ac1', '1 year', 'month')Prefer derive on the DataSet unless you need the shifted value inside a larger query; the declaration also gives you the checks and the caption.
Deriving previous period
Section titled “Deriving previous period”A DataSet can declare what a previous-period slot means instead of supplying it. derive makes bino produce the column, assert checks a column the query already returns. Both take the same shape, keyed by the slot (pp1 to pp4):
apiVersion: bino.bi/v1alpha1
kind: DataSet
metadata:
name: sales_monthly
spec:
query: |
SELECT region, regionIndex, ac1, pl1, date FROM sales -- no pp column here
derive:
pp1: { from: ac1, shift: 1 month, grain: month } # previous month
pp2: { from: ac1, shift: 1 year, grain: month } # same month last year
assert:
pp3: { from: pl1, shift: 1 year, grain: month } # pp3 comes from the query; bino checks itEach declaration has three attributes, all required:
| Attribute | Value | Meaning |
|---|---|---|
from | one of ac1…ac4, pp1…pp4, fc1…fc4, pl1…pl4 | The slot whose value is copied from the earlier row. |
shift | <n> <unit>, n ≥ 1, unit day, week, month, quarter or year | How far back the earlier row lies. |
grain | day, week, month, quarter or year | The period one row of the dataset stands for. It is declared, never inferred from the dates. |
Grain says what a row is, shift says which earlier row to compare against: on a monthly table, shift: 1 month is the previous month and shift: 1 year the same month last year.
The declarations work with query, prql and source alike. A slot may appear in derive or in assert, not both (dataset-derive-conflict), and a derived slot must not be returned by the query. Two declared slots are independent of each other.
How a row is matched
Section titled “How a row is matched”For every row bino computes two keys:
- identity — the values of every column except
dateand the sixteen slot columns, so all dimension columns with their index twins,operationandsetname; - period —
datetruncated to the grain (date_trunc(grain, date::DATE); a week starts on Monday).
The slot receives the from value of the row that has the same identity and whose period equals this row's period minus shift, provided that period plus shift gives this row's period again. Without such a row the slot is NULL. With from: ac1, shift: 1 month, grain: month and region as the only dimension:
region | date | ac1 | period | period − shift | pp1 |
|---|---|---|---|---|---|
| North | 2024-01-31 | 100 | 2024-01 | 2023-12 | NULL — no such row |
| North | 2024-02-29 | 110 | 2024-02 | 2024-01 | 100 |
| North | 2024-03-31 | 120 | 2024-03 | 2024-02 | 110 |
| South | 2024-03-31 | 80 | 2024-03 | 2024-02 | NULL — South has no February row |
| West | 2024-02-29 | 70 | 2024-02 | 2024-01 | NULL — West is new in February |
| West | 2024-03-31 | NULL | 2024-03 | 2024-02 | 70 — added: West has no March row, its February value is kept visible |
What follows from the rule:
- Periods are compared, not dates, so month ends line up (
2024-03-31finds2024-02-29) and the day within the period does not matter. - Choose a shift that is a whole number of grains. A shift shorter than the grain (
1 weekon grainmonth) never matches. A month shift on graindaymatches by day of month, and the round-trip condition leaves the 31st empty when the prior month is shorter. - Every identity must have at most one row per period; duplicates are an error (see below).
- Rows before the first shifted period have no prior row: the first month of a table has no
pp1, its first year nopp2. If that is every row, the build warns. - Index twins and control columns are part of the identity:
categoryIndex,rowGroupIndex,operationandsetnamemust be the same in both periods. An index computed per period (a rank by value) changes the identity and breaks the match; compute indexes from a stable attribute. - An identity that is new in this period keeps its row with an empty slot.
- An identity that existed one shift earlier but has no row in this period gets a row: the dimension columns of the earlier row, every measure
NULL, the slot filled, and thedatethe data uses for that period. So a region that stopped selling still showsac1 = NULL, pp1 = 563and the drop stays visible. Rows are only added for periods the data covers; nothing is created after the last period.assertnever adds rows, it checks the rows the query supplied.
Caption. The engine captions pp1…pp4 as PY by default. That is right for a year shift; for any other unit bino emits an Internationalization bundle for the artefact's language that captions the slot PP. It is placed before your own manifests, so a project override of global.pp1 still wins.
Checks and diagnostics
Section titled “Checks and diagnostics”After the query runs, bino checks every row — not the validation sample — and reports through the same channel as data validation. Four of the checks are errors that fail bino build regardless of --data-validation, because a declared expectation is not a sampled type check:
| Check | Message | Outcome |
|---|---|---|
| Derived slot supplied | slot pp2 is already in the query result; use assert: for a supplied slot | error, fails the build even under --data-validation warn |
| Source column missing | slot ac1 (from of pp2) is missing from the query result, column "date" is missing from the query result, slot pp3 is asserted but missing from the query result | error, fails the build even under --data-validation warn |
| Duplicate identity in period | duplicate rows for identity region=North in period 2024-03-01 (grain month) | error, fails the build even under --data-validation warn |
| Assert mismatch | assert pp3: 4 row(s) differ from pl1 shifted by 1 year (grain month); first at identity region=North, period 2024-03-01 | error, fails the build even under --data-validation warn |
| Derived slot empty | pp2 derived from ac1 is null on every row — the query window has no prior period | warning |
The assert comparison tolerates relative noise below 1e-9; rows whose shifted value is NULL have nothing to compare and are skipped. --warn-on-query-errors downgrades the four errors like any other query failure, so preview and bino lint --execute-queries show them as warnings.
Data Validation
Section titled “Data Validation”bino can validate query results against the standard schema at build time. This catches common data issues early:
Validation checks
Section titled “Validation checks”| Check | Description |
|---|---|
| Type validation | String fields (category, rowGroup, etc.) must be strings |
| Type validation | Number fields (ac1, categoryIndex, etc.) must be numbers |
| Enum validation | operation must be "+" or "-" |
| Date format | date must be ISO 8601 (YYYY-MM-DD, or YYYY-MM-DDThh:mm:ss with an optional Z or ±hh:mm offset) |
| Dependent required | Dimension pairs must appear together in both directions: rowGroup requires rowGroupIndex and rowGroupIndex requires rowGroup, and likewise for category, subCategory, columnGroup and columnSubGroup |
The derive/assert checks are not part of this sampled validation: they run on every row, and their four error cases fail the build in every validation mode.
Enabling validation
Section titled “Enabling validation”Data validation is enabled by default in warn mode. Configure via CLI flags:
# Log warnings and continue (default)
bino build --data-validation=warn
# Treat validation errors as fatal
bino build --data-validation=fail
# Skip validation entirely
bino build --data-validation=offFor lint-only validation (without building), use --execute-queries:
bino lint --execute-queriesSample size
Section titled “Sample size”For efficiency, bino validates only the first N rows per dataset. Configure via environment variable:
BNR_DATA_VALIDATION_SAMPLE_SIZE=100 bino buildDefault sample size is 10 rows.
Conditional inclusion with constraints
Section titled “Conditional inclusion with constraints”DataSet documents support metadata.constraints to conditionally include them for specific artefacts, modes, or environments.
Preview-only datasets
Section titled “Preview-only datasets”Include debug or diagnostic datasets only during development:
apiVersion: bino.bi/v1alpha1
kind: DataSet
metadata:
name: debug_metrics
constraints:
- mode==preview
spec:
query: |
SELECT * FROM raw_data LIMIT 100
dependencies:
- raw_dataFormat-specific datasets
Section titled “Format-specific datasets”Use different datasets for different output formats:
# Detailed dataset for print (A4/Letter)
apiVersion: bino.bi/v1alpha1
kind: DataSet
metadata:
name: sales_summary
constraints:
- spec.format in [a4,letter]
spec:
query: |
SELECT region, product, month, revenue, costs, margin
FROM sales_fact
GROUP BY region, product, month
---
# Summarized dataset for screen (XGA)
apiVersion: bino.bi/v1alpha1
kind: DataSet
metadata:
name: sales_summary
constraints:
- field: spec.format
operator: "=="
value: xga
spec:
query: |
SELECT region, SUM(revenue) as total_revenue
FROM sales_fact
GROUP BY regionFor the full constraint syntax and operators, see Constraints and Scoped Names.
Inline DataSet definitions
Section titled “Inline DataSet definitions”Instead of defining DataSets as separate documents, you can define them inline directly within components. This is useful for simple, component-specific data transformations.
Inline DataSet in a Chart
Section titled “Inline DataSet in a Chart”apiVersion: bino.bi/v1alpha1
kind: ChartStructure
metadata:
name: sales_chart
spec:
dataset:
query: |
SELECT region, SUM(amount) as total
FROM sales_csv
GROUP BY region
dependencies:
- sales_csv
chartTitle: Sales by Region
chartType: bar
# ... chart configurationInline DataSet with inline DataSource
Section titled “Inline DataSet with inline DataSource”You can nest inline DataSource definitions within an inline DataSet. Use the @inline(N) syntax to reference them by index:
apiVersion: bino.bi/v1alpha1
kind: ChartStructure
metadata:
name: quick_chart
spec:
dataset:
dependencies:
- type: csv
path: ./data/sales.csv
query: |
SELECT region, SUM(amount) as total
FROM @inline(0)
GROUP BY region
chartTitle: Sales Overview
# ... chart configurationThe @inline(0) reference points to the first item in the dependencies array (0-indexed). When you have multiple inline DataSources:
spec:
dataset:
dependencies:
- type: csv
path: ./data/orders.csv
- type: csv
path: ./data/customers.csv
query: |
SELECT c.name, COUNT(*) as order_count
FROM @inline(0) o
JOIN @inline(1) c ON o.customer_id = c.id
GROUP BY c.nameMixing inline and named references
Section titled “Mixing inline and named references”You can mix inline DataSource definitions with named references:
spec:
dataset:
dependencies:
- sales_csv # Named reference to existing DataSource
- type: csv # Inline definition
path: ./data/rates.csv
query: |
SELECT s.*, r.rate
FROM sales_csv s
JOIN @inline(1) r ON s.currency = r.codeNote: Named references use their document name directly in SQL, while inline definitions use @inline(N).
Direct source pass-through
Section titled “Direct source pass-through”For simple cases where you want to use a DataSource directly without transformation, use the source field:
apiVersion: bino.bi/v1alpha1
kind: DataSet
metadata:
name: sales_passthrough
spec:
source: sales_csvThis is equivalent to SELECT * FROM sales_csv but more explicit and efficient. The source field is mutually exclusive with query and prql.
Inline source pass-through
Section titled “Inline source pass-through”You can also use inline DataSource definitions with source:
apiVersion: bino.bi/v1alpha1
kind: Table
metadata:
name: raw_data_table
spec:
dataset:
source:
type: csv
path: ./data/report.csv
# ... table configurationHow inline definitions work
Section titled “How inline definitions work”When bino processes inline definitions:
- Materialization: Inline definitions are converted to synthetic documents during YAML loading
- Hash-based naming: Generated documents get unique names like
_inline_datasource_a1b2c3d4 - Deduplication: Identical inline definitions share the same generated document
- Labels: Generated documents have
bino.bi/generated: "true"andbino.bi/inline: "true"labels
This means inline definitions are fully compatible with caching, the dependency graph, and all other bino features.
Naming restrictions
Section titled “Naming restrictions”User-defined document names must not start with _inline_ as this prefix is reserved for generated inline definitions. The lint rule inline-naming-conflict enforces this. Names starting with bino_ or _bino_ are reserved as well, for the functions and macros bino registers in every session and the views it creates for derive/assert (reserved-name-prefix).
Attribute Reference
Section titled “Attribute Reference”Common Metadata
Section titled “Common Metadata”| Attribute | Type | Required | Default | Description |
|---|---|---|---|---|
apiVersion | string | yes | — | Must be bino.bi/v1alpha1. |
kind | string | yes | — | Must be DataSet. |
metadata.name | string | yes | — | Unique identifier, and the table name the query is registered under. Pattern: ^(@[a-z0-9][a-z0-9_-]*/([a-z0-9][a-z0-9_-]*/)?)?[A-Za-z0-9_]([-A-Za-z0-9_]*[A-Za-z0-9_])?$ — a plain name, optionally prefixed with a registry scope such as @acme/ or @acme/kit/. |
metadata.labels | object | no | — | Key-value pairs for categorization and constraint matching. |
metadata.annotations | object | no | — | Arbitrary key-value metadata, not used by the system. |
metadata.description | string | no | — | Free-form description. |
metadata.constraints | array | no | — | Conditional inclusion rules. See Constraints. |
Spec Attributes
Section titled “Spec Attributes”At least one of query, prql, or source is required.
source is mutually exclusive with query and prql — combining them is rejected by the lint rule dataset-source-exclusive.
Setting query and prql together is accepted by the schema, but only prql is executed (the precedence is source before prql before query).
derive and assert combine with any of the three; the same slot in both is rejected by the lint rule dataset-derive-conflict.
| Attribute | Type | Required | Default | Description | Sample |
|---|---|---|---|---|---|
spec.query | string or object | conditional | — | SQL query executed against DuckDB, inline or from an external file. DataSource and DataSet names are the table names; inline dependencies are addressed as @inline(N). Required if prql and source are not set. | query: SELECT * FROM sales |
spec.query.$file | string | conditional | — | Path to an external .sql file, resolved relative to the manifest. Used instead of an inline string. See External query files. | $file: ./queries/sales_summary.sql |
spec.prql | string or object | conditional | — | PRQL query, inline or from an external file, compiled to SQL by DuckDB's prql extension. Required if query and source are not set. | prql: from sales_csv |
spec.prql.$file | string | conditional | — | Path to an external .prql file, resolved relative to the manifest. Used instead of an inline string. See External query files. | $file: ./queries/customer_orders.prql |
spec.source | string or object | conditional | — | Direct DataSource pass-through, equivalent to SELECT * FROM <name>. A string name or an inline DataSource definition. Required if query and prql are not set. See Direct source pass-through. | source: sales_csv |
spec.dependencies | array | no | — | List of DataSource dependencies. Feeds the dependency graph and the cache digest, so a changed input file invalidates the dataset. | see below |
spec.dependencies[] | string or object | no | — | Either the metadata.name of a DataSource, or an inline DataSource definition referenced in the query as @inline(N) with N the zero-based index. | - sales_csv |
spec.derive | object | no | — | Previous-period slots bino produces from another slot shifted back in time, keyed by pp1…pp4. The query must not return the slot. See Deriving previous period. | derive: { pp2: { from: ac1, shift: 1 year, grain: month } } |
spec.derive.<pp>.from | string | yes | — | Slot the earlier row is read from: one of ac1…ac4, pp1…pp4, fc1…fc4, pl1…pl4. | from: ac1 |
spec.derive.<pp>.shift | string | yes | — | How far back, <n> <unit> with the unit day, week, month, quarter or year. A year shift keeps the PY caption, any other unit is captioned PP. | shift: 1 month |
spec.derive.<pp>.grain | string | yes | — | The period one row stands for: day, week, month, quarter or year. Never inferred. | grain: month |
spec.assert | object | no | — | Previous-period slots the query supplies and bino checks, same shape as derive. A mismatch on any row with a prior period fails the build even under --data-validation warn. | assert: { pp3: { from: pl1, shift: 1 year, grain: month } } |
An inline DataSource — as spec.source or as an item of spec.dependencies — takes the full DataSource spec (type with the values csv, excel, parquet, inline, postgres_query, mysql_query, plus path, content, columns, connection, delimiter, sample, …). Those attributes are documented in the DataSource reference.
External file reference
Section titled “External file reference”spec:
query:
$file: ./queries/sales_summary.sqlDependencies with inline DataSources
Section titled “Dependencies with inline DataSources”spec:
dependencies:
- sales_csv # named reference
- type: csv # inline definition
path: ./data/extra.csv
query: |
SELECT * FROM sales_csv
UNION ALL
SELECT * FROM @inline(1)