Skip to content

Data model

One of the core principles of Reporting as Code is that data transformations are declarative and reproducible. In bino, you never manually export, paste, or reshape data - you define the sources and the SQL, and the engine handles the rest.

bino uses an embedded analytical engine. DataSource and DataSet manifests describe how raw data becomes analytical tables used by layouts and reports.

Each DataSource creates a view named after metadata.name. Depending on spec.type, bino loads data from:

  • Inline rows embedded in the manifest (type: inline)
  • Local files and directories (type: csv, excel, parquet)
  • Remote databases queried via SQL (type: postgres_query, mysql_query)
  • Object storage endpoints via connection secrets (S3, GCS, R2, Azure, HTTP, HuggingFace)

The ephemeral flag controls caching behavior:

  • When omitted, bino infers ephemerality from source type (remote and database sources are treated as changing).
  • Set ephemeral: false to explicitly allow caching for otherwise dynamic-looking sources.

See the DataSource reference for full details.

DataSet manifests define SQL queries that run on top of datasources. The query is executed in the query engine, and the results are reused by layouts and charts.

Key ideas:

  • spec.query is SQL; every datasource is queryable by its metadata.name in any dataset query.
  • spec.dependencies lists the datasources the query reads, so the dataset cache notices when they change and the dependency graph shows them.
  • bino lint reports a datasource the query reads but the list misses (dataset-dependency-undeclared).

A dataset can also declare what a previous-period slot means instead of supplying it: spec.derive names the slot bino fills from another slot shifted back in time (pp1: { from: ac1, shift: 1 month, grain: month }), and spec.assert checks a slot the query already returns. grain is the period one row stands for, shift how far back the compared row lies; rows are matched by their dimension values and by the date truncated to the grain, and the caption follows the shift (PY for a year, PP otherwise). See Deriving previous period.

See the DataSet reference for examples.

External storage and APIs (S3, GCS, HTTP, R2, Azure, HuggingFace) are configured via ConnectionSecret manifests. A connection secret has a spec.type (for example s3 or http) and a type-specific block (s3, http, azure, …).

Examples:

  • S3 access keys via spec.s3.keyIdFromEnv and spec.s3.secretFromEnv
  • HTTP basic auth via spec.http.usernameFromEnv and spec.http.passwordFromEnv
  • HuggingFace token via spec.huggingface.tokenFromEnv

Database credentials (postgres, mysql) are also stored in connection secrets, while non-credential connection details live in DataSource.spec.connection. See the ConnectionSecret reference for full schemas and examples.