Run your data warehouse end-to-end.
Best-practice guides organized by role. Each one teaches the practice first, then shows how Kirimana operationalizes it. Pick your role to see the track built for you.
The catalogue is open; reading a guide requires a (free) account. Sign in or request access .
32 guides in this track · updated with every release
Architecture & design
- Design a contract-first data warehouse 🔒
Describe every dataset once, as a data contract, and generate pipelines, tests and documentation from that single description instead of scanning what already got built.
- Choose your silver modelling technique 🔒
A decision framework for flat, Data Vault 2.0, or Kimball-conformed silver — why the choice is project-wide, what each option costs, and why you should decide before the first silver table exists.
- Model bronze on the source, silver and gold on the business 🔒
Bronze mirrors the source system verbatim; silver and gold model the consumer's information needs. Keeping that boundary sharp is what stops legacy source structures from calcifying into your warehouse.
- Design a naming standard your platform enforces 🔒
Naming conventions in a wiki die; conventions compiled into the platform live. How to design catalog, schema and table names as configuration, keep environments name-identical, and freeze the standard once data exists.
- Plan a multi-domain warehouse with hub-and-spoke governance 🔒
Give domains real autonomy inside central platform standards: when to split a domain, how to make cross-domain consumption an explicit reviewed dependency, and how to federate contracts without buying a central catalog.
Data governance
- Enforce ownership and classification from day one 🔒
Make a named owner and a data classification a structural requirement on every dataset, so governance cannot decay into 'owner: unknown'.
- Handle personal data correctly in a contract-first warehouse 🔒
Identify personally identifiable information at the source boundary, author it fail-closed with an audited human override, and keep evidence you can hand to an auditor.
- Set up approval workflows that scale with your team 🔒
Distinguish routine from sensitive changes, size the approver count to the risk, and keep the whole approval flow inside the pull requests engineers already work in.
- Govern AI access to your data 🔒
Replace blanket AI bans and blanket AI trust with a per-dataset policy, an explicit autonomy ladder, and an audit log of every model call with cost and caller.
- Run governance where the work happens: in pull requests 🔒
Turn classification raises, cross-domain consumption, and access rules into pull-request status checks with a sticky comment as the audit trail, replacing after-the-fact review meetings.
Data management
- Onboard a new data source the right way 🔒
Treat source onboarding as a governance event, not a plumbing task: introspect, classify, and contract before the first load, and capture ownership at the door.
- Manage schema evolution without breaking consumers 🔒
Evolve schemas additively, verify impact before every change, and continuously reconcile the declared contract against what the warehouse actually does.
- Keep history: choosing a historization strategy 🔒
A decision framework for slowly changing dimension type 2 (SCD2), Data Vault satellites, and snapshots, anchored in grain and business-key stability rather than tooling defaults.
- Set retention and deletion policies you can prove 🔒
Tie retention to classification, implement erasure as an auditable workflow rather than a script, and keep evidence an auditor can verify without trusting your word.
Data engineering
- Build your medallion end-to-end: first source to first star 🔒
Ship the smallest complete value chain, one source through bronze, silver, and gold to a report, before you scale out. Vertical slices beat layer-by-layer buildout.
- Ingestion patterns: files, APIs, databases and streams 🔒
Normalise every ingestion path into one universal shape: landing zone to bronze, with idempotent loads and provenance columns on every row.
- Build silver with Data Vault 2.0 🔒
Model silver as hubs, links, and satellites with a clean raw-vault/business-vault split, and know when Data Vault is the wrong tool.
- Build gold star schemas BI can trust 🔒
Model gold as an explicit Kimball star with declared grain, conformed dimensions, and surrogate keys, generated from contracts so every BI tool computes the same number.
- Test your data: quality gates and invariants 🔒
Build a test pyramid for data, declare quality as contract metadata so tests travel with the dataset, and fail closed on breach in production paths.
- Backfill and reprocess without fear 🔒
Treat idempotence as the entry requirement, run backfills as a first-class audited flow instead of one-off scripts, and verify after every reprocess.
DevOps for data
- Put your warehouse in git: everything-as-code 🔒
Keep the entire warehouse definition in one repository, use trunk-based development with short-lived branches, and treat environments as configuration rather than branches.
- Gate every pull request 🔒
Build a continuous integration pipeline for data that validates schemas, lints governance, runs project health checks, and counts required approvals, so nothing merges unreviewed.
- Manage secrets and connections across environments 🔒
Keep every credential out of code and YAML, resolve secrets from a vault at run time, and drive multiple environments from one codebase with named targets.
- Deploy the platform: from laptop to cloud 🔒
Develop against an embedded local database with a seconds-long apply loop, then deploy the same contracts to the cloud warehouse, growing from one pip-installed binary to scheduled and containerized only when needed.
DataOps
- Monitor warehouse health continuously 🔒
Turn platform health into a ranked, fix-hinted punch-list that gates pull requests and keeps a tracking issue in sync, instead of a dashboard nobody reads.
- Catch schema drift before your consumers do 🔒
Reconcile the live warehouse against your contracts on a schedule, treat drift as an incident to triage, and fold accepted drift back into the contract deliberately.
- Define and track SLAs on data products 🔒
Declare freshness and completeness as measurable promises inside the contract, check them on a schedule, and route every breach to an owned incident instead of an email thread.
- Run data incidents like an SRE 🔒
Classify severity, route incidents to the owning team through your ITSM tool, and run blameless postmortems where lineage shows the blast radius and every incident traces to a dataset and a change.
- Use lineage for impact analysis, not documentation 🔒
Declare lineage in the contract at design time, run what-if impact analysis before every schema change, and hand auditors a deterministic map of where personal data flows.
Release management
- Promote changes forward-only: dev, test, prod 🔒
Keep one main branch and treat environments as cursors on its history. Promotion moves a cursor forward, never sideways or back, and production promotion sits behind approvals engineers cannot bypass.
- Release with a manifest: know exactly what runs where 🔒
Maintain one release manifest as the single truth of which version of every data contract runs in which environment, and make that truth visible to reviewers on every pull request.
- Handle hotfixes without breaking your promotion model 🔒
A hotfix travels the same forward-only path as any change, just faster. Keep the diff small, expedite the review, promote immediately, tag what you ship, and track hotfix frequency as a health signal.