Configure Databricks Secret Scopes

Project YAML never carries a secret value. Anywhere a credential is needed, the YAML holds a ${vault:…} reference that resolves at run time. When your control plane runs next to a Databricks workspace, the natural place to keep those secrets is Databricks Secret Scopes — and Kirimana resolves refs directly against them, so a secret you already manage in the workspace is a secret Kirimana can read without a second copy anywhere.

How refs map to scopes

A ${vault:…} ref has a scheme, a secret id, and a key. With the Databricks backend, the secret id is the scope name and the key is the secret key within that scope:

auth:
  kind: bearer
  value_ref: "${vault:jira-credentials:bot-token}"

This resolves scope jira-credentials, key bot-token, and reads it from the workspace with GET /api/2.0/secrets/get. The value Databricks returns is base64-encoded on the wire; Kirimana decodes it and hands your run the plaintext. The provider holds no plaintext in Kirimana’s own storage — it forwards each read to Databricks and keeps the value in memory only for the lifetime of the run, with a per-process cache so the same secret isn’t re-fetched dozens of times during one apply.

Setup

Two things need to be true.

The secret exists in a scope. A workspace operator stores it in a Databricks Secret Scope — either a Databricks-managed scope or one backed by Azure Key Vault or AWS Secrets Manager. Kirimana reads whichever kind you use; the ref shape is identical.

Kirimana can read the scope. Reads authenticate as the Databricks service principal, and the SP needs READ on the scope’s ACL. If a ref fails to resolve, the error distinguishes the cause: a missing secret (not found), an expired or revoked token (authentication failed — refresh or rotate the token), or a scope the SP may not read (access denied — grant the SP READ on the scope). You are told which of the three it is, so you fix the right thing.

Once the scope and the ACL are in place, confirm the wiring end to end:

kiri databricks health --target prod

The vault-scope probe verifies that every ${vault:…} ref in kiri.yml resolves. A green probe means your refs are wired; a FAIL points at the ref that isn’t.

Rotation

Rotation is a re-set at the workspace. Because a ${vault:…} ref is indirection — the YAML points at a scope and key, never at a value — writing a new value over the same scope/key in Databricks means every ref resolves to the new value on the next run. No YAML changes, no redeploy, no contract edits.

After rotating a platform credential such as the service-principal token, re-run kiri databricks health to confirm the new value is readable through the whole chain before you rely on it.

When to choose this backend

Databricks Secret Scopes are the right home for secrets when Kirimana runs alongside a Databricks deployment and you want one place to manage credentials. Because the scope can itself be backed by a cloud key vault, you get the workspace’s own ACL model on top of your existing secret store without Kirimana ever holding a copy. Kirimana is in private beta at v1.0.0-beta.1; the Secret Scopes backend is available today.