Registering a key source

A key source is the component that hands the model key to a serving process. This page describes the key-source interface in src/kllm/keyed/tee/keysource.py and how a separate package registers a key source through the kllm.keysource entry-point group.

Built-in and registered sources

Name	Provided by	Behavior
passphrase	kllm (CLI)	kllm serve --key KEYFILE unlocks the key file with its passphrase. For development; refused in production mode.
env	kllm.keyed.tee.keysource	Reads KLLM_MODEL_KEY (64 hexadecimal digits or 32 raw bytes). For development; refused in production mode.
cdh	kLLM Enterprise	Inside a confidential VM, the Confidential Data Hub attests the VM to the deployer’s key broker, which releases the key only to a VM whose measurements match the deployer’s policy.
gramine	kLLM Enterprise	A key source that runs inside a confidential VM; described in the kLLM Enterprise documentation.
any other name	A package that registers it	Loaded from the kllm.keysource entry-point group.

The open-source package registers no entry points. Without kLLM Enterprise, a request for cdh fails with key source 'cdh' needs kLLM Enterprise (kllm-enterprise).

The interface

A key source is a callable that takes no arguments and returns a kllm.keyed.keys.ModelKey. kLLM resolves it by name:

from importlib.metadata import entry_points

def external(name):
    for ep in entry_points(group="kllm.keysource", name=name):
        return ep.load()
    raise KeySourceUnavailable(...)

external(name) returns the loaded object, and the caller invokes it immediately: external(name)(). The source should:

• return a ModelKey built from exactly 32 bytes. keysource.parse_key(raw) accepts 32 raw bytes or 64 hexadecimal digits, strips surrounding white space and raises ValueError for anything else;

• raise an exception with a precise message when the key cannot be obtained. A KeySourceUnavailable (a subclass of kllm.trust.TrustUnavailable) is printed by kllm to standard error and ends the command with exit status 2;

• never write the key to disk, to the log or to the environment of a child process.

Where key sources are called

Two code paths resolve key sources.

kllm serve --key-source cdh

Provider.attested_key() calls external("cdh")(). kllm serve prints the key’s fingerprint, then calls prepare_serve(..., key_source="cdh"), which checks the fingerprint against the sealed model and decrypts the permuted tokenizer. This is the only key source that production mode accepts.

keysource.load_key(pop=True)

Reads KLLM_KEY_SOURCE (default env). env reads KLLM_MODEL_KEY and, with pop=True, removes it from the environment; under production mode it raises RuntimeError. Any other value is resolved with external(name)(). load_key is a library function for serving processes; the kllm commands of 0.1.0.dev0 do not call it.

Important

In 0.1.0.dev0, kllm serve --key-source accepts only passphrase and cdh, and production mode accepts only keys released through the name cdh. A key source under another name is reachable through load_key() with KLLM_KEY_SOURCE, not through the kllm serve command line.

Registering a key source

A key source lives in its own package and declares an entry point in the kllm.keysource group. With a pyproject.toml:

[build-system]
requires = ["setuptools>=64"]
build-backend = "setuptools.build_meta"

[project]
name = "example-kllm-keysource"
version = "0.1.0"
dependencies = ["kllm[keyed]"]

[project.entry-points."kllm.keysource"]
example = "example_keysource:load_key"

[tool.setuptools]
py-modules = ["example_keysource"]

The module returns a ModelKey:

"""example_keysource: obtain the kLLM model key from a local secret service."""

from kllm.keyed.tee.keysource import KeySourceUnavailable, parse_key


def fetch_secret() -> bytes:
    """Return the key bytes from the deployment's secret service."""
    raise NotImplementedError


def load_key():
    try:
        raw = fetch_secret()
    except OSError as exc:
        raise KeySourceUnavailable(f"example key source: {exc}") from exc
    return parse_key(raw)

Install the package in the same environment as kLLM and check that the entry point resolves:

$ pip install ./example-kllm-keysource
$ KLLM_KEY_SOURCE=example python -c \
    "from kllm.keyed.tee import keysource; print(keysource.load_key().fingerprint)"

The command prints the key’s fingerprint, which must match the fingerprint recorded in the sealed model’s metadata (config.json for LLMs, kllm.json for pipelines).

Security requirements

A key source decides who can serve a sealed model, so it carries the deployment’s trust boundary.

• Release the key only into the trusted domain. Outside a confidential VM, any key source exposes the key to the host; see Threat model.

• Bind the release to evidence about the receiving environment, as the cdh source does with attestation, rather than to a static credential stored next to the model.

• Production mode exists to refuse development sources. A new source that is meant for production cannot be selected through kllm serve --production in 0.1.0.dev0, because that mode requires --key-source cdh.

See Confidential VM deployment for the deployment that uses the cdh source, and Settings for the environment variables involved.

See also

• Architecture overview for where key sources sit in the source tree.

• Model keys for keys and key files.

• License for the separately licensed Enterprise key sources.