YAML Configuration Reference

Purpose

This page is a reference. Use it to look up field names, expected shapes, and common examples. For guided learning, start with Configuration Guide, Multi-Profile, or Workspaces.

File Names

File	Meaning	Selected By
`mlknife-compose.yaml`	Default compose configuration	No profile flag
`mlknife-compose.prod.yaml`	Named profile that inherits from default	`--profile prod`
`mlknife-compose.batch-training.yaml`	Pipeline or environment variant	`--profile batch-training`

Root Schema

mlknife-compose.yaml

name: image-classifier
author: ml-team
backend: aws

parameters: {}
services: {}
modules: {}

Field	Type	Required	Description
`name`	string	Required	Logical stack name.
`author`	string	Required	Owner or team responsible for the stack.
`backend`	string	Optional	Provider backend, such as `aws` or `gcp`. Defaults to `aws` when omitted.
`workspace`	string	Optional	Deployment workspace. CLI `--workspace` takes precedence.
`role`	string	Optional	Provider execution role or equivalent deployment identity.
`parent_profile`	string	Profile only	Parent profile to inherit from. Defaults to `default`.
`parameters`	object	Optional	Reusable values referenced by services and modules.
`services`	object	Optional	Provider-managed infrastructure services.
`modules`	object	Optional	Pipeline modules or workflow steps.
`executors`	object	Optional	Reusable execution templates for modules.
`module_defaults`	object	Optional	Defaults applied to modules.
`disabled_services`	list	Optional	Service names excluded from the resolved stack.
`disabled_modules`	list	Optional	Module names excluded from the resolved stack.

Parameters

Use parameters for values reused across services and modules. References have two resolution phases: deploy-time references are resolved while ModelKnife builds the stack, and run-time references are resolved when an orchestrated pipeline run starts.

parameters:
  environment: dev
  data_path: s3://example-${parameters.environment}/images
  run_output_path: s3://example-${parameters.environment}/runs/${runtime.run_id}
  audit_prefix: ${runtime.workspace}/${runtime.pipeline_name}/${runtime.started_at}

Reference	Phase	Meaning
`${parameters.environment}`	Deploy time	Reference another parameter.
`${env.ENVIRONMENT}`	Deploy time	Reference an environment variable or value loaded from `.env`.
`${services.name.outputs.key}`	Deploy time	Reference outputs produced by a deployed service.
`${runtime.run_id}`	Run time	Shared id for the current pipeline run.
`${runtime.started_at}`	Run time	Timestamp for when the current pipeline run started.
`${runtime.pipeline_name}`	Run time	Logical pipeline name for the current run.
`${runtime.workspace}`	Run time	Workspace selected for the current run.
`${runtime.provider}`	Run time	Provider backend for the current run.

Resolution Phases

${parameters.*}, ${env.*}, and ${services.*} are deploy-time references. ${runtime.*} values are run-time references: they are intentionally preserved during compose-file resolution and filled by the orchestration layer when a pipeline run starts. All modules in the same run receive the same runtime context.

Run Overrides

Use run overrides when you want one pipeline execution to use different parameters without editing or redeploying the compose file. Overrides are passed to the same deployed pipeline for that execution only. Scheduled runs still use the parameters defined in YAML.

mk p run --param start_date=2026-06-01
mk p run --param input_path=s3://bucket/input
mk p run --module-param prepare_training_data.limit=1000

CLI Option	Scope	Behavior
`--param key=value`	Pipeline run	Overrides a pipeline parameter, then updates module job parameters that reference it.
`--module-param module.key=value`	One module	Overrides or adds a job parameter for the named module only.

Override Order

ModelKnife starts from the compose file, applies --param to the pipeline parameter set, resolves module job parameters that reference those pipeline parameters, then applies --module-param to the named module. --module-param is the most specific override.

Services

Services describe managed infrastructure. The common shape is provider-neutral where possible, with provider-specific fields under provider.<name>.

services:
  artifacts_bucket:
    type: object_storage_bucket
    configuration:
      bucket_name: image-classifier-${parameters.environment}-artifacts
      versioning: true
    provider:
      aws:
        force_destroy: false

Field	Type	Description
`type`	string	Service kind, such as `object_storage_bucket` or `function`.
`repository`	string	Source repository path when the service needs code.
`configuration`	object	Provider-neutral service configuration.
`provider`	object	Provider-specific overrides, keyed by provider name.
`depends_on`	list	Service dependencies.
`tags`	object	User metadata. ModelKnife also adds managed metadata such as workspace where supported.

Modules

Modules describe pipeline steps and their dependencies.

modules:
  prepare_training_data:
    type: etl
    repository: ../src
    entry_point: jobs/prepare_training_data.py
    depends_on: []
    job_parameters:
      input_path: ${parameters.data_path}/raw
      output_path: ${parameters.run_output_path}/prepared

Field	Type	Description
`type`	string	Module kind.
`repository`	string	Source code path.
`entry_point`	string	Script, function, or executable entry point.
`executor`	reference/object	Execution environment or reference to an executor.
`depends_on`	list	Module dependency names.
`job_parameters`	object	Runtime parameters passed to the module.

Profiles

A profile file inherits from the default compose file, then overrides matching fields.

mlknife-compose.prod.yaml

parent_profile: default

parameters:
  environment: prod

services:
  artifacts_bucket:
    configuration:
      versioning: true

Clearing Inherited Sections

Use modules: {} or services: {} when a profile should intentionally clear inherited modules or services.

Workspace Interaction

Workspace is not a YAML field. It is selected by CLI flag and passed into ModelKnife-managed deployment isolation.

mk s deploy --workspace prod --profile prod