YAML Configuration Reference

Purpose

This page is a reference. Use it to look up field names, expected shapes, and common examples. For guided learning, start with Configuration Guide, Multi-Profile, or Workspaces.

File Names

File	Meaning	Selected By
`mlknife-compose.yaml`	Default compose configuration	No profile flag
`mlknife-compose.prod.yaml`	Named profile that inherits from default	`--profile prod`
`mlknife-compose.batch-training.yaml`	Pipeline or environment variant	`--profile batch-training`

Root Schema

mlknife-compose.yaml

name: image-classifier
author: ml-team
backend: aws

deployment:
  enabled: true

parameters: {}
workspace_overrides: {}
services: {}
modules: {}

Field	Type	Required	Description
`name`	string	Required	Logical stack name.
`author`	string	Required	Owner or team responsible for the stack.
`backend`	string	Optional	Provider backend, such as `aws` or `gcp`. Defaults to `aws` when omitted.
`workspace`	string	Optional	Deployment workspace. CLI `--workspace` takes precedence.
`role`	string	Optional	Provider execution role or equivalent deployment identity.
`deployment`	object	Optional	Deploy command guard. Set `deployment.enabled: false` to block `mk s deploy` and `mk p deploy` for an abstract/base compose file.
`parent_profile`	string	Profile only	Parent profile to inherit from. Defaults to `default`.
`parameters`	object	Optional	Reusable values referenced by services and modules.
`workspace_overrides`	object	Optional	Workspace-specific compose overrides applied after profile merge and before placeholder resolution.
`services`	object	Optional	Provider-managed infrastructure services.
`modules`	object	Optional	Pipeline modules or workflow steps.
`executors`	object	Optional	Reusable execution templates for modules.
`module_defaults`	object	Optional	Defaults applied to modules.
`disabled_services`	list	Optional	Service names excluded from the resolved stack.
`disabled_modules`	list	Optional	Module names excluded from the resolved stack.

Deployment Guard

Use deployment.enabled: false on a base compose file that should be inherited by profiles but never deployed directly. The guard only blocks deploy commands; status, show, delete, and rollback commands are unchanged.

# mlknife-compose.yaml
name: naoo-i18n
author: platform
deployment:
  enabled: false
  reason: Base compose is not deployable. Use -p dev or -p prod.
  next_steps:
    - mk s deploy -p dev
    - mk s deploy -p prod

# mlknife-compose.dev.yaml
deployment:
  enabled: true

Parameters

Use parameters for values reused across services and modules. References have two resolution phases: deploy-time references are resolved while ModelKnife builds the stack, and run-time references are resolved when an orchestrated pipeline run starts.

parameters:
  environment: dev
  workspace_artifacts: s3://example-${system.workspace}/artifacts
  data_path: s3://example-${parameters.environment}/images
  run_output_path: s3://example-${parameters.environment}/runs/${runtime.run_id}
  audit_prefix: ${system.workspace}/${runtime.pipeline_name}/${runtime.started_at}

Reference	Phase	Meaning
`${parameters.environment}`	Deploy time	Reference another parameter.
`${env.ENVIRONMENT}`	Deploy time	Reference an environment variable or value loaded from `.env`.
`${system.workspace}`	Deploy time	Workspace selected by `--workspace`, compose `workspace`, or the current user default.
`${system.provider}`	Deploy time	Resolved provider backend, such as `aws` or `gcp`.
`${system.region}`	Deploy time	Provider session region when region is needed by the compose file.
`${services.name.outputs.key}`	Deploy time	Reference outputs produced by a deployed service.
`${runtime.run_id}`	Run time	Shared id for the current pipeline run.
`${runtime.started_at}`	Run time	Timestamp for when the current pipeline run started.
`${runtime.pipeline_name}`	Run time	Logical pipeline name for the current run.

Resolution Phases

${parameters.*}, ${env.*}, ${system.*}, and ${services.*} are deploy-time references. ${runtime.*} values are run-time references: they are intentionally preserved during compose-file resolution and filled by the orchestration layer when a pipeline run starts. All modules in the same run receive the same runtime context.

Use ${system.workspace} and ${system.provider} for workspace and provider values in compose configuration. ${runtime.workspace} and ${runtime.provider} are not supported compose placeholders.

${system.profile} is not supported. Profiles select configuration files; workspaces, providers, and regions describe the deployment target.

Run Overrides

Use run overrides when you want one pipeline execution to use different parameters without editing or redeploying the compose file. Overrides are passed to the same deployed pipeline for that execution only. Scheduled runs still use the parameters defined in YAML.

mk p run --param start_date=2026-06-01
mk p run --param input_path=s3://bucket/input
mk p run --module-param prepare_training_data.limit=1000

CLI Option	Scope	Behavior
`--param key=value`	Pipeline run	Overrides a pipeline parameter, then updates module job parameters that reference it.
`--module-param module.key=value`	One module	Overrides or adds a job parameter for the named module only.

Override Order

ModelKnife starts from the compose file, applies --param to the pipeline parameter set, resolves module job parameters that reference those pipeline parameters, then applies --module-param to the named module. --module-param is the most specific override.

Workspace Overrides

Use workspace_overrides for values that differ by selected workspace but should stay in the same compose profile. The override key must match the resolved workspace name from --workspace, compose workspace, or the current user default.

parameters:
  HERMES_ONLINE_INDEX_QUEUE_URL: ""

workspace_overrides:
  gwliu:
    parameters:
      HERMES_ONLINE_INDEX_QUEUE_URL: "https://sqs.eu-west-1.amazonaws.com/123/dev.fifo"
  prod:
    parameters:
      HERMES_ONLINE_INDEX_QUEUE_URL: "https://sqs.eu-west-1.amazonaws.com/456/prod.fifo"

services:
  online_worker:
    type: function:lambda_function
    repository: ../modules
    configuration:
      function_name: online-worker-${system.workspace}
      runtime: python3.9
      entry_point: worker.lambda_handler
      code_path: lambda_services
      environment:
        ONLINE_INDEX_QUEUE_URL: ${parameters.HERMES_ONLINE_INDEX_QUEUE_URL}

For mk s deploy -p infr-services -w gwliu, the final ONLINE_INDEX_QUEUE_URL is the value from workspace_overrides.gwliu.parameters.HERMES_ONLINE_INDEX_QUEUE_URL.

Step	Precedence	Notes
Default compose	Lowest	Base values from `mlknife-compose.yaml`.
Profile compose	Middle	Values from `mlknife-compose.<profile>.yaml` merge over default.
Workspace override	Highest	Values from `workspace_overrides.<workspace>` merge over the selected profile.
Placeholder resolution	After merge	`${parameters.}`, `${system.}`, `${env.*}`, and service output references resolve after the effective compose is assembled.

Recommended Use

Prefer profiles for large deployment variants such as dev versus prod. Prefer workspace_overrides for small workspace-specific values inside the same profile, especially external URLs, account-local resource names, or integration switches.

Services

Services describe managed infrastructure. The common shape is provider-neutral where possible, with provider-specific fields under provider.<name>.

services:
  artifacts_bucket:
    type: object_storage_bucket
    configuration:
      bucket_name: image-classifier-${parameters.environment}-artifacts
      versioning: true
    provider:
      aws:
        force_destroy: false

Field	Type	Description
`type`	string	Service kind, such as `object_storage_bucket` or `function`.
`repository`	string	Source repository path when the service needs code.
`configuration`	object	Provider-neutral service configuration.
`provider`	object	Provider-specific overrides, keyed by provider name.
`depends_on`	list	Service dependencies.
`tags`	object	User metadata. ModelKnife also adds managed metadata such as workspace where supported.

Modules

Modules describe pipeline steps and their dependencies.

modules:
  prepare_training_data:
    type: etl
    repository: ../src
    entry_point: jobs/prepare_training_data.py
    depends_on: []
    job_parameters:
      input_path: ${parameters.data_path}/raw
      output_path: ${parameters.run_output_path}/prepared

Field	Type	Description
`type`	string	Module kind.
`repository`	string	Source code path.
`entry_point`	string	Script, function, or executable entry point.
`executor`	reference/object	Execution environment or reference to an executor.
`depends_on`	list	Module dependency names.
`job_parameters`	object	Runtime parameters passed to the module.

Profiles

A profile file inherits from the default compose file, then overrides matching fields.

mlknife-compose.prod.yaml

parent_profile: default

parameters:
  environment: prod

services:
  artifacts_bucket:
    configuration:
      versioning: true

Clearing Inherited Sections

Use modules: {} or services: {} when a profile should intentionally clear inherited modules or services.

Workspace Interaction

Workspace is selected by CLI flag, compose workspace, or the current user default, then passed into ModelKnife-managed deployment isolation. CLI --workspace takes precedence over the compose field.

mk s deploy --workspace prod --profile prod