Services Reference

services:
  web_api:
    type: container_service
    repository: "../api"
    configuration:
      service_name: "web-api-${system.workspace}"
      port: 8080

The minimal configuration builds ../api/Dockerfile, pushes a source-hash-tagged image to a ModelKnife-managed ECR repository, and exposes the service through an HTTP ALB URL.

services:
  web_api:
    type: container_service
    repository: "../api"
    configuration:
      service_name: "web-api-${system.workspace}"
      image:
        build_context: "."
        dockerfile: "Dockerfile"
      port: 8080
      cpu: 512
      memory: 1024
      desired_count: 2
      environment:
        ENV: "${system.workspace}"
      secrets:
        DATABASE_URL: "arn:aws:secretsmanager:eu-west-1:123456789012:secret:db-url"
      health_check:
        path: "/health"
      autoscaling:
        enabled: true
        min_count: 2
        max_count: 10
        cpu_target: 60
      provider:
        aws:
          task_policy:
            statements:
              - effect: Allow
                actions:
                  - dynamodb:Query
                resources:
                  - arn:aws:dynamodb:eu-west-1:123456789012:table/users

services:
  web_api:
    type: container_service
    repository: "../api"
    configuration:
      service_name: "web-api-${system.workspace}"
      port: 8080

  public_api:
    type: api_gateway_v2
    depends_on:
      - web_api
    configuration:
      api_name: "public-api-${system.workspace}"
      routes:
        - route_key: "$default"
          http_endpoint: "${services.web_api.outputs.service_url}"

Default Behavior

• Image: build context ., Dockerfile Dockerfile, source-hash image tag.
• Compute: cpu: 512, memory: 1024.
• Count: 2 for prod workspace, otherwise 1.
• Network: default VPC, public subnets, public IP assignment, generated security groups.
• Load balancer: one HTTP Application Load Balancer per service, listener port 80.
• Health check: /health.
• Logs: CloudWatch Logs group /aws/ecs/modelknife/<service_name>, retention 14 days.
• Deployment: ECS deployment circuit breaker with rollback enabled.
• Deletion: removes runtime resources, keeps ECR repository and log group by default.

Configuration Parameters

• service_name (string, required) - Logical container service name.
• image (object, optional) - Either build_context/dockerfile or an existing uri.
• port (integer, required) - Container port and target group port.
• cpu (integer, optional) - Fargate CPU units. Defaults to 512.
• memory (integer, optional) - Fargate memory in MiB. Defaults to 1024.
• desired_count (integer, optional) - ECS desired count. Defaults by workspace.
• environment (object, optional) - Plain container environment variables.
• secrets (object, optional) - ECS secret references from Secrets Manager or SSM.
• health_check (object, optional) - ALB health check path and thresholds.
• autoscaling (object, optional) - Simple CPU target tracking autoscaling.
• logging (object, optional) - CloudWatch log group retention and stream prefix.
• deletion (object, optional) - Whether to delete retained ECR/log resources.
• provider.aws (object, optional) - AWS-specific network and IAM settings.

Outputs

• service_url
• alb_dns_name
• cluster_name
• ecs_service_name
• task_definition_arn
• ecr_repository_uri
• image_uri
• target_group_arn
• load_balancer_security_group_id
• service_security_group_id
• log_group_name

services:
  simple_function:
    type: lambda_function
    repository: "../src"
    configuration:
      function_name: "simple-${parameters.environment}"
      runtime: "python3.11"
      entry_point: "app.lambda_handler"
      code_path: "python-lambda"
      # Uses defaults: memory_size=128MB, timeout=30s

services:
  api_function:
    type: lambda_function
    repository: "../src"
    configuration:
        function_name: "api-${parameters.environment}"
        runtime: "python3.11"
        entry_point: "app.lambda_handler"
        code_path: "python-lambda"
        extra_py_files:
          - "./enrichment_core:enrichment_core"
        memory_size: 512      # Override default (128 MB)
        timeout: 60           # Override default (30 seconds)
        environment:
          API_KEY: "${parameters.api_key}"

services:
  # Python: entry point = "filename.function_name"
  # Expects requirements.txt in src/ directory
  python_api:
    type: lambda_function
    repository: "../src"
    configuration:
        function_name: "python-api-${parameters.environment}"
        runtime: "python3.11"
        entry_point: "app.lambda_handler"  # Calls lambda_handler() in app.py
        code_path: "python_lambda/"
        build_strategy: "auto"
        provider:
          aws:
            dependency_artifact: layer

  # Node.js: entry point = "filename.function_name"
  nodejs_api:
    type: lambda_function
    repository: "../src"
    configuration:
      function_name: "nodejs-api-${parameters.environment}"
      runtime: "nodejs18.x"
      entry_point: "index.handler"  # Calls exports.handler in index.js
      code_path: "python_lambda"

  # Java: entry point = "package.Class::method"
  java_api:
    type: lambda_function
    repository: "../src"
    configuration:
        function_name: "java-api-${parameters.environment}"
        runtime: "java11"
        entry_point: "com.example.Handler::handleRequest"
        code_path: "java-lambda"
        memory_size: 1024
        timeout: 60

  # Go: entry point = executable name (typically "main")
  go_api:
    type: lambda_function
    repository: "../src"
    configuration:
        function_name: "go-api-${parameters.environment}"
        runtime: "go1.x"
        entry_point: "main"  # Compiled binary name
        code_path: "go-lambda/api"

Project Organization

The following folder structure shows how to organize code based on the above examples. The final code path is resolved as: configuration_file_directory/repository/code_path

• configuration_file_directory: Directory containing mlknife-compose.yaml (e.g., conf/)
• repository: Code repository path relative to the configuration file directory (e.g., "../src")
• code_path: Specific code directory (e.g., "python-lambda", "go-lambda/api")

my-lambda-project/
├── conf/
│   └── mlknife-compose.yaml          # Configuration file
└── src/                              # Source code directory
    ├── python-lambda/                # Python Lambda functions
    │   ├── app.py                    # Main entry point file
    │   ├── processor.py              # Additional modules
    │   └── requirements.txt          # Python dependencies
    ├── enrichment_core/              # Shared Python package copied by extra_py_files
    │   └── __init__.py
    ├── nodejs-lambda/                # Node.js Lambda functions
    │   ├── index.js                  # Main entry point file
    │   ├── utils.js                  # Helper modules
    │   └── package.json              # Node.js dependencies
    ├── java-lambda/                  # Java Lambda functions
    │   ├── src/
    │   │   └── main/
    │   │       └── java/
    │   │           └── com/
    │   │               └── example/
    │   │                   └── Handler.java
    │   ├── pom.xml                   # Maven dependencies
    │   └── build.gradle              # Gradle dependencies (alternative)
    └── go-lambda/                    # Go Lambda functions
        ├── api/
        │   ├── main.go               # Main entry point file
        │   └── go.mod                # Go module file
        └── processor/
            ├── main.go
            └── go.mod

services:
  scheduled_processor:
    type: lambda_function
    repository: "../src"
    configuration:
        function_name: "processor-${parameters.environment}"
        runtime: "python3.11"
        entry_point: "processor.main"
        code_path: "python-lambda"
        extra_py_files:
          - "./enrichment_core:enrichment_core"
        memory_size: 1024
        timeout: 300

        # EventBridge scheduling
        schedule:
          cron: "0 8 * * 1-5"   # Weekdays at 8 AM
          timezone: "America/New_York"
          enabled: true

        provider:
          aws:
            # VPC access for private resources
            vpc_config:
              subnet_ids: ["subnet-12345678"]
              security_group_ids: ["sg-abcdef12"]

            # Custom layers
            layers:
              - "arn: aws:lambda:us-east-1:123456789012:layer:utils:1"

        environment:
          DB_HOST: "${parameters.database_host}"

Canonical Configuration Parameters

• table_name (string, required) - Physical table name.
• partition_key and sort_key (strings) - Primary key fields; sort_key is optional.
• attributes (mapping) - Key/index attribute names mapped to string, number, or binary.
• indexes (list) - Each index requires name (or index_name) and partition_key; sort_key is optional.
• billing_mode - ON_DEMAND (default) or PROVISIONED.
• capacity - Positive read_capacity and write_capacity values for provisioned mode.
• ttl - Mapping with boolean enabled; attribute is required when enabled.
• change_stream - Mapping with boolean enabled and optional view_type.
• backup and encryption - Provider-neutral mappings with an optional boolean enabled.
• provider.aws - AWS-only features such as auto scaling, local secondary indexes, VPC/access settings, deletion protection, and table class.

AWS Compatibility Fields Used in the Advanced Examples

Basic Configuration

• function_name (string, optional) - Lambda function name
• repository (string, optional) - Code repository location (required, service level)

Service Configuration

• build_strategy (string, optional) - Python build strategy: "local", "docker", "auto" (default: "auto")
• code_path (string, optional) - Path within repository
• extra_py_files (list of strings, optional) - Additional Python files or directories to copy into the Lambda package. Each entry uses source[:target], for example "./enrichment_core:enrichment_core". The source is resolved from the repository root or the code_path directory, and the target is the import path inside the packaged Lambda artifact.
• environment (string, optional) - Environment variables (key-value pairs, default: empty)
• entry_point (string, optional) - Entry point or handler function (format varies by runtime, see Entry point Formats section)
• runtime (string, optional) - Runtime (python3.8-3.12, nodejs18.x, java11, go1.x, etc.)

Performance Settings

• memory_size (string, optional) - Memory in MB (128-10240, 64MB increments, default: 128)
• timeout (string, optional) - Max execution time in seconds (1-900, default: 30)

Security Configuration

• vpc_config (string, optional) - VPC access (subnet_ids, security_group_ids, default: none)

Scheduling Configuration

• schedule (string, optional) - EventBridge cron scheduling (cron, timezone, enabled, default: none)

Advanced Options

• build_layer (string, optional) - Separate dependencies into reusable layer (default: false)
• layers (string, optional) - List of Lambda layer ARNs (default: empty)

Parameter Validation

• Source ARNs: Kinesis streams, SQS queues, and DynamoDB streams are supported.
• Target ARNs: Lambda, SQS, SNS, EventBridge event buses, Kinesis, Firehose, and Step Functions are supported.
• Source parameter keys: kinesis_stream_parameters, sqs_queue_parameters, and dynamo_db_stream_parameters. Target parameter keys are lambda_function_parameters and sqs_queue_parameters.

Common Validation Errors

• Parameter name contains invalid characters: Use only alphanumeric characters, hyphens, underscores, and periods
  Example: Change "my bucket" to "my-bucket"
• Parameter value exceeds maximum length: Reduce the parameter value to within the allowed range
  Example: Function names must be 64 characters or less
• Service reference not found: Ensure the referenced service exists and is properly named
  Example: Check that ${services.my-service.outputs.arn} references an existing service

Entry point Formats by Runtime

• Python: filename.function_name (e.g., app.lambda_handler calls lambda_handler() in app.py)
• Node.js: filename.function_name (e.g., index.handler calls exports.handler in index.js)
• Java: package.Class::method (e.g., com.example.Handler::handleRequest)
• Go: main or bootstrap (executable name, typically main)
• .NET: Assembly::Namespace.Class::Method (e.g., MyApp::MyApp.Function::Handler)
• Ruby: filename.method_name (e.g., lambda_function.lambda_handler)

Runtime-Specific Features

• Python (3.8-3.12): Automatically installs dependencies from requirements.txt located in the code_path directory. Uses local pip for pure Python packages (fast) or Docker for native dependencies (reliable). Auto-detects packages like numpy, pandas, scikit-learn that require compiled C extensions. Supports build_strategy, build_layer, and extra_py_files options.
• Node.js (14.x-20.x): Automatic npm install for package.json dependencies. Supports ES modules and CommonJS.
• Java (8, 11, 17, 21): Auto-detects Maven (pom.xml) or Gradle (build.gradle) and runs build automatically. Maven: mvn clean package -DskipTests, Gradle: ./gradlew build -x test.
• Go (1.x, provided.al2): Compiles to bootstrap executable. Binary must be named according to handler value.
• Other Runtimes: .NET (dotnet6-8), Ruby (2.7-3.3) with automatic dependency resolution

Advanced Features

• EventBridge Scheduling: Cron expressions with timezone support ("0 8 * * 1-5" = weekdays 8AM)
• VPC Access: Private subnet access with vpc_config
• Layer Management: Automatic layer creation and reuse for dependencies

services:
  model_endpoint:
    type: sagemaker_endpoint
    repository: "."
    configuration:
      endpoint_name: "ml-model-${parameters.environment}"
      model_name: "my-model"
      code_path: "inference"
      instance_type: "ml.m5.large"
      initial_instance_count: 1
      initial_variant_weight: 1.0
      requirements_file: "requirements.txt"
      environment:
        MODEL_VERSION: "v1.0"
      enable_network_isolation: false

Configuration Parameters

Basic Configuration

• endpoint_name (string, required) - SageMaker endpoint name
• instance_type (string, required) - SageMaker instance type for hosting
• model_name (string, required) - Model name for the endpoint

Service Configuration

• code_path (string, required) - Path to inference code directory
• initial_instance_count (integer, optional, default: 1) - Number of instances to start with
• initial_variant_weight (number, optional, default: 1.0) - Initial traffic weight; must be non-negative
• requirements_file (string, optional) - Python dependencies file

Advanced Options

• environment (mapping, optional) - String environment variables for the model container
• model_data_url (string, optional) - S3 URL for a pre-trained model artifact
• image_uri (string, optional) - Custom inference container image
• role_arn (string, optional) - Custom SageMaker execution role
• accelerator_type (string, optional) - Supported Elastic Inference accelerator type
• volume_size_in_gb (integer, optional) - Positive endpoint volume size
• enable_network_isolation (boolean, default: false) - Disable outbound network access from the model container
• vpc_config (mapping, optional) - VPC subnet and security group settings
• data_capture_config (mapping, optional) - SageMaker data capture settings

services:
  data_lake:
    type: s3_bucket
    configuration:
      bucket_name: "ml-data-${parameters.environment}"
      versioning: true

services:
  secure_data_lake:
    type: s3_bucket
    configuration:
      bucket_name: "ml-secure-data-${parameters.environment}"
      versioning: true
      encryption:
        SSEAlgorithm: "aws:kms"
        KMSMasterKeyID: "alias/s3-encryption-key"
        BucketKeyEnabled: true
      lifecycle_configuration:
        - Id: "DeleteOldVersions"
          Status: "Enabled"
          NoncurrentVersionExpiration:
            NoncurrentDays: 30
        - Id: "TransitionToIA"
          Status: "Enabled"
          Transitions:
            - Days: 30
              StorageClass: "STANDARD_IA"
            - Days: 90
              StorageClass: "GLACIER"

Configuration Parameters

Basic Configuration

• bucket_name (string, required) - S3 bucket name (must be globally unique, 3-63 characters, letters/numbers/hyphens/periods only)

Service Configuration

• BucketKeyEnabled (string, optional) - Use S3 Bucket Keys to reduce KMS costs (boolean)
• Expiration (string, optional) - Object expiration settings
• Id (string, optional) - Unique rule identifier
• KMSMasterKeyID (string, optional) - KMS key ID or alias (required for aws:kms)
• NoncurrentVersionExpiration (string, optional) - Expiration for non-current versions
• SSEAlgorithm (string, optional) - Encryption algorithm ("AES256" or "aws:kms")
• Status (string, optional) - Rule status ("Enabled" or "Disabled")
• Transitions (string, optional) - List of transition rules with Days and StorageClass

Storage Configuration

• versioning (boolean, optional, default: false) - Enable object versioning (cannot be disabled once enabled, only suspended)

Parameter Validation

• bucket_name: 3-63 characters, lowercase letters, numbers, hyphens, and periods only
  Examples: my-data-bucket, company-logs-2024, ml-model-artifacts
• versioning: Boolean value
  Examples: true, false

Common Validation Errors

• Parameter name contains invalid characters: Use only alphanumeric characters, hyphens, underscores, and periods
  Example: Change "my bucket" to "my-bucket"
• Parameter value exceeds maximum length: Reduce the parameter value to within the allowed range
  Example: Function names must be 64 characters or less
• Service reference not found: Ensure the referenced service exists and is properly named
  Example: Check that ${services.my-service.outputs.arn} references an existing service

Implementation Notes

• Bucket names must be globally unique across all AWS accounts
• Versioning cannot be disabled once enabled, only suspended
• Lifecycle rules are processed in the order they appear in the configuration
• Encryption settings apply to all objects in the bucket by default
• Use KMS encryption for enhanced security and compliance requirements

services:
  feature_store:
    type: dynamodb_table
    configuration:
      table_name: "ml-features-${parameters.environment}"
      attributes:
        feature_id: string
        timestamp: string
        user_id: string
      partition_key: "feature_id"
      sort_key: "timestamp"
      billing_mode: ON_DEMAND
      indexes:
        - name: "user-index"
          partition_key: "user_id"
      ttl:
        enabled: true
        attribute: expires_at
      change_stream:
        enabled: true
        view_type: NEW_AND_OLD_IMAGES
      backup:
        point_in_time_recovery: true
      encryption:
        enabled: true
        kms_key_id: alias/aws/dynamodb
      provider:
        aws:
          deletion_protection: true

services:
  advanced_table:
    type: dynamodb_table
    configuration:
      table_name: "ml-features-${parameters.environment}"
      attribute_definitions:
        feature_id: "S"
        timestamp: "S"
        user_id: "S"
        score: "N"
      partition_key: "feature_id"
      sort_key: "timestamp"
      billing_mode: "PROVISIONED"
      provisioned_throughput:
        ReadCapacityUnits: 10
        WriteCapacityUnits: 10
      auto_scaling:
        table:
          read_capacity:
            min_capacity: 5
            max_capacity: 100
            target_utilization: 70.0
          write_capacity:
            min_capacity: 5
            max_capacity: 100
            target_utilization: 70.0
        global_secondary_indexes:
          user-index:
            read_capacity:
              min_capacity: 5
              max_capacity: 50
              target_utilization: 70.0
      server_side_encryption:
        enabled: true
        kms_key_id: "alias/dynamodb-key"
      global_secondary_indexes:
        - index_name: "user-index"
          partition_key: "user_id"
          sort_key: "score"
          projection:
            projection_type: "ALL"
      local_secondary_indexes:
        - index_name: "score-index"
          sort_key: "score"
          projection:
            projection_type: "KEYS_ONLY"
      stream_specification:
        stream_enabled: true
        stream_view_type: "NEW_AND_OLD_IMAGES"
      backup_configuration:
        on_demand_backup:
          backup_name: "ml-features-backup"
        scheduled_backup:
          schedule_expression: "cron(0 2 * * ? *)"
          retention_period_days: 30
      table_class: "STANDARD"

# Table-level auto-scaling only
services:
  scalable_table:
    type: dynamodb_table
    configuration:
      table_name: "user-sessions-${parameters.environment}"
      attribute_definitions:
        session_id: "S"
        user_id: "S"
      partition_key: "session_id"
      billing_mode: "PROVISIONED"
      provisioned_throughput:
        ReadCapacityUnits: 5
        WriteCapacityUnits: 5
      auto_scaling:
        table:
          read_capacity:
            min_capacity: 5
            max_capacity: 200
            target_utilization: 70.0
            scale_in_cooldown: 300   # 5 minutes
            scale_out_cooldown: 60   # 1 minute
          write_capacity:
            min_capacity: 5
            max_capacity: 100
            target_utilization: 80.0

---

# GSI auto-scaling configuration
services:
  multi_index_table:
    type: dynamodb_table
    configuration:
      table_name: "product-catalog-${parameters.environment}"
      attribute_definitions:
        product_id: "S"
        category: "S"
        price: "N"
        brand: "S"
      partition_key: "product_id"
      billing_mode: "PROVISIONED"
      provisioned_throughput:
        ReadCapacityUnits: 10
        WriteCapacityUnits: 10
      global_secondary_indexes:
        - index_name: "category-price-index"
          partition_key: "category"
          sort_key: "price"
          projection:
            projection_type: "ALL"
          provisioned_throughput:
            ReadCapacityUnits: 5
            WriteCapacityUnits: 5
        - index_name: "brand-index"
          partition_key: "brand"
          projection:
            projection_type: "KEYS_ONLY"
          provisioned_throughput:
            ReadCapacityUnits: 3
            WriteCapacityUnits: 3
      auto_scaling:
        table:
          read_capacity:
            min_capacity: 5
            max_capacity: 100
            target_utilization: 70.0
          write_capacity:
            min_capacity: 5
            max_capacity: 50
            target_utilization: 70.0
        global_secondary_indexes:
          category-price-index:
            read_capacity:
              min_capacity: 5
              max_capacity: 50
              target_utilization: 75.0
            write_capacity:
              min_capacity: 5
              max_capacity: 25
              target_utilization: 75.0
          brand-index:
            read_capacity:
              min_capacity: 3
              max_capacity: 20
              target_utilization: 80.0
            write_capacity:
              min_capacity: 3
              max_capacity: 10
              target_utilization: 80.0

# Comprehensive index configuration
services:
  analytics_table:
    type: dynamodb_table
    configuration:
      table_name: "user-analytics-${parameters.environment}"
      attribute_definitions:
        user_id: "S"        # Table partition key
        timestamp: "S"      # Table sort key
        event_type: "S"     # GSI partition key
        session_id: "S"     # GSI sort key
        score: "N"          # LSI sort key
        region: "S"         # Additional GSI partition key
      partition_key: "user_id"
      sort_key: "timestamp"
      billing_mode: "PAY_PER_REQUEST"

      # Global Secondary Indexes (different partition key)
      global_secondary_indexes:
        # GSI with both partition and sort key
        - index_name: "event-session-index"
          partition_key: "event_type"
          sort_key: "session_id"
          projection:
            projection_type: "ALL"  # Include all attributes

        # GSI with partition key only
        - index_name: "region-index"
          partition_key: "region"
          projection:
            projection_type: "KEYS_ONLY"  # Only key attributes

        # GSI with selective attribute projection
        - index_name: "event-timestamp-index"
          partition_key: "event_type"
          sort_key: "timestamp"
          projection:
            projection_type: "INCLUDE"
            non_key_attributes:
              - "user_id"
              - "score"
              - "session_id"

      # Local Secondary Indexes (same partition key, different sort key)
      local_secondary_indexes:
        # LSI for querying by score within user_id
        - index_name: "user-score-index"
          sort_key: "score"
          projection:
            projection_type: "KEYS_ONLY"

        # LSI for querying by event_type within user_id
        - index_name: "user-event-index"
          sort_key: "event_type"
          projection:
            projection_type: "ALL"

---

# Provisioned billing with GSI-specific capacity
services:
  ecommerce_table:
    type: dynamodb_table
    configuration:
      table_name: "orders-${parameters.environment}"
      attribute_definitions:
        order_id: "S"
        customer_id: "S"
        order_date: "S"
        status: "S"
        total_amount: "N"
      partition_key: "order_id"
      sort_key: "order_date"
      billing_mode: "PROVISIONED"
      provisioned_throughput:
        ReadCapacityUnits: 20
        WriteCapacityUnits: 10

      global_secondary_indexes:
        # Customer orders index with custom capacity
        - index_name: "customer-date-index"
          partition_key: "customer_id"
          sort_key: "order_date"
          projection:
            projection_type: "ALL"
          provisioned_throughput:
            ReadCapacityUnits: 10  # Different from table capacity
            WriteCapacityUnits: 5

        # Status index for order management
        - index_name: "status-date-index"
          partition_key: "status"
          sort_key: "order_date"
          projection:
            projection_type: "INCLUDE"
            non_key_attributes:
              - "customer_id"
              - "total_amount"
          provisioned_throughput:
            ReadCapacityUnits: 5
            WriteCapacityUnits: 2

      local_secondary_indexes:
        # Sort orders by total amount within order_id
        - index_name: "order-amount-index"
          sort_key: "total_amount"
          projection:
            projection_type: "INCLUDE"
            non_key_attributes:
              - "customer_id"
              - "status"

Configuration Parameters

Basic Configuration

• index_name (string, required) - Unique index name
• index_name (string, required) - Unique index name
• table_name (string, required) - DynamoDB table name (supports environment variables). Must be 3-255 characters, alphanumeric with underscores, hyphens, and periods allowed.
• global_secondary_indexes.{index_name} (string, optional) - Per-GSI scaling configuration
• stream_view_type (string, optional) - Stream content: "KEYS_ONLY", "NEW_IMAGE", "OLD_IMAGE", "NEW_AND_OLD_IMAGES"

Service Configuration

• partition_key (string, required) - Primary partition key attribute name. Must be defined in attribute_definitions.
• partition_key (string, required) - GSI partition key (must be in attribute_definitions)
• sort_key (string, required) - LSI sort key (different from table sort_key, must be in attribute_definitions)
• cross_account_access (string, optional) - Cross-account access configuration with trusted_account_ids and access_level
• enabled (boolean, optional) - Enable/disable encryption
• iam_policy_templates (string, optional) - List of predefined policy templates: read_only, read_write, stream_consumer, ml_feature_store, admin
• on_demand_backup (string, optional) - Manual backup configuration with backup_name
• point_in_time_recovery (string, optional, default: false) - Enable continuous backups for 35 days with point-in-time recovery.
• projection (string, optional) - Projection configuration: projection_type ("ALL", "KEYS_ONLY", "INCLUDE") and non_key_attributes for INCLUDE
• projection (string, optional) - Projection configuration
• resource_policy (string, optional) - Custom resource-based policy document
• route_table_ids (string, optional) - List of route table IDs
• sort_key (string, optional) - Sort key attribute name. Must be defined in attribute_definitions and different from partition_key.
• sort_key (string, optional) - GSI sort key (must be in attribute_definitions)
• stream_enabled (boolean, optional) - Enable/disable streams
• table.read_capacity/write_capacity (string, optional) - Table-level scaling with min_capacity, max_capacity, target_utilization (0-100)
• vpc_endpoint_id (string, optional) - VPC endpoint ID (vpce-*)

Performance Settings

• provisioned_throughput (string, required) - Capacity settings with ReadCapacityUnits and WriteCapacityUnits (minimum 1 each).
• billing_mode (string, optional, canonical default: "ON_DEMAND") - PAY_PER_REQUEST is accepted as a backward-compatible alias; use PROVISIONED with capacity settings for predictable capacity.
• provisioned_throughput (string, optional) - GSI-specific capacity (PROVISIONED billing only)

Security Configuration

• kms_key_id (string, optional) - KMS key ID, alias, or ARN. Use "alias/aws/dynamodb" for AWS managed key

Network Configuration

• security_group_ids (string, optional) - List of security group IDs
• subnet_ids (string, optional) - List of subnet IDs

Scheduling Configuration

• scheduled_backup (string, optional) - Automated backup with schedule_expression (cron format) and retention_period_days (1-35)

Parameter Validation

• table_name: 3-255 characters, alphanumeric, hyphens, underscores, and periods only
  Examples: user-sessions, product_catalog, event.logs
• billing_mode: PAY_PER_REQUEST or PROVISIONED
  Examples: PAY_PER_REQUEST, PROVISIONED

Common Validation Errors

• Parameter name contains invalid characters: Use only alphanumeric characters, hyphens, underscores, and periods
  Example: Change "my bucket" to "my-bucket"
• Parameter value exceeds maximum length: Reduce the parameter value to within the allowed range
  Example: Function names must be 64 characters or less
• Service reference not found: Ensure the referenced service exists and is properly named
  Example: Check that ${services.my-service.outputs.arn} references an existing service

Implementation Notes

• Billing Mode Selection: Use PAY_PER_REQUEST for unpredictable workloads and PROVISIONED with auto-scaling for consistent traffic patterns.
• Auto-Scaling: Requires PROVISIONED billing mode and appropriate IAM permissions for CloudWatch and Application Auto Scaling.
• Global Secondary Indexes (GSI):
  • Maximum 20 GSIs per table
  • Each GSI consumes additional read/write capacity and storage
  • Can have different partition and sort keys from the main table
  • Can be created after table creation
  • In PROVISIONED billing mode, each GSI can have independent capacity settings
  • Projection types: ALL (all attributes), KEYS_ONLY (key attributes only), INCLUDE (specified attributes)
• Local Secondary Indexes (LSI):
  • Maximum 10 LSIs per table
  • Can only be created at table creation time (not after)
  • Must use the same partition key as the main table
  • Must have a different sort key from the main table
  • Share read/write capacity with the main table
  • Total item size limit of 400KB per partition key value (including all LSI items)
• Index Validation Rules:
  • All partition and sort keys used in indexes must be defined in attribute_definitions
  • Index names must be unique within the table
  • LSI sort key cannot be the same as the table sort key
  • For INCLUDE projection, non_key_attributes must not include key attributes
• Encryption: Encryption at rest is enabled by default with AWS managed keys. Customer managed KMS keys provide additional control but incur extra costs.
• Streams: Stream records are retained for 24 hours. Use for real-time processing, replication, or analytics.
• Table Class: STANDARD_INFREQUENT_ACCESS reduces storage costs by ~60% but increases access costs. Suitable for infrequently accessed data.
• Capacity Planning: For PROVISIONED billing, consider query patterns when setting GSI capacity. Read-heavy GSIs may need higher read capacity than the main table.

services:
  user_features:
    type: sagemaker_feature_store
    configuration:
      feature_group_name: "user-features-${parameters.environment}"
      record_identifier_name: "user_id"
      event_time_feature_name: "event_time"
      feature_definitions:
        user_id: "String"
        age: "Integral"
        income: "Fractional"
        event_time: "String"
      online_store_config:
        enable_online_store: true
      offline_store_config:
        s3_storage_config:
          s3_uri: "s3://${services.feature_bucket.bucket_name}/features/"

Configuration Parameters

Basic Configuration

• event_time_feature_name (string, optional) - Timestamp field name
• feature_group_name (string, optional) - Feature group name in SageMaker
• record_identifier_name (string, optional) - Primary key field name

Service Configuration

• feature_definitions (string, optional) - Feature schema with data types
• offline_store_config (string, optional) - Offline store configuration for batch processing
• online_store_config (string, optional) - Online store configuration for real-time access

services:
  analytics_db:
    type: glue_database
    configuration:
      database_name: "analytics_${parameters.environment}"
      description: "Analytics data warehouse database"

services:
  data_catalog:
    type: glue_database
    configuration:
      database_name: "data_catalog_${parameters.environment}"
      description: "Centralized data catalog for ML features"
      location_uri: "s3://${services.data_lake.outputs.bucket_name}/catalog/"

Configuration Parameters

Basic Configuration

• database_name (string, optional) - Glue database name (lowercase, underscore-separated)

Service Configuration

• description (string, optional) - Database description (optional)
• location_uri (string, optional) - Default storage location S3 URI (optional)

Parameter Validation

• Names: Must follow AWS naming conventions (alphanumeric, hyphens, underscores)
• Required Parameters: All required parameters must be provided
• Type Validation: Parameters must match expected data types
• Range Validation: Numeric parameters must be within allowed ranges

Database Management

• Schema Registry: Central location for table schemas and metadata
• Table Organization: Tables are organized under databases
• Integration: Works with Athena, EMR, and other analytics services

services:
  events_table:
    type: glue_table
    configuration:
      table_name: "events"
      database_name: "${services.analytics_db.outputs.database_name}"
      storage_location: "s3://${services.data_lake.outputs.bucket_name}/events/"
      columns:
        event_id: "string"
        user_id: "string"
        event_type: "string"
        timestamp: "bigint"
        amount: "double"
        is_premium: "boolean"

services:
  partitioned_events:
    type: glue_table
    configuration:
      table_name: "partitioned_events"
      database_name: "${services.analytics_db.outputs.database_name}"
      storage_location: "s3://${services.data_lake.outputs.bucket_name}/partitioned-events/"
      columns:
        event_id: "string"
        user_id: "string"
        event_type: "string"
        timestamp: "bigint"
        year: "int"
        month: "int"
        day: "int"
      partition_keys: ["year", "month", "day"]
      description: "Events table partitioned by date for efficient querying"

services:
  json_table:
    type: glue_table
    configuration:
      table_name: "json_events"
      database_name: "${services.analytics_db.outputs.database_name}"
      storage_location: "s3://${services.data_lake.outputs.bucket_name}/json-events/"
      columns:
        event_id: "string"
        payload: "struct<user_id:string,action:string,metadata:map<string,string>>"
        timestamp: "timestamp"
      serde_info:
        serialization_library: "org.apache.hive.hcatalog.data.JsonSerDe"
        "serialization.format": "1"
      input_format: "org.apache.hadoop.mapred.TextInputFormat"
      output_format: "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat"

services:
  complex_table:
    type: glue_table
    configuration:
      table_name: "complex_data"
      database_name: "${services.analytics_db.outputs.database_name}"
      storage_location: "s3://${services.data_lake.outputs.bucket_name}/complex-data/"
      columns:
        id: "string"
        tags: "array<string>"
        metadata: "map<string,string>"
        user_profile: "struct<name:string,age:int,preferences:array<string>>"
        scores: "array<double>"
        created_at: "timestamp"
        is_active: "boolean"

Configuration Parameters

Basic Configuration

• database_name (string, optional) - Glue database name (reference to database service)
• table_name (string, optional) - Glue table name (lowercase, underscore-separated)
• table_type (string, optional) - Table type ("EXTERNAL_TABLE" or "VIRTUAL_VIEW", default: "EXTERNAL_TABLE")

Service Configuration

• description (string, optional) - Table description (optional)
• input_format (string, optional) - Input format class name (optional)
• output_format (string, optional) - Output format class name (optional)
• parameters (string, optional) - Additional table parameters (optional)
• partition_keys (string, optional) - List of column names to use as partitions (optional)
• serde_info (mapping, optional) - SerDe configuration. When set, it must include serialization_library; additional string keys are passed as SerDe parameters.
• storage_location (string, optional) - S3 URI where table data is stored

Parameter Validation

• Names: Must follow AWS naming conventions (alphanumeric, hyphens, underscores)
• Required Parameters: All required parameters must be provided
• Type Validation: Parameters must match expected data types
• Range Validation: Numeric parameters must be within allowed ranges

Supported Data Types

• Primitive Types: string, int, bigint, double, float, boolean, timestamp, date, binary, decimal
• Complex Types: array<type>, map<key_type,value_type>, struct<field:type,field:type>
• Examples: array<string>, map<string,int>, struct<name:string,age:int>

Partitioning

• Partition Keys: List of column names that define table partitions
• Performance: Partitioning improves query performance by limiting data scanned
• Common Patterns: Date-based partitioning (year, month, day) or categorical partitioning
• Validation: Partition key columns must be defined in the columns section

SerDe Configuration

• JSON SerDe: org.apache.hive.hcatalog.data.JsonSerDe for JSON data
• Parquet: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe
• CSV/TSV: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
• Custom Parameters: Configure SerDe behavior through additional keys in serde_info

Configuration Parameters

Basic Configuration

• database_name (string, optional) - Parent Glue database name
• table_name (string, optional) - Glue table name

Service Configuration

• columns (string, optional) - Table column definitions in simplified format (column_name: type)
• input_format (string, optional) - Data input format class (optional)
• output_format (string, optional) - Data output format class (optional)
• partition_keys (string, optional) - List of column names to use as partition keys (optional)
• serde_info (mapping, optional) - Serialization/deserialization library and SerDe-specific parameters
• storage_location (string, optional) - S3 location for table data

Simplified Configuration

• Columns: Use simple key-value format: column_name: "type"
• Partition Keys: List column names that exist in the columns definition
• Convention: Follows ModelKnife's "less configuration" principle
• Deployer Mapping: Automatically converts to AWS Glue API format

services:
  product_search:
    type: search_service
    configuration:
      service_name: "product-search-${parameters.environment}"
      search_type: "product_search"
      environment: "production"
      performance_tier: "high_performance"
      access_level: "private"
      indices:
        - name: "products"
          fields:
            - name: "title"
              type: "text"
              analyzer: "english"
              copy_to: ["search_all"]
            - name: "category"
              type: "keyword"
              facetable: true
            - name: "price"
              type: "number"
              facetable: true
            - name: "brand"
              type: "keyword"
              facetable: true
            - name: "search_all"
              type: "search_as_you_type"
              max_shingle_size: 3
      languages: ["english"]

services:
  document_search:
    type: search_service
    configuration:
      service_name: "doc-search-${parameters.environment}"
      search_type: "vector_search"
      environment: "production"
      performance_tier: "balanced"
      access_level: "team_access"
      embedding_config:
        model_id: "amazon.titan-embed-text-v1"
        service: "bedrock"
        batch_size: 25
        auto_vectorize: true
      indices:
        - name: "documents"
          fields:
            - name: "content"
              type: "text"
              analyzer: "english"
            - name: "title"
              type: "text"
              analyzer: "english"
            - name: "embedding"
              type: "vector"
              dimensions: 1536
              similarity_function: "cosine"
            - name: "document_type"
              type: "keyword"
              facetable: true
            - name: "created_date"
              type: "date"
              facetable: true
      languages: ["english"]

services:
  hybrid_search:
    type: search_service
    configuration:
      service_name: "hybrid-search-${parameters.environment}"
      search_type: "hybrid_search"
      environment: "production"
      performance_tier: "high_performance"
      access_level: "public"
      embedding_config:
        model_id: "amazon.titan-embed-text-v1"
        service: "bedrock"
        batch_size: 50
        auto_vectorize: true
      indices:
        - name: "content"
          fields:
            - name: "title"
              type: "text"
              analyzer: "english"
              searchable: true
            - name: "content"
              type: "text"
              analyzer: "english"
              searchable: true
            - name: "embedding"
              type: "vector"
              dimensions: 1536
              similarity_function: "cosine"
            - name: "tags"
              type: "keyword"
              facetable: true
            - name: "category"
              type: "keyword"
              facetable: true
      languages: ["english", "spanish"]

Business-Oriented Search Types

• full_text_search - Optimized for text search and analysis with advanced text processing
• vector_search - Optimized for semantic/vector search with embedding integration
• product_search - Optimized for e-commerce search with faceting and filtering
• log_search - Optimized for time-series log analysis and monitoring
• document_search - Optimized for document content search and retrieval
• hybrid_search - Combines full-text and vector search for comprehensive results

Parameter Validation

• Names: Must follow AWS naming conventions (alphanumeric, hyphens, underscores)
• Required Parameters: All required parameters must be provided
• Type Validation: Parameters must match expected data types
• Range Validation: Numeric parameters must be within allowed ranges

Configuration Parameters

Basic Configuration

• search_type (string, required) - Business-oriented search type (see above)
• service_name (string, required) - Search service name (1-32 characters, lowercase, alphanumeric and hyphens)

Service Configuration

• access_level (string, optional) - Access level ("public", "private", "team_access")
• data_sources (string, optional) - Data ingestion source configurations
• embedding_config (string, optional) - Embedding model configuration for vector search
• environment (string, optional) - Deployment environment ("development", "staging", "production")
• indices (string, optional) - Index configurations with field definitions
• languages (string, optional) - Supported languages for text analysis
• performance_tier (string, optional) - Performance tier ("development", "balanced", "high_performance", "cost_optimized")
• standby_replicas (boolean, optional) - Explicitly enable or disable OpenSearch Serverless standby replicas. When omitted, ModelKnife derives the setting from the environment and performance tier.
• serverless_vector_acceleration (string, optional) - Vector acceleration mode for vector collections: ENABLED, DISABLED, or ALLOWED.
• delete_collection_on_delete (boolean, default: false) - Controls whether deleting the ModelKnife service also removes the OpenSearch collection and indexed data. The safe default retains the collection.

Field Types

• text - Full-text searchable content with analyzers
• keyword - Exact match, faceting, and filtering
• number - Numeric values for range queries and faceting
• date - Date/timestamp fields for temporal filtering
• vector - Vector embeddings for semantic search
• search_as_you_type - Auto-complete and search suggestions

Performance Tiers

• development - Cost-optimized for development and testing
• balanced - Balanced performance and cost for most use cases
• high_performance - Performance-optimized for production workloads
• cost_optimized - Minimum cost configuration for light usage

Access Levels

• public - Public internet access with authentication
• private - Private VPC access only
• team_access - Team-based access control with IAM integration

Embedding Configuration

• model_id - Embedding model identifier (e.g., "amazon.titan-embed-text-v1")
• service - Embedding service ("bedrock", "openai", "huggingface")
• batch_size - Batch size for embedding generation (1-100)
• timeout_seconds - Request timeout for embedding service
• auto_vectorize - Automatically generate embeddings for text fields

Implementation Notes

• Search service automatically selects optimal infrastructure (serverless vs managed) based on configuration
• Vector search types require embedding_config for automatic vectorization
• Field configurations determine index mappings and query capabilities
• Performance tiers affect underlying infrastructure provisioning and costs
• Access levels control network access patterns and authentication requirements
• Multi-language support affects text analyzers and tokenization strategies
• Data sources enable automatic ingestion and transformation pipelines

services:
  ml_api:
    type: api_gateway
    configuration:
      api_name: "ml-api-${parameters.environment}"
      stage_name: "prod"
      resources:
        - path: "/predict"
          method: "POST"
          integration_type: "lambda"
          lambda_function: "${services.inference_lambda.function_name}"
        - path: "/health"
          method: "GET"
          integration_type: "mock"

Configuration Parameters

Basic Configuration

• api_name (string, optional) - API Gateway REST API name
• stage_name (string, optional) - Deployment stage name (e.g., dev, prod)

Service Configuration

• resources (string, optional) - API resource definitions with paths and methods

Network Configuration

• cors_enabled (string, optional) - Enable CORS for cross-origin requests (optional)

services:
  my_api:
    type: api_gateway_v2
    configuration:
      api_name: "my-api-${parameters.environment}"
      stage_name: "dev"
      resources:
        # Lambda integration
        - path: "/users"
          methods: ["GET", "POST"]
          integration_type: "lambda"
          lambda_function: "${services.user_lambda.function_name}"
        
        # HTTP integration (public endpoints)
        - path: "/external"
          methods: ["GET"]
          integration_type: "http"
          integration_uri: "https://api.external-service.com/data"
        
        # Mock integration (testing)
        - path: "/health"
          methods: ["GET"]
          integration_type: "mock"
          mock_response:
            status_code: 200
            response_body: '{"status": "healthy"}'

Private Integration

The integration_type: "private" provides a simplified and more intuitive way to configure private resource integrations. It automatically handles path extraction, parameter conversion, and VPC Link setup, reducing configuration complexity and errors.

Why Private Integration?

Connect API Gateway to internal services (ECS, EKS, EC2) without exposing them to the internet. Ideal for security, compliance, and enterprise architectures requiring private backend connectivity.

services:
  enrichment_api:
    type: api_gateway_v2
    configuration:
      api_name: "enrichment-api-${parameters.environment}"
      stage_name: "dev"
      resources:
        - path: "/users"
          methods: ["GET"]
          integration_type: "private"
          integration_uri: "arn:aws:elasticloadbalancing:eu-west-1:123456789012:listener/net/my-nlb/abcd1234/efgh5678#/v1/users"
          vpc_link_id: "abc123"
        
        - path: "/users/{id}"
          methods: ["GET"]
          integration_type: "private"
          integration_uri: "arn:aws:elasticloadbalancing:eu-west-1:123456789012:listener/net/my-nlb/abcd1234/efgh5678#/v1/users/{id}"
          vpc_link_id: "abc123"

services:
  internal_api:
    type: api_gateway_v2
    configuration:
      api_name: "internal-api-${parameters.environment}"
      stage_name: "dev"
      resources:
        - path: "/api/data"
          methods: ["GET"]
          integration_type: "private"
          integration_uri: "https://internal-service.vpc.local/v1/data"
          vpc_link_id: "abc123"
        
        - path: "/api/data/{category}"
          methods: ["GET"]
          integration_type: "private"
          integration_uri: "https://internal-service.vpc.local/v1/data/{category}"
          vpc_link_id: "abc123"

Private Integration URI Formats

Authentication

services:
  secure_api:
    type: api_gateway_v2
    configuration:
      api_name: "secure-api-${parameters.environment}"
      stage_name: "prod"
      authorizers:
        - name: "jwt-auth"
          type: "JWT"
          configuration:
            # Cognito User Pool
            issuer: "https://cognito-idp.us-east-1.amazonaws.com/us-east-1_XXXXXXXXX"
            audience: ["your-app-client-id"]
            
            # Google OAuth (alternative)
            # issuer: "https://accounts.google.com"
            # audience: ["your-google-client-id.apps.googleusercontent.com"]
            
            # Auth0 (alternative)
            # issuer: "https://your-domain.auth0.com/"
            # audience: ["your-auth0-api-identifier"]
      resources:
        - path: "/protected"
          methods: ["GET", "POST"]
          integration_type: "lambda"
          lambda_function: "${services.api_lambda.function_name}"
          authorizer: "jwt-auth"

services:
  auth_api:
    type: api_gateway_v2
    configuration:
      api_name: "auth-api-${parameters.environment}"
      stage_name: "prod"
      authorizers:
        - name: "custom-auth"
          type: "REQUEST"
          configuration:
            authorizer_uri: "${services.auth_lambda.function_arn}"
            identity_sources: ["$request.header.Authorization"]
            authorizer_result_ttl_in_seconds: 300
          
          # Simplified format (alternative)
          # lambda_function: "custom-authorizer"
          # cache_ttl: 300
          # identity_sources: ["$request.header.Authorization", "$request.header.X-API-Key"]
      resources:
        - path: "/protected"
          methods: ["GET", "POST"]
          integration_type: "lambda"
          lambda_function: "${services.api_lambda.function_name}"
          authorizer: "custom-auth"

Advanced Features

services:
  production_api:
    type: api_gateway_v2
    configuration:
      api_name: "production-api-${parameters.environment}"
      stage_name: "prod"

      # CORS configuration
      cors:
        allow_credentials: true
        allow_headers: ["Content-Type", "Authorization"]
        allow_methods: ["GET", "POST", "PUT", "DELETE"]
        allow_origins: ["https://myapp.com"]
        max_age: 3600

      # Rate limiting
      throttling:
        burst_limit: 5000
        rate_limit: 2000.0

      resources:
        - path: "/api/{proxy+}"
          methods: ["GET", "POST", "PUT", "DELETE", "PATCH", "HEAD", "OPTIONS"]
          integration_type: "lambda"
          lambda_function: "${services.api_lambda.outputs.function_name}"

Configuration Reference

• api_name - HTTP API name (supports variables)
• stage_name - Deployment stage (default: "dev")
• description - API description
• resources - Route definitions with paths and methods
• authorizers - JWT and REQUEST authorizer definitions
• cors - Cross-origin resource sharing settings
• throttling - Rate limiting configuration

Integration Types

HTTP API v2 supports four integration types for connecting routes to backend services:

• lambda - AWS Lambda proxy integration for serverless functions

  integration_type: "lambda"
  lambda_function: "my-function-name"

• http - HTTP proxy integration for public HTTP/HTTPS endpoints

  integration_type: "http"
  integration_uri: "https://api.external.com/endpoint"

• private Simplified private resource integration

  integration_type: "private"
  integration_uri: "arn:aws:elasticloadbalancing:region:account:listener/net/nlb/id/listener-id#/v1/path/{id}"
  vpc_link_id: "vpc-link-id"

• mock - Mock responses for testing and prototyping

  integration_type: "mock"
  mock_response:
    status_code: 200
    response_body: '{"status": "ok"}'

Authorizer Types

• JWT Authorizers: Support Cognito User Pools and OIDC providers (Auth0, Google, etc.)
• REQUEST Authorizers: Custom Lambda-based authorization with configurable identity sources
• Route-level Authorization: Apply different authorizers to different routes

CORS & Throttling

• cors.allow_credentials - Allow cookies and authorization headers
• cors.allow_headers/methods/origins - Configure allowed headers, methods, and origins
• throttling.burst_limit - Maximum concurrent requests
• throttling.rate_limit - Steady-state requests per second

Key Benefits

• Cost: Up to 70% cheaper than REST APIs
• Performance: Up to 60% faster response times
• Configuration: Simple and intuitive resource definitions
• Features: Modern JWT/REQUEST authorizers, enhanced CORS, better throttling

services:
  complete_api:
    type: api_gateway_v2
    configuration:
      api_name: "complete-api-${parameters.environment}"
      stage_name: "prod"
      description: "Complete HTTP API with all features"

      # JWT and REQUEST authorizers
      authorizers:
        - name: "jwt-auth"
          type: "JWT"
          configuration:
            issuer: "https://cognito-idp.us-east-1.amazonaws.com/us-east-1_XXXXXXXXX"
            audience: ["client-id"]
        - name: "custom-auth"
          type: "REQUEST"
          configuration:
            authorizer_uri: "${services.auth_lambda.function_arn}"
            identity_sources: ["$request.header.X-API-Key"]
            authorizer_result_ttl_in_seconds: 300

      # CORS configuration
      cors:
        allow_credentials: true
        allow_headers: ["Content-Type", "Authorization"]
        allow_methods: ["GET", "POST", "PUT", "DELETE"]
        allow_origins: ["https://myapp.com"]
        max_age: 3600

      # Throttling limits
      throttling:
        burst_limit: 5000
        rate_limit: 2000.0

      resources:
        # Public endpoints
        - path: "/health"
          methods: ["GET"]
          integration_type: "mock"
          mock_response:
            status_code: 200
            response_body: '{"status": "healthy"}'

        # JWT protected endpoints
        - path: "/users"
          methods: ["GET", "POST"]
          integration_type: "lambda"
          lambda_function: "${services.user_lambda.function_name}"
          authorizer: "jwt-auth"

        - path: "/users/{id}"
          methods: ["GET", "PUT", "DELETE"]
          integration_type: "lambda"
          lambda_function: "${services.user_lambda.function_name}"
          authorizer: "jwt-auth"

        # Custom auth protected endpoints
        - path: "/admin/stats"
          methods: ["GET"]
          integration_type: "lambda"
          lambda_function: "${services.admin_lambda.function_name}"
          authorizer: "custom-auth"

        # HTTP proxy integration
        - path: "/external/{proxy+}"
          methods: ["GET", "POST"]
          integration_type: "http"
          integration_uri: "https://api.external.com/{proxy}"

      # Stage-specific configuration
      stage_configuration:
        auto_deploy: true
        throttling_burst_limit: 4000
        throttling_rate_limit: 1500.0

services:
  event_stream:
    type: kinesis_stream
    configuration:
      stream_name: "events-${parameters.environment}"
      shard_count: 2
      retention_period: 48

services:
  encrypted_stream:
    type: kinesis_stream
    configuration:
      stream_name: "secure-events-${parameters.environment}"
      shard_count: 4
      retention_period: 168  # 7 days
      encryption_type: "KMS"
      kms_key_id: "alias/aws/kinesis"

Configuration Parameters

Basic Configuration

• encryption_type (string, optional) - Encryption type ("KMS" for server-side encryption)
• stream_name (string, optional) - Kinesis stream name (supports environment variables)

Service Configuration

• retention_period (string, optional) - Data retention in hours (24-8760)
• role_arn (string, optional) - Custom IAM role ARN (optional, uses global default if not specified)
• shard_count (string, optional) - Number of shards (1-1000), affects throughput capacity

Security Configuration

• kms_key_id (string, optional) - KMS key for encryption (alias or key ID)

Parameter Validation

• stream_name: 1-128 characters, alphanumeric, hyphens, underscores, and periods only
  Examples: event-stream, user_activity, log.stream
• shard_count: 1-1000 shards
  Examples: 1, 5, 100

Common Validation Errors

• Parameter name contains invalid characters: Use only alphanumeric characters, hyphens, underscores, and periods
  Example: Change "my bucket" to "my-bucket"
• Parameter value exceeds maximum length: Reduce the parameter value to within the allowed range
  Example: Function names must be 64 characters or less
• Service reference not found: Ensure the referenced service exists and is properly named
  Example: Check that ${services.my-service.outputs.arn} references an existing service

IAM Integration

• Default: Uses global kinesis_stream_role with appropriate permissions
• Custom: Provide role_arn parameter for custom IAM role
• Validation: Custom roles are validated for correct permissions

services:
  order_worker:
    type: lambda_function
    repository: "./src"
    configuration:
      function_name: "order-worker-${system.workspace}"
      runtime: python3.11
      entry_point: handler.main
      code_path: order_worker

  order_events:
    type: sqs_queue
    depends_on: [order_worker]
    configuration:
      queue_name: "order-events-${system.workspace}"
      visibility_timeout: 120
      message_retention_period: 345600
      receive_message_wait_time_seconds: 20
      maximum_message_size: 262144
      kms_master_key_id: alias/aws/sqs
      redrive_policy:
        dead_letter_target_arn: "arn:aws:sqs:eu-central-1:123456789012:order-events-dlq"
        max_receive_count: 5
      provider:
        aws:
          lambda_triggers:
            - function_name: "${services.order_worker.outputs.function_name}"
              batch_size: 10
              maximum_batching_window_in_seconds: 5
              enabled: true
              function_response_types: [ReportBatchItemFailures]
              maximum_concurrency: 10
      tags:
        Workload: orders

services:
  billing_events:
    type: queue:sqs_queue
    configuration:
      queue_name: "billing-events-${system.workspace}.fifo"
      fifo: true
      content_based_deduplication: true
      deduplication_scope: messageGroup
      fifo_throughput_limit: perMessageGroupId

Configuration Parameters

• queue_name (string, required) - 1-80 alphanumeric, hyphen, or underscore characters. FIFO names must end in .fifo.
• fifo (boolean, default: false) - Creates a FIFO queue. A .fifo name and this flag must agree.
• content_based_deduplication (boolean, default: false) - Enables content-based deduplication for FIFO queues.
• visibility_timeout (integer, default: 30) - Visibility timeout in seconds, from 0 through 43,200.
• message_retention_period (integer, default: 345600) - Retention in seconds, from 60 through 1,209,600.
• delay_seconds (integer, default: 0) - Delivery delay in seconds, from 0 through 900.
• receive_message_wait_time_seconds (integer, default: 0) - Long-polling wait, from 0 through 20 seconds.
• maximum_message_size (integer, default: 262144) - Maximum message size in bytes, from 1,024 through 262,144.
• kms_master_key_id (string, optional) - Customer-managed key id/ARN or an AWS managed alias such as alias/aws/sqs.
• kms_data_key_reuse_period_seconds (integer, optional) - KMS data-key reuse period, from 60 through 86,400 seconds.
• deduplication_scope (FIFO only) - messageGroup or queue.
• fifo_throughput_limit (FIFO only) - perMessageGroupId or perQueue.
• redrive_policy (mapping, optional) - Requires dead_letter_target_arn and max_receive_count from 1 through 1,000.
• provider.aws.lambda_triggers (list, optional) - Creates or reconciles Lambda event-source mappings. Each item requires function_name and may set batch_size, maximum_batching_window_in_seconds, enabled, function_response_types, maximum_concurrency, and filter_criteria.
• tags (mapping, optional) - Additional string tags merged with service-level tags.

Outputs

References may use queue_name, queue_url, queue_arn, and console_url. A successful deployment with Lambda triggers also reports lambda_trigger_uuids.

services:
  data_pipeline:
    type: kinesis_firehose
    configuration:
      delivery_stream_name: "events-firehose-${parameters.environment}"
      destination_s3_bucket: "${services.data_lake.outputs.bucket_name}"
      prefix: "events/"
      buffering_size: 5
      buffering_interval: 300
      compression_format: "GZIP"

services:
  stream_to_s3:
    type: kinesis_firehose
    configuration:
      delivery_stream_name: "stream-to-s3-${parameters.environment}"
      source_service: "event_stream"  # Reference to Kinesis stream service
      destination_s3_bucket: "${services.data_lake.outputs.bucket_name}"
      prefix: "processed-events/"
      buffering_size: 10
      buffering_interval: 60
      compression_format: "GZIP"

services:
  stream_to_s3_alt:
    type: kinesis_firehose
    configuration:
      delivery_stream_name: "stream-alt-${parameters.environment}"
      source_stream_arn: "${services.event_stream.outputs.stream_arn}"
      destination_s3_bucket: "${services.data_lake.outputs.bucket_name}"
      prefix: "processed-events/"
      buffering_size: 10
      buffering_interval: 60
      compression_format: "GZIP"

services:
  advanced_firehose:
    type: kinesis_firehose
    configuration:
      delivery_stream_name: "advanced-firehose-${parameters.environment}"
      source_stream_arn: "${services.event_stream.outputs.stream_arn}"
      destination_s3_bucket: "${services.data_lake.outputs.bucket_name}"
      prefix: "events/year=!{timestamp:yyyy}/month=!{timestamp:MM}/day=!{timestamp:dd}/"
      error_output_prefix: "errors/"
      buffering_size: 64
      buffering_interval: 300
      compression_format: "GZIP"

      # Convert JSON to Parquet format
      data_format_conversion:
        enabled: true
        output_format: "parquet"
        schema_database: "${services.analytics_db.outputs.database_name}"
        schema_table: "events"

      # Custom IAM role for cross-account access
      role_arn: "arn:aws:iam::123456789012:role/FirehoseDeliveryRole"

services:
  high_throughput_firehose:
    type: kinesis_firehose
    configuration:
      delivery_stream_name: "high-throughput-${parameters.environment}"
      source_stream_arn: "${services.high_volume_stream.outputs.stream_arn}"
      destination_s3_bucket: "${services.data_lake.outputs.bucket_name}"
      prefix: "high-volume-data/"

      # Optimize for high throughput
      buffering_size: 128     # Buffer size in MB
      buffering_interval: 60  # Buffer interval in seconds
      compression_format: "GZIP"

      tags:
        Environment: "${parameters.environment}"
        DataType: "HighVolume"
        CostCenter: "Analytics"

Configuration Parameters

Basic Configuration

• delivery_stream_name (string, required) - Firehose delivery stream name (1-64 characters, alphanumeric, hyphens, underscores)

Service Configuration

• buffering_interval (integer, optional) - Buffer interval in seconds (60-900, default: 60)
• buffering_size (integer, optional) - Buffer size in MB (1-512, default: 64)
• compression_format (string, optional) - Data compression format (GZIP, ZIP, Snappy, HADOOP_SNAPPY, UNCOMPRESSED, default: GZIP)
• data_format_conversion (string, optional) - Convert data format (JSON to Parquet/ORC, optional)
• destination_s3_bucket (string, required) - Target S3 bucket for data delivery
• prefix (string, optional) - S3 key prefix for data organization (supports dynamic partitioning)
• error_output_prefix (string, optional) - S3 prefix for error records (optional)
• role_arn (string, optional) - Custom IAM role ARN (optional, uses global default if not specified)
• source_service (string, optional) - Source Kinesis stream service name (alternative to source_stream_arn)
• source_stream_arn (string, optional) - Source Kinesis stream ARN (alternative to source_service, for stream-to-S3 mode)
• partition_strategy (string, optional) - Optional partitioning strategy passed to the Firehose deployment adapter

Advanced Options

• tags (string, optional) - Resource tags (optional)

For backward compatibility, the loader also accepts buffer_size, buffer_interval, and destination_s3_prefix. New configurations should use the canonical names shown above.

Parameter Validation

• bucket_name: 3-63 characters, lowercase letters, numbers, hyphens, and periods only
  Examples: my-data-bucket, company-logs-2024, ml-model-artifacts
• versioning: Boolean value
  Examples: true, false

Common Validation Errors

• Parameter name contains invalid characters: Use only alphanumeric characters, hyphens, underscores, and periods
  Example: Change "my bucket" to "my-bucket"
• Parameter value exceeds maximum length: Reduce the parameter value to within the allowed range
  Example: Function names must be 64 characters or less
• Service reference not found: Ensure the referenced service exists and is properly named
  Example: Check that ${services.my-service.outputs.arn} references an existing service

Data Format Conversion

• Supported Formats: Convert JSON input to Parquet or ORC format
• Schema Integration: Uses AWS Glue Data Catalog for schema information
• Configuration:
  • enabled - Enable format conversion (true/false)
  • output_format - Target format ("parquet" or "orc")
  • schema_database - Glue database name for schema
  • schema_table - Glue table name for schema
• Benefits: Improved query performance and reduced storage costs

Dynamic Partitioning

• Time-based Partitioning: Use timestamp expressions in S3 prefix
  • year=!{timestamp:yyyy}/month=!{timestamp:MM}/ - Year/month partitions
  • !{timestamp:yyyy/MM/dd/HH}/ - Hourly partitions
• Content-based Partitioning: Extract values from record content
  • event_type=!{partitionKeyFromQuery:event_type}/ - Partition by event type
• Performance: Improves query performance by reducing data scanned

Buffering and Delivery

• Buffer Size: Amount of data (1-128 MB) to buffer before delivery
• Buffer Interval: Maximum time (60-900 seconds) to wait before delivery
• Delivery Trigger: Whichever condition is met first triggers delivery
• Optimization: Larger buffers reduce S3 PUT costs but increase latency

Compression Options

• GZIP: Good compression ratio, widely supported (recommended)
• ZIP: Compatible with many tools, moderate compression
• Snappy: Fast compression/decompression, lower ratio
• HADOOP_SNAPPY: Hadoop-compatible Snappy format
• UNCOMPRESSED: No compression, fastest delivery

Delivery Modes

• Direct PUT: Applications write directly to Firehose (no source configuration needed)
• Kinesis Data Streams Source: Firehose reads from existing stream
  • source_service - Reference to Kinesis stream service name (recommended)
  • source_stream_arn - Direct ARN reference (alternative approach)
• Use Cases: Direct PUT for simple ingestion, stream source for complex processing pipelines

Error Handling

• Error Records: Failed records are delivered to error output prefix
• Retry Logic: Automatic retries for transient failures
• Monitoring: CloudWatch metrics for delivery success/failure rates

IAM Integration

• Default: Uses global kinesis_firehose_role with appropriate permissions
• Custom: Provide role_arn parameter for custom IAM role
• Permissions: Role needs S3 write permissions and Kinesis read permissions (if using source stream)
• Cross-Account: Custom roles enable cross-account S3 delivery

services:
  event_processor:
    type: eventbridge_pipe
    configuration:
      pipe_name: "event-processor-${parameters.environment}"
      source_service: "event_stream"
      target_service: "processor_lambda"
      description: "Process events from stream to Lambda"

services:
  external_pipe:
    type: eventbridge_pipe
    configuration:
      pipe_name: "external-pipe-${parameters.environment}"
      source_arn: "arn:aws:kinesis:us-east-1:123456789012:stream/external-stream"
      target_arn: "arn:aws:lambda:us-east-1:123456789012:function:external-processor"
      description: "Process events from external Kinesis stream"

services:
  filtered_processor:
    type: eventbridge_pipe
    configuration:
      pipe_name: "filtered-processor-${parameters.environment}"
      source_service: "event_stream"
      target_service: "processor_lambda"
      description: "Process only high-priority events"
      source_parameters:
        kinesis_stream_parameters:
          batch_size: 10
          starting_position: "LATEST"
          maximum_batching_window_in_seconds: 5
      filter_criteria:
        filters:
          - pattern: '{"event_type": ["click", "purchase", "signup"]}'
          - pattern: '{"priority": ["high", "critical"]}'

services:
  sqs_processor:
    type: eventbridge_pipe
    configuration:
      pipe_name: "sqs-processor-${parameters.environment}"
      source_service: "message_queue"
      target_service: "message_processor"
      description: "Process SQS messages asynchronously"
      source_parameters:
        sqs_queue_parameters:
          batch_size: 5
      target_parameters:
        lambda_function_parameters:
          invocation_type: "FIRE_AND_FORGET"

services:
  # Primary processing pipe
  db_stream_processor:
    type: eventbridge_pipe
    configuration:
      pipe_name: "db-stream-processor-${parameters.environment}"
      source_arn: "${services.user_table.outputs.stream_arn}"
      target_service: "user_change_processor"
      description: "Process user table changes"
      source_parameters:
        dynamo_db_stream_parameters:
          batch_size: 5
          starting_position: "LATEST"
      filter_criteria:
        filters:
          - pattern: '{"eventName": ["INSERT", "MODIFY"]}'

  # Analytics pipe
  db_analytics_pipe:
    type: eventbridge_pipe
    configuration:
      pipe_name: "db-analytics-pipe-${parameters.environment}"
      source_arn: "${services.user_table.outputs.stream_arn}"
      target_service: "analytics_processor"
      description: "Send user changes to analytics"
      filter_criteria:
        filters:
          - pattern: '{"eventName": ["INSERT", "REMOVE"]}'

services:
  # Multi-stage processing with enrichment
  enriched_processor:
    type: eventbridge_pipe
    configuration:
      pipe_name: "enriched-processor-${parameters.environment}"
      source_service: "raw_events_stream"
      target_service: "enriched_processor_lambda"
      description: "Enrich and process raw events"

      source_parameters:
        kinesis_stream_parameters:
          batch_size: 25
          starting_position: "TRIM_HORIZON"
          maximum_batching_window_in_seconds: 5

      target_parameters:
        lambda_function_parameters:
          invocation_type: "REQUEST_RESPONSE"

      filter_criteria:
        filters:
          - pattern: '{"source": ["web", "mobile"]}'
          - pattern: '{"event_type": {"exists": true}}'
          - pattern: '{"user_id": {"exists": true}}'

      # Custom IAM role for cross-account access
      role_arn: "arn:aws:iam::123456789012:role/EventBridgePipeRole"

Configuration Parameters

Basic Configuration

• pipe_name (string, optional) - EventBridge pipe name (1-64 characters, alphanumeric, periods, hyphens, underscores)

Service Configuration

• description (string, optional) - Pipe description (optional)
• enrichment (string, optional) - Event enrichment configuration (optional)
• filter_criteria (string, optional) - Event filtering patterns using JSON pattern matching (optional)
• role_arn (string, optional) - Custom IAM role ARN (optional, uses global default if not specified)
• source_arn (string, optional) - Source ARN (alternative to source_service, for external or direct ARN specification)
• source_parameters (string, optional) - Source-specific configuration parameters (optional)
• source_service (string, optional) - Name of source service (simplified approach, mutually exclusive with source_arn)
• target_arn (string, optional) - Target ARN (alternative to target_service, for external or direct ARN specification)
• target_parameters (string, optional) - Target-specific configuration parameters (optional)
• target_service (string, optional) - Name of target service (simplified approach, mutually exclusive with target_arn)

Advanced Options

• tags (string, optional) - Resource tags (optional)

Parameter Validation

• function_name: Must be 1-64 characters, alphanumeric and hyphens only
  Examples: my-function, data-processor-prod, api-handler-v2
• memory_size: 128-10240 MB in 64MB increments
  Examples: 128, 512, 1024
• timeout: 1-900 seconds
  Examples: 30, 300, 900
• runtime: python3.8, python3.9, python3.10, python3.11, python3.12, nodejs18.x, java11, go1.x
  Examples: python3.11, nodejs18.x, java11

Common Validation Errors

• Parameter name contains invalid characters: Use only alphanumeric characters, hyphens, underscores, and periods
  Example: Change "my bucket" to "my-bucket"
• Parameter value exceeds maximum length: Reduce the parameter value to within the allowed range
  Example: Function names must be 64 characters or less
• Service reference not found: Ensure the referenced service exists and is properly named
  Example: Check that ${services.my-service.outputs.arn} references an existing service

Source Parameters

• Kinesis Stream Parameters (kinesis_stream_parameters):
  • batch_size - Number of records per batch (1-10000, default varies by source)
  • starting_position - Where to start reading (TRIM_HORIZON, LATEST, AT_TIMESTAMP)
  • maximum_batching_window_in_seconds - Maximum time to wait for batch (0-300 seconds)
• DynamoDB Stream Parameters (dynamo_db_stream_parameters):
  • batch_size - Number of records per batch (1-1000)
  • starting_position - Where to start reading (TRIM_HORIZON, LATEST)
  • maximum_batching_window_in_seconds - Maximum time to wait for batch
• SQS Parameters (sqs_queue_parameters):
  • batch_size - Number of messages per batch (1-10)
  • maximum_batching_window_in_seconds - Maximum time to wait for batch

Target Parameters

• Lambda Function Parameters (lambda_function_parameters):
  • invocation_type - How to invoke Lambda (REQUEST_RESPONSE, FIRE_AND_FORGET)
• SQS Parameters (sqs_queue_parameters):
  • message_group_id - Message group ID for FIFO queues

Event Sources & Targets

• Supported Sources:
  • Kinesis Data Streams
  • Amazon SQS queues
  • DynamoDB Streams
• Supported Targets:
  • AWS Lambda functions
  • Amazon SQS queues
  • Amazon SNS topics
  • EventBridge event buses
  • Kinesis Data Streams
  • Kinesis Data Firehose
  • AWS Step Functions state machines

Event Filtering

• JSON Pattern Matching: Use JSON patterns to filter events based on content
• Multiple Filters: Apply multiple filter patterns (OR logic between filters)
• Pattern Examples:
  • {"event_type": ["purchase", "signup"]} - Match specific event types
  • {"amount": [{"numeric": [">", 100]}]} - Numeric comparisons
  • {"user_id": {"exists": true}} - Check field existence
  • {"source": {"prefix": "web-"}} - String prefix matching
• Performance: Filtering reduces downstream processing and costs

Simplified Configuration Benefits

• Service Names: Use source_service and target_service instead of complex ARNs
• Automatic ARN Resolution: Deployer automatically resolves service names to ARNs during deployment
• Convention over Configuration: Follows ModelKnife's principle of minimal configuration
• Flexibility: Can still use source_arn and target_arn for external resources or direct ARN specification
• Validation: Service names are validated against defined services in the same configuration

IAM Integration

• Default: Uses global eventbridge_pipe_role with appropriate permissions
• Custom: Provide role_arn parameter for custom IAM role (useful for cross-account access)
• Permissions: Role needs permissions to read from source and write to target
• Validation: Custom roles are validated for correct permissions during deployment

backend: gcp
services:
  event_handler:
    type: function:cloud_function
    repository: "./src"
    configuration:
      function_name: "event-handler-${system.workspace}"
      runtime: python3.11
      entry_point: handler.handle
      code_path: event_handler
      timeout: 120
      memory_size: 512
      environment:
        LOG_LEVEL: INFO
      provider:
        gcp:
          region: europe-west1
          gen2: true
          trigger_http: true
          allow_unauthenticated: false
          memory: 512Mi
          service_account: runtime@example-project.iam.gserviceaccount.com

  artifact_store:
    type: object_storage_bucket:gcs_bucket
    configuration:
      bucket_name: "example-artifacts-${system.workspace}"
      versioning: true
      encryption:
        default_kms_key_name: projects/example/locations/europe/keyRings/app/cryptoKeys/data
      lifecycle_configuration:
        - action: {type: Delete}
          condition: {age: 90}
      provider:
        gcp:
          location: EUROPE-WEST1
          storage_class: STANDARD
          uniform_bucket_level_access: true
          public_access_prevention: enforced

  sessions:
    type: nosql_table:firestore_collection
    configuration:
      table_name: "sessions-${system.workspace}"
      ttl:
        enabled: true
        attribute: expires_at
      provider:
        gcp:
          database: "(default)"
          location: europe-west1
          create_database_if_missing: true
          create_placeholder_document: false
          delete_documents_on_delete: false

Provider-Specific Behavior

• Cloud Function: requires function_name, runtime, entry_point, code_path, and a service-level repository. Only HTTP-triggered functions are currently supported. A dotted entry point such as handler.handle is translated to handle.
• Cloud Storage: uses the shared bucket fields documented under S3 for bucket_name, versioning, encryption, and lifecycle_configuration; GCP extensions belong under provider.gcp.
• Firestore: uses table_name as the collection name. Firestore creates a collection when its first document is written unless create_placeholder_document is enabled.
• Safe deletion: Firestore documents are retained by default. delete_documents_on_delete opts into document removal. delete_database_on_delete is allowed only for a dedicated database, never (default).

Outputs

• Cloud Function: function_name, region, and url when returned by GCP.
• Cloud Storage: bucket_name and bucket_uri.
• Firestore: database, table_name, collection, and collection provisioning metadata.

Configuration Parameters

services:
  iris_inference:
    type: lambda_function
    repository: "./src"
    configuration:
      function_name: iris-inference
      entry_point: iris_inference_with_model.lambda_handler
      runtime: python3.9
      code_path: "lambda_functions/"

      # ModelKnife discovers requirements.txt inside code_path.
      build_strategy: auto      # auto | local | docker
      provider:
        aws:
          dependency_artifact: layer  # layer | zip

      timeout: 300
      memory_size: 512
      environment:
        MODEL_S3_PATH: "s3://my-bucket/models/"

Build Configuration Options

Build Strategy Details

# LOCAL Strategy - Fast, pure Python only
services:
  simple_lambda:
    type: lambda_function
    repository: "./src"
    configuration:
      function_name: simple-lambda
      runtime: python3.11
      entry_point: handler.main
      code_path: simple_lambda  # Put requirements.txt in this directory.
      build_strategy: local     # Fast pip install for pure Python packages.

# DOCKER Strategy - Native binaries supported
  ml_lambda:
    type: lambda_function
    repository: "./src"
    configuration:
      function_name: ml-lambda
      runtime: python3.11
      entry_point: inference.predict
      code_path: ml_lambda      # requirements.txt contains pandas/numpy/etc.
      build_strategy: docker    # Linux-compatible build.
      provider:
        aws:
          dependency_artifact: layer

Performance Optimizations

# First deployment - full build
🔧 Building dependencies with DOCKER strategy
📦 Creating Lambda layer ZIP: 42.3 MB
🚀 Publishing layer to AWS: mlknife-layer-76daf7e3
✅ Function deployed: 35.2 seconds

# Source-only change - optimized
🚀 Skipping dependency ZIP build - using existing layer
🎯 Layer already has ARN, skipping layer publishing
✅ Function updated: 16.1 seconds (50% faster!)

Docker Build Process

# Simplified equivalent of ModelKnife's automatic dependency build:
docker run --rm --platform linux/amd64 \
  --entrypoint "" \
  -v /local/build:/asset \
  -w /asset \
  public.ecr.aws/lambda/python:3.11 \
  bash -lc "pip install --no-cache-dir --disable-pip-version-check \
    -r /asset/requirements.txt -t /asset"

# Benefits:
# ✅ Linux-compatible binaries
# ✅ Exact Lambda runtime environment
# ✅ Handles complex native dependencies
# ✅ Automatic platform detection (arm64/amd64)

Common Use Cases

pandas==2.0.3
numpy==1.24.3
scikit-learn==1.3.0
joblib==1.3.1

requests==2.31.0
boto3==1.28.17
pydantic==2.1.1

Requirements File Best Practices

Troubleshooting

# Deploy with detailed logging
mk s deploy --service iris_inference --verbose

# Check detailed deployment status
mk s status --service iris_inference --details

# Validate configuration without deploying
mk s validate --service iris_inference