IAM Configuration System

ModelKnife uses a configuration-driven IAM system with standardized role naming. Understanding this architecture helps administrators customize permissions and troubleshoot access issues.

Configuration Overview

ModelKnife uses two configuration files to manage IAM resources:

  • System Config: Standardized roles and policies (same for all accounts)
  • User Config: Account-specific customizations (admin-only)

Configuration Architecture

The IAM system uses a two-file architecture that separates standard configuration from customizations:

File Structure

# IAM configuration location
~/.mlknife/iam-config/
├── system-config.json                    # Standard roles and policies
└── user-config-{account_id}-{region}.json  # Account customizations (optional)

System Configuration

The system configuration defines standardized IAM resources used across all AWS accounts:

system-config.json (Generated Automatically) 
{
  "version": "1.0",
  "description": "Standardized IAM configuration",
  "environment": {
    "account_id": "123456789012",
    "region": "us-east-1"
  },
  "iam_roles": {
    "lambda_execution_role": {
      "role_name_template": "mlknife-lambda-execution",
      "description": "Lambda execution role",
      "trust_policy": {
        "Version": "2012-10-17",
        "Statement": [{
          "Effect": "Allow",
          "Principal": { "Service": "lambda.amazonaws.com" },
          "Action": "sts:AssumeRole"
        }]
      }
    },
    "step_functions_role": {
      "role_name_template": "mlknife-stepfunctions",
      "description": "Step Functions execution role",
      "trust_policy": {
        "Version": "2012-10-17",
        "Statement": [{
          "Effect": "Allow",
          "Principal": { "Service": "states.amazonaws.com" },
          "Action": "sts:AssumeRole"
        }]
      }
    }
  },
  "iam_groups": {
    "mlknife-developers": {
      "description": "Developer access group",
      "attached_policies": ["mlknife-developer-policy"]
    },
    "mlknife-admin": {
      "description": "Administrator access group",
      "attached_policies": ["mlknife-admin-policy"]
    }
  }
}

Standardized Role Naming

All IAM roles use consistent, predictable names across AWS accounts:

Role Naming Pattern

Format: mlknife-{service}

Service Roles:
  • mlknife-stepfunctions
  • mlknife-lambda-execution
  • mlknife-sagemaker-exec
  • mlknife-glue-job
Team Groups:
  • mlknife-developers
  • mlknife-admin

User Configuration (Admin-Only)

Administrators can create account-specific customizations when needed:

user-config-123456789012-us-east-1.json (Optional) 
{
  "version": "1.0",
  "description": "Account-specific customizations",
  "custom_policies": {
    "additional-s3-access": {
      "description": "Custom S3 bucket access",
      "policy_document": {
        "Version": "2012-10-17",
        "Statement": [{
          "Effect": "Allow",
          "Action": ["s3:GetObject", "s3:PutObject"],
          "Resource": "arn:aws:s3:::company-bucket/*"
        }]
      }
    }
  },
  "role_overrides": {
    "lambda_execution_role": {
      "additional_policies": ["additional-s3-access"]
    }
  }
}

Admin-Only Feature

Only users in the mlknife-admin group can create or modify user configuration files. This ensures security and consistency across the team.

Configuration Generation Process

When mk setup init runs, it follows this process:

Setup Process

  1. Load system configuration: Reads standardized role definitions from source code
  2. Detect environment: Gets AWS account ID and region
  3. Generate roles: Creates IAM roles using the standardized names
  4. Create groups: Sets up team access groups with appropriate policies
  5. Save configuration: Stores generated config in ~/.mlknife/iam-config/

Configuration Commands

# View system configuration
mk setup status --detail

# Reset configuration to defaults (admin only)
mk setup reset --confirm

# View configuration file locations
ls ~/.mlknife/iam-config/

Advanced Configuration

Configuration File Locations

# System configuration (auto-generated)
~/.mlknife/iam-config/system-config.json

# User configuration (admin-created)
~/.mlknife/iam-config/user-config-{account_id}-{region}.json

# Source configuration (in ModelKnife installation)
{mlknife_install}/mlknife/utils/iam/defaults/system_config.py

Configuration Precedence

When multiple configuration sources exist, ModelKnife uses this precedence order:

  1. User config overrides: Account-specific customizations take highest priority
  2. Generated system config: Standard configuration for the account
  3. Default system config: Built-in defaults from source code

Regenerating Configuration

Administrators can regenerate configuration from the latest source code:

# Regenerate from source (preserves user config)
mk setup reset --confirm
mk setup init

Troubleshooting Configuration Issues

Configuration File Problems

Missing Configuration Files

Problem: mk setup status shows "configuration not found"

Solution: Run mk setup init to generate configuration files

Outdated Configuration

Problem: Role names don't match current standardized format

Solution: Regenerate configuration:

mk setup reset --confirm
mk setup init

Role Creation Issues

Role Already Exists

Problem: IAM role creation fails with "already exists" error

Solution: Check existing role configuration and update if needed

# Check existing roles in AWS
aws iam list-roles --query 'Roles[?starts_with(RoleName, `mlknife-`)].RoleName'

# Update existing roles instead of creating new ones
mk setup init --update

Debugging Configuration

# View current configuration
mk setup status --detail

# Check configuration file syntax
python -m json.tool ~/.mlknife/iam-config/system-config.json

# Validate IAM resources in AWS
aws iam list-roles --query 'Roles[?starts_with(RoleName, `mlknife-`)].{Name:RoleName,Created:CreateDate}' --output table

Related Documentation

See Also