Team Access Management
Set up role-based access control for your ModelKnife team using AWS IAM groups and permissions
Overview
ModelKnife provides team-based access control through AWS IAM groups. This allows you to manage who can deploy services, run pipelines, and perform administrative tasks.
Two-Tier Access Model
ModelKnife uses a simple two-tier access model designed for most ML teams:
- Developers: Can deploy and manage ML pipelines and services
- Administrators: Can manage team membership and AWS resources
Quick Navigation
This guide covers the complete setup process:
- Prerequisites - AWS account setup verification
- AWS Console Setup - Step-by-step IAM user creation and permission configuration
- Required IAM Permissions - Complete policy details
- Setup Process Explanation - What
mk setup init
creates - Team Management - Adding and managing team members
Prerequisites
Before running mk setup init
, ensure your AWS account and IAM user are properly configured with the necessary permissions.
Important
The first admin user who runs mk setup init
needs comprehensive IAM permissions to create AWS resources and manage team access. This is a one-time setup requirement.
1. AWS Account Setup Verification
Verify your AWS account is ready for ModelKnife setup:
# Check your AWS connection
aws sts get-caller-identity
Expected output should show your account ID, user ARN, and user ID:
{
"UserId": "AIDACKCEVSQ6C2EXAMPLE",
"Account": "123456789012",
"Arn": "arn:aws:iam::123456789012:user/your-username"
}
2. AWS Console Setup Instructions
This section provides detailed step-by-step instructions for setting up IAM permissions through the AWS Management Console. Follow these steps if you prefer using the web interface over CLI commands.
Security Best Practice
Do not use your AWS root account for ModelKnife setup. Create a dedicated IAM user with administrative permissions instead. The steps below will guide you through creating this user.
Two Setup Options
You can set up IAM permissions using either:
- Option A: AWS Managed Policy (AdministratorAccess) - Quick but very broad permissions
- Option B (Recommended): Custom Policy - Minimum required permissions for better security
Step 1: Access the AWS IAM Console
- Sign in to the AWS Management Console
- Navigate to the IAM service by:
- Typing "IAM" in the search bar at the top
- Or clicking "Services" → "Security, Identity, & Compliance" → "IAM"
- You should see the IAM dashboard with navigation options on the left
Step 2: Create IAM User (If Needed)
If you don't already have an IAM user for ModelKnife administration:
- Navigate to Users:
- Click "Users" in the left navigation panel
- Click the "Add users" button
- Configure User Details:
- Enter username:
modelknife-admin
(or your preferred name) - Check both access types:
- ✅ "Programmatic access" (for AWS CLI)
- ✅ "AWS Management Console access" (for web interface)
- For console access, choose "Custom password" and set a secure password
- Uncheck "Require password reset" if you don't want to change it on first login
- Click "Next: Permissions"
- Enter username:
- Set Permissions (Choose One Option):
- Add Tags (Optional):
- Add tags if your organization requires them
- Example: Key="Purpose", Value="ModelKnife Administration"
- Click "Next: Review"
- Review and Create:
- Review all settings
- Click "Create user"
- Important: Download the CSV file with credentials or copy the Access Key ID and Secret Access Key
- Store these credentials securely - you won't be able to see the secret key again
Step 3: Create Custom Policy (Recommended Option)
If you chose Option B above, follow these detailed steps to create the custom policy:
- Navigate to Policies:
- In the IAM console, click "Policies" in the left navigation
- Click the "Create policy" button
- Choose JSON Editor:
- Click the "JSON" tab (not the Visual editor)
- You'll see a default policy template
- Replace with ModelKnife Policy:
- Delete all existing content in the JSON editor
- Copy the complete policy from the code block in section 5 below
- Paste it into the JSON editor
- The policy should start with
{"Version": "2012-10-17"...
- Validate and Continue:
- Click "Next: Tags" (AWS will validate the JSON automatically)
- If you see any JSON errors, double-check that you copied the complete policy
- Add Tags (Optional):
- Add tags if required by your organization
- Click "Next: Review"
- Name and Create Policy:
- Policy name:
ModelKnifeAdminPolicy
- Description:
Minimum required permissions for ModelKnife setup and administration
- Review the policy summary to ensure it includes IAM permissions
- Click "Create policy"
- Policy name:
Step 4: Attach Policy to Existing User
If you already have an IAM user and need to add ModelKnife permissions:
- Navigate to Your User:
- Click "Users" in the left navigation
- Click on your username from the list
- Add Permissions:
- Click the "Permissions" tab
- Click "Add permissions" button
- Select "Attach existing policies directly"
- Select Policy:
- For AWS managed policy: Search for "AdministratorAccess" and check it
- For custom policy: Search for "ModelKnifeAdminPolicy" and check it
- Click "Next: Review"
- Review and Apply:
- Review the permissions being added
- Click "Add permissions"
Step 5: Generate Access Keys (If Needed)
If your user doesn't have programmatic access keys for AWS CLI:
- Access Security Credentials:
- In your user's details page, click the "Security credentials" tab
- Scroll down to "Access keys" section
- Create Access Key:
- Click "Create access key"
- Select "Command Line Interface (CLI)" as the use case
- Check the confirmation checkbox
- Click "Next"
- Add Description (Optional):
- Add a description like "ModelKnife CLI access"
- Click "Create access key"
- Save Credentials:
- Important: Copy both the Access Key ID and Secret Access Key
- Or click "Download .csv file" to save them
- Store these securely - you won't be able to see the secret key again
- Click "Done"
Step 6: Verify Console Setup
After completing the console setup, verify everything is configured correctly:
Test your setup with these commands:
# Test AWS connection
aws sts get-caller-identity
# Test IAM permissions
aws iam get-user
# Test policy listing (should not give permission errors)
aws iam list-policies --max-items 1
Console Setup Troubleshooting
Error: "Policy document is invalid"
Cause: JSON syntax error in custom policy
Solution:
- Ensure you copied the complete policy JSON from section 5
- Check for missing commas, brackets, or quotes
- Use a JSON validator to check syntax
- Try copying the policy again from the source
Error: "Cannot find policy ModelKnifeAdminPolicy"
Cause: Custom policy not created or named differently
Solution:
- Go to IAM → Policies and verify the policy exists
- Check the exact policy name (case-sensitive)
- If missing, recreate the policy following the Step 3 instructions above
- Refresh the user permissions page
Error: "Access keys not working with AWS CLI"
Cause: Incorrect AWS CLI configuration
Solution:
- Run
aws configure
and re-enter credentials - Verify access key ID and secret key are correct
- Check that the user has programmatic access enabled
- Ensure no extra spaces in the credentials
Error: "User already exists" when creating user
Cause: Username already taken in your AWS account
Solution:
- Choose a different username (e.g., "mk-admin", "modelknife-setup")
- Or use an existing user and just attach the policy
- Check if the existing user already has the required permissions
Console Setup Tips
- Browser Compatibility: Use a modern browser (Chrome, Firefox, Safari, Edge)
- Multiple Tabs: Keep the policy creation tab open while creating users
- Credential Security: Never share access keys or store them in code repositories
- Policy Testing: Use AWS IAM Policy Simulator to test permissions before applying
- Documentation: Keep a record of the username and policy name for future reference
Security Best Practices
- Unique Usernames: Use descriptive names like "modelknife-admin" instead of generic names
- Strong Passwords: Use complex passwords for console access
- Access Key Rotation: Regularly rotate access keys (every 90 days recommended)
- Principle of Least Privilege: Use the custom policy instead of AdministratorAccess when possible
- MFA Recommendation: Consider enabling Multi-Factor Authentication for admin users
3. AWS CLI Configuration
Configure AWS CLI with your IAM user credentials:
# Configure AWS CLI
aws configure
# Or configure a specific profile
aws configure --profile modelknife-admin
You'll need to provide:
- AWS Access Key ID: From your IAM user
- AWS Secret Access Key: From your IAM user
- Default region: Your preferred AWS region (e.g., us-east-1)
- Default output format: json (recommended)
4. Required IAM Permissions
The admin user needs comprehensive IAM permissions to run mk setup init
. This section provides the complete list of required permissions organized by category, along with explanations for why each permission is needed.
Principle of Least Privilege
These permissions follow the principle of least privilege - they provide exactly what's needed for ModelKnife setup, no more. The setup process creates service-specific roles that handle AWS service permissions, so your admin user only needs IAM management capabilities.
Complete IAM Permissions Policy
Use this complete policy for your admin user. You can either copy the JSON below or download the ready-to-use policy file:
You can download the policy file directly to use in the AWS Console:
Or copy and paste this JSON directly into a custom IAM policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "IAMRoleManagement",
"Effect": "Allow",
"Action": [
"iam:CreateRole",
"iam:GetRole",
"iam:AttachRolePolicy",
"iam:DetachRolePolicy",
"iam:PutRolePolicy",
"iam:GetRolePolicy",
"iam:DeleteRolePolicy",
"iam:ListAttachedRolePolicies",
"iam:ListRolePolicies",
"iam:PassRole"
],
"Resource": "*",
"Condition": {
"StringLike": {
"iam:RoleName": "mlknife-*"
}
}
},
{
"Sid": "IAMGroupManagement",
"Effect": "Allow",
"Action": [
"iam:CreateGroup",
"iam:GetGroup",
"iam:AttachGroupPolicy",
"iam:DetachGroupPolicy",
"iam:AddUserToGroup",
"iam:RemoveUserFromGroup",
"iam:ListGroupsForUser"
],
"Resource": "*",
"Condition": {
"StringLike": {
"iam:GroupName": "mlknife-*"
}
}
},
{
"Sid": "IAMPolicyManagement",
"Effect": "Allow",
"Action": [
"iam:CreatePolicy",
"iam:GetPolicy",
"iam:GetPolicyVersion"
],
"Resource": "*",
"Condition": {
"StringLike": {
"iam:PolicyName": "MLKnife*"
}
}
},
{
"Sid": "IAMReadOnlyOperations",
"Effect": "Allow",
"Action": [
"iam:GetUser",
"iam:GetAccountSummary",
"iam:ListUsers",
"iam:ListGroups",
"iam:ListPolicies",
"iam:ListRoles"
],
"Resource": "*"
},
{
"Sid": "STSOperations",
"Effect": "Allow",
"Action": [
"sts:GetCallerIdentity"
],
"Resource": "*"
}
]
}
Permission Categories Explained
Alternative: Using AWS Managed Policies
If you prefer to use AWS managed policies instead of the custom policy above, you can attach these policies to your admin user:
# Option 1: Full administrative access (broadest permissions)
arn:aws:iam::aws:policy/AdministratorAccess
# Option 2: IAM-specific administrative access (more restrictive)
arn:aws:iam::aws:policy/IAMFullAccess
How to Use the Copied Policy
After copying the policy JSON above, follow these steps in the AWS Console:
- Go to AWS IAM Policies
- Click "Create policy"
- Click the "JSON" tab
- Paste the copied policy JSON
- Click "Next: Tags" → "Next: Review"
- Name it "ModelKnifeAdminPolicy"
- Click "Create policy"
- Attach the policy to your IAM user
Managed Policy Limitations
AdministratorAccess: Very broad permissions but may cause "Invalid principal" errors in some AWS regions/accounts due to service principal restrictions.
IAMFullAccess: Provides all IAM permissions but lacks STS permissions needed for account identification.
Recommendation: Use the custom policy above for the most reliable setup experience.
Additional Policy Examples
For specific organizational requirements, here are additional policy examples you can use:
Minimal IAM-Only Policy
If your organization requires the absolute minimum permissions, use this IAM-only policy. Note that this may require additional permissions during setup:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"iam:CreateRole",
"iam:GetRole",
"iam:AttachRolePolicy",
"iam:PutRolePolicy",
"iam:PassRole",
"iam:CreateGroup",
"iam:GetGroup",
"iam:AttachGroupPolicy",
"iam:CreatePolicy",
"iam:GetPolicy",
"iam:GetUser",
"iam:AddUserToGroup",
"sts:GetCallerIdentity"
],
"Resource": "*"
}
]
}
Organization-Scoped Policy
For organizations that want to restrict ModelKnife resources to specific naming patterns:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"iam:CreateRole",
"iam:GetRole",
"iam:AttachRolePolicy",
"iam:PutRolePolicy",
"iam:PassRole"
],
"Resource": "arn:aws:iam::*:role/mlknife-*"
},
{
"Effect": "Allow",
"Action": [
"iam:CreateGroup",
"iam:GetGroup",
"iam:AttachGroupPolicy"
],
"Resource": "arn:aws:iam::*:group/mlknife-*"
},
{
"Effect": "Allow",
"Action": [
"iam:CreatePolicy",
"iam:GetPolicy"
],
"Resource": "arn:aws:iam::*:policy/MLKnife*"
},
{
"Effect": "Allow",
"Action": [
"iam:GetUser",
"iam:AddUserToGroup",
"sts:GetCallerIdentity"
],
"Resource": "*"
}
]
}
What mk setup init Creates For You
The setup command automatically creates all AWS service permissions and roles. You don't need to manually configure:
- 9 IAM service roles (SageMaker, Lambda, Glue, Step Functions, etc.)
- Service-specific policies for S3, DynamoDB, OpenSearch, etc.
- Team access groups with appropriate permissions
- All role trust relationships and policy attachments
You only need IAM management permissions - the setup handles everything else automatically.
Why Only IAM Permissions?
The setup process is designed to be simple - you only provide IAM management permissions, and ModelKnife creates everything else:
- You provide: IAM permissions to create roles, groups, and policies
- Setup creates: All service roles with appropriate AWS service permissions
- Result: Team members get permissions through groups, not direct user policies
This approach is more secure and manageable than giving broad permissions to individual users.
6. Permission Verification Steps
Before running mk setup init
, verify that your IAM permissions are correctly configured using these validation commands:
Basic AWS Access Verification
# 1. Verify AWS connection and identity
aws sts get-caller-identity
# 2. Check your current IAM user details
aws iam get-user
# 3. Verify AWS region configuration
aws configure get region
IAM Permission Validation
Test specific IAM permissions required for ModelKnife setup:
# Test role management permissions
aws iam list-roles --max-items 1
# Test group management permissions
aws iam list-groups --max-items 1
# Test policy management permissions
aws iam list-policies --scope Local --max-items 1
# Test user management permissions
aws iam list-users --max-items 1
Advanced Permission Testing
For more thorough validation, test actual resource creation (these commands create and immediately delete test resources):
Optional Advanced Testing
These commands create temporary resources to test permissions. They're optional but provide the most accurate validation.
# Test role creation (creates and deletes a test role)
aws iam create-role --role-name mlknife-test-role \
--assume-role-policy-document '{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {"Service": "lambda.amazonaws.com"},
"Action": "sts:AssumeRole"
}]
}' && aws iam delete-role --role-name mlknife-test-role
# Test group creation (creates and deletes a test group)
aws iam create-group --group-name mlknife-test-group && \
aws iam delete-group --group-name mlknife-test-group
Expected Verification Results
All verification commands should complete successfully:
Returns your AWS account ID, user ARN, and user ID
Returns your IAM user details without permission errors
Returns list of existing IAM roles (proves read access)
Returns list of existing IAM groups
Successfully creates and deletes test resources
Troubleshooting Permission Issues
Error: "AccessDenied" on list operations
Cause: Missing basic IAM read permissions
Solution:
- Ensure your IAM user has the custom policy from section 5
- Or attach
IAMReadOnlyAccess
managed policy as minimum - Contact your AWS administrator to grant permissions
Error: "User not found" or "Invalid credentials"
Cause: AWS CLI not configured or invalid credentials
Solution:
- Run
aws configure
to set up credentials - Check credentials:
aws configure list
- Verify access keys are active in IAM console
Error: "Cannot create role" during advanced testing
Cause: Missing role creation permissions
Solution:
- Verify the custom policy includes all IAM role permissions
- Check that the policy is attached to your user account
- Ensure you're using the correct AWS profile/credentials
- Try the minimal policy validation commands first
Policy Validation Commands
Use these commands to validate your policy is working correctly before running mk setup init
:
# Validate policy JSON syntax (if you have the file locally)
aws iam validate-policy-document --policy-document file://modelknife-admin-policy.json
# Test specific IAM permissions
aws iam simulate-principal-policy \
--policy-source-arn $(aws sts get-caller-identity --query Arn --output text) \
--action-names iam:CreateRole iam:CreateGroup iam:CreatePolicy \
--resource-arns "*"
# Quick permission check - should return without errors
aws iam get-account-summary --query 'SummaryMap.{Users:Users,Groups:Groups,Roles:Roles}' --output table
Copy-Paste Policy Verification
After copying and pasting the policy JSON, verify it was applied correctly:
# List policies attached to your user
aws iam list-attached-user-policies --user-name $(aws sts get-caller-identity --query 'Arn' --output text | cut -d'/' -f2)
# Get the policy document to verify it matches
aws iam get-policy --policy-arn arn:aws:iam::$(aws sts get-caller-identity --query Account --output text):policy/ModelKnifeAdminPolicy \
--query 'Policy.{PolicyName:PolicyName,Description:Description}' --output table
# Test the policy allows basic operations
aws iam list-roles --max-items 1 --query 'Roles[0].RoleName' --output text
iam:CreateRole
and iam:DeleteRole
permissionsAutomated Permission Verification
For convenience, here are comprehensive validation commands that test all required permissions:
# Comprehensive permission validation script
# Save this as validate-permissions.sh and run: bash validate-permissions.sh
#!/bin/bash
echo "=== ModelKnife IAM Permission Validation ==="
echo "Testing all required permissions for mk setup init..."
echo
# Test 1: Basic AWS access
echo "1. Testing AWS connection..."
if aws sts get-caller-identity > /dev/null 2>&1; then
echo " ✅ AWS connection successful"
USER_ARN=$(aws sts get-caller-identity --query 'Arn' --output text)
echo " User: $USER_ARN"
else
echo " ❌ AWS connection failed"
exit 1
fi
# Test 2: IAM read permissions
echo "2. Testing IAM read permissions..."
aws iam get-user > /dev/null 2>&1 && echo " ✅ IAM user read access" || echo " ❌ IAM user read failed"
aws iam list-roles --max-items 1 > /dev/null 2>&1 && echo " ✅ IAM role list access" || echo " ❌ IAM role list failed"
aws iam list-groups --max-items 1 > /dev/null 2>&1 && echo " ✅ IAM group list access" || echo " ❌ IAM group list failed"
# Test 3: IAM creation permissions (creates and deletes test resources)
echo "3. Testing IAM creation permissions..."
if aws iam create-role --role-name mlknife-test-role --assume-role-policy-document '{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {"Service": "lambda.amazonaws.com"},
"Action": "sts:AssumeRole"
}]
}' > /dev/null 2>&1; then
echo " ✅ IAM role creation successful"
aws iam delete-role --role-name mlknife-test-role > /dev/null 2>&1
echo " ✅ IAM role deletion successful"
else
echo " ❌ IAM role creation failed"
fi
if aws iam create-group --group-name mlknife-test-group > /dev/null 2>&1; then
echo " ✅ IAM group creation successful"
aws iam delete-group --group-name mlknife-test-group > /dev/null 2>&1
echo " ✅ IAM group deletion successful"
else
echo " ❌ IAM group creation failed"
fi
echo
echo "=== Validation Complete ==="
echo "If all tests show ✅, you're ready to run 'mk setup init'"
Or run individual validation commands:
# Quick validation commands
aws sts get-caller-identity # Should return your user info
aws iam get-user # Should return your IAM user details
aws iam list-roles --max-items 1 # Should list roles without errors
aws iam list-groups --max-items 1 # Should list groups without errors
# Test policy attachment (using a harmless AWS managed policy)
aws iam attach-user-policy --user-name YOUR_USERNAME --policy-arn arn:aws:iam::aws:policy/IAMReadOnlyAccess
aws iam detach-user-policy --user-name YOUR_USERNAME --policy-arn arn:aws:iam::aws:policy/IAMReadOnlyAccess
This script will test all permissions and provide a detailed report of what works and what needs to be fixed.
Permission Validation Tips
- Start Simple: Begin with basic verification commands before advanced testing
- Check Policies: Use AWS IAM console to review attached policies
- Test Incrementally: If one command fails, fix that before proceeding
- Use Simulation: AWS IAM Policy Simulator can test permissions without creating resources