Quick Start Guide

Prerequisites

Before you begin, ensure you have:

• Python 3.8+ installed on your system
• AWS CLI installed and configured with your credentials
• AWS Account with appropriate permissions for SageMaker, Lambda, S3, Step Functions, and IAM
• Git access to the ModelKnife repository (contact admin for access)

Step 1: Install ModelKnife

Install ModelKnife (see the Installation Guide for OS‑specific steps and options):

pip install git+ssh://git@github.com/naoo-AI/modelknife.git

Verify your CLI is available:

mk --help

Step 2: Team Setup (First Time)

If you're the first person setting up ModelKnife for your team, run the initialization command:

mk setup init

Details on admin vs developer setup are in Team Setup. Permission scope and least‑privilege roles are described in Installation ▸ AWS Permissions Setup.

Step 3: Verify Your Access

Check that everything is set up correctly:

mk setup status
mk team status

You should see your access level (Developer or Admin) and AWS configuration details.

Step 4: Create Your First Configuration

Create a small ML configuration with one serving service and one training pipeline. This keeps the first run simple while still showing ModelKnife's dual approach: stable services + iterative ML workflows.

name: mk-iris-quickstart
	author: your-name
	description: Minimal Iris example with one service and one pipeline
	
	parameters:
	  environment: dev
	  base_path: s3://${env.MLKNIFE_BUCKET}/${parameters.environment}/quickstart
	  data_path: ${parameters.base_path}
	
	executors:
	  python_processor:
	    type: sagemaker_processor
	    class: sagemaker.sklearn.processing.SKLearnProcessor
	    instance_type: ml.c5.xlarge
	    framework_version: 0.23-1
	
	services:
	  iris-inference-lambda:
	    type: function:lambda_function
	    repository: ./
	    configuration:
	      function_name: iris-inference-${parameters.environment}
	      runtime: python3.9
	      entry_point: iris_inference.lambda_handler
	      code_path: ./lambda_functions
	      provider:
	        aws:
	          dependency_artifact: layer
	      timeout: 30
	      memory_size: 256
      environment:
        MODEL_S3_PATH: ${parameters.data_path}/iris/model
    tags:
      service_type: model_inference
      model_type: iris_classifier
	
	modules:
	  prepare-iris:
	    executor: ${executors.python_processor}
	    repository: ./sagemaker_scripts
	    entry_point: iris_prepare.py
	    output_names: output_path
	    job_parameters:
	      output_path: ${parameters.data_path}/iris/features
	    description: "Prepare a small Iris feature dataset"
	
	  train-iris:
	    executor: ${executors.python_processor}
    repository: ./sagemaker_scripts
    entry_point: iris_train.py
    input_names: features_path
    output_names: model_output_path
    job_parameters:
      features_path: ${parameters.data_path}/iris/features
      model_output_path: ${parameters.data_path}/iris/model
    description: "SageMaker: train a simple classifier on Iris"
    depends_on:
      - prepare-iris

What This Configuration Does

The configuration demonstrates ModelKnife's dual lifecycle approach without introducing extra cloud services on the first run:

Services (Infrastructure - Deploy Once)

• iris-inference-lambda: A Lambda function for real-time model inference. It loads the trained model artifact from S3.

Pipelines (ML Workflows - Iterate Frequently)

• prepare-iris: Generates a small Iris CSV at ${parameters.data_path}/iris/features.
• train-iris (SageMaker): Reads the CSV from ${parameters.data_path}/iris/features, trains a multiclass logistic regression model, and saves iris_model.pkl to ${parameters.data_path}/iris/model.

This shows the complete ML lifecycle: the pipeline modules handle batch training, while the service provides real-time inference. ModelKnife uses your account's default roles configured via mk setup init, so no explicit role ARNs are required.

About the Iris Example

The Iris module is intentionally minimal: it writes a tiny feature dataset and trains a simple classifier. This lets you validate your environment with the least friction before expanding your pipeline.

cp -R docs/website/examples/quickstart-template ./my-mlknife-stack
	cd my-mlknife-stack
	
	export MLKNIFE_BUCKET=your-existing-artifact-bucket
	
	mk s deploy
	mk p deploy
	mk p run

Step 5: Deploy Infrastructure Services

Deploy the stable infrastructure services (Lambda function in this example):

mk s deploy

This creates your Lambda function for real-time model inference. You only need to do this once per environment - services provide stable infrastructure.

Step 6: Deploy ML Pipeline

Deploy your ML processing modules:

mk p deploy

This creates your SageMaker processing jobs and Step Functions workflow.

Step 7: Run Your Pipeline

Execute your ML pipeline:

mk p run

Monitor the execution:

mk p status
mk p runs

Step 8: Test Your Inference Service

Once the model is trained, you can test the Lambda function:

# Test the Lambda function with sample Iris data
aws lambda invoke \
  --function-name iris-inference-dev \
  --cli-binary-format raw-in-base64-out \
  --payload '{"sepal_length":5.1,"sepal_width":3.5,"petal_length":1.4,"petal_width":0.2}' \
  response.json

cat response.json

You should see a prediction like: {"prediction": "setosa", "confidence": 0.95}

Step 9: Visualize Your Workflow

Generate an interactive visualization of your pipeline:

mk p visualize

This opens an interactive DAG (Directed Acyclic Graph) in your browser showing your pipeline structure and dependencies.

Hands-on: Iris Quickstart

The quickest way to validate your setup is the Iris quickstart. It requires no input data; the first module creates the dataset and the second trains the model.

1. Copy the template and set your artifact bucket

cp -R docs/website/examples/quickstart-template ./my-mlknife-stack
	cd my-mlknife-stack
	
	export MLKNIFE_BUCKET=your-existing-artifact-bucket

2. Deploy and run both Iris modules

mk s deploy
	mk p deploy
	mk p run

3. Monitor outputs

mk p status
mk p runs --limit 5

# Outputs:
	# - Dataset: s3://$MLKNIFE_BUCKET/dev/quickstart/iris/features/iris.csv
	# - Model:   s3://$MLKNIFE_BUCKET/dev/quickstart/iris/model/iris_model.pkl

Common Commands

Here are the most frequently used ModelKnife commands:

# Deploy pipeline
mk p deploy

# Run entire pipeline
mk p run

# Run specific modules
mk p run --modules data-preprocessing,model-training

# Check status
mk p status

# View execution history
mk p runs

# Schedule pipeline
mk p schedule set --cron "0 9 * * 1-5" --timezone UTC

# Deploy all services
mk s deploy

# Deploy specific service
mk s deploy --service feature_store

# Check service status
mk s status

# Validate configuration
mk s validate

# View configuration
mk conf show

# Check team status
mk team status

# List team members
mk team list-users

Next Steps

Now that you have ModelKnife running, explore these topics:

• Core Concepts - Understanding services, pipelines, modules, and executors
• Configuration Reference - Complete YAML configuration guide
• Examples - Real-world ML pipeline examples
• Team Setup - Advanced team management and access control