Pricing Docs MCP

CLI Tool

Run human QA tests from your terminal with the Runhuman CLI.

What is the CLI?

The Runhuman CLI is a command-line interface for managing QA tests, projects, templates, and API keys. It provides both interactive and JSON output modes, making it perfect for local development, CI/CD pipelines, and automation.

Why use the CLI?

  • 🚀 Fast and efficient terminal-based workflow
  • 🤖 JSON output mode for CI/CD integration
  • 📦 Local configuration management
  • 🎯 Interactive prompts for common tasks
  • 🔧 Complete control over all Runhuman features

Installation

# Install globally
npm install -g runhuman

# Or use with npx (no installation required)
npx runhuman --help

Requirements: Node.js 18+

Quick Start

# 1. Login with your API key
runhuman login

# 2. Create your first test
runhuman create https://myapp.com -d "Test the checkout flow"

# 3. Check the status
runhuman status <jobId>

# 4. Get results
runhuman results <jobId>

Core Commands

Job Management

Create and manage QA test jobs.

create [url]

Create a new QA test job.

# Basic usage
runhuman create https://myapp.com -d "Test checkout flow"

# With template
runhuman create https://myapp.com --template tmpl_abc123

# Synchronous (wait for result)
runhuman create https://myapp.com -d "Test login" --sync

# With custom output schema
runhuman create https://myapp.com -d "Test" --schema ./schema.json

# JSON output for scripting
runhuman create https://myapp.com -d "Test" --json

Options:

  • -d, --description <text> - Test instructions for the human tester
  • --template <id> - Use a pre-defined template
  • --duration <seconds> - Target test duration (default: 300)
  • --screen-size <size> - Screen size: desktop, mobile, tablet, laptop
  • --schema <path> - Path to JSON schema file for structured output
  • --sync - Wait for result before exiting (blocks until complete)
  • --json - Output as JSON instead of interactive display

Example schema file:

{
  "loginWorks": {
    "type": "boolean",
    "description": "Does login succeed?"
  },
  "redirectsToHome": {
    "type": "boolean",
    "description": "Redirects to dashboard after login?"
  },
  "issues": {
    "type": "array",
    "description": "List of any bugs or issues found"
  }
}

status <jobId>

Check the status of a test job.

runhuman status job_abc123
runhuman status job_abc123 --json

Shows current job status: pending, claimed, in_progress, completed, failed.

wait <jobId>

Wait for a job to complete with live status updates.

runhuman wait job_abc123
runhuman wait job_abc123 --timeout 300

Options:

  • --timeout <seconds> - Maximum wait time (default: 600)
  • --json - Output as JSON

results <jobId>

Display detailed test results.

runhuman results job_abc123
runhuman results job_abc123 --json  # For scripting

Shows:

  • Test success/failure
  • Structured data from output schema
  • Tester’s written feedback
  • Screenshots and video URL
  • Cost and duration

list

List all your test jobs.

# List all jobs
runhuman list

# Filter by status
runhuman list --status completed

# Filter by project
runhuman list --project proj_abc123

# Limit results
runhuman list --limit 20

# JSON output
runhuman list --json

Options:

  • --status <status> - Filter: pending, claimed, in_progress, completed, failed
  • --project <id> - Filter by project ID
  • --limit <number> - Maximum number of jobs (default: 50)
  • --offset <number> - Pagination offset
  • --json - Output as JSON

delete <jobId>

Delete a job permanently.

runhuman delete job_abc123
runhuman delete job_abc123 --force  # Skip confirmation

Options:

  • --force - Skip confirmation prompt

Authentication

Manage your authentication and account.

login

Authenticate with your API key.

runhuman login

You’ll be prompted to enter your API key. Get your key from the API Keys dashboard.

The key is stored securely in ~/.config/runhuman/config.json with restricted permissions (0600).

logout

Clear your authentication credentials.

runhuman logout

whoami

Show your current user information.

runhuman whoami
runhuman whoami --json

Displays:

  • Email
  • Account ID
  • Account creation date

tokens balance

Check your remaining token/credit balance.

runhuman tokens balance
runhuman tokens balance --json

tokens history

View your token usage history.

runhuman tokens history
runhuman tokens history --limit 50

Projects

Organize your tests into projects.

projects list

List all your projects.

runhuman projects list
runhuman projects ls  # alias
runhuman projects list --json

projects create <name>

Create a new project.

runhuman projects create "My App"
runhuman projects create "My App" -d "Production QA testing"
runhuman projects create "My App" --default-url https://myapp.com

Options:

  • -d, --description <text> - Project description
  • --default-url <url> - Default URL for tests
  • --github-repo <owner/repo> - Link GitHub repository

projects show <projectId>

Show details of a specific project.

runhuman projects show proj_abc123
runhuman projects info proj_abc123  # alias

projects update <projectId>

Update a project.

runhuman projects update proj_abc123 --name "New Name"
runhuman projects update proj_abc123 -d "Updated description"

Options:

  • --name <name> - New project name
  • -d, --description <text> - New description
  • --default-url <url> - New default URL
  • --github-repo <owner/repo> - New GitHub repository

projects delete <projectId>

Delete a project permanently.

runhuman projects delete proj_abc123
runhuman projects delete proj_abc123 --force

API Keys

Manage API keys for projects.

keys list

List all API keys for a project.

runhuman keys list --project proj_abc123
runhuman keys ls --project proj_abc123  # alias

Options:

  • --project <id> - Project ID (required)
  • --show-keys - Display full unmasked keys (use with caution)
  • --json - Output as JSON

keys create <projectId> <name>

Create a new API key.

runhuman keys create proj_abc123 "Production Key"
runhuman keys create proj_abc123 "CI/CD Key"

⚠️ Important: The full key is only shown once. Save it immediately.

keys show <keyId>

Show details of a specific API key.

runhuman keys show key_abc123
runhuman keys show key_abc123 --show-keys  # Display full key

keys delete <keyId>

Delete/revoke an API key.

runhuman keys delete key_abc123
runhuman keys revoke key_abc123  # alias
runhuman keys rm key_abc123      # alias

Templates

Create reusable test templates.

templates list

List all templates for a project.

runhuman templates list --project proj_abc123
runhuman templates ls --project proj_abc123  # alias

templates create <name>

Create a new template.

runhuman templates create "Smoke Test" \
  --project proj_abc123 \
  -d "Basic smoke test template" \
  --schema ./smoke-test-schema.json

Options:

  • --project <id> - Project ID (required)
  • -d, --description <text> - Template description
  • --duration <seconds> - Default test duration
  • --schema <path> - Path to output schema JSON file

templates show <templateId>

Show template details.

runhuman templates show tmpl_abc123

templates update <templateId>

Update a template.

runhuman templates update tmpl_abc123 --name "New Name"
runhuman templates update tmpl_abc123 -d "Updated description"

templates delete <templateId>

Delete a template.

runhuman templates delete tmpl_abc123

GitHub Integration

Connect GitHub repositories and test issues.

Link a GitHub repository to your project.

runhuman github link owner/repo --project proj_abc123

github repos

List all linked GitHub repositories.

runhuman github repos
runhuman github repos --json

github issues <repo>

List issues for a repository.

runhuman github issues owner/repo
runhuman github issues owner/repo --label bug
runhuman github issues owner/repo --state open

Options:

  • --label <label> - Filter by label
  • --state <state> - Filter by state: open, closed, all
  • --limit <number> - Maximum number of issues

github test <number>

Create a test job for a specific GitHub issue.

runhuman github test 42 --repo owner/repo

Options:

  • --repo <owner/repo> - Repository (required)
  • --template <id> - Use a template
  • --sync - Wait for result

github bulk-test

Test multiple GitHub issues at once.

runhuman github bulk-test --repo owner/repo --label needs-testing
runhuman github bulk-test --repo owner/repo --numbers 42,43,44

Options:

  • --repo <owner/repo> - Repository (required)
  • --label <label> - Test all issues with this label
  • --numbers <list> - Comma-separated issue numbers
  • --limit <number> - Maximum number of issues to test

Configuration

Manage CLI configuration.

config get <key>

Get a configuration value.

runhuman config get apiUrl
runhuman config get defaultProject

config set <key> <value>

Set a configuration value.

runhuman config set defaultProject proj_abc123
runhuman config set color false
runhuman config set apiUrl https://api.runhuman.com

Common settings:

  • apiUrl - API base URL
  • defaultProject - Default project ID
  • color - Enable/disable colored output (true/false)
  • outputFormat - Output format: interactive or json

config list

List all configuration values.

runhuman config list
runhuman config ls  # alias

Shows the full configuration hierarchy with source information.

config reset

Reset configuration to defaults.

# Reset project config only
runhuman config reset --project

# Reset global config
runhuman config reset --global

# Reset everything
runhuman config reset --all

Project Setup

Initialize and manage project setup.

init

Interactive project initialization wizard.

runhuman init

Creates a .runhumanrc file in your project directory with:

  • Default project ID
  • Common settings
  • Template configurations

watch [patterns]

Watch files and auto-create tests on changes.

# Watch all HTML files
runhuman watch "*.html"

# Watch multiple patterns
runhuman watch "*.html" "*.js"

# With custom test configuration
runhuman watch "*.html" --template tmpl_abc123

Options:

  • --template <id> - Template to use for auto-created tests
  • --description <text> - Default description
  • --project <id> - Project to use

Configuration System

The CLI uses a 5-level configuration hierarchy (highest to lowest priority):

1. CLI Flags (Highest Priority)

runhuman create --project proj_123 https://example.com

2. Environment Variables

export RUNHUMAN_API_KEY="your_key"
export RUNHUMAN_API_URL="https://runhuman.com"
export RUNHUMAN_PROJECT_ID="proj_abc123"
export RUNHUMAN_NO_COLOR="true"

Available environment variables:

  • RUNHUMAN_API_KEY - API authentication key
  • RUNHUMAN_API_URL - API base URL
  • RUNHUMAN_PROJECT_ID - Default project ID
  • RUNHUMAN_NO_COLOR - Disable colored output

3. Project Config

Create .runhumanrc in your project root:

{
  "defaultProject": "proj_abc123",
  "defaultTemplate": "tmpl_smoke_test",
  "color": true,
  "outputFormat": "interactive"
}

4. Global Config

Stored in ~/.config/runhuman/config.json:

{
  "apiUrl": "https://runhuman.com",
  "apiKey": "your_key_here",
  "color": true
}

Created automatically on first runhuman login.

5. Built-in Defaults (Lowest Priority)

Default values are used when nothing else is configured.

CI/CD Integration

The CLI is designed for seamless CI/CD integration with JSON output mode.

GitHub Actions

name: QA Test
on: [push]

jobs:
  qa-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Install Runhuman CLI
        run: npm install -g runhuman

      - name: Run Human QA Test
        env:
          RUNHUMAN_API_KEY: ${{ secrets.RUNHUMAN_API_KEY }}
        run: |
          runhuman create ${{ env.PREVIEW_URL }} \
            --description "Test main user flow" \
            --template "smoke-test" \
            --sync \
            --json > result.json

      - name: Check Result
        run: |
          if [ "$(jq -r '.result.passed' result.json)" = "true" ]; then
            echo "✅ QA test passed"
            exit 0
          else
            echo "❌ QA test failed"
            jq '.result' result.json
            exit 1
          fi

GitLab CI

qa-test:
  stage: test
  script:
    - npm install -g runhuman
    - |
      runhuman create $STAGING_URL \
        --description "Smoke test" \
        --sync \
        --json > result.json
    - |
      if [ "$(jq -r '.result.passed' result.json)" = "true" ]; then
        exit 0
      else
        exit 1
      fi

CircleCI

version: 2.1
jobs:
  qa-test:
    docker:
      - image: node:18
    steps:
      - checkout
      - run:
          name: Install CLI
          command: npm install -g runhuman
      - run:
          name: Run QA Test
          command: |
            runhuman create https://myapp.com \
              --template smoke-test \
              --sync \
              --json

Jenkins

pipeline {
  agent any
  stages {
    stage('QA Test') {
      steps {
        sh 'npm install -g runhuman'
        sh '''
          runhuman create ${STAGING_URL} \
            --description "Test checkout" \
            --sync \
            --json > result.json
        '''
        script {
          def result = readJSON file: 'result.json'
          if (!result.result.passed) {
            error("QA test failed")
          }
        }
      }
    }
  }
}

Advanced Usage

Parallel Test Execution

Create multiple jobs and wait for all to complete:

#!/bin/bash

# Create jobs
JOB1=$(runhuman create https://myapp.com/page1 -d "Test" --json | jq -r '.jobId')
JOB2=$(runhuman create https://myapp.com/page2 -d "Test" --json | jq -r '.jobId')
JOB3=$(runhuman create https://myapp.com/page3 -d "Test" --json | jq -r '.jobId')

# Wait for all to complete
runhuman wait $JOB1 --json &
runhuman wait $JOB2 --json &
runhuman wait $JOB3 --json &

wait

# Get results
runhuman results $JOB1
runhuman results $JOB2
runhuman results $JOB3

Template-Based Testing

Create reusable templates for common test scenarios:

# Create a smoke test template
runhuman templates create "Smoke Test" \
  --project proj_abc123 \
  -d "Quick smoke test of critical paths" \
  --duration 300 \
  --schema smoke-test-schema.json

# Use the template
runhuman create https://myapp.com --template tmpl_smoke_test

Watch Mode for Local Development

Automatically test on file changes:

# Watch HTML files, auto-test on changes
runhuman watch "dist/*.html" \
  --template tmpl_smoke_test \
  --description "Auto smoke test"

When a file changes, a new test job is created automatically.

JSON Output Mode

All commands support --json flag for machine-readable output.

Example:

runhuman create https://example.com -d "Test" --sync --json

Output:

{
  "jobId": "job_abc123",
  "status": "completed",
  "result": {
    "passed": true,
    "explanation": "All tests passed successfully",
    "data": {
      "loginWorks": true,
      "checkoutWorks": true
    }
  },
  "costUsd": 0.18,
  "testDurationSeconds": 120
}

Exit codes:

  • 0 - Success
  • 1 - Test failed or command error
  • 2 - Authentication error
  • 3 - Not found
  • 4 - Timeout

Troubleshooting

Authentication Errors

# Error: Unauthorized (401)
# Solution: Login again
runhuman logout
runhuman login

Network Timeouts

# Error: Connection timeout
# Solution: Check API URL and network
runhuman config get apiUrl
ping runhuman.com

Rate Limiting

# Error: Rate limit exceeded (429)
# Solution: Wait and retry, or check your plan limits
runhuman tokens balance

Config File Issues

# Reset config if corrupted
runhuman config reset --all

# Or manually delete
rm -rf ~/.config/runhuman

Missing Dependencies

# Reinstall CLI
npm uninstall -g runhuman
npm install -g runhuman

# Or use npx (no install needed)
npx runhuman --help

Full Command Reference

Jobs (7 commands)
  • runhuman create [url] - Create QA test
  • runhuman status <jobId> - Check job status
  • runhuman wait <jobId> - Wait for completion
  • runhuman results <jobId> - Show results
  • runhuman list - List jobs
  • runhuman delete <jobId> - Delete job
  • runhuman watch [patterns] - Auto-create tests on file changes
Authentication (4 commands)
  • runhuman login - Login with API key
  • runhuman logout - Logout
  • runhuman whoami - Show current user
  • runhuman tokens balance - Show token balance
  • runhuman tokens history - Show usage history
Projects (5 commands)
  • runhuman projects list - List projects
  • runhuman projects create <name> - Create project
  • runhuman projects show <id> - Show project
  • runhuman projects update <id> - Update project
  • runhuman projects delete <id> - Delete project
API Keys (4 commands)
  • runhuman keys list - List keys
  • runhuman keys create <projectId> <name> - Create key
  • runhuman keys show <keyId> - Show key details
  • runhuman keys delete <keyId> - Delete key
Templates (5 commands)
  • runhuman templates list - List templates
  • runhuman templates create <name> - Create template
  • runhuman templates show <id> - Show template
  • runhuman templates update <id> - Update template
  • runhuman templates delete <id> - Delete template
GitHub (5 commands)
  • runhuman github link <repo> - Link repository
  • runhuman github repos - List repos
  • runhuman github issues <repo> - List issues
  • runhuman github test <number> - Test issue
  • runhuman github bulk-test - Test multiple issues
Configuration (4 commands)
  • runhuman config get <key> - Get config value
  • runhuman config set <key> <value> - Set config value
  • runhuman config list - List all config
  • runhuman config reset - Reset config
Setup (1 command)
  • runhuman init - Initialize project

Total: 45 commands

Next Steps

TopicLink
REST API documentationAPI Reference
GitHub Actions integrationGitHub Actions
Practical examplesCookbook
Full technical specReference
npm packagenpmjs.com/package/runhuman
Source codeGitHub