Workflow CLI Overview

Workflow CLI Overview

The Workflow CLI provides a declarative, workflow-driven approach to managing OpenSearch migrations on Kubernetes. Instead of running individual commands, you define your migration configuration and submit it as a workflow that Argo Workflows orchestrates.

What is the Workflow CLI

The Workflow CLI is a command-line tool that you run from within the Migration Console pod. It lets you:

• Configure migrations using declarative YAML
• Submit migrations as Argo Workflows
• Monitor workflow execution and progress
• Manage workflow lifecycle (approve, stop, retry)

To use the Workflow CLI, first connect to the Migration Console pod:

For EKS deployments, configure kubectl:

aws eks update-kubeconfig --region <REGION> --name migration-eks-cluster-<STAGE>-<REGION>

Connect to the Migration Console pod:

kubectl exec -it migration-console-0 -n ma -- /bin/bash

Then run workflow commands from within the pod.

Key concepts

Declarative configuration

Instead of running commands step-by-step, you declare what you want to migrate. The configuration schema defines source clusters, target clusters, authentication, snapshot repositories, and migration options.

To see the complete configuration schema for your installed version:

workflow configure sample

The system determines how to execute the migration based on your configuration.

Argo Workflows orchestration

Argo Workflows provides:

• Parallel execution: Multiple migration tasks run simultaneously
• Retry logic: Automatic retry on transient failures
• Progress tracking: Visual workflow status
• Resource management: Controlled pod scheduling
• Approval gates: Human checkpoints before critical operations

You don't interact with Argo Workflows directly—the Workflow CLI handles this for you.

State management

Workflow state and coordination data are stored in a distributed key-value store:

• Configuration: Runtime settings accessible to all workflow steps
• Coordination: Synchronization between parallel tasks
• Status: Shared state for monitoring progress

This state is managed automatically; you don't need to interact with it directly.

How it works

1. Configuration phase

Create a migration configuration using the workflow configure commands:

# Load a sample configuration as your starting point
workflow configure sample --load

# Edit the configuration
workflow configure edit

# View the configuration
workflow configure view

2. Submission phase

Submit your configuration as a workflow:

workflow submit

This command:

• Validates your configuration
• Transforms it to run as an Argo Workflow
• Initializes workflow state
• Submits to Argo Workflows
• Prints the workflow name for tracking (runs asynchronously)

3. Execution phase

Argo Workflows executes the migration in stages:

1. Initialize: Set up workflow environment
2. Create snapshots: Snapshot source cluster indexes
3. Register snapshots: Make snapshots available to target
4. Migrate metadata: Transfer index templates and settings
5. Load documents: Bulk load documents from snapshots
6. Cleanup: Remove temporary state

4. Monitoring phase

Track progress with status commands:

# Interactive TUI to view progress, check logs, and approve steps
workflow manage

# View workflow progress
workflow status

# Approve manual steps if needed
workflow approve <TASK_NAMES>

The workflow manage TUI

The workflow manage command launches an interactive terminal interface for monitoring and controlling your workflow:

• Live status view: See all workflow steps and their current state
• Log viewer: Drill into logs for any step without leaving the interface
• Approval handling: Approve pending steps directly from the TUI
• Progress tracking: Watch steps transition through states in real-time

This is the recommended way to monitor long-running migrations.

Approval gates

Certain workflow steps require manual approval before proceeding. This provides checkpoints to verify the migration is progressing correctly.

When approvals happen

• Before metadata migration (to verify snapshot completed correctly)
• Before document backfill (to verify metadata migrated correctly)
• At other critical transition points depending on your configuration

What approvals mean

An approval gate indicates the workflow has completed a phase and is waiting for you to verify the results before continuing. The ⟳ status symbol indicates a step waiting for approval.

How to handle approvals

# Using the TUI (recommended)
workflow manage

# Approve a specific step
workflow approve source.target.metadataMigrate

# Approve multiple steps matching a pattern
workflow approve "*.metadataMigrate"

# Approve multiple specific steps
workflow approve step.a step.b step.c

Error recovery

When a workflow fails (indicated by ✗ status):

1. Check the status

workflow status

Identify which step failed and note the step name.

2. Check the logs

# View all logs
workflow output

# View logs for a specific step using label selectors
workflow output -l task=<task-name>

3. Fix the configuration

If the failure was due to configuration issues:

workflow configure edit

4. Resubmit

After fixing the issue:

workflow submit

The workflow will restart. For document backfill, RFS checkpoints allow resuming from where it left off rather than starting over.

Command reference

Configuration commands

Command	Description
workflow configure sample	Display sample configuration schema for your version
workflow configure sample --load	Load sample as starting point
workflow configure edit	Edit configuration in default editor
workflow configure view	Display current configuration
workflow configure clear	Clear current configuration

Execution commands

Command	Description
workflow submit	Submit workflow for execution

Management commands

Command	Description
workflow manage	Interactive TUI: approve steps, view logs, monitor progress
workflow approve <step>	Approve a single step that is currently blocked
workflow approve "*.pattern"	Approve all matching steps (glob pattern)
workflow approve a b c	Approve multiple steps
workflow stop	Stop a running workflow

Monitoring commands

Command	Description
workflow status	Show workflow progress and task states
workflow output	View workflow logs across all pods
workflow output --follow	Stream logs as they are written
workflow output -l key=value	Filter logs using label selectors

Label selectors for workflow output use tab-completion to discover available keys and values for your workflow. The -l flag auto-prefixes keys with migrations.opensearch.org/.

Help commands

Command	Description
workflow --help	Show general help
workflow configure --help	Show configure subcommand help
workflow -vv status	Enable verbose logging

Benefits

Declarative configuration

• Define what to migrate, not how
• Configuration is version-controlled
• Easy to review and audit
• Repeatable migrations

Workflow orchestration

• Automatic parallelism
• Built-in retry logic
• State management
• Progress tracking

Kubernetes-native

• Leverages Kubernetes scheduling
• Resource limits and requests
• Integration with Kubernetes ecosystem
• Scalable execution

Limitations

• Limited to backfill migrations (no live capture/replay)
• Requires Kubernetes deployment
• Requires Argo Workflows installation
• Configuration changes require workflow resubmission

Next steps

1. Workflow CLI Getting Started - Step-by-step tutorial for your first workflow
2. Backfill Workflow - Detailed backfill configuration and execution
3. Troubleshooting - Common issues and solutions

Related pages

• Deploying to Kubernetes - Generic Kubernetes deployment
• Deploying to EKS - AWS EKS deployment
• Migration Paths - Supported versions and compatibility