feat: add comprehensive secrets management system (#17)

BREAKING
Implement server-side secrets management with automatic redaction, audit
logging, and fail-safe security. Secrets are resolved during execution
and never reach LLM context.

Core features:
- Five-namespace variable system (inputs, metadata, blocks,
__internal__, secrets)
- EnvVarSecretProvider with WORKFLOW_SECRET_* prefix
- Automatic output redaction for credential protection
- Comprehensive audit trail for compliance
- Fail-fast behavior for missing secrets

Changes:
- Add secrets/ module with provider, redactor, audit, exceptions
- Integrate secrets into VariableResolver with async resolution
- Update all executors to support secret substitution
- Add 8 comprehensive test workflows for secrets validation
- Update 30+ snapshot tests for five-namespace variable system

Security:
- Server-side resolution prevents credential leakage to LLM
- Pattern-based redactor sanitizes all outputs
- Structured audit logging for security compliance
- Fail-fast on missing secrets prevents silent failures

Author: ibacalu

.github/workflows/release.yml

@@ -13,8 +13,26 @@ jobs:
       issues: write         # to be able to comment on released issues
       pull-requests: write  # to be able to comment on released pull requests
     steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+
+      - name: Setup Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.12'
+
+      - name: Install packaging library
+        run: pip install packaging
+
       - name: Semantic Release
         uses: qtsone/actions/release@main
         with:
           github-token: ${{ secrets.GITHUB_TOKEN }}
           ssh-key: ${{ secrets.DEPLOY_KEY }}
+          skip-checkout: 'true'
+          extra-plugins: |
+            @semantic-release/changelog
+            @semantic-release/git
+            @semantic-release/exec

.releaserc

@@ -17,11 +17,18 @@
         "changelogTitle": "# Changelog\n\nAll notable changes to this project will be documented in this file."
       }
     ],
+    [
+      "@semantic-release/exec",
+      {
+        "prepareCmd": "python scripts/update-version.py ${nextRelease.version}"
+      }
+    ],
     [
       "@semantic-release/git",
       {
         "assets": [
-          "CHANGELOG.md"
+          "CHANGELOG.md",
+          "src/workflows_mcp/__init__.py"
         ],
         "message": "chore(release): version ${nextRelease.version}\n\n${nextRelease.notes}"
       }

README.md

@@ -66,6 +66,54 @@ All `env` variables are optional. The `--refresh` flag is recommended to ensure
 
 Restart your LLM client, and you're ready to go!
 
+## ✨ Features
+
+- **DAG-Based Workflows**: Automatic dependency resolution and parallel execution
+- **🔐 Secrets Management**: Server-side credential handling with automatic redaction (v5.0.0+)
+- **Workflow Composition**: Reusable workflows via Workflow blocks
+- **Conditional Execution**: Boolean expressions for dynamic control flow
+- **Variable Resolution**: Five-namespace system (inputs, metadata, blocks, secrets, __internal__)
+- **File Operations**: CreateFile, ReadFile, RenderTemplate with Jinja2
+- **HTTP Integration**: HttpCall for REST API interactions
+- **Interactive Workflows**: Prompt blocks for user input
+- **MCP Integration**: Exposed as tools for LLM agents via Model Context Protocol
+
+### 🔐 Secrets Management
+
+Securely manage API keys, passwords, and tokens in workflows:
+
+```yaml
+blocks:
+  - id: call_github_api
+    type: HttpCall
+    inputs:
+      url: "https://api.github.com/user"
+      headers:
+        Authorization: "Bearer {{secrets.GITHUB_TOKEN}}"
+```
+
+**Key Features:**
+
+- ✅ **Server-side resolution** - Secrets never reach LLM context
+- ✅ **Automatic redaction** - Secret values sanitized from all outputs
+- ✅ **Fail-fast behavior** - Missing secrets cause immediate workflow failure
+- ✅ **Audit logging** - Comprehensive tracking for compliance
+
+**Configuration:**
+
+```json
+{
+  "mcpServers": {
+    "workflows": {
+      "env": {
+        "WORKFLOW_SECRET_GITHUB_TOKEN": "ghp_xxx",
+        "WORKFLOW_SECRET_OPENAI_API_KEY": "sk-xxx"
+      }
+    }
+  }
+}
+```
+
 ## What Can You Do With It?
 
 ### Built-in Workflows
@@ -212,6 +260,7 @@ Your custom workflows override built-in ones if they have the same name. Later d
 - **WORKFLOWS_TEMPLATE_PATHS** - Comma-separated list of additional template directories
 - **WORKFLOWS_MAX_RECURSION_DEPTH** - Maximum workflow recursion depth (default: 50, range: 1-10000)
 - **WORKFLOWS_LOG_LEVEL** - Logging level (DEBUG, INFO, WARNING, ERROR, CRITICAL)
+- **WORKFLOW_SECRET_<NAME>** - Define secrets for workflow execution (e.g., `WORKFLOW_SECRET_GITHUB_TOKEN`)
 
 ## Example Usage with Claude
 
@@ -287,7 +336,7 @@ The server uses a **fractal execution model** where workflows and blocks share t
 - **WorkflowRunner** - Orchestrates workflow execution
 - **BlockOrchestrator** - Executes individual blocks with error handling
 - **DAGResolver** - Resolves dependencies and computes parallel execution waves
-- **Variable Resolution** - Four-namespace variable system (inputs, blocks, metadata, internal)
+- **Variable Resolution** - Five-namespace variable system (inputs, blocks, metadata, secrets, __internal__)
 - **Checkpoint System** - Pause/resume support for interactive workflows
 
 Workflows execute in **waves**—blocks with no dependencies or whose dependencies are satisfied run in parallel within each wave, maximizing efficiency.

scripts/update-version.py

@@ -0,0 +1,141 @@
+#!/usr/bin/env python3
+"""Update version in src/workflows_mcp/__init__.py
+
+Usage: python scripts/update-version.py <version>
+
+This script uses the official Python packaging library for robust version validation,
+supporting all PEP 440 version formats including:
+- Standard releases: 1.2.3
+- Pre-releases: 1.2.3a1, 1.2.3b2, 1.2.3rc1
+- Post-releases: 1.2.3.post1
+- Dev releases: 1.2.3.dev1
+- Epochs: 1!1.0
+- Local versions: 1.2.3+local.version
+"""
+
+import sys
+from pathlib import Path
+
+try:
+    from packaging.version import InvalidVersion, Version
+except ImportError:
+    print("Error: 'packaging' library not found", file=sys.stderr)
+    print("Install it with: pip install packaging", file=sys.stderr)
+    sys.exit(1)
+
+
+def validate_version(version_string: str) -> Version:
+    """Validate version string using PEP 440 specification.
+
+    Args:
+        version_string: Version string to validate
+
+    Returns:
+        Validated Version object
+
+    Raises:
+        InvalidVersion: If version string is invalid
+    """
+    try:
+        return Version(version_string)
+    except InvalidVersion as e:
+        print(f"Error: Invalid version format: {version_string}", file=sys.stderr)
+        print(f"Reason: {e}", file=sys.stderr)
+        print(
+            "\nValid formats include:",
+            "  - Standard: 1.2.3",
+            "  - Pre-release: 1.2.3a1, 1.2.3b2, 1.2.3rc1",
+            "  - Post-release: 1.2.3.post1",
+            "  - Dev release: 1.2.3.dev1",
+            "  - With epoch: 1!1.0",
+            "  - With local: 1.2.3+local.version",
+            sep="\n",
+            file=sys.stderr,
+        )
+        sys.exit(1)
+
+
+def update_version_in_file(file_path: Path, new_version: str) -> tuple[str, bool]:
+    """Update __version__ variable in a Python file.
+
+    Args:
+        file_path: Path to the Python file
+        new_version: New version string
+
+    Returns:
+        Tuple of (old_version, was_updated)
+    """
+    if not file_path.exists():
+        print(f"Error: {file_path} not found", file=sys.stderr)
+        sys.exit(1)
+
+    content = file_path.read_text(encoding="utf-8")
+
+    old_version = None
+    updated = False
+
+    # Find and update __version__ line while preserving all formatting
+    for line in content.splitlines():
+        # Match __version__ = "..." with any amount of whitespace
+        if line.strip().startswith("__version__") and "=" in line:
+            # Extract old version (everything between quotes)
+            parts = line.split("=", 1)
+            if len(parts) == 2:
+                value_part = parts[1].strip()
+                if value_part.startswith('"') or value_part.startswith("'"):
+                    quote_char = value_part[0]
+                    # Find the version string between quotes
+                    start = value_part.index(quote_char)
+                    end = value_part.index(quote_char, start + 1)
+                    old_version = value_part[start + 1 : end]
+
+                    if old_version != new_version:
+                        # Replace only the version string in the entire content
+                        # This preserves all whitespace and line endings
+                        old_full_line = line
+                        new_value_part = (
+                            value_part[: start + 1] + new_version + value_part[end:]
+                        )
+                        new_full_line = parts[0] + "= " + new_value_part
+                        content = content.replace(old_full_line, new_full_line, 1)
+                        updated = True
+                    break
+
+    if old_version is None:
+        print(f"Error: Could not find __version__ in {file_path}", file=sys.stderr)
+        sys.exit(1)
+
+    if updated:
+        file_path.write_text(content, encoding="utf-8")
+
+    return old_version, updated
+
+
+def main() -> None:
+    """Update version in __init__.py file."""
+    if len(sys.argv) != 2:
+        print("Error: Version argument required", file=sys.stderr)
+        print("Usage: python scripts/update-version.py <version>", file=sys.stderr)
+        sys.exit(1)
+
+    new_version_str = sys.argv[1]
+
+    # Validate using official Python packaging library (PEP 440)
+    new_version = validate_version(new_version_str)
+
+    print(f"\n📦 Updating version to {new_version}\n")
+
+    # Update __init__.py
+    init_file = Path("src/workflows_mcp/__init__.py")
+    old_version, was_updated = update_version_in_file(init_file, str(new_version))
+
+    if was_updated:
+        print(f"✓ {init_file}: {old_version} → {new_version}")
+    else:
+        print(f"  {init_file}: already {new_version}")
+
+    print("\n✅ Version update complete\n")
+
+
+if __name__ == "__main__":
+    main()

src/workflows_mcp/engine/orchestrator.py

@@ -5,6 +5,7 @@
 2. Catch ExecutionPaused → signal pause to caller
 3. Determine operation outcome (for Shell: exit_code)
 4. Return structured BlockExecution result
+5. Resolve secrets in inputs and redact secrets from outputs
 
 This is the bridge between executors (which return BaseModel or raise exceptions)
 and the workflow execution layer (which needs Metadata).
@@ -13,7 +14,7 @@
 from __future__ import annotations
 
 from datetime import UTC, datetime
-from typing import Any
+from typing import TYPE_CHECKING, Any
 
 from pydantic import BaseModel
 
@@ -23,6 +24,9 @@
 from .executor_base import BlockExecutor
 from .metadata import Metadata
 
+if TYPE_CHECKING:
+    from .secrets import SecretProvider, SecretRedactor
+
 
 class BlockExecution(BaseModel):
     """
@@ -56,9 +60,26 @@ class BlockOrchestrator:
     - Catch exceptions → create Metadata
     - Catch ExecutionPaused → signal pause
     - Determine operation outcome (e.g., Shell exit_code)
+    - Resolve secrets in inputs before execution
+    - Redact secrets from outputs after execution
     - Return BlockExecution with output + metadata
     """
 
+    def __init__(
+        self,
+        secret_provider: SecretProvider | None = None,
+        secret_redactor: SecretRedactor | None = None,
+    ):
+        """
+        Initialize block orchestrator with optional secret management.
+
+        Args:
+            secret_provider: Optional secret provider for resolving {{secrets.*}}
+            secret_redactor: Optional secret redactor for output sanitization
+        """
+        self.secret_provider = secret_provider
+        self.secret_redactor = secret_redactor
+
     async def execute_block(
         self,
         executor: BlockExecutor,
@@ -95,6 +116,14 @@ async def execute_block(
             # Execute block
             output = await executor.execute(inputs, context)
 
+            # Redact secrets from output (if redactor is configured)
+            if self.secret_redactor:
+                # Convert output to dict, redact, then reconstruct
+                output_dict = output.model_dump()
+                redacted_dict = self.secret_redactor.redact(output_dict)
+                # Reconstruct output with redacted values
+                output = type(output)(**redacted_dict)
+
             # Success! Determine operation outcome
             completed_at = datetime.now(UTC).isoformat()
             execution_time_ms = datetime.now(UTC).timestamp() * 1000 - start_time_ms