SearchAPI MCP Server - Refactoring Notes

Overview

This codebase has been refactored following FAST MCP best practices for production-ready Model Context Protocol servers. The refactoring focuses on:

Configuration Management - Pydantic-based validation and environment variable handling
Resilient HTTP Client - Connection pooling, retry logic, and circuit breaker pattern
Performance Optimization - Multi-level caching with TTL
Error Handling - Structured error responses with proper classification
Monitoring & Observability - Metrics collection and health checks
Production Readiness - Logging, graceful degradation, and fail-safe design

Architecture Changes

File Structure

searchAPI-mcp/
├── config.py                    # NEW: Configuration management with Pydantic
├── client.py                    # NEW: HTTP client with resilience patterns
├── mcp_server_refactored.py     # NEW: Refactored MCP server
├── mcp_server.py                # ORIGINAL: Legacy server (kept for reference)
├── test_refactored.py           # NEW: Test suite
├── requirements.txt             # UPDATED: Added Pydantic dependencies
├── .env.example                 # EXISTING: Environment template
└── README.md                    # EXISTING: Main documentation

Key Components

1. Configuration Module (`config.py`)

Following FAST MCP Best Practice: Externalized configuration with startup validation

Features:

Pydantic models for type-safe configuration
Environment variable validation at startup
Configurable timeouts, retries, cache settings
Prevents invalid API keys (e.g., "your_api_key_here")

from config import load_config

config = load_config()  # Raises ValueError if invalid

2. HTTP Client (`client.py`)

Following FAST MCP Best Practices: Resilient communication with external APIs

Components:

a) CacheManager

In-memory cache with TTL (default: 1 hour)
LRU eviction when max size reached
Reduces API calls and improves latency
Cache hit rate tracking

b) CircuitBreaker

Prevents cascading failures
Opens after 5 consecutive failures
Auto-recovery after 60 seconds
Half-open state for gradual recovery

c) MetricsCollector

Request count and latency tracking
P95/P99 latency percentiles
Cache hit rate monitoring
Error classification and counting

d) SearchAPIClient

Connection pooling (default: 10 connections)
Exponential backoff retry (max 3 retries)
Structured error responses
Health check endpoint
Automatic resource cleanup

3. MCP Server (`mcp_server_refactored.py`)

Following FAST MCP Best Practices: Production-ready tool design

Improvements:

Comprehensive Tool Descriptions: Each tool has detailed docstrings explaining:
- Purpose and use cases
- All parameters with examples
- Return value structure
- Usage examples
- Cross-references to related tools
Input Validation: Validates required parameters before making API calls
Better Error Messages: Structured error responses with type classification
Health Check Tool: New health_check() tool for monitoring
Enhanced Date Tool: get_current_time() includes travel date suggestions
Google AI Mode: New tool for AI-powered search with overview generation

New Features

1. Health Check & Monitoring

# Check service health
await health_check()

# Returns:
{
  "api_status": {
    "status": "healthy",
    "latency_ms": 245.3,
    "circuit_breaker": "closed"
  },
  "cache_stats": {
    "size": 42,
    "max_size": 1000,
    "ttl": 3600
  },
  "metrics": {
    "request_count": 150,
    "error_count": 2,
    "error_rate": 0.013,
    "cache_hit_rate": 0.35,
    "latency": {
      "avg": 0.234,
      "p95": 0.456,
      "p99": 0.678
    }
  }
}

2. Google AI Mode Search

New tool for AI-powered search with comprehensive overviews:

await search_google_ai_mode(
    q="How does photosynthesis work?",
    location="San Francisco, CA"
)

# Returns:
{
  "text_blocks": [
    {
      "type": "header",
      "answer": "Photosynthesis Overview",
      "reference_indexes": [1, 3]
    },
    {
      "type": "paragraph",
      "answer": "Photosynthesis is the process...",
      "reference_indexes": [1, 2, 3]
    },
    {
      "type": "ordered_list",
      "items": [...]
    }
  ],
  "markdown": "## Photosynthesis Overview\n\nPhotosynthesis...[^1][^2]",
  "reference_links": [
    {
      "index": 1,
      "title": "Photosynthesis - Wikipedia",
      "link": "https://...",
      "source": "Wikipedia"
    }
  ],
  "web_results": [...],
  "local_results": [...]
}

3. Response Caching

Automatic caching of API responses:

Default TTL: 1 hour (3600 seconds)
Cache Size: 1000 items (LRU eviction)
Configurable: Via environment variables

Benefits:

Faster response times for repeated queries
Reduced API costs
Lower API rate limit impact

4. Circuit Breaker

Automatic failure protection:

Threshold: 5 consecutive failures
Recovery Time: 60 seconds
States: Closed → Open → Half-Open → Closed

Prevents wasting resources on unavailable services.

5. Retry Logic

Exponential backoff retry:

Max Retries: 3 attempts
Backoff: 1s → 2s → 4s
Smart Retry: Doesn't retry 4xx client errors

Configuration

Environment Variables

Create a .env file:

# Required
SEARCHAPI_API_KEY=your_actual_api_key

# Optional - HTTP Client
TIMEOUT=30.0                    # Request timeout (seconds)
MAX_RETRIES=3                   # Retry attempts
RETRY_BACKOFF=1.0              # Backoff multiplier
POOL_CONNECTIONS=10            # Connection pool size
POOL_MAXSIZE=10                # Max connections per pool

# Optional - Cache
ENABLE_CACHE=true              # Enable response caching
CACHE_TTL=3600                 # Cache lifetime (seconds)
CACHE_MAX_SIZE=1000            # Max cached items

# Optional - Monitoring
ENABLE_METRICS=true            # Enable metrics collection
LOG_LEVEL=INFO                 # Logging level

# Optional - Server
MCP_TRANSPORT=stdio            # MCP transport type

Claude Desktop Configuration

Using Refactored Server:

{
  "mcpServers": {
    "searchapi": {
      "command": "uv",
      "args": [
        "run",
        "--with",
        "mcp[cli]",
        "/path/to/searchAPI-mcp/mcp_server_refactored.py"
      ],
      "env": {
        "SEARCHAPI_API_KEY": "your_api_key_here",
        "ENABLE_CACHE": "true",
        "CACHE_TTL": "3600",
        "LOG_LEVEL": "INFO"
      }
    }
  }
}

Installation

1. Install Dependencies

# Install updated requirements
pip install -r requirements.txt

# Or install individually
pip install httpx>=0.24.0 fastmcp>=0.1.0 python-dotenv>=1.0.0 pydantic>=2.0.0 pydantic-settings>=2.0.0

2. Configure Environment

# Copy example env file
cp .env.example .env

# Edit with your API key
nano .env

3. Run Server

# Run refactored server
python mcp_server_refactored.py

# Or with uv
uv run mcp_server_refactored.py

Testing

Run Tests

# Basic test runner
python test_refactored.py

# Full test suite with pytest
pip install pytest pytest-asyncio
pytest test_refactored.py -v

# With coverage
pytest test_refactored.py --cov=. --cov-report=html

Test Coverage

The test suite includes:

✓ Configuration validation
✓ Cache operations (set, get, expiration, eviction)
✓ Metrics collection (requests, errors, latency)
✓ Circuit breaker behavior
✓ HTTP client initialization
✓ Request caching
✓ Health checks
✓ Error handling

Performance Improvements

Benchmarks (Estimated)

Metric	Original	Refactored	Improvement
Cache Hit Response	N/A	<10ms	∞
Repeated Queries	~300ms	~10ms	30x faster
Connection Setup	Per request	Pooled	~50ms saved
Failed Requests	No retry	3 retries	+resilience
Circuit Break	No	Yes	+stability

Cache Hit Rate

Expected cache hit rates:

Repeated Queries: ~80-90%
Exploratory Use: ~20-40%
Production: ~50-70%

Migration Guide

From Original to Refactored

No Changes Required for:

Tool names and parameters
API responses
MCP client configuration (except file path)

Optional Enhancements:

Configure cache TTL based on use case
Adjust retry settings for network conditions
Enable/disable metrics collection
Set custom logging levels

Breaking Changes:

None - fully backward compatible

Best Practices Applied

1. Single Responsibility Principle

Each module has one clear purpose:

config.py → Configuration
client.py → HTTP communication
mcp_server_refactored.py → MCP tool definitions

2. Defense in Depth

Multiple security layers:

Input validation (Pydantic)
API key validation
Error handling
Rate limiting (via retry backoff)

3. Fail-Safe Design

Circuit breaker prevents cascading failures
Cache fallback for degraded service
Graceful error responses
No crashes on invalid config

4. Performance Optimization

Connection pooling
Response caching
Async/await throughout
Efficient retry logic

5. Observability

Structured logging
Metrics collection
Health checks
Request timing

Troubleshooting

Issue: Configuration Error

ValueError: Invalid API key. Please set SEARCHAPI_API_KEY environment variable.

Solution: Set valid API key in .env file or environment.

Issue: Circuit Breaker Open

error: "Circuit breaker is OPEN - service unavailable"

Solution: Wait 60 seconds for auto-recovery or check SearchAPI service status.

Issue: Cache Not Working

Check:

ENABLE_CACHE=true in environment
Queries are identical (parameters must match exactly)
Cache hasn't expired (default 1 hour TTL)

Issue: Slow First Request

Expected: First request is slower due to:

Connection pool initialization
Circuit breaker warmup
No cache hit

Subsequent requests will be much faster.

Future Enhancements

Potential improvements:

Redis cache backend for multi-instance deployments
Prometheus metrics export
OpenTelemetry tracing
Rate limit handling with backoff
Request queue for high-volume scenarios
WebSocket transport support
OAuth authentication for SearchAPI

References

Support

For issues or questions:

Check this document first
Review test suite for examples
Check configuration in .env
Review logs for error details
Open GitHub issue with logs and config (sanitize API keys!)

Last Updated: 2025-11-15 Version: 1.0.0 (Refactored)

FilesExpand file tree

REFACTOR_NOTES.md

Latest commit

History