Skip to content

README.md

Latest commit

 

History

History
386 lines (309 loc) · 10.6 KB

README.md

File metadata and controls

386 lines (309 loc) · 10.6 KB

NLP Order Chatbot - Production Ready System

A complete production-ready chatbot system that processes natural language orders via Telegram and provides an admin dashboard for order management.

🚀 Features

Customer Experience

  • Natural Language Processing: Customers send plain English messages via Telegram
  • Intelligent Order Extraction: Uses Google Gemini AI with regex-based MCP fallback for reliability
  • Bank Reference Extraction: Automatically extracts bank payment references from customer messages
  • Instant Confirmation: Formatted order confirmation with order ID and total
  • Error Handling: Clear guidance when order details are missing

Admin Experience

  • Real-time Dashboard: View all orders with formatted order forms
  • Stock Reference: Quick view of available products and prices
  • One-Click Actions: Verify payments, add waybills, mark as delivered
  • Status Tracking: Clear order progression from pending → delivered
  • Auto-refresh: Dashboard updates every 30 seconds
  • Bank Reference Visibility: Shows full bank references for easy payment verification
  • Order Amount Display: Shows order amounts in the order list table

Technical Features

  • 99%+ Reliability: Regex-based fallback ensures no orders are lost
  • Pluggable Architecture: Swap LLM providers, add channels easily
  • Production Ready: Docker containerized with health checks
  • Real-time Updates: Webhook processing with async architecture
  • Complete Audit Trail: Track every order change with timestamps

📋 Quick Start

Prerequisites

  • Docker & Docker Compose
  • Telegram Bot Token (from @BotFather)
  • Google Gemini API Key (from Google AI Studio)

1. Clone and Setup

git clone <repository>
cd nlp-order-chatbot

2. Configure Environment

# Copy environment template
cp .env.example .env

# Edit .env with your API keys
nano .env

Required environment variables:

TELEGRAM_TOKEN=your_telegram_bot_token_here
GEMINI_API_KEY=your_gemini_api_key_here

3. Start the System

# Run the setup script
./scripts/setup.sh

# This will:
# - Build all Docker containers
# - Start PostgreSQL database
# - Launch MCP tools service
# - Start FastAPI backend
# - Launch admin dashboard
# - Seed the database with products

4. Setup Telegram Webhook

# Configure webhook for your bot
./scripts/setup_webhook.sh

5. Test the System

# Run comprehensive system tests
./scripts/test_system.sh

🎯 Usage Examples

Customer Order Flow

Customer sends:

"Hi, I'm Sarah Johnson, I want 1 case, phone 0821234567, bank ref :fnb1234, Pick n Pay Rosebank, Johannesburg, Gauteng"

Bot responds:

✅ Order received!

Most simple order form for team SA 
*Order form*
Names: Sarah Johnson
Order (ID): OR1700123456
Stock to collect: 1x 24 bottles
Cell nr: 0821234567
*Pudo Locker name:* Pick n Pay Rosebank
*Your suburb:* Johannesburg
Gauteng
Total: R180

Please transfer R180 to our banking details and send us the reference number.

Admin Dashboard Flow

  1. View Orders: See all incoming orders with formatted forms and bank references
  2. Verify Payment: Click "✓ Verify Payment", enter or confirm bank reference
  3. Add Waybill: Click "📦 Add Waybill", enter Pudo tracking number
  4. Mark Delivered: Click "✅ Mark Delivered" after drop-off

🏗️ Architecture

graph TD
    A[Telegram User] -->|Messages| B[Telegram API]
    B --> C[FastAPI Backend]
    C --> D[(PostgreSQL Database)]
    C --> E[LLM Service - Gemini]
    E -->|Fallback| F[MCP Tools Service]
    F --> D
    C --> G[Admin Dashboard]
    H[Admin User] --> G
    G --> C

    style A fill:#4285F4,stroke:#333
    style B fill:#34A853,stroke:#333
    style C fill:#FBBC05,stroke:#333
    style D fill:#EA4335,stroke:#333
    style E fill:#8E44AD,stroke:#333
    style F fill:#2C3E50,stroke:#333
    style G fill:#E67E22,stroke:#333
    style H fill:#9B59B6,stroke:#333
Loading

Services Architecture

  1. Telegram Integration Layer

    • Receives messages via webhook
    • Processes natural language orders
    • Sends formatted confirmations

  2. FastAPI Backend Service

    • Main API server (port 8000)
    • Order processing and validation
    • Admin endpoints for order management
    • Health checks and monitoring

  3. LLM Service with Fallback

    • Primary: Google Gemini AI for order extraction
    • Fallback: Regex-based MCP tools for reliability
    • Product catalog management

  4. MCP Tools Service

    • Lightweight regex-based extraction
    • Product database access
    • Order validation utilities
    • Fallback when LLM fails

  5. PostgreSQL Database

    • Stores orders, customers, products
    • Order status tracking
    • Payment and fulfillment data

  6. Admin Dashboard

    • Next.js React application (port 3000)
    • Real-time order monitoring
    • Payment verification interface
    • Fulfillment management

Data Flow

  1. Order Creation:

    • Customer sends Telegram message
    • FastAPI receives webhook
    • LLM Service extracts order details
    • Fallback to MCP Tools if needed
    • Order saved to PostgreSQL
    • Confirmation sent to customer

  2. Order Management:

    • Admin views orders in dashboard
    • Payment verified with bank reference
    • Waybill added for fulfillment
    • Order marked as delivered
    • Customer notifications sent

📊 Available Products

Product Price Aliases
24 Bottles Case R180 "1 case", "24 bottles", "case"
36 Bottles R220 "36 bottles"
20 Sprays R150 "20 sprays", "20 spray bottles"
24 Bottles + 20 Sprays R180 "24 bottles plus 20 sprays"
48 Bottles R360 "48 bottles", "2 cases"

🔧 Development

Project Structure

nlp-order-chatbot/
├── backend/                 # FastAPI application
│   ├── main.py             # Main API server
│   ├── models.py           # Database models
│   ├── telegram_handler.py # Telegram integration
│   ├── llm_service.py      # Gemini + MCP integration
│   ├── admin_api.py        # Admin endpoints
│   └── database.py         # DB connection & utilities
├── mcp-tools/              # Fallback extraction service
│   └── simple_tools.py     # Regex-based extraction
├── admin-dashboard/        # Next.js admin interface
│   └── pages/index.tsx     # Main dashboard page
├── scripts/                # Utility scripts
├── docker-compose.yml      # Service orchestration
└── README.md

Local Development

# Start services in development mode
docker-compose up --build

# View logs
docker-compose logs -f backend
docker-compose logs -f mcp-tools

# Access services
# Admin Dashboard: http://localhost:3000
# API Documentation: http://localhost:8000/docs
# API Health: http://localhost:8000/health
# MCP Tools: http://localhost:8080/health

Database Access

# Connect to PostgreSQL
docker-compose exec postgres psql -U orderuser -d orders

# View orders
SELECT order_display_id, customer_name, status, total_amount, payment_reference FROM orders;

# View products
SELECT sku, name, price FROM products;

🔍 API Endpoints

Admin Endpoints

  • GET /admin/orders - List all orders with bank references and amounts
  • POST /admin/orders/{id}/verify-payment - Verify payment with bank reference
  • POST /admin/orders/{id}/waybill - Add waybill number
  • POST /admin/orders/{id}/mark-delivered - Mark as delivered
  • GET /admin/dashboard/stats - Dashboard statistics
  • GET /admin/products - List available products

System Endpoints

  • GET /health - System health check
  • POST /webhook/telegram - Telegram webhook
  • POST /admin/test-extraction - Test order extraction
  • GET /admin/mcp-status - MCP service status

🛡️ Production Deployment

Environment Variables

# Production database
DATABASE_URL=postgresql://user:pass@prod-db:5432/orders

# Production webhook
TELEGRAM_WEBHOOK_URL=https://yourdomain.com/webhook/telegram

# CORS configuration
CORS_ORIGINS=https://yourdomain.com,https://admin.yourdomain.com

# SSL/Security
SSL_VERIFY=true
SECRET_KEY=your-secret-key-here

Health Checks

All services include comprehensive health checks:

  • Database connectivity
  • API responsiveness
  • MCP tools availability
  • LLM service status

Monitoring

Built-in endpoints for monitoring:

  • /health - Overall system health
  • /admin/dashboard/stats - Business metrics
  • /admin/mcp-status - Fallback service status

🚨 Troubleshooting

Common Issues

Services not starting:

# Check logs
docker-compose logs

# Rebuild containers
docker-compose down && docker-compose up --build

Database connection errors:

# Reset database
docker-compose down -v
docker-compose up postgres

Telegram webhook issues:

# Remove and reset webhook
curl -X POST "https://api.telegram.org/bot<TOKEN>/deleteWebhook"
./scripts/setup_webhook.sh

Order extraction failures:

  • Check Gemini API key is valid
  • Verify MCP tools service is running
  • Test with /admin/test-extraction endpoint

Bank Reference Extraction Issues

If bank references are not being extracted from customer messages:

  1. Check the message format: Ensure customers include bank reference in formats like:

    • "bank ref :fnb1234"
    • "bank reference: abc123"
    • "payment ref : xyz789"

  2. Test extraction: Use the admin test endpoint:

    POST /admin/test-extraction
{
  "text": "Your test message with bank ref :test123"
}

  3. Check MCP tools logs:

    docker-compose logs -f mcp-tools

Support

  • Check service logs: docker-compose logs -f
  • Test individual components with provided scripts
  • Verify environment variables are correctly set
  • Ensure all required ports are available

📈 Scaling

The system is designed for horizontal scaling:

  • Stateless services: All components can be replicated
  • Database connection pooling: Ready for multiple instances
  • Queue-ready: Easy to add Redis for background processing
  • Load balancer ready: Standard HTTP interfaces

🔐 Security

  • Input validation on all endpoints
  • SQL injection protection via SQLAlchemy
  • Rate limiting ready (add middleware)
  • CORS configuration for production
  • Environment-based configuration
  • No secrets in code or containers

📞 Getting Help

  1. Check service health: ./scripts/test_system.sh
  2. View logs: docker-compose logs -f
  3. Test individual components: Use the provided scripts
  4. API documentation: http://localhost:8000/docs

System is ready for production use with minimal configuration changes.
⚡