Skip to content

Latest commit

 

History

History
386 lines (320 loc) · 11.9 KB

File metadata and controls

386 lines (320 loc) · 11.9 KB

हमारी आवाज़, हमारे अधिकार (Our Voice, Our Rights)

MGNREGA Transparency Dashboard — मनरेगा पारदर्शिता डैशबोर्ड

Live: configure via sslip.io, e.g. https://app.3-109-118-59.sslip.io

Hindi‑first, offline‑ready dashboard for Uttar Pradesh MGNREGA data. Designed for low‑literacy users with voice, big touch targets, and simple comparisons.


✨ What’s in this version

  • Latest‑month KPIs only (no confusing FY sums on home)
  • Detail pages per KPI with: compare (prev month/year, state average, other district), trends, and pie breakdowns
  • Instant FY and rolling‑12 tiles (precomputed with fallback calculation)
  • Offline mode: Service Worker (Network‑First) + IndexedDB cache of API responses; offline banner
  • Resilience: CSV seed + ingest worker with retries/backoff; graceful fallback when API is down
  • Security: HTTPS (Caddy), CSP/HSTS/Referrer‑Policy/X‑Frame‑Options; rate limits on /locate and compare
  • Observability: Prometheus /metrics, health /api/healthz & /api/ready
  • Low‑literacy UX: Hindi by default, guided tour, big cards, simple pies, voice narration

🏗 Architecture

Browser (React PWA)
   └─ Caddy (TLS, security headers)
       └─ Frontend (Nginx serving SPA)
           └─ FastAPI (backend)
               ├─ Redis (cache)
               └─ Postgres (DB)
                     ▲
                     └─ Ingest Worker (data.gov.in, CSV fallback)

🔌 API (selected)

  • GET /api/districts — districts (UP)
  • GET /api/districts/{code}/summary — latest month summary
  • GET /api/districts/{code}/current — latest raw month
  • GET /api/districts/{code}/compare — vs state average (latest month)
  • GET /api/districts/{code}/compare-current?baseline=prev_month|prev_year|state_avg
  • GET /api/districts/{code}/compare-district?other=... — vs other district
  • GET /api/districts/{code}/rollup?fy=YYYY-YYYY — FY aggregate (precomputed + fallback)
  • GET /api/districts/{code}/rolling?months=12 — rolling N months (precomputed + fallback)
  • GET /api/last-updated — freshness + ingest source (api/csv)
  • GET /metrics — Prometheus

🔐 Security

  • HTTPS via Caddy and free hostname (sslip.io)
  • CSP, HSTS, Referrer‑Policy, X‑Frame‑Options, X‑Content‑Type‑Options
  • Rate limits on geolocate (/locate) and compare endpoints
  • No PII stored; only public program data

🚀 Deploy on EC2 (recommended)

Prerequisites: Ubuntu 22.04 VM, Docker + compose plugin, open ports 22/80/443.

  1. Clone and env
sudo mkdir -p /opt/ovr && sudo chown ubuntu:ubuntu /opt/ovr
cd /opt/ovr
git clone https://github.com/ikrishanaa/Our-Voice-Our-Rights.git .
cat > .env << 'EOF'
DATA_GOV_API_KEY=YOUR_API_KEY
DATA_GOV_RESOURCE_ID=ee03643a-ee4c-48c2-ac30-9f2ff26ab722
# optional
# ALERT_WEBHOOK=
# VITE_PLAUSIBLE_DOMAIN=
EOF
  1. Bind frontend to loopback (Caddy will expose HTTPS)
sed -i 's/"3001:80"/"127.0.0.1:3001:80"/' docker-compose.yml
  1. Build, start, seed
docker compose build --no-cache
docker compose up -d
docker compose run --rm seed
  1. Caddy HTTPS (sslip.io). If Elastic IP is A.B.C.D, hostname is app.A-B-C-D.sslip.io.
sudo apt -y install debian-keyring debian-archive-keyring apt-transport-https
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | \
  sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | \
  sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo apt update && sudo apt -y install caddy

# Replace the hostname below
sudo bash -c "cat > /etc/caddy/Caddyfile" <<'EOF'
app.A-B-C-D.sslip.io {
  encode gzip
  header {
    Strict-Transport-Security "max-age=31536000; includeSubDomains"
    X-Frame-Options "SAMEORIGIN"
    X-Content-Type-Options "nosniff"
    Referrer-Policy "strict-origin-when-cross-origin"
    Content-Security-Policy "default-src 'self'; script-src 'self' 'unsafe-inline' 'unsafe-eval' https://plausible.io; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com; img-src 'self' data:; font-src 'self' https://fonts.gstatic.com data:; connect-src 'self' https://app.A-B-C-D.sslip.io http://127.0.0.1:3001 https://plausible.io; frame-ancestors 'self'; form-action 'self'; base-uri 'self';"
    Permissions-Policy "geolocation=(self), microphone=()"
  }
  reverse_proxy 127.0.0.1:3001
}
EOF
sudo systemctl reload caddy

Open https://app.A-B-C-D.sslip.io

Note: CSP is set by Caddy. Keep CSP disabled in frontend/nginx to avoid duplicate policies.

Update deployment

cd /opt/ovr
git pull
# rebuild what changed
docker compose build
docker compose up -d

Backup DB (daily)

sudo mkdir -p /opt/ovr/backups && sudo chown ubuntu:ubuntu /opt/ovr/backups
cat > /opt/ovr/bin/pg_backup.sh << 'EOF'
#!/usr/bin/env bash
set -euo pipefail
TS=$(date +%F_%H%M)
docker exec -i mgnrega-db pg_dump -U mgnrega_user mgnrega_db | gzip > /opt/ovr/backups/pg_${TS}.sql.gz
find /opt/ovr/backups -type f -name "pg_*.sql.gz" -mtime +14 -delete
EOF
chmod +x /opt/ovr/bin/pg_backup.sh
(crontab -l 2>/dev/null; echo "15 2 * * * /opt/ovr/bin/pg_backup.sh") | crontab -

🧑‍💻 Local development

Backend

cd backend
python -m venv venv && source venv/bin/activate
pip install -r requirements.txt
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

Frontend

cd frontend
npm install
npm run dev

⚙️ Environment variables

Required

  • DATA_GOV_API_KEY — your data.gov.in API key
  • DATA_GOV_RESOURCE_IDee03643a-ee4c-48c2-ac30-9f2ff26ab722

Optional

  • ALERT_WEBHOOK — POSTs a JSON alert if ingest fails
  • VITE_PLAUSIBLE_DOMAIN — enable privacy‑safe analytics

🩺 Ops quick sheet

  • Health: /api/healthz and /api/ready
  • Metrics: /metrics (Prometheus)
  • Logs: docker compose logs -f backend (or any service)
  • Seed: docker compose run --rm seed

🧭 Troubleshooting

  • Districts dropdown empty in Incognito: ensure DB is seeded and CSP allows same‑origin connect-src; prefer CSP only in Caddy.
  • Rolling shows 0: backend now falls back to raw aggregation; rebuild backend and refresh.
  • HTTPS fails: ports 80/443 allowed in security group + UFW; sudo systemctl status caddy.


Production Deployment guide on VM - EC2

Production Deployment Guide (VM)

This guide deploys the app to a self‑managed VM (AWS EC2 shown, works similarly for other VPS). Target stack:

  • Ubuntu 22.04 LTS
  • Docker + compose plugin
  • Caddy as TLS reverse proxy (free hostname via sslip.io)
  • Postgres + Redis in Docker

0) Prerequisites

  • Open inbound: 22, 80, 443 in cloud security group and UFW
  • Elastic/Public IP
  • Your data.gov.in credentials (API key, resource id)

1) Create VM (AWS console)

  • EC2 → Launch instance
    • AMI: Ubuntu 22.04 LTS (x86)
    • Instance type: t3.small or t3.medium
    • Key pair: create/download .pem
    • Security group: allow 22/80/443
    • Storage: 60–80GB
  • Allocate Elastic IP → associate with instance

2) First login & base setup

chmod 600 ~/Downloads/ovr-prod.pem
ssh -i ~/Downloads/ovr-prod.pem ubuntu@ELASTIC_IP

# Updates
sudo apt update && sudo apt -y upgrade

# Optional swap (2G) on small instances
sudo fallocate -l 2G /swapfile && sudo chmod 600 /swapfile && sudo mkswap /swapfile \
  && echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab && sudo swapon /swapfile

# UFW
sudo apt -y install ufw
sudo ufw allow OpenSSH && sudo ufw allow 80/tcp && sudo ufw allow 443/tcp
sudo ufw --force enable

3) Install Docker + compose plugin

sudo apt -y install ca-certificates curl gnupg lsb-release
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
 https://download.docker.com/linux/ubuntu $(. /etc/os-release; echo $VERSION_CODENAME) stable" \
| sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
sudo apt -y install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin git
sudo usermod -aG docker $USER
newgrp docker

4) Clone app and set env

sudo mkdir -p /opt/ovr && sudo chown ubuntu:ubuntu /opt/ovr
cd /opt/ovr
git clone https://github.com/ikrishanaa/Our-Voice-Our-Rights.git .

# Minimal env (two required)
cat > .env << 'EOF'
DATA_GOV_API_KEY=YOUR_API_KEY
DATA_GOV_RESOURCE_ID=ee03643a-ee4c-48c2-ac30-9f2ff26ab722
# optional
# ALERT_WEBHOOK=
# VITE_PLAUSIBLE_DOMAIN=
EOF

5) Bind frontend to loopback (served via Caddy)

sed -i 's/"3001:80"/"127.0.0.1:3001:80"/' docker-compose.yml

6) Build, start, and seed

docker compose build --no-cache
docker compose up -d
# one-time seed
docker compose run --rm seed

Check:

docker compose ps
curl -s http://127.0.0.1:3001/api/healthz || true

7) Configure HTTPS with Caddy (sslip.io)

Free hostname pattern: app.A-B-C-D.sslip.io for Elastic IP A.B.C.D.

Install Caddy:

sudo apt -y install debian-keyring debian-archive-keyring apt-transport-https
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | \
  sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | \
  sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo apt update && sudo apt -y install caddy

Write Caddyfile (replace hostname):

sudo bash -c "cat > /etc/caddy/Caddyfile" <<'EOF'
app.A-B-C-D.sslip.io {
  encode gzip
  header {
    Strict-Transport-Security "max-age=31536000; includeSubDomains"
    X-Frame-Options "SAMEORIGIN"
    X-Content-Type-Options "nosniff"
    Referrer-Policy "strict-origin-when-cross-origin"
    Content-Security-Policy "default-src 'self'; script-src 'self' 'unsafe-inline' 'unsafe-eval' https://plausible.io; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com; img-src 'self' data:; font-src 'self' https://fonts.gstatic.com data:; connect-src 'self' https://app.A-B-C-D.sslip.io http://127.0.0.1:3001 https://plausible.io; frame-ancestors 'self'; form-action 'self'; base-uri 'self';"
    Permissions-Policy "geolocation=(self), microphone=()"
  }
  reverse_proxy 127.0.0.1:3001
}
EOF
sudo systemctl reload caddy

Verify:

curl -I https://app.A-B-C-D.sslip.io

8) Operations

  • Update:
cd /opt/ovr
git pull
docker compose build
docker compose up -d
  • Logs:
docker compose logs -f backend
  • Health:
curl -s https://app.A-B-C-D.sslip.io/api/ready

9) Backups (Postgres daily)

sudo mkdir -p /opt/ovr/backups && sudo chown ubuntu:ubuntu /opt/ovr/backups
cat > /opt/ovr/bin/pg_backup.sh << 'EOF'
#!/usr/bin/env bash
set -euo pipefail
TS=$(date +%F_%H%M)
docker exec -i mgnrega-db pg_dump -U mgnrega_user mgnrega_db | gzip > /opt/ovr/backups/pg_${TS}.sql.gz
find /opt/ovr/backups -type f -name "pg_*.sql.gz" -mtime +14 -delete
EOF
chmod +x /opt/ovr/bin/pg_backup.sh
(crontab -l 2>/dev/null; echo "15 2 * * * /opt/ovr/bin/pg_backup.sh") | crontab -

10) Troubleshooting

  • District list empty in Incognito
    • Ensure DB seeded (docker compose run --rm seed)
    • Avoid duplicate CSP policies; keep CSP only in Caddy; rebuild frontend if you removed CSP from nginx
  • Rolling shows 0
    • Rebuild backend with latest fallback (docker compose build backend && docker compose up -d backend)
  • TLS fails
    • Security group + UFW allow 80/443; sudo systemctl status caddy

11) Hardening checklist

  • Change DB credentials, rotate regularly
  • Move backups to S3 or external storage
  • Add fail2ban and SSH key‑only login
  • Add alerts via ALERT_WEBHOOK for ingest failures
  • Enable Prometheus scraping of /metrics

📄 License & Credits

Public‑good project to make MGNREGA data accessible for citizens. Hindi‑first design, UP scope in Phase 1.