Skip to content
Open

Tx #37

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,8 @@ PostgreSQL (with optional PostGIS) is the primary store for transcripts, callsig
python radioshaq/infrastructure/local/run_alembic.py upgrade head
```

**Docker Compose overrides:** The `radioshaq/infrastructure/local/docker-compose.yml` Postgres services accept env vars to customize credentials and ports. Defaults: `POSTGRES_USER=radioshaq`, `POSTGRES_PASSWORD=radioshaq`, `POSTGRES_DB=radioshaq`, `POSTGRES_PORT=5434` (main); `POSTGRES_TEST_*` for the test DB (port 5433). If you override these, set `RADIOSHAQ_DATABASE__POSTGRES_URL` (and `POSTGRES_*` for Alembic) to match.

See [Quick Start](quick-start.md) for credentials and troubleshooting.

---
Expand Down
2 changes: 1 addition & 1 deletion docs/quick-start.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ Add `--hindsight` to also start the Hindsight container (semantic memory). Or st
--8<-- "snippets/postgres-up.sh"
```

If you use a different host/port or URL, set `RADIOSHAQ_DATABASE__POSTGRES_URL` before running migrations and the API.
If you use a different host/port or URL, set `RADIOSHAQ_DATABASE__POSTGRES_URL` before running migrations and the API. To customize Docker Postgres credentials or ports, set `POSTGRES_USER`, `POSTGRES_PASSWORD`, `POSTGRES_DB`, `POSTGRES_PORT` (and matching `RADIOSHAQ_DATABASE__POSTGRES_URL`) before running `docker compose up`; see [Configuration → Database](configuration.md#database).

---

Expand Down
31 changes: 19 additions & 12 deletions radioshaq/infrastructure/local/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,15 @@
# Docker Compose for RadioShaq local development
#
# Postgres is mapped to host port 5434 to avoid conflict with a local PostgreSQL
# already using 5432. Set DATABASE_URL=postgresql+asyncpg://radioshaq:radioshaq@127.0.0.1:5434/radioshaq
# (or use default in config).
# already using 5432. Default URL: postgresql+asyncpg://radioshaq:radioshaq@localhost:5434/radioshaq
# (or use RADIOSHAQ_DATABASE__POSTGRES_URL).
#
# Override Postgres credentials/ports via env vars (defaults shown):
# POSTGRES_USER=radioshaq POSTGRES_PASSWORD=radioshaq POSTGRES_DB=radioshaq
# POSTGRES_PORT=5434 (host port for main Postgres)
# POSTGRES_TEST_USER=radioshaq POSTGRES_TEST_PASSWORD=radioshaq POSTGRES_TEST_DB=radioshaq_test
# POSTGRES_TEST_PORT=5433 (host port for test Postgres)
# If you change these, set RADIOSHAQ_DATABASE__POSTGRES_URL to match.
#
# Usage:
# docker compose up -d postgres # Start Postgres on 5434
Expand All @@ -21,20 +28,20 @@ services:
container_name: radioshaq-postgres
restart: unless-stopped
environment:
POSTGRES_USER: radioshaq
POSTGRES_PASSWORD: radioshaq
POSTGRES_DB: radioshaq
POSTGRES_USER: ${POSTGRES_USER:-radioshaq}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-radioshaq}
POSTGRES_DB: ${POSTGRES_DB:-radioshaq}
LANG: en_US.utf8
LC_ALL: en_US.utf8
ports:
- "5434:5432"
- "${POSTGRES_PORT:-5434}:5432"
volumes:
# Persist database data
- postgres_data:/var/lib/postgresql/data
# Initial setup scripts
- ./postgres/init:/docker-entrypoint-initdb.d
healthcheck:
test: ["CMD-SHELL", "pg_isready -U radioshaq -d radioshaq"]
test: ["CMD-SHELL", "pg_isready -U $${POSTGRES_USER} -d $${POSTGRES_DB}"]
interval: 10s
timeout: 5s
retries: 5
Expand All @@ -50,15 +57,15 @@ services:
container_name: radioshaq-postgres-test
restart: unless-stopped
environment:
POSTGRES_USER: radioshaq
POSTGRES_PASSWORD: radioshaq
POSTGRES_DB: radioshaq_test
POSTGRES_USER: ${POSTGRES_TEST_USER:-radioshaq}
POSTGRES_PASSWORD: ${POSTGRES_TEST_PASSWORD:-radioshaq}
POSTGRES_DB: ${POSTGRES_TEST_DB:-radioshaq_test}
ports:
- "5433:5432"
- "${POSTGRES_TEST_PORT:-5433}:5432"
volumes:
- postgres_test_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U radioshaq -d radioshaq_test"]
test: ["CMD-SHELL", "pg_isready -U $${POSTGRES_USER} -d $${POSTGRES_DB}"]
interval: 10s
timeout: 5s
retries: 5
Expand Down
25 changes: 0 additions & 25 deletions radioshaq/loguru/__init__.py

This file was deleted.

1 change: 1 addition & 0 deletions radioshaq/radioshaq/api/routes/bus.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ class OptOutBody(BaseModel):
async def publish_inbound(
request: Request,
body: dict[str, Any] = Body(..., embed=False),
Comment on lines 25 to 27
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Breaking change: publish_inbound now requires JWT auth

Adding _user: TokenPayload = Depends(get_current_user) to this endpoint is a good security improvement, but it is a breaking change for any Lambda function or external service that currently calls POST /bus/inbound (or wherever this router is mounted) without a JWT token. Those callers will now receive 401 Unauthorized silently.

It would be worth documenting (in the PR description or changelog):

  • Which callers are affected (e.g. AWS Lambda integrations)
  • The migration path (how to obtain and pass a valid JWT)
  • Whether a service-account token should be pre-generated and distributed to existing callers

_user: TokenPayload = Depends(get_current_user),
) -> dict[str, Any]:
"""
Accept an inbound message (e.g. from Lambda) and publish to MessageBus.
Expand Down
7 changes: 4 additions & 3 deletions radioshaq/radioshaq/api/routes/config_routes.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,9 +69,10 @@ async def get_config_llm(
out = _llm_config_dict(config, redact=True)
override = getattr(request.app.state, "llm_config_override", None)
if override:
for k in _LLM_SECRET_KEYS:
override.pop(k, None)
out = {**out, **override}
sanitized_override = {
k: v for k, v in dict(override).items() if k not in _LLM_SECRET_KEYS
}
out = {**out, **sanitized_override}
out["_meta"] = {"config_applies_after": CONFIG_APPLIES_AFTER_RESTART}
return out

Expand Down
20 changes: 15 additions & 5 deletions radioshaq/radioshaq/api/routes/emergency.py
Original file line number Diff line number Diff line change
Expand Up @@ -92,10 +92,14 @@ async def create_emergency_request(
async def _get_pending_emergency_count(request: Request) -> int:
"""Return number of pending emergency events (for use by pending-count and SSE stream)."""
db = getattr(request.app.state, "db", None)
if db is None or not hasattr(db, "get_pending_coordination_events"):
if db is None or not hasattr(db, "count_pending_coordination_events"):
if db is not None and hasattr(db, "get_pending_coordination_events"):
events = await db.get_pending_coordination_events(
max_results=1000, event_type="emergency"
)
return len(events)
return 0
events = await db.get_pending_coordination_events(max_results=1000, event_type="emergency")
return len(events)
return await db.count_pending_coordination_events(event_type="emergency", status="pending")


@router.get("/pending-count")
Expand Down Expand Up @@ -203,7 +207,10 @@ async def approve_emergency_event(
message_bus = getattr(request.app.state, "message_bus", None)
if not message_bus or not hasattr(message_bus, "publish_outbound"):
await db.update_coordination_event(event_id, status="pending")
return {"ok": True, "event_id": event_id, "status": "pending", "sent": False, "detail": "Message bus not available"}
raise HTTPException(
status_code=503,
detail="Message bus not available; approval rolled back to pending",
)
from radioshaq.vendor.nanobot.bus.events import OutboundMessage
content = extra.get("message") or event.get("notes") or "Emergency notification from RadioShaq."
try:
Expand All @@ -222,7 +229,10 @@ async def approve_emergency_event(
raise HTTPException(status_code=503, detail=f"Outbound bus error: {exc}") from exc
if not ok:
await db.update_coordination_event(event_id, status="pending")
return {"ok": True, "event_id": event_id, "status": "pending", "sent": False, "detail": "Outbound queue full"}
raise HTTPException(
status_code=503,
detail="Outbound queue full; approval rolled back to pending",
)
await db.update_coordination_event(
event_id,
status="approved",
Expand Down
22 changes: 16 additions & 6 deletions radioshaq/radioshaq/api/routes/twilio.py
Original file line number Diff line number Diff line change
Expand Up @@ -145,14 +145,24 @@ async def _handle_inbound(request: Request, explicit_channel: str) -> Response:
"signature_validated": signature_validated,
}

# Opt-out handling (STOP): record directly if DB available, then acknowledge.
# Opt-out handling (STOP): record in DB first; only acknowledge if persisted (TCPA/CASL).
if body.upper() in _OPTOUT_KEYWORDS:
db = getattr(request.app.state, "db", None)
if db is not None and hasattr(db, "record_opt_out_by_phone"):
try:
await db.record_opt_out_by_phone(from_phone, channel)
except Exception as e:
logger.warning("Opt-out record failed (channel={} phone={}): {}", channel, from_phone, e)
if db is None or not hasattr(db, "record_opt_out_by_phone"):
raise HTTPException(
status_code=503,
detail="Opt-out not available; database not configured",
)
try:
await db.record_opt_out_by_phone(from_phone, channel)
except Exception as e:
logger.error(
"Opt-out record failed (channel={} phone={}): {}", channel, from_phone, e
)
raise HTTPException(
status_code=500,
detail="Opt-out could not be recorded; please try again",
) from e
Comment on lines +148 to +165
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

STOP opt-out silently lost on DB failure

Returning a 503 when the database is unavailable causes the STOP request to be silently dropped. Twilio does not automatically retry SMS webhooks by default on 5xx responses — it logs an error and moves on (a fallback URL must be explicitly configured in the Twilio console for retries). This means:

  1. User sends STOP
  2. DB is temporarily down → Twilio receives 503
  3. Twilio does not retry; the opt-out is never recorded
  4. User receives no TwiML acknowledgment
  5. User continues receiving messages, unaware the opt-out failed

This is arguably a TCPA/CASL regression from the previous code, which at least sent the user a confirmation message (even though the opt-out wasn't persisted). With the current approach, when the DB is degraded, there is no user-facing acknowledgment whatsoever and the opt-out record is lost.

Consider implementing a durable retry mechanism (e.g., writing to a dead-letter queue or a fallback store) so opt-out requests survive transient DB outages, and always return a TwiML 200 to Twilio once the opt-out has been enqueued for reliable delivery.

return _twiml_response("You have been opted out. Reply START to re-subscribe.")

inbound = InboundMessage(
Expand Down
2 changes: 1 addition & 1 deletion radioshaq/radioshaq/audio/asr_plugin/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ def _is_voxtral_like_model_id(model_id: str) -> bool:
if not model_id or not model_id.strip():
return False
s = model_id.strip().lower()
return s == "voxtral" or "voxtral" in s or s.startswith("shakods/") or s.startswith("mistralai/voxtral")
return s == "voxtral" or "voxtral" in s or s.startswith("radioshaq/") or s.startswith("mistralai/voxtral")


def transcribe_audio(
Expand Down
25 changes: 18 additions & 7 deletions radioshaq/radioshaq/config/schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
from radioshaq.constants import ASR_LANGUAGE_AUTO, ASR_LANGUAGE_VALUES

logger = logging.getLogger(__name__)
INSECURE_JWT_SECRETS = frozenset({"dev-secret", "dev-secret-change-in-production", "test"})


class Mode(StrEnum):
Expand Down Expand Up @@ -124,7 +125,7 @@ class DatabaseConfig(BaseModel):

# PostgreSQL with PostGIS (default port 5434 matches docker-compose to avoid host 5432 conflict)
postgres_url: str = Field(
default="postgresql+asyncpg://radioshaq:radioshaq@127.0.0.1:5434/radioshaq",
default="postgresql+asyncpg://radioshaq:radioshaq@localhost:5434/radioshaq",
description="PostgreSQL connection URL with asyncpg driver",
)
Comment thread
Josephrp marked this conversation as resolved.
postgres_pool_size: int = Field(default=10, ge=1, le=100)
Expand Down Expand Up @@ -183,11 +184,11 @@ class JWTConfig(BaseModel):
@field_validator("secret_key")
@classmethod
def validate_secret(cls, v: str) -> str:
"""Warn about insecure secrets."""
if v in ("dev-secret", "dev-secret-change-in-production", "test"):
# In production, this should raise an error
pass
return v
"""Normalize and require a non-empty secret string."""
value = (v or "").strip()
if not value:
raise ValueError("jwt.secret_key must not be empty")
return value


class LLMConfig(BaseModel):
Expand Down Expand Up @@ -860,7 +861,17 @@ def expand_path(cls, v: str | Path) -> Path:

@model_validator(mode="after")
def create_directories(self) -> Config:
"""Ensure workspace and data directories exist."""
"""Ensure directories exist and block insecure runtime defaults."""
secret = (self.jwt.secret_key or "").strip()
if secret in INSECURE_JWT_SECRETS:
if self.debug:
logger.warning(
"Using insecure jwt.secret_key in debug mode; set RADIOSHAQ_JWT__SECRET_KEY before production"
)
else:
raise ValueError(
"Insecure jwt.secret_key configured. Set RADIOSHAQ_JWT__SECRET_KEY to a strong secret or enable debug for local development."
)
self.workspace_dir.mkdir(parents=True, exist_ok=True)
self.data_dir.mkdir(parents=True, exist_ok=True)
return self
Comment on lines 861 to 877
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

INSECURE_JWT_SECRETS check in create_directories conflates two concerns

create_directories validates directory creation and enforces a security policy on the JWT secret. If the secret check raises ValueError, the directories are never created, leaving workspace_dir and data_dir potentially absent. Consider reordering so directory creation happens first (or in a separate validator), and then raising on insecure secrets — or breaking these into two @model_validator methods with clearer names.

Additionally, the validate_secret field validator already strips and validates the key, so (self.jwt.secret_key or "").strip() here is redundant — self.jwt.secret_key is already stripped at that point.

Expand Down
17 changes: 16 additions & 1 deletion radioshaq/radioshaq/database/postgres_gis.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
from typing import Any, Sequence

from geoalchemy2.functions import ST_DWithin, ST_GeogFromText
from sqlalchemy import delete, select, text, update
from sqlalchemy import delete, func, select, text, update
from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine
from sqlalchemy.orm import sessionmaker

Expand Down Expand Up @@ -625,6 +625,21 @@ async def get_pending_coordination_events(
result = await session.execute(query)
return [row.to_dict() for row in result.scalars()]

async def count_pending_coordination_events(
self,
event_type: str | None = None,
status: str = "pending",
) -> int:
"""Return the number of coordination events matching the filters (COUNT(*) query)."""
async with self.async_session() as session:
query = select(func.count()).select_from(CoordinationEvent).where(
CoordinationEvent.status == status
)
if event_type:
query = query.where(CoordinationEvent.event_type == event_type)
result = await session.execute(query)
return result.scalar_one() or 0

async def get_emergency_events_with_locations(
self,
since: str | None = None,
Expand Down
1 change: 1 addition & 0 deletions radioshaq/web-interface/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
# API base URL (default: http://localhost:8000)
# VITE_RADIOSHAQ_API=http://localhost:8000

# Auth token is acquired at runtime (POST /auth/token) and applied with setApiToken(...)
# Optional: Bearer token for authenticated API calls
# VITE_RADIOSHAQ_TOKEN=

Expand Down
4 changes: 2 additions & 2 deletions radioshaq/web-interface/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ npm install
## Environment

- `VITE_RADIOSHAQ_API` – API base URL (default: `http://localhost:8000`)
- `VITE_RADIOSHAQ_TOKEN` – Optional Bearer token for authenticated API calls
- Authentication token is set at runtime via login (`/auth/token`) and `setApiToken(...)`; it is not read from `VITE_` build-time env vars
- `VITE_GOOGLE_MAPS_API_KEY` – Optional. Google Maps JavaScript API key for the **Map** page, **Radio** page field map panel, and **Transcripts** “View on map”. Create a key in [Google Cloud Console](https://console.cloud.google.com/apis/credentials), enable **Maps JavaScript API**, and restrict the key by HTTP referrer to your app origin(s) to avoid misuse. If unset, map features show a short message instead of a map.

## Run
Expand All @@ -21,7 +21,7 @@ npm install
npm run dev
```

Open http://localhost:3000. The dev server proxies `/api` and `/ws` to the RadioShaq API (port 8000).
Open <http://localhost:3000>. The dev server proxies `/api` and `/ws` to the RadioShaq API (port 8000).

## Build

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ function OperatorMapGoogle({
mapRef.current = null;
infoWindowRef.current = null;
};
}, []);
}, [clearMarkers]);

useEffect(() => {
const map = mapRef.current;
Expand Down
7 changes: 6 additions & 1 deletion radioshaq/web-interface/src/locales/es.json
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,9 @@
"updateLocation": "Actualizar mi ubicación",
"lat": "Latitud",
"lng": "Longitud",
"setLocation": "Establecer ubicación",
"latInvalid": "La latitud debe estar entre -90 y 90.",
"lngInvalid": "La longitud debe estar entre -180 y 180.",
"locationUpdated": "Ubicación actualizada.",
"viewOnMap": "Ver en mapa",
"noLocationsForCallsigns": "No hay ubicaciones para estos indicativos.",
Expand All @@ -54,7 +57,9 @@
"approve": "Aprobar y enviar",
"reject": "Rechazar",
"allowNotifications": "Permitir notificaciones",
"autoRefresh": "La lista se actualiza cada 12 s; la app también comprueba cada 15 s y reproduce una alerta y muestra una notificación cuando llegan nuevos mensajes."
"autoRefresh": "La lista se actualiza cada 12 s; la app también comprueba cada 15 s y reproduce una alerta y muestra una notificación cuando llegan nuevos mensajes.",
"mapTitle": "Eventos en el mapa",
"viewOnMap": "Ver en mapa"
},
"license": {
"title": "Se requiere aceptación de la licencia",
Expand Down
7 changes: 6 additions & 1 deletion radioshaq/web-interface/src/locales/fr.json
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,9 @@
"updateLocation": "Mettre à jour ma position",
"lat": "Latitude",
"lng": "Longitude",
"setLocation": "Définir la position",
"latInvalid": "La latitude doit être comprise entre -90 et 90.",
"lngInvalid": "La longitude doit être comprise entre -180 et 180.",
"locationUpdated": "Position mise à jour.",
"viewOnMap": "Voir sur la carte",
"noLocationsForCallsigns": "Aucune position pour ces indicatifs.",
Expand All @@ -54,7 +57,9 @@
"approve": "Approuver et envoyer",
"reject": "Rejeter",
"allowNotifications": "Autoriser les notifications",
"autoRefresh": "La liste s'actualise toutes les 12 s ; l'app vérifie aussi toutes les 15 s et joue une alerte et affiche une notification à l'arrivée de nouveaux messages."
"autoRefresh": "La liste s'actualise toutes les 12 s ; l'app vérifie aussi toutes les 15 s et joue une alerte et affiche une notification à l'arrivée de nouveaux messages.",
"mapTitle": "Événements sur la carte",
"viewOnMap": "Voir sur la carte"
},
"license": {
"title": "Acceptation de la licence requise",
Expand Down
2 changes: 1 addition & 1 deletion radioshaq/web-interface/src/main.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import App from './App';

if (import.meta.env.VITE_SHAKODS_API != null || import.meta.env.VITE_SHAKODS_TOKEN != null) {
console.warn(
'Deprecated: VITE_SHAKODS_API and VITE_SHAKODS_TOKEN are no longer used. Use VITE_RADIOSHAQ_API and VITE_RADIOSHAQ_TOKEN instead.'
'Deprecated: VITE_SHAKODS_API and VITE_SHAKODS_TOKEN are no longer used. Use VITE_RADIOSHAQ_API and runtime token login (/auth/token) instead.'
);
}

Expand Down
Loading
Loading