Add support for TTS endpoint

[TheTerrasque]

Would be nice to support the TTS endpoint too, for example to use with https://github.com/remsky/Kokoro-FastAPI/

[mostlygeek]

I pushed a commit into the support-tts branch which adds v1/audio/speech to the supported OpenAI endpoints. I couldn't get Kokoro-FastAPI working with podman as an upstream. So not tested, but it should work. If you get it working please share your configs.

[TheTerrasque]

Thanks!

Got it working, took some time to get podman running (didn't have it installed), but it worked. Takes about 30 seconds to start the backend.

Config:

    cmd: > 
      podman run --rm  
      --name kokoro-tts   
      -v /path/to/Kokoro-FastAPI/api/src:/app/api/src   
      -v /path/to/Kokoro-FastAPI/Kokoro-82M:/app/Kokoro-82M  
      -p 8880:8880   
      -e PYTHONPATH=/app:/app/Kokoro-82M   
      --gpus all   
      ghcr.io/remsky/kokoro-fastapi:v0.0.5
    ttl: 7200
    proxy: http://127.0.0.1:8880

[mostlygeek]

Excellent. Glad to hear it is working and thanks for sharing the configuration.

[mostlygeek]

Here is a working example with docker and https://github.com/remsky/Kokoro-FastAPI/

models:
  kokoro-tts:
    name: "kokoro TTS"
    useModelName: "tts-1"
    cmd: |
      docker run --rm --name ${MODEL_ID}
      -p ${PORT}:8880
      --gpus 'device=0'
      --env 'API_LOG_LEVEL=INFO'
      ghcr.io/remsky/kokoro-fastapi-gpu:latest
    cmdStop: docker stop ${MODEL_ID}

Test it with

curl -v \
 -H "Content-Type: application/json" \
 -d '{"input": "hello, this is a test of kokoro TTS","stream": true, "model": "kokoro-tts"}' \
 http://10.0.1.50:8080/v1/audio/speech | play -t mp3 -