Skip to content

✨(devex) add LIVEKIT_INTERNAL_URL for server-to-server communication#1172

Closed
cameledev wants to merge 1 commit intomainfrom
fix-livekit-url
Closed

✨(devex) add LIVEKIT_INTERNAL_URL for server-to-server communication#1172
cameledev wants to merge 1 commit intomainfrom
fix-livekit-url

Conversation

@cameledev
Copy link
Copy Markdown
Collaborator

NB: this requires extra attention, as the following is proposed by AI, and I don't have full understanding of the implications !

Problem

Docker Compose's extra_hosts: host-gateway maps 127.0.0.1.nip.io to 172.17.0.1 (the docker0 bridge). This worked with Docker Compose plugin 5.0.2, but after an upgrade to 5.1.0, the DOCKER-FORWARD chain now isolates bridge networks by default — traffic from the compose network (172.18.0.0/16) to the docker0 bridge (172.17.0.1) is dropped.

This broke all server-side LiveKit API calls (telephony SIP dispatch, recording egress) because the backend container used LIVEKIT_API_URL (http://127.0.0.1.nip.io:7880) for both:

  • Sending the URL to the browser (which needs a host-accessible address)
  • Making server-to-server API calls (which should use container-to-container networking)

Fix

Added LIVEKIT_INTERNAL_URL setting for server-to-server LiveKit communication. In development, this is set to http://livekit:7880 (Docker DNS), while LIVEKIT_API_URL remains http://127.0.0.1.nip.io:7880 for the browser. create_livekit_client() uses the internal URL when set, falling back to LIVEKIT_API_URL for backwards compatibility.

Original error

 "OPTIONS /api/v1.0/rooms/6045b9ce-3a7d-4b66-b79f-947103dc6dc2/start-recording/ HTTP/1.1" 200 0
Cannot connect to host 127.0.0.1.nip.io:7880 ssl:default [Connect call failed ('172.17.0.1', 7880)]
Internal Server Error: /api/v1.0/rooms/webhooks-livekit/
Traceback (most recent call last):
  File "/app/.venv/lib/python3.13/site-packages/aiohttp/connector.py", line 1298, in _wrap_create_connection
    sock = await aiohappyeyeballs.start_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/app/.venv/lib/python3.13/site-packages/aiohappyeyeballs/impl.py", line 122, in start_connection
    raise first_exception
  File "/app/.venv/lib/python3.13/site-packages/aiohappyeyeballs/impl.py", line 73, in start_connection
    sock = await _connect_sock(
           ^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/app/.venv/lib/python3.13/site-packages/aiohappyeyeballs/impl.py", line 208, in _connect_sock
    await loop.sock_connect(sock, address)
  File "/usr/local/lib/python3.13/asyncio/selector_events.py", line 641, in sock_connect
    return await fut
           ^^^^^^^^^
  File "/usr/local/lib/python3.13/asyncio/selector_events.py", line 681, in _sock_connect_cb
    raise OSError(err, f'Connect call failed {address}')
TimeoutError: [Errno 110] Connect call failed ('172.17.0.1', 7880)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/.venv/lib/python3.13/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/app/.venv/lib/python3.13/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/app/.venv/lib/python3.13/site-packages/django/views/decorators/csrf.py", line 65, in _view_wrapper
    return view_func(request, *args, **kwargs)
  File "/app/.venv/lib/python3.13/site-packages/rest_framework/viewsets.py", line 125, in view
    return self.dispatch(request, *args, **kwargs)
           ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.13/site-packages/rest_framework/views.py", line 515, in dispatch
    response = self.handle_exception(exc)
  File "/app/.venv/lib/python3.13/site-packages/rest_framework/views.py", line 475, in handle_exception
    self.raise_uncaught_exception(exc)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^
  File "/app/.venv/lib/python3.13/site-packages/rest_framework/views.py", line 486, in raise_uncaught_exception
    raise exc
  File "/app/.venv/lib/python3.13/site-packages/rest_framework/views.py", line 512, in dispatch
    response = handler(request, *args, **kwargs)
  File "/app/core/api/viewsets.py", line 494, in webhooks_livekit
    livekit_events_service.receive(request)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/app/core/services/livekit_events.py", line 145, in receive
    handler(data)
    ~~~~~~~^^^^^^
  File "/app/core/services/livekit_events.py", line 220, in _handle_room_started
    self.telephony_service.create_dispatch_rule(room)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/app/.venv/lib/python3.13/site-packages/asgiref/sync.py", line 325, in __call__
    return call_result.result()
           ~~~~~~~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.13/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.13/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/app/.venv/lib/python3.13/site-packages/asgiref/sync.py", line 365, in main_wrap
    result = await awaitable
             ^^^^^^^^^^^^^^^
  File "/app/core/services/telephony.py", line 52, in create_dispatch_rule
    await lkapi.sip.create_sip_dispatch_rule(create=request)
  File "/app/.venv/lib/python3.13/site-packages/livekit/api/sip_service.py", line 559, in create_sip_dispatch_rule
    return await self.create_dispatch_rule(create)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.13/site-packages/livekit/api/sip_service.py", line 532, in create_dispatch_rule
    return await self._client.request(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/app/.venv/lib/python3.13/site-packages/livekit/api/twirp_client.py", line 121, in request
    async with self._session.post(
               ~~~~~~~~~~~~~~~~~~^
        url, headers=headers, data=serialized_data, timeout=timeout
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ) as resp:
    ^
  File "/app/.venv/lib/python3.13/site-packages/aiohttp/client.py", line 1510, in __aenter__
    self._resp: _RetType = await self._coro
                           ^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.13/site-packages/aiohttp/client.py", line 779, in _request
    resp = await handler(req)
           ^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.13/site-packages/aiohttp/client.py", line 734, in _connect_and_send_request
    conn = await self._connector.connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        req, traces=traces, timeout=real_timeout
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/app/.venv/lib/python3.13/site-packages/aiohttp/connector.py", line 672, in connect
    proto = await self._create_connection(req, traces, timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.13/site-packages/aiohttp/connector.py", line 1239, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.13/site-packages/aiohttp/connector.py", line 1611, in _create_direct_connection
    raise last_exc
  File "/app/.venv/lib/python3.13/site-packages/aiohttp/connector.py", line 1580, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<7 lines>...
    )
    ^
  File "/app/.venv/lib/python3.13/site-packages/aiohttp/connector.py", line 1321, in _wrap_create_connection
    raise client_error(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host 127.0.0.1.nip.io:7880 ssl:default [Connect call failed ('172.17.0.1', 7880)]```

Introduced a LIVEKIT_INTERNAL_URL environment variable for
container-to-container LiveKit communication, using http://livekit:7880 in development.
LIVEKIT_API_URL continues to serve browser-facing requests, and
create_livekit_client() prefers the internal URL when set, preserving
 backwards compatibility.
@sonarqubecloud
Copy link
Copy Markdown

# Internal URL for server-to-server LiveKit API calls (e.g. container-to-container).
# Defaults to LIVEKIT_API_URL when not set.
LIVEKIT_INTERNAL_URL = values.Value(
"", environ_name="LIVEKIT_INTERNAL_URL", environ_prefix=None
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

default shoud be None, not ""

Comment thread src/backend/core/utils.py
Comment on lines +215 to +219
configuration = {
**(custom_configuration or settings.LIVEKIT_CONFIGURATION),
"url": settings.LIVEKIT_INTERNAL_URL
or (custom_configuration or settings.LIVEKIT_CONFIGURATION)["url"],
}
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It’s a bit hard to follow, something like this might be clearer with an explicit comment:

Suggested change
configuration = {
**(custom_configuration or settings.LIVEKIT_CONFIGURATION),
"url": settings.LIVEKIT_INTERNAL_URL
or (custom_configuration or settings.LIVEKIT_CONFIGURATION)["url"],
}
configuration = custom_configuration or settings.LIVEKIT_CONFIGURATION
# ... clear commment it's necessary for the docker compose stack
if settings.LIVEKIT_INTERNAL_URL is not None:
configuration["url"] = settings.LIVEKIT_INTERNAL_URL

@cameledev
Copy link
Copy Markdown
Collaborator Author

The problem was a firewall issue on my local machine...

@cameledev cameledev closed this Mar 24, 2026
@lebaudantoine lebaudantoine deleted the fix-livekit-url branch March 28, 2026 13:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants