Skip to content

[BUG] Compose watch can recreate a depends_on service and leave it stuck in Created #13717

@jackmpcollins

Description

@jackmpcollins

Description

Compose watch/reconcile appears to mishandle depends_on when both the watched service and its dependency have develop.watch configured.

I reduced this to a minimal public repro in https://github.com/jackmpcollins/compose-watch-backend-recreate-repro

Current behavior:

  • I run docker compose up --build --watch
  • I touch only frontend/trigger.txt
  • Compose reports that it is rebuilding frontend
  • Compose also rebuilds/reconciles backend
  • backend is recreated and then left stuck in Created
  • frontend stays up, but requests return 502
  • frontend logs show lookup backend on 127.0.0.11:53: no such host

Expected behavior:

  • touching only the frontend trigger file should not strand backend
  • either only frontend should be recreated, or if backend is touched by reconciliation it should come
    back up cleanly

This seems related to:

Steps To Reproduce

I reduced this to a minimal public repro in https://github.com/jackmpcollins/compose-watch-backend-recreate-repro


Setup

Minimal docker-compose.yml:

services:
  backend:
    build: ./backend
    develop:
      watch:
        - action: restart
          path: ./backend/Dockerfile

  frontend:
    depends_on:
      - backend
    build: ./frontend
    ports:
      - "3002:3000"
    develop:
      watch:
        - action: rebuild
          path: ./frontend/trigger.txt

backend/Dockerfile:

FROM hashicorp/http-echo:1.0.0

CMD ["-text=backend ok", "-listen=:8000"]

frontend/Dockerfile:

FROM caddy:2.10.2-alpine

COPY Caddyfile /etc/caddy/Caddyfile

frontend/Caddyfile:

:3000 {
      reverse_proxy backend:8000
}

frontend/trigger.txt:

touch me to trigger a frontend rebuild

Repro

Repro path 1:

docker compose down
docker compose up -d --build
docker compose watch frontend

Repro path 2:

docker compose down
docker compose up --build --watch

In another shell:

touch frontend/trigger.txt
docker compose ps
docker compose ps -a
curl -i http://localhost:3002/
docker compose logs --since=1m frontend

Observed watch output includes:

Rebuilding service(s) ["frontend"] after changes were detected...
Container <project>-backend-1 Recreate
Container <project>-backend-1 Recreated
Container <project>-frontend-1 Recreate
Container <project>-frontend-1 Recreated

With docker compose up --build --watch, I also see:

backend-1 has been recreated
backend-1 ... received interrupt, shutting down...
backend-1 exited with code 2

Observed broken state after that:

  • docker compose ps no longer lists backend
  • docker compose ps -a shows backend in Created
  • curl http://localhost:3002/ returns 502
  • frontend logs show lookup backend on 127.0.0.11:53: no such host

Ablation notes:

  • removing backend.develop.watch makes the bug disappear
  • removing frontend.depends_on makes the bug disappear
  • removing frontend.develop.watch makes the bug disappear
  • changing backend to a pure image: service makes the bug disappear
  • the backend watch action does not need to be rebuild; restart is enough

Compose Version

Docker Compose version v5.1.1
Docker Compose version v5.1.1

Docker Environment

Client:
 Version:    29.3.1
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  agent: Docker AI Agent Runner (Docker Inc.)
    Version:  v1.34.0
    Path:     /Users/jack/.docker/cli-plugins/docker-agent
  ai: Docker AI Agent - Ask Gordon (Docker Inc.)
    Version:  v1.20.1
    Path:     /Users/jack/.docker/cli-plugins/docker-ai
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.32.1-desktop.1
    Path:     /Users/jack/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v5.1.1
    Path:     /Users/jack/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.47
    Path:     /Users/jack/.docker/cli-plugins/docker-debug
  desktop: Docker Desktop commands (Docker Inc.)
    Version:  v0.3.0
    Path:     /Users/jack/.docker/cli-plugins/docker-desktop
  dhi: CLI for managing Docker Hardened Images (Docker Inc.)
    Version:  v0.0.2
    Path:     /Users/jack/.docker/cli-plugins/docker-dhi
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.31
    Path:     /Users/jack/.docker/cli-plugins/docker-extension
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.4.0
    Path:     /Users/jack/.docker/cli-plugins/docker-init
  mcp: Docker MCP Plugin (Docker Inc.)
    Version:  v0.40.3
    Path:     /Users/jack/.docker/cli-plugins/docker-mcp
  model: Docker Model Runner (Docker Inc.)
    Version:  v1.1.28
    Path:     /Users/jack/.docker/cli-plugins/docker-model
  offload: Docker Offload (Docker Inc.)
    Version:  v0.5.77
    Path:     /Users/jack/.docker/cli-plugins/docker-offload
  pass: Docker Pass Secrets Manager Plugin (beta) (Docker Inc.)
    Version:  v0.0.24
    Path:     /Users/jack/.docker/cli-plugins/docker-pass
  sandbox: Docker Sandbox (Docker Inc.)
    Version:  v0.12.0
    Path:     /Users/jack/.docker/cli-plugins/docker-sandbox
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/jack/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.20.3
    Path:     /Users/jack/.docker/cli-plugins/docker-scout

Server:
 Containers: 2
  Running: 1
  Paused: 0
  Stopped: 1
 Images: 37
 Server Version: 29.3.1
 Storage Driver: overlayfs
  driver-type: io.containerd.snapshotter.v1
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 CDI spec directories:
  /etc/cdi
  /var/run/cdi
 Discovered Devices:
  cdi: docker.com/gpu=webgpu
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: dea7da592f5d1d2b7755e3a161be07f43fad8f75
 runc version: v1.3.4-0-gd6d73eb8
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.12.76-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 12
 Total Memory: 23.43GiB
 Name: docker-desktop
 ID: 48c0ae5a-cc67-41ed-9230-7e1bc099096d
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///Users/jack/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  ::1/128
  127.0.0.0/8
 Live Restore Enabled: false
 Firewall Backend: iptables

Anything else?

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions