Alert when a Docker container restarts or exits

A container that restarts in a loop or exits with a non-zero code is easy to miss if you only look at docker ps once a day. This guide shows you how to alert when a Docker container restarts or exits by reading the restart count and exit code from the Docker API, comparing them with the last known values, and sending a notification from a small script.

The check runs from a systemd timer every minute, keeps its state in /var/lib/container-watch, and works for any container on the host, including ones started by Docker Compose.

Before you start

You need:

• A Linux VPS running Ubuntu 22.04/24.04, Debian 12, or AlmaLinux/Rocky 8–9 with Docker installed.
• Root or a user in the docker group, plus sudo.
• A notification channel. Telegram is used in the examples because it needs only a bot token and a chat ID, but the same script works with email or an HTTP webhook.
• Basic comfort with bash and systemctl.

If you have not connected to the server yet, the browser console in your VPS provider's control panel works for the first login.

Understand restart counts and exit codes

Docker tracks two things you care about. RestartCount increases every time the restart policy brings a container back up. State.ExitCode records why the process stopped the last time it ran.

docker ps -a --format 'table {{.Names}}\t{{.Status}}\t{{.Image}}'

docker inspect --format '{{.Name}} restarts={{.RestartCount}} status={{.State.Status}} exit={{.State.ExitCode}} oom={{.State.OOMKilled}}' $(docker ps -aq)

Common exit codes:

• 0 — the process finished normally. Fine for a one-shot job, a problem for a long-running service.
• 1 — a general application error. Check the logs.
• 137 — killed with SIGKILL, often the OOM killer. Confirm with OOMKilled=true.
• 143 — killed with SIGTERM, usually a normal stop or a redeploy.

For live events you can stream the Docker event bus instead:

docker events --filter event=die --filter event=restart --filter event=oom

That is useful while debugging, but it only reports while it is running. For alerting, a periodic check against docker inspect survives reboots and missed windows.

Write the watchdog script

Create /usr/local/bin/container-watch.sh. It loops over every container, writes the current values to a state file, and compares them with the previous run.

sudo nano /usr/local/bin/container-watch.sh

#!/usr/bin/env bash
set -uo pipefail

STATE_DIR=/var/lib/container-watch
mkdir -p "$STATE_DIR"

TELEGRAM_TOKEN="${TELEGRAM_TOKEN:-}"
TELEGRAM_CHAT_ID="${TELEGRAM_CHAT_ID:-}"

notify() {
  local msg="$1"
  logger -t container-watch "$msg"
  if [[ -n "$TELEGRAM_TOKEN" && -n "$TELEGRAM_CHAT_ID" ]]; then
    curl -fsS --max-time 10 \
      -X POST "https://api.telegram.org/bot${TELEGRAM_TOKEN}/sendMessage" \
      --data-urlencode "chat_id=${TELEGRAM_CHAT_ID}" \
      --data-urlencode "text=${msg}" >/dev/null \
      || logger -t container-watch "telegram send failed"
  fi
}

for cid in $(docker ps -aq); do
  read -r name restarts status exitcode oom < <(
    docker inspect --format '{{.Name}} {{.RestartCount}} {{.State.Status}} {{.State.ExitCode}} {{.State.OOMKilled}}' "$cid"
  )
  name="${name#/}"

  state_file="$STATE_DIR/${cid}.state"
  prev=""
  [[ -f "$state_file" ]] && prev=$(cat "$state_file")
  echo "$restarts $status $exitcode" > "$state_file"

  # First run for this container: store a baseline, do not alert.
  [[ -z "$prev" ]] && continue
  read -r prev_restarts prev_status prev_exit <<< "$prev"

  if (( restarts > prev_restarts )); then
    notify "Container ${name} restarted (count ${prev_restarts} -> ${restarts}, status ${status})"
  fi

  if [[ "$status" == "exited" && "$prev_status" != "exited" ]]; then
    notify "Container ${name} exited with code ${exitcode} (OOMKilled=${oom})"
  fi
done

Make it executable:

sudo chmod 700 /usr/local/bin/container-watch.sh

The state file is keyed by container ID, so a container that is recreated by docker compose up --force-recreate starts from a fresh baseline instead of firing a false alert.

Send alerts to Telegram, email or a webhook

Put the credentials in a root-only environment file so they never sit in the script itself.

sudo tee /etc/container-watch.env >/dev/null <<'EOF'
TELEGRAM_TOKEN=123456789:AAExampleTokenFromBotFather
TELEGRAM_CHAT_ID=123456789
EOF
sudo chmod 600 /etc/container-watch.env

To use email instead, replace the curl call with:

echo "$msg" | mail -s "Container alert on $(hostname)" ops@example.com

For a generic webhook, post JSON:

curl -fsS -X POST -H 'Content-Type: application/json' \
  -d "{\"text\":\"${msg}\"}" https://example.com/hooks/containers

Run the check every minute with a systemd timer

Create the service unit:

# /etc/systemd/system/container-watch.service
[Unit]
Description=Check Docker containers for restarts and exits

[Service]
Type=oneshot
EnvironmentFile=/etc/container-watch.env
ExecStart=/usr/local/bin/container-watch.sh

Create the timer:

# /etc/systemd/system/container-watch.timer
[Unit]
Description=Run container-watch every minute

[Timer]
OnBootSec=2min
OnUnitActiveSec=1min
AccuracySec=10s

[Install]
WantedBy=timers.target

Enable both:

sudo systemctl daemon-reload
sudo systemctl enable --now container-watch.timer
systemctl list-timers container-watch.timer

The timer fires one minute apart and runs the script as root, so it can read the Docker socket without extra group changes.

Check it works

Start a throwaway container, take a baseline, then kill it and run the check again.

docker run -d --name watchtest --restart unless-stopped alpine sleep 3600
sudo /usr/local/bin/container-watch.sh   # baseline, no alert expected
docker kill watchtest
sleep 5
sudo /usr/local/bin/container-watch.sh   # restart alert expected

Now test the exit path with a container that fails on purpose:

docker run --name exit-test alpine sh -c 'exit 1'
sudo /usr/local/bin/container-watch.sh

You should receive a message about exit-test with exit code 1. Confirm the timer is running and check the log:

systemctl status container-watch.timer
journalctl -t container-watch -n 20 --no-pager

Clean up the test containers when you are done:

docker rm -f watchtest exit-test
sudo rm -f /var/lib/container-watch/*.state

Troubleshooting

• No alert when a container dies. The container has restart policy no, so it stays in exited and RestartCount never changes. The script handles this through the exit-state check — verify the state file in /var/lib/container-watch is being written.
• Telegram returns HTTP 400 or 403. The chat ID is wrong, or the bot has never been messaged. Open the bot in Telegram, send /start, then read the chat ID from https://api.telegram.org/bot<TOKEN>/getUpdates.
• Alerts repeat after every reboot. The state directory is on a tmpfs path such as /tmp or /run. Keep it under /var/lib so the baseline survives reboots.
• The script exits early with no output. set -e combined with a failing docker inspect stops the loop. The script above uses set -uo pipefail without -e for that reason; keep it that way.
• A redeployed container triggers a false restart alert. Compose recreates containers with a new ID, so the state file is new and the first run is treated as a baseline. If you see repeats, check that the container is not being recreated on every deploy.

Next steps

• Diagnose a service that will not stay up with Fix a Docker container that exits immediately.
• Add an external view of the same services with Self-host Uptime Kuma for uptime and alert monitoring.
• Track trends over time with Monitor Docker containers with Prometheus, cAdvisor and Grafana.
• Keep the evidence around with View and rotate Docker container logs.