8 Commits
Author SHA1 Message Date
Europium 63 57100d0a7b Inspector 0.9.3: all times follow the stack's TZ; docs explain why it matters 2026-10-28 01:33:53 -07:00
Europium 63 b4da527732 Inspector 0.9.2: settings validation, sign-in hardening and other fixes 2026-10-28 01:06:34 -07:00
Europium 63 284ba2ea15 Inspector 0.9.1: guide explains VPN apps and Portainer setup; dependency and token fixes 2026-10-28 00:30:40 -07:00
Europium 63 bd78835f4a Inspector 0.9.0: edit every container option and the Uptime Kuma / Portainer integrations in the dashboard 2026-10-28 00:03:19 -07:00
Europium 63 7013a13cba Update logo in README 2026-10-27 16:44:33 -07:00
Europium 63 c603e86e75 Release v0.8.0 — Portainer-first install, in-app guide
- Install from Portainer with no command line: paste the compose, set a
  folder and a password, deploy. Inspector creates its own config and
  detects Docker socket permissions by itself.
- Illustrated how-to guide inside the app (Help tab) and in the repo.
- Containers added in the UI are watched right away, no restart.
- maintenance_window switch on the container form.
- README with logo and screenshots; INSTALL rewritten for Portainer.
2026-10-27 15:45:33 -07:00
Europium 63 4c05a4d565 Release v0.7.4 — compact container list in settings rail 2026-10-27 15:11:35 -07:00
Europium 63 88f6d06df9 Release v0.7.3 — Containers under Settings, per-channel priorities, restart_on cards
- Containers tab moved into the settings rail (All containers grid + per-container entries).
- Per-platform priority references for Gotify and Pushover.
- restart_on redesigned as toggleable cards with descriptions and quick presets.
- Fix: unchecking every restart_on status now saves observe-only.
2026-10-27 14:45:17 -07:00
30 changed files with 2263 additions and 362 deletions

No files matched your search

+203
View File
@@ -7,6 +7,209 @@ Version numbers follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html
--- ---
## [0.9.3] — 2026-10-28
### Changed
- **Every time follows the `TZ` of the Inspector stack.** Before, the dashboard,
Events page and alerts were always in US Pacific time while the maintenance window
and logs used `TZ`, so anyone outside Pacific saw times hours off (e.g. a 02:00
Berlin window shown as 5:00 PM). Now the dashboard, the Events page (including
where each day starts), alert times, cooldown times and logs all use `TZ`.
- Alert times include the zone abbreviation (`10-28 09:27 AM CET`), and "Cooldown
until" in alerts, event messages and logs is a local time instead of a UTC
timestamp.
- The active time zone is shown at the bottom of every Settings page, on the Events
page and on the Maintenance page.
- **If you never set `TZ`**, times are now shown in UTC instead of Pacific. Set `TZ`
in the stack (e.g. `America/New_York`). An unrecognized name logs a warning and
falls back to UTC.
### Docs
- INSTALL: `TZ` is now marked as needed, with a "Why the time zone matters" section
(maintenance window, alerts, dashboard, logs), how to find your zone name, and a
troubleshooting row. Guide: a Time zone section and a first-login check. Compose
and README updated.
---
## [0.9.2] — 2026-10-28
Fixes from a code and security review. None of them change how containers are
monitored or recovered.
### Fixed
- **A bad value saved from Settings can no longer stop Inspector from starting.**
The forms now reject invalid times (maintenance start/end), unknown log levels and
out-of-range numbers (for example `interval_seconds: 0`, which made the loop spin).
If `config.yml` already has such a value, Inspector starts with the default and
logs a warning instead of crashing in a restart loop.
- **The dashboard password can no longer be cleared.** Saving an empty password (or
username) is refused, and an empty password never signs anyone in. Before, clearing
it left the running dashboard open to anyone.
- **Passwords with non-ASCII characters (é, ü, £ …) now work.** Before, every login
attempt with one crashed.
- The Events page no longer errors for a moment after a container is removed.
- The HTTP check hint no longer suggests `302` as an expected status; redirects are
followed, so the final page's code is what counts.
### Security
- **Sign-in throttling:** after 5 failed sign-ins (form or Basic Auth) from one address
within 5 minutes, that address is refused for the rest of the window.
- **Changing the dashboard username or password signs out existing sessions.**
Expired sessions are cleaned up.
- Request bodies are limited to 1 MB; an invalid `Content-Length` gets a 400.
- The Gotify token is sent in the `X-Gotify-Key` header instead of the URL, so it no
longer appears in Gotify or reverse-proxy access logs.
- Portainer token environment variables must be named `PORTAINER_…` (a name already in
your `config.yml` keeps working), so Test connection can't be used to send other
container secrets to an arbitrary URL.
- Single quotes are escaped in HTML, and the delete confirmations build their text safely.
- Two settings saves at the same moment can no longer overwrite each other.
- `requests` updated to 2.32.5 (CVE-2024-47081).
---
## [0.9.1] — 2026-10-28
### Fixed
- **A dependency on a disabled container no longer blocks recovery forever.** If
`depends_on` named a container that is disabled in Inspector, the dependent app was
never restarted. Inspector now only waits for dependencies it is watching, and the
container page warns when the selected dependency is disabled.
- **`PORTAINER_API_TOKEN` now reaches the container.** The public `compose.yml` didn't
pass it in, so setting it as a Portainer stack variable had no effect. Existing
stacks need the line `- PORTAINER_API_TOKEN=${PORTAINER_API_TOKEN:-}` added under
`environment:` (Stacks → inspector → Editor).
- The "depends_on … is disabled or not found" warning was logged on every check
(since 0.8.0). It is now logged once, and again only when the container settings
change.
### Changed
- Guide: new sections **6. Apps behind a VPN container** (why depends_on matters, how to
spot a VPN-routed app, setup, what Inspector does) and **7. Portainer stack redeploy**
(why it's needed, what a redeploy does to the whole stack, requirements, where to find
every value, adding the token, the other settings, Test connection errors).
- Clearer form hints: `vpn_routed` only affects logging; `depends_on` works for any
dependency; where to find the endpoint ID.
---
## [0.9.0] — 2026-10-28
### Added
- **Every container option is now in the dashboard.** The container page has three
new groups:
- **Reachability**: HTTP check (URL, timeout, expected status) and Uptime Kuma
monitor.
- **Dependencies**: `depends_on` (pick from your containers) and `vpn_routed`.
- **Recovery**: `recovery_action` with plain-language descriptions, plus the
Portainer environment and stack name for stack redeploys.
- **Integrations settings.** New **Uptime Kuma** and **Portainer** pages under
Settings. Portainer environments can be added, edited, renamed (containers
that use them are updated) and deleted (refused while a container uses one).
The API token can be an environment variable name (recommended) or a masked
value saved in `config.yml`.
- **Test connection** buttons for Uptime Kuma (lists monitors) and each Portainer
environment (lists stacks on the endpoint), using the values in the form even
before saving.
- The All containers grid shows each container's extras (HTTP check, Uptime Kuma,
dependency, recovery action).
- Guide: new sections on reachability checks and apps behind a VPN container.
### Changed
- Uptime Kuma and Portainer changes take effect on the next check; Inspector
rebuilds those clients when their settings change instead of needing a restart.
- Saving a container form unchanged now leaves `config.yml` byte-identical. Blank
optional fields are removed instead of written as empty values (`color: ''`,
`maintenance_window: false`), and an unchanged `restart_on` list keeps its
order and the comments after it. An explicit `recovery_action: restart_container`
is kept.
---
## [0.8.0] — 2026-10-27
### Added
- **Install from Portainer with no command line.** Mount one folder per purpose
(`/app/config`, `/app/logs`, `/app/data`) and deploy. If there's no config yet,
Inspector writes a starter `config.yml` (dashboard on, event history on, no
containers). Dashboard login comes from `INSPECTOR_USERNAME` / `INSPECTOR_PASSWORD`;
without a password, one is generated and printed once in the container logs.
- **Docker socket access is automatic.** A new entrypoint starts as root only long
enough to read the socket's group and fix ownership of the mounted folders, then
runs Inspector as `PUID:PGID` (default `1000:1000`) with that group added.
`DOCKER_GID` / `group_add` are no longer needed.
- **In-app guide** at `/guide/` with screenshots, linked from a new **Help** tab
in the header. Also in the repo at `src/inspector/assets/guide/guide.html`.
- **`maintenance_window` switch** on the container form, so containers can opt in
to the daily maintenance window without editing YAML.
### Fixed
- Containers added from the UI are now watched on the next check. Before, the
loop only logged that a restart was needed, despite what the UI said.
Containers removed or disabled from the UI are dropped the same way.
- Status page could fail to render while a container was being added or removed
(it iterated the live state dict and assumed every entry was still in config).
### Changed
- Config path: `--config`, then `$INSPECTOR_CONFIG`, then `/app/config.yml` if
present (existing single-file setups keep working unchanged), then
`/app/config/config.yml`.
- Public `compose.yml`, `INSTALL.md`, and `README.md` rewritten for Portainer
stacks; README gains the logo and screenshots.
### Upgrade notes
- Existing deployments need no changes. To adopt the simpler layout, move
`config.yml` into the new config folder and switch to the new compose file.
---
## [0.7.4] — 2026-10-27
### Changed
- **Settings rail container list is compact.** Per-container entries are indented
under "All containers" and fit on one line: colored dot, name, then enabled /
disabled in small muted text. The max-error stat is gone. Long names truncate
with an ellipsis, and the status stays visible.
---
## [0.7.3] — 2026-10-27
### Changed
- **Containers moved into Settings.** The top-nav Containers tab is gone; the
settings rail now has an "All containers" entry (the tile grid at `/containers`)
plus one entry per container (`/containers/<name>`). Both views render inside
the same rail layout, and the Settings tab stays highlighted while you browse them.
- **Per-channel priority reference.** The shared priority block is replaced by one
reference per platform: Gotify (0 silent · 1–3 low · 4–7 normal · 8–10 high) and
Pushover (-2 Lowest · -1 Low · 0 Normal · 1 High · 2 Emergency), with the
Inspector → Pushover mapping noted under the Pushover one.
- **restart_on redesign.** The pill row is replaced by a grid of toggleable cards,
each with a color accent and a plain-language description of the status, plus
"Recommended" (UNHEALTHY + NOT RUNNING) and "Observe only" quick buttons.
### Fixed
- Unchecking every `restart_on` status now saves `restart_on: []` (observe only).
Previously an all-unchecked submit silently kept the old list.
---
## [0.7.2] — 2026-10-27 ## [0.7.2] — 2026-10-27
### Added ### Added
+9 -4
View File
@@ -19,9 +19,14 @@ ARG INSPECTOR_UID=1000
ARG INSPECTOR_GID=1000 ARG INSPECTOR_GID=1000
RUN groupadd --gid ${INSPECTOR_GID} inspector \ RUN groupadd --gid ${INSPECTOR_GID} inspector \
&& useradd --uid ${INSPECTOR_UID} --gid inspector --home-dir /app --shell /usr/sbin/nologin inspector \ && useradd --uid ${INSPECTOR_UID} --gid inspector --home-dir /app --shell /usr/sbin/nologin inspector \
&& mkdir -p /app/logs /app/data \ && mkdir -p /app/config /app/logs /app/data \
&& chown -R inspector:inspector /app/logs /app/data && chown -R inspector:inspector /app/config /app/logs /app/data
USER inspector COPY docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh
RUN chmod 755 /usr/local/bin/docker-entrypoint.sh
CMD ["python", "-m", "inspector.main", "--config", "/app/config.yml"] # Starts as root only to detect the Docker socket's group and fix mounted
# folder ownership, then drops to PUID:PGID (default 1000:1000).
# See docker-entrypoint.sh.
ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"]
CMD ["python", "-m", "inspector.main"]
+144 -50
View File
@@ -1,70 +1,164 @@
# Install # Installing Inspector with Portainer
Inspector watches Docker containers, restarts the ones you tell it to, This guide uses only the Portainer web UI. You don't need SSH, a terminal, or a
and notifies you via Gotify and/or Pushover. All configuration lives in config file: Inspector creates its own settings on first start and everything
a single `config.yml` that you can edit by hand or from the built-in web else is done in its dashboard.
settings page.
## Requirements **You need:** a Docker host managed by Portainer (CE or BE, 2.x), and a few
minutes. Gotify or Pushover are optional, for phone alerts.
- Docker Engine 24+ and `docker compose` v2. ---
- A machine that can reach `/var/run/docker.sock` (any Docker host).
- Optional: a Gotify server and/or a Pushover account for notifications.
## Quickstart ## 1. Create the stack
```bash 1. In Portainer, open your environment (usually **local**) and click
# 1. Grab the compose file and example config **Stacks → + Add stack**.
mkdir -p ~/docker/inspector && cd ~/docker/inspector 2. **Name:** `inspector`
curl -L -o compose.yml https://gitea.euronvault.com/euronvault/inspector/raw/branch/main/compose.yml 3. **Build method:** *Web editor*. Paste the contents of
curl -L -o config.yml https://gitea.euronvault.com/euronvault/inspector/raw/branch/main/config.example.yml [`compose.yml`](./compose.yml):
# 2. Edit config.yml — at minimum, list the containers you want watched ```yaml
# under the `containers:` key. services:
$EDITOR config.yml inspector:
image: gitea.euronvault.com/euronvault/inspector:${INSPECTOR_TAG:-latest}
container_name: inspector
restart: unless-stopped
ports:
- "${INSPECTOR_PORT:-8080}:8080"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ${INSPECTOR_DATA:-/opt/inspector}/config:/app/config
- ${INSPECTOR_DATA:-/opt/inspector}/logs:/app/logs
- ${INSPECTOR_DATA:-/opt/inspector}/data:/app/data
environment:
- TZ=${TZ:-UTC}
- INSPECTOR_PASSWORD=${INSPECTOR_PASSWORD:-}
- PUID=${PUID:-1000}
- PGID=${PGID:-1000}
```
# 3. Set your Docker group GID so the container can read the socket. ## 2. Set the environment variables
echo "DOCKER_GID=$(getent group docker | cut -d: -f3)" > .env
# 4. Pull and run Scroll down to **Environment variables**. Click **Advanced mode** and paste,
docker compose pull then edit the values:
docker compose up -d
docker compose logs -f inspector ```
INSPECTOR_DATA=/mnt/user/appdata/inspector
INSPECTOR_PASSWORD=pick-a-password
TZ=America/Los_Angeles
``` ```
If `dashboard.enabled: true` in your config, visit | Variable | What to put | Required |
`http://<host>:8080/` after startup to see the dashboard, and |---|---|---|
`http://<host>:8080/settings` to edit the config from the browser. | `INSPECTOR_DATA` | Folder on your server where Inspector keeps its settings, logs, and history. **Unraid:** `/mnt/user/appdata/inspector`. **Other Linux:** `/opt/inspector`. Created automatically. | Yes |
| `INSPECTOR_PASSWORD` | Password for the dashboard (username `admin`). Used only on first start; change it later in the dashboard. Leave it out and Inspector generates one and prints it in the container logs. | Recommended |
| `TZ` | **Your time zone**, e.g. `America/New_York` or `Europe/Berlin`. Without it Inspector uses UTC, so the maintenance window, alerts and dashboard times are off by your UTC offset. See [Why the time zone matters](#why-the-time-zone-matters). | **Yes** (default `UTC`) |
| `INSPECTOR_PORT` | Port on your server for the dashboard. Change it if `8080` is taken. | No (default `8080`) |
| `INSPECTOR_TAG` | Version to run, e.g. `0.8.0`. See [Updating](#4-updating). | No (default `latest`) |
| `PUID` / `PGID` | User and group that own Inspector's files. Unraid users often use `99` / `100` so the files match other appdata. | No (default `1000` / `1000`) |
| `PORTAINER_API_TOKEN` | A Portainer access token. Only needed if you use **Portainer stack redeploy** for VPN-routed apps; see the in-app guide (Help → Portainer). | No |
## Pinning a version ### Why the time zone matters
Prefer an immutable tag over `latest` in production: Inspector uses `TZ` for every time it shows or acts on. If it's wrong, things
still work, but at the wrong hour or with confusing times:
```bash - **The maintenance window runs in this time zone.** A window of `02:00–05:00`
# .env with `TZ` left at UTC runs at 02:00 UTC, which is the middle of the evening in
INSPECTOR_TAG=0.7.0 the Americas or mid-morning in Asia. Inspector would then restart containers
``` during your nightly backups, and ignore real problems during the day.
- **Alerts** (Gotify / Pushover) show the time of the problem in this zone, with
its abbreviation, e.g. `10-28 03:15 AM CET`.
- **The dashboard and Events page** show times in this zone, and the Events
page's days run midnight to midnight in this zone.
- **Logs** are stamped in this zone.
Images are published at `gitea.euronvault.com/euronvault/inspector`. **How to set it:** use your zone's name from the
[list of time zones](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones)
("TZ identifier" column), e.g. `America/New_York`, `America/Chicago`,
`Europe/London`, `Europe/Berlin`, `Australia/Sydney`. Abbreviations like `EST` or
`CET` don't work. On Unraid, **Settings → Date and Time** shows the zone your
server uses.
## Upgrading **How to check it:** the bottom of every Settings page shows
`time zone: <name>`. If it says `UTC` and you're not on UTC, set `TZ` in the
stack and click **Update the stack**. If the name you set isn't recognized,
Inspector logs a warning and uses UTC.
```bash ## 3. Deploy and sign in
docker compose pull
docker compose up -d
```
Your `config.yml` is persisted outside the container, so upgrades don't 1. Click **Deploy the stack**.
touch it. The settings page keeps a `.bak` of the previous file next to 2. Open `http://<your-server-ip>:8080` (or the port you chose).
`config.yml` on every save (when the directory is bind-mounted; see the 3. Sign in as `admin` with your `INSPECTOR_PASSWORD`.
README for the Docker caveat). - No password set? Go to **Containers → inspector → Logs** in Portainer. Near
the top is a box labeled *Inspector dashboard login* with the generated
password.
4. Click **Help** in the dashboard header for the full how-to guide. The short
version:
- **Settings → + Add container**: add each container you want watched, using
its exact name from Portainer, and click **Recommended** for restart rules.
- **Settings → Notifications**: turn on Gotify and/or Pushover, click
**Send test**, then **Save**.
- Restart Inspector once after changing anything marked
**requires restart** (Portainer → Containers → tick `inspector` → Restart).
## Uninstall That's it. Inspector picks up new containers on its next check (within a minute).
```bash ---
docker compose down
rm -rf logs data config.yml compose.yml .env
```
Inspector doesn't store anything outside the directory it was started ## 4. Updating
from.
1. **Stacks → inspector → Editor**.
2. Click **Update the stack** and turn on **Re-pull image and redeploy**.
3. Confirm.
Your settings and history live in `INSPECTOR_DATA`, so updates never touch them.
**Pinning a version.** `latest` always gets the newest release. To stay on a
specific version, set `INSPECTOR_TAG` (e.g. `0.8.0`) in the stack's environment
variables, then update the stack. Available versions are on the
[releases page](https://gitea.euronvault.com/euronvault/inspector/releases)
and in the [CHANGELOG](./CHANGELOG.md).
**Rolling back.** Set `INSPECTOR_TAG` to the previous version and update the
stack with **Re-pull image and redeploy** on.
## 5. Uninstalling
1. **Stacks → inspector → Delete this stack**.
2. Delete the `INSPECTOR_DATA` folder (on Unraid, through the appdata share) if
you don't want to keep the settings and history.
Inspector never changes your other containers' files or settings. It only
starts and restarts the ones you add to it.
---
## Troubleshooting
| What you see | What to do |
|---|---|
| Stack fails with `pull access denied` or `not found` | Check the image line is exactly `gitea.euronvault.com/euronvault/inspector:...` and that `INSPECTOR_TAG` (if set) is a real version. |
| Port already in use | Set `INSPECTOR_PORT` to a free port (e.g. `8095`) and update the stack. |
| Logs say *Permission denied on /var/run/docker.sock* | Make sure the stack has no `user:` line, then update the stack. Inspector needs to start normally to set up socket access. |
| Red **READ-ONLY** badge at the bottom of Settings | The config folder mount ends in `:ro` or the folder isn't writable. Remove `:ro` and update the stack. |
| Can't sign in | Look for the generated password in the container logs. If you've lost it, open `INSPECTOR_DATA/config/config.yml` (e.g. via your Unraid share), set a new `password:` under `dashboard:`, and restart Inspector. |
| Times are a few hours off, or the maintenance window runs at the wrong time | `TZ` is missing or wrong. Check `time zone:` at the bottom of Settings, set `TZ` to your zone name (e.g. `Europe/London`), and update the stack. |
| A container shows **CONTAINER NOT FOUND** | The name in Inspector doesn't match Docker. Copy the exact name from Portainer → Containers. |
## Security notes
- Inspector needs the Docker socket to restart containers. Anything with access
to that socket effectively has full control of the host, so keep the
dashboard on your home network (or behind a VPN / reverse proxy with auth)
and use a real password.
- Inspector's process runs as an unprivileged user. It starts as root only long
enough to detect the Docker socket's group and fix ownership of its own
folders.
## Moving from an older (0.7.x) setup
Older setups mounted a single file at `/app/config.yml` and set `group_add` /
`DOCKER_GID`. That still works unchanged. To switch to the simpler layout:
move your `config.yml` into `<INSPECTOR_DATA>/config/`, replace the stack with
the compose above, and update the stack.
+82 -112
View File
@@ -1,139 +1,109 @@
# Inspector <p align="center">
<img src="docs/logo_outline.png" alt="Inspector logo" width="140">
</p>
Inspector is a small, opinionated Docker-watchdog. It polls the Docker <h1 align="center">Inspector</h1>
daemon, applies per-container restart rules, pings Gotify and/or
Pushover when things go wrong, and gives you a dashboard that shows
what's happening right now and what happened earlier today.
Built for home-lab and small-fleet operators who want "it just works" <p align="center"><b>Monitor. Restart. Report.</b><br>
container recovery without standing up a full observability stack. A small Docker watchdog for home servers: it watches the containers you pick,
restarts them when they break, and tells your phone about it.</p>
## What Inspector does <p align="center">
<a href="./INSTALL.md">Install with Portainer</a> ·
<a href="./src/inspector/assets/guide/guide.html">How-to guide</a> ·
<a href="./CHANGELOG.md">Changelog</a>
</p>
- Polls each configured container on a fixed interval and classifies it: ![Inspector status page](src/inspector/assets/guide/status.png)
`HEALTHY`, `UNHEALTHY`, `STARTING`, `NOT RUNNING`, `ERROR`,
`UNREACHABLE`, `CONTAINER NOT FOUND`, `NO HEALTH CHECK`, `UNKNOWN`.
- Restarts a container when it crosses a configured error threshold —
with cooldowns, grace periods, and an hourly restart cap so a flapping
container doesn't DDoS your host.
- Opt-in HTTP reachability checks per container (useful for VPN-routed
containers whose tunnel drops while Docker still reports "healthy").
- Opt-in Uptime Kuma integration as a second reachability source.
- Advanced recovery via Portainer stack redeploy for containers whose
network namespace is tied to a VPN sidecar.
- Sends notifications on transitions, not on every poll — one line when
something breaks, one line when it recovers.
- Optional SQLite event history + a daily/weekly digest.
- Scheduled maintenance windows (don't act during 2–6 AM).
- Manual notification mute per container, from the dashboard.
- Embedded web dashboard + events timeline + **web settings page** for
editing `config.yml` without SSH.
## What Inspector is not ## What it does
- Not a metrics system. It doesn't scrape Prometheus or push - **Watches the containers you choose.** Each check sorts a container into
dashboards; use Grafana / Netdata / your real monitoring stack for healthy, unhealthy, not running, starting, and so on.
that. - **Restarts them when they break**, with sensible brakes: a container has to fail
- Not an orchestrator. It only acts on containers you've explicitly several checks in a row first, there's a cooldown between restarts, a grace
listed in `config.yml`. period while it boots, and an hourly cap so a crash-looping app can't take over
- Not multi-host. One Inspector per Docker socket. your server.
- **Alerts you** through [Gotify](https://gotify.net) and/or
[Pushover](https://pushover.net): once when something breaks, once when it's
restarted, once when it recovers. Not on every check.
- **Is set up entirely in the browser.** Add, edit, and remove containers, set up
notifications (with a **Send test** button), and change every setting from the
dashboard. No YAML editing needed.
- **Keeps a history.** The Events page shows a day-by-day timeline of failures,
restarts, and recoveries, and an optional digest summarizes it for you.
- **Stays out of the way when you're working.** Put a container in maintenance or
mute its alerts with one click, or set a nightly maintenance window for backups
and updates.
- **Works in your time zone.** Set `TZ` in the stack and the maintenance window,
alerts, dashboard and logs all follow it.
## Install ## Install
See [`INSTALL.md`](./INSTALL.md). TL;DR: Inspector is built to be installed from **Portainer**: paste one compose file into
**Stacks → Add stack**, set a folder and a password, deploy. Inspector creates its
own config and sets up Docker access by itself.
```bash **→ [Step-by-step Portainer install](./INSTALL.md)**
mkdir -p ~/docker/inspector && cd ~/docker/inspector
curl -L -o compose.yml https://gitea.euronvault.com/euronvault/inspector/raw/branch/main/compose.yml
curl -L -o config.yml https://gitea.euronvault.com/euronvault/inspector/raw/branch/main/config.example.yml
$EDITOR config.yml
echo "DOCKER_GID=$(getent group docker | cut -d: -f3)" > .env
docker compose up -d
```
## Configuration ## Using it
Everything lives in `config.yml` next to the compose file. Start from Once it's running, click **Help** in the dashboard header for the illustrated
[`config.example.yml`](./config.example.yml) — every key is commented. guide (the same guide is in this repo at
Highlights: [`src/inspector/assets/guide/guide.html`](./src/inspector/assets/guide/guide.html);
download it and open it in a browser to read it here).
- `inspector.interval_seconds` — poll cadence. | Add a container | Restart rules |
- `inspector.dry_run` — classify + log, never restart. | ------------------------------------------------------------------- | -------------------------------------------------------------------- |
- `notifications.gotify` / `notifications.pushover` — enable either, | ![New container form](src/inspector/assets/guide/container-new.png) | ![Container settings](src/inspector/assets/guide/container-edit.png) |
both, or neither.
- `containers.<name>.max_error_count` / `cooldown_seconds` /
`grace_period_seconds` / `max_restarts_per_hour` — the restart knobs.
- `containers.<name>.restart_on` — which statuses trigger a restart
(leave empty for observe-only).
- `containers.<name>.http_check` — opt-in HTTP ping.
- `containers.<name>.recovery_action` — `restart_container` (default)
or the advanced Portainer-based options.
### Dashboard and settings page | Notifications | Events history |
| ------------------------------------------------------------------------------- | --------------------------------------------------------- |
| ![Notification settings](src/inspector/assets/guide/settings-notifications.png) | ![Events timeline](src/inspector/assets/guide/events.png) |
If `dashboard.enabled: true`, Inspector exposes a small web UI on ## What it isn't
`:8080`:
- `/` — container cards with current status, error bar, phase - **Not a monitoring/metrics system.** No graphs of CPU or memory; use Grafana,
(cooldown / grace / failing / observe only / maintenance), and Netdata, or Uptime Kuma for that.
maintenance / mute toggles. - **Not an orchestrator.** It only touches the containers you add, and only to
- `/events` — per-day timeline of status changes, restarts, recoveries, restart them.
and failures (requires `database.enabled: true`). - **One Docker host per Inspector.** Run one copy on each server.
- `/settings` — forms for every top-level config section plus one card
per container. Saves preserve comments and key order in `config.yml`
and keep a `.bak`. Fields that take effect on the next loop iteration
(per-container thresholds, `dry_run`, `heartbeat`, `interval_seconds`)
hot-apply; fields bound at startup are labeled **requires restart**.
Note on `.bak` inside Docker: when `config.yml` is bind-mounted as a ## Advanced options
single file (the compose default), the `.bak` lives in the container's
overlay fs — the live file still updates cleanly, but the backup is
lost when the container is removed. To keep the `.bak` on the host,
bind-mount the directory instead of the single file.
### Notifications Most people never need these. They are all on each container's page in the
dashboard (and in `config.yml`, see
[`config.example.yml`](./config.example.yml) for every option with comments).
Both channels are independent. If both are enabled, every alert goes to - **HTTP checks**: mark a container `UNREACHABLE` when a URL stops answering,
both. Inspector sends on transitions: even if Docker says it's healthy (useful for apps behind a VPN container).
- **Uptime Kuma**: use a Kuma status page as an extra reachability signal.
**Test connection** lists the monitors Inspector can see.
- **Dependencies** (`depends_on`): don't restart an app while the container it
relies on (e.g. a VPN) is down.
- **Portainer stack redeploy**: for containers that share a VPN container's
network, recover by redeploying the whole stack instead of restarting one
container. Set up the Portainer connection under **Settings → Portainer**;
**Test connection** lists your stacks.
``` ## Security
🚨 first failure threshold crossed
🔁 container restart triggered
✅ restart / start completed
✅ container recovered
🛑 restart failed / hourly limit exceeded
❓ configured container not found
```
Gotify messages are Markdown; Pushover doesn't render Markdown so the Inspector needs the Docker socket (`/var/run/docker.sock`) to restart containers,
`**bold**` passthrough is cosmetic — bodies are identical. and that socket is effectively root on your server. So:
Pushover priorities map from Inspector's Gotify-style scale (default 5) - Keep the dashboard on your home network, or put it behind a VPN or a reverse
to Pushover's `-2..2` range: `<=2 → -1`, `3..4 → 0`, `5..6 → 1`, proxy with authentication. Don't expose port 8080 to the internet.
`>=7 → 2`. Priority 2 (emergency) ships default `retry` / `expire` and - Use a real dashboard password. Repeated failed sign-ins from one address are
your Pushover client prompts for acknowledgement. blocked for a few minutes.
- Inspector itself runs as an unprivileged user; it starts as root only to detect
the socket's permissions and fix ownership of its own folders.
## Docker socket security ## Versions
Inspector talks to `/var/run/docker.sock`. That socket is effectively Releases follow `MAJOR.MINOR.PATCH`. Every release is published as an image tag;
root on your host: anything that can write to it can start, stop, `latest` is the newest. See the [CHANGELOG](./CHANGELOG.md) for what changed, and
inspect, and (through bind mounts) read any file as root. Inspector [INSTALL.md](./INSTALL.md#4-updating) for updating, pinning, and rolling back.
mounts it read-write because it needs to restart containers.
Follow the usual home-lab precautions:
- Don't publish the dashboard port on an untrusted network. Use a
reverse proxy with auth, a tailnet, or SSH tunneling.
- Set a strong `dashboard.password`.
- Store Portainer / Gotify / Pushover tokens in the config file on a
host-only path with permissions `600`.
## Versioning
Semantic versioning (`MAJOR.MINOR.PATCH`). Image tags are published for
every release; `latest` tracks the newest tag. Pin a specific version
in production — see `INSTALL.md`.
## License ## License
[MIT](./LICENSE). [MIT](./LICENSE) © 2026 Europium 63
+31 -17
View File
@@ -1,29 +1,43 @@
# Inspector compose file for buddies / self-hosters. # Inspector — Portainer stack
# Pulls the published image from gitea.euronvault.com.
# #
# Before `docker compose up -d`: # Paste this into Portainer: Stacks -> Add stack -> Web editor.
# 1. Copy config.example.yml -> config.yml and edit it. # Then set these under "Environment variables" (Advanced mode lets you paste them):
# 2. Set DOCKER_GID to your host's docker group GID: #
# getent group docker | cut -d: -f3 # INSPECTOR_DATA Folder on your server for Inspector's files.
# Export it in your shell or put it in a .env file next to this file. # Unraid: /mnt/user/appdata/inspector Other: /opt/inspector
# 3. Optionally set INSPECTOR_TAG to pin a version # INSPECTOR_PASSWORD Dashboard password (username is "admin").
# (default: latest). Prefer a pinned X.Y.Z for production. # Only used on first start; change it later in Settings.
# TZ Your time zone name, e.g. America/New_York, Europe/Berlin.
# Important: the maintenance window, alerts, dashboard and
# logs all use it. Left unset, everything is in UTC.
#
# Optional:
# INSPECTOR_PORT Port for the dashboard on your server (default 8080).
# INSPECTOR_TAG Image version to run (default "latest"; e.g. 0.8.0 to pin).
# PUID / PGID User/group Inspector runs as (default 1000/1000).
# Unraid users often prefer 99 / 100.
# PORTAINER_API_TOKEN Portainer access token, only if you use stack redeploy
# (Settings -> Portainer in the dashboard).
#
# No config file to create, no docker group ID to look up: Inspector creates
# its config on first start and detects Docker socket permissions by itself.
services: services:
inspector: inspector:
image: gitea.euronvault.com/euronvault/inspector:${INSPECTOR_TAG:-latest} image: gitea.euronvault.com/euronvault/inspector:${INSPECTOR_TAG:-latest}
container_name: inspector container_name: inspector
restart: unless-stopped restart: unless-stopped
# GID of the docker group on the host so the non-root user inside the
# container can read /var/run/docker.sock. Required.
group_add:
- "${DOCKER_GID:-999}"
ports: ports:
# Only published if dashboard.enabled is true in config.yml.
- "${INSPECTOR_PORT:-8080}:8080" - "${INSPECTOR_PORT:-8080}:8080"
volumes: volumes:
# Lets Inspector see and restart your containers. Required.
- /var/run/docker.sock:/var/run/docker.sock - /var/run/docker.sock:/var/run/docker.sock
- ./config.yml:/app/config.yml # Settings (config.yml), logs, and event history. Kept across updates.
- ./logs:/app/logs - ${INSPECTOR_DATA:-/opt/inspector}/config:/app/config
- ./data:/app/data - ${INSPECTOR_DATA:-/opt/inspector}/logs:/app/logs
- ${INSPECTOR_DATA:-/opt/inspector}/data:/app/data
environment: environment:
- TZ=${TZ:-UTC} - TZ=${TZ:-UTC}
- INSPECTOR_PASSWORD=${INSPECTOR_PASSWORD:-}
- PUID=${PUID:-1000}
- PGID=${PGID:-1000}
- PORTAINER_API_TOKEN=${PORTAINER_API_TOKEN:-}
+6 -2
View File
@@ -1,5 +1,9 @@
# Inspector example configuration. # Inspector example configuration: every option, with comments.
# Copy to config.yml, edit, and mount into the container at /app/config.yml. #
# You normally don't need this file. Inspector writes its own config.yml to
# INSPECTOR_DATA/config/ on first start, and the dashboard edits it for you.
# Use this as a reference for the advanced options that are YAML-only
# (http_check, uptime_kuma, depends_on, Portainer stack redeploy).
inspector: inspector:
interval_seconds: 60 interval_seconds: 60
+38
View File
@@ -0,0 +1,38 @@
#!/bin/sh
# Inspector container entrypoint.
#
# Started as root (the default): find the group that owns the Docker socket,
# make the mounted folders writable, then drop to an unprivileged user with
# that group added. No DOCKER_GID / group_add needed.
#
# Started as non-root (compose `user:` / older group_add setups): run as-is.
set -e
if [ "$(id -u)" != "0" ]; then
exec "$@"
fi
PUID="${PUID:-1000}"
PGID="${PGID:-1000}"
SOCK=/var/run/docker.sock
groups="$PGID"
if [ -S "$SOCK" ]; then
sock_gid="$(stat -c %g "$SOCK")"
groups="$groups,$sock_gid"
else
echo "warning: $SOCK is not mounted; Inspector cannot see any containers" >&2
fi
# Mounted folders are often created by Docker as root. Hand them (not their
# contents) to the runtime user so Inspector can write config, logs and data.
for d in /app/config /app/logs /app/data; do
if [ -d "$d" ] && [ "$(stat -c %u "$d")" != "$PUID" ]; then
chown "$PUID:$PGID" "$d" 2>/dev/null || echo "warning: could not chown $d" >&2
fi
done
if [ -f /app/config/config.yml ] && [ "$(stat -c %u /app/config/config.yml)" != "$PUID" ]; then
chown "$PUID:$PGID" /app/config/config.yml 2>/dev/null || true
fi
exec setpriv --reuid="$PUID" --regid="$PGID" --groups="$groups" "$@"
Executable
BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 17 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 25 KiB

+1 -1
View File
@@ -1,5 +1,5 @@
docker==7.1.0 docker==7.1.0
PyYAML==6.0.2 PyYAML==6.0.2
requests==2.32.3 requests==2.32.5
colorlog==6.9.0 colorlog==6.9.0
ruamel.yaml==0.18.6 ruamel.yaml==0.18.6
+1 -1
View File
@@ -1 +1 @@
__version__ = "0.7.2" __version__ = "0.9.3"
Binary file not shown.

After

Width:  |  Height:  |  Size: 49 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 70 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 73 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 45 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 27 KiB

+461
View File
@@ -0,0 +1,461 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Inspector — How to use</title>
<style>
:root {
--bg: #0b1220; --card: #111c30; --card-2: #0f1829; --border: #1e293b;
--ink: #e2e8f0; --ink-2: #cbd5e1; --muted: #8494a9;
--blue: #60a5fa; --green: #34d399; --amber: #fbbf24; --red: #f87171; --violet: #a78bfa;
}
* { box-sizing: border-box; }
html, body { margin: 0; background: var(--bg); color: var(--ink);
font-family: Inter, -apple-system, "Segoe UI", Roboto, "Helvetica Neue", sans-serif;
font-size: 16px; line-height: 1.6; }
a { color: var(--blue); }
.top { position: sticky; top: 0; z-index: 5; background: #060b16; border-bottom: 1px solid var(--border); }
.top .in { max-width: 1100px; margin: 0 auto; padding: 0.8rem 1rem; display: flex; align-items: center; gap: 1rem; }
.top .brand { font-weight: 800; font-size: 1.05rem; }
.top .brand span { color: var(--muted); font-weight: 400; margin-left: 0.4rem; }
.top a.back { margin-left: auto; font-size: 0.9rem; text-decoration: none; border: 1px solid var(--border);
padding: 0.35rem 0.8rem; border-radius: 8px; color: var(--ink-2); }
.wrap { max-width: 1100px; margin: 0 auto; padding: 2rem 1rem 4rem; display: grid;
grid-template-columns: 220px minmax(0, 1fr); gap: 2.5rem; }
nav.toc { position: sticky; top: 4.5rem; align-self: start; font-size: 0.88rem; }
nav.toc b { display: block; color: var(--muted); text-transform: uppercase; font-size: 0.7rem;
letter-spacing: 0.08em; margin-bottom: 0.5rem; }
nav.toc a { display: block; color: var(--ink-2); text-decoration: none; padding: 0.25rem 0; }
nav.toc a:hover { color: var(--blue); }
main { min-width: 0; }
h1 { font-size: 1.9rem; margin: 0 0 0.4rem; line-height: 1.2; }
.lede { color: var(--ink-2); font-size: 1.05rem; margin: 0 0 2rem; }
section { margin-bottom: 3rem; scroll-margin-top: 4.5rem; }
h2 { font-size: 1.35rem; margin: 0 0 0.6rem; padding-top: 0.5rem; border-top: 1px solid var(--border); }
h3 { font-size: 1.02rem; margin: 1.4rem 0 0.4rem; }
p, li { color: var(--ink-2); }
code { background: var(--card-2); border: 1px solid var(--border); padding: 0.05em 0.35em;
border-radius: 5px; font-size: 0.88em; color: var(--ink); }
figure { margin: 1rem 0 1.4rem; }
figure img { width: 100%; height: auto; display: block; border-radius: 12px; border: 1px solid var(--border); }
figcaption { color: var(--muted); font-size: 0.82rem; margin-top: 0.4rem; }
.steps { counter-reset: s; list-style: none; padding: 0; }
.steps li { counter-increment: s; position: relative; padding-left: 2.3rem; margin-bottom: 0.7rem; }
.steps li::before { content: counter(s); position: absolute; left: 0; top: 0.05rem; width: 1.6rem; height: 1.6rem;
border-radius: 50%; background: rgba(96,165,250,0.15); color: var(--blue); font-weight: 700;
font-size: 0.85rem; display: flex; align-items: center; justify-content: center; }
.note { background: var(--card); border: 1px solid var(--border); border-left: 3px solid var(--blue);
border-radius: 10px; padding: 0.75rem 1rem; margin: 1rem 0; color: var(--ink-2); }
.note.warn { border-left-color: var(--amber); }
table { width: 100%; border-collapse: collapse; margin: 0.8rem 0 1.2rem; font-size: 0.92rem; }
th, td { text-align: left; padding: 0.5rem 0.7rem; border-bottom: 1px solid var(--border); vertical-align: top; }
th { color: var(--muted); font-weight: 600; font-size: 0.78rem; text-transform: uppercase; letter-spacing: 0.04em; }
td:first-child { white-space: nowrap; }
.tag { display: inline-block; font-size: 0.72rem; font-weight: 700; padding: 0.05rem 0.45rem; border-radius: 999px; }
.t-green { background: rgba(52,211,153,0.15); color: var(--green); }
.t-red { background: rgba(248,113,113,0.15); color: var(--red); }
.t-amber { background: rgba(251,191,36,0.15); color: var(--amber); }
.t-blue { background: rgba(96,165,250,0.15); color: var(--blue); }
.t-violet { background: rgba(167,139,250,0.15); color: var(--violet); }
@media (max-width: 820px) {
.wrap { grid-template-columns: 1fr; gap: 1rem; padding-top: 1.2rem; }
nav.toc { position: static; }
}
</style>
</head>
<body>
<div class="top"><div class="in">
<div class="brand">Inspector<span>How to use</span></div>
<a class="back" href="/">Open dashboard</a>
</div></div>
<div class="wrap">
<nav class="toc">
<b>On this page</b>
<a href="#first-login">1. First login</a>
<a href="#add">2. Add your containers</a>
<a href="#status">3. The Status page</a>
<a href="#notifications">4. Notifications</a>
<a href="#containers">5. Managing containers</a>
<a href="#vpn">6. Apps behind a VPN</a>
<a href="#portainer">7. Portainer stack redeploy</a>
<a href="#events">8. Events history</a>
<a href="#settings">9. Other settings</a>
<a href="#restart">10. "Requires restart" fields</a>
<a href="#trouble">11. Troubleshooting</a>
</nav>
<main>
<h1>Using Inspector</h1>
<p class="lede">Inspector watches the Docker containers you pick, restarts them when they break,
and pings your phone when something goes wrong or recovers. Everything is set from this web
dashboard; you never need to touch a config file.</p>
<section id="first-login">
<h2>1. First login</h2>
<p>When Inspector starts for the first time it creates its config file and a dashboard login.</p>
<ol class="steps">
<li>If you set <code>INSPECTOR_PASSWORD</code> in the Portainer stack, sign in with
<code>admin</code> (or your <code>INSPECTOR_USERNAME</code>) and that password.</li>
<li>If you didn't, Inspector generated a password. In Portainer open
<b>Containers → inspector → Logs</b>. Near the top you'll see a box labeled
<i>Inspector dashboard login</i> with the username and password.</li>
<li>Open <code>http://&lt;your-server&gt;:8080</code> and sign in.</li>
<li>Change the password under <b>Settings → Dashboard</b> if you like.</li>
<li>Check the <b>time zone</b>: at the bottom of any Settings page it says
<code>time zone: …</code>. If that's not your zone, see <a href="#settings">Time zone</a>.</li>
</ol>
</section>
<section id="add">
<h2>2. Add your containers</h2>
<p>Inspector only watches containers you add. Go to <b>Settings</b> and click
<b>+ Add container</b> at the bottom of the left menu (or <b>+ New container</b> on the
All containers page).</p>
<figure><img src="container-new.png" alt="New container form">
<figcaption>The New container form.</figcaption></figure>
<ol class="steps">
<li><b>name</b>: the container's name exactly as Portainer shows it
(for example <code>sonarr</code>). If it doesn't match, the card shows
<span class="tag t-amber">CONTAINER NOT FOUND</span>.</li>
<li>Leave <b>enabled</b> on.</li>
<li>Under <b>Restart when status is…</b> click <b>Recommended</b>
(restarts on <code>UNHEALTHY</code> and <code>NOT RUNNING</code>), or
<b>Observe only</b> if you just want alerts without restarts.</li>
<li>Click <b>Create container</b>. Inspector starts watching it on its next check
(within a minute by default). No restart needed.</li>
</ol>
<h3>What the numbers mean</h3>
<table>
<tr><th>Field</th><th>Meaning</th><th>Default</th></tr>
<tr><td>max_error_count</td><td>How many bad checks in a row before Inspector restarts the container. Stops one blip from causing a restart.</td><td>5</td></tr>
<tr><td>cooldown_seconds</td><td>Minimum time between two restarts of the same container.</td><td>300</td></tr>
<tr><td>grace_period_seconds</td><td>After a restart, ignore failures for this long so the app can boot.</td><td>120</td></tr>
<tr><td>max_restarts_per_hour</td><td>Safety cap. After this many restarts in an hour Inspector stops restarting and alerts you instead.</td><td>3</td></tr>
<tr><td>maintenance_window</td><td>Follow the daily maintenance window from Settings → Maintenance.</td><td>off</td></tr>
<tr><td>color</td><td>Card color on the dashboard. Pick a swatch or leave blank for automatic.</td><td>auto</td></tr>
</table>
<div class="note">With the default 60-second check interval and <code>max_error_count</code> 5, a
container has to be broken for about five minutes before Inspector restarts it. Lower the count
for faster recovery.</div>
</section>
<section id="status">
<h2>3. The Status page</h2>
<figure><img src="status.png" alt="Status page">
<figcaption>Status: one card per watched container.</figcaption></figure>
<p>The top row counts containers, healthy ones, ones with issues, and ones in maintenance.
Each card shows:</p>
<ul>
<li><b>Status badge</b>: what Docker reports right now, e.g.
<span class="tag t-green">HEALTHY</span> <span class="tag t-red">NOT RUNNING</span>
<span class="tag t-violet">NO HEALTH CHECK</span> (the image has no health check; that's normal
for many apps and only means Inspector can tell running from stopped, not healthy from unhealthy).</li>
<li><b>Errors bar</b>: bad checks in a row out of <code>max_error_count</code>. When it fills,
Inspector restarts the container.</li>
<li><b>Cooldown until / Last restart</b>: when Inspector last restarted it and when it's allowed to
again.</li>
<li><b>Observe only</b>: shown when the container has no restart triggers; Inspector alerts but
never restarts it.</li>
<li><b>Enter maintenance</b>: pause Inspector for that container while you work on it
(updates, migrations). Click again to exit.</li>
<li><b>Mute notifications</b>: keep watching and restarting, but stop alerts for a while
(12 hours by default).</li>
</ul>
<p>The page refreshes itself every 30 seconds.</p>
</section>
<section id="notifications">
<h2>4. Notifications</h2>
<p>Inspector can alert you through <a href="https://gotify.net">Gotify</a>,
<a href="https://pushover.net">Pushover</a>, or both. Set them up under
<b>Settings → Notifications</b>.</p>
<figure><img src="settings-notifications.png" alt="Notifications settings">
<figcaption>Each channel has its own priority reference, a Send test button, and a Save button.</figcaption></figure>
<ol class="steps">
<li>Turn on <b>enabled</b> for the channel you use.</li>
<li><b>Gotify</b>: enter your server URL and an <i>application</i> token (create one in Gotify
under Apps). <b>Pushover</b>: enter your application API token and your user key.</li>
<li>Click <b>Send test</b>. You should get a message on your phone within seconds. If it fails,
the reason appears next to the button.</li>
<li>Click <b>Save</b>, then restart Inspector once (see <a href="#restart">section 10</a>),
because turning a channel on or off is a "requires restart" field.</li>
</ol>
<p>You'll get one alert when a container first fails, when Inspector restarts it, and when it
recovers. You won't get one on every check.</p>
</section>
<section id="containers">
<h2>5. Managing containers</h2>
<p><b>Settings → All containers</b> shows every watched container as a tile. Click a tile (or a
name in the left menu) to edit it.</p>
<figure><img src="containers.png" alt="All containers grid">
<figcaption>All containers.</figcaption></figure>
<figure><img src="container-edit.png" alt="Container settings">
<figcaption>Editing one container. Each restart trigger is a card you click on or off.</figcaption></figure>
<p>Changes apply on the next check. <b>Delete container</b> stops Inspector watching it; it
does <i>not</i> touch the container itself.</p>
<h3>Restart triggers</h3>
<table>
<tr><th>Status</th><th>When it happens</th></tr>
<tr><td>UNHEALTHY</td><td>The container's own health check is failing.</td></tr>
<tr><td>NOT RUNNING</td><td>Stopped, exited, or crashed.</td></tr>
<tr><td>UNREACHABLE</td><td>Running, but its HTTP check or Uptime Kuma monitor says it's down (see below).</td></tr>
<tr><td>ERROR</td><td>Inspector couldn't check it (Docker API error).</td></tr>
<tr><td>STARTING</td><td>Still in its health-check start period. Usually leave this off.</td></tr>
<tr><td>HEALTHY / NO HEALTH CHECK</td><td>Normal states; almost never a reason to restart.</td></tr>
</table>
<h3>Reachability: catching apps that look fine but aren't</h3>
<p>Docker only knows whether a container is running and whether its own health check passes. An
app can pass both and still be unusable, for example when it routes through a VPN container
whose tunnel dropped. Under <b>Reachability</b> on the container page you can add:</p>
<ul>
<li><b>HTTP check</b>: a URL Inspector requests every check (for example
<code>http://192.168.1.10:9696/ping</code>). Set <b>expected_status</b> to the code the app
returns when it's healthy; most return 200, some return 401 or 302.</li>
<li><b>Uptime Kuma monitor</b>: if you already run Uptime Kuma, name one of its monitors. Set up
the connection first in <b>Settings → Uptime Kuma</b>; <b>Test connection</b> lists the
monitor names it can see.</li>
</ul>
<p>A failed check reports <b>UNREACHABLE</b>. Tick UNREACHABLE in the restart triggers if you want
Inspector to act on it.</p>
<figure><img src="container-advanced.png" alt="Reachability, Dependencies and Recovery groups">
<figcaption>Bottom of a container's page: an app behind a VPN container with an HTTP check and stack-redeploy recovery.</figcaption></figure>
</section>
<section id="vpn">
<h2>6. Apps behind a VPN container</h2>
<h3>Why these apps need special handling</h3>
<p>A common setup routes download and indexer apps through a VPN container. Those apps don't get a
network of their own: their compose file says <code>network_mode: container:&lt;vpn&gt;</code> (or
<code>service:&lt;vpn&gt;</code>) and all their traffic goes through the VPN container. That causes
two problems for a watchdog:</p>
<ul>
<li><b>When the VPN breaks, every app behind it breaks at the same time.</b> Restarting the apps
does nothing while the tunnel is down. Without help, Inspector would restart each app over and
over, use up their hourly restart limits, and flood you with alerts, all for a problem in a
different container.</li>
<li><b>When the VPN container is recreated</b> (an image update, or someone redeploys its stack),
it gets a new container ID. The apps still point at the old one, so a normal restart fails with
errors like <i>"No such container"</i> or <i>"joining network namespace"</i>. The only fix is
to recreate the apps too, which is what a Portainer stack redeploy does
(<a href="#portainer">section 7</a>).</li>
</ul>
<h3>How to tell if an app is behind a VPN</h3>
<ul>
<li>In Portainer open <b>Stacks → your stack → Editor</b> and look for
<code>network_mode: container:…</code> or <code>network_mode: service:…</code> on the app.</li>
<li>Or open <b>Containers → the app</b>: under <b>Network</b> it shows the VPN container instead
of a normal network, and it has no ports of its own (they're published on the VPN container).</li>
</ul>
<h3>Setting it up</h3>
<ol class="steps">
<li><b>Add the VPN container to Inspector and keep it enabled</b> (restart on UNHEALTHY and NOT
RUNNING is a good start). Inspector can only wait for containers it is watching: if the VPN
container is disabled, its apps are recovered normally, as if there were no dependency, and the
container page shows a warning.</li>
<li>On each app behind it, set <b>Dependencies → depends_on</b> to the VPN container.</li>
<li>Turn on <b>vpn_routed</b>. This only changes the log: Inspector writes down when it holds off
on the app because the VPN is down. The waiting itself comes from depends_on.</li>
<li>Add an <b>HTTP check</b> (<a href="#containers">section 5</a>) using your server's IP and the
port published on the VPN container, and tick <b>UNREACHABLE</b>. Docker often still reports
these apps as healthy when the tunnel is down; the HTTP check notices.</li>
<li>Under <b>Recovery</b>, choose <b>Restart, then redeploy stack if that fails</b> and enter the
stack name. This needs the Portainer connection from <a href="#portainer">section 7</a>.</li>
</ol>
<h3>What Inspector then does</h3>
<p>Every check, it looks at the VPN container first. While the VPN is anything other than healthy
(or starting, or running without a health check), the apps that depend on it are not restarted;
Inspector fixes the VPN first. Once the VPN is healthy again, the apps are handled normally: a
restart first, and a stack redeploy if the restart fails or doesn't help.</p>
<p class="note"><b>depends_on isn't only for VPNs.</b> Use it for any "B needs A" pair, such as an
app and its database: Inspector won't restart the app while its database is down.</p>
</section>
<section id="portainer">
<h2>7. Portainer stack redeploy</h2>
<h3>Why Inspector needs Portainer</h3>
<p>Inspector talks to Docker directly, so it can restart or start a container. It can't
<i>recreate</i> one: that needs the container's compose definition, which lives in Portainer. A
stack redeploy asks Portainer to do exactly what its <b>Update the stack</b> button does. That's
how VPN-routed apps get reattached to a recreated VPN container. You only need this if you use one
of the two "redeploy" recovery actions.</p>
<p class="note warn"><b>A redeploy affects the whole stack.</b> Portainer pulls the latest images
and recreates <i>every</i> container in that stack, not just the failing app, so all of them blink
off for a moment. That's why <b>Restart, then redeploy stack if that fails</b> is the recommended
choice: most problems are fixed by the plain restart, and the redeploy is only used when that
doesn't work.</p>
<h3>Requirements</h3>
<ul>
<li>The app must belong to a stack that was <b>created in Portainer</b>. In <b>Stacks</b>, stacks
created elsewhere (for example with <code>docker compose</code> on the command line) are marked
<b>Limited</b>, and Portainer can't redeploy them.</li>
<li>An access token for a Portainer user who can manage that stack. An administrator's token is
the simplest.</li>
</ul>
<h3>Step 1: find the values</h3>
<table>
<tr><th>Setting</th><th>Where to find it</th></tr>
<tr><td>base_url</td><td>The address you open Portainer at, without anything after the port:
usually <code>http://&lt;server-ip&gt;:9000</code> or <code>https://&lt;server-ip&gt;:9443</code>.
Use the server's IP, not <code>localhost</code>: Inspector runs in its own container.</td></tr>
<tr><td>endpoint_id</td><td>On Portainer's <b>Home</b> page, click your environment (often
<b>local</b>). The browser address then ends like <code>#!/2/docker/dashboard</code>: the
number after <code>#!/</code> is the endpoint ID (here <code>2</code>).</td></tr>
<tr><td>stack name</td><td><b>Stacks</b> list, <b>Name</b> column, spelled exactly the same.
This goes on each app's container page, not on the Portainer page.</td></tr>
<tr><td>verify_ssl</td><td>Leave off for <code>http://</code> and for Portainer's built-in
self-signed certificate on 9443. Turn on only if Portainer has a real certificate.</td></tr>
<tr><td>API token</td><td>Click your username (top right) → <b>My account</b> →
<b>Access tokens</b> → <b>Add access token</b>. Name it <code>inspector</code> and copy the
token straight away: Portainer shows it only once.</td></tr>
</table>
<h3>Step 2: give Inspector the token</h3>
<p><b>Recommended: an environment variable.</b> The token then never lands in
<code>config.yml</code> or its backups.</p>
<ol class="steps">
<li>In Portainer open <b>Stacks → inspector → Editor</b>.</li>
<li>Check that the compose file has this line under <code>environment:</code> (it's in the
compose file from version 0.9.1 on; add it if your stack is older):
<code>- PORTAINER_API_TOKEN=${PORTAINER_API_TOKEN:-}</code></li>
<li>Under <b>Environment variables</b> click <b>Add an environment variable</b>: name
<code>PORTAINER_API_TOKEN</code>, value the token.</li>
<li>Click <b>Update the stack</b>. Inspector restarts with the token available.</li>
</ol>
<p><b>Alternative: save it in Inspector.</b> In the next step choose <b>Saved in config.yml</b>
and paste the token. It is masked in the dashboard, but anyone who can read Inspector's config
folder (or a backup of it) can read the token.</p>
<h3>Step 3: connect Inspector</h3>
<ol class="steps">
<li>Open <b>Settings → Portainer</b>. Under <b>Add environment</b>, give it a name (your own
label, e.g. <code>home</code>) and fill in base_url, endpoint_id and verify_ssl.</li>
<li><b>API token stored as</b>: <i>Environment variable</i> with the name
<code>PORTAINER_API_TOKEN</code>, or <i>Saved in config.yml</i> with the token.</li>
<li>Click <b>Test connection</b>. It should list the stacks on that endpoint; check that your
app's stack is there. Then <b>Save new environment</b>.</li>
<li>At the top of the page turn <b>enabled</b> on, pick the <b>default_environment</b>, and
<b>Save Portainer</b>.</li>
<li>On each app's container page, set the recovery action and stack name
(<a href="#vpn">section 6</a>). Leave <b>portainer_environment</b> on <i>Default</i> unless
you connected more than one Portainer.</li>
</ol>
<figure><img src="settings-portainer.png" alt="Portainer settings">
<figcaption>Settings → Portainer with one environment; the token comes from an environment variable.</figcaption></figure>
<h3>The other settings on the page</h3>
<table>
<tr><th>Setting</th><th>What it does</th></tr>
<tr><td>default_environment</td><td>Which connection apps use when they don't pick one. With a
single environment you can leave it on Automatic.</td></tr>
<tr><td>stack_redeploy_cooldown_seconds</td><td>Minimum time between two redeploys of the same
stack (default 300). Several apps often share a stack; this stops them triggering a redeploy
each.</td></tr>
<tr><td>post_redeploy_wait_seconds</td><td>How long Inspector waits after a redeploy for the
stack to come up before judging it again (default 90).</td></tr>
</table>
<h3>If Test connection fails</h3>
<table>
<tr><th>Message</th><th>Fix</th></tr>
<tr><td>environment variable … is not set</td><td>The compose line from step 2 is missing, or
the stack wasn't updated after adding the variable.</td></tr>
<tr><td>the API token was rejected</td><td>Token copied wrong, deleted, or from a user without
access. Create a new one.</td></tr>
<tr><td>no stacks found on endpoint N</td><td>Wrong endpoint ID; check the number in the
Portainer address again.</td></tr>
<tr><td>request failed …</td><td>Wrong address or port, <code>http</code> vs <code>https</code>
mixed up, or <code>localhost</code> used instead of the server IP. With https and a
self-signed certificate, turn verify_ssl off.</td></tr>
</table>
</section>
<section id="events">
<h2>8. Events history</h2>
<p>The <b>Events</b> tab is a day-by-day timeline of failures, restarts, recoveries, mutes, and
maintenance. Use <b>Prev day</b> / <b>Next day</b> to look back.</p>
<figure><img src="events.png" alt="Events timeline">
<figcaption>Here a container stopped, Inspector restarted it, and it recovered a few seconds later.</figcaption></figure>
</section>
<section id="settings">
<h2>9. Other settings</h2>
<figure><img src="settings-inspector.png" alt="Inspector loop settings">
<figcaption>Settings → Inspector loop.</figcaption></figure>
<table>
<tr><th>Section</th><th>What it's for</th></tr>
<tr><td>Inspector loop</td><td><b>interval_seconds</b>: how often to check (60 is fine).
<b>dry_run</b>: log and alert but never actually restart; handy while you're trying it out.</td></tr>
<tr><td>Dashboard</td><td>Login username and password for this web UI.</td></tr>
<tr><td>Event history</td><td>How many days of events to keep.</td></tr>
<tr><td>Health summary</td><td>A periodic digest (e.g. daily) of what happened, sent through your notification channels.</td></tr>
<tr><td>Maintenance</td><td>A daily quiet window (e.g. 02:00–05:00, in your <a href="#settings">time zone</a>) so nightly backups or updates don't trigger restarts. Turn on <b>maintenance_window</b> on each container that should follow it.</td></tr>
<tr><td>Notification mute</td><td>Turns the per-container <b>Mute notifications</b> buttons on or off, and sets how long a mute lasts.</td></tr>
<tr><td>Uptime Kuma</td><td>Connection to your Uptime Kuma status page, used by container reachability checks.</td></tr>
<tr><td>Portainer</td><td>Connection to Portainer, used to redeploy stacks. See <a href="#portainer">section 7</a>.</td></tr>
</table>
<p>At the bottom of every settings page is a strip showing where the config file lives,
whether Inspector can write to it, and which time zone it is using.</p>
<h3>Time zone</h3>
<p>Inspector uses the <code>TZ</code> setting of its Portainer stack for every time it shows or acts on:</p>
<ul>
<li><b>The maintenance window runs in this zone.</b> A 02:00–05:00 window with the wrong zone
runs hours off, so Inspector could restart containers during your backups and ignore
problems at other times.</li>
<li>Alert times (with the zone, e.g. <code>03:15 AM CET</code>), dashboard times, the Events
page's days (midnight to midnight), and logs.</li>
</ul>
<p>To change it: in Portainer open <b>Stacks → inspector → Editor</b>, set the <code>TZ</code>
environment variable to your zone's name, e.g. <code>America/New_York</code>,
<code>Europe/London</code> or <code>Australia/Sydney</code> (names from the
<a href="https://en.wikipedia.org/wiki/List_of_tz_database_time_zones">list of time zones</a>;
abbreviations like <code>EST</code> don't work), then click <b>Update the stack</b>. Without
<code>TZ</code>, Inspector uses UTC.</p>
</section>
<section id="restart">
<h2>10. "Requires restart" fields</h2>
<p>Most changes apply on the next check. Fields marked
<span class="tag t-violet">REQUIRES RESTART</span> (like turning a notification channel or the
dashboard on or off) only take effect after Inspector restarts:</p>
<ol class="steps">
<li>In Portainer go to <b>Containers</b>.</li>
<li>Tick <b>inspector</b> and click <b>Restart</b>.</li>
</ol>
<p>Your settings and history are kept; they live in the folders mounted into the container.</p>
</section>
<section id="trouble">
<h2>11. Troubleshooting</h2>
<table>
<tr><th>Problem</th><th>Fix</th></tr>
<tr><td>Red <b>READ-ONLY</b> badge at the bottom of Settings</td>
<td>Inspector can't save changes. In the stack, make sure the config folder mount does not end in
<code>:ro</code>, then update the stack.</td></tr>
<tr><td>A card says <b>CONTAINER NOT FOUND</b></td>
<td>The name doesn't match a container. Check the exact name in Portainer → Containers, delete
the entry and add it again with the right name.</td></tr>
<tr><td>Logs say <i>Permission denied on /var/run/docker.sock</i></td>
<td>Remove any <code>user:</code> line from the stack so Inspector starts normally (it detects the
socket permissions itself), then update the stack.</td></tr>
<tr><td>Send test fails</td>
<td>The message next to the button says why: wrong URL, wrong token (HTTP 401), or the server can't
be reached from your Docker host.</td></tr>
<tr><td>Forgot the dashboard password</td>
<td>Open <code>config.yml</code> in the stack's config folder (e.g. through your Unraid share),
set <code>dashboard: password:</code> to something new, and restart Inspector.</td></tr>
<tr><td>No alerts arrive</td>
<td>Check the channel is enabled, Send test works, Inspector was restarted after enabling it,
and the container isn't muted or in maintenance.</td></tr>
</table>
</section>
</main>
</div>
</body>
</html>
Binary file not shown.

After

Width:  |  Height:  |  Size: 34 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 63 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 42 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 33 KiB

+56
View File
@@ -0,0 +1,56 @@
"""First-run helpers: find config.yml and write a starter one if it's missing.
Lets Inspector be installed from Portainer with nothing on the host but an
empty folder mounted at /app/config.
"""
from __future__ import annotations
import json
import os
import secrets
from importlib import resources
from pathlib import Path
LEGACY_CONFIG_PATH = Path("/app/config.yml")
DEFAULT_CONFIG_PATH = Path("/app/config/config.yml")
def resolve_config_path(cli_value: str | None, env: dict | None = None) -> Path:
"""--config wins, then $INSPECTOR_CONFIG, then the legacy single-file
mount if present, then the folder mount."""
env = os.environ if env is None else env
if cli_value:
return Path(cli_value)
if env.get("INSPECTOR_CONFIG"):
return Path(env["INSPECTOR_CONFIG"])
if LEGACY_CONFIG_PATH.exists():
return LEGACY_CONFIG_PATH
return DEFAULT_CONFIG_PATH
def write_starter_config(path: Path, env: dict | None = None) -> str | None:
"""Write the starter config to `path`.
Dashboard credentials come from INSPECTOR_USERNAME / INSPECTOR_PASSWORD.
Returns the generated password when none was supplied (so the caller can
log it once), otherwise None.
"""
env = os.environ if env is None else env
username = env.get("INSPECTOR_USERNAME") or "admin"
password = env.get("INSPECTOR_PASSWORD") or ""
generated = None
if not password:
generated = password = secrets.token_urlsafe(12)
template = (resources.files(__package__) / "starter_config.yml").read_text(encoding="utf-8")
# json.dumps gives a valid YAML double-quoted scalar for any input.
text = template.replace("__USERNAME__", json.dumps(username)).replace(
"__PASSWORD__", json.dumps(password)
)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(text, encoding="utf-8")
try:
path.chmod(0o600)
except OSError:
pass
return generated
+34 -4
View File
@@ -1,10 +1,12 @@
from __future__ import annotations from __future__ import annotations
import io import io
import logging
import os import os
import shutil import shutil
import threading import threading
from dataclasses import dataclass, field from dataclasses import dataclass, field
from datetime import time
from pathlib import Path from pathlib import Path
import yaml import yaml
@@ -23,6 +25,30 @@ VALID_STATUSES = {
"UNREACHABLE", "UNREACHABLE",
} }
log = logging.getLogger("inspector.config")
# Names logging.setLevel() accepts; anything else would crash at startup.
VALID_LOG_LEVELS = {"CRITICAL", "FATAL", "ERROR", "WARN", "WARNING", "INFO", "DEBUG", "NOTSET"}
def valid_hhmm(value: str) -> bool:
"""True if `value` parses the way the maintenance tracker parses it ("HH:MM")."""
try:
hour, minute = str(value).split(":")
time(int(hour), int(minute))
return True
except (TypeError, ValueError):
return False
def _startup_safe(value, default, ok: bool, key: str):
"""Return `value`, or `default` with a warning when it would crash Inspector."""
if ok:
return value
log.warning("config: invalid %s %r; using %r instead", key, value, default)
return default
VALID_RECOVERY_ACTIONS = { VALID_RECOVERY_ACTIONS = {
"restart_container", "restart_container",
"start_container", "start_container",
@@ -190,9 +216,11 @@ def build_config_from_raw(raw: dict) -> AppConfig:
to validate an edited raw dict before writing it back to disk. to validate an edited raw dict before writing it back to disk.
""" """
insp_raw = raw.get("inspector", {}) or {} insp_raw = raw.get("inspector", {}) or {}
interval = int(insp_raw.get("interval_seconds", 60))
log_level = str(insp_raw.get("log_level", "INFO")).upper()
inspector = InspectorConfig( inspector = InspectorConfig(
interval_seconds=int(insp_raw.get("interval_seconds", 60)), interval_seconds=_startup_safe(interval, 1, interval >= 1, "inspector.interval_seconds"),
log_level=str(insp_raw.get("log_level", "INFO")).upper(), log_level=_startup_safe(log_level, "INFO", log_level in VALID_LOG_LEVELS, "inspector.log_level"),
dry_run=bool(insp_raw.get("dry_run", False)), dry_run=bool(insp_raw.get("dry_run", False)),
log_dir=str(insp_raw.get("log_dir", "/app/logs")), log_dir=str(insp_raw.get("log_dir", "/app/logs")),
log_filename=str(insp_raw.get("log_filename", "inspector.log")), log_filename=str(insp_raw.get("log_filename", "inspector.log")),
@@ -237,10 +265,12 @@ def build_config_from_raw(raw: dict) -> AppConfig:
) )
maint_raw = raw.get("maintenance") or {} maint_raw = raw.get("maintenance") or {}
maint_start = str(maint_raw.get("start", "00:00"))
maint_end = str(maint_raw.get("end", "06:00"))
maintenance = MaintenanceConfig( maintenance = MaintenanceConfig(
enabled=bool(maint_raw.get("enabled", False)), enabled=bool(maint_raw.get("enabled", False)),
start=str(maint_raw.get("start", "00:00")), start=_startup_safe(maint_start, "00:00", valid_hhmm(maint_start), "maintenance.start"),
end=str(maint_raw.get("end", "06:00")), end=_startup_safe(maint_end, "06:00", valid_hhmm(maint_end), "maintenance.end"),
dir=str(maint_raw.get("dir", "/app/data/maintenance")), dir=str(maint_raw.get("dir", "/app/data/maintenance")),
max_hours=int(maint_raw.get("max_hours", 8)), max_hours=int(maint_raw.get("max_hours", 8)),
) )
+997 -123
View File
File diff suppressed because it is too large. Load diff
+45 -6
View File
@@ -1,14 +1,49 @@
from __future__ import annotations from __future__ import annotations
from datetime import datetime, timezone import logging
from zoneinfo import ZoneInfo import os
from datetime import datetime, timezone, tzinfo
from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
PACIFIC = ZoneInfo("America/Los_Angeles") log = logging.getLogger("inspector.formatting")
def format_timestamp(value: datetime | str | None) -> str: def _resolve_local_tz(env: dict | None = None) -> tuple[tzinfo, str]:
"""Render a timestamp in US Pacific time, e.g. '06-14 03:45 PM'. """The zone every displayed time uses, plus its name.
Follows the same rules as the C library (which drives log timestamps and
the maintenance window), so all times Inspector shows agree:
TZ set to a zone name -> that zone; TZ unset -> the container's
/etc/localtime; anything else -> UTC.
"""
env = os.environ if env is None else env
name = (env.get("TZ") or "").strip().lstrip(":")
if name:
try:
return ZoneInfo(name), name
except (ZoneInfoNotFoundError, ValueError):
log.warning(
"TZ=%r is not a known time zone name (e.g. Europe/Berlin); showing times in UTC",
name,
)
return timezone.utc, "UTC"
try:
with open("/etc/localtime", "rb") as fh:
zone = ZoneInfo.from_file(fh, key="localtime")
label = datetime.now(zone).tzname() or "localtime"
return zone, label
except (OSError, ValueError):
return timezone.utc, "UTC"
# Resolved once at import; the container's TZ doesn't change while it runs.
LOCAL_TZ, LOCAL_TZ_NAME = _resolve_local_tz()
def format_timestamp(value: datetime | str | None, *, with_zone: bool = False) -> str:
"""Render a timestamp in the configured local zone, e.g. '06-14 03:45 PM'
(with_zone adds the abbreviation: '06-14 03:45 PM CET').
Accepts a datetime, an ISO 8601 string (as stored by EventStore / Accepts a datetime, an ISO 8601 string (as stored by EventStore /
ContainerState), or None/"" (returns ""). Naive datetimes/strings are ContainerState), or None/"" (returns ""). Naive datetimes/strings are
@@ -24,4 +59,8 @@ def format_timestamp(value: datetime | str | None) -> str:
return value return value
if value.tzinfo is None: if value.tzinfo is None:
value = value.replace(tzinfo=timezone.utc) value = value.replace(tzinfo=timezone.utc)
return value.astimezone(PACIFIC).strftime("%m-%d %I:%M %p") local = value.astimezone(LOCAL_TZ)
text = local.strftime("%m-%d %I:%M %p")
if with_zone:
text += f" {local.tzname()}"
return text
+62 -20
View File
@@ -2,6 +2,7 @@ from __future__ import annotations
import argparse import argparse
import logging import logging
import os
import signal import signal
import sys import sys
import time import time
@@ -9,6 +10,7 @@ from datetime import datetime
from pathlib import Path from pathlib import Path
from . import __version__ from . import __version__
from .bootstrap import resolve_config_path, write_starter_config
from .config import AppConfig, ConfigRef, ContainerConfig, load_config, load_raw from .config import AppConfig, ConfigRef, ContainerConfig, load_config, load_raw
from .dashboard import DashboardServer from .dashboard import DashboardServer
from .docker_client import build_client, get_container_status from .docker_client import build_client, get_container_status
@@ -168,7 +170,7 @@ def _check_one(
message=( message=(
f"**{name}** is back to **HEALTHY** (was *{state.previous_status}*).\n\n" f"**{name}** is back to **HEALTHY** (was *{state.previous_status}*).\n\n"
f"- Errors before recovery: **{state.error_count}**\n" f"- Errors before recovery: **{state.error_count}**\n"
f"- Time: {format_timestamp(now)}" f"- Time: {format_timestamp(now, with_zone=True)}"
), ),
priority=3, priority=3,
) )
@@ -194,7 +196,7 @@ def _check_one(
f"**{name}** is missing from Docker.\n\n" f"**{name}** is missing from Docker.\n\n"
+ (f"- Detail: {result.message}\n" if result.message else "") + (f"- Detail: {result.message}\n" if result.message else "")
+ f"- Last known status: *{state.previous_status or 'unknown'}*\n" + f"- Last known status: *{state.previous_status or 'unknown'}*\n"
f"- Time: {format_timestamp(now)}" f"- Time: {format_timestamp(now, with_zone=True)}"
), ),
priority=7, priority=7,
) )
@@ -238,7 +240,7 @@ def _check_one(
+ ".\n\n" + ".\n\n"
f"- Error count: **{state.error_count}/{container_config.max_error_count}**\n" f"- Error count: **{state.error_count}/{container_config.max_error_count}**\n"
f"- Previous status: *{state.previous_status or 'unknown'}*\n" f"- Previous status: *{state.previous_status or 'unknown'}*\n"
f"- Time: {format_timestamp(now)}" f"- Time: {format_timestamp(now, with_zone=True)}"
), ),
priority=5, priority=5,
) )
@@ -274,7 +276,7 @@ def _check_one(
f"**{container_config.max_restarts_per_hour}** restart(s)/hour.\n\n" f"**{container_config.max_restarts_per_hour}** restart(s)/hour.\n\n"
f"- Current status: **{result.status}**\n" f"- Current status: **{result.status}**\n"
f"- Restarts paused until the hourly window clears\n" f"- Restarts paused until the hourly window clears\n"
f"- Time: {format_timestamp(now)}" f"- Time: {format_timestamp(now, with_zone=True)}"
), ),
priority=9, priority=9,
) )
@@ -291,8 +293,10 @@ def _check_one(
if ok: if ok:
dep = container_config.depends_on dep = container_config.depends_on
if dep: # Only wait on a dependency Inspector is watching. A disabled one has
dep_status = states[dep].current_status if dep in states else None # no state, and waiting on it would block recovery forever.
if dep and dep in states:
dep_status = states[dep].current_status
if dep_status not in (HEALTHY, STARTING, NO_HEALTH_CHECK): if dep_status not in (HEALTHY, STARTING, NO_HEALTH_CHECK):
if container_config.vpn_routed: if container_config.vpn_routed:
log.info( log.info(
@@ -398,8 +402,9 @@ def run_loop(config_ref: ConfigRef) -> int:
except Exception as exc: except Exception as exc:
if "Permission denied" in str(exc): if "Permission denied" in str(exc):
log.critical( log.critical(
"Permission denied on /var/run/docker.sock. Set DOCKER_GID to " "Permission denied on /var/run/docker.sock. Start the container "
"the host docker group GID: stat -c '%g' /var/run/docker.sock" "as root (the default) so it can detect the socket's group, or set "
"group_add to the socket's GID: stat -c '%g' /var/run/docker.sock"
) )
else: else:
log.critical("Cannot reach Docker daemon: %s", exc) log.critical("Cannot reach Docker daemon: %s", exc)
@@ -414,6 +419,7 @@ def run_loop(config_ref: ConfigRef) -> int:
check_order = _topological_order(enabled_configs) check_order = _topological_order(enabled_configs)
portainer_client = PortainerClient(app.integrations.portainer) portainer_client = PortainerClient(app.integrations.portainer)
integrations_cfg = app.integrations
stack_cooldowns: dict[str, datetime] = {} stack_cooldowns: dict[str, datetime] = {}
dashboard = DashboardServer( dashboard = DashboardServer(
@@ -437,18 +443,31 @@ def run_loop(config_ref: ConfigRef) -> int:
notifier = _build_notifier(app) notifier = _build_notifier(app)
# Rebuild enabled_configs / check_order from current config so edits # Rebuild enabled_configs / check_order from current config so edits
# to per-container settings apply next cycle. Containers added via # to per-container settings apply next cycle. Containers added (or
# the settings page are logged as 'requires restart' because the # re-enabled) from the settings page get a fresh state here; `states`
# states dict is built once at startup. # is shared with the dashboard, so it is updated in place.
current_enabled = {c.name: c for c in app.containers if c.enabled} current_enabled = {c.name: c for c in app.containers if c.enabled}
new_names = current_enabled.keys() - states.keys() new_names = current_enabled.keys() - states.keys()
if new_names: if new_names:
log.warning( for new_name in new_names:
"Containers added in config require Inspector restart to begin monitoring: %s", states[new_name] = ContainerState(name=new_name)
sorted(new_names), name_width = max(name_width, *(len(n) for n in new_names))
) log.info("Now watching newly added container(s): %s", sorted(new_names))
enabled_configs = {n: c for n, c in current_enabled.items() if n in states} for gone in states.keys() - current_enabled.keys():
check_order = _topological_order(enabled_configs) states.pop(gone, None)
log.info("Stopped watching container (removed or disabled): %s", gone)
if current_enabled != enabled_configs:
# Only re-order (and re-log ordering warnings) when the config changed.
check_order = _topological_order(current_enabled)
enabled_configs = current_enabled
# Integration clients hold their config; rebuild them when the
# settings page changes Uptime Kuma / Portainer.
if app.integrations != integrations_cfg:
integrations_cfg = app.integrations
uptime_kuma_client = UptimeKumaClient(integrations_cfg.uptime_kuma)
portainer_client = PortainerClient(integrations_cfg.portainer)
log.info("Integration settings changed; reloaded Uptime Kuma / Portainer clients")
uptime_kuma_snapshot = uptime_kuma_client.fetch() uptime_kuma_snapshot = uptime_kuma_client.fetch()
for name in check_order: for name in check_order:
@@ -495,14 +514,37 @@ def run_loop(config_ref: ConfigRef) -> int:
def main(argv: list[str] | None = None) -> int: def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(prog="inspector", description="Docker container health watcher") parser = argparse.ArgumentParser(prog="inspector", description="Docker container health watcher")
parser.add_argument("--config", "-c", default="/app/config.yml", help="Path to config.yml") parser.add_argument(
"--config", "-c", default=None,
help="Path to config.yml (default: $INSPECTOR_CONFIG, /app/config.yml if present, "
"else /app/config/config.yml)",
)
parser.add_argument("--version", action="version", version=f"inspector {__version__}") parser.add_argument("--version", action="version", version=f"inspector {__version__}")
args = parser.parse_args(argv) args = parser.parse_args(argv)
config_path = Path(args.config) config_path = resolve_config_path(args.config)
if not config_path.exists(): if not config_path.exists():
print(f"config file not found: {config_path}", file=sys.stderr) # First run with an empty config folder: write a starter config so the
# dashboard comes up and everything else can be set from the web UI.
try:
generated_password = write_starter_config(config_path)
except OSError as exc:
print(
f"config file not found: {config_path} (and could not create a starter one: {exc})",
file=sys.stderr,
)
return 2 return 2
print(f"Created starter config at {config_path}", file=sys.stderr)
if generated_password:
print(
"=" * 64 + "\n"
" Inspector dashboard login (generated on first start)\n"
f" username: {os.environ.get('INSPECTOR_USERNAME') or 'admin'}\n"
f" password: {generated_password}\n"
" Change it under Settings -> Dashboard.\n"
+ "=" * 64,
file=sys.stderr,
)
app = load_config(config_path) app = load_config(config_path)
raw = load_raw(config_path) raw = load_raw(config_path)
+2 -1
View File
@@ -33,9 +33,10 @@ class GotifyNotifier:
}, },
} }
try: try:
# Header, not ?token=, so the token stays out of access logs.
resp = requests.post( resp = requests.post(
url, url,
params={"token": self._config.token}, headers={"X-Gotify-Key": self._config.token},
json=payload, json=payload,
timeout=self._timeout, timeout=self._timeout,
) )
+15 -15
View File
@@ -228,7 +228,7 @@ def _perform_restart_then_redeploy(
f"**{result.status}** check(s).\n\n" f"**{result.status}** check(s).\n\n"
f"- If it doesn't recover, Inspector escalates to a Portainer stack redeploy\n" f"- If it doesn't recover, Inspector escalates to a Portainer stack redeploy\n"
f"- Cooldown: **{container_config.cooldown_seconds}s** after restart\n" f"- Cooldown: **{container_config.cooldown_seconds}s** after restart\n"
f"- Time: {format_timestamp(now)}" f"- Time: {format_timestamp(now, with_zone=True)}"
), ),
priority=6, priority=6,
) )
@@ -242,7 +242,7 @@ def _perform_restart_then_redeploy(
state.restart_limit_notified = False state.restart_limit_notified = False
log.info( log.info(
"%-*s restart succeeded; cooldown until %s (will redeploy if still failing)", "%-*s restart succeeded; cooldown until %s (will redeploy if still failing)",
width, name, state.cooldown_until.isoformat(), width, name, format_timestamp(state.cooldown_until, with_zone=True),
) )
eventstore.record( eventstore.record(
container=name, container=name,
@@ -250,15 +250,15 @@ def _perform_restart_then_redeploy(
old_status=state.previous_status, old_status=state.previous_status,
new_status=result.status, new_status=result.status,
error_count=state.error_count, error_count=state.error_count,
message=f"restart succeeded; cooldown until {state.cooldown_until.isoformat()}", message=f"restart succeeded; cooldown until {format_timestamp(state.cooldown_until, with_zone=True)}",
) )
notifier.notify( notifier.notify(
title=f"✅ Inspector: {name} restart completed", title=f"✅ Inspector: {name} restart completed",
message=( message=(
f"**{name}** was successfully restarted by Docker.\n\n" f"**{name}** was successfully restarted by Docker.\n\n"
f"- Previous status: *{result.status}*\n" f"- Previous status: *{result.status}*\n"
f"- Cooldown until: {state.cooldown_until.isoformat()}\n" f"- Cooldown until: {format_timestamp(state.cooldown_until, with_zone=True)}\n"
f"- Time: {format_timestamp(now)}" f"- Time: {format_timestamp(now, with_zone=True)}"
), ),
priority=3, priority=3,
) )
@@ -321,7 +321,7 @@ def _perform_docker_action(
f"{verb.capitalize()}ing **{name}** after **{state.error_count}** consecutive " f"{verb.capitalize()}ing **{name}** after **{state.error_count}** consecutive "
f"**{result.status}** check(s).\n\n" f"**{result.status}** check(s).\n\n"
f"- Cooldown: **{container_config.cooldown_seconds}s** after {verb}\n" f"- Cooldown: **{container_config.cooldown_seconds}s** after {verb}\n"
f"- Time: {format_timestamp(now)}" f"- Time: {format_timestamp(now, with_zone=True)}"
), ),
priority=6, priority=6,
) )
@@ -335,22 +335,22 @@ def _perform_docker_action(
state.record_restart(now, container_config.cooldown_seconds) state.record_restart(now, container_config.cooldown_seconds)
state.last_notification_state = "RESTARTED" state.last_notification_state = "RESTARTED"
state.restart_limit_notified = False state.restart_limit_notified = False
log.info("%-*s %s succeeded; cooldown until %s", width, name, verb, state.cooldown_until.isoformat()) log.info("%-*s %s succeeded; cooldown until %s", width, name, verb, format_timestamp(state.cooldown_until, with_zone=True))
eventstore.record( eventstore.record(
container=name, container=name,
event_type=EVENT_RESTART, event_type=EVENT_RESTART,
old_status=state.previous_status, old_status=state.previous_status,
new_status=result.status, new_status=result.status,
error_count=state.error_count, error_count=state.error_count,
message=f"{verb} succeeded; cooldown until {state.cooldown_until.isoformat()}", message=f"{verb} succeeded; cooldown until {format_timestamp(state.cooldown_until, with_zone=True)}",
) )
notifier.notify( notifier.notify(
title=f"✅ Inspector: {name} {verb} completed", title=f"✅ Inspector: {name} {verb} completed",
message=( message=(
f"**{name}** was successfully {verb}ed by Docker.\n\n" f"**{name}** was successfully {verb}ed by Docker.\n\n"
f"- Previous status: *{result.status}*\n" f"- Previous status: *{result.status}*\n"
f"- Cooldown until: {state.cooldown_until.isoformat()}\n" f"- Cooldown until: {format_timestamp(state.cooldown_until, with_zone=True)}\n"
f"- Time: {format_timestamp(now)}" f"- Time: {format_timestamp(now, with_zone=True)}"
), ),
priority=3, priority=3,
) )
@@ -362,7 +362,7 @@ def _perform_docker_action(
f"{verb.capitalize()} of **{name}** failed.\n\n" f"{verb.capitalize()} of **{name}** failed.\n\n"
f"- Status: **{result.status}**\n" f"- Status: **{result.status}**\n"
f"- Error count: **{state.error_count}**\n" f"- Error count: **{state.error_count}**\n"
f"- Time: {format_timestamp(now)}\n\n" f"- Time: {format_timestamp(now, with_zone=True)}\n\n"
f"```\n{err}\n```" f"```\n{err}\n```"
), ),
priority=9, priority=9,
@@ -469,7 +469,7 @@ def _perform_portainer_redeploy(
f"Redeploying Portainer stack **{stack_name}** (environment **{environment}**) " f"Redeploying Portainer stack **{stack_name}** (environment **{environment}**) "
f"to recover **{name}**.\n\n" f"to recover **{name}**.\n\n"
f"- Status: **{result.status}**\n" f"- Status: **{result.status}**\n"
f"- Time: {format_timestamp(now)}" f"- Time: {format_timestamp(now, with_zone=True)}"
), ),
priority=6, priority=6,
) )
@@ -483,7 +483,7 @@ def _perform_portainer_redeploy(
message=( message=(
f"Portainer redeploy of **{stack_name}** (environment **{environment}**) failed.\n\n" f"Portainer redeploy of **{stack_name}** (environment **{environment}**) failed.\n\n"
f"- Triggered by: **{name}** ({result.status})\n" f"- Triggered by: **{name}** ({result.status})\n"
f"- Time: {format_timestamp(now)}\n\n" f"- Time: {format_timestamp(now, with_zone=True)}\n\n"
f"```\n{redeploy_result.message}\n```" f"```\n{redeploy_result.message}\n```"
), ),
priority=9, priority=9,
@@ -516,7 +516,7 @@ def _perform_portainer_redeploy(
f"successfully redeployed to recover **{name}**.\n\n" f"successfully redeployed to recover **{name}**.\n\n"
f"- Triggered by: *{result.status}*\n" f"- Triggered by: *{result.status}*\n"
f"- Post-redeploy wait: {wait_seconds}s\n" f"- Post-redeploy wait: {wait_seconds}s\n"
f"- Time: {format_timestamp(now)}" f"- Time: {format_timestamp(now, with_zone=True)}"
), ),
priority=4, priority=4,
) )
@@ -536,7 +536,7 @@ def _perform_portainer_redeploy(
f"Portainer stack **{stack_name}** was redeployed, but the HTTP recheck " f"Portainer stack **{stack_name}** was redeployed, but the HTTP recheck "
f"for **{name}** still reports a failure.\n\n" f"for **{name}** still reports a failure.\n\n"
f"- Recheck result: *{recheck.message}*\n" f"- Recheck result: *{recheck.message}*\n"
f"- Time: {format_timestamp(now)}" f"- Time: {format_timestamp(now, with_zone=True)}"
), ),
priority=6, priority=6,
) )
+64
View File
@@ -0,0 +1,64 @@
# Inspector configuration.
#
# Inspector wrote this starter file on first start. You can edit everything
# from the web UI (Settings), or by hand. Containers are added from
# Settings -> Containers -> Add container.
inspector:
interval_seconds: 60
log_level: INFO
dry_run: false
log_dir: /app/logs
log_filename: inspector.log
log_retention_days: 30
heartbeat: false
notifications:
# Gotify: https://gotify.net
gotify:
enabled: false
url: ""
token: ""
priority: 5
# Pushover: https://pushover.net
pushover:
enabled: false
token: ""
user_key: ""
priority: 5
device: ""
sound: ""
# Event history (powers the Events page and the health summary digest).
database:
enabled: true
path: /app/data/events.db
retention_days: 30
health_summary:
enabled: false
interval_hours: 24
priority: 3
maintenance:
enabled: false
start: "00:00"
end: "06:00"
dir: /app/data/maintenance
max_hours: 8
notification_mute:
enabled: true
dir: /app/data/muted
default_hours: 12
dashboard:
enabled: true
host: 0.0.0.0
port: 8080
username: __USERNAME__
password: __PASSWORD__
# Containers Inspector watches. Add them from the web UI.
containers:
+12 -6
View File
@@ -41,20 +41,26 @@ class UptimeKumaClient:
and bool(self._config.status_page_slug) and bool(self._config.status_page_slug)
) )
def _heartbeat_url(self) -> str:
return f"{self._config.url}/api/status-page/heartbeat/{self._config.status_page_slug}"
def fetch(self) -> HeartbeatSnapshot: def fetch(self) -> HeartbeatSnapshot:
if not self.enabled: if not self.enabled:
return _EMPTY return _EMPTY
url = f"{self._config.url}/api/status-page/heartbeat/{self._config.status_page_slug}"
try: try:
resp = requests.get(url, timeout=self._timeout) return self.fetch_or_raise()
resp.raise_for_status()
data = resp.json()
return self._parse(data)
except Exception as exc: except Exception as exc:
log.warning("Uptime Kuma heartbeat fetch failed (%s): %s", url, exc) log.warning("Uptime Kuma heartbeat fetch failed (%s): %s", self._heartbeat_url(), exc)
return _EMPTY return _EMPTY
def fetch_or_raise(self) -> HeartbeatSnapshot:
"""Like fetch(), but lets request/parse errors propagate (used by the
settings page's "Test connection" button)."""
resp = requests.get(self._heartbeat_url(), timeout=self._timeout)
resp.raise_for_status()
return self._parse(resp.json())
def _parse(self, data: dict) -> HeartbeatSnapshot: def _parse(self, data: dict) -> HeartbeatSnapshot:
# Expected shape: # Expected shape:
# {"publicGroupList": [{"monitorList": [{"id": 1, "name": "..."}]}], # {"publicGroupList": [{"monitorList": [{"id": 1, "name": "..."}]}],