inspector/CHANGELOG.md

Changelog

All notable changes to Inspector are documented here.

Format follows Keep a Changelog. Version numbers follow Semantic Versioning.

---

[0.7.1] — 2026-10-27

Changed

• Settings page redesigned to a classic SaaS layout (left rail + content pane):
  • Left rail groups sections under Core / Data / Scheduling with a live subtitle under each item (e.g. "60s · dry-run on", "gotify on · pushover off", "on · :8080") so you can read state without clicking in.
  • Each section is URL-addressable: /settings/inspector, /settings/notifications, /settings/dashboard, /settings/database, /settings/health_summary, /settings/maintenance, /settings/notification_mute.
  • Gotify and Pushover now share a single Notifications section with one Save button per channel, so the fan-out model is visible at a glance.
  • Save flash is a prominent green banner with a success check.
• Containers moved to a dedicated top-level route (/containers): a grid of tiles showing status, enabled, max errors, cooldown, and restart_on at a glance. Click a tile for the per-container edit form at /containers/<name>. Keeps the Settings page small when buddies have many containers.
• Top navigation adds a Containers tab alongside Status / Events / Settings.

Internal

• POST /settings/gotify and POST /settings/pushover now redirect back to /settings/notifications (the grouped page).
• POST /containers/<name> added as a parallel entry point to the existing settings backend for container edits.

---

[0.7.0] — 2026-10-26

Added

• Public packaging under euronvault/inspector on Gitea with a dedicated image at gitea.euronvault.com/euronvault/inspector:0.7.0 (plus :latest and :sha-<short> tags). Dev continues on eugene/inspector.
• scripts/sync_public.sh with an explicit allowlist (scripts/public_allowlist.txt) and grep-based denylist (scripts/public_denylist_patterns.txt) that refuses to publish any file matching a personal-string pattern (lab IPs, lab container names, lab env var names, agent-workflow scaffolding paths).
• Sanitized public/ staging tree: generic README, INSTALL guide, MIT LICENSE (Copyright (c) 2026 Europium 63), config.example.yml with two neutral example containers, and a compose.yml that pulls from gitea.euronvault.com/euronvault/inspector.
• Web settings page at /settings for editing config.yml without SSH: per-section forms for Inspector loop, Gotify, Pushover, Dashboard, Database, Health summary, Maintenance, Notification mute, and per-container tuning (thresholds, cooldowns, restart_on, color).
• Saves round-trip through ruamel.yaml so comments and key order in config.yml are preserved. Each save keeps a .bak of the prior file. (In the Docker container .bak lives in the overlay fs because config.yml is a single-file bind mount; the live file is still updated cleanly.)
• In-process hot reload via a new ConfigRef: edits to container-level settings and inspector.dry_run/heartbeat/interval_seconds take effect on the next loop iteration without a container restart. Fields requiring restart are explicitly labeled in the UI.
• CSRF token on all cookie-authenticated POSTs (Settings forms, maintenance toggle, mute toggle). Basic Auth clients bypass the check because they prove identity on every request.
• Secret fields (Gotify token, Pushover token/user_key, dashboard password) render masked and treat the __UNCHANGED__ sentinel as "keep existing"; real secret values are never echoed into HTML.

Changed

• main.run_loop() now takes a ConfigRef, re-reads config each iteration, and rebuilds enabled_configs/check_order so edits apply without a restart.
• DashboardServer constructor simplified: now takes ConfigRef, states, maintenance, notification_mute (per-section configs are resolved live via ConfigRef.get()).
• Added ruamel.yaml==0.18.6 to requirements.txt.

---

[0.6.0] — 2026-10-26

Added

• Pushover notifications as a second independent channel alongside Gotify. Enable either, both, or neither under notifications.pushover; alerts fan out to every enabled channel. Supports optional device and sound overrides.
• Priority mapping from Inspector's Gotify-style integer scale to Pushover's -2..2 range: <=2 → -1, 3..4 → 0, 5..6 → 1, >=7 → 2. Emergency tier (2) ships default retry/expire values and triggers the usual Pushover acknowledgement prompt.
• MultiNotifier internally fans each notify() call out to every enabled channel; a crash in one channel never prevents the others from running.

---

[0.5.5] — 2026-06-18

Changed

• Timestamp display format shortened: year and timezone abbreviation (PST/PDT) removed. Timestamps now render as MM-DD HH:MM AM/PM (e.g. 06-18 09:30 AM).

---

[0.5.4] — 2026-06-17

Added

• Dashboard cards now surface scheduled maintenance for containers opted into the recurring daily window (maintenance_window: true):
  • while idle, a meta-row 🔧 Next maintenance: in <countdown> (<start> – <end>) shows when the next window begins, with the window times in the dashboard's local format;
  • while the window is active, the phase row shows 🔧 In maintenance — ends in <countdown> so it's clear when normal monitoring resumes.
• All countdowns reuse the existing 1-second client-side ticker (no page reload needed).
• /api/status now returns maintenance_scheduled (bool) and maintenance_window_start / maintenance_window_end (ISO UTC or null) per container.

---

[0.5.3] — 2026-06-15

Added

• Dashboard status cards now show a phase row below the error bar, indicating what Inspector is currently doing for each container:
  • ⏱ Cooldown — Xs remaining — post-restart cooldown is active (countdown to expiry)
  • ⏱ Grace period — Xs remaining — container is booting post-restart; failures not counted
  • ⚠ Restart limit — resets in Xs — hourly restart cap hit; countdown to when it clears
  • ⚠ Failing X/Y — next check in Zs — error count is accumulating toward the restart threshold
  • 👁 Observe only — restart_on: [] is set; Inspector tracks status but never acts
  • 🔧 In maintenance — container is in a maintenance window
  • Next check in Zs — container is healthy; soft reminder of the monitoring cadence
• All countdown timers update every second in the browser without a page reload (JS setInterval); the existing 30 s auto-refresh keeps data fresh.
• /api/status now returns phase, phase_until (ISO UTC or null), last_check_at (ISO UTC or null), and interval_seconds per container entry.

---

[0.5.2] — 2026-06-15

Added

• Portainer environments now support api_token: "ptr_..." to embed the API token directly in config.yml, as an alternative to api_token_env (env var lookup). api_token takes precedence when both are set. Useful when injecting env vars into the Inspector container is inconvenient.
• Improved token-not-found error message: now mentions both api_token and api_token_env options so the fix is self-evident in the logs.
• _find_stack() now logs a debug line with how many stacks were scanned and which endpoint_id was used, aiding multi-environment Portainer diagnosis.

---

[0.5.1] — 2026-06-15

Fixed

• health_summary digest crash: KeyError: 'portainer_redeploy_failed' on the first periodic digest when event history contained Portainer redeploy events. health_summary._LABELS was missing the three Portainer event types (portainer_redeploy, portainer_redeploy_failed, portainer_redeploy_skipped) that were added to SUMMARY_EVENT_TYPES in v0.4.0.

---

[0.5.0] — 2026-06-16

Added

• New per-container recovery_action: restart_then_redeploy — tiered recovery for VPN-routed containers. Inspector first tries a plain docker restart (which fixes the common case where the VPN sidecar was merely restarted, keeping the same container id) and only escalates to a Portainer stack redeploy when the restart errors (e.g. the stale-namespace "No such container" case after the sidecar is recreated) or the container is still failing on the next recovery cycle. Escalation is non-blocking — it happens on the following cycle, not via an inline wait. Requires portainer_stack_name like portainer_stack_redeploy, and honors the same depends_on gating.

[0.4.2] — 2026-06-15

Added

• Result notifications for recovery actions: a successful docker restart / start now sends a ✅ … completed notification, and a successful Portainer stack redeploy sends ✅ … redeployed. A failed post-redeploy HTTP recheck sends ⚠️ … recheck failed.
• "Testing the Portainer API" guide in the README (uses python3 + requests, since the container image has no curl).

Changed

• Portainer API errors now include the HTTP status code and response body in the failure message/log (e.g. HTTP 401: {"message":"Invalid JWT token"}), making misconfigured tokens/endpoints far easier to diagnose. Added intermediate debug logging for stack lookup and a warning when a stack name isn't found.

---

[0.4.1] — 2026-06-15

Fixed

• Per-container restart_on: [] (empty list) now correctly disables all restart triggers for that container. Previously, an empty list was treated as "not set" and silently fell back to the default [UNHEALTHY, NOT RUNNING], so a container couldn't be configured as observe-only (e.g. for depends_on status tracking without Inspector restarting it itself). Omitting restart_on entirely still defaults to [UNHEALTHY, NOT RUNNING].

---

[0.4.0] — 2026-06-15

Added

• New per-container recovery_action: restart_container (default, unchanged behavior), start_container (calls docker start instead of docker restart), and portainer_stack_redeploy.
• portainer_stack_redeploy recovers containers whose network_mode: container:<vpn> namespace goes stale after the VPN sidecar (e.g. Gluetun) is recreated — the "joining network namespace of container: No such container" failure a plain restart can't fix. It redeploys the owning Portainer stack via the Portainer API. New integrations.portainer config: named environments (token loaded from api_token_env, never logged or stored), default_environment, stack_redeploy_cooldown_seconds, post_redeploy_wait_seconds. New per-container portainer_environment / portainer_stack_name.
• New per-container depends_on / vpn_routed: containers are checked (and recovered) in dependency order each cycle, so a VPN sidecar is recovered before its dependents. Recovery is deferred — without consuming the container's cooldown — while its depends_on container isn't healthy/starting; vpn_routed: true logs why.
• Dry-run extended to cover the new actions, including the Portainer redeploy flow (DRY RUN: would resolve/redeploy/wait Portainer ...).
• New event types: portainer_redeploy, portainer_redeploy_failed, portainer_redeploy_skipped.
• See Portainer stack redeploy recovery in the README.
• New per-container notification mute: notification_mute config (enabled, dir, default_hours, default 12h auto-expiring), dashboard "Mute notifications"/"Unmute notifications" button + "Muted" badge, POST /notifications/<name>/mute|unmute. Muting silences outbound Gotify notifications only — monitoring, failure counting, restarts/recovery, cooldowns, event history, and the health-summary digest continue unaffected. New event types notification_mute_enter/exit. See Muting notifications in the README.

All new fields are optional and backward-compatible; existing configs are unaffected. recovery_action: recreate_container is not implemented and is rejected at config load.

---

[0.3.1] — 2026-06-14

Changed

• http_check now classifies outcomes into three explicit tiers: the configured expected_status = up, a connection refused / timeout / DNS failure = unreachable (reported as the synthetic status code 0), and any other code = "responding but unexpected". Connection-level failures were already treated as failed checks (→ UNREACHABLE); this makes the distinction explicit in the result (HttpCheckResult.status_code) and in the log / notification messages. Matching remains exact — the README now documents how to find a container's actual status code.

---

[0.3.0] — 2026-06-14

Added

• Opt-in "unreachable" detection per container: even when Docker reports a container as healthy/running, Inspector can independently verify it's actually reachable via an HTTP/HTML health check (http_check) and/or a shared Uptime Kuma status page (integrations.uptime_kuma + per-container uptime_kuma.monitor_name). A failing check on an otherwise-healthy container surfaces as a new UNREACHABLE status (red badge on the dashboard); opt it into a container's restart_on to have it count toward failures/restarts like any other status. Both checks default to disabled and never crash Inspector.
• Events page (/events) redesigned as a daily view: shows one calendar day (12:00 AM-11:59 PM US Pacific) at a time, defaulting to today, with "Prev day" / "Next day" navigation (?date=YYYY-MM-DD, also supported by /api/events). The timeline now always renders all 24 hourly markers (12 AM..11 PM) so you can scroll directly to a time of day, plus a trailing "11 PM (yesterday)" section for the 1-hour lookback overlap from the previous day.

Changed

• All dashboard and Gotify notification timestamps now render in US Pacific time (PDT/PST, DST-aware) instead of UTC, e.g. 2026-06-14 03:45 PM PDT. JSON API endpoints are unaffected and continue to return raw UTC ISO 8601 timestamps.

---

[0.2.6] — 2026-06-14

Changed

• Gotify notifications reformatted as Markdown (extras.client::display.contentType: text/markdown) with richer context per message: error counts and thresholds, previous/current status, Docker error details for failed restarts, restart/cooldown info, and a formatted timestamp. The health-history digest is now a Markdown bulleted list with a header.
• Dashboard timestamps (container "Last restart" / "Cooldown until", and the events timeline) now render as 12-hour time with AM/PM (e.g. 2026-06-14 03:45 PM UTC) instead of raw ISO 8601.

Fixed

• "Last restart" no longer reverts to "Never" after Inspector itself restarts: on startup, with database.enabled: true, each container's most recent recorded restart (and any still-active cooldown) is restored from the event history database.

---

[0.2.5] — 2026-06-14

Added

• Per-container identity colors on the dashboard: each container card gets a soft background tint and colored name from a 16-color auto-assigned palette, and the same color carries over to that container's chip on the events timeline. Override the auto-assigned color with an optional color: "#rrggbb" field per container in config.yml.

---

[0.2.4] — 2026-06-14

Fixed

• Sign out: removed the redirect-if-authenticated on GET /login so sign out always reaches the login form, even with a cached browser Basic Auth session.
• Mobile header: version/refresh text now hides on narrow screens, and sign-out renders as a small outlined button beside the brand.
• Status page error bar: shows 100% green when a container has zero errors (previously rendered as an invisible 0-width bar) and a proportional red fill once errors are detected.
• Container cards: hide the cooldown row when no cooldown is active, and show "Never" instead of "—" when a container has no restart history.
• Events page: the "event history not enabled" message now includes the database: enabled: true config snippet needed to turn it on.

---

[0.2.3] — 2026-06-14

Changed

• Replaced the browser-native Basic Auth prompt with a styled dark-theme login page backed by a session cookie (24h expiry); HTTP Basic Auth still works for API clients. Added a "Sign out" link, and made the dashboard header, stats grid, and card grid responsive on mobile.

---

[0.2.2] — 2026-06-14

Added

• Dashboard pages now set a full favicon/app-icon set (favicon.ico, favicon.svg, favicon-96x96.png, apple-touch-icon.png, web app manifest icons, site.webmanifest) generated from the Inspector icon. Served from new GET /static/<file> endpoints alongside the existing icon route (Basic Auth required, same as every other dashboard route).

---

[0.2.1] — 2026-06-14

Changed

• Web dashboard redesigned: dark "card grid" theme. Each container gets its own card (status badge, error-count progress bar, cooldown/last-restart details, maintenance badge and toggle button) instead of a plain table; the events page renders a colour-coded vertical timeline instead of a table. The header now shows the Inspector icon plus an "Inspector" / "Monitor. Restart. Report." wordmark set in Google Sans Flex (loaded from Google Fonts; falls back to the system sans-serif if unavailable). New GET /static/icon.png endpoint (Basic Auth required, same as every other dashboard route) serves the icon from a new src/inspector/assets/ package directory.

---

[0.2.0] — 2026-06-14

Added

• Optional embedded web dashboard (dashboard.enabled, default false), protected by HTTP Basic Auth (dashboard.username / dashboard.password). If enabled with an empty password, Inspector logs an error and refuses to start the dashboard (fails closed). Runs in the same process as the monitoring loop — no extra container, zero new dependencies (stdlib http.server).
  • Status page (/, JSON at /api/status) — live per-container table: current status, error count vs. threshold, cooldown expiry, last restart time, and whether it's in a maintenance window. Auto-refreshes every 30 seconds.
  • Events page (/events, JSON at /api/events) — last 100 rows from event history, if database.enabled: true.
  • Maintenance toggle — when maintenance.enabled: true, per-container "Enter/Exit maintenance" buttons POST /maintenance/<name>/enter|exit, touching/removing the same marker files described under "Ad-hoc marker files". Returns 409 if maintenance windows are disabled, 404 for an unknown container. New module src/inspector/dashboard.py, new config section dashboard: in config.example.yml.
• MaintenanceTracker.peek(name) — read-only maintenance check (no transition logging, no eventstore writes) used by the dashboard.
• tests/test_dashboard.py — unit tests covering the disabled no-op, the empty-password refusal, Basic Auth enforcement, the status page and API, the maintenance toggle (enter/exit, disabled-config 409, unknown-container 404), and the events page/API with and without event history enabled.

---

[0.1.11] — 2026-06-14

Added

• Optional maintenance windows (maintenance.enabled, default false). Two complementary mechanisms, both gated by this one flag:

  • Recurring daily window — containers with maintenance_window: true are "in maintenance" every day between maintenance.start and maintenance.end (default 00:00–06:00, local time via TZ, may wrap past midnight).
  • Ad-hoc marker files — touch/rm <maintenance.dir>/<container> (default /app/data/maintenance/<container>) for unplanned maintenance. A marker older than maintenance.max_hours (default 8) auto-expires.

  While either mechanism is active for a container, Inspector is fully silent for it: no failure counting, restarts, notifications, status-change logging, or event recording. Only one-time "entering/exiting maintenance window" log lines and new maintenance_enter/maintenance_exit events (excluded from the health-history digest) mark the transitions. State (error counts, cooldowns, restart history) carries over unchanged across maintenance. New module src/inspector/maintenance.py, new config section maintenance: in config.example.yml, new per-container maintenance_window field.

• tests/test_maintenance.py — unit tests covering the disabled no-op, marker-file activation and auto-expiry, the recurring window (including midnight wraparound), enter/exit transition logging and event recording, and safe handling of marker-check errors.

---

[0.1.10] — 2026-06-14

Added

• Optional periodic health-history digest (health_summary.enabled, default false). When enabled, Inspector logs a per-container summary of notable activity (failures, restarts, restart-limit hits, recoveries, not-found events — status changes are excluded) since the last digest, every health_summary.interval_hours (default 24). The first digest fires immediately on startup, covering the last interval_hours. The digest is always logged; it is pushed to Gotify (at health_summary.priority, default 3) only if any container had activity in the window. Depends on database.enabled: true for real data — if health_summary.enabled is true but event history isn't enabled, Inspector logs a one-time startup warning and digests report "no issues" for every container. New module src/inspector/health_summary.py, new config section health_summary: in config.example.yml.
• EventStore.summarize(since) — per-container counts of notable event types at/after a timestamp, and EventStore.enabled property.
• tests/test_health_summary.py — unit tests covering the disabled no-op, the database-disabled warning, immediate first report with a capped window, interval gating, message formatting, and Gotify activity gating.

---

[0.1.9] — 2026-06-13

Added

• Optional SQLite event history (database.enabled, default false). When enabled, Inspector persists notable per-container events (status changes, recoveries, failures, restarts, restart-limit hits, container-not-found) to a local SQLite database at database.path (default /app/data/events.db), surviving container restarts. Old events are pruned according to database.retention_days (default 30) on startup and once per 24h thereafter. The database is opened in WAL mode for future concurrent read access (e.g. a read-only dashboard). A database error is logged but never crashes Inspector. New module src/inspector/eventstore.py, new config section database: in config.example.yml, new ./data volume mount in compose.yml / compose.prod.yml, new /app/data directory in Dockerfile.
• tests/test_eventstore.py — unit tests covering the disabled no-op, row insertion, WAL mode, and retention pruning.

Changed

• _check_one() signature gains an eventstore: EventStore parameter (always passed; no-ops internally when database.enabled is false), mirroring the existing notifier: GotifyNotifier pattern.
• README "What Inspector does not do" updated: live ContainerState is still in-memory and resets on restart, but event history can now persist via the new opt-in SQLite database.

---

[0.1.8] — 2026-06-13

Changed

• Log format overhauled for readability: container names are now left-padded into a fixed-width column, the redundant logger-name field (inspector:) was dropped from both the file and console formatters, and a blank line now brackets each failure → restart → recovery "incident" so it stands out as a block. Message wording adjusted accordingly (e.g. gotify Restarting due to 3 consecutive UNHEALTHY checks instead of Restarting gotify due to 3 consecutive UNHEALTHY checks).

---

[0.1.7] — 2026-06-13

Fixed

• Docker socket permission errors at startup no longer crash with a raw Python traceback. run_loop() now catches the connection failure and logs an actionable message pointing at DOCKER_GID (stat -c '%g' /var/run/docker.sock).

---

[0.1.6] — 2026-06-13

Added

• tests/test_config.py — 4 unit tests covering config loading, defaults, and validation (non-mapping containers rejected, unknown restart_on statuses rejected).
• tests/test_health.py — 8 unit tests covering all Docker status classifications (HEALTHY, UNHEALTHY, STARTING, NO HEALTH CHECK, NOT RUNNING, UNKNOWN, missing state, ERROR constant).
• tests/test_state.py — 10 unit tests covering failure counting, restart decision logic, and the full per-container state machine (dry-run, grace period, cooldown, hourly restart limit, recovery).

Changed

• CI workflow now runs the full unit test suite (python3 -m unittest discover -s tests -v) before building the image.

---

[0.1.5] — 2026-06-13

Changed

• CI and release workflows reverted to runs-on: ubuntu-latest (the self-hosted runner approach introduced in v0.1.3 was abandoned — the Gitea-hosted runner handles builds reliably).
• Gitea runner setup guide (docs/gitea_runner_setup.md, ops/gitea-runner/) updated with label-mismatch troubleshooting and clearer registration instructions.

Added

• README documents the verified release baseline (v0.1.5) and the step-by-step release flow.

---

[0.1.4] — 2026-06-13

Added

• Inspector logo and wordmark assets: logo/inspector_icon.{png,svg}, logo/inspector_wordmark.{png,svg}, logo/inspector_icon_chip.{png,svg}.
• README header now displays the Inspector wordmark image.

Changed

• Previous logo files moved to logo/archive/.
• .gitignore — added act_data/ (local act runner data directory).

---

[0.1.3] — 2026-06-13

Changed

• CI and release workflows switched to runs-on: self-hosted.
• CI simplified to use docker build directly (removed docker/setup-buildx-action dependency).

---

[0.1.2] — 2026-06-13

Changed

• Release workflow rewritten to use native docker login / docker push shell commands, replacing the docker/build-push-action third-party action.

Added

• workflow_dispatch trigger on the release workflow — enables manual image builds without a tag push (useful for testing the CI publish path).

---

[0.1.1] — 2026-06-13

Fixed

• Release workflow secret names corrected: GITEA_REGISTRY_USER → REGISTRY_USER, GITEA_REGISTRY_TOKEN → REGISTRY_TOKEN. The v0.1.0 release workflow referenced the wrong secret names, causing the publish step to fail.

---

[0.1.0] — 2026-06-13

Added

• Initial Inspector v1: self-hosted Python Docker container health watcher.
• Multi-container support via a single config.yml — replaces the old pattern of one script + one systemd service per watched container.
• Per-container state machine with independent failure counters.
• Health status classification: HEALTHY, UNHEALTHY, STARTING, NO HEALTH CHECK, NOT RUNNING, CONTAINER NOT FOUND, ERROR, UNKNOWN.
• Restart logic: fires after max_error_count consecutive failures.
• Cooldown: post-restart window that blocks further restarts.
• Grace period: post-restart window that suppresses failure counting while a container boots.
• Hourly restart cap (max_restarts_per_hour) to break endless loops; fires a critical Gotify alert when hit.
• Optional Gotify notifications on state transitions (not per-loop).
• Dry-run mode: logs decisions without executing restarts.
• Rotating log file (TimedRotatingFileHandler, midnight, 30-day retention) with colored console output and plain-text file output.
• Dockerfile — python:3.12-slim base, non-root inspector user (UID/GID 1000).
• compose.yml — dev compose: builds from source, mounts Docker socket.
• compose.prod.yml — prod compose: pulls published image from Gitea registry, no local build.
• requirements.txt — pinned: docker==7.1.0, PyYAML==6.0.2, requests==2.32.3, colorlog==6.9.0.
• Makefile with build, smoke, compose-config, tag, push-local, release, clean, version, help targets.
• Gitea Actions CI (ci.yml): byte-compile + Docker build on every push.
• Gitea Actions release (release.yml): build and publish four image tags on v*.*.* tag push.
• scripts/bump_version.sh — updates __version__ in src/inspector/__init__.py.
• scripts/smoke_test.sh — end-to-end dry-run test against the live Docker socket.
• README with full feature documentation, configuration reference, Portainer deployment guide, and troubleshooting table.