Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
6aed0cf9a7 | ||
|
|
57100d0a7b | ||
|
|
b4da527732 |
No files matched your search
@@ -7,6 +7,92 @@ Version numbers follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## [0.9.4] — 2026-10-28
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- **Containers without a Docker health check now "recover" properly.** Many images
|
||||||
|
don't define a health check, so Docker reports them as `NO HEALTH CHECK` even when
|
||||||
|
they work. Inspector only treated `HEALTHY` as back to normal, so for these
|
||||||
|
containers:
|
||||||
|
- the error count never reset, and one-off failures days apart added up to a restart;
|
||||||
|
- no "recovered" alert was sent;
|
||||||
|
- with *Restart, then redeploy*, the next unrelated problem skipped the restart and
|
||||||
|
redeployed the whole stack.
|
||||||
|
|
||||||
|
Now `NO HEALTH CHECK` counts as working (unless you made it a restart trigger): the
|
||||||
|
count resets, the "recovered" alert is sent ("back to running (no health check)"),
|
||||||
|
and the next problem starts with a plain restart. Containers with a health check
|
||||||
|
behave exactly as before.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## [0.9.3] — 2026-10-28
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- **Every time follows the `TZ` of the Inspector stack.** Before, the dashboard,
|
||||||
|
Events page and alerts were always in US Pacific time while the maintenance window
|
||||||
|
and logs used `TZ`, so anyone outside Pacific saw times hours off (e.g. a 02:00
|
||||||
|
Berlin window shown as 5:00 PM). Now the dashboard, the Events page (including
|
||||||
|
where each day starts), alert times, cooldown times and logs all use `TZ`.
|
||||||
|
- Alert times include the zone abbreviation (`10-28 09:27 AM CET`), and "Cooldown
|
||||||
|
until" in alerts, event messages and logs is a local time instead of a UTC
|
||||||
|
timestamp.
|
||||||
|
- The active time zone is shown at the bottom of every Settings page, on the Events
|
||||||
|
page and on the Maintenance page.
|
||||||
|
- **If you never set `TZ`**, times are now shown in UTC instead of Pacific. Set `TZ`
|
||||||
|
in the stack (e.g. `America/New_York`). An unrecognized name logs a warning and
|
||||||
|
falls back to UTC.
|
||||||
|
|
||||||
|
### Docs
|
||||||
|
|
||||||
|
- INSTALL: `TZ` is now marked as needed, with a "Why the time zone matters" section
|
||||||
|
(maintenance window, alerts, dashboard, logs), how to find your zone name, and a
|
||||||
|
troubleshooting row. Guide: a Time zone section and a first-login check. Compose
|
||||||
|
and README updated.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## [0.9.2] — 2026-10-28
|
||||||
|
|
||||||
|
Fixes from a code and security review. None of them change how containers are
|
||||||
|
monitored or recovered.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- **A bad value saved from Settings can no longer stop Inspector from starting.**
|
||||||
|
The forms now reject invalid times (maintenance start/end), unknown log levels and
|
||||||
|
out-of-range numbers (for example `interval_seconds: 0`, which made the loop spin).
|
||||||
|
If `config.yml` already has such a value, Inspector starts with the default and
|
||||||
|
logs a warning instead of crashing in a restart loop.
|
||||||
|
- **The dashboard password can no longer be cleared.** Saving an empty password (or
|
||||||
|
username) is refused, and an empty password never signs anyone in. Before, clearing
|
||||||
|
it left the running dashboard open to anyone.
|
||||||
|
- **Passwords with non-ASCII characters (é, ü, £ …) now work.** Before, every login
|
||||||
|
attempt with one crashed.
|
||||||
|
- The Events page no longer errors for a moment after a container is removed.
|
||||||
|
- The HTTP check hint no longer suggests `302` as an expected status; redirects are
|
||||||
|
followed, so the final page's code is what counts.
|
||||||
|
|
||||||
|
### Security
|
||||||
|
|
||||||
|
- **Sign-in throttling:** after 5 failed sign-ins (form or Basic Auth) from one address
|
||||||
|
within 5 minutes, that address is refused for the rest of the window.
|
||||||
|
- **Changing the dashboard username or password signs out existing sessions.**
|
||||||
|
Expired sessions are cleaned up.
|
||||||
|
- Request bodies are limited to 1 MB; an invalid `Content-Length` gets a 400.
|
||||||
|
- The Gotify token is sent in the `X-Gotify-Key` header instead of the URL, so it no
|
||||||
|
longer appears in Gotify or reverse-proxy access logs.
|
||||||
|
- Portainer token environment variables must be named `PORTAINER_…` (a name already in
|
||||||
|
your `config.yml` keeps working), so Test connection can't be used to send other
|
||||||
|
container secrets to an arbitrary URL.
|
||||||
|
- Single quotes are escaped in HTML, and the delete confirmations build their text safely.
|
||||||
|
- Two settings saves at the same moment can no longer overwrite each other.
|
||||||
|
- `requests` updated to 2.32.5 (CVE-2024-47081).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## [0.9.1] — 2026-10-28
|
## [0.9.1] — 2026-10-28
|
||||||
|
|
||||||
### Fixed
|
### Fixed
|
||||||
|
|||||||
+29
-1
@@ -52,12 +52,39 @@ TZ=America/Los_Angeles
|
|||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `INSPECTOR_DATA` | Folder on your server where Inspector keeps its settings, logs, and history. **Unraid:** `/mnt/user/appdata/inspector`. **Other Linux:** `/opt/inspector`. Created automatically. | Yes |
|
| `INSPECTOR_DATA` | Folder on your server where Inspector keeps its settings, logs, and history. **Unraid:** `/mnt/user/appdata/inspector`. **Other Linux:** `/opt/inspector`. Created automatically. | Yes |
|
||||||
| `INSPECTOR_PASSWORD` | Password for the dashboard (username `admin`). Used only on first start; change it later in the dashboard. Leave it out and Inspector generates one and prints it in the container logs. | Recommended |
|
| `INSPECTOR_PASSWORD` | Password for the dashboard (username `admin`). Used only on first start; change it later in the dashboard. Leave it out and Inspector generates one and prints it in the container logs. | Recommended |
|
||||||
| `TZ` | Your [timezone](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones), so times on the dashboard are local. | Recommended |
|
| `TZ` | **Your time zone**, e.g. `America/New_York` or `Europe/Berlin`. Without it Inspector uses UTC, so the maintenance window, alerts and dashboard times are off by your UTC offset. See [Why the time zone matters](#why-the-time-zone-matters). | **Yes** (default `UTC`) |
|
||||||
| `INSPECTOR_PORT` | Port on your server for the dashboard. Change it if `8080` is taken. | No (default `8080`) |
|
| `INSPECTOR_PORT` | Port on your server for the dashboard. Change it if `8080` is taken. | No (default `8080`) |
|
||||||
| `INSPECTOR_TAG` | Version to run, e.g. `0.8.0`. See [Updating](#4-updating). | No (default `latest`) |
|
| `INSPECTOR_TAG` | Version to run, e.g. `0.8.0`. See [Updating](#4-updating). | No (default `latest`) |
|
||||||
| `PUID` / `PGID` | User and group that own Inspector's files. Unraid users often use `99` / `100` so the files match other appdata. | No (default `1000` / `1000`) |
|
| `PUID` / `PGID` | User and group that own Inspector's files. Unraid users often use `99` / `100` so the files match other appdata. | No (default `1000` / `1000`) |
|
||||||
| `PORTAINER_API_TOKEN` | A Portainer access token. Only needed if you use **Portainer stack redeploy** for VPN-routed apps; see the in-app guide (Help → Portainer). | No |
|
| `PORTAINER_API_TOKEN` | A Portainer access token. Only needed if you use **Portainer stack redeploy** for VPN-routed apps; see the in-app guide (Help → Portainer). | No |
|
||||||
|
|
||||||
|
### Why the time zone matters
|
||||||
|
|
||||||
|
Inspector uses `TZ` for every time it shows or acts on. If it's wrong, things
|
||||||
|
still work, but at the wrong hour or with confusing times:
|
||||||
|
|
||||||
|
- **The maintenance window runs in this time zone.** A window of `02:00–05:00`
|
||||||
|
with `TZ` left at UTC runs at 02:00 UTC, which is the middle of the evening in
|
||||||
|
the Americas or mid-morning in Asia. Inspector would then restart containers
|
||||||
|
during your nightly backups, and ignore real problems during the day.
|
||||||
|
- **Alerts** (Gotify / Pushover) show the time of the problem in this zone, with
|
||||||
|
its abbreviation, e.g. `10-28 03:15 AM CET`.
|
||||||
|
- **The dashboard and Events page** show times in this zone, and the Events
|
||||||
|
page's days run midnight to midnight in this zone.
|
||||||
|
- **Logs** are stamped in this zone.
|
||||||
|
|
||||||
|
**How to set it:** use your zone's name from the
|
||||||
|
[list of time zones](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones)
|
||||||
|
("TZ identifier" column), e.g. `America/New_York`, `America/Chicago`,
|
||||||
|
`Europe/London`, `Europe/Berlin`, `Australia/Sydney`. Abbreviations like `EST` or
|
||||||
|
`CET` don't work. On Unraid, **Settings → Date and Time** shows the zone your
|
||||||
|
server uses.
|
||||||
|
|
||||||
|
**How to check it:** the bottom of every Settings page shows
|
||||||
|
`time zone: <name>`. If it says `UTC` and you're not on UTC, set `TZ` in the
|
||||||
|
stack and click **Update the stack**. If the name you set isn't recognized,
|
||||||
|
Inspector logs a warning and uses UTC.
|
||||||
|
|
||||||
## 3. Deploy and sign in
|
## 3. Deploy and sign in
|
||||||
|
|
||||||
1. Click **Deploy the stack**.
|
1. Click **Deploy the stack**.
|
||||||
@@ -116,6 +143,7 @@ starts and restarts the ones you add to it.
|
|||||||
| Logs say *Permission denied on /var/run/docker.sock* | Make sure the stack has no `user:` line, then update the stack. Inspector needs to start normally to set up socket access. |
|
| Logs say *Permission denied on /var/run/docker.sock* | Make sure the stack has no `user:` line, then update the stack. Inspector needs to start normally to set up socket access. |
|
||||||
| Red **READ-ONLY** badge at the bottom of Settings | The config folder mount ends in `:ro` or the folder isn't writable. Remove `:ro` and update the stack. |
|
| Red **READ-ONLY** badge at the bottom of Settings | The config folder mount ends in `:ro` or the folder isn't writable. Remove `:ro` and update the stack. |
|
||||||
| Can't sign in | Look for the generated password in the container logs. If you've lost it, open `INSPECTOR_DATA/config/config.yml` (e.g. via your Unraid share), set a new `password:` under `dashboard:`, and restart Inspector. |
|
| Can't sign in | Look for the generated password in the container logs. If you've lost it, open `INSPECTOR_DATA/config/config.yml` (e.g. via your Unraid share), set a new `password:` under `dashboard:`, and restart Inspector. |
|
||||||
|
| Times are a few hours off, or the maintenance window runs at the wrong time | `TZ` is missing or wrong. Check `time zone:` at the bottom of Settings, set `TZ` to your zone name (e.g. `Europe/London`), and update the stack. |
|
||||||
| A container shows **CONTAINER NOT FOUND** | The name in Inspector doesn't match Docker. Copy the exact name from Portainer → Containers. |
|
| A container shows **CONTAINER NOT FOUND** | The name in Inspector doesn't match Docker. Copy the exact name from Portainer → Containers. |
|
||||||
|
|
||||||
## Security notes
|
## Security notes
|
||||||
|
|||||||
@@ -35,6 +35,8 @@ restarts them when they break, and tells your phone about it.</p>
|
|||||||
- **Stays out of the way when you're working.** Put a container in maintenance or
|
- **Stays out of the way when you're working.** Put a container in maintenance or
|
||||||
mute its alerts with one click, or set a nightly maintenance window for backups
|
mute its alerts with one click, or set a nightly maintenance window for backups
|
||||||
and updates.
|
and updates.
|
||||||
|
- **Works in your time zone.** Set `TZ` in the stack and the maintenance window,
|
||||||
|
alerts, dashboard and logs all follow it.
|
||||||
|
|
||||||
## Install
|
## Install
|
||||||
|
|
||||||
@@ -91,7 +93,8 @@ and that socket is effectively root on your server. So:
|
|||||||
|
|
||||||
- Keep the dashboard on your home network, or put it behind a VPN or a reverse
|
- Keep the dashboard on your home network, or put it behind a VPN or a reverse
|
||||||
proxy with authentication. Don't expose port 8080 to the internet.
|
proxy with authentication. Don't expose port 8080 to the internet.
|
||||||
- Use a real dashboard password.
|
- Use a real dashboard password. Repeated failed sign-ins from one address are
|
||||||
|
blocked for a few minutes.
|
||||||
- Inspector itself runs as an unprivileged user; it starts as root only to detect
|
- Inspector itself runs as an unprivileged user; it starts as root only to detect
|
||||||
the socket's permissions and fix ownership of its own folders.
|
the socket's permissions and fix ownership of its own folders.
|
||||||
|
|
||||||
|
|||||||
+3
-1
@@ -7,7 +7,9 @@
|
|||||||
# Unraid: /mnt/user/appdata/inspector Other: /opt/inspector
|
# Unraid: /mnt/user/appdata/inspector Other: /opt/inspector
|
||||||
# INSPECTOR_PASSWORD Dashboard password (username is "admin").
|
# INSPECTOR_PASSWORD Dashboard password (username is "admin").
|
||||||
# Only used on first start; change it later in Settings.
|
# Only used on first start; change it later in Settings.
|
||||||
# TZ Your timezone, e.g. America/Los_Angeles
|
# TZ Your time zone name, e.g. America/New_York, Europe/Berlin.
|
||||||
|
# Important: the maintenance window, alerts, dashboard and
|
||||||
|
# logs all use it. Left unset, everything is in UTC.
|
||||||
#
|
#
|
||||||
# Optional:
|
# Optional:
|
||||||
# INSPECTOR_PORT Port for the dashboard on your server (default 8080).
|
# INSPECTOR_PORT Port for the dashboard on your server (default 8080).
|
||||||
|
|||||||
+1
-1
@@ -1,5 +1,5 @@
|
|||||||
docker==7.1.0
|
docker==7.1.0
|
||||||
PyYAML==6.0.2
|
PyYAML==6.0.2
|
||||||
requests==2.32.3
|
requests==2.32.5
|
||||||
colorlog==6.9.0
|
colorlog==6.9.0
|
||||||
ruamel.yaml==0.18.6
|
ruamel.yaml==0.18.6
|
||||||
@@ -1 +1 @@
|
|||||||
__version__ = "0.9.1"
|
__version__ = "0.9.4"
|
||||||
@@ -103,6 +103,8 @@ dashboard; you never need to touch a config file.</p>
|
|||||||
<i>Inspector dashboard login</i> with the username and password.</li>
|
<i>Inspector dashboard login</i> with the username and password.</li>
|
||||||
<li>Open <code>http://<your-server>:8080</code> and sign in.</li>
|
<li>Open <code>http://<your-server>:8080</code> and sign in.</li>
|
||||||
<li>Change the password under <b>Settings → Dashboard</b> if you like.</li>
|
<li>Change the password under <b>Settings → Dashboard</b> if you like.</li>
|
||||||
|
<li>Check the <b>time zone</b>: at the bottom of any Settings page it says
|
||||||
|
<code>time zone: …</code>. If that's not your zone, see <a href="#settings">Time zone</a>.</li>
|
||||||
</ol>
|
</ol>
|
||||||
</section>
|
</section>
|
||||||
|
|
||||||
@@ -392,13 +394,29 @@ maintenance. Use <b>Prev day</b> / <b>Next day</b> to look back.</p>
|
|||||||
<tr><td>Dashboard</td><td>Login username and password for this web UI.</td></tr>
|
<tr><td>Dashboard</td><td>Login username and password for this web UI.</td></tr>
|
||||||
<tr><td>Event history</td><td>How many days of events to keep.</td></tr>
|
<tr><td>Event history</td><td>How many days of events to keep.</td></tr>
|
||||||
<tr><td>Health summary</td><td>A periodic digest (e.g. daily) of what happened, sent through your notification channels.</td></tr>
|
<tr><td>Health summary</td><td>A periodic digest (e.g. daily) of what happened, sent through your notification channels.</td></tr>
|
||||||
<tr><td>Maintenance</td><td>A daily quiet window (e.g. 02:00–05:00) so nightly backups or updates don't trigger restarts. Turn on <b>maintenance_window</b> on each container that should follow it.</td></tr>
|
<tr><td>Maintenance</td><td>A daily quiet window (e.g. 02:00–05:00, in your <a href="#settings">time zone</a>) so nightly backups or updates don't trigger restarts. Turn on <b>maintenance_window</b> on each container that should follow it.</td></tr>
|
||||||
<tr><td>Notification mute</td><td>Turns the per-container <b>Mute notifications</b> buttons on or off, and sets how long a mute lasts.</td></tr>
|
<tr><td>Notification mute</td><td>Turns the per-container <b>Mute notifications</b> buttons on or off, and sets how long a mute lasts.</td></tr>
|
||||||
<tr><td>Uptime Kuma</td><td>Connection to your Uptime Kuma status page, used by container reachability checks.</td></tr>
|
<tr><td>Uptime Kuma</td><td>Connection to your Uptime Kuma status page, used by container reachability checks.</td></tr>
|
||||||
<tr><td>Portainer</td><td>Connection to Portainer, used to redeploy stacks. See <a href="#portainer">section 7</a>.</td></tr>
|
<tr><td>Portainer</td><td>Connection to Portainer, used to redeploy stacks. See <a href="#portainer">section 7</a>.</td></tr>
|
||||||
</table>
|
</table>
|
||||||
<p>At the bottom of every settings page is a strip showing where the config file lives and
|
<p>At the bottom of every settings page is a strip showing where the config file lives,
|
||||||
whether Inspector can write to it.</p>
|
whether Inspector can write to it, and which time zone it is using.</p>
|
||||||
|
|
||||||
|
<h3>Time zone</h3>
|
||||||
|
<p>Inspector uses the <code>TZ</code> setting of its Portainer stack for every time it shows or acts on:</p>
|
||||||
|
<ul>
|
||||||
|
<li><b>The maintenance window runs in this zone.</b> A 02:00–05:00 window with the wrong zone
|
||||||
|
runs hours off, so Inspector could restart containers during your backups and ignore
|
||||||
|
problems at other times.</li>
|
||||||
|
<li>Alert times (with the zone, e.g. <code>03:15 AM CET</code>), dashboard times, the Events
|
||||||
|
page's days (midnight to midnight), and logs.</li>
|
||||||
|
</ul>
|
||||||
|
<p>To change it: in Portainer open <b>Stacks → inspector → Editor</b>, set the <code>TZ</code>
|
||||||
|
environment variable to your zone's name, e.g. <code>America/New_York</code>,
|
||||||
|
<code>Europe/London</code> or <code>Australia/Sydney</code> (names from the
|
||||||
|
<a href="https://en.wikipedia.org/wiki/List_of_tz_database_time_zones">list of time zones</a>;
|
||||||
|
abbreviations like <code>EST</code> don't work), then click <b>Update the stack</b>. Without
|
||||||
|
<code>TZ</code>, Inspector uses UTC.</p>
|
||||||
</section>
|
</section>
|
||||||
|
|
||||||
<section id="restart">
|
<section id="restart">
|
||||||
|
|||||||
+34
-4
@@ -1,10 +1,12 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import io
|
import io
|
||||||
|
import logging
|
||||||
import os
|
import os
|
||||||
import shutil
|
import shutil
|
||||||
import threading
|
import threading
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
|
from datetime import time
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
import yaml
|
import yaml
|
||||||
@@ -23,6 +25,30 @@ VALID_STATUSES = {
|
|||||||
"UNREACHABLE",
|
"UNREACHABLE",
|
||||||
}
|
}
|
||||||
|
|
||||||
|
log = logging.getLogger("inspector.config")
|
||||||
|
|
||||||
|
# Names logging.setLevel() accepts; anything else would crash at startup.
|
||||||
|
VALID_LOG_LEVELS = {"CRITICAL", "FATAL", "ERROR", "WARN", "WARNING", "INFO", "DEBUG", "NOTSET"}
|
||||||
|
|
||||||
|
|
||||||
|
def valid_hhmm(value: str) -> bool:
|
||||||
|
"""True if `value` parses the way the maintenance tracker parses it ("HH:MM")."""
|
||||||
|
try:
|
||||||
|
hour, minute = str(value).split(":")
|
||||||
|
time(int(hour), int(minute))
|
||||||
|
return True
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _startup_safe(value, default, ok: bool, key: str):
|
||||||
|
"""Return `value`, or `default` with a warning when it would crash Inspector."""
|
||||||
|
if ok:
|
||||||
|
return value
|
||||||
|
log.warning("config: invalid %s %r; using %r instead", key, value, default)
|
||||||
|
return default
|
||||||
|
|
||||||
|
|
||||||
VALID_RECOVERY_ACTIONS = {
|
VALID_RECOVERY_ACTIONS = {
|
||||||
"restart_container",
|
"restart_container",
|
||||||
"start_container",
|
"start_container",
|
||||||
@@ -190,9 +216,11 @@ def build_config_from_raw(raw: dict) -> AppConfig:
|
|||||||
to validate an edited raw dict before writing it back to disk.
|
to validate an edited raw dict before writing it back to disk.
|
||||||
"""
|
"""
|
||||||
insp_raw = raw.get("inspector", {}) or {}
|
insp_raw = raw.get("inspector", {}) or {}
|
||||||
|
interval = int(insp_raw.get("interval_seconds", 60))
|
||||||
|
log_level = str(insp_raw.get("log_level", "INFO")).upper()
|
||||||
inspector = InspectorConfig(
|
inspector = InspectorConfig(
|
||||||
interval_seconds=int(insp_raw.get("interval_seconds", 60)),
|
interval_seconds=_startup_safe(interval, 1, interval >= 1, "inspector.interval_seconds"),
|
||||||
log_level=str(insp_raw.get("log_level", "INFO")).upper(),
|
log_level=_startup_safe(log_level, "INFO", log_level in VALID_LOG_LEVELS, "inspector.log_level"),
|
||||||
dry_run=bool(insp_raw.get("dry_run", False)),
|
dry_run=bool(insp_raw.get("dry_run", False)),
|
||||||
log_dir=str(insp_raw.get("log_dir", "/app/logs")),
|
log_dir=str(insp_raw.get("log_dir", "/app/logs")),
|
||||||
log_filename=str(insp_raw.get("log_filename", "inspector.log")),
|
log_filename=str(insp_raw.get("log_filename", "inspector.log")),
|
||||||
@@ -237,10 +265,12 @@ def build_config_from_raw(raw: dict) -> AppConfig:
|
|||||||
)
|
)
|
||||||
|
|
||||||
maint_raw = raw.get("maintenance") or {}
|
maint_raw = raw.get("maintenance") or {}
|
||||||
|
maint_start = str(maint_raw.get("start", "00:00"))
|
||||||
|
maint_end = str(maint_raw.get("end", "06:00"))
|
||||||
maintenance = MaintenanceConfig(
|
maintenance = MaintenanceConfig(
|
||||||
enabled=bool(maint_raw.get("enabled", False)),
|
enabled=bool(maint_raw.get("enabled", False)),
|
||||||
start=str(maint_raw.get("start", "00:00")),
|
start=_startup_safe(maint_start, "00:00", valid_hhmm(maint_start), "maintenance.start"),
|
||||||
end=str(maint_raw.get("end", "06:00")),
|
end=_startup_safe(maint_end, "06:00", valid_hhmm(maint_end), "maintenance.end"),
|
||||||
dir=str(maint_raw.get("dir", "/app/data/maintenance")),
|
dir=str(maint_raw.get("dir", "/app/data/maintenance")),
|
||||||
max_hours=int(maint_raw.get("max_hours", 8)),
|
max_hours=int(maint_raw.get("max_hours", 8)),
|
||||||
)
|
)
|
||||||
|
|||||||
+212
-64
@@ -1,5 +1,7 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import functools
|
||||||
|
import hashlib
|
||||||
import json
|
import json
|
||||||
import logging
|
import logging
|
||||||
import os
|
import os
|
||||||
@@ -21,12 +23,15 @@ from .config import (
|
|||||||
AppConfig,
|
AppConfig,
|
||||||
ConfigRef,
|
ConfigRef,
|
||||||
ContainerConfig,
|
ContainerConfig,
|
||||||
|
VALID_LOG_LEVELS,
|
||||||
HttpCheckConfig,
|
HttpCheckConfig,
|
||||||
UptimeKumaMonitorConfig,
|
UptimeKumaMonitorConfig,
|
||||||
build_config_from_raw,
|
build_config_from_raw,
|
||||||
|
valid_hhmm,
|
||||||
write_config_raw,
|
write_config_raw,
|
||||||
)
|
)
|
||||||
from .formatting import PACIFIC, format_timestamp
|
from . import formatting
|
||||||
|
from .formatting import format_timestamp
|
||||||
from .health import CONTAINER_NOT_FOUND, ERROR, HEALTHY, NOT_RUNNING, UNHEALTHY, UNREACHABLE
|
from .health import CONTAINER_NOT_FOUND, ERROR, HEALTHY, NOT_RUNNING, UNHEALTHY, UNREACHABLE
|
||||||
from .maintenance import MaintenanceTracker
|
from .maintenance import MaintenanceTracker
|
||||||
from .notification_mute import NotificationMuteTracker
|
from .notification_mute import NotificationMuteTracker
|
||||||
@@ -36,6 +41,9 @@ from .state import ContainerState, utcnow
|
|||||||
log = logging.getLogger("inspector.dashboard")
|
log = logging.getLogger("inspector.dashboard")
|
||||||
|
|
||||||
_SESSION_MAX_AGE = 86400 # seconds a login cookie stays valid
|
_SESSION_MAX_AGE = 86400 # seconds a login cookie stays valid
|
||||||
|
_MAX_BODY_BYTES = 1024 * 1024 # largest request body accepted
|
||||||
|
_LOGIN_MAX_FAILURES = 5 # failed logins per client before it is throttled
|
||||||
|
_LOGIN_FAILURE_WINDOW = 300 # seconds a failed login counts toward the limit
|
||||||
_UNCHANGED_SECRET_PLACEHOLDER = "__UNCHANGED__" # UI sentinel: keep existing secret value
|
_UNCHANGED_SECRET_PLACEHOLDER = "__UNCHANGED__" # UI sentinel: keep existing secret value
|
||||||
|
|
||||||
# Config keys that require an Inspector container restart to take effect.
|
# Config keys that require an Inspector container restart to take effect.
|
||||||
@@ -84,7 +92,13 @@ def _color_map(
|
|||||||
states: dict[str, ContainerState],
|
states: dict[str, ContainerState],
|
||||||
containers: dict[str, ContainerConfig],
|
containers: dict[str, ContainerConfig],
|
||||||
) -> dict[str, str]:
|
) -> dict[str, str]:
|
||||||
return {name: _resolve_color(containers[name], i) for i, name in enumerate(states)}
|
# list() snapshot + skip: the inspector loop adds/removes states from another
|
||||||
|
# thread, and a container removed in the UI keeps its state until the next check.
|
||||||
|
return {
|
||||||
|
name: _resolve_color(containers[name], i)
|
||||||
|
for i, name in enumerate(list(states))
|
||||||
|
if name in containers
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
def _compute_phase(
|
def _compute_phase(
|
||||||
@@ -167,20 +181,20 @@ def _status_snapshot(
|
|||||||
|
|
||||||
|
|
||||||
def _parse_date_param(value: str | None) -> date:
|
def _parse_date_param(value: str | None) -> date:
|
||||||
"""Parse a ?date=YYYY-MM-DD query param; falls back to "today" in
|
"""Parse a ?date=YYYY-MM-DD query param; falls back to "today" in the
|
||||||
America/Los_Angeles on missing/invalid input."""
|
configured time zone on missing/invalid input."""
|
||||||
if value:
|
if value:
|
||||||
try:
|
try:
|
||||||
return date.fromisoformat(value)
|
return date.fromisoformat(value)
|
||||||
except ValueError:
|
except ValueError:
|
||||||
pass
|
pass
|
||||||
return datetime.now(PACIFIC).date()
|
return datetime.now(formatting.LOCAL_TZ).date()
|
||||||
|
|
||||||
|
|
||||||
def _day_bounds_utc(day: date) -> tuple[datetime, datetime]:
|
def _day_bounds_utc(day: date) -> tuple[datetime, datetime]:
|
||||||
"""UTC [start, end) bounds for a Pacific calendar day, extended one hour
|
"""UTC [start, end) bounds for a local calendar day, extended one hour
|
||||||
into the previous day for the lookback-overlap."""
|
into the previous day for the lookback-overlap."""
|
||||||
midnight = datetime.combine(day, datetime.min.time(), tzinfo=PACIFIC)
|
midnight = datetime.combine(day, datetime.min.time(), tzinfo=formatting.LOCAL_TZ)
|
||||||
start = (midnight - timedelta(hours=1)).astimezone(timezone.utc)
|
start = (midnight - timedelta(hours=1)).astimezone(timezone.utc)
|
||||||
end = (midnight + timedelta(days=1)).astimezone(timezone.utc)
|
end = (midnight + timedelta(days=1)).astimezone(timezone.utc)
|
||||||
return start, end
|
return start, end
|
||||||
@@ -205,8 +219,8 @@ def _read_events_range(db_path: str, start: datetime, end: datetime) -> list[dic
|
|||||||
|
|
||||||
|
|
||||||
def _hour_markers(day: date) -> list[datetime]:
|
def _hour_markers(day: date) -> list[datetime]:
|
||||||
"""24 Pacific-local hour boundaries for `day`, 00:00..23:00 ascending."""
|
"""24 local hour boundaries for `day`, 00:00..23:00 ascending."""
|
||||||
midnight = datetime.combine(day, datetime.min.time(), tzinfo=PACIFIC)
|
midnight = datetime.combine(day, datetime.min.time(), tzinfo=formatting.LOCAL_TZ)
|
||||||
return [midnight + timedelta(hours=h) for h in range(24)]
|
return [midnight + timedelta(hours=h) for h in range(24)]
|
||||||
|
|
||||||
|
|
||||||
@@ -226,7 +240,7 @@ def _group_events_by_hour(
|
|||||||
|
|
||||||
for ev in events:
|
for ev in events:
|
||||||
try:
|
try:
|
||||||
ts = datetime.fromisoformat(ev["ts"]).astimezone(PACIFIC)
|
ts = datetime.fromisoformat(ev["ts"]).astimezone(formatting.LOCAL_TZ)
|
||||||
except (ValueError, KeyError):
|
except (ValueError, KeyError):
|
||||||
continue
|
continue
|
||||||
if prev_hour_start <= ts < markers[0]:
|
if prev_hour_start <= ts < markers[0]:
|
||||||
@@ -246,9 +260,31 @@ def _escape(value: object) -> str:
|
|||||||
.replace("<", "<")
|
.replace("<", "<")
|
||||||
.replace(">", ">")
|
.replace(">", ">")
|
||||||
.replace('"', """)
|
.replace('"', """)
|
||||||
|
.replace("'", "'")
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _js_string_attr(text: str) -> str:
|
||||||
|
"""A JS string literal safe to put inside a double-quoted HTML attribute."""
|
||||||
|
return _escape(json.dumps(text))
|
||||||
|
|
||||||
|
|
||||||
|
def _creds_match(username: str, password: str, cfg) -> bool:
|
||||||
|
"""Constant-time credential check. Never accepts an empty configured
|
||||||
|
password, and works for non-ASCII input (compare_digest on str doesn't)."""
|
||||||
|
if not cfg.password:
|
||||||
|
return False
|
||||||
|
user_ok = secrets.compare_digest(username.encode("utf-8"), cfg.username.encode("utf-8"))
|
||||||
|
pass_ok = secrets.compare_digest(password.encode("utf-8"), cfg.password.encode("utf-8"))
|
||||||
|
return user_ok and pass_ok
|
||||||
|
|
||||||
|
|
||||||
|
def _cred_fingerprint(cfg) -> str:
|
||||||
|
"""Changes whenever the dashboard username or password changes, which
|
||||||
|
invalidates every session created with the old credentials."""
|
||||||
|
return hashlib.sha256(f"{cfg.username}\0{cfg.password}".encode("utf-8")).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
_STATUS_BADGE_CLASS = {
|
_STATUS_BADGE_CLASS = {
|
||||||
HEALTHY: "healthy",
|
HEALTHY: "healthy",
|
||||||
UNHEALTHY: "unhealthy",
|
UNHEALTHY: "unhealthy",
|
||||||
@@ -1076,12 +1112,15 @@ def _render_events_page(
|
|||||||
|
|
||||||
prev_day = (selected_day - timedelta(days=1)).isoformat()
|
prev_day = (selected_day - timedelta(days=1)).isoformat()
|
||||||
next_day = (selected_day + timedelta(days=1)).isoformat()
|
next_day = (selected_day + timedelta(days=1)).isoformat()
|
||||||
today = datetime.now(PACIFIC).date()
|
today = datetime.now(formatting.LOCAL_TZ).date()
|
||||||
day_label = selected_day.strftime("%A, %B %-d, %Y")
|
day_label = selected_day.strftime("%A, %B %-d, %Y")
|
||||||
|
|
||||||
parts.append('<div class="events-nav">\n')
|
parts.append('<div class="events-nav">\n')
|
||||||
parts.append(f'<a class="day-nav" href="/events?date={prev_day}">← Prev day</a>\n')
|
parts.append(f'<a class="day-nav" href="/events?date={prev_day}">← Prev day</a>\n')
|
||||||
parts.append(f'<span class="day-label">{_escape(day_label)}</span>\n')
|
parts.append(
|
||||||
|
f'<span class="day-label">{_escape(day_label)}'
|
||||||
|
f' <span style="color:var(--muted); font-weight:400; font-size:0.8em;">· times in {_escape(formatting.LOCAL_TZ_NAME)}</span></span>\n'
|
||||||
|
)
|
||||||
if selected_day >= today:
|
if selected_day >= today:
|
||||||
parts.append('<span class="day-nav disabled">Next day →</span>\n')
|
parts.append('<span class="day-nav disabled">Next day →</span>\n')
|
||||||
else:
|
else:
|
||||||
@@ -1146,10 +1185,13 @@ def _render_event_item(ev: dict, container_colors: dict[str, str] | None) -> str
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def _render_login_page(error: bool = False) -> bytes:
|
def _render_login_page(error: bool = False, throttled: bool = False) -> bytes:
|
||||||
error_html = (
|
if throttled:
|
||||||
'<div class="login-error">Incorrect username or password.</div>\n' if error else ""
|
error_html = '<div class="login-error">Too many failed sign-ins. Try again in a few minutes.</div>\n'
|
||||||
)
|
elif error:
|
||||||
|
error_html = '<div class="login-error">Incorrect username or password.</div>\n'
|
||||||
|
else:
|
||||||
|
error_html = ""
|
||||||
head = (
|
head = (
|
||||||
"<!DOCTYPE html>\n"
|
"<!DOCTYPE html>\n"
|
||||||
'<html lang="en">\n<head>\n'
|
'<html lang="en">\n<head>\n'
|
||||||
@@ -1243,6 +1285,14 @@ def _form_float(form: dict, key: str, default: float = 0.0) -> float:
|
|||||||
return default
|
return default
|
||||||
|
|
||||||
|
|
||||||
|
def _int_in(form: dict, key: str, default: int, lo: int, hi: int) -> int:
|
||||||
|
"""Read an int form field and reject values outside lo..hi."""
|
||||||
|
value = _form_int(form, key, default)
|
||||||
|
if not lo <= value <= hi:
|
||||||
|
raise _SettingsError(f"{key} must be between {lo} and {hi} (got {value})")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
def _set_or_drop(section: dict, key: str, value) -> None:
|
def _set_or_drop(section: dict, key: str, value) -> None:
|
||||||
"""Store value, or remove the key when it is blank/False so config.yml
|
"""Store value, or remove the key when it is blank/False so config.yml
|
||||||
doesn't fill up with empty defaults."""
|
doesn't fill up with empty defaults."""
|
||||||
@@ -1270,8 +1320,10 @@ def _apply_container_advanced(raw: dict, name: str, cfg: dict, form: dict) -> No
|
|||||||
hc["enabled"] = hc_enabled
|
hc["enabled"] = hc_enabled
|
||||||
hc["url"] = hc_url
|
hc["url"] = hc_url
|
||||||
timeout = _form_float(form, "http_check_timeout_seconds", 5.0)
|
timeout = _form_float(form, "http_check_timeout_seconds", 5.0)
|
||||||
|
if not 0.5 <= timeout <= 120:
|
||||||
|
raise _SettingsError(f"HTTP check timeout must be between 0.5 and 120 seconds (got {timeout:g})")
|
||||||
hc["timeout_seconds"] = int(timeout) if timeout.is_integer() else timeout
|
hc["timeout_seconds"] = int(timeout) if timeout.is_integer() else timeout
|
||||||
hc["expected_status"] = _form_int(form, "http_check_expected_status", 200)
|
hc["expected_status"] = _int_in(form, "http_check_expected_status", 200, 100, 599)
|
||||||
cfg["http_check"] = hc
|
cfg["http_check"] = hc
|
||||||
else:
|
else:
|
||||||
cfg.pop("http_check", None)
|
cfg.pop("http_check", None)
|
||||||
@@ -1314,6 +1366,15 @@ def _apply_container_advanced(raw: dict, name: str, cfg: dict, form: dict) -> No
|
|||||||
cfg.pop("portainer_stack_name", None)
|
cfg.pop("portainer_stack_name", None)
|
||||||
|
|
||||||
|
|
||||||
|
_PORTAINER_ENV_NAME_RE = re.compile(r"^PORTAINER_[A-Z0-9_]+$")
|
||||||
|
|
||||||
|
|
||||||
|
def _token_env_allowed(name: str, saved: str = "") -> bool:
|
||||||
|
"""Only PORTAINER_* variables (or the name already in config.yml) may be read
|
||||||
|
as a token, so the dashboard can't be used to send other container secrets out."""
|
||||||
|
return bool(_PORTAINER_ENV_NAME_RE.match(name)) or (bool(saved) and name == saved)
|
||||||
|
|
||||||
|
|
||||||
def _apply_portainer_env(raw: dict, form: dict) -> None:
|
def _apply_portainer_env(raw: dict, form: dict) -> None:
|
||||||
"""Add, edit, rename or delete one integrations.portainer.environments entry."""
|
"""Add, edit, rename or delete one integrations.portainer.environments entry."""
|
||||||
pt = raw.setdefault("integrations", {}).setdefault("portainer", {})
|
pt = raw.setdefault("integrations", {}).setdefault("portainer", {})
|
||||||
@@ -1350,7 +1411,7 @@ def _apply_portainer_env(raw: dict, form: dict) -> None:
|
|||||||
|
|
||||||
env = envs.get(original) or {}
|
env = envs.get(original) or {}
|
||||||
env["base_url"] = base_url
|
env["base_url"] = base_url
|
||||||
env["endpoint_id"] = _form_int(form, "endpoint_id", 1)
|
env["endpoint_id"] = _int_in(form, "endpoint_id", 1, 1, 1_000_000)
|
||||||
env["verify_ssl"] = _form_bool(form, "verify_ssl")
|
env["verify_ssl"] = _form_bool(form, "verify_ssl")
|
||||||
if _form_str(form, "token_source", "env") == "inline":
|
if _form_str(form, "token_source", "env") == "inline":
|
||||||
_apply_secret(form, env, "api_token", "api_token")
|
_apply_secret(form, env, "api_token", "api_token")
|
||||||
@@ -1361,6 +1422,8 @@ def _apply_portainer_env(raw: dict, form: dict) -> None:
|
|||||||
token_env = _form_str(form, "api_token_env", "").strip()
|
token_env = _form_str(form, "api_token_env", "").strip()
|
||||||
if not token_env:
|
if not token_env:
|
||||||
raise _SettingsError("enter the name of the environment variable holding the API token")
|
raise _SettingsError("enter the name of the environment variable holding the API token")
|
||||||
|
if not _token_env_allowed(token_env, env.get("api_token_env", "")):
|
||||||
|
raise _SettingsError("the environment variable name must start with PORTAINER_ (e.g. PORTAINER_API_TOKEN)")
|
||||||
env["api_token_env"] = token_env
|
env["api_token_env"] = token_env
|
||||||
env.pop("api_token", None)
|
env.pop("api_token", None)
|
||||||
|
|
||||||
@@ -1376,57 +1439,81 @@ def _apply_portainer_env(raw: dict, form: dict) -> None:
|
|||||||
pt["environments"] = envs
|
pt["environments"] = envs
|
||||||
|
|
||||||
|
|
||||||
|
_SAVE_LOCK = threading.Lock()
|
||||||
|
|
||||||
|
|
||||||
|
def _serialized(fn):
|
||||||
|
"""Run config read-modify-write functions one at a time, so two saves at
|
||||||
|
the same moment can't overwrite each other's changes."""
|
||||||
|
@functools.wraps(fn)
|
||||||
|
def wrapper(*args, **kwargs):
|
||||||
|
with _SAVE_LOCK:
|
||||||
|
return fn(*args, **kwargs)
|
||||||
|
return wrapper
|
||||||
|
|
||||||
|
|
||||||
|
@_serialized
|
||||||
def _apply_settings_update(config_ref: ConfigRef, section: str, form: dict) -> None:
|
def _apply_settings_update(config_ref: ConfigRef, section: str, form: dict) -> None:
|
||||||
"""Validate the form, merge into the raw config, write atomically, swap ref."""
|
"""Validate the form, merge into the raw config, write atomically, swap ref."""
|
||||||
raw = config_ref.get_raw()
|
raw = config_ref.get_raw()
|
||||||
|
|
||||||
if section == "inspector":
|
if section == "inspector":
|
||||||
insp = raw.setdefault("inspector", {})
|
insp = raw.setdefault("inspector", {})
|
||||||
insp["interval_seconds"] = _form_int(form, "interval_seconds", 60)
|
insp["interval_seconds"] = _int_in(form, "interval_seconds", 60, 1, 86400)
|
||||||
insp["log_level"] = _form_str(form, "log_level", "INFO").upper()
|
level = _form_str(form, "log_level", "INFO").strip().upper()
|
||||||
|
if level not in VALID_LOG_LEVELS:
|
||||||
|
raise _SettingsError("log_level must be one of DEBUG, INFO, WARNING, ERROR, CRITICAL")
|
||||||
|
insp["log_level"] = level
|
||||||
insp["dry_run"] = _form_bool(form, "dry_run")
|
insp["dry_run"] = _form_bool(form, "dry_run")
|
||||||
insp["heartbeat"] = _form_bool(form, "heartbeat")
|
insp["heartbeat"] = _form_bool(form, "heartbeat")
|
||||||
insp["log_retention_days"] = _form_int(form, "log_retention_days", 30)
|
insp["log_retention_days"] = _int_in(form, "log_retention_days", 30, 1, 3650)
|
||||||
elif section == "gotify":
|
elif section == "gotify":
|
||||||
notif = raw.setdefault("notifications", {})
|
notif = raw.setdefault("notifications", {})
|
||||||
gotify = notif.setdefault("gotify", {})
|
gotify = notif.setdefault("gotify", {})
|
||||||
gotify["enabled"] = _form_bool(form, "enabled")
|
gotify["enabled"] = _form_bool(form, "enabled")
|
||||||
gotify["url"] = _form_str(form, "url", "")
|
gotify["url"] = _form_str(form, "url", "")
|
||||||
_apply_secret(form, gotify, "token", "token")
|
_apply_secret(form, gotify, "token", "token")
|
||||||
gotify["priority"] = _form_int(form, "priority", 5)
|
gotify["priority"] = _int_in(form, "priority", 5, 0, 10)
|
||||||
elif section == "pushover":
|
elif section == "pushover":
|
||||||
notif = raw.setdefault("notifications", {})
|
notif = raw.setdefault("notifications", {})
|
||||||
pushover = notif.setdefault("pushover", {})
|
pushover = notif.setdefault("pushover", {})
|
||||||
pushover["enabled"] = _form_bool(form, "enabled")
|
pushover["enabled"] = _form_bool(form, "enabled")
|
||||||
_apply_secret(form, pushover, "token", "token")
|
_apply_secret(form, pushover, "token", "token")
|
||||||
_apply_secret(form, pushover, "user_key", "user_key")
|
_apply_secret(form, pushover, "user_key", "user_key")
|
||||||
pushover["priority"] = _form_int(form, "priority", 5)
|
pushover["priority"] = _int_in(form, "priority", 5, 0, 10)
|
||||||
pushover["device"] = _form_str(form, "device", "")
|
pushover["device"] = _form_str(form, "device", "")
|
||||||
pushover["sound"] = _form_str(form, "sound", "")
|
pushover["sound"] = _form_str(form, "sound", "")
|
||||||
elif section == "dashboard":
|
elif section == "dashboard":
|
||||||
dash = raw.setdefault("dashboard", {})
|
dash = raw.setdefault("dashboard", {})
|
||||||
dash["enabled"] = _form_bool(form, "enabled")
|
dash["enabled"] = _form_bool(form, "enabled")
|
||||||
dash["username"] = _form_str(form, "username", "admin")
|
dash["username"] = _form_str(form, "username", "admin").strip()
|
||||||
|
if not dash["username"]:
|
||||||
|
raise _SettingsError("the dashboard username can't be empty")
|
||||||
_apply_secret(form, dash, "password", "password")
|
_apply_secret(form, dash, "password", "password")
|
||||||
|
if not dash.get("password"):
|
||||||
|
raise _SettingsError("the dashboard password can't be empty")
|
||||||
elif section == "database":
|
elif section == "database":
|
||||||
db = raw.setdefault("database", {})
|
db = raw.setdefault("database", {})
|
||||||
db["enabled"] = _form_bool(form, "enabled")
|
db["enabled"] = _form_bool(form, "enabled")
|
||||||
db["retention_days"] = _form_int(form, "retention_days", 30)
|
db["retention_days"] = _int_in(form, "retention_days", 30, 1, 3650)
|
||||||
elif section == "health_summary":
|
elif section == "health_summary":
|
||||||
hs = raw.setdefault("health_summary", {})
|
hs = raw.setdefault("health_summary", {})
|
||||||
hs["enabled"] = _form_bool(form, "enabled")
|
hs["enabled"] = _form_bool(form, "enabled")
|
||||||
hs["interval_hours"] = _form_int(form, "interval_hours", 24)
|
hs["interval_hours"] = _int_in(form, "interval_hours", 24, 1, 8760)
|
||||||
hs["priority"] = _form_int(form, "priority", 3)
|
hs["priority"] = _int_in(form, "priority", 3, 0, 10)
|
||||||
elif section == "maintenance":
|
elif section == "maintenance":
|
||||||
m = raw.setdefault("maintenance", {})
|
m = raw.setdefault("maintenance", {})
|
||||||
m["enabled"] = _form_bool(form, "enabled")
|
m["enabled"] = _form_bool(form, "enabled")
|
||||||
m["start"] = _form_str(form, "start", "00:00")
|
for key, default in (("start", "00:00"), ("end", "06:00")):
|
||||||
m["end"] = _form_str(form, "end", "06:00")
|
value = _form_str(form, key, default).strip()
|
||||||
m["max_hours"] = _form_int(form, "max_hours", 8)
|
if not valid_hhmm(value):
|
||||||
|
raise _SettingsError(f"{key} must be a 24-hour time like 02:30 (got {value!r})")
|
||||||
|
m[key] = value
|
||||||
|
m["max_hours"] = _int_in(form, "max_hours", 8, 1, 720)
|
||||||
elif section == "notification_mute":
|
elif section == "notification_mute":
|
||||||
nm = raw.setdefault("notification_mute", {})
|
nm = raw.setdefault("notification_mute", {})
|
||||||
nm["enabled"] = _form_bool(form, "enabled")
|
nm["enabled"] = _form_bool(form, "enabled")
|
||||||
nm["default_hours"] = _form_int(form, "default_hours", 12)
|
nm["default_hours"] = _int_in(form, "default_hours", 12, 1, 720)
|
||||||
elif section.startswith("container:"):
|
elif section.startswith("container:"):
|
||||||
name = section.split(":", 1)[1]
|
name = section.split(":", 1)[1]
|
||||||
containers = raw.get("containers") or {}
|
containers = raw.get("containers") or {}
|
||||||
@@ -1434,10 +1521,10 @@ def _apply_settings_update(config_ref: ConfigRef, section: str, form: dict) -> N
|
|||||||
raise _SettingsError(f"unknown container {name!r}")
|
raise _SettingsError(f"unknown container {name!r}")
|
||||||
cfg = containers[name] or {}
|
cfg = containers[name] or {}
|
||||||
cfg["enabled"] = _form_bool(form, "enabled")
|
cfg["enabled"] = _form_bool(form, "enabled")
|
||||||
cfg["max_error_count"] = _form_int(form, "max_error_count", 5)
|
cfg["max_error_count"] = _int_in(form, "max_error_count", 5, 1, 1000)
|
||||||
cfg["cooldown_seconds"] = _form_int(form, "cooldown_seconds", 300)
|
cfg["cooldown_seconds"] = _int_in(form, "cooldown_seconds", 300, 0, 86400)
|
||||||
cfg["grace_period_seconds"] = _form_int(form, "grace_period_seconds", 120)
|
cfg["grace_period_seconds"] = _int_in(form, "grace_period_seconds", 120, 0, 86400)
|
||||||
cfg["max_restarts_per_hour"] = _form_int(form, "max_restarts_per_hour", 3)
|
cfg["max_restarts_per_hour"] = _int_in(form, "max_restarts_per_hour", 3, 1, 100)
|
||||||
_set_or_drop(cfg, "color", _form_str(form, "color", "").strip())
|
_set_or_drop(cfg, "color", _form_str(form, "color", "").strip())
|
||||||
if form.get("restart_on_present"): # only our own form carries this field
|
if form.get("restart_on_present"): # only our own form carries this field
|
||||||
_set_or_drop(cfg, "maintenance_window", _form_bool(form, "maintenance_window"))
|
_set_or_drop(cfg, "maintenance_window", _form_bool(form, "maintenance_window"))
|
||||||
@@ -1467,8 +1554,8 @@ def _apply_settings_update(config_ref: ConfigRef, section: str, form: dict) -> N
|
|||||||
if default_env and default_env not in (pt.get("environments") or {}):
|
if default_env and default_env not in (pt.get("environments") or {}):
|
||||||
raise _SettingsError(f"unknown Portainer environment {default_env!r}")
|
raise _SettingsError(f"unknown Portainer environment {default_env!r}")
|
||||||
_set_or_drop(pt, "default_environment", default_env)
|
_set_or_drop(pt, "default_environment", default_env)
|
||||||
pt["stack_redeploy_cooldown_seconds"] = _form_int(form, "stack_redeploy_cooldown_seconds", 300)
|
pt["stack_redeploy_cooldown_seconds"] = _int_in(form, "stack_redeploy_cooldown_seconds", 300, 0, 86400)
|
||||||
pt["post_redeploy_wait_seconds"] = _form_int(form, "post_redeploy_wait_seconds", 90)
|
pt["post_redeploy_wait_seconds"] = _int_in(form, "post_redeploy_wait_seconds", 90, 0, 3600)
|
||||||
elif section == "portainer_env":
|
elif section == "portainer_env":
|
||||||
_apply_portainer_env(raw, form)
|
_apply_portainer_env(raw, form)
|
||||||
else:
|
else:
|
||||||
@@ -1492,6 +1579,7 @@ def _apply_settings_update(config_ref: ConfigRef, section: str, form: dict) -> N
|
|||||||
_CONTAINER_NAME_RE = re.compile(r"^[a-zA-Z0-9][a-zA-Z0-9_-]{0,62}$")
|
_CONTAINER_NAME_RE = re.compile(r"^[a-zA-Z0-9][a-zA-Z0-9_-]{0,62}$")
|
||||||
|
|
||||||
|
|
||||||
|
@_serialized
|
||||||
def _apply_container_create(config_ref: ConfigRef, form: dict) -> str:
|
def _apply_container_create(config_ref: ConfigRef, form: dict) -> str:
|
||||||
"""Insert a new container under raw['containers']. Returns the new name."""
|
"""Insert a new container under raw['containers']. Returns the new name."""
|
||||||
name = _form_str(form, "name", "").strip()
|
name = _form_str(form, "name", "").strip()
|
||||||
@@ -1507,10 +1595,10 @@ def _apply_container_create(config_ref: ConfigRef, form: dict) -> str:
|
|||||||
|
|
||||||
cfg = {
|
cfg = {
|
||||||
"enabled": _form_bool(form, "enabled"),
|
"enabled": _form_bool(form, "enabled"),
|
||||||
"max_error_count": _form_int(form, "max_error_count", 5),
|
"max_error_count": _int_in(form, "max_error_count", 5, 1, 1000),
|
||||||
"cooldown_seconds": _form_int(form, "cooldown_seconds", 300),
|
"cooldown_seconds": _int_in(form, "cooldown_seconds", 300, 0, 86400),
|
||||||
"grace_period_seconds": _form_int(form, "grace_period_seconds", 120),
|
"grace_period_seconds": _int_in(form, "grace_period_seconds", 120, 0, 86400),
|
||||||
"max_restarts_per_hour": _form_int(form, "max_restarts_per_hour", 3),
|
"max_restarts_per_hour": _int_in(form, "max_restarts_per_hour", 3, 1, 100),
|
||||||
"color": _form_str(form, "color", ""),
|
"color": _form_str(form, "color", ""),
|
||||||
"maintenance_window": _form_bool(form, "maintenance_window"),
|
"maintenance_window": _form_bool(form, "maintenance_window"),
|
||||||
}
|
}
|
||||||
@@ -1536,6 +1624,7 @@ def _apply_container_create(config_ref: ConfigRef, form: dict) -> str:
|
|||||||
return name
|
return name
|
||||||
|
|
||||||
|
|
||||||
|
@_serialized
|
||||||
def _apply_container_delete(config_ref: ConfigRef, name: str) -> None:
|
def _apply_container_delete(config_ref: ConfigRef, name: str) -> None:
|
||||||
raw = config_ref.get_raw()
|
raw = config_ref.get_raw()
|
||||||
containers = raw.get("containers") or {}
|
containers = raw.get("containers") or {}
|
||||||
@@ -1583,7 +1672,7 @@ def _run_test_notification(channel: str, form: dict, app: AppConfig) -> dict:
|
|||||||
try:
|
try:
|
||||||
resp = requests.post(
|
resp = requests.post(
|
||||||
f"{url}/message",
|
f"{url}/message",
|
||||||
params={"token": token},
|
headers={"X-Gotify-Key": token},
|
||||||
json={"title": title, "message": message, "priority": priority},
|
json={"title": title, "message": message, "priority": priority},
|
||||||
timeout=5.0,
|
timeout=5.0,
|
||||||
)
|
)
|
||||||
@@ -1665,6 +1754,9 @@ def _run_test_integration(kind: str, form: dict, app: AppConfig) -> dict:
|
|||||||
return {"ok": False, "error": "API token required"}
|
return {"ok": False, "error": "API token required"}
|
||||||
else:
|
else:
|
||||||
var = _form_str(form, "api_token_env", "").strip()
|
var = _form_str(form, "api_token_env", "").strip()
|
||||||
|
saved = app.integrations.portainer.environments.get(_form_str(form, "env_original", ""))
|
||||||
|
if var and not _token_env_allowed(var, saved.api_token_env if saved else ""):
|
||||||
|
return {"ok": False, "error": "the environment variable name must start with PORTAINER_"}
|
||||||
token = os.environ.get(var, "") if var else ""
|
token = os.environ.get(var, "") if var else ""
|
||||||
if not token:
|
if not token:
|
||||||
return {"ok": False, "error": f"environment variable {var!r} is not set in the Inspector container"}
|
return {"ok": False, "error": f"environment variable {var!r} is not set in the Inspector container"}
|
||||||
@@ -1974,6 +2066,7 @@ def _render_diag_strip(config_ref: ConfigRef) -> str:
|
|||||||
return (
|
return (
|
||||||
'<div class="diag-strip">'
|
'<div class="diag-strip">'
|
||||||
f'<span>config: <code>{_escape(str(path))}</code></span>'
|
f'<span>config: <code>{_escape(str(path))}</code></span>'
|
||||||
|
f'<span>time zone: <code>{_escape(formatting.LOCAL_TZ_NAME)}</code></span>'
|
||||||
f"<span>{badge}{hint}</span>"
|
f"<span>{badge}{hint}</span>"
|
||||||
f'<span style="margin-left:auto;">Inspector v{__version__}</span>'
|
f'<span style="margin-left:auto;">Inspector v{__version__}</span>'
|
||||||
"</div>\n"
|
"</div>\n"
|
||||||
@@ -2113,6 +2206,8 @@ def _settings_body_for(section: str, app: AppConfig, csrf_token: str) -> str:
|
|||||||
return (
|
return (
|
||||||
'<h3 class="pane-title">Scheduled maintenance window</h3>'
|
'<h3 class="pane-title">Scheduled maintenance window</h3>'
|
||||||
'<p class="pane-sub">Daily quiet hours. Containers that opt in (<code>maintenance_window: true</code> in their config) are not acted on during the window.</p>'
|
'<p class="pane-sub">Daily quiet hours. Containers that opt in (<code>maintenance_window: true</code> in their config) are not acted on during the window.</p>'
|
||||||
|
f'<p class="pane-sub" style="margin-top:-1rem;">Times are in <b>{_escape(formatting.LOCAL_TZ_NAME)}</b>, '
|
||||||
|
"from the <code>TZ</code> setting of the Inspector stack. If that's not your time zone, the window runs at the wrong hours.</p>"
|
||||||
+ _settings_form("maintenance", "Maintenance", body, csrf_token)
|
+ _settings_form("maintenance", "Maintenance", body, csrf_token)
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -2203,7 +2298,7 @@ def _portainer_env_form(env_name: str, env, csrf_token: str, *, suggest_name: st
|
|||||||
if env is not None:
|
if env is not None:
|
||||||
delete = (
|
delete = (
|
||||||
'<button type="submit" name="delete" value="1" class="btn-danger" '
|
'<button type="submit" name="delete" value="1" class="btn-danger" '
|
||||||
f"onclick=\"return confirm('Delete Portainer environment {_escape(env_name)}?');\">Delete</button>"
|
f'onclick="return confirm({_js_string_attr(f"Delete Portainer environment {env_name}?")});">Delete</button>'
|
||||||
)
|
)
|
||||||
label = "environment" if env is not None else "new environment"
|
label = "environment" if env is not None else "new environment"
|
||||||
return (
|
return (
|
||||||
@@ -2440,7 +2535,7 @@ def _render_container_advanced(name: str, cfg, app: AppConfig) -> str:
|
|||||||
+ '<div class="grid-2">'
|
+ '<div class="grid-2">'
|
||||||
+ _settings_field("timeout_seconds", "http_check_timeout_seconds", timeout_str, kind="number")
|
+ _settings_field("timeout_seconds", "http_check_timeout_seconds", timeout_str, kind="number")
|
||||||
+ _settings_field("expected_status", "http_check_expected_status", str(hc.expected_status), kind="number",
|
+ _settings_field("expected_status", "http_check_expected_status", str(hc.expected_status), kind="number",
|
||||||
hint="Exact HTTP code a healthy service returns (often 200; some apps return 401 or 302)")
|
hint="Exact HTTP code a healthy service returns (often 200; some apps return 401). Redirects are followed, so use the final page's code.")
|
||||||
+ "</div></div>"
|
+ "</div></div>"
|
||||||
'<div class="form-group"><h4>Uptime Kuma monitor</h4>'
|
'<div class="form-group"><h4>Uptime Kuma monitor</h4>'
|
||||||
+ _settings_field("enabled", "uptime_kuma_enabled", "on" if ukm.enabled else "", kind="switch")
|
+ _settings_field("enabled", "uptime_kuma_enabled", "on" if ukm.enabled else "", kind="switch")
|
||||||
@@ -2580,7 +2675,7 @@ def _render_container_form(
|
|||||||
delete_btn = (
|
delete_btn = (
|
||||||
f'<form method="post" action="/containers/{_escape(name)}/delete" '
|
f'<form method="post" action="/containers/{_escape(name)}/delete" '
|
||||||
'style="display:inline;" '
|
'style="display:inline;" '
|
||||||
f"onsubmit=\"return confirm('Delete container {_escape(name)} from config.yml?');\">"
|
f'onsubmit="return confirm({_js_string_attr(f"Delete container {name} from config.yml?")});">'
|
||||||
f'<input type="hidden" name="csrf" value="{_escape(csrf_token)}">'
|
f'<input type="hidden" name="csrf" value="{_escape(csrf_token)}">'
|
||||||
'<button type="submit" class="btn-danger">Delete container</button>'
|
'<button type="submit" class="btn-danger">Delete container</button>'
|
||||||
"</form>"
|
"</form>"
|
||||||
@@ -2684,8 +2779,39 @@ class _DashboardHTTPServer(ThreadingHTTPServer):
|
|||||||
self.states = states
|
self.states = states
|
||||||
self.maintenance = maintenance
|
self.maintenance = maintenance
|
||||||
self.notification_mute = notification_mute
|
self.notification_mute = notification_mute
|
||||||
# token -> (expiry_timestamp, csrf_token)
|
# token -> (expiry_timestamp, csrf_token, credential fingerprint)
|
||||||
self._sessions: dict[str, tuple[float, str]] = {}
|
self._sessions: dict[str, tuple[float, str, str]] = {}
|
||||||
|
# client address -> timestamps of recent failed logins
|
||||||
|
self._failed_logins: dict[str, list[float]] = {}
|
||||||
|
self._failed_lock = threading.Lock()
|
||||||
|
|
||||||
|
def login_throttled(self, client: str) -> bool:
|
||||||
|
cutoff = time.time() - _LOGIN_FAILURE_WINDOW
|
||||||
|
with self._failed_lock:
|
||||||
|
recent = [t for t in self._failed_logins.get(client, []) if t > cutoff]
|
||||||
|
if recent:
|
||||||
|
self._failed_logins[client] = recent
|
||||||
|
else:
|
||||||
|
self._failed_logins.pop(client, None)
|
||||||
|
return len(recent) >= _LOGIN_MAX_FAILURES
|
||||||
|
|
||||||
|
def note_login_failure(self, client: str) -> None:
|
||||||
|
with self._failed_lock:
|
||||||
|
self._failed_logins.setdefault(client, []).append(time.time())
|
||||||
|
|
||||||
|
def clear_login_failures(self, client: str) -> None:
|
||||||
|
with self._failed_lock:
|
||||||
|
self._failed_logins.pop(client, None)
|
||||||
|
|
||||||
|
def new_session(self) -> str:
|
||||||
|
now = time.time()
|
||||||
|
for tok, entry in list(self._sessions.items()): # drop expired sessions
|
||||||
|
if entry[0] <= now:
|
||||||
|
self._sessions.pop(tok, None)
|
||||||
|
token = secrets.token_urlsafe(32)
|
||||||
|
fp = _cred_fingerprint(self.current_app().dashboard)
|
||||||
|
self._sessions[token] = (now + _SESSION_MAX_AGE, secrets.token_urlsafe(32), fp)
|
||||||
|
return token
|
||||||
|
|
||||||
def current_app(self) -> AppConfig:
|
def current_app(self) -> AppConfig:
|
||||||
return self.config_ref.get()
|
return self.config_ref.get()
|
||||||
@@ -2710,29 +2836,47 @@ class _Handler(BaseHTTPRequestHandler):
|
|||||||
def _is_authenticated(self) -> bool:
|
def _is_authenticated(self) -> bool:
|
||||||
# Session cookie
|
# Session cookie
|
||||||
now = time.time()
|
now = time.time()
|
||||||
|
cfg = self.server.current_app().dashboard
|
||||||
token = self._session_token()
|
token = self._session_token()
|
||||||
if token:
|
if token:
|
||||||
entry = self.server._sessions.get(token)
|
entry = self.server._sessions.get(token)
|
||||||
if entry and now < entry[0]:
|
if entry and now < entry[0] and entry[2] == _cred_fingerprint(cfg):
|
||||||
return True
|
return True
|
||||||
self.server._sessions.pop(token, None)
|
self.server._sessions.pop(token, None)
|
||||||
|
|
||||||
# Basic Auth
|
# Basic Auth
|
||||||
header = self.headers.get("Authorization", "")
|
header = self.headers.get("Authorization", "")
|
||||||
if header.startswith("Basic "):
|
if header.startswith("Basic "):
|
||||||
|
client = self.client_address[0]
|
||||||
|
if self.server.login_throttled(client):
|
||||||
|
return False
|
||||||
try:
|
try:
|
||||||
decoded = b64decode(header[len("Basic "):]).decode("utf-8")
|
decoded = b64decode(header[len("Basic "):]).decode("utf-8")
|
||||||
username, _, password = decoded.partition(":")
|
username, _, password = decoded.partition(":")
|
||||||
except Exception:
|
except Exception:
|
||||||
username, password = "", ""
|
username, password = "", ""
|
||||||
cfg = self.server.current_app().dashboard
|
if _creds_match(username, password, cfg):
|
||||||
if secrets.compare_digest(username, cfg.username) and secrets.compare_digest(
|
self.server.clear_login_failures(client)
|
||||||
password, cfg.password
|
|
||||||
):
|
|
||||||
return True
|
return True
|
||||||
|
self.server.note_login_failure(client)
|
||||||
|
|
||||||
return False
|
return False
|
||||||
|
|
||||||
|
def _read_form(self) -> dict | None:
|
||||||
|
"""Parse the urlencoded request body, or send 400/413 and return None."""
|
||||||
|
try:
|
||||||
|
length = int(self.headers.get("Content-Length", 0))
|
||||||
|
except ValueError:
|
||||||
|
length = -1
|
||||||
|
if length < 0:
|
||||||
|
self._send_bytes(HTTPStatus.BAD_REQUEST, "text/plain; charset=utf-8", b"bad Content-Length")
|
||||||
|
return None
|
||||||
|
if length > _MAX_BODY_BYTES:
|
||||||
|
self._send_bytes(HTTPStatus.REQUEST_ENTITY_TOO_LARGE, "text/plain; charset=utf-8", b"request too large")
|
||||||
|
return None
|
||||||
|
raw = self.rfile.read(length) if length > 0 else b""
|
||||||
|
return parse_qs(raw.decode("utf-8", errors="replace"))
|
||||||
|
|
||||||
def _csrf_token_for_session(self) -> str:
|
def _csrf_token_for_session(self) -> str:
|
||||||
"""Return the CSRF token bound to the current session, or '' if none."""
|
"""Return the CSRF token bound to the current session, or '' if none."""
|
||||||
token = self._session_token()
|
token = self._session_token()
|
||||||
@@ -2890,18 +3034,21 @@ class _Handler(BaseHTTPRequestHandler):
|
|||||||
server = self.server
|
server = self.server
|
||||||
|
|
||||||
if path == "/login":
|
if path == "/login":
|
||||||
length = int(self.headers.get("Content-Length", 0))
|
data = self._read_form()
|
||||||
raw = self.rfile.read(length) if length > 0 else b""
|
if data is None:
|
||||||
data = parse_qs(raw.decode("utf-8", errors="replace"))
|
return
|
||||||
|
client = self.client_address[0]
|
||||||
|
if server.login_throttled(client):
|
||||||
|
self._send_bytes(
|
||||||
|
HTTPStatus.TOO_MANY_REQUESTS, "text/html; charset=utf-8", _render_login_page(throttled=True)
|
||||||
|
)
|
||||||
|
return
|
||||||
username = data.get("username", [""])[0]
|
username = data.get("username", [""])[0]
|
||||||
password = data.get("password", [""])[0]
|
password = data.get("password", [""])[0]
|
||||||
cfg = server.current_app().dashboard
|
cfg = server.current_app().dashboard
|
||||||
if secrets.compare_digest(username, cfg.username) and secrets.compare_digest(
|
if _creds_match(username, password, cfg):
|
||||||
password, cfg.password
|
server.clear_login_failures(client)
|
||||||
):
|
token = server.new_session()
|
||||||
token = secrets.token_urlsafe(32)
|
|
||||||
csrf = secrets.token_urlsafe(32)
|
|
||||||
server._sessions[token] = (time.time() + _SESSION_MAX_AGE, csrf)
|
|
||||||
self.send_response(HTTPStatus.SEE_OTHER)
|
self.send_response(HTTPStatus.SEE_OTHER)
|
||||||
self.send_header("Location", "/")
|
self.send_header("Location", "/")
|
||||||
self.send_header(
|
self.send_header(
|
||||||
@@ -2911,6 +3058,7 @@ class _Handler(BaseHTTPRequestHandler):
|
|||||||
self.send_header("Content-Length", "0")
|
self.send_header("Content-Length", "0")
|
||||||
self.end_headers()
|
self.end_headers()
|
||||||
else:
|
else:
|
||||||
|
server.note_login_failure(client)
|
||||||
self._send_bytes(
|
self._send_bytes(
|
||||||
HTTPStatus.UNAUTHORIZED, "text/html; charset=utf-8", _render_login_page(error=True)
|
HTTPStatus.UNAUTHORIZED, "text/html; charset=utf-8", _render_login_page(error=True)
|
||||||
)
|
)
|
||||||
@@ -2919,9 +3067,9 @@ class _Handler(BaseHTTPRequestHandler):
|
|||||||
if not self._check_auth():
|
if not self._check_auth():
|
||||||
return
|
return
|
||||||
|
|
||||||
length = int(self.headers.get("Content-Length", 0))
|
form = self._read_form()
|
||||||
raw_body = self.rfile.read(length) if length > 0 else b""
|
if form is None:
|
||||||
form = parse_qs(raw_body.decode("utf-8", errors="replace"))
|
return
|
||||||
|
|
||||||
# Only cookie-authenticated requests need a CSRF token. Basic Auth
|
# Only cookie-authenticated requests need a CSRF token. Basic Auth
|
||||||
# clients prove identity on every request so they're not subject to
|
# clients prove identity on every request so they're not subject to
|
||||||
|
|||||||
@@ -1,14 +1,49 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from datetime import datetime, timezone
|
import logging
|
||||||
from zoneinfo import ZoneInfo
|
import os
|
||||||
|
from datetime import datetime, timezone, tzinfo
|
||||||
|
from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
|
||||||
|
|
||||||
|
|
||||||
PACIFIC = ZoneInfo("America/Los_Angeles")
|
log = logging.getLogger("inspector.formatting")
|
||||||
|
|
||||||
|
|
||||||
def format_timestamp(value: datetime | str | None) -> str:
|
def _resolve_local_tz(env: dict | None = None) -> tuple[tzinfo, str]:
|
||||||
"""Render a timestamp in US Pacific time, e.g. '06-14 03:45 PM'.
|
"""The zone every displayed time uses, plus its name.
|
||||||
|
|
||||||
|
Follows the same rules as the C library (which drives log timestamps and
|
||||||
|
the maintenance window), so all times Inspector shows agree:
|
||||||
|
TZ set to a zone name -> that zone; TZ unset -> the container's
|
||||||
|
/etc/localtime; anything else -> UTC.
|
||||||
|
"""
|
||||||
|
env = os.environ if env is None else env
|
||||||
|
name = (env.get("TZ") or "").strip().lstrip(":")
|
||||||
|
if name:
|
||||||
|
try:
|
||||||
|
return ZoneInfo(name), name
|
||||||
|
except (ZoneInfoNotFoundError, ValueError):
|
||||||
|
log.warning(
|
||||||
|
"TZ=%r is not a known time zone name (e.g. Europe/Berlin); showing times in UTC",
|
||||||
|
name,
|
||||||
|
)
|
||||||
|
return timezone.utc, "UTC"
|
||||||
|
try:
|
||||||
|
with open("/etc/localtime", "rb") as fh:
|
||||||
|
zone = ZoneInfo.from_file(fh, key="localtime")
|
||||||
|
label = datetime.now(zone).tzname() or "localtime"
|
||||||
|
return zone, label
|
||||||
|
except (OSError, ValueError):
|
||||||
|
return timezone.utc, "UTC"
|
||||||
|
|
||||||
|
|
||||||
|
# Resolved once at import; the container's TZ doesn't change while it runs.
|
||||||
|
LOCAL_TZ, LOCAL_TZ_NAME = _resolve_local_tz()
|
||||||
|
|
||||||
|
|
||||||
|
def format_timestamp(value: datetime | str | None, *, with_zone: bool = False) -> str:
|
||||||
|
"""Render a timestamp in the configured local zone, e.g. '06-14 03:45 PM'
|
||||||
|
(with_zone adds the abbreviation: '06-14 03:45 PM CET').
|
||||||
|
|
||||||
Accepts a datetime, an ISO 8601 string (as stored by EventStore /
|
Accepts a datetime, an ISO 8601 string (as stored by EventStore /
|
||||||
ContainerState), or None/"" (returns ""). Naive datetimes/strings are
|
ContainerState), or None/"" (returns ""). Naive datetimes/strings are
|
||||||
@@ -24,4 +59,8 @@ def format_timestamp(value: datetime | str | None) -> str:
|
|||||||
return value
|
return value
|
||||||
if value.tzinfo is None:
|
if value.tzinfo is None:
|
||||||
value = value.replace(tzinfo=timezone.utc)
|
value = value.replace(tzinfo=timezone.utc)
|
||||||
return value.astimezone(PACIFIC).strftime("%m-%d %I:%M %p")
|
local = value.astimezone(LOCAL_TZ)
|
||||||
|
text = local.strftime("%m-%d %I:%M %p")
|
||||||
|
if with_zone:
|
||||||
|
text += f" {local.tzname()}"
|
||||||
|
return text
|
||||||
+30
-11
@@ -63,6 +63,19 @@ def _request_stop(signum, _frame):
|
|||||||
_stop = True
|
_stop = True
|
||||||
|
|
||||||
|
|
||||||
|
def _is_normal(status: str | None, container_config: ContainerConfig) -> bool:
|
||||||
|
"""True when `status` means the container is working.
|
||||||
|
|
||||||
|
HEALTHY always counts. NO HEALTH CHECK (running, but the image defines no
|
||||||
|
health check) counts too, unless the user made it a restart trigger.
|
||||||
|
Many images have no health check, and without this their error count and
|
||||||
|
restart episode would never reset.
|
||||||
|
"""
|
||||||
|
if status == HEALTHY:
|
||||||
|
return True
|
||||||
|
return status == NO_HEALTH_CHECK and NO_HEALTH_CHECK not in container_config.restart_on
|
||||||
|
|
||||||
|
|
||||||
def _apply_reachability_override(
|
def _apply_reachability_override(
|
||||||
container_config: ContainerConfig,
|
container_config: ContainerConfig,
|
||||||
result: HealthResult,
|
result: HealthResult,
|
||||||
@@ -158,19 +171,25 @@ def _check_one(
|
|||||||
message=result.message,
|
message=result.message,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
is_normal = _is_normal(state.current_status, container_config)
|
||||||
if (
|
if (
|
||||||
state.previous_status is not None
|
state.previous_status is not None
|
||||||
and state.previous_status != HEALTHY
|
and not _is_normal(state.previous_status, container_config)
|
||||||
and state.current_status == HEALTHY
|
and is_normal
|
||||||
|
# HEALTHY here means "a recovery was already announced", for either status.
|
||||||
and state.last_notification_state != HEALTHY
|
and state.last_notification_state != HEALTHY
|
||||||
):
|
):
|
||||||
log.info("%-*s recovered: %s -> HEALTHY", width, name, state.previous_status)
|
log.info("%-*s recovered: %s -> %s", width, name, state.previous_status, state.current_status)
|
||||||
|
back_to = (
|
||||||
|
"**HEALTHY**" if state.current_status == HEALTHY
|
||||||
|
else "**running** (no health check)"
|
||||||
|
)
|
||||||
notifier.notify(
|
notifier.notify(
|
||||||
title=f"✅ Inspector: {name} recovered",
|
title=f"✅ Inspector: {name} recovered",
|
||||||
message=(
|
message=(
|
||||||
f"**{name}** is back to **HEALTHY** (was *{state.previous_status}*).\n\n"
|
f"**{name}** is back to {back_to} (was *{state.previous_status}*).\n\n"
|
||||||
f"- Errors before recovery: **{state.error_count}**\n"
|
f"- Errors before recovery: **{state.error_count}**\n"
|
||||||
f"- Time: {format_timestamp(now)}"
|
f"- Time: {format_timestamp(now, with_zone=True)}"
|
||||||
),
|
),
|
||||||
priority=3,
|
priority=3,
|
||||||
)
|
)
|
||||||
@@ -178,7 +197,7 @@ def _check_one(
|
|||||||
container=name,
|
container=name,
|
||||||
event_type=EVENT_RECOVERY,
|
event_type=EVENT_RECOVERY,
|
||||||
old_status=state.previous_status,
|
old_status=state.previous_status,
|
||||||
new_status=HEALTHY,
|
new_status=state.current_status,
|
||||||
error_count=state.error_count,
|
error_count=state.error_count,
|
||||||
)
|
)
|
||||||
state.last_notification_state = HEALTHY
|
state.last_notification_state = HEALTHY
|
||||||
@@ -196,7 +215,7 @@ def _check_one(
|
|||||||
f"**{name}** is missing from Docker.\n\n"
|
f"**{name}** is missing from Docker.\n\n"
|
||||||
+ (f"- Detail: {result.message}\n" if result.message else "")
|
+ (f"- Detail: {result.message}\n" if result.message else "")
|
||||||
+ f"- Last known status: *{state.previous_status or 'unknown'}*\n"
|
+ f"- Last known status: *{state.previous_status or 'unknown'}*\n"
|
||||||
f"- Time: {format_timestamp(now)}"
|
f"- Time: {format_timestamp(now, with_zone=True)}"
|
||||||
),
|
),
|
||||||
priority=7,
|
priority=7,
|
||||||
)
|
)
|
||||||
@@ -240,7 +259,7 @@ def _check_one(
|
|||||||
+ ".\n\n"
|
+ ".\n\n"
|
||||||
f"- Error count: **{state.error_count}/{container_config.max_error_count}**\n"
|
f"- Error count: **{state.error_count}/{container_config.max_error_count}**\n"
|
||||||
f"- Previous status: *{state.previous_status or 'unknown'}*\n"
|
f"- Previous status: *{state.previous_status or 'unknown'}*\n"
|
||||||
f"- Time: {format_timestamp(now)}"
|
f"- Time: {format_timestamp(now, with_zone=True)}"
|
||||||
),
|
),
|
||||||
priority=5,
|
priority=5,
|
||||||
)
|
)
|
||||||
@@ -253,9 +272,9 @@ def _check_one(
|
|||||||
message=result.message,
|
message=result.message,
|
||||||
)
|
)
|
||||||
state.last_notification_state = result.status
|
state.last_notification_state = result.status
|
||||||
elif result.status == HEALTHY:
|
elif is_normal:
|
||||||
if state.error_count != 0:
|
if state.error_count != 0:
|
||||||
log.info("%-*s healthy again; resetting error count", width, name)
|
log.info("%-*s back to normal; resetting error count", width, name)
|
||||||
state.error_count = 0
|
state.error_count = 0
|
||||||
state.restarted_this_episode = False
|
state.restarted_this_episode = False
|
||||||
|
|
||||||
@@ -276,7 +295,7 @@ def _check_one(
|
|||||||
f"**{container_config.max_restarts_per_hour}** restart(s)/hour.\n\n"
|
f"**{container_config.max_restarts_per_hour}** restart(s)/hour.\n\n"
|
||||||
f"- Current status: **{result.status}**\n"
|
f"- Current status: **{result.status}**\n"
|
||||||
f"- Restarts paused until the hourly window clears\n"
|
f"- Restarts paused until the hourly window clears\n"
|
||||||
f"- Time: {format_timestamp(now)}"
|
f"- Time: {format_timestamp(now, with_zone=True)}"
|
||||||
),
|
),
|
||||||
priority=9,
|
priority=9,
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -33,9 +33,10 @@ class GotifyNotifier:
|
|||||||
},
|
},
|
||||||
}
|
}
|
||||||
try:
|
try:
|
||||||
|
# Header, not ?token=, so the token stays out of access logs.
|
||||||
resp = requests.post(
|
resp = requests.post(
|
||||||
url,
|
url,
|
||||||
params={"token": self._config.token},
|
headers={"X-Gotify-Key": self._config.token},
|
||||||
json=payload,
|
json=payload,
|
||||||
timeout=self._timeout,
|
timeout=self._timeout,
|
||||||
)
|
)
|
||||||
|
|||||||
+17
-16
@@ -175,7 +175,8 @@ def _perform_restart_then_redeploy(
|
|||||||
- the container is still failing on a later recovery cycle
|
- the container is still failing on a later recovery cycle
|
||||||
(`state.restarted_this_episode` is already set), meaning the restart didn't stick.
|
(`state.restarted_this_episode` is already set), meaning the restart didn't stick.
|
||||||
|
|
||||||
The episode flag resets when the container is next seen HEALTHY (see main._check_one).
|
The episode flag resets when the container is next seen working: HEALTHY, or
|
||||||
|
NO HEALTH CHECK when that isn't a restart trigger (see main._is_normal).
|
||||||
"""
|
"""
|
||||||
name = container_config.name
|
name = container_config.name
|
||||||
|
|
||||||
@@ -228,7 +229,7 @@ def _perform_restart_then_redeploy(
|
|||||||
f"**{result.status}** check(s).\n\n"
|
f"**{result.status}** check(s).\n\n"
|
||||||
f"- If it doesn't recover, Inspector escalates to a Portainer stack redeploy\n"
|
f"- If it doesn't recover, Inspector escalates to a Portainer stack redeploy\n"
|
||||||
f"- Cooldown: **{container_config.cooldown_seconds}s** after restart\n"
|
f"- Cooldown: **{container_config.cooldown_seconds}s** after restart\n"
|
||||||
f"- Time: {format_timestamp(now)}"
|
f"- Time: {format_timestamp(now, with_zone=True)}"
|
||||||
),
|
),
|
||||||
priority=6,
|
priority=6,
|
||||||
)
|
)
|
||||||
@@ -242,7 +243,7 @@ def _perform_restart_then_redeploy(
|
|||||||
state.restart_limit_notified = False
|
state.restart_limit_notified = False
|
||||||
log.info(
|
log.info(
|
||||||
"%-*s restart succeeded; cooldown until %s (will redeploy if still failing)",
|
"%-*s restart succeeded; cooldown until %s (will redeploy if still failing)",
|
||||||
width, name, state.cooldown_until.isoformat(),
|
width, name, format_timestamp(state.cooldown_until, with_zone=True),
|
||||||
)
|
)
|
||||||
eventstore.record(
|
eventstore.record(
|
||||||
container=name,
|
container=name,
|
||||||
@@ -250,15 +251,15 @@ def _perform_restart_then_redeploy(
|
|||||||
old_status=state.previous_status,
|
old_status=state.previous_status,
|
||||||
new_status=result.status,
|
new_status=result.status,
|
||||||
error_count=state.error_count,
|
error_count=state.error_count,
|
||||||
message=f"restart succeeded; cooldown until {state.cooldown_until.isoformat()}",
|
message=f"restart succeeded; cooldown until {format_timestamp(state.cooldown_until, with_zone=True)}",
|
||||||
)
|
)
|
||||||
notifier.notify(
|
notifier.notify(
|
||||||
title=f"✅ Inspector: {name} restart completed",
|
title=f"✅ Inspector: {name} restart completed",
|
||||||
message=(
|
message=(
|
||||||
f"**{name}** was successfully restarted by Docker.\n\n"
|
f"**{name}** was successfully restarted by Docker.\n\n"
|
||||||
f"- Previous status: *{result.status}*\n"
|
f"- Previous status: *{result.status}*\n"
|
||||||
f"- Cooldown until: {state.cooldown_until.isoformat()}\n"
|
f"- Cooldown until: {format_timestamp(state.cooldown_until, with_zone=True)}\n"
|
||||||
f"- Time: {format_timestamp(now)}"
|
f"- Time: {format_timestamp(now, with_zone=True)}"
|
||||||
),
|
),
|
||||||
priority=3,
|
priority=3,
|
||||||
)
|
)
|
||||||
@@ -321,7 +322,7 @@ def _perform_docker_action(
|
|||||||
f"{verb.capitalize()}ing **{name}** after **{state.error_count}** consecutive "
|
f"{verb.capitalize()}ing **{name}** after **{state.error_count}** consecutive "
|
||||||
f"**{result.status}** check(s).\n\n"
|
f"**{result.status}** check(s).\n\n"
|
||||||
f"- Cooldown: **{container_config.cooldown_seconds}s** after {verb}\n"
|
f"- Cooldown: **{container_config.cooldown_seconds}s** after {verb}\n"
|
||||||
f"- Time: {format_timestamp(now)}"
|
f"- Time: {format_timestamp(now, with_zone=True)}"
|
||||||
),
|
),
|
||||||
priority=6,
|
priority=6,
|
||||||
)
|
)
|
||||||
@@ -335,22 +336,22 @@ def _perform_docker_action(
|
|||||||
state.record_restart(now, container_config.cooldown_seconds)
|
state.record_restart(now, container_config.cooldown_seconds)
|
||||||
state.last_notification_state = "RESTARTED"
|
state.last_notification_state = "RESTARTED"
|
||||||
state.restart_limit_notified = False
|
state.restart_limit_notified = False
|
||||||
log.info("%-*s %s succeeded; cooldown until %s", width, name, verb, state.cooldown_until.isoformat())
|
log.info("%-*s %s succeeded; cooldown until %s", width, name, verb, format_timestamp(state.cooldown_until, with_zone=True))
|
||||||
eventstore.record(
|
eventstore.record(
|
||||||
container=name,
|
container=name,
|
||||||
event_type=EVENT_RESTART,
|
event_type=EVENT_RESTART,
|
||||||
old_status=state.previous_status,
|
old_status=state.previous_status,
|
||||||
new_status=result.status,
|
new_status=result.status,
|
||||||
error_count=state.error_count,
|
error_count=state.error_count,
|
||||||
message=f"{verb} succeeded; cooldown until {state.cooldown_until.isoformat()}",
|
message=f"{verb} succeeded; cooldown until {format_timestamp(state.cooldown_until, with_zone=True)}",
|
||||||
)
|
)
|
||||||
notifier.notify(
|
notifier.notify(
|
||||||
title=f"✅ Inspector: {name} {verb} completed",
|
title=f"✅ Inspector: {name} {verb} completed",
|
||||||
message=(
|
message=(
|
||||||
f"**{name}** was successfully {verb}ed by Docker.\n\n"
|
f"**{name}** was successfully {verb}ed by Docker.\n\n"
|
||||||
f"- Previous status: *{result.status}*\n"
|
f"- Previous status: *{result.status}*\n"
|
||||||
f"- Cooldown until: {state.cooldown_until.isoformat()}\n"
|
f"- Cooldown until: {format_timestamp(state.cooldown_until, with_zone=True)}\n"
|
||||||
f"- Time: {format_timestamp(now)}"
|
f"- Time: {format_timestamp(now, with_zone=True)}"
|
||||||
),
|
),
|
||||||
priority=3,
|
priority=3,
|
||||||
)
|
)
|
||||||
@@ -362,7 +363,7 @@ def _perform_docker_action(
|
|||||||
f"{verb.capitalize()} of **{name}** failed.\n\n"
|
f"{verb.capitalize()} of **{name}** failed.\n\n"
|
||||||
f"- Status: **{result.status}**\n"
|
f"- Status: **{result.status}**\n"
|
||||||
f"- Error count: **{state.error_count}**\n"
|
f"- Error count: **{state.error_count}**\n"
|
||||||
f"- Time: {format_timestamp(now)}\n\n"
|
f"- Time: {format_timestamp(now, with_zone=True)}\n\n"
|
||||||
f"```\n{err}\n```"
|
f"```\n{err}\n```"
|
||||||
),
|
),
|
||||||
priority=9,
|
priority=9,
|
||||||
@@ -469,7 +470,7 @@ def _perform_portainer_redeploy(
|
|||||||
f"Redeploying Portainer stack **{stack_name}** (environment **{environment}**) "
|
f"Redeploying Portainer stack **{stack_name}** (environment **{environment}**) "
|
||||||
f"to recover **{name}**.\n\n"
|
f"to recover **{name}**.\n\n"
|
||||||
f"- Status: **{result.status}**\n"
|
f"- Status: **{result.status}**\n"
|
||||||
f"- Time: {format_timestamp(now)}"
|
f"- Time: {format_timestamp(now, with_zone=True)}"
|
||||||
),
|
),
|
||||||
priority=6,
|
priority=6,
|
||||||
)
|
)
|
||||||
@@ -483,7 +484,7 @@ def _perform_portainer_redeploy(
|
|||||||
message=(
|
message=(
|
||||||
f"Portainer redeploy of **{stack_name}** (environment **{environment}**) failed.\n\n"
|
f"Portainer redeploy of **{stack_name}** (environment **{environment}**) failed.\n\n"
|
||||||
f"- Triggered by: **{name}** ({result.status})\n"
|
f"- Triggered by: **{name}** ({result.status})\n"
|
||||||
f"- Time: {format_timestamp(now)}\n\n"
|
f"- Time: {format_timestamp(now, with_zone=True)}\n\n"
|
||||||
f"```\n{redeploy_result.message}\n```"
|
f"```\n{redeploy_result.message}\n```"
|
||||||
),
|
),
|
||||||
priority=9,
|
priority=9,
|
||||||
@@ -516,7 +517,7 @@ def _perform_portainer_redeploy(
|
|||||||
f"successfully redeployed to recover **{name}**.\n\n"
|
f"successfully redeployed to recover **{name}**.\n\n"
|
||||||
f"- Triggered by: *{result.status}*\n"
|
f"- Triggered by: *{result.status}*\n"
|
||||||
f"- Post-redeploy wait: {wait_seconds}s\n"
|
f"- Post-redeploy wait: {wait_seconds}s\n"
|
||||||
f"- Time: {format_timestamp(now)}"
|
f"- Time: {format_timestamp(now, with_zone=True)}"
|
||||||
),
|
),
|
||||||
priority=4,
|
priority=4,
|
||||||
)
|
)
|
||||||
@@ -536,7 +537,7 @@ def _perform_portainer_redeploy(
|
|||||||
f"Portainer stack **{stack_name}** was redeployed, but the HTTP recheck "
|
f"Portainer stack **{stack_name}** was redeployed, but the HTTP recheck "
|
||||||
f"for **{name}** still reports a failure.\n\n"
|
f"for **{name}** still reports a failure.\n\n"
|
||||||
f"- Recheck result: *{recheck.message}*\n"
|
f"- Recheck result: *{recheck.message}*\n"
|
||||||
f"- Time: {format_timestamp(now)}"
|
f"- Time: {format_timestamp(now, with_zone=True)}"
|
||||||
),
|
),
|
||||||
priority=6,
|
priority=6,
|
||||||
)
|
)
|
||||||
|
|||||||
Reference in new issue
Block a user