From 8c3a402594801d78cf07cbb4834412ac503eda92 Mon Sep 17 00:00:00 2001 From: Alexey <247128645+axkurcom@users.noreply.github.com> Date: Tue, 25 Aug 2026 13:04:50 +0300 Subject: [PATCH] Docs for WEB Debug Co-Authored-By: brekotis <93345790+brekotis@users.noreply.github.com> --- docs/Architecture/API/API.md | 8 +++---- docs/Config_params/CONFIG_PARAMS.en.md | 28 +++++++++++++++++++++--- docs/WEB/WEB_PROXY.en.md | 30 ++++++++++++++++++++++++-- 3 files changed, 57 insertions(+), 9 deletions(-) diff --git a/docs/Architecture/API/API.md b/docs/Architecture/API/API.md index e9944ef..fc44383 100644 --- a/docs/Architecture/API/API.md +++ b/docs/Architecture/API/API.md @@ -1524,7 +1524,7 @@ The revision is verified again after preparation. With `failure_policy=rollback` ## WEB Proxy Management -The API provides partial operational control for WEB mode; it does not expose a dedicated `/v1/web` resource. +The API provides partial operational control for WEB mode. It does not expose a mutable `/v1/web` resource, but it serves bounded read-only HTML diagnostics at `GET /web-status`. | Operation | Current contract | | --- | --- | @@ -1535,13 +1535,13 @@ The API provides partial operational control for WEB mode; it does not expose a | Manage access users | Use `/v1/users`. Creating a user does not add it to `web.vhosts.profiles`; profile membership remains file-managed. | | Disable one user | `POST /v1/users/{username}/disable` updates admission immediately and cancels the user's active sessions. | | Rotate a profiled user's secret | Use `/v1/users/{username}/rotate-secret`; the config watcher rebuilds WEB capabilities from the new access snapshot. The API returns the secret, not a `tg://webproxy` link. | -| Read WEB-specific runtime statistics | No WEB-specific endpoint exists in the current API surface. | +| Read WEB-specific runtime diagnostics | Use authenticated `GET /web-status`; filters cover client IP, process session ID, User-Agent, and non-secret key fingerprint, with optional grouping and expandable request-to-response details. | -`web.enabled`, `web.carrier`, `web.timeouts`, vhosts, profiles, and decoy snapshots are runtime-generation fields. A changed carrier applies only to newly issued bridge sessions; existing sessions retain their creation-time carrier. WEB listener inventory and trust policy, plus all `[web.limits]`, are process-owned. A successful reload can therefore activate the runtime-owned subset while reporting the process-owned subset as deferred. +`web.enabled`, `web.carrier`, `web.debug`, `web.timeouts`, vhosts, profiles, and decoy snapshots are runtime-generation fields. A changed carrier applies only to newly issued bridge sessions; existing sessions retain their creation-time carrier. WEB listener inventory and trust policy, plus all `[web.limits]`, are process-owned. A successful reload can therefore activate the runtime-owned subset while reporting the process-owned subset as deferred. Before deleting a user referenced by a WEB profile, remove and apply the profile first. User mutations validate the complete resulting configuration, so a dangling WEB profile is rejected rather than persisted. -The API whitelist is evaluated against the direct TCP peer and does not use the WEB listener's `X-Forwarded-For` policy. Keep the API on a separate loopback or private bind, use a narrow whitelist and a non-empty exact `auth_header`, and do not expose it through the public WEB vhost. +The API whitelist is evaluated against the direct TCP peer and does not use the WEB listener's `X-Forwarded-For` policy. `/web-status` inherits API enablement, whitelist, gray action, and exact authorization-header checks; it accepts only `GET`, normalizes a trailing slash, sets `no-store` and restrictive browser security headers, caps each page at 8 MiB, and permits at most two concurrent renderers. Keep the API on a separate loopback or private bind, use a narrow whitelist and a non-empty exact `auth_header`, and do not expose it through the public WEB vhost. Deployment, TLS-terminator examples, links, and WEB-specific verification are documented in the [WEB proxy guide](../../WEB/WEB_PROXY.en.md). diff --git a/docs/Config_params/CONFIG_PARAMS.en.md b/docs/Config_params/CONFIG_PARAMS.en.md index a5c0dd3..98cacab 100644 --- a/docs/Config_params/CONFIG_PARAMS.en.md +++ b/docs/Config_params/CONFIG_PARAMS.en.md @@ -2557,12 +2557,32 @@ WEB mode carries Telegram Desktop MTProxy traffic through HTTPS terminated by an | --- | --- | --- | --- | | `enabled` | `bool` | `false` | `✔` | | `carrier` | `"https"` or `"https-lanes"` | `"https"` | `✔` | +| `debug` | table | disabled, bounded defaults | `✔` | | `limits` | table | bounded defaults | `✘` | | `timeouts` | table | bounded defaults | `✔` | | `vhosts` | array of tables | `[]` | `✔` | `enabled = true` requires at least one network-eligible WEB listener, at least one vhost, and at least one profile in every vhost. `carrier = "https"` preserves the serialized HTTPS transport. `carrier = "https-lanes"` gives stream zero and every logical stream independent uplink sequencing, downlink cursors, retries, and long polls; it requires `max_http_handlers >= 2` and public HTTP/2 on the TLS terminator to remove application-level inter-stream head-of-line blocking. A reload applies `carrier` only to newly issued bridge sessions. Disabling WEB stops issuance of new bridge and session credentials after reload; use the users API to revoke one user's active sessions. +# [web.debug] + +This hot-reloadable table controls the process-owned server-side WEB debug recorder exposed as authenticated HTML at `GET /web-status` on the API listener. Collection is disabled by default. Retained and in-flight records remain bounded by restart-only values in `[web.limits]`. + +| Key | Type | Default | Description | +| --- | --- | --- | --- | +| `enabled` | `bool` | `false` | Enables WEB HTTP, frame, and lifecycle debug records. | +| `capture_lifecycle` | `bool` | `true` | Records typed bridge, session, stream, handshake, relay, and close events. | +| `capture_headers` | `bool` | `true` | Retains header names and only allowlisted non-credential values. | +| `capture_timings` | `bool` | `true` | Retains request-body, response-ready, and response-body Hyper timing points. | +| `capture_frames` | `bool` | `true` | Parses bounded carrier bodies into frame type, stream ID, length, WINDOW, and error metadata without retaining frame payload separately. | +| `body_capture` | `"off"`, `"metadata"`, `"prefix"`, or `"full"` | `"metadata"` | Controls request and response body byte retention. | +| `body_prefix_bytes` | `usize` | `4096` | Prefix retained for recognized WEB bodies in `prefix` mode. | +| `decoy_body_prefix_bytes` | `usize` | `4096` | Maximum retained prefix for ordinary decoy traffic in both `prefix` and `full` modes. | +| `default_window_secs` | `u64` | `180` | Default observation window rendered by `/web-status`. | +| `max_window_secs` | `u64` | `3600` | Largest observation window accepted by `/web-status`; validated at no more than 86400. | + +Changing `enabled` or any capture field clears retained records and rejects commits started under the previous policy epoch. Changing only the default or maximum observation window preserves compatible retained records. `full` retains a complete recognized carrier body only up to `web.limits.max_body_bytes`; decoy bodies always remain prefix-bounded. A prefix that depends on a simultaneously increased restart-only capacity is deferred with `web.debug` until restart. URI queries are never retained, credential header values are omitted, body copies are scrubbed for known WEB capabilities and bearer tokens, and profile keys are represented only by a domain-separated 16-hex fingerprint. + # [web.limits] These process-wide ceilings make every WEB registry, queue, request body, static snapshot, and admission path bounded. All values are validated together. Per-owner limits cannot exceed global limits, queue reserves must preserve control-frame progress, body reservations must fit their global budget, and all declared byte ceilings must fit `memory_envelope_bytes`. Changing any value in this table requires a process restart. @@ -2597,7 +2617,9 @@ These process-wide ceilings make every WEB registry, queue, request body, static | `max_static_files` | `usize` | `4096` | Static snapshot entries across all vhosts. | | `max_static_file_bytes` | `usize` | `8388608` | Maximum bytes in one static file. | | `max_static_bytes` | `usize` | `67108864` | Static snapshot bytes across all vhosts. | -| `memory_envelope_bytes` | `usize` | `805306368` | Declared envelope for HTTP heads, bodies, queues, and static snapshots; maximum 4 GiB. | +| `debug_records_capacity` | `usize` | `65536` | Maximum retained WEB debug record count. | +| `debug_bytes_global` | `usize` | `67108864` | Retained plus in-flight WEB debug byte ceiling; minimum 4096. | +| `memory_envelope_bytes` | `usize` | `805306368` | Declared envelope for HTTP heads, bodies, queues, static snapshots, and bounded debug/status buffers; maximum 4 GiB. | | `new_bootstraps_per_minute` | `u32` | `1200` | Sustained process-wide bootstrap issuance rate. | | `new_bootstraps_burst` | `u32` | `256` | Process-wide bootstrap issuance burst. | | `new_sessions_per_minute` | `u32` | `600` | Sustained process-wide session creation rate. | @@ -2655,9 +2677,9 @@ Profile limits must be non-zero and no greater than their corresponding global l ## WEB lifecycle and API management -- The config watcher and generation reload apply `web.enabled`, `web.carrier`, `web.timeouts`, vhosts, profiles, and decoy snapshots without a process restart. Existing sessions keep their acquisition-time carrier, limits, and deadlines; newly issued bridge sessions use the active generation. +- The config watcher and generation reload apply `web.enabled`, `web.carrier`, `web.debug`, `web.timeouts`, vhosts, profiles, and decoy snapshots without a process restart. Existing sessions keep their acquisition-time carrier, limits, and deadlines; newly issued bridge sessions use the active generation. - WEB listener inventory and trust policy under `server.listeners`, and every `web.limits` value, are process-owned and restart-required. -- There is no dedicated `/v1/web` endpoint. `GET /v1/config` omits `[web]`, and `PATCH /v1/config` rejects a `web` key with `400 section_not_editable`. +- There is no mutable `/v1/web` resource. `GET /web-status` provides authenticated read-only HTML diagnostics; `GET /v1/config` omits `[web]`, and `PATCH /v1/config` rejects a `web` key with `400 section_not_editable`. - To manage WEB policy remotely, update the owned TOML file and call `POST /v1/system/reload`; inspect `GET /v1/system/reload/{id}` and its `deferred_process_fields`. Restart Telemt when it contains `server.listeners` or `web.limits`. - Existing access users can be created, changed, rotated, enabled, disabled, and deleted through `/v1/users`. Creating a user does not add a WEB profile. Disabling a user immediately updates admission and cancels that user's active sessions. - `PATCH /v1/config` can persist `server.listeners`, including WEB listener fields, but a changed WEB listener does not become active until process restart. diff --git a/docs/WEB/WEB_PROXY.en.md b/docs/WEB/WEB_PROXY.en.md index 1bf9592..7d926bf 100644 --- a/docs/WEB/WEB_PROXY.en.md +++ b/docs/WEB/WEB_PROXY.en.md @@ -186,7 +186,7 @@ The frontend or `defaults` section must also set `timeout client` above the long | --- | --- | | WEB listener inventory, bind address, and trust policy | Process-owned; restart Telemt. | | Any `[web.limits]` value | Process-owned memory/resource contract; restart Telemt. | -| `web.enabled`, `web.carrier`, timeouts, vhosts, profiles, and decoys | Applied by the config watcher or a runtime generation reload. | +| `web.enabled`, `web.carrier`, `web.debug`, timeouts, vhosts, profiles, and decoys | Applied by the config watcher or a runtime generation reload. | | Existing HTTP connections and WEB sessions | Keep their acquisition-time carrier, limits, and deadlines; newly issued bridge sessions use the active carrier. New logical streams use the active relay generation. | | Process shutdown | Uses the latest reloaded `web.timeouts.shutdown_secs`. | @@ -194,13 +194,14 @@ Each logical stream keeps its session's creation-time client IP and owns a proce ## API management -API management is available, but it is intentionally partial. There is no dedicated `/v1/web` endpoint and no WEB-specific runtime statistics endpoint. +API management is available, but it is intentionally partial. There is no mutable `/v1/web` resource; the API listener exposes the read-only HTML debug view at `/web-status`. | Operation | API support | | --- | --- | | Read or patch `[web]`, vhosts, profiles, decoys, timeouts, or limits | No. `GET /v1/config` omits `[web]`; `PATCH /v1/config` returns `400 section_not_editable` for `web`. | | Persist `server.listeners` | Yes, through `PATCH /v1/config`, but a changed WEB listener remains deferred until process restart. | | Apply an externally edited WEB configuration | Yes, through `POST /v1/system/reload`, then inspect the operation status. | +| Inspect bounded server-side WEB request and lifecycle details | Yes, through authenticated `GET /web-status`. | | Manage `[access.users]` | Yes, through `/v1/users`. User creation does not create a WEB profile. | | Revoke one user | Yes. `/v1/users/{username}/disable` updates admission immediately and cancels that user's active sessions. | @@ -217,6 +218,30 @@ read_only = false The API whitelist checks the direct TCP peer and does not trust `X-Forwarded-For`. Changes to `[server.api]` itself require a process restart. +### Server-side WEB debug view + +Enable bounded collection in the owned configuration file: + +```toml +[web.debug] +enabled = true +capture_lifecycle = true +capture_headers = true +capture_timings = true +capture_frames = true +body_capture = "metadata" +body_prefix_bytes = 4096 +decoy_body_prefix_bytes = 4096 +default_window_secs = 180 +max_window_secs = 3600 +``` + +Open `http://127.0.0.1:9091/web-status` with the same direct-peer whitelist and exact `Authorization` header used by the API. A trailing slash is accepted. Only `GET` is allowed. The page supports `window_secs`, canonical `ip`, numeric `session`, case-insensitive `user_agent`, and `key` filters. Repeat `group_by=ip`, `group_by=session`, `group_by=user_agent`, or `group_by=key` to build grouped summaries; `limit` is restricted to `1..=1000`. Each row links to an exact-record view and expands into method, path, sanitized headers, body metadata or bytes, timing points, parsed frames, and typed lifecycle events. + +The process-owned ring survives runtime generation replacement. Capture-policy changes clear incompatible retained records; window-only changes do not. The ring defaults to 65536 records and 64 MiB retained plus in-flight bytes, the HTML response is capped at 8 MiB, grouping is capped at 1024 groups, and no more than two response bodies retain page permits concurrently. Change `web.limits.debug_records_capacity` or `web.limits.debug_bytes_global` only with a process restart. A hot prefix that fits only a simultaneously increased restart-only capacity is deferred until that restart. + +`body_capture = "off"` omits bodies, `metadata` retains lengths and terminal states, `prefix` retains configured prefixes, and `full` retains recognized carrier bodies up to `web.limits.max_body_bytes`. Ordinary decoy bodies remain limited by `decoy_body_prefix_bytes` even in `full` mode. Queries and raw capabilities are never stored; credential header values are omitted; known WEB capabilities and bearer tokens are scrubbed from captured bodies; the displayed key is a non-secret domain-separated fingerprint. Timing ends at Hyper body polling and does not claim kernel flush or TCP acknowledgment. + After an administrator or configuration system atomically updates the TOML file, set `TELEMT_API_AUTH` to the exact value configured in `auth_header` and submit an observable generation reload: ```bash @@ -274,6 +299,7 @@ See the complete [Control API contract](../Architecture/API/API.md) for request | WEB configuration is valid on disk but listener behavior did not change | Inspect reload `deferred_process_fields`; listener and `[web.limits]` changes require restart. | | Carrier requests reach the decoy | Verify exact vhost, link secret mode, direct proxy CIDR, and one parseable `X-Forwarded-For` value. | | Long polls disconnect near a fixed interval | Raise NGINX/HAProxy client, server, send, and read timeouts above `web.timeouts.long_poll_secs`. | +| `/web-status` is empty | Confirm `[web.debug].enabled = true`, apply the configuration, select a window within `max_window_secs`, and generate new WEB traffic after the policy change. | | `https-lanes` works but streams still block each other | Confirm public HTTP/2 negotiation, preserve `X-Lane-ID`, and provide enough TLS-terminator upstream connections for concurrent private HTTP/1.1 polls. | | Telegram Desktop rejects the link | Omit the port, use a valid FQDN, port 443 externally, and only `plain` or `dd` secret mode. | | One node works but a load-balanced pool is intermittent | Add complete-vhost affinity; WEB credential registries are process-local. |