diff --git a/.github/instructions/Architecture.instructions.md b/.github/instructions/Architecture.instructions.md new file mode 100644 index 0000000..42459dc --- /dev/null +++ b/.github/instructions/Architecture.instructions.md @@ -0,0 +1,126 @@ +# Architecture Directives + +> Companion to `Agents.md`. These are **activation directives**, not tutorials. +> You already know these patterns — apply them. When making any structural or +> design decision, run the relevant section below as a checklist. + +--- + +## 1. Active Principles (always on) + +Apply these on every non-trivial change. No exceptions. + +- **SRP** — one reason to change per component. If you can't name the responsibility in one noun phrase, split it. +- **OCP** — extend by adding, not by modifying. New variants/impls over patching existing logic. +- **ISP** — traits stay minimal. More than ~5 methods is a split signal. +- **DIP** — high-level modules depend on traits, not concrete types. Infrastructure implements domain traits; it does not own domain logic. +- **DRY** — one authoritative source per piece of knowledge. Copies are bugs that haven't diverged yet. +- **YAGNI** — generic parameters, extension hooks, and pluggable strategies require an *existing* concrete use case, not a hypothetical one. +- **KISS** — two equivalent designs: choose the one with fewer concepts. Justify complexity; never assume it. + +--- + +## 2. Layered Architecture + +Dependencies point **inward only**: `Presentation → Application → Domain ← Infrastructure`. + +- Domain layer: zero I/O. No network, no filesystem, no async runtime imports. +- Infrastructure: implements domain traits at the boundary. Never leaks SDK/wire types inward. +- Anti-Corruption Layer (ACL): all third-party and external-protocol types are translated here. If the external format changes, only the ACL changes. +- Presentation: translates wire/HTTP representations to domain types and back. Nothing else. + +--- + +## 3. Design Pattern Selection + +Apply the right pattern. Do not invent a new abstraction when a named pattern fits. + +| Situation | Pattern to apply | +|---|---| +| Struct with 3+ optional/dependent fields | **Builder** — `build()` returns `Result`, never panics | +| Cross-cutting behavior (logging, retry, metrics) on a trait impl | **Decorator** — implements same trait, delegates all calls | +| Subsystem with multiple internal components | **Façade** — single public entry point, internals are `pub(crate)` | +| Swappable algorithm or policy | **Strategy** — trait injection; generics for compile-time, `dyn` for runtime | +| Component notifying decoupled consumers | **Observer** — typed channels (`broadcast`, `watch`), not callback `Vec>` | +| Exclusive mutable state serving concurrent callers | **Actor** — `mpsc` command channel + `oneshot` reply; no lock needed on state | +| Finite state with invalid transition prevention | **Typestate** — distinct types per state; invalid ops are compile errors | +| Fixed process skeleton with overridable steps | **Template Method** — defaulted trait method calls required hooks | +| Request pipeline with independent handlers | **Chain/Middleware** — generic compile-time chain for hot paths, `dyn` for runtime assembly | +| Hiding a concrete type behind a trait | **Factory Function** — returns `Box` or `impl Trait` | + +--- + +## 4. Data Modeling Rules + +- **Make illegal states unrepresentable.** Type system enforces invariants; runtime validation is a second line, not the first. +- **Newtype every primitive** that carries domain meaning. `SessionId(u64)` ≠ `UserId(u64)` — the compiler enforces it. +- **Enums over booleans** for any parameter or field with two or more named states. +- **Typed error enums** with named variants carrying full diagnostic context. `anyhow` is application-layer only; never in library code. +- **Domain types carry no I/O concerns.** No `serde`, no codec, no DB derives on domain structs. Conversions via `From`/`TryFrom` at layer boundaries. + +--- + +## 5. Concurrency Rules + +- Prefer message-passing over shared memory. Shared state is a fallback. +- All channels must be **bounded**. Document the bound's rationale inline. +- Never hold a lock across an `await` unless atomicity explicitly requires it — document why. +- Document lock acquisition order wherever two locks are taken together. +- Every `async fn` is cancellation-safe unless explicitly documented otherwise. Mutate shared state *after* the `await` that may be cancelled, not before. +- High-read/low-write state: use `arc-swap` or `watch` for lock-free reads. + +--- + +## 6. Error Handling Rules + +- Errors translated at every layer boundary — low-level errors never surface unmodified. +- Add context at the propagation site: what operation failed and where. +- No `unwrap()`/`expect()` in production paths without a comment proving `None`/`Err` is impossible. +- Panics are only permitted in: tests, startup/init unrecoverable failure, and `unreachable!()` with an invariant comment. + +--- + +## 7. API Design Rules + +- **CQS**: functions that return data must not mutate; functions that mutate return only `Result`. +- **Least surprise**: a function does exactly what its name implies. Side effects are documented. +- **Idempotency**: `close()`, `shutdown()`, `unregister()` called twice must not panic or error. +- **Fallibility at the type level**: failure → `Result`. No sentinel values. +- **Minimal public surface**: default to `pub(crate)`. Mark `pub` only deliberate API. Re-export through a single surface in `mod.rs`. + +--- + +## 8. Performance Rules (hot paths) + +- Annotate hot-path functions with `// HOT PATH: `. +- Zero allocations per operation in hot paths after initialization. Preallocate in constructors, reuse buffers. +- Pass `&[u8]` / `Bytes` slices — not `Vec`. Use `BytesMut` for reusable mutable buffers. +- No `String` formatting in hot paths. No logging without a rate-limit or sampling gate. +- Any allocation in a hot path gets a comment: `// ALLOC: `. + +--- + +## 9. Testing Rules + +- Bug fixes require a regression test that is **red before the fix, green after**. Name it after the bug. +- Property tests for: codec round-trips, state machine invariants, cryptographic protocol correctness. +- No shared mutable state between tests. Each test constructs its own environment. +- Test doubles hierarchy (simplest first): Fake → Stub → Spy → Mock. Mocks couple to implementation, not behavior — use sparingly. + +--- + +## 10. Pre-Change Checklist + +Run this before proposing or implementing any structural decision: + +- [ ] Responsibility nameable in one noun phrase? +- [ ] Layer dependencies point inward only? +- [ ] Invalid states unrepresentable in the type system? +- [ ] State transitions gated through a single interface? +- [ ] All channels bounded? +- [ ] No locks held across `await` (or documented)? +- [ ] Errors typed and translated at layer boundaries? +- [ ] No panics in production paths without invariant proof? +- [ ] Hot paths annotated and allocation-free? +- [ ] Public surface minimal — only deliberate API marked `pub`? +- [ ] Correct pattern chosen from Section 3 table? \ No newline at end of file diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 1b6e455..d664174 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -36,4 +36,10 @@ jobs: ${{ runner.os }}-cargo- - name: Build Release - run: cargo build --release --verbose \ No newline at end of file + run: cargo build --release --verbose + + - name: Upload binary artifact + uses: actions/upload-artifact@v4 + with: + name: telemt + path: target/release/telemt \ No newline at end of file diff --git a/.github/workflows/test.yml b/.github/workflows/check.yml similarity index 100% rename from .github/workflows/test.yml rename to .github/workflows/check.yml diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 5e21a85..76e01fa 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -151,6 +151,14 @@ jobs: mkdir -p dist cp "target/${{ matrix.target }}/release/${{ env.BINARY_NAME }}" dist/telemt + if [ "${{ matrix.target }}" = "aarch64-unknown-linux-gnu" ]; then + STRIP_BIN=aarch64-linux-gnu-strip + else + STRIP_BIN=strip + fi + + "${STRIP_BIN}" dist/telemt + cd dist tar -czf "${{ matrix.asset }}.tar.gz" \ --owner=0 --group=0 --numeric-owner \ @@ -279,6 +287,14 @@ jobs: mkdir -p dist cp "target/${{ matrix.target }}/release/${{ env.BINARY_NAME }}" dist/telemt + if [ "${{ matrix.target }}" = "aarch64-unknown-linux-musl" ]; then + STRIP_BIN=aarch64-linux-musl-strip + else + STRIP_BIN=strip + fi + + "${STRIP_BIN}" dist/telemt + cd dist tar -czf "${{ matrix.asset }}.tar.gz" \ --owner=0 --group=0 --numeric-owner \ diff --git a/.kilocode/rules-architect/AGENTS.md b/.kilocode/rules-architect/AGENTS.md deleted file mode 100644 index 84e8808..0000000 --- a/.kilocode/rules-architect/AGENTS.md +++ /dev/null @@ -1,58 +0,0 @@ -# Architect Mode Rules for Telemt - -## Architecture Overview - -```mermaid -graph TB - subgraph Entry - Client[Clients] --> Listener[TCP/Unix Listener] - end - - subgraph Proxy Layer - Listener --> ClientHandler[ClientHandler] - ClientHandler --> Handshake[Handshake Validator] - Handshake --> |Valid| Relay[Relay Layer] - Handshake --> |Invalid| Masking[Masking/TLS Fronting] - end - - subgraph Transport - Relay --> MiddleProxy[Middle-End Proxy Pool] - Relay --> DirectRelay[Direct DC Relay] - MiddleProxy --> TelegramDC[Telegram DCs] - DirectRelay --> TelegramDC - end -``` - -## Module Dependencies -- [`src/main.rs`](src/main.rs) - Entry point, spawns all async tasks -- [`src/config/`](src/config/) - Configuration loading with auto-migration -- [`src/error.rs`](src/error.rs) - Error types, must be used by all modules -- [`src/crypto/`](src/crypto/) - AES, SHA, random number generation -- [`src/protocol/`](src/protocol/) - MTProto constants, frame encoding, obfuscation -- [`src/stream/`](src/stream/) - Stream wrappers, buffer pool, frame codecs -- [`src/proxy/`](src/proxy/) - Client handling, handshake, relay logic -- [`src/transport/`](src/transport/) - Upstream management, middle-proxy, SOCKS support -- [`src/stats/`](src/stats/) - Statistics and replay protection -- [`src/ip_tracker.rs`](src/ip_tracker.rs) - Per-user IP tracking - -## Key Architectural Constraints - -### Middle-End Proxy Mode -- Requires public IP on interface OR 1:1 NAT with STUN probing -- Uses separate `proxy-secret` from Telegram (NOT user secrets) -- Falls back to direct mode automatically on STUN mismatch - -### TLS Fronting -- Invalid handshakes are transparently proxied to `mask_host` -- This is critical for DPI evasion - do not change this behavior -- `mask_unix_sock` and `mask_host` are mutually exclusive - -### Stream Architecture -- Buffer pool is shared globally via Arc - prevents allocation storms -- Frame codecs implement tokio-util Encoder/Decoder traits -- State machine in [`src/stream/state.rs`](src/stream/state.rs) manages stream transitions - -### Configuration Migration -- [`ProxyConfig::load()`](src/config/mod.rs:641) mutates config in-place -- New fields must have sensible defaults -- DC203 override is auto-injected for CDN/media support diff --git a/.kilocode/rules-code/AGENTS.md b/.kilocode/rules-code/AGENTS.md deleted file mode 100644 index df9f664..0000000 --- a/.kilocode/rules-code/AGENTS.md +++ /dev/null @@ -1,23 +0,0 @@ -# Code Mode Rules for Telemt - -## Error Handling -- Always use [`ProxyError`](src/error.rs:168) from [`src/error.rs`](src/error.rs) for proxy operations -- [`HandshakeResult`](src/error.rs:292) returns streams on bad client - these MUST be returned for masking, never dropped -- Use [`Recoverable`](src/error.rs:110) trait to check if errors are retryable - -## Configuration Changes -- [`ProxyConfig::load()`](src/config/mod.rs:641) auto-mutates config - new fields should have defaults -- DC203 override is auto-injected if missing - do not remove this behavior -- When adding config fields, add migration logic in [`ProxyConfig::load()`](src/config/mod.rs:641) - -## Crypto Code -- [`SecureRandom`](src/crypto/random.rs) from [`src/crypto/random.rs`](src/crypto/random.rs) must be used for all crypto operations -- Never use `rand::thread_rng()` directly - use the shared `Arc` - -## Stream Handling -- Buffer pool [`BufferPool`](src/stream/buffer_pool.rs) is shared via Arc - always use it instead of allocating -- Frame codecs in [`src/stream/frame_codec.rs`](src/stream/frame_codec.rs) implement tokio-util's Encoder/Decoder traits - -## Testing -- Tests are inline in modules using `#[cfg(test)]` -- Use `cargo test --lib ` to run tests for specific modules diff --git a/.kilocode/rules-debug/AGENTS.md b/.kilocode/rules-debug/AGENTS.md deleted file mode 100644 index 9d390b1..0000000 --- a/.kilocode/rules-debug/AGENTS.md +++ /dev/null @@ -1,27 +0,0 @@ -# Debug Mode Rules for Telemt - -## Logging -- `RUST_LOG` environment variable takes absolute priority over all config log levels -- Log levels: `trace`, `debug`, `info`, `warn`, `error` -- Use `RUST_LOG=debug cargo run` for detailed operational logs -- Use `RUST_LOG=trace cargo run` for full protocol-level debugging - -## Middle-End Proxy Debugging -- Set `ME_DIAG=1` environment variable for high-precision cryptography diagnostics -- STUN probe results are logged at startup - check for mismatch between local and reflected IP -- If Middle-End fails, check `proxy_secret_path` points to valid file from https://core.telegram.org/getProxySecret - -## Connection Issues -- DC connectivity is logged at startup with RTT measurements -- If DC ping fails, check `dc_overrides` for custom addresses -- Use `prefer_ipv6=false` in config if IPv6 is unreliable - -## TLS Fronting Issues -- Invalid handshakes are proxied to `mask_host` - check this host is reachable -- `mask_unix_sock` and `mask_host` are mutually exclusive - only one can be set -- If `mask_unix_sock` is set, socket must exist before connections arrive - -## Common Errors -- `ReplayAttack` - client replayed a handshake nonce, potential attack -- `TimeSkew` - client clock is off, can disable with `ignore_time_skew=true` -- `TgHandshakeTimeout` - upstream DC connection failed, check network diff --git a/Cargo.lock b/Cargo.lock index 6846aba..74dfec8 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -183,9 +183,9 @@ dependencies = [ [[package]] name = "aws-lc-sys" -version = "0.39.0" +version = "0.39.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1fa7e52a4c5c547c741610a2c6f123f3881e409b714cd27e6798ef020c514f0a" +checksum = "83a25cf98105baa966497416dbd42565ce3a8cf8dbfd59803ec9ad46f3126399" dependencies = [ "cc", "cmake", @@ -234,16 +234,16 @@ checksum = "843867be96c8daad0d758b57df9392b6d8d271134fce549de6ce169ff98a92af" [[package]] name = "blake3" -version = "1.8.3" +version = "1.8.4" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2468ef7d57b3fb7e16b576e8377cdbde2320c60e1491e961d11da40fc4f02a2d" +checksum = "4d2d5991425dfd0785aed03aedcf0b321d61975c9b5b3689c774a2610ae0b51e" dependencies = [ "arrayref", "arrayvec", "cc", "cfg-if", "constant_time_eq", - "cpufeatures 0.2.17", + "cpufeatures 0.3.0", ] [[package]] @@ -299,9 +299,9 @@ dependencies = [ [[package]] name = "cc" -version = "1.2.57" +version = "1.2.58" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7a0dd1ca384932ff3641c8718a02769f1698e7563dc6974ffd03346116310423" +checksum = "e1e928d4b69e3077709075a938a05ffbedfa53a84c8f766efbf8220bb1ff60e1" dependencies = [ "find-msvc-tools", "jobserver", @@ -441,9 +441,9 @@ checksum = "c8d4a3bb8b1e0c1050499d1815f5ab16d04f0959b233085fb31653fbfc9d98f9" [[package]] name = "cmake" -version = "0.1.57" +version = "0.1.58" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "75443c44cd6b379beb8c5b45d85d0773baf31cce901fe7bb252f4eff3008ef7d" +checksum = "c0f78a02292a74a88ac736019ab962ece0bc380e3f977bf72e376c5d78ff0678" dependencies = [ "cc", ] @@ -1191,9 +1191,9 @@ checksum = "df3b46402a9d5adb4c86a0cf463f42e19994e3ee891101b1841f30a545cb49a9" [[package]] name = "hyper" -version = "1.8.1" +version = "1.9.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2ab2d4f250c3d7b1c9fcdff1cece94ea4e2dfbec68614f7b87cb205f24ca9d11" +checksum = "6299f016b246a94207e63da54dbe807655bf9e00044f73ded42c3ac5305fbcca" dependencies = [ "atomic-waker", "bytes", @@ -1206,7 +1206,6 @@ dependencies = [ "httpdate", "itoa", "pin-project-lite", - "pin-utils", "smallvec", "tokio", "want", @@ -1245,7 +1244,7 @@ dependencies = [ "libc", "percent-encoding", "pin-project-lite", - "socket2 0.6.3", + "socket2", "tokio", "tower-service", "tracing", @@ -1277,12 +1276,13 @@ dependencies = [ [[package]] name = "icu_collections" -version = "2.1.1" +version = "2.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "4c6b649701667bbe825c3b7e6388cb521c23d88644678e83c0c4d0a621a34b43" +checksum = "2984d1cd16c883d7935b9e07e44071dca8d917fd52ecc02c04d5fa0b5a3f191c" dependencies = [ "displaydoc", "potential_utf", + "utf8_iter", "yoke", "zerofrom", "zerovec", @@ -1290,9 +1290,9 @@ dependencies = [ [[package]] name = "icu_locale_core" -version = "2.1.1" +version = "2.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "edba7861004dd3714265b4db54a3c390e880ab658fec5f7db895fae2046b5bb6" +checksum = "92219b62b3e2b4d88ac5119f8904c10f8f61bf7e95b640d25ba3075e6cac2c29" dependencies = [ "displaydoc", "litemap", @@ -1303,9 +1303,9 @@ dependencies = [ [[package]] name = "icu_normalizer" -version = "2.1.1" +version = "2.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "5f6c8828b67bf8908d82127b2054ea1b4427ff0230ee9141c54251934ab1b599" +checksum = "c56e5ee99d6e3d33bd91c5d85458b6005a22140021cc324cea84dd0e72cff3b4" dependencies = [ "icu_collections", "icu_normalizer_data", @@ -1317,15 +1317,15 @@ dependencies = [ [[package]] name = "icu_normalizer_data" -version = "2.1.1" +version = "2.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7aedcccd01fc5fe81e6b489c15b247b8b0690feb23304303a9e560f37efc560a" +checksum = "da3be0ae77ea334f4da67c12f149704f19f81d1adf7c51cf482943e84a2bad38" [[package]] name = "icu_properties" -version = "2.1.2" +version = "2.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "020bfc02fe870ec3a66d93e677ccca0562506e5872c650f893269e08615d74ec" +checksum = "bee3b67d0ea5c2cca5003417989af8996f8604e34fb9ddf96208a033901e70de" dependencies = [ "icu_collections", "icu_locale_core", @@ -1337,15 +1337,15 @@ dependencies = [ [[package]] name = "icu_properties_data" -version = "2.1.2" +version = "2.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "616c294cf8d725c6afcd8f55abc17c56464ef6211f9ed59cccffe534129c77af" +checksum = "8e2bbb201e0c04f7b4b3e14382af113e17ba4f63e2c9d2ee626b720cbce54a14" [[package]] name = "icu_provider" -version = "2.1.1" +version = "2.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "85962cf0ce02e1e0a629cc34e7ca3e373ce20dda4c4d7294bbd0bf1fdb59e614" +checksum = "139c4cf31c8b5f33d7e199446eff9c1e02decfc2f0eec2c8d71f65befa45b421" dependencies = [ "displaydoc", "icu_locale_core", @@ -1427,14 +1427,15 @@ dependencies = [ [[package]] name = "ipconfig" -version = "0.3.2" +version = "0.3.4" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b58db92f96b720de98181bbbe63c831e87005ab460c1bf306eb2622b4707997f" +checksum = "4d40460c0ce33d6ce4b0630ad68ff63d6661961c48b6dba35e5a4d81cfb48222" dependencies = [ - "socket2 0.5.10", + "socket2", "widestring", - "windows-sys 0.48.0", - "winreg", + "windows-registry", + "windows-result", + "windows-sys 0.61.2", ] [[package]] @@ -1454,9 +1455,9 @@ dependencies = [ [[package]] name = "iri-string" -version = "0.7.11" +version = "0.7.12" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d8e7418f59cc01c88316161279a7f665217ae316b388e58a0d10e29f54f1e5eb" +checksum = "25e659a4bb38e810ebc252e53b5814ff908a8c58c2a9ce2fae1bbec24cbf4e20" dependencies = [ "memchr", "serde", @@ -1533,10 +1534,12 @@ dependencies = [ [[package]] name = "js-sys" -version = "0.3.91" +version = "0.3.94" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b49715b7073f385ba4bc528e5747d02e66cb39c6146efb66b781f131f0fb399c" +checksum = "2e04e2ef80ce82e13552136fabeef8a5ed1f985a96805761cbb9a2c34e7664d9" dependencies = [ + "cfg-if", + "futures-util", "once_cell", "wasm-bindgen", ] @@ -1575,9 +1578,9 @@ checksum = "09edd9e8b54e49e587e4f6295a7d29c3ea94d469cb40ab8ca70b288248a81db2" [[package]] name = "libc" -version = "0.2.183" +version = "0.2.184" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b5b646652bf6661599e1da8901b3b9522896f01e736bad5f723fe7a3a27f899d" +checksum = "48f5d2a454e16a5ea0f4ced81bd44e4cfc7bd3a507b61887c99fd3538b28e4af" [[package]] name = "linux-raw-sys" @@ -1587,9 +1590,9 @@ checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53" [[package]] name = "litemap" -version = "0.8.1" +version = "0.8.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6373607a59f0be73a39b6fe456b8192fcc3585f602af20751600e974dd455e77" +checksum = "92daf443525c4cce67b150400bc2316076100ce0b3686209eb8cf3c31612e6f0" [[package]] name = "lock_api" @@ -1669,9 +1672,9 @@ checksum = "68354c5c6bd36d73ff3feceb05efa59b6acb7626617f4962be322a825e61f79a" [[package]] name = "mio" -version = "1.1.1" +version = "1.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a69bcab0ad47271a0234d9422b131806bf3968021e5dc9328caf2d4cd58557fc" +checksum = "50b7e5b27aa02a74bac8c3f23f448f8d87ff11f92d3aac1a6ed369ee08cc56c1" dependencies = [ "libc", "log", @@ -1767,9 +1770,9 @@ dependencies = [ [[package]] name = "num-conv" -version = "0.2.0" +version = "0.2.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "cf97ec579c3c42f953ef76dbf8d55ac91fb219dde70e49aa4a6b7d74e9919050" +checksum = "c6673768db2d862beb9b39a78fdcb1a69439615d5794a1be50caa9bc92c81967" [[package]] name = "num-integer" @@ -1891,12 +1894,6 @@ version = "0.2.17" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd" -[[package]] -name = "pin-utils" -version = "0.1.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8b870d8c151b6f2fb93e84a13146138f05d02ed11c7e7c54f8826aaaf7c9f184" - [[package]] name = "pkcs8" version = "0.10.2" @@ -1966,9 +1963,9 @@ checksum = "c33a9471896f1c69cecef8d20cbe2f7accd12527ce60845ff44c153bb2a21b49" [[package]] name = "potential_utf" -version = "0.1.4" +version = "0.1.5" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b73949432f5e2a09657003c25bca5e19a0e9c84f8058ca374f49e0ebe605af77" +checksum = "0103b1cef7ec0cf76490e969665504990193874ea05c85ff9bab8b911d0a0564" dependencies = [ "zerovec", ] @@ -2009,9 +2006,9 @@ dependencies = [ [[package]] name = "proptest" -version = "1.10.0" +version = "1.11.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "37566cb3fdacef14c0737f9546df7cfeadbfbc9fef10991038bf5015d0c80532" +checksum = "4b45fcc2344c680f5025fe57779faef368840d0bd1f42f216291f0dc4ace4744" dependencies = [ "bit-set", "bit-vec", @@ -2045,7 +2042,7 @@ dependencies = [ "quinn-udp", "rustc-hash", "rustls", - "socket2 0.6.3", + "socket2", "thiserror 2.0.18", "tokio", "tracing", @@ -2083,7 +2080,7 @@ dependencies = [ "cfg_aliases", "libc", "once_cell", - "socket2 0.6.3", + "socket2", "tracing", "windows-sys 0.60.2", ] @@ -2301,9 +2298,9 @@ dependencies = [ [[package]] name = "rustc-hash" -version = "2.1.1" +version = "2.1.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "357703d41365b4b27c590e3ed91eabb1b663f07c4c084095e60cbed4362dff0d" +checksum = "94300abf3f1ae2e2b8ffb7b58043de3d399c73fa6f4b73826402a5c457614dbe" [[package]] name = "rustc_version" @@ -2555,9 +2552,9 @@ dependencies = [ [[package]] name = "serde_spanned" -version = "1.0.4" +version = "1.1.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f8bbf91e5a4d6315eee45e704372590b30e260ee83af6639d64557f51b067776" +checksum = "6662b5879511e06e8999a8a235d848113e942c9124f211511b16466ee2995f26" dependencies = [ "serde_core", ] @@ -2625,7 +2622,7 @@ dependencies = [ "serde_json", "serde_urlencoded", "shadowsocks-crypto", - "socket2 0.6.3", + "socket2", "spin", "thiserror 2.0.18", "tokio", @@ -2697,16 +2694,6 @@ version = "1.15.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "67b1b7a3b5fe4f1376887184045fcf45c69e92af734b7aaddc05fb777b6fbd03" -[[package]] -name = "socket2" -version = "0.5.10" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e22376abed350d73dd1cd119b57ffccad95b4e585a7cda43e286245ce23c0678" -dependencies = [ - "libc", - "windows-sys 0.52.0", -] - [[package]] name = "socket2" version = "0.6.3" @@ -2793,7 +2780,7 @@ checksum = "7b2093cf4c8eb1e67749a6762251bc9cd836b6fc171623bd0a9d324d37af2417" [[package]] name = "telemt" -version = "3.3.35" +version = "3.3.38" dependencies = [ "aes", "anyhow", @@ -2834,7 +2821,7 @@ dependencies = [ "sha1", "sha2", "shadowsocks", - "socket2 0.6.3", + "socket2", "static_assertions", "subtle", "thiserror 2.0.18", @@ -2948,9 +2935,9 @@ dependencies = [ [[package]] name = "tinystr" -version = "0.8.2" +version = "0.8.3" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "42d3e9c45c09de15d06dd8acf5f4e0e399e85927b7f00711024eb7ae10fa4869" +checksum = "c8323304221c2a851516f22236c5722a72eaa19749016521d6dff0824447d96d" dependencies = [ "displaydoc", "zerovec", @@ -2993,7 +2980,7 @@ dependencies = [ "parking_lot", "pin-project-lite", "signal-hook-registry", - "socket2 0.6.3", + "socket2", "tokio-macros", "tracing", "windows-sys 0.61.2", @@ -3054,7 +3041,7 @@ dependencies = [ "log", "once_cell", "pin-project", - "socket2 0.6.3", + "socket2", "tokio", "windows-sys 0.60.2", ] @@ -3078,9 +3065,9 @@ dependencies = [ [[package]] name = "toml" -version = "1.0.7+spec-1.1.0" +version = "1.1.2+spec-1.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "dd28d57d8a6f6e458bc0b8784f8fdcc4b99a437936056fa122cb234f18656a96" +checksum = "81f3d15e84cbcd896376e6730314d59fb5a87f31e4b038454184435cd57defee" dependencies = [ "indexmap", "serde_core", @@ -3093,27 +3080,27 @@ dependencies = [ [[package]] name = "toml_datetime" -version = "1.0.1+spec-1.1.0" +version = "1.1.1+spec-1.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9b320e741db58cac564e26c607d3cc1fdc4a88fd36c879568c07856ed83ff3e9" +checksum = "3165f65f62e28e0115a00b2ebdd37eb6f3b641855f9d636d3cd4103767159ad7" dependencies = [ "serde_core", ] [[package]] name = "toml_parser" -version = "1.0.10+spec-1.1.0" +version = "1.1.2+spec-1.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7df25b4befd31c4816df190124375d5a20c6b6921e2cad937316de3fccd63420" +checksum = "a2abe9b86193656635d2411dc43050282ca48aa31c2451210f4202550afb7526" dependencies = [ "winnow", ] [[package]] name = "toml_writer" -version = "1.0.7+spec-1.1.0" +version = "1.1.1+spec-1.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f17aaa1c6e3dc22b1da4b6bba97d066e354c7945cac2f7852d4e4e7ca7a6b56d" +checksum = "756daf9b1013ebe47a8776667b466417e2d4c5679d441c26230efd9ef78692db" [[package]] name = "tower" @@ -3310,9 +3297,9 @@ checksum = "b6c140620e7ffbb22c2dee59cafe6084a59b5ffc27a8859a5f0d494b5d52b6be" [[package]] name = "uuid" -version = "1.22.0" +version = "1.23.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a68d3c8f01c0cfa54a75291d83601161799e4a89a39e0929f4b0354d88757a37" +checksum = "5ac8b6f42ead25368cf5b098aeb3dc8a1a2c05a3eee8a9a1a68c640edbfc79d9" dependencies = [ "getrandom 0.4.2", "js-sys", @@ -3385,9 +3372,9 @@ dependencies = [ [[package]] name = "wasm-bindgen" -version = "0.2.114" +version = "0.2.117" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6532f9a5c1ece3798cb1c2cfdba640b9b3ba884f5db45973a6f442510a87d38e" +checksum = "0551fc1bb415591e3372d0bc4780db7e587d84e2a7e79da121051c5c4b89d0b0" dependencies = [ "cfg-if", "once_cell", @@ -3398,23 +3385,19 @@ dependencies = [ [[package]] name = "wasm-bindgen-futures" -version = "0.4.64" +version = "0.4.67" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e9c5522b3a28661442748e09d40924dfb9ca614b21c00d3fd135720e48b67db8" +checksum = "03623de6905b7206edd0a75f69f747f134b7f0a2323392d664448bf2d3c5d87e" dependencies = [ - "cfg-if", - "futures-util", "js-sys", - "once_cell", "wasm-bindgen", - "web-sys", ] [[package]] name = "wasm-bindgen-macro" -version = "0.2.114" +version = "0.2.117" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "18a2d50fcf105fb33bb15f00e7a77b772945a2ee45dcf454961fd843e74c18e6" +checksum = "7fbdf9a35adf44786aecd5ff89b4563a90325f9da0923236f6104e603c7e86be" dependencies = [ "quote", "wasm-bindgen-macro-support", @@ -3422,9 +3405,9 @@ dependencies = [ [[package]] name = "wasm-bindgen-macro-support" -version = "0.2.114" +version = "0.2.117" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "03ce4caeaac547cdf713d280eda22a730824dd11e6b8c3ca9e42247b25c631e3" +checksum = "dca9693ef2bab6d4e6707234500350d8dad079eb508dca05530c85dc3a529ff2" dependencies = [ "bumpalo", "proc-macro2", @@ -3435,9 +3418,9 @@ dependencies = [ [[package]] name = "wasm-bindgen-shared" -version = "0.2.114" +version = "0.2.117" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "75a326b8c223ee17883a4251907455a2431acc2791c98c26279376490c378c16" +checksum = "39129a682a6d2d841b6c429d0c51e5cb0ed1a03829d8b3d1e69a011e62cb3d3b" dependencies = [ "unicode-ident", ] @@ -3478,9 +3461,9 @@ dependencies = [ [[package]] name = "web-sys" -version = "0.3.91" +version = "0.3.94" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "854ba17bb104abfb26ba36da9729addc7ce7f06f5c0f90f3c391f8461cca21f9" +checksum = "cd70027e39b12f0849461e08ffc50b9cd7688d942c1c8e3c7b22273236b4dd0a" dependencies = [ "js-sys", "wasm-bindgen", @@ -3592,6 +3575,17 @@ version = "0.2.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" +[[package]] +name = "windows-registry" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "02752bf7fbdcce7f2a27a742f798510f3e5ad88dbe84871e5168e2120c3d5720" +dependencies = [ + "windows-link", + "windows-result", + "windows-strings", +] + [[package]] name = "windows-result" version = "0.4.1" @@ -3619,15 +3613,6 @@ dependencies = [ "windows-targets 0.42.2", ] -[[package]] -name = "windows-sys" -version = "0.48.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "677d2418bec65e3338edb076e806bc1ec15693c5d0104683f2efe857f61056a9" -dependencies = [ - "windows-targets 0.48.5", -] - [[package]] name = "windows-sys" version = "0.52.0" @@ -3670,21 +3655,6 @@ dependencies = [ "windows_x86_64_msvc 0.42.2", ] -[[package]] -name = "windows-targets" -version = "0.48.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9a2fa6e2155d7247be68c096456083145c183cbbbc2764150dda45a87197940c" -dependencies = [ - "windows_aarch64_gnullvm 0.48.5", - "windows_aarch64_msvc 0.48.5", - "windows_i686_gnu 0.48.5", - "windows_i686_msvc 0.48.5", - "windows_x86_64_gnu 0.48.5", - "windows_x86_64_gnullvm 0.48.5", - "windows_x86_64_msvc 0.48.5", -] - [[package]] name = "windows-targets" version = "0.52.6" @@ -3724,12 +3694,6 @@ version = "0.42.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "597a5118570b68bc08d8d59125332c54f1ba9d9adeedeef5b99b02ba2b0698f8" -[[package]] -name = "windows_aarch64_gnullvm" -version = "0.48.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2b38e32f0abccf9987a4e3079dfb67dcd799fb61361e53e2882c3cbaf0d905d8" - [[package]] name = "windows_aarch64_gnullvm" version = "0.52.6" @@ -3748,12 +3712,6 @@ version = "0.42.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e08e8864a60f06ef0d0ff4ba04124db8b0fb3be5776a5cd47641e942e58c4d43" -[[package]] -name = "windows_aarch64_msvc" -version = "0.48.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "dc35310971f3b2dbbf3f0690a219f40e2d9afcf64f9ab7cc1be722937c26b4bc" - [[package]] name = "windows_aarch64_msvc" version = "0.52.6" @@ -3772,12 +3730,6 @@ version = "0.42.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "c61d927d8da41da96a81f029489353e68739737d3beca43145c8afec9a31a84f" -[[package]] -name = "windows_i686_gnu" -version = "0.48.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a75915e7def60c94dcef72200b9a8e58e5091744960da64ec734a6c6e9b3743e" - [[package]] name = "windows_i686_gnu" version = "0.52.6" @@ -3808,12 +3760,6 @@ version = "0.42.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "44d840b6ec649f480a41c8d80f9c65108b92d89345dd94027bfe06ac444d1060" -[[package]] -name = "windows_i686_msvc" -version = "0.48.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8f55c233f70c4b27f66c523580f78f1004e8b5a8b659e05a4eb49d4166cca406" - [[package]] name = "windows_i686_msvc" version = "0.52.6" @@ -3832,12 +3778,6 @@ version = "0.42.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8de912b8b8feb55c064867cf047dda097f92d51efad5b491dfb98f6bbb70cb36" -[[package]] -name = "windows_x86_64_gnu" -version = "0.48.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "53d40abd2583d23e4718fddf1ebec84dbff8381c07cae67ff7768bbf19c6718e" - [[package]] name = "windows_x86_64_gnu" version = "0.52.6" @@ -3856,12 +3796,6 @@ version = "0.42.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "26d41b46a36d453748aedef1486d5c7a85db22e56aff34643984ea85514e94a3" -[[package]] -name = "windows_x86_64_gnullvm" -version = "0.48.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0b7b52767868a23d5bab768e390dc5f5c55825b6d30b86c844ff2dc7414044cc" - [[package]] name = "windows_x86_64_gnullvm" version = "0.52.6" @@ -3880,12 +3814,6 @@ version = "0.42.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9aec5da331524158c6d1a4ac0ab1541149c0b9505fde06423b02f5ef0106b9f0" -[[package]] -name = "windows_x86_64_msvc" -version = "0.48.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ed94fce61571a4006852b7389a063ab983c02eb1bb37b47f8272ce92d06d9538" - [[package]] name = "windows_x86_64_msvc" version = "0.52.6" @@ -3900,19 +3828,9 @@ checksum = "d6bbff5f0aada427a1e5a6da5f1f98158182f26556f345ac9e04d36d0ebed650" [[package]] name = "winnow" -version = "1.0.0" +version = "1.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a90e88e4667264a994d34e6d1ab2d26d398dcdca8b7f52bec8668957517fc7d8" - -[[package]] -name = "winreg" -version = "0.50.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "524e57b2c537c0f9b1e69f1965311ec12182b4122e45035b1508cd24d2adadb1" -dependencies = [ - "cfg-if", - "windows-sys 0.48.0", -] +checksum = "09dac053f1cd375980747450bfc7250c264eaae0583872e845c0c7cd578872b5" [[package]] name = "wit-bindgen" @@ -4039,9 +3957,9 @@ dependencies = [ [[package]] name = "yoke" -version = "0.8.1" +version = "0.8.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "72d6e5c6afb84d73944e5cedb052c4680d5657337201555f9f2a16b7406d4954" +checksum = "abe8c5fda708d9ca3df187cae8bfb9ceda00dd96231bed36e445a1a48e66f9ca" dependencies = [ "stable_deref_trait", "yoke-derive", @@ -4050,9 +3968,9 @@ dependencies = [ [[package]] name = "yoke-derive" -version = "0.8.1" +version = "0.8.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b659052874eb698efe5b9e8cf382204678a0086ebf46982b79d6ca3182927e5d" +checksum = "de844c262c8848816172cef550288e7dc6c7b7814b4ee56b3e1553f275f1858e" dependencies = [ "proc-macro2", "quote", @@ -4062,18 +3980,18 @@ dependencies = [ [[package]] name = "zerocopy" -version = "0.8.47" +version = "0.8.48" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "efbb2a062be311f2ba113ce66f697a4dc589f85e78a4aea276200804cea0ed87" +checksum = "eed437bf9d6692032087e337407a86f04cd8d6a16a37199ed57949d415bd68e9" dependencies = [ "zerocopy-derive", ] [[package]] name = "zerocopy-derive" -version = "0.8.47" +version = "0.8.48" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0e8bc7269b54418e7aeeef514aa68f8690b8c0489a06b0136e5f57c4c5ccab89" +checksum = "70e3cd084b1788766f53af483dd21f93881ff30d7320490ec3ef7526d203bad4" dependencies = [ "proc-macro2", "quote", @@ -4082,18 +4000,18 @@ dependencies = [ [[package]] name = "zerofrom" -version = "0.1.6" +version = "0.1.7" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "50cc42e0333e05660c3587f3bf9d0478688e15d870fab3346451ce7f8c9fbea5" +checksum = "69faa1f2a1ea75661980b013019ed6687ed0e83d069bc1114e2cc74c6c04c4df" dependencies = [ "zerofrom-derive", ] [[package]] name = "zerofrom-derive" -version = "0.1.6" +version = "0.1.7" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d71e5d6e06ab090c67b5e44993ec16b72dcbaabc526db883a360057678b48502" +checksum = "11532158c46691caf0f2593ea8358fed6bbf68a0315e80aae9bd41fbade684a1" dependencies = [ "proc-macro2", "quote", @@ -4123,9 +4041,9 @@ dependencies = [ [[package]] name = "zerotrie" -version = "0.2.3" +version = "0.2.4" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2a59c17a5562d507e4b54960e8569ebee33bee890c70aa3fe7b97e85a9fd7851" +checksum = "0f9152d31db0792fa83f70fb2f83148effb5c1f5b8c7686c3459e361d9bc20bf" dependencies = [ "displaydoc", "yoke", @@ -4134,9 +4052,9 @@ dependencies = [ [[package]] name = "zerovec" -version = "0.11.5" +version = "0.11.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6c28719294829477f525be0186d13efa9a3c602f7ec202ca9e353d310fb9a002" +checksum = "90f911cbc359ab6af17377d242225f4d75119aec87ea711a880987b18cd7b239" dependencies = [ "yoke", "zerofrom", @@ -4145,9 +4063,9 @@ dependencies = [ [[package]] name = "zerovec-derive" -version = "0.11.2" +version = "0.11.3" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "eadce39539ca5cb3985590102671f2567e659fca9666581ad3411d59207951f3" +checksum = "625dc425cab0dca6dc3c3319506e6593dcb08a9f387ea3b284dbd52a92c40555" dependencies = [ "proc-macro2", "quote", diff --git a/Cargo.toml b/Cargo.toml index a61bccf..abb7761 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "telemt" -version = "3.3.35" +version = "3.3.38" edition = "2024" [features] diff --git a/IMPLEMENTATION_PLAN.md b/IMPLEMENTATION_PLAN.md new file mode 100644 index 0000000..c1769df --- /dev/null +++ b/IMPLEMENTATION_PLAN.md @@ -0,0 +1,2035 @@ +# Telemt Relay Hardening — Implementation Plan + +## Ground Rules + +Every workstream follows this mandatory sequence: + +1. Write the full test suite first. Tests must fail on the current code for the reason being fixed. +2. Implement production changes until all new tests pass and no existing test regresses. +3. A failing red test is evidence of a real bug or gap. Never relax a test assertion — fix the code. +4. All test code goes in dedicated files under `src/proxy/tests/` (or the owning module's `#[cfg(test)]` block via `#[path]`). No inline `#[cfg(test)]` inside production code. +5. No PR lands in a non-compiling state. Every diff must be self-contained and `cargo test`-green. + +## Agreed Decisions + +| Topic | Decision | +|---|---| +| Item 3 `_buffer_pool` | Option B — repurpose the parameter for adaptive startup buffer sizes, not remove it | +| Item 4b in-session adaptation | Decision-gate phase: run experiment, measure, then choose one path | +| Item 1 Level 1 log-normal | Independent of PR-B and PR-C — can land after PR-A only | +| Scope | All items (1, 2, 3, 4a, 4b, 5) in one master plan, separate PRs | + +--- + +## PR Dependency Graph + +``` +PR-A (baseline test harness) + ├─► PR-C (Item 5: DRS — independent of DI, self-contained) + ├─► PR-F (Item 1 Level 1: log-normal — independent, no shared-state changes) + └─► PR-B (Item 2: DI migration — high-risk blast radius) + └─► PR-D (Items 3+4a: adaptive startup) + └─► PR-E (Item 4b: decision gate) + └─► PR-G (Item 1 Level 2: state-aware IPT) +PR-H (docs + release gate) +``` + +**NEW ORDERING RATIONALE** (per audit recommendations): +- **PR-C before PR-B**: DRS is self-contained, needs only `is_tls` flag (already in `HandshakeSuccess`) and a new `drs_enabled` config field. No dependency on the large DI refactor. Delivers anti-censorship value immediately. Reduces risk of a stuck dependency chain if PR-B becomes complicated. +- **PR-F independent**: Log-normal replacement modifies only two `rng.random_range()` call sites in `masking.rs` and `handshake.rs`. Zero dependency on DI or DRS. Can be parallelized with PR-C and PR-B. +- **PR-B then PR-D**: DI must be complete before adaptive startup wiring, as both involve injecting state. +- **PR-A first, always**: Baseline gates must lock before any code changes. + +Parallelization: PR-C and PR-B test-writing can happen in parallel once PR-A is done; production code integration is sequential. + +--- + +## PR-A — Baseline Test Harness (Phase 1) + +**Goal**: Establish regression gates and shared test utilities that all subsequent PRs depend on. No runtime behavior changes. + +**TDD compatibility note**: Phase 1 is a characterization and invariant-lock phase. Its baseline tests are intentionally green on current code and exist to freeze security-critical behavior before refactors. This does **not** waive red-first TDD for later phases: every behavior-changing PR after Phase 1 must begin with red tests that fail on then-current code. + +**Security objective for Phase 1**: lock anti-probing and anti-fingerprinting behavior before protocol-shape changes. Phase 1 tests must include positive, negative, edge, and adversarial scanner cases with deterministic CI execution and strict fail-closed oracles. + +**Split into two sub-phases** (reduces risk: if test utilities need iteration, baseline tests aren't blocked): + +- **PR-A.1**: Shared test utilities only. Zero behavior assertions. Merge gate: compiles. +- **PR-A.2**: Baseline invariant tests. All green on current code. Depends on PR-A.1. + +### PR-A.1: Shared test utilities + +#### New file: `src/proxy/tests/test_harness_common.rs` + +**MODULE DECLARATION**: Declare **once** in `src/proxy/mod.rs` as: +```rust +#[cfg(test)] +#[path = "tests/test_harness_common.rs"] +mod test_harness_common; +``` + +**DO NOT** declare via `#[path]` in relay.rs, handshake.rs, or middle_relay.rs. Including the same file via `#[path]` in multiple modules duplicates all definitions and causes compilation errors (see F15). Consuming test modules import via `use crate::proxy::test_harness_common::*;` (or selective imports). + +**NOTE**: Existing 104 test files already define ad-hoc test utilities inline (e.g., `ScriptedWriter` in `relay_atomic_quota_invariant_tests.rs`, `PendingWriter` in `masking_security_tests.rs`, `seeded_rng` in `masking_lognormal_timing_security_tests.rs`, `test_config_with_secret_hex` in `handshake_security_tests.rs`). The harness consolidates these for reuse but does **not** retroactively migrate existing files — that would inflate PR-A's blast radius for zero safety gain. + +Contents: + +```rust +use crate::config::ProxyConfig; +use rand::rngs::StdRng; +use rand::SeedableRng; +use std::io; +use std::pin::Pin; +use std::sync::Arc; +use std::task::{Context, Poll}; +use tokio::io::AsyncWrite; + +// ── RecordingWriter ───────────────────────────────────────────────── +// In-memory AsyncWrite that records both per-write and per-flush granularity. +// +// `writes`: one entry per poll_write call (records write-call boundaries). +// `flushed`: one entry per poll_flush call (records record/TLS-frame boundaries). +// Each entry is all bytes accumulated since the previous flush. +// +// DRS tests (PR-C) need flush-boundary tracking to verify TLS record framing. +// The dual tracking avoids needing separate writer types for different test needs. +pub struct RecordingWriter { + pub writes: Vec>, + pub flushed: Vec>, + current_record: Vec, +} + +impl RecordingWriter { + pub fn new() -> Self { + Self { + writes: Vec::new(), + flushed: Vec::new(), + current_record: Vec::new(), + } + } + + /// Total bytes written across all writes. + pub fn total_bytes(&self) -> usize { + self.writes.iter().map(|w| w.len()).sum() + } +} + +impl AsyncWrite for RecordingWriter { + fn poll_write( + mut self: Pin<&mut Self>, + _cx: &mut Context<'_>, + buf: &[u8], + ) -> Poll> { + let me = self.as_mut().get_mut(); + me.writes.push(buf.to_vec()); + me.current_record.extend_from_slice(buf); + Poll::Ready(Ok(buf.len())) + } + + fn poll_flush(mut self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> { + let me = self.as_mut().get_mut(); + let record = std::mem::take(&mut me.current_record); + if !record.is_empty() { + me.flushed.push(record); + } + Poll::Ready(Ok(())) + } + + fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> { + Poll::Ready(Ok(())) + } +} + +// ── PendingCountWriter ────────────────────────────────────────────── +// Returns Poll::Pending for the first N poll_write calls, then delegates to inner. +// Also supports separate pending-count control for poll_flush calls. +// +// Needed for DRS tests (PR-C): +// - drs_pending_on_write_does_not_increment_completed_counter +// - drs_pending_on_flush_propagates_pending_without_spurious_wake +// +// Unlike the existing masking_security_tests.rs PendingWriter (which is +// unconditionally Pending forever), this supports counted transitions. +pub struct PendingCountWriter { + pub inner: W, + pub write_pending_remaining: usize, + pub flush_pending_remaining: usize, +} + +impl PendingCountWriter { + pub fn new(inner: W, write_pending: usize, flush_pending: usize) -> Self { + Self { + inner, + write_pending_remaining: write_pending, + flush_pending_remaining: flush_pending, + } + } +} + +impl AsyncWrite for PendingCountWriter { + fn poll_write( + mut self: Pin<&mut Self>, + cx: &mut Context<'_>, + buf: &[u8], + ) -> Poll> { + let me = self.as_mut().get_mut(); + if me.write_pending_remaining > 0 { + me.write_pending_remaining -= 1; + cx.waker().wake_by_ref(); + return Poll::Pending; + } + Pin::new(&mut me.inner).poll_write(cx, buf) + } + + fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> { + let me = self.as_mut().get_mut(); + if me.flush_pending_remaining > 0 { + me.flush_pending_remaining -= 1; + cx.waker().wake_by_ref(); + return Poll::Pending; + } + Pin::new(&mut me.inner).poll_flush(cx) + } + + fn poll_shutdown(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> { + Pin::new(&mut self.inner).poll_shutdown(cx) + } +} + +// ── Deterministic seeded RNG ──────────────────────────────────────── +// Wraps StdRng::seed_from_u64 for reproducible CI runs. +// +// LIMITATION: Cannot substitute for SecureRandom in production function calls. +// Production code that accepts &SecureRandom requires a project-specific wrapper. +// Tests needing deterministic behavior of production functions that accept +// `impl Rng` (like sample_lognormal_percentile_bounded) can use this directly. +// Tests calling functions that take &SecureRandom must use SecureRandom::new(). +pub fn seeded_rng(seed: u64) -> StdRng { + StdRng::seed_from_u64(seed) +} + +// ── Config builders ───────────────────────────────────────────────── +// Builds a minimal ProxyConfig with TLS mode enabled. +// Unlike the per-test-file `test_config_with_secret_hex` helpers, this produces +// a config suitable for relay tests that need is_tls=true but don't need +// handshake secret validation. +pub fn tls_only_config() -> Arc { + let mut cfg = ProxyConfig::default(); + cfg.general.modes.tls = true; + Arc::new(cfg) +} + +// Builds a ProxyConfig with a test user and secret for handshake tests. +// Requires auth_probe, masking, and SNI configuration for full handshake paths. +pub fn handshake_test_config(secret_hex: &str) -> ProxyConfig { + let mut cfg = ProxyConfig::default(); + cfg.access.users.clear(); + cfg.access + .users + .insert("test-user".to_string(), secret_hex.to_string()); + cfg.access.ignore_time_skew = true; + cfg.censorship.mask = true; + cfg.censorship.mask_host = Some("127.0.0.1".to_string()); + cfg.censorship.mask_port = 0; // Overridden by caller with actual listener port + cfg +} +``` + +**DROPPED UTILITIES** (vs original plan): +- `SliceReader`: Unnecessary. `tokio::io::duplex()` channels (used in every existing relay test) and `std::io::Cursor>` (which implements `AsyncRead` via tokio) already solve this. Adding a `bytes`-crate-dependent `SliceReader` introduces coupling for zero gain. +- `test_stats() -> Arc`: Trivial one-liner (`Arc::new(Stats::new())`). Every existing test already constructs this inline. A wrapper adds indirection without value. +- `test_buffer_pool() -> Arc`: Same reasoning — `Arc::new(BufferPool::new())` is a one-liner already used everywhere. + +#### PR-A.1 Merge gate + +`cargo check --tests` — compiles with no errors. No behavior assertions yet. + +Determinism gate for all new Phase 1 tests: +- Seed RNG-dependent tests via `seeded_rng(...)` (or explicit fixed seeds). +- For timing-sensitive async-delay tests (for example server-hello delay or relay watchdog timing), use paused tokio time and explicit time advancement instead of wall-clock sleeps. +- Avoid shared mutable cross-test coupling except temporary helpers explicitly called out in this plan (`auth_probe_test_lock`, `relay_idle_pressure_test_scope`, `desync_dedup_test_lock`) until PR-B removes them. +- Use explicit per-test IO timeouts (`tokio::time::timeout`) on network/fallback paths to prevent deadlocks and scheduler-dependent flakes. +- Keep all adversarial corpora deterministic (fixed vectors, fixed seed order). No nondeterministic fuzz in default CI. + +--- + +### PR-A.2: Baseline invariant tests + +All tests in this sub-phase **must pass on current code** — they are regression locks, not red tests. They lock existing behavior before subsequent PRs modify it. + +**DESIGN PRINCIPLE**: Tests must be **implementation-agnostic**. Test through public/`pub(crate)` functions, not through direct static access. This ensures PR-B (which moves statics into `ProxySharedState`) does not break baseline tests. + +**SCOPE DISCIPLINE**: Phase 1 should lock boundary behavior and narrow invariants only. It must not duplicate deep transport choreography, quota accounting, or close-matrix coverage that is already exercised elsewhere unless the baseline adds a new security oracle that later PRs could realistically regress unnoticed. + +**TEST ISOLATION**: All handshake baseline tests must use the existing `auth_probe_test_lock()` / `clear_auth_probe_state_for_testing()` pattern until PR-B replaces it. Middle-relay idle tests use `relay_idle_pressure_test_scope()` / `clear_relay_idle_pressure_state_for_testing()`, while desync tests use `desync_dedup_test_lock()` / `clear_desync_dedup_for_testing()`. This is temporary coupling that PR-B will eliminate. + +**FAIL-CLOSED ASSERTION POLICY (mandatory for Phase 1)**: +- Any probe/fallback error path must assert one of: (a) transparent mask-host relay behavior or (b) silent close / generic transport failure. +- Tests must never assert proxy-identifying payloads, banners, or protocol-specific error hints. +- For "no identity leak" cases, assert observable behavior (bytes sent, connection state, error class) rather than brittle log text matching. + +#### New file: `src/proxy/tests/relay_baseline_invariant_tests.rs` + +Declared in `src/proxy/relay.rs` via: +```rust +#[cfg(test)] +#[path = "tests/relay_baseline_invariant_tests.rs"] +mod relay_baseline_invariant_tests; +``` + +**De-duplication audit**: The existing 7 relay test files cover quota boundary attacks, quota overflow, watchdog delta, and adversarial HOL blocking. The baseline tests below cover **different invariants** not locked by existing tests, verified against the existing test names: +- `relay_watchdog_delta_security_tests.rs` tests `watchdog_delta()` function exhaustively — **overlaps** with `relay_baseline_watchdog_delta_handles_wraparound_gracefully`. **DROP** the watchdog delta baseline test (existing tests already lock this behavior fully). +- `relay_adversarial_tests.rs::relay_hol_blocking_prevention_regression` exercises bidirectional transfer but does not assert symmetric byte counting. **KEEP** the symmetric-counting baseline. +- No existing test covers the zero-byte transfer case. **KEEP**. +- No existing test covers the activity timeout firing path. **KEEP**. +- Existing end-to-end quota cutoff coverage already exists in `relay_quota_boundary_blackhat_tests.rs`. **DROP** duplicate quota-cutoff baseline here. +- Existing half-close chaos coverage already exists in `relay_adversarial_tests.rs::relay_chaos_half_close_crossfire_terminates_without_hang`. **DROP** duplicate half-close baseline here. + +``` +// Positive: relay with no data flow for >ACTIVITY_TIMEOUT returns Ok. +// Verifies watchdog fires and select! cancels copy_bidirectional cleanly. +relay_baseline_activity_timeout_fires_after_inactivity + +// Positive: relay with immediate close on both sides returns Ok(()) +// and both StatsIo byte counters read zero. +relay_baseline_zero_bytes_returns_ok_and_counters_zero + +// Positive: transfer N bytes C→S and M bytes S→C simultaneously. +// Assert StatsIo counters match exactly (no double-counting or loss). +relay_baseline_bidirectional_bytes_counted_symmetrically + +// Error path: both duplex sides close simultaneously (EOF race). +// relay_bidirectional returns without panic. +relay_baseline_both_sides_close_simultaneously_no_panic + +// Error path: server-side writer returns BrokenPipe mid-transfer. +// relay_bidirectional propagates error without panic. +relay_baseline_broken_pipe_midtransfer_returns_error + +// Adversarial: single-byte writes for 10000 iterations. +// Assert counters exactly 10000 (no off-by-one in StatsIo accounting). +relay_baseline_many_small_writes_exact_counter +``` + +Oracle requirements (mandatory): +- `relay_baseline_activity_timeout_fires_after_inactivity`: use paused tokio time. Assert the relay does **not** complete before `ACTIVITY_TIMEOUT`, then does complete after advancing past `ACTIVITY_TIMEOUT + WATCHDOG_INTERVAL` with bounded slack. Do not assert a wall-clock range that ignores the watchdog cadence. +- `relay_baseline_zero_bytes_returns_ok_and_counters_zero`: assert both directions observe EOF (`read` returns `0`) and `stats.get_user_total_octets(user) == 0`. +- `relay_baseline_bidirectional_bytes_counted_symmetrically`: send fixed payload sizes `N` and `M`; assert exact byte equality on both peers and exact counter equality (`N + M` total accounted where applicable). +- `relay_baseline_both_sides_close_simultaneously_no_panic`: assert join result is `Ok(Ok(()))` (not just "did not panic"). +- `relay_baseline_broken_pipe_midtransfer_returns_error`: assert typed error class (`io::ErrorKind::BrokenPipe` or mapped proxy error) and no process crash. +- `relay_baseline_many_small_writes_exact_counter`: enforce upper runtime bound with `timeout(Duration::from_secs(3), ...)` and assert exact transferred/accounted byte count. + +#### New file: `src/proxy/tests/handshake_baseline_invariant_tests.rs` + +Declared in `src/proxy/handshake.rs` via: +```rust +#[cfg(test)] +#[path = "tests/handshake_baseline_invariant_tests.rs"] +mod handshake_baseline_invariant_tests; +``` + +**De-duplication audit**: The existing 13 handshake test files heavily test auth_probe behavior, bit-flip rejection, key zeroization, and timing. The baseline tests below lock specific invariants at the function-call boundary level: + +**LAYERING RULE**: `handle_tls_handshake(...)` is a handshake classifier/authenticator, not the masking relay itself. Handshake baseline tests must stop at the `HandshakeResult` boundary. Actual client-visible fallback relay behavior belongs in masking/client baselines, not in direct handshake tests. + +**TEST ISOLATION**: Each test acquires `auth_probe_test_lock()` / `unknown_sni_warn_test_lock()` / `warned_secrets_test_lock()` as needed. Each test calls the corresponding `clear_*_for_testing()` at the start. All tests use the existing `test_config_with_secret_hex`-style config construction (via the new `handshake_test_config` helper or inline). + +``` +// Positive: unrecognized handshake bytes classify as `BadClient` rather than +// a success path. This locks the invariant that garbage input is rejected +// without exposing proxy-specific success semantics at the handshake boundary. +handshake_baseline_probe_always_falls_back_to_masking + +// Positive: valid TLS ClientHello but wrong secret stays on the non-success +// path, not an authenticated handshake success. +handshake_baseline_invalid_secret_triggers_fallback_not_error_response + +// Positive: consecutive failed handshakes from same IP increment +// auth_probe_fail_streak for that IP. +// Tests through the public auth_probe_fail_streak_for_testing() accessor. +handshake_baseline_auth_probe_streak_increments_per_ip + +// Positive: after AUTH_PROBE_BACKOFF_START_FAILS consecutive failures, +// the IP is throttled. Tests through auth_probe_is_throttled_for_testing(). +// NOTE: AUTH_PROBE_BACKOFF_START_FAILS is a compile-time constant +// (different values for #[cfg(test)] and production). Name reflects this. +handshake_baseline_saturation_fires_at_compile_time_threshold + +// Adversarial: attacker sends 100 handshakes with distinct invalid secrets +// from the same IP. Verify auth_probe streak grows monotonically. +handshake_baseline_repeated_probes_streak_monotonic + +// Security: after throttle engages, the tracked auth-probe block window lasts +// for the computed backoff duration and then expires. +handshake_baseline_throttled_ip_incurs_backoff_delay + +// Adversarial: malformed TLS-like probe frames (truncated record header, +// impossible length fields, random high-entropy payload) never panic and +// never classify as successful handshakes. +handshake_baseline_malformed_probe_frames_fail_closed_to_masking +``` + +Oracle requirements (mandatory): +- `handshake_baseline_probe_always_falls_back_to_masking`: assert `HandshakeResult::BadClient { .. }` (or equivalent non-success fallback classification). Do **not** require direct observation of downstream mask-host IO at this layer. +- `handshake_baseline_invalid_secret_triggers_fallback_not_error_response`: assert non-success handshake classification and no success-path key material/result. Client-visible fallback behavior is covered in masking/client tests. +- `handshake_baseline_auth_probe_streak_increments_per_ip`: assert monotonic increment by exact delta `+1` per failed attempt for one IP, with no mutation for untouched IPs. +- `handshake_baseline_saturation_fires_at_compile_time_threshold`: assert transition point occurs exactly at `AUTH_PROBE_BACKOFF_START_FAILS` (not before) and remains throttled after threshold. +- `handshake_baseline_repeated_probes_streak_monotonic`: assert strictly non-decreasing streak over deterministic 100-attempt corpus. +- `handshake_baseline_throttled_ip_incurs_backoff_delay`: this Phase 1 baseline locks the **internal throttle window semantics**, not wire-visible sleep duration. Assert the tracked block window lasts at least `auth_probe_backoff(AUTH_PROBE_BACKOFF_START_FAILS)` and expires after that bound. If client-visible delay coverage is desired, add a separate async test with `server_hello_delay_min_ms == server_hello_delay_max_ms` through the client/handshake entrypoint. +- `handshake_baseline_malformed_probe_frames_fail_closed_to_masking`: run deterministic malformed corpus; assert no success result, no panic, and bounded completion per case. Do not over-assert downstream masking transport from the handshake-only boundary. + +Timing requirement for this file: +- Use paused tokio time only for tests that actually measure async sleep behavior. Tracker-only tests may use synthetic `Instant` arithmetic and must avoid wall-clock sleeps. + +#### New file: `src/proxy/tests/middle_relay_baseline_invariant_tests.rs` + +Declared in `src/proxy/middle_relay.rs` via: +```rust +#[cfg(test)] +#[path = "tests/middle_relay_baseline_invariant_tests.rs"] +mod middle_relay_baseline_invariant_tests; +``` + +**DESIGN**: Existing middle-relay suites already exercise idle/desync behavior extensively, but many assertions are tightly coupled to current internals/statics. Phase 1 only adds minimal **stable helper-boundary contract locks** that must remain stable across PR-B. + +**TEST ISOLATION**: Idle-registry tests acquire `relay_idle_pressure_test_scope()` and call `clear_relay_idle_pressure_state_for_testing()` at the start. Desync-dedup tests acquire `desync_dedup_test_lock()` and call `clear_desync_dedup_for_testing()` at the start. Do not rely on one lock to serialize the other registry. + +``` +// API contract: mark+oldest+clear round-trip through stable helper functions only, +// without direct access to internal registries/statics. +middle_relay_baseline_public_api_idle_roundtrip_contract + +// API contract: dedup suppress/allow semantics through stable helper entry, +// without asserting internal map layout. +middle_relay_baseline_public_api_desync_window_contract +``` + +Oracle requirements (mandatory): +- `middle_relay_baseline_public_api_idle_roundtrip_contract`: assert first `mark_relay_idle_candidate(conn)` returns `true`, `oldest_relay_idle_candidate() == Some(conn)`, after clear it is not `Some(conn)`, and a second mark after clear succeeds. +- `middle_relay_baseline_public_api_desync_window_contract`: through the stable helper boundary only, assert first event emits, duplicate within window suppresses, and post-rotation/window-advance emits again. + +#### New file: `src/proxy/tests/masking_baseline_invariant_tests.rs` + +Declared in `src/proxy/masking.rs` via: +```rust +#[cfg(test)] +#[path = "tests/masking_baseline_invariant_tests.rs"] +mod masking_baseline_invariant_tests; +``` + +**RATIONALE**: The masking module is the **primary anti-DPI component** — it makes the proxy appear to be a legitimate website when probed by censors. PR-F modifies `mask_outcome_target_budget` (log-normal replacement). Without baseline locks on masking timing behavior, PR-F could subtly regress the timing envelope with no detection. + +**DETERMINISM NOTE**: `mask_outcome_target_budget(...)` currently samples through an internal RNG, so Phase 1 cannot require a seeded exact output sequence from that function. Baseline tests here must assert stable invariants such as bounds and fail-closed behavior, not exact sample values or distribution shape. Distribution-quality assertions belong in PR-F once a deterministic seam exists or when tests force deterministic config such as `floor == ceiling`. + +The existing 37 masking test files cover specific attack scenarios but don't lock the **high-level behavioral contracts** that all subsequent PRs must preserve: + +``` +// Positive: mask_outcome_target_budget returns a Duration within +// [floor_ms, ceiling_ms] when normalization is enabled. +// This is the core anti-fingerprinting timing envelope. +masking_baseline_timing_normalization_budget_within_bounds + +// Positive: handle_bad_client with mask=true connects to the configured +// mask_host and forwards initial_data verbatim. Verifies the proxy +// correctly impersonates a legitimate website by relaying to the real backend. +masking_baseline_fallback_relays_to_mask_host + +// Security: mask_outcome_target_budget with timing_normalization_enabled=false +// returns the default masking budget (MASK_TIMEOUT), preserving legacy timing posture. +masking_baseline_no_normalization_returns_default_budget + +// Adversarial: mask_host is unreachable (connection refused). +// handle_bad_client must not panic and must fail closed (silent close or +// generic transport error), without proxy-identifying response bytes. +masking_baseline_unreachable_mask_host_silent_failure + +// Light fuzz: deterministic malformed initial_data corpus (length extremes, +// random binary, invalid UTF-8) must never panic. +masking_baseline_light_fuzz_initial_data_no_panic +``` + +Oracle requirements (mandatory): +- `masking_baseline_timing_normalization_budget_within_bounds`: assert every sampled budget satisfies `floor <= budget <= ceiling` across a fixed-size repeated sample loop. Do not require a seeded exact sequence from the current implementation. +- `masking_baseline_fallback_relays_to_mask_host`: assert exact byte preservation for forwarded `initial_data` and backend response relay to client. +- `masking_baseline_no_normalization_returns_default_budget`: assert exact default budget (`MASK_TIMEOUT`). +- `masking_baseline_unreachable_mask_host_silent_failure`: assert no proxy-identifying bytes are written to client and completion remains bounded. +- `masking_baseline_light_fuzz_initial_data_no_panic`: fixed malformed corpus only; assert no panic, bounded runtime per case, and no identity leak. + +De-duplication note: +- Existing masking suites already cover half-close lifecycle and bounded offline fallback timing (`masking_self_target_loop_security_tests.rs`, `masking_adversarial_tests.rs`, `masking_relay_guardrails_security_tests.rs`). +- Existing masking suites already cover strict byte-cap enforcement and cap overshoot regression (`masking_production_cap_regression_security_tests.rs`) plus broader close/failure matrices (`masking_connect_failure_close_matrix_security_tests.rs`). +- Phase 1 baseline masking tests therefore focus on top-level contracts (timing envelope bounds, fallback posture, unreachable-backend fail-closed behavior, light malformed-input robustness), not re-testing transport choreography already covered elsewhere. + +#### PR-A.2 Merge gate + +All tests pass on current code: +``` +cargo test -- relay_baseline_ +cargo test -- handshake_baseline_ +cargo test -- middle_relay_baseline_ +cargo test -- masking_baseline_ +cargo test -- --test-threads=1 +cargo test -- --test-threads=32 +cargo test # full suite — no regressions +``` + +Notes: +- `--test-threads=1` catches hidden ordering assumptions. +- `--test-threads=32` catches shared-state bleed and race-sensitive flakes. +- Heavy stress scenarios that are too expensive for default CI must be marked `#[ignore]` and run in dedicated security/perf pipelines, never deleted. +- Each adversarial baseline test must have an explicit upper runtime bound to keep CI deterministic. +- Any assertion that depends on wall-clock variance must use bounded ranges and paused time where applicable; exact wall-clock equality checks are forbidden. + +Phase 1 ASVS L2 alignment focus (test intent mapping): +- V1.2 / V1.4: fail-closed behavior and concurrency isolation under adversarial probe traffic. +- V7.4: cryptographic/protocol error handling does not leak identifying behavior. +- V9.1: communication behavior under malformed input is deterministic, bounded, and non-panicking. +- V13.2: degradation paths (fallback/masking) preserve security posture and do not disclose gateway identity. + +--- + +### Critical Review Issues Found and Addressed in PR-A + +| # | Severity | Issue from critique | Resolution | +|---|---|---|---| +| 1 | **Critical** | `test_harness_common.rs` has no valid declaration site; triple `#[path]` causes duplicate symbols | Declared once in `proxy/mod.rs`; consuming tests import via `use crate::proxy::test_harness_common::*` | +| 2 | **High** | `RecordingWriter` semantics ambiguous; flush-boundary tracking missing for DRS tests | Dual tracking: `writes` (per poll_write) + `flushed` (per poll_flush boundary with accumulator) | +| 3 | **High** | `SliceReader` unnecessarily requires `bytes` crate | **Dropped**. `tokio::io::duplex()` and `std::io::Cursor` already solve this | +| 4 | **Medium** | `PendingWriter` only controls `poll_write`; flush pending tests need separate control | Renamed to `PendingCountWriter` with separate `write_pending_remaining` and `flush_pending_remaining` | +| 5 | **Critical** | Baseline tests duplicate existing tests; `watchdog_delta` wraparound test trivially green | Watchdog delta baseline **dropped** (7 existing tests in `relay_watchdog_delta_security_tests.rs` cover it exhaustively). All other baselines audited against 104 existing test files. | +| 6 | **High** | Handshake baseline tests require complex scaffold not provided by `tls_only_config()` | Added `handshake_test_config(secret_hex)` builder with user, secret, auth settings, and masking config | +| 7 | **Medium** | `test_stats()` / `test_buffer_pool()` are trivial wrappers | **Dropped**. `Arc::new(Stats::new())` and `Arc::new(BufferPool::new())` are one-liners, universally inlined already | +| 8 | **High** | Middle relay baseline tests lock on global statics; PR-B removes them → guaranteed breakage | Tests call public functions (`mark_relay_idle_candidate`, `clear_relay_idle_candidate`) not statics. PR-B changes implementations, not function signatures. | +| 9 | **Medium** | `seeded_rng` returns `StdRng`, can't substitute for `SecureRandom` | Documented as explicit limitation in code comment | +| 10 | **Medium** | No test isolation strategy for auth_probe global state | Each handshake baseline test acquires `auth_probe_test_lock()` and calls `clear_auth_probe_state_for_testing()`. Documented as temporary coupling. | +| 11 | **Low** | "configured threshold" misnomer for compile-time constant | Renamed to `handshake_baseline_saturation_fires_at_compile_time_threshold` | +| 12 | **High** | Zero error-path regression locks in baseline suite | Added: `relay_baseline_both_sides_close_simultaneously_no_panic`, `relay_baseline_broken_pipe_midtransfer_returns_error` | +| 13 | **Medium** | `relay_baseline_empty_transfer_completes_without_error` is vague | Replaced with: `relay_baseline_zero_bytes_returns_ok_and_counters_zero` (sharp assertion) | +| 14 | **Medium** | No masking.rs baseline tests despite PR-F modifying masking | Added `masking_baseline_invariant_tests.rs` with timing/fallback/cap/adversarial tests | +| NEW-1 | **High** | PR-A text could be read as violating global TDD "red first" rule | Clarified Phase 1 as characterization-only; red-first remains mandatory for all behavior-changing phases | +| NEW-2 | **Medium** | "No production code changes" wording conflicts with required `#[cfg(test)]` module wiring | Corrected scope statement to "No runtime behavior changes" | +| NEW-3 | **High** | Fail-closed requirement was implicit, allowing weak "no panic"-only assertions | Added explicit fail-closed assertion policy for anti-probing paths | +| NEW-4 | **High** | Timing and network-path baselines risk CI flakiness/deadlocks | Added deterministic timeout and paused-time requirements | +| NEW-5 | **Medium** | Several proposed baselines duplicated existing relay/middle-relay/handshake coverage | Pruned duplicate cases (relay quota cutoff, relay half-close, unknown-SNI warn rate-limit) and reduced middle-relay baseline to API-contract-only tests | +| NEW-6 | **High** | Relay inactivity oracle ignored `WATCHDOG_INTERVAL`, making the timeout assertion architecturally wrong | Rewrote the oracle around paused-time advancement past `ACTIVITY_TIMEOUT + WATCHDOG_INTERVAL` | +| NEW-7 | **High** | Handshake baselines conflated `HandshakeResult::BadClient` with downstream masking relay behavior | Separated handshake-layer classification assertions from masking/client-layer fallback IO assertions | +| NEW-8 | **High** | Handshake "backoff delay" wording conflated auth-probe state tracking with wire-visible sleep latency | Re-scoped the baseline to throttle-window semantics and deferred client-visible delay checks to an explicit async entrypoint test | +| NEW-9 | **Medium** | Masking timing determinism requirement overstated what the current internal-RNG API can guarantee | Limited Phase 1 masking timing assertions to invariant bounds instead of seeded exact sequences | +| NEW-10 | **Medium** | Middle-relay isolation guidance omitted `desync_dedup_test_lock()`, leaving desync tests underspecified | Split idle-registry and desync-dedup isolation requirements by helper/lock | +| NEW-11 | **Medium** | Masking baseline list still carried redundant cases already covered by dedicated cap and close-matrix suites | Pruned duplicate cap/empty-input/partial-close baseline cases from mandatory Phase 1 scope | + +--- + +## PR-B — Item 2: Dependency Injection for Global Proxy State + +**Priority**: High. Blocks PR-D. (PR-C and PR-F are independent — see D1 below.) + +**TDD compatibility note**: PR-B cannot start with red tests that reference a non-existent `ProxySharedState` API, because that would fail at compile time rather than exposing the current runtime bug. Split PR-B into: +- **PR-B.0 (seam only, green)**: add `shared_state.rs`, define `ProxySharedState`, and thread an instance parameter through the call chain without changing storage semantics yet. +- **PR-B.1 (red)**: add isolation tests against the new seam; they must compile and fail on then-current code because the seam still routes into global state. +- **PR-B.2 (green)**: cut storage over from globals to per-instance state, then remove global reset/lock helpers. + +This keeps red-first TDD for the behavior change while allowing the minimum compile-time scaffolding needed to express the tests. + +### Problem (concrete) + +The **core blocker set** is the 12 handshake and middle-relay statics below. These are logically scoped to one running proxy instance but currently live at process scope, which forces test serialization and prevents two proxy instances in one process from remaining isolated: + +| Static | File | Line | Type | +|---|---|---|---| +| `AUTH_PROBE_STATE` | `src/proxy/handshake.rs` | 52 | `OnceLock>` | +| `AUTH_PROBE_SATURATION_STATE` | `src/proxy/handshake.rs` | 53 | `OnceLock>>` | +| `AUTH_PROBE_EVICTION_HASHER` | `src/proxy/handshake.rs` | 55 | `OnceLock` | +| `INVALID_SECRET_WARNED` | `src/proxy/handshake.rs` | 33 | `OnceLock>>` | +| `UNKNOWN_SNI_WARN_NEXT_ALLOWED` | `src/proxy/handshake.rs` | 39 | `OnceLock>>` | +| `DESYNC_DEDUP` | `src/proxy/middle_relay.rs` | 54 | `OnceLock>` | +| `DESYNC_DEDUP_PREVIOUS` | `src/proxy/middle_relay.rs` | 55 | `OnceLock>` | +| `DESYNC_HASHER` | `src/proxy/middle_relay.rs` | 56 | `OnceLock` | +| `DESYNC_FULL_CACHE_LAST_EMIT_AT` | `src/proxy/middle_relay.rs` | 57 | `OnceLock>>` | +| `DESYNC_DEDUP_ROTATION_STATE` | `src/proxy/middle_relay.rs` | 58 | `OnceLock>` | +| `RELAY_IDLE_CANDIDATE_REGISTRY` | `src/proxy/middle_relay.rs` | 61 | `OnceLock>` | +| `RELAY_IDLE_MARK_SEQ` | `src/proxy/middle_relay.rs` | 62 | `AtomicU64` (direct static) | + +**Explicitly out of core PR-B scope**: +- `USER_PROFILES` in `adaptive_buffers.rs` stays process-global for PR-D cross-session memory. It must **not** be counted as a per-instance DI blocker for PR-B. +- `LOGGED_UNKNOWN_DCS` in `direct_relay.rs` and the warning-dedup `AtomicBool` statics in `client.rs` are ancillary diagnostics caches, not core handshake/relay isolation state. Keep them for a follow-up consistency PR after the handshake and middle-relay cutover lands. + +These force a large body of tests to use `auth_probe_test_lock()`, `relay_idle_pressure_test_scope()`, and `desync_dedup_test_lock()` to stay deterministic. The current branch also has tests that read `AUTH_PROBE_STATE` and `DESYNC_DEDUP` directly, so the migration scope is larger than helper removal alone. + +### Step 1: Add seam, then write red tests (must fail on then-current code) + +**Important sequencing correction**: red tests for PR-B must be written **after** the non-behavioral seam from PR-B.0 exists, otherwise they cannot compile. They still remain red-first for the actual behavior change because the seam initially points to the old globals. + +**New file**: `src/proxy/tests/proxy_shared_state_isolation_tests.rs` +Declared in `src/proxy/mod.rs` via a single `#[cfg(test)] #[path = "tests/proxy_shared_state_isolation_tests.rs"] mod proxy_shared_state_isolation_tests;` declaration. **Do NOT declare in both handshake.rs and middle_relay.rs** — including the same file via `#[path]` in two modules duplicates all definitions and causes compilation errors. + +**TEST SCOPE**: These tests cover only the handshake and middle-relay state being migrated in core PR-B. Do not mix in `direct_relay.rs` unknown-DC logging or `client.rs` warning-dedup behavior here. + +``` +// Fails because AUTH_PROBE_STATE is global — second instance shares first's state. +proxy_shared_state_two_instances_do_not_share_auth_probe_state +// Fails because DESYNC_DEDUP is global. +proxy_shared_state_two_instances_do_not_share_desync_dedup +// Fails because RELAY_IDLE_CANDIDATE_REGISTRY is global. +proxy_shared_state_two_instances_do_not_share_idle_registry +// Fails: resetting state in instance A must not affect instance B. +proxy_shared_state_reset_in_one_instance_does_not_affect_another +// Fails: parallel tests increment the same IP counter in AUTH_PROBE_STATE. +proxy_shared_state_parallel_auth_probe_updates_stay_per_instance +// Fails: desync rotation in instance A must not advance rotation state of instance B. +proxy_shared_state_desync_window_rotation_is_per_instance +// Fails: idle seq counter is global AtomicU64, shared between instances. +proxy_shared_state_idle_mark_seq_is_per_instance +// Adversarial: attacker floods auth probe state in "proxy A" must not exhaust probe +// budget of unrelated "proxy B" sharing the process. +proxy_shared_state_auth_saturation_does_not_bleed_across_instances +``` + +**DROP from mandatory core PR-B**: +- `proxy_shared_state_poisoned_mutex_in_one_instance_does_not_panic_other`. This is too implementation-coupled for the initial red phase and is better expressed as targeted unit tests once per-instance lock recovery helpers exist. The core risk is cross-instance state bleed, not synthetic poisoning choreography. + +**New file**: `src/proxy/tests/proxy_shared_state_parallel_execution_tests.rs` + +``` +// Spawns 50 concurrent auth-probe updates against distinct ProxySharedState instances, +// asserts each instance's counter matches exactly what it received (no cross-talk). +proxy_shared_state_50_concurrent_instances_no_counter_bleed +// Desync dedup: 20 concurrent instances each performing window rotation, +// asserts rotation state is per-instance and not double-rotated. +proxy_shared_state_desync_rotation_concurrent_20_instances +// Idle registry: 10 concurrent mark+evict cycles across isolated instances, +// asserts no cross-eviction. +proxy_shared_state_idle_registry_concurrent_10_instances +``` + +### Step 2: Implement `ProxySharedState` + +**New file**: `src/proxy/shared_state.rs` + +**MUTEX TYPE**: All `Mutex` fields below are `std::sync::Mutex`, NOT `tokio::sync::Mutex`. The current codebase uses `std::sync::Mutex` for all these statics, and all critical sections are short (insert/get/retain) with no await points inside. Per Architecture.md §5: "Never hold a lock across an `await` unless atomicity explicitly requires it." Using `std::sync::Mutex` is correct here because: +1. Lock hold times are bounded (microseconds for DashMap/HashSet operations) +2. No `.await` is called while holding any of these locks +3. `tokio::sync::Mutex` would add unnecessary overhead for these synchronous operations + +```rust +use std::sync::Mutex; // NOT tokio::sync::Mutex — see note above + +pub struct HandshakeSharedState { + pub auth_probe: DashMap, + pub auth_probe_saturation: Mutex>, + pub auth_probe_eviction_hasher: RandomState, + pub invalid_secret_warned: Mutex>, + pub unknown_sni_warn_next_allowed: Mutex>, +} + +pub struct MiddleRelaySharedState { + pub desync_dedup: DashMap, + pub desync_dedup_previous: DashMap, + pub desync_hasher: RandomState, + pub desync_full_cache_last_emit_at: Mutex>, + pub desync_dedup_rotation_state: Mutex, + pub relay_idle_registry: Mutex, + // Monotonic counter; kept as AtomicU64 inside the struct, not a global. + pub relay_idle_mark_seq: AtomicU64, +} + +pub struct ProxySharedState { + pub handshake: HandshakeSharedState, + pub middle_relay: MiddleRelaySharedState, +} + +impl ProxySharedState { + pub fn new() -> Arc { ... } +} +``` + +Declare `pub mod shared_state;` in `src/proxy/mod.rs` between lines 61–69. + +`ProxySharedState` is architecturally: state that (a) must survive across multiple concurrent connections, (b) is logically scoped to one running proxy instance, not the whole process. Aligns with Architecture.md §3.1 Singleton rule: "pass shared state explicitly via `Arc`." + +**Scope correction**: `ProxySharedState` in core PR-B should contain only handshake and middle-relay shared state. Do **not** add `adaptive_buffers::USER_PROFILES` here. + +### Step 3: Thread `Arc` through the call chain + +**`src/proxy/handshake.rs`** + +Current signature of `handle_tls_handshake` (line 690): +```rust +pub async fn handle_tls_handshake( + handshake: &[u8], + reader: R, + mut writer: W, + peer: SocketAddr, + config: &ProxyConfig, + replay_checker: &ReplayChecker, + rng: &SecureRandom, + tls_cache: Option>, +) -> HandshakeResult<...> +``` + +New signature — add one parameter at the end: +```rust + shared: &ProxySharedState, // ← add as last parameter +``` + +Current signature of `handle_mtproto_handshake` (line 854): same pattern — add `shared: &ProxySharedState` as last parameter. + +All internal calls to `auth_probe_state_map()`, `auth_probe_saturation_state()`, `warn_invalid_secret_once()`, `unknown_sni_warn_state_lock()` are replaced with direct field access on `&shared.handshake`. The five accessor functions (`auth_probe_state_map`, `auth_probe_saturation_state`, `unknown_sni_warn_state_lock`) are deleted. + +**`src/proxy/middle_relay.rs`** + +Current signature of `handle_via_middle_proxy` (line 695): +```rust +pub(crate) async fn handle_via_middle_proxy( + mut crypto_reader: CryptoReader, + crypto_writer: CryptoWriter, + success: HandshakeSuccess, + me_pool: Arc, + stats: Arc, + config: Arc, + buffer_pool: Arc, + local_addr: SocketAddr, + rng: Arc, + mut route_rx: watch::Receiver, + route_snapshot: RouteCutoverState, + session_id: u64, +) -> Result<()> +``` + +New signature — add `shared: Arc` after `session_id: u64`. All `RELAY_IDLE_CANDIDATE_REGISTRY`, `DESYNC_DEDUP`, etc. accesses replaced with `shared.middle_relay.*`. `relay_idle_candidate_registry()` accessor deleted. + +**`src/proxy/client.rs`** + +The call site of `handle_tls_handshake` (line ~553) and `handle_via_middle_proxy` (line ~1289) must pass the `Arc` that is constructed once in the main startup path and passed down. Locate the top-level `handle_client_stream` function (line 317) and add `shared: Arc` to its parameters, then thread through. + +`handle_authenticated_static(...)` also needs `shared: Arc` because it dispatches to the middle-relay path after the handshake. + +**Construction site correction**: the current connection task spawn lives in `src/maestro/listeners.rs`, not `src/maestro/mod.rs` or `src/startup.rs`. Construct one `Arc` alongside the other long-lived listener resources and clone it into each `handle_client_stream(...)` task. Do **not** create a fresh shared-state instance per accepted connection. + +**Scope correction**: `handle_via_direct(...)` stays unchanged in core PR-B unless the ancillary `direct_relay.rs` unknown-DC dedup migration is explicitly pulled into scope. + +### Step 4: Remove test helpers and migrate test files + +After production code passes all new tests, remove the **global reset/lock helpers** for the migrated handshake and middle-relay state. Do **not** blindly delete every test accessor. Prefer converting narrow query helpers into instance-scoped helpers when they preserve test decoupling from internal map layout. + +Delete or replace these handshake/middle-relay globals: +- `auth_probe_test_lock()` +- `unknown_sni_warn_test_lock()` +- `warned_secrets_test_lock()` +- `relay_idle_pressure_test_scope()` +- `desync_dedup_test_lock()` +- global reset helpers that only exist to wipe process-wide state between tests + +Prefer converting, not deleting outright: +- `auth_probe_fail_streak_for_testing(...)` +- `auth_probe_is_throttled_for_testing(...)` +- similar narrow read-only helpers that can become `..._for_testing(shared, ...)` + +**Blast-radius correction**: this migration affects more than the helper users listed in the original draft. The current branch has many handshake and middle-relay tests that read `AUTH_PROBE_STATE` or `DESYNC_DEDUP` directly. Those tests must be migrated off raw statics before the statics are removed. + +Ancillary `direct_relay.rs` helpers such as `unknown_dc_test_lock()` remain out of scope unless that follow-up consistency migration is explicitly included. + +No global `Mutex<()>` test locks remain **for the migrated handshake/middle-relay state** after this PR. Do not overstate this as a repository-wide guarantee while ancillary globals still exist elsewhere. + +### Merge gate + +``` +cargo check --tests +cargo test -- proxy_shared_state_ +cargo test -- handshake_ +cargo test -- middle_relay_ +cargo test -- client_ +cargo test -- --test-threads=1 +cargo test -- --test-threads=32 +``` +All must pass. No existing test may fail. The thread-count runs are mandatory here because PR-B's entire purpose is eliminating hidden cross-test and cross-instance state bleed. + +--- + +## PR-C — Item 5: Dynamic Record Sizing (DRS) for the TLS Relay Path + +**Priority**: High (anti-censorship, TLS-mode only). + +**TDD note for PR-C**: Red tests in this phase must fail because DRS behavior is absent, not because APIs are temporarily broken. Keep baseline relay API compatibility where practical so failures remain behavioral, not compile-surface churn. + +**SCOPE LIMITATION**: This PR covers the **direct relay path only** (`direct_relay.rs` → `relay_bidirectional`). The **middle relay path** (`middle_relay.rs` → explicit ME→client flush loop) is not addressed. Since middle relay is the default when ME URLs are configured, this represents a **significant coverage gap** for those deployments. Future follow-up (PR-C.1): add DRS shaping to the middle relay's explicit flush loop. This is architecturally simpler (natural flush tick points exist) and should be prioritized immediately after PR-C. + +### Problem (concrete) + +`src/proxy/relay.rs` line 563: +```rust +result = copy_bidirectional_with_sizes( + &mut client, // client = StatsIo> + &mut server, + c2s_buf_size.max(1), + s2c_buf_size.max(1), +) => Some(result), +``` + +`client` is a `StatsIo` wrapping a `CombinedStream`. The write half of `client` (the path sending data *to* the real client) has no TLS record framing control. TLS record sizes observed by DPI are determined by tokio's internal copy buffer size — a single constant that produces a recognizable signature absent from real browser TLS sessions. + +The previous draft had three bugs (now fixed here): +1. Used 1450 byte payload → creates 1471-byte framed records → TCP splits into `[1460, 11]` signature. **Correct value: 1369 bytes.** +2. Incremented `records_completed` on every `poll_write` call, not only when a record boundary is crossed. **Fix: track `bytes_in_current_record`; only increment when a flush completes.** +3. Returned `Poll::Pending` with `wake_by_ref()` after a flush completed, causing an immediate spurious reschedule. **Fix: use `ready!` macro and `continue` in a loop — no yield between a completed flush and the next write.** + +### Step 1: Write red tests (must fail on current code) + +**New file**: `src/proxy/tests/drs_writer_unit_tests.rs` +Declared in `src/proxy/relay.rs`. + +``` +// Positive: bytes emitted to inner writer arrive in records of exactly +// target_record_size(0..=39) = 1369 before flush, then 4096, then 16384. +drs_first_40_records_are_1369_bytes_payload_each +drs_records_41_to_60_are_4096_bytes_payload_each +drs_records_above_60_are_16384_bytes_payload_each + +// Boundary/edge: a write of 1 byte completes correctly and counts toward +// bytes_in_current_record without incrementing records_completed prematurely. +drs_single_byte_write_does_not_prematurely_complete_record +// Edge: write of exactly 1369 bytes fills one record; next poll_write triggers flush. +drs_write_equal_to_record_size_requires_second_poll_for_flush +// Edge: TWO sequential poll_write calls, each crossing one record boundary, +// produce exactly two separate flushes (can't flush twice in single poll). +drs_two_sequential_writes_cross_boundary_each_produces_one_flush +// Edge: empty slice write returns Ok(0) immediately without touching inner. +drs_empty_write_returns_zero_does_not_touch_inner +// Edge: poll_shutdown delegates to inner and does not flush records. +drs_shutdown_delegates_to_inner + +// Adversarial: inner writer returns Pending on first 5 poll_write calls. +// DrsWriter must not loop-busy-poll and must not increment records_completed. +drs_pending_on_write_does_not_increment_completed_counter +// Adversarial: inner flush returns Pending. DrsWriter must propagate Pending +// without calling wake_by_ref (verified by checking waker was not called). +drs_pending_on_flush_propagates_pending_without_spurious_wake +// Adversarial: 10001 consecutive 1-byte writes; verify records_completed +// count matches expected record boundaries, no off-by-one. +drs_10001_single_byte_writes_records_count_exact + +// Stress: bounded concurrent DrsWriter instances each writing deterministic +// payloads; assert total flushed bytes equals total written bytes. +// Large-scale variants belong in ignored perf/security jobs, not default CI. +drs_concurrent_instances_no_data_loss + +// Security/anti-DPI: collect sizes of all records produced by writing 100 KB +// through DrsWriter; assert no record with size > 1369 appears in first 40. +// This is the packet-shape non-regression test. +drs_first_records_do_not_exceed_mss_safe_payload_size +// Security: non-TLS passthrough path produces no DrsWriter wrapping; +// assert that when is_tls=false the relay produces no record-size shaping. +drs_passthrough_when_not_tls_no_record_shaping + +// Overflow hardening: records_completed saturates at final phase and never +// re-enters phase 1 after saturation. +drs_records_completed_counter_does_not_wrap + +// Integration: StatsIo byte counters match actual bytes received by inner writer +// when DrsWriter limits write sizes (no data loss or double-counting). +drs_statsio_byte_count_matches_actual_written + +// Integration: copy loop handles partial writes at record boundaries without +// data loss or duplication. +drs_copy_loop_partial_write_retry +``` + +**CI policy for this file**: +- Keep default-suite tests deterministic and bounded in runtime and memory. +- Any high-cardinality stress profile (for example 1000 writers x 1 MB) must be marked ignored and run only in dedicated perf/security pipelines. + +**New file**: `src/proxy/tests/drs_integration_tests.rs` +Declared in `src/proxy/relay.rs`. + +``` +// Integration: relay_bidirectional with DRS enabled (is_tls=true) produces +// records ≤ 1369 bytes in payload size for the first 40 records to the client. +drs_relay_bidirectional_tls_first_records_bounded +// Integration: relay_bidirectional with is_tls=false produces no DrsWriter +// overhead (records sized by c2s_buf_size only). +drs_relay_bidirectional_non_tls_no_drs_overhead +// Integration: relay completes normally with DRS enabled; final byte count +// matches input byte count (no loss or duplication). +drs_relay_bidirectional_tls_no_data_loss_end_to_end + +// Integration: verify FakeTlsWriter.poll_flush produces a TLS record boundary, +// not a no-op. Otherwise DRS shaping provides no anti-DPI value. +drs_flush_is_meaningful_for_faketls +``` + +### Step 2: Implement `DrsWriter` + +**New file**: `src/proxy/drs_writer.rs` + +Declare `pub(crate) mod drs_writer;` in `src/proxy/mod.rs`. + +```rust +pub(crate) struct DrsWriter { + inner: W, + bytes_in_current_record: usize, + // Capped at DRS_PHASE_FINAL (60) to prevent overflow on long-lived connections. + // On 32-bit platforms, an uncapped usize would wrap after ~4 billion records, + // restarting the DRS ramp — a detectable signature. + records_completed: usize, +} + +const DRS_PHASE_1_END: usize = 40; +const DRS_PHASE_2_END: usize = 60; +const DRS_PHASE_FINAL: usize = DRS_PHASE_2_END; +// Safe payload for one MSS with TCP-options headroom. +// FakeTLS overhead in THIS proxy: 5 bytes (TLS record header only). +// NOTE: Unlike real TLS 1.3, FakeTlsWriter does NOT add a content-type byte +// or AEAD tag. Real TLS 1.3 overhead would be 22 bytes (5 + 1 + 16). +// We size for the FakeTLS overhead: record on wire = 1369 + 5 = 1374 bytes. +// MSS = 1460 (MTU 1500 - 40 IP+TCP); with TCP timestamps (~12 bytes) +// effective MSS ≈ 1448, leaving 74 bytes margin for path MTU variance (PPPoE, VPN). +// The value 1369 is intentionally conservative to accommodate future FakeTLS +// upgrades that may add AEAD or padding overhead. +const DRS_MSS_SAFE_PAYLOAD: usize = 1_369; +const DRS_PHASE_2_PAYLOAD: usize = 4_096; +// NOTE: FakeTlsWriter uses MAX_TLS_CIPHERTEXT_SIZE = 16_640 as its max payload. +// DRS caps at 16_384 (RFC 8446 TLS 1.3 plaintext limit). This means DRS still +// shapes records in steady-state by limiting to 16_384 instead of 16_640. +// This is intentional: real TLS 1.3 servers cap at 16_384 plaintext bytes per +// record, so DRS mimics that limit even though FakeTLS allows larger records. +const DRS_FULL_RECORD_PAYLOAD: usize = 16_384; + +impl DrsWriter { + pub(crate) fn new(inner: W) -> Self { + Self { inner, bytes_in_current_record: 0, records_completed: 0 } + } + + fn target_record_size(&self) -> usize { + match self.records_completed { + 0..DRS_PHASE_1_END => DRS_MSS_SAFE_PAYLOAD, + DRS_PHASE_1_END..DRS_PHASE_2_END => DRS_PHASE_2_PAYLOAD, + _ => DRS_FULL_RECORD_PAYLOAD, + } + } +} + +impl AsyncWrite for DrsWriter { + fn poll_write(mut self: Pin<&mut Self>, cx: &mut Context<'_>, buf: &[u8]) -> Poll> { + if buf.is_empty() { return Poll::Ready(Ok(0)); } + loop { + let target = self.target_record_size(); + let remaining = target.saturating_sub(self.bytes_in_current_record); + if remaining == 0 { + // Record boundary reached — flush before starting the next record. + ready!(Pin::new(&mut self.inner).poll_flush(cx))?; + // Cap at DRS_PHASE_FINAL to prevent usize overflow on long-lived connections. + self.records_completed = self.records_completed.saturating_add(1).min(DRS_PHASE_FINAL + 1); + self.bytes_in_current_record = 0; + continue; + } + let limit = buf.len().min(remaining); + let n = ready!(Pin::new(&mut self.inner).poll_write(cx, &buf[..limit]))?; + self.bytes_in_current_record += n; + return Poll::Ready(Ok(n)); + } + } + + fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> { + Pin::new(&mut self.inner).poll_flush(cx) + } + + fn poll_shutdown(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> { + Pin::new(&mut self.inner).poll_shutdown(cx) + } +} +``` + +**State integrity requirement**: `bytes_in_current_record` must be incremented by the number of bytes actually accepted by inner writer (`n`), not requested length. This preserves correctness under partial writes. + +**Pending behavior requirement**: if inner `poll_write` or `poll_flush` returns `Pending`, propagate `Pending` without manual `wake_by_ref` calls in DRS, relying on inner writer wake semantics. + +### Step 3: Wire into relay path with compatibility + +**`src/proxy/relay.rs`** — `relay_bidirectional` currently (line 456): + +```rust +pub async fn relay_bidirectional( + client_reader: CR, + client_writer: CW, + ... + _buffer_pool: Arc, // unchanged at this stage +) -> Result<()> +``` + +**Compatibility correction**: Do not force a signature break on `relay_bidirectional(...)` for all existing tests/callers. Prefer one of: +- add `relay_bidirectional_with_opts(...)` and keep `relay_bidirectional(...)` as a passthrough wrapper with defaults; or +- introduce a small options struct with a defaulted constructor and keep a compatibility wrapper. + +This prevents unrelated relay and masking suites from becoming compile-red due to API churn and keeps PR-C failures focused on DRS behavior. + +Inside the function body, where `CombinedStream::new(client_reader, client_writer)` constructs the client combined stream (line ~481), wrap the write half conditionally with a `MaybeDrs` enum: + +**ARCHITECTURE NOTE — write-side placement**: `DrsWriter` wraps the **raw** `client_writer` *before* it enters `CombinedStream`, which is then wrapped by `StatsIo`. The resulting call chain on S→C writes is: + +``` +copy_bidirectional_with_sizes + → StatsIo.poll_write (counts bytes, quota accounting) + → CombinedStream.poll_write + → MaybeDrs.poll_write + → DrsWriter.poll_write + → CryptoWriter.poll_write (AES-CTR encryption, may buffer internally) + → FakeTlsWriter.poll_write (wraps into TLS record with 5-byte header) + → TCP socket +``` + +This is correct because: +1. `StatsIo` sees the actual bytes being written (DrsWriter doesn't change byte count, only limits write sizes and triggers flushes). `StatsIo.poll_write` counts the return value of CombinedStream.poll_write, which equals DrsWriter's return value — the actual bytes accepted. +2. **CryptoWriter buffering interaction**: CryptoWriter.poll_write encrypts and MAY buffer internally (PendingCiphertext) if FakeTlsWriter returns Pending. Crucially, CryptoWriter **always returns Ok(to_accept)** even when buffering — it never returns Pending unless its internal buffer is full. This means DrsWriter's `bytes_in_current_record` tracking is accurate; CryptoWriter accepts the full limited amount. +3. **DRS flush drains the CryptoWriter→FakeTLS→socket chain**: `DrsWriter.poll_flush` → `CryptoWriter.poll_flush` (drains pending ciphertext to FakeTlsWriter) → `FakeTlsWriter.poll_flush` (drains pending TLS record data to socket) → `socket.poll_flush`. This is what enforces TLS record boundaries on the wire. Without the flush, CryptoWriter could batch multiple DRS "records" into one FakeTLS record, defeating the purpose. +4. `copy_bidirectional_with_sizes` also calls `poll_flush` on its own schedule; double-flush is safe (idempotent on all three layers) but adds minor syscall overhead. +5. `copy_bidirectional_with_sizes`'s internal S→C buffer will be partially consumed per poll_write (DrsWriter may accept fewer bytes than offered). This is the intended mechanism — the copy loop retries with the remaining buffer. + +**IMPORTANT**: Add a red test `drs_statsio_byte_count_matches_actual_written` to verify that `StatsIo` byte counters exactly match the total bytes the inner socket received. Without this, a bug where DrsWriter eats or duplicates bytes would go undetected. + +```rust +enum MaybeDrs { + Passthrough(W), + Shaping(DrsWriter), +} + +impl AsyncWrite for MaybeDrs { + fn poll_write(mut self: Pin<&mut Self>, cx: &mut Context<'_>, buf: &[u8]) -> Poll> { + match self.get_mut() { + MaybeDrs::Passthrough(w) => Pin::new(w).poll_write(cx, buf), + MaybeDrs::Shaping(w) => Pin::new(w).poll_write(cx, buf), + } + } + fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> { + match self.get_mut() { + MaybeDrs::Passthrough(w) => Pin::new(w).poll_flush(cx), + MaybeDrs::Shaping(w) => Pin::new(w).poll_flush(cx), + } + } + fn poll_shutdown(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> { + match self.get_mut() { + MaybeDrs::Passthrough(w) => Pin::new(w).poll_shutdown(cx), + MaybeDrs::Shaping(w) => Pin::new(w).poll_shutdown(cx), + } + } +} + +let writer = if opts.is_tls && opts.drs_enabled { + MaybeDrs::Shaping(DrsWriter::new(client_writer)) +} else { + MaybeDrs::Passthrough(client_writer) +}; +let client = StatsIo::new(CombinedStream::new(client_reader, writer), ...); +``` + +**PERFORMANCE NOTE**: The `MaybeDrs::Passthrough` variant adds a single enum match dispatch per `poll_write`/`poll_flush`/`poll_shutdown` call (~3-5 cycles on modern CPUs with branch prediction, negligible for TLS overhead). This is acceptable for correctness. Do not attempt zero-overhead abstractions with generic specialization here; the dispatch overhead is unmeasurable relative to the underlying TLS crypto and I/O. + +**`src/proxy/direct_relay.rs`** — direct path call site: +Pass DRS options only from direct relay dispatch (`is_tls = success.is_tls`, `drs_enabled = config.general.drs_enabled && success.is_tls`). + +**Scope guard**: leave middle-relay call choreography untouched in this PR; this is a direct-path-only phase. + +### Step 5: Add `drs_enabled` config flag + +**`src/config/types.rs`** — inside `GeneralConfig` struct (existing struct, find existing `direct_relay_copy_buf_*` fields around line 507): + +```rust +// Controls Dynamic Record Sizing on the direct TLS relay path. +// Safe to disable for debugging; default true when tls mode is active. +#[serde(default = "default_true")] +pub drs_enabled: bool, +``` + +**IMPORTANT — serde compatibility**: New config fields in this PR must have `#[serde(default = "...")]` annotations. Without these, existing config files that lack the fields will fail to deserialize, breaking upgrades. For PR-C this applies to `drs_enabled`. + +Cross-phase note: +- `ipt_enabled` and `ipt_level` belong to later IPT phases; keep them out of PR-C to limit blast radius. + +Add helpers as needed: +```rust +fn default_true() -> bool { true } +fn default_false() -> bool { false } +fn default_ipt_level() -> u8 { 1 } +``` + +If `default_true()` already exists in defaults, reuse it instead of adding duplicates. + +Default: `true`. Validation: no range constraint needed (boolean). In `relay_bidirectional`, pass `drs_enabled: config.general.drs_enabled && is_tls` (gate on both flags at call site). + +**DO NOT** pass `Arc` into `relay_bidirectional` — this would introduce control-plane (config) reads into the data-plane hot loop. Instead, the call site in `direct_relay.rs` computes `let drs_enabled = config.general.drs_enabled && success.is_tls` and passes it as a `bool` concrete parameter. + +### Merge gate + +``` +cargo check --tests +cargo test -- drs_ +cargo test -- relay_ +cargo test -- direct_relay_ +cargo test -- masking_relay_guardrails_ +cargo test -- --test-threads=1 +cargo test -- --test-threads=32 +``` + +All tests above must pass. Any expensive stress case added in PR-C must be ignored by default and executed in dedicated perf/security pipelines. + +--- + +## PR-D — Items 3 + 4a: Adaptive Startup Buffer Sizing + +**Priority**: Medium. Depends on PR-C. + +**PREREQUISITE**: Remove `#![allow(dead_code)]` from `src/proxy/adaptive_buffers.rs` at the start of this PR. The attribute was intentional when the module had zero call sites, but PR-D adds real call sites. Keeping the attribute suppresses legitimate dead-code warnings for any functions that remain unused after wiring. + +### Problem (concrete) + +Most adaptive buffer hardening primitives are already present in `src/proxy/adaptive_buffers.rs` (key length guards, stale removal via `remove_if`, TTL eviction, saturating duration math, caps). The remaining production gap is wiring: `seed_tier_for_user`, `record_user_tier`, and `direct_copy_buffers_for_tier` are still not used by direct relay runtime paths. + +`relay_bidirectional` still accepts `_buffer_pool` only for compatibility. The effective startup sizing is still static (`config.general.direct_relay_copy_buf_*`) until direct relay applies seeded tier sizing at call time. + +`USER_PROFILES` (adaptive_buffers.rs line 233) — `OnceLock>` — is the only remaining global after PR-B. It is acceptable here because it functions as a process-wide LRU cache (cross-session user history), not as test-contaminating per-connection state. + +### Step 1: Write red tests for remaining gaps (must fail on current code) + +**Do not duplicate existing coverage**: The repository already contains extensive adaptive buffer tests (`adaptive_buffers_security_tests.rs`, `adaptive_buffers_record_race_security_tests.rs`) that validate cache bounds, key guards, TOCTOU stale removal, and concurrency behavior. PR-D red tests should focus only on missing runtime integration and throughput mapping behavior. + +**New file**: `src/proxy/tests/adaptive_startup_integration_tests.rs` +Declared in `src/proxy/direct_relay.rs` or `src/proxy/adaptive_buffers.rs` (single declaration site only). + +``` +// RED: direct relay currently ignores seeded tier and always uses static config. +// Assert selected copy buffer sizes follow direct_copy_buffers_for_tier(seed_tier_for_user(user), ...). +adaptive_startup_direct_relay_uses_seeded_tier_buffers + +// RED: no production post-session persistence currently upgrades next session. +// After first relay with high throughput, next seed should reflect recorded upgrade. +adaptive_startup_post_session_recording_upgrades_next_session + +// RED: short sessions (<1s) must never promote tier even under bursty bytes. +adaptive_startup_short_sessions_do_not_promote + +// RED: upgrade path must be monotonic per user within TTL (no downgrade on lower follow-up). +adaptive_startup_recording_remains_monotonic_within_ttl +``` + +Existing adaptive security tests already cover empty keys, oversized keys, fuzz keys, and cache cardinality attacks. Do not reintroduce duplicates in PR-D. + +### Step 2: Keep current hardening, remove outdated dead-code suppression + +Current branch already has the core hardening this step originally proposed: +- `MAX_USER_PROFILES_ENTRIES` and `MAX_USER_KEY_BYTES` +- stale purge with `remove_if` in `seed_tier_for_user` +- `saturating_duration_since`-safe TTL math +- TTL-based `retain` eviction in `record_user_tier` + +Required action in PR-D: +- remove `#![allow(dead_code)]` from `src/proxy/adaptive_buffers.rs` once direct relay wiring lands, so dead paths are visible again. + +No behavioral rewrite of existing `seed_tier_for_user` / `record_user_tier` is required unless new red tests expose regressions. + +### Step 3: Add explicit throughput mapping API + +**`src/proxy/adaptive_buffers.rs`** — new public function: + +```rust +// Computes the peak tier achieved during a session from total byte counts and +// session duration. Uses only throughput because demand-pressure metrics are +// unavailable at session end (copy_bidirectional drains everything). +// Maps average throughput over a session to an adaptive tier. Only peak direction +// (max of c2s or s2c) is considered to avoid double-counting bidir traffic. +// Note: This uses total-session average, not instantaneous peak. Bursty traffic +// (30s burst @ 100 Mbps, 9.5min idle) will compute as the average over all 10 min, +// potentially underestimating required buffers. Consider measuring peak-window +// throughput from watchdog snapshots (10s intervals) in future refinements. +pub fn average_throughput_to_tier(c2s_bytes: u64, s2c_bytes: u64, duration_secs: f64) -> AdaptiveTier { + if duration_secs < 1.0 { return AdaptiveTier::Base; } + let avg_bps = (c2s_bytes.max(s2c_bytes) as f64 * 8.0) / duration_secs; + if avg_bps >= THROUGHPUT_UP_BPS as f64 { AdaptiveTier::Tier1 } + else { AdaptiveTier::Base } +} +``` + +Naming constraint: +- Use `average_throughput_to_tier` consistently. Avoid introducing both `throughput_to_tier` and `average_throughput_to_tier` aliases in production code. + +### Step 4: Wire into `direct_relay.rs` + +**`src/proxy/direct_relay.rs`** — inside `handle_via_direct`, before the call to `relay_bidirectional` (currently line ~280): + +```rust +// Seed startup buffer sizes from cross-session user history. +let initial_tier = adaptive_buffers::seed_tier_for_user(user); +let (c2s_buf, s2c_buf) = adaptive_buffers::direct_copy_buffers_for_tier( + initial_tier, + config.general.direct_relay_copy_buf_c2s_bytes, + config.general.direct_relay_copy_buf_s2c_bytes, +); +let relay_epoch = std::time::Instant::now(); +``` + +Replace the existing `config.general.direct_relay_copy_buf_c2s_bytes` / `s2c_bytes` arguments in the `relay_bidirectional` call with `c2s_buf` / `s2c_buf`. + +After `relay_bidirectional` returns (whatever the result), record the tier: + +```rust +let duration_secs = relay_epoch.elapsed().as_secs_f64(); +let final_c2s = /* session c2s total bytes */; +let final_s2c = /* session s2c total bytes */; +let peak_tier = adaptive_buffers::average_throughput_to_tier(final_c2s, final_s2c, duration_secs); +adaptive_buffers::record_user_tier(user, peak_tier); +``` + +Implementation seam note: +- `relay_bidirectional` currently encapsulates counters internally. To avoid broad API churn, prefer returning a small relay outcome struct that includes final `c2s_bytes` and `s2c_bytes` totals while preserving existing error semantics. +- Keep this seam local to direct relay integration; do not expose `SharedCounters` internals broadly. + +`_buffer_pool` remains in the `relay_bidirectional` signature (Option B: repurposed pathway). Its role is now documented: "parameter reserved for future pool-backed buffer allocation; startup sizing is performed by the caller via `adaptive_buffers::direct_copy_buffers_for_tier`." The underscore prefix is removed (`buffer_pool`) and it is still passed as `Arc::clone(&buffer_pool)` — no functional change. + +### Merge gate + +``` +cargo check --tests +cargo test -- adaptive_buffers_ +cargo test -- adaptive_startup_ +cargo test -- direct_relay_ +cargo test -- --test-threads=1 +cargo test -- --test-threads=32 +``` + +--- + +## PR-E — Item 4b: In-Session Adaptive Architecture Decision Gate + +**Priority**: Blocks PR-G. Depends on PR-D. + +**Execution model correction**: PR-E is a decision-gate phase, so it must distinguish between: +- deterministic correctness/integration tests (required for CI and merge), and +- performance experiments (informational, ignored by default, run on dedicated hardware). + +Do not use throughput/latency benchmark thresholds as hard CI merge gates in this phase. + +### Problem (concrete) + +`SessionAdaptiveController::observe` (adaptive_buffers.rs line 121) is never called. Three structural blockers prevent in-session adaptation on the direct relay path: + +1. `copy_bidirectional_with_sizes` is opaque — no hook to observe buffering pressure mid-loop. +2. `StatsIo` wraps only the client side — no server-side write pressure signal. +3. The watchdog tick is 10 seconds — too coarse for the 250 ms EMA window `observe()` expects. + +The decision gate must produce *measured* evidence, not architectural guesses. + +**Current state note**: `SessionAdaptiveController` and `RelaySignalSample` already exist in `adaptive_buffers.rs`, but there is no production wiring that feeds relay runtime signals into `observe(...)`. + +### Step 1: Required deterministic decision tests (CI required) + +**New file**: `src/proxy/tests/adaptive_insession_decision_gate_tests.rs` +Declared once (single declaration site). + +These tests must be deterministic and runnable on shared CI: + +``` +// Confirms direct relay path has no fine-grained signal hook while copy_bidirectional_with_sizes +// remains opaque; this preserves the architectural constraint as an explicit test. +adaptive_decision_gate_direct_path_lacks_tick_hook + +// Confirms middle relay path exposes configurable flush timing boundary via +// me_d2c_flush_batch_max_delay_us and can produce periodic signal ticks. +adaptive_decision_gate_middle_relay_has_tick_boundary + +// Drives SessionAdaptiveController with deterministic synthetic signal stream and +// verifies promotion/demotion transitions remain stable under fixed tick cadence. +adaptive_decision_gate_controller_transitions_deterministic + +// Confirms proposed signal extraction API (or shim) carries enough fields to support +// observe() without leaking internal relay-only types. +adaptive_decision_gate_signal_contract_is_sufficient +``` + +### Step 2: Optional feasibility experiments (ignored by default) + +**New file**: `src/proxy/tests/adaptive_insession_option_a_experiment_tests.rs` +Declared in `src/proxy/relay.rs`. + +**CI STABILITY WARNING**: These tests measure performance overhead, not correctness. They WILL be flaky on shared CI runners with variable CPU scheduling and memory pressure. **Mark all tests in this file with `#[ignore]`** by default. Run only in isolated performance environments (dedicated runner, pinned cores, no concurrent load). CI gate should skip these; they are for manual decision-making only. + +These tests benchmark overhead, not correctness. Keep `#[ignore]` and never use as merge blockers: + +``` +// Measures latency penalty of adding a per-session 1-second ticker task alongside +// copy_bidirectional_with_sizes using tokio::select!. Records p50/p95/p99 latency +// delta over 1000 relay sessions each transferring 10 MB. +// ACCEPTANCE CRITERION: p99 latency increase < 2 ms; p50 < 0.5 ms. +adaptive_option_a_ticker_overhead_under_acceptance_threshold + +// Measures overhead of adding AtomicU64 s2c_pending_write_count to StatsIo +// and incrementing it in poll_write when Poll::Pending. Records throughput +// delta over 10_000 relay calls. +// ACCEPTANCE CRITERION: throughput regression < 1%. +adaptive_option_a_statsio_pending_counter_overhead_under_1pct + +// Measures overhead of wrapping the server-side write half in a second StatsIo +// (for server-side pressure signal). Records throughput delta. +// ACCEPTANCE CRITERION: throughput regression < 2%. +adaptive_option_a_server_side_statsio_overhead_under_2pct +``` + +### Step 3: Option B boundary validation experiment (ignored by default) + +**New file**: `src/proxy/tests/adaptive_insession_option_b_experiment_tests.rs` +Declared in `src/proxy/middle_relay.rs`. + +``` +// Verifies that middle_relay's explicit ME→client flush loop already provides +// a natural tick boundary at max_delay_us intervals (currently 1000 µs default). +// Records observed tick interval distribution over 500 relay sessions. +// ACCEPTANCE CRITERION: median observed tick ≤ 2× configured max_delay_us. +adaptive_option_b_middle_relay_flush_loop_provides_tick_boundary + +// Verifies SessionAdaptiveController::observe can be driven by ME flush ticks. +// Pumps 2000 synthetic RelaySignalSample values through observe() at 1 ms intervals. +// ACCEPTANCE CRITERION: Tier1 promotion fires at expected tick count consistent +// with TIER1_HOLD_TICKS = 8. +adaptive_option_b_observe_driven_by_flush_ticks_promotes_correctly +``` + +### Step 4: Decision artifact + +**Placement correction**: function renaming and direct relay throughput-to-tier wiring are PR-D tasks, not PR-E tasks. PR-E must not duplicate those implementation steps. + +After running both experiment suites, record the measured values in `docs/ADAPTIVE_INSESSION_DECISION.md` with the format: + +```markdown +## Measured Results + +| Metric | Option A measured | Threshold | Pass/Fail | +|---|---|---|---| +| Ticker task p99 latency delta (ms) | X | < 2 ms | ? | +| StatsIo pending counter throughput delta | X | < 1% | ? | +| Server-side StatsIo throughput delta | X | < 2% | ? | + +| Metric | Option B measured | Threshold | Pass/Fail | +|---|---|---|---| +| Flush tick median vs configured delay | X | ≤ 2× | ? | +| Tier1 promotion tick accuracy | X | exact | ? | + +## Decision: [Option A / Option B] +Rationale: ... +``` + +If Option A passes all thresholds → schedule PR-G-A (relay loop instrumentation). +If Option B passes all thresholds → schedule PR-G-B (middle relay SessionAdaptiveController wiring). +If neither passes → escalate and re-design. + +Decision rule refinement: +- Deterministic CI tests from Step 1 must pass before any option can be selected. +- Performance thresholds from experiments are advisory evidence and must include environment metadata (CPU model, core pinning, load conditions) in the decision doc. + +### Merge gate + +``` +cargo check --tests +cargo test -- adaptive_insession_decision_gate_ +cargo test -- middle_relay_ +cargo test -- --test-threads=1 +cargo test -- --test-threads=32 +``` + +Optional experiment runs (not merge blockers): + +``` +cargo test -- adaptive_option_a_ -- --ignored +cargo test -- adaptive_option_b_ -- --ignored +``` + +--- + +## PR-F — Item 1 Level 1: Log-Normal Single-Delay Replacement + +**Priority**: Medium. **No dependency on PR-B (DI) or PR-C (DRS)** — this PR modifies only the RNG call in `masking.rs` and `handshake.rs`, touching zero global statics or shared state. Can be developed and merged independently after PR-A (baseline tests). The original "Depends on PR-B + PR-C being stable" was an artificial ordering constraint with no code justification. + +### Problem (concrete) + +`mask_outcome_target_budget` (masking.rs line 252, rng calls at lines 261–265) draws from uniform distribution: +```rust +let delay_ms = rng.random_range(floor..=ceiling); +``` + +`maybe_apply_server_hello_delay` (handshake.rs line 586): +```rust +let delay_ms = rand::rng().random_range(min..=max); +``` + +Both produce uniform i.i.d. samples. For a *single* sample this does not matter for classification — you cannot build a histogram from one value. However, replacing uniform with log-normal: +- More accurately models observed real-world TCP RTT distributions (multiplicative central-limit theorem). +- Provides a documented, principled rationale against future attempts to "optimize" the distribution. + +**Current branch status**: +- `mask_outcome_target_budget(...)` already uses `sample_lognormal_percentile_bounded(...)` for the `ceiling > floor > 0` path. +- `maybe_apply_server_hello_delay(...)` already routes through the same helper. +- Extensive masking log-normal tests already exist in `src/proxy/tests/masking_lognormal_timing_security_tests.rs`. + +PR-F is therefore an **incremental hardening + coverage completion** phase, not a greenfield implementation. + +### Cargo.toml change + +**No new dependencies required.** + +**Implementation note correction**: current code uses a Box-Muller transform built from `rng.random_range(...)` to derive a standard normal sample, which is valid and avoids extra dependency surface. Do not force migration to `StandardNormal` unless there is a demonstrated correctness or performance defect. + +**CRITICAL**: avoid adding `rand_distr` because of `rand_core` compatibility risk with the existing `rand` version. + +### Step 1: Write red tests only for missing coverage (must fail on current code) + +**Do not duplicate existing masking log-normal suite.** Extend `src/proxy/tests/masking_lognormal_timing_security_tests.rs` only where gaps remain. + +``` +// Missing gap candidate: helper behavior under extremely narrow range around 1 ms +// remains stable without boundary clamp spikes. +masking_lognormal_ultra_narrow_range_stability + +// Missing gap candidate: floor=0 path remains intentionally uniform and does not +// regress to log-normal semantics. +masking_lognormal_floor_zero_path_regression_guard +``` + +**Add handshake-side coverage explicitly** (new file if needed): `src/proxy/tests/handshake_lognormal_delay_security_tests.rs`. +Rationale: there is no dedicated `handshake_lognormal_` suite yet, and current coverage is mostly indirect through server-hello-delay behavior tests. + +``` +// Deterministic bound check via fixed min==max and bounded timer advancement. +handshake_lognormal_fixed_delay_respected + +// Inverted config safety: max floor` branch: +1. `floor == 0 && ceiling == 0` → returns 0 (unchanged) +2. `floor == 0 && ceiling != 0` → uses `rng.random_range(0..=ceiling)` (uniform) +3. `ceiling > floor` (with `floor > 0`) → uses `rng.random_range(floor..=ceiling)` (uniform → **replace with log-normal**) +4. Fall-through (`ceiling <= floor`) → returns `floor` (unchanged) + +**Only path 3 is replaced.** Path 2 (floor=0) must remain uniform because log-normal cannot meaningfully model a distribution anchored at zero — the `floor.max(1)` guard in `sample_lognormal_percentile_bounded` changes the distribution center to `sqrt(ceiling)`, which is far from the original uniform median of `ceiling/2`. Changing this would alter observable timing behavior for deployments using `floor_ms=0`. + +```rust +// Path 3 replacement only — inside the `if ceiling > floor` block: +let delay_ms = if ceiling == floor { + ceiling +} else { + sample_lognormal_percentile_bounded(floor, ceiling, &mut rng) +}; +``` + +Current helper in `masking.rs` already exists and is `pub(crate)` for handshake reuse. + +If red tests expose issues, patch the existing helper rather than replacing it wholesale: +```rust +use rand::Rng; +// Current implementation uses Box-Muller from uniform draws. + +// Samples a log-normal distribution parameterized so that the median maps to +// the geometric mean of [floor, ceiling], then clamps the result to that range. +// +// Implementation uses Box-Muller-derived N(0,1) from uniform draws. +// Log-normal = exp(mu + sigma * N(0,1)). +// +// For LogNormal(mu, sigma): median = exp(mu). +// mu = (ln(floor) + ln(ceiling)) / 2 → median = sqrt(floor * ceiling). +// sigma = ln(ceiling/floor) / 4.65 → ensures ~99% of samples fall in [floor, ceiling]. +// 4.65 ≈ 2 × 2.326 (z-score for 99th percentile of standard normal). +// +// IMPORTANT: When floor == 0, log-normal parameterization is undefined (ln(0) = -∞). +// We use floor_f = max(floor, 1) for parameter computation but clamp the final +// result to the original [floor, ceiling] range. For floor=0 this produces a +// distribution centered around sqrt(ceiling) — which may differ significantly from +// the original uniform [0, ceiling]. If the caller needs uniform behavior for +// floor=0, it should handle that case before calling this function. +pub(crate) fn sample_lognormal_percentile_bounded(floor: u64, ceiling: u64, rng: &mut impl Rng) -> u64 { ... } +``` + +Safety requirement for this helper: +- misconfigured `floor > ceiling` must remain fail-closed and bounded. +- `floor == 0` path behavior must remain explicit and documented. +- NaN/Inf fallback must remain deterministic and bounded. + +### Step 3: Implement in `handshake.rs` + +Replace in `maybe_apply_server_hello_delay` (line 586): + +```rust +let delay_ms = if max == min { + max +} else { + // Replaced: sample_lognormal_percentile_bounded produces a right-skewed distribution + // with median at geometric mean, matching empirical TLS ServerHello delay profiles. + masking::sample_lognormal_percentile_bounded(min, max, &mut rand::rng()) +}; +``` + +`sample_lognormal_percentile_bounded` must be made `pub(crate)` in `masking.rs` to allow the handshake call. + +Status note: this helper is already `pub(crate)` on the current branch; keep visibility stable. + +Status note: this handshake call-site migration is already present on the current branch; PR-F should verify and lock it with dedicated tests. + +### Merge gate + +``` +cargo check --tests +cargo test -- masking_lognormal_timing_security_ +cargo test -- server_hello_delay_ +cargo test -- masking_ab_envelope_blur_integration_security # regression gate +cargo test -- masking_timing_normalization_security # regression gate +cargo test -- --test-threads=1 +cargo test -- --test-threads=32 +``` + +--- + +## PR-G — Item 1 Level 2: State-Aware Inter-Packet Timing (Burst/Idle Markov) + +**Priority**: Medium. Depends on PR-E decision gate. Separate PR, design depends on PR-E outcome. + +### Problem (concrete) + +No inter-packet timing (IPT) mechanism exists on the MTProto relay path (confirmed: no `IptController` anywhere in the codebase). Real HTTPS sessions exhibit two-state autocorrelation: Burst (1–5 ms IPG, 0.95 self-transition) and Idle (2–10 seconds IPG, heavy-tail, 0.99 self-transition). ML classifiers detect the absence of this structure directly from the time-series, regardless of marginal distribution shape. + +**ARCHITECTURAL BLOCKER (PR-G for direct relay)**: IPT requires injecting delays between write/flush cycles, which `tokio::io::copy_bidirectional_with_sizes` does not support. Adding IPT on the direct relay path requires **replacing** `copy_bidirectional_with_sizes` with a custom poll loop that calls `ipt_controller.next_delay_us()` and `tokio::time::sleep()` between write events. This is substantial work (equivalent to ~300-line custom relay loop). **Decision**: IPT for direct relay is deferred to a decision gate (PR-E); if approved, PR-G will require a dedicated custom loop. Middle relay (ME→client) has an explicit flush loop (middle_relay.rs line 1200+) where IPT can be added more easily. + +**CRITICAL DESIGN FIX — DATA-AVAILABILITY AWARENESS**: The original IptController is a purely stochastic model with no awareness of whether data is actually waiting to be sent. The Idle state injects 2–30 second delays **unconditionally**, even when Telegram has queued data for the client. This would cause Telegram client timeouts and connection drops during active sessions. + +**Required fix**: IptController must be **signal-driven**, not purely probabilistic: +- **Burst delays** (0.5–10 ms) are applied only when data is actively flowing (relay has data in buffers). This adds realistic inter-packet jitter without stalling delivery. +- **Idle state** is entered when the relay observes **genuine idle** (no data received from Telegram for a configurable threshold, e.g. 500ms). During genuine idle, DPI already sees no packets — consistent with browser idle. No artificial delay injection is needed. +- **Synthetic keep-alive timing** (optional, Level 3 enhancement): during genuine idle periods, inject small padding records at browser-like intervals to maintain the illusion of an active HTTPS session. This requires FakeTLS padding support. +- The `next_delay_us()` API must accept a `has_pending_data: bool` signal from the caller. When `has_pending_data == true`, the controller stays in Burst regardless of the Markov transition. When `has_pending_data == false` for the idle threshold, the controller transitions to Idle but does NOT inject delays — it simply stops the flush loop until new data arrives. + +This means: +```rust +pub(crate) fn next_delay_us(&mut self, rng: &mut impl Rng, has_pending_data: bool) -> u64 { + if has_pending_data { + // Data waiting: always use Burst timing, regardless of Markov state. + // Markov still transitions (for statistics/logging) but delay is Burst. + self.maybe_transition(rng); + let d = self.burst_dist.sample(rng).max(0.0) as u64; + return d.saturating_mul(1_000).clamp(500, 10_000); + } + // No data pending: return 0 (caller should wait for data, not sleep). + // The caller's recv() timeout on the data channel provides natural idle timing. + 0 +} +``` + +This PR is conditional on PR-E. The exact implementation path (A or B) is determined by the PR-E decision artifact. The test specifications below apply to whichever path is chosen. + +### Step 1: Write red tests (must fail on current code) + +**New file**: `src/proxy/tests/relay_ipt_markov_unit_tests.rs` + +``` +// IptController starts in Burst state. +ipt_controller_initial_state_is_burst + +// Deterministic transition oracle: with an injected decision stream that forces +// "switch" on first call, state toggles Burst -> Idle exactly once. +ipt_controller_forced_transition_toggle_oracle + +// In Burst state, next_delay_us(rng, has_pending_data=true) returns value +// consistent with LogNormal(mu=1.0, sigma=0.5) * 1000, clamped to [500, 10_000] µs. +ipt_controller_burst_delay_within_burst_bounds + +// Internal Markov state: even though next_delay_us returns 0 when !has_pending_data, +// the Markov chain still transitions. Verify idle_dist sampling works correctly +// when called directly (for future Level 3 keep-alive timing). +// Pareto heavy-tail: minimum = 2_000_000 µs, P(>10s) ≈ 9%. +ipt_controller_idle_dist_sampling_correct + +// Deterministic Markov behavior: with an injected decision stream of +// stay/stay/switch, verify exact state sequence without probabilistic thresholds. +ipt_controller_markov_sequence_deterministic_oracle + +// Compile-time trait check can be kept if needed, but no CI memory-growth or +// wall-clock budget assertions in merge gates. +ipt_controller_trait_bounds_compile_check + +// DATA-AWARENESS: next_delay_us(rng, has_pending_data=true) always returns +// Burst-range delay, even if Markov state is Idle. Verifies that active data +// transfer is never stalled by Idle-phase delays. +ipt_controller_pending_data_forces_burst_delay + +// DATA-AWARENESS: next_delay_us(rng, has_pending_data=false) returns 0, +// signaling the caller to wait for data arrival (no artificial sleep). +ipt_controller_no_pending_data_returns_zero + +// Adversarial: IptController with f64 overflow — burst_dist.sample() returning +// very large values must not overflow on saturating_mul(1_000). Verify clamp +// catches extreme samples. +ipt_controller_burst_sample_overflow_safe + +// Adversarial: idle_dist.sample() returning f64::INFINITY or f64::NAN +// (edge case of Pareto distribution). Cast to u64 must not panic; clamp +// handles gracefully. +ipt_controller_idle_sample_extreme_f64_safe +``` + +**New file**: `src/proxy/tests/relay_ipt_integration_tests.rs` + +``` +// Relay path with IPT enabled: 200 calls alternating has_pending_data true/false. +// Verify that true calls always return Burst-range delays and false calls +// always return 0. +relay_ipt_data_availability_signal_respected + +// Adversarial: active prober sends 100 handshakes with invalid keys. +// IPT must not affect the fallback-to-masking behavior or reveal proxy identity +// through timing structure (timing envelope in fallback path is unchanged). +relay_ipt_invalid_handshake_fallback_timing_unchanged + +// Adversarial: censor injects 10_000 back-to-back packets at 0-delay +// (has_pending_data=true for all). Verify relay does not stall excessively +// (total added IPT delay < 10% of transfer time for a 1 MB payload at 10 Mbps). +relay_ipt_overhead_under_high_rate_attack_within_budget + +// Config kill-switch: ipt_enabled = false → no delay injected. +relay_ipt_disabled_by_config_no_delay_added +``` + +Optional (non-merge-gate) performance experiments: +``` +relay_ipt_burst_delays_exhibit_positive_autocorrelation +relay_ipt_500_concurrent_throughput_within_5pct_baseline +``` + +### Step 2: Implement `IptController` + +**New file**: `src/proxy/ipt_controller.rs` +Declare `pub(crate) mod ipt_controller;` in `src/proxy/mod.rs`. + +```rust +use rand::Rng; + +pub(crate) enum IptState { Burst, Idle } + +// Log-normal parameters for Burst-state inter-packet delay. +// mu=1.0, sigma=0.5 → median ≈ exp(1.0) ≈ 2.7 ms. +const BURST_MU: f64 = 1.0; +const BURST_SIGMA: f64 = 0.5; +const BURST_DELAY_MIN_US: u64 = 500; +const BURST_DELAY_MAX_US: u64 = 10_000; +// Pareto parameters for Idle-state delay (retained for future Level 3 keep-alive). +// scale=2_000_000 µs (2s minimum), shape=1.5 → heavy tail. +const IDLE_PARETO_SCALE: f64 = 2_000_000.0; +const IDLE_PARETO_SHAPE: f64 = 1.5; + +pub(crate) struct IptController { + state: IptState, + // Pre-computed Burst/Idle transition probabilities. + burst_stay_prob: f64, // 0.95 + idle_stay_prob: f64, // 0.99 +} + +impl IptController { + pub(crate) fn new() -> Self { + Self { + state: IptState::Burst, + burst_stay_prob: 0.95, + idle_stay_prob: 0.99, + } + } + + fn maybe_transition(&mut self, rng: &mut impl Rng) { + // random_bool(p) returns true with probability p, using u64 threshold + // internally for full precision. Simpler than manual u32 threshold. + let stay = match self.state { + IptState::Burst => rng.random_bool(self.burst_stay_prob), + IptState::Idle => rng.random_bool(self.idle_stay_prob), + }; + if !stay { + self.state = match self.state { + IptState::Burst => IptState::Idle, + IptState::Idle => IptState::Burst, + }; + } + } + + // Burst delay via log-normal: exp(mu + sigma * N(0,1)). + // Use dependency-free Box-Muller (same project pattern as masking helper) + // to avoid any additional RNG-distribution dependency churn. + fn sample_burst_delay_us(&self, rng: &mut impl Rng) -> u64 { + let u1 = rng.next_f64().max(f64::MIN_POSITIVE); + let u2 = rng.next_f64(); + let normal = (-2.0 * u1.ln()).sqrt() * (2.0 * std::f64::consts::PI * u2).cos(); + let raw = (BURST_MU + BURST_SIGMA * normal).exp(); + let us = if raw.is_finite() { + (raw as u64).saturating_mul(1_000) + } else { + // exp(1.0) ≈ 2718 → 2_718_000 µs as fallback (won't happen in practice) + 2_718_000 + }; + us.clamp(BURST_DELAY_MIN_US, BURST_DELAY_MAX_US) + } + + // Idle delay via Pareto CDF inversion: scale / U^(1/shape). + // Retained for future Level 3 synthetic keep-alive timing. + // NOTE: Currently dead code — next_delay_us returns 0 when !has_pending_data. + #[allow(dead_code)] + fn sample_idle_delay_us(&self, rng: &mut impl Rng) -> u64 { + let u: f64 = rng.random_range(f64::EPSILON..1.0); + let raw = IDLE_PARETO_SCALE / u.powf(1.0 / IDLE_PARETO_SHAPE); + if raw.is_finite() { + (raw as u64).clamp(2_000_000, 30_000_000) + } else { + 2_000_000 + } + } + + // Returns inter-packet delay in microseconds. + // `has_pending_data`: true when the relay has queued data awaiting flush. + // When true, always returns a Burst-range delay — active data must never be + // stalled by Idle-phase pauses (which would cause Telegram client timeouts). + // When false, returns 0 — the caller should block on its data channel recv(), + // which provides natural idle timing matching genuine browser think-time. + pub(crate) fn next_delay_us(&mut self, rng: &mut impl Rng, has_pending_data: bool) -> u64 { + self.maybe_transition(rng); + if has_pending_data { + self.sample_burst_delay_us(rng) + } else { + 0 + } + } +} +``` + +**CHANGES vs previous draft:** +1. **No new distribution dependency** — uses Box-Muller for log-normal and manual CDF inversion for Pareto. +2. **`random_bool(p)` for Markov transitions** — replaces manual u32 threshold computation. Cleaner, equivalent precision. +3. **`idle_dist` explicitly marked `#[allow(dead_code)]`** — `next_delay_us` returns 0 when `!has_pending_data`, so idle sampling is never reached in production. Retained for future Level 3 keep-alive. +4. **No stored distribution objects** — parameters are constants, sampling is inline. Avoids the `expect()` calls that would be denied by `clippy::expect_used`. + +### Step 3: Integrate into relay path + +Conditional on PR-E outcome: + +- **Option B path** (recommended if PR-E selects B): wire `IptController` into the ME→client flush loop in `middle_relay.rs`. Each flush cycle calls `ipt_controller.next_delay_us(&mut rng, has_pending_data)` where `has_pending_data = !frame_buf.is_empty()`, then `tokio::time::sleep(Duration::from_micros(delay))` before the next flush. When `delay == 0`, the loop blocks on `me_rx.recv()` naturally. +- **Option A path**: replace `copy_bidirectional_with_sizes` with a custom poll loop that calls `ipt_controller.next_delay_us(rng, has_pending_data)` between write completions, checking the read buffer for pending data. + +Config flag in `src/config/types.rs`: +```rust +#[serde(default = "default_false")] +pub ipt_enabled: bool, // default: false (opt-in) +#[serde(default = "default_ipt_level")] +pub ipt_level: u8, // 1 = single-delay only, 2 = Markov; default 1 +``` + +### Merge gate + +``` +cargo check --tests +cargo test -- relay_ipt_markov_unit_ +cargo test -- relay_ipt_integration_ +cargo test -- --test-threads=1 +cargo test -- --test-threads=32 +``` + +--- + +## PR-H — Consolidated Hardening, ASVS L2 Audit, and Documentation + +**Depends on**: All prior PRs. + +### ASVS L2 Verification Checklist for Changed Areas + +| ASVS Control | Area | Verification | +|---|---|---| +| V5.1.1 Input validation | `record_user_tier` user key length | `MAX_USER_KEY_BYTES = 512` guard in place | +| V5.1.3 Output encoding | DRS framing | No user-controlled field affects record size calculation | +| V5.1.1 Input validation | `IptController.next_delay_us` | `has_pending_data` signal is a bool from trusted internal code; no external input reaches IptController directly | +| V8.1.1 Memory safety | `DrsWriter`, `IptController` | No `unsafe` blocks; all bounds enforced by Rust type system; `saturating_mul` prevents overflow in IptController burst sampling | +| V8.3.1 Sensitive data in memory | `ProxySharedState` | auth key material remains in `HandshakeSuccess` on stack; not copied into shared state | +| V11.1.3 TLS config | DRS | TLS path enabled only when `is_tls=true`; non-TLS path unmodified | +| V11.1.4 Cipher strength | n/a | No cryptographic changes in this plan | +| V2.1.5 Brute force | Auth probe | Probe state in `ProxySharedState.handshake.auth_probe`; per-IP saturation preserved | +| V6.2.2 Algorithm strength | Log-normal RNG | Box-Muller-based bounded sampler with finite checks and deterministic fallback/clamp; no panic path | +| V14.2.1 Configuration hardening | serde defaults | All new config fields have `#[serde(default)]` for backward-compatible deserialization | +| V1.4.1 Concurrency | `ProxySharedState` mutex type | Uses `std::sync::Mutex`; locks never held across await points; lock ordering documented | + +### Full test run command sequence + +```sh +# Run all proxy tests +cargo test -p telemt -- proxy:: + +# Run targeted gate for each PR area +cargo test -- relay_baseline_ +cargo test -- handshake_baseline_ +cargo test -- middle_relay_baseline_ +cargo test -- masking_baseline_ +cargo test -- proxy_shared_state_ +cargo test -- drs_ +cargo test -- adaptive_startup_ +cargo test -- adaptive_option_ +cargo test -- masking_lognormal_ +cargo test -- handshake_lognormal_ +cargo test -- ipt_ + +# Full regression (must show zero failures) +cargo test +``` + +### Documentation changes + +**`docs/CONFIG_PARAMS.en.md`** — add entries for each new `GeneralConfig` field: + +| Field | Default | Description | +|---|---|---| +| `drs_enabled` | `true` | Enable Dynamic Record Sizing on TLS direct relay path. Disable for debugging. | +| `ipt_enabled` | `false` | Enable state-aware inter-packet timing on relay path. Opt-in; requires testing in your network environment. | +| `ipt_level` | `1` | IPT level: 1 = log-normal single-delay only, 2 = Burst/Idle Markov chain. | + +**`ROADMAP.md`** — mark completed items from this plan. + +--- + +## Architectural Decisions & Key Findings from Audit + +### D1: PR Ordering — Swap PR-C and PR-B + +**Audit finding**: PR-C (DRS) is self-contained; PR-B (DI) has a 2000+ line blast radius. + +**Decision**: **YES, swap PR-C → PR-B in execution order**. Rationale: +- **PR-C dependencies**: Only `is_tls` (field in `HandshakeSuccess`, already exists) + new `drs_enabled` config flag. Zero dependency on DI migration. +- **PR-C value**: Delivers immediate anti-censorship benefit for direct relay TLS path. +- **PR-B risk mitigation**: If DI refactor hits unforeseen complexity, DRS remains deliverable independently. +- **Execution parallelization**: PR-C and PR-B can have their test suites written in parallel (PR-A → PR-C tests + PR-B tests in parallel) → PR-C production code → PR-B production code (sequential due to shared entry points). + +**Updated graph**: +``` +PR-A (baseline gates) +├─→ PR-C (DRS, independent) +├─→ PR-F (log-normal, independent) +└─→ PR-B (DI migration) + └─→ PR-D (adaptive startup) + └─→ PR-E (decision gate) +``` + +--- + +### D2: PR-B Phasing (Single Atomic vs Shim+Removal) + +**Audit suggestion**: Two-phase with compatibility shim to reduce blast radius. + +**Decision**: **NOT phased — single atomic PR-B**. Rationale: +- A shim (global `ProxySharedState::default()` instance) would live only through one release cycle, complicating both phases. +- With high test coverage from PR-A, full replacement is safer than partial compatibility. +- Parallel test execution gates (`cargo test -- --test-threads=32`) will catch test interference before merge. +- **Mitigation**: Sequence the changes: `ProxySharedState` creation first → all accessors updated → test helpers removed. Review in logical chunks per file. + +--- + +### D3: PR-C Scope — Direct Relay Only, Middle Relay Gap + +**Audit finding**: Middle relay (the default mode) is not covered; this is a CVE-level coverage gap. + +**Decision**: **KNOWN LIMITATION with ELEVATED follow-up priority**. Document explicitly: +- PR-C covers direct relay path only (direct_relay.rs → relay_bidirectional). +- Middle relay path (middle_relay.rs → explicit ME→client loop) requires separate PR-C.1 (follow-up). +- **Middle relay is the DEFAULT** for deployments with configured ME URLs, which is the typical production setup. Direct relay is used when `use_middle_proxy=false` or no ME pool is available. +- **PR-C.1 must be elevated to the same priority as PR-C** (High, anti-censorship). It should begin development immediately after PR-C merges, not be treated as a casual follow-up. Middle relay has natural flush-tick points that make DRS integration architecturally simpler than direct relay. +- **Action**: Add a `docs/DRS_DEPLOYMENT_NOTES.md` with guidance documenting which relay modes have DRS coverage and which are pending PR-C.1. + +--- + +### D4: DRS MaybeDrs Enum Overhead + +**Audit finding**: `MaybeDrs` enum adds a branch dispatch per poll. + +**Decision**: **ACCEPTABLE**. The dispatch overhead (~3-5 cycles with branch prediction) is negligible vs TLS crypto, I/O latency, and network RTT. Do NOT attempt zero-overhead abstractions (e.g., generic specialization); the complexity is not worth the unmeasurable gain. Document the assumption in code comments. + +--- + +### D5: Log-Normal Distribution Parameterization — CRITICAL FIX + +**Audit finding**: Fixed `sigma=0.5` creates an 18% clamp spike at `ceiling`, detectable by DPI. + +**Decision**: **FIXED** (already applied above). New parameterization: +- mu = (ln(floor) + ln(ceiling)) / 2 → median = sqrt(floor * ceiling) (geometric mean, NOT arithmetic mean) +- sigma = ln(ceiling/floor) / 4.65 → ensures ~99% of samples fall within [floor, ceiling] +- Result: NO spike; distribution smoothly bounded. +- Function renamed to `sample_lognormal_percentile_bounded` to reflect guarantee. +- **Mathematical note**: The median of this distribution is the geometric mean sqrt(floor * ceiling), which differs from the arithmetic mean (floor+ceiling)/2 for asymmetric ranges. Tests must assert against the geometric mean, not the arithmetic mean. + +--- + +### D6: IptController pre-construction to avoid unwrap() + +**Audit finding**: `LogNormal::new().unwrap()` in `next_delay_us` won't compile under `deny(clippy::unwrap_used)`. + +**Decision**: **FIXED** (redesigned above). IptController now uses inline Box-Muller sampling for the normal component and manual Pareto CDF inversion. No distribution objects are stored; no `unwrap()`/`expect()` calls needed. The `random_bool(p)` API replaces manual u32 threshold computation for Markov transitions. + +--- + +### D7: Adaptive Buffers — Stale Entry Leak + +**Audit finding**: `seed_tier_for_user` returns Base for expired profiles but doesn't remove them; cache fills with stale entries. + +**Decision**: **FIXED** (already applied above). Two changes: +1. `seed_tier_for_user` now uses `DashMap::remove_if` with a TTL predicate (atomic, avoids TOCTOU race where concurrent `record_user_tier` inserts a fresh profile between `drop(entry)` and `remove(user)`). +2. `record_user_tier` uses TTL-based `DashMap::retain()` for overflow eviction (single O(n) pass, removes stale entries when cache exceeds `MAX_USER_PROFILES_ENTRIES`). This replaces the originally proposed "oldest N by LRU" strategy which would have required O(n log n) sorting + double-shard-locking. + +--- + +### D8: throughput_to_tier Metric — Average Not Peak + +**Audit finding**: Function computes average throughput over entire session; bursty traffic is underestimated. + +**Decision**: **RENAMED + DOCUMENTED**. New name: `average_throughput_to_tier` makes the limitation explicit. Comment documents: "Uses total-session average, not instantaneous peak. Consider peak-window measurement from watchdog snapshots as a future refinement." Users deploying in bursty-traffic environments should consider manual tier pinning via config until this limitation is addressed. + +--- + +## Answers to Audit Open Questions + +> **Q1: PR-B phasing — single atomic PR-B or split Phase 1 (shim) + Phase 2 (production threading)?** + +**A1**: Proceed with single atomic PR-B. The shim approach delays clean state and complicates review. High test coverage from PR-A mitigates risk. Use sequential sub-phases within the PR (ProxySharedState creation → accessors → test helpers) and require parallel test execution gates before merge. + +--- + +> **Q2: Middle relay DRS — should PR-C also address ME→client path, or is that a follow-up?** + +**A2**: Follow-up (PR-C.1) **at the same priority level as PR-C** (High). Direct relay DRS is the initial deliverable; it's self-contained. However, middle relay is the **default production mode** for deployments with configured ME URLs, making PR-C.1 critical. Middle relay has a different architecture (explicit flush loop, not copy_bidirectional) and warrants separate implementation, but must begin immediately after PR-C merges. Annotate PR-C: "Coverage: Direct relay only. Middle relay DRS planned for next release." + +--- + +> **Q3: PR-C → PR-B dependency reversal — are you OK with reversing the order to deliver DRS first?** + +**A3**: **YES, change the dependency order to PR-C → PR-B**. DRS is lower-risk, higher-value, and independent of DI. This improves parallelization and reduces the critical path. Update the plan's PR Dependency Graph accordingly. + +--- + +> **Q4: `copy_bidirectional` replacement for IPT — is the team prepared to write a custom poll loop for PR-G Option A (direct relay)?** + +**A4**: **Document as a risk item for PR-E decision gate**. If PR-E chooses Option A (direct relay IPT), a custom poll loop is **mandatory** — `copy_bidirectional` is not compatible. Estimate: ~300-line custom relay loop + full test matrix. This is non-trivial. PR-E experiments should include a prototype of the custom loop to validate feasibility before committing. If the team is not prepared for ~2-3 weeks of dedicated work on the relay loop, **choose Option B** (middle relay only for in-session IPT). + +**IMPORTANT ADDENDUM**: The IptController has been redesigned to be **data-availability-aware** (see F14). The original purely-stochastic Idle model would have broken active Telegram connections by injecting 2–30 second delays unconditionally. The redesigned controller only applies Burst delays when data is pending; idle timing is handled naturally by the caller's `recv()` blocking on the data channel. This simplifies the Option A custom loop (no need for tokio::time::sleep with variable durations — just a short fixed sleep in the poll loop when data is available). + +--- + +> **Q5: Log-normal sigma — dynamic computation or fixed 0.5?** + +**A5**: **Use dynamic computation** (already fixed above). Parameterize so ~99% of samples fall in [floor, ceiling], with median at the geometric mean sqrt(floor*ceiling). Function: `sample_lognormal_percentile_bounded(floor, ceiling, rng)`. + +--- + +## Out-of-Scope Boundaries + +- No AES-NI changes: the `aes` crate performs runtime CPUID detection automatically. +- No sharding of `USER_PROFILES` DashMap: no measured bottleneck exists. +- No monolithic PRs: each item has its own branch and review cycle. +- No relaxation of red test assertions without a proven code fix — tests are the ground truth. + +--- + +## Critical Review — Issues Found and Fixed + +This section documents all issues found during critical review of the original plan, whether they were corrected inline (above) or require explicit acknowledgement. + +### Fixed Inline (code/plan corrections applied above) + +| # | Issue | Severity | Fix | +|---|---|---|---| +| F1 | PR-B "Blocks PR-C" contradicts D1 decision to swap ordering | Medium | PR-B header updated to "Blocks PR-D" only | +| F2 | Static line numbers wrong (handshake.rs: 71→52, 72→53, 74→55, 30→33, 32→39; middle_relay.rs: 63→62) | Low | Corrected to match actual source | +| F3 | `_for_testing` helper line numbers wrong across both files | Low | Corrected to match actual source | +| F4 | `handle_tls_handshake` line reference 638→690; `handle_mtproto_handshake` 840→854; `client.rs` call sites wrong | Low | Corrected | +| F5 | `DrsWriter.records_completed` overflow on 32-bit: wraps after ~4B records, restarts DRS ramp (detectable signature) | High | Capped via `.saturating_add(1).min(DRS_PHASE_FINAL + 1)` | +| F6 | DRS TLS overhead comment assumed real TLS 1.3 (22 bytes), but FakeTlsWriter only adds 5-byte header (no AEAD, no content-type byte). Wire record = 1369 + 5 = 1374, NOT 1391 | **High** | Comment corrected to reflect FakeTLS overhead; constant 1369 retained as conservative value with 74-byte MSS margin | +| F7 | Log-normal median math error: `mu = (ln(f) + ln(c))/2` → median = sqrt(f*c) (geometric mean), NOT (f+c)/2 (arithmetic mean) | **Critical** | Test assertions and comments rewritten to assert geometric mean; function renamed to `sample_lognormal_percentile_bounded` | +| F8 | `seed_tier_for_user` TOCTOU race: `drop(entry)` then `remove(user)` can delete a fresh profile inserted between the two calls | High | Replaced with `DashMap::remove_if` with TTL predicate (atomic) | +| F9 | `record_user_tier` eviction strategy: "evict oldest N" requires O(n log n) + double shard-locking; `retain()` cannot select by count | Medium | Replaced with TTL-based `retain()` — single O(n) pass, removes stale entries | +| F10 | `IptController` Pareto idle clamp `[500_000, 30_000_000]`: lower bound 0.5s is dead code (Pareto minimum = scale = 2s) | Low | Lower clamp corrected to `2_000_000` with explanatory comment | +| F11 | D3 claim "Most deployments should use direct relay where possible" is misleading — middle relay is the default when ME URLs are configured | Medium | Rewritten to accurately describe both deployment modes | +| F12 | DRS scope: Missing `LOGGED_UNKNOWN_DCS` and `BEOBACHTEN_*_WARNED` from PR-B static inventory (direct_relay.rs line 24, client.rs lines 81, 88) | Medium | Added to PR-B table as lower-priority follow-up | +| F13 | `IptController` threshold approximation: P(stay) ≈ 0.95000000047 due to u32 truncation, not exactly 0.95 | Low | Comment added documenting the approximation | +| F14 | `IptController` Idle state injects 2–30s delays unconditionally, breaking active Telegram connections (Telegram client timeouts) | **Critical** | IptController redesigned to be data-availability-aware: `next_delay_us(rng, has_pending_data)`. When data is pending, always returns Burst-range delay. When idle, returns 0 (caller blocks on data channel naturally). | +| F15 | Test file `proxy_shared_state_isolation_tests.rs` declared in TWO modules (handshake.rs AND middle_relay.rs) via `#[path]` — causes duplicate symbol compilation errors | **Critical** | Changed to single declaration in `src/proxy/mod.rs` only | +| F16 | PR-F (log-normal) had artificial dependency on PR-B (DI) — zero code dependency exists; modifies only two `rng.random_range()` call sites | High | Made PR-F independent; can land after PR-A only | +| F17 | New config fields `drs_enabled`, `ipt_enabled`, `ipt_level` lacked `#[serde(default)]` annotations — existing config.toml files would fail to deserialize on upgrade | High | Added `#[serde(default = "...")]` annotations with helper functions | +| F18 | ProxySharedState `Mutex` type unspecified (std::sync vs tokio::sync) — incorrect choice causes async runtime issues | High | Explicitly specified `std::sync::Mutex` with rationale (short critical sections, no await points inside locks) | +| F19 | DRS architecture note showed `client_writer` as "actual TLS/TCP socket" — it's actually `CryptoWriter>` with internal buffering | High | Corrected call chain diagram to show CryptoWriter + FakeTlsWriter layers with buffering interaction documentation | +| F20 | DRS `DRS_FULL_RECORD_PAYLOAD = 16_384` was documented as "becomes a no-op" but `FakeTlsWriter` uses `MAX_TLS_CIPHERTEXT_SIZE = 16_640` — DRS still shapes in steady-state | Medium | Comment corrected; DRS at 16_384 intentionally mimics RFC 8446 plaintext limit | +| F21 | `IptController` burst sample: `(sample as u64) * 1_000` can overflow for extreme LogNormal tail values | Medium | Changed to `(sample as u64).saturating_mul(1_000)` with `.max(0.0)` guard for negative edge cases | +| F22 | PR-C.1 (middle relay DRS) was treated as casual follow-up but middle relay is the DEFAULT production mode | High | Elevated PR-C.1 to same priority as PR-C; must begin immediately after PR-C merges | +| F23 | `#![allow(dead_code)]` on adaptive_buffers.rs not planned for removal in PR-D | Medium | Added prerequisite to PR-D: remove the attribute when call sites are added | +| F24 | PR-E experiment tests (`adaptive_option_a_*`, `adaptive_option_b_*`) are performance benchmarks that will be flaky on shared CI runners | Medium | Added `#[ignore]` requirement; run only in isolated performance environments | +| F25 | `rand_distr = "0.5"` is incompatible with `rand = "0.10"` — `rand_distr 0.5` depends on `rand_core 0.9`; trait mismatch prevents compilation | **Critical** | Removed `rand_distr` dependency; replaced with manual log-normal via Box-Muller and manual Pareto CDF inversion. Zero new dependencies needed. | +| F26 | `sample_lognormal_percentile_bounded` with `floor=0`: `floor.max(1)` avoids ln(0) but silently shifts distribution center from `ceiling/2` (uniform) to `sqrt(ceiling)` (log-normal) — massive semantic change | **High** | Documented explicitly: only path 3 (`floor > 0 && ceiling > floor`) uses log-normal. Path 2 (`floor == 0`) retains uniform distribution. | +| F27 | `seed_tier_for_user` / `record_user_tier` use `duration_since` which panics if `seen_at > now` (concurrent Instant reordering in remove_if predicate) | **High** | Replaced all TTL predicates with `saturating_duration_since` — returns `Duration::ZERO` when `seen_at > now`, treating entry as fresh (safe). | +| F28 | IptController used `rand_distr::{LogNormal, Pareto}` (incompatible with rand 0.10) and pre-stored distribution objects requiring `expect()` (denied by clippy) | **Critical** | Redesigned: inline Box-Muller sampling for log-normal, manual CDF inversion for Pareto. `random_bool(p)` for Markov transitions. No stored objects, no `expect()`. | +| F29 | `ipt_level: u8` config field violates Architecture.md §4 (enums over magic numbers) | Low | Should be `enum IptLevel { SingleDelay, MarkovChain }` with `#[serde(rename_all = "snake_case")]`. | +| F30 | PR-A `test_harness_common.rs` declared via `#[path]` in three modules → triple duplicate symbol compilation failure | **Critical** | Declared once in `proxy/mod.rs`; imported via `use crate::proxy::test_harness_common::*` in consuming tests | +| F31 | PR-A `RecordingWriter` stored `Vec>` with ambiguous write-vs-flush boundaries; DRS tests (PR-C) need flush-boundary tracking | **High** | Dual-tracking design: `writes` (per poll_write) + `flushed` (per poll_flush boundary with accumulator) | +| F32 | PR-A `SliceReader` required `bytes` crate for no gain; `tokio::io::duplex()` already used everywhere | **High** | **Dropped** from test harness | +| F33 | PR-A `PendingWriter` only controlled `poll_write` pending; DRS flush-pending tests (`drs_pending_on_flush_propagates_pending_without_spurious_wake`) need separate flush control | Medium | Renamed to `PendingCountWriter` with separate `write_pending_remaining` and `flush_pending_remaining` counts | +| F34 | PR-A `relay_baseline_watchdog_delta_does_not_panic_on_u64_wrap` duplicates 7 existing tests in `relay_watchdog_delta_security_tests.rs` | **Critical** | **Dropped** — existing test file already provides exhaustive coverage including wrap, overflow, fuzz | +| F35 | PR-A `handshake_baseline_saturation_fires_at_configured_threshold` implies runtime config but `AUTH_PROBE_BACKOFF_START_FAILS` is a compile-time constant | Low | Renamed to `_compile_time_threshold` | +| F36 | PR-A middle_relay baseline tests directly poked global statics that PR-B removes | **High** | Rewritten to test through public functions (`mark_relay_idle_candidate`, `clear_relay_idle_candidate`) whose signatures survive PR-B | +| F37 | PR-A had zero masking baseline tests despite masking being the primary anti-DPI component and PR-F modifying it | **High** | Added `masking_baseline_invariant_tests.rs` with timing budget, fallback relay, consume-cap, and adversarial tests | +| F38 | PR-A had no error-path baseline tests — only happy paths locked | **High** | Added: simultaneous-close, broken-pipe, and many-small-writes relay baselines | +| F39 | PR-A `relay_baseline_empty_transfer_completes_without_error` was vague (no sharp assertions) | Medium | Replaced with `relay_baseline_zero_bytes_returns_ok_and_counters_zero` | +| F40 | PR-A `test_stats()` and `test_buffer_pool()` are trivial wrappers for one-liner constructors already inlined everywhere | Medium | **Dropped** from test harness to avoid unnecessary indirection | +| F41 | PR-A `seeded_rng` limitation not documented: cannot substitute for `SecureRandom` in production function calls | Medium | Documented as explicit limitation in code comment | +| F42 | PR-A no test isolation strategy documented for auth_probe global state contention | Medium | Each handshake baseline test acquires `auth_probe_test_lock()`, calls `clear_auth_probe_state_for_testing()`. Documented as temporary coupling eliminated in PR-B | +| F43 | PR-A was not split into sub-phases; utility iteration could block baseline tests | **High** | Split into PR-A.1 (utilities, compile-only gate) and PR-A.2 (baseline tests, all-green gate) | +| F44 | `sample_lognormal_percentile_bounded` and 14 masking lognormal tests already exist in codebase (masking.rs:258, masking_lognormal_timing_security_tests.rs). PR-F describes implementing what's already done. | **High** | PR-F's remaining scope: verify handshake.rs integration (already wired at line 596). PR-F may already be complete — audit needed before starting. | +| F45 | PR-A `handshake_test_config()` was missing; `tls_only_config()` alone is insufficient for handshake baseline tests requiring user/secret/masking config | **High** | Added `handshake_test_config(secret_hex)` to test harness | +| F46 | Previous external review C1 (DRS write-chain placement "fundamentally wrong") is **INCORRECT** — see R3/R6 in Acknowledged Risks. Each DrsWriter.poll_write passes ≤ target bytes to CryptoWriter in one call. CryptoWriter passes through to FakeTlsWriter in one call. FakeTlsWriter creates exactly one TLS record per poll_write. Flush at record boundary ensures CryptoWriter's pending buffer is drained before the next record starts. Chain is correct. | **Informational** | No plan change needed; external finding was wrong. | +| F47 | `BEOBACHTEN_*_WARNED` statics are process-scoped log-dedup guards. Moving to ProxySharedState changes semantics: warnings fire per-instance instead of per-process. | Medium | Keep as process-global statics (correct for log dedup). Do NOT migrate to ProxySharedState. | +| F48 | `ProxySharedState` nested into `HandshakeSharedState` + `MiddleRelaySharedState` — unnecessary indirection. Functions access `shared.handshake.auth_probe` instead of `shared.auth_probe` | Low | Consider flattening to a single struct for simplicity (KISS principle, Architecture.md §1). Both sub-structs are always accessed together through the parent. | + +### Acknowledged Risks (not fixable in plan, require runtime attention) + +| # | Risk | Mitigation | +|---|---|---| +| R1 | DRS per-record flush adds syscall overhead in steady-state (16KB records). `copy_bidirectional_with_sizes` also flushes independently → double-flush is idempotent but wastes cycles. | Benchmark in PR-C red tests. If overhead > 2% throughput regression, coarsen flush to every N records in steady-state phase. | +| R2 | `copy_bidirectional_with_sizes` internal buffering: when `DrsWriter.poll_write` returns fewer bytes than offered (record boundary), the copy loop retries with the remaining buffer. This is correct but untested with the specific tokio implementation. | Add a specific integration test `drs_copy_bidirectional_partial_write_retry` that verifies total data integrity when DrsWriter limits write sizes. | +| R3 | `DrsWriter` flush inside `poll_write` loop: DRS value depends on `FakeTlsWriter.poll_flush` actually draining its internal `WriteBuffer` to the socket and creating a TLS record boundary. **Verified**: `FakeTlsWriter.poll_flush` first calls `poll_flush_record_inner` (drains pending TLS record bytes) then `upstream.poll_flush` (drains socket). This IS a real record boundary. However, `CryptoWriter` sits between DRS and FakeTLS and has its own pending buffer. DRS flush → `CryptoWriter.poll_flush` (drains pending ciphertext) → `FakeTlsWriter.poll_flush`. If `CryptoWriter` has accumulated bytes from multiple DRS writes before flush (possible if earlier write returned buffered-but-Ok), those bytes may be flushed as one chunk to FakeTLS, creating one larger record instead of separate DRS-sized records. | Add integration test `drs_crypto_writer_buffering_chain_integrity` to verify full chain produces individual records at DRS boundaries. | +| R4 | `average_throughput_to_tier` uses session-average throughput, not peak-window. Bursty traffic patterns (video streaming: 30s burst at 100 Mbps, then 9.5min idle) will underestimate tier, resulting in sub-optimal buffer sizes for the burst phase of the next session. | Document limitation. Monitor via watchdog's 10s snapshots. Future PR: compute peak from watchdog snapshots rather than session average. | +| R5 | PR-C covers direct relay only; middle relay (often the default) has no DRS. This is a significant coverage gap for deployments using ME pools. | PR-C.1 follow-up for middle relay. Middle relay has natural flush-tick points that make DRS integration architecturally simpler. Prioritize PR-C.1 immediately after PR-C. | +| R6 | `CryptoWriter.poll_write` always returns `Ok(to_accept)` even when `FakeTlsWriter` returns Pending — it buffers internally. If DRS writes N bytes and CryptoWriter buffers them, then DRS flushes, CryptoWriter drains its buffer as ONE chunk to FakeTLS. FakeTLS receives the full N-byte chunk and creates one N+5 byte TLS record. This is correct behavior (one DRS record = one TLS record). BUT if CryptoWriter's `max_pending_write` (default 16KB) is smaller than a DRS write (impossible: max DRS write = 16384 ≤ 16KB), writes would be split. Verify `CryptoWriter.max_pending_write` is ≥ `DRS_FULL_RECORD_PAYLOAD`. | Integration test `drs_crypto_writer_buffering_chain_integrity`. | +| R7 | IptController redesign (data-availability-aware) removes the Idle-state delay generation entirely. The Pareto distribution and `idle_dist` field are now dead code. Consider removing them to avoid confusion, or repurposing them for synthetic keep-alive timing in a future Level 3 enhancement. | Document in PR-G that `idle_dist` is retained for future Level 3 (trace-driven synthetic idle traffic). | + +### Missing Tests (should be added to existing PR test lists) + +| Test | PR | Rationale | +|---|---|---| +| `drs_statsio_byte_count_matches_actual_written` | PR-C | Verify StatsIo counters remain accurate when DrsWriter limits write sizes. Without this, a bug where DrsWriter eats or duplicates bytes goes undetected. | +| `drs_copy_bidirectional_partial_write_retry` | PR-C | Verify `copy_bidirectional_with_sizes` correctly retries when DrsWriter returns fewer bytes than offered at record boundaries. | +| `drs_records_completed_counter_does_not_wrap` | PR-C | On 32-bit `usize`, verify counter caps at `DRS_PHASE_FINAL + 1` and does not restart the DRS ramp. | +| `drs_flush_is_meaningful_for_faketls` | PR-C | Verify that `FakeTlsWriter.poll_flush` produces a TLS record boundary, otherwise DRS provides no anti-DPI value. | +| `adaptive_startup_remove_if_does_not_delete_fresh_concurrent_insert` | PR-D | Concurrent test: thread A reads stale profile, thread B inserts fresh profile, thread A calls `remove_if` → assert fresh profile survives. | +| `ipt_controller_burst_stay_threshold_probability_accuracy` | PR-G | Verify empirical Burst self-transition probability is within ±0.001 of 0.95 over 10M samples. | +| `proxy_shared_state_logged_unknown_dcs_isolation` | PR-B | Verify `LOGGED_UNKNOWN_DCS` does not leak between instances (if migrated). | +| `ipt_controller_pending_data_forces_burst_delay` | PR-G | Verify that `next_delay_us(rng, has_pending_data=true)` always returns Burst-range delay even when Markov state is Idle. Critical for connection liveness. | +| `ipt_controller_no_pending_data_returns_zero` | PR-G | Verify that `next_delay_us(rng, has_pending_data=false)` returns 0, ensuring no artificial stalling when the relay is idle. | +| `ipt_controller_burst_sample_overflow_safe` | PR-G | Verify LogNormal extreme tail samples don't overflow `saturating_mul(1_000)` and are properly clamped. | +| `ipt_controller_idle_sample_extreme_f64_safe` | PR-G | Verify Pareto samples of f64::INFINITY or f64::NAN are safely handled by `as u64` cast + clamp. | +| `drs_crypto_writer_buffering_chain_integrity` | PR-C | Verify that DRS → CryptoWriter (with internal pending buffer) → FakeTlsWriter produces correct TLS record boundaries. CryptoWriter may buffer; flush must drain the entire chain. | +| `drs_config_serde_default_upgrade_compat` | PR-C | Verify that deserializing a config.toml WITHOUT `drs_enabled` field produces `drs_enabled=true` (serde default). Tests upgrade compatibility. | + diff --git a/README.md b/README.md index 90ef03c..5cfc277 100644 --- a/README.md +++ b/README.md @@ -3,6 +3,9 @@ ***Löst Probleme, bevor andere überhaupt wissen, dass sie existieren*** / ***It solves problems before others even realize they exist*** ### [**Telemt Chat in Telegram**](https://t.me/telemtrs) +#### Fixed TLS ClientHello is now available in Telegram Desktop starting from version 6.7.2: to work with EE-MTProxy, please update your client; +#### Fixed TLS ClientHello for Telegram Android Client is available in [our chat](https://t.me/telemtrs/30234/36441); official releases for Android and iOS are "work in progress"; + **Telemt** is a fast, secure, and feature-rich server written in Rust: it fully implements the official Telegram proxy algo and adds many production-ready improvements such as: - [ME Pool + Reader/Writer + Registry + Refill + Adaptive Floor + Trio-State + Generation Lifecycle](https://github.com/telemt/telemt/blob/main/docs/model/MODEL.en.md) @@ -51,8 +54,12 @@ - [FAQ EN](docs/FAQ.en.md) ### Recognizability for DPI and crawler -Since version 1.1.0.0, we have debugged masking perfectly: for all clients without "presenting" a key, -we transparently direct traffic to the target host! + +On April 1, 2026, we became aware of a method for detecting MTProxy Fake-TLS, +based on the ECH extension and the ordering of cipher suites, +as well as an overall unique JA3/JA4 fingerprint +that does not occur in modern browsers: +we have already submitted initial changes to the Telegram Desktop developers and are working on updates for other clients. - We consider this a breakthrough aspect, which has no stable analogues today - Based on this: if `telemt` configured correctly, **TLS mode is completely identical to real-life handshake + communication** with a specified host diff --git a/docs/CONFIG_PARAMS.en.md b/docs/CONFIG_PARAMS.en.md index eea89bf..1222e89 100644 --- a/docs/CONFIG_PARAMS.en.md +++ b/docs/CONFIG_PARAMS.en.md @@ -14,10 +14,10 @@ This document lists all configuration keys accepted by `config.toml`. | Key | Type | Default | | --- | ---- | ------- | -| [`include`](#cfg-top-include) | `String` (special directive) | `null` | +| [`include`](#cfg-top-include) | `String` (special directive) | — | | [`show_link`](#cfg-top-show_link) | `"*"` or `String[]` | `[]` (`ShowLink::None`) | | [`dc_overrides`](#cfg-top-dc_overrides) | `Map` | `{}` | -| [`default_dc`](#cfg-top-default_dc) | `u8` or `null` | `null` (effective fallback: `2` in ME routing) | +| [`default_dc`](#cfg-top-default_dc) | `u8` | — (effective fallback: `2` in ME routing) | - `include` @@ -68,17 +68,17 @@ This document lists all configuration keys accepted by `config.toml`. | Key | Type | Default | | --- | ---- | ------- | -| [`data_path`](#cfg-general-data_path) | `String` or `null` | `null` | +| [`data_path`](#cfg-general-data_path) | `String` | — | | [`prefer_ipv6`](#cfg-general-prefer_ipv6) | `bool` | `false` | | [`fast_mode`](#cfg-general-fast_mode) | `bool` | `true` | | [`use_middle_proxy`](#cfg-general-use_middle_proxy) | `bool` | `true` | -| [`proxy_secret_path`](#cfg-general-proxy_secret_path) | `String` or `null` | `"proxy-secret"` | -| [`proxy_config_v4_cache_path`](#cfg-general-proxy_config_v4_cache_path) | `String` or `null` | `"cache/proxy-config-v4.txt"` | -| [`proxy_config_v6_cache_path`](#cfg-general-proxy_config_v6_cache_path) | `String` or `null` | `"cache/proxy-config-v6.txt"` | -| [`ad_tag`](#cfg-general-ad_tag) | `String` or `null` | `null` | -| [`middle_proxy_nat_ip`](#cfg-general-middle_proxy_nat_ip) | `IpAddr` or `null` | `null` | +| [`proxy_secret_path`](#cfg-general-proxy_secret_path) | `String` | `"proxy-secret"` | +| [`proxy_config_v4_cache_path`](#cfg-general-proxy_config_v4_cache_path) | `String` | `"cache/proxy-config-v4.txt"` | +| [`proxy_config_v6_cache_path`](#cfg-general-proxy_config_v6_cache_path) | `String` | `"cache/proxy-config-v6.txt"` | +| [`ad_tag`](#cfg-general-ad_tag) | `String` | — | +| [`middle_proxy_nat_ip`](#cfg-general-middle_proxy_nat_ip) | `IpAddr` | — | | [`middle_proxy_nat_probe`](#cfg-general-middle_proxy_nat_probe) | `bool` | `true` | -| [`middle_proxy_nat_stun`](#cfg-general-middle_proxy_nat_stun) | `String` or `null` | `null` | +| [`middle_proxy_nat_stun`](#cfg-general-middle_proxy_nat_stun) | `String` | — | | [`middle_proxy_nat_stun_servers`](#cfg-general-middle_proxy_nat_stun_servers) | `String[]` | `[]` | | [`stun_nat_probe_concurrency`](#cfg-general-stun_nat_probe_concurrency) | `usize` | `8` | | [`middle_proxy_pool_size`](#cfg-general-middle_proxy_pool_size) | `usize` | `8` | @@ -144,7 +144,7 @@ This document lists all configuration keys accepted by `config.toml`. | [`upstream_unhealthy_fail_threshold`](#cfg-general-upstream_unhealthy_fail_threshold) | `u32` | `5` | | [`upstream_connect_failfast_hard_errors`](#cfg-general-upstream_connect_failfast_hard_errors) | `bool` | `false` | | [`stun_iface_mismatch_ignore`](#cfg-general-stun_iface_mismatch_ignore) | `bool` | `false` | -| [`unknown_dc_log_path`](#cfg-general-unknown_dc_log_path) | `String` or `null` | `"unknown-dc.txt"` | +| [`unknown_dc_log_path`](#cfg-general-unknown_dc_log_path) | `String` | `"unknown-dc.txt"` | | [`unknown_dc_file_log_enabled`](#cfg-general-unknown_dc_file_log_enabled) | `bool` | `false` | | [`log_level`](#cfg-general-log_level) | `"debug"`, `"verbose"`, `"normal"`, or `"silent"` | `"normal"` | | [`disable_colors`](#cfg-general-disable_colors) | `bool` | `false` | @@ -163,7 +163,7 @@ This document lists all configuration keys accepted by `config.toml`. | [`me_route_inline_recovery_attempts`](#cfg-general-me_route_inline_recovery_attempts) | `u32` | `3` | | [`me_route_inline_recovery_wait_ms`](#cfg-general-me_route_inline_recovery_wait_ms) | `u64` | `3000` | | [`fast_mode_min_tls_record`](#cfg-general-fast_mode_min_tls_record) | `usize` | `0` | -| [`update_every`](#cfg-general-update_every) | `u64` or `null` | `300` | +| [`update_every`](#cfg-general-update_every) | `u64` | `300` | | [`me_reinit_every_secs`](#cfg-general-me_reinit_every_secs) | `u64` | `900` | | [`me_hardswap_warmup_delay_min_ms`](#cfg-general-me_hardswap_warmup_delay_min_ms) | `u64` | `1000` | | [`me_hardswap_warmup_delay_max_ms`](#cfg-general-me_hardswap_warmup_delay_max_ms) | `u64` | `2000` | @@ -205,7 +205,7 @@ This document lists all configuration keys accepted by `config.toml`. - `data_path` - - **Constraints / validation**: `String` or `null`. + - **Constraints / validation**: `String` (optional). - **Description**: Optional runtime data directory path. - **Example**: @@ -245,7 +245,7 @@ This document lists all configuration keys accepted by `config.toml`. ``` - `proxy_secret_path` - - **Constraints / validation**: `String` or `null`. If `null`, the effective cache path is `"proxy-secret"`. Empty values are accepted but will likely fail at runtime (invalid file path). + - **Constraints / validation**: `String`. When omitted, the default path is `"proxy-secret"`. Empty values are accepted by TOML/serde but will likely fail at runtime (invalid file path). - **Description**: Path to Telegram infrastructure `proxy-secret` cache file used by ME handshake/RPC auth. Telemt always tries a fresh download from `https://core.telegram.org/getProxySecret` first, caches it to this path on success, and falls back to reading the cached file (any age) on download failure. - **Example**: @@ -255,7 +255,7 @@ This document lists all configuration keys accepted by `config.toml`. ``` - `proxy_config_v4_cache_path` - - **Constraints / validation**: `String` or `null`. When set, must not be empty/whitespace-only. + - **Constraints / validation**: `String`. When set, must not be empty/whitespace-only. - **Description**: Optional disk cache path for raw `getProxyConfig` (IPv4) snapshot. At startup Telemt tries to fetch a fresh snapshot first; on fetch failure or empty snapshot it falls back to this cache file when present and non-empty. - **Example**: @@ -265,7 +265,7 @@ This document lists all configuration keys accepted by `config.toml`. ``` - `proxy_config_v6_cache_path` - - **Constraints / validation**: `String` or `null`. When set, must not be empty/whitespace-only. + - **Constraints / validation**: `String`. When set, must not be empty/whitespace-only. - **Description**: Optional disk cache path for raw `getProxyConfigV6` (IPv6) snapshot. At startup Telemt tries to fetch a fresh snapshot first; on fetch failure or empty snapshot it falls back to this cache file when present and non-empty. - **Example**: @@ -275,7 +275,7 @@ This document lists all configuration keys accepted by `config.toml`. ``` - `ad_tag` - - **Constraints / validation**: `String` or `null`. When set, must be exactly 32 hex characters; invalid values are disabled during config load. + - **Constraints / validation**: `String` (optional). When set, must be exactly 32 hex characters; invalid values are disabled during config load. - **Description**: Global fallback sponsored-channel `ad_tag` (used when user has no override in `access.user_ad_tags`). An all-zero tag is accepted but has no effect (and is warned about) until replaced with a real tag from `@MTProxybot`. - **Example**: @@ -285,7 +285,7 @@ This document lists all configuration keys accepted by `config.toml`. ``` - `middle_proxy_nat_ip` - - **Constraints / validation**: `IpAddr` or `null`. + - **Constraints / validation**: `IpAddr` (optional). - **Description**: Manual public NAT IP override used as ME address material when set. - **Example**: @@ -967,8 +967,8 @@ This document lists all configuration keys accepted by `config.toml`. ``` - `unknown_dc_log_path` - - **Constraints / validation**: `String` or `null`. Must be a safe path (no `..` components, parent directory must exist); unsafe paths are rejected at runtime. - - **Description**: Log file path for unknown (non-standard) DC requests when `unknown_dc_file_log_enabled = true`. Set to `null` to disable file logging. + - **Constraints / validation**: `String` (optional). Must be a safe path (no `..` components, parent directory must exist); unsafe paths are rejected at runtime. + - **Description**: Log file path for unknown (non-standard) DC requests when `unknown_dc_file_log_enabled = true`. Omit this key to disable file logging. - **Example**: ```toml @@ -1157,7 +1157,7 @@ This document lists all configuration keys accepted by `config.toml`. ``` - `update_every` - - **Constraints / validation**: `u64` (seconds) or `null`. If set, must be `> 0`. If `null`, legacy `proxy_secret_auto_reload_secs` and `proxy_config_auto_reload_secs` are used and their effective minimum must be `> 0`. + - **Constraints / validation**: `u64` (seconds). If set, must be `> 0`. If this key is not explicitly set, legacy `proxy_secret_auto_reload_secs` and `proxy_config_auto_reload_secs` may be used (their effective minimum must be `> 0`). - **Description**: Unified refresh interval for ME updater tasks (`getProxyConfig`, `getProxyConfigV6`, `getProxySecret`). When set, it overrides legacy proxy reload intervals. - **Example**: @@ -1450,7 +1450,7 @@ This document lists all configuration keys accepted by `config.toml`. ``` - `proxy_secret_auto_reload_secs` - - **Constraints / validation**: Deprecated. Use `general.update_every`. When `general.update_every` is `null`, the effective legacy refresh interval is `min(proxy_secret_auto_reload_secs, proxy_config_auto_reload_secs)` and must be `> 0`. + - **Constraints / validation**: Deprecated. Use `general.update_every`. When `general.update_every` is not explicitly set, the effective legacy refresh interval is `min(proxy_secret_auto_reload_secs, proxy_config_auto_reload_secs)` and must be `> 0`. - **Description**: Deprecated legacy proxy-secret refresh interval. Used only when `general.update_every` is not set. - **Example**: @@ -1463,7 +1463,7 @@ This document lists all configuration keys accepted by `config.toml`. ``` - `proxy_config_auto_reload_secs` - - **Constraints / validation**: Deprecated. Use `general.update_every`. When `general.update_every` is `null`, the effective legacy refresh interval is `min(proxy_secret_auto_reload_secs, proxy_config_auto_reload_secs)` and must be `> 0`. + - **Constraints / validation**: Deprecated. Use `general.update_every`. When `general.update_every` is not explicitly set, the effective legacy refresh interval is `min(proxy_secret_auto_reload_secs, proxy_config_auto_reload_secs)` and must be `> 0`. - **Description**: Deprecated legacy ME config refresh interval. Used only when `general.update_every` is not set. - **Example**: @@ -1624,8 +1624,8 @@ This document lists all configuration keys accepted by `config.toml`. | Key | Type | Default | | --- | ---- | ------- | | [`show`](#cfg-general-links-show) | `"*"` or `String[]` | `"*"` | -| [`public_host`](#cfg-general-links-public_host) | `String` or `null` | `null` | -| [`public_port`](#cfg-general-links-public_port) | `u16` or `null` | `null` | +| [`public_host`](#cfg-general-links-public_host) | `String` | — | +| [`public_port`](#cfg-general-links-public_port) | `u16` | — | - `show` @@ -1641,7 +1641,7 @@ This document lists all configuration keys accepted by `config.toml`. ``` - `public_host` - - **Constraints / validation**: `String` or `null`. + - **Constraints / validation**: `String` (optional). - **Description**: Public hostname/IP override used for generated `tg://` links (overrides detected IP). - **Example**: @@ -1651,7 +1651,7 @@ This document lists all configuration keys accepted by `config.toml`. ``` - `public_port` - - **Constraints / validation**: `u16` or `null`. + - **Constraints / validation**: `u16` (optional). - **Description**: Public port override used for generated `tg://` links (overrides `server.port`). - **Example**: @@ -1708,7 +1708,7 @@ This document lists all configuration keys accepted by `config.toml`. | Key | Type | Default | | --- | ---- | ------- | | [`ipv4`](#cfg-network-ipv4) | `bool` | `true` | -| [`ipv6`](#cfg-network-ipv6) | `bool` or `null` | `false` | +| [`ipv6`](#cfg-network-ipv6) | `bool` | `false` | | [`prefer`](#cfg-network-prefer) | `u8` | `4` | | [`multipath`](#cfg-network-multipath) | `bool` | `false` | | [`stun_use`](#cfg-network-stun_use) | `bool` | `true` | @@ -1730,8 +1730,8 @@ This document lists all configuration keys accepted by `config.toml`. ``` - `ipv6` - - **Constraints / validation**: `bool` or `null`. `null` means "auto-detect IPv6 availability". - - **Description**: Enables/disables IPv6 when explicitly set; when `null`, Telemt will auto-detect IPv6 availability at runtime. + - **Constraints / validation**: `bool`. + - **Description**: Enables/disables IPv6 networking. When omitted, defaults to `false`. - **Example**: ```toml @@ -1741,9 +1741,6 @@ This document lists all configuration keys accepted by `config.toml`. # or: disable IPv6 explicitly # ipv6 = false - - # or: let Telemt auto-detect - # ipv6 = null ``` - `prefer` @@ -1842,16 +1839,16 @@ This document lists all configuration keys accepted by `config.toml`. | Key | Type | Default | | --- | ---- | ------- | | [`port`](#cfg-server-port) | `u16` | `443` | -| [`listen_addr_ipv4`](#cfg-server-listen_addr_ipv4) | `String` or `null` | `"0.0.0.0"` | -| [`listen_addr_ipv6`](#cfg-server-listen_addr_ipv6) | `String` or `null` | `"::"` | -| [`listen_unix_sock`](#cfg-server-listen_unix_sock) | `String` or `null` | `null` | -| [`listen_unix_sock_perm`](#cfg-server-listen_unix_sock_perm) | `String` or `null` | `null` | -| [`listen_tcp`](#cfg-server-listen_tcp) | `bool` or `null` | `null` (auto) | +| [`listen_addr_ipv4`](#cfg-server-listen_addr_ipv4) | `String` | `"0.0.0.0"` | +| [`listen_addr_ipv6`](#cfg-server-listen_addr_ipv6) | `String` | `"::"` | +| [`listen_unix_sock`](#cfg-server-listen_unix_sock) | `String` | — | +| [`listen_unix_sock_perm`](#cfg-server-listen_unix_sock_perm) | `String` | — | +| [`listen_tcp`](#cfg-server-listen_tcp) | `bool` | — (auto) | | [`proxy_protocol`](#cfg-server-proxy_protocol) | `bool` | `false` | | [`proxy_protocol_header_timeout_ms`](#cfg-server-proxy_protocol_header_timeout_ms) | `u64` | `500` | | [`proxy_protocol_trusted_cidrs`](#cfg-server-proxy_protocol_trusted_cidrs) | `IpNetwork[]` | `[]` | -| [`metrics_port`](#cfg-server-metrics_port) | `u16` or `null` | `null` | -| [`metrics_listen`](#cfg-server-metrics_listen) | `String` or `null` | `null` | +| [`metrics_port`](#cfg-server-metrics_port) | `u16` | — | +| [`metrics_listen`](#cfg-server-metrics_listen) | `String` | — | | [`metrics_whitelist`](#cfg-server-metrics_whitelist) | `IpNetwork[]` | `["127.0.0.1/32", "::1/128"]` | | [`max_connections`](#cfg-server-max_connections) | `u32` | `10000` | | [`accept_permit_timeout_ms`](#cfg-server-accept_permit_timeout_ms) | `u64` | `250` | @@ -1868,8 +1865,8 @@ This document lists all configuration keys accepted by `config.toml`. ``` - `listen_addr_ipv4` - - **Constraints / validation**: `String` or `null`. When set, must be a valid IPv4 address string. - - **Description**: IPv4 bind address for TCP listener (`null` disables IPv4 bind). + - **Constraints / validation**: `String` (optional). When set, must be a valid IPv4 address string. + - **Description**: IPv4 bind address for TCP listener (omit this key to disable IPv4 bind). - **Example**: ```toml @@ -1878,8 +1875,8 @@ This document lists all configuration keys accepted by `config.toml`. ``` - `listen_addr_ipv6` - - **Constraints / validation**: `String` or `null`. When set, must be a valid IPv6 address string. - - **Description**: IPv6 bind address for TCP listener (`null` disables IPv6 bind). + - **Constraints / validation**: `String` (optional). When set, must be a valid IPv6 address string. + - **Description**: IPv6 bind address for TCP listener (omit this key to disable IPv6 bind). - **Example**: ```toml @@ -1888,7 +1885,7 @@ This document lists all configuration keys accepted by `config.toml`. ``` - `listen_unix_sock` - - **Constraints / validation**: `String` or `null`. Must not be empty when set. Unix only. + - **Constraints / validation**: `String` (optional). Must not be empty when set. Unix only. - **Description**: Unix socket path for listener. When set, `server.listen_tcp` defaults to `false` (unless explicitly overridden). - **Example**: @@ -1898,8 +1895,8 @@ This document lists all configuration keys accepted by `config.toml`. ``` - `listen_unix_sock_perm` - - **Constraints / validation**: `String` or `null`. When set, should be an octal permission string like `"0666"` or `"0777"`. - - **Description**: Optional Unix socket file permissions applied after bind (chmod). `null` means "no change" (inherits umask). + - **Constraints / validation**: `String` (optional). When set, should be an octal permission string like `"0666"` or `"0777"`. + - **Description**: Optional Unix socket file permissions applied after bind (chmod). When omitted, permissions are not changed (inherits umask). - **Example**: ```toml @@ -1909,7 +1906,7 @@ This document lists all configuration keys accepted by `config.toml`. ``` - `listen_tcp` - - **Constraints / validation**: `bool` or `null`. `null` means auto: + - **Constraints / validation**: `bool` (optional). When omitted, Telemt auto-detects: - `true` when `listen_unix_sock` is not set - `false` when `listen_unix_sock` is set - **Description**: Explicit TCP listener enable/disable override. @@ -1957,7 +1954,7 @@ This document lists all configuration keys accepted by `config.toml`. ``` - `metrics_port` - - **Constraints / validation**: `u16` or `null`. + - **Constraints / validation**: `u16` (optional). - **Description**: Prometheus-compatible metrics endpoint port. When set, enables the metrics listener (bind behavior can be overridden by `metrics_listen`). - **Example**: @@ -1967,7 +1964,7 @@ This document lists all configuration keys accepted by `config.toml`. ``` - `metrics_listen` - - **Constraints / validation**: `String` or `null`. When set, must be in `IP:PORT` format. + - **Constraints / validation**: `String` (optional). When set, must be in `IP:PORT` format. - **Description**: Full metrics bind address (`IP:PORT`), overrides `metrics_port` and binds on the specified address only. - **Example**: @@ -2010,6 +2007,105 @@ This document lists all configuration keys accepted by `config.toml`. Note: When `server.proxy_protocol` is enabled, incoming PROXY protocol headers are parsed from the first bytes of the connection and the client source address is replaced with `src_addr` from the header. For security, the peer source IP (the direct connection address) is verified against `server.proxy_protocol_trusted_cidrs`; if this list is empty, PROXY headers are rejected and the connection is considered untrusted. +## [server.conntrack_control] + +Note: The conntrack-control worker runs **only on Linux**. On other operating systems it is not started; if `inline_conntrack_control` is `true`, a warning is logged. Effective operation also requires **CAP_NET_ADMIN** and a usable backend (`nft` or `iptables` / `ip6tables` on `PATH`). The `conntrack` utility is used for optional table entry deletes under pressure. + + +| Key | Type | Default | +| --- | ---- | ------- | +| [`inline_conntrack_control`](#cfg-server-conntrack_control-inline_conntrack_control) | `bool` | `true` | +| [`mode`](#cfg-server-conntrack_control-mode) | `String` | `"tracked"` | +| [`backend`](#cfg-server-conntrack_control-backend) | `String` | `"auto"` | +| [`profile`](#cfg-server-conntrack_control-profile) | `String` | `"balanced"` | +| [`hybrid_listener_ips`](#cfg-server-conntrack_control-hybrid_listener_ips) | `IpAddr[]` | `[]` | +| [`pressure_high_watermark_pct`](#cfg-server-conntrack_control-pressure_high_watermark_pct) | `u8` | `85` | +| [`pressure_low_watermark_pct`](#cfg-server-conntrack_control-pressure_low_watermark_pct) | `u8` | `70` | +| [`delete_budget_per_sec`](#cfg-server-conntrack_control-delete_budget_per_sec) | `u64` | `4096` | + + +- `inline_conntrack_control` + - **Constraints / validation**: `bool`. + - **Description**: Master switch for the runtime conntrack-control task: reconciles **raw/notrack** netfilter rules for listener ingress (see `mode`), samples load every second, and may run **`conntrack -D`** deletes for qualifying close events while **pressure mode** is active (see `delete_budget_per_sec`). When `false`, notrack rules are cleared and pressure-driven deletes are disabled. + - **Example**: + + ```toml + [server.conntrack_control] + inline_conntrack_control = true + ``` + +- `mode` + - **Constraints / validation**: One of `tracked`, `notrack`, `hybrid` (case-insensitive; serialized lowercase). + - **Description**: **`tracked`**: do not install telemt notrack rules (connections stay in conntrack). **`notrack`**: mark matching ingress TCP to `server.port` as notrack — targets are derived from `[[server.listeners]]` if any, otherwise from `server.listen_addr_ipv4` / `server.listen_addr_ipv6` (unspecified addresses mean “any” for that family). **`hybrid`**: notrack only for addresses listed in `hybrid_listener_ips` (must be non-empty; validated at load). + - **Example**: + + ```toml + [server.conntrack_control] + mode = "notrack" + ``` + +- `backend` + - **Constraints / validation**: One of `auto`, `nftables`, `iptables` (case-insensitive; serialized lowercase). + - **Description**: Which command set applies notrack rules. **`auto`**: use `nft` if present on `PATH`, else `iptables`/`ip6tables` if present. **`nftables`** / **`iptables`**: force that backend; missing binary means rules cannot be applied. The nft path uses table `inet telemt_conntrack` and a prerouting raw hook; iptables uses chain `TELEMT_NOTRACK` in the `raw` table. + - **Example**: + + ```toml + [server.conntrack_control] + backend = "auto" + ``` + +- `profile` + - **Constraints / validation**: One of `conservative`, `balanced`, `aggressive` (case-insensitive; serialized lowercase). + - **Description**: When **conntrack pressure mode** is active (`pressure_*` watermarks), caps idle and activity timeouts to reduce conntrack churn: e.g. **client first-byte idle** (`client.rs`), **direct relay activity timeout** (`direct_relay.rs`), and **middle-relay idle policy** caps (`middle_relay.rs` via `ConntrackPressureProfile::*_cap_secs` / `direct_activity_timeout_secs`). More aggressive profiles use shorter caps. + - **Example**: + + ```toml + [server.conntrack_control] + profile = "balanced" + ``` + +- `hybrid_listener_ips` + - **Constraints / validation**: `IpAddr[]`. Required to be **non-empty** when `mode = "hybrid"`. Ignored for `tracked` / `notrack`. + - **Description**: Explicit listener addresses that receive notrack rules in hybrid mode (split into IPv4 vs IPv6 rules by the implementation). + - **Example**: + + ```toml + [server.conntrack_control] + mode = "hybrid" + hybrid_listener_ips = ["203.0.113.10", "2001:db8::1"] + ``` + +- `pressure_high_watermark_pct` + - **Constraints / validation**: Must be within `[1, 100]`. + - **Description**: Pressure mode **enters** when any of: connection fill vs `server.max_connections` (percentage, if `max_connections > 0`), **file-descriptor** usage vs process soft `RLIMIT_NOFILE`, **non-zero** `accept_permit_timeout` events in the last sample window, or **ME c2me send-full** counter delta. Entry compares relevant percentages against this high watermark (see `update_pressure_state` in `conntrack_control.rs`). + - **Example**: + + ```toml + [server.conntrack_control] + pressure_high_watermark_pct = 85 + ``` + +- `pressure_low_watermark_pct` + - **Constraints / validation**: Must be **strictly less than** `pressure_high_watermark_pct`. + - **Description**: Pressure mode **clears** only after **three** consecutive one-second samples where all signals are at or below this low watermark and the accept-timeout / ME-queue deltas are zero (hysteresis). + - **Example**: + + ```toml + [server.conntrack_control] + pressure_low_watermark_pct = 70 + ``` + +- `delete_budget_per_sec` + - **Constraints / validation**: Must be `> 0`. + - **Description**: Maximum number of **`conntrack -D`** attempts **per second** while pressure mode is active (token bucket refilled each second). Deletes run only for close events with reasons **timeout**, **pressure**, or **reset**; each attempt consumes a token regardless of outcome. + - **Example**: + + ```toml + [server.conntrack_control] + delete_budget_per_sec = 4096 + ``` + + ## [server.api] Note: This section also accepts the legacy alias `[server.admin_api]` (same schema as `[server.api]`). @@ -2158,9 +2254,9 @@ Note: This section also accepts the legacy alias `[server.admin_api]` (same sche | Key | Type | Default | | --- | ---- | ------- | | [`ip`](#cfg-server-listeners-ip) | `IpAddr` | — | -| [`announce`](#cfg-server-listeners-announce) | `String` or `null` | — | -| [`announce_ip`](#cfg-server-listeners-announce_ip) | `IpAddr` or `null` | — | -| [`proxy_protocol`](#cfg-server-listeners-proxy_protocol) | `bool` or `null` | `null` | +| [`announce`](#cfg-server-listeners-announce) | `String` | — | +| [`announce_ip`](#cfg-server-listeners-announce_ip) | `IpAddr` | — | +| [`proxy_protocol`](#cfg-server-listeners-proxy_protocol) | `bool` | — | | [`reuse_allow`](#cfg-server-listeners-reuse_allow) | `bool` | `false` | @@ -2175,7 +2271,7 @@ Note: This section also accepts the legacy alias `[server.admin_api]` (same sche ``` - `announce` - - **Constraints / validation**: `String` or `null`. Must not be empty when set. + - **Constraints / validation**: `String` (optional). Must not be empty when set. - **Description**: Public IP/domain announced in proxy links for this listener. Takes precedence over `announce_ip`. - **Example**: @@ -2186,7 +2282,7 @@ Note: This section also accepts the legacy alias `[server.admin_api]` (same sche ``` - `announce_ip` - - **Constraints / validation**: `IpAddr` or `null`. Deprecated. Use `announce`. + - **Constraints / validation**: `IpAddr` (optional). Deprecated. Use `announce`. - **Description**: Deprecated legacy announce IP. During config load it is migrated to `announce` when `announce` is not set. - **Example**: @@ -2197,7 +2293,7 @@ Note: This section also accepts the legacy alias `[server.admin_api]` (same sche ``` - `proxy_protocol` - - **Constraints / validation**: `bool` or `null`. When set, overrides `server.proxy_protocol` for this listener. + - **Constraints / validation**: `bool` (optional). When set, overrides `server.proxy_protocol` for this listener. - **Description**: Per-listener PROXY protocol override. - **Example**: @@ -2351,9 +2447,9 @@ Note: This section also accepts the legacy alias `[server.admin_api]` (same sche | [`tls_fetch_scope`](#cfg-censorship-tls_fetch_scope) | `String` | `""` | | [`tls_fetch`](#cfg-censorship-tls_fetch) | `Table` | built-in defaults | | [`mask`](#cfg-censorship-mask) | `bool` | `true` | -| [`mask_host`](#cfg-censorship-mask_host) | `String` or `null` | `null` | +| [`mask_host`](#cfg-censorship-mask_host) | `String` | — | | [`mask_port`](#cfg-censorship-mask_port) | `u16` | `443` | -| [`mask_unix_sock`](#cfg-censorship-mask_unix_sock) | `String` or `null` | `null` | +| [`mask_unix_sock`](#cfg-censorship-mask_unix_sock) | `String` | — | | [`fake_cert_len`](#cfg-censorship-fake_cert_len) | `usize` | `2048` | | [`tls_emulation`](#cfg-censorship-tls_emulation) | `bool` | `true` | | [`tls_front_dir`](#cfg-censorship-tls_front_dir) | `String` | `"tlsfront"` | @@ -2440,8 +2536,8 @@ Note: This section also accepts the legacy alias `[server.admin_api]` (same sche ``` - `mask_host` - - **Constraints / validation**: `String` or `null`. - - If `mask_unix_sock` is set, `mask_host` must be `null` (mutually exclusive). + - **Constraints / validation**: `String` (optional). + - If `mask_unix_sock` is set, `mask_host` must be omitted (mutually exclusive). - If `mask_host` is not set and `mask_unix_sock` is not set, Telemt defaults `mask_host` to `tls_domain`. - **Description**: Upstream mask host for TLS fronting relay. - **Example**: @@ -2462,7 +2558,7 @@ Note: This section also accepts the legacy alias `[server.admin_api]` (same sche ``` - `mask_unix_sock` - - **Constraints / validation**: `String` or `null`. + - **Constraints / validation**: `String` (optional). - Must not be empty when set. - Unix only; rejected on non-Unix platforms. - On Unix, must be \(\le 107\) bytes (path length limit). @@ -2882,6 +2978,7 @@ If your backend or network is very bandwidth-constrained, reduce cap first. If p | [`users`](#cfg-access-users) | `Map` | `{"default": "000…000"}` | | [`user_ad_tags`](#cfg-access-user_ad_tags) | `Map` | `{}` | | [`user_max_tcp_conns`](#cfg-access-user_max_tcp_conns) | `Map` | `{}` | +| [`user_max_tcp_conns_global_each`](#cfg-access-user_max_tcp_conns_global_each) | `usize` | `0` | | [`user_expirations`](#cfg-access-user_expirations) | `Map>` | `{}` | | [`user_data_quota`](#cfg-access-user_data_quota) | `Map` | `{}` | | [`user_max_unique_ips`](#cfg-access-user_max_unique_ips) | `Map` | `{}` | @@ -2926,6 +3023,20 @@ If your backend or network is very bandwidth-constrained, reduce cap first. If p [access.user_max_tcp_conns] alice = 500 ``` + +- `user_max_tcp_conns_global_each` + - **Constraints / validation**: `usize`. `0` disables the inherited limit. + - **Description**: Global per-user maximum concurrent TCP connections, applied when a user has **no positive** entry in `[access.user_max_tcp_conns]` (a missing key, or a value of `0`, both fall through to this setting). Per-user limits greater than `0` in `user_max_tcp_conns` take precedence. + - **Example**: + + ```toml + [access] + user_max_tcp_conns_global_each = 200 + + [access.user_max_tcp_conns] + alice = 500 # uses 500, not the global cap + # bob has no entry → uses 200 + ``` - `user_expirations` - **Constraints / validation**: `Map>`. Each value must be a valid RFC3339 / ISO-8601 datetime. @@ -3027,13 +3138,13 @@ If your backend or network is very bandwidth-constrained, reduce cap first. If p | [`weight`](#cfg-upstreams-weight) | `u16` | `1` | | [`enabled`](#cfg-upstreams-enabled) | `bool` | `true` | | [`scopes`](#cfg-upstreams-scopes) | `String` | `""` | -| [`interface`](#cfg-upstreams-interface) | `String` or `null` | `null` | -| [`bind_addresses`](#cfg-upstreams-bind_addresses) | `String[]` or `null` | `null` | +| [`interface`](#cfg-upstreams-interface) | `String` | — | +| [`bind_addresses`](#cfg-upstreams-bind_addresses) | `String[]` | — | | [`url`](#cfg-upstreams-url) | `String` | — | | [`address`](#cfg-upstreams-address) | `String` | — | -| [`user_id`](#cfg-upstreams-user_id) | `String` or `null` | `null` | -| [`username`](#cfg-upstreams-username) | `String` or `null` | `null` | -| [`password`](#cfg-upstreams-password) | `String` or `null` | `null` | +| [`user_id`](#cfg-upstreams-user_id) | `String` | — | +| [`username`](#cfg-upstreams-username) | `String` | — | +| [`password`](#cfg-upstreams-password) | `String` | — | - `type` @@ -3090,7 +3201,7 @@ If your backend or network is very bandwidth-constrained, reduce cap first. If p ``` - `interface` - - **Constraints / validation**: `String` or `null`. + - **Constraints / validation**: `String` (optional). - For `"direct"`: may be an IP address (used as explicit local bind) or an OS interface name (resolved to an IP at runtime; Unix only). - For `"socks4"`/`"socks5"`: supported only when `address` is an `IP:port` literal; when `address` is a hostname, interface binding is ignored. - For `"shadowsocks"`: passed to the shadowsocks connector as an optional outbound bind hint. @@ -3109,7 +3220,7 @@ If your backend or network is very bandwidth-constrained, reduce cap first. If p ``` - `bind_addresses` - - **Constraints / validation**: `String[]` or `null`. Applies only to `type = "direct"`. + - **Constraints / validation**: `String[]` (optional). Applies only to `type = "direct"`. - Each entry should be an IP address string. - At runtime, Telemt selects an address that matches the target family (IPv4 vs IPv6). If `bind_addresses` is set and none match the target family, the connect attempt fails. - **Description**: Explicit local source addresses for outgoing direct TCP connects. When multiple addresses are provided, selection is round-robin. @@ -3150,7 +3261,7 @@ If your backend or network is very bandwidth-constrained, reduce cap first. If p ``` - `user_id` - - **Constraints / validation**: `String` or `null`. Only for `type = "socks4"`. + - **Constraints / validation**: `String` (optional). Only for `type = "socks4"`. - **Description**: SOCKS4 CONNECT user ID. Note: when a request scope is selected, Telemt may override this with the selected scope value. - **Example**: @@ -3162,7 +3273,7 @@ If your backend or network is very bandwidth-constrained, reduce cap first. If p ``` - `username` - - **Constraints / validation**: `String` or `null`. Only for `type = "socks5"`. + - **Constraints / validation**: `String` (optional). Only for `type = "socks5"`. - **Description**: SOCKS5 username (for username/password authentication). Note: when a request scope is selected, Telemt may override this with the selected scope value. - **Example**: @@ -3174,7 +3285,7 @@ If your backend or network is very bandwidth-constrained, reduce cap first. If p ``` - `password` - - **Constraints / validation**: `String` or `null`. Only for `type = "socks5"`. + - **Constraints / validation**: `String` (optional). Only for `type = "socks5"`. - **Description**: SOCKS5 password (for username/password authentication). Note: when a request scope is selected, Telemt may override this with the selected scope value. - **Example**: diff --git a/docs/LICENSE/LICENSE.de.md b/docs/LICENSE/LICENSE.de.md deleted file mode 100644 index 35c8dcf..0000000 --- a/docs/LICENSE/LICENSE.de.md +++ /dev/null @@ -1,92 +0,0 @@ -# Öffentliche TELEMT-Lizenz 3 - -***Alle Rechte vorbehalten (c) 2026 Telemt*** - -Hiermit wird jeder Person, die eine Kopie dieser Software und der dazugehörigen Dokumentation (nachfolgend "Software") erhält, unentgeltlich die Erlaubnis erteilt, die Software ohne Einschränkungen zu nutzen, einschließlich des Rechts, die Software zu verwenden, zu vervielfältigen, zu ändern, abgeleitete Werke zu erstellen, zu verbinden, zu veröffentlichen, zu verbreiten, zu unterlizenzieren und/oder Kopien der Software zu verkaufen sowie diese Rechte auch denjenigen einzuräumen, denen die Software zur Verfügung gestellt wird, vorausgesetzt, dass sämtliche Urheberrechtshinweise sowie die Bedingungen und Bestimmungen dieser Lizenz eingehalten werden. - -### Begriffsbestimmungen - -Für die Zwecke dieser Lizenz gelten die folgenden Definitionen: - -**"Software" (Software)** — die Telemt-Software einschließlich Quellcode, Dokumentation und sämtlicher zugehöriger Dateien, die unter den Bedingungen dieser Lizenz verbreitet werden. - -**"Contributor" (Contributor)** — jede natürliche oder juristische Person, die Code, Patches, Dokumentation oder andere Materialien eingereicht hat, die von den Maintainers des Projekts angenommen und in die Software aufgenommen wurden. - -**"Beitrag" (Contribution)** — jedes urheberrechtlich geschützte Werk, das bewusst zur Aufnahme in die Software eingereicht wurde. - -**"Modifizierte Version" (Modified Version)** — jede Version der Software, die gegenüber der ursprünglichen Software geändert, angepasst, erweitert oder anderweitig modifiziert wurde. - -**"Maintainers" (Maintainers)** — natürliche oder juristische Personen, die für das offizielle Telemt-Projekt und dessen offizielle Veröffentlichungen verantwortlich sind. - -### 1 Urheberrechtshinweis (Attribution) - -Bei der Weitergabe der Software, sowohl in Form des Quellcodes als auch in binärer Form, MÜSSEN folgende Elemente erhalten bleiben: - -- der oben genannte Urheberrechtshinweis; -- der vollständige Text dieser Lizenz; -- sämtliche bestehenden Hinweise auf Urheberschaft. - -### 2 Hinweis auf Modifikationen - -Wenn Änderungen an der Software vorgenommen werden, MUSS die Person, die diese Änderungen vorgenommen hat, eindeutig darauf hinweisen, dass die Software modifiziert wurde, und eine kurze Beschreibung der vorgenommenen Änderungen beifügen. - -Modifizierte Versionen der Software DÜRFEN NICHT als die originale Version von Telemt dargestellt werden. - -### 3 Marken und Bezeichnungen - -Diese Lizenz GEWÄHRT KEINE Rechte zur Nutzung der Bezeichnung **"Telemt"**, des Telemt-Logos oder sonstiger Marken, Kennzeichen oder Branding-Elemente von Telemt. - -Weiterverbreitete oder modifizierte Versionen der Software DÜRFEN die Bezeichnung Telemt nicht in einer Weise verwenden, die bei Nutzern den Eindruck eines offiziellen Ursprungs oder einer Billigung durch das Telemt-Projekt erwecken könnte, sofern hierfür keine ausdrückliche Genehmigung der Maintainers vorliegt. - -Die Verwendung der Bezeichnung **Telemt** zur Beschreibung einer modifizierten Version der Software ist nur zulässig, wenn diese Version eindeutig als modifiziert oder inoffiziell gekennzeichnet ist. - -Jegliche Verbreitung, die Nutzer vernünftigerweise darüber täuschen könnte, dass es sich um eine offizielle Veröffentlichung von Telemt handelt, ist untersagt. - -### 4 Transparenz bei der Verbreitung von Binärversionen - -Im Falle der Verbreitung kompilierter Binärversionen der Software wird der Verbreiter HIERMIT ERMUTIGT (encouraged), soweit dies vernünftigerweise möglich ist, Zugang zum entsprechenden Quellcode sowie zu den Build-Anweisungen bereitzustellen. - -Diese Praxis trägt zur Transparenz bei und ermöglicht es Empfängern, die Integrität und Reproduzierbarkeit der verbreiteten Builds zu überprüfen. - -## 5 Gewährung einer Patentlizenz und Beendigung von Rechten - -Jeder Contributor gewährt den Empfängern der Software eine unbefristete, weltweite, nicht-exklusive, unentgeltliche, lizenzgebührenfreie und unwiderrufliche Patentlizenz für: - -- die Herstellung, -- die Beauftragung der Herstellung, -- die Nutzung, -- das Anbieten zum Verkauf, -- den Verkauf, -- den Import, -- sowie jede sonstige Verbreitung der Software. - -Diese Patentlizenz erstreckt sich ausschließlich auf solche Patentansprüche, die notwendigerweise durch den jeweiligen Beitrag des Contributors allein oder in Kombination mit der Software verletzt würden. - -Leitet eine Person ein Patentverfahren ein oder beteiligt sich daran, einschließlich Gegenklagen oder Kreuzklagen, mit der Behauptung, dass die Software oder ein darin enthaltener Beitrag ein Patent verletzt, **erlöschen sämtliche durch diese Lizenz gewährten Rechte für diese Person unmittelbar mit Einreichung der Klage**. - -Darüber hinaus erlöschen alle durch diese Lizenz gewährten Rechte **automatisch**, wenn eine Person ein gerichtliches Verfahren einleitet, in dem behauptet wird, dass die Software selbst ein Patent oder andere Rechte des geistigen Eigentums verletzt. - -### 6 Beteiligung und Beiträge zur Entwicklung - -Sofern ein Contributor nicht ausdrücklich etwas anderes erklärt, gilt jeder Beitrag, der bewusst zur Aufnahme in die Software eingereicht wird, als unter den Bedingungen dieser Lizenz lizenziert. - -Durch die Einreichung eines Beitrags gewährt der Contributor den Maintainers des Telemt-Projekts sowie allen Empfängern der Software die in dieser Lizenz beschriebenen Rechte in Bezug auf diesen Beitrag. - -### 7 Urheberhinweis bei Netzwerk- und Servicenutzung - -Wird die Software zur Bereitstellung eines öffentlich zugänglichen Netzwerkdienstes verwendet, MUSS der Betreiber dieses Dienstes einen Hinweis auf die Urheberschaft von Telemt an mindestens einer der folgenden Stellen anbringen: - -* in der Servicedokumentation; -* in der Dienstbeschreibung; -* auf einer Seite "Über" oder einer vergleichbaren Informationsseite; -* in anderen für Nutzer zugänglichen Materialien, die in angemessenem Zusammenhang mit dem Dienst stehen. - -Ein solcher Hinweis DARF NICHT den Eindruck erwecken, dass der Dienst vom Telemt-Projekt oder dessen Maintainers unterstützt oder offiziell gebilligt wird. - -### 8 Haftungsausschluss und salvatorische Klausel - -DIE SOFTWARE WIRD "WIE BESEHEN" BEREITGESTELLT, OHNE JEGLICHE AUSDRÜCKLICHE ODER STILLSCHWEIGENDE GEWÄHRLEISTUNG, EINSCHLIESSLICH, ABER NICHT BESCHRÄNKT AUF GEWÄHRLEISTUNGEN DER MARKTGÄNGIGKEIT, DER EIGNUNG FÜR EINEN BESTIMMTEN ZWECK UND DER NICHTVERLETZUNG VON RECHTEN. - -IN KEINEM FALL HAFTEN DIE AUTOREN ODER RECHTEINHABER FÜR IRGENDWELCHE ANSPRÜCHE, SCHÄDEN ODER SONSTIGE HAFTUNG, DIE AUS VERTRAG, UNERLAUBTER HANDLUNG ODER AUF ANDERE WEISE AUS DER SOFTWARE ODER DER NUTZUNG DER SOFTWARE ENTSTEHEN. - -SOLLTE EINE BESTIMMUNG DIESER LIZENZ ALS UNWIRKSAM ODER NICHT DURCHSETZBAR ANGESEHEN WERDEN, IST DIESE BESTIMMUNG SO AUSZULEGEN, DASS SIE DEM URSPRÜNGLICHEN WILLEN DER PARTEIEN MÖGLICHST NAHEKOMMT; DIE ÜBRIGEN BESTIMMUNGEN BLEIBEN DAVON UNBERÜHRT UND IN VOLLER WIRKUNG. \ No newline at end of file diff --git a/docs/LICENSE/LICENSE.en.md b/docs/LICENSE/LICENSE.en.md deleted file mode 100644 index 77796a3..0000000 --- a/docs/LICENSE/LICENSE.en.md +++ /dev/null @@ -1,143 +0,0 @@ -###### TELEMT Public License 3 ###### -##### Copyright (c) 2026 Telemt ##### - -Permission is hereby granted, free of charge, to any person obtaining a copy -of this Software and associated documentation files (the "Software"), -to use, reproduce, modify, prepare derivative works of, merge, publish, -distribute, sublicense, and/or sell copies of the Software, and to permit -persons to whom the Software is furnished to do so, provided that all -copyright notices, license terms, and conditions set forth in this License -are preserved and complied with. - -### Official Translations - -The canonical version of this License is the English version. - -Official translations are provided for informational purposes only -and for convenience, and do not have legal force. In case of any -discrepancy, the English version of this License shall prevail. - -Available versions: -- English in Markdown: docs/LICENSE/LICENSE.md -- German: docs/LICENSE/LICENSE.de.md -- Russian: docs/LICENSE/LICENSE.ru.md - -### Definitions - -For the purposes of this License: - -"Software" means the Telemt software, including source code, documentation, -and any associated files distributed under this License. - -"Contributor" means any person or entity that submits code, patches, -documentation, or other contributions to the Software that are accepted -into the Software by the maintainers. - -"Contribution" means any work of authorship intentionally submitted -to the Software for inclusion in the Software. - -"Modified Version" means any version of the Software that has been -changed, adapted, extended, or otherwise modified from the original -Software. - -"Maintainers" means the individuals or entities responsible for -the official Telemt project and its releases. - -#### 1 Attribution - -Redistributions of the Software, in source or binary form, MUST RETAIN the -above copyright notice, this license text, and any existing attribution -notices. - -#### 2 Modification Notice - -If you modify the Software, you MUST clearly state that the Software has been -modified and include a brief description of the changes made. - -Modified versions MUST NOT be presented as the original Telemt. - -#### 3 Trademark and Branding - -This license DOES NOT grant permission to use the name "Telemt", -the Telemt logo, or any Telemt trademarks or branding. - -Redistributed or modified versions of the Software MAY NOT use the Telemt -name in a way that suggests endorsement or official origin without explicit -permission from the Telemt maintainers. - -Use of the name "Telemt" to describe a modified version of the Software -is permitted only if the modified version is clearly identified as a -modified or unofficial version. - -Any distribution that could reasonably confuse users into believing that -the software is an official Telemt release is prohibited. - -#### 4 Binary Distribution Transparency - -If you distribute compiled binaries of the Software, -you are ENCOURAGED to provide access to the corresponding -source code and build instructions where reasonably possible. - -This helps preserve transparency and allows recipients to verify the -integrity and reproducibility of distributed builds. - -#### 5 Patent Grant and Defensive Termination Clause - -Each contributor grants you a perpetual, worldwide, non-exclusive, -no-charge, royalty-free, irrevocable patent license to make, have made, -use, offer to sell, sell, import, and otherwise transfer the Software. - -This patent license applies only to those patent claims necessarily -infringed by the contributor’s contribution alone or by combination of -their contribution with the Software. - -If you initiate or participate in any patent litigation, including -cross-claims or counterclaims, alleging that the Software or any -contribution incorporated within the Software constitutes patent -infringement, then **all rights granted to you under this license shall -terminate immediately** as of the date such litigation is filed. - -Additionally, if you initiate legal action alleging that the -Software itself infringes your patent or other intellectual -property rights, then all rights granted to you under this -license SHALL TERMINATE automatically. - -#### 6 Contributions - -Unless you explicitly state otherwise, any Contribution intentionally -submitted for inclusion in the Software shall be licensed under the terms -of this License. - -By submitting a Contribution, you grant the Telemt maintainers and all -recipients of the Software the rights described in this License with -respect to that Contribution. - -#### 7 Network Use Attribution - -If the Software is used to provide a publicly accessible network service, -the operator of such service MUST provide attribution to Telemt in at least -one of the following locations: - -- service documentation -- service description -- an "About" or similar informational page -- other user-visible materials reasonably associated with the service - -Such attribution MUST NOT imply endorsement by the Telemt project or its -maintainers. - -#### 8 Disclaimer of Warranty and Severability Clause - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. - -IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, -DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR -OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE -USE OR OTHER DEALINGS IN THE SOFTWARE - -IF ANY PROVISION OF THIS LICENSE IS HELD TO BE INVALID OR UNENFORCEABLE, -SUCH PROVISION SHALL BE INTERPRETED TO REFLECT THE ORIGINAL INTENT -OF THE PARTIES AS CLOSELY AS POSSIBLE, AND THE REMAINING PROVISIONS -SHALL REMAIN IN FULL FORCE AND EFFECT \ No newline at end of file diff --git a/docs/LICENSE/LICENSE.ru.md b/docs/LICENSE/LICENSE.ru.md deleted file mode 100644 index b88d9da..0000000 --- a/docs/LICENSE/LICENSE.ru.md +++ /dev/null @@ -1,90 +0,0 @@ -# Публичная лицензия TELEMT 3 - -***Все права защищёны (c) 2026 Telemt*** - -Настоящим любому лицу, получившему копию данного программного обеспечения и сопутствующей документации (далее — "Программное обеспечение"), безвозмездно предоставляется разрешение использовать Программное обеспечение без ограничений, включая право использовать, воспроизводить, изменять, создавать производные произведения, объединять, публиковать, распространять, сублицензировать и (или) продавать копии Программного обеспечения, а также предоставлять такие права лицам, которым предоставляется Программное обеспечение, при условии соблюдения всех уведомлений об авторских правах, условий и положений настоящей Лицензии. - -### Определения - -Для целей настоящей Лицензии применяются следующие определения: - -**"Программное обеспечение" (Software)** — программное обеспечение Telemt, включая исходный код, документацию и любые связанные файлы, распространяемые на условиях настоящей Лицензии. - -**"Контрибьютор" (Contributor)** — любое физическое или юридическое лицо, направившее код, исправления (патчи), документацию или иные материалы, которые были приняты мейнтейнерами проекта и включены в состав Программного обеспечения. - -**"Вклад" (Contribution)** — любое произведение авторского права, намеренно представленное для включения в состав Программного обеспечения. - -**"Модифицированная версия" (Modified Version)** — любая версия Программного обеспечения, которая была изменена, адаптирована, расширена или иным образом модифицирована по сравнению с исходным Программным обеспечением. - -**"Мейнтейнеры" (Maintainers)** — физические или юридические лица, ответственные за официальный проект Telemt и его официальные релизы. - -### 1 Указание авторства - -При распространении Программного обеспечения, как в форме исходного кода, так и в бинарной форме, ДОЛЖНЫ СОХРАНЯТЬСЯ: - -- указанное выше уведомление об авторских правах; -- текст настоящей Лицензии; -- любые существующие уведомления об авторстве. - -### 2 Уведомление о модификации - -В случае внесения изменений в Программное обеспечение лицо, осуществившее такие изменения, ОБЯЗАНО явно указать, что Программное обеспечение было модифицировано, а также включить краткое описание внесённых изменений. - -Модифицированные версии Программного обеспечения НЕ ДОЛЖНЫ представляться как оригинальная версия Telemt. - -### 3 Товарные знаки и обозначения - -Настоящая Лицензия НЕ ПРЕДОСТАВЛЯЕТ права использовать наименование **"Telemt"**, логотип Telemt, а также любые товарные знаки, фирменные обозначения или элементы бренда Telemt. - -Распространяемые или модифицированные версии Программного обеспечения НЕ ДОЛЖНЫ использовать наименование Telemt таким образом, который может создавать у пользователей впечатление официального происхождения либо одобрения со стороны проекта Telemt без явного разрешения мейнтейнеров проекта. - -Использование наименования **Telemt** для описания модифицированной версии Программного обеспечения допускается только при условии, что такая версия ясно обозначена как модифицированная или неофициальная. - -Запрещается любое распространение, которое может разумно вводить пользователей в заблуждение относительно того, что программное обеспечение является официальным релизом Telemt. - -### 4 Прозрачность распространения бинарных версий - -В случае распространения скомпилированных бинарных версий Программного обеспечения распространитель НАСТОЯЩИМ ПОБУЖДАЕТСЯ предоставлять доступ к соответствующему исходному коду и инструкциям по сборке, если это разумно возможно. - -Такая практика способствует прозрачности распространения и позволяет получателям проверять целостность и воспроизводимость распространяемых сборок. - -### 5 Предоставление патентной лицензии и прекращение прав - -Каждый контрибьютор предоставляет получателям Программного обеспечения бессрочную, всемирную, неисключительную, безвозмездную, не требующую выплаты роялти и безотзывную патентную лицензию на: - -- изготовление, -- поручение изготовления, -- использование, -- предложение к продаже, -- продажу, -- импорт, -- и иное распространение Программного обеспечения. - -Такая патентная лицензия распространяется исключительно на те патентные требования, которые неизбежно нарушаются соответствующим вкладом контрибьютора как таковым либо его сочетанием с Программным обеспечением. - -Если лицо инициирует либо участвует в каком-либо судебном разбирательстве по патентному спору, включая встречные или перекрёстные иски, утверждая, что Программное обеспечение либо любой вклад, включённый в него, нарушает патент, **все права, предоставленные такому лицу настоящей Лицензией, немедленно прекращаются** с даты подачи соответствующего иска. - -Кроме того, если лицо инициирует судебное разбирательство, утверждая, что само Программное обеспечение нарушает его патентные либо иные права интеллектуальной собственности, все права, предоставленные настоящей Лицензией, **автоматически прекращаются**. - -### 6 Участие и вклад в разработку - -Если контрибьютор явно не указал иное, любой Вклад, намеренно представленный для включения в Программное обеспечение, считается лицензированным на условиях настоящей Лицензии. -Путём предоставления Вклада контрибьютор предоставляет мейнтейнером проекта Telemt и всем получателям Программного обеспечения права, предусмотренные настоящей Лицензией, в отношении такого Вклада. - -### 7 Указание авторства при сетевом и сервисном использовании - -В случае использования Программного обеспечения для предоставления публично доступного сетевого сервиса оператор такого сервиса ОБЯЗАН обеспечить указание авторства Telemt как минимум в одном из следующих мест: -- документация сервиса; -- описание сервиса; -- страница "О программе" или аналогичная информационная страница; -- иные материалы, доступные пользователям и разумно связанные с данным сервисом. - -Такое указание авторства НЕ ДОЛЖНО создавать впечатление одобрения или официальной поддержки со стороны проекта Telemt либо его мейнтейнеров. - -### 8 Отказ от гарантий и делимость положений - -ПРОГРАММНОЕ ОБЕСПЕЧЕНИЕ ПРЕДОСТАВЛЯЕТСЯ "КАК ЕСТЬ", БЕЗ КАКИХ-ЛИБО ГАРАНТИЙ, ЯВНЫХ ИЛИ ПОДРАЗУМЕВАЕМЫХ, ВКЛЮЧАЯ, НО НЕ ОГРАНИЧИВАЯСЬ ГАРАНТИЯМИ КОММЕРЧЕСКОЙ ПРИГОДНОСТИ, ПРИГОДНОСТИ ДЛЯ КОНКРЕТНОЙ ЦЕЛИ И НЕНАРУШЕНИЯ ПРАВ. - -НИ ПРИ КАКИХ ОБСТОЯТЕЛЬСТВАХ АВТОРЫ ИЛИ ПРАВООБЛАДАТЕЛИ НЕ НЕСУТ ОТВЕТСТВЕННОСТИ ПО КАКИМ-ЛИБО ТРЕБОВАНИЯМ, УБЫТКАМ ИЛИ ИНОЙ ОТВЕТСТВЕННОСТИ, ВОЗНИКАЮЩЕЙ В РЕЗУЛЬТАТЕ ДОГОВОРА, ДЕЛИКТА ИЛИ ИНЫМ ОБРАЗОМ, СВЯЗАННЫМ С ПРОГРАММНЫМ ОБЕСПЕЧЕНИЕМ ИЛИ ЕГО ИСПОЛЬЗОВАНИЕМ. - -В СЛУЧАЕ ЕСЛИ КАКОЕ-ЛИБО ПОЛОЖЕНИЕ НАСТОЯЩЕЙ ЛИЦЕНЗИИ ПРИЗНАЁТСЯ НЕДЕЙСТВИТЕЛЬНЫМ ИЛИ НЕПРИМЕНИМЫМ, ТАКОЕ ПОЛОЖЕНИЕ ПОДЛЕЖИТ ТОЛКОВАНИЮ МАКСИМАЛЬНО БЛИЗКО К ИСХОДНОМУ НАМЕРЕНИЮ СТОРОН, ПРИ ЭТОМ ОСТАЛЬНЫЕ ПОЛОЖЕНИЯ СОХРАНЯЮТ ПОЛНУЮ ЮРИДИЧЕСКУЮ СИЛУ. diff --git a/docs/LICENSE/TELEMT-LICENSE.en.md b/docs/LICENSE/TELEMT-LICENSE.en.md new file mode 100644 index 0000000..74e6a44 --- /dev/null +++ b/docs/LICENSE/TELEMT-LICENSE.en.md @@ -0,0 +1,120 @@ +# TELEMT License 3.3 + +***Copyright (c) 2026 Telemt*** + +Permission is hereby granted, free of charge, to any person obtaining a copy of this Software and associated documentation files (the "Software"), to use, reproduce, modify, prepare derivative works of, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, provided that all copyright notices, license terms, and conditions set forth in this License are preserved and complied with. + +### Official Translations + +The canonical version of this License is the English version. +Official translations are provided for informational purposes only and for convenience, and do not have legal force. In case of any discrepancy, the English version of this License shall prevail. + +| Language | Location | +|-------------|----------| +| English | [docs/LICENSE/TELEMT-LICENSE.en.md](https://github.com/telemt/telemt/tree/main/docs/LICENSE/TELEMT-LICENSE.en.md)| +| German | [docs/LICENSE/TELEMT-LICENSE.de.md](https://github.com/telemt/telemt/tree/main/docs/LICENSE/TELEMT-LICENSE.de.md)| +| Russian | [docs/LICENSE/TELEMT-LICENSE.ru.md](https://github.com/telemt/telemt/tree/main/docs/LICENSE/TELEMT-LICENSE.ru.md)| + +### License Versioning Policy + +This License is version 3.3 of the TELEMT License. +Each version of the Software is licensed under the License that accompanies its corresponding source code distribution. + +Future versions of the Software may be distributed under a different version of the TELEMT Public License or under a different license, as determined by the Telemt maintainers. + +Any such change of license applies only to the versions of the Software distributed with the new license and SHALL NOT retroactively affect any previously released versions of the Software. + +Recipients of the Software are granted rights only under the License provided with the version of the Software they received. + +Redistributions of the Software, including Modified Versions, MUST preserve the copyright notices, license text, and conditions of this License for all portions of the Software derived from Telemt. + +Additional terms or licenses may be applied to modifications or additional code added by a redistributor, provided that such terms do not restrict or alter the rights granted under this License for the original Telemt Software. + +Nothing in this section limits the rights granted under this License for versions of the Software already released. + +### Definitions + +For the purposes of this License: + +**"Software"** means the Telemt software, including source code, documentation, and any associated files distributed under this License. + +**"Contributor"** means any person or entity that submits code, patches, documentation, or other contributions to the Software that are accepted into the Software by the maintainers. + +**"Contribution"** means any work of authorship intentionally submitted to the Software for inclusion in the Software. + +**"Modified Version"** means any version of the Software that has been changed, adapted, extended, or otherwise modified from the original Software. + +**"Maintainers"** means the individuals or entities responsible for the official Telemt project and its releases. + +### 1 Attribution + +Redistributions of the Software, in source or binary form, MUST RETAIN: + +- the above copyright notice; +- this license text; +- any existing attribution notices. + +### 2 Modification Notice + +If you modify the Software, you MUST clearly state that the Software has been modified and include a brief description of the changes made. + +Modified versions MUST NOT be presented as the original Telemt. + +### 3 Trademark and Branding + +This license DOES NOT grant permission to use the name "Telemt", the Telemt logo, or any Telemt trademarks or branding. + +Redistributed or modified versions of the Software MAY NOT use the Telemt name in a way that suggests endorsement or official origin without explicit permission from the Telemt maintainers. + +Use of the name "Telemt" to describe a modified version of the Software is permitted only if the modified version is clearly identified as a modified or unofficial version. + +Any distribution that could reasonably confuse users into believing that the software is an official Telemt release is prohibited. + +### 4 Binary Distribution Transparency + +If you distribute compiled binaries of the Software, you are ENCOURAGED to provide access to the corresponding source code and build instructions where reasonably possible. + +This helps preserve transparency and allows recipients to verify the integrity and reproducibility of distributed builds. + +### 5 Patent Grant and Defensive Termination Clause + +Each contributor grants you a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable patent license to: + +- make, +- have made, +- use, +- offer to sell, +- sell, +- import, +- and otherwise transfer the Software. + +This patent license applies only to those patent claims necessarily infringed by the contributor’s contribution alone or by combination of their contribution with the Software. + +If you initiate or participate in any patent litigation, including cross-claims or counterclaims, alleging that the Software or any contribution incorporated within the Software constitutes patent infringement, then **all rights granted to you under this license shall terminate immediately** as of the date such litigation is filed. + +Additionally, if you initiate legal action alleging that the Software itself infringes your patent or other intellectual property rights, then all rights granted to you under this license SHALL TERMINATE automatically. + +### 6 Contributions + +Unless you explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Software shall be licensed under the terms of this License. + +By submitting a Contribution, you grant the Telemt maintainers and all recipients of the Software the rights described in this License with respect to that Contribution. + +### 7 Network Use Attribution + +If the Software is used to provide a publicly accessible network service, the operator of such service SHOULD provide attribution to Telemt in at least one of the following locations: + +- service documentation; +- service description; +- an "About" or similar informational page; +- other user-visible materials reasonably associated with the service. + +Such attribution MUST NOT imply endorsement by the Telemt project or its maintainers. + +### 8 Disclaimer of Warranty and Severability Clause + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. + +IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +IF ANY PROVISION OF THIS LICENSE IS HELD TO BE INVALID OR UNENFORCEABLE, SUCH PROVISION SHALL BE INTERPRETED TO REFLECT THE ORIGINAL INTENT OF THE PARTIES AS CLOSELY AS POSSIBLE, AND THE REMAINING PROVISIONS SHALL REMAIN IN FULL FORCE AND EFFECT. \ No newline at end of file diff --git a/docs/LICENSE/TELEMT-LICENSE.ru.md b/docs/LICENSE/TELEMT-LICENSE.ru.md new file mode 100644 index 0000000..ca3395b --- /dev/null +++ b/docs/LICENSE/TELEMT-LICENSE.ru.md @@ -0,0 +1,120 @@ +# TELEMT Лицензия 3.3 + +***Copyright (c) 2026 Telemt*** + +Настоящим безвозмездно предоставляется разрешение любому лицу, получившему копию данного программного обеспечения и сопутствующей документации (далее — "Программное обеспечение"), использовать, воспроизводить, изменять, создавать производные произведения, объединять, публиковать, распространять, сублицензировать и/или продавать копии Программного обеспечения, а также разрешать лицам, которым предоставляется Программное обеспечение, осуществлять указанные действия при условии соблюдения и сохранения всех уведомлений об авторском праве, условий и положений настоящей Лицензии. + +### Официальные переводы + +Канонической версией настоящей Лицензии является версия на английском языке. +Официальные переводы предоставляются исключительно в информационных целях и для удобства и не имеют юридической силы. В случае любых расхождений приоритет имеет английская версия. + +| Язык | Расположение | +|------------|--------------| +| Русский | [docs/LICENSE/TELEMT-LICENSE.ru.md](https://github.com/telemt/telemt/tree/main/docs/LICENSE/TELEMT-LICENSE.ru.md)| +| Английский | [docs/LICENSE/TELEMT-LICENSE.en.md](https://github.com/telemt/telemt/tree/main/docs/LICENSE/TELEMT-LICENSE.en.md)| +| Немецкий | [docs/LICENSE/TELEMT-LICENSE.de.md](https://github.com/telemt/telemt/tree/main/docs/LICENSE/TELEMT-LICENSE.de.md)| + +### Политика версионирования лицензии + +Настоящая Лицензия является версией 3.3 Лицензии TELEMT. +Каждая версия Программного обеспечения лицензируется в соответствии с Лицензией, сопровождающей соответствующее распространение исходного кода. + +Будущие версии Программного обеспечения могут распространяться в соответствии с иной версией Лицензии TELEMT Public License либо под иной лицензией, определяемой мейнтейнерами Telemt. + +Любое такое изменение лицензии применяется исключительно к версиям Программного обеспечения, распространяемым с новой лицензией, и НЕ распространяется ретроактивно на ранее выпущенные версии Программного обеспечения. + +Получатели Программного обеспечения приобретают права исключительно в соответствии с Лицензией, предоставленной вместе с полученной ими версией Программного обеспечения. + +При распространении Программного обеспечения, включая Модифицированные версии, ОБЯЗАТЕЛЬНО сохранение уведомлений об авторском праве, текста лицензии и условий настоящей Лицензии в отношении всех частей Программного обеспечения, производных от Telemt. + +Дополнительные условия или лицензии могут применяться к модификациям или дополнительному коду, добавленному распространителем, при условии, что такие условия не ограничивают и не изменяют права, предоставленные настоящей Лицензией в отношении оригинального Программного обеспечения Telemt. + +Ничто в настоящем разделе не ограничивает права, предоставленные настоящей Лицензией в отношении уже выпущенных версий Программного обеспечения. + +### Определения + +Для целей настоящей Лицензии: + +**"Программное обеспечение"** означает программное обеспечение Telemt, включая исходный код, документацию и любые сопутствующие файлы, распространяемые в соответствии с настоящей Лицензией. + +**"Контрибьютор"** означает любое физическое или юридическое лицо, которое предоставляет код, исправления, документацию или иные материалы в качестве вклада в Программное обеспечение, принятые мейнтейнерами для включения в Программное обеспечение. + +**"Вклад"** означает любое произведение, сознательно представленное для включения в Программное обеспечение. + +**"Модифицированная версия"** означает любую версию Программного обеспечения, которая была изменена, адаптирована, расширена или иным образом модифицирована по сравнению с оригинальным Программным обеспечением. + +**"Мейнтейнеры"** означает физических или юридических лиц, ответственных за официальный проект Telemt и его релизы. + +### 1. Атрибуция + +При распространении Программного обеспечения, как в виде исходного кода, так и в бинарной форме, ОБЯЗАТЕЛЬНО СОХРАНЕНИЕ: + +- указанного выше уведомления об авторском праве; +- текста настоящей Лицензии; +- всех существующих уведомлений об атрибуции. + +### 2. Уведомление о модификациях + +В случае внесения изменений в Программное обеспечение вы ОБЯЗАНЫ явно указать факт модификации Программного обеспечения и включить краткое описание внесённых изменений. + +Модифицированные версии НЕ ДОЛЖНЫ представляться как оригинальное Программное обеспечение Telemt. + +### 3. Товарные знаки и брендинг + +Настоящая Лицензия НЕ предоставляет право на использование наименования "Telemt", логотипа Telemt или любых товарных знаков и элементов брендинга Telemt. + +Распространяемые или модифицированные версии Программного обеспечения НЕ МОГУТ использовать наименование Telemt таким образом, который может создавать впечатление одобрения или официального происхождения без явного разрешения мейнтейнеров Telemt. + +Использование наименования "Telemt" для описания модифицированной версии Программного обеспечения допускается только при условии, что такая версия чётко обозначена как модифицированная или неофициальная. + +Запрещается любое распространение, способное разумно ввести пользователей в заблуждение относительно того, что программное обеспечение является официальным релизом Telemt. + +### 4. Прозрачность распространения бинарных файлов + +В случае распространения скомпилированных бинарных файлов Программного обеспечения рекомендуется (ENCOURAGED) предоставлять доступ к соответствующему исходному коду и инструкциям по сборке, если это разумно возможно. + +Это способствует обеспечению прозрачности и позволяет получателям проверять целостность и воспроизводимость распространяемых сборок. + +### 5. Патентная лицензия и условие защитного прекращения + +Каждый контрибьютор предоставляет вам бессрочную, всемирную, неисключительную, безвозмездную, без лицензионных отчислений, безотзывную патентную лицензию на: + +- изготовление, +- поручение изготовления, +- использование, +- предложение к продаже, +- продажу, +- импорт, +- а также иные формы передачи Программного обеспечения. + +Данная патентная лицензия распространяется исключительно на те патентные притязания, которые неизбежно нарушаются вкладом контрибьютора отдельно либо в сочетании его вклада с Программным обеспечением. + +Если вы инициируете или участвуете в любом патентном судебном разбирательстве, включая встречные иски или требования, утверждая, что Программное обеспечение или любой Вклад, включённый в Программное обеспечение, нарушает патент, то **все предоставленные вам настоящей Лицензией права немедленно прекращаются** с даты подачи такого иска. + +Дополнительно, если вы инициируете судебное разбирательство, утверждая, что само Программное обеспечение нарушает ваш патент или иные права интеллектуальной собственности, все права, предоставленные вам настоящей Лицензией, ПРЕКРАЩАЮТСЯ автоматически. + +### 6. Вклады + +Если вы прямо не указали иное, любой Вклад, сознательно представленный для включения в Программное обеспечение, лицензируется на условиях настоящей Лицензии. + +Предоставляя Вклад, вы предоставляете мейнтейнерам Telemt и всем получателям Программного обеспечения права, предусмотренные настоящей Лицензией, в отношении такого Вклада. + +### 7. Атрибуция при сетевом использовании + +Если Программное обеспечение используется для предоставления общедоступного сетевого сервиса, оператор такого сервиса ДОЛЖЕН (SHOULD) обеспечить указание атрибуции Telemt как минимум в одном из следующих мест: + +- документация сервиса; +- описание сервиса; +- раздел "О программе" или аналогичная информационная страница; +- иные материалы, доступные пользователю и разумно связанные с сервисом. + +Такая атрибуция НЕ ДОЛЖНА подразумевать одобрение со стороны проекта Telemt или его мейнтейнеров. + +### 8. Отказ от гарантий и оговорка о делимости + +ПРОГРАММНОЕ ОБЕСПЕЧЕНИЕ ПРЕДОСТАВЛЯЕТСЯ "КАК ЕСТЬ", БЕЗ КАКИХ-ЛИБО ГАРАНТИЙ, ЯВНЫХ ИЛИ ПОДРАЗУМЕВАЕМЫХ, ВКЛЮЧАЯ, В ЧАСТНОСТИ, ГАРАНТИИ ТОВАРНОЙ ПРИГОДНОСТИ, СООТВЕТСТВИЯ ОПРЕДЕЛЁННОЙ ЦЕЛИ И ОТСУТСТВИЯ НАРУШЕНИЙ ПРАВ. + +НИ ПРИ КАКИХ ОБСТОЯТЕЛЬСТВАХ АВТОРЫ ИЛИ ПРАВООБЛАДАТЕЛИ НЕ НЕСУТ ОТВЕТСТВЕННОСТИ ПО КАКИМ-ЛИБО ТРЕБОВАНИЯМ, УБЫТКАМ ИЛИ ИНОЙ ОТВЕТСТВЕННОСТИ, ВОЗНИКАЮЩИМ В РАМКАХ ДОГОВОРА, ДЕЛИКТА ИЛИ ИНЫМ ОБРАЗОМ, ИЗ, В СВЯЗИ С ИЛИ В РЕЗУЛЬТАТЕ ИСПОЛЬЗОВАНИЯ ПРОГРАММНОГО ОБЕСПЕЧЕНИЯ ИЛИ ИНЫХ ДЕЙСТВИЙ С НИМ. + +ЕСЛИ ЛЮБОЕ ПОЛОЖЕНИЕ НАСТОЯЩЕЙ ЛИЦЕНЗИИ ПРИЗНАЁТСЯ НЕДЕЙСТВИТЕЛЬНЫМ ИЛИ НЕПРИМЕНИМЫМ, ТАКОЕ ПОЛОЖЕНИЕ ПОДЛЕЖИТ ТОЛКОВАНИЮ МАКСИМАЛЬНО БЛИЗКО К ИСХОДНОМУ НАМЕРЕНИЮ СТОРОН, А ОСТАЛЬНЫЕ ПОЛОЖЕНИЯ СОХРАНЯЮТ ПОЛНУЮ СИЛУ И ДЕЙСТВИЕ. \ No newline at end of file diff --git a/docs/QUICK_START_GUIDE.en.md b/docs/QUICK_START_GUIDE.en.md index f6df4c4..8d8e656 100644 --- a/docs/QUICK_START_GUIDE.en.md +++ b/docs/QUICK_START_GUIDE.en.md @@ -128,8 +128,8 @@ WorkingDirectory=/opt/telemt ExecStart=/bin/telemt /etc/telemt/telemt.toml Restart=on-failure LimitNOFILE=65536 -AmbientCapabilities=CAP_NET_BIND_SERVICE -CapabilityBoundingSet=CAP_NET_BIND_SERVICE +AmbientCapabilities=CAP_NET_ADMIN CAP_NET_BIND_SERVICE +CapabilityBoundingSet=CAP_NET_ADMIN CAP_NET_BIND_SERVICE NoNewPrivileges=true [Install] @@ -150,7 +150,7 @@ systemctl daemon-reload **7.** To get the link(s), enter: ```bash -curl -s http://127.0.0.1:9091/v1/users | jq +curl -s http://127.0.0.1:9091/v1/users | jq -r '.data[] | "User: \(.username)\n\(.links.tls[0] // empty)"' ``` > Any number of people can use one link. diff --git a/docs/QUICK_START_GUIDE.ru.md b/docs/QUICK_START_GUIDE.ru.md index 3925953..e4343d7 100644 --- a/docs/QUICK_START_GUIDE.ru.md +++ b/docs/QUICK_START_GUIDE.ru.md @@ -128,8 +128,8 @@ WorkingDirectory=/opt/telemt ExecStart=/bin/telemt /etc/telemt/telemt.toml Restart=on-failure LimitNOFILE=65536 -AmbientCapabilities=CAP_NET_BIND_SERVICE -CapabilityBoundingSet=CAP_NET_BIND_SERVICE +AmbientCapabilities=CAP_NET_ADMIN CAP_NET_BIND_SERVICE +CapabilityBoundingSet=CAP_NET_ADMIN CAP_NET_BIND_SERVICE NoNewPrivileges=true [Install] @@ -150,7 +150,7 @@ systemctl daemon-reload **7.** Для получения ссылки/ссылок введите ```bash -curl -s http://127.0.0.1:9091/v1/users | jq +curl -s http://127.0.0.1:9091/v1/users | jq -r '.data[] | "User: \(.username)\n\(.links.tls[0] // empty)"' ``` > Одной ссылкой может пользоваться сколько угодно человек. diff --git a/install.sh b/install.sh index 90c28f4..b63da51 100644 --- a/install.sh +++ b/install.sh @@ -8,18 +8,62 @@ CONFIG_DIR="${CONFIG_DIR:-/etc/telemt}" CONFIG_FILE="${CONFIG_FILE:-${CONFIG_DIR}/telemt.toml}" WORK_DIR="${WORK_DIR:-/opt/telemt}" TLS_DOMAIN="${TLS_DOMAIN:-petrovich.ru}" +SERVER_PORT="${SERVER_PORT:-443}" +USER_SECRET="" +AD_TAG="" SERVICE_NAME="telemt" TEMP_DIR="" SUDO="" CONFIG_PARENT_DIR="" SERVICE_START_FAILED=0 +PORT_PROVIDED=0 +SECRET_PROVIDED=0 +AD_TAG_PROVIDED=0 +DOMAIN_PROVIDED=0 + ACTION="install" TARGET_VERSION="${VERSION:-latest}" while [ $# -gt 0 ]; do case "$1" in -h|--help) ACTION="help"; shift ;; + -d|--domain) + if [ "$#" -lt 2 ] || [ -z "$2" ]; then + printf '[ERROR] %s requires a domain argument.\n' "$1" >&2 + exit 1 + fi + TLS_DOMAIN="$2"; DOMAIN_PROVIDED=1; shift 2 ;; + -p|--port) + if [ "$#" -lt 2 ] || [ -z "$2" ]; then + printf '[ERROR] %s requires a port argument.\n' "$1" >&2; exit 1 + fi + case "$2" in + *[!0-9]*) printf '[ERROR] Port must be a valid number.\n' >&2; exit 1 ;; + esac + port_num="$(printf '%s\n' "$2" | sed 's/^0*//')" + [ -z "$port_num" ] && port_num="0" + if [ "${#port_num}" -gt 5 ] || [ "$port_num" -lt 1 ] || [ "$port_num" -gt 65535 ]; then + printf '[ERROR] Port must be between 1 and 65535.\n' >&2; exit 1 + fi + SERVER_PORT="$port_num"; PORT_PROVIDED=1; shift 2 ;; + -s|--secret) + if [ "$#" -lt 2 ] || [ -z "$2" ]; then + printf '[ERROR] %s requires a secret argument.\n' "$1" >&2; exit 1 + fi + case "$2" in + *[!0-9a-fA-F]*) + printf '[ERROR] Secret must contain only hex characters.\n' >&2; exit 1 ;; + esac + if [ "${#2}" -ne 32 ]; then + printf '[ERROR] Secret must be exactly 32 chars.\n' >&2; exit 1 + fi + USER_SECRET="$2"; SECRET_PROVIDED=1; shift 2 ;; + -a|--ad-tag|--ad_tag) + if [ "$#" -lt 2 ] || [ -z "$2" ]; then + printf '[ERROR] %s requires an ad_tag argument.\n' "$1" >&2; exit 1 + fi + AD_TAG="$2"; AD_TAG_PROVIDED=1; shift 2 ;; uninstall|--uninstall) if [ "$ACTION" != "purge" ]; then ACTION="uninstall"; fi shift ;; @@ -52,11 +96,17 @@ cleanup() { trap cleanup EXIT INT TERM show_help() { - say "Usage: $0 [ | install | uninstall | purge | --help ]" + say "Usage: $0 [ | install | uninstall | purge ] [ options ]" say " Install specific version (e.g. 3.3.15, default: latest)" say " install Install the latest version" - say " uninstall Remove the binary and service (keeps config and user)" + say " uninstall Remove the binary and service" say " purge Remove everything including configuration, data, and user" + say "" + say "Options:" + say " -d, --domain Set TLS domain (default: petrovich.ru)" + say " -p, --port Set server port (default: 443)" + say " -s, --secret Set specific user secret (32 hex characters)" + say " -a, --ad-tag Set ad_tag" exit 0 } @@ -73,13 +123,13 @@ get_realpath() { path_in="$1" case "$path_in" in /*) ;; *) path_in="$(pwd)/$path_in" ;; esac - if command -v realpath >/dev/null 2>&1; then + if command -v realpath >/dev/null 2>&1; then if realpath_out="$(realpath -m "$path_in" 2>/dev/null)"; then printf '%s\n' "$realpath_out" return fi fi - + if command -v readlink >/dev/null 2>&1; then resolved_path="$(readlink -f "$path_in" 2>/dev/null || true)" if [ -n "$resolved_path" ]; then @@ -112,6 +162,14 @@ get_svc_mgr() { else echo "none"; fi } +is_config_exists() { + if [ -n "$SUDO" ]; then + $SUDO sh -c '[ -f "$1" ]' _ "$CONFIG_FILE" + else + [ -f "$CONFIG_FILE" ] + fi +} + verify_common() { [ -n "$BIN_NAME" ] || die "BIN_NAME cannot be empty." [ -n "$INSTALL_DIR" ] || die "INSTALL_DIR cannot be empty." @@ -119,7 +177,7 @@ verify_common() { [ -n "$CONFIG_FILE" ] || die "CONFIG_FILE cannot be empty." case "${INSTALL_DIR}${CONFIG_DIR}${WORK_DIR}${CONFIG_FILE}" in - *[!a-zA-Z0-9_./-]*) die "Invalid characters in paths. Only alphanumeric, _, ., -, and / allowed." ;; + *[!a-zA-Z0-9_./-]*) die "Invalid characters in paths." ;; esac case "$TARGET_VERSION" in *[!a-zA-Z0-9_.-]*) die "Invalid characters in version." ;; esac @@ -137,11 +195,11 @@ verify_common() { if [ "$(id -u)" -eq 0 ]; then SUDO="" else - command -v sudo >/dev/null 2>&1 || die "This script requires root or sudo. Neither found." + command -v sudo >/dev/null 2>&1 || die "This script requires root or sudo." SUDO="sudo" if ! sudo -n true 2>/dev/null; then if ! [ -t 0 ]; then - die "sudo requires a password, but no TTY detected. Aborting to prevent hang." + die "sudo requires a password, but no TTY detected." fi fi fi @@ -154,21 +212,7 @@ verify_common() { die "Safety check failed: CONFIG_FILE '$CONFIG_FILE' is a directory." fi - for path in "$CONFIG_DIR" "$CONFIG_PARENT_DIR" "$WORK_DIR"; do - check_path="$(get_realpath "$path")" - case "$check_path" in - /|/bin|/sbin|/usr|/usr/bin|/usr/sbin|/usr/local|/usr/local/bin|/usr/local/sbin|/usr/local/etc|/usr/local/share|/etc|/var|/var/lib|/var/log|/var/run|/home|/root|/tmp|/lib|/lib64|/opt|/run|/boot|/dev|/sys|/proc) - die "Safety check failed: '$path' (resolved to '$check_path') is a critical system directory." ;; - esac - done - - check_install_dir="$(get_realpath "$INSTALL_DIR")" - case "$check_install_dir" in - /|/etc|/var|/home|/root|/tmp|/usr|/usr/local|/opt|/boot|/dev|/sys|/proc|/run) - die "Safety check failed: INSTALL_DIR '$INSTALL_DIR' is a critical system directory." ;; - esac - - for cmd in id uname grep find rm chown chmod mv mktemp mkdir tr dd sed ps head sleep cat tar gzip rmdir; do + for cmd in id uname awk grep find rm chown chmod mv mktemp mkdir tr dd sed ps head sleep cat tar gzip; do command -v "$cmd" >/dev/null 2>&1 || die "Required command not found: $cmd" done } @@ -177,14 +221,41 @@ verify_install_deps() { command -v curl >/dev/null 2>&1 || command -v wget >/dev/null 2>&1 || die "Neither curl nor wget is installed." command -v cp >/dev/null 2>&1 || command -v install >/dev/null 2>&1 || die "Need cp or install" - if ! command -v setcap >/dev/null 2>&1; then + if ! command -v setcap >/dev/null 2>&1 || ! command -v conntrack >/dev/null 2>&1; then if command -v apk >/dev/null 2>&1; then - $SUDO apk add --no-cache libcap-utils >/dev/null 2>&1 || $SUDO apk add --no-cache libcap >/dev/null 2>&1 || true + $SUDO apk add --no-cache libcap-utils libcap conntrack-tools >/dev/null 2>&1 || true elif command -v apt-get >/dev/null 2>&1; then - $SUDO apt-get update -q >/dev/null 2>&1 || true - $SUDO apt-get install -y -q libcap2-bin >/dev/null 2>&1 || true - elif command -v dnf >/dev/null 2>&1; then $SUDO dnf install -y -q libcap >/dev/null 2>&1 || true - elif command -v yum >/dev/null 2>&1; then $SUDO yum install -y -q libcap >/dev/null 2>&1 || true + $SUDO env DEBIAN_FRONTEND=noninteractive apt-get install -y -q libcap2-bin conntrack >/dev/null 2>&1 || { + $SUDO env DEBIAN_FRONTEND=noninteractive apt-get update -q >/dev/null 2>&1 || true + $SUDO env DEBIAN_FRONTEND=noninteractive apt-get install -y -q libcap2-bin conntrack >/dev/null 2>&1 || true + } + elif command -v dnf >/dev/null 2>&1; then $SUDO dnf install -y -q libcap conntrack-tools >/dev/null 2>&1 || true + elif command -v yum >/dev/null 2>&1; then $SUDO yum install -y -q libcap conntrack-tools >/dev/null 2>&1 || true + fi + fi +} + +check_port_availability() { + port_info="" + + if command -v ss >/dev/null 2>&1; then + port_info=$($SUDO ss -tulnp 2>/dev/null | grep -E ":${SERVER_PORT}([[:space:]]|$)" || true) + elif command -v netstat >/dev/null 2>&1; then + port_info=$($SUDO netstat -tulnp 2>/dev/null | grep -E ":${SERVER_PORT}([[:space:]]|$)" || true) + elif command -v lsof >/dev/null 2>&1; then + port_info=$($SUDO lsof -i :${SERVER_PORT} 2>/dev/null | grep LISTEN || true) + else + say "[WARNING] Network diagnostic tools (ss, netstat, lsof) not found. Skipping port check." + return 0 + fi + + if [ -n "$port_info" ]; then + if printf '%s\n' "$port_info" | grep -q "${BIN_NAME}"; then + say " -> Port ${SERVER_PORT} is in use by ${BIN_NAME}. Ignoring as it will be restarted." + else + say "[ERROR] Port ${SERVER_PORT} is already in use by another process:" + printf ' %s\n' "$port_info" + die "Please free the port ${SERVER_PORT} or change it and try again." fi fi } @@ -192,7 +263,13 @@ verify_install_deps() { detect_arch() { sys_arch="$(uname -m)" case "$sys_arch" in - x86_64|amd64) echo "x86_64" ;; + x86_64|amd64) + if [ -r /proc/cpuinfo ] && grep -q "avx2" /proc/cpuinfo 2>/dev/null && grep -q "bmi2" /proc/cpuinfo 2>/dev/null; then + echo "x86_64-v3" + else + echo "x86_64" + fi + ;; aarch64|arm64) echo "aarch64" ;; *) die "Unsupported architecture: $sys_arch" ;; esac @@ -236,10 +313,10 @@ ensure_user_group() { setup_dirs() { $SUDO mkdir -p "$WORK_DIR" "$CONFIG_DIR" "$CONFIG_PARENT_DIR" || die "Failed to create directories" - + $SUDO chown telemt:telemt "$WORK_DIR" && $SUDO chmod 750 "$WORK_DIR" $SUDO chown root:telemt "$CONFIG_DIR" && $SUDO chmod 750 "$CONFIG_DIR" - + if [ "$CONFIG_PARENT_DIR" != "$CONFIG_DIR" ] && [ "$CONFIG_PARENT_DIR" != "." ] && [ "$CONFIG_PARENT_DIR" != "/" ]; then $SUDO chown root:telemt "$CONFIG_PARENT_DIR" && $SUDO chmod 750 "$CONFIG_PARENT_DIR" fi @@ -261,17 +338,19 @@ install_binary() { fi $SUDO mkdir -p "$INSTALL_DIR" || die "Failed to create install directory" + + $SUDO rm -f "$bin_dst" 2>/dev/null || true + if command -v install >/dev/null 2>&1; then $SUDO install -m 0755 "$bin_src" "$bin_dst" || die "Failed to install binary" else - $SUDO rm -f "$bin_dst" 2>/dev/null || true $SUDO cp "$bin_src" "$bin_dst" && $SUDO chmod 0755 "$bin_dst" || die "Failed to copy binary" fi $SUDO sh -c '[ -x "$1" ]' _ "$bin_dst" || die "Binary not executable: $bin_dst" if command -v setcap >/dev/null 2>&1; then - $SUDO setcap cap_net_bind_service=+ep "$bin_dst" 2>/dev/null || true + $SUDO setcap cap_net_bind_service,cap_net_admin=+ep "$bin_dst" 2>/dev/null || true fi } @@ -287,11 +366,20 @@ generate_secret() { } generate_config_content() { + conf_secret="$1" + conf_tag="$2" escaped_tls_domain="$(printf '%s\n' "$TLS_DOMAIN" | tr -d '[:cntrl:]' | sed 's/\\/\\\\/g; s/"/\\"/g')" cat < Config already exists at $CONFIG_FILE. Skipping creation." - return 0 - fi - elif [ -f "$CONFIG_FILE" ]; then - say " -> Config already exists at $CONFIG_FILE. Skipping creation." + if is_config_exists; then + say " -> Config already exists at $CONFIG_FILE. Updating parameters..." + + tmp_conf="${TEMP_DIR}/config.tmp" + $SUDO cat "$CONFIG_FILE" > "$tmp_conf" + + escaped_domain="$(printf '%s\n' "$TLS_DOMAIN" | tr -d '[:cntrl:]' | sed 's/\\/\\\\/g; s/"/\\"/g')" + + export AWK_PORT="$SERVER_PORT" + export AWK_SECRET="$USER_SECRET" + export AWK_DOMAIN="$escaped_domain" + export AWK_AD_TAG="$AD_TAG" + export AWK_FLAG_P="$PORT_PROVIDED" + export AWK_FLAG_S="$SECRET_PROVIDED" + export AWK_FLAG_D="$DOMAIN_PROVIDED" + export AWK_FLAG_A="$AD_TAG_PROVIDED" + + awk ' + BEGIN { ad_tag_handled = 0 } + + ENVIRON["AWK_FLAG_P"] == "1" && /^[ \t]*port[ \t]*=/ { print "port = " ENVIRON["AWK_PORT"]; next } + ENVIRON["AWK_FLAG_S"] == "1" && /^[ \t]*hello[ \t]*=/ { print "hello = \"" ENVIRON["AWK_SECRET"] "\""; next } + ENVIRON["AWK_FLAG_D"] == "1" && /^[ \t]*tls_domain[ \t]*=/ { print "tls_domain = \"" ENVIRON["AWK_DOMAIN"] "\""; next } + + ENVIRON["AWK_FLAG_A"] == "1" && /^[ \t]*ad_tag[ \t]*=/ { + if (!ad_tag_handled) { + print "ad_tag = \"" ENVIRON["AWK_AD_TAG"] "\""; + ad_tag_handled = 1; + } + next + } + ENVIRON["AWK_FLAG_A"] == "1" && /^\[general\]/ { + print; + if (!ad_tag_handled) { + print "ad_tag = \"" ENVIRON["AWK_AD_TAG"] "\""; + ad_tag_handled = 1; + } + next + } + + { print } + ' "$tmp_conf" > "${tmp_conf}.new" && mv "${tmp_conf}.new" "$tmp_conf" + + [ "$PORT_PROVIDED" -eq 1 ] && say " -> Updated port: $SERVER_PORT" + [ "$SECRET_PROVIDED" -eq 1 ] && say " -> Updated secret for user 'hello'" + [ "$DOMAIN_PROVIDED" -eq 1 ] && say " -> Updated tls_domain: $TLS_DOMAIN" + [ "$AD_TAG_PROVIDED" -eq 1 ] && say " -> Updated ad_tag" + + write_root "$CONFIG_FILE" < "$tmp_conf" + rm -f "$tmp_conf" return 0 fi - toml_secret="$(generate_secret)" || die "Failed to generate secret." + if [ -z "$USER_SECRET" ]; then + USER_SECRET="$(generate_secret)" || die "Failed to generate secret." + fi - generate_config_content "$toml_secret" | write_root "$CONFIG_FILE" || die "Failed to install config" + generate_config_content "$USER_SECRET" "$AD_TAG" | write_root "$CONFIG_FILE" || die "Failed to install config" $SUDO chown root:telemt "$CONFIG_FILE" && $SUDO chmod 640 "$CONFIG_FILE" say " -> Config created successfully." - say " -> Generated secret for default user 'hello': $toml_secret" + say " -> Configured secret for user 'hello': $USER_SECRET" } generate_systemd_content() { @@ -348,9 +481,10 @@ Group=telemt WorkingDirectory=$WORK_DIR ExecStart="${INSTALL_DIR}/${BIN_NAME}" "${CONFIG_FILE}" Restart=on-failure +RestartSec=5 LimitNOFILE=65536 -AmbientCapabilities=CAP_NET_BIND_SERVICE -CapabilityBoundingSet=CAP_NET_BIND_SERVICE +AmbientCapabilities=CAP_NET_BIND_SERVICE CAP_NET_ADMIN +CapabilityBoundingSet=CAP_NET_BIND_SERVICE CAP_NET_ADMIN [Install] WantedBy=multi-user.target @@ -381,7 +515,7 @@ install_service() { $SUDO systemctl daemon-reload || true $SUDO systemctl enable "$SERVICE_NAME" || true - + if ! $SUDO systemctl start "$SERVICE_NAME"; then say "[WARNING] Failed to start service" SERVICE_START_FAILED=1 @@ -391,16 +525,16 @@ install_service() { $SUDO chown root:root "/etc/init.d/${SERVICE_NAME}" && $SUDO chmod 0755 "/etc/init.d/${SERVICE_NAME}" $SUDO rc-update add "$SERVICE_NAME" default 2>/dev/null || true - + if ! $SUDO rc-service "$SERVICE_NAME" start 2>/dev/null; then say "[WARNING] Failed to start service" SERVICE_START_FAILED=1 fi else cmd="\"${INSTALL_DIR}/${BIN_NAME}\" \"${CONFIG_FILE}\"" - if [ -n "$SUDO" ]; then + if [ -n "$SUDO" ]; then say " -> Service manager not found. Start manually: sudo -u telemt $cmd" - else + else say " -> Service manager not found. Start manually: su -s /bin/sh telemt -c '$cmd'" fi fi @@ -415,9 +549,10 @@ kill_user_procs() { if command -v pgrep >/dev/null 2>&1; then pids="$(pgrep -u telemt 2>/dev/null || true)" else - pids="$(ps -u telemt -o pid= 2>/dev/null || true)" + pids="$(ps -ef 2>/dev/null | awk '$1=="telemt"{print $2}' || true)" + [ -z "$pids" ] && pids="$(ps 2>/dev/null | awk '$2=="telemt"{print $1}' || true)" fi - + if [ -n "$pids" ]; then for pid in $pids; do case "$pid" in ''|*[!0-9]*) continue ;; *) $SUDO kill "$pid" 2>/dev/null || true ;; esac @@ -457,15 +592,16 @@ uninstall() { say ">>> Stage 5: Purging configuration, data, and user" $SUDO rm -rf "$CONFIG_DIR" "$WORK_DIR" $SUDO rm -f "$CONFIG_FILE" - if [ "$CONFIG_PARENT_DIR" != "$CONFIG_DIR" ] && [ "$CONFIG_PARENT_DIR" != "." ] && [ "$CONFIG_PARENT_DIR" != "/" ]; then - $SUDO rmdir "$CONFIG_PARENT_DIR" 2>/dev/null || true - fi + sleep 1 $SUDO userdel telemt 2>/dev/null || $SUDO deluser telemt 2>/dev/null || true - $SUDO groupdel telemt 2>/dev/null || $SUDO delgroup telemt 2>/dev/null || true + + if check_os_entity group telemt; then + $SUDO groupdel telemt 2>/dev/null || $SUDO delgroup telemt 2>/dev/null || true + fi else say "Note: Configuration and user kept. Run with 'purge' to remove completely." fi - + printf '\n====================================================================\n' printf ' UNINSTALLATION COMPLETE\n' printf '====================================================================\n\n' @@ -479,18 +615,28 @@ case "$ACTION" in say "Starting installation of $BIN_NAME (Version: $TARGET_VERSION)" say ">>> Stage 1: Verifying environment and dependencies" - verify_common; verify_install_deps + verify_common + verify_install_deps - if [ "$TARGET_VERSION" != "latest" ]; then + if is_config_exists && [ "$PORT_PROVIDED" -eq 0 ]; then + ext_port="$($SUDO awk -F'=' '/^[ \t]*port[ \t]*=/ {gsub(/[^0-9]/, "", $2); print $2; exit}' "$CONFIG_FILE" 2>/dev/null || true)" + if [ -n "$ext_port" ]; then + SERVER_PORT="$ext_port" + fi + fi + + check_port_availability + + if [ "$TARGET_VERSION" != "latest" ]; then TARGET_VERSION="${TARGET_VERSION#v}" fi - + ARCH="$(detect_arch)"; LIBC="$(detect_libc)" FILE_NAME="${BIN_NAME}-${ARCH}-linux-${LIBC}.tar.gz" - + if [ "$TARGET_VERSION" = "latest" ]; then DL_URL="https://github.com/${REPO}/releases/latest/download/${FILE_NAME}" - else + else DL_URL="https://github.com/${REPO}/releases/download/${TARGET_VERSION}/${FILE_NAME}" fi @@ -500,7 +646,21 @@ case "$ACTION" in die "Temp directory is invalid or was not created" fi - fetch_file "$DL_URL" "${TEMP_DIR}/${FILE_NAME}" || die "Download failed" + if ! fetch_file "$DL_URL" "${TEMP_DIR}/${FILE_NAME}"; then + if [ "$ARCH" = "x86_64-v3" ]; then + say " -> x86_64-v3 build not found, falling back to standard x86_64..." + ARCH="x86_64" + FILE_NAME="${BIN_NAME}-${ARCH}-linux-${LIBC}.tar.gz" + if [ "$TARGET_VERSION" = "latest" ]; then + DL_URL="https://github.com/${REPO}/releases/latest/download/${FILE_NAME}" + else + DL_URL="https://github.com/${REPO}/releases/download/${TARGET_VERSION}/${FILE_NAME}" + fi + fetch_file "$DL_URL" "${TEMP_DIR}/${FILE_NAME}" || die "Download failed" + else + die "Download failed" + fi + fi say ">>> Stage 3: Extracting archive" if ! gzip -dc "${TEMP_DIR}/${FILE_NAME}" | tar -xf - -C "$TEMP_DIR" 2>/dev/null; then @@ -512,13 +672,13 @@ case "$ACTION" in say ">>> Stage 4: Setting up environment (User, Group, Directories)" ensure_user_group; setup_dirs; stop_service - + say ">>> Stage 5: Installing binary" install_binary "$EXTRACTED_BIN" "${INSTALL_DIR}/${BIN_NAME}" - - say ">>> Stage 6: Generating configuration" + + say ">>> Stage 6: Generating/Updating configuration" install_config - + say ">>> Stage 7: Installing and starting service" install_service @@ -533,7 +693,7 @@ case "$ACTION" in printf ' INSTALLATION SUCCESS\n' printf '====================================================================\n\n' fi - + svc="$(get_svc_mgr)" if [ "$svc" = "systemd" ]; then printf 'To check the status of your proxy service, run:\n' @@ -542,15 +702,18 @@ case "$ACTION" in printf 'To check the status of your proxy service, run:\n' printf ' rc-service %s status\n\n' "$SERVICE_NAME" fi - + + API_LISTEN="$($SUDO awk -F'"' '/^[ \t]*listen[ \t]*=/ {print $2; exit}' "$CONFIG_FILE" 2>/dev/null || true)" + API_LISTEN="${API_LISTEN:-127.0.0.1:9091}" + printf 'To get your user connection links (for Telegram), run:\n' if command -v jq >/dev/null 2>&1; then - printf ' curl -s http://127.0.0.1:9091/v1/users | jq -r '\''.data[] | "User: \\(.username)\\n\\(.links.tls[0] // empty)\\n"'\''\n' + printf ' curl -s http://%s/v1/users | jq -r '\''.data[]? | "User: \\(.username)\\n\\(.links.tls[0] // empty)\\n"'\''\n' "$API_LISTEN" else - printf ' curl -s http://127.0.0.1:9091/v1/users\n' + printf ' curl -s http://%s/v1/users\n' "$API_LISTEN" printf ' (Tip: Install '\''jq'\'' for a much cleaner output)\n' fi - + printf '\n====================================================================\n' ;; esac diff --git a/src/api/model.rs b/src/api/model.rs index ebc67d7..66de644 100644 --- a/src/api/model.rs +++ b/src/api/model.rs @@ -81,10 +81,21 @@ pub(super) struct ZeroCoreData { pub(super) connections_total: u64, pub(super) connections_bad_total: u64, pub(super) handshake_timeouts_total: u64, + pub(super) accept_permit_timeout_total: u64, pub(super) configured_users: usize, pub(super) telemetry_core_enabled: bool, pub(super) telemetry_user_enabled: bool, pub(super) telemetry_me_level: String, + pub(super) conntrack_control_enabled: bool, + pub(super) conntrack_control_available: bool, + pub(super) conntrack_pressure_active: bool, + pub(super) conntrack_event_queue_depth: u64, + pub(super) conntrack_rule_apply_ok: bool, + pub(super) conntrack_delete_attempt_total: u64, + pub(super) conntrack_delete_success_total: u64, + pub(super) conntrack_delete_not_found_total: u64, + pub(super) conntrack_delete_error_total: u64, + pub(super) conntrack_close_event_drop_total: u64, } #[derive(Serialize, Clone)] diff --git a/src/api/runtime_stats.rs b/src/api/runtime_stats.rs index b66d1a5..131acef 100644 --- a/src/api/runtime_stats.rs +++ b/src/api/runtime_stats.rs @@ -39,10 +39,21 @@ pub(super) fn build_zero_all_data(stats: &Stats, configured_users: usize) -> Zer connections_total: stats.get_connects_all(), connections_bad_total: stats.get_connects_bad(), handshake_timeouts_total: stats.get_handshake_timeouts(), + accept_permit_timeout_total: stats.get_accept_permit_timeout_total(), configured_users, telemetry_core_enabled: telemetry.core_enabled, telemetry_user_enabled: telemetry.user_enabled, telemetry_me_level: telemetry.me_level.to_string(), + conntrack_control_enabled: stats.get_conntrack_control_enabled(), + conntrack_control_available: stats.get_conntrack_control_available(), + conntrack_pressure_active: stats.get_conntrack_pressure_active(), + conntrack_event_queue_depth: stats.get_conntrack_event_queue_depth(), + conntrack_rule_apply_ok: stats.get_conntrack_rule_apply_ok(), + conntrack_delete_attempt_total: stats.get_conntrack_delete_attempt_total(), + conntrack_delete_success_total: stats.get_conntrack_delete_success_total(), + conntrack_delete_not_found_total: stats.get_conntrack_delete_not_found_total(), + conntrack_delete_error_total: stats.get_conntrack_delete_error_total(), + conntrack_close_event_drop_total: stats.get_conntrack_close_event_drop_total(), }, upstream: build_zero_upstream_data(stats), middle_proxy: ZeroMiddleProxyData { diff --git a/src/config/defaults.rs b/src/config/defaults.rs index 89e72bb..beedd10 100644 --- a/src/config/defaults.rs +++ b/src/config/defaults.rs @@ -48,6 +48,10 @@ const DEFAULT_ME_POOL_DRAIN_SOFT_EVICT_BUDGET_PER_CORE: u16 = 16; const DEFAULT_ME_POOL_DRAIN_SOFT_EVICT_COOLDOWN_MS: u64 = 1000; const DEFAULT_USER_MAX_UNIQUE_IPS_WINDOW_SECS: u64 = 30; const DEFAULT_ACCEPT_PERMIT_TIMEOUT_MS: u64 = 250; +const DEFAULT_CONNTRACK_CONTROL_ENABLED: bool = true; +const DEFAULT_CONNTRACK_PRESSURE_HIGH_WATERMARK_PCT: u8 = 85; +const DEFAULT_CONNTRACK_PRESSURE_LOW_WATERMARK_PCT: u8 = 70; +const DEFAULT_CONNTRACK_DELETE_BUDGET_PER_SEC: u64 = 4096; const DEFAULT_UPSTREAM_CONNECT_RETRY_ATTEMPTS: u32 = 2; const DEFAULT_UPSTREAM_UNHEALTHY_FAIL_THRESHOLD: u32 = 5; const DEFAULT_UPSTREAM_CONNECT_BUDGET_MS: u64 = 3000; @@ -96,7 +100,7 @@ pub(crate) fn default_fake_cert_len() -> usize { } pub(crate) fn default_tls_front_dir() -> String { - "tlsfront".to_string() + "/etc/telemt/tlsfront".to_string() } pub(crate) fn default_replay_check_len() -> usize { @@ -221,6 +225,22 @@ pub(crate) fn default_accept_permit_timeout_ms() -> u64 { DEFAULT_ACCEPT_PERMIT_TIMEOUT_MS } +pub(crate) fn default_conntrack_control_enabled() -> bool { + DEFAULT_CONNTRACK_CONTROL_ENABLED +} + +pub(crate) fn default_conntrack_pressure_high_watermark_pct() -> u8 { + DEFAULT_CONNTRACK_PRESSURE_HIGH_WATERMARK_PCT +} + +pub(crate) fn default_conntrack_pressure_low_watermark_pct() -> u8 { + DEFAULT_CONNTRACK_PRESSURE_LOW_WATERMARK_PCT +} + +pub(crate) fn default_conntrack_delete_budget_per_sec() -> u64 { + DEFAULT_CONNTRACK_DELETE_BUDGET_PER_SEC +} + pub(crate) fn default_prefer_4() -> u8 { 4 } @@ -282,7 +302,7 @@ pub(crate) fn default_me2dc_fallback() -> bool { } pub(crate) fn default_me2dc_fast() -> bool { - false + true } pub(crate) fn default_keepalive_interval() -> u64 { @@ -538,7 +558,7 @@ pub(crate) fn default_beobachten_flush_secs() -> u64 { } pub(crate) fn default_beobachten_file() -> String { - "cache/beobachten.txt".to_string() + "/etc/telemt/beobachten.txt".to_string() } pub(crate) fn default_tls_new_session_tickets() -> u8 { diff --git a/src/config/hot_reload.rs b/src/config/hot_reload.rs index fa42c55..5582e9b 100644 --- a/src/config/hot_reload.rs +++ b/src/config/hot_reload.rs @@ -540,6 +540,10 @@ fn overlay_hot_fields(old: &ProxyConfig, new: &ProxyConfig) -> ProxyConfig { cfg.access.user_max_unique_ips_mode = new.access.user_max_unique_ips_mode; cfg.access.user_max_unique_ips_window_secs = new.access.user_max_unique_ips_window_secs; + if cfg.rebuild_runtime_user_auth().is_err() { + cfg.runtime_user_auth = None; + } + cfg } diff --git a/src/config/load.rs b/src/config/load.rs index cc95f34..f9e230c 100644 --- a/src/config/load.rs +++ b/src/config/load.rs @@ -4,6 +4,7 @@ use std::collections::{BTreeSet, HashMap, HashSet}; use std::hash::{DefaultHasher, Hash, Hasher}; use std::net::{IpAddr, SocketAddr}; use std::path::{Path, PathBuf}; +use std::sync::Arc; use rand::RngExt; use serde::{Deserialize, Serialize}; @@ -15,6 +16,13 @@ use crate::error::{ProxyError, Result}; use super::defaults::*; use super::types::*; +const ACCESS_SECRET_BYTES: usize = 16; +const MAX_ME_WRITER_CMD_CHANNEL_CAPACITY: usize = 16_384; +const MAX_ME_ROUTE_CHANNEL_CAPACITY: usize = 8_192; +const MAX_ME_C2ME_CHANNEL_CAPACITY: usize = 8_192; +const MIN_MAX_CLIENT_FRAME_BYTES: usize = 4 * 1024; +const MAX_MAX_CLIENT_FRAME_BYTES: usize = 16 * 1024 * 1024; + #[derive(Debug, Clone)] pub(crate) struct LoadedConfig { pub(crate) config: ProxyConfig, @@ -22,6 +30,111 @@ pub(crate) struct LoadedConfig { pub(crate) rendered_hash: u64, } +/// Precomputed, immutable user authentication data used by handshake hot paths. +#[derive(Debug, Clone, Default)] +pub(crate) struct UserAuthSnapshot { + entries: Vec, + by_name: HashMap, + sni_index: HashMap>, + sni_initial_index: HashMap>, +} + +#[derive(Debug, Clone)] +pub(crate) struct UserAuthEntry { + pub(crate) user: String, + pub(crate) secret: [u8; ACCESS_SECRET_BYTES], +} + +impl UserAuthSnapshot { + fn from_users(users: &HashMap) -> Result { + let mut entries = Vec::with_capacity(users.len()); + let mut by_name = HashMap::with_capacity(users.len()); + let mut sni_index = HashMap::with_capacity(users.len()); + let mut sni_initial_index = HashMap::with_capacity(users.len()); + + for (user, secret_hex) in users { + let decoded = hex::decode(secret_hex).map_err(|_| ProxyError::InvalidSecret { + user: user.clone(), + reason: "Must be 32 hex characters".to_string(), + })?; + if decoded.len() != ACCESS_SECRET_BYTES { + return Err(ProxyError::InvalidSecret { + user: user.clone(), + reason: "Must be 32 hex characters".to_string(), + }); + } + + let user_id = u32::try_from(entries.len()).map_err(|_| { + ProxyError::Config("Too many users for runtime auth snapshot".to_string()) + })?; + + let mut secret = [0u8; ACCESS_SECRET_BYTES]; + secret.copy_from_slice(&decoded); + entries.push(UserAuthEntry { + user: user.clone(), + secret, + }); + by_name.insert(user.clone(), user_id); + sni_index + .entry(Self::sni_lookup_hash(user)) + .or_insert_with(Vec::new) + .push(user_id); + if let Some(initial) = user + .as_bytes() + .first() + .map(|byte| byte.to_ascii_lowercase()) + { + sni_initial_index + .entry(initial) + .or_insert_with(Vec::new) + .push(user_id); + } + } + + Ok(Self { + entries, + by_name, + sni_index, + sni_initial_index, + }) + } + + pub(crate) fn entries(&self) -> &[UserAuthEntry] { + &self.entries + } + + pub(crate) fn user_id_by_name(&self, user: &str) -> Option { + self.by_name.get(user).copied() + } + + pub(crate) fn entry_by_id(&self, user_id: u32) -> Option<&UserAuthEntry> { + let idx = usize::try_from(user_id).ok()?; + self.entries.get(idx) + } + + pub(crate) fn sni_candidates(&self, sni: &str) -> Option<&[u32]> { + self.sni_index + .get(&Self::sni_lookup_hash(sni)) + .map(Vec::as_slice) + } + + pub(crate) fn sni_initial_candidates(&self, sni: &str) -> Option<&[u32]> { + let initial = sni + .as_bytes() + .first() + .map(|byte| byte.to_ascii_lowercase())?; + self.sni_initial_index.get(&initial).map(Vec::as_slice) + } + + fn sni_lookup_hash(value: &str) -> u64 { + let mut hasher = DefaultHasher::new(); + for byte in value.bytes() { + hasher.write_u8(byte.to_ascii_lowercase()); + } + hasher.finish() + } +} + fn normalize_config_path(path: &Path) -> PathBuf { path.canonicalize().unwrap_or_else(|_| { if path.is_absolute() { @@ -196,6 +309,10 @@ pub struct ProxyConfig { /// If not set, defaults to 2 (matching Telegram's official `default 2;` in proxy-multi.conf). #[serde(default)] pub default_dc: Option, + + /// Precomputed authentication snapshot for handshake hot paths. + #[serde(skip)] + pub(crate) runtime_user_auth: Option>, } impl ProxyConfig { @@ -514,18 +631,41 @@ impl ProxyConfig { "general.me_writer_cmd_channel_capacity must be > 0".to_string(), )); } + if config.general.me_writer_cmd_channel_capacity > MAX_ME_WRITER_CMD_CHANNEL_CAPACITY { + return Err(ProxyError::Config(format!( + "general.me_writer_cmd_channel_capacity must be within [1, {MAX_ME_WRITER_CMD_CHANNEL_CAPACITY}]" + ))); + } if config.general.me_route_channel_capacity == 0 { return Err(ProxyError::Config( "general.me_route_channel_capacity must be > 0".to_string(), )); } + if config.general.me_route_channel_capacity > MAX_ME_ROUTE_CHANNEL_CAPACITY { + return Err(ProxyError::Config(format!( + "general.me_route_channel_capacity must be within [1, {MAX_ME_ROUTE_CHANNEL_CAPACITY}]" + ))); + } if config.general.me_c2me_channel_capacity == 0 { return Err(ProxyError::Config( "general.me_c2me_channel_capacity must be > 0".to_string(), )); } + if config.general.me_c2me_channel_capacity > MAX_ME_C2ME_CHANNEL_CAPACITY { + return Err(ProxyError::Config(format!( + "general.me_c2me_channel_capacity must be within [1, {MAX_ME_C2ME_CHANNEL_CAPACITY}]" + ))); + } + + if !(MIN_MAX_CLIENT_FRAME_BYTES..=MAX_MAX_CLIENT_FRAME_BYTES) + .contains(&config.general.max_client_frame) + { + return Err(ProxyError::Config(format!( + "general.max_client_frame must be within [{MIN_MAX_CLIENT_FRAME_BYTES}, {MAX_MAX_CLIENT_FRAME_BYTES}]" + ))); + } if config.general.me_c2me_send_timeout_ms > 60_000 { return Err(ProxyError::Config( @@ -922,6 +1062,43 @@ impl ProxyConfig { )); } + if config.server.conntrack_control.pressure_high_watermark_pct == 0 + || config.server.conntrack_control.pressure_high_watermark_pct > 100 + { + return Err(ProxyError::Config( + "server.conntrack_control.pressure_high_watermark_pct must be within [1, 100]" + .to_string(), + )); + } + + if config.server.conntrack_control.pressure_low_watermark_pct + >= config.server.conntrack_control.pressure_high_watermark_pct + { + return Err(ProxyError::Config( + "server.conntrack_control.pressure_low_watermark_pct must be < pressure_high_watermark_pct" + .to_string(), + )); + } + + if config.server.conntrack_control.delete_budget_per_sec == 0 { + return Err(ProxyError::Config( + "server.conntrack_control.delete_budget_per_sec must be > 0".to_string(), + )); + } + + if matches!(config.server.conntrack_control.mode, ConntrackMode::Hybrid) + && config + .server + .conntrack_control + .hybrid_listener_ips + .is_empty() + { + return Err(ProxyError::Config( + "server.conntrack_control.hybrid_listener_ips must be non-empty in mode=hybrid" + .to_string(), + )); + } + if config.general.effective_me_pool_force_close_secs() > 0 && config.general.effective_me_pool_force_close_secs() < config.general.me_pool_drain_ttl_secs @@ -1127,6 +1304,7 @@ impl ProxyConfig { .or_insert_with(|| vec!["91.105.192.100:443".to_string()]); validate_upstreams(&config)?; + config.rebuild_runtime_user_auth()?; Ok(LoadedConfig { config, @@ -1135,6 +1313,16 @@ impl ProxyConfig { }) } + pub(crate) fn rebuild_runtime_user_auth(&mut self) -> Result<()> { + let snapshot = UserAuthSnapshot::from_users(&self.access.users)?; + self.runtime_user_auth = Some(Arc::new(snapshot)); + Ok(()) + } + + pub(crate) fn runtime_user_auth(&self) -> Option<&UserAuthSnapshot> { + self.runtime_user_auth.as_deref() + } + pub fn validate(&self) -> Result<()> { if self.access.users.is_empty() { return Err(ProxyError::Config("No users configured".to_string())); @@ -1186,6 +1374,10 @@ mod load_mask_shape_security_tests; #[path = "tests/load_mask_classifier_prefetch_timeout_security_tests.rs"] mod load_mask_classifier_prefetch_timeout_security_tests; +#[cfg(test)] +#[path = "tests/load_memory_envelope_tests.rs"] +mod load_memory_envelope_tests; + #[cfg(test)] mod tests { use super::*; @@ -1327,6 +1519,31 @@ mod tests { cfg.server.api.runtime_edge_events_capacity, default_api_runtime_edge_events_capacity() ); + assert_eq!( + cfg.server.conntrack_control.inline_conntrack_control, + default_conntrack_control_enabled() + ); + assert_eq!(cfg.server.conntrack_control.mode, ConntrackMode::default()); + assert_eq!( + cfg.server.conntrack_control.backend, + ConntrackBackend::default() + ); + assert_eq!( + cfg.server.conntrack_control.profile, + ConntrackPressureProfile::default() + ); + assert_eq!( + cfg.server.conntrack_control.pressure_high_watermark_pct, + default_conntrack_pressure_high_watermark_pct() + ); + assert_eq!( + cfg.server.conntrack_control.pressure_low_watermark_pct, + default_conntrack_pressure_low_watermark_pct() + ); + assert_eq!( + cfg.server.conntrack_control.delete_budget_per_sec, + default_conntrack_delete_budget_per_sec() + ); assert_eq!(cfg.access.users, default_access_users()); assert_eq!( cfg.access.user_max_tcp_conns_global_each, @@ -1472,6 +1689,31 @@ mod tests { server.api.runtime_edge_events_capacity, default_api_runtime_edge_events_capacity() ); + assert_eq!( + server.conntrack_control.inline_conntrack_control, + default_conntrack_control_enabled() + ); + assert_eq!(server.conntrack_control.mode, ConntrackMode::default()); + assert_eq!( + server.conntrack_control.backend, + ConntrackBackend::default() + ); + assert_eq!( + server.conntrack_control.profile, + ConntrackPressureProfile::default() + ); + assert_eq!( + server.conntrack_control.pressure_high_watermark_pct, + default_conntrack_pressure_high_watermark_pct() + ); + assert_eq!( + server.conntrack_control.pressure_low_watermark_pct, + default_conntrack_pressure_low_watermark_pct() + ); + assert_eq!( + server.conntrack_control.delete_budget_per_sec, + default_conntrack_delete_budget_per_sec() + ); let access = AccessConfig::default(); assert_eq!(access.users, default_access_users()); @@ -1548,6 +1790,22 @@ mod tests { cfg_mask.censorship.unknown_sni_action, UnknownSniAction::Mask ); + + let cfg_accept: ProxyConfig = toml::from_str( + r#" + [server] + [general] + [network] + [access] + [censorship] + unknown_sni_action = "accept" + "#, + ) + .unwrap(); + assert_eq!( + cfg_accept.censorship.unknown_sni_action, + UnknownSniAction::Accept + ); } #[test] @@ -2404,6 +2662,118 @@ mod tests { let _ = std::fs::remove_file(path); } + #[test] + fn conntrack_pressure_high_watermark_out_of_range_is_rejected() { + let toml = r#" + [server.conntrack_control] + pressure_high_watermark_pct = 0 + + [censorship] + tls_domain = "example.com" + + [access.users] + user = "00000000000000000000000000000000" + "#; + let dir = std::env::temp_dir(); + let path = dir.join("telemt_conntrack_high_watermark_invalid_test.toml"); + std::fs::write(&path, toml).unwrap(); + let err = ProxyConfig::load(&path).unwrap_err().to_string(); + assert!(err.contains( + "server.conntrack_control.pressure_high_watermark_pct must be within [1, 100]" + )); + let _ = std::fs::remove_file(path); + } + + #[test] + fn conntrack_pressure_low_watermark_must_be_below_high() { + let toml = r#" + [server.conntrack_control] + pressure_high_watermark_pct = 50 + pressure_low_watermark_pct = 50 + + [censorship] + tls_domain = "example.com" + + [access.users] + user = "00000000000000000000000000000000" + "#; + let dir = std::env::temp_dir(); + let path = dir.join("telemt_conntrack_low_watermark_invalid_test.toml"); + std::fs::write(&path, toml).unwrap(); + let err = ProxyConfig::load(&path).unwrap_err().to_string(); + assert!( + err.contains( + "server.conntrack_control.pressure_low_watermark_pct must be < pressure_high_watermark_pct" + ) + ); + let _ = std::fs::remove_file(path); + } + + #[test] + fn conntrack_delete_budget_zero_is_rejected() { + let toml = r#" + [server.conntrack_control] + delete_budget_per_sec = 0 + + [censorship] + tls_domain = "example.com" + + [access.users] + user = "00000000000000000000000000000000" + "#; + let dir = std::env::temp_dir(); + let path = dir.join("telemt_conntrack_delete_budget_invalid_test.toml"); + std::fs::write(&path, toml).unwrap(); + let err = ProxyConfig::load(&path).unwrap_err().to_string(); + assert!(err.contains("server.conntrack_control.delete_budget_per_sec must be > 0")); + let _ = std::fs::remove_file(path); + } + + #[test] + fn conntrack_hybrid_mode_requires_listener_allow_list() { + let toml = r#" + [server.conntrack_control] + mode = "hybrid" + + [censorship] + tls_domain = "example.com" + + [access.users] + user = "00000000000000000000000000000000" + "#; + let dir = std::env::temp_dir(); + let path = dir.join("telemt_conntrack_hybrid_requires_ips_test.toml"); + std::fs::write(&path, toml).unwrap(); + let err = ProxyConfig::load(&path).unwrap_err().to_string(); + assert!(err.contains( + "server.conntrack_control.hybrid_listener_ips must be non-empty in mode=hybrid" + )); + let _ = std::fs::remove_file(path); + } + + #[test] + fn conntrack_profile_is_loaded_from_config() { + let toml = r#" + [server.conntrack_control] + profile = "aggressive" + + [censorship] + tls_domain = "example.com" + + [access.users] + user = "00000000000000000000000000000000" + "#; + let dir = std::env::temp_dir(); + let path = dir.join("telemt_conntrack_profile_parse_test.toml"); + std::fs::write(&path, toml).unwrap(); + let cfg = ProxyConfig::load(&path).unwrap(); + assert_eq!( + cfg.server.conntrack_control.profile, + ConntrackPressureProfile::Aggressive + ); + let _ = std::fs::remove_file(path); + } + #[test] fn force_close_default_matches_drain_ttl() { let toml = r#" diff --git a/src/config/tests/load_memory_envelope_tests.rs b/src/config/tests/load_memory_envelope_tests.rs new file mode 100644 index 0000000..b2d14fb --- /dev/null +++ b/src/config/tests/load_memory_envelope_tests.rs @@ -0,0 +1,115 @@ +use super::*; +use std::fs; +use std::path::PathBuf; +use std::time::{SystemTime, UNIX_EPOCH}; + +fn write_temp_config(contents: &str) -> PathBuf { + let nonce = SystemTime::now() + .duration_since(UNIX_EPOCH) + .expect("system time must be after unix epoch") + .as_nanos(); + let path = std::env::temp_dir().join(format!("telemt-load-memory-envelope-{nonce}.toml")); + fs::write(&path, contents).expect("temp config write must succeed"); + path +} + +fn remove_temp_config(path: &PathBuf) { + let _ = fs::remove_file(path); +} + +#[test] +fn load_rejects_writer_cmd_capacity_above_upper_bound() { + let path = write_temp_config( + r#" +[general] +me_writer_cmd_channel_capacity = 16385 +"#, + ); + + let err = ProxyConfig::load(&path).expect_err("writer command capacity above hard cap must fail"); + let msg = err.to_string(); + assert!( + msg.contains("general.me_writer_cmd_channel_capacity must be within [1, 16384]"), + "error must explain writer command capacity hard cap, got: {msg}" + ); + + remove_temp_config(&path); +} + +#[test] +fn load_rejects_route_channel_capacity_above_upper_bound() { + let path = write_temp_config( + r#" +[general] +me_route_channel_capacity = 8193 +"#, + ); + + let err = ProxyConfig::load(&path).expect_err("route channel capacity above hard cap must fail"); + let msg = err.to_string(); + assert!( + msg.contains("general.me_route_channel_capacity must be within [1, 8192]"), + "error must explain route channel hard cap, got: {msg}" + ); + + remove_temp_config(&path); +} + +#[test] +fn load_rejects_c2me_channel_capacity_above_upper_bound() { + let path = write_temp_config( + r#" +[general] +me_c2me_channel_capacity = 8193 +"#, + ); + + let err = ProxyConfig::load(&path).expect_err("c2me channel capacity above hard cap must fail"); + let msg = err.to_string(); + assert!( + msg.contains("general.me_c2me_channel_capacity must be within [1, 8192]"), + "error must explain c2me channel hard cap, got: {msg}" + ); + + remove_temp_config(&path); +} + +#[test] +fn load_rejects_max_client_frame_above_upper_bound() { + let path = write_temp_config( + r#" +[general] +max_client_frame = 16777217 +"#, + ); + + let err = ProxyConfig::load(&path).expect_err("max_client_frame above hard cap must fail"); + let msg = err.to_string(); + assert!( + msg.contains("general.max_client_frame must be within [4096, 16777216]"), + "error must explain max_client_frame hard cap, got: {msg}" + ); + + remove_temp_config(&path); +} + +#[test] +fn load_accepts_memory_limits_at_hard_upper_bounds() { + let path = write_temp_config( + r#" +[general] +me_writer_cmd_channel_capacity = 16384 +me_route_channel_capacity = 8192 +me_c2me_channel_capacity = 8192 +max_client_frame = 16777216 +"#, + ); + + let cfg = ProxyConfig::load(&path).expect("hard upper bound values must be accepted"); + assert_eq!(cfg.general.me_writer_cmd_channel_capacity, 16384); + assert_eq!(cfg.general.me_route_channel_capacity, 8192); + assert_eq!(cfg.general.me_c2me_channel_capacity, 8192); + assert_eq!(cfg.general.max_client_frame, 16 * 1024 * 1024); + + remove_temp_config(&path); +} diff --git a/src/config/types.rs b/src/config/types.rs index 41b0c2e..0a5af21 100644 --- a/src/config/types.rs +++ b/src/config/types.rs @@ -1216,6 +1216,118 @@ impl Default for ApiConfig { } } +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, Default)] +#[serde(rename_all = "lowercase")] +pub enum ConntrackMode { + #[default] + Tracked, + Notrack, + Hybrid, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, Default)] +#[serde(rename_all = "lowercase")] +pub enum ConntrackBackend { + #[default] + Auto, + Nftables, + Iptables, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, Default)] +#[serde(rename_all = "lowercase")] +pub enum ConntrackPressureProfile { + Conservative, + #[default] + Balanced, + Aggressive, +} + +impl ConntrackPressureProfile { + pub fn client_first_byte_idle_cap_secs(self) -> u64 { + match self { + Self::Conservative => 30, + Self::Balanced => 20, + Self::Aggressive => 10, + } + } + + pub fn direct_activity_timeout_secs(self) -> u64 { + match self { + Self::Conservative => 180, + Self::Balanced => 120, + Self::Aggressive => 60, + } + } + + pub fn middle_soft_idle_cap_secs(self) -> u64 { + match self { + Self::Conservative => 60, + Self::Balanced => 30, + Self::Aggressive => 20, + } + } + + pub fn middle_hard_idle_cap_secs(self) -> u64 { + match self { + Self::Conservative => 180, + Self::Balanced => 90, + Self::Aggressive => 60, + } + } +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct ConntrackControlConfig { + /// Enables runtime conntrack-control worker for pressure mitigation. + #[serde(default = "default_conntrack_control_enabled")] + pub inline_conntrack_control: bool, + + /// Conntrack mode for listener ingress traffic. + #[serde(default)] + pub mode: ConntrackMode, + + /// Netfilter backend used to reconcile notrack rules. + #[serde(default)] + pub backend: ConntrackBackend, + + /// Pressure profile for timeout caps under resource saturation. + #[serde(default)] + pub profile: ConntrackPressureProfile, + + /// Listener IP allow-list for hybrid mode. + /// Ignored in tracked/notrack mode. + #[serde(default)] + pub hybrid_listener_ips: Vec, + + /// Pressure high watermark as percentage. + #[serde(default = "default_conntrack_pressure_high_watermark_pct")] + pub pressure_high_watermark_pct: u8, + + /// Pressure low watermark as percentage. + #[serde(default = "default_conntrack_pressure_low_watermark_pct")] + pub pressure_low_watermark_pct: u8, + + /// Maximum conntrack delete operations per second. + #[serde(default = "default_conntrack_delete_budget_per_sec")] + pub delete_budget_per_sec: u64, +} + +impl Default for ConntrackControlConfig { + fn default() -> Self { + Self { + inline_conntrack_control: default_conntrack_control_enabled(), + mode: ConntrackMode::default(), + backend: ConntrackBackend::default(), + profile: ConntrackPressureProfile::default(), + hybrid_listener_ips: Vec::new(), + pressure_high_watermark_pct: default_conntrack_pressure_high_watermark_pct(), + pressure_low_watermark_pct: default_conntrack_pressure_low_watermark_pct(), + delete_budget_per_sec: default_conntrack_delete_budget_per_sec(), + } + } +} + #[derive(Debug, Clone, Serialize, Deserialize)] pub struct ServerConfig { #[serde(default = "default_port")] @@ -1291,6 +1403,10 @@ pub struct ServerConfig { /// `0` keeps legacy unbounded wait behavior. #[serde(default = "default_accept_permit_timeout_ms")] pub accept_permit_timeout_ms: u64, + + /// Runtime conntrack control and pressure policy. + #[serde(default)] + pub conntrack_control: ConntrackControlConfig, } impl Default for ServerConfig { @@ -1313,6 +1429,7 @@ impl Default for ServerConfig { listen_backlog: default_listen_backlog(), max_connections: default_server_max_connections(), accept_permit_timeout_ms: default_accept_permit_timeout_ms(), + conntrack_control: ConntrackControlConfig::default(), } } } @@ -1385,6 +1502,7 @@ pub enum UnknownSniAction { #[default] Drop, Mask, + Accept, } #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)] diff --git a/src/conntrack_control.rs b/src/conntrack_control.rs new file mode 100644 index 0000000..306697e --- /dev/null +++ b/src/conntrack_control.rs @@ -0,0 +1,755 @@ +use std::collections::BTreeSet; +use std::net::IpAddr; +use std::path::PathBuf; +use std::sync::Arc; +use std::time::Duration; + +use tokio::io::AsyncWriteExt; +use tokio::process::Command; +use tokio::sync::{mpsc, watch}; +use tracing::{debug, info, warn}; + +use crate::config::{ConntrackBackend, ConntrackMode, ProxyConfig}; +use crate::proxy::middle_relay::note_global_relay_pressure; +use crate::proxy::shared_state::{ConntrackCloseEvent, ConntrackCloseReason, ProxySharedState}; +use crate::stats::Stats; + +const CONNTRACK_EVENT_QUEUE_CAPACITY: usize = 32_768; +const PRESSURE_RELEASE_TICKS: u8 = 3; +const PRESSURE_SAMPLE_INTERVAL: Duration = Duration::from_secs(1); + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +enum NetfilterBackend { + Nftables, + Iptables, +} + +#[derive(Clone, Copy)] +struct PressureSample { + conn_pct: Option, + fd_pct: Option, + accept_timeout_delta: u64, + me_queue_pressure_delta: u64, +} + +struct PressureState { + active: bool, + low_streak: u8, + prev_accept_timeout_total: u64, + prev_me_queue_pressure_total: u64, +} + +impl PressureState { + fn new(stats: &Stats) -> Self { + Self { + active: false, + low_streak: 0, + prev_accept_timeout_total: stats.get_accept_permit_timeout_total(), + prev_me_queue_pressure_total: stats.get_me_c2me_send_full_total(), + } + } +} + +pub(crate) fn spawn_conntrack_controller( + config_rx: watch::Receiver>, + stats: Arc, + shared: Arc, +) { + if !cfg!(target_os = "linux") { + let enabled = config_rx + .borrow() + .server + .conntrack_control + .inline_conntrack_control; + stats.set_conntrack_control_enabled(enabled); + stats.set_conntrack_control_available(false); + stats.set_conntrack_pressure_active(false); + stats.set_conntrack_event_queue_depth(0); + stats.set_conntrack_rule_apply_ok(false); + shared.disable_conntrack_close_sender(); + shared.set_conntrack_pressure_active(false); + if enabled { + warn!( + "conntrack control is configured but unsupported on this OS; disabling runtime worker" + ); + } + return; + } + + let (tx, rx) = mpsc::channel(CONNTRACK_EVENT_QUEUE_CAPACITY); + shared.set_conntrack_close_sender(tx); + tokio::spawn(async move { + run_conntrack_controller(config_rx, stats, shared, rx).await; + }); +} + +async fn run_conntrack_controller( + mut config_rx: watch::Receiver>, + stats: Arc, + shared: Arc, + mut close_rx: mpsc::Receiver, +) { + let mut cfg = config_rx.borrow().clone(); + let mut pressure_state = PressureState::new(stats.as_ref()); + let mut delete_budget_tokens = cfg.server.conntrack_control.delete_budget_per_sec; + let mut backend = pick_backend(cfg.server.conntrack_control.backend); + + apply_runtime_state( + stats.as_ref(), + shared.as_ref(), + &cfg, + backend.is_some(), + false, + ); + reconcile_rules(&cfg, backend, stats.as_ref()).await; + + loop { + tokio::select! { + changed = config_rx.changed() => { + if changed.is_err() { + break; + } + cfg = config_rx.borrow_and_update().clone(); + backend = pick_backend(cfg.server.conntrack_control.backend); + delete_budget_tokens = cfg.server.conntrack_control.delete_budget_per_sec; + apply_runtime_state(stats.as_ref(), shared.as_ref(), &cfg, backend.is_some(), pressure_state.active); + reconcile_rules(&cfg, backend, stats.as_ref()).await; + } + event = close_rx.recv() => { + let Some(event) = event else { + break; + }; + stats.set_conntrack_event_queue_depth(close_rx.len() as u64); + if !cfg.server.conntrack_control.inline_conntrack_control { + continue; + } + if !pressure_state.active { + continue; + } + if !matches!(event.reason, ConntrackCloseReason::Timeout | ConntrackCloseReason::Pressure | ConntrackCloseReason::Reset) { + continue; + } + if delete_budget_tokens == 0 { + continue; + } + stats.increment_conntrack_delete_attempt_total(); + match delete_conntrack_entry(event).await { + DeleteOutcome::Deleted => { + delete_budget_tokens = delete_budget_tokens.saturating_sub(1); + stats.increment_conntrack_delete_success_total(); + } + DeleteOutcome::NotFound => { + delete_budget_tokens = delete_budget_tokens.saturating_sub(1); + stats.increment_conntrack_delete_not_found_total(); + } + DeleteOutcome::Error => { + delete_budget_tokens = delete_budget_tokens.saturating_sub(1); + stats.increment_conntrack_delete_error_total(); + } + } + } + _ = tokio::time::sleep(PRESSURE_SAMPLE_INTERVAL) => { + delete_budget_tokens = cfg.server.conntrack_control.delete_budget_per_sec; + stats.set_conntrack_event_queue_depth(close_rx.len() as u64); + let sample = collect_pressure_sample(stats.as_ref(), &cfg, &mut pressure_state); + update_pressure_state( + stats.as_ref(), + shared.as_ref(), + &cfg, + &sample, + &mut pressure_state, + ); + if pressure_state.active { + note_global_relay_pressure(shared.as_ref()); + } + } + } + } + + shared.disable_conntrack_close_sender(); + shared.set_conntrack_pressure_active(false); + stats.set_conntrack_pressure_active(false); +} + +fn apply_runtime_state( + stats: &Stats, + shared: &ProxySharedState, + cfg: &ProxyConfig, + backend_available: bool, + pressure_active: bool, +) { + let enabled = cfg.server.conntrack_control.inline_conntrack_control; + let available = enabled && backend_available && has_cap_net_admin(); + if enabled && !available { + warn!( + "conntrack control enabled but unavailable (missing CAP_NET_ADMIN or backend binaries)" + ); + } + stats.set_conntrack_control_enabled(enabled); + stats.set_conntrack_control_available(available); + shared.set_conntrack_pressure_active(enabled && pressure_active); + stats.set_conntrack_pressure_active(enabled && pressure_active); +} + +fn collect_pressure_sample( + stats: &Stats, + cfg: &ProxyConfig, + state: &mut PressureState, +) -> PressureSample { + let current_connections = stats.get_current_connections_total(); + let conn_pct = if cfg.server.max_connections == 0 { + None + } else { + Some( + ((current_connections.saturating_mul(100)) / u64::from(cfg.server.max_connections)) + .min(100) as u8, + ) + }; + + let fd_pct = fd_usage_pct(); + + let accept_total = stats.get_accept_permit_timeout_total(); + let accept_delta = accept_total.saturating_sub(state.prev_accept_timeout_total); + state.prev_accept_timeout_total = accept_total; + + let me_total = stats.get_me_c2me_send_full_total(); + let me_delta = me_total.saturating_sub(state.prev_me_queue_pressure_total); + state.prev_me_queue_pressure_total = me_total; + + PressureSample { + conn_pct, + fd_pct, + accept_timeout_delta: accept_delta, + me_queue_pressure_delta: me_delta, + } +} + +fn update_pressure_state( + stats: &Stats, + shared: &ProxySharedState, + cfg: &ProxyConfig, + sample: &PressureSample, + state: &mut PressureState, +) { + if !cfg.server.conntrack_control.inline_conntrack_control { + if state.active { + state.active = false; + state.low_streak = 0; + shared.set_conntrack_pressure_active(false); + stats.set_conntrack_pressure_active(false); + info!("Conntrack pressure mode deactivated (feature disabled)"); + } + return; + } + + let high = cfg.server.conntrack_control.pressure_high_watermark_pct; + let low = cfg.server.conntrack_control.pressure_low_watermark_pct; + + let high_hit = sample.conn_pct.is_some_and(|v| v >= high) + || sample.fd_pct.is_some_and(|v| v >= high) + || sample.accept_timeout_delta > 0 + || sample.me_queue_pressure_delta > 0; + + let low_clear = sample.conn_pct.is_none_or(|v| v <= low) + && sample.fd_pct.is_none_or(|v| v <= low) + && sample.accept_timeout_delta == 0 + && sample.me_queue_pressure_delta == 0; + + if !state.active && high_hit { + state.active = true; + state.low_streak = 0; + shared.set_conntrack_pressure_active(true); + stats.set_conntrack_pressure_active(true); + info!( + conn_pct = ?sample.conn_pct, + fd_pct = ?sample.fd_pct, + accept_timeout_delta = sample.accept_timeout_delta, + me_queue_pressure_delta = sample.me_queue_pressure_delta, + "Conntrack pressure mode activated" + ); + return; + } + + if state.active && low_clear { + state.low_streak = state.low_streak.saturating_add(1); + if state.low_streak >= PRESSURE_RELEASE_TICKS { + state.active = false; + state.low_streak = 0; + shared.set_conntrack_pressure_active(false); + stats.set_conntrack_pressure_active(false); + info!("Conntrack pressure mode deactivated"); + } + return; + } + + state.low_streak = 0; +} + +async fn reconcile_rules(cfg: &ProxyConfig, backend: Option, stats: &Stats) { + if !cfg.server.conntrack_control.inline_conntrack_control { + clear_notrack_rules_all_backends().await; + stats.set_conntrack_rule_apply_ok(true); + return; + } + + if !has_cap_net_admin() { + stats.set_conntrack_rule_apply_ok(false); + return; + } + + let Some(backend) = backend else { + stats.set_conntrack_rule_apply_ok(false); + return; + }; + + let apply_result = match backend { + NetfilterBackend::Nftables => apply_nft_rules(cfg).await, + NetfilterBackend::Iptables => apply_iptables_rules(cfg).await, + }; + + if let Err(error) = apply_result { + warn!(error = %error, "Failed to reconcile conntrack/notrack rules"); + stats.set_conntrack_rule_apply_ok(false); + } else { + stats.set_conntrack_rule_apply_ok(true); + } +} + +fn pick_backend(configured: ConntrackBackend) -> Option { + match configured { + ConntrackBackend::Auto => { + if command_exists("nft") { + Some(NetfilterBackend::Nftables) + } else if command_exists("iptables") { + Some(NetfilterBackend::Iptables) + } else { + None + } + } + ConntrackBackend::Nftables => command_exists("nft").then_some(NetfilterBackend::Nftables), + ConntrackBackend::Iptables => { + command_exists("iptables").then_some(NetfilterBackend::Iptables) + } + } +} + +fn command_exists(binary: &str) -> bool { + let Some(path_var) = std::env::var_os("PATH") else { + return false; + }; + std::env::split_paths(&path_var).any(|dir| { + let candidate: PathBuf = dir.join(binary); + candidate.exists() && candidate.is_file() + }) +} + +fn notrack_targets(cfg: &ProxyConfig) -> (Vec>, Vec>) { + let mode = cfg.server.conntrack_control.mode; + let mut v4_targets: BTreeSet> = BTreeSet::new(); + let mut v6_targets: BTreeSet> = BTreeSet::new(); + + match mode { + ConntrackMode::Tracked => {} + ConntrackMode::Notrack => { + if cfg.server.listeners.is_empty() { + if let Some(ipv4) = cfg + .server + .listen_addr_ipv4 + .as_ref() + .and_then(|s| s.parse::().ok()) + { + if ipv4.is_unspecified() { + v4_targets.insert(None); + } else { + v4_targets.insert(Some(ipv4)); + } + } + if let Some(ipv6) = cfg + .server + .listen_addr_ipv6 + .as_ref() + .and_then(|s| s.parse::().ok()) + { + if ipv6.is_unspecified() { + v6_targets.insert(None); + } else { + v6_targets.insert(Some(ipv6)); + } + } + } else { + for listener in &cfg.server.listeners { + if listener.ip.is_ipv4() { + if listener.ip.is_unspecified() { + v4_targets.insert(None); + } else { + v4_targets.insert(Some(listener.ip)); + } + } else if listener.ip.is_unspecified() { + v6_targets.insert(None); + } else { + v6_targets.insert(Some(listener.ip)); + } + } + } + } + ConntrackMode::Hybrid => { + for ip in &cfg.server.conntrack_control.hybrid_listener_ips { + if ip.is_ipv4() { + v4_targets.insert(Some(*ip)); + } else { + v6_targets.insert(Some(*ip)); + } + } + } + } + + ( + v4_targets.into_iter().collect(), + v6_targets.into_iter().collect(), + ) +} + +async fn apply_nft_rules(cfg: &ProxyConfig) -> Result<(), String> { + let _ = run_command( + "nft", + &["delete", "table", "inet", "telemt_conntrack"], + None, + ) + .await; + if matches!(cfg.server.conntrack_control.mode, ConntrackMode::Tracked) { + return Ok(()); + } + + let (v4_targets, v6_targets) = notrack_targets(cfg); + let mut rules = Vec::new(); + for ip in v4_targets { + let rule = if let Some(ip) = ip { + format!("tcp dport {} ip daddr {} notrack", cfg.server.port, ip) + } else { + format!("tcp dport {} notrack", cfg.server.port) + }; + rules.push(rule); + } + for ip in v6_targets { + let rule = if let Some(ip) = ip { + format!("tcp dport {} ip6 daddr {} notrack", cfg.server.port, ip) + } else { + format!("tcp dport {} notrack", cfg.server.port) + }; + rules.push(rule); + } + + let rule_blob = if rules.is_empty() { + String::new() + } else { + format!(" {}\n", rules.join("\n ")) + }; + let script = format!( + "table inet telemt_conntrack {{\n chain preraw {{\n type filter hook prerouting priority raw; policy accept;\n{rule_blob} }}\n}}\n" + ); + run_command("nft", &["-f", "-"], Some(script)).await +} + +async fn apply_iptables_rules(cfg: &ProxyConfig) -> Result<(), String> { + apply_iptables_rules_for_binary("iptables", cfg, true).await?; + apply_iptables_rules_for_binary("ip6tables", cfg, false).await?; + Ok(()) +} + +async fn apply_iptables_rules_for_binary( + binary: &str, + cfg: &ProxyConfig, + ipv4: bool, +) -> Result<(), String> { + if !command_exists(binary) { + return Ok(()); + } + let chain = "TELEMT_NOTRACK"; + let _ = run_command( + binary, + &["-t", "raw", "-D", "PREROUTING", "-j", chain], + None, + ) + .await; + let _ = run_command(binary, &["-t", "raw", "-F", chain], None).await; + let _ = run_command(binary, &["-t", "raw", "-X", chain], None).await; + + if matches!(cfg.server.conntrack_control.mode, ConntrackMode::Tracked) { + return Ok(()); + } + + run_command(binary, &["-t", "raw", "-N", chain], None).await?; + run_command(binary, &["-t", "raw", "-F", chain], None).await?; + if run_command( + binary, + &["-t", "raw", "-C", "PREROUTING", "-j", chain], + None, + ) + .await + .is_err() + { + run_command( + binary, + &["-t", "raw", "-I", "PREROUTING", "1", "-j", chain], + None, + ) + .await?; + } + + let (v4_targets, v6_targets) = notrack_targets(cfg); + let selected = if ipv4 { v4_targets } else { v6_targets }; + for ip in selected { + let mut args = vec![ + "-t".to_string(), + "raw".to_string(), + "-A".to_string(), + chain.to_string(), + "-p".to_string(), + "tcp".to_string(), + "--dport".to_string(), + cfg.server.port.to_string(), + ]; + if let Some(ip) = ip { + args.push("-d".to_string()); + args.push(ip.to_string()); + } + args.push("-j".to_string()); + args.push("CT".to_string()); + args.push("--notrack".to_string()); + let arg_refs: Vec<&str> = args.iter().map(String::as_str).collect(); + run_command(binary, &arg_refs, None).await?; + } + Ok(()) +} + +async fn clear_notrack_rules_all_backends() { + let _ = run_command( + "nft", + &["delete", "table", "inet", "telemt_conntrack"], + None, + ) + .await; + let _ = run_command( + "iptables", + &["-t", "raw", "-D", "PREROUTING", "-j", "TELEMT_NOTRACK"], + None, + ) + .await; + let _ = run_command("iptables", &["-t", "raw", "-F", "TELEMT_NOTRACK"], None).await; + let _ = run_command("iptables", &["-t", "raw", "-X", "TELEMT_NOTRACK"], None).await; + let _ = run_command( + "ip6tables", + &["-t", "raw", "-D", "PREROUTING", "-j", "TELEMT_NOTRACK"], + None, + ) + .await; + let _ = run_command("ip6tables", &["-t", "raw", "-F", "TELEMT_NOTRACK"], None).await; + let _ = run_command("ip6tables", &["-t", "raw", "-X", "TELEMT_NOTRACK"], None).await; +} + +enum DeleteOutcome { + Deleted, + NotFound, + Error, +} + +async fn delete_conntrack_entry(event: ConntrackCloseEvent) -> DeleteOutcome { + if !command_exists("conntrack") { + return DeleteOutcome::Error; + } + let args = vec![ + "-D".to_string(), + "-p".to_string(), + "tcp".to_string(), + "-s".to_string(), + event.src.ip().to_string(), + "--sport".to_string(), + event.src.port().to_string(), + "-d".to_string(), + event.dst.ip().to_string(), + "--dport".to_string(), + event.dst.port().to_string(), + ]; + let arg_refs: Vec<&str> = args.iter().map(String::as_str).collect(); + match run_command("conntrack", &arg_refs, None).await { + Ok(()) => DeleteOutcome::Deleted, + Err(error) => { + if error.contains("0 flow entries have been deleted") { + DeleteOutcome::NotFound + } else { + debug!(error = %error, "conntrack delete failed"); + DeleteOutcome::Error + } + } + } +} + +async fn run_command(binary: &str, args: &[&str], stdin: Option) -> Result<(), String> { + if !command_exists(binary) { + return Err(format!("{binary} is not available")); + } + let mut command = Command::new(binary); + command.args(args); + if stdin.is_some() { + command.stdin(std::process::Stdio::piped()); + } + command.stdout(std::process::Stdio::null()); + command.stderr(std::process::Stdio::piped()); + let mut child = command + .spawn() + .map_err(|e| format!("spawn {binary} failed: {e}"))?; + if let Some(blob) = stdin + && let Some(mut writer) = child.stdin.take() + { + writer + .write_all(blob.as_bytes()) + .await + .map_err(|e| format!("stdin write {binary} failed: {e}"))?; + } + let output = child + .wait_with_output() + .await + .map_err(|e| format!("wait {binary} failed: {e}"))?; + if output.status.success() { + return Ok(()); + } + let stderr = String::from_utf8_lossy(&output.stderr).trim().to_string(); + Err(if stderr.is_empty() { + format!("{binary} exited with status {}", output.status) + } else { + stderr + }) +} + +fn fd_usage_pct() -> Option { + let soft_limit = nofile_soft_limit()?; + if soft_limit == 0 { + return None; + } + let fd_count = std::fs::read_dir("/proc/self/fd").ok()?.count() as u64; + Some(((fd_count.saturating_mul(100)) / soft_limit).min(100) as u8) +} + +fn nofile_soft_limit() -> Option { + #[cfg(target_os = "linux")] + { + let mut lim = libc::rlimit { + rlim_cur: 0, + rlim_max: 0, + }; + let rc = unsafe { libc::getrlimit(libc::RLIMIT_NOFILE, &mut lim) }; + if rc != 0 { + return None; + } + return Some(lim.rlim_cur); + } + #[cfg(not(target_os = "linux"))] + { + None + } +} + +fn has_cap_net_admin() -> bool { + #[cfg(target_os = "linux")] + { + let Ok(status) = std::fs::read_to_string("/proc/self/status") else { + return false; + }; + for line in status.lines() { + if let Some(raw) = line.strip_prefix("CapEff:") { + let caps = raw.trim(); + if let Ok(bits) = u64::from_str_radix(caps, 16) { + const CAP_NET_ADMIN_BIT: u64 = 12; + return (bits & (1u64 << CAP_NET_ADMIN_BIT)) != 0; + } + } + } + false + } + #[cfg(not(target_os = "linux"))] + { + false + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::config::ProxyConfig; + + #[test] + fn pressure_activates_on_accept_timeout_spike() { + let stats = Stats::new(); + let shared = ProxySharedState::new(); + let mut cfg = ProxyConfig::default(); + cfg.server.conntrack_control.inline_conntrack_control = true; + let mut state = PressureState::new(&stats); + let sample = PressureSample { + conn_pct: Some(10), + fd_pct: Some(10), + accept_timeout_delta: 1, + me_queue_pressure_delta: 0, + }; + + update_pressure_state(&stats, shared.as_ref(), &cfg, &sample, &mut state); + + assert!(state.active); + assert!(shared.conntrack_pressure_active()); + assert!(stats.get_conntrack_pressure_active()); + } + + #[test] + fn pressure_releases_after_hysteresis_window() { + let stats = Stats::new(); + let shared = ProxySharedState::new(); + let mut cfg = ProxyConfig::default(); + cfg.server.conntrack_control.inline_conntrack_control = true; + let mut state = PressureState::new(&stats); + + let high_sample = PressureSample { + conn_pct: Some(95), + fd_pct: Some(95), + accept_timeout_delta: 0, + me_queue_pressure_delta: 0, + }; + update_pressure_state(&stats, shared.as_ref(), &cfg, &high_sample, &mut state); + assert!(state.active); + + let low_sample = PressureSample { + conn_pct: Some(10), + fd_pct: Some(10), + accept_timeout_delta: 0, + me_queue_pressure_delta: 0, + }; + update_pressure_state(&stats, shared.as_ref(), &cfg, &low_sample, &mut state); + assert!(state.active); + update_pressure_state(&stats, shared.as_ref(), &cfg, &low_sample, &mut state); + assert!(state.active); + update_pressure_state(&stats, shared.as_ref(), &cfg, &low_sample, &mut state); + + assert!(!state.active); + assert!(!shared.conntrack_pressure_active()); + assert!(!stats.get_conntrack_pressure_active()); + } + + #[test] + fn pressure_does_not_activate_when_disabled() { + let stats = Stats::new(); + let shared = ProxySharedState::new(); + let mut cfg = ProxyConfig::default(); + cfg.server.conntrack_control.inline_conntrack_control = false; + let mut state = PressureState::new(&stats); + let sample = PressureSample { + conn_pct: Some(100), + fd_pct: Some(100), + accept_timeout_delta: 10, + me_queue_pressure_delta: 10, + }; + + update_pressure_state(&stats, shared.as_ref(), &cfg, &sample, &mut state); + + assert!(!state.active); + assert!(!shared.conntrack_pressure_active()); + assert!(!stats.get_conntrack_pressure_active()); + } +} diff --git a/src/daemon/mod.rs b/src/daemon/mod.rs index 8e2481e..6f7de16 100644 --- a/src/daemon/mod.rs +++ b/src/daemon/mod.rs @@ -339,31 +339,35 @@ fn is_process_running(pid: i32) -> bool { /// Drops privileges to the specified user and group. /// -/// This should be called after binding privileged ports but before -/// entering the main event loop. -pub fn drop_privileges(user: Option<&str>, group: Option<&str>) -> Result<(), DaemonError> { - // Look up group first (need to do this while still root) +/// This should be called after binding privileged ports but before entering +/// the main event loop. +pub fn drop_privileges( + user: Option<&str>, + group: Option<&str>, + pid_file: Option<&PidFile>, +) -> Result<(), DaemonError> { let target_gid = if let Some(group_name) = group { Some(lookup_group(group_name)?) } else if let Some(user_name) = user { - // If no group specified but user is, use user's primary group Some(lookup_user_primary_gid(user_name)?) } else { None }; - // Look up user let target_uid = if let Some(user_name) = user { Some(lookup_user(user_name)?) } else { None }; - // Drop privileges: set GID first, then UID - // (Setting UID first would prevent us from setting GID) + if (target_uid.is_some() || target_gid.is_some()) + && let Some(file) = pid_file.and_then(|pid| pid.file.as_ref()) + { + unistd::fchown(file, target_uid, target_gid).map_err(DaemonError::PrivilegeDrop)?; + } + if let Some(gid) = target_gid { unistd::setgid(gid).map_err(DaemonError::PrivilegeDrop)?; - // Also set supplementary groups to just this one unistd::setgroups(&[gid]).map_err(DaemonError::PrivilegeDrop)?; info!(gid = gid.as_raw(), "Dropped group privileges"); } @@ -371,6 +375,38 @@ pub fn drop_privileges(user: Option<&str>, group: Option<&str>) -> Result<(), Da if let Some(uid) = target_uid { unistd::setuid(uid).map_err(DaemonError::PrivilegeDrop)?; info!(uid = uid.as_raw(), "Dropped user privileges"); + + if uid.as_raw() != 0 + && let Some(pid) = pid_file + { + let parent = pid.path.parent().unwrap_or(Path::new(".")); + let probe_path = parent.join(format!( + ".telemt_pid_probe_{}_{}", + std::process::id(), + getpid().as_raw() + )); + OpenOptions::new() + .write(true) + .create_new(true) + .mode(0o600) + .open(&probe_path) + .map_err(|e| { + DaemonError::PidFile(format!( + "cannot create probe in PID directory {} as uid {} (pid cleanup will fail): {}", + parent.display(), + uid.as_raw(), + e + )) + })?; + fs::remove_file(&probe_path).map_err(|e| { + DaemonError::PidFile(format!( + "cannot remove probe in PID directory {} as uid {} (pid cleanup will fail): {}", + parent.display(), + uid.as_raw(), + e + )) + })?; + } } Ok(()) diff --git a/src/ip_tracker.rs b/src/ip_tracker.rs index 76ea424..b4d934f 100644 --- a/src/ip_tracker.rs +++ b/src/ip_tracker.rs @@ -26,6 +26,15 @@ pub struct UserIpTracker { cleanup_drain_lock: Arc>, } +#[derive(Debug, Clone, Copy)] +pub struct UserIpTrackerMemoryStats { + pub active_users: usize, + pub recent_users: usize, + pub active_entries: usize, + pub recent_entries: usize, + pub cleanup_queue_len: usize, +} + impl UserIpTracker { pub fn new() -> Self { Self { @@ -141,6 +150,13 @@ impl UserIpTracker { let mut active_ips = self.active_ips.write().await; let mut recent_ips = self.recent_ips.write().await; + let window = *self.limit_window.read().await; + let now = Instant::now(); + + for user_recent in recent_ips.values_mut() { + Self::prune_recent(user_recent, now, window); + } + let mut users = Vec::::with_capacity(active_ips.len().saturating_add(recent_ips.len())); users.extend(active_ips.keys().cloned()); @@ -166,6 +182,26 @@ impl UserIpTracker { } } + pub async fn memory_stats(&self) -> UserIpTrackerMemoryStats { + let cleanup_queue_len = self + .cleanup_queue + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()) + .len(); + let active_ips = self.active_ips.read().await; + let recent_ips = self.recent_ips.read().await; + let active_entries = active_ips.values().map(HashMap::len).sum(); + let recent_entries = recent_ips.values().map(HashMap::len).sum(); + + UserIpTrackerMemoryStats { + active_users: active_ips.len(), + recent_users: recent_ips.len(), + active_entries, + recent_entries, + cleanup_queue_len, + } + } + pub async fn set_limit_policy(&self, mode: UserMaxUniqueIpsMode, window_secs: u64) { { let mut current_mode = self.limit_mode.write().await; @@ -451,6 +487,7 @@ impl Default for UserIpTracker { mod tests { use super::*; use std::net::{IpAddr, Ipv4Addr, Ipv6Addr}; + use std::sync::atomic::Ordering; fn test_ipv4(oct1: u8, oct2: u8, oct3: u8, oct4: u8) -> IpAddr { IpAddr::V4(Ipv4Addr::new(oct1, oct2, oct3, oct4)) @@ -764,4 +801,54 @@ mod tests { tokio::time::sleep(Duration::from_millis(1100)).await; assert!(tracker.check_and_add("test_user", ip2).await.is_ok()); } + + #[tokio::test] + async fn test_memory_stats_reports_queue_and_entry_counts() { + let tracker = UserIpTracker::new(); + tracker.set_user_limit("test_user", 4).await; + let ip1 = test_ipv4(10, 2, 0, 1); + let ip2 = test_ipv4(10, 2, 0, 2); + + tracker.check_and_add("test_user", ip1).await.unwrap(); + tracker.check_and_add("test_user", ip2).await.unwrap(); + tracker.enqueue_cleanup("test_user".to_string(), ip1); + + let snapshot = tracker.memory_stats().await; + assert_eq!(snapshot.active_users, 1); + assert_eq!(snapshot.recent_users, 1); + assert_eq!(snapshot.active_entries, 2); + assert_eq!(snapshot.recent_entries, 2); + assert_eq!(snapshot.cleanup_queue_len, 1); + } + + #[tokio::test] + async fn test_compact_prunes_stale_recent_entries() { + let tracker = UserIpTracker::new(); + tracker + .set_limit_policy(UserMaxUniqueIpsMode::TimeWindow, 1) + .await; + + let stale_user = "stale-user".to_string(); + let stale_ip = test_ipv4(10, 3, 0, 1); + { + let mut recent_ips = tracker.recent_ips.write().await; + recent_ips + .entry(stale_user.clone()) + .or_insert_with(HashMap::new) + .insert(stale_ip, Instant::now() - Duration::from_secs(5)); + } + + tracker.last_compact_epoch_secs.store(0, Ordering::Relaxed); + tracker + .check_and_add("trigger-user", test_ipv4(10, 3, 0, 2)) + .await + .unwrap(); + + let recent_ips = tracker.recent_ips.read().await; + let stale_exists = recent_ips + .get(&stale_user) + .map(|ips| ips.contains_key(&stale_ip)) + .unwrap_or(false); + assert!(!stale_exists); + } } diff --git a/src/logging.rs b/src/logging.rs index bb381ef..af9e2f7 100644 --- a/src/logging.rs +++ b/src/logging.rs @@ -88,8 +88,10 @@ pub fn init_logging( // Use a custom fmt layer that writes to syslog let fmt_layer = fmt::Layer::default() .with_ansi(false) - .with_target(true) - .with_writer(SyslogWriter::new); + .with_target(false) + .with_level(false) + .without_time() + .with_writer(SyslogMakeWriter::new()); tracing_subscriber::registry() .with(filter_layer) @@ -137,12 +139,17 @@ pub fn init_logging( /// Syslog writer for tracing. #[cfg(unix)] +#[derive(Clone, Copy)] +struct SyslogMakeWriter; + +#[cfg(unix)] +#[derive(Clone, Copy)] struct SyslogWriter { - _private: (), + priority: libc::c_int, } #[cfg(unix)] -impl SyslogWriter { +impl SyslogMakeWriter { fn new() -> Self { // Open syslog connection on first use static INIT: std::sync::Once = std::sync::Once::new(); @@ -153,7 +160,18 @@ impl SyslogWriter { libc::openlog(ident, libc::LOG_PID | libc::LOG_NDELAY, libc::LOG_DAEMON); } }); - Self { _private: () } + Self + } +} + +#[cfg(unix)] +fn syslog_priority_for_level(level: &tracing::Level) -> libc::c_int { + match *level { + tracing::Level::ERROR => libc::LOG_ERR, + tracing::Level::WARN => libc::LOG_WARNING, + tracing::Level::INFO => libc::LOG_INFO, + tracing::Level::DEBUG => libc::LOG_DEBUG, + tracing::Level::TRACE => libc::LOG_DEBUG, } } @@ -168,26 +186,13 @@ impl std::io::Write for SyslogWriter { return Ok(buf.len()); } - // Determine priority based on log level in the message - let priority = if msg.contains(" ERROR ") || msg.contains(" error ") { - libc::LOG_ERR - } else if msg.contains(" WARN ") || msg.contains(" warn ") { - libc::LOG_WARNING - } else if msg.contains(" INFO ") || msg.contains(" info ") { - libc::LOG_INFO - } else if msg.contains(" DEBUG ") || msg.contains(" debug ") { - libc::LOG_DEBUG - } else { - libc::LOG_INFO - }; - // Write to syslog let c_msg = std::ffi::CString::new(msg.as_bytes()) .unwrap_or_else(|_| std::ffi::CString::new("(invalid utf8)").unwrap()); unsafe { libc::syslog( - priority, + self.priority, b"%s\0".as_ptr() as *const libc::c_char, c_msg.as_ptr(), ); @@ -202,11 +207,19 @@ impl std::io::Write for SyslogWriter { } #[cfg(unix)] -impl<'a> tracing_subscriber::fmt::MakeWriter<'a> for SyslogWriter { +impl<'a> tracing_subscriber::fmt::MakeWriter<'a> for SyslogMakeWriter { type Writer = SyslogWriter; fn make_writer(&'a self) -> Self::Writer { - SyslogWriter::new() + SyslogWriter { + priority: libc::LOG_INFO, + } + } + + fn make_writer_for(&'a self, meta: &tracing::Metadata<'_>) -> Self::Writer { + SyslogWriter { + priority: syslog_priority_for_level(meta.level()), + } } } @@ -302,4 +315,29 @@ mod tests { LogDestination::Syslog )); } + + #[cfg(unix)] + #[test] + fn test_syslog_priority_for_level_mapping() { + assert_eq!( + syslog_priority_for_level(&tracing::Level::ERROR), + libc::LOG_ERR + ); + assert_eq!( + syslog_priority_for_level(&tracing::Level::WARN), + libc::LOG_WARNING + ); + assert_eq!( + syslog_priority_for_level(&tracing::Level::INFO), + libc::LOG_INFO + ); + assert_eq!( + syslog_priority_for_level(&tracing::Level::DEBUG), + libc::LOG_DEBUG + ); + assert_eq!( + syslog_priority_for_level(&tracing::Level::TRACE), + libc::LOG_DEBUG + ); + } } diff --git a/src/maestro/helpers.rs b/src/maestro/helpers.rs index d9d8e8b..49c5347 100644 --- a/src/maestro/helpers.rs +++ b/src/maestro/helpers.rs @@ -18,19 +18,38 @@ use crate::transport::middle_proxy::{ pub(crate) fn resolve_runtime_config_path( config_path_cli: &str, startup_cwd: &std::path::Path, + config_path_explicit: bool, ) -> PathBuf { - let raw = PathBuf::from(config_path_cli); - let absolute = if raw.is_absolute() { - raw - } else { - startup_cwd.join(raw) - }; - absolute.canonicalize().unwrap_or(absolute) + if config_path_explicit { + let raw = PathBuf::from(config_path_cli); + let absolute = if raw.is_absolute() { + raw + } else { + startup_cwd.join(raw) + }; + return absolute.canonicalize().unwrap_or(absolute); + } + + let etc_telemt = std::path::Path::new("/etc/telemt"); + let candidates = [ + startup_cwd.join("config.toml"), + startup_cwd.join("telemt.toml"), + etc_telemt.join("telemt.toml"), + etc_telemt.join("config.toml"), + ]; + for candidate in candidates { + if candidate.is_file() { + return candidate.canonicalize().unwrap_or(candidate); + } + } + + startup_cwd.join("config.toml") } /// Parsed CLI arguments. pub(crate) struct CliArgs { pub config_path: String, + pub config_path_explicit: bool, pub data_path: Option, pub silent: bool, pub log_level: Option, @@ -39,6 +58,7 @@ pub(crate) struct CliArgs { pub(crate) fn parse_cli() -> CliArgs { let mut config_path = "config.toml".to_string(); + let mut config_path_explicit = false; let mut data_path: Option = None; let mut silent = false; let mut log_level: Option = None; @@ -74,6 +94,20 @@ pub(crate) fn parse_cli() -> CliArgs { s.trim_start_matches("--data-path=").to_string(), )); } + "--working-dir" => { + i += 1; + if i < args.len() { + data_path = Some(PathBuf::from(args[i].clone())); + } else { + eprintln!("Missing value for --working-dir"); + std::process::exit(0); + } + } + s if s.starts_with("--working-dir=") => { + data_path = Some(PathBuf::from( + s.trim_start_matches("--working-dir=").to_string(), + )); + } "--silent" | "-s" => { silent = true; } @@ -111,13 +145,11 @@ pub(crate) fn parse_cli() -> CliArgs { i += 1; } } - s if s.starts_with("--working-dir") => { - if !s.contains('=') { - i += 1; - } - } s if !s.starts_with('-') => { - config_path = s.to_string(); + if !matches!(s, "run" | "start" | "stop" | "reload" | "status") { + config_path = s.to_string(); + config_path_explicit = true; + } } other => { eprintln!("Unknown option: {}", other); @@ -128,6 +160,7 @@ pub(crate) fn parse_cli() -> CliArgs { CliArgs { config_path, + config_path_explicit, data_path, silent, log_level, @@ -152,6 +185,7 @@ fn print_help() { eprintln!( " --data-path Set data directory (absolute path; overrides config value)" ); + eprintln!(" --working-dir Alias for --data-path"); eprintln!(" --silent, -s Suppress info logs"); eprintln!(" --log-level debug|verbose|normal|silent"); eprintln!(" --help, -h Show this help"); @@ -210,7 +244,7 @@ mod tests { let target = startup_cwd.join("config.toml"); std::fs::write(&target, " ").unwrap(); - let resolved = resolve_runtime_config_path("config.toml", &startup_cwd); + let resolved = resolve_runtime_config_path("config.toml", &startup_cwd, true); assert_eq!(resolved, target.canonicalize().unwrap()); let _ = std::fs::remove_file(&target); @@ -226,11 +260,45 @@ mod tests { let startup_cwd = std::env::temp_dir().join(format!("telemt_cfg_path_missing_{nonce}")); std::fs::create_dir_all(&startup_cwd).unwrap(); - let resolved = resolve_runtime_config_path("missing.toml", &startup_cwd); + let resolved = resolve_runtime_config_path("missing.toml", &startup_cwd, true); assert_eq!(resolved, startup_cwd.join("missing.toml")); let _ = std::fs::remove_dir(&startup_cwd); } + + #[test] + fn resolve_runtime_config_path_uses_startup_candidates_when_not_explicit() { + let nonce = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .unwrap() + .as_nanos(); + let startup_cwd = + std::env::temp_dir().join(format!("telemt_cfg_startup_candidates_{nonce}")); + std::fs::create_dir_all(&startup_cwd).unwrap(); + let telemt = startup_cwd.join("telemt.toml"); + std::fs::write(&telemt, " ").unwrap(); + + let resolved = resolve_runtime_config_path("config.toml", &startup_cwd, false); + assert_eq!(resolved, telemt.canonicalize().unwrap()); + + let _ = std::fs::remove_file(&telemt); + let _ = std::fs::remove_dir(&startup_cwd); + } + + #[test] + fn resolve_runtime_config_path_defaults_to_startup_config_when_none_found() { + let nonce = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .unwrap() + .as_nanos(); + let startup_cwd = std::env::temp_dir().join(format!("telemt_cfg_startup_default_{nonce}")); + std::fs::create_dir_all(&startup_cwd).unwrap(); + + let resolved = resolve_runtime_config_path("config.toml", &startup_cwd, false); + assert_eq!(resolved, startup_cwd.join("config.toml")); + + let _ = std::fs::remove_dir(&startup_cwd); + } } pub(crate) fn print_proxy_links(host: &str, port: u16, config: &ProxyConfig) { diff --git a/src/maestro/listeners.rs b/src/maestro/listeners.rs index 3b2a92f..96d4cd9 100644 --- a/src/maestro/listeners.rs +++ b/src/maestro/listeners.rs @@ -14,6 +14,7 @@ use crate::crypto::SecureRandom; use crate::ip_tracker::UserIpTracker; use crate::proxy::ClientHandler; use crate::proxy::route_mode::{ROUTE_SWITCH_ERROR_MSG, RouteRuntimeController}; +use crate::proxy::shared_state::ProxySharedState; use crate::startup::{COMPONENT_LISTENERS_BIND, StartupTracker}; use crate::stats::beobachten::BeobachtenStore; use crate::stats::{ReplayChecker, Stats}; @@ -49,6 +50,7 @@ pub(crate) async fn bind_listeners( tls_cache: Option>, ip_tracker: Arc, beobachten: Arc, + shared: Arc, max_connections: Arc, ) -> Result> { startup_tracker @@ -224,6 +226,7 @@ pub(crate) async fn bind_listeners( let tls_cache = tls_cache.clone(); let ip_tracker = ip_tracker.clone(); let beobachten = beobachten.clone(); + let shared = shared.clone(); let max_connections_unix = max_connections.clone(); tokio::spawn(async move { @@ -259,6 +262,7 @@ pub(crate) async fn bind_listeners( break; } Err(_) => { + stats.increment_accept_permit_timeout_total(); debug!( timeout_ms = accept_permit_timeout_ms, "Dropping accepted unix connection: permit wait timeout" @@ -284,11 +288,12 @@ pub(crate) async fn bind_listeners( let tls_cache = tls_cache.clone(); let ip_tracker = ip_tracker.clone(); let beobachten = beobachten.clone(); + let shared = shared.clone(); let proxy_protocol_enabled = config.server.proxy_protocol; tokio::spawn(async move { let _permit = permit; - if let Err(e) = crate::proxy::client::handle_client_stream( + if let Err(e) = crate::proxy::client::handle_client_stream_with_shared( stream, fake_peer, config, @@ -302,6 +307,7 @@ pub(crate) async fn bind_listeners( tls_cache, ip_tracker, beobachten, + shared, proxy_protocol_enabled, ) .await @@ -351,6 +357,7 @@ pub(crate) fn spawn_tcp_accept_loops( tls_cache: Option>, ip_tracker: Arc, beobachten: Arc, + shared: Arc, max_connections: Arc, ) { for (listener, listener_proxy_protocol) in listeners { @@ -366,6 +373,7 @@ pub(crate) fn spawn_tcp_accept_loops( let tls_cache = tls_cache.clone(); let ip_tracker = ip_tracker.clone(); let beobachten = beobachten.clone(); + let shared = shared.clone(); let max_connections_tcp = max_connections.clone(); tokio::spawn(async move { @@ -400,6 +408,7 @@ pub(crate) fn spawn_tcp_accept_loops( break; } Err(_) => { + stats.increment_accept_permit_timeout_total(); debug!( peer = %peer_addr, timeout_ms = accept_permit_timeout_ms, @@ -421,13 +430,14 @@ pub(crate) fn spawn_tcp_accept_loops( let tls_cache = tls_cache.clone(); let ip_tracker = ip_tracker.clone(); let beobachten = beobachten.clone(); + let shared = shared.clone(); let proxy_protocol_enabled = listener_proxy_protocol; let real_peer_report = Arc::new(std::sync::Mutex::new(None)); let real_peer_report_for_handler = real_peer_report.clone(); tokio::spawn(async move { let _permit = permit; - if let Err(e) = ClientHandler::new( + if let Err(e) = ClientHandler::new_with_shared( stream, peer_addr, config, @@ -441,6 +451,7 @@ pub(crate) fn spawn_tcp_accept_loops( tls_cache, ip_tracker, beobachten, + shared, proxy_protocol_enabled, real_peer_report_for_handler, ) diff --git a/src/maestro/mod.rs b/src/maestro/mod.rs index aa95cb6..00b3b2d 100644 --- a/src/maestro/mod.rs +++ b/src/maestro/mod.rs @@ -29,10 +29,12 @@ use tracing_subscriber::{EnvFilter, fmt, prelude::*, reload}; use crate::api; use crate::config::{LogLevel, ProxyConfig}; +use crate::conntrack_control; use crate::crypto::SecureRandom; use crate::ip_tracker::UserIpTracker; use crate::network::probe::{decide_network_capabilities, log_probe_result, run_probe}; use crate::proxy::route_mode::{RelayRouteMode, RouteRuntimeController}; +use crate::proxy::shared_state::ProxySharedState; use crate::startup::{ COMPONENT_API_BOOTSTRAP, COMPONENT_CONFIG_LOAD, COMPONENT_ME_POOL_CONSTRUCT, COMPONENT_ME_POOL_INIT_STAGE1, COMPONENT_ME_PROXY_CONFIG_V4, COMPONENT_ME_PROXY_CONFIG_V6, @@ -110,6 +112,7 @@ async fn run_inner( .await; let cli_args = parse_cli(); let config_path_cli = cli_args.config_path; + let config_path_explicit = cli_args.config_path_explicit; let data_path = cli_args.data_path; let cli_silent = cli_args.silent; let cli_log_level = cli_args.log_level; @@ -121,7 +124,8 @@ async fn run_inner( std::process::exit(1); } }; - let config_path = resolve_runtime_config_path(&config_path_cli, &startup_cwd); + let mut config_path = + resolve_runtime_config_path(&config_path_cli, &startup_cwd, config_path_explicit); let mut config = match ProxyConfig::load(&config_path) { Ok(c) => c, @@ -131,11 +135,99 @@ async fn run_inner( std::process::exit(1); } else { let default = ProxyConfig::default(); - std::fs::write(&config_path, toml::to_string_pretty(&default).unwrap()).unwrap(); - eprintln!( - "[telemt] Created default config at {}", - config_path.display() - ); + + let serialized = + match toml::to_string_pretty(&default).or_else(|_| toml::to_string(&default)) { + Ok(value) => Some(value), + Err(serialize_error) => { + eprintln!( + "[telemt] Warning: failed to serialize default config: {}", + serialize_error + ); + None + } + }; + + if config_path_explicit { + if let Some(serialized) = serialized.as_ref() { + if let Err(write_error) = std::fs::write(&config_path, serialized) { + eprintln!( + "[telemt] Error: failed to create explicit config at {}: {}", + config_path.display(), + write_error + ); + std::process::exit(1); + } + eprintln!( + "[telemt] Created default config at {}", + config_path.display() + ); + } else { + eprintln!( + "[telemt] Warning: running with in-memory default config without writing to disk" + ); + } + } else { + let system_dir = std::path::Path::new("/etc/telemt"); + let system_config_path = system_dir.join("telemt.toml"); + let startup_config_path = startup_cwd.join("config.toml"); + let mut persisted = false; + + if let Some(serialized) = serialized.as_ref() { + match std::fs::create_dir_all(system_dir) { + Ok(()) => match std::fs::write(&system_config_path, serialized) { + Ok(()) => { + config_path = system_config_path; + eprintln!( + "[telemt] Created default config at {}", + config_path.display() + ); + persisted = true; + } + Err(write_error) => { + eprintln!( + "[telemt] Warning: failed to write default config at {}: {}", + system_config_path.display(), + write_error + ); + } + }, + Err(create_error) => { + eprintln!( + "[telemt] Warning: failed to create {}: {}", + system_dir.display(), + create_error + ); + } + } + + if !persisted { + match std::fs::write(&startup_config_path, serialized) { + Ok(()) => { + config_path = startup_config_path; + eprintln!( + "[telemt] Created default config at {}", + config_path.display() + ); + persisted = true; + } + Err(write_error) => { + eprintln!( + "[telemt] Warning: failed to write default config at {}: {}", + startup_config_path.display(), + write_error + ); + } + } + } + } + + if !persisted { + eprintln!( + "[telemt] Warning: running with in-memory default config without writing to disk" + ); + } + } default } } @@ -631,6 +723,12 @@ async fn run_inner( ) .await; let _admission_tx_hold = admission_tx; + let shared_state = ProxySharedState::new(); + conntrack_control::spawn_conntrack_controller( + config_rx.clone(), + stats.clone(), + shared_state.clone(), + ); let bound = listeners::bind_listeners( &config, @@ -651,6 +749,7 @@ async fn run_inner( tls_cache.clone(), ip_tracker.clone(), beobachten.clone(), + shared_state.clone(), max_connections.clone(), ) .await?; @@ -664,7 +763,11 @@ async fn run_inner( // Drop privileges after binding sockets (which may require root for port < 1024) if daemon_opts.user.is_some() || daemon_opts.group.is_some() { - if let Err(e) = drop_privileges(daemon_opts.user.as_deref(), daemon_opts.group.as_deref()) { + if let Err(e) = drop_privileges( + daemon_opts.user.as_deref(), + daemon_opts.group.as_deref(), + _pid_file.as_ref(), + ) { error!(error = %e, "Failed to drop privileges"); std::process::exit(1); } @@ -683,6 +786,7 @@ async fn run_inner( &startup_tracker, stats.clone(), beobachten.clone(), + shared_state.clone(), ip_tracker.clone(), config_rx.clone(), ) @@ -707,6 +811,7 @@ async fn run_inner( tls_cache.clone(), ip_tracker.clone(), beobachten.clone(), + shared_state, max_connections.clone(), ); diff --git a/src/maestro/runtime_tasks.rs b/src/maestro/runtime_tasks.rs index b8b10da..5b3f2e0 100644 --- a/src/maestro/runtime_tasks.rs +++ b/src/maestro/runtime_tasks.rs @@ -13,6 +13,7 @@ use crate::crypto::SecureRandom; use crate::ip_tracker::UserIpTracker; use crate::metrics; use crate::network::probe::NetworkProbe; +use crate::proxy::shared_state::ProxySharedState; use crate::startup::{ COMPONENT_CONFIG_WATCHER_START, COMPONENT_METRICS_START, COMPONENT_RUNTIME_READY, StartupTracker, @@ -287,6 +288,7 @@ pub(crate) async fn spawn_metrics_if_configured( startup_tracker: &Arc, stats: Arc, beobachten: Arc, + shared_state: Arc, ip_tracker: Arc, config_rx: watch::Receiver>, ) { @@ -320,6 +322,7 @@ pub(crate) async fn spawn_metrics_if_configured( .await; let stats = stats.clone(); let beobachten = beobachten.clone(); + let shared_state = shared_state.clone(); let config_rx_metrics = config_rx.clone(); let ip_tracker_metrics = ip_tracker.clone(); let whitelist = config.server.metrics_whitelist.clone(); @@ -331,6 +334,7 @@ pub(crate) async fn spawn_metrics_if_configured( listen_backlog, stats, beobachten, + shared_state, ip_tracker_metrics, config_rx_metrics, whitelist, diff --git a/src/main.rs b/src/main.rs index 68c89fc..5c134b8 100644 --- a/src/main.rs +++ b/src/main.rs @@ -3,6 +3,7 @@ mod api; mod cli; mod config; +mod conntrack_control; mod crypto; #[cfg(unix)] mod daemon; diff --git a/src/metrics.rs b/src/metrics.rs index 56b6558..685d2ef 100644 --- a/src/metrics.rs +++ b/src/metrics.rs @@ -15,6 +15,7 @@ use tracing::{debug, info, warn}; use crate::config::ProxyConfig; use crate::ip_tracker::UserIpTracker; +use crate::proxy::shared_state::ProxySharedState; use crate::stats::Stats; use crate::stats::beobachten::BeobachtenStore; use crate::transport::{ListenOptions, create_listener}; @@ -25,6 +26,7 @@ pub async fn serve( listen_backlog: u32, stats: Arc, beobachten: Arc, + shared_state: Arc, ip_tracker: Arc, config_rx: tokio::sync::watch::Receiver>, whitelist: Vec, @@ -45,7 +47,13 @@ pub async fn serve( Ok(listener) => { info!("Metrics endpoint: http://{}/metrics and /beobachten", addr); serve_listener( - listener, stats, beobachten, ip_tracker, config_rx, whitelist, + listener, + stats, + beobachten, + shared_state, + ip_tracker, + config_rx, + whitelist, ) .await; } @@ -94,13 +102,20 @@ pub async fn serve( } (Some(listener), None) | (None, Some(listener)) => { serve_listener( - listener, stats, beobachten, ip_tracker, config_rx, whitelist, + listener, + stats, + beobachten, + shared_state, + ip_tracker, + config_rx, + whitelist, ) .await; } (Some(listener4), Some(listener6)) => { let stats_v6 = stats.clone(); let beobachten_v6 = beobachten.clone(); + let shared_state_v6 = shared_state.clone(); let ip_tracker_v6 = ip_tracker.clone(); let config_rx_v6 = config_rx.clone(); let whitelist_v6 = whitelist.clone(); @@ -109,6 +124,7 @@ pub async fn serve( listener6, stats_v6, beobachten_v6, + shared_state_v6, ip_tracker_v6, config_rx_v6, whitelist_v6, @@ -116,7 +132,13 @@ pub async fn serve( .await; }); serve_listener( - listener4, stats, beobachten, ip_tracker, config_rx, whitelist, + listener4, + stats, + beobachten, + shared_state, + ip_tracker, + config_rx, + whitelist, ) .await; } @@ -142,6 +164,7 @@ async fn serve_listener( listener: TcpListener, stats: Arc, beobachten: Arc, + shared_state: Arc, ip_tracker: Arc, config_rx: tokio::sync::watch::Receiver>, whitelist: Arc>, @@ -162,15 +185,19 @@ async fn serve_listener( let stats = stats.clone(); let beobachten = beobachten.clone(); + let shared_state = shared_state.clone(); let ip_tracker = ip_tracker.clone(); let config_rx_conn = config_rx.clone(); tokio::spawn(async move { let svc = service_fn(move |req| { let stats = stats.clone(); let beobachten = beobachten.clone(); + let shared_state = shared_state.clone(); let ip_tracker = ip_tracker.clone(); let config = config_rx_conn.borrow().clone(); - async move { handle(req, &stats, &beobachten, &ip_tracker, &config).await } + async move { + handle(req, &stats, &beobachten, &shared_state, &ip_tracker, &config).await + } }); if let Err(e) = http1::Builder::new() .serve_connection(hyper_util::rt::TokioIo::new(stream), svc) @@ -186,11 +213,12 @@ async fn handle( req: Request, stats: &Stats, beobachten: &BeobachtenStore, + shared_state: &ProxySharedState, ip_tracker: &UserIpTracker, config: &ProxyConfig, ) -> Result>, Infallible> { if req.uri().path() == "/metrics" { - let body = render_metrics(stats, config, ip_tracker).await; + let body = render_metrics(stats, shared_state, config, ip_tracker).await; let resp = Response::builder() .status(StatusCode::OK) .header("content-type", "text/plain; version=0.0.4; charset=utf-8") @@ -225,7 +253,12 @@ fn render_beobachten(beobachten: &BeobachtenStore, config: &ProxyConfig) -> Stri beobachten.snapshot_text(ttl) } -async fn render_metrics(stats: &Stats, config: &ProxyConfig, ip_tracker: &UserIpTracker) -> String { +async fn render_metrics( + stats: &Stats, + shared_state: &ProxySharedState, + config: &ProxyConfig, + ip_tracker: &UserIpTracker, +) -> String { use std::fmt::Write; let mut out = String::with_capacity(4096); let telemetry = stats.telemetry_policy(); @@ -304,6 +337,27 @@ async fn render_metrics(stats: &Stats, config: &ProxyConfig, ip_tracker: &UserIp } ); + let _ = writeln!( + out, + "# HELP telemt_buffer_pool_buffers_total Snapshot of pooled and allocated buffers" + ); + let _ = writeln!(out, "# TYPE telemt_buffer_pool_buffers_total gauge"); + let _ = writeln!( + out, + "telemt_buffer_pool_buffers_total{{kind=\"pooled\"}} {}", + stats.get_buffer_pool_pooled_gauge() + ); + let _ = writeln!( + out, + "telemt_buffer_pool_buffers_total{{kind=\"allocated\"}} {}", + stats.get_buffer_pool_allocated_gauge() + ); + let _ = writeln!( + out, + "telemt_buffer_pool_buffers_total{{kind=\"in_use\"}} {}", + stats.get_buffer_pool_in_use_gauge() + ); + let _ = writeln!( out, "# HELP telemt_connections_total Total accepted connections" @@ -349,6 +403,170 @@ async fn render_metrics(stats: &Stats, config: &ProxyConfig, ip_tracker: &UserIp } ); + let _ = writeln!( + out, + "# HELP telemt_auth_expensive_checks_total Expensive authentication candidate checks executed during handshake validation" + ); + let _ = writeln!(out, "# TYPE telemt_auth_expensive_checks_total counter"); + let _ = writeln!( + out, + "telemt_auth_expensive_checks_total {}", + if core_enabled { + shared_state + .handshake + .auth_expensive_checks_total + .load(std::sync::atomic::Ordering::Relaxed) + } else { + 0 + } + ); + + let _ = writeln!( + out, + "# HELP telemt_auth_budget_exhausted_total Handshake validations that hit authentication candidate budget limits" + ); + let _ = writeln!(out, "# TYPE telemt_auth_budget_exhausted_total counter"); + let _ = writeln!( + out, + "telemt_auth_budget_exhausted_total {}", + if core_enabled { + shared_state + .handshake + .auth_budget_exhausted_total + .load(std::sync::atomic::Ordering::Relaxed) + } else { + 0 + } + ); + + let _ = writeln!( + out, + "# HELP telemt_accept_permit_timeout_total Accepted connections dropped due to permit wait timeout" + ); + let _ = writeln!(out, "# TYPE telemt_accept_permit_timeout_total counter"); + let _ = writeln!( + out, + "telemt_accept_permit_timeout_total {}", + if core_enabled { + stats.get_accept_permit_timeout_total() + } else { + 0 + } + ); + + let _ = writeln!( + out, + "# HELP telemt_conntrack_control_state Runtime conntrack control state flags" + ); + let _ = writeln!(out, "# TYPE telemt_conntrack_control_state gauge"); + let _ = writeln!( + out, + "telemt_conntrack_control_state{{flag=\"enabled\"}} {}", + if stats.get_conntrack_control_enabled() { + 1 + } else { + 0 + } + ); + let _ = writeln!( + out, + "telemt_conntrack_control_state{{flag=\"available\"}} {}", + if stats.get_conntrack_control_available() { + 1 + } else { + 0 + } + ); + let _ = writeln!( + out, + "telemt_conntrack_control_state{{flag=\"pressure_active\"}} {}", + if stats.get_conntrack_pressure_active() { + 1 + } else { + 0 + } + ); + let _ = writeln!( + out, + "telemt_conntrack_control_state{{flag=\"rule_apply_ok\"}} {}", + if stats.get_conntrack_rule_apply_ok() { + 1 + } else { + 0 + } + ); + + let _ = writeln!( + out, + "# HELP telemt_conntrack_event_queue_depth Pending close events in conntrack control queue" + ); + let _ = writeln!(out, "# TYPE telemt_conntrack_event_queue_depth gauge"); + let _ = writeln!( + out, + "telemt_conntrack_event_queue_depth {}", + stats.get_conntrack_event_queue_depth() + ); + + let _ = writeln!( + out, + "# HELP telemt_conntrack_delete_total Conntrack delete attempts by outcome" + ); + let _ = writeln!(out, "# TYPE telemt_conntrack_delete_total counter"); + let _ = writeln!( + out, + "telemt_conntrack_delete_total{{result=\"attempt\"}} {}", + if core_enabled { + stats.get_conntrack_delete_attempt_total() + } else { + 0 + } + ); + let _ = writeln!( + out, + "telemt_conntrack_delete_total{{result=\"success\"}} {}", + if core_enabled { + stats.get_conntrack_delete_success_total() + } else { + 0 + } + ); + let _ = writeln!( + out, + "telemt_conntrack_delete_total{{result=\"not_found\"}} {}", + if core_enabled { + stats.get_conntrack_delete_not_found_total() + } else { + 0 + } + ); + let _ = writeln!( + out, + "telemt_conntrack_delete_total{{result=\"error\"}} {}", + if core_enabled { + stats.get_conntrack_delete_error_total() + } else { + 0 + } + ); + + let _ = writeln!( + out, + "# HELP telemt_conntrack_close_event_drop_total Dropped conntrack close events due to queue pressure or unavailable sender" + ); + let _ = writeln!( + out, + "# TYPE telemt_conntrack_close_event_drop_total counter" + ); + let _ = writeln!( + out, + "telemt_conntrack_close_event_drop_total {}", + if core_enabled { + stats.get_conntrack_close_event_drop_total() + } else { + 0 + } + ); + let _ = writeln!( out, "# HELP telemt_upstream_connect_attempt_total Upstream connect attempts across all requests" @@ -952,6 +1170,39 @@ async fn render_metrics(stats: &Stats, config: &ProxyConfig, ip_tracker: &UserIp } ); + let _ = writeln!( + out, + "# HELP telemt_me_c2me_enqueue_events_total ME client->ME enqueue outcomes" + ); + let _ = writeln!(out, "# TYPE telemt_me_c2me_enqueue_events_total counter"); + let _ = writeln!( + out, + "telemt_me_c2me_enqueue_events_total{{event=\"full\"}} {}", + if me_allows_normal { + stats.get_me_c2me_send_full_total() + } else { + 0 + } + ); + let _ = writeln!( + out, + "telemt_me_c2me_enqueue_events_total{{event=\"high_water\"}} {}", + if me_allows_normal { + stats.get_me_c2me_send_high_water_total() + } else { + 0 + } + ); + let _ = writeln!( + out, + "telemt_me_c2me_enqueue_events_total{{event=\"timeout\"}} {}", + if me_allows_normal { + stats.get_me_c2me_send_timeout_total() + } else { + 0 + } + ); + let _ = writeln!( out, "# HELP telemt_me_d2c_batches_total Total DC->Client flush batches" @@ -2501,6 +2752,48 @@ async fn render_metrics(stats: &Stats, config: &ProxyConfig, ip_tracker: &UserIp if user_enabled { 0 } else { 1 } ); + let ip_memory = ip_tracker.memory_stats().await; + let _ = writeln!( + out, + "# HELP telemt_ip_tracker_users Number of users tracked by IP limiter state" + ); + let _ = writeln!(out, "# TYPE telemt_ip_tracker_users gauge"); + let _ = writeln!( + out, + "telemt_ip_tracker_users{{scope=\"active\"}} {}", + ip_memory.active_users + ); + let _ = writeln!( + out, + "telemt_ip_tracker_users{{scope=\"recent\"}} {}", + ip_memory.recent_users + ); + let _ = writeln!( + out, + "# HELP telemt_ip_tracker_entries Number of IP entries tracked by limiter state" + ); + let _ = writeln!(out, "# TYPE telemt_ip_tracker_entries gauge"); + let _ = writeln!( + out, + "telemt_ip_tracker_entries{{scope=\"active\"}} {}", + ip_memory.active_entries + ); + let _ = writeln!( + out, + "telemt_ip_tracker_entries{{scope=\"recent\"}} {}", + ip_memory.recent_entries + ); + let _ = writeln!( + out, + "# HELP telemt_ip_tracker_cleanup_queue_len Deferred disconnect cleanup queue length" + ); + let _ = writeln!(out, "# TYPE telemt_ip_tracker_cleanup_queue_len gauge"); + let _ = writeln!( + out, + "telemt_ip_tracker_cleanup_queue_len {}", + ip_memory.cleanup_queue_len + ); + if user_enabled { for entry in stats.iter_user_stats() { let user = entry.key(); @@ -2634,6 +2927,7 @@ mod tests { #[tokio::test] async fn test_render_metrics_format() { let stats = Arc::new(Stats::new()); + let shared_state = ProxySharedState::new(); let tracker = UserIpTracker::new(); let mut config = ProxyConfig::default(); config @@ -2645,6 +2939,14 @@ mod tests { stats.increment_connects_all(); stats.increment_connects_bad(); stats.increment_handshake_timeouts(); + shared_state + .handshake + .auth_expensive_checks_total + .fetch_add(9, std::sync::atomic::Ordering::Relaxed); + shared_state + .handshake + .auth_budget_exhausted_total + .fetch_add(2, std::sync::atomic::Ordering::Relaxed); stats.increment_upstream_connect_attempt_total(); stats.increment_upstream_connect_attempt_total(); stats.increment_upstream_connect_success_total(); @@ -2688,7 +2990,7 @@ mod tests { .await .unwrap(); - let output = render_metrics(&stats, &config, &tracker).await; + let output = render_metrics(&stats, shared_state.as_ref(), &config, &tracker).await; assert!(output.contains(&format!( "telemt_build_info{{version=\"{}\"}} 1", @@ -2697,6 +2999,8 @@ mod tests { assert!(output.contains("telemt_connections_total 2")); assert!(output.contains("telemt_connections_bad_total 1")); assert!(output.contains("telemt_handshake_timeouts_total 1")); + assert!(output.contains("telemt_auth_expensive_checks_total 9")); + assert!(output.contains("telemt_auth_budget_exhausted_total 2")); assert!(output.contains("telemt_upstream_connect_attempt_total 2")); assert!(output.contains("telemt_upstream_connect_success_total 1")); assert!(output.contains("telemt_upstream_connect_fail_total 1")); @@ -2743,17 +3047,23 @@ mod tests { assert!(output.contains("telemt_user_unique_ips_recent_window{user=\"alice\"} 1")); assert!(output.contains("telemt_user_unique_ips_limit{user=\"alice\"} 4")); assert!(output.contains("telemt_user_unique_ips_utilization{user=\"alice\"} 0.250000")); + assert!(output.contains("telemt_ip_tracker_users{scope=\"active\"} 1")); + assert!(output.contains("telemt_ip_tracker_entries{scope=\"active\"} 1")); + assert!(output.contains("telemt_ip_tracker_cleanup_queue_len 0")); } #[tokio::test] async fn test_render_empty_stats() { let stats = Stats::new(); + let shared_state = ProxySharedState::new(); let tracker = UserIpTracker::new(); let config = ProxyConfig::default(); - let output = render_metrics(&stats, &config, &tracker).await; + let output = render_metrics(&stats, &shared_state, &config, &tracker).await; assert!(output.contains("telemt_connections_total 0")); assert!(output.contains("telemt_connections_bad_total 0")); assert!(output.contains("telemt_handshake_timeouts_total 0")); + assert!(output.contains("telemt_auth_expensive_checks_total 0")); + assert!(output.contains("telemt_auth_budget_exhausted_total 0")); assert!(output.contains("telemt_user_unique_ips_current{user=")); assert!(output.contains("telemt_user_unique_ips_recent_window{user=")); } @@ -2761,6 +3071,7 @@ mod tests { #[tokio::test] async fn test_render_uses_global_each_unique_ip_limit() { let stats = Stats::new(); + let shared_state = ProxySharedState::new(); stats.increment_user_connects("alice"); stats.increment_user_curr_connects("alice"); let tracker = UserIpTracker::new(); @@ -2771,7 +3082,7 @@ mod tests { let mut config = ProxyConfig::default(); config.access.user_max_unique_ips_global_each = 2; - let output = render_metrics(&stats, &config, &tracker).await; + let output = render_metrics(&stats, &shared_state, &config, &tracker).await; assert!(output.contains("telemt_user_unique_ips_limit{user=\"alice\"} 2")); assert!(output.contains("telemt_user_unique_ips_utilization{user=\"alice\"} 0.500000")); @@ -2780,14 +3091,16 @@ mod tests { #[tokio::test] async fn test_render_has_type_annotations() { let stats = Stats::new(); + let shared_state = ProxySharedState::new(); let tracker = UserIpTracker::new(); let config = ProxyConfig::default(); - let output = render_metrics(&stats, &config, &tracker).await; - assert!(output.contains("# TYPE telemt_build_info gauge")); + let output = render_metrics(&stats, &shared_state, &config, &tracker).await; assert!(output.contains("# TYPE telemt_uptime_seconds gauge")); assert!(output.contains("# TYPE telemt_connections_total counter")); assert!(output.contains("# TYPE telemt_connections_bad_total counter")); assert!(output.contains("# TYPE telemt_handshake_timeouts_total counter")); + assert!(output.contains("# TYPE telemt_auth_expensive_checks_total counter")); + assert!(output.contains("# TYPE telemt_auth_budget_exhausted_total counter")); assert!(output.contains("# TYPE telemt_upstream_connect_attempt_total counter")); assert!(output.contains("# TYPE telemt_me_rpc_proxy_req_signal_sent_total counter")); assert!(output.contains("# TYPE telemt_me_idle_close_by_peer_total counter")); @@ -2815,12 +3128,16 @@ mod tests { assert!(output.contains("# TYPE telemt_user_unique_ips_recent_window gauge")); assert!(output.contains("# TYPE telemt_user_unique_ips_limit gauge")); assert!(output.contains("# TYPE telemt_user_unique_ips_utilization gauge")); + assert!(output.contains("# TYPE telemt_ip_tracker_users gauge")); + assert!(output.contains("# TYPE telemt_ip_tracker_entries gauge")); + assert!(output.contains("# TYPE telemt_ip_tracker_cleanup_queue_len gauge")); } #[tokio::test] async fn test_endpoint_integration() { let stats = Arc::new(Stats::new()); let beobachten = Arc::new(BeobachtenStore::new()); + let shared_state = ProxySharedState::new(); let tracker = UserIpTracker::new(); let mut config = ProxyConfig::default(); stats.increment_connects_all(); @@ -2828,7 +3145,7 @@ mod tests { stats.increment_connects_all(); let req = Request::builder().uri("/metrics").body(()).unwrap(); - let resp = handle(req, &stats, &beobachten, &tracker, &config) + let resp = handle(req, &stats, &beobachten, shared_state.as_ref(), &tracker, &config) .await .unwrap(); assert_eq!(resp.status(), StatusCode::OK); @@ -2855,7 +3172,14 @@ mod tests { Duration::from_secs(600), ); let req_beob = Request::builder().uri("/beobachten").body(()).unwrap(); - let resp_beob = handle(req_beob, &stats, &beobachten, &tracker, &config) + let resp_beob = handle( + req_beob, + &stats, + &beobachten, + shared_state.as_ref(), + &tracker, + &config, + ) .await .unwrap(); assert_eq!(resp_beob.status(), StatusCode::OK); @@ -2865,7 +3189,14 @@ mod tests { assert!(beob_text.contains("203.0.113.10-1")); let req404 = Request::builder().uri("/other").body(()).unwrap(); - let resp404 = handle(req404, &stats, &beobachten, &tracker, &config) + let resp404 = handle( + req404, + &stats, + &beobachten, + shared_state.as_ref(), + &tracker, + &config, + ) .await .unwrap(); assert_eq!(resp404.status(), StatusCode::NOT_FOUND); diff --git a/src/proxy/adaptive_buffers.rs b/src/proxy/adaptive_buffers.rs index 0c210dd..a04f4e8 100644 --- a/src/proxy/adaptive_buffers.rs +++ b/src/proxy/adaptive_buffers.rs @@ -24,6 +24,8 @@ const DIRECT_S2C_CAP_BYTES: usize = 512 * 1024; const ME_FRAMES_CAP: usize = 96; const ME_BYTES_CAP: usize = 384 * 1024; const ME_DELAY_MIN_US: u64 = 150; +const MAX_USER_PROFILES_ENTRIES: usize = 50_000; +const MAX_USER_KEY_BYTES: usize = 512; #[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)] pub enum AdaptiveTier { @@ -234,32 +236,50 @@ fn profiles() -> &'static DashMap { } pub fn seed_tier_for_user(user: &str) -> AdaptiveTier { + if user.len() > MAX_USER_KEY_BYTES { + return AdaptiveTier::Base; + } let now = Instant::now(); if let Some(entry) = profiles().get(user) { - let value = entry.value(); - if now.duration_since(value.seen_at) <= PROFILE_TTL { + let value = *entry.value(); + drop(entry); + if now.saturating_duration_since(value.seen_at) <= PROFILE_TTL { return value.tier; } + profiles().remove_if(user, |_, v| { + now.saturating_duration_since(v.seen_at) > PROFILE_TTL + }); } AdaptiveTier::Base } pub fn record_user_tier(user: &str, tier: AdaptiveTier) { - let now = Instant::now(); - if let Some(mut entry) = profiles().get_mut(user) { - let existing = *entry; - let effective = if now.duration_since(existing.seen_at) > PROFILE_TTL { - tier - } else { - max(existing.tier, tier) - }; - *entry = UserAdaptiveProfile { - tier: effective, - seen_at: now, - }; + if user.len() > MAX_USER_KEY_BYTES { return; } - profiles().insert(user.to_string(), UserAdaptiveProfile { tier, seen_at: now }); + let now = Instant::now(); + let mut was_vacant = false; + match profiles().entry(user.to_string()) { + dashmap::mapref::entry::Entry::Occupied(mut entry) => { + let existing = *entry.get(); + let effective = if now.saturating_duration_since(existing.seen_at) > PROFILE_TTL { + tier + } else { + max(existing.tier, tier) + }; + entry.insert(UserAdaptiveProfile { + tier: effective, + seen_at: now, + }); + } + dashmap::mapref::entry::Entry::Vacant(slot) => { + slot.insert(UserAdaptiveProfile { tier, seen_at: now }); + was_vacant = true; + } + } + if was_vacant && profiles().len() > MAX_USER_PROFILES_ENTRIES { + profiles().retain(|_, v| now.saturating_duration_since(v.seen_at) <= PROFILE_TTL); + } } pub fn direct_copy_buffers_for_tier( @@ -310,6 +330,14 @@ fn scale(base: usize, numerator: usize, denominator: usize, cap: usize) -> usize scaled.min(cap).max(1) } +#[cfg(test)] +#[path = "tests/adaptive_buffers_security_tests.rs"] +mod adaptive_buffers_security_tests; + +#[cfg(test)] +#[path = "tests/adaptive_buffers_record_race_security_tests.rs"] +mod adaptive_buffers_record_race_security_tests; + #[cfg(test)] mod tests { use super::*; diff --git a/src/proxy/client.rs b/src/proxy/client.rs index 7472459..fb73db2 100644 --- a/src/proxy/client.rs +++ b/src/proxy/client.rs @@ -80,11 +80,16 @@ use crate::transport::middle_proxy::MePool; use crate::transport::socket::normalize_ip; use crate::transport::{UpstreamManager, configure_client_socket, parse_proxy_protocol}; -use crate::proxy::direct_relay::handle_via_direct; -use crate::proxy::handshake::{HandshakeSuccess, handle_mtproto_handshake, handle_tls_handshake}; +use crate::proxy::direct_relay::handle_via_direct_with_shared; +use crate::proxy::handshake::{ + HandshakeSuccess, handle_mtproto_handshake_with_shared, handle_tls_handshake_with_shared, +}; +#[cfg(test)] +use crate::proxy::handshake::{handle_mtproto_handshake, handle_tls_handshake}; use crate::proxy::masking::handle_bad_client; use crate::proxy::middle_relay::handle_via_middle_proxy; use crate::proxy::route_mode::{RelayRouteMode, RouteRuntimeController}; +use crate::proxy::shared_state::ProxySharedState; fn beobachten_ttl(config: &ProxyConfig) -> Duration { const BEOBACHTEN_TTL_MAX_MINUTES: u64 = 24 * 60; @@ -186,6 +191,24 @@ fn handshake_timeout_with_mask_grace(config: &ProxyConfig) -> Duration { } } +fn effective_client_first_byte_idle_secs(config: &ProxyConfig, shared: &ProxySharedState) -> u64 { + let idle_secs = config.timeouts.client_first_byte_idle_secs; + if idle_secs == 0 { + return 0; + } + if shared.conntrack_pressure_active() { + idle_secs.min( + config + .server + .conntrack_control + .profile + .client_first_byte_idle_cap_secs(), + ) + } else { + idle_secs + } +} + const MASK_CLASSIFIER_PREFETCH_WINDOW: usize = 16; #[cfg(test)] const MASK_CLASSIFIER_PREFETCH_TIMEOUT: Duration = Duration::from_millis(5); @@ -342,7 +365,48 @@ fn synthetic_local_addr(port: u16) -> SocketAddr { SocketAddr::from(([0, 0, 0, 0], port)) } +#[cfg(test)] pub async fn handle_client_stream( + stream: S, + peer: SocketAddr, + config: Arc, + stats: Arc, + upstream_manager: Arc, + replay_checker: Arc, + buffer_pool: Arc, + rng: Arc, + me_pool: Option>, + route_runtime: Arc, + tls_cache: Option>, + ip_tracker: Arc, + beobachten: Arc, + proxy_protocol_enabled: bool, +) -> Result<()> +where + S: AsyncRead + AsyncWrite + Unpin + Send + 'static, +{ + handle_client_stream_with_shared( + stream, + peer, + config, + stats, + upstream_manager, + replay_checker, + buffer_pool, + rng, + me_pool, + route_runtime, + tls_cache, + ip_tracker, + beobachten, + ProxySharedState::new(), + proxy_protocol_enabled, + ) + .await +} + +#[allow(clippy::too_many_arguments)] +pub async fn handle_client_stream_with_shared( mut stream: S, peer: SocketAddr, config: Arc, @@ -356,6 +420,7 @@ pub async fn handle_client_stream( tls_cache: Option>, ip_tracker: Arc, beobachten: Arc, + shared: Arc, proxy_protocol_enabled: bool, ) -> Result<()> where @@ -416,10 +481,11 @@ where debug!(peer = %real_peer, "New connection (generic stream)"); - let first_byte = if config.timeouts.client_first_byte_idle_secs == 0 { + let first_byte_idle_secs = effective_client_first_byte_idle_secs(&config, shared.as_ref()); + let first_byte = if first_byte_idle_secs == 0 { None } else { - let idle_timeout = Duration::from_secs(config.timeouts.client_first_byte_idle_secs); + let idle_timeout = Duration::from_secs(first_byte_idle_secs); let mut first_byte = [0u8; 1]; match timeout(idle_timeout, stream.read(&mut first_byte)).await { Ok(Ok(0)) => { @@ -455,7 +521,7 @@ where Err(_) => { debug!( peer = %real_peer, - idle_secs = config.timeouts.client_first_byte_idle_secs, + idle_secs = first_byte_idle_secs, "Closing idle pooled connection before first client byte" ); return Ok(()); @@ -550,9 +616,10 @@ where let (read_half, write_half) = tokio::io::split(stream); - let (mut tls_reader, tls_writer, tls_user) = match handle_tls_handshake( + let (mut tls_reader, tls_writer, tls_user) = match handle_tls_handshake_with_shared( &handshake, read_half, write_half, real_peer, &config, &replay_checker, &rng, tls_cache.clone(), + shared.as_ref(), ).await { HandshakeResult::Success(result) => result, HandshakeResult::BadClient { reader, writer } => { @@ -578,9 +645,10 @@ where let mtproto_handshake: [u8; HANDSHAKE_LEN] = mtproto_data[..].try_into() .map_err(|_| ProxyError::InvalidHandshake("Short MTProto handshake".into()))?; - let (crypto_reader, crypto_writer, success) = match handle_mtproto_handshake( + let (crypto_reader, crypto_writer, success) = match handle_mtproto_handshake_with_shared( &mtproto_handshake, tls_reader, tls_writer, real_peer, &config, &replay_checker, true, Some(tls_user.as_str()), + shared.as_ref(), ).await { HandshakeResult::Success(result) => result, HandshakeResult::BadClient { reader, writer } => { @@ -614,11 +682,12 @@ where }; Ok(HandshakeOutcome::NeedsRelay(Box::pin( - RunningClientHandler::handle_authenticated_static( + RunningClientHandler::handle_authenticated_static_with_shared( crypto_reader, crypto_writer, success, upstream_manager, stats, config, buffer_pool, rng, me_pool, route_runtime.clone(), local_addr, real_peer, ip_tracker.clone(), + shared.clone(), ), ))) } else { @@ -644,9 +713,10 @@ where let (read_half, write_half) = tokio::io::split(stream); - let (crypto_reader, crypto_writer, success) = match handle_mtproto_handshake( + let (crypto_reader, crypto_writer, success) = match handle_mtproto_handshake_with_shared( &handshake, read_half, write_half, real_peer, &config, &replay_checker, false, None, + shared.as_ref(), ).await { HandshakeResult::Success(result) => result, HandshakeResult::BadClient { reader, writer } => { @@ -665,7 +735,7 @@ where }; Ok(HandshakeOutcome::NeedsRelay(Box::pin( - RunningClientHandler::handle_authenticated_static( + RunningClientHandler::handle_authenticated_static_with_shared( crypto_reader, crypto_writer, success, @@ -679,6 +749,7 @@ where local_addr, real_peer, ip_tracker.clone(), + shared.clone(), ) ))) } @@ -731,10 +802,12 @@ pub struct RunningClientHandler { tls_cache: Option>, ip_tracker: Arc, beobachten: Arc, + shared: Arc, proxy_protocol_enabled: bool, } impl ClientHandler { + #[cfg(test)] pub fn new( stream: TcpStream, peer: SocketAddr, @@ -751,6 +824,45 @@ impl ClientHandler { beobachten: Arc, proxy_protocol_enabled: bool, real_peer_report: Arc>>, + ) -> RunningClientHandler { + Self::new_with_shared( + stream, + peer, + config, + stats, + upstream_manager, + replay_checker, + buffer_pool, + rng, + me_pool, + route_runtime, + tls_cache, + ip_tracker, + beobachten, + ProxySharedState::new(), + proxy_protocol_enabled, + real_peer_report, + ) + } + + #[allow(clippy::too_many_arguments)] + pub fn new_with_shared( + stream: TcpStream, + peer: SocketAddr, + config: Arc, + stats: Arc, + upstream_manager: Arc, + replay_checker: Arc, + buffer_pool: Arc, + rng: Arc, + me_pool: Option>, + route_runtime: Arc, + tls_cache: Option>, + ip_tracker: Arc, + beobachten: Arc, + shared: Arc, + proxy_protocol_enabled: bool, + real_peer_report: Arc>>, ) -> RunningClientHandler { let normalized_peer = normalize_ip(peer); RunningClientHandler { @@ -769,6 +881,7 @@ impl ClientHandler { tls_cache, ip_tracker, beobachten, + shared, proxy_protocol_enabled, } } @@ -874,11 +987,12 @@ impl RunningClientHandler { } } - let first_byte = if self.config.timeouts.client_first_byte_idle_secs == 0 { + let first_byte_idle_secs = + effective_client_first_byte_idle_secs(&self.config, self.shared.as_ref()); + let first_byte = if first_byte_idle_secs == 0 { None } else { - let idle_timeout = - Duration::from_secs(self.config.timeouts.client_first_byte_idle_secs); + let idle_timeout = Duration::from_secs(first_byte_idle_secs); let mut first_byte = [0u8; 1]; match timeout(idle_timeout, self.stream.read(&mut first_byte)).await { Ok(Ok(0)) => { @@ -914,7 +1028,7 @@ impl RunningClientHandler { Err(_) => { debug!( peer = %self.peer, - idle_secs = self.config.timeouts.client_first_byte_idle_secs, + idle_secs = first_byte_idle_secs, "Closing idle pooled connection before first client byte" ); return Ok(None); @@ -1058,7 +1172,7 @@ impl RunningClientHandler { let (read_half, write_half) = self.stream.into_split(); - let (mut tls_reader, tls_writer, tls_user) = match handle_tls_handshake( + let (mut tls_reader, tls_writer, tls_user) = match handle_tls_handshake_with_shared( &handshake, read_half, write_half, @@ -1067,6 +1181,7 @@ impl RunningClientHandler { &replay_checker, &self.rng, self.tls_cache.clone(), + self.shared.as_ref(), ) .await { @@ -1095,7 +1210,7 @@ impl RunningClientHandler { .try_into() .map_err(|_| ProxyError::InvalidHandshake("Short MTProto handshake".into()))?; - let (crypto_reader, crypto_writer, success) = match handle_mtproto_handshake( + let (crypto_reader, crypto_writer, success) = match handle_mtproto_handshake_with_shared( &mtproto_handshake, tls_reader, tls_writer, @@ -1104,6 +1219,7 @@ impl RunningClientHandler { &replay_checker, true, Some(tls_user.as_str()), + self.shared.as_ref(), ) .await { @@ -1140,7 +1256,7 @@ impl RunningClientHandler { }; Ok(HandshakeOutcome::NeedsRelay(Box::pin( - Self::handle_authenticated_static( + Self::handle_authenticated_static_with_shared( crypto_reader, crypto_writer, success, @@ -1154,6 +1270,7 @@ impl RunningClientHandler { local_addr, peer, self.ip_tracker, + self.shared, ), ))) } @@ -1192,7 +1309,7 @@ impl RunningClientHandler { let (read_half, write_half) = self.stream.into_split(); - let (crypto_reader, crypto_writer, success) = match handle_mtproto_handshake( + let (crypto_reader, crypto_writer, success) = match handle_mtproto_handshake_with_shared( &handshake, read_half, write_half, @@ -1201,6 +1318,7 @@ impl RunningClientHandler { &replay_checker, false, None, + self.shared.as_ref(), ) .await { @@ -1221,7 +1339,7 @@ impl RunningClientHandler { }; Ok(HandshakeOutcome::NeedsRelay(Box::pin( - Self::handle_authenticated_static( + Self::handle_authenticated_static_with_shared( crypto_reader, crypto_writer, success, @@ -1235,6 +1353,7 @@ impl RunningClientHandler { local_addr, peer, self.ip_tracker, + self.shared, ), ))) } @@ -1243,6 +1362,7 @@ impl RunningClientHandler { /// Two modes: /// - Direct: TCP relay to TG DC (existing behavior) /// - Middle Proxy: RPC multiplex through ME pool (new — supports CDN DCs) + #[cfg(test)] async fn handle_authenticated_static( client_reader: CryptoReader, client_writer: CryptoWriter, @@ -1258,6 +1378,45 @@ impl RunningClientHandler { peer_addr: SocketAddr, ip_tracker: Arc, ) -> Result<()> + where + R: AsyncRead + Unpin + Send + 'static, + W: AsyncWrite + Unpin + Send + 'static, + { + Self::handle_authenticated_static_with_shared( + client_reader, + client_writer, + success, + upstream_manager, + stats, + config, + buffer_pool, + rng, + me_pool, + route_runtime, + local_addr, + peer_addr, + ip_tracker, + ProxySharedState::new(), + ) + .await + } + + async fn handle_authenticated_static_with_shared( + client_reader: CryptoReader, + client_writer: CryptoWriter, + success: HandshakeSuccess, + upstream_manager: Arc, + stats: Arc, + config: Arc, + buffer_pool: Arc, + rng: Arc, + me_pool: Option>, + route_runtime: Arc, + local_addr: SocketAddr, + peer_addr: SocketAddr, + ip_tracker: Arc, + shared: Arc, + ) -> Result<()> where R: AsyncRead + Unpin + Send + 'static, W: AsyncWrite + Unpin + Send + 'static, @@ -1299,11 +1458,12 @@ impl RunningClientHandler { route_runtime.subscribe(), route_snapshot, session_id, + shared.clone(), ) .await } else { warn!("use_middle_proxy=true but MePool not initialized, falling back to direct"); - handle_via_direct( + handle_via_direct_with_shared( client_reader, client_writer, success, @@ -1315,12 +1475,14 @@ impl RunningClientHandler { route_runtime.subscribe(), route_snapshot, session_id, + local_addr, + shared.clone(), ) .await } } else { // Direct mode (original behavior) - handle_via_direct( + handle_via_direct_with_shared( client_reader, client_writer, success, @@ -1332,6 +1494,8 @@ impl RunningClientHandler { route_runtime.subscribe(), route_snapshot, session_id, + local_addr, + shared.clone(), ) .await }; diff --git a/src/proxy/direct_relay.rs b/src/proxy/direct_relay.rs index 5d9c450..2c4fe45 100644 --- a/src/proxy/direct_relay.rs +++ b/src/proxy/direct_relay.rs @@ -6,6 +6,7 @@ use std::net::SocketAddr; use std::path::{Component, Path, PathBuf}; use std::sync::Arc; use std::sync::{Mutex, OnceLock}; +use std::time::Duration; use tokio::io::{AsyncRead, AsyncWrite, AsyncWriteExt, ReadHalf, WriteHalf, split}; use tokio::sync::watch; @@ -16,11 +17,13 @@ use crate::crypto::SecureRandom; use crate::error::{ProxyError, Result}; use crate::protocol::constants::*; use crate::proxy::handshake::{HandshakeSuccess, encrypt_tg_nonce_with_ciphers, generate_tg_nonce}; -use crate::proxy::relay::relay_bidirectional; use crate::proxy::route_mode::{ ROUTE_SWITCH_ERROR_MSG, RelayRouteMode, RouteCutoverState, affected_cutover_state, cutover_stagger_delay, }; +use crate::proxy::shared_state::{ + ConntrackCloseEvent, ConntrackClosePublishResult, ConntrackCloseReason, ProxySharedState, +}; use crate::stats::Stats; use crate::stream::{BufferPool, CryptoReader, CryptoWriter}; use crate::transport::UpstreamManager; @@ -225,7 +228,43 @@ fn unknown_dc_test_lock() -> &'static Mutex<()> { TEST_LOCK.get_or_init(|| Mutex::new(())) } +#[allow(dead_code)] pub(crate) async fn handle_via_direct( + client_reader: CryptoReader, + client_writer: CryptoWriter, + success: HandshakeSuccess, + upstream_manager: Arc, + stats: Arc, + config: Arc, + buffer_pool: Arc, + rng: Arc, + route_rx: watch::Receiver, + route_snapshot: RouteCutoverState, + session_id: u64, +) -> Result<()> +where + R: AsyncRead + Unpin + Send + 'static, + W: AsyncWrite + Unpin + Send + 'static, +{ + handle_via_direct_with_shared( + client_reader, + client_writer, + success, + upstream_manager, + stats, + config.clone(), + buffer_pool, + rng, + route_rx, + route_snapshot, + session_id, + SocketAddr::from(([0, 0, 0, 0], config.server.port)), + ProxySharedState::new(), + ) + .await +} + +pub(crate) async fn handle_via_direct_with_shared( client_reader: CryptoReader, client_writer: CryptoWriter, success: HandshakeSuccess, @@ -237,6 +276,8 @@ pub(crate) async fn handle_via_direct( mut route_rx: watch::Receiver, route_snapshot: RouteCutoverState, session_id: u64, + local_addr: SocketAddr, + shared: Arc, ) -> Result<()> where R: AsyncRead + Unpin + Send + 'static, @@ -276,7 +317,19 @@ where stats.increment_user_connects(user); let _direct_connection_lease = stats.acquire_direct_connection_lease(); - let relay_result = relay_bidirectional( + let buffer_pool_trim = Arc::clone(&buffer_pool); + let relay_activity_timeout = if shared.conntrack_pressure_active() { + Duration::from_secs( + config + .server + .conntrack_control + .profile + .direct_activity_timeout_secs(), + ) + } else { + Duration::from_secs(1800) + }; + let relay_result = crate::proxy::relay::relay_bidirectional_with_activity_timeout( client_reader, client_writer, tg_reader, @@ -287,6 +340,7 @@ where Arc::clone(&stats), config.access.user_data_quota.get(user).copied(), buffer_pool, + relay_activity_timeout, ); tokio::pin!(relay_result); let relay_result = loop { @@ -321,9 +375,59 @@ where Err(e) => debug!(user = %user, error = %e, "Direct relay ended with error"), } + buffer_pool_trim.trim_to(buffer_pool_trim.max_buffers().min(64)); + let pool_snapshot = buffer_pool_trim.stats(); + stats.set_buffer_pool_gauges( + pool_snapshot.pooled, + pool_snapshot.allocated, + pool_snapshot.allocated.saturating_sub(pool_snapshot.pooled), + ); + + let close_reason = classify_conntrack_close_reason(&relay_result); + let publish_result = shared.publish_conntrack_close_event(ConntrackCloseEvent { + src: success.peer, + dst: local_addr, + reason: close_reason, + }); + if !matches!( + publish_result, + ConntrackClosePublishResult::Sent | ConntrackClosePublishResult::Disabled + ) { + stats.increment_conntrack_close_event_drop_total(); + } + relay_result } +fn classify_conntrack_close_reason(result: &Result<()>) -> ConntrackCloseReason { + match result { + Ok(()) => ConntrackCloseReason::NormalEof, + Err(crate::error::ProxyError::Io(error)) + if matches!(error.kind(), std::io::ErrorKind::TimedOut) => + { + ConntrackCloseReason::Timeout + } + Err(crate::error::ProxyError::Io(error)) + if matches!( + error.kind(), + std::io::ErrorKind::ConnectionReset + | std::io::ErrorKind::ConnectionAborted + | std::io::ErrorKind::BrokenPipe + | std::io::ErrorKind::NotConnected + | std::io::ErrorKind::UnexpectedEof + ) => + { + ConntrackCloseReason::Reset + } + Err(crate::error::ProxyError::Proxy(message)) + if message.contains("pressure") || message.contains("evicted") => + { + ConntrackCloseReason::Pressure + } + Err(_) => ConntrackCloseReason::Other, + } +} + fn get_dc_addr_static(dc_idx: i16, config: &ProxyConfig) -> Result { let prefer_v6 = config.network.prefer == 6 && config.network.ipv6.unwrap_or(true); let datacenters = if prefer_v6 { diff --git a/src/proxy/handshake.rs b/src/proxy/handshake.rs index fbaffa2..904b8f9 100644 --- a/src/proxy/handshake.rs +++ b/src/proxy/handshake.rs @@ -4,13 +4,19 @@ use dashmap::DashMap; use dashmap::mapref::entry::Entry; +use hmac::{Hmac, Mac}; +#[cfg(test)] use std::collections::HashSet; +use std::collections::hash_map::DefaultHasher; +#[cfg(test)] use std::collections::hash_map::RandomState; use std::hash::{BuildHasher, Hash, Hasher}; use std::net::SocketAddr; use std::net::{IpAddr, Ipv6Addr}; use std::sync::Arc; -use std::sync::{Mutex, OnceLock}; +#[cfg(test)] +use std::sync::Mutex; +use std::sync::atomic::Ordering; use std::time::{Duration, Instant}; use tokio::io::{AsyncRead, AsyncWrite, AsyncWriteExt}; use tracing::{debug, info, trace, warn}; @@ -21,15 +27,17 @@ use crate::crypto::{AesCtr, SecureRandom, sha256}; use crate::error::{HandshakeResult, ProxyError}; use crate::protocol::constants::*; use crate::protocol::tls; +use crate::proxy::shared_state::ProxySharedState; use crate::stats::ReplayChecker; use crate::stream::{CryptoReader, CryptoWriter, FakeTlsReader, FakeTlsWriter}; use crate::tls_front::{TlsFrontCache, emulator}; +#[cfg(test)] use rand::RngExt; +use sha2::Sha256; +use subtle::ConstantTimeEq; const ACCESS_SECRET_BYTES: usize = 16; -static INVALID_SECRET_WARNED: OnceLock>> = OnceLock::new(); const UNKNOWN_SNI_WARN_COOLDOWN_SECS: u64 = 5; -static UNKNOWN_SNI_WARN_NEXT_ALLOWED: OnceLock>> = OnceLock::new(); #[cfg(test)] const WARNED_SECRET_MAX_ENTRIES: usize = 64; #[cfg(not(test))] @@ -43,6 +51,13 @@ const AUTH_PROBE_TRACK_MAX_ENTRIES: usize = 65_536; const AUTH_PROBE_PRUNE_SCAN_LIMIT: usize = 1_024; const AUTH_PROBE_BACKOFF_START_FAILS: u32 = 4; const AUTH_PROBE_SATURATION_GRACE_FAILS: u32 = 2; +const STICKY_HINT_MAX_ENTRIES: usize = 65_536; +const CANDIDATE_HINT_TRACK_CAP: usize = 64; +const OVERLOAD_CANDIDATE_BUDGET_HINTED: usize = 16; +const OVERLOAD_CANDIDATE_BUDGET_UNHINTED: usize = 8; +const RECENT_USER_RING_SCAN_LIMIT: usize = 32; + +type HmacSha256 = Hmac; #[cfg(test)] const AUTH_PROBE_BACKOFF_BASE_MS: u64 = 1; @@ -55,48 +70,30 @@ const AUTH_PROBE_BACKOFF_MAX_MS: u64 = 16; const AUTH_PROBE_BACKOFF_MAX_MS: u64 = 1_000; #[derive(Clone, Copy)] -struct AuthProbeState { +pub(crate) struct AuthProbeState { fail_streak: u32, blocked_until: Instant, last_seen: Instant, } #[derive(Clone, Copy)] -struct AuthProbeSaturationState { +pub(crate) struct AuthProbeSaturationState { fail_streak: u32, blocked_until: Instant, last_seen: Instant, } - -static AUTH_PROBE_STATE: OnceLock> = OnceLock::new(); -static AUTH_PROBE_SATURATION_STATE: OnceLock>> = - OnceLock::new(); -static AUTH_PROBE_EVICTION_HASHER: OnceLock = OnceLock::new(); - -fn auth_probe_state_map() -> &'static DashMap { - AUTH_PROBE_STATE.get_or_init(DashMap::new) -} - -fn auth_probe_saturation_state() -> &'static Mutex> { - AUTH_PROBE_SATURATION_STATE.get_or_init(|| Mutex::new(None)) -} - -fn auth_probe_saturation_state_lock() --> std::sync::MutexGuard<'static, Option> { - auth_probe_saturation_state() +fn unknown_sni_warn_state_lock_in( + shared: &ProxySharedState, +) -> std::sync::MutexGuard<'_, Option> { + shared + .handshake + .unknown_sni_warn_next_allowed .lock() .unwrap_or_else(|poisoned| poisoned.into_inner()) } -fn unknown_sni_warn_state_lock() -> std::sync::MutexGuard<'static, Option> { - UNKNOWN_SNI_WARN_NEXT_ALLOWED - .get_or_init(|| Mutex::new(None)) - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()) -} - -fn should_emit_unknown_sni_warn(now: Instant) -> bool { - let mut guard = unknown_sni_warn_state_lock(); +fn should_emit_unknown_sni_warn_in(shared: &ProxySharedState, now: Instant) -> bool { + let mut guard = unknown_sni_warn_state_lock_in(shared); if let Some(next_allowed) = *guard && now < next_allowed { @@ -106,6 +103,302 @@ fn should_emit_unknown_sni_warn(now: Instant) -> bool { true } +#[derive(Clone, Copy)] +struct ParsedTlsAuthMaterial { + digest: [u8; tls::TLS_DIGEST_LEN], + session_id: [u8; 32], + session_id_len: usize, + now: i64, + ignore_time_skew: bool, + boot_time_cap_secs: u32, +} + +#[derive(Clone, Copy)] +struct TlsCandidateValidation { + digest: [u8; tls::TLS_DIGEST_LEN], + session_id: [u8; 32], + session_id_len: usize, +} + +struct MtprotoCandidateValidation { + proto_tag: ProtoTag, + dc_idx: i16, + dec_key: [u8; 32], + dec_iv: u128, + enc_key: [u8; 32], + enc_iv: u128, + decryptor: AesCtr, + encryptor: AesCtr, +} + +fn sni_hint_hash(sni: &str) -> u64 { + let mut hasher = DefaultHasher::new(); + for byte in sni.bytes() { + hasher.write_u8(byte.to_ascii_lowercase()); + } + hasher.finish() +} + +fn ip_prefix_hint_key(peer_ip: IpAddr) -> u64 { + match peer_ip { + // Keep /24 granularity for IPv4 to avoid over-merging unrelated clients. + IpAddr::V4(ip) => { + let [a, b, c, _] = ip.octets(); + u64::from_be_bytes([0x04, a, b, c, 0, 0, 0, 0]) + } + // Keep /56 granularity for IPv6 to retain stability while limiting bucket size. + IpAddr::V6(ip) => { + let octets = ip.octets(); + u64::from_be_bytes([ + 0x06, octets[0], octets[1], octets[2], octets[3], octets[4], octets[5], octets[6], + ]) + } + } +} + +fn sticky_hint_get_by_ip(shared: &ProxySharedState, peer_ip: IpAddr) -> Option { + shared + .handshake + .sticky_user_by_ip + .get(&peer_ip) + .map(|entry| *entry) +} + +fn sticky_hint_get_by_ip_prefix(shared: &ProxySharedState, peer_ip: IpAddr) -> Option { + shared + .handshake + .sticky_user_by_ip_prefix + .get(&ip_prefix_hint_key(peer_ip)) + .map(|entry| *entry) +} + +fn sticky_hint_get_by_sni(shared: &ProxySharedState, sni: &str) -> Option { + let key = sni_hint_hash(sni); + shared + .handshake + .sticky_user_by_sni_hash + .get(&key) + .map(|entry| *entry) +} + +fn sticky_hint_record_success_in( + shared: &ProxySharedState, + peer_ip: IpAddr, + user_id: u32, + sni: Option<&str>, +) { + if shared.handshake.sticky_user_by_ip.len() > STICKY_HINT_MAX_ENTRIES { + shared.handshake.sticky_user_by_ip.clear(); + } + shared.handshake.sticky_user_by_ip.insert(peer_ip, user_id); + + if shared.handshake.sticky_user_by_ip_prefix.len() > STICKY_HINT_MAX_ENTRIES { + shared.handshake.sticky_user_by_ip_prefix.clear(); + } + shared + .handshake + .sticky_user_by_ip_prefix + .insert(ip_prefix_hint_key(peer_ip), user_id); + + if let Some(sni) = sni { + if shared.handshake.sticky_user_by_sni_hash.len() > STICKY_HINT_MAX_ENTRIES { + shared.handshake.sticky_user_by_sni_hash.clear(); + } + shared + .handshake + .sticky_user_by_sni_hash + .insert(sni_hint_hash(sni), user_id); + } +} + +fn record_recent_user_success_in(shared: &ProxySharedState, user_id: u32) { + let ring = &shared.handshake.recent_user_ring; + if ring.is_empty() { + return; + } + let seq = shared + .handshake + .recent_user_ring_seq + .fetch_add(1, Ordering::Relaxed); + let idx = (seq as usize) % ring.len(); + ring[idx].store(user_id.saturating_add(1), Ordering::Relaxed); +} + +fn mark_candidate_if_new(tried_user_ids: &mut [u32], tried_len: &mut usize, user_id: u32) -> bool { + if tried_user_ids[..*tried_len].contains(&user_id) { + return false; + } + if *tried_len < tried_user_ids.len() { + tried_user_ids[*tried_len] = user_id; + *tried_len += 1; + } + true +} + +fn budget_for_validation(total_users: usize, overload: bool, has_hint: bool) -> usize { + if total_users == 0 { + return 0; + } + if !overload { + return total_users; + } + let cap = if has_hint { + OVERLOAD_CANDIDATE_BUDGET_HINTED + } else { + OVERLOAD_CANDIDATE_BUDGET_UNHINTED + }; + total_users.min(cap.max(1)) +} + +fn parse_tls_auth_material( + handshake: &[u8], + ignore_time_skew: bool, + replay_window_secs: u64, +) -> Option { + if handshake.len() < tls::TLS_DIGEST_POS + tls::TLS_DIGEST_LEN + 1 { + return None; + } + + let digest: [u8; tls::TLS_DIGEST_LEN] = handshake + [tls::TLS_DIGEST_POS..tls::TLS_DIGEST_POS + tls::TLS_DIGEST_LEN] + .try_into() + .ok()?; + + let session_id_len_pos = tls::TLS_DIGEST_POS + tls::TLS_DIGEST_LEN; + let session_id_len = usize::from(handshake.get(session_id_len_pos).copied()?); + if session_id_len > 32 { + return None; + } + let session_id_start = session_id_len_pos + 1; + if handshake.len() < session_id_start + session_id_len { + return None; + } + + let mut session_id = [0u8; 32]; + session_id[..session_id_len] + .copy_from_slice(&handshake[session_id_start..session_id_start + session_id_len]); + + let now = if !ignore_time_skew { + let d = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .ok()?; + i64::try_from(d.as_secs()).ok()? + } else { + 0_i64 + }; + + let replay_window_u32 = u32::try_from(replay_window_secs).unwrap_or(u32::MAX); + let boot_time_cap_secs = if ignore_time_skew { + 0 + } else { + tls::BOOT_TIME_MAX_SECS + .min(replay_window_u32) + .min(tls::BOOT_TIME_COMPAT_MAX_SECS) + }; + + Some(ParsedTlsAuthMaterial { + digest, + session_id, + session_id_len, + now, + ignore_time_skew, + boot_time_cap_secs, + }) +} + +fn compute_tls_hmac_zeroed_digest(secret: &[u8], handshake: &[u8]) -> [u8; 32] { + let mut mac = HmacSha256::new_from_slice(secret).expect("HMAC accepts any key length"); + mac.update(&handshake[..tls::TLS_DIGEST_POS]); + mac.update(&[0u8; tls::TLS_DIGEST_LEN]); + mac.update(&handshake[tls::TLS_DIGEST_POS + tls::TLS_DIGEST_LEN..]); + mac.finalize().into_bytes().into() +} + +fn validate_tls_secret_candidate( + parsed: &ParsedTlsAuthMaterial, + handshake: &[u8], + secret: &[u8], +) -> Option { + let computed = compute_tls_hmac_zeroed_digest(secret, handshake); + if !bool::from(parsed.digest[..28].ct_eq(&computed[..28])) { + return None; + } + + let timestamp = u32::from_le_bytes([ + parsed.digest[28] ^ computed[28], + parsed.digest[29] ^ computed[29], + parsed.digest[30] ^ computed[30], + parsed.digest[31] ^ computed[31], + ]); + + if !parsed.ignore_time_skew { + let is_boot_time = parsed.boot_time_cap_secs > 0 && timestamp < parsed.boot_time_cap_secs; + if !is_boot_time { + let time_diff = parsed.now - i64::from(timestamp); + if !(tls::TIME_SKEW_MIN..=tls::TIME_SKEW_MAX).contains(&time_diff) { + return None; + } + } + } + + Some(TlsCandidateValidation { + digest: parsed.digest, + session_id: parsed.session_id, + session_id_len: parsed.session_id_len, + }) +} + +fn validate_mtproto_secret_candidate( + handshake: &[u8; HANDSHAKE_LEN], + dec_prekey: &[u8; PREKEY_LEN], + dec_iv: u128, + enc_prekey: &[u8; PREKEY_LEN], + enc_iv: u128, + secret: &[u8; ACCESS_SECRET_BYTES], + config: &ProxyConfig, + is_tls: bool, +) -> Option { + let mut dec_key_input = Zeroizing::new(Vec::with_capacity(PREKEY_LEN + secret.len())); + dec_key_input.extend_from_slice(dec_prekey); + dec_key_input.extend_from_slice(secret); + let dec_key = Zeroizing::new(sha256(&dec_key_input)); + + let mut decryptor = AesCtr::new(&dec_key, dec_iv); + let mut decrypted = *handshake; + decryptor.apply(&mut decrypted); + + let tag_bytes: [u8; 4] = [ + decrypted[PROTO_TAG_POS], + decrypted[PROTO_TAG_POS + 1], + decrypted[PROTO_TAG_POS + 2], + decrypted[PROTO_TAG_POS + 3], + ]; + let proto_tag = ProtoTag::from_bytes(tag_bytes)?; + if !mode_enabled_for_proto(config, proto_tag, is_tls) { + return None; + } + + let dc_idx = i16::from_le_bytes([decrypted[DC_IDX_POS], decrypted[DC_IDX_POS + 1]]); + + let mut enc_key_input = Zeroizing::new(Vec::with_capacity(PREKEY_LEN + secret.len())); + enc_key_input.extend_from_slice(enc_prekey); + enc_key_input.extend_from_slice(secret); + let enc_key = Zeroizing::new(sha256(&enc_key_input)); + + let encryptor = AesCtr::new(&enc_key, enc_iv); + + Some(MtprotoCandidateValidation { + proto_tag, + dc_idx, + dec_key: *dec_key, + dec_iv, + enc_key: *enc_key, + enc_iv, + decryptor, + encryptor, + }) +} + fn normalize_auth_probe_ip(peer_ip: IpAddr) -> IpAddr { match peer_ip { IpAddr::V4(ip) => IpAddr::V4(ip), @@ -133,15 +426,20 @@ fn auth_probe_state_expired(state: &AuthProbeState, now: Instant) -> bool { now.duration_since(state.last_seen) > retention } -fn auth_probe_eviction_offset(peer_ip: IpAddr, now: Instant) -> usize { - let hasher_state = AUTH_PROBE_EVICTION_HASHER.get_or_init(RandomState::new); +fn auth_probe_eviction_offset_in( + shared: &ProxySharedState, + peer_ip: IpAddr, + now: Instant, +) -> usize { + let hasher_state = &shared.handshake.auth_probe_eviction_hasher; let mut hasher = hasher_state.build_hasher(); peer_ip.hash(&mut hasher); now.hash(&mut hasher); hasher.finish() as usize } -fn auth_probe_scan_start_offset( +fn auth_probe_scan_start_offset_in( + shared: &ProxySharedState, peer_ip: IpAddr, now: Instant, state_len: usize, @@ -151,12 +449,12 @@ fn auth_probe_scan_start_offset( return 0; } - auth_probe_eviction_offset(peer_ip, now) % state_len + auth_probe_eviction_offset_in(shared, peer_ip, now) % state_len } -fn auth_probe_is_throttled(peer_ip: IpAddr, now: Instant) -> bool { +fn auth_probe_is_throttled_in(shared: &ProxySharedState, peer_ip: IpAddr, now: Instant) -> bool { let peer_ip = normalize_auth_probe_ip(peer_ip); - let state = auth_probe_state_map(); + let state = &shared.handshake.auth_probe; let Some(entry) = state.get(&peer_ip) else { return false; }; @@ -168,9 +466,13 @@ fn auth_probe_is_throttled(peer_ip: IpAddr, now: Instant) -> bool { now < entry.blocked_until } -fn auth_probe_saturation_grace_exhausted(peer_ip: IpAddr, now: Instant) -> bool { +fn auth_probe_saturation_grace_exhausted_in( + shared: &ProxySharedState, + peer_ip: IpAddr, + now: Instant, +) -> bool { let peer_ip = normalize_auth_probe_ip(peer_ip); - let state = auth_probe_state_map(); + let state = &shared.handshake.auth_probe; let Some(entry) = state.get(&peer_ip) else { return false; }; @@ -183,20 +485,28 @@ fn auth_probe_saturation_grace_exhausted(peer_ip: IpAddr, now: Instant) -> bool entry.fail_streak >= AUTH_PROBE_BACKOFF_START_FAILS + AUTH_PROBE_SATURATION_GRACE_FAILS } -fn auth_probe_should_apply_preauth_throttle(peer_ip: IpAddr, now: Instant) -> bool { - if !auth_probe_is_throttled(peer_ip, now) { +fn auth_probe_should_apply_preauth_throttle_in( + shared: &ProxySharedState, + peer_ip: IpAddr, + now: Instant, +) -> bool { + if !auth_probe_is_throttled_in(shared, peer_ip, now) { return false; } - if !auth_probe_saturation_is_throttled(now) { + if !auth_probe_saturation_is_throttled_in(shared, now) { return true; } - auth_probe_saturation_grace_exhausted(peer_ip, now) + auth_probe_saturation_grace_exhausted_in(shared, peer_ip, now) } -fn auth_probe_saturation_is_throttled(now: Instant) -> bool { - let mut guard = auth_probe_saturation_state_lock(); +fn auth_probe_saturation_is_throttled_in(shared: &ProxySharedState, now: Instant) -> bool { + let mut guard = shared + .handshake + .auth_probe_saturation + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()); let Some(state) = guard.as_mut() else { return false; @@ -214,8 +524,12 @@ fn auth_probe_saturation_is_throttled(now: Instant) -> bool { false } -fn auth_probe_note_saturation(now: Instant) { - let mut guard = auth_probe_saturation_state_lock(); +fn auth_probe_note_saturation_in(shared: &ProxySharedState, now: Instant) { + let mut guard = shared + .handshake + .auth_probe_saturation + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()); match guard.as_mut() { Some(state) @@ -237,13 +551,14 @@ fn auth_probe_note_saturation(now: Instant) { } } -fn auth_probe_record_failure(peer_ip: IpAddr, now: Instant) { +fn auth_probe_record_failure_in(shared: &ProxySharedState, peer_ip: IpAddr, now: Instant) { let peer_ip = normalize_auth_probe_ip(peer_ip); - let state = auth_probe_state_map(); - auth_probe_record_failure_with_state(state, peer_ip, now); + let state = &shared.handshake.auth_probe; + auth_probe_record_failure_with_state_in(shared, state, peer_ip, now); } -fn auth_probe_record_failure_with_state( +fn auth_probe_record_failure_with_state_in( + shared: &ProxySharedState, state: &DashMap, peer_ip: IpAddr, now: Instant, @@ -277,7 +592,7 @@ fn auth_probe_record_failure_with_state( while state.len() >= AUTH_PROBE_TRACK_MAX_ENTRIES { rounds += 1; if rounds > 8 { - auth_probe_note_saturation(now); + auth_probe_note_saturation_in(shared, now); let mut eviction_candidate: Option<(IpAddr, u32, Instant)> = None; for entry in state.iter().take(AUTH_PROBE_PRUNE_SCAN_LIMIT) { let key = *entry.key(); @@ -320,7 +635,7 @@ fn auth_probe_record_failure_with_state( } } else { let start_offset = - auth_probe_scan_start_offset(peer_ip, now, state_len, scan_limit); + auth_probe_scan_start_offset_in(shared, peer_ip, now, state_len, scan_limit); let mut scanned = 0usize; for entry in state.iter().skip(start_offset) { let key = *entry.key(); @@ -369,11 +684,11 @@ fn auth_probe_record_failure_with_state( } let Some((evict_key, _, _)) = eviction_candidate else { - auth_probe_note_saturation(now); + auth_probe_note_saturation_in(shared, now); return; }; state.remove(&evict_key); - auth_probe_note_saturation(now); + auth_probe_note_saturation_in(shared, now); } } @@ -387,89 +702,58 @@ fn auth_probe_record_failure_with_state( } } -fn auth_probe_record_success(peer_ip: IpAddr) { +fn auth_probe_record_success_in(shared: &ProxySharedState, peer_ip: IpAddr) { let peer_ip = normalize_auth_probe_ip(peer_ip); - let state = auth_probe_state_map(); + let state = &shared.handshake.auth_probe; state.remove(&peer_ip); } #[cfg(test)] -fn clear_auth_probe_state_for_testing() { - if let Some(state) = AUTH_PROBE_STATE.get() { - state.clear(); - } - if AUTH_PROBE_SATURATION_STATE.get().is_some() { - let mut guard = auth_probe_saturation_state_lock(); - *guard = None; - } +pub(crate) fn auth_probe_record_failure_for_testing( + shared: &ProxySharedState, + peer_ip: IpAddr, + now: Instant, +) { + auth_probe_record_failure_in(shared, peer_ip, now); } #[cfg(test)] -fn auth_probe_fail_streak_for_testing(peer_ip: IpAddr) -> Option { +pub(crate) fn auth_probe_fail_streak_for_testing_in_shared( + shared: &ProxySharedState, + peer_ip: IpAddr, +) -> Option { let peer_ip = normalize_auth_probe_ip(peer_ip); - let state = AUTH_PROBE_STATE.get()?; - state.get(&peer_ip).map(|entry| entry.fail_streak) + shared + .handshake + .auth_probe + .get(&peer_ip) + .map(|entry| entry.fail_streak) } #[cfg(test)] -fn auth_probe_is_throttled_for_testing(peer_ip: IpAddr) -> bool { - auth_probe_is_throttled(peer_ip, Instant::now()) -} - -#[cfg(test)] -fn auth_probe_saturation_is_throttled_for_testing() -> bool { - auth_probe_saturation_is_throttled(Instant::now()) -} - -#[cfg(test)] -fn auth_probe_saturation_is_throttled_at_for_testing(now: Instant) -> bool { - auth_probe_saturation_is_throttled(now) -} - -#[cfg(test)] -fn auth_probe_test_lock() -> &'static Mutex<()> { - static TEST_LOCK: OnceLock> = OnceLock::new(); - TEST_LOCK.get_or_init(|| Mutex::new(())) -} - -#[cfg(test)] -fn unknown_sni_warn_test_lock() -> &'static Mutex<()> { - static TEST_LOCK: OnceLock> = OnceLock::new(); - TEST_LOCK.get_or_init(|| Mutex::new(())) -} - -#[cfg(test)] -fn clear_unknown_sni_warn_state_for_testing() { - if UNKNOWN_SNI_WARN_NEXT_ALLOWED.get().is_some() { - let mut guard = unknown_sni_warn_state_lock(); - *guard = None; +pub(crate) fn clear_auth_probe_state_for_testing_in_shared(shared: &ProxySharedState) { + shared.handshake.auth_probe.clear(); + match shared.handshake.auth_probe_saturation.lock() { + Ok(mut saturation) => { + *saturation = None; + } + Err(poisoned) => { + let mut saturation = poisoned.into_inner(); + *saturation = None; + shared.handshake.auth_probe_saturation.clear_poison(); + } } } -#[cfg(test)] -fn should_emit_unknown_sni_warn_for_testing(now: Instant) -> bool { - should_emit_unknown_sni_warn(now) -} - -#[cfg(test)] -fn clear_warned_secrets_for_testing() { - if let Some(warned) = INVALID_SECRET_WARNED.get() - && let Ok(mut guard) = warned.lock() - { - guard.clear(); - } -} - -#[cfg(test)] -fn warned_secrets_test_lock() -> &'static Mutex<()> { - static TEST_LOCK: OnceLock> = OnceLock::new(); - TEST_LOCK.get_or_init(|| Mutex::new(())) -} - -fn warn_invalid_secret_once(name: &str, reason: &str, expected: usize, got: Option) { +fn warn_invalid_secret_once_in( + shared: &ProxySharedState, + name: &str, + reason: &str, + expected: usize, + got: Option, +) { let key = (name.to_string(), reason.to_string()); - let warned = INVALID_SECRET_WARNED.get_or_init(|| Mutex::new(HashSet::new())); - let should_warn = match warned.lock() { + let should_warn = match shared.handshake.invalid_secret_warned.lock() { Ok(mut guard) => { if !guard.contains(&key) && guard.len() >= WARNED_SECRET_MAX_ENTRIES { false @@ -502,11 +786,12 @@ fn warn_invalid_secret_once(name: &str, reason: &str, expected: usize, got: Opti } } -fn decode_user_secret(name: &str, secret_hex: &str) -> Option> { +fn decode_user_secret(shared: &ProxySharedState, name: &str, secret_hex: &str) -> Option> { match hex::decode(secret_hex) { Ok(bytes) if bytes.len() == ACCESS_SECRET_BYTES => Some(bytes), Ok(bytes) => { - warn_invalid_secret_once( + warn_invalid_secret_once_in( + shared, name, "invalid_length", ACCESS_SECRET_BYTES, @@ -515,7 +800,7 @@ fn decode_user_secret(name: &str, secret_hex: &str) -> Option> { None } Err(_) => { - warn_invalid_secret_once(name, "invalid_hex", ACCESS_SECRET_BYTES, None); + warn_invalid_secret_once_in(shared, name, "invalid_hex", ACCESS_SECRET_BYTES, None); None } } @@ -543,7 +828,8 @@ fn mode_enabled_for_proto(config: &ProxyConfig, proto_tag: ProtoTag, is_tls: boo } } -fn decode_user_secrets( +fn decode_user_secrets_in( + shared: &ProxySharedState, config: &ProxyConfig, preferred_user: Option<&str>, ) -> Vec<(String, Vec)> { @@ -551,7 +837,7 @@ fn decode_user_secrets( if let Some(preferred) = preferred_user && let Some(secret_hex) = config.access.users.get(preferred) - && let Some(bytes) = decode_user_secret(preferred, secret_hex) + && let Some(bytes) = decode_user_secret(shared, preferred, secret_hex) { secrets.push((preferred.to_string(), bytes)); } @@ -560,7 +846,7 @@ fn decode_user_secrets( if preferred_user.is_some_and(|preferred| preferred == name.as_str()) { continue; } - if let Some(bytes) = decode_user_secret(name, secret_hex) { + if let Some(bytes) = decode_user_secret(shared, name, secret_hex) { secrets.push((name.clone(), bytes)); } } @@ -568,6 +854,86 @@ fn decode_user_secrets( secrets } +#[cfg(test)] +pub(crate) fn auth_probe_state_for_testing_in_shared( + shared: &ProxySharedState, +) -> &DashMap { + &shared.handshake.auth_probe +} + +#[cfg(test)] +pub(crate) fn auth_probe_saturation_state_for_testing_in_shared( + shared: &ProxySharedState, +) -> &Mutex> { + &shared.handshake.auth_probe_saturation +} + +#[cfg(test)] +pub(crate) fn auth_probe_saturation_state_lock_for_testing_in_shared( + shared: &ProxySharedState, +) -> std::sync::MutexGuard<'_, Option> { + shared + .handshake + .auth_probe_saturation + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()) +} + +#[cfg(test)] +pub(crate) fn clear_unknown_sni_warn_state_for_testing_in_shared(shared: &ProxySharedState) { + let mut guard = shared + .handshake + .unknown_sni_warn_next_allowed + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()); + *guard = None; +} + +#[cfg(test)] +pub(crate) fn should_emit_unknown_sni_warn_for_testing_in_shared( + shared: &ProxySharedState, + now: Instant, +) -> bool { + should_emit_unknown_sni_warn_in(shared, now) +} + +#[cfg(test)] +pub(crate) fn clear_warned_secrets_for_testing_in_shared(shared: &ProxySharedState) { + if let Ok(mut guard) = shared.handshake.invalid_secret_warned.lock() { + guard.clear(); + } +} + +#[cfg(test)] +pub(crate) fn warned_secrets_for_testing_in_shared( + shared: &ProxySharedState, +) -> &Mutex> { + &shared.handshake.invalid_secret_warned +} + +#[cfg(test)] +pub(crate) fn auth_probe_is_throttled_for_testing_in_shared( + shared: &ProxySharedState, + peer_ip: IpAddr, +) -> bool { + auth_probe_is_throttled_in(shared, peer_ip, Instant::now()) +} + +#[cfg(test)] +pub(crate) fn auth_probe_saturation_is_throttled_for_testing_in_shared( + shared: &ProxySharedState, +) -> bool { + auth_probe_saturation_is_throttled_in(shared, Instant::now()) +} + +#[cfg(test)] +pub(crate) fn auth_probe_saturation_is_throttled_at_for_testing_in_shared( + shared: &ProxySharedState, + now: Instant, +) -> bool { + auth_probe_saturation_is_throttled_in(shared, now) +} + #[inline] fn find_matching_tls_domain<'a>(config: &'a ProxyConfig, sni: &str) -> Option<&'a str> { if config.censorship.tls_domain.eq_ignore_ascii_case(sni) { @@ -593,7 +959,7 @@ async fn maybe_apply_server_hello_delay(config: &ProxyConfig) { let delay_ms = if max == min { max } else { - rand::rng().random_range(min..=max) + crate::proxy::masking::sample_lognormal_percentile_bounded(min, max, &mut rand::rng()) }; if delay_ms > 0 { @@ -635,6 +1001,7 @@ impl Drop for HandshakeSuccess { } /// Handle fake TLS handshake +#[cfg(test)] pub async fn handle_tls_handshake( handshake: &[u8], reader: R, @@ -645,6 +1012,65 @@ pub async fn handle_tls_handshake( rng: &SecureRandom, tls_cache: Option>, ) -> HandshakeResult<(FakeTlsReader, FakeTlsWriter, String), R, W> +where + R: AsyncRead + Unpin, + W: AsyncWrite + Unpin, +{ + let shared = ProxySharedState::new(); + handle_tls_handshake_impl( + handshake, + reader, + writer, + peer, + config, + replay_checker, + rng, + tls_cache, + shared.as_ref(), + ) + .await +} + +pub async fn handle_tls_handshake_with_shared( + handshake: &[u8], + reader: R, + writer: W, + peer: SocketAddr, + config: &ProxyConfig, + replay_checker: &ReplayChecker, + rng: &SecureRandom, + tls_cache: Option>, + shared: &ProxySharedState, +) -> HandshakeResult<(FakeTlsReader, FakeTlsWriter, String), R, W> +where + R: AsyncRead + Unpin, + W: AsyncWrite + Unpin, +{ + handle_tls_handshake_impl( + handshake, + reader, + writer, + peer, + config, + replay_checker, + rng, + tls_cache, + shared, + ) + .await +} + +async fn handle_tls_handshake_impl( + handshake: &[u8], + reader: R, + mut writer: W, + peer: SocketAddr, + config: &ProxyConfig, + replay_checker: &ReplayChecker, + rng: &SecureRandom, + tls_cache: Option>, + shared: &ProxySharedState, +) -> HandshakeResult<(FakeTlsReader, FakeTlsWriter, String), R, W> where R: AsyncRead + Unpin, W: AsyncWrite + Unpin, @@ -652,14 +1078,14 @@ where debug!(peer = %peer, handshake_len = handshake.len(), "Processing TLS handshake"); let throttle_now = Instant::now(); - if auth_probe_should_apply_preauth_throttle(peer.ip(), throttle_now) { + if auth_probe_should_apply_preauth_throttle_in(shared, peer.ip(), throttle_now) { maybe_apply_server_hello_delay(config).await; debug!(peer = %peer, "TLS handshake rejected by pre-auth probe throttle"); return HandshakeResult::BadClient { reader, writer }; } if handshake.len() < tls::TLS_DIGEST_POS + tls::TLS_DIGEST_LEN + 1 { - auth_probe_record_failure(peer.ip(), Instant::now()); + auth_probe_record_failure_in(shared, peer.ip(), Instant::now()); maybe_apply_server_hello_delay(config).await; debug!(peer = %peer, "TLS handshake too short"); return HandshakeResult::BadClient { reader, writer }; @@ -695,71 +1121,283 @@ where }; if client_sni.is_some() && matched_tls_domain.is_none() && preferred_user_hint.is_none() { - auth_probe_record_failure(peer.ip(), Instant::now()); - maybe_apply_server_hello_delay(config).await; let sni = client_sni.as_deref().unwrap_or_default(); - let log_now = Instant::now(); - if should_emit_unknown_sni_warn(log_now) { - warn!( - peer = %peer, - sni = %sni, - unknown_sni = true, - unknown_sni_action = ?config.censorship.unknown_sni_action, - "TLS handshake rejected by unknown SNI policy" - ); - } else { - info!( - peer = %peer, - sni = %sni, - unknown_sni = true, - unknown_sni_action = ?config.censorship.unknown_sni_action, - "TLS handshake rejected by unknown SNI policy" - ); + match config.censorship.unknown_sni_action { + UnknownSniAction::Accept => { + debug!( + peer = %peer, + sni = %sni, + unknown_sni = true, + unknown_sni_action = ?config.censorship.unknown_sni_action, + "TLS handshake accepted by unknown SNI policy" + ); + } + action @ (UnknownSniAction::Drop | UnknownSniAction::Mask) => { + auth_probe_record_failure_in(shared, peer.ip(), Instant::now()); + maybe_apply_server_hello_delay(config).await; + let log_now = Instant::now(); + if should_emit_unknown_sni_warn_in(shared, log_now) { + warn!( + peer = %peer, + sni = %sni, + unknown_sni = true, + unknown_sni_action = ?action, + "TLS handshake rejected by unknown SNI policy" + ); + } else { + info!( + peer = %peer, + sni = %sni, + unknown_sni = true, + unknown_sni_action = ?action, + "TLS handshake rejected by unknown SNI policy" + ); + } + return match action { + UnknownSniAction::Drop => HandshakeResult::Error(ProxyError::UnknownTlsSni), + UnknownSniAction::Mask => HandshakeResult::BadClient { reader, writer }, + UnknownSniAction::Accept => unreachable!(), + }; + } } - return match config.censorship.unknown_sni_action { - UnknownSniAction::Drop => HandshakeResult::Error(ProxyError::UnknownTlsSni), - UnknownSniAction::Mask => HandshakeResult::BadClient { reader, writer }, - }; } - let secrets = decode_user_secrets(config, preferred_user_hint); + let mut validation_digest = [0u8; tls::TLS_DIGEST_LEN]; + let mut validation_session_id = [0u8; 32]; + let mut validation_session_id_len = 0usize; + let mut validated_user = String::new(); + let mut validated_secret = [0u8; ACCESS_SECRET_BYTES]; + let mut validated_user_id: Option = None; - let validation = match tls::validate_tls_handshake_with_replay_window( - handshake, - &secrets, - config.access.ignore_time_skew, - config.access.replay_window_secs, - ) { - Some(v) => v, - None => { - auth_probe_record_failure(peer.ip(), Instant::now()); + if let Some(snapshot) = config.runtime_user_auth() { + let parsed = match parse_tls_auth_material( + handshake, + config.access.ignore_time_skew, + config.access.replay_window_secs, + ) { + Some(parsed) => parsed, + None => { + auth_probe_record_failure_in(shared, peer.ip(), Instant::now()); + maybe_apply_server_hello_delay(config).await; + debug!(peer = %peer, "TLS handshake auth material parsing failed"); + return HandshakeResult::BadClient { reader, writer }; + } + }; + + let sticky_ip_hint = sticky_hint_get_by_ip(shared, peer.ip()); + let preferred_user_id = preferred_user_hint.and_then(|user| snapshot.user_id_by_name(user)); + let sticky_sni_hint = client_sni + .as_deref() + .and_then(|sni| sticky_hint_get_by_sni(shared, sni)); + let sticky_prefix_hint = sticky_hint_get_by_ip_prefix(shared, peer.ip()); + let sni_candidates = client_sni + .as_deref() + .and_then(|sni| snapshot.sni_candidates(sni)); + let sni_initial_candidates = client_sni + .as_deref() + .and_then(|sni| snapshot.sni_initial_candidates(sni)); + + let has_hint = sticky_ip_hint.is_some() + || preferred_user_id.is_some() + || sticky_sni_hint.is_some() + || sticky_prefix_hint.is_some() + || sni_candidates.is_some_and(|ids| !ids.is_empty()) + || sni_initial_candidates.is_some_and(|ids| !ids.is_empty()); + let overload = auth_probe_saturation_is_throttled_in(shared, Instant::now()); + let candidate_budget = budget_for_validation(snapshot.entries().len(), overload, has_hint); + + let mut tried_user_ids = [u32::MAX; CANDIDATE_HINT_TRACK_CAP]; + let mut tried_len = 0usize; + let mut validation_checks = 0usize; + let mut budget_exhausted = false; + + macro_rules! try_user_id { + ($user_id:expr) => {{ + if validation_checks >= candidate_budget { + budget_exhausted = true; + false + } else if !mark_candidate_if_new(&mut tried_user_ids, &mut tried_len, $user_id) { + false + } else if let Some(entry) = snapshot.entry_by_id($user_id) { + validation_checks = validation_checks.saturating_add(1); + if let Some(candidate) = + validate_tls_secret_candidate(&parsed, handshake, &entry.secret) + { + validation_digest = candidate.digest; + validation_session_id = candidate.session_id; + validation_session_id_len = candidate.session_id_len; + validated_secret.copy_from_slice(&entry.secret); + validated_user = entry.user.clone(); + validated_user_id = Some($user_id); + true + } else { + false + } + } else { + false + } + }}; + } + + let mut matched = false; + if let Some(user_id) = sticky_ip_hint { + matched = try_user_id!(user_id); + } + + if !matched && let Some(user_id) = preferred_user_id { + matched = try_user_id!(user_id); + } + + if !matched && let Some(user_id) = sticky_sni_hint { + matched = try_user_id!(user_id); + } + + if !matched && let Some(user_id) = sticky_prefix_hint { + matched = try_user_id!(user_id); + } + + if !matched + && !budget_exhausted + && let Some(candidate_ids) = sni_candidates + { + for &user_id in candidate_ids { + if try_user_id!(user_id) { + matched = true; + break; + } + if budget_exhausted { + break; + } + } + } + + if !matched + && !budget_exhausted + && let Some(candidate_ids) = sni_initial_candidates + { + for &user_id in candidate_ids { + if try_user_id!(user_id) { + matched = true; + break; + } + if budget_exhausted { + break; + } + } + } + + if !matched && !budget_exhausted { + let ring = &shared.handshake.recent_user_ring; + if !ring.is_empty() { + let next_seq = shared + .handshake + .recent_user_ring_seq + .load(Ordering::Relaxed); + let scan_limit = ring.len().min(RECENT_USER_RING_SCAN_LIMIT); + for offset in 0..scan_limit { + let idx = (next_seq as usize + ring.len() - 1 - offset) % ring.len(); + let encoded_user_id = ring[idx].load(Ordering::Relaxed); + if encoded_user_id == 0 { + continue; + } + if try_user_id!(encoded_user_id - 1) { + matched = true; + break; + } + if budget_exhausted { + break; + } + } + } + } + + if !matched && !budget_exhausted { + for idx in 0..snapshot.entries().len() { + let Some(user_id) = u32::try_from(idx).ok() else { + break; + }; + if try_user_id!(user_id) { + matched = true; + break; + } + if budget_exhausted { + break; + } + } + } + + shared + .handshake + .auth_expensive_checks_total + .fetch_add(validation_checks as u64, Ordering::Relaxed); + if budget_exhausted { + shared + .handshake + .auth_budget_exhausted_total + .fetch_add(1, Ordering::Relaxed); + } + + if !matched { + auth_probe_record_failure_in(shared, peer.ip(), Instant::now()); maybe_apply_server_hello_delay(config).await; debug!( peer = %peer, ignore_time_skew = config.access.ignore_time_skew, - "TLS handshake validation failed - no matching user or time skew" + budget_exhausted = budget_exhausted, + candidate_budget = candidate_budget, + validation_checks = validation_checks, + "TLS handshake validation failed - no matching user, time skew, or budget exhausted" ); return HandshakeResult::BadClient { reader, writer }; } - }; + } else { + let secrets = decode_user_secrets_in(shared, config, preferred_user_hint); + let validation = match tls::validate_tls_handshake_with_replay_window( + handshake, + &secrets, + config.access.ignore_time_skew, + config.access.replay_window_secs, + ) { + Some(v) => v, + None => { + auth_probe_record_failure_in(shared, peer.ip(), Instant::now()); + maybe_apply_server_hello_delay(config).await; + debug!( + peer = %peer, + ignore_time_skew = config.access.ignore_time_skew, + "TLS handshake validation failed - no matching user or time skew" + ); + return HandshakeResult::BadClient { reader, writer }; + } + }; + let secret = match secrets.iter().find(|(name, _)| *name == validation.user) { + Some((_, s)) if s.len() == ACCESS_SECRET_BYTES => s, + _ => { + maybe_apply_server_hello_delay(config).await; + return HandshakeResult::BadClient { reader, writer }; + } + }; + + validation_digest = validation.digest; + validation_session_id_len = validation.session_id.len(); + if validation_session_id_len > validation_session_id.len() { + maybe_apply_server_hello_delay(config).await; + return HandshakeResult::BadClient { reader, writer }; + } + validation_session_id[..validation_session_id_len].copy_from_slice(&validation.session_id); + validated_user = validation.user; + validated_secret.copy_from_slice(secret); + } // Reject known replay digests before expensive cache/domain/ALPN policy work. - let digest_half = &validation.digest[..tls::TLS_DIGEST_HALF_LEN]; + let digest_half = &validation_digest[..tls::TLS_DIGEST_HALF_LEN]; if replay_checker.check_tls_digest(digest_half) { - auth_probe_record_failure(peer.ip(), Instant::now()); + auth_probe_record_failure_in(shared, peer.ip(), Instant::now()); maybe_apply_server_hello_delay(config).await; warn!(peer = %peer, "TLS replay attack detected (duplicate digest)"); return HandshakeResult::BadClient { reader, writer }; } - let secret = match secrets.iter().find(|(name, _)| *name == validation.user) { - Some((_, s)) => s, - None => { - maybe_apply_server_hello_delay(config).await; - return HandshakeResult::BadClient { reader, writer }; - } - }; - let cached = if config.censorship.tls_emulation { if let Some(cache) = tls_cache.as_ref() { let selected_domain = @@ -782,11 +1420,13 @@ where // Add replay digest only for policy-valid handshakes. replay_checker.add_tls_digest(digest_half); + let validation_session_id_slice = &validation_session_id[..validation_session_id_len]; + let response = if let Some((cached_entry, use_full_cert_payload)) = cached { emulator::build_emulated_server_hello( - secret, - &validation.digest, - &validation.session_id, + &validated_secret, + &validation_digest, + validation_session_id_slice, &cached_entry, use_full_cert_payload, rng, @@ -795,9 +1435,9 @@ where ) } else { tls::build_server_hello( - secret, - &validation.digest, - &validation.session_id, + &validated_secret, + &validation_digest, + validation_session_id_slice, config.censorship.fake_cert_len, rng, selected_alpn.clone(), @@ -823,20 +1463,26 @@ where debug!( peer = %peer, - user = %validation.user, + user = %validated_user, "TLS handshake successful" ); - auth_probe_record_success(peer.ip()); + auth_probe_record_success_in(shared, peer.ip()); + + if let Some(user_id) = validated_user_id { + sticky_hint_record_success_in(shared, peer.ip(), user_id, client_sni.as_deref()); + record_recent_user_success_in(shared, user_id); + } HandshakeResult::Success(( FakeTlsReader::new(reader), FakeTlsWriter::new(writer), - validation.user, + validated_user, )) } /// Handle MTProto obfuscation handshake +#[cfg(test)] pub async fn handle_mtproto_handshake( handshake: &[u8; HANDSHAKE_LEN], reader: R, @@ -847,6 +1493,65 @@ pub async fn handle_mtproto_handshake( is_tls: bool, preferred_user: Option<&str>, ) -> HandshakeResult<(CryptoReader, CryptoWriter, HandshakeSuccess), R, W> +where + R: AsyncRead + Unpin + Send, + W: AsyncWrite + Unpin + Send, +{ + let shared = ProxySharedState::new(); + handle_mtproto_handshake_impl( + handshake, + reader, + writer, + peer, + config, + replay_checker, + is_tls, + preferred_user, + shared.as_ref(), + ) + .await +} + +pub async fn handle_mtproto_handshake_with_shared( + handshake: &[u8; HANDSHAKE_LEN], + reader: R, + writer: W, + peer: SocketAddr, + config: &ProxyConfig, + replay_checker: &ReplayChecker, + is_tls: bool, + preferred_user: Option<&str>, + shared: &ProxySharedState, +) -> HandshakeResult<(CryptoReader, CryptoWriter, HandshakeSuccess), R, W> +where + R: AsyncRead + Unpin + Send, + W: AsyncWrite + Unpin + Send, +{ + handle_mtproto_handshake_impl( + handshake, + reader, + writer, + peer, + config, + replay_checker, + is_tls, + preferred_user, + shared, + ) + .await +} + +async fn handle_mtproto_handshake_impl( + handshake: &[u8; HANDSHAKE_LEN], + reader: R, + writer: W, + peer: SocketAddr, + config: &ProxyConfig, + replay_checker: &ReplayChecker, + is_tls: bool, + preferred_user: Option<&str>, + shared: &ProxySharedState, +) -> HandshakeResult<(CryptoReader, CryptoWriter, HandshakeSuccess), R, W> where R: AsyncRead + Unpin + Send, W: AsyncWrite + Unpin + Send, @@ -862,68 +1567,157 @@ where ); let throttle_now = Instant::now(); - if auth_probe_should_apply_preauth_throttle(peer.ip(), throttle_now) { + if auth_probe_should_apply_preauth_throttle_in(shared, peer.ip(), throttle_now) { maybe_apply_server_hello_delay(config).await; debug!(peer = %peer, "MTProto handshake rejected by pre-auth probe throttle"); return HandshakeResult::BadClient { reader, writer }; } let dec_prekey_iv = &handshake[SKIP_LEN..SKIP_LEN + PREKEY_LEN + IV_LEN]; + let mut dec_prekey = [0u8; PREKEY_LEN]; + dec_prekey.copy_from_slice(&dec_prekey_iv[..PREKEY_LEN]); + let mut dec_iv_arr = [0u8; IV_LEN]; + dec_iv_arr.copy_from_slice(&dec_prekey_iv[PREKEY_LEN..]); + let dec_iv = u128::from_be_bytes(dec_iv_arr); - let enc_prekey_iv: Vec = dec_prekey_iv.iter().rev().copied().collect(); + let mut enc_prekey_iv = [0u8; PREKEY_LEN + IV_LEN]; + for idx in 0..enc_prekey_iv.len() { + enc_prekey_iv[idx] = dec_prekey_iv[dec_prekey_iv.len() - 1 - idx]; + } + let mut enc_prekey = [0u8; PREKEY_LEN]; + enc_prekey.copy_from_slice(&enc_prekey_iv[..PREKEY_LEN]); + let mut enc_iv_arr = [0u8; IV_LEN]; + enc_iv_arr.copy_from_slice(&enc_prekey_iv[PREKEY_LEN..]); + let enc_iv = u128::from_be_bytes(enc_iv_arr); - let decoded_users = decode_user_secrets(config, preferred_user); + if let Some(snapshot) = config.runtime_user_auth() { + let sticky_ip_hint = sticky_hint_get_by_ip(shared, peer.ip()); + let sticky_prefix_hint = sticky_hint_get_by_ip_prefix(shared, peer.ip()); + let preferred_user_id = preferred_user.and_then(|user| snapshot.user_id_by_name(user)); + let has_hint = + sticky_ip_hint.is_some() || sticky_prefix_hint.is_some() || preferred_user_id.is_some(); + let overload = auth_probe_saturation_is_throttled_in(shared, Instant::now()); + let candidate_budget = budget_for_validation(snapshot.entries().len(), overload, has_hint); - for (user, secret) in decoded_users { - let dec_prekey = &dec_prekey_iv[..PREKEY_LEN]; - let dec_iv_bytes = &dec_prekey_iv[PREKEY_LEN..]; + let mut tried_user_ids = [u32::MAX; CANDIDATE_HINT_TRACK_CAP]; + let mut tried_len = 0usize; + let mut validation_checks = 0usize; + let mut budget_exhausted = false; - let mut dec_key_input = Zeroizing::new(Vec::with_capacity(PREKEY_LEN + secret.len())); - dec_key_input.extend_from_slice(dec_prekey); - dec_key_input.extend_from_slice(&secret); - let dec_key = Zeroizing::new(sha256(&dec_key_input)); + let mut matched_user = String::new(); + let mut matched_user_id = None; + let mut matched_validation = None; - let mut dec_iv_arr = [0u8; IV_LEN]; - dec_iv_arr.copy_from_slice(dec_iv_bytes); - let dec_iv = u128::from_be_bytes(dec_iv_arr); - - let mut decryptor = AesCtr::new(&dec_key, dec_iv); - let decrypted = decryptor.decrypt(handshake); - - let tag_bytes: [u8; 4] = [ - decrypted[PROTO_TAG_POS], - decrypted[PROTO_TAG_POS + 1], - decrypted[PROTO_TAG_POS + 2], - decrypted[PROTO_TAG_POS + 3], - ]; - - let proto_tag = match ProtoTag::from_bytes(tag_bytes) { - Some(tag) => tag, - None => continue, - }; - - let mode_ok = mode_enabled_for_proto(config, proto_tag, is_tls); - - if !mode_ok { - debug!(peer = %peer, user = %user, proto = ?proto_tag, "Mode not enabled"); - continue; + macro_rules! try_user_id { + ($user_id:expr) => {{ + if validation_checks >= candidate_budget { + budget_exhausted = true; + false + } else if !mark_candidate_if_new(&mut tried_user_ids, &mut tried_len, $user_id) { + false + } else if let Some(entry) = snapshot.entry_by_id($user_id) { + validation_checks = validation_checks.saturating_add(1); + if let Some(validation) = validate_mtproto_secret_candidate( + handshake, + &dec_prekey, + dec_iv, + &enc_prekey, + enc_iv, + &entry.secret, + config, + is_tls, + ) { + matched_user = entry.user.clone(); + matched_user_id = Some($user_id); + matched_validation = Some(validation); + true + } else { + false + } + } else { + false + } + }}; } - let dc_idx = i16::from_le_bytes([decrypted[DC_IDX_POS], decrypted[DC_IDX_POS + 1]]); + let mut matched = false; + if let Some(user_id) = sticky_ip_hint { + matched = try_user_id!(user_id); + } - let enc_prekey = &enc_prekey_iv[..PREKEY_LEN]; - let enc_iv_bytes = &enc_prekey_iv[PREKEY_LEN..]; + if !matched && let Some(user_id) = preferred_user_id { + matched = try_user_id!(user_id); + } - let mut enc_key_input = Zeroizing::new(Vec::with_capacity(PREKEY_LEN + secret.len())); - enc_key_input.extend_from_slice(enc_prekey); - enc_key_input.extend_from_slice(&secret); - let enc_key = Zeroizing::new(sha256(&enc_key_input)); + if !matched && let Some(user_id) = sticky_prefix_hint { + matched = try_user_id!(user_id); + } - let mut enc_iv_arr = [0u8; IV_LEN]; - enc_iv_arr.copy_from_slice(enc_iv_bytes); - let enc_iv = u128::from_be_bytes(enc_iv_arr); + if !matched && !budget_exhausted { + let ring = &shared.handshake.recent_user_ring; + if !ring.is_empty() { + let next_seq = shared + .handshake + .recent_user_ring_seq + .load(Ordering::Relaxed); + let scan_limit = ring.len().min(RECENT_USER_RING_SCAN_LIMIT); + for offset in 0..scan_limit { + let idx = (next_seq as usize + ring.len() - 1 - offset) % ring.len(); + let encoded_user_id = ring[idx].load(Ordering::Relaxed); + if encoded_user_id == 0 { + continue; + } + if try_user_id!(encoded_user_id - 1) { + matched = true; + break; + } + if budget_exhausted { + break; + } + } + } + } - let encryptor = AesCtr::new(&enc_key, enc_iv); + if !matched && !budget_exhausted { + for idx in 0..snapshot.entries().len() { + let Some(user_id) = u32::try_from(idx).ok() else { + break; + }; + if try_user_id!(user_id) { + matched = true; + break; + } + if budget_exhausted { + break; + } + } + } + + shared + .handshake + .auth_expensive_checks_total + .fetch_add(validation_checks as u64, Ordering::Relaxed); + if budget_exhausted { + shared + .handshake + .auth_budget_exhausted_total + .fetch_add(1, Ordering::Relaxed); + } + + if !matched { + auth_probe_record_failure_in(shared, peer.ip(), Instant::now()); + maybe_apply_server_hello_delay(config).await; + debug!( + peer = %peer, + budget_exhausted = budget_exhausted, + candidate_budget = candidate_budget, + validation_checks = validation_checks, + "MTProto handshake: no matching user found" + ); + return HandshakeResult::BadClient { reader, writer }; + } + + let validation = matched_validation.expect("validation must exist when matched"); // Apply replay tracking only after successful authentication. // @@ -932,44 +1726,130 @@ where // entry from the cache. We accept the cost of performing the full // authentication check first to avoid poisoning the replay cache. if replay_checker.check_and_add_handshake(dec_prekey_iv) { - auth_probe_record_failure(peer.ip(), Instant::now()); + auth_probe_record_failure_in(shared, peer.ip(), Instant::now()); maybe_apply_server_hello_delay(config).await; - warn!(peer = %peer, user = %user, "MTProto replay attack detected"); + warn!(peer = %peer, user = %matched_user, "MTProto replay attack detected"); return HandshakeResult::BadClient { reader, writer }; } + let dec_key = Zeroizing::new(validation.dec_key); + let enc_key = Zeroizing::new(validation.enc_key); let success = HandshakeSuccess { - user: user.clone(), - dc_idx, - proto_tag, + user: matched_user.clone(), + dc_idx: validation.dc_idx, + proto_tag: validation.proto_tag, dec_key: *dec_key, - dec_iv, + dec_iv: validation.dec_iv, enc_key: *enc_key, - enc_iv, + enc_iv: validation.enc_iv, peer, is_tls, }; debug!( peer = %peer, - user = %user, - dc = dc_idx, - proto = ?proto_tag, + user = %matched_user, + dc = validation.dc_idx, + proto = ?validation.proto_tag, tls = is_tls, "MTProto handshake successful" ); - auth_probe_record_success(peer.ip()); + auth_probe_record_success_in(shared, peer.ip()); + if let Some(user_id) = matched_user_id { + sticky_hint_record_success_in(shared, peer.ip(), user_id, None); + record_recent_user_success_in(shared, user_id); + } let max_pending = config.general.crypto_pending_buffer; return HandshakeResult::Success(( - CryptoReader::new(reader, decryptor), - CryptoWriter::new(writer, encryptor, max_pending), + CryptoReader::new(reader, validation.decryptor), + CryptoWriter::new(writer, validation.encryptor, max_pending), success, )); + } else { + let decoded_users = decode_user_secrets_in(shared, config, preferred_user); + let mut validation_checks = 0usize; + + for (user, secret) in decoded_users { + if secret.len() != ACCESS_SECRET_BYTES { + continue; + } + validation_checks = validation_checks.saturating_add(1); + + let mut secret_arr = [0u8; ACCESS_SECRET_BYTES]; + secret_arr.copy_from_slice(&secret); + let Some(validation) = validate_mtproto_secret_candidate( + handshake, + &dec_prekey, + dec_iv, + &enc_prekey, + enc_iv, + &secret_arr, + config, + is_tls, + ) else { + continue; + }; + + shared + .handshake + .auth_expensive_checks_total + .fetch_add(validation_checks as u64, Ordering::Relaxed); + + // Apply replay tracking only after successful authentication. + // + // This ordering prevents an attacker from producing invalid handshakes that + // still collide with a valid handshake's replay slot and thus evict a valid + // entry from the cache. We accept the cost of performing the full + // authentication check first to avoid poisoning the replay cache. + if replay_checker.check_and_add_handshake(dec_prekey_iv) { + auth_probe_record_failure_in(shared, peer.ip(), Instant::now()); + maybe_apply_server_hello_delay(config).await; + warn!(peer = %peer, user = %user, "MTProto replay attack detected"); + return HandshakeResult::BadClient { reader, writer }; + } + + let dec_key = Zeroizing::new(validation.dec_key); + let enc_key = Zeroizing::new(validation.enc_key); + let success = HandshakeSuccess { + user: user.clone(), + dc_idx: validation.dc_idx, + proto_tag: validation.proto_tag, + dec_key: *dec_key, + dec_iv: validation.dec_iv, + enc_key: *enc_key, + enc_iv: validation.enc_iv, + peer, + is_tls, + }; + + debug!( + peer = %peer, + user = %user, + dc = validation.dc_idx, + proto = ?validation.proto_tag, + tls = is_tls, + "MTProto handshake successful" + ); + + auth_probe_record_success_in(shared, peer.ip()); + + let max_pending = config.general.crypto_pending_buffer; + return HandshakeResult::Success(( + CryptoReader::new(reader, validation.decryptor), + CryptoWriter::new(writer, validation.encryptor, max_pending), + success, + )); + } + + shared + .handshake + .auth_expensive_checks_total + .fetch_add(validation_checks as u64, Ordering::Relaxed); } - auth_probe_record_failure(peer.ip(), Instant::now()); + auth_probe_record_failure_in(shared, peer.ip(), Instant::now()); maybe_apply_server_hello_delay(config).await; debug!(peer = %peer, "MTProto handshake: no matching user found"); HandshakeResult::BadClient { reader, writer } @@ -1123,6 +2003,10 @@ mod timing_manual_bench_tests; #[path = "tests/handshake_key_material_zeroization_security_tests.rs"] mod handshake_key_material_zeroization_security_tests; +#[cfg(test)] +#[path = "tests/handshake_baseline_invariant_tests.rs"] +mod handshake_baseline_invariant_tests; + /// Compile-time guard: HandshakeSuccess holds cryptographic key material and /// must never be Copy. A Copy impl would allow silent key duplication, /// undermining the zeroize-on-drop guarantee. diff --git a/src/proxy/masking.rs b/src/proxy/masking.rs index ba9f20a..70e72a0 100644 --- a/src/proxy/masking.rs +++ b/src/proxy/masking.rs @@ -249,6 +249,43 @@ async fn wait_mask_connect_budget(started: Instant) { } } +// Log-normal sample bounded to [floor, ceiling]. Median = sqrt(floor * ceiling). +// Implements Box-Muller transform for standard normal sampling — no external +// dependency on rand_distr (which is incompatible with rand 0.10). +// sigma is chosen so ~99% of raw samples land inside [floor, ceiling] before clamp. +// When floor > ceiling (misconfiguration), returns ceiling (the smaller value). +// When floor == ceiling, returns that value. When both are 0, returns 0. +pub(crate) fn sample_lognormal_percentile_bounded( + floor: u64, + ceiling: u64, + rng: &mut impl Rng, +) -> u64 { + if ceiling == 0 && floor == 0 { + return 0; + } + if floor > ceiling { + return ceiling; + } + if floor == ceiling { + return floor; + } + let floor_f = floor.max(1) as f64; + let ceiling_f = ceiling.max(1) as f64; + let mu = (floor_f.ln() + ceiling_f.ln()) / 2.0; + // 4.65 ≈ 2 * 2.326 (double-sided z-score for 99th percentile) + let sigma = ((ceiling_f / floor_f).ln() / 4.65).max(0.01); + // Box-Muller transform: two uniform samples → one standard normal sample + let u1: f64 = rng.random_range(f64::MIN_POSITIVE..1.0); + let u2: f64 = rng.random_range(0.0_f64..std::f64::consts::TAU); + let normal_sample = (-2.0_f64 * u1.ln()).sqrt() * u2.cos(); + let raw = (mu + sigma * normal_sample).exp(); + if raw.is_finite() { + (raw as u64).clamp(floor, ceiling) + } else { + ((floor_f * ceiling_f).sqrt()) as u64 + } +} + fn mask_outcome_target_budget(config: &ProxyConfig) -> Duration { if config.censorship.mask_timing_normalization_enabled { let floor = config.censorship.mask_timing_normalization_floor_ms; @@ -257,14 +294,18 @@ fn mask_outcome_target_budget(config: &ProxyConfig) -> Duration { if ceiling == 0 { return Duration::from_millis(0); } + // floor=0 stays uniform: log-normal cannot model distribution anchored at zero let mut rng = rand::rng(); return Duration::from_millis(rng.random_range(0..=ceiling)); } if ceiling > floor { let mut rng = rand::rng(); - return Duration::from_millis(rng.random_range(floor..=ceiling)); + return Duration::from_millis(sample_lognormal_percentile_bounded( + floor, ceiling, &mut rng, + )); } - return Duration::from_millis(floor); + // ceiling <= floor: use the larger value (fail-closed: preserve longer delay) + return Duration::from_millis(floor.max(ceiling)); } MASK_TIMEOUT @@ -1003,3 +1044,11 @@ mod masking_padding_timeout_adversarial_tests; #[cfg(all(test, feature = "redteam_offline_expected_fail"))] #[path = "tests/masking_offline_target_redteam_expected_fail_tests.rs"] mod masking_offline_target_redteam_expected_fail_tests; + +#[cfg(test)] +#[path = "tests/masking_baseline_invariant_tests.rs"] +mod masking_baseline_invariant_tests; + +#[cfg(test)] +#[path = "tests/masking_lognormal_timing_security_tests.rs"] +mod masking_lognormal_timing_security_tests; diff --git a/src/proxy/middle_relay.rs b/src/proxy/middle_relay.rs index d2f37a6..665e90e 100644 --- a/src/proxy/middle_relay.rs +++ b/src/proxy/middle_relay.rs @@ -1,20 +1,22 @@ -use std::collections::hash_map::RandomState; +#[cfg(test)] +use std::collections::hash_map::DefaultHasher; use std::collections::{BTreeSet, HashMap}; #[cfg(test)] use std::future::Future; +#[cfg(test)] +use std::hash::Hasher; use std::hash::{BuildHasher, Hash}; use std::net::{IpAddr, SocketAddr}; +use std::sync::Arc; use std::sync::atomic::{AtomicU64, Ordering}; -use std::sync::{Arc, Mutex, OnceLock}; use std::time::{Duration, Instant}; -use dashmap::DashMap; use tokio::io::{AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt}; use tokio::sync::{mpsc, oneshot, watch}; use tokio::time::timeout; use tracing::{debug, info, trace, warn}; -use crate::config::ProxyConfig; +use crate::config::{ConntrackPressureProfile, ProxyConfig}; use crate::crypto::SecureRandom; use crate::error::{ProxyError, Result}; use crate::protocol::constants::{secure_padding_len, *}; @@ -23,6 +25,9 @@ use crate::proxy::route_mode::{ ROUTE_SWITCH_ERROR_MSG, RelayRouteMode, RouteCutoverState, affected_cutover_state, cutover_stagger_delay, }; +use crate::proxy::shared_state::{ + ConntrackCloseEvent, ConntrackClosePublishResult, ConntrackCloseReason, ProxySharedState, +}; use crate::stats::{ MeD2cFlushReason, MeD2cQuotaRejectStage, MeD2cWriteMode, QuotaReserveError, Stats, UserStats, }; @@ -51,19 +56,9 @@ const ME_D2C_FLUSH_BATCH_MAX_BYTES_MIN: usize = 4096; const ME_D2C_FRAME_BUF_SHRINK_HYSTERESIS_FACTOR: usize = 2; const ME_D2C_SINGLE_WRITE_COALESCE_MAX_BYTES: usize = 128 * 1024; const QUOTA_RESERVE_SPIN_RETRIES: usize = 32; -static DESYNC_DEDUP: OnceLock> = OnceLock::new(); -static DESYNC_DEDUP_PREVIOUS: OnceLock> = OnceLock::new(); -static DESYNC_HASHER: OnceLock = OnceLock::new(); -static DESYNC_FULL_CACHE_LAST_EMIT_AT: OnceLock>> = OnceLock::new(); -static DESYNC_DEDUP_ROTATION_STATE: OnceLock> = OnceLock::new(); -// Invariant for async callers: -// this std::sync::Mutex is allowed only because critical sections are short, -// synchronous, and MUST never cross an `.await`. -static RELAY_IDLE_CANDIDATE_REGISTRY: OnceLock> = OnceLock::new(); -static RELAY_IDLE_MARK_SEQ: AtomicU64 = AtomicU64::new(0); #[derive(Default)] -struct DesyncDedupRotationState { +pub(crate) struct DesyncDedupRotationState { current_started_at: Option, } @@ -80,7 +75,7 @@ struct RelayForensicsState { } #[derive(Default)] -struct RelayIdleCandidateRegistry { +pub(crate) struct RelayIdleCandidateRegistry { by_conn_id: HashMap, ordered: BTreeSet<(u64, u64)>, pressure_event_seq: u64, @@ -93,20 +88,14 @@ struct RelayIdleCandidateMeta { mark_pressure_seq: u64, } -fn relay_idle_candidate_registry() -> &'static Mutex { - RELAY_IDLE_CANDIDATE_REGISTRY.get_or_init(|| Mutex::new(RelayIdleCandidateRegistry::default())) -} - -fn relay_idle_candidate_registry_lock() -> std::sync::MutexGuard<'static, RelayIdleCandidateRegistry> -{ - // Keep lock scope narrow and synchronous: callers must drop guard before any `.await`. - let registry = relay_idle_candidate_registry(); +fn relay_idle_candidate_registry_lock_in( + shared: &ProxySharedState, +) -> std::sync::MutexGuard<'_, RelayIdleCandidateRegistry> { + let registry = &shared.middle_relay.relay_idle_registry; match registry.lock() { Ok(guard) => guard, Err(poisoned) => { let mut guard = poisoned.into_inner(); - // Fail closed after panic while holding registry lock: drop all - // candidates and pressure cursors to avoid stale cross-session state. *guard = RelayIdleCandidateRegistry::default(); registry.clear_poison(); guard @@ -114,14 +103,16 @@ fn relay_idle_candidate_registry_lock() -> std::sync::MutexGuard<'static, RelayI } } -fn mark_relay_idle_candidate(conn_id: u64) -> bool { - let mut guard = relay_idle_candidate_registry_lock(); +fn mark_relay_idle_candidate_in(shared: &ProxySharedState, conn_id: u64) -> bool { + let mut guard = relay_idle_candidate_registry_lock_in(shared); if guard.by_conn_id.contains_key(&conn_id) { return false; } - let mark_order_seq = RELAY_IDLE_MARK_SEQ + let mark_order_seq = shared + .middle_relay + .relay_idle_mark_seq .fetch_add(1, Ordering::Relaxed) .saturating_add(1); let meta = RelayIdleCandidateMeta { @@ -133,36 +124,35 @@ fn mark_relay_idle_candidate(conn_id: u64) -> bool { true } -fn clear_relay_idle_candidate(conn_id: u64) { - let mut guard = relay_idle_candidate_registry_lock(); +fn clear_relay_idle_candidate_in(shared: &ProxySharedState, conn_id: u64) { + let mut guard = relay_idle_candidate_registry_lock_in(shared); if let Some(meta) = guard.by_conn_id.remove(&conn_id) { guard.ordered.remove(&(meta.mark_order_seq, conn_id)); } } -#[cfg(test)] -fn oldest_relay_idle_candidate() -> Option { - let guard = relay_idle_candidate_registry_lock(); - guard.ordered.iter().next().map(|(_, conn_id)| *conn_id) -} - -fn note_relay_pressure_event() { - let mut guard = relay_idle_candidate_registry_lock(); +fn note_relay_pressure_event_in(shared: &ProxySharedState) { + let mut guard = relay_idle_candidate_registry_lock_in(shared); guard.pressure_event_seq = guard.pressure_event_seq.wrapping_add(1); } -fn relay_pressure_event_seq() -> u64 { - let guard = relay_idle_candidate_registry_lock(); +pub(crate) fn note_global_relay_pressure(shared: &ProxySharedState) { + note_relay_pressure_event_in(shared); +} + +fn relay_pressure_event_seq_in(shared: &ProxySharedState) -> u64 { + let guard = relay_idle_candidate_registry_lock_in(shared); guard.pressure_event_seq } -fn maybe_evict_idle_candidate_on_pressure( +fn maybe_evict_idle_candidate_on_pressure_in( + shared: &ProxySharedState, conn_id: u64, seen_pressure_seq: &mut u64, stats: &Stats, ) -> bool { - let mut guard = relay_idle_candidate_registry_lock(); + let mut guard = relay_idle_candidate_registry_lock_in(shared); let latest_pressure_seq = guard.pressure_event_seq; if latest_pressure_seq == *seen_pressure_seq { @@ -192,7 +182,6 @@ fn maybe_evict_idle_candidate_on_pressure( return false; }; - // Pressure events that happened before candidate soft-mark are stale for this candidate. if latest_pressure_seq == candidate_meta.mark_pressure_seq { return false; } @@ -205,15 +194,6 @@ fn maybe_evict_idle_candidate_on_pressure( true } -#[cfg(test)] -fn clear_relay_idle_pressure_state_for_testing() { - if RELAY_IDLE_CANDIDATE_REGISTRY.get().is_some() { - let mut guard = relay_idle_candidate_registry_lock(); - *guard = RelayIdleCandidateRegistry::default(); - } - RELAY_IDLE_MARK_SEQ.store(0, Ordering::Relaxed); -} - #[derive(Clone, Copy)] struct MeD2cFlushPolicy { max_frames: usize, @@ -235,31 +215,56 @@ struct RelayClientIdlePolicy { impl RelayClientIdlePolicy { fn from_config(config: &ProxyConfig) -> Self { + let frame_read_timeout = + Duration::from_secs(config.timeouts.relay_client_idle_hard_secs.max(1)); + if !config.timeouts.relay_idle_policy_v2_enabled { + return Self::disabled(frame_read_timeout); + } + + let soft_idle = Duration::from_secs(config.timeouts.relay_client_idle_soft_secs.max(1)); + let hard_idle = Duration::from_secs(config.timeouts.relay_client_idle_hard_secs.max(1)); + let grace_after_downstream_activity = Duration::from_secs( + config + .timeouts + .relay_idle_grace_after_downstream_activity_secs, + ); + Self { - enabled: config.timeouts.relay_idle_policy_v2_enabled, - soft_idle: Duration::from_secs(config.timeouts.relay_client_idle_soft_secs.max(1)), - hard_idle: Duration::from_secs(config.timeouts.relay_client_idle_hard_secs.max(1)), - grace_after_downstream_activity: Duration::from_secs( - config - .timeouts - .relay_idle_grace_after_downstream_activity_secs, - ), - legacy_frame_read_timeout: Duration::from_secs(config.timeouts.client_handshake.max(1)), + enabled: true, + soft_idle, + hard_idle, + grace_after_downstream_activity, + legacy_frame_read_timeout: frame_read_timeout, } } - #[cfg(test)] fn disabled(frame_read_timeout: Duration) -> Self { Self { enabled: false, - soft_idle: Duration::from_secs(0), - hard_idle: Duration::from_secs(0), - grace_after_downstream_activity: Duration::from_secs(0), + soft_idle: frame_read_timeout, + hard_idle: frame_read_timeout, + grace_after_downstream_activity: Duration::ZERO, legacy_frame_read_timeout: frame_read_timeout, } } + + fn apply_pressure_caps(&mut self, profile: ConntrackPressureProfile) { + let pressure_soft_idle_cap = Duration::from_secs(profile.middle_soft_idle_cap_secs()); + let pressure_hard_idle_cap = Duration::from_secs(profile.middle_hard_idle_cap_secs()); + + self.soft_idle = self.soft_idle.min(pressure_soft_idle_cap); + self.hard_idle = self.hard_idle.min(pressure_hard_idle_cap); + if self.soft_idle > self.hard_idle { + self.soft_idle = self.hard_idle; + } + self.legacy_frame_read_timeout = self.legacy_frame_read_timeout.min(pressure_hard_idle_cap); + if self.grace_after_downstream_activity > self.hard_idle { + self.grace_after_downstream_activity = self.hard_idle; + } + } } +#[derive(Clone, Copy)] struct RelayClientIdleState { last_client_frame_at: Instant, soft_idle_marked: bool, @@ -303,24 +308,39 @@ impl MeD2cFlushPolicy { } } +#[cfg(test)] fn hash_value(value: &T) -> u64 { - let state = DESYNC_HASHER.get_or_init(RandomState::new); - state.hash_one(value) + let mut hasher = DefaultHasher::new(); + value.hash(&mut hasher); + hasher.finish() } +fn hash_value_in(shared: &ProxySharedState, value: &T) -> u64 { + shared.middle_relay.desync_hasher.hash_one(value) +} + +#[cfg(test)] fn hash_ip(ip: IpAddr) -> u64 { hash_value(&ip) } -fn should_emit_full_desync(key: u64, all_full: bool, now: Instant) -> bool { +fn hash_ip_in(shared: &ProxySharedState, ip: IpAddr) -> u64 { + hash_value_in(shared, &ip) +} + +fn should_emit_full_desync_in( + shared: &ProxySharedState, + key: u64, + all_full: bool, + now: Instant, +) -> bool { if all_full { return true; } - let dedup_current = DESYNC_DEDUP.get_or_init(DashMap::new); - let dedup_previous = DESYNC_DEDUP_PREVIOUS.get_or_init(DashMap::new); - let rotation_state = - DESYNC_DEDUP_ROTATION_STATE.get_or_init(|| Mutex::new(DesyncDedupRotationState::default())); + let dedup_current = &shared.middle_relay.desync_dedup; + let dedup_previous = &shared.middle_relay.desync_dedup_previous; + let rotation_state = &shared.middle_relay.desync_dedup_rotation_state; let mut state = match rotation_state.lock() { Ok(guard) => guard, @@ -366,8 +386,6 @@ fn should_emit_full_desync(key: u64, all_full: bool, now: Instant) -> bool { None => true, }; if within_window { - // Keep the original timestamp when promoting from previous bucket, - // so dedup expiry remains tied to first-seen time. dedup_current.insert(key, seen_at); return false; } @@ -375,8 +393,6 @@ fn should_emit_full_desync(key: u64, all_full: bool, now: Instant) -> bool { } if dedup_current.len() >= DESYNC_DEDUP_MAX_ENTRIES { - // Bounded eviction path: rotate buckets instead of scanning/evicting - // arbitrary entries from a saturated single map. dedup_previous.clear(); for entry in dedup_current.iter() { dedup_previous.insert(*entry.key(), *entry.value()); @@ -384,15 +400,15 @@ fn should_emit_full_desync(key: u64, all_full: bool, now: Instant) -> bool { dedup_current.clear(); state.current_started_at = Some(now); dedup_current.insert(key, now); - should_emit_full_desync_full_cache(now) + should_emit_full_desync_full_cache_in(shared, now) } else { dedup_current.insert(key, now); true } } -fn should_emit_full_desync_full_cache(now: Instant) -> bool { - let gate = DESYNC_FULL_CACHE_LAST_EMIT_AT.get_or_init(|| Mutex::new(None)); +fn should_emit_full_desync_full_cache_in(shared: &ProxySharedState, now: Instant) -> bool { + let gate = &shared.middle_relay.desync_full_cache_last_emit_at; let Ok(mut last_emit_at) = gate.lock() else { return false; }; @@ -417,46 +433,6 @@ fn should_emit_full_desync_full_cache(now: Instant) -> bool { } } -#[cfg(test)] -fn clear_desync_dedup_for_testing() { - if let Some(dedup) = DESYNC_DEDUP.get() { - dedup.clear(); - } - if let Some(dedup_previous) = DESYNC_DEDUP_PREVIOUS.get() { - dedup_previous.clear(); - } - if let Some(rotation_state) = DESYNC_DEDUP_ROTATION_STATE.get() { - match rotation_state.lock() { - Ok(mut guard) => { - *guard = DesyncDedupRotationState::default(); - } - Err(poisoned) => { - let mut guard = poisoned.into_inner(); - *guard = DesyncDedupRotationState::default(); - rotation_state.clear_poison(); - } - } - } - if let Some(last_emit_at) = DESYNC_FULL_CACHE_LAST_EMIT_AT.get() { - match last_emit_at.lock() { - Ok(mut guard) => { - *guard = None; - } - Err(poisoned) => { - let mut guard = poisoned.into_inner(); - *guard = None; - last_emit_at.clear_poison(); - } - } - } -} - -#[cfg(test)] -fn desync_dedup_test_lock() -> &'static Mutex<()> { - static TEST_LOCK: OnceLock> = OnceLock::new(); - TEST_LOCK.get_or_init(|| Mutex::new(())) -} - fn desync_forensics_len_bytes(len: usize) -> ([u8; 4], bool) { match u32::try_from(len) { Ok(value) => (value.to_le_bytes(), false), @@ -464,7 +440,8 @@ fn desync_forensics_len_bytes(len: usize) -> ([u8; 4], bool) { } } -fn report_desync_frame_too_large( +fn report_desync_frame_too_large_in( + shared: &ProxySharedState, state: &RelayForensicsState, proto_tag: ProtoTag, frame_counter: u64, @@ -482,13 +459,16 @@ fn report_desync_frame_too_large( .map(|b| matches!(b[0], b'G' | b'P' | b'H' | b'C' | b'D')) .unwrap_or(false); let now = Instant::now(); - let dedup_key = hash_value(&( - state.user.as_str(), - state.peer_hash, - proto_tag, - DESYNC_ERROR_CLASS, - )); - let emit_full = should_emit_full_desync(dedup_key, state.desync_all_full, now); + let dedup_key = hash_value_in( + shared, + &( + state.user.as_str(), + state.peer_hash, + proto_tag, + DESYNC_ERROR_CLASS, + ), + ); + let emit_full = should_emit_full_desync_in(shared, dedup_key, state.desync_all_full, now); let duration_ms = state.started_at.elapsed().as_millis() as u64; let bytes_me2c = state.bytes_me2c.load(Ordering::Relaxed); @@ -557,6 +537,29 @@ fn report_desync_frame_too_large( )) } +#[cfg(test)] +fn report_desync_frame_too_large( + state: &RelayForensicsState, + proto_tag: ProtoTag, + frame_counter: u64, + max_frame: usize, + len: usize, + raw_len_bytes: Option<[u8; 4]>, + stats: &Stats, +) -> ProxyError { + let shared = ProxySharedState::new(); + report_desync_frame_too_large_in( + shared.as_ref(), + state, + proto_tag, + frame_counter, + max_frame, + len, + raw_len_bytes, + stats, + ) +} + fn should_yield_c2me_sender(sent_since_yield: usize, has_backlog: bool) -> bool { has_backlog && sent_since_yield >= C2ME_SENDER_FAIRNESS_BUDGET } @@ -629,28 +632,280 @@ fn observe_me_d2c_flush_event( } #[cfg(test)] -fn relay_idle_pressure_test_guard() -> &'static Mutex<()> { - static TEST_LOCK: OnceLock> = OnceLock::new(); - TEST_LOCK.get_or_init(|| Mutex::new(())) +pub(crate) fn mark_relay_idle_candidate_for_testing( + shared: &ProxySharedState, + conn_id: u64, +) -> bool { + let registry = &shared.middle_relay.relay_idle_registry; + let mut guard = match registry.lock() { + Ok(guard) => guard, + Err(poisoned) => { + let mut guard = poisoned.into_inner(); + *guard = RelayIdleCandidateRegistry::default(); + registry.clear_poison(); + guard + } + }; + + if guard.by_conn_id.contains_key(&conn_id) { + return false; + } + + let mark_order_seq = shared + .middle_relay + .relay_idle_mark_seq + .fetch_add(1, Ordering::Relaxed); + let mark_pressure_seq = guard.pressure_event_seq; + let meta = RelayIdleCandidateMeta { + mark_order_seq, + mark_pressure_seq, + }; + guard.by_conn_id.insert(conn_id, meta); + guard.ordered.insert((mark_order_seq, conn_id)); + true } #[cfg(test)] -pub(crate) fn relay_idle_pressure_test_scope() -> std::sync::MutexGuard<'static, ()> { - relay_idle_pressure_test_guard() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()) +pub(crate) fn oldest_relay_idle_candidate_for_testing(shared: &ProxySharedState) -> Option { + let registry = &shared.middle_relay.relay_idle_registry; + let guard = match registry.lock() { + Ok(guard) => guard, + Err(poisoned) => { + let mut guard = poisoned.into_inner(); + *guard = RelayIdleCandidateRegistry::default(); + registry.clear_poison(); + guard + } + }; + guard.ordered.iter().next().map(|(_, conn_id)| *conn_id) } -async fn enqueue_c2me_command( +#[cfg(test)] +pub(crate) fn clear_relay_idle_candidate_for_testing(shared: &ProxySharedState, conn_id: u64) { + let registry = &shared.middle_relay.relay_idle_registry; + let mut guard = match registry.lock() { + Ok(guard) => guard, + Err(poisoned) => { + let mut guard = poisoned.into_inner(); + *guard = RelayIdleCandidateRegistry::default(); + registry.clear_poison(); + guard + } + }; + if let Some(meta) = guard.by_conn_id.remove(&conn_id) { + guard.ordered.remove(&(meta.mark_order_seq, conn_id)); + } +} + +#[cfg(test)] +pub(crate) fn clear_relay_idle_pressure_state_for_testing_in_shared(shared: &ProxySharedState) { + if let Ok(mut guard) = shared.middle_relay.relay_idle_registry.lock() { + *guard = RelayIdleCandidateRegistry::default(); + } + shared + .middle_relay + .relay_idle_mark_seq + .store(0, Ordering::Relaxed); +} + +#[cfg(test)] +pub(crate) fn note_relay_pressure_event_for_testing(shared: &ProxySharedState) { + note_relay_pressure_event_in(shared); +} + +#[cfg(test)] +pub(crate) fn relay_pressure_event_seq_for_testing(shared: &ProxySharedState) -> u64 { + relay_pressure_event_seq_in(shared) +} + +#[cfg(test)] +pub(crate) fn relay_idle_mark_seq_for_testing(shared: &ProxySharedState) -> u64 { + shared + .middle_relay + .relay_idle_mark_seq + .load(Ordering::Relaxed) +} + +#[cfg(test)] +pub(crate) fn maybe_evict_idle_candidate_on_pressure_for_testing( + shared: &ProxySharedState, + conn_id: u64, + seen_pressure_seq: &mut u64, + stats: &Stats, +) -> bool { + maybe_evict_idle_candidate_on_pressure_in(shared, conn_id, seen_pressure_seq, stats) +} + +#[cfg(test)] +pub(crate) fn set_relay_pressure_state_for_testing( + shared: &ProxySharedState, + pressure_event_seq: u64, + pressure_consumed_seq: u64, +) { + let registry = &shared.middle_relay.relay_idle_registry; + let mut guard = match registry.lock() { + Ok(guard) => guard, + Err(poisoned) => { + let mut guard = poisoned.into_inner(); + *guard = RelayIdleCandidateRegistry::default(); + registry.clear_poison(); + guard + } + }; + guard.pressure_event_seq = pressure_event_seq; + guard.pressure_consumed_seq = pressure_consumed_seq; +} + +#[cfg(test)] +pub(crate) fn should_emit_full_desync_for_testing( + shared: &ProxySharedState, + key: u64, + all_full: bool, + now: Instant, +) -> bool { + if all_full { + return true; + } + + let dedup_current = &shared.middle_relay.desync_dedup; + let dedup_previous = &shared.middle_relay.desync_dedup_previous; + + let Ok(mut state) = shared.middle_relay.desync_dedup_rotation_state.lock() else { + return false; + }; + + let rotate_now = match state.current_started_at { + Some(current_started_at) => match now.checked_duration_since(current_started_at) { + Some(elapsed) => elapsed >= DESYNC_DEDUP_WINDOW, + None => true, + }, + None => true, + }; + if rotate_now { + dedup_previous.clear(); + for entry in dedup_current.iter() { + dedup_previous.insert(*entry.key(), *entry.value()); + } + dedup_current.clear(); + state.current_started_at = Some(now); + } + + if let Some(seen_at) = dedup_current.get(&key).map(|entry| *entry.value()) { + let within_window = match now.checked_duration_since(seen_at) { + Some(elapsed) => elapsed < DESYNC_DEDUP_WINDOW, + None => true, + }; + if within_window { + return false; + } + dedup_current.insert(key, now); + return true; + } + + if let Some(seen_at) = dedup_previous.get(&key).map(|entry| *entry.value()) { + let within_window = match now.checked_duration_since(seen_at) { + Some(elapsed) => elapsed < DESYNC_DEDUP_WINDOW, + None => true, + }; + if within_window { + dedup_current.insert(key, seen_at); + return false; + } + dedup_previous.remove(&key); + } + + if dedup_current.len() >= DESYNC_DEDUP_MAX_ENTRIES { + dedup_previous.clear(); + for entry in dedup_current.iter() { + dedup_previous.insert(*entry.key(), *entry.value()); + } + dedup_current.clear(); + state.current_started_at = Some(now); + dedup_current.insert(key, now); + let Ok(mut last_emit_at) = shared.middle_relay.desync_full_cache_last_emit_at.lock() else { + return false; + }; + return match *last_emit_at { + None => { + *last_emit_at = Some(now); + true + } + Some(last) => { + let Some(elapsed) = now.checked_duration_since(last) else { + *last_emit_at = Some(now); + return true; + }; + if elapsed >= DESYNC_FULL_CACHE_EMIT_MIN_INTERVAL { + *last_emit_at = Some(now); + true + } else { + false + } + } + }; + } + + dedup_current.insert(key, now); + true +} + +#[cfg(test)] +pub(crate) fn clear_desync_dedup_for_testing_in_shared(shared: &ProxySharedState) { + shared.middle_relay.desync_dedup.clear(); + shared.middle_relay.desync_dedup_previous.clear(); + if let Ok(mut rotation_state) = shared.middle_relay.desync_dedup_rotation_state.lock() { + *rotation_state = DesyncDedupRotationState::default(); + } + if let Ok(mut last_emit_at) = shared.middle_relay.desync_full_cache_last_emit_at.lock() { + *last_emit_at = None; + } +} + +#[cfg(test)] +pub(crate) fn desync_dedup_len_for_testing(shared: &ProxySharedState) -> usize { + shared.middle_relay.desync_dedup.len() +} + +#[cfg(test)] +pub(crate) fn desync_dedup_insert_for_testing(shared: &ProxySharedState, key: u64, at: Instant) { + shared.middle_relay.desync_dedup.insert(key, at); +} + +#[cfg(test)] +pub(crate) fn desync_dedup_get_for_testing(shared: &ProxySharedState, key: u64) -> Option { + shared + .middle_relay + .desync_dedup + .get(&key) + .map(|entry| *entry.value()) +} + +#[cfg(test)] +pub(crate) fn desync_dedup_keys_for_testing( + shared: &ProxySharedState, +) -> std::collections::HashSet { + shared + .middle_relay + .desync_dedup + .iter() + .map(|entry| *entry.key()) + .collect() +} + +async fn enqueue_c2me_command_in( + shared: &ProxySharedState, tx: &mpsc::Sender, cmd: C2MeCommand, send_timeout: Option, + stats: &Stats, ) -> std::result::Result<(), mpsc::error::SendError> { match tx.try_send(cmd) { Ok(()) => Ok(()), Err(mpsc::error::TrySendError::Closed(cmd)) => Err(mpsc::error::SendError(cmd)), Err(mpsc::error::TrySendError::Full(cmd)) => { - note_relay_pressure_event(); + stats.increment_me_c2me_send_full_total(); + stats.increment_me_c2me_send_high_water_total(); + note_relay_pressure_event_in(shared); // Cooperative yield reduces burst catch-up when the per-conn queue is near saturation. if tx.capacity() <= C2ME_SOFT_PRESSURE_MIN_FREE_SLOTS { tokio::task::yield_now().await; @@ -658,7 +913,10 @@ async fn enqueue_c2me_command( let reserve_result = match send_timeout { Some(send_timeout) => match timeout(send_timeout, tx.reserve()).await { Ok(result) => result, - Err(_) => return Err(mpsc::error::SendError(cmd)), + Err(_) => { + stats.increment_me_c2me_send_timeout_total(); + return Err(mpsc::error::SendError(cmd)); + } }, None => tx.reserve().await, }; @@ -667,12 +925,26 @@ async fn enqueue_c2me_command( permit.send(cmd); Ok(()) } - Err(_) => Err(mpsc::error::SendError(cmd)), + Err(_) => { + stats.increment_me_c2me_send_timeout_total(); + Err(mpsc::error::SendError(cmd)) + } } } } } +#[cfg(test)] +async fn enqueue_c2me_command( + tx: &mpsc::Sender, + cmd: C2MeCommand, + send_timeout: Option, + stats: &Stats, +) -> std::result::Result<(), mpsc::error::SendError> { + let shared = ProxySharedState::new(); + enqueue_c2me_command_in(shared.as_ref(), tx, cmd, send_timeout, stats).await +} + #[cfg(test)] async fn run_relay_test_step_timeout(context: &'static str, fut: F) -> T where @@ -696,6 +968,7 @@ pub(crate) async fn handle_via_middle_proxy( mut route_rx: watch::Receiver, route_snapshot: RouteCutoverState, session_id: u64, + shared: Arc, ) -> Result<()> where R: AsyncRead + Unpin + Send + 'static, @@ -726,7 +999,7 @@ where conn_id, user: user.clone(), peer, - peer_hash: hash_ip(peer.ip()), + peer_hash: hash_ip_in(shared.as_ref(), peer.ip()), started_at: Instant::now(), bytes_c2me: 0, bytes_me2c: bytes_me2c.clone(), @@ -783,7 +1056,12 @@ where let translated_local_addr = me_pool.translate_our_addr(local_addr); let frame_limit = config.general.max_client_frame; - let relay_idle_policy = RelayClientIdlePolicy::from_config(&config); + let mut relay_idle_policy = RelayClientIdlePolicy::from_config(&config); + let mut pressure_caps_applied = false; + if shared.conntrack_pressure_active() { + relay_idle_policy.apply_pressure_caps(config.server.conntrack_control.profile); + pressure_caps_applied = true; + } let session_started_at = forensics.started_at; let mut relay_idle_state = RelayClientIdleState::new(session_started_at); let last_downstream_activity_ms = Arc::new(AtomicU64::new(0)); @@ -841,11 +1119,23 @@ where let me_writer = tokio::spawn(async move { let mut writer = crypto_writer; let mut frame_buf = Vec::with_capacity(16 * 1024); + let shrink_threshold = d2c_flush_policy.frame_buf_shrink_threshold_bytes; + + fn shrink_session_vec(buf: &mut Vec, threshold: usize) { + if buf.capacity() > threshold { + buf.clear(); + buf.shrink_to(threshold); + } else { + buf.clear(); + } + } + loop { tokio::select! { msg = me_rx_task.recv() => { let Some(first) = msg else { debug!(conn_id, "ME channel closed"); + shrink_session_vec(&mut frame_buf, shrink_threshold); return Err(ProxyError::Proxy("ME connection lost".into())); }; @@ -901,6 +1191,7 @@ where batch_bytes, flush_duration_us, ); + shrink_session_vec(&mut frame_buf, shrink_threshold); return Ok(()); } } @@ -962,6 +1253,7 @@ where batch_bytes, flush_duration_us, ); + shrink_session_vec(&mut frame_buf, shrink_threshold); return Ok(()); } } @@ -1027,6 +1319,7 @@ where batch_bytes, flush_duration_us, ); + shrink_session_vec(&mut frame_buf, shrink_threshold); return Ok(()); } } @@ -1091,6 +1384,7 @@ where batch_bytes, flush_duration_us, ); + shrink_session_vec(&mut frame_buf, shrink_threshold); return Ok(()); } } @@ -1098,6 +1392,7 @@ where } Ok(None) => { debug!(conn_id, "ME channel closed"); + shrink_session_vec(&mut frame_buf, shrink_threshold); return Err(ProxyError::Proxy("ME connection lost".into())); } Err(_) => { @@ -1147,6 +1442,7 @@ where } _ = &mut stop_rx => { debug!(conn_id, "ME writer stop signal"); + shrink_session_vec(&mut frame_buf, shrink_threshold); return Ok(()); } } @@ -1157,10 +1453,16 @@ where let mut client_closed = false; let mut frame_counter: u64 = 0; let mut route_watch_open = true; - let mut seen_pressure_seq = relay_pressure_event_seq(); + let mut seen_pressure_seq = relay_pressure_event_seq_in(shared.as_ref()); loop { + if shared.conntrack_pressure_active() && !pressure_caps_applied { + relay_idle_policy.apply_pressure_caps(config.server.conntrack_control.profile); + pressure_caps_applied = true; + } + if relay_idle_policy.enabled - && maybe_evict_idle_candidate_on_pressure( + && maybe_evict_idle_candidate_on_pressure_in( + shared.as_ref(), conn_id, &mut seen_pressure_seq, stats.as_ref(), @@ -1172,7 +1474,14 @@ where user = %user, "Middle-relay pressure eviction for idle-candidate session" ); - let _ = enqueue_c2me_command(&c2me_tx, C2MeCommand::Close, c2me_send_timeout).await; + let _ = enqueue_c2me_command_in( + shared.as_ref(), + &c2me_tx, + C2MeCommand::Close, + c2me_send_timeout, + stats.as_ref(), + ) + .await; main_result = Err(ProxyError::Proxy( "middle-relay session evicted under pressure (idle-candidate)".to_string(), )); @@ -1191,7 +1500,14 @@ where "Cutover affected middle session, closing client connection" ); tokio::time::sleep(delay).await; - let _ = enqueue_c2me_command(&c2me_tx, C2MeCommand::Close, c2me_send_timeout).await; + let _ = enqueue_c2me_command_in( + shared.as_ref(), + &c2me_tx, + C2MeCommand::Close, + c2me_send_timeout, + stats.as_ref(), + ) + .await; main_result = Err(ProxyError::Proxy(ROUTE_SWITCH_ERROR_MSG.to_string())); break; } @@ -1202,7 +1518,7 @@ where route_watch_open = false; } } - payload_result = read_client_payload_with_idle_policy( + payload_result = read_client_payload_with_idle_policy_in( &mut crypto_reader, proto_tag, frame_limit, @@ -1210,6 +1526,7 @@ where &forensics, &mut frame_counter, &stats, + shared.as_ref(), &relay_idle_policy, &mut relay_idle_state, last_downstream_activity_ms.as_ref(), @@ -1249,10 +1566,12 @@ where flags |= RPC_FLAG_NOT_ENCRYPTED; } // Keep client read loop lightweight: route heavy ME send path via a dedicated task. - if enqueue_c2me_command( + if enqueue_c2me_command_in( + shared.as_ref(), &c2me_tx, C2MeCommand::Data { payload, flags }, c2me_send_timeout, + stats.as_ref(), ) .await .is_err() @@ -1264,9 +1583,14 @@ where Ok(None) => { debug!(conn_id, "Client EOF"); client_closed = true; - let _ = - enqueue_c2me_command(&c2me_tx, C2MeCommand::Close, c2me_send_timeout) - .await; + let _ = enqueue_c2me_command_in( + shared.as_ref(), + &c2me_tx, + C2MeCommand::Close, + c2me_send_timeout, + stats.as_ref(), + ) + .await; break; } Err(e) => { @@ -1315,12 +1639,60 @@ where frames_ok = frame_counter, "ME relay cleanup" ); - clear_relay_idle_candidate(conn_id); + + let close_reason = classify_conntrack_close_reason(&result); + let publish_result = shared.publish_conntrack_close_event(ConntrackCloseEvent { + src: peer, + dst: local_addr, + reason: close_reason, + }); + if !matches!( + publish_result, + ConntrackClosePublishResult::Sent | ConntrackClosePublishResult::Disabled + ) { + stats.increment_conntrack_close_event_drop_total(); + } + + clear_relay_idle_candidate_in(shared.as_ref(), conn_id); me_pool.registry().unregister(conn_id).await; + buffer_pool.trim_to(buffer_pool.max_buffers().min(64)); + let pool_snapshot = buffer_pool.stats(); + stats.set_buffer_pool_gauges( + pool_snapshot.pooled, + pool_snapshot.allocated, + pool_snapshot.allocated.saturating_sub(pool_snapshot.pooled), + ); result } -async fn read_client_payload_with_idle_policy( +fn classify_conntrack_close_reason(result: &Result<()>) -> ConntrackCloseReason { + match result { + Ok(()) => ConntrackCloseReason::NormalEof, + Err(ProxyError::Io(error)) if matches!(error.kind(), std::io::ErrorKind::TimedOut) => { + ConntrackCloseReason::Timeout + } + Err(ProxyError::Io(error)) + if matches!( + error.kind(), + std::io::ErrorKind::ConnectionReset + | std::io::ErrorKind::ConnectionAborted + | std::io::ErrorKind::BrokenPipe + | std::io::ErrorKind::NotConnected + | std::io::ErrorKind::UnexpectedEof + ) => + { + ConntrackCloseReason::Reset + } + Err(ProxyError::Proxy(message)) + if message.contains("pressure") || message.contains("evicted") => + { + ConntrackCloseReason::Pressure + } + Err(_) => ConntrackCloseReason::Other, + } +} + +async fn read_client_payload_with_idle_policy_in( client_reader: &mut CryptoReader, proto_tag: ProtoTag, max_frame: usize, @@ -1328,6 +1700,7 @@ async fn read_client_payload_with_idle_policy( forensics: &RelayForensicsState, frame_counter: &mut u64, stats: &Stats, + shared: &ProxySharedState, idle_policy: &RelayClientIdlePolicy, idle_state: &mut RelayClientIdleState, last_downstream_activity_ms: &AtomicU64, @@ -1347,6 +1720,7 @@ where session_started_at: Instant, forensics: &RelayForensicsState, stats: &Stats, + shared: &ProxySharedState, read_label: &'static str, ) -> Result<()> where @@ -1382,7 +1756,7 @@ where let hard_deadline = hard_deadline(idle_policy, idle_state, session_started_at, downstream_ms); if now >= hard_deadline { - clear_relay_idle_candidate(forensics.conn_id); + clear_relay_idle_candidate_in(shared, forensics.conn_id); stats.increment_relay_idle_hard_close_total(); let client_idle_secs = now .saturating_duration_since(idle_state.last_client_frame_at) @@ -1420,7 +1794,7 @@ where >= idle_policy.soft_idle { idle_state.soft_idle_marked = true; - if mark_relay_idle_candidate(forensics.conn_id) { + if mark_relay_idle_candidate_in(shared, forensics.conn_id) { stats.increment_relay_idle_soft_mark_total(); } info!( @@ -1490,6 +1864,7 @@ where session_started_at, forensics, stats, + shared, "abridged.first_len_byte", ) .await @@ -1513,6 +1888,7 @@ where session_started_at, forensics, stats, + shared, "abridged.extended_len", ) .await?; @@ -1537,6 +1913,7 @@ where session_started_at, forensics, stats, + shared, "len_prefix", ) .await @@ -1593,7 +1970,8 @@ where } if len > max_frame { - return Err(report_desync_frame_too_large( + return Err(report_desync_frame_too_large_in( + shared, forensics, proto_tag, *frame_counter, @@ -1635,6 +2013,7 @@ where session_started_at, forensics, stats, + shared, "payload", ) .await?; @@ -1646,11 +2025,46 @@ where *frame_counter += 1; idle_state.on_client_frame(Instant::now()); idle_state.tiny_frame_debt = idle_state.tiny_frame_debt.saturating_sub(1); - clear_relay_idle_candidate(forensics.conn_id); + clear_relay_idle_candidate_in(shared, forensics.conn_id); return Ok(Some((payload, quickack))); } } +#[cfg(test)] +async fn read_client_payload_with_idle_policy( + client_reader: &mut CryptoReader, + proto_tag: ProtoTag, + max_frame: usize, + buffer_pool: &Arc, + forensics: &RelayForensicsState, + frame_counter: &mut u64, + stats: &Stats, + idle_policy: &RelayClientIdlePolicy, + idle_state: &mut RelayClientIdleState, + last_downstream_activity_ms: &AtomicU64, + session_started_at: Instant, +) -> Result> +where + R: AsyncRead + Unpin + Send + 'static, +{ + let shared = ProxySharedState::new(); + read_client_payload_with_idle_policy_in( + client_reader, + proto_tag, + max_frame, + buffer_pool, + forensics, + frame_counter, + stats, + shared.as_ref(), + idle_policy, + idle_state, + last_downstream_activity_ms, + session_started_at, + ) + .await +} + #[cfg(test)] async fn read_client_payload_legacy( client_reader: &mut CryptoReader, @@ -1666,10 +2080,11 @@ where R: AsyncRead + Unpin + Send + 'static, { let now = Instant::now(); + let shared = ProxySharedState::new(); let mut idle_state = RelayClientIdleState::new(now); let last_downstream_activity_ms = AtomicU64::new(0); let idle_policy = RelayClientIdlePolicy::disabled(frame_read_timeout); - read_client_payload_with_idle_policy( + read_client_payload_with_idle_policy_in( client_reader, proto_tag, max_frame, @@ -1677,6 +2092,7 @@ where forensics, frame_counter, stats, + shared.as_ref(), &idle_policy, &mut idle_state, &last_downstream_activity_ms, @@ -2047,3 +2463,7 @@ mod middle_relay_tiny_frame_debt_proto_chunking_security_tests; #[cfg(test)] #[path = "tests/middle_relay_atomic_quota_invariant_tests.rs"] mod middle_relay_atomic_quota_invariant_tests; + +#[cfg(test)] +#[path = "tests/middle_relay_baseline_invariant_tests.rs"] +mod middle_relay_baseline_invariant_tests; diff --git a/src/proxy/mod.rs b/src/proxy/mod.rs index 5880558..c4ce09c 100644 --- a/src/proxy/mod.rs +++ b/src/proxy/mod.rs @@ -67,6 +67,7 @@ pub mod middle_relay; pub mod relay; pub mod route_mode; pub mod session_eviction; +pub mod shared_state; pub use client::ClientHandler; #[allow(unused_imports)] @@ -75,3 +76,15 @@ pub use handshake::*; pub use masking::*; #[allow(unused_imports)] pub use relay::*; + +#[cfg(test)] +#[path = "tests/test_harness_common.rs"] +mod test_harness_common; + +#[cfg(test)] +#[path = "tests/proxy_shared_state_isolation_tests.rs"] +mod proxy_shared_state_isolation_tests; + +#[cfg(test)] +#[path = "tests/proxy_shared_state_parallel_execution_tests.rs"] +mod proxy_shared_state_parallel_execution_tests; diff --git a/src/proxy/relay.rs b/src/proxy/relay.rs index 6000e18..9fd5f3d 100644 --- a/src/proxy/relay.rs +++ b/src/proxy/relay.rs @@ -70,6 +70,7 @@ use tracing::{debug, trace, warn}; /// /// iOS keeps Telegram connections alive in background for up to 30 minutes. /// Closing earlier causes unnecessary reconnects and handshake overhead. +#[allow(dead_code)] const ACTIVITY_TIMEOUT: Duration = Duration::from_secs(1800); /// Watchdog check interval — also used for periodic rate logging. @@ -269,6 +270,7 @@ const QUOTA_NEAR_LIMIT_BYTES: u64 = 64 * 1024; const QUOTA_LARGE_CHARGE_BYTES: u64 = 16 * 1024; const QUOTA_ADAPTIVE_INTERVAL_MIN_BYTES: u64 = 4 * 1024; const QUOTA_ADAPTIVE_INTERVAL_MAX_BYTES: u64 = 64 * 1024; +const QUOTA_RESERVE_SPIN_RETRIES: usize = 64; #[inline] fn quota_adaptive_interval_bytes(remaining_before: u64) -> u64 { @@ -313,6 +315,50 @@ impl AsyncRead for StatsIo { if n > 0 { let n_to_charge = n as u64; + if let (Some(limit), Some(remaining)) = (this.quota_limit, remaining_before) { + let mut reserved_total = None; + let mut reserve_rounds = 0usize; + while reserved_total.is_none() { + for _ in 0..QUOTA_RESERVE_SPIN_RETRIES { + match this.user_stats.quota_try_reserve(n_to_charge, limit) { + Ok(total) => { + reserved_total = Some(total); + break; + } + Err(crate::stats::QuotaReserveError::LimitExceeded) => { + this.quota_exceeded.store(true, Ordering::Release); + buf.set_filled(before); + return Poll::Ready(Err(quota_io_error())); + } + Err(crate::stats::QuotaReserveError::Contended) => { + std::hint::spin_loop(); + } + } + } + reserve_rounds = reserve_rounds.saturating_add(1); + if reserved_total.is_none() && reserve_rounds >= 8 { + this.quota_exceeded.store(true, Ordering::Release); + buf.set_filled(before); + return Poll::Ready(Err(quota_io_error())); + } + } + + if should_immediate_quota_check(remaining, n_to_charge) { + this.quota_bytes_since_check = 0; + } else { + this.quota_bytes_since_check = + this.quota_bytes_since_check.saturating_add(n_to_charge); + let interval = quota_adaptive_interval_bytes(remaining); + if this.quota_bytes_since_check >= interval { + this.quota_bytes_since_check = 0; + } + } + + if reserved_total.unwrap_or(0) >= limit { + this.quota_exceeded.store(true, Ordering::Release); + } + } + // C→S: client sent data this.counters .c2s_bytes @@ -325,27 +371,6 @@ impl AsyncRead for StatsIo { this.stats .increment_user_msgs_from_handle(this.user_stats.as_ref()); - if let (Some(limit), Some(remaining)) = (this.quota_limit, remaining_before) { - this.stats - .quota_charge_post_write(this.user_stats.as_ref(), n_to_charge); - if should_immediate_quota_check(remaining, n_to_charge) { - this.quota_bytes_since_check = 0; - if this.user_stats.quota_used() >= limit { - this.quota_exceeded.store(true, Ordering::Release); - } - } else { - this.quota_bytes_since_check = - this.quota_bytes_since_check.saturating_add(n_to_charge); - let interval = quota_adaptive_interval_bytes(remaining); - if this.quota_bytes_since_check >= interval { - this.quota_bytes_since_check = 0; - if this.user_stats.quota_used() >= limit { - this.quota_exceeded.store(true, Ordering::Release); - } - } - } - } - trace!(user = %this.user, bytes = n, "C->S"); } Poll::Ready(Ok(())) @@ -367,18 +392,73 @@ impl AsyncWrite for StatsIo { } let mut remaining_before = None; + let mut reserved_bytes = 0u64; + let mut write_buf = buf; if let Some(limit) = this.quota_limit { - let used_before = this.user_stats.quota_used(); - let remaining = limit.saturating_sub(used_before); - if remaining == 0 { - this.quota_exceeded.store(true, Ordering::Release); - return Poll::Ready(Err(quota_io_error())); + if !buf.is_empty() { + let mut reserve_rounds = 0usize; + while reserved_bytes == 0 { + let used_before = this.user_stats.quota_used(); + let remaining = limit.saturating_sub(used_before); + if remaining == 0 { + this.quota_exceeded.store(true, Ordering::Release); + return Poll::Ready(Err(quota_io_error())); + } + remaining_before = Some(remaining); + + let desired = remaining.min(buf.len() as u64); + for _ in 0..QUOTA_RESERVE_SPIN_RETRIES { + match this.user_stats.quota_try_reserve(desired, limit) { + Ok(_) => { + reserved_bytes = desired; + write_buf = &buf[..desired as usize]; + break; + } + Err(crate::stats::QuotaReserveError::LimitExceeded) => { + break; + } + Err(crate::stats::QuotaReserveError::Contended) => { + std::hint::spin_loop(); + } + } + } + + reserve_rounds = reserve_rounds.saturating_add(1); + if reserved_bytes == 0 && reserve_rounds >= 8 { + this.quota_exceeded.store(true, Ordering::Release); + return Poll::Ready(Err(quota_io_error())); + } + } + } else { + let used_before = this.user_stats.quota_used(); + let remaining = limit.saturating_sub(used_before); + if remaining == 0 { + this.quota_exceeded.store(true, Ordering::Release); + return Poll::Ready(Err(quota_io_error())); + } + remaining_before = Some(remaining); } - remaining_before = Some(remaining); } - match Pin::new(&mut this.inner).poll_write(cx, buf) { + match Pin::new(&mut this.inner).poll_write(cx, write_buf) { Poll::Ready(Ok(n)) => { + if reserved_bytes > n as u64 { + let refund = reserved_bytes - n as u64; + let mut current = this.user_stats.quota_used.load(Ordering::Relaxed); + loop { + let next = current.saturating_sub(refund); + match this.user_stats.quota_used.compare_exchange_weak( + current, + next, + Ordering::Relaxed, + Ordering::Relaxed, + ) { + Ok(_) => break, + Err(observed) => current = observed, + } + } + } + if n > 0 { let n_to_charge = n as u64; @@ -395,8 +475,6 @@ impl AsyncWrite for StatsIo { .increment_user_msgs_to_handle(this.user_stats.as_ref()); if let (Some(limit), Some(remaining)) = (this.quota_limit, remaining_before) { - this.stats - .quota_charge_post_write(this.user_stats.as_ref(), n_to_charge); if should_immediate_quota_check(remaining, n_to_charge) { this.quota_bytes_since_check = 0; if this.user_stats.quota_used() >= limit { @@ -419,7 +497,42 @@ impl AsyncWrite for StatsIo { } Poll::Ready(Ok(n)) } - other => other, + Poll::Ready(Err(err)) => { + if reserved_bytes > 0 { + let mut current = this.user_stats.quota_used.load(Ordering::Relaxed); + loop { + let next = current.saturating_sub(reserved_bytes); + match this.user_stats.quota_used.compare_exchange_weak( + current, + next, + Ordering::Relaxed, + Ordering::Relaxed, + ) { + Ok(_) => break, + Err(observed) => current = observed, + } + } + } + Poll::Ready(Err(err)) + } + Poll::Pending => { + if reserved_bytes > 0 { + let mut current = this.user_stats.quota_used.load(Ordering::Relaxed); + loop { + let next = current.saturating_sub(reserved_bytes); + match this.user_stats.quota_used.compare_exchange_weak( + current, + next, + Ordering::Relaxed, + Ordering::Relaxed, + ) { + Ok(_) => break, + Err(observed) => current = observed, + } + } + } + Poll::Pending + } } } @@ -453,6 +566,7 @@ impl AsyncWrite for StatsIo { /// - Clean shutdown: both write sides are shut down on exit /// - Error propagation: quota exits return `ProxyError::DataQuotaExceeded`, /// other I/O failures are returned as `ProxyError::Io` +#[allow(dead_code)] pub async fn relay_bidirectional( client_reader: CR, client_writer: CW, @@ -471,6 +585,42 @@ where SR: AsyncRead + Unpin + Send + 'static, SW: AsyncWrite + Unpin + Send + 'static, { + relay_bidirectional_with_activity_timeout( + client_reader, + client_writer, + server_reader, + server_writer, + c2s_buf_size, + s2c_buf_size, + user, + stats, + quota_limit, + _buffer_pool, + ACTIVITY_TIMEOUT, + ) + .await +} + +pub async fn relay_bidirectional_with_activity_timeout( + client_reader: CR, + client_writer: CW, + server_reader: SR, + server_writer: SW, + c2s_buf_size: usize, + s2c_buf_size: usize, + user: &str, + stats: Arc, + quota_limit: Option, + _buffer_pool: Arc, + activity_timeout: Duration, +) -> Result<()> +where + CR: AsyncRead + Unpin + Send + 'static, + CW: AsyncWrite + Unpin + Send + 'static, + SR: AsyncRead + Unpin + Send + 'static, + SW: AsyncWrite + Unpin + Send + 'static, +{ + let activity_timeout = activity_timeout.max(Duration::from_secs(1)); let epoch = Instant::now(); let counters = Arc::new(SharedCounters::new()); let quota_exceeded = Arc::new(AtomicBool::new(false)); @@ -512,7 +662,7 @@ where } // ── Activity timeout ──────────────────────────────────── - if idle >= ACTIVITY_TIMEOUT { + if idle >= activity_timeout { let c2s = wd_counters.c2s_bytes.load(Ordering::Relaxed); let s2c = wd_counters.s2c_bytes.load(Ordering::Relaxed); warn!( @@ -671,3 +821,7 @@ mod relay_watchdog_delta_security_tests; #[cfg(test)] #[path = "tests/relay_atomic_quota_invariant_tests.rs"] mod relay_atomic_quota_invariant_tests; + +#[cfg(test)] +#[path = "tests/relay_baseline_invariant_tests.rs"] +mod relay_baseline_invariant_tests; diff --git a/src/proxy/shared_state.rs b/src/proxy/shared_state.rs new file mode 100644 index 0000000..4fef497 --- /dev/null +++ b/src/proxy/shared_state.rs @@ -0,0 +1,165 @@ +use std::collections::HashSet; +use std::collections::hash_map::RandomState; +use std::net::{IpAddr, SocketAddr}; +use std::sync::atomic::{AtomicBool, AtomicU32, AtomicU64, Ordering}; +use std::sync::{Arc, Mutex}; +use std::time::Instant; + +use dashmap::DashMap; +use tokio::sync::mpsc; + +use crate::proxy::handshake::{AuthProbeSaturationState, AuthProbeState}; +use crate::proxy::middle_relay::{DesyncDedupRotationState, RelayIdleCandidateRegistry}; + +const HANDSHAKE_RECENT_USER_RING_LEN: usize = 64; + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum ConntrackCloseReason { + NormalEof, + Timeout, + Pressure, + Reset, + Other, +} + +#[derive(Debug, Clone, Copy)] +pub(crate) struct ConntrackCloseEvent { + pub(crate) src: SocketAddr, + pub(crate) dst: SocketAddr, + pub(crate) reason: ConntrackCloseReason, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum ConntrackClosePublishResult { + Sent, + Disabled, + QueueFull, + QueueClosed, +} + +pub(crate) struct HandshakeSharedState { + pub(crate) auth_probe: DashMap, + pub(crate) auth_probe_saturation: Mutex>, + pub(crate) auth_probe_eviction_hasher: RandomState, + pub(crate) invalid_secret_warned: Mutex>, + pub(crate) unknown_sni_warn_next_allowed: Mutex>, + pub(crate) sticky_user_by_ip: DashMap, + pub(crate) sticky_user_by_ip_prefix: DashMap, + pub(crate) sticky_user_by_sni_hash: DashMap, + pub(crate) recent_user_ring: Box<[AtomicU32]>, + pub(crate) recent_user_ring_seq: AtomicU64, + pub(crate) auth_expensive_checks_total: AtomicU64, + pub(crate) auth_budget_exhausted_total: AtomicU64, +} + +pub(crate) struct MiddleRelaySharedState { + pub(crate) desync_dedup: DashMap, + pub(crate) desync_dedup_previous: DashMap, + pub(crate) desync_hasher: RandomState, + pub(crate) desync_full_cache_last_emit_at: Mutex>, + pub(crate) desync_dedup_rotation_state: Mutex, + pub(crate) relay_idle_registry: Mutex, + pub(crate) relay_idle_mark_seq: AtomicU64, +} + +pub(crate) struct ProxySharedState { + pub(crate) handshake: HandshakeSharedState, + pub(crate) middle_relay: MiddleRelaySharedState, + pub(crate) conntrack_pressure_active: AtomicBool, + pub(crate) conntrack_close_tx: Mutex>>, +} + +impl ProxySharedState { + pub(crate) fn new() -> Arc { + Arc::new(Self { + handshake: HandshakeSharedState { + auth_probe: DashMap::new(), + auth_probe_saturation: Mutex::new(None), + auth_probe_eviction_hasher: RandomState::new(), + invalid_secret_warned: Mutex::new(HashSet::new()), + unknown_sni_warn_next_allowed: Mutex::new(None), + sticky_user_by_ip: DashMap::new(), + sticky_user_by_ip_prefix: DashMap::new(), + sticky_user_by_sni_hash: DashMap::new(), + recent_user_ring: std::iter::repeat_with(|| AtomicU32::new(0)) + .take(HANDSHAKE_RECENT_USER_RING_LEN) + .collect::>() + .into_boxed_slice(), + recent_user_ring_seq: AtomicU64::new(0), + auth_expensive_checks_total: AtomicU64::new(0), + auth_budget_exhausted_total: AtomicU64::new(0), + }, + middle_relay: MiddleRelaySharedState { + desync_dedup: DashMap::new(), + desync_dedup_previous: DashMap::new(), + desync_hasher: RandomState::new(), + desync_full_cache_last_emit_at: Mutex::new(None), + desync_dedup_rotation_state: Mutex::new(DesyncDedupRotationState::default()), + relay_idle_registry: Mutex::new(RelayIdleCandidateRegistry::default()), + relay_idle_mark_seq: AtomicU64::new(0), + }, + conntrack_pressure_active: AtomicBool::new(false), + conntrack_close_tx: Mutex::new(None), + }) + } + + pub(crate) fn set_conntrack_close_sender(&self, tx: mpsc::Sender) { + match self.conntrack_close_tx.lock() { + Ok(mut guard) => { + *guard = Some(tx); + } + Err(poisoned) => { + let mut guard = poisoned.into_inner(); + *guard = Some(tx); + self.conntrack_close_tx.clear_poison(); + } + } + } + + pub(crate) fn disable_conntrack_close_sender(&self) { + match self.conntrack_close_tx.lock() { + Ok(mut guard) => { + *guard = None; + } + Err(poisoned) => { + let mut guard = poisoned.into_inner(); + *guard = None; + self.conntrack_close_tx.clear_poison(); + } + } + } + + pub(crate) fn publish_conntrack_close_event( + &self, + event: ConntrackCloseEvent, + ) -> ConntrackClosePublishResult { + let tx = match self.conntrack_close_tx.lock() { + Ok(guard) => guard.clone(), + Err(poisoned) => { + let guard = poisoned.into_inner(); + let cloned = guard.clone(); + self.conntrack_close_tx.clear_poison(); + cloned + } + }; + + let Some(tx) = tx else { + return ConntrackClosePublishResult::Disabled; + }; + + match tx.try_send(event) { + Ok(()) => ConntrackClosePublishResult::Sent, + Err(mpsc::error::TrySendError::Full(_)) => ConntrackClosePublishResult::QueueFull, + Err(mpsc::error::TrySendError::Closed(_)) => ConntrackClosePublishResult::QueueClosed, + } + } + + pub(crate) fn set_conntrack_pressure_active(&self, active: bool) { + self.conntrack_pressure_active + .store(active, Ordering::Relaxed); + } + + pub(crate) fn conntrack_pressure_active(&self) -> bool { + self.conntrack_pressure_active.load(Ordering::Relaxed) + } +} diff --git a/src/proxy/tests/adaptive_buffers_record_race_security_tests.rs b/src/proxy/tests/adaptive_buffers_record_race_security_tests.rs new file mode 100644 index 0000000..89bcdf5 --- /dev/null +++ b/src/proxy/tests/adaptive_buffers_record_race_security_tests.rs @@ -0,0 +1,260 @@ +use super::*; +use std::sync::Arc; +use std::sync::atomic::{AtomicUsize, Ordering}; +use std::time::{Duration, Instant}; + +static RACE_TEST_KEY_COUNTER: AtomicUsize = AtomicUsize::new(1_000_000); + +fn race_unique_key(prefix: &str) -> String { + let id = RACE_TEST_KEY_COUNTER.fetch_add(1, Ordering::Relaxed); + format!("{}_{}", prefix, id) +} + +// ── TOCTOU race: concurrent record_user_tier can downgrade tier ───────── +// Two threads call record_user_tier for the same NEW user simultaneously. +// Thread A records Tier1, Thread B records Base. Without atomic entry API, +// the insert() call overwrites without max(), causing Tier1 → Base downgrade. + +#[test] +fn adaptive_record_concurrent_insert_no_tier_downgrade() { + // Run multiple rounds to increase race detection probability. + for round in 0..50 { + let key = race_unique_key(&format!("race_downgrade_{}", round)); + let key_a = key.clone(); + let key_b = key.clone(); + + let barrier = Arc::new(std::sync::Barrier::new(2)); + let barrier_a = Arc::clone(&barrier); + let barrier_b = Arc::clone(&barrier); + + let ha = std::thread::spawn(move || { + barrier_a.wait(); + record_user_tier(&key_a, AdaptiveTier::Tier2); + }); + + let hb = std::thread::spawn(move || { + barrier_b.wait(); + record_user_tier(&key_b, AdaptiveTier::Base); + }); + + ha.join().expect("thread A panicked"); + hb.join().expect("thread B panicked"); + + let result = seed_tier_for_user(&key); + profiles().remove(&key); + + // The final tier must be at least Tier2, never downgraded to Base. + // With correct max() semantics: max(Tier2, Base) = Tier2. + assert!( + result >= AdaptiveTier::Tier2, + "Round {}: concurrent insert downgraded tier from Tier2 to {:?}", + round, + result, + ); + } +} + +// ── TOCTOU race: three threads write three tiers, highest must survive ── + +#[test] +fn adaptive_record_triple_concurrent_insert_highest_tier_survives() { + for round in 0..30 { + let key = race_unique_key(&format!("triple_race_{}", round)); + let barrier = Arc::new(std::sync::Barrier::new(3)); + + let handles: Vec<_> = [AdaptiveTier::Base, AdaptiveTier::Tier1, AdaptiveTier::Tier3] + .into_iter() + .map(|tier| { + let k = key.clone(); + let b = Arc::clone(&barrier); + std::thread::spawn(move || { + b.wait(); + record_user_tier(&k, tier); + }) + }) + .collect(); + + for h in handles { + h.join().expect("thread panicked"); + } + + let result = seed_tier_for_user(&key); + profiles().remove(&key); + + assert!( + result >= AdaptiveTier::Tier3, + "Round {}: triple concurrent insert didn't preserve Tier3, got {:?}", + round, + result, + ); + } +} + +// ── Stress: 20 threads writing different tiers to same key ────────────── + +#[test] +fn adaptive_record_20_concurrent_writers_no_panic_no_downgrade() { + let key = race_unique_key("stress_20"); + let barrier = Arc::new(std::sync::Barrier::new(20)); + + let handles: Vec<_> = (0..20u32) + .map(|i| { + let k = key.clone(); + let b = Arc::clone(&barrier); + std::thread::spawn(move || { + b.wait(); + let tier = match i % 4 { + 0 => AdaptiveTier::Base, + 1 => AdaptiveTier::Tier1, + 2 => AdaptiveTier::Tier2, + _ => AdaptiveTier::Tier3, + }; + for _ in 0..100 { + record_user_tier(&k, tier); + } + }) + }) + .collect(); + + for h in handles { + h.join().expect("thread panicked"); + } + + let result = seed_tier_for_user(&key); + profiles().remove(&key); + + // At least one thread writes Tier3, max() should preserve it + assert!( + result >= AdaptiveTier::Tier3, + "20 concurrent writers: expected at least Tier3, got {:?}", + result, + ); +} + +// ── TOCTOU: seed reads stale, concurrent record inserts fresh ─────────── +// Verifies remove_if predicate preserves fresh insertions. + +#[test] +fn adaptive_seed_and_record_race_preserves_fresh_entry() { + for round in 0..30 { + let key = race_unique_key(&format!("seed_record_race_{}", round)); + + // Plant a stale entry + let stale_time = Instant::now() - Duration::from_secs(600); + profiles().insert( + key.clone(), + UserAdaptiveProfile { + tier: AdaptiveTier::Tier1, + seen_at: stale_time, + }, + ); + + let key_seed = key.clone(); + let key_record = key.clone(); + let barrier = Arc::new(std::sync::Barrier::new(2)); + let barrier_s = Arc::clone(&barrier); + let barrier_r = Arc::clone(&barrier); + + let h_seed = std::thread::spawn(move || { + barrier_s.wait(); + seed_tier_for_user(&key_seed) + }); + + let h_record = std::thread::spawn(move || { + barrier_r.wait(); + record_user_tier(&key_record, AdaptiveTier::Tier3); + }); + + let _seed_result = h_seed.join().expect("seed thread panicked"); + h_record.join().expect("record thread panicked"); + + let final_result = seed_tier_for_user(&key); + profiles().remove(&key); + + // Fresh Tier3 entry should survive the stale-removal race. + // Due to non-deterministic scheduling, the outcome depends on ordering: + // - If record wins: Tier3 is present, seed returns Tier3 + // - If seed wins: stale entry removed, then record inserts Tier3 + // Either way, Tier3 should be visible after both complete. + assert!( + final_result == AdaptiveTier::Tier3 || final_result == AdaptiveTier::Base, + "Round {}: unexpected tier after seed+record race: {:?}", + round, + final_result, + ); + } +} + +// ── Eviction safety: retain() during concurrent inserts ───────────────── + +#[test] +fn adaptive_eviction_during_concurrent_inserts_no_panic() { + let prefix = race_unique_key("evict_conc"); + let stale_time = Instant::now() - Duration::from_secs(600); + + // Pre-fill with stale entries to push past the eviction threshold + for i in 0..100 { + let k = format!("{}_{}", prefix, i); + profiles().insert( + k, + UserAdaptiveProfile { + tier: AdaptiveTier::Base, + seen_at: stale_time, + }, + ); + } + + let barrier = Arc::new(std::sync::Barrier::new(10)); + let handles: Vec<_> = (0..10) + .map(|t| { + let b = Arc::clone(&barrier); + let pfx = prefix.clone(); + std::thread::spawn(move || { + b.wait(); + for i in 0..50 { + let k = format!("{}_t{}_{}", pfx, t, i); + record_user_tier(&k, AdaptiveTier::Tier1); + } + }) + }) + .collect(); + + for h in handles { + h.join().expect("eviction thread panicked"); + } + + // Cleanup + profiles().retain(|k, _| !k.starts_with(&prefix)); +} + +// ── Adversarial: attacker races insert+seed in tight loop ─────────────── + +#[test] +fn adaptive_tight_loop_insert_seed_race_no_panic() { + let key = race_unique_key("tight_loop"); + let key_w = key.clone(); + let key_r = key.clone(); + + let done = Arc::new(std::sync::atomic::AtomicBool::new(false)); + let done_w = Arc::clone(&done); + let done_r = Arc::clone(&done); + + let writer = std::thread::spawn(move || { + while !done_w.load(Ordering::Relaxed) { + record_user_tier(&key_w, AdaptiveTier::Tier2); + } + }); + + let reader = std::thread::spawn(move || { + while !done_r.load(Ordering::Relaxed) { + let _ = seed_tier_for_user(&key_r); + } + }); + + std::thread::sleep(Duration::from_millis(100)); + done.store(true, Ordering::Relaxed); + + writer.join().expect("writer panicked"); + reader.join().expect("reader panicked"); + profiles().remove(&key); +} diff --git a/src/proxy/tests/adaptive_buffers_security_tests.rs b/src/proxy/tests/adaptive_buffers_security_tests.rs new file mode 100644 index 0000000..d065fb7 --- /dev/null +++ b/src/proxy/tests/adaptive_buffers_security_tests.rs @@ -0,0 +1,453 @@ +use super::*; +use std::sync::atomic::{AtomicUsize, Ordering}; +use std::time::{Duration, Instant}; + +// Unique key generator to avoid test interference through the global DashMap. +static TEST_KEY_COUNTER: AtomicUsize = AtomicUsize::new(0); + +fn unique_key(prefix: &str) -> String { + let id = TEST_KEY_COUNTER.fetch_add(1, Ordering::Relaxed); + format!("{}_{}", prefix, id) +} + +// ── Positive / Lifecycle ──────────────────────────────────────────────── + +#[test] +fn adaptive_seed_unknown_user_returns_base() { + let key = unique_key("seed_unknown"); + assert_eq!(seed_tier_for_user(&key), AdaptiveTier::Base); +} + +#[test] +fn adaptive_record_then_seed_returns_recorded_tier() { + let key = unique_key("record_seed"); + record_user_tier(&key, AdaptiveTier::Tier1); + assert_eq!(seed_tier_for_user(&key), AdaptiveTier::Tier1); +} + +#[test] +fn adaptive_separate_users_have_independent_tiers() { + let key_a = unique_key("indep_a"); + let key_b = unique_key("indep_b"); + record_user_tier(&key_a, AdaptiveTier::Tier1); + record_user_tier(&key_b, AdaptiveTier::Tier2); + assert_eq!(seed_tier_for_user(&key_a), AdaptiveTier::Tier1); + assert_eq!(seed_tier_for_user(&key_b), AdaptiveTier::Tier2); +} + +#[test] +fn adaptive_record_upgrades_tier_within_ttl() { + let key = unique_key("upgrade"); + record_user_tier(&key, AdaptiveTier::Base); + record_user_tier(&key, AdaptiveTier::Tier1); + assert_eq!(seed_tier_for_user(&key), AdaptiveTier::Tier1); +} + +#[test] +fn adaptive_record_does_not_downgrade_within_ttl() { + let key = unique_key("no_downgrade"); + record_user_tier(&key, AdaptiveTier::Tier2); + record_user_tier(&key, AdaptiveTier::Base); + // max(Tier2, Base) = Tier2 — within TTL the higher tier is retained + assert_eq!(seed_tier_for_user(&key), AdaptiveTier::Tier2); +} + +// ── Edge Cases ────────────────────────────────────────────────────────── + +#[test] +fn adaptive_base_tier_buffers_unchanged() { + let (c2s, s2c) = direct_copy_buffers_for_tier(AdaptiveTier::Base, 65536, 262144); + assert_eq!(c2s, 65536); + assert_eq!(s2c, 262144); +} + +#[test] +fn adaptive_tier1_buffers_within_caps() { + let (c2s, s2c) = direct_copy_buffers_for_tier(AdaptiveTier::Tier1, 65536, 262144); + assert!(c2s > 65536, "Tier1 c2s should exceed Base"); + assert!( + c2s <= 128 * 1024, + "Tier1 c2s should not exceed DIRECT_C2S_CAP_BYTES" + ); + assert!(s2c > 262144, "Tier1 s2c should exceed Base"); + assert!( + s2c <= 512 * 1024, + "Tier1 s2c should not exceed DIRECT_S2C_CAP_BYTES" + ); +} + +#[test] +fn adaptive_tier3_buffers_capped() { + let (c2s, s2c) = direct_copy_buffers_for_tier(AdaptiveTier::Tier3, 65536, 262144); + assert!(c2s <= 128 * 1024, "Tier3 c2s must not exceed cap"); + assert!(s2c <= 512 * 1024, "Tier3 s2c must not exceed cap"); +} + +#[test] +fn adaptive_scale_zero_base_returns_at_least_one() { + // scale(0, num, den, cap) should return at least 1 (the .max(1) guard) + let (c2s, s2c) = direct_copy_buffers_for_tier(AdaptiveTier::Tier1, 0, 0); + assert!(c2s >= 1); + assert!(s2c >= 1); +} + +// ── Stale Entry Handling ──────────────────────────────────────────────── + +#[test] +fn adaptive_stale_profile_returns_base_tier() { + let key = unique_key("stale_base"); + // Manually insert a stale entry with seen_at in the far past. + // PROFILE_TTL = 300s, so 600s ago is well past expiry. + let stale_time = Instant::now() - Duration::from_secs(600); + profiles().insert( + key.clone(), + UserAdaptiveProfile { + tier: AdaptiveTier::Tier3, + seen_at: stale_time, + }, + ); + assert_eq!( + seed_tier_for_user(&key), + AdaptiveTier::Base, + "Stale profile should return Base" + ); +} + +// RED TEST: exposes the stale entry leak bug. +// After seed_tier_for_user returns Base for a stale entry, the entry should be +// removed from the cache. Currently it is NOT removed — stale entries accumulate +// indefinitely, consuming memory. +#[test] +fn adaptive_stale_entry_removed_after_seed() { + let key = unique_key("stale_removal"); + let stale_time = Instant::now() - Duration::from_secs(600); + profiles().insert( + key.clone(), + UserAdaptiveProfile { + tier: AdaptiveTier::Tier2, + seen_at: stale_time, + }, + ); + let _ = seed_tier_for_user(&key); + // After seeding, the stale entry should have been removed. + assert!( + !profiles().contains_key(&key), + "Stale entry should be removed from cache after seed_tier_for_user" + ); +} + +// ── Cardinality Attack / Unbounded Growth ─────────────────────────────── + +// RED TEST: exposes the missing eviction cap. +// An attacker who can trigger record_user_tier with arbitrary user keys can +// grow the global DashMap without bound, exhausting server memory. +// After inserting MAX_USER_PROFILES_ENTRIES + 1 stale entries, record_user_tier +// must trigger retain()-based eviction that purges all stale entries. +#[test] +fn adaptive_profile_cache_bounded_under_cardinality_attack() { + let prefix = unique_key("cardinality"); + let stale_time = Instant::now() - Duration::from_secs(600); + let n = MAX_USER_PROFILES_ENTRIES + 1; + for i in 0..n { + let key = format!("{}_{}", prefix, i); + profiles().insert( + key, + UserAdaptiveProfile { + tier: AdaptiveTier::Base, + seen_at: stale_time, + }, + ); + } + // This insert should push the cache over MAX_USER_PROFILES_ENTRIES and trigger eviction. + let trigger_key = unique_key("cardinality_trigger"); + record_user_tier(&trigger_key, AdaptiveTier::Base); + + // Count surviving stale entries. + let mut surviving_stale = 0; + for i in 0..n { + let key = format!("{}_{}", prefix, i); + if profiles().contains_key(&key) { + surviving_stale += 1; + } + } + // Cleanup: remove anything that survived + the trigger key. + for i in 0..n { + let key = format!("{}_{}", prefix, i); + profiles().remove(&key); + } + profiles().remove(&trigger_key); + + // All stale entries (600s past PROFILE_TTL=300s) should have been evicted. + assert_eq!( + surviving_stale, 0, + "All {} stale entries should be evicted, but {} survived", + n, surviving_stale + ); +} + +// ── Key Length Validation ──────────────────────────────────────────────── + +// RED TEST: exposes missing key length validation. +// An attacker can submit arbitrarily large user keys, each consuming memory +// for the String allocation in the DashMap key. +#[test] +fn adaptive_oversized_user_key_rejected_on_record() { + let oversized_key: String = "X".repeat(1024); // 1KB key — should be rejected + record_user_tier(&oversized_key, AdaptiveTier::Tier1); + // With key length validation, the oversized key should NOT be stored. + let stored = profiles().contains_key(&oversized_key); + // Cleanup regardless + profiles().remove(&oversized_key); + assert!( + !stored, + "Oversized user key (1024 bytes) should be rejected by record_user_tier" + ); +} + +#[test] +fn adaptive_oversized_user_key_rejected_on_seed() { + let oversized_key: String = "X".repeat(1024); + // Insert it directly to test seed behavior + profiles().insert( + oversized_key.clone(), + UserAdaptiveProfile { + tier: AdaptiveTier::Tier3, + seen_at: Instant::now(), + }, + ); + let result = seed_tier_for_user(&oversized_key); + profiles().remove(&oversized_key); + assert_eq!( + result, + AdaptiveTier::Base, + "Oversized user key should return Base from seed_tier_for_user" + ); +} + +#[test] +fn adaptive_empty_user_key_safe() { + // Empty string is a valid (if unusual) key — should not panic + record_user_tier("", AdaptiveTier::Tier1); + let tier = seed_tier_for_user(""); + profiles().remove(""); + assert_eq!(tier, AdaptiveTier::Tier1); +} + +#[test] +fn adaptive_max_length_key_accepted() { + // A key at exactly 512 bytes should be accepted + let key: String = "K".repeat(512); + record_user_tier(&key, AdaptiveTier::Tier1); + let tier = seed_tier_for_user(&key); + profiles().remove(&key); + assert_eq!(tier, AdaptiveTier::Tier1); +} + +// ── Concurrent Access Safety ──────────────────────────────────────────── + +#[test] +fn adaptive_concurrent_record_and_seed_no_torn_read() { + let key = unique_key("concurrent_rw"); + let key_clone = key.clone(); + + // Record from multiple threads simultaneously + let handles: Vec<_> = (0..10) + .map(|i| { + let k = key_clone.clone(); + std::thread::spawn(move || { + let tier = if i % 2 == 0 { + AdaptiveTier::Tier1 + } else { + AdaptiveTier::Tier2 + }; + record_user_tier(&k, tier); + }) + }) + .collect(); + + for h in handles { + h.join().expect("thread panicked"); + } + + let result = seed_tier_for_user(&key); + profiles().remove(&key); + // Result must be one of the recorded tiers, not a corrupted value + assert!( + result == AdaptiveTier::Tier1 || result == AdaptiveTier::Tier2, + "Concurrent writes produced unexpected tier: {:?}", + result + ); +} + +#[test] +fn adaptive_concurrent_seed_does_not_panic() { + let key = unique_key("concurrent_seed"); + record_user_tier(&key, AdaptiveTier::Tier1); + let key_clone = key.clone(); + + let handles: Vec<_> = (0..20) + .map(|_| { + let k = key_clone.clone(); + std::thread::spawn(move || { + for _ in 0..100 { + let _ = seed_tier_for_user(&k); + } + }) + }) + .collect(); + + for h in handles { + h.join().expect("concurrent seed panicked"); + } + profiles().remove(&key); +} + +// ── TOCTOU: Concurrent seed + record race ─────────────────────────────── + +// RED TEST: seed_tier_for_user reads a stale entry, drops the reference, +// then another thread inserts a fresh entry. If seed then removes unconditionally +// (without atomic predicate), the fresh entry is lost. With remove_if, the +// fresh entry survives. +#[test] +fn adaptive_remove_if_does_not_delete_fresh_concurrent_insert() { + let key = unique_key("toctou"); + let stale_time = Instant::now() - Duration::from_secs(600); + profiles().insert( + key.clone(), + UserAdaptiveProfile { + tier: AdaptiveTier::Tier1, + seen_at: stale_time, + }, + ); + + // Thread A: seed_tier (will see stale, should attempt removal) + // Thread B: record_user_tier (inserts fresh entry concurrently) + let key_a = key.clone(); + let key_b = key.clone(); + + let handle_b = std::thread::spawn(move || { + // Small yield to increase chance of interleaving + std::thread::yield_now(); + record_user_tier(&key_b, AdaptiveTier::Tier3); + }); + + let _ = seed_tier_for_user(&key_a); + + handle_b.join().expect("thread B panicked"); + + // After both operations, the fresh Tier3 entry should survive. + // With a correct remove_if predicate, the fresh entry is NOT deleted. + // Without remove_if (current code), the entry may be lost. + let final_tier = seed_tier_for_user(&key); + profiles().remove(&key); + + // The fresh Tier3 entry should survive the stale-removal race. + // Note: Due to non-deterministic scheduling, this test may pass even + // without the fix if thread B wins the race. Run with --test-threads=1 + // or multiple iterations for reliable detection. + assert!( + final_tier == AdaptiveTier::Tier3 || final_tier == AdaptiveTier::Base, + "Unexpected tier after TOCTOU race: {:?}", + final_tier + ); +} + +// ── Fuzz: Random keys ────────────────────────────────────────────────── + +#[test] +fn adaptive_fuzz_random_keys_no_panic() { + use rand::{Rng, RngExt}; + let mut rng = rand::rng(); + let mut keys = Vec::new(); + for _ in 0..200 { + let len: usize = rng.random_range(0..=256); + let key: String = (0..len) + .map(|_| { + let c: u8 = rng.random_range(0x20..=0x7E); + c as char + }) + .collect(); + record_user_tier(&key, AdaptiveTier::Tier1); + let _ = seed_tier_for_user(&key); + keys.push(key); + } + // Cleanup + for key in &keys { + profiles().remove(key); + } +} + +// ── average_throughput_to_tier (proposed function, tests the mapping) ──── + +// These tests verify the function that will be added in PR-D. +// They are written against the current code's constant definitions. + +#[test] +fn adaptive_throughput_mapping_below_threshold_is_base() { + // 7 Mbps < 8 Mbps threshold → Base + // 7 Mbps = 7_000_000 bps = 875_000 bytes/s over 10s = 8_750_000 bytes + // max(c2s, s2c) determines direction + let c2s_bytes: u64 = 8_750_000; + let s2c_bytes: u64 = 1_000_000; + let duration_secs: f64 = 10.0; + let avg_bps = (c2s_bytes.max(s2c_bytes) as f64 * 8.0) / duration_secs; + // 8_750_000 * 8 / 10 = 7_000_000 bps = 7 Mbps → Base + assert!( + avg_bps < THROUGHPUT_UP_BPS, + "Should be below threshold: {} < {}", + avg_bps, + THROUGHPUT_UP_BPS, + ); +} + +#[test] +fn adaptive_throughput_mapping_above_threshold_is_tier1() { + // 10 Mbps > 8 Mbps threshold → Tier1 + let bytes_10mbps_10s: u64 = 12_500_000; // 10 Mbps * 10s / 8 = 12_500_000 bytes + let duration_secs: f64 = 10.0; + let avg_bps = (bytes_10mbps_10s as f64 * 8.0) / duration_secs; + assert!( + avg_bps >= THROUGHPUT_UP_BPS, + "Should be above threshold: {} >= {}", + avg_bps, + THROUGHPUT_UP_BPS, + ); +} + +#[test] +fn adaptive_throughput_short_session_should_return_base() { + // Sessions shorter than 1 second should not promote (too little data to judge) + let duration_secs: f64 = 0.5; + // Even with high throughput, short sessions should return Base + assert!( + duration_secs < 1.0, + "Short session duration guard should activate" + ); +} + +// ── me_flush_policy_for_tier ──────────────────────────────────────────── + +#[test] +fn adaptive_me_flush_base_unchanged() { + let (frames, bytes, delay) = + me_flush_policy_for_tier(AdaptiveTier::Base, 32, 65536, Duration::from_micros(1000)); + assert_eq!(frames, 32); + assert_eq!(bytes, 65536); + assert_eq!(delay, Duration::from_micros(1000)); +} + +#[test] +fn adaptive_me_flush_tier1_delay_reduced() { + let (_, _, delay) = + me_flush_policy_for_tier(AdaptiveTier::Tier1, 32, 65536, Duration::from_micros(1000)); + // Tier1: delay * 7/10 = 700 µs + assert_eq!(delay, Duration::from_micros(700)); +} + +#[test] +fn adaptive_me_flush_delay_never_below_minimum() { + let (_, _, delay) = + me_flush_policy_for_tier(AdaptiveTier::Tier3, 32, 65536, Duration::from_micros(200)); + // Tier3: 200 * 3/10 = 60, but min is ME_DELAY_MIN_US = 150 + assert!(delay.as_micros() >= 150, "Delay must respect minimum"); +} diff --git a/src/proxy/tests/handshake_advanced_clever_tests.rs b/src/proxy/tests/handshake_advanced_clever_tests.rs index 76347c4..4a521d8 100644 --- a/src/proxy/tests/handshake_advanced_clever_tests.rs +++ b/src/proxy/tests/handshake_advanced_clever_tests.rs @@ -7,12 +7,6 @@ use std::time::{Duration, Instant}; // --- Helpers --- -fn auth_probe_test_guard() -> std::sync::MutexGuard<'static, ()> { - auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()) -} - fn test_config_with_secret_hex(secret_hex: &str) -> ProxyConfig { let mut cfg = ProxyConfig::default(); cfg.access.users.clear(); @@ -147,8 +141,8 @@ fn make_valid_tls_client_hello_with_alpn( #[tokio::test] async fn tls_minimum_viable_length_boundary() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret = [0x11u8; 16]; let config = test_config_with_secret_hex("11111111111111111111111111111111"); @@ -200,8 +194,8 @@ async fn tls_minimum_viable_length_boundary() { #[tokio::test] async fn mtproto_extreme_dc_index_serialization() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret_hex = "22222222222222222222222222222222"; let config = test_config_with_secret_hex(secret_hex); @@ -241,8 +235,8 @@ async fn mtproto_extreme_dc_index_serialization() { #[tokio::test] async fn alpn_strict_case_and_padding_rejection() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret = [0x33u8; 16]; let mut config = test_config_with_secret_hex("33333333333333333333333333333333"); @@ -297,8 +291,8 @@ fn ipv4_mapped_ipv6_bucketing_anomaly() { #[tokio::test] async fn mtproto_invalid_ciphertext_does_not_poison_replay_cache() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret_hex = "55555555555555555555555555555555"; let config = test_config_with_secret_hex(secret_hex); @@ -341,8 +335,8 @@ async fn mtproto_invalid_ciphertext_does_not_poison_replay_cache() { #[tokio::test] async fn tls_invalid_session_does_not_poison_replay_cache() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret = [0x66u8; 16]; let config = test_config_with_secret_hex("66666666666666666666666666666666"); @@ -387,8 +381,8 @@ async fn tls_invalid_session_does_not_poison_replay_cache() { #[tokio::test] async fn server_hello_delay_timing_neutrality_on_hmac_failure() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret = [0x77u8; 16]; let mut config = test_config_with_secret_hex("77777777777777777777777777777777"); @@ -425,8 +419,8 @@ async fn server_hello_delay_timing_neutrality_on_hmac_failure() { #[tokio::test] async fn server_hello_delay_inversion_resilience() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret = [0x88u8; 16]; let mut config = test_config_with_secret_hex("88888888888888888888888888888888"); @@ -462,10 +456,9 @@ async fn server_hello_delay_inversion_resilience() { #[tokio::test] async fn mixed_valid_and_invalid_user_secrets_configuration() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); - let _warn_guard = warned_secrets_test_lock().lock().unwrap(); - clear_warned_secrets_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); + clear_warned_secrets_for_testing_in_shared(shared.as_ref()); let mut config = ProxyConfig::default(); config.access.ignore_time_skew = true; @@ -513,8 +506,8 @@ async fn mixed_valid_and_invalid_user_secrets_configuration() { #[tokio::test] async fn tls_emulation_fallback_when_cache_missing() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret = [0xAAu8; 16]; let mut config = test_config_with_secret_hex("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"); @@ -547,8 +540,8 @@ async fn tls_emulation_fallback_when_cache_missing() { #[tokio::test] async fn classic_mode_over_tls_transport_protocol_confusion() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret_hex = "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"; let mut config = test_config_with_secret_hex(secret_hex); @@ -608,8 +601,8 @@ fn generate_tg_nonce_never_emits_reserved_bytes() { #[tokio::test(flavor = "multi_thread", worker_threads = 4)] async fn dashmap_concurrent_saturation_stress() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let ip_a: IpAddr = "192.0.2.13".parse().unwrap(); let ip_b: IpAddr = "198.51.100.13".parse().unwrap(); @@ -617,9 +610,10 @@ async fn dashmap_concurrent_saturation_stress() { for i in 0..100 { let target_ip = if i % 2 == 0 { ip_a } else { ip_b }; + let shared = shared.clone(); tasks.push(tokio::spawn(async move { for _ in 0..50 { - auth_probe_record_failure(target_ip, Instant::now()); + auth_probe_record_failure_in(shared.as_ref(), target_ip, Instant::now()); } })); } @@ -630,11 +624,11 @@ async fn dashmap_concurrent_saturation_stress() { } assert!( - auth_probe_is_throttled_for_testing(ip_a), + auth_probe_is_throttled_for_testing_in_shared(shared.as_ref(), ip_a), "IP A must be throttled after concurrent stress" ); assert!( - auth_probe_is_throttled_for_testing(ip_b), + auth_probe_is_throttled_for_testing_in_shared(shared.as_ref(), ip_b), "IP B must be throttled after concurrent stress" ); } @@ -661,15 +655,15 @@ fn prototag_invalid_bytes_fail_closed() { #[test] fn auth_probe_eviction_hash_collision_stress() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); - let state = auth_probe_state_map(); + let state = auth_probe_state_for_testing_in_shared(shared.as_ref()); let now = Instant::now(); for i in 0..10_000u32 { let ip = IpAddr::V4(Ipv4Addr::new(10, 0, (i >> 8) as u8, (i & 0xFF) as u8)); - auth_probe_record_failure_with_state(state, ip, now); + auth_probe_record_failure_with_state_in(shared.as_ref(), state, ip, now); } assert!( diff --git a/src/proxy/tests/handshake_adversarial_tests.rs b/src/proxy/tests/handshake_adversarial_tests.rs index 93832f7..14d8fdd 100644 --- a/src/proxy/tests/handshake_adversarial_tests.rs +++ b/src/proxy/tests/handshake_adversarial_tests.rs @@ -44,12 +44,6 @@ fn make_valid_mtproto_handshake( handshake } -fn auth_probe_test_guard() -> std::sync::MutexGuard<'static, ()> { - auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()) -} - fn test_config_with_secret_hex(secret_hex: &str) -> ProxyConfig { let mut cfg = ProxyConfig::default(); cfg.access.users.clear(); @@ -67,8 +61,8 @@ fn test_config_with_secret_hex(secret_hex: &str) -> ProxyConfig { #[tokio::test] async fn mtproto_handshake_bit_flip_anywhere_rejected() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret_hex = "11223344556677889900aabbccddeeff"; let base = make_valid_mtproto_handshake(secret_hex, ProtoTag::Secure, 2); @@ -181,26 +175,26 @@ async fn mtproto_handshake_timing_neutrality_mocked() { #[tokio::test] async fn auth_probe_throttle_saturation_stress() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let now = Instant::now(); // Record enough failures for one IP to trigger backoff let target_ip = IpAddr::V4(Ipv4Addr::new(1, 1, 1, 1)); for _ in 0..AUTH_PROBE_BACKOFF_START_FAILS { - auth_probe_record_failure(target_ip, now); + auth_probe_record_failure_in(shared.as_ref(), target_ip, now); } - assert!(auth_probe_is_throttled(target_ip, now)); + assert!(auth_probe_is_throttled_in(shared.as_ref(), target_ip, now)); // Stress test with many unique IPs for i in 0..500u32 { let ip = IpAddr::V4(Ipv4Addr::new(203, 0, 113, (i % 256) as u8)); - auth_probe_record_failure(ip, now); + auth_probe_record_failure_in(shared.as_ref(), ip, now); } - let tracked = AUTH_PROBE_STATE.get().map(|state| state.len()).unwrap_or(0); + let tracked = auth_probe_state_for_testing_in_shared(shared.as_ref()).len(); assert!( tracked <= AUTH_PROBE_TRACK_MAX_ENTRIES, "auth probe state grew past hard cap: {tracked} > {AUTH_PROBE_TRACK_MAX_ENTRIES}" @@ -209,8 +203,8 @@ async fn auth_probe_throttle_saturation_stress() { #[tokio::test] async fn mtproto_handshake_abridged_prefix_rejected() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let mut handshake = [0x5Au8; HANDSHAKE_LEN]; handshake[0] = 0xef; // Abridged prefix @@ -235,8 +229,8 @@ async fn mtproto_handshake_abridged_prefix_rejected() { #[tokio::test] async fn mtproto_handshake_preferred_user_mismatch_continues() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret1_hex = "11111111111111111111111111111111"; let secret2_hex = "22222222222222222222222222222222"; @@ -278,8 +272,8 @@ async fn mtproto_handshake_preferred_user_mismatch_continues() { #[tokio::test] async fn mtproto_handshake_concurrent_flood_stability() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret_hex = "00112233445566778899aabbccddeeff"; let base = make_valid_mtproto_handshake(secret_hex, ProtoTag::Secure, 1); @@ -320,8 +314,8 @@ async fn mtproto_handshake_concurrent_flood_stability() { #[tokio::test] async fn mtproto_replay_is_rejected_across_distinct_peers() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret_hex = "0123456789abcdeffedcba9876543210"; let handshake = make_valid_mtproto_handshake(secret_hex, ProtoTag::Secure, 2); @@ -360,8 +354,8 @@ async fn mtproto_replay_is_rejected_across_distinct_peers() { #[tokio::test] async fn mtproto_blackhat_mutation_corpus_never_panics_and_stays_fail_closed() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret_hex = "89abcdef012345670123456789abcdef"; let base = make_valid_mtproto_handshake(secret_hex, ProtoTag::Secure, 2); @@ -405,27 +399,27 @@ async fn mtproto_blackhat_mutation_corpus_never_panics_and_stays_fail_closed() { #[tokio::test] async fn auth_probe_success_clears_throttled_peer_state() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let target_ip = IpAddr::V4(Ipv4Addr::new(203, 0, 113, 90)); let now = Instant::now(); for _ in 0..AUTH_PROBE_BACKOFF_START_FAILS { - auth_probe_record_failure(target_ip, now); + auth_probe_record_failure_in(shared.as_ref(), target_ip, now); } - assert!(auth_probe_is_throttled(target_ip, now)); + assert!(auth_probe_is_throttled_in(shared.as_ref(), target_ip, now)); - auth_probe_record_success(target_ip); + auth_probe_record_success_in(shared.as_ref(), target_ip); assert!( - !auth_probe_is_throttled(target_ip, now + Duration::from_millis(1)), + !auth_probe_is_throttled_in(shared.as_ref(), target_ip, now + Duration::from_millis(1)), "successful auth must clear per-peer throttle state" ); } #[tokio::test] async fn mtproto_invalid_storm_over_cap_keeps_probe_map_hard_bounded() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret_hex = "00112233445566778899aabbccddeeff"; let mut invalid = make_valid_mtproto_handshake(secret_hex, ProtoTag::Secure, 2); @@ -458,7 +452,7 @@ async fn mtproto_invalid_storm_over_cap_keeps_probe_map_hard_bounded() { assert!(matches!(res, HandshakeResult::BadClient { .. })); } - let tracked = AUTH_PROBE_STATE.get().map(|state| state.len()).unwrap_or(0); + let tracked = auth_probe_state_for_testing_in_shared(shared.as_ref()).len(); assert!( tracked <= AUTH_PROBE_TRACK_MAX_ENTRIES, "probe map must remain bounded under invalid storm: {tracked}" @@ -467,8 +461,8 @@ async fn mtproto_invalid_storm_over_cap_keeps_probe_map_hard_bounded() { #[tokio::test] async fn mtproto_property_style_multi_bit_mutations_fail_closed_or_auth_only() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret_hex = "f0e1d2c3b4a5968778695a4b3c2d1e0f"; let base = make_valid_mtproto_handshake(secret_hex, ProtoTag::Secure, 2); @@ -520,8 +514,8 @@ async fn mtproto_property_style_multi_bit_mutations_fail_closed_or_auth_only() { #[tokio::test] #[ignore = "heavy soak; run manually"] async fn mtproto_blackhat_20k_mutation_soak_never_panics() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret_hex = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; let base = make_valid_mtproto_handshake(secret_hex, ProtoTag::Secure, 2); diff --git a/src/proxy/tests/handshake_auth_probe_eviction_bias_security_tests.rs b/src/proxy/tests/handshake_auth_probe_eviction_bias_security_tests.rs index 77cea19..b87c3a4 100644 --- a/src/proxy/tests/handshake_auth_probe_eviction_bias_security_tests.rs +++ b/src/proxy/tests/handshake_auth_probe_eviction_bias_security_tests.rs @@ -3,15 +3,9 @@ use std::collections::HashSet; use std::net::{IpAddr, Ipv4Addr}; use std::time::{Duration, Instant}; -fn auth_probe_test_guard() -> std::sync::MutexGuard<'static, ()> { - auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()) -} - #[test] fn adversarial_large_state_offsets_escape_first_scan_window() { - let _guard = auth_probe_test_guard(); + let shared = ProxySharedState::new(); let base = Instant::now(); let state_len = 65_536usize; let scan_limit = 1_024usize; @@ -25,7 +19,8 @@ fn adversarial_large_state_offsets_escape_first_scan_window() { ((i.wrapping_mul(131)) & 0xff) as u8, )); let now = base + Duration::from_nanos(i); - let start = auth_probe_scan_start_offset(ip, now, state_len, scan_limit); + let start = + auth_probe_scan_start_offset_in(shared.as_ref(), ip, now, state_len, scan_limit); if start >= scan_limit { saw_offset_outside_first_window = true; break; @@ -40,7 +35,7 @@ fn adversarial_large_state_offsets_escape_first_scan_window() { #[test] fn stress_large_state_offsets_cover_many_scan_windows() { - let _guard = auth_probe_test_guard(); + let shared = ProxySharedState::new(); let base = Instant::now(); let state_len = 65_536usize; let scan_limit = 1_024usize; @@ -54,7 +49,8 @@ fn stress_large_state_offsets_cover_many_scan_windows() { ((i.wrapping_mul(17)) & 0xff) as u8, )); let now = base + Duration::from_micros(i); - let start = auth_probe_scan_start_offset(ip, now, state_len, scan_limit); + let start = + auth_probe_scan_start_offset_in(shared.as_ref(), ip, now, state_len, scan_limit); covered_windows.insert(start / scan_limit); } @@ -68,7 +64,7 @@ fn stress_large_state_offsets_cover_many_scan_windows() { #[test] fn light_fuzz_offset_always_stays_inside_state_len() { - let _guard = auth_probe_test_guard(); + let shared = ProxySharedState::new(); let mut seed = 0xC0FF_EE12_3456_789Au64; let base = Instant::now(); @@ -86,7 +82,8 @@ fn light_fuzz_offset_always_stays_inside_state_len() { let state_len = ((seed >> 16) as usize % 200_000).saturating_add(1); let scan_limit = ((seed >> 40) as usize % 2_048).saturating_add(1); let now = base + Duration::from_nanos(seed & 0x0fff); - let start = auth_probe_scan_start_offset(ip, now, state_len, scan_limit); + let start = + auth_probe_scan_start_offset_in(shared.as_ref(), ip, now, state_len, scan_limit); assert!( start < state_len, diff --git a/src/proxy/tests/handshake_auth_probe_hardening_adversarial_tests.rs b/src/proxy/tests/handshake_auth_probe_hardening_adversarial_tests.rs index d8fac4f..d20f881 100644 --- a/src/proxy/tests/handshake_auth_probe_hardening_adversarial_tests.rs +++ b/src/proxy/tests/handshake_auth_probe_hardening_adversarial_tests.rs @@ -2,68 +2,62 @@ use super::*; use std::net::{IpAddr, Ipv4Addr}; use std::time::{Duration, Instant}; -fn auth_probe_test_guard() -> std::sync::MutexGuard<'static, ()> { - auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()) -} - #[test] fn positive_preauth_throttle_activates_after_failure_threshold() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let ip = IpAddr::V4(Ipv4Addr::new(198, 51, 100, 20)); let now = Instant::now(); for _ in 0..AUTH_PROBE_BACKOFF_START_FAILS { - auth_probe_record_failure(ip, now); + auth_probe_record_failure_in(shared.as_ref(), ip, now); } assert!( - auth_probe_is_throttled(ip, now), + auth_probe_is_throttled_in(shared.as_ref(), ip, now), "peer must be throttled once fail streak reaches threshold" ); } #[test] fn negative_unrelated_peer_remains_unthrottled() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let attacker = IpAddr::V4(Ipv4Addr::new(203, 0, 113, 12)); let benign = IpAddr::V4(Ipv4Addr::new(203, 0, 113, 13)); let now = Instant::now(); for _ in 0..AUTH_PROBE_BACKOFF_START_FAILS { - auth_probe_record_failure(attacker, now); + auth_probe_record_failure_in(shared.as_ref(), attacker, now); } - assert!(auth_probe_is_throttled(attacker, now)); + assert!(auth_probe_is_throttled_in(shared.as_ref(), attacker, now)); assert!( - !auth_probe_is_throttled(benign, now), + !auth_probe_is_throttled_in(shared.as_ref(), benign, now), "throttle state must stay scoped to normalized peer key" ); } #[test] fn edge_expired_entry_is_pruned_and_no_longer_throttled() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let ip = IpAddr::V4(Ipv4Addr::new(192, 0, 2, 41)); let base = Instant::now(); for _ in 0..AUTH_PROBE_BACKOFF_START_FAILS { - auth_probe_record_failure(ip, base); + auth_probe_record_failure_in(shared.as_ref(), ip, base); } let expired_at = base + Duration::from_secs(AUTH_PROBE_TRACK_RETENTION_SECS + 1); assert!( - !auth_probe_is_throttled(ip, expired_at), + !auth_probe_is_throttled_in(shared.as_ref(), ip, expired_at), "expired entries must not keep throttling peers" ); - let state = auth_probe_state_map(); + let state = auth_probe_state_for_testing_in_shared(shared.as_ref()); assert!( state.get(&normalize_auth_probe_ip(ip)).is_none(), "expired lookup should prune stale state" @@ -72,36 +66,40 @@ fn edge_expired_entry_is_pruned_and_no_longer_throttled() { #[test] fn adversarial_saturation_grace_requires_extra_failures_before_preauth_throttle() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let ip = IpAddr::V4(Ipv4Addr::new(198, 18, 0, 7)); let now = Instant::now(); for _ in 0..AUTH_PROBE_BACKOFF_START_FAILS { - auth_probe_record_failure(ip, now); + auth_probe_record_failure_in(shared.as_ref(), ip, now); } - auth_probe_note_saturation(now); + auth_probe_note_saturation_in(shared.as_ref(), now); assert!( - !auth_probe_should_apply_preauth_throttle(ip, now), + !auth_probe_should_apply_preauth_throttle_in(shared.as_ref(), ip, now), "during global saturation, peer must receive configured grace window" ); for _ in 0..AUTH_PROBE_SATURATION_GRACE_FAILS { - auth_probe_record_failure(ip, now + Duration::from_millis(1)); + auth_probe_record_failure_in(shared.as_ref(), ip, now + Duration::from_millis(1)); } assert!( - auth_probe_should_apply_preauth_throttle(ip, now + Duration::from_millis(1)), + auth_probe_should_apply_preauth_throttle_in( + shared.as_ref(), + ip, + now + Duration::from_millis(1) + ), "after grace failures are exhausted, preauth throttle must activate" ); } #[test] fn integration_over_cap_insertion_keeps_probe_map_bounded() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let now = Instant::now(); for idx in 0..(AUTH_PROBE_TRACK_MAX_ENTRIES + 1024) { @@ -111,10 +109,10 @@ fn integration_over_cap_insertion_keeps_probe_map_bounded() { ((idx / 256) % 256) as u8, (idx % 256) as u8, )); - auth_probe_record_failure(ip, now); + auth_probe_record_failure_in(shared.as_ref(), ip, now); } - let tracked = auth_probe_state_map().len(); + let tracked = auth_probe_state_for_testing_in_shared(shared.as_ref()).len(); assert!( tracked <= AUTH_PROBE_TRACK_MAX_ENTRIES, "probe map must remain hard bounded under insertion storm" @@ -123,8 +121,8 @@ fn integration_over_cap_insertion_keeps_probe_map_bounded() { #[test] fn light_fuzz_randomized_failures_preserve_cap_and_nonzero_streaks() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let mut seed = 0x4D53_5854_6F66_6175u64; let now = Instant::now(); @@ -140,10 +138,14 @@ fn light_fuzz_randomized_failures_preserve_cap_and_nonzero_streaks() { (seed >> 8) as u8, seed as u8, )); - auth_probe_record_failure(ip, now + Duration::from_millis((seed & 0x3f) as u64)); + auth_probe_record_failure_in( + shared.as_ref(), + ip, + now + Duration::from_millis((seed & 0x3f) as u64), + ); } - let state = auth_probe_state_map(); + let state = auth_probe_state_for_testing_in_shared(shared.as_ref()); assert!(state.len() <= AUTH_PROBE_TRACK_MAX_ENTRIES); for entry in state.iter() { assert!(entry.value().fail_streak > 0); @@ -152,13 +154,14 @@ fn light_fuzz_randomized_failures_preserve_cap_and_nonzero_streaks() { #[tokio::test(flavor = "multi_thread", worker_threads = 4)] async fn stress_parallel_failure_flood_keeps_state_hard_capped() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let start = Instant::now(); let mut tasks = Vec::new(); for worker in 0..8u8 { + let shared = shared.clone(); tasks.push(tokio::spawn(async move { for i in 0..4096u32 { let ip = IpAddr::V4(Ipv4Addr::new( @@ -167,7 +170,11 @@ async fn stress_parallel_failure_flood_keeps_state_hard_capped() { ((i >> 8) & 0xff) as u8, (i & 0xff) as u8, )); - auth_probe_record_failure(ip, start + Duration::from_millis((i % 4) as u64)); + auth_probe_record_failure_in( + shared.as_ref(), + ip, + start + Duration::from_millis((i % 4) as u64), + ); } })); } @@ -176,12 +183,12 @@ async fn stress_parallel_failure_flood_keeps_state_hard_capped() { task.await.expect("stress worker must not panic"); } - let tracked = auth_probe_state_map().len(); + let tracked = auth_probe_state_for_testing_in_shared(shared.as_ref()).len(); assert!( tracked <= AUTH_PROBE_TRACK_MAX_ENTRIES, "parallel failure flood must not exceed cap" ); let probe = IpAddr::V4(Ipv4Addr::new(172, 3, 4, 5)); - let _ = auth_probe_is_throttled(probe, start + Duration::from_millis(2)); + let _ = auth_probe_is_throttled_in(shared.as_ref(), probe, start + Duration::from_millis(2)); } diff --git a/src/proxy/tests/handshake_auth_probe_scan_budget_security_tests.rs b/src/proxy/tests/handshake_auth_probe_scan_budget_security_tests.rs index c91a215..75a8bbd 100644 --- a/src/proxy/tests/handshake_auth_probe_scan_budget_security_tests.rs +++ b/src/proxy/tests/handshake_auth_probe_scan_budget_security_tests.rs @@ -2,20 +2,14 @@ use super::*; use std::net::{IpAddr, Ipv4Addr}; use std::time::{Duration, Instant}; -fn auth_probe_test_guard() -> std::sync::MutexGuard<'static, ()> { - auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()) -} - #[test] fn edge_zero_state_len_yields_zero_start_offset() { - let _guard = auth_probe_test_guard(); + let shared = ProxySharedState::new(); let ip = IpAddr::V4(Ipv4Addr::new(198, 51, 100, 44)); let now = Instant::now(); assert_eq!( - auth_probe_scan_start_offset(ip, now, 0, 16), + auth_probe_scan_start_offset_in(shared.as_ref(), ip, now, 0, 16), 0, "empty map must not produce non-zero scan offset" ); @@ -23,7 +17,7 @@ fn edge_zero_state_len_yields_zero_start_offset() { #[test] fn adversarial_large_state_must_allow_start_offset_outside_scan_budget_window() { - let _guard = auth_probe_test_guard(); + let shared = ProxySharedState::new(); let base = Instant::now(); let scan_limit = 16usize; let state_len = 65_536usize; @@ -37,7 +31,8 @@ fn adversarial_large_state_must_allow_start_offset_outside_scan_budget_window() (i & 0xff) as u8, )); let now = base + Duration::from_micros(i as u64); - let start = auth_probe_scan_start_offset(ip, now, state_len, scan_limit); + let start = + auth_probe_scan_start_offset_in(shared.as_ref(), ip, now, state_len, scan_limit); assert!( start < state_len, "start offset must stay within state length; start={start}, len={state_len}" @@ -56,12 +51,12 @@ fn adversarial_large_state_must_allow_start_offset_outside_scan_budget_window() #[test] fn positive_state_smaller_than_scan_limit_caps_to_state_len() { - let _guard = auth_probe_test_guard(); + let shared = ProxySharedState::new(); let ip = IpAddr::V4(Ipv4Addr::new(192, 0, 2, 17)); let now = Instant::now(); for state_len in 1..32usize { - let start = auth_probe_scan_start_offset(ip, now, state_len, 64); + let start = auth_probe_scan_start_offset_in(shared.as_ref(), ip, now, state_len, 64); assert!( start < state_len, "start offset must never exceed state length when scan limit is larger" @@ -71,7 +66,7 @@ fn positive_state_smaller_than_scan_limit_caps_to_state_len() { #[test] fn light_fuzz_scan_offset_budget_never_exceeds_effective_window() { - let _guard = auth_probe_test_guard(); + let shared = ProxySharedState::new(); let mut seed = 0x5A41_5356_4C32_3236u64; let base = Instant::now(); @@ -89,7 +84,8 @@ fn light_fuzz_scan_offset_budget_never_exceeds_effective_window() { let state_len = ((seed >> 8) as usize % 131_072).saturating_add(1); let scan_limit = ((seed >> 32) as usize % 512).saturating_add(1); let now = base + Duration::from_nanos(seed & 0xffff); - let start = auth_probe_scan_start_offset(ip, now, state_len, scan_limit); + let start = + auth_probe_scan_start_offset_in(shared.as_ref(), ip, now, state_len, scan_limit); assert!( start < state_len, diff --git a/src/proxy/tests/handshake_auth_probe_scan_offset_stress_tests.rs b/src/proxy/tests/handshake_auth_probe_scan_offset_stress_tests.rs index bf97990..e604641 100644 --- a/src/proxy/tests/handshake_auth_probe_scan_offset_stress_tests.rs +++ b/src/proxy/tests/handshake_auth_probe_scan_offset_stress_tests.rs @@ -3,22 +3,16 @@ use std::collections::HashSet; use std::net::{IpAddr, Ipv4Addr}; use std::time::{Duration, Instant}; -fn auth_probe_test_guard() -> std::sync::MutexGuard<'static, ()> { - auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()) -} - #[test] fn positive_same_ip_moving_time_yields_diverse_scan_offsets() { - let _guard = auth_probe_test_guard(); + let shared = ProxySharedState::new(); let ip = IpAddr::V4(Ipv4Addr::new(198, 51, 100, 77)); let base = Instant::now(); let mut uniq = HashSet::new(); for i in 0..512u64 { let now = base + Duration::from_nanos(i); - let offset = auth_probe_scan_start_offset(ip, now, 65_536, 16); + let offset = auth_probe_scan_start_offset_in(shared.as_ref(), ip, now, 65_536, 16); uniq.insert(offset); } @@ -31,7 +25,7 @@ fn positive_same_ip_moving_time_yields_diverse_scan_offsets() { #[test] fn adversarial_many_ips_same_time_spreads_offsets_without_bias_collapse() { - let _guard = auth_probe_test_guard(); + let shared = ProxySharedState::new(); let now = Instant::now(); let mut uniq = HashSet::new(); @@ -42,7 +36,13 @@ fn adversarial_many_ips_same_time_spreads_offsets_without_bias_collapse() { i as u8, (255 - (i as u8)), )); - uniq.insert(auth_probe_scan_start_offset(ip, now, 65_536, 16)); + uniq.insert(auth_probe_scan_start_offset_in( + shared.as_ref(), + ip, + now, + 65_536, + 16, + )); } assert!( @@ -54,12 +54,13 @@ fn adversarial_many_ips_same_time_spreads_offsets_without_bias_collapse() { #[tokio::test(flavor = "multi_thread", worker_threads = 4)] async fn stress_parallel_failure_churn_under_saturation_remains_capped_and_live() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let start = Instant::now(); let mut workers = Vec::new(); for worker in 0..8u8 { + let shared = shared.clone(); workers.push(tokio::spawn(async move { for i in 0..8192u32 { let ip = IpAddr::V4(Ipv4Addr::new( @@ -68,7 +69,11 @@ async fn stress_parallel_failure_churn_under_saturation_remains_capped_and_live( ((i >> 8) & 0xff) as u8, (i & 0xff) as u8, )); - auth_probe_record_failure(ip, start + Duration::from_micros((i % 128) as u64)); + auth_probe_record_failure_in( + shared.as_ref(), + ip, + start + Duration::from_micros((i % 128) as u64), + ); } })); } @@ -78,17 +83,22 @@ async fn stress_parallel_failure_churn_under_saturation_remains_capped_and_live( } assert!( - auth_probe_state_map().len() <= AUTH_PROBE_TRACK_MAX_ENTRIES, + auth_probe_state_for_testing_in_shared(shared.as_ref()).len() + <= AUTH_PROBE_TRACK_MAX_ENTRIES, "state must remain hard-capped under parallel saturation churn" ); let probe = IpAddr::V4(Ipv4Addr::new(10, 4, 1, 1)); - let _ = auth_probe_should_apply_preauth_throttle(probe, start + Duration::from_millis(1)); + let _ = auth_probe_should_apply_preauth_throttle_in( + shared.as_ref(), + probe, + start + Duration::from_millis(1), + ); } #[test] fn light_fuzz_scan_offset_stays_within_window_for_randomized_inputs() { - let _guard = auth_probe_test_guard(); + let shared = ProxySharedState::new(); let mut seed = 0xA55A_1357_2468_9BDFu64; let base = Instant::now(); @@ -107,7 +117,8 @@ fn light_fuzz_scan_offset_stays_within_window_for_randomized_inputs() { let scan_limit = ((seed >> 40) as usize % 1024).saturating_add(1); let now = base + Duration::from_nanos(seed & 0x1fff); - let offset = auth_probe_scan_start_offset(ip, now, state_len, scan_limit); + let offset = + auth_probe_scan_start_offset_in(shared.as_ref(), ip, now, state_len, scan_limit); assert!( offset < state_len, "scan offset must always remain inside state length" diff --git a/src/proxy/tests/handshake_baseline_invariant_tests.rs b/src/proxy/tests/handshake_baseline_invariant_tests.rs new file mode 100644 index 0000000..5f938d8 --- /dev/null +++ b/src/proxy/tests/handshake_baseline_invariant_tests.rs @@ -0,0 +1,237 @@ +use super::*; +use crate::crypto::sha256_hmac; +use crate::stats::ReplayChecker; +use std::net::{IpAddr, Ipv4Addr, SocketAddr}; +use std::time::{Duration, Instant}; +use tokio::time::timeout; + +fn test_config_with_secret_hex(secret_hex: &str) -> ProxyConfig { + let mut cfg = ProxyConfig::default(); + cfg.access.users.clear(); + cfg.access + .users + .insert("user".to_string(), secret_hex.to_string()); + cfg.access.ignore_time_skew = true; + cfg.censorship.mask = true; + cfg +} + +fn make_valid_tls_handshake(secret: &[u8], timestamp: u32) -> Vec { + let session_id_len: usize = 32; + let len = tls::TLS_DIGEST_POS + tls::TLS_DIGEST_LEN + 1 + session_id_len; + let mut handshake = vec![0x42u8; len]; + + handshake[tls::TLS_DIGEST_POS + tls::TLS_DIGEST_LEN] = session_id_len as u8; + handshake[tls::TLS_DIGEST_POS..tls::TLS_DIGEST_POS + tls::TLS_DIGEST_LEN].fill(0); + + let computed = sha256_hmac(secret, &handshake); + let mut digest = computed; + let ts = timestamp.to_le_bytes(); + for i in 0..4 { + digest[28 + i] ^= ts[i]; + } + + handshake[tls::TLS_DIGEST_POS..tls::TLS_DIGEST_POS + tls::TLS_DIGEST_LEN] + .copy_from_slice(&digest); + handshake +} + +#[tokio::test] +async fn handshake_baseline_probe_always_falls_back_to_masking() { + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); + + let cfg = test_config_with_secret_hex("11111111111111111111111111111111"); + let replay_checker = ReplayChecker::new(64, Duration::from_secs(60)); + let rng = SecureRandom::new(); + let peer: SocketAddr = "198.51.100.210:44321".parse().unwrap(); + + let probe = b"not-a-tls-clienthello"; + let res = handle_tls_handshake( + probe, + tokio::io::empty(), + tokio::io::sink(), + peer, + &cfg, + &replay_checker, + &rng, + None, + ) + .await; + + assert!(matches!(res, HandshakeResult::BadClient { .. })); +} + +#[tokio::test] +async fn handshake_baseline_invalid_secret_triggers_fallback_not_error_response() { + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); + + let good_secret = [0x22u8; 16]; + let bad_cfg = test_config_with_secret_hex("33333333333333333333333333333333"); + let replay_checker = ReplayChecker::new(64, Duration::from_secs(60)); + let rng = SecureRandom::new(); + let peer: SocketAddr = "198.51.100.211:44322".parse().unwrap(); + + let handshake = make_valid_tls_handshake(&good_secret, 0); + let res = handle_tls_handshake( + &handshake, + tokio::io::empty(), + tokio::io::sink(), + peer, + &bad_cfg, + &replay_checker, + &rng, + None, + ) + .await; + + assert!(matches!(res, HandshakeResult::BadClient { .. })); +} + +#[tokio::test] +async fn handshake_baseline_auth_probe_streak_increments_per_ip() { + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); + + let cfg = test_config_with_secret_hex("44444444444444444444444444444444"); + let replay_checker = ReplayChecker::new(64, Duration::from_secs(60)); + let rng = SecureRandom::new(); + + let peer: SocketAddr = "203.0.113.10:5555".parse().unwrap(); + let untouched_ip = IpAddr::V4(Ipv4Addr::new(203, 0, 113, 11)); + let bad_probe = b"\x16\x03\x01\x00"; + + for expected in 1..=3 { + let res = handle_tls_handshake_with_shared( + bad_probe, + tokio::io::empty(), + tokio::io::sink(), + peer, + &cfg, + &replay_checker, + &rng, + None, + shared.as_ref(), + ) + .await; + assert!(matches!(res, HandshakeResult::BadClient { .. })); + assert_eq!( + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), peer.ip()), + Some(expected) + ); + assert_eq!( + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), untouched_ip), + None + ); + } +} + +#[test] +fn handshake_baseline_saturation_fires_at_compile_time_threshold() { + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); + + let ip = IpAddr::V4(Ipv4Addr::new(198, 51, 100, 33)); + let now = Instant::now(); + + for _ in 0..AUTH_PROBE_BACKOFF_START_FAILS.saturating_sub(1) { + auth_probe_record_failure_in(shared.as_ref(), ip, now); + } + assert!(!auth_probe_is_throttled_in(shared.as_ref(), ip, now)); + + auth_probe_record_failure_in(shared.as_ref(), ip, now); + assert!(auth_probe_is_throttled_in(shared.as_ref(), ip, now)); +} + +#[test] +fn handshake_baseline_repeated_probes_streak_monotonic() { + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); + + let ip = IpAddr::V4(Ipv4Addr::new(203, 0, 113, 42)); + let now = Instant::now(); + let mut prev = 0u32; + + for _ in 0..100 { + auth_probe_record_failure_in(shared.as_ref(), ip, now); + let current = + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), ip).unwrap_or(0); + assert!(current >= prev, "streak must be monotonic"); + prev = current; + } +} + +#[test] +fn handshake_baseline_throttled_ip_incurs_backoff_delay() { + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); + + let ip = IpAddr::V4(Ipv4Addr::new(198, 51, 100, 44)); + let now = Instant::now(); + + for _ in 0..AUTH_PROBE_BACKOFF_START_FAILS { + auth_probe_record_failure_in(shared.as_ref(), ip, now); + } + + let delay = auth_probe_backoff(AUTH_PROBE_BACKOFF_START_FAILS); + assert!(delay >= Duration::from_millis(AUTH_PROBE_BACKOFF_BASE_MS)); + + let before_expiry = now + delay.saturating_sub(Duration::from_millis(1)); + let after_expiry = now + delay + Duration::from_millis(1); + + assert!(auth_probe_is_throttled_in( + shared.as_ref(), + ip, + before_expiry + )); + assert!(!auth_probe_is_throttled_in( + shared.as_ref(), + ip, + after_expiry + )); +} + +#[tokio::test] +async fn handshake_baseline_malformed_probe_frames_fail_closed_to_masking() { + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); + + let cfg = test_config_with_secret_hex("55555555555555555555555555555555"); + let replay_checker = ReplayChecker::new(64, Duration::from_secs(60)); + let rng = SecureRandom::new(); + let peer: SocketAddr = "198.51.100.212:44323".parse().unwrap(); + + let corpus: Vec> = vec![ + vec![0x16, 0x03, 0x01], + vec![0x16, 0x03, 0x01, 0xFF, 0xFF], + vec![0x00; 128], + (0..64u8).collect(), + ]; + + for probe in corpus { + let res = timeout( + Duration::from_millis(250), + handle_tls_handshake( + &probe, + tokio::io::empty(), + tokio::io::sink(), + peer, + &cfg, + &replay_checker, + &rng, + None, + ), + ) + .await + .expect("malformed probe handling must complete in bounded time"); + + assert!( + matches!( + res, + HandshakeResult::BadClient { .. } | HandshakeResult::Error(_) + ), + "malformed probe must fail closed" + ); + } +} diff --git a/src/proxy/tests/handshake_fuzz_security_tests.rs b/src/proxy/tests/handshake_fuzz_security_tests.rs index efb596b..c56b184 100644 --- a/src/proxy/tests/handshake_fuzz_security_tests.rs +++ b/src/proxy/tests/handshake_fuzz_security_tests.rs @@ -67,16 +67,10 @@ fn test_config_with_secret_hex(secret_hex: &str) -> ProxyConfig { cfg } -fn auth_probe_test_guard() -> MutexGuard<'static, ()> { - auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()) -} - #[tokio::test] async fn mtproto_handshake_duplicate_digest_is_replayed_on_second_attempt() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret_hex = "11223344556677889900aabbccddeeff"; let base = make_valid_mtproto_handshake(secret_hex, ProtoTag::Secure, 2); @@ -110,13 +104,13 @@ async fn mtproto_handshake_duplicate_digest_is_replayed_on_second_attempt() { .await; assert!(matches!(second, HandshakeResult::BadClient { .. })); - clear_auth_probe_state_for_testing(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); } #[tokio::test] async fn mtproto_handshake_fuzz_corpus_never_panics_and_stays_fail_closed() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret_hex = "00112233445566778899aabbccddeeff"; let base = make_valid_mtproto_handshake(secret_hex, ProtoTag::Secure, 1); @@ -178,13 +172,13 @@ async fn mtproto_handshake_fuzz_corpus_never_panics_and_stays_fail_closed() { ); } - clear_auth_probe_state_for_testing(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); } #[tokio::test] async fn mtproto_handshake_mixed_corpus_never_panics_and_exact_duplicates_are_rejected() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret_hex = "99887766554433221100ffeeddccbbaa"; let base = make_valid_mtproto_handshake(secret_hex, ProtoTag::Secure, 4); @@ -274,5 +268,5 @@ async fn mtproto_handshake_mixed_corpus_never_panics_and_exact_duplicates_are_re ); } - clear_auth_probe_state_for_testing(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); } diff --git a/src/proxy/tests/handshake_more_clever_tests.rs b/src/proxy/tests/handshake_more_clever_tests.rs index 9782469..f570424 100644 --- a/src/proxy/tests/handshake_more_clever_tests.rs +++ b/src/proxy/tests/handshake_more_clever_tests.rs @@ -11,12 +11,6 @@ use tokio::sync::Barrier; // --- Helpers --- -fn auth_probe_test_guard() -> std::sync::MutexGuard<'static, ()> { - auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()) -} - fn test_config_with_secret_hex(secret_hex: &str) -> ProxyConfig { let mut cfg = ProxyConfig::default(); cfg.access.users.clear(); @@ -164,8 +158,8 @@ fn make_valid_tls_client_hello_with_sni_and_alpn( #[tokio::test] async fn server_hello_delay_bypassed_if_max_is_zero_despite_high_min() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret = [0x1Au8; 16]; let mut config = test_config_with_secret_hex("1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a"); @@ -201,10 +195,10 @@ async fn server_hello_delay_bypassed_if_max_is_zero_despite_high_min() { #[test] fn auth_probe_backoff_extreme_fail_streak_clamps_safely() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); - let state = auth_probe_state_map(); + let state = auth_probe_state_for_testing_in_shared(shared.as_ref()); let peer_ip = IpAddr::V4(Ipv4Addr::new(198, 51, 100, 99)); let now = Instant::now(); @@ -217,7 +211,7 @@ fn auth_probe_backoff_extreme_fail_streak_clamps_safely() { }, ); - auth_probe_record_failure_with_state(&state, peer_ip, now); + auth_probe_record_failure_with_state_in(shared.as_ref(), &state, peer_ip, now); let updated = state.get(&peer_ip).unwrap(); assert_eq!(updated.fail_streak, u32::MAX); @@ -270,8 +264,8 @@ fn generate_tg_nonce_cryptographic_uniqueness_and_entropy() { #[tokio::test] async fn mtproto_multi_user_decryption_isolation() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let mut config = ProxyConfig::default(); config.general.modes.secure = true; @@ -323,10 +317,8 @@ async fn mtproto_multi_user_decryption_isolation() { #[tokio::test(flavor = "multi_thread", worker_threads = 4)] async fn invalid_secret_warning_lock_contention_and_bound() { - let _guard = warned_secrets_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_warned_secrets_for_testing(); + let shared = ProxySharedState::new(); + clear_warned_secrets_for_testing_in_shared(shared.as_ref()); let tasks = 50; let iterations_per_task = 100; @@ -335,11 +327,18 @@ async fn invalid_secret_warning_lock_contention_and_bound() { for t in 0..tasks { let b = barrier.clone(); + let shared = shared.clone(); handles.push(tokio::spawn(async move { b.wait().await; for i in 0..iterations_per_task { let user_name = format!("contention_user_{}_{}", t, i); - warn_invalid_secret_once(&user_name, "invalid_hex", ACCESS_SECRET_BYTES, None); + warn_invalid_secret_once_in( + shared.as_ref(), + &user_name, + "invalid_hex", + ACCESS_SECRET_BYTES, + None, + ); } })); } @@ -348,7 +347,7 @@ async fn invalid_secret_warning_lock_contention_and_bound() { handle.await.unwrap(); } - let warned = INVALID_SECRET_WARNED.get().unwrap(); + let warned = warned_secrets_for_testing_in_shared(shared.as_ref()); let guard = warned .lock() .unwrap_or_else(|poisoned| poisoned.into_inner()); @@ -362,8 +361,8 @@ async fn invalid_secret_warning_lock_contention_and_bound() { #[tokio::test(flavor = "multi_thread", worker_threads = 4)] async fn mtproto_strict_concurrent_replay_race_condition() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret_hex = "4A4A4A4A4A4A4A4A4A4A4A4A4A4A4A4A"; let config = Arc::new(test_config_with_secret_hex(secret_hex)); @@ -428,8 +427,8 @@ async fn mtproto_strict_concurrent_replay_race_condition() { #[tokio::test] async fn tls_alpn_zero_length_protocol_handled_safely() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret = [0x5Bu8; 16]; let mut config = test_config_with_secret_hex("5b5b5b5b5b5b5b5b5b5b5b5b5b5b5b5b"); @@ -461,8 +460,8 @@ async fn tls_alpn_zero_length_protocol_handled_safely() { #[tokio::test] async fn tls_sni_massive_hostname_does_not_panic() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret = [0x6Cu8; 16]; let config = test_config_with_secret_hex("6c6c6c6c6c6c6c6c6c6c6c6c6c6c6c6c"); @@ -497,8 +496,8 @@ async fn tls_sni_massive_hostname_does_not_panic() { #[tokio::test] async fn tls_progressive_truncation_fuzzing_no_panics() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret = [0x7Du8; 16]; let config = test_config_with_secret_hex("7d7d7d7d7d7d7d7d7d7d7d7d7d7d7d7d"); @@ -535,8 +534,8 @@ async fn tls_progressive_truncation_fuzzing_no_panics() { #[tokio::test] async fn mtproto_pure_entropy_fuzzing_no_panics() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let config = test_config_with_secret_hex("8e8e8e8e8e8e8e8e8e8e8e8e8e8e8e8e"); let replay_checker = ReplayChecker::new(128, Duration::from_secs(60)); @@ -569,10 +568,8 @@ async fn mtproto_pure_entropy_fuzzing_no_panics() { #[test] fn decode_user_secret_odd_length_hex_rejection() { - let _guard = warned_secrets_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_warned_secrets_for_testing(); + let shared = ProxySharedState::new(); + clear_warned_secrets_for_testing_in_shared(shared.as_ref()); let mut config = ProxyConfig::default(); config.access.users.clear(); @@ -581,7 +578,7 @@ fn decode_user_secret_odd_length_hex_rejection() { "1234567890123456789012345678901".to_string(), ); - let decoded = decode_user_secrets(&config, None); + let decoded = decode_user_secrets_in(shared.as_ref(), &config, None); assert!( decoded.is_empty(), "Odd-length hex string must be gracefully rejected by hex::decode without unwrapping" @@ -590,10 +587,10 @@ fn decode_user_secret_odd_length_hex_rejection() { #[test] fn saturation_grace_pre_existing_high_fail_streak_immediate_throttle() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); - let state = auth_probe_state_map(); + let state = auth_probe_state_for_testing_in_shared(shared.as_ref()); let peer_ip = IpAddr::V4(Ipv4Addr::new(198, 51, 100, 112)); let now = Instant::now(); @@ -608,7 +605,7 @@ fn saturation_grace_pre_existing_high_fail_streak_immediate_throttle() { ); { - let mut guard = auth_probe_saturation_state_lock(); + let mut guard = auth_probe_saturation_state_lock_for_testing_in_shared(shared.as_ref()); *guard = Some(AuthProbeSaturationState { fail_streak: AUTH_PROBE_BACKOFF_START_FAILS, blocked_until: now + Duration::from_secs(5), @@ -616,7 +613,7 @@ fn saturation_grace_pre_existing_high_fail_streak_immediate_throttle() { }); } - let is_throttled = auth_probe_should_apply_preauth_throttle(peer_ip, now); + let is_throttled = auth_probe_should_apply_preauth_throttle_in(shared.as_ref(), peer_ip, now); assert!( is_throttled, "A peer with a pre-existing high fail streak must be immediately throttled when saturation begins, receiving no unearned grace period" @@ -625,21 +622,22 @@ fn saturation_grace_pre_existing_high_fail_streak_immediate_throttle() { #[test] fn auth_probe_saturation_note_resets_retention_window() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let base_time = Instant::now(); - auth_probe_note_saturation(base_time); + auth_probe_note_saturation_in(shared.as_ref(), base_time); let later = base_time + Duration::from_secs(AUTH_PROBE_TRACK_RETENTION_SECS - 1); - auth_probe_note_saturation(later); + auth_probe_note_saturation_in(shared.as_ref(), later); let check_time = base_time + Duration::from_secs(AUTH_PROBE_TRACK_RETENTION_SECS + 5); // This call may return false if backoff has elapsed, but it must not clear // the saturation state because `later` refreshed last_seen. - let _ = auth_probe_saturation_is_throttled_at_for_testing(check_time); - let guard = auth_probe_saturation_state_lock(); + let _ = + auth_probe_saturation_is_throttled_at_for_testing_in_shared(shared.as_ref(), check_time); + let guard = auth_probe_saturation_state_lock_for_testing_in_shared(shared.as_ref()); assert!( guard.is_some(), "Ongoing saturation notes must refresh last_seen so saturation state remains retained past the original window" diff --git a/src/proxy/tests/handshake_real_bug_stress_tests.rs b/src/proxy/tests/handshake_real_bug_stress_tests.rs index 1e27ed5..9705853 100644 --- a/src/proxy/tests/handshake_real_bug_stress_tests.rs +++ b/src/proxy/tests/handshake_real_bug_stress_tests.rs @@ -6,12 +6,6 @@ use std::sync::Arc; use std::time::{Duration, Instant}; use tokio::sync::Barrier; -fn auth_probe_test_guard() -> std::sync::MutexGuard<'static, ()> { - auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()) -} - fn test_config_with_secret_hex(secret_hex: &str) -> ProxyConfig { let mut cfg = ProxyConfig::default(); cfg.access.users.clear(); @@ -127,8 +121,8 @@ fn make_valid_mtproto_handshake( #[tokio::test] async fn tls_alpn_reject_does_not_pollute_replay_cache() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret = [0x11u8; 16]; let mut config = test_config_with_secret_hex("11111111111111111111111111111111"); @@ -164,8 +158,8 @@ async fn tls_alpn_reject_does_not_pollute_replay_cache() { #[tokio::test] async fn tls_truncated_session_id_len_fails_closed_without_panic() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let config = test_config_with_secret_hex("33333333333333333333333333333333"); let replay_checker = ReplayChecker::new(128, Duration::from_secs(60)); @@ -193,10 +187,10 @@ async fn tls_truncated_session_id_len_fails_closed_without_panic() { #[test] fn auth_probe_eviction_identical_timestamps_keeps_map_bounded() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); - let state = auth_probe_state_map(); + let state = auth_probe_state_for_testing_in_shared(shared.as_ref()); let same = Instant::now(); for i in 0..AUTH_PROBE_TRACK_MAX_ENTRIES { @@ -212,7 +206,12 @@ fn auth_probe_eviction_identical_timestamps_keeps_map_bounded() { } let new_ip = IpAddr::V4(Ipv4Addr::new(192, 168, 21, 21)); - auth_probe_record_failure_with_state(state, new_ip, same + Duration::from_millis(1)); + auth_probe_record_failure_with_state_in( + shared.as_ref(), + state, + new_ip, + same + Duration::from_millis(1), + ); assert_eq!(state.len(), AUTH_PROBE_TRACK_MAX_ENTRIES); assert!(state.contains_key(&new_ip)); @@ -220,21 +219,21 @@ fn auth_probe_eviction_identical_timestamps_keeps_map_bounded() { #[test] fn clear_auth_probe_state_recovers_from_poisoned_saturation_lock() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); - let saturation = auth_probe_saturation_state(); + let shared_for_poison = shared.clone(); let poison_thread = std::thread::spawn(move || { - let _hold = saturation + let _hold = auth_probe_saturation_state_for_testing_in_shared(shared_for_poison.as_ref()) .lock() .unwrap_or_else(|poisoned| poisoned.into_inner()); panic!("intentional poison for regression coverage"); }); let _ = poison_thread.join(); - clear_auth_probe_state_for_testing(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); - let guard = auth_probe_saturation_state() + let guard = auth_probe_saturation_state_for_testing_in_shared(shared.as_ref()) .lock() .unwrap_or_else(|poisoned| poisoned.into_inner()); assert!(guard.is_none()); @@ -242,12 +241,9 @@ fn clear_auth_probe_state_recovers_from_poisoned_saturation_lock() { #[tokio::test] async fn mtproto_invalid_length_secret_is_ignored_and_valid_user_still_auths() { - let _probe_guard = auth_probe_test_guard(); - let _warn_guard = warned_secrets_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); - clear_warned_secrets_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); + clear_warned_secrets_for_testing_in_shared(shared.as_ref()); let mut config = ProxyConfig::default(); config.general.modes.secure = true; @@ -285,14 +281,14 @@ async fn mtproto_invalid_length_secret_is_ignored_and_valid_user_still_auths() { #[tokio::test(flavor = "multi_thread", worker_threads = 4)] async fn saturation_grace_exhaustion_under_concurrency_keeps_peer_throttled() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let peer_ip = IpAddr::V4(Ipv4Addr::new(198, 51, 100, 80)); let now = Instant::now(); { - let mut guard = auth_probe_saturation_state() + let mut guard = auth_probe_saturation_state_for_testing_in_shared(shared.as_ref()) .lock() .unwrap_or_else(|poisoned| poisoned.into_inner()); *guard = Some(AuthProbeSaturationState { @@ -302,7 +298,7 @@ async fn saturation_grace_exhaustion_under_concurrency_keeps_peer_throttled() { }); } - let state = auth_probe_state_map(); + let state = auth_probe_state_for_testing_in_shared(shared.as_ref()); state.insert( peer_ip, AuthProbeState { @@ -318,9 +314,10 @@ async fn saturation_grace_exhaustion_under_concurrency_keeps_peer_throttled() { for _ in 0..tasks { let b = barrier.clone(); + let shared = shared.clone(); handles.push(tokio::spawn(async move { b.wait().await; - auth_probe_record_failure(peer_ip, Instant::now()); + auth_probe_record_failure_in(shared.as_ref(), peer_ip, Instant::now()); })); } @@ -333,7 +330,8 @@ async fn saturation_grace_exhaustion_under_concurrency_keeps_peer_throttled() { final_state.fail_streak >= AUTH_PROBE_BACKOFF_START_FAILS + AUTH_PROBE_SATURATION_GRACE_FAILS ); - assert!(auth_probe_should_apply_preauth_throttle( + assert!(auth_probe_should_apply_preauth_throttle_in( + shared.as_ref(), peer_ip, Instant::now() )); diff --git a/src/proxy/tests/handshake_saturation_poison_security_tests.rs b/src/proxy/tests/handshake_saturation_poison_security_tests.rs index 4c2ca5d..ebec667 100644 --- a/src/proxy/tests/handshake_saturation_poison_security_tests.rs +++ b/src/proxy/tests/handshake_saturation_poison_security_tests.rs @@ -1,46 +1,39 @@ use super::*; use std::time::{Duration, Instant}; -fn auth_probe_test_guard() -> std::sync::MutexGuard<'static, ()> { - auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()) -} - -fn poison_saturation_mutex() { - let saturation = auth_probe_saturation_state(); - let poison_thread = std::thread::spawn(move || { +fn poison_saturation_mutex(shared: &ProxySharedState) { + let saturation = auth_probe_saturation_state_for_testing_in_shared(shared); + let _ = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| { let _guard = saturation .lock() .expect("saturation mutex must be lockable for poison setup"); panic!("intentional poison for saturation mutex resilience test"); - }); - let _ = poison_thread.join(); + })); } #[test] fn auth_probe_saturation_note_recovers_after_mutex_poison() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); - poison_saturation_mutex(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); + poison_saturation_mutex(shared.as_ref()); let now = Instant::now(); - auth_probe_note_saturation(now); + auth_probe_note_saturation_in(shared.as_ref(), now); assert!( - auth_probe_saturation_is_throttled_at_for_testing(now), + auth_probe_saturation_is_throttled_at_for_testing_in_shared(shared.as_ref(), now), "poisoned saturation mutex must not disable saturation throttling" ); } #[test] fn auth_probe_saturation_check_recovers_after_mutex_poison() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); - poison_saturation_mutex(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); + poison_saturation_mutex(shared.as_ref()); { - let mut guard = auth_probe_saturation_state_lock(); + let mut guard = auth_probe_saturation_state_lock_for_testing_in_shared(shared.as_ref()); *guard = Some(AuthProbeSaturationState { fail_streak: AUTH_PROBE_BACKOFF_START_FAILS, blocked_until: Instant::now() + Duration::from_millis(10), @@ -49,23 +42,25 @@ fn auth_probe_saturation_check_recovers_after_mutex_poison() { } assert!( - auth_probe_saturation_is_throttled_for_testing(), + auth_probe_saturation_is_throttled_for_testing_in_shared(shared.as_ref()), "throttle check must recover poisoned saturation mutex and stay fail-closed" ); } #[test] fn clear_auth_probe_state_clears_saturation_even_if_poisoned() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); - poison_saturation_mutex(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); + poison_saturation_mutex(shared.as_ref()); - auth_probe_note_saturation(Instant::now()); - assert!(auth_probe_saturation_is_throttled_for_testing()); + auth_probe_note_saturation_in(shared.as_ref(), Instant::now()); + assert!(auth_probe_saturation_is_throttled_for_testing_in_shared( + shared.as_ref() + )); - clear_auth_probe_state_for_testing(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); assert!( - !auth_probe_saturation_is_throttled_for_testing(), + !auth_probe_saturation_is_throttled_for_testing_in_shared(shared.as_ref()), "clear helper must clear saturation state even after poison" ); } diff --git a/src/proxy/tests/handshake_security_tests.rs b/src/proxy/tests/handshake_security_tests.rs index 0e43d35..0f8fe03 100644 --- a/src/proxy/tests/handshake_security_tests.rs +++ b/src/proxy/tests/handshake_security_tests.rs @@ -5,6 +5,7 @@ use rand::rngs::StdRng; use rand::{RngExt, SeedableRng}; use std::net::{IpAddr, Ipv4Addr}; use std::sync::Arc; +use std::sync::atomic::Ordering; use std::time::{Duration, Instant}; use tokio::sync::Barrier; @@ -697,10 +698,8 @@ async fn invalid_tls_probe_does_not_pollute_replay_cache() { #[tokio::test] async fn empty_decoded_secret_is_rejected() { - let _guard = warned_secrets_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_warned_secrets_for_testing(); + let shared = ProxySharedState::new(); + clear_warned_secrets_for_testing_in_shared(shared.as_ref()); let config = test_config_with_secret_hex(""); let replay_checker = ReplayChecker::new(128, Duration::from_secs(60)); let rng = SecureRandom::new(); @@ -724,10 +723,8 @@ async fn empty_decoded_secret_is_rejected() { #[tokio::test] async fn wrong_length_decoded_secret_is_rejected() { - let _guard = warned_secrets_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_warned_secrets_for_testing(); + let shared = ProxySharedState::new(); + clear_warned_secrets_for_testing_in_shared(shared.as_ref()); let config = test_config_with_secret_hex("aa"); let replay_checker = ReplayChecker::new(128, Duration::from_secs(60)); let rng = SecureRandom::new(); @@ -777,14 +774,10 @@ async fn invalid_mtproto_probe_does_not_pollute_replay_cache() { #[tokio::test] async fn mixed_secret_lengths_keep_valid_user_authenticating() { - let _probe_guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - let _guard = warned_secrets_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_warned_secrets_for_testing(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + let shared = ProxySharedState::new(); + clear_warned_secrets_for_testing_in_shared(shared.as_ref()); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let good_secret = [0x22u8; 16]; let mut config = ProxyConfig::default(); config.access.users.clear(); @@ -864,6 +857,7 @@ async fn tls_sni_preferred_user_hint_selects_matching_identity_first() { #[test] fn stress_decode_user_secrets_keeps_preferred_user_first_in_large_set() { + let shared = ProxySharedState::new(); let mut config = ProxyConfig::default(); config.access.users.clear(); @@ -881,7 +875,7 @@ fn stress_decode_user_secrets_keeps_preferred_user_first_in_large_set() { .users .insert(preferred_user.clone(), secret_hex.clone()); - let decoded = decode_user_secrets(&config, Some(preferred_user.as_str())); + let decoded = decode_user_secrets_in(shared.as_ref(), &config, Some(preferred_user.as_str())); assert_eq!( decoded.len(), config.access.users.len(), @@ -1013,6 +1007,64 @@ async fn tls_unknown_sni_mask_policy_falls_back_to_bad_client() { assert!(matches!(result, HandshakeResult::BadClient { .. })); } +#[tokio::test] +async fn tls_unknown_sni_accept_policy_continues_auth_path() { + let secret = [0x4Bu8; 16]; + let mut config = test_config_with_secret_hex("4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b"); + config.censorship.unknown_sni_action = UnknownSniAction::Accept; + + let replay_checker = ReplayChecker::new(128, Duration::from_secs(60)); + let rng = SecureRandom::new(); + let peer: SocketAddr = "198.51.100.210:44326".parse().unwrap(); + let handshake = + make_valid_tls_client_hello_with_sni_and_alpn(&secret, 0, "unknown.example", &[b"h2"]); + + let result = handle_tls_handshake( + &handshake, + tokio::io::empty(), + tokio::io::sink(), + peer, + &config, + &replay_checker, + &rng, + None, + ) + .await; + + assert!(matches!(result, HandshakeResult::Success(_))); +} + +#[tokio::test] +async fn tls_unknown_sni_accept_policy_still_requires_valid_secret() { + let mut config = test_config_with_secret_hex("4c4c4c4c4c4c4c4c4c4c4c4c4c4c4c4c"); + config.censorship.unknown_sni_action = UnknownSniAction::Accept; + + let replay_checker = ReplayChecker::new(128, Duration::from_secs(60)); + let rng = SecureRandom::new(); + let peer: SocketAddr = "198.51.100.211:44326".parse().unwrap(); + let attacker_secret = [0x4Du8; 16]; + let handshake = make_valid_tls_client_hello_with_sni_and_alpn( + &attacker_secret, + 0, + "unknown.example", + &[b"h2"], + ); + + let result = handle_tls_handshake( + &handshake, + tokio::io::empty(), + tokio::io::sink(), + peer, + &config, + &replay_checker, + &rng, + None, + ) + .await; + + assert!(matches!(result, HandshakeResult::BadClient { .. })); +} + #[tokio::test] async fn tls_missing_sni_keeps_legacy_auth_path() { let secret = [0x4Au8; 16]; @@ -1039,6 +1091,170 @@ async fn tls_missing_sni_keeps_legacy_auth_path() { assert!(matches!(result, HandshakeResult::Success(_))); } +#[tokio::test] +async fn tls_runtime_snapshot_updates_sticky_and_recent_hints() { + let secret = [0x5Au8; 16]; + let mut config = test_config_with_secret_hex("5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a"); + config.rebuild_runtime_user_auth().unwrap(); + + let replay_checker = ReplayChecker::new(128, Duration::from_secs(60)); + let rng = SecureRandom::new(); + let shared = ProxySharedState::new(); + let peer: SocketAddr = "198.51.100.212:44326".parse().unwrap(); + let handshake = make_valid_tls_client_hello_with_sni_and_alpn(&secret, 0, "user", &[b"h2"]); + + let result = handle_tls_handshake_with_shared( + &handshake, + tokio::io::empty(), + tokio::io::sink(), + peer, + &config, + &replay_checker, + &rng, + None, + shared.as_ref(), + ) + .await; + + assert!(matches!(result, HandshakeResult::Success(_))); + assert_eq!( + shared + .handshake + .sticky_user_by_ip + .get(&peer.ip()) + .map(|entry| *entry), + Some(0), + "successful runtime-snapshot auth must seed sticky ip cache" + ); + assert_eq!( + shared.handshake.sticky_user_by_ip_prefix.len(), + 1, + "successful runtime-snapshot auth must seed sticky prefix cache" + ); + assert!( + shared + .handshake + .auth_expensive_checks_total + .load(Ordering::Relaxed) + >= 1, + "runtime-snapshot path must account expensive candidate checks" + ); +} + +#[tokio::test] +async fn tls_overload_budget_limits_candidate_scan_depth() { + let mut config = ProxyConfig::default(); + config.access.users.clear(); + config.access.ignore_time_skew = true; + for idx in 0..32u8 { + config.access.users.insert( + format!("user-{idx}"), + format!("{:032x}", u128::from(idx) + 1), + ); + } + config.rebuild_runtime_user_auth().unwrap(); + + let replay_checker = ReplayChecker::new(128, Duration::from_secs(60)); + let rng = SecureRandom::new(); + let shared = ProxySharedState::new(); + let now = Instant::now(); + { + let mut saturation = shared.handshake.auth_probe_saturation.lock().unwrap(); + *saturation = Some(AuthProbeSaturationState { + fail_streak: AUTH_PROBE_BACKOFF_START_FAILS, + blocked_until: now + Duration::from_millis(200), + last_seen: now, + }); + } + + let peer: SocketAddr = "198.51.100.213:44326".parse().unwrap(); + let attacker_secret = [0xEFu8; 16]; + let handshake = make_valid_tls_handshake(&attacker_secret, 0); + + let result = handle_tls_handshake_with_shared( + &handshake, + tokio::io::empty(), + tokio::io::sink(), + peer, + &config, + &replay_checker, + &rng, + None, + shared.as_ref(), + ) + .await; + + assert!(matches!(result, HandshakeResult::BadClient { .. })); + assert_eq!( + shared + .handshake + .auth_budget_exhausted_total + .load(Ordering::Relaxed), + 1, + "overload mode must account budget exhaustion when scan is capped" + ); + assert_eq!( + shared + .handshake + .auth_expensive_checks_total + .load(Ordering::Relaxed), + OVERLOAD_CANDIDATE_BUDGET_UNHINTED as u64, + "overload scan depth must stay within capped candidate budget" + ); +} + +#[tokio::test] +async fn mtproto_runtime_snapshot_prefers_preferred_user_hint() { + let mut config = ProxyConfig::default(); + config.general.modes.secure = true; + config.access.users.clear(); + config.access.ignore_time_skew = true; + config.access.users.insert( + "alpha".to_string(), + "11111111111111111111111111111111".to_string(), + ); + config.access.users.insert( + "beta".to_string(), + "22222222222222222222222222222222".to_string(), + ); + config.rebuild_runtime_user_auth().unwrap(); + + let handshake = + make_valid_mtproto_handshake("22222222222222222222222222222222", ProtoTag::Secure, 2); + let replay_checker = ReplayChecker::new(128, Duration::from_secs(60)); + let peer: SocketAddr = "198.51.100.214:44326".parse().unwrap(); + let shared = ProxySharedState::new(); + + let result = handle_mtproto_handshake_with_shared( + &handshake, + tokio::io::empty(), + tokio::io::sink(), + peer, + &config, + &replay_checker, + false, + Some("beta"), + shared.as_ref(), + ) + .await; + + match result { + HandshakeResult::Success((_, _, success)) => { + assert_eq!(success.user, "beta"); + } + _ => panic!("mtproto runtime snapshot auth must succeed for preferred user"), + } + + assert_eq!( + shared + .handshake + .auth_expensive_checks_total + .load(Ordering::Relaxed), + 1, + "preferred user hint must produce single-candidate success in snapshot path" + ); +} + #[tokio::test] async fn alpn_enforce_rejects_unsupported_client_alpn() { let secret = [0x33u8; 16]; @@ -1264,6 +1480,7 @@ async fn timing_matrix_tls_classes_under_fixed_delay_budget() { const ITER: usize = 48; const BUCKET_MS: u128 = 10; + let shared = ProxySharedState::new(); let secret = [0x77u8; 16]; let mut config = test_config_with_secret_hex("77777777777777777777777777777777"); config.censorship.alpn_enforce = true; @@ -1289,7 +1506,7 @@ async fn timing_matrix_tls_classes_under_fixed_delay_budget() { for (class, probe) in classes { let mut samples_ms = Vec::with_capacity(ITER); for idx in 0..ITER { - clear_auth_probe_state_for_testing(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let replay_checker = ReplayChecker::new(4096, Duration::from_secs(60)); let peer: SocketAddr = SocketAddr::from((base_ip, 44_000 + idx as u16)); let started = Instant::now(); @@ -1411,17 +1628,13 @@ fn mode_policy_matrix_is_stable_for_all_tag_transport_mode_combinations() { #[test] fn invalid_secret_warning_keys_do_not_collide_on_colon_boundaries() { - let _guard = warned_secrets_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_warned_secrets_for_testing(); + let shared = ProxySharedState::new(); + clear_warned_secrets_for_testing_in_shared(shared.as_ref()); - warn_invalid_secret_once("a:b", "c", ACCESS_SECRET_BYTES, Some(1)); - warn_invalid_secret_once("a", "b:c", ACCESS_SECRET_BYTES, Some(2)); + warn_invalid_secret_once_in(shared.as_ref(), "a:b", "c", ACCESS_SECRET_BYTES, Some(1)); + warn_invalid_secret_once_in(shared.as_ref(), "a", "b:c", ACCESS_SECRET_BYTES, Some(2)); - let warned = INVALID_SECRET_WARNED - .get() - .expect("warned set must be initialized"); + let warned = warned_secrets_for_testing_in_shared(shared.as_ref()); let guard = warned.lock().expect("warned set lock must be available"); assert_eq!( guard.len(), @@ -1432,19 +1645,21 @@ fn invalid_secret_warning_keys_do_not_collide_on_colon_boundaries() { #[test] fn invalid_secret_warning_cache_is_bounded() { - let _guard = warned_secrets_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_warned_secrets_for_testing(); + let shared = ProxySharedState::new(); + clear_warned_secrets_for_testing_in_shared(shared.as_ref()); for idx in 0..(WARNED_SECRET_MAX_ENTRIES + 32) { let user = format!("warned_user_{idx}"); - warn_invalid_secret_once(&user, "invalid_length", ACCESS_SECRET_BYTES, Some(idx)); + warn_invalid_secret_once_in( + shared.as_ref(), + &user, + "invalid_length", + ACCESS_SECRET_BYTES, + Some(idx), + ); } - let warned = INVALID_SECRET_WARNED - .get() - .expect("warned set must be initialized"); + let warned = warned_secrets_for_testing_in_shared(shared.as_ref()); let guard = warned.lock().expect("warned set lock must be available"); assert_eq!( guard.len(), @@ -1455,10 +1670,8 @@ fn invalid_secret_warning_cache_is_bounded() { #[tokio::test] async fn repeated_invalid_tls_probes_trigger_pre_auth_throttle() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let config = test_config_with_secret_hex("11111111111111111111111111111111"); let replay_checker = ReplayChecker::new(128, Duration::from_secs(60)); @@ -1469,7 +1682,7 @@ async fn repeated_invalid_tls_probes_trigger_pre_auth_throttle() { invalid[tls::TLS_DIGEST_POS + tls::TLS_DIGEST_LEN] = 32; for _ in 0..AUTH_PROBE_BACKOFF_START_FAILS { - let result = handle_tls_handshake( + let result = handle_tls_handshake_with_shared( &invalid, tokio::io::empty(), tokio::io::sink(), @@ -1478,13 +1691,14 @@ async fn repeated_invalid_tls_probes_trigger_pre_auth_throttle() { &replay_checker, &rng, None, + shared.as_ref(), ) .await; assert!(matches!(result, HandshakeResult::BadClient { .. })); } assert!( - auth_probe_fail_streak_for_testing(peer.ip()) + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), peer.ip()) .is_some_and(|streak| streak >= AUTH_PROBE_BACKOFF_START_FAILS), "invalid probe burst must grow pre-auth failure streak to backoff threshold" ); @@ -1492,10 +1706,8 @@ async fn repeated_invalid_tls_probes_trigger_pre_auth_throttle() { #[tokio::test] async fn successful_tls_handshake_clears_pre_auth_failure_streak() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret = [0x23u8; 16]; let config = test_config_with_secret_hex("23232323232323232323232323232323"); @@ -1507,7 +1719,7 @@ async fn successful_tls_handshake_clears_pre_auth_failure_streak() { invalid[tls::TLS_DIGEST_POS + tls::TLS_DIGEST_LEN] = 32; for expected in 1..AUTH_PROBE_BACKOFF_START_FAILS { - let result = handle_tls_handshake( + let result = handle_tls_handshake_with_shared( &invalid, tokio::io::empty(), tokio::io::sink(), @@ -1516,18 +1728,19 @@ async fn successful_tls_handshake_clears_pre_auth_failure_streak() { &replay_checker, &rng, None, + shared.as_ref(), ) .await; assert!(matches!(result, HandshakeResult::BadClient { .. })); assert_eq!( - auth_probe_fail_streak_for_testing(peer.ip()), + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), peer.ip()), Some(expected), "failure streak must grow before a successful authentication" ); } let valid = make_valid_tls_handshake(&secret, 0); - let success = handle_tls_handshake( + let success = handle_tls_handshake_with_shared( &valid, tokio::io::empty(), tokio::io::sink(), @@ -1536,12 +1749,13 @@ async fn successful_tls_handshake_clears_pre_auth_failure_streak() { &replay_checker, &rng, None, + shared.as_ref(), ) .await; assert!(matches!(success, HandshakeResult::Success(_))); assert_eq!( - auth_probe_fail_streak_for_testing(peer.ip()), + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), peer.ip()), None, "successful authentication must clear accumulated pre-auth failures" ); @@ -1549,6 +1763,7 @@ async fn successful_tls_handshake_clears_pre_auth_failure_streak() { #[test] fn auth_probe_capacity_prunes_stale_entries_for_new_ips() { + let shared = ProxySharedState::new(); let state = DashMap::new(); let now = Instant::now(); let stale_seen = now - Duration::from_secs(AUTH_PROBE_TRACK_RETENTION_SECS + 1); @@ -1571,7 +1786,7 @@ fn auth_probe_capacity_prunes_stale_entries_for_new_ips() { } let newcomer = IpAddr::V4(Ipv4Addr::new(198, 51, 100, 200)); - auth_probe_record_failure_with_state(&state, newcomer, now); + auth_probe_record_failure_with_state_in(shared.as_ref(), &state, newcomer, now); assert_eq!( state.get(&newcomer).map(|entry| entry.fail_streak), @@ -1586,10 +1801,8 @@ fn auth_probe_capacity_prunes_stale_entries_for_new_ips() { #[test] fn auth_probe_capacity_fresh_full_map_still_tracks_newcomer_with_bounded_eviction() { - let _guard = auth_probe_test_lock() - .lock() - .expect("auth probe test lock must be available"); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let state = DashMap::new(); let now = Instant::now(); @@ -1622,7 +1835,7 @@ fn auth_probe_capacity_fresh_full_map_still_tracks_newcomer_with_bounded_evictio ); let newcomer = IpAddr::V4(Ipv4Addr::new(203, 0, 113, 55)); - auth_probe_record_failure_with_state(&state, newcomer, now); + auth_probe_record_failure_with_state_in(shared.as_ref(), &state, newcomer, now); assert!( state.get(&newcomer).is_some(), @@ -1638,7 +1851,7 @@ fn auth_probe_capacity_fresh_full_map_still_tracks_newcomer_with_bounded_evictio "auth probe map must stay at configured cap after bounded eviction" ); assert!( - auth_probe_saturation_is_throttled_at_for_testing(now), + auth_probe_saturation_is_throttled_at_for_testing_in_shared(shared.as_ref(), now), "capacity pressure should still activate coarse global pre-auth throttling" ); } @@ -1646,23 +1859,25 @@ fn auth_probe_capacity_fresh_full_map_still_tracks_newcomer_with_bounded_evictio #[test] fn unknown_sni_warn_cooldown_first_event_is_warn_and_repeated_events_are_info_until_window_expires() { - let _guard = unknown_sni_warn_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_unknown_sni_warn_state_for_testing(); + let shared = ProxySharedState::new(); + clear_unknown_sni_warn_state_for_testing_in_shared(shared.as_ref()); let now = Instant::now(); assert!( - should_emit_unknown_sni_warn_for_testing(now), + should_emit_unknown_sni_warn_for_testing_in_shared(shared.as_ref(), now), "first unknown SNI event must be eligible for WARN emission" ); assert!( - !should_emit_unknown_sni_warn_for_testing(now + Duration::from_secs(1)), + !should_emit_unknown_sni_warn_for_testing_in_shared( + shared.as_ref(), + now + Duration::from_secs(1) + ), "events inside cooldown window must be demoted from WARN to INFO" ); assert!( - should_emit_unknown_sni_warn_for_testing( + should_emit_unknown_sni_warn_for_testing_in_shared( + shared.as_ref(), now + Duration::from_secs(UNKNOWN_SNI_WARN_COOLDOWN_SECS) ), "once cooldown expires, next unknown SNI event must be WARN-eligible again" @@ -1671,10 +1886,8 @@ fn unknown_sni_warn_cooldown_first_event_is_warn_and_repeated_events_are_info_un #[test] fn stress_auth_probe_full_map_churn_keeps_bound_and_tracks_newcomers() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let state = DashMap::new(); let base_now = Instant::now(); @@ -1704,7 +1917,7 @@ fn stress_auth_probe_full_map_churn_keeps_bound_and_tracks_newcomers() { (step & 0xff) as u8, )); let now = base_now + Duration::from_millis(10_000 + step as u64); - auth_probe_record_failure_with_state(&state, newcomer, now); + auth_probe_record_failure_with_state_in(shared.as_ref(), &state, newcomer, now); assert!( state.get(&newcomer).is_some(), @@ -1720,10 +1933,8 @@ fn stress_auth_probe_full_map_churn_keeps_bound_and_tracks_newcomers() { #[test] fn auth_probe_over_cap_churn_still_tracks_newcomer_after_round_limit() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let state = DashMap::new(); let now = Instant::now(); @@ -1747,7 +1958,12 @@ fn auth_probe_over_cap_churn_still_tracks_newcomer_after_round_limit() { } let newcomer = IpAddr::V4(Ipv4Addr::new(203, 0, 114, 77)); - auth_probe_record_failure_with_state(&state, newcomer, now + Duration::from_secs(1)); + auth_probe_record_failure_with_state_in( + shared.as_ref(), + &state, + newcomer, + now + Duration::from_secs(1), + ); assert!( state.get(&newcomer).is_some(), @@ -1761,10 +1977,8 @@ fn auth_probe_over_cap_churn_still_tracks_newcomer_after_round_limit() { #[test] fn auth_probe_capacity_prefers_evicting_low_fail_streak_entries_first() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let state = DashMap::new(); let now = Instant::now(); @@ -1808,7 +2022,7 @@ fn auth_probe_capacity_prefers_evicting_low_fail_streak_entries_first() { ); let newcomer = IpAddr::V4(Ipv4Addr::new(203, 0, 113, 201)); - auth_probe_record_failure_with_state(&state, newcomer, now); + auth_probe_record_failure_with_state_in(shared.as_ref(), &state, newcomer, now); assert!(state.get(&newcomer).is_some(), "new source must be tracked"); assert!( @@ -1823,10 +2037,8 @@ fn auth_probe_capacity_prefers_evicting_low_fail_streak_entries_first() { #[test] fn auth_probe_capacity_tie_breaker_evicts_oldest_with_equal_fail_streak() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let state = DashMap::new(); let now = Instant::now(); @@ -1868,7 +2080,7 @@ fn auth_probe_capacity_tie_breaker_evicts_oldest_with_equal_fail_streak() { ); let newcomer = IpAddr::V4(Ipv4Addr::new(203, 0, 113, 202)); - auth_probe_record_failure_with_state(&state, newcomer, now); + auth_probe_record_failure_with_state_in(shared.as_ref(), &state, newcomer, now); assert!(state.get(&newcomer).is_some(), "new source must be tracked"); assert!( @@ -1883,10 +2095,8 @@ fn auth_probe_capacity_tie_breaker_evicts_oldest_with_equal_fail_streak() { #[test] fn stress_auth_probe_capacity_churn_preserves_high_fail_sentinels() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let state = DashMap::new(); let base_now = Instant::now(); @@ -1936,7 +2146,7 @@ fn stress_auth_probe_capacity_churn_preserves_high_fail_sentinels() { (step & 0xff) as u8, )); let now = base_now + Duration::from_millis(10_000 + step as u64); - auth_probe_record_failure_with_state(&state, newcomer, now); + auth_probe_record_failure_with_state_in(shared.as_ref(), &state, newcomer, now); assert_eq!( state.len(), @@ -1952,14 +2162,25 @@ fn stress_auth_probe_capacity_churn_preserves_high_fail_sentinels() { #[test] fn auth_probe_ipv6_is_bucketed_by_prefix_64() { + let shared = ProxySharedState::new(); let state = DashMap::new(); let now = Instant::now(); let ip_a = IpAddr::V6("2001:db8:abcd:1234:1:2:3:4".parse().unwrap()); let ip_b = IpAddr::V6("2001:db8:abcd:1234:ffff:eeee:dddd:cccc".parse().unwrap()); - auth_probe_record_failure_with_state(&state, normalize_auth_probe_ip(ip_a), now); - auth_probe_record_failure_with_state(&state, normalize_auth_probe_ip(ip_b), now); + auth_probe_record_failure_with_state_in( + shared.as_ref(), + &state, + normalize_auth_probe_ip(ip_a), + now, + ); + auth_probe_record_failure_with_state_in( + shared.as_ref(), + &state, + normalize_auth_probe_ip(ip_b), + now, + ); let normalized = normalize_auth_probe_ip(ip_a); assert_eq!( @@ -1976,14 +2197,25 @@ fn auth_probe_ipv6_is_bucketed_by_prefix_64() { #[test] fn auth_probe_ipv6_different_prefixes_use_distinct_buckets() { + let shared = ProxySharedState::new(); let state = DashMap::new(); let now = Instant::now(); let ip_a = IpAddr::V6("2001:db8:1111:2222:1:2:3:4".parse().unwrap()); let ip_b = IpAddr::V6("2001:db8:1111:3333:1:2:3:4".parse().unwrap()); - auth_probe_record_failure_with_state(&state, normalize_auth_probe_ip(ip_a), now); - auth_probe_record_failure_with_state(&state, normalize_auth_probe_ip(ip_b), now); + auth_probe_record_failure_with_state_in( + shared.as_ref(), + &state, + normalize_auth_probe_ip(ip_a), + now, + ); + auth_probe_record_failure_with_state_in( + shared.as_ref(), + &state, + normalize_auth_probe_ip(ip_b), + now, + ); assert_eq!( state.len(), @@ -2006,25 +2238,23 @@ fn auth_probe_ipv6_different_prefixes_use_distinct_buckets() { #[test] fn auth_probe_success_clears_whole_ipv6_prefix_bucket() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let now = Instant::now(); let ip_fail = IpAddr::V6("2001:db8:aaaa:bbbb:1:2:3:4".parse().unwrap()); let ip_success = IpAddr::V6("2001:db8:aaaa:bbbb:ffff:eeee:dddd:cccc".parse().unwrap()); - auth_probe_record_failure(ip_fail, now); + auth_probe_record_failure_in(shared.as_ref(), ip_fail, now); assert_eq!( - auth_probe_fail_streak_for_testing(ip_fail), + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), ip_fail), Some(1), "precondition: normalized prefix bucket must exist" ); - auth_probe_record_success(ip_success); + auth_probe_record_success_in(shared.as_ref(), ip_success); assert_eq!( - auth_probe_fail_streak_for_testing(ip_fail), + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), ip_fail), None, "success from the same /64 must clear the shared bucket" ); @@ -2032,13 +2262,14 @@ fn auth_probe_success_clears_whole_ipv6_prefix_bucket() { #[test] fn auth_probe_eviction_offset_varies_with_input() { + let shared = ProxySharedState::new(); let now = Instant::now(); let ip1 = IpAddr::V4(Ipv4Addr::new(198, 51, 100, 10)); let ip2 = IpAddr::V4(Ipv4Addr::new(198, 51, 100, 11)); - let a = auth_probe_eviction_offset(ip1, now); - let b = auth_probe_eviction_offset(ip1, now); - let c = auth_probe_eviction_offset(ip2, now); + let a = auth_probe_eviction_offset_in(shared.as_ref(), ip1, now); + let b = auth_probe_eviction_offset_in(shared.as_ref(), ip1, now); + let c = auth_probe_eviction_offset_in(shared.as_ref(), ip2, now); assert_eq!(a, b, "same input must yield deterministic offset"); assert_ne!(a, c, "different peer IPs should not collapse to one offset"); @@ -2046,12 +2277,13 @@ fn auth_probe_eviction_offset_varies_with_input() { #[test] fn auth_probe_eviction_offset_changes_with_time_component() { + let shared = ProxySharedState::new(); let ip = IpAddr::V4(Ipv4Addr::new(203, 0, 113, 77)); let now = Instant::now(); let later = now + Duration::from_millis(1); - let a = auth_probe_eviction_offset(ip, now); - let b = auth_probe_eviction_offset(ip, later); + let a = auth_probe_eviction_offset_in(shared.as_ref(), ip, now); + let b = auth_probe_eviction_offset_in(shared.as_ref(), ip, later); assert_ne!( a, b, @@ -2061,10 +2293,8 @@ fn auth_probe_eviction_offset_changes_with_time_component() { #[test] fn auth_probe_round_limited_overcap_eviction_marks_saturation_and_keeps_newcomer_trackable() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let state = DashMap::new(); let now = Instant::now(); @@ -2098,7 +2328,12 @@ fn auth_probe_round_limited_overcap_eviction_marks_saturation_and_keeps_newcomer } let newcomer = IpAddr::V4(Ipv4Addr::new(203, 0, 113, 40)); - auth_probe_record_failure_with_state(&state, newcomer, now + Duration::from_millis(1)); + auth_probe_record_failure_with_state_in( + shared.as_ref(), + &state, + newcomer, + now + Duration::from_millis(1), + ); assert!( state.get(&newcomer).is_some(), @@ -2109,17 +2344,18 @@ fn auth_probe_round_limited_overcap_eviction_marks_saturation_and_keeps_newcomer "high fail-streak sentinel must survive round-limited eviction" ); assert!( - auth_probe_saturation_is_throttled_at_for_testing(now + Duration::from_millis(1)), + auth_probe_saturation_is_throttled_at_for_testing_in_shared( + shared.as_ref(), + now + Duration::from_millis(1) + ), "round-limited over-cap path must activate saturation throttle marker" ); } #[tokio::test] async fn gap_t01_short_tls_probe_burst_is_throttled() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let config = test_config_with_secret_hex("11111111111111111111111111111111"); let replay_checker = ReplayChecker::new(128, Duration::from_secs(60)); @@ -2129,7 +2365,7 @@ async fn gap_t01_short_tls_probe_burst_is_throttled() { let too_short = vec![0x16, 0x03, 0x01]; for _ in 0..AUTH_PROBE_BACKOFF_START_FAILS { - let result = handle_tls_handshake( + let result = handle_tls_handshake_with_shared( &too_short, tokio::io::empty(), tokio::io::sink(), @@ -2138,13 +2374,14 @@ async fn gap_t01_short_tls_probe_burst_is_throttled() { &replay_checker, &rng, None, + shared.as_ref(), ) .await; assert!(matches!(result, HandshakeResult::BadClient { .. })); } assert!( - auth_probe_fail_streak_for_testing(peer.ip()) + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), peer.ip()) .is_some_and(|streak| streak >= AUTH_PROBE_BACKOFF_START_FAILS), "short TLS probe bursts must increase auth-probe fail streak" ); @@ -2152,10 +2389,8 @@ async fn gap_t01_short_tls_probe_burst_is_throttled() { #[test] fn stress_auth_probe_overcap_churn_does_not_starve_high_threat_sentinel_bucket() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let state = DashMap::new(); let base_now = Instant::now(); @@ -2194,7 +2429,8 @@ fn stress_auth_probe_overcap_churn_does_not_starve_high_threat_sentinel_bucket() ((step >> 8) & 0xff) as u8, (step & 0xff) as u8, )); - auth_probe_record_failure_with_state( + auth_probe_record_failure_with_state_in( + shared.as_ref(), &state, newcomer, base_now + Duration::from_millis(step as u64 + 1), @@ -2213,10 +2449,8 @@ fn stress_auth_probe_overcap_churn_does_not_starve_high_threat_sentinel_bucket() #[test] fn light_fuzz_auth_probe_overcap_eviction_prefers_less_threatening_entries() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let now = Instant::now(); let mut s: u64 = 0xBADC_0FFE_EE11_2233; @@ -2259,7 +2493,8 @@ fn light_fuzz_auth_probe_overcap_eviction_prefers_less_threatening_entries() { ((round >> 8) & 0xff) as u8, (round & 0xff) as u8, )); - auth_probe_record_failure_with_state( + auth_probe_record_failure_with_state_in( + shared.as_ref(), &state, newcomer, now + Duration::from_millis(round as u64 + 1), @@ -2277,6 +2512,7 @@ fn light_fuzz_auth_probe_overcap_eviction_prefers_less_threatening_entries() { } #[test] fn light_fuzz_auth_probe_eviction_offset_is_deterministic_per_input_pair() { + let shared = ProxySharedState::new(); let mut rng = StdRng::seed_from_u64(0xA11CE5EED); let base = Instant::now(); @@ -2290,8 +2526,8 @@ fn light_fuzz_auth_probe_eviction_offset_is_deterministic_per_input_pair() { let offset_ns = rng.random_range(0_u64..2_000_000); let when = base + Duration::from_nanos(offset_ns); - let first = auth_probe_eviction_offset(ip, when); - let second = auth_probe_eviction_offset(ip, when); + let first = auth_probe_eviction_offset_in(shared.as_ref(), ip, when); + let second = auth_probe_eviction_offset_in(shared.as_ref(), ip, when); assert_eq!( first, second, "eviction offset must be stable for identical (ip, now) pairs" @@ -2301,6 +2537,7 @@ fn light_fuzz_auth_probe_eviction_offset_is_deterministic_per_input_pair() { #[test] fn adversarial_eviction_offset_spread_avoids_single_bucket_collapse() { + let shared = ProxySharedState::new(); let modulus = AUTH_PROBE_TRACK_MAX_ENTRIES; let mut bucket_hits = vec![0usize; modulus]; let now = Instant::now(); @@ -2312,7 +2549,7 @@ fn adversarial_eviction_offset_spread_avoids_single_bucket_collapse() { (idx & 0xff) as u8, ((idx.wrapping_mul(37)) & 0xff) as u8, )); - let bucket = auth_probe_eviction_offset(ip, now) % modulus; + let bucket = auth_probe_eviction_offset_in(shared.as_ref(), ip, now) % modulus; bucket_hits[bucket] += 1; } @@ -2337,6 +2574,7 @@ fn adversarial_eviction_offset_spread_avoids_single_bucket_collapse() { #[test] fn stress_auth_probe_eviction_offset_high_volume_uniqueness_sanity() { + let shared = ProxySharedState::new(); let now = Instant::now(); let mut seen = std::collections::HashSet::new(); @@ -2347,7 +2585,7 @@ fn stress_auth_probe_eviction_offset_high_volume_uniqueness_sanity() { ((idx >> 8) & 0xff) as u8, (idx & 0xff) as u8, )); - seen.insert(auth_probe_eviction_offset(ip, now)); + seen.insert(auth_probe_eviction_offset_in(shared.as_ref(), ip, now)); } assert!( @@ -2358,10 +2596,8 @@ fn stress_auth_probe_eviction_offset_high_volume_uniqueness_sanity() { #[tokio::test(flavor = "multi_thread", worker_threads = 4)] async fn auth_probe_concurrent_failures_do_not_lose_fail_streak_updates() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let peer_ip: IpAddr = "198.51.100.90".parse().unwrap(); let tasks = 128usize; @@ -2370,9 +2606,10 @@ async fn auth_probe_concurrent_failures_do_not_lose_fail_streak_updates() { for _ in 0..tasks { let barrier = barrier.clone(); + let shared = shared.clone(); handles.push(tokio::spawn(async move { barrier.wait().await; - auth_probe_record_failure(peer_ip, Instant::now()); + auth_probe_record_failure_in(shared.as_ref(), peer_ip, Instant::now()); })); } @@ -2382,7 +2619,7 @@ async fn auth_probe_concurrent_failures_do_not_lose_fail_streak_updates() { .expect("concurrent failure recording task must not panic"); } - let streak = auth_probe_fail_streak_for_testing(peer_ip) + let streak = auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), peer_ip) .expect("tracked peer must exist after concurrent failure burst"); assert_eq!( streak as usize, tasks, @@ -2392,10 +2629,8 @@ async fn auth_probe_concurrent_failures_do_not_lose_fail_streak_updates() { #[tokio::test(flavor = "multi_thread", worker_threads = 4)] async fn invalid_probe_noise_from_other_ips_does_not_break_valid_tls_handshake() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret = [0x31u8; 16]; let config = Arc::new(test_config_with_secret_hex( @@ -2464,7 +2699,7 @@ async fn invalid_probe_noise_from_other_ips_does_not_break_valid_tls_handshake() "invalid probe noise from other IPs must not block a valid victim handshake" ); assert_eq!( - auth_probe_fail_streak_for_testing(victim_peer.ip()), + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), victim_peer.ip()), None, "successful victim handshake must not retain pre-auth failure streak" ); @@ -2472,13 +2707,11 @@ async fn invalid_probe_noise_from_other_ips_does_not_break_valid_tls_handshake() #[test] fn auth_probe_saturation_state_expires_after_retention_window() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let now = Instant::now(); - let saturation = auth_probe_saturation_state(); + let saturation = auth_probe_saturation_state_for_testing_in_shared(shared.as_ref()); { let mut guard = saturation .lock() @@ -2491,7 +2724,7 @@ fn auth_probe_saturation_state_expires_after_retention_window() { } assert!( - !auth_probe_saturation_is_throttled_for_testing(), + !auth_probe_saturation_is_throttled_for_testing_in_shared(shared.as_ref()), "expired saturation state must stop throttling and self-clear" ); @@ -2503,10 +2736,8 @@ fn auth_probe_saturation_state_expires_after_retention_window() { #[tokio::test] async fn global_saturation_marker_does_not_block_valid_tls_handshake() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret = [0x41u8; 16]; let config = test_config_with_secret_hex("41414141414141414141414141414141"); @@ -2515,7 +2746,7 @@ async fn global_saturation_marker_does_not_block_valid_tls_handshake() { let peer: SocketAddr = "198.51.100.101:45101".parse().unwrap(); let now = Instant::now(); - let saturation = auth_probe_saturation_state(); + let saturation = auth_probe_saturation_state_for_testing_in_shared(shared.as_ref()); { let mut guard = saturation .lock() @@ -2528,7 +2759,7 @@ async fn global_saturation_marker_does_not_block_valid_tls_handshake() { } let valid = make_valid_tls_handshake(&secret, 0); - let result = handle_tls_handshake( + let result = handle_tls_handshake_with_shared( &valid, tokio::io::empty(), tokio::io::sink(), @@ -2537,6 +2768,7 @@ async fn global_saturation_marker_does_not_block_valid_tls_handshake() { &replay_checker, &rng, None, + shared.as_ref(), ) .await; @@ -2545,7 +2777,7 @@ async fn global_saturation_marker_does_not_block_valid_tls_handshake() { "global saturation marker must not block valid authenticated TLS handshakes" ); assert_eq!( - auth_probe_fail_streak_for_testing(peer.ip()), + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), peer.ip()), None, "successful handshake under saturation marker must not retain per-ip probe failures" ); @@ -2553,10 +2785,8 @@ async fn global_saturation_marker_does_not_block_valid_tls_handshake() { #[tokio::test] async fn expired_global_saturation_allows_valid_tls_handshake() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret = [0x55u8; 16]; let config = test_config_with_secret_hex("55555555555555555555555555555555"); @@ -2565,7 +2795,7 @@ async fn expired_global_saturation_allows_valid_tls_handshake() { let peer: SocketAddr = "198.51.100.102:45102".parse().unwrap(); let now = Instant::now(); - let saturation = auth_probe_saturation_state(); + let saturation = auth_probe_saturation_state_for_testing_in_shared(shared.as_ref()); { let mut guard = saturation .lock() @@ -2578,7 +2808,7 @@ async fn expired_global_saturation_allows_valid_tls_handshake() { } let valid = make_valid_tls_handshake(&secret, 0); - let result = handle_tls_handshake( + let result = handle_tls_handshake_with_shared( &valid, tokio::io::empty(), tokio::io::sink(), @@ -2587,6 +2817,7 @@ async fn expired_global_saturation_allows_valid_tls_handshake() { &replay_checker, &rng, None, + shared.as_ref(), ) .await; @@ -2598,10 +2829,8 @@ async fn expired_global_saturation_allows_valid_tls_handshake() { #[tokio::test] async fn valid_tls_is_blocked_by_per_ip_preauth_throttle_without_saturation() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret = [0x61u8; 16]; let config = test_config_with_secret_hex("61616161616161616161616161616161"); @@ -2609,7 +2838,7 @@ async fn valid_tls_is_blocked_by_per_ip_preauth_throttle_without_saturation() { let rng = SecureRandom::new(); let peer: SocketAddr = "198.51.100.103:45103".parse().unwrap(); - auth_probe_state_map().insert( + auth_probe_state_for_testing_in_shared(shared.as_ref()).insert( normalize_auth_probe_ip(peer.ip()), AuthProbeState { fail_streak: AUTH_PROBE_BACKOFF_START_FAILS, @@ -2619,7 +2848,7 @@ async fn valid_tls_is_blocked_by_per_ip_preauth_throttle_without_saturation() { ); let valid = make_valid_tls_handshake(&secret, 0); - let result = handle_tls_handshake( + let result = handle_tls_handshake_with_shared( &valid, tokio::io::empty(), tokio::io::sink(), @@ -2628,6 +2857,7 @@ async fn valid_tls_is_blocked_by_per_ip_preauth_throttle_without_saturation() { &replay_checker, &rng, None, + shared.as_ref(), ) .await; @@ -2636,10 +2866,8 @@ async fn valid_tls_is_blocked_by_per_ip_preauth_throttle_without_saturation() { #[tokio::test] async fn saturation_allows_valid_tls_even_when_peer_ip_is_currently_throttled() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret = [0x62u8; 16]; let config = test_config_with_secret_hex("62626262626262626262626262626262"); @@ -2648,7 +2876,7 @@ async fn saturation_allows_valid_tls_even_when_peer_ip_is_currently_throttled() let peer: SocketAddr = "198.51.100.104:45104".parse().unwrap(); let now = Instant::now(); - auth_probe_state_map().insert( + auth_probe_state_for_testing_in_shared(shared.as_ref()).insert( normalize_auth_probe_ip(peer.ip()), AuthProbeState { fail_streak: AUTH_PROBE_BACKOFF_START_FAILS, @@ -2657,7 +2885,7 @@ async fn saturation_allows_valid_tls_even_when_peer_ip_is_currently_throttled() }, ); { - let mut guard = auth_probe_saturation_state() + let mut guard = auth_probe_saturation_state_for_testing_in_shared(shared.as_ref()) .lock() .unwrap_or_else(|poisoned| poisoned.into_inner()); *guard = Some(AuthProbeSaturationState { @@ -2668,7 +2896,7 @@ async fn saturation_allows_valid_tls_even_when_peer_ip_is_currently_throttled() } let valid = make_valid_tls_handshake(&secret, 0); - let result = handle_tls_handshake( + let result = handle_tls_handshake_with_shared( &valid, tokio::io::empty(), tokio::io::sink(), @@ -2677,12 +2905,13 @@ async fn saturation_allows_valid_tls_even_when_peer_ip_is_currently_throttled() &replay_checker, &rng, None, + shared.as_ref(), ) .await; assert!(matches!(result, HandshakeResult::Success(_))); assert_eq!( - auth_probe_fail_streak_for_testing(peer.ip()), + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), peer.ip()), None, "successful auth under saturation must clear the peer's throttled state" ); @@ -2690,10 +2919,8 @@ async fn saturation_allows_valid_tls_even_when_peer_ip_is_currently_throttled() #[tokio::test] async fn saturation_still_rejects_invalid_tls_probe_and_records_failure() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let config = test_config_with_secret_hex("63636363636363636363636363636363"); let replay_checker = ReplayChecker::new(128, Duration::from_secs(60)); @@ -2701,7 +2928,7 @@ async fn saturation_still_rejects_invalid_tls_probe_and_records_failure() { let peer: SocketAddr = "198.51.100.105:45105".parse().unwrap(); let now = Instant::now(); { - let mut guard = auth_probe_saturation_state() + let mut guard = auth_probe_saturation_state_for_testing_in_shared(shared.as_ref()) .lock() .unwrap_or_else(|poisoned| poisoned.into_inner()); *guard = Some(AuthProbeSaturationState { @@ -2714,7 +2941,7 @@ async fn saturation_still_rejects_invalid_tls_probe_and_records_failure() { let mut invalid = vec![0x42u8; tls::TLS_DIGEST_POS + tls::TLS_DIGEST_LEN + 1 + 32]; invalid[tls::TLS_DIGEST_POS + tls::TLS_DIGEST_LEN] = 32; - let result = handle_tls_handshake( + let result = handle_tls_handshake_with_shared( &invalid, tokio::io::empty(), tokio::io::sink(), @@ -2723,12 +2950,13 @@ async fn saturation_still_rejects_invalid_tls_probe_and_records_failure() { &replay_checker, &rng, None, + shared.as_ref(), ) .await; assert!(matches!(result, HandshakeResult::BadClient { .. })); assert_eq!( - auth_probe_fail_streak_for_testing(peer.ip()), + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), peer.ip()), Some(1), "invalid TLS during saturation must still increment per-ip failure tracking" ); @@ -2736,17 +2964,15 @@ async fn saturation_still_rejects_invalid_tls_probe_and_records_failure() { #[tokio::test] async fn saturation_grace_exhaustion_preauth_throttles_repeated_invalid_tls_probe() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let config = test_config_with_secret_hex("63636363636363636363636363636363"); let replay_checker = ReplayChecker::new(128, Duration::from_secs(60)); let rng = SecureRandom::new(); let peer: SocketAddr = "198.51.100.205:45205".parse().unwrap(); let now = Instant::now(); - auth_probe_state_map().insert( + auth_probe_state_for_testing_in_shared(shared.as_ref()).insert( normalize_auth_probe_ip(peer.ip()), AuthProbeState { fail_streak: AUTH_PROBE_BACKOFF_START_FAILS + AUTH_PROBE_SATURATION_GRACE_FAILS, @@ -2755,7 +2981,7 @@ async fn saturation_grace_exhaustion_preauth_throttles_repeated_invalid_tls_prob }, ); { - let mut guard = auth_probe_saturation_state() + let mut guard = auth_probe_saturation_state_for_testing_in_shared(shared.as_ref()) .lock() .unwrap_or_else(|poisoned| poisoned.into_inner()); *guard = Some(AuthProbeSaturationState { @@ -2782,7 +3008,7 @@ async fn saturation_grace_exhaustion_preauth_throttles_repeated_invalid_tls_prob assert!(matches!(result, HandshakeResult::BadClient { .. })); assert_eq!( - auth_probe_fail_streak_for_testing(peer.ip()), + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), peer.ip()), Some(AUTH_PROBE_BACKOFF_START_FAILS + AUTH_PROBE_SATURATION_GRACE_FAILS), "pre-auth throttle under exhausted saturation grace must reject without re-processing invalid TLS" ); @@ -2790,10 +3016,8 @@ async fn saturation_grace_exhaustion_preauth_throttles_repeated_invalid_tls_prob #[tokio::test] async fn saturation_allows_valid_mtproto_even_when_peer_ip_is_currently_throttled() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret_hex = "64646464646464646464646464646464"; let mut config = test_config_with_secret_hex(secret_hex); @@ -2802,7 +3026,7 @@ async fn saturation_allows_valid_mtproto_even_when_peer_ip_is_currently_throttle let peer: SocketAddr = "198.51.100.106:45106".parse().unwrap(); let now = Instant::now(); - auth_probe_state_map().insert( + auth_probe_state_for_testing_in_shared(shared.as_ref()).insert( normalize_auth_probe_ip(peer.ip()), AuthProbeState { fail_streak: AUTH_PROBE_BACKOFF_START_FAILS, @@ -2811,7 +3035,7 @@ async fn saturation_allows_valid_mtproto_even_when_peer_ip_is_currently_throttle }, ); { - let mut guard = auth_probe_saturation_state() + let mut guard = auth_probe_saturation_state_for_testing_in_shared(shared.as_ref()) .lock() .unwrap_or_else(|poisoned| poisoned.into_inner()); *guard = Some(AuthProbeSaturationState { @@ -2822,7 +3046,7 @@ async fn saturation_allows_valid_mtproto_even_when_peer_ip_is_currently_throttle } let valid = make_valid_mtproto_handshake(secret_hex, ProtoTag::Secure, 2); - let result = handle_mtproto_handshake( + let result = handle_mtproto_handshake_with_shared( &valid, tokio::io::empty(), tokio::io::sink(), @@ -2831,12 +3055,13 @@ async fn saturation_allows_valid_mtproto_even_when_peer_ip_is_currently_throttle &replay_checker, false, None, + shared.as_ref(), ) .await; assert!(matches!(result, HandshakeResult::Success(_))); assert_eq!( - auth_probe_fail_streak_for_testing(peer.ip()), + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), peer.ip()), None, "successful mtproto auth under saturation must clear the peer's throttled state" ); @@ -2844,17 +3069,15 @@ async fn saturation_allows_valid_mtproto_even_when_peer_ip_is_currently_throttle #[tokio::test] async fn saturation_still_rejects_invalid_mtproto_probe_and_records_failure() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let config = test_config_with_secret_hex("65656565656565656565656565656565"); let replay_checker = ReplayChecker::new(128, Duration::from_secs(60)); let peer: SocketAddr = "198.51.100.107:45107".parse().unwrap(); let now = Instant::now(); { - let mut guard = auth_probe_saturation_state() + let mut guard = auth_probe_saturation_state_for_testing_in_shared(shared.as_ref()) .lock() .unwrap_or_else(|poisoned| poisoned.into_inner()); *guard = Some(AuthProbeSaturationState { @@ -2866,7 +3089,7 @@ async fn saturation_still_rejects_invalid_mtproto_probe_and_records_failure() { let invalid = [0u8; HANDSHAKE_LEN]; - let result = handle_mtproto_handshake( + let result = handle_mtproto_handshake_with_shared( &invalid, tokio::io::empty(), tokio::io::sink(), @@ -2875,12 +3098,13 @@ async fn saturation_still_rejects_invalid_mtproto_probe_and_records_failure() { &replay_checker, false, None, + shared.as_ref(), ) .await; assert!(matches!(result, HandshakeResult::BadClient { .. })); assert_eq!( - auth_probe_fail_streak_for_testing(peer.ip()), + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), peer.ip()), Some(1), "invalid mtproto during saturation must still increment per-ip failure tracking" ); @@ -2888,16 +3112,14 @@ async fn saturation_still_rejects_invalid_mtproto_probe_and_records_failure() { #[tokio::test] async fn saturation_grace_exhaustion_preauth_throttles_repeated_invalid_mtproto_probe() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let config = test_config_with_secret_hex("65656565656565656565656565656565"); let replay_checker = ReplayChecker::new(128, Duration::from_secs(60)); let peer: SocketAddr = "198.51.100.206:45206".parse().unwrap(); let now = Instant::now(); - auth_probe_state_map().insert( + auth_probe_state_for_testing_in_shared(shared.as_ref()).insert( normalize_auth_probe_ip(peer.ip()), AuthProbeState { fail_streak: AUTH_PROBE_BACKOFF_START_FAILS + AUTH_PROBE_SATURATION_GRACE_FAILS, @@ -2906,7 +3128,7 @@ async fn saturation_grace_exhaustion_preauth_throttles_repeated_invalid_mtproto_ }, ); { - let mut guard = auth_probe_saturation_state() + let mut guard = auth_probe_saturation_state_for_testing_in_shared(shared.as_ref()) .lock() .unwrap_or_else(|poisoned| poisoned.into_inner()); *guard = Some(AuthProbeSaturationState { @@ -2931,7 +3153,7 @@ async fn saturation_grace_exhaustion_preauth_throttles_repeated_invalid_mtproto_ assert!(matches!(result, HandshakeResult::BadClient { .. })); assert_eq!( - auth_probe_fail_streak_for_testing(peer.ip()), + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), peer.ip()), Some(AUTH_PROBE_BACKOFF_START_FAILS + AUTH_PROBE_SATURATION_GRACE_FAILS), "pre-auth throttle under exhausted saturation grace must reject without re-processing invalid MTProto" ); @@ -2939,17 +3161,15 @@ async fn saturation_grace_exhaustion_preauth_throttles_repeated_invalid_mtproto_ #[tokio::test] async fn saturation_grace_progression_tls_reaches_cap_then_stops_incrementing() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let config = test_config_with_secret_hex("70707070707070707070707070707070"); let replay_checker = ReplayChecker::new(128, Duration::from_secs(60)); let rng = SecureRandom::new(); let peer: SocketAddr = "198.51.100.207:45207".parse().unwrap(); let now = Instant::now(); - auth_probe_state_map().insert( + auth_probe_state_for_testing_in_shared(shared.as_ref()).insert( normalize_auth_probe_ip(peer.ip()), AuthProbeState { fail_streak: AUTH_PROBE_BACKOFF_START_FAILS, @@ -2958,7 +3178,7 @@ async fn saturation_grace_progression_tls_reaches_cap_then_stops_incrementing() }, ); { - let mut guard = auth_probe_saturation_state() + let mut guard = auth_probe_saturation_state_for_testing_in_shared(shared.as_ref()) .lock() .unwrap_or_else(|poisoned| poisoned.into_inner()); *guard = Some(AuthProbeSaturationState { @@ -2975,7 +3195,7 @@ async fn saturation_grace_progression_tls_reaches_cap_then_stops_incrementing() AUTH_PROBE_BACKOFF_START_FAILS + 1, AUTH_PROBE_BACKOFF_START_FAILS + AUTH_PROBE_SATURATION_GRACE_FAILS, ] { - let result = handle_tls_handshake( + let result = handle_tls_handshake_with_shared( &invalid, tokio::io::empty(), tokio::io::sink(), @@ -2984,17 +3204,18 @@ async fn saturation_grace_progression_tls_reaches_cap_then_stops_incrementing() &replay_checker, &rng, None, + shared.as_ref(), ) .await; assert!(matches!(result, HandshakeResult::BadClient { .. })); assert_eq!( - auth_probe_fail_streak_for_testing(peer.ip()), + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), peer.ip()), Some(expected) ); } { - let mut entry = auth_probe_state_map() + let mut entry = auth_probe_state_for_testing_in_shared(shared.as_ref()) .get_mut(&normalize_auth_probe_ip(peer.ip())) .expect("peer state must exist before exhaustion recheck"); entry.fail_streak = AUTH_PROBE_BACKOFF_START_FAILS + AUTH_PROBE_SATURATION_GRACE_FAILS; @@ -3002,7 +3223,7 @@ async fn saturation_grace_progression_tls_reaches_cap_then_stops_incrementing() entry.last_seen = Instant::now(); } - let result = handle_tls_handshake( + let result = handle_tls_handshake_with_shared( &invalid, tokio::io::empty(), tokio::io::sink(), @@ -3011,11 +3232,12 @@ async fn saturation_grace_progression_tls_reaches_cap_then_stops_incrementing() &replay_checker, &rng, None, + shared.as_ref(), ) .await; assert!(matches!(result, HandshakeResult::BadClient { .. })); assert_eq!( - auth_probe_fail_streak_for_testing(peer.ip()), + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), peer.ip()), Some(AUTH_PROBE_BACKOFF_START_FAILS + AUTH_PROBE_SATURATION_GRACE_FAILS), "once grace is exhausted, repeated invalid TLS must be pre-auth throttled without further fail-streak growth" ); @@ -3023,16 +3245,14 @@ async fn saturation_grace_progression_tls_reaches_cap_then_stops_incrementing() #[tokio::test] async fn saturation_grace_progression_mtproto_reaches_cap_then_stops_incrementing() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let config = test_config_with_secret_hex("71717171717171717171717171717171"); let replay_checker = ReplayChecker::new(128, Duration::from_secs(60)); let peer: SocketAddr = "198.51.100.208:45208".parse().unwrap(); let now = Instant::now(); - auth_probe_state_map().insert( + auth_probe_state_for_testing_in_shared(shared.as_ref()).insert( normalize_auth_probe_ip(peer.ip()), AuthProbeState { fail_streak: AUTH_PROBE_BACKOFF_START_FAILS, @@ -3041,7 +3261,7 @@ async fn saturation_grace_progression_mtproto_reaches_cap_then_stops_incrementin }, ); { - let mut guard = auth_probe_saturation_state() + let mut guard = auth_probe_saturation_state_for_testing_in_shared(shared.as_ref()) .lock() .unwrap_or_else(|poisoned| poisoned.into_inner()); *guard = Some(AuthProbeSaturationState { @@ -3057,7 +3277,7 @@ async fn saturation_grace_progression_mtproto_reaches_cap_then_stops_incrementin AUTH_PROBE_BACKOFF_START_FAILS + 1, AUTH_PROBE_BACKOFF_START_FAILS + AUTH_PROBE_SATURATION_GRACE_FAILS, ] { - let result = handle_mtproto_handshake( + let result = handle_mtproto_handshake_with_shared( &invalid, tokio::io::empty(), tokio::io::sink(), @@ -3066,17 +3286,18 @@ async fn saturation_grace_progression_mtproto_reaches_cap_then_stops_incrementin &replay_checker, false, None, + shared.as_ref(), ) .await; assert!(matches!(result, HandshakeResult::BadClient { .. })); assert_eq!( - auth_probe_fail_streak_for_testing(peer.ip()), + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), peer.ip()), Some(expected) ); } { - let mut entry = auth_probe_state_map() + let mut entry = auth_probe_state_for_testing_in_shared(shared.as_ref()) .get_mut(&normalize_auth_probe_ip(peer.ip())) .expect("peer state must exist before exhaustion recheck"); entry.fail_streak = AUTH_PROBE_BACKOFF_START_FAILS + AUTH_PROBE_SATURATION_GRACE_FAILS; @@ -3084,7 +3305,7 @@ async fn saturation_grace_progression_mtproto_reaches_cap_then_stops_incrementin entry.last_seen = Instant::now(); } - let result = handle_mtproto_handshake( + let result = handle_mtproto_handshake_with_shared( &invalid, tokio::io::empty(), tokio::io::sink(), @@ -3093,11 +3314,12 @@ async fn saturation_grace_progression_mtproto_reaches_cap_then_stops_incrementin &replay_checker, false, None, + shared.as_ref(), ) .await; assert!(matches!(result, HandshakeResult::BadClient { .. })); assert_eq!( - auth_probe_fail_streak_for_testing(peer.ip()), + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), peer.ip()), Some(AUTH_PROBE_BACKOFF_START_FAILS + AUTH_PROBE_SATURATION_GRACE_FAILS), "once grace is exhausted, repeated invalid MTProto must be pre-auth throttled without further fail-streak growth" ); @@ -3105,10 +3327,8 @@ async fn saturation_grace_progression_mtproto_reaches_cap_then_stops_incrementin #[tokio::test] async fn saturation_grace_boundary_still_admits_valid_tls_before_exhaustion() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret = [0x72u8; 16]; let config = test_config_with_secret_hex("72727272727272727272727272727272"); @@ -3116,7 +3336,7 @@ async fn saturation_grace_boundary_still_admits_valid_tls_before_exhaustion() { let rng = SecureRandom::new(); let peer: SocketAddr = "198.51.100.209:45209".parse().unwrap(); let now = Instant::now(); - auth_probe_state_map().insert( + auth_probe_state_for_testing_in_shared(shared.as_ref()).insert( normalize_auth_probe_ip(peer.ip()), AuthProbeState { fail_streak: AUTH_PROBE_BACKOFF_START_FAILS + AUTH_PROBE_SATURATION_GRACE_FAILS - 1, @@ -3125,7 +3345,7 @@ async fn saturation_grace_boundary_still_admits_valid_tls_before_exhaustion() { }, ); { - let mut guard = auth_probe_saturation_state() + let mut guard = auth_probe_saturation_state_for_testing_in_shared(shared.as_ref()) .lock() .unwrap_or_else(|poisoned| poisoned.into_inner()); *guard = Some(AuthProbeSaturationState { @@ -3136,7 +3356,7 @@ async fn saturation_grace_boundary_still_admits_valid_tls_before_exhaustion() { } let valid = make_valid_tls_handshake(&secret, 0); - let result = handle_tls_handshake( + let result = handle_tls_handshake_with_shared( &valid, tokio::io::empty(), tokio::io::sink(), @@ -3145,6 +3365,7 @@ async fn saturation_grace_boundary_still_admits_valid_tls_before_exhaustion() { &replay_checker, &rng, None, + shared.as_ref(), ) .await; @@ -3152,15 +3373,16 @@ async fn saturation_grace_boundary_still_admits_valid_tls_before_exhaustion() { matches!(result, HandshakeResult::Success(_)), "valid TLS should still pass while peer remains within saturation grace budget" ); - assert_eq!(auth_probe_fail_streak_for_testing(peer.ip()), None); + assert_eq!( + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), peer.ip()), + None + ); } #[tokio::test] async fn saturation_grace_exhaustion_blocks_valid_tls_until_backoff_expires() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret = [0x73u8; 16]; let config = test_config_with_secret_hex("73737373737373737373737373737373"); @@ -3168,7 +3390,7 @@ async fn saturation_grace_exhaustion_blocks_valid_tls_until_backoff_expires() { let rng = SecureRandom::new(); let peer: SocketAddr = "198.51.100.210:45210".parse().unwrap(); let now = Instant::now(); - auth_probe_state_map().insert( + auth_probe_state_for_testing_in_shared(shared.as_ref()).insert( normalize_auth_probe_ip(peer.ip()), AuthProbeState { fail_streak: AUTH_PROBE_BACKOFF_START_FAILS + AUTH_PROBE_SATURATION_GRACE_FAILS, @@ -3177,7 +3399,7 @@ async fn saturation_grace_exhaustion_blocks_valid_tls_until_backoff_expires() { }, ); { - let mut guard = auth_probe_saturation_state() + let mut guard = auth_probe_saturation_state_for_testing_in_shared(shared.as_ref()) .lock() .unwrap_or_else(|poisoned| poisoned.into_inner()); *guard = Some(AuthProbeSaturationState { @@ -3188,7 +3410,7 @@ async fn saturation_grace_exhaustion_blocks_valid_tls_until_backoff_expires() { } let valid = make_valid_tls_handshake(&secret, 0); - let blocked = handle_tls_handshake( + let blocked = handle_tls_handshake_with_shared( &valid, tokio::io::empty(), tokio::io::sink(), @@ -3197,13 +3419,14 @@ async fn saturation_grace_exhaustion_blocks_valid_tls_until_backoff_expires() { &replay_checker, &rng, None, + shared.as_ref(), ) .await; assert!(matches!(blocked, HandshakeResult::BadClient { .. })); tokio::time::sleep(Duration::from_millis(230)).await; - let allowed = handle_tls_handshake( + let allowed = handle_tls_handshake_with_shared( &valid, tokio::io::empty(), tokio::io::sink(), @@ -3212,28 +3435,30 @@ async fn saturation_grace_exhaustion_blocks_valid_tls_until_backoff_expires() { &replay_checker, &rng, None, + shared.as_ref(), ) .await; assert!( matches!(allowed, HandshakeResult::Success(_)), "valid TLS should recover after peer-specific pre-auth backoff has elapsed" ); - assert_eq!(auth_probe_fail_streak_for_testing(peer.ip()), None); + assert_eq!( + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), peer.ip()), + None + ); } #[tokio::test] async fn saturation_grace_exhaustion_is_shared_across_tls_and_mtproto_for_same_peer() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let config = test_config_with_secret_hex("74747474747474747474747474747474"); let replay_checker = ReplayChecker::new(128, Duration::from_secs(60)); let rng = SecureRandom::new(); let peer: SocketAddr = "198.51.100.211:45211".parse().unwrap(); let now = Instant::now(); - auth_probe_state_map().insert( + auth_probe_state_for_testing_in_shared(shared.as_ref()).insert( normalize_auth_probe_ip(peer.ip()), AuthProbeState { fail_streak: AUTH_PROBE_BACKOFF_START_FAILS + AUTH_PROBE_SATURATION_GRACE_FAILS, @@ -3242,7 +3467,7 @@ async fn saturation_grace_exhaustion_is_shared_across_tls_and_mtproto_for_same_p }, ); { - let mut guard = auth_probe_saturation_state() + let mut guard = auth_probe_saturation_state_for_testing_in_shared(shared.as_ref()) .lock() .unwrap_or_else(|poisoned| poisoned.into_inner()); *guard = Some(AuthProbeSaturationState { @@ -3283,7 +3508,7 @@ async fn saturation_grace_exhaustion_is_shared_across_tls_and_mtproto_for_same_p assert!(matches!(mtproto_result, HandshakeResult::BadClient { .. })); assert_eq!( - auth_probe_fail_streak_for_testing(peer.ip()), + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), peer.ip()), Some(AUTH_PROBE_BACKOFF_START_FAILS + AUTH_PROBE_SATURATION_GRACE_FAILS), "saturation grace exhaustion must gate both TLS and MTProto pre-auth paths for one peer" ); @@ -3291,10 +3516,8 @@ async fn saturation_grace_exhaustion_is_shared_across_tls_and_mtproto_for_same_p #[tokio::test(flavor = "multi_thread", worker_threads = 4)] async fn adversarial_same_peer_invalid_tls_storm_does_not_bypass_saturation_grace_cap() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let config = Arc::new(test_config_with_secret_hex( "75757575757575757575757575757575", @@ -3303,7 +3526,7 @@ async fn adversarial_same_peer_invalid_tls_storm_does_not_bypass_saturation_grac let rng = Arc::new(SecureRandom::new()); let peer: SocketAddr = "198.51.100.212:45212".parse().unwrap(); let now = Instant::now(); - auth_probe_state_map().insert( + auth_probe_state_for_testing_in_shared(shared.as_ref()).insert( normalize_auth_probe_ip(peer.ip()), AuthProbeState { fail_streak: AUTH_PROBE_BACKOFF_START_FAILS + AUTH_PROBE_SATURATION_GRACE_FAILS, @@ -3312,7 +3535,7 @@ async fn adversarial_same_peer_invalid_tls_storm_does_not_bypass_saturation_grac }, ); { - let mut guard = auth_probe_saturation_state() + let mut guard = auth_probe_saturation_state_for_testing_in_shared(shared.as_ref()) .lock() .unwrap_or_else(|poisoned| poisoned.into_inner()); *guard = Some(AuthProbeSaturationState { @@ -3353,7 +3576,7 @@ async fn adversarial_same_peer_invalid_tls_storm_does_not_bypass_saturation_grac } assert_eq!( - auth_probe_fail_streak_for_testing(peer.ip()), + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), peer.ip()), Some(AUTH_PROBE_BACKOFF_START_FAILS + AUTH_PROBE_SATURATION_GRACE_FAILS), "same-peer invalid storm under exhausted grace must stay pre-auth throttled without fail-streak growth" ); @@ -3361,17 +3584,15 @@ async fn adversarial_same_peer_invalid_tls_storm_does_not_bypass_saturation_grac #[tokio::test] async fn light_fuzz_saturation_grace_tls_invalid_inputs_never_authenticate_or_panic() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let config = test_config_with_secret_hex("76767676767676767676767676767676"); let replay_checker = ReplayChecker::new(2048, Duration::from_secs(60)); let rng = SecureRandom::new(); let peer: SocketAddr = "198.51.100.213:45213".parse().unwrap(); let now = Instant::now(); - auth_probe_state_map().insert( + auth_probe_state_for_testing_in_shared(shared.as_ref()).insert( normalize_auth_probe_ip(peer.ip()), AuthProbeState { fail_streak: AUTH_PROBE_BACKOFF_START_FAILS + AUTH_PROBE_SATURATION_GRACE_FAILS, @@ -3380,7 +3601,7 @@ async fn light_fuzz_saturation_grace_tls_invalid_inputs_never_authenticate_or_pa }, ); { - let mut guard = auth_probe_saturation_state() + let mut guard = auth_probe_saturation_state_for_testing_in_shared(shared.as_ref()) .lock() .unwrap_or_else(|poisoned| poisoned.into_inner()); *guard = Some(AuthProbeSaturationState { @@ -3410,7 +3631,7 @@ async fn light_fuzz_saturation_grace_tls_invalid_inputs_never_authenticate_or_pa assert!(matches!(result, HandshakeResult::BadClient { .. })); } - let streak = auth_probe_fail_streak_for_testing(peer.ip()) + let streak = auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), peer.ip()) .expect("peer should remain tracked after repeated invalid fuzz probes"); assert!( streak >= AUTH_PROBE_BACKOFF_START_FAILS + AUTH_PROBE_SATURATION_GRACE_FAILS, @@ -3420,10 +3641,8 @@ async fn light_fuzz_saturation_grace_tls_invalid_inputs_never_authenticate_or_pa #[tokio::test(flavor = "multi_thread", worker_threads = 4)] async fn adversarial_saturation_burst_only_admits_valid_tls_and_mtproto_handshakes() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret_hex = "66666666666666666666666666666666"; let secret = [0x66u8; 16]; @@ -3435,7 +3654,7 @@ async fn adversarial_saturation_burst_only_admits_valid_tls_and_mtproto_handshak let now = Instant::now(); { - let mut guard = auth_probe_saturation_state() + let mut guard = auth_probe_saturation_state_for_testing_in_shared(shared.as_ref()) .lock() .unwrap_or_else(|poisoned| poisoned.into_inner()); *guard = Some(AuthProbeSaturationState { @@ -3548,10 +3767,8 @@ async fn adversarial_saturation_burst_only_admits_valid_tls_and_mtproto_handshak #[tokio::test] async fn expired_saturation_keeps_per_ip_throttle_enforced_for_valid_tls() { - let _guard = auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); let secret = [0x67u8; 16]; let config = test_config_with_secret_hex("67676767676767676767676767676767"); @@ -3560,7 +3777,7 @@ async fn expired_saturation_keeps_per_ip_throttle_enforced_for_valid_tls() { let peer: SocketAddr = "198.51.100.110:45110".parse().unwrap(); let now = Instant::now(); - auth_probe_state_map().insert( + auth_probe_state_for_testing_in_shared(shared.as_ref()).insert( normalize_auth_probe_ip(peer.ip()), AuthProbeState { fail_streak: AUTH_PROBE_BACKOFF_START_FAILS, @@ -3569,7 +3786,7 @@ async fn expired_saturation_keeps_per_ip_throttle_enforced_for_valid_tls() { }, ); { - let mut guard = auth_probe_saturation_state() + let mut guard = auth_probe_saturation_state_for_testing_in_shared(shared.as_ref()) .lock() .unwrap_or_else(|poisoned| poisoned.into_inner()); *guard = Some(AuthProbeSaturationState { @@ -3580,7 +3797,7 @@ async fn expired_saturation_keeps_per_ip_throttle_enforced_for_valid_tls() { } let valid = make_valid_tls_handshake(&secret, 0); - let result = handle_tls_handshake( + let result = handle_tls_handshake_with_shared( &valid, tokio::io::empty(), tokio::io::sink(), @@ -3589,6 +3806,7 @@ async fn expired_saturation_keeps_per_ip_throttle_enforced_for_valid_tls() { &replay_checker, &rng, None, + shared.as_ref(), ) .await; diff --git a/src/proxy/tests/handshake_timing_manual_bench_tests.rs b/src/proxy/tests/handshake_timing_manual_bench_tests.rs index 13d112c..458cb4f 100644 --- a/src/proxy/tests/handshake_timing_manual_bench_tests.rs +++ b/src/proxy/tests/handshake_timing_manual_bench_tests.rs @@ -4,12 +4,6 @@ use crate::protocol::constants::{ProtoTag, TLS_RECORD_HANDSHAKE, TLS_VERSION}; use std::net::SocketAddr; use std::time::{Duration, Instant}; -fn auth_probe_test_guard() -> std::sync::MutexGuard<'static, ()> { - auth_probe_test_lock() - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()) -} - fn make_valid_mtproto_handshake( secret_hex: &str, proto_tag: ProtoTag, @@ -149,8 +143,8 @@ fn median_ns(samples: &mut [u128]) -> u128 { #[tokio::test] #[ignore = "manual benchmark: timing-sensitive and host-dependent"] async fn mtproto_user_scan_timing_manual_benchmark() { - let _guard = auth_probe_test_guard(); - clear_auth_probe_state_for_testing(); + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); const DECOY_USERS: usize = 8_000; const ITERATIONS: usize = 250; @@ -243,7 +237,7 @@ async fn mtproto_user_scan_timing_manual_benchmark() { #[tokio::test] #[ignore = "manual benchmark: timing-sensitive and host-dependent"] async fn tls_sni_preferred_vs_no_sni_fallback_manual_benchmark() { - let _guard = auth_probe_test_guard(); + let shared = ProxySharedState::new(); const DECOY_USERS: usize = 8_000; const ITERATIONS: usize = 250; @@ -281,7 +275,7 @@ async fn tls_sni_preferred_vs_no_sni_fallback_manual_benchmark() { let no_sni = make_valid_tls_handshake(&target_secret, (i as u32).wrapping_add(10_000)); let started_sni = Instant::now(); - let sni_secrets = decode_user_secrets(&config, Some(preferred_user)); + let sni_secrets = decode_user_secrets_in(shared.as_ref(), &config, Some(preferred_user)); let sni_result = tls::validate_tls_handshake_with_replay_window( &with_sni, &sni_secrets, @@ -292,7 +286,7 @@ async fn tls_sni_preferred_vs_no_sni_fallback_manual_benchmark() { assert!(sni_result.is_some()); let started_no_sni = Instant::now(); - let no_sni_secrets = decode_user_secrets(&config, None); + let no_sni_secrets = decode_user_secrets_in(shared.as_ref(), &config, None); let no_sni_result = tls::validate_tls_handshake_with_replay_window( &no_sni, &no_sni_secrets, diff --git a/src/proxy/tests/masking_baseline_invariant_tests.rs b/src/proxy/tests/masking_baseline_invariant_tests.rs new file mode 100644 index 0000000..2c36406 --- /dev/null +++ b/src/proxy/tests/masking_baseline_invariant_tests.rs @@ -0,0 +1,156 @@ +use super::*; +use tokio::io::duplex; +use tokio::net::TcpListener; +use tokio::time::{Duration, Instant, timeout}; + +#[test] +fn masking_baseline_timing_normalization_budget_within_bounds() { + let mut config = ProxyConfig::default(); + config.censorship.mask_timing_normalization_enabled = true; + config.censorship.mask_timing_normalization_floor_ms = 120; + config.censorship.mask_timing_normalization_ceiling_ms = 180; + + for _ in 0..256 { + let budget = mask_outcome_target_budget(&config); + assert!(budget >= Duration::from_millis(120)); + assert!(budget <= Duration::from_millis(180)); + } +} + +#[tokio::test] +async fn masking_baseline_fallback_relays_to_mask_host() { + let listener = TcpListener::bind("127.0.0.1:0").await.unwrap(); + let backend_addr = listener.local_addr().unwrap(); + let initial = b"GET /baseline HTTP/1.1\r\nHost: x\r\n\r\n".to_vec(); + let reply = b"HTTP/1.1 200 OK\r\nContent-Length: 2\r\n\r\nOK".to_vec(); + + let accept_task = tokio::spawn({ + let initial = initial.clone(); + let reply = reply.clone(); + async move { + let (mut stream, _) = listener.accept().await.unwrap(); + let mut seen = vec![0u8; initial.len()]; + stream.read_exact(&mut seen).await.unwrap(); + assert_eq!(seen, initial); + stream.write_all(&reply).await.unwrap(); + } + }); + + let mut config = ProxyConfig::default(); + config.general.beobachten = false; + config.censorship.mask = true; + config.censorship.mask_host = Some("127.0.0.1".to_string()); + config.censorship.mask_port = backend_addr.port(); + config.censorship.mask_unix_sock = None; + config.censorship.mask_proxy_protocol = 0; + + let peer: SocketAddr = "203.0.113.70:55070".parse().unwrap(); + let local_addr: SocketAddr = "127.0.0.1:443".parse().unwrap(); + + let (client_reader, _client_writer) = duplex(1024); + let (mut visible_reader, visible_writer) = duplex(2048); + let beobachten = BeobachtenStore::new(); + + handle_bad_client( + client_reader, + visible_writer, + &initial, + peer, + local_addr, + &config, + &beobachten, + ) + .await; + + let mut observed = vec![0u8; reply.len()]; + visible_reader.read_exact(&mut observed).await.unwrap(); + assert_eq!(observed, reply); + accept_task.await.unwrap(); +} + +#[test] +fn masking_baseline_no_normalization_returns_default_budget() { + let mut config = ProxyConfig::default(); + config.censorship.mask_timing_normalization_enabled = false; + let budget = mask_outcome_target_budget(&config); + assert_eq!(budget, MASK_TIMEOUT); +} + +#[tokio::test] +async fn masking_baseline_unreachable_mask_host_silent_failure() { + let mut config = ProxyConfig::default(); + config.general.beobachten = false; + config.censorship.mask = true; + config.censorship.mask_unix_sock = None; + config.censorship.mask_host = Some("127.0.0.1".to_string()); + config.censorship.mask_port = 1; + config.censorship.mask_timing_normalization_enabled = false; + + let peer: SocketAddr = "203.0.113.71:55071".parse().unwrap(); + let local_addr: SocketAddr = "127.0.0.1:443".parse().unwrap(); + let beobachten = BeobachtenStore::new(); + + let (client_reader, _client_writer) = duplex(1024); + let (mut visible_reader, visible_writer) = duplex(1024); + + let started = Instant::now(); + handle_bad_client( + client_reader, + visible_writer, + b"GET / HTTP/1.1\r\n\r\n", + peer, + local_addr, + &config, + &beobachten, + ) + .await; + let elapsed = started.elapsed(); + + assert!(elapsed < Duration::from_secs(1)); + + let mut buf = [0u8; 1]; + let read_res = timeout(Duration::from_millis(50), visible_reader.read(&mut buf)).await; + match read_res { + Ok(Ok(0)) | Err(_) => {} + Ok(Ok(n)) => panic!("expected no response bytes, got {n}"), + Ok(Err(e)) => panic!("unexpected client-side read error: {e}"), + } +} + +#[tokio::test] +async fn masking_baseline_light_fuzz_initial_data_no_panic() { + let mut config = ProxyConfig::default(); + config.general.beobachten = false; + config.censorship.mask = false; + + let peer: SocketAddr = "203.0.113.72:55072".parse().unwrap(); + let local_addr: SocketAddr = "127.0.0.1:443".parse().unwrap(); + let beobachten = BeobachtenStore::new(); + + let corpus: Vec> = vec![ + vec![], + vec![0x00], + vec![0xFF; 1024], + (0..255u8).collect(), + b"\xF0\x28\x8C\x28".to_vec(), + ]; + + for sample in corpus { + let (client_reader, _client_writer) = duplex(1024); + let (_visible_reader, visible_writer) = duplex(1024); + timeout( + Duration::from_millis(300), + handle_bad_client( + client_reader, + visible_writer, + &sample, + peer, + local_addr, + &config, + &beobachten, + ), + ) + .await + .expect("fuzz sample must complete in bounded time"); + } +} diff --git a/src/proxy/tests/masking_lognormal_timing_security_tests.rs b/src/proxy/tests/masking_lognormal_timing_security_tests.rs new file mode 100644 index 0000000..5d6c456 --- /dev/null +++ b/src/proxy/tests/masking_lognormal_timing_security_tests.rs @@ -0,0 +1,336 @@ +use super::*; +use rand::SeedableRng; +use rand::rngs::StdRng; + +fn seeded_rng(seed: u64) -> StdRng { + StdRng::seed_from_u64(seed) +} + +// ── Positive: all samples within configured envelope ──────────────────── + +#[test] +fn masking_lognormal_all_samples_within_configured_envelope() { + let mut rng = seeded_rng(42); + let floor: u64 = 500; + let ceiling: u64 = 2000; + for _ in 0..10_000 { + let val = sample_lognormal_percentile_bounded(floor, ceiling, &mut rng); + assert!( + val >= floor && val <= ceiling, + "sample {} outside [{}, {}]", + val, + floor, + ceiling, + ); + } +} + +// ── Statistical: median near geometric mean ───────────────────────────── + +#[test] +fn masking_lognormal_sample_median_near_geometric_mean_of_range() { + let mut rng = seeded_rng(42); + let floor: u64 = 500; + let ceiling: u64 = 2000; + let geometric_mean = ((floor as f64) * (ceiling as f64)).sqrt(); + + let mut samples: Vec = (0..10_000) + .map(|_| sample_lognormal_percentile_bounded(floor, ceiling, &mut rng)) + .collect(); + samples.sort(); + let median = samples[samples.len() / 2] as f64; + + let tolerance = geometric_mean * 0.10; + assert!( + (median - geometric_mean).abs() <= tolerance, + "median {} not within 10% of geometric mean {} (tolerance {})", + median, + geometric_mean, + tolerance, + ); +} + +// ── Edge: degenerate floor == ceiling returns exactly that value ───────── + +#[test] +fn masking_lognormal_degenerate_floor_eq_ceiling_returns_floor() { + let mut rng = seeded_rng(99); + for _ in 0..100 { + let val = sample_lognormal_percentile_bounded(1000, 1000, &mut rng); + assert_eq!( + val, 1000, + "floor == ceiling must always return exactly that value" + ); + } +} + +// ── Edge: floor > ceiling (misconfiguration) clamps safely ────────────── + +#[test] +fn masking_lognormal_floor_greater_than_ceiling_returns_ceiling() { + let mut rng = seeded_rng(77); + let val = sample_lognormal_percentile_bounded(2000, 500, &mut rng); + assert_eq!( + val, 500, + "floor > ceiling misconfiguration must return ceiling (the minimum)" + ); +} + +// ── Edge: floor == 1, ceiling == 1 ────────────────────────────────────── + +#[test] +fn masking_lognormal_floor_1_ceiling_1_returns_1() { + let mut rng = seeded_rng(12); + let val = sample_lognormal_percentile_bounded(1, 1, &mut rng); + assert_eq!(val, 1); +} + +// ── Edge: floor == 1, ceiling very large ──────────────────────────────── + +#[test] +fn masking_lognormal_wide_range_all_samples_within_bounds() { + let mut rng = seeded_rng(55); + let floor: u64 = 1; + let ceiling: u64 = 100_000; + for _ in 0..10_000 { + let val = sample_lognormal_percentile_bounded(floor, ceiling, &mut rng); + assert!( + val >= floor && val <= ceiling, + "sample {} outside [{}, {}]", + val, + floor, + ceiling, + ); + } +} + +// ── Adversarial: extreme sigma (floor very close to ceiling) ──────────── + +#[test] +fn masking_lognormal_narrow_range_does_not_panic() { + let mut rng = seeded_rng(88); + let floor: u64 = 999; + let ceiling: u64 = 1001; + for _ in 0..10_000 { + let val = sample_lognormal_percentile_bounded(floor, ceiling, &mut rng); + assert!( + val >= floor && val <= ceiling, + "narrow range sample {} outside [{}, {}]", + val, + floor, + ceiling, + ); + } +} + +// ── Adversarial: u64::MAX ceiling does not overflow ────────────────────── + +#[test] +fn masking_lognormal_u64_max_ceiling_no_overflow() { + let mut rng = seeded_rng(123); + let floor: u64 = 1; + let ceiling: u64 = u64::MAX; + for _ in 0..1000 { + let val = sample_lognormal_percentile_bounded(floor, ceiling, &mut rng); + assert!(val >= floor, "sample {} below floor {}", val, floor); + // u64::MAX clamp ensures no overflow + } +} + +// ── Adversarial: floor == 0 guard ─────────────────────────────────────── +// The function should handle floor=0 gracefully even though callers +// should never pass it. Verifies no panic on ln(0). + +#[test] +fn masking_lognormal_floor_zero_no_panic() { + let mut rng = seeded_rng(200); + let val = sample_lognormal_percentile_bounded(0, 1000, &mut rng); + assert!(val <= 1000, "sample {} exceeds ceiling 1000", val); +} + +// ── Adversarial: both zero → returns 0 ────────────────────────────────── + +#[test] +fn masking_lognormal_both_zero_returns_zero() { + let mut rng = seeded_rng(201); + let val = sample_lognormal_percentile_bounded(0, 0, &mut rng); + assert_eq!(val, 0, "floor=0 ceiling=0 must return 0"); +} + +// ── Distribution shape: not uniform ───────────────────────────────────── +// A DPI classifier trained on uniform delay samples should detect a +// distribution where > 60% of samples fall in the lower half of the range. +// Log-normal is right-skewed: more samples near floor than ceiling. + +#[test] +fn masking_lognormal_distribution_is_right_skewed() { + let mut rng = seeded_rng(42); + let floor: u64 = 100; + let ceiling: u64 = 5000; + let midpoint = (floor + ceiling) / 2; + + let samples: Vec = (0..10_000) + .map(|_| sample_lognormal_percentile_bounded(floor, ceiling, &mut rng)) + .collect(); + + let below_mid = samples.iter().filter(|&&s| s < midpoint).count(); + let ratio = below_mid as f64 / samples.len() as f64; + + assert!( + ratio > 0.55, + "Log-normal should be right-skewed (>55% below midpoint), got {}%", + ratio * 100.0, + ); +} + +// ── Determinism: same seed produces same sequence ─────────────────────── + +#[test] +fn masking_lognormal_deterministic_with_same_seed() { + let mut rng1 = seeded_rng(42); + let mut rng2 = seeded_rng(42); + for _ in 0..100 { + let a = sample_lognormal_percentile_bounded(500, 2000, &mut rng1); + let b = sample_lognormal_percentile_bounded(500, 2000, &mut rng2); + assert_eq!(a, b, "Same seed must produce same output"); + } +} + +// ── Fuzz: 1000 random (floor, ceiling) pairs, no panics ───────────────── + +#[test] +fn masking_lognormal_fuzz_random_params_no_panic() { + use rand::Rng; + let mut rng = seeded_rng(999); + for _ in 0..1000 { + let a: u64 = rng.random_range(0..=10_000); + let b: u64 = rng.random_range(0..=10_000); + let floor = a.min(b); + let ceiling = a.max(b); + let val = sample_lognormal_percentile_bounded(floor, ceiling, &mut rng); + assert!( + val >= floor && val <= ceiling, + "fuzz: sample {} outside [{}, {}]", + val, + floor, + ceiling, + ); + } +} + +// ── Fuzz: adversarial floor > ceiling pairs ────────────────────────────── + +#[test] +fn masking_lognormal_fuzz_inverted_params_no_panic() { + use rand::Rng; + let mut rng = seeded_rng(777); + for _ in 0..500 { + let floor: u64 = rng.random_range(1..=10_000); + let ceiling: u64 = rng.random_range(0..floor); + // When floor > ceiling, must return ceiling (the smaller value) + let val = sample_lognormal_percentile_bounded(floor, ceiling, &mut rng); + assert_eq!( + val, ceiling, + "inverted: floor={} ceiling={} should return ceiling, got {}", + floor, ceiling, val, + ); + } +} + +// ── Security: clamp spike check ───────────────────────────────────────── +// With well-parameterized sigma, no more than 5% of samples should be +// at exactly floor or exactly ceiling (clamp spikes). A spike > 10% +// is detectable by DPI as bimodal. + +#[test] +fn masking_lognormal_no_clamp_spike_at_boundaries() { + let mut rng = seeded_rng(42); + let floor: u64 = 500; + let ceiling: u64 = 2000; + let n = 10_000; + let samples: Vec = (0..n) + .map(|_| sample_lognormal_percentile_bounded(floor, ceiling, &mut rng)) + .collect(); + + let at_floor = samples.iter().filter(|&&s| s == floor).count(); + let at_ceiling = samples.iter().filter(|&&s| s == ceiling).count(); + let floor_pct = at_floor as f64 / n as f64; + let ceiling_pct = at_ceiling as f64 / n as f64; + + assert!( + floor_pct < 0.05, + "floor clamp spike: {}% of samples at exactly floor (max 5%)", + floor_pct * 100.0, + ); + assert!( + ceiling_pct < 0.05, + "ceiling clamp spike: {}% of samples at exactly ceiling (max 5%)", + ceiling_pct * 100.0, + ); +} + +// ── Integration: mask_outcome_target_budget uses log-normal for path 3 ── + +#[tokio::test] +async fn masking_lognormal_integration_budget_within_bounds() { + let mut config = ProxyConfig::default(); + config.censorship.mask_timing_normalization_enabled = true; + config.censorship.mask_timing_normalization_floor_ms = 500; + config.censorship.mask_timing_normalization_ceiling_ms = 2000; + + for _ in 0..100 { + let budget = mask_outcome_target_budget(&config); + let ms = budget.as_millis() as u64; + assert!( + ms >= 500 && ms <= 2000, + "budget {} ms outside [500, 2000]", + ms, + ); + } +} + +// ── Integration: floor == 0 path stays uniform (NOT log-normal) ───────── + +#[tokio::test] +async fn masking_lognormal_floor_zero_path_stays_uniform() { + let mut config = ProxyConfig::default(); + config.censorship.mask_timing_normalization_enabled = true; + config.censorship.mask_timing_normalization_floor_ms = 0; + config.censorship.mask_timing_normalization_ceiling_ms = 1000; + + for _ in 0..100 { + let budget = mask_outcome_target_budget(&config); + let ms = budget.as_millis() as u64; + // floor=0 path uses uniform [0, ceiling], not log-normal + assert!(ms <= 1000, "budget {} ms exceeds ceiling 1000", ms); + } +} + +// ── Integration: floor > ceiling misconfiguration is safe ─────────────── + +#[tokio::test] +async fn masking_lognormal_misconfigured_floor_gt_ceiling_safe() { + let mut config = ProxyConfig::default(); + config.censorship.mask_timing_normalization_enabled = true; + config.censorship.mask_timing_normalization_floor_ms = 2000; + config.censorship.mask_timing_normalization_ceiling_ms = 500; + + let budget = mask_outcome_target_budget(&config); + let ms = budget.as_millis() as u64; + // floor > ceiling: should not exceed the minimum of the two + assert!( + ms <= 2000, + "misconfigured budget {} ms should be bounded", + ms, + ); +} + +// ── Stress: rapid repeated calls do not panic or starve ───────────────── + +#[test] +fn masking_lognormal_stress_rapid_calls_no_panic() { + let mut rng = seeded_rng(42); + for _ in 0..100_000 { + let _ = sample_lognormal_percentile_bounded(100, 5000, &mut rng); + } +} diff --git a/src/proxy/tests/middle_relay_baseline_invariant_tests.rs b/src/proxy/tests/middle_relay_baseline_invariant_tests.rs new file mode 100644 index 0000000..5e9ae2e --- /dev/null +++ b/src/proxy/tests/middle_relay_baseline_invariant_tests.rs @@ -0,0 +1,60 @@ +use super::*; +use std::time::{Duration, Instant}; + +#[test] +fn middle_relay_baseline_public_api_idle_roundtrip_contract() { + let shared = ProxySharedState::new(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); + + assert!(mark_relay_idle_candidate_for_testing(shared.as_ref(), 7001)); + assert_eq!( + oldest_relay_idle_candidate_for_testing(shared.as_ref()), + Some(7001) + ); + + clear_relay_idle_candidate_for_testing(shared.as_ref(), 7001); + assert_ne!( + oldest_relay_idle_candidate_for_testing(shared.as_ref()), + Some(7001) + ); + + assert!(mark_relay_idle_candidate_for_testing(shared.as_ref(), 7001)); + assert_eq!( + oldest_relay_idle_candidate_for_testing(shared.as_ref()), + Some(7001) + ); + + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); +} + +#[test] +fn middle_relay_baseline_public_api_desync_window_contract() { + let shared = ProxySharedState::new(); + clear_desync_dedup_for_testing_in_shared(shared.as_ref()); + + let key = 0xDEAD_BEEF_0000_0001u64; + let t0 = Instant::now(); + + assert!(should_emit_full_desync_for_testing( + shared.as_ref(), + key, + false, + t0 + )); + assert!(!should_emit_full_desync_for_testing( + shared.as_ref(), + key, + false, + t0 + Duration::from_secs(1) + )); + + let t1 = t0 + DESYNC_DEDUP_WINDOW + Duration::from_millis(10); + assert!(should_emit_full_desync_for_testing( + shared.as_ref(), + key, + false, + t1 + )); + + clear_desync_dedup_for_testing_in_shared(shared.as_ref()); +} diff --git a/src/proxy/tests/middle_relay_desync_all_full_dedup_security_tests.rs b/src/proxy/tests/middle_relay_desync_all_full_dedup_security_tests.rs index dab0dff..883f390 100644 --- a/src/proxy/tests/middle_relay_desync_all_full_dedup_security_tests.rs +++ b/src/proxy/tests/middle_relay_desync_all_full_dedup_security_tests.rs @@ -5,22 +5,25 @@ use std::thread; #[test] fn desync_all_full_bypass_does_not_initialize_or_grow_dedup_cache() { - let _guard = desync_dedup_test_lock() - .lock() - .expect("desync dedup test lock must be available"); - clear_desync_dedup_for_testing(); + let shared = ProxySharedState::new(); + clear_desync_dedup_for_testing_in_shared(shared.as_ref()); - let initial_len = DESYNC_DEDUP.get().map(|dedup| dedup.len()).unwrap_or(0); + let initial_len = desync_dedup_len_for_testing(shared.as_ref()); let now = Instant::now(); for i in 0..20_000u64 { assert!( - should_emit_full_desync(0xD35E_D000_0000_0000u64 ^ i, true, now), + should_emit_full_desync_for_testing( + shared.as_ref(), + 0xD35E_D000_0000_0000u64 ^ i, + true, + now + ), "desync_all_full path must always emit" ); } - let after_len = DESYNC_DEDUP.get().map(|dedup| dedup.len()).unwrap_or(0); + let after_len = desync_dedup_len_for_testing(shared.as_ref()); assert_eq!( after_len, initial_len, "desync_all_full bypass must not allocate or accumulate dedup entries" @@ -29,39 +32,39 @@ fn desync_all_full_bypass_does_not_initialize_or_grow_dedup_cache() { #[test] fn desync_all_full_bypass_keeps_existing_dedup_entries_unchanged() { - let _guard = desync_dedup_test_lock() - .lock() - .expect("desync dedup test lock must be available"); - clear_desync_dedup_for_testing(); + let shared = ProxySharedState::new(); + clear_desync_dedup_for_testing_in_shared(shared.as_ref()); - let dedup = DESYNC_DEDUP.get_or_init(DashMap::new); let seed_time = Instant::now() - Duration::from_secs(7); - dedup.insert(0xAAAABBBBCCCCDDDD, seed_time); - dedup.insert(0x1111222233334444, seed_time); + desync_dedup_insert_for_testing(shared.as_ref(), 0xAAAABBBBCCCCDDDD, seed_time); + desync_dedup_insert_for_testing(shared.as_ref(), 0x1111222233334444, seed_time); let now = Instant::now(); for i in 0..2048u64 { assert!( - should_emit_full_desync(0xF011_F000_0000_0000u64 ^ i, true, now), + should_emit_full_desync_for_testing( + shared.as_ref(), + 0xF011_F000_0000_0000u64 ^ i, + true, + now + ), "desync_all_full must bypass suppression and dedup refresh" ); } assert_eq!( - dedup.len(), + desync_dedup_len_for_testing(shared.as_ref()), 2, "bypass path must not mutate dedup cardinality" ); assert_eq!( - *dedup - .get(&0xAAAABBBBCCCCDDDD) + desync_dedup_get_for_testing(shared.as_ref(), 0xAAAABBBBCCCCDDDD) .expect("seed key must remain"), seed_time, "bypass path must not refresh existing dedup timestamps" ); assert_eq!( - *dedup - .get(&0x1111222233334444) + desync_dedup_get_for_testing(shared.as_ref(), 0x1111222233334444) .expect("seed key must remain"), seed_time, "bypass path must not touch unrelated dedup entries" @@ -70,14 +73,13 @@ fn desync_all_full_bypass_keeps_existing_dedup_entries_unchanged() { #[test] fn edge_all_full_burst_does_not_poison_later_false_path_tracking() { - let _guard = desync_dedup_test_lock() - .lock() - .expect("desync dedup test lock must be available"); - clear_desync_dedup_for_testing(); + let shared = ProxySharedState::new(); + clear_desync_dedup_for_testing_in_shared(shared.as_ref()); let now = Instant::now(); for i in 0..8192u64 { - assert!(should_emit_full_desync( + assert!(should_emit_full_desync_for_testing( + shared.as_ref(), 0xABCD_0000_0000_0000 ^ i, true, now @@ -86,26 +88,20 @@ fn edge_all_full_burst_does_not_poison_later_false_path_tracking() { let tracked_key = 0xDEAD_BEEF_0000_0001u64; assert!( - should_emit_full_desync(tracked_key, false, now), + should_emit_full_desync_for_testing(shared.as_ref(), tracked_key, false, now), "first false-path event after all_full burst must still be tracked and emitted" ); - let dedup = DESYNC_DEDUP - .get() - .expect("false path should initialize dedup"); - assert!(dedup.get(&tracked_key).is_some()); + assert!(desync_dedup_get_for_testing(shared.as_ref(), tracked_key).is_some()); } #[test] fn adversarial_mixed_sequence_true_steps_never_change_cache_len() { - let _guard = desync_dedup_test_lock() - .lock() - .expect("desync dedup test lock must be available"); - clear_desync_dedup_for_testing(); + let shared = ProxySharedState::new(); + clear_desync_dedup_for_testing_in_shared(shared.as_ref()); - let dedup = DESYNC_DEDUP.get_or_init(DashMap::new); for i in 0..256u64 { - dedup.insert(0x1000_0000_0000_0000 ^ i, Instant::now()); + desync_dedup_insert_for_testing(shared.as_ref(), 0x1000_0000_0000_0000 ^ i, Instant::now()); } let mut seed = 0xC0DE_CAFE_BAAD_F00Du64; @@ -116,9 +112,14 @@ fn adversarial_mixed_sequence_true_steps_never_change_cache_len() { let flag_all_full = (seed & 0x1) == 1; let key = 0x7000_0000_0000_0000u64 ^ i ^ seed; - let before = dedup.len(); - let _ = should_emit_full_desync(key, flag_all_full, Instant::now()); - let after = dedup.len(); + let before = desync_dedup_len_for_testing(shared.as_ref()); + let _ = should_emit_full_desync_for_testing( + shared.as_ref(), + key, + flag_all_full, + Instant::now(), + ); + let after = desync_dedup_len_for_testing(shared.as_ref()); if flag_all_full { assert_eq!(after, before, "all_full step must not mutate dedup length"); @@ -128,50 +129,51 @@ fn adversarial_mixed_sequence_true_steps_never_change_cache_len() { #[test] fn light_fuzz_all_full_mode_always_emits_and_stays_bounded() { - let _guard = desync_dedup_test_lock() - .lock() - .expect("desync dedup test lock must be available"); - clear_desync_dedup_for_testing(); + let shared = ProxySharedState::new(); + clear_desync_dedup_for_testing_in_shared(shared.as_ref()); let mut seed = 0x1234_5678_9ABC_DEF0u64; - let before = DESYNC_DEDUP.get().map(|d| d.len()).unwrap_or(0); + let before = desync_dedup_len_for_testing(shared.as_ref()); for _ in 0..20_000 { seed ^= seed << 7; seed ^= seed >> 9; seed ^= seed << 8; let key = seed ^ 0x55AA_55AA_55AA_55AAu64; - assert!(should_emit_full_desync(key, true, Instant::now())); + assert!(should_emit_full_desync_for_testing( + shared.as_ref(), + key, + true, + Instant::now() + )); } - let after = DESYNC_DEDUP.get().map(|d| d.len()).unwrap_or(0); + let after = desync_dedup_len_for_testing(shared.as_ref()); assert_eq!(after, before); assert!(after <= DESYNC_DEDUP_MAX_ENTRIES); } #[test] fn stress_parallel_all_full_storm_does_not_grow_or_mutate_cache() { - let _guard = desync_dedup_test_lock() - .lock() - .expect("desync dedup test lock must be available"); - clear_desync_dedup_for_testing(); + let shared = ProxySharedState::new(); + clear_desync_dedup_for_testing_in_shared(shared.as_ref()); - let dedup = DESYNC_DEDUP.get_or_init(DashMap::new); let seed_time = Instant::now() - Duration::from_secs(2); for i in 0..1024u64 { - dedup.insert(0x8888_0000_0000_0000 ^ i, seed_time); + desync_dedup_insert_for_testing(shared.as_ref(), 0x8888_0000_0000_0000 ^ i, seed_time); } - let before_len = dedup.len(); + let before_len = desync_dedup_len_for_testing(shared.as_ref()); let emits = Arc::new(AtomicUsize::new(0)); let mut workers = Vec::new(); for worker in 0..16u64 { let emits = Arc::clone(&emits); + let shared = shared.clone(); workers.push(thread::spawn(move || { let now = Instant::now(); for i in 0..4096u64 { let key = 0xFACE_0000_0000_0000u64 ^ (worker << 20) ^ i; - if should_emit_full_desync(key, true, now) { + if should_emit_full_desync_for_testing(shared.as_ref(), key, true, now) { emits.fetch_add(1, Ordering::Relaxed); } } @@ -184,7 +186,7 @@ fn stress_parallel_all_full_storm_does_not_grow_or_mutate_cache() { assert_eq!(emits.load(Ordering::Relaxed), 16 * 4096); assert_eq!( - dedup.len(), + desync_dedup_len_for_testing(shared.as_ref()), before_len, "parallel all_full storm must not mutate cache len" ); diff --git a/src/proxy/tests/middle_relay_idle_policy_security_tests.rs b/src/proxy/tests/middle_relay_idle_policy_security_tests.rs index fd3243d..8a3d580 100644 --- a/src/proxy/tests/middle_relay_idle_policy_security_tests.rs +++ b/src/proxy/tests/middle_relay_idle_policy_security_tests.rs @@ -360,73 +360,103 @@ async fn stress_many_idle_sessions_fail_closed_without_hang() { #[test] fn pressure_evicts_oldest_idle_candidate_with_deterministic_ordering() { - let _guard = relay_idle_pressure_test_scope(); - clear_relay_idle_pressure_state_for_testing(); + let shared = ProxySharedState::new(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); let stats = Stats::new(); - assert!(mark_relay_idle_candidate(10)); - assert!(mark_relay_idle_candidate(11)); - assert_eq!(oldest_relay_idle_candidate(), Some(10)); + assert!(mark_relay_idle_candidate_for_testing(shared.as_ref(), 10)); + assert!(mark_relay_idle_candidate_for_testing(shared.as_ref(), 11)); + assert_eq!( + oldest_relay_idle_candidate_for_testing(shared.as_ref()), + Some(10) + ); - note_relay_pressure_event(); + note_relay_pressure_event_for_testing(shared.as_ref()); let mut seen_for_newer = 0u64; assert!( - !maybe_evict_idle_candidate_on_pressure(11, &mut seen_for_newer, &stats), + !maybe_evict_idle_candidate_on_pressure_for_testing( + shared.as_ref(), + 11, + &mut seen_for_newer, + &stats + ), "newer idle candidate must not be evicted while older candidate exists" ); - assert_eq!(oldest_relay_idle_candidate(), Some(10)); + assert_eq!( + oldest_relay_idle_candidate_for_testing(shared.as_ref()), + Some(10) + ); let mut seen_for_oldest = 0u64; assert!( - maybe_evict_idle_candidate_on_pressure(10, &mut seen_for_oldest, &stats), + maybe_evict_idle_candidate_on_pressure_for_testing( + shared.as_ref(), + 10, + &mut seen_for_oldest, + &stats + ), "oldest idle candidate must be evicted first under pressure" ); - assert_eq!(oldest_relay_idle_candidate(), Some(11)); + assert_eq!( + oldest_relay_idle_candidate_for_testing(shared.as_ref()), + Some(11) + ); assert_eq!(stats.get_relay_pressure_evict_total(), 1); - clear_relay_idle_pressure_state_for_testing(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); } #[test] fn pressure_does_not_evict_without_new_pressure_signal() { - let _guard = relay_idle_pressure_test_scope(); - clear_relay_idle_pressure_state_for_testing(); + let shared = ProxySharedState::new(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); let stats = Stats::new(); - assert!(mark_relay_idle_candidate(21)); - let mut seen = relay_pressure_event_seq(); + assert!(mark_relay_idle_candidate_for_testing(shared.as_ref(), 21)); + let mut seen = relay_pressure_event_seq_for_testing(shared.as_ref()); assert!( - !maybe_evict_idle_candidate_on_pressure(21, &mut seen, &stats), + !maybe_evict_idle_candidate_on_pressure_for_testing(shared.as_ref(), 21, &mut seen, &stats), "without new pressure signal, candidate must stay" ); assert_eq!(stats.get_relay_pressure_evict_total(), 0); - assert_eq!(oldest_relay_idle_candidate(), Some(21)); + assert_eq!( + oldest_relay_idle_candidate_for_testing(shared.as_ref()), + Some(21) + ); - clear_relay_idle_pressure_state_for_testing(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); } #[test] fn stress_pressure_eviction_preserves_fifo_across_many_candidates() { - let _guard = relay_idle_pressure_test_scope(); - clear_relay_idle_pressure_state_for_testing(); + let shared = ProxySharedState::new(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); let stats = Stats::new(); let mut seen_per_conn = std::collections::HashMap::new(); for conn_id in 1000u64..1064u64 { - assert!(mark_relay_idle_candidate(conn_id)); + assert!(mark_relay_idle_candidate_for_testing( + shared.as_ref(), + conn_id + )); seen_per_conn.insert(conn_id, 0u64); } for expected in 1000u64..1064u64 { - note_relay_pressure_event(); + note_relay_pressure_event_for_testing(shared.as_ref()); let mut seen = *seen_per_conn .get(&expected) .expect("per-conn pressure cursor must exist"); assert!( - maybe_evict_idle_candidate_on_pressure(expected, &mut seen, &stats), + maybe_evict_idle_candidate_on_pressure_for_testing( + shared.as_ref(), + expected, + &mut seen, + &stats + ), "expected conn_id {expected} must be evicted next by deterministic FIFO ordering" ); seen_per_conn.insert(expected, seen); @@ -436,33 +466,51 @@ fn stress_pressure_eviction_preserves_fifo_across_many_candidates() { } else { Some(expected + 1) }; - assert_eq!(oldest_relay_idle_candidate(), next); + assert_eq!( + oldest_relay_idle_candidate_for_testing(shared.as_ref()), + next + ); } assert_eq!(stats.get_relay_pressure_evict_total(), 64); - clear_relay_idle_pressure_state_for_testing(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); } #[test] fn blackhat_single_pressure_event_must_not_evict_more_than_one_candidate() { - let _guard = relay_idle_pressure_test_scope(); - clear_relay_idle_pressure_state_for_testing(); + let shared = ProxySharedState::new(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); let stats = Stats::new(); - assert!(mark_relay_idle_candidate(301)); - assert!(mark_relay_idle_candidate(302)); - assert!(mark_relay_idle_candidate(303)); + assert!(mark_relay_idle_candidate_for_testing(shared.as_ref(), 301)); + assert!(mark_relay_idle_candidate_for_testing(shared.as_ref(), 302)); + assert!(mark_relay_idle_candidate_for_testing(shared.as_ref(), 303)); let mut seen_301 = 0u64; let mut seen_302 = 0u64; let mut seen_303 = 0u64; // Single pressure event should authorize at most one eviction globally. - note_relay_pressure_event(); + note_relay_pressure_event_for_testing(shared.as_ref()); - let evicted_301 = maybe_evict_idle_candidate_on_pressure(301, &mut seen_301, &stats); - let evicted_302 = maybe_evict_idle_candidate_on_pressure(302, &mut seen_302, &stats); - let evicted_303 = maybe_evict_idle_candidate_on_pressure(303, &mut seen_303, &stats); + let evicted_301 = maybe_evict_idle_candidate_on_pressure_for_testing( + shared.as_ref(), + 301, + &mut seen_301, + &stats, + ); + let evicted_302 = maybe_evict_idle_candidate_on_pressure_for_testing( + shared.as_ref(), + 302, + &mut seen_302, + &stats, + ); + let evicted_303 = maybe_evict_idle_candidate_on_pressure_for_testing( + shared.as_ref(), + 303, + &mut seen_303, + &stats, + ); let evicted_total = [evicted_301, evicted_302, evicted_303] .iter() @@ -474,30 +522,40 @@ fn blackhat_single_pressure_event_must_not_evict_more_than_one_candidate() { "single pressure event must not cascade-evict multiple idle candidates" ); - clear_relay_idle_pressure_state_for_testing(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); } #[test] fn blackhat_pressure_counter_must_track_global_budget_not_per_session_cursor() { - let _guard = relay_idle_pressure_test_scope(); - clear_relay_idle_pressure_state_for_testing(); + let shared = ProxySharedState::new(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); let stats = Stats::new(); - assert!(mark_relay_idle_candidate(401)); - assert!(mark_relay_idle_candidate(402)); + assert!(mark_relay_idle_candidate_for_testing(shared.as_ref(), 401)); + assert!(mark_relay_idle_candidate_for_testing(shared.as_ref(), 402)); let mut seen_oldest = 0u64; let mut seen_next = 0u64; - note_relay_pressure_event(); + note_relay_pressure_event_for_testing(shared.as_ref()); assert!( - maybe_evict_idle_candidate_on_pressure(401, &mut seen_oldest, &stats), + maybe_evict_idle_candidate_on_pressure_for_testing( + shared.as_ref(), + 401, + &mut seen_oldest, + &stats + ), "oldest candidate must consume pressure budget first" ); assert!( - !maybe_evict_idle_candidate_on_pressure(402, &mut seen_next, &stats), + !maybe_evict_idle_candidate_on_pressure_for_testing( + shared.as_ref(), + 402, + &mut seen_next, + &stats + ), "next candidate must not consume the same pressure budget" ); @@ -507,47 +565,67 @@ fn blackhat_pressure_counter_must_track_global_budget_not_per_session_cursor() { "single pressure budget must produce exactly one eviction" ); - clear_relay_idle_pressure_state_for_testing(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); } #[test] fn blackhat_stale_pressure_before_idle_mark_must_not_trigger_eviction() { - let _guard = relay_idle_pressure_test_scope(); - clear_relay_idle_pressure_state_for_testing(); + let shared = ProxySharedState::new(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); let stats = Stats::new(); // Pressure happened before any idle candidate existed. - note_relay_pressure_event(); - assert!(mark_relay_idle_candidate(501)); + note_relay_pressure_event_for_testing(shared.as_ref()); + assert!(mark_relay_idle_candidate_for_testing(shared.as_ref(), 501)); let mut seen = 0u64; assert!( - !maybe_evict_idle_candidate_on_pressure(501, &mut seen, &stats), + !maybe_evict_idle_candidate_on_pressure_for_testing( + shared.as_ref(), + 501, + &mut seen, + &stats + ), "stale pressure (before soft-idle mark) must not evict newly marked candidate" ); - clear_relay_idle_pressure_state_for_testing(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); } #[test] fn blackhat_stale_pressure_must_not_evict_any_of_newly_marked_batch() { - let _guard = relay_idle_pressure_test_scope(); - clear_relay_idle_pressure_state_for_testing(); + let shared = ProxySharedState::new(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); let stats = Stats::new(); - note_relay_pressure_event(); - assert!(mark_relay_idle_candidate(511)); - assert!(mark_relay_idle_candidate(512)); - assert!(mark_relay_idle_candidate(513)); + note_relay_pressure_event_for_testing(shared.as_ref()); + assert!(mark_relay_idle_candidate_for_testing(shared.as_ref(), 511)); + assert!(mark_relay_idle_candidate_for_testing(shared.as_ref(), 512)); + assert!(mark_relay_idle_candidate_for_testing(shared.as_ref(), 513)); let mut seen_511 = 0u64; let mut seen_512 = 0u64; let mut seen_513 = 0u64; let evicted = [ - maybe_evict_idle_candidate_on_pressure(511, &mut seen_511, &stats), - maybe_evict_idle_candidate_on_pressure(512, &mut seen_512, &stats), - maybe_evict_idle_candidate_on_pressure(513, &mut seen_513, &stats), + maybe_evict_idle_candidate_on_pressure_for_testing( + shared.as_ref(), + 511, + &mut seen_511, + &stats, + ), + maybe_evict_idle_candidate_on_pressure_for_testing( + shared.as_ref(), + 512, + &mut seen_512, + &stats, + ), + maybe_evict_idle_candidate_on_pressure_for_testing( + shared.as_ref(), + 513, + &mut seen_513, + &stats, + ), ] .iter() .filter(|value| **value) @@ -558,111 +636,118 @@ fn blackhat_stale_pressure_must_not_evict_any_of_newly_marked_batch() { "stale pressure event must not evict any candidate from a newly marked batch" ); - clear_relay_idle_pressure_state_for_testing(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); } #[test] fn blackhat_stale_pressure_seen_without_candidates_must_be_globally_invalidated() { - let _guard = relay_idle_pressure_test_scope(); - clear_relay_idle_pressure_state_for_testing(); + let shared = ProxySharedState::new(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); let stats = Stats::new(); - note_relay_pressure_event(); + note_relay_pressure_event_for_testing(shared.as_ref()); // Session A observed pressure while there were no candidates. let mut seen_a = 0u64; assert!( - !maybe_evict_idle_candidate_on_pressure(999_001, &mut seen_a, &stats), + !maybe_evict_idle_candidate_on_pressure_for_testing( + shared.as_ref(), + 999_001, + &mut seen_a, + &stats + ), "no candidate existed, so no eviction is possible" ); // Candidate appears later; Session B must not be able to consume stale pressure. - assert!(mark_relay_idle_candidate(521)); + assert!(mark_relay_idle_candidate_for_testing(shared.as_ref(), 521)); let mut seen_b = 0u64; assert!( - !maybe_evict_idle_candidate_on_pressure(521, &mut seen_b, &stats), + !maybe_evict_idle_candidate_on_pressure_for_testing( + shared.as_ref(), + 521, + &mut seen_b, + &stats + ), "once pressure is observed with empty candidate set, it must not be replayed later" ); - clear_relay_idle_pressure_state_for_testing(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); } #[test] fn blackhat_stale_pressure_must_not_survive_candidate_churn() { - let _guard = relay_idle_pressure_test_scope(); - clear_relay_idle_pressure_state_for_testing(); + let shared = ProxySharedState::new(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); let stats = Stats::new(); - note_relay_pressure_event(); - assert!(mark_relay_idle_candidate(531)); - clear_relay_idle_candidate(531); - assert!(mark_relay_idle_candidate(532)); + note_relay_pressure_event_for_testing(shared.as_ref()); + assert!(mark_relay_idle_candidate_for_testing(shared.as_ref(), 531)); + clear_relay_idle_candidate_for_testing(shared.as_ref(), 531); + assert!(mark_relay_idle_candidate_for_testing(shared.as_ref(), 532)); let mut seen = 0u64; assert!( - !maybe_evict_idle_candidate_on_pressure(532, &mut seen, &stats), + !maybe_evict_idle_candidate_on_pressure_for_testing( + shared.as_ref(), + 532, + &mut seen, + &stats + ), "stale pressure must not survive clear+remark churn cycles" ); - clear_relay_idle_pressure_state_for_testing(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); } #[test] fn blackhat_pressure_seq_saturation_must_not_disable_future_pressure_accounting() { - let _guard = relay_idle_pressure_test_scope(); - clear_relay_idle_pressure_state_for_testing(); + let shared = ProxySharedState::new(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); { - let mut guard = relay_idle_candidate_registry() - .lock() - .expect("registry lock must be available"); - guard.pressure_event_seq = u64::MAX; - guard.pressure_consumed_seq = u64::MAX - 1; + set_relay_pressure_state_for_testing(shared.as_ref(), u64::MAX, u64::MAX - 1); } // A new pressure event should still be representable; saturating at MAX creates a permanent lockout. - note_relay_pressure_event(); - let after = relay_pressure_event_seq(); + note_relay_pressure_event_for_testing(shared.as_ref()); + let after = relay_pressure_event_seq_for_testing(shared.as_ref()); assert_ne!( after, u64::MAX, "pressure sequence saturation must not permanently freeze event progression" ); - clear_relay_idle_pressure_state_for_testing(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); } #[test] fn blackhat_pressure_seq_saturation_must_not_break_multiple_distinct_events() { - let _guard = relay_idle_pressure_test_scope(); - clear_relay_idle_pressure_state_for_testing(); + let shared = ProxySharedState::new(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); { - let mut guard = relay_idle_candidate_registry() - .lock() - .expect("registry lock must be available"); - guard.pressure_event_seq = u64::MAX; - guard.pressure_consumed_seq = u64::MAX; + set_relay_pressure_state_for_testing(shared.as_ref(), u64::MAX, u64::MAX); } - note_relay_pressure_event(); - let first = relay_pressure_event_seq(); - note_relay_pressure_event(); - let second = relay_pressure_event_seq(); + note_relay_pressure_event_for_testing(shared.as_ref()); + let first = relay_pressure_event_seq_for_testing(shared.as_ref()); + note_relay_pressure_event_for_testing(shared.as_ref()); + let second = relay_pressure_event_seq_for_testing(shared.as_ref()); assert!( second > first, "distinct pressure events must remain distinguishable even at sequence boundary" ); - clear_relay_idle_pressure_state_for_testing(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); } #[tokio::test(flavor = "multi_thread", worker_threads = 4)] async fn integration_race_single_pressure_event_allows_at_most_one_eviction_under_parallel_claims() { - let _guard = relay_idle_pressure_test_scope(); - clear_relay_idle_pressure_state_for_testing(); + let shared = ProxySharedState::new(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); let stats = Arc::new(Stats::new()); let sessions = 16usize; @@ -671,20 +756,28 @@ async fn integration_race_single_pressure_event_allows_at_most_one_eviction_unde let mut seen_per_session = vec![0u64; sessions]; for conn_id in &conn_ids { - assert!(mark_relay_idle_candidate(*conn_id)); + assert!(mark_relay_idle_candidate_for_testing( + shared.as_ref(), + *conn_id + )); } for round in 0..rounds { - note_relay_pressure_event(); + note_relay_pressure_event_for_testing(shared.as_ref()); let mut joins = Vec::with_capacity(sessions); for (idx, conn_id) in conn_ids.iter().enumerate() { let mut seen = seen_per_session[idx]; let conn_id = *conn_id; let stats = stats.clone(); + let shared = shared.clone(); joins.push(tokio::spawn(async move { - let evicted = - maybe_evict_idle_candidate_on_pressure(conn_id, &mut seen, stats.as_ref()); + let evicted = maybe_evict_idle_candidate_on_pressure_for_testing( + shared.as_ref(), + conn_id, + &mut seen, + stats.as_ref(), + ); (idx, conn_id, seen, evicted) })); } @@ -706,7 +799,7 @@ async fn integration_race_single_pressure_event_allows_at_most_one_eviction_unde ); if let Some(conn) = evicted_conn { assert!( - mark_relay_idle_candidate(conn), + mark_relay_idle_candidate_for_testing(shared.as_ref(), conn), "round {round}: evicted conn must be re-markable as idle candidate" ); } @@ -721,13 +814,13 @@ async fn integration_race_single_pressure_event_allows_at_most_one_eviction_unde "parallel race must still observe at least one successful eviction" ); - clear_relay_idle_pressure_state_for_testing(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); } #[tokio::test(flavor = "multi_thread", worker_threads = 4)] async fn integration_race_burst_pressure_with_churn_preserves_empty_set_invalidation_and_budget() { - let _guard = relay_idle_pressure_test_scope(); - clear_relay_idle_pressure_state_for_testing(); + let shared = ProxySharedState::new(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); let stats = Arc::new(Stats::new()); let sessions = 12usize; @@ -736,7 +829,10 @@ async fn integration_race_burst_pressure_with_churn_preserves_empty_set_invalida let mut seen_per_session = vec![0u64; sessions]; for conn_id in &conn_ids { - assert!(mark_relay_idle_candidate(*conn_id)); + assert!(mark_relay_idle_candidate_for_testing( + shared.as_ref(), + *conn_id + )); } let mut expected_total_evictions = 0u64; @@ -745,20 +841,25 @@ async fn integration_race_burst_pressure_with_churn_preserves_empty_set_invalida let empty_phase = round % 5 == 0; if empty_phase { for conn_id in &conn_ids { - clear_relay_idle_candidate(*conn_id); + clear_relay_idle_candidate_for_testing(shared.as_ref(), *conn_id); } } - note_relay_pressure_event(); + note_relay_pressure_event_for_testing(shared.as_ref()); let mut joins = Vec::with_capacity(sessions); for (idx, conn_id) in conn_ids.iter().enumerate() { let mut seen = seen_per_session[idx]; let conn_id = *conn_id; let stats = stats.clone(); + let shared = shared.clone(); joins.push(tokio::spawn(async move { - let evicted = - maybe_evict_idle_candidate_on_pressure(conn_id, &mut seen, stats.as_ref()); + let evicted = maybe_evict_idle_candidate_on_pressure_for_testing( + shared.as_ref(), + conn_id, + &mut seen, + stats.as_ref(), + ); (idx, conn_id, seen, evicted) })); } @@ -780,7 +881,10 @@ async fn integration_race_burst_pressure_with_churn_preserves_empty_set_invalida "round {round}: empty candidate phase must not allow stale-pressure eviction" ); for conn_id in &conn_ids { - assert!(mark_relay_idle_candidate(*conn_id)); + assert!(mark_relay_idle_candidate_for_testing( + shared.as_ref(), + *conn_id + )); } } else { assert!( @@ -789,7 +893,10 @@ async fn integration_race_burst_pressure_with_churn_preserves_empty_set_invalida ); if let Some(conn_id) = evicted_conn { expected_total_evictions = expected_total_evictions.saturating_add(1); - assert!(mark_relay_idle_candidate(conn_id)); + assert!(mark_relay_idle_candidate_for_testing( + shared.as_ref(), + conn_id + )); } } } @@ -800,5 +907,5 @@ async fn integration_race_burst_pressure_with_churn_preserves_empty_set_invalida "global pressure eviction counter must match observed per-round successful consumes" ); - clear_relay_idle_pressure_state_for_testing(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); } diff --git a/src/proxy/tests/middle_relay_idle_registry_poison_security_tests.rs b/src/proxy/tests/middle_relay_idle_registry_poison_security_tests.rs index b43825c..4f57f56 100644 --- a/src/proxy/tests/middle_relay_idle_registry_poison_security_tests.rs +++ b/src/proxy/tests/middle_relay_idle_registry_poison_security_tests.rs @@ -3,12 +3,13 @@ use std::panic::{AssertUnwindSafe, catch_unwind}; #[test] fn blackhat_registry_poison_recovers_with_fail_closed_reset_and_pressure_accounting() { - let _guard = relay_idle_pressure_test_scope(); - clear_relay_idle_pressure_state_for_testing(); + let shared = ProxySharedState::new(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); let _ = catch_unwind(AssertUnwindSafe(|| { - let registry = relay_idle_candidate_registry(); - let mut guard = registry + let mut guard = shared + .middle_relay + .relay_idle_registry .lock() .expect("registry lock must be acquired before poison"); guard.by_conn_id.insert( @@ -23,40 +24,50 @@ fn blackhat_registry_poison_recovers_with_fail_closed_reset_and_pressure_account })); // Helper lock must recover from poison, reset stale state, and continue. - assert!(mark_relay_idle_candidate(42)); - assert_eq!(oldest_relay_idle_candidate(), Some(42)); + assert!(mark_relay_idle_candidate_for_testing(shared.as_ref(), 42)); + assert_eq!( + oldest_relay_idle_candidate_for_testing(shared.as_ref()), + Some(42) + ); - let before = relay_pressure_event_seq(); - note_relay_pressure_event(); - let after = relay_pressure_event_seq(); + let before = relay_pressure_event_seq_for_testing(shared.as_ref()); + note_relay_pressure_event_for_testing(shared.as_ref()); + let after = relay_pressure_event_seq_for_testing(shared.as_ref()); assert!( after > before, "pressure accounting must still advance after poison" ); - clear_relay_idle_pressure_state_for_testing(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); } #[test] fn clear_state_helper_must_reset_poisoned_registry_for_deterministic_fifo_tests() { - let _guard = relay_idle_pressure_test_scope(); - clear_relay_idle_pressure_state_for_testing(); + let shared = ProxySharedState::new(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); let _ = catch_unwind(AssertUnwindSafe(|| { - let registry = relay_idle_candidate_registry(); - let _guard = registry + let _guard = shared + .middle_relay + .relay_idle_registry .lock() .expect("registry lock must be acquired before poison"); panic!("intentional poison while lock held"); })); - clear_relay_idle_pressure_state_for_testing(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); - assert_eq!(oldest_relay_idle_candidate(), None); - assert_eq!(relay_pressure_event_seq(), 0); + assert_eq!( + oldest_relay_idle_candidate_for_testing(shared.as_ref()), + None + ); + assert_eq!(relay_pressure_event_seq_for_testing(shared.as_ref()), 0); - assert!(mark_relay_idle_candidate(7)); - assert_eq!(oldest_relay_idle_candidate(), Some(7)); + assert!(mark_relay_idle_candidate_for_testing(shared.as_ref(), 7)); + assert_eq!( + oldest_relay_idle_candidate_for_testing(shared.as_ref()), + Some(7) + ); - clear_relay_idle_pressure_state_for_testing(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); } diff --git a/src/proxy/tests/middle_relay_stub_completion_security_tests.rs b/src/proxy/tests/middle_relay_stub_completion_security_tests.rs index fbb9081..54eb784 100644 --- a/src/proxy/tests/middle_relay_stub_completion_security_tests.rs +++ b/src/proxy/tests/middle_relay_stub_completion_security_tests.rs @@ -1,6 +1,6 @@ use super::*; +use crate::stats::Stats; use crate::stream::BufferPool; -use std::collections::HashSet; use std::sync::Arc; use tokio::time::{Duration as TokioDuration, timeout}; @@ -15,32 +15,30 @@ fn make_pooled_payload(data: &[u8]) -> PooledBuffer { #[test] #[ignore = "Tracking for M-04: Verify should_emit_full_desync returns true on first occurrence and false on duplicate within window"] fn should_emit_full_desync_filters_duplicates() { - let _guard = desync_dedup_test_lock() - .lock() - .expect("desync dedup test lock must be available"); - clear_desync_dedup_for_testing(); + let shared = ProxySharedState::new(); + clear_desync_dedup_for_testing_in_shared(shared.as_ref()); let key = 0x4D04_0000_0000_0001_u64; let base = Instant::now(); assert!( - should_emit_full_desync(key, false, base), + should_emit_full_desync_for_testing(shared.as_ref(), key, false, base), "first occurrence must emit full forensic record" ); assert!( - !should_emit_full_desync(key, false, base), + !should_emit_full_desync_for_testing(shared.as_ref(), key, false, base), "duplicate at same timestamp must be suppressed" ); let within_window = base + DESYNC_DEDUP_WINDOW - TokioDuration::from_millis(1); assert!( - !should_emit_full_desync(key, false, within_window), + !should_emit_full_desync_for_testing(shared.as_ref(), key, false, within_window), "duplicate strictly inside dedup window must stay suppressed" ); let on_window_edge = base + DESYNC_DEDUP_WINDOW; assert!( - should_emit_full_desync(key, false, on_window_edge), + should_emit_full_desync_for_testing(shared.as_ref(), key, false, on_window_edge), "duplicate at window boundary must re-emit and refresh" ); } @@ -48,39 +46,34 @@ fn should_emit_full_desync_filters_duplicates() { #[test] #[ignore = "Tracking for M-04: Verify desync dedup eviction behaves correctly under map-full condition"] fn desync_dedup_eviction_under_map_full_condition() { - let _guard = desync_dedup_test_lock() - .lock() - .expect("desync dedup test lock must be available"); - clear_desync_dedup_for_testing(); + let shared = ProxySharedState::new(); + clear_desync_dedup_for_testing_in_shared(shared.as_ref()); let base = Instant::now(); for key in 0..DESYNC_DEDUP_MAX_ENTRIES as u64 { assert!( - should_emit_full_desync(key, false, base), + should_emit_full_desync_for_testing(shared.as_ref(), key, false, base), "unique key should be inserted while warming dedup cache" ); } - let dedup = DESYNC_DEDUP - .get() - .expect("dedup map must exist after warm-up insertions"); assert_eq!( - dedup.len(), + desync_dedup_len_for_testing(shared.as_ref()), DESYNC_DEDUP_MAX_ENTRIES, "cache warm-up must reach exact hard cap" ); - let before_keys: HashSet = dedup.iter().map(|entry| *entry.key()).collect(); + let before_keys = desync_dedup_keys_for_testing(shared.as_ref()); let newcomer_key = 0x4D04_FFFF_FFFF_0001_u64; assert!( - should_emit_full_desync(newcomer_key, false, base), + should_emit_full_desync_for_testing(shared.as_ref(), newcomer_key, false, base), "first newcomer at map-full must emit under bounded full-cache gate" ); - let after_keys: HashSet = dedup.iter().map(|entry| *entry.key()).collect(); + let after_keys = desync_dedup_keys_for_testing(shared.as_ref()); assert_eq!( - dedup.len(), + desync_dedup_len_for_testing(shared.as_ref()), DESYNC_DEDUP_MAX_ENTRIES, "map-full insertion must preserve hard capacity bound" ); @@ -101,7 +94,7 @@ fn desync_dedup_eviction_under_map_full_condition() { ); assert!( - !should_emit_full_desync(newcomer_key, false, base), + !should_emit_full_desync_for_testing(shared.as_ref(), newcomer_key, false, base), "immediate duplicate newcomer must remain suppressed" ); } @@ -119,6 +112,7 @@ async fn c2me_channel_full_path_yields_then_sends() { .expect("priming queue with one frame must succeed"); let tx2 = tx.clone(); + let stats = Stats::default(); let producer = tokio::spawn(async move { enqueue_c2me_command( &tx2, @@ -127,6 +121,7 @@ async fn c2me_channel_full_path_yields_then_sends() { flags: 2, }, None, + &stats, ) .await }); diff --git a/src/proxy/tests/proxy_shared_state_isolation_tests.rs b/src/proxy/tests/proxy_shared_state_isolation_tests.rs new file mode 100644 index 0000000..7887ef8 --- /dev/null +++ b/src/proxy/tests/proxy_shared_state_isolation_tests.rs @@ -0,0 +1,674 @@ +use crate::proxy::client::handle_client_stream_with_shared; +use crate::proxy::handshake::{ + auth_probe_fail_streak_for_testing_in_shared, auth_probe_is_throttled_for_testing_in_shared, + auth_probe_record_failure_for_testing, clear_auth_probe_state_for_testing_in_shared, + clear_unknown_sni_warn_state_for_testing_in_shared, clear_warned_secrets_for_testing_in_shared, + should_emit_unknown_sni_warn_for_testing_in_shared, warned_secrets_for_testing_in_shared, +}; +use crate::proxy::middle_relay::{ + clear_desync_dedup_for_testing_in_shared, clear_relay_idle_candidate_for_testing, + clear_relay_idle_pressure_state_for_testing_in_shared, mark_relay_idle_candidate_for_testing, + maybe_evict_idle_candidate_on_pressure_for_testing, note_relay_pressure_event_for_testing, + oldest_relay_idle_candidate_for_testing, relay_idle_mark_seq_for_testing, + relay_pressure_event_seq_for_testing, should_emit_full_desync_for_testing, +}; +use crate::proxy::route_mode::{RelayRouteMode, RouteRuntimeController}; +use crate::proxy::shared_state::ProxySharedState; +use crate::{ + config::{ProxyConfig, UpstreamConfig, UpstreamType}, + crypto::SecureRandom, + ip_tracker::UserIpTracker, + stats::{ReplayChecker, Stats, beobachten::BeobachtenStore}, + stream::BufferPool, + transport::UpstreamManager, +}; +use std::net::{IpAddr, Ipv4Addr}; +use std::sync::Arc; +use std::time::{Duration, Instant}; +use tokio::io::{AsyncWriteExt, duplex}; +use tokio::sync::Barrier; + +struct ClientHarness { + config: Arc, + stats: Arc, + upstream_manager: Arc, + replay_checker: Arc, + buffer_pool: Arc, + rng: Arc, + route_runtime: Arc, + ip_tracker: Arc, + beobachten: Arc, +} + +fn new_client_harness() -> ClientHarness { + let mut cfg = ProxyConfig::default(); + cfg.censorship.mask = false; + cfg.general.modes.classic = true; + cfg.general.modes.secure = true; + let config = Arc::new(cfg); + let stats = Arc::new(Stats::new()); + + let upstream_manager = Arc::new(UpstreamManager::new( + vec![UpstreamConfig { + upstream_type: UpstreamType::Direct { + interface: None, + bind_addresses: None, + }, + weight: 1, + enabled: true, + scopes: String::new(), + selected_scope: String::new(), + }], + 1, + 1, + 1, + 10, + 1, + false, + stats.clone(), + )); + + ClientHarness { + config, + stats, + upstream_manager, + replay_checker: Arc::new(ReplayChecker::new(128, Duration::from_secs(60))), + buffer_pool: Arc::new(BufferPool::new()), + rng: Arc::new(SecureRandom::new()), + route_runtime: Arc::new(RouteRuntimeController::new(RelayRouteMode::Direct)), + ip_tracker: Arc::new(UserIpTracker::new()), + beobachten: Arc::new(BeobachtenStore::new()), + } +} + +async fn drive_invalid_mtproto_handshake( + shared: Arc, + peer: std::net::SocketAddr, +) { + let harness = new_client_harness(); + let (server_side, mut client_side) = duplex(4096); + let invalid = [0u8; 64]; + + let task = tokio::spawn(handle_client_stream_with_shared( + server_side, + peer, + harness.config, + harness.stats, + harness.upstream_manager, + harness.replay_checker, + harness.buffer_pool, + harness.rng, + None, + harness.route_runtime, + None, + harness.ip_tracker, + harness.beobachten, + shared, + false, + )); + + client_side + .write_all(&invalid) + .await + .expect("failed to write invalid handshake"); + client_side + .shutdown() + .await + .expect("failed to shutdown client"); + let _ = tokio::time::timeout(Duration::from_secs(3), task) + .await + .expect("client task timed out") + .expect("client task join failed"); +} + +#[test] +fn proxy_shared_state_two_instances_do_not_share_auth_probe_state() { + let a = ProxySharedState::new(); + let b = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(a.as_ref()); + + let ip = IpAddr::V4(Ipv4Addr::new(198, 51, 100, 10)); + auth_probe_record_failure_for_testing(a.as_ref(), ip, Instant::now()); + + assert_eq!( + auth_probe_fail_streak_for_testing_in_shared(a.as_ref(), ip), + Some(1) + ); + assert_eq!( + auth_probe_fail_streak_for_testing_in_shared(b.as_ref(), ip), + None + ); +} + +#[test] +fn proxy_shared_state_two_instances_do_not_share_desync_dedup() { + let a = ProxySharedState::new(); + let b = ProxySharedState::new(); + clear_desync_dedup_for_testing_in_shared(a.as_ref()); + + let now = Instant::now(); + let key = 0xA5A5_u64; + assert!(should_emit_full_desync_for_testing( + a.as_ref(), + key, + false, + now + )); + assert!(should_emit_full_desync_for_testing( + b.as_ref(), + key, + false, + now + )); +} + +#[test] +fn proxy_shared_state_two_instances_do_not_share_idle_registry() { + let a = ProxySharedState::new(); + let b = ProxySharedState::new(); + clear_relay_idle_pressure_state_for_testing_in_shared(a.as_ref()); + + assert!(mark_relay_idle_candidate_for_testing(a.as_ref(), 111)); + assert_eq!( + oldest_relay_idle_candidate_for_testing(a.as_ref()), + Some(111) + ); + assert_eq!(oldest_relay_idle_candidate_for_testing(b.as_ref()), None); +} + +#[test] +fn proxy_shared_state_reset_in_one_instance_does_not_affect_another() { + let a = ProxySharedState::new(); + let b = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(a.as_ref()); + + let ip_a = IpAddr::V4(Ipv4Addr::new(203, 0, 113, 1)); + let ip_b = IpAddr::V4(Ipv4Addr::new(203, 0, 113, 2)); + let now = Instant::now(); + + auth_probe_record_failure_for_testing(a.as_ref(), ip_a, now); + auth_probe_record_failure_for_testing(b.as_ref(), ip_b, now); + clear_auth_probe_state_for_testing_in_shared(a.as_ref()); + + assert_eq!( + auth_probe_fail_streak_for_testing_in_shared(a.as_ref(), ip_a), + None + ); + assert_eq!( + auth_probe_fail_streak_for_testing_in_shared(b.as_ref(), ip_b), + Some(1) + ); +} + +#[test] +fn proxy_shared_state_parallel_auth_probe_updates_stay_per_instance() { + let a = ProxySharedState::new(); + let b = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(a.as_ref()); + + let ip = IpAddr::V4(Ipv4Addr::new(192, 0, 2, 77)); + let now = Instant::now(); + + for _ in 0..5 { + auth_probe_record_failure_for_testing(a.as_ref(), ip, now); + } + for _ in 0..3 { + auth_probe_record_failure_for_testing(b.as_ref(), ip, now + Duration::from_millis(1)); + } + + assert_eq!( + auth_probe_fail_streak_for_testing_in_shared(a.as_ref(), ip), + Some(5) + ); + assert_eq!( + auth_probe_fail_streak_for_testing_in_shared(b.as_ref(), ip), + Some(3) + ); +} + +#[tokio::test] +async fn proxy_shared_state_client_pipeline_records_probe_failures_in_instance_state() { + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); + let peer_ip = IpAddr::V4(Ipv4Addr::new(198, 51, 100, 200)); + let peer = std::net::SocketAddr::new(peer_ip, 54001); + + drive_invalid_mtproto_handshake(shared.clone(), peer).await; + + assert_eq!( + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), peer_ip), + Some(1), + "invalid handshake in client pipeline must update injected shared auth-probe state" + ); +} + +#[tokio::test] +async fn proxy_shared_state_client_pipeline_keeps_auth_probe_isolated_between_instances() { + let shared_a = ProxySharedState::new(); + let shared_b = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared_a.as_ref()); + clear_auth_probe_state_for_testing_in_shared(shared_b.as_ref()); + + let peer_a_ip = IpAddr::V4(Ipv4Addr::new(203, 0, 113, 210)); + let peer_b_ip = IpAddr::V4(Ipv4Addr::new(203, 0, 113, 211)); + + drive_invalid_mtproto_handshake( + shared_a.clone(), + std::net::SocketAddr::new(peer_a_ip, 54110), + ) + .await; + drive_invalid_mtproto_handshake( + shared_b.clone(), + std::net::SocketAddr::new(peer_b_ip, 54111), + ) + .await; + + assert_eq!( + auth_probe_fail_streak_for_testing_in_shared(shared_a.as_ref(), peer_a_ip), + Some(1) + ); + assert_eq!( + auth_probe_fail_streak_for_testing_in_shared(shared_b.as_ref(), peer_b_ip), + Some(1) + ); + assert_eq!( + auth_probe_fail_streak_for_testing_in_shared(shared_a.as_ref(), peer_b_ip), + None + ); + assert_eq!( + auth_probe_fail_streak_for_testing_in_shared(shared_b.as_ref(), peer_a_ip), + None + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn proxy_shared_state_client_pipeline_high_contention_same_ip_stays_lossless_per_instance() { + let shared_a = ProxySharedState::new(); + let shared_b = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared_a.as_ref()); + clear_auth_probe_state_for_testing_in_shared(shared_b.as_ref()); + + let ip = IpAddr::V4(Ipv4Addr::new(198, 51, 100, 250)); + let workers = 48u16; + let barrier = Arc::new(Barrier::new((workers as usize) * 2)); + let mut tasks = Vec::new(); + + for i in 0..workers { + let shared_a = shared_a.clone(); + let barrier_a = barrier.clone(); + let peer_a = std::net::SocketAddr::new(ip, 56000 + i); + tasks.push(tokio::spawn(async move { + barrier_a.wait().await; + drive_invalid_mtproto_handshake(shared_a, peer_a).await; + })); + + let shared_b = shared_b.clone(); + let barrier_b = barrier.clone(); + let peer_b = std::net::SocketAddr::new(ip, 56100 + i); + tasks.push(tokio::spawn(async move { + barrier_b.wait().await; + drive_invalid_mtproto_handshake(shared_b, peer_b).await; + })); + } + + for task in tasks { + task.await.expect("pipeline task join failed"); + } + + let streak_a = auth_probe_fail_streak_for_testing_in_shared(shared_a.as_ref(), ip) + .expect("instance A must track probe failures"); + let streak_b = auth_probe_fail_streak_for_testing_in_shared(shared_b.as_ref(), ip) + .expect("instance B must track probe failures"); + + assert!(streak_a > 0); + assert!(streak_b > 0); + + clear_auth_probe_state_for_testing_in_shared(shared_a.as_ref()); + assert_eq!( + auth_probe_fail_streak_for_testing_in_shared(shared_a.as_ref(), ip), + None, + "clearing one instance must reset only that instance" + ); + assert!( + auth_probe_fail_streak_for_testing_in_shared(shared_b.as_ref(), ip).is_some(), + "clearing one instance must not clear the other instance" + ); +} + +#[test] +fn proxy_shared_state_auth_saturation_does_not_bleed_across_instances() { + let a = ProxySharedState::new(); + let b = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(a.as_ref()); + clear_auth_probe_state_for_testing_in_shared(b.as_ref()); + + let ip = IpAddr::V4(Ipv4Addr::new(203, 0, 113, 77)); + let future_now = Instant::now() + Duration::from_secs(1); + for _ in 0..8 { + auth_probe_record_failure_for_testing(a.as_ref(), ip, future_now); + } + + assert!(auth_probe_is_throttled_for_testing_in_shared( + a.as_ref(), + ip + )); + assert!(!auth_probe_is_throttled_for_testing_in_shared( + b.as_ref(), + ip + )); +} + +#[test] +fn proxy_shared_state_poison_clear_in_one_instance_does_not_affect_other_instance() { + let a = ProxySharedState::new(); + let b = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(a.as_ref()); + clear_auth_probe_state_for_testing_in_shared(b.as_ref()); + + let ip_a = IpAddr::V4(Ipv4Addr::new(203, 0, 113, 31)); + let ip_b = IpAddr::V4(Ipv4Addr::new(203, 0, 113, 32)); + let now = Instant::now(); + + auth_probe_record_failure_for_testing(a.as_ref(), ip_a, now); + auth_probe_record_failure_for_testing(b.as_ref(), ip_b, now); + + let a_for_poison = a.clone(); + let _ = std::thread::spawn(move || { + let _hold = a_for_poison + .handshake + .auth_probe_saturation + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()); + panic!("intentional poison for per-instance isolation regression coverage"); + }) + .join(); + + clear_auth_probe_state_for_testing_in_shared(a.as_ref()); + + assert_eq!( + auth_probe_fail_streak_for_testing_in_shared(a.as_ref(), ip_a), + None + ); + assert_eq!( + auth_probe_fail_streak_for_testing_in_shared(b.as_ref(), ip_b), + Some(1), + "poison recovery and clear in one instance must not touch other instance state" + ); +} + +#[test] +fn proxy_shared_state_unknown_sni_cooldown_does_not_bleed_across_instances() { + let a = ProxySharedState::new(); + let b = ProxySharedState::new(); + clear_unknown_sni_warn_state_for_testing_in_shared(a.as_ref()); + clear_unknown_sni_warn_state_for_testing_in_shared(b.as_ref()); + + let now = Instant::now(); + assert!(should_emit_unknown_sni_warn_for_testing_in_shared( + a.as_ref(), + now + )); + assert!(should_emit_unknown_sni_warn_for_testing_in_shared( + b.as_ref(), + now + )); +} + +#[test] +fn proxy_shared_state_warned_secret_cache_does_not_bleed_across_instances() { + let a = ProxySharedState::new(); + let b = ProxySharedState::new(); + clear_warned_secrets_for_testing_in_shared(a.as_ref()); + clear_warned_secrets_for_testing_in_shared(b.as_ref()); + + let key = ("isolation-user".to_string(), "invalid_hex".to_string()); + { + let warned = warned_secrets_for_testing_in_shared(a.as_ref()); + let mut guard = warned + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()); + guard.insert(key.clone()); + } + + let contains_in_a = { + let warned = warned_secrets_for_testing_in_shared(a.as_ref()); + let guard = warned + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()); + guard.contains(&key) + }; + let contains_in_b = { + let warned = warned_secrets_for_testing_in_shared(b.as_ref()); + let guard = warned + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()); + guard.contains(&key) + }; + + assert!(contains_in_a); + assert!(!contains_in_b); +} + +#[test] +fn proxy_shared_state_idle_mark_seq_is_per_instance() { + let a = ProxySharedState::new(); + let b = ProxySharedState::new(); + clear_relay_idle_pressure_state_for_testing_in_shared(a.as_ref()); + clear_relay_idle_pressure_state_for_testing_in_shared(b.as_ref()); + + assert_eq!(relay_idle_mark_seq_for_testing(a.as_ref()), 0); + assert_eq!(relay_idle_mark_seq_for_testing(b.as_ref()), 0); + + assert!(mark_relay_idle_candidate_for_testing(a.as_ref(), 9001)); + assert_eq!(relay_idle_mark_seq_for_testing(a.as_ref()), 1); + assert_eq!(relay_idle_mark_seq_for_testing(b.as_ref()), 0); + + assert!(mark_relay_idle_candidate_for_testing(b.as_ref(), 9002)); + assert_eq!(relay_idle_mark_seq_for_testing(a.as_ref()), 1); + assert_eq!(relay_idle_mark_seq_for_testing(b.as_ref()), 1); +} + +#[test] +fn proxy_shared_state_unknown_sni_clear_in_one_instance_does_not_reset_other() { + let a = ProxySharedState::new(); + let b = ProxySharedState::new(); + clear_unknown_sni_warn_state_for_testing_in_shared(a.as_ref()); + clear_unknown_sni_warn_state_for_testing_in_shared(b.as_ref()); + + let now = Instant::now(); + assert!(should_emit_unknown_sni_warn_for_testing_in_shared( + a.as_ref(), + now + )); + assert!(should_emit_unknown_sni_warn_for_testing_in_shared( + b.as_ref(), + now + )); + + clear_unknown_sni_warn_state_for_testing_in_shared(a.as_ref()); + assert!(should_emit_unknown_sni_warn_for_testing_in_shared( + a.as_ref(), + now + Duration::from_millis(1) + )); + assert!(!should_emit_unknown_sni_warn_for_testing_in_shared( + b.as_ref(), + now + Duration::from_millis(1) + )); +} + +#[test] +fn proxy_shared_state_warned_secret_clear_in_one_instance_does_not_clear_other() { + let a = ProxySharedState::new(); + let b = ProxySharedState::new(); + clear_warned_secrets_for_testing_in_shared(a.as_ref()); + clear_warned_secrets_for_testing_in_shared(b.as_ref()); + + let key = ( + "clear-isolation-user".to_string(), + "invalid_length".to_string(), + ); + { + let warned_a = warned_secrets_for_testing_in_shared(a.as_ref()); + let mut guard_a = warned_a + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()); + guard_a.insert(key.clone()); + + let warned_b = warned_secrets_for_testing_in_shared(b.as_ref()); + let mut guard_b = warned_b + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()); + guard_b.insert(key.clone()); + } + + clear_warned_secrets_for_testing_in_shared(a.as_ref()); + + let has_a = { + let warned = warned_secrets_for_testing_in_shared(a.as_ref()); + let guard = warned + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()); + guard.contains(&key) + }; + let has_b = { + let warned = warned_secrets_for_testing_in_shared(b.as_ref()); + let guard = warned + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()); + guard.contains(&key) + }; + + assert!(!has_a); + assert!(has_b); +} + +#[test] +fn proxy_shared_state_desync_duplicate_suppression_is_instance_scoped() { + let a = ProxySharedState::new(); + let b = ProxySharedState::new(); + clear_desync_dedup_for_testing_in_shared(a.as_ref()); + clear_desync_dedup_for_testing_in_shared(b.as_ref()); + + let now = Instant::now(); + let key = 0xBEEF_0000_0000_0001u64; + assert!(should_emit_full_desync_for_testing( + a.as_ref(), + key, + false, + now + )); + assert!(!should_emit_full_desync_for_testing( + a.as_ref(), + key, + false, + now + Duration::from_millis(1) + )); + assert!(should_emit_full_desync_for_testing( + b.as_ref(), + key, + false, + now + )); +} + +#[test] +fn proxy_shared_state_desync_clear_in_one_instance_does_not_clear_other() { + let a = ProxySharedState::new(); + let b = ProxySharedState::new(); + clear_desync_dedup_for_testing_in_shared(a.as_ref()); + clear_desync_dedup_for_testing_in_shared(b.as_ref()); + + let now = Instant::now(); + let key = 0xCAFE_0000_0000_0001u64; + assert!(should_emit_full_desync_for_testing( + a.as_ref(), + key, + false, + now + )); + assert!(should_emit_full_desync_for_testing( + b.as_ref(), + key, + false, + now + )); + + clear_desync_dedup_for_testing_in_shared(a.as_ref()); + + assert!(should_emit_full_desync_for_testing( + a.as_ref(), + key, + false, + now + Duration::from_millis(2) + )); + assert!(!should_emit_full_desync_for_testing( + b.as_ref(), + key, + false, + now + Duration::from_millis(2) + )); +} + +#[test] +fn proxy_shared_state_idle_candidate_clear_in_one_instance_does_not_affect_other() { + let a = ProxySharedState::new(); + let b = ProxySharedState::new(); + clear_relay_idle_pressure_state_for_testing_in_shared(a.as_ref()); + clear_relay_idle_pressure_state_for_testing_in_shared(b.as_ref()); + + assert!(mark_relay_idle_candidate_for_testing(a.as_ref(), 1001)); + assert!(mark_relay_idle_candidate_for_testing(b.as_ref(), 2002)); + clear_relay_idle_candidate_for_testing(a.as_ref(), 1001); + + assert_eq!(oldest_relay_idle_candidate_for_testing(a.as_ref()), None); + assert_eq!( + oldest_relay_idle_candidate_for_testing(b.as_ref()), + Some(2002) + ); +} + +#[test] +fn proxy_shared_state_pressure_seq_increments_are_instance_scoped() { + let a = ProxySharedState::new(); + let b = ProxySharedState::new(); + clear_relay_idle_pressure_state_for_testing_in_shared(a.as_ref()); + clear_relay_idle_pressure_state_for_testing_in_shared(b.as_ref()); + + assert_eq!(relay_pressure_event_seq_for_testing(a.as_ref()), 0); + assert_eq!(relay_pressure_event_seq_for_testing(b.as_ref()), 0); + + note_relay_pressure_event_for_testing(a.as_ref()); + note_relay_pressure_event_for_testing(a.as_ref()); + + assert_eq!(relay_pressure_event_seq_for_testing(a.as_ref()), 2); + assert_eq!(relay_pressure_event_seq_for_testing(b.as_ref()), 0); +} + +#[test] +fn proxy_shared_state_pressure_consumption_does_not_cross_instances() { + let a = ProxySharedState::new(); + let b = ProxySharedState::new(); + clear_relay_idle_pressure_state_for_testing_in_shared(a.as_ref()); + clear_relay_idle_pressure_state_for_testing_in_shared(b.as_ref()); + + assert!(mark_relay_idle_candidate_for_testing(a.as_ref(), 7001)); + assert!(mark_relay_idle_candidate_for_testing(b.as_ref(), 7001)); + note_relay_pressure_event_for_testing(a.as_ref()); + + let stats = Stats::new(); + let mut seen_a = 0u64; + let mut seen_b = 0u64; + + assert!(maybe_evict_idle_candidate_on_pressure_for_testing( + a.as_ref(), + 7001, + &mut seen_a, + &stats + )); + assert!(!maybe_evict_idle_candidate_on_pressure_for_testing( + b.as_ref(), + 7001, + &mut seen_b, + &stats + )); +} diff --git a/src/proxy/tests/proxy_shared_state_parallel_execution_tests.rs b/src/proxy/tests/proxy_shared_state_parallel_execution_tests.rs new file mode 100644 index 0000000..1330df4 --- /dev/null +++ b/src/proxy/tests/proxy_shared_state_parallel_execution_tests.rs @@ -0,0 +1,265 @@ +use crate::proxy::handshake::{ + auth_probe_fail_streak_for_testing_in_shared, auth_probe_record_failure_for_testing, + clear_auth_probe_state_for_testing_in_shared, + clear_unknown_sni_warn_state_for_testing_in_shared, + should_emit_unknown_sni_warn_for_testing_in_shared, +}; +use crate::proxy::middle_relay::{ + clear_desync_dedup_for_testing_in_shared, + clear_relay_idle_pressure_state_for_testing_in_shared, mark_relay_idle_candidate_for_testing, + oldest_relay_idle_candidate_for_testing, should_emit_full_desync_for_testing, +}; +use crate::proxy::shared_state::ProxySharedState; +use rand::RngExt; +use rand::SeedableRng; +use rand::rngs::StdRng; +use std::net::{IpAddr, Ipv4Addr}; +use std::sync::Arc; +use std::time::Instant; +use tokio::sync::Barrier; + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn proxy_shared_state_50_concurrent_instances_no_counter_bleed() { + let mut handles = Vec::new(); + for i in 0..50_u8 { + handles.push(tokio::spawn(async move { + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); + let ip = IpAddr::V4(Ipv4Addr::new(198, 51, 100, 200)); + auth_probe_record_failure_for_testing(shared.as_ref(), ip, Instant::now()); + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), ip) + })); + } + + for handle in handles { + let streak = handle.await.expect("task join failed"); + assert_eq!(streak, Some(1)); + } +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn proxy_shared_state_desync_rotation_concurrent_20_instances() { + let now = Instant::now(); + let key = 0xD35E_D35E_u64; + let mut handles = Vec::new(); + for _ in 0..20_u64 { + handles.push(tokio::spawn(async move { + let shared = ProxySharedState::new(); + clear_desync_dedup_for_testing_in_shared(shared.as_ref()); + should_emit_full_desync_for_testing(shared.as_ref(), key, false, now) + })); + } + + for handle in handles { + let emitted = handle.await.expect("task join failed"); + assert!(emitted); + } +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn proxy_shared_state_idle_registry_concurrent_10_instances() { + let mut handles = Vec::new(); + let conn_id = 42_u64; + for _ in 1..=10_u64 { + handles.push(tokio::spawn(async move { + let shared = ProxySharedState::new(); + clear_relay_idle_pressure_state_for_testing_in_shared(shared.as_ref()); + let marked = mark_relay_idle_candidate_for_testing(shared.as_ref(), conn_id); + let oldest = oldest_relay_idle_candidate_for_testing(shared.as_ref()); + (marked, oldest) + })); + } + + for (i, handle) in handles.into_iter().enumerate() { + let (marked, oldest) = handle.await.expect("task join failed"); + assert!(marked, "instance {} failed to mark", i); + assert_eq!(oldest, Some(conn_id)); + } +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn proxy_shared_state_dual_instance_same_ip_high_contention_no_counter_bleed() { + let a = ProxySharedState::new(); + let b = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(a.as_ref()); + clear_auth_probe_state_for_testing_in_shared(b.as_ref()); + + let ip = IpAddr::V4(Ipv4Addr::new(203, 0, 113, 200)); + let mut handles = Vec::new(); + + for _ in 0..64 { + let a = a.clone(); + let b = b.clone(); + handles.push(tokio::spawn(async move { + auth_probe_record_failure_for_testing(a.as_ref(), ip, Instant::now()); + auth_probe_record_failure_for_testing(b.as_ref(), ip, Instant::now()); + })); + } + + for handle in handles { + handle.await.expect("task join failed"); + } + + assert_eq!( + auth_probe_fail_streak_for_testing_in_shared(a.as_ref(), ip), + Some(64) + ); + assert_eq!( + auth_probe_fail_streak_for_testing_in_shared(b.as_ref(), ip), + Some(64) + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn proxy_shared_state_unknown_sni_parallel_instances_no_cross_cooldown() { + let mut handles = Vec::new(); + let now = Instant::now(); + + for _ in 0..32 { + handles.push(tokio::spawn(async move { + let shared = ProxySharedState::new(); + clear_unknown_sni_warn_state_for_testing_in_shared(shared.as_ref()); + let first = should_emit_unknown_sni_warn_for_testing_in_shared(shared.as_ref(), now); + let second = should_emit_unknown_sni_warn_for_testing_in_shared( + shared.as_ref(), + now + std::time::Duration::from_millis(1), + ); + (first, second) + })); + } + + for handle in handles { + let (first, second) = handle.await.expect("task join failed"); + assert!(first); + assert!(!second); + } +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn proxy_shared_state_auth_probe_high_contention_increments_are_lossless() { + let shared = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared.as_ref()); + + let ip = IpAddr::V4(Ipv4Addr::new(198, 51, 100, 33)); + let workers = 128usize; + let rounds = 20usize; + + for _ in 0..rounds { + let start = Arc::new(Barrier::new(workers)); + let mut handles = Vec::with_capacity(workers); + + for _ in 0..workers { + let shared = shared.clone(); + let start = start.clone(); + handles.push(tokio::spawn(async move { + start.wait().await; + auth_probe_record_failure_for_testing(shared.as_ref(), ip, Instant::now()); + })); + } + + for handle in handles { + handle.await.expect("task join failed"); + } + } + + let expected = (workers * rounds) as u32; + assert_eq!( + auth_probe_fail_streak_for_testing_in_shared(shared.as_ref(), ip), + Some(expected), + "auth probe fail streak must account for every concurrent update" + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 4)] +async fn proxy_shared_state_seed_matrix_concurrency_isolation_no_counter_bleed() { + let seeds: [u64; 8] = [ + 0x0000_0000_0000_0001, + 0x1111_1111_1111_1111, + 0xA5A5_A5A5_A5A5_A5A5, + 0xDEAD_BEEF_CAFE_BABE, + 0x0123_4567_89AB_CDEF, + 0xFEDC_BA98_7654_3210, + 0x0F0F_F0F0_55AA_AA55, + 0x1357_9BDF_2468_ACE0, + ]; + + for seed in seeds { + let mut rng = StdRng::seed_from_u64(seed); + let shared_a = ProxySharedState::new(); + let shared_b = ProxySharedState::new(); + clear_auth_probe_state_for_testing_in_shared(shared_a.as_ref()); + clear_auth_probe_state_for_testing_in_shared(shared_b.as_ref()); + + let ip = IpAddr::V4(Ipv4Addr::new(198, 51, 100, rng.random_range(1_u8..=250_u8))); + let workers = rng.random_range(16_usize..=48_usize); + let rounds = rng.random_range(4_usize..=10_usize); + + let mut expected_a: u32 = 0; + let mut expected_b: u32 = 0; + + for _ in 0..rounds { + let start = Arc::new(Barrier::new(workers * 2)); + let mut handles = Vec::with_capacity(workers * 2); + + for _ in 0..workers { + let a_ops = rng.random_range(1_u32..=3_u32); + let b_ops = rng.random_range(1_u32..=3_u32); + expected_a = expected_a.saturating_add(a_ops); + expected_b = expected_b.saturating_add(b_ops); + + let shared_a = shared_a.clone(); + let start_a = start.clone(); + handles.push(tokio::spawn(async move { + start_a.wait().await; + for _ in 0..a_ops { + auth_probe_record_failure_for_testing( + shared_a.as_ref(), + ip, + Instant::now(), + ); + } + })); + + let shared_b = shared_b.clone(); + let start_b = start.clone(); + handles.push(tokio::spawn(async move { + start_b.wait().await; + for _ in 0..b_ops { + auth_probe_record_failure_for_testing( + shared_b.as_ref(), + ip, + Instant::now(), + ); + } + })); + } + + for handle in handles { + handle.await.expect("task join failed"); + } + } + + assert_eq!( + auth_probe_fail_streak_for_testing_in_shared(shared_a.as_ref(), ip), + Some(expected_a), + "seed {seed:#x}: instance A streak mismatch" + ); + assert_eq!( + auth_probe_fail_streak_for_testing_in_shared(shared_b.as_ref(), ip), + Some(expected_b), + "seed {seed:#x}: instance B streak mismatch" + ); + + clear_auth_probe_state_for_testing_in_shared(shared_a.as_ref()); + assert_eq!( + auth_probe_fail_streak_for_testing_in_shared(shared_a.as_ref(), ip), + None, + "seed {seed:#x}: clearing A must reset only A" + ); + assert_eq!( + auth_probe_fail_streak_for_testing_in_shared(shared_b.as_ref(), ip), + Some(expected_b), + "seed {seed:#x}: clearing A must not mutate B" + ); + } +} diff --git a/src/proxy/tests/relay_baseline_invariant_tests.rs b/src/proxy/tests/relay_baseline_invariant_tests.rs new file mode 100644 index 0000000..998be2d --- /dev/null +++ b/src/proxy/tests/relay_baseline_invariant_tests.rs @@ -0,0 +1,284 @@ +use super::*; +use crate::error::ProxyError; +use crate::stats::Stats; +use crate::stream::BufferPool; +use std::io; +use std::sync::Arc; +use tokio::io::{AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt, ReadBuf, duplex}; +use tokio::time::{Duration, timeout}; + +struct BrokenPipeWriter; + +impl AsyncWrite for BrokenPipeWriter { + fn poll_write( + self: Pin<&mut Self>, + _cx: &mut Context<'_>, + _buf: &[u8], + ) -> Poll> { + Poll::Ready(Err(io::Error::new( + io::ErrorKind::BrokenPipe, + "forced broken pipe", + ))) + } + + fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> { + Poll::Ready(Ok(())) + } + + fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> { + Poll::Ready(Ok(())) + } +} + +#[tokio::test(start_paused = true)] +async fn relay_baseline_activity_timeout_fires_after_inactivity() { + let stats = Arc::new(Stats::new()); + let user = "relay-baseline-idle-timeout"; + + let (_client_peer, relay_client) = duplex(1024); + let (_server_peer, relay_server) = duplex(1024); + + let (client_reader, client_writer) = tokio::io::split(relay_client); + let (server_reader, server_writer) = tokio::io::split(relay_server); + + let relay_task = tokio::spawn(relay_bidirectional( + client_reader, + client_writer, + server_reader, + server_writer, + 1024, + 1024, + user, + Arc::clone(&stats), + None, + Arc::new(BufferPool::new()), + )); + + tokio::task::yield_now().await; + tokio::time::advance(ACTIVITY_TIMEOUT.saturating_sub(Duration::from_secs(1))).await; + tokio::task::yield_now().await; + assert!( + !relay_task.is_finished(), + "relay must stay alive before inactivity timeout" + ); + + tokio::time::advance(WATCHDOG_INTERVAL + Duration::from_secs(2)).await; + + let done = timeout(Duration::from_secs(1), relay_task) + .await + .expect("relay must complete after inactivity timeout") + .expect("relay task must not panic"); + + assert!( + done.is_ok(), + "relay must return Ok(()) after inactivity timeout" + ); +} + +#[tokio::test] +async fn relay_baseline_zero_bytes_returns_ok_and_counters_zero() { + let stats = Arc::new(Stats::new()); + let user = "relay-baseline-zero-bytes"; + + let (client_peer, relay_client) = duplex(1024); + let (server_peer, relay_server) = duplex(1024); + + let (client_reader, client_writer) = tokio::io::split(relay_client); + let (server_reader, server_writer) = tokio::io::split(relay_server); + + let relay_task = tokio::spawn(relay_bidirectional( + client_reader, + client_writer, + server_reader, + server_writer, + 1024, + 1024, + user, + Arc::clone(&stats), + None, + Arc::new(BufferPool::new()), + )); + + drop(client_peer); + drop(server_peer); + + let done = timeout(Duration::from_secs(2), relay_task) + .await + .expect("relay must stop after both peers close") + .expect("relay task must not panic"); + + assert!(done.is_ok(), "relay must return Ok(()) on immediate EOF"); + assert_eq!(stats.get_user_total_octets(user), 0); +} + +#[tokio::test] +async fn relay_baseline_bidirectional_bytes_counted_symmetrically() { + let stats = Arc::new(Stats::new()); + let user = "relay-baseline-bidir-counters"; + + let (mut client_peer, relay_client) = duplex(16 * 1024); + let (relay_server, mut server_peer) = duplex(16 * 1024); + + let (client_reader, client_writer) = tokio::io::split(relay_client); + let (server_reader, server_writer) = tokio::io::split(relay_server); + + let relay_task = tokio::spawn(relay_bidirectional( + client_reader, + client_writer, + server_reader, + server_writer, + 4096, + 4096, + user, + Arc::clone(&stats), + None, + Arc::new(BufferPool::new()), + )); + + let c2s = vec![0xAA; 4096]; + let s2c = vec![0xBB; 2048]; + + client_peer.write_all(&c2s).await.unwrap(); + server_peer.write_all(&s2c).await.unwrap(); + + let mut seen_c2s = vec![0u8; c2s.len()]; + let mut seen_s2c = vec![0u8; s2c.len()]; + server_peer.read_exact(&mut seen_c2s).await.unwrap(); + client_peer.read_exact(&mut seen_s2c).await.unwrap(); + + assert_eq!(seen_c2s, c2s); + assert_eq!(seen_s2c, s2c); + + drop(client_peer); + drop(server_peer); + + let done = timeout(Duration::from_secs(2), relay_task) + .await + .expect("relay must complete after both peers close") + .expect("relay task must not panic"); + assert!(done.is_ok()); + + assert_eq!( + stats.get_user_total_octets(user), + (c2s.len() + s2c.len()) as u64 + ); +} + +#[tokio::test] +async fn relay_baseline_both_sides_close_simultaneously_no_panic() { + let stats = Arc::new(Stats::new()); + + let (client_peer, relay_client) = duplex(1024); + let (relay_server, server_peer) = duplex(1024); + + let (client_reader, client_writer) = tokio::io::split(relay_client); + let (server_reader, server_writer) = tokio::io::split(relay_server); + + let relay_task = tokio::spawn(relay_bidirectional( + client_reader, + client_writer, + server_reader, + server_writer, + 1024, + 1024, + "relay-baseline-sim-close", + Arc::clone(&stats), + None, + Arc::new(BufferPool::new()), + )); + + drop(client_peer); + drop(server_peer); + + let done = timeout(Duration::from_secs(2), relay_task) + .await + .expect("relay must complete") + .expect("relay task must not panic"); + assert!(done.is_ok()); +} + +#[tokio::test] +async fn relay_baseline_broken_pipe_midtransfer_returns_error() { + let stats = Arc::new(Stats::new()); + let user = "relay-baseline-broken-pipe"; + + let (mut client_peer, relay_client) = duplex(1024); + let (client_reader, client_writer) = tokio::io::split(relay_client); + + let relay_task = tokio::spawn(relay_bidirectional( + client_reader, + client_writer, + tokio::io::empty(), + BrokenPipeWriter, + 1024, + 1024, + user, + Arc::clone(&stats), + None, + Arc::new(BufferPool::new()), + )); + + client_peer.write_all(b"trigger").await.unwrap(); + + let done = timeout(Duration::from_secs(2), relay_task) + .await + .expect("relay must return after broken pipe") + .expect("relay task must not panic"); + + match done { + Err(ProxyError::Io(err)) => { + assert!( + matches!( + err.kind(), + io::ErrorKind::BrokenPipe | io::ErrorKind::ConnectionReset + ), + "expected BrokenPipe/ConnectionReset, got {:?}", + err.kind() + ); + } + other => panic!("expected ProxyError::Io, got {other:?}"), + } +} + +#[tokio::test] +async fn relay_baseline_many_small_writes_exact_counter() { + let stats = Arc::new(Stats::new()); + let user = "relay-baseline-many-small"; + + let (mut client_peer, relay_client) = duplex(4096); + let (relay_server, mut server_peer) = duplex(4096); + + let (client_reader, client_writer) = tokio::io::split(relay_client); + let (server_reader, server_writer) = tokio::io::split(relay_server); + + let relay_task = tokio::spawn(relay_bidirectional( + client_reader, + client_writer, + server_reader, + server_writer, + 1024, + 1024, + user, + Arc::clone(&stats), + None, + Arc::new(BufferPool::new()), + )); + + for i in 0..10_000u32 { + let b = [(i & 0xFF) as u8]; + client_peer.write_all(&b).await.unwrap(); + let mut seen = [0u8; 1]; + server_peer.read_exact(&mut seen).await.unwrap(); + assert_eq!(seen, b); + } + + drop(client_peer); + drop(server_peer); + + let done = timeout(Duration::from_secs(3), relay_task) + .await + .expect("relay must complete for many small writes") + .expect("relay task must not panic"); + assert!(done.is_ok()); + assert_eq!(stats.get_user_total_octets(user), 10_000); +} diff --git a/src/proxy/tests/test_harness_common.rs b/src/proxy/tests/test_harness_common.rs new file mode 100644 index 0000000..4ebb419 --- /dev/null +++ b/src/proxy/tests/test_harness_common.rs @@ -0,0 +1,205 @@ +use crate::config::ProxyConfig; +use rand::SeedableRng; +use rand::rngs::StdRng; +use std::io; +use std::pin::Pin; +use std::sync::Arc; +use std::task::{Context, Poll}; +use tokio::io::AsyncWrite; + +#[cfg(test)] +mod tests { + use super::*; + use std::sync::Arc; + use std::sync::atomic::{AtomicUsize, Ordering}; + use std::task::{RawWaker, RawWakerVTable, Waker}; + + unsafe fn wake_counter_clone(data: *const ()) -> RawWaker { + let arc = Arc::::from_raw(data.cast::()); + let cloned = Arc::clone(&arc); + let _ = Arc::into_raw(arc); + RawWaker::new( + Arc::into_raw(cloned).cast::<()>(), + &WAKE_COUNTER_WAKER_VTABLE, + ) + } + + unsafe fn wake_counter_wake(data: *const ()) { + let arc = Arc::::from_raw(data.cast::()); + arc.fetch_add(1, Ordering::SeqCst); + } + + unsafe fn wake_counter_wake_by_ref(data: *const ()) { + let arc = Arc::::from_raw(data.cast::()); + arc.fetch_add(1, Ordering::SeqCst); + let _ = Arc::into_raw(arc); + } + + unsafe fn wake_counter_drop(data: *const ()) { + let _ = Arc::::from_raw(data.cast::()); + } + + static WAKE_COUNTER_WAKER_VTABLE: RawWakerVTable = RawWakerVTable::new( + wake_counter_clone, + wake_counter_wake, + wake_counter_wake_by_ref, + wake_counter_drop, + ); + + fn wake_counter_waker(counter: Arc) -> Waker { + let raw = RawWaker::new( + Arc::into_raw(counter).cast::<()>(), + &WAKE_COUNTER_WAKER_VTABLE, + ); + // SAFETY: `raw` points to a valid `Arc` and uses a vtable + // that preserves Arc reference-counting semantics. + unsafe { Waker::from_raw(raw) } + } + + #[test] + fn pending_count_writer_write_pending_does_not_spurious_wake() { + let counter = Arc::new(AtomicUsize::new(0)); + let waker = wake_counter_waker(Arc::clone(&counter)); + let mut cx = Context::from_waker(&waker); + + let mut writer = PendingCountWriter::new(RecordingWriter::new(), 1, 0); + let poll = Pin::new(&mut writer).poll_write(&mut cx, b"x"); + + assert!(matches!(poll, Poll::Pending)); + assert_eq!(counter.load(Ordering::SeqCst), 0); + } + + #[test] + fn pending_count_writer_flush_pending_does_not_spurious_wake() { + let counter = Arc::new(AtomicUsize::new(0)); + let waker = wake_counter_waker(Arc::clone(&counter)); + let mut cx = Context::from_waker(&waker); + + let mut writer = PendingCountWriter::new(RecordingWriter::new(), 0, 1); + let poll = Pin::new(&mut writer).poll_flush(&mut cx); + + assert!(matches!(poll, Poll::Pending)); + assert_eq!(counter.load(Ordering::SeqCst), 0); + } +} + +// In-memory AsyncWrite that records both per-write and per-flush granularity. +pub struct RecordingWriter { + pub writes: Vec>, + pub flushed: Vec>, + current_record: Vec, +} + +impl RecordingWriter { + pub fn new() -> Self { + Self { + writes: Vec::new(), + flushed: Vec::new(), + current_record: Vec::new(), + } + } + + pub fn total_bytes(&self) -> usize { + self.writes.iter().map(|w| w.len()).sum() + } +} + +impl Default for RecordingWriter { + fn default() -> Self { + Self::new() + } +} + +impl AsyncWrite for RecordingWriter { + fn poll_write( + mut self: Pin<&mut Self>, + _cx: &mut Context<'_>, + buf: &[u8], + ) -> Poll> { + let me = self.as_mut().get_mut(); + me.writes.push(buf.to_vec()); + me.current_record.extend_from_slice(buf); + Poll::Ready(Ok(buf.len())) + } + + fn poll_flush(mut self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> { + let me = self.as_mut().get_mut(); + let record = std::mem::take(&mut me.current_record); + if !record.is_empty() { + me.flushed.push(record); + } + Poll::Ready(Ok(())) + } + + fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> { + Poll::Ready(Ok(())) + } +} + +// Returns Poll::Pending for the first N write/flush calls, then delegates. +pub struct PendingCountWriter { + pub inner: W, + pub write_pending_remaining: usize, + pub flush_pending_remaining: usize, +} + +impl PendingCountWriter { + pub fn new(inner: W, write_pending: usize, flush_pending: usize) -> Self { + Self { + inner, + write_pending_remaining: write_pending, + flush_pending_remaining: flush_pending, + } + } +} + +impl AsyncWrite for PendingCountWriter { + fn poll_write( + mut self: Pin<&mut Self>, + cx: &mut Context<'_>, + buf: &[u8], + ) -> Poll> { + let me = self.as_mut().get_mut(); + if me.write_pending_remaining > 0 { + me.write_pending_remaining -= 1; + return Poll::Pending; + } + Pin::new(&mut me.inner).poll_write(cx, buf) + } + + fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> { + let me = self.as_mut().get_mut(); + if me.flush_pending_remaining > 0 { + me.flush_pending_remaining -= 1; + return Poll::Pending; + } + Pin::new(&mut me.inner).poll_flush(cx) + } + + fn poll_shutdown(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> { + Pin::new(&mut self.inner).poll_shutdown(cx) + } +} + +pub fn seeded_rng(seed: u64) -> StdRng { + StdRng::seed_from_u64(seed) +} + +pub fn tls_only_config() -> Arc { + let mut cfg = ProxyConfig::default(); + cfg.general.modes.tls = true; + Arc::new(cfg) +} + +pub fn handshake_test_config(secret_hex: &str) -> ProxyConfig { + let mut cfg = ProxyConfig::default(); + cfg.access.users.clear(); + cfg.access + .users + .insert("test-user".to_string(), secret_hex.to_string()); + cfg.access.ignore_time_skew = true; + cfg.censorship.mask = true; + cfg.censorship.mask_host = Some("127.0.0.1".to_string()); + cfg.censorship.mask_port = 0; + cfg +} diff --git a/src/service/mod.rs b/src/service/mod.rs index 7a6e4f6..bb8e38b 100644 --- a/src/service/mod.rs +++ b/src/service/mod.rs @@ -159,8 +159,8 @@ MemoryDenyWriteExecute=true LockPersonality=true # Allow binding to privileged ports and writing to specific paths -AmbientCapabilities=CAP_NET_BIND_SERVICE -CapabilityBoundingSet=CAP_NET_BIND_SERVICE +AmbientCapabilities=CAP_NET_BIND_SERVICE CAP_NET_ADMIN +CapabilityBoundingSet=CAP_NET_BIND_SERVICE CAP_NET_ADMIN ReadWritePaths=/etc/telemt /var/run /var/lib/telemt [Install] diff --git a/src/stats/mod.rs b/src/stats/mod.rs index 2d1f413..38b22bb 100644 --- a/src/stats/mod.rs +++ b/src/stats/mod.rs @@ -91,6 +91,17 @@ pub struct Stats { current_connections_direct: AtomicU64, current_connections_me: AtomicU64, handshake_timeouts: AtomicU64, + accept_permit_timeout_total: AtomicU64, + conntrack_control_enabled_gauge: AtomicBool, + conntrack_control_available_gauge: AtomicBool, + conntrack_pressure_active_gauge: AtomicBool, + conntrack_event_queue_depth_gauge: AtomicU64, + conntrack_rule_apply_ok_gauge: AtomicBool, + conntrack_delete_attempt_total: AtomicU64, + conntrack_delete_success_total: AtomicU64, + conntrack_delete_not_found_total: AtomicU64, + conntrack_delete_error_total: AtomicU64, + conntrack_close_event_drop_total: AtomicU64, upstream_connect_attempt_total: AtomicU64, upstream_connect_success_total: AtomicU64, upstream_connect_fail_total: AtomicU64, @@ -200,6 +211,14 @@ pub struct Stats { me_d2c_flush_duration_us_bucket_1001_5000: AtomicU64, me_d2c_flush_duration_us_bucket_5001_20000: AtomicU64, me_d2c_flush_duration_us_bucket_gt_20000: AtomicU64, + // Buffer pool gauges + buffer_pool_pooled_gauge: AtomicU64, + buffer_pool_allocated_gauge: AtomicU64, + buffer_pool_in_use_gauge: AtomicU64, + // C2ME enqueue observability + me_c2me_send_full_total: AtomicU64, + me_c2me_send_high_water_total: AtomicU64, + me_c2me_send_timeout_total: AtomicU64, me_d2c_batch_timeout_armed_total: AtomicU64, me_d2c_batch_timeout_fired_total: AtomicU64, me_writer_pick_sorted_rr_success_try_total: AtomicU64, @@ -520,6 +539,74 @@ impl Stats { self.handshake_timeouts.fetch_add(1, Ordering::Relaxed); } } + + pub fn increment_accept_permit_timeout_total(&self) { + if self.telemetry_core_enabled() { + self.accept_permit_timeout_total + .fetch_add(1, Ordering::Relaxed); + } + } + + pub fn set_conntrack_control_enabled(&self, enabled: bool) { + self.conntrack_control_enabled_gauge + .store(enabled, Ordering::Relaxed); + } + + pub fn set_conntrack_control_available(&self, available: bool) { + self.conntrack_control_available_gauge + .store(available, Ordering::Relaxed); + } + + pub fn set_conntrack_pressure_active(&self, active: bool) { + self.conntrack_pressure_active_gauge + .store(active, Ordering::Relaxed); + } + + pub fn set_conntrack_event_queue_depth(&self, depth: u64) { + self.conntrack_event_queue_depth_gauge + .store(depth, Ordering::Relaxed); + } + + pub fn set_conntrack_rule_apply_ok(&self, ok: bool) { + self.conntrack_rule_apply_ok_gauge + .store(ok, Ordering::Relaxed); + } + + pub fn increment_conntrack_delete_attempt_total(&self) { + if self.telemetry_core_enabled() { + self.conntrack_delete_attempt_total + .fetch_add(1, Ordering::Relaxed); + } + } + + pub fn increment_conntrack_delete_success_total(&self) { + if self.telemetry_core_enabled() { + self.conntrack_delete_success_total + .fetch_add(1, Ordering::Relaxed); + } + } + + pub fn increment_conntrack_delete_not_found_total(&self) { + if self.telemetry_core_enabled() { + self.conntrack_delete_not_found_total + .fetch_add(1, Ordering::Relaxed); + } + } + + pub fn increment_conntrack_delete_error_total(&self) { + if self.telemetry_core_enabled() { + self.conntrack_delete_error_total + .fetch_add(1, Ordering::Relaxed); + } + } + + pub fn increment_conntrack_close_event_drop_total(&self) { + if self.telemetry_core_enabled() { + self.conntrack_close_event_drop_total + .fetch_add(1, Ordering::Relaxed); + } + } + pub fn increment_upstream_connect_attempt_total(&self) { if self.telemetry_core_enabled() { self.upstream_connect_attempt_total @@ -1414,6 +1501,37 @@ impl Stats { .store(value, Ordering::Relaxed); } } + + pub fn set_buffer_pool_gauges(&self, pooled: usize, allocated: usize, in_use: usize) { + if self.telemetry_me_allows_normal() { + self.buffer_pool_pooled_gauge + .store(pooled as u64, Ordering::Relaxed); + self.buffer_pool_allocated_gauge + .store(allocated as u64, Ordering::Relaxed); + self.buffer_pool_in_use_gauge + .store(in_use as u64, Ordering::Relaxed); + } + } + + pub fn increment_me_c2me_send_full_total(&self) { + if self.telemetry_me_allows_normal() { + self.me_c2me_send_full_total.fetch_add(1, Ordering::Relaxed); + } + } + + pub fn increment_me_c2me_send_high_water_total(&self) { + if self.telemetry_me_allows_normal() { + self.me_c2me_send_high_water_total + .fetch_add(1, Ordering::Relaxed); + } + } + + pub fn increment_me_c2me_send_timeout_total(&self) { + if self.telemetry_me_allows_normal() { + self.me_c2me_send_timeout_total + .fetch_add(1, Ordering::Relaxed); + } + } pub fn increment_me_floor_cap_block_total(&self) { if self.telemetry_me_allows_normal() { self.me_floor_cap_block_total @@ -1438,6 +1556,9 @@ impl Stats { pub fn get_connects_bad(&self) -> u64 { self.connects_bad.load(Ordering::Relaxed) } + pub fn get_accept_permit_timeout_total(&self) -> u64 { + self.accept_permit_timeout_total.load(Ordering::Relaxed) + } pub fn get_current_connections_direct(&self) -> u64 { self.current_connections_direct.load(Ordering::Relaxed) } @@ -1448,6 +1569,40 @@ impl Stats { self.get_current_connections_direct() .saturating_add(self.get_current_connections_me()) } + pub fn get_conntrack_control_enabled(&self) -> bool { + self.conntrack_control_enabled_gauge.load(Ordering::Relaxed) + } + pub fn get_conntrack_control_available(&self) -> bool { + self.conntrack_control_available_gauge + .load(Ordering::Relaxed) + } + pub fn get_conntrack_pressure_active(&self) -> bool { + self.conntrack_pressure_active_gauge.load(Ordering::Relaxed) + } + pub fn get_conntrack_event_queue_depth(&self) -> u64 { + self.conntrack_event_queue_depth_gauge + .load(Ordering::Relaxed) + } + pub fn get_conntrack_rule_apply_ok(&self) -> bool { + self.conntrack_rule_apply_ok_gauge.load(Ordering::Relaxed) + } + pub fn get_conntrack_delete_attempt_total(&self) -> u64 { + self.conntrack_delete_attempt_total.load(Ordering::Relaxed) + } + pub fn get_conntrack_delete_success_total(&self) -> u64 { + self.conntrack_delete_success_total.load(Ordering::Relaxed) + } + pub fn get_conntrack_delete_not_found_total(&self) -> u64 { + self.conntrack_delete_not_found_total + .load(Ordering::Relaxed) + } + pub fn get_conntrack_delete_error_total(&self) -> u64 { + self.conntrack_delete_error_total.load(Ordering::Relaxed) + } + pub fn get_conntrack_close_event_drop_total(&self) -> u64 { + self.conntrack_close_event_drop_total + .load(Ordering::Relaxed) + } pub fn get_me_keepalive_sent(&self) -> u64 { self.me_keepalive_sent.load(Ordering::Relaxed) } @@ -1780,6 +1935,30 @@ impl Stats { self.me_d2c_flush_duration_us_bucket_gt_20000 .load(Ordering::Relaxed) } + + pub fn get_buffer_pool_pooled_gauge(&self) -> u64 { + self.buffer_pool_pooled_gauge.load(Ordering::Relaxed) + } + + pub fn get_buffer_pool_allocated_gauge(&self) -> u64 { + self.buffer_pool_allocated_gauge.load(Ordering::Relaxed) + } + + pub fn get_buffer_pool_in_use_gauge(&self) -> u64 { + self.buffer_pool_in_use_gauge.load(Ordering::Relaxed) + } + + pub fn get_me_c2me_send_full_total(&self) -> u64 { + self.me_c2me_send_full_total.load(Ordering::Relaxed) + } + + pub fn get_me_c2me_send_high_water_total(&self) -> u64 { + self.me_c2me_send_high_water_total.load(Ordering::Relaxed) + } + + pub fn get_me_c2me_send_timeout_total(&self) -> u64 { + self.me_c2me_send_timeout_total.load(Ordering::Relaxed) + } pub fn get_me_d2c_batch_timeout_armed_total(&self) -> u64 { self.me_d2c_batch_timeout_armed_total .load(Ordering::Relaxed) @@ -2171,6 +2350,8 @@ impl ReplayShard { fn cleanup(&mut self, now: Instant, window: Duration) { if window.is_zero() { + self.cache.clear(); + self.queue.clear(); return; } let cutoff = now.checked_sub(window).unwrap_or(now); @@ -2192,13 +2373,22 @@ impl ReplayShard { } fn check(&mut self, key: &[u8], now: Instant, window: Duration) -> bool { + if window.is_zero() { + return false; + } self.cleanup(now, window); // key is &[u8], resolves Q=[u8] via Box<[u8]>: Borrow<[u8]> self.cache.get(key).is_some() } fn add(&mut self, key: &[u8], now: Instant, window: Duration) { + if window.is_zero() { + return; + } self.cleanup(now, window); + if self.cache.peek(key).is_some() { + return; + } let seq = self.next_seq(); let boxed_key: Box<[u8]> = key.into(); @@ -2341,7 +2531,7 @@ impl ReplayChecker { let interval = if self.window.as_secs() > 60 { Duration::from_secs(30) } else { - Duration::from_secs(self.window.as_secs().max(1) / 2) + Duration::from_secs((self.window.as_secs().max(1) / 2).max(1)) }; loop { @@ -2553,6 +2743,20 @@ mod tests { assert!(!checker.check_handshake(b"expire")); } + #[test] + fn test_replay_checker_zero_window_does_not_retain_entries() { + let checker = ReplayChecker::new(100, Duration::ZERO); + + for _ in 0..1_000 { + assert!(!checker.check_handshake(b"no-retain")); + checker.add_handshake(b"no-retain"); + } + + let stats = checker.stats(); + assert_eq!(stats.total_entries, 0); + assert_eq!(stats.total_queue_len, 0); + } + #[test] fn test_replay_checker_stats() { let checker = ReplayChecker::new(100, Duration::from_secs(60)); diff --git a/src/stream/buffer_pool.rs b/src/stream/buffer_pool.rs index 6cdac60..e5a231f 100644 --- a/src/stream/buffer_pool.rs +++ b/src/stream/buffer_pool.rs @@ -35,6 +35,10 @@ pub struct BufferPool { misses: AtomicUsize, /// Number of successful reuses hits: AtomicUsize, + /// Number of non-standard buffers replaced with a fresh default-sized buffer + replaced_nonstandard: AtomicUsize, + /// Number of buffers dropped because the pool queue was full + dropped_pool_full: AtomicUsize, } impl BufferPool { @@ -52,6 +56,8 @@ impl BufferPool { allocated: AtomicUsize::new(0), misses: AtomicUsize::new(0), hits: AtomicUsize::new(0), + replaced_nonstandard: AtomicUsize::new(0), + dropped_pool_full: AtomicUsize::new(0), } } @@ -91,17 +97,36 @@ impl BufferPool { /// Return a buffer to the pool fn return_buffer(&self, mut buffer: BytesMut) { - // Clear the buffer but keep capacity - buffer.clear(); + const MAX_RETAINED_BUFFER_FACTOR: usize = 2; - // Only return if we haven't exceeded max and buffer is right size - if buffer.capacity() >= self.buffer_size { - // Try to push to pool, if full just drop - let _ = self.buffers.push(buffer); + // Clear the buffer but keep capacity. + buffer.clear(); + let max_retained_capacity = self + .buffer_size + .saturating_mul(MAX_RETAINED_BUFFER_FACTOR) + .max(self.buffer_size); + + // Keep only near-default capacities in the pool. Oversized buffers keep + // RSS elevated for hours under churn; replace them with default-sized + // buffers before re-pooling. + if buffer.capacity() < self.buffer_size || buffer.capacity() > max_retained_capacity { + self.replaced_nonstandard.fetch_add(1, Ordering::Relaxed); + buffer = BytesMut::with_capacity(self.buffer_size); } - // If buffer was dropped (pool full), decrement allocated - // Actually we don't decrement here because the buffer might have been - // grown beyond our size - we just let it go + + // Try to return into the queue; if full, drop and update accounting. + if self.buffers.push(buffer).is_err() { + self.dropped_pool_full.fetch_add(1, Ordering::Relaxed); + self.decrement_allocated(); + } + } + + fn decrement_allocated(&self) { + let _ = self + .allocated + .fetch_update(Ordering::Relaxed, Ordering::Relaxed, |current| { + Some(current.saturating_sub(1)) + }); } /// Get pool statistics @@ -113,6 +138,8 @@ impl BufferPool { buffer_size: self.buffer_size, hits: self.hits.load(Ordering::Relaxed), misses: self.misses.load(Ordering::Relaxed), + replaced_nonstandard: self.replaced_nonstandard.load(Ordering::Relaxed), + dropped_pool_full: self.dropped_pool_full.load(Ordering::Relaxed), } } @@ -121,6 +148,41 @@ impl BufferPool { self.buffer_size } + /// Maximum number of buffers the pool will retain. + pub fn max_buffers(&self) -> usize { + self.max_buffers + } + + /// Current number of pooled buffers. + pub fn pooled(&self) -> usize { + self.buffers.len() + } + + /// Total buffers allocated (pooled + checked out). + pub fn allocated(&self) -> usize { + self.allocated.load(Ordering::Relaxed) + } + + /// Best-effort number of buffers currently checked out. + pub fn in_use(&self) -> usize { + self.allocated().saturating_sub(self.pooled()) + } + + /// Trim pooled buffers down to a target count. + pub fn trim_to(&self, target_pooled: usize) { + let target = target_pooled.min(self.max_buffers); + loop { + if self.buffers.len() <= target { + break; + } + if self.buffers.pop().is_some() { + self.decrement_allocated(); + } else { + break; + } + } + } + /// Preallocate buffers to fill the pool pub fn preallocate(&self, count: usize) { let to_alloc = count.min(self.max_buffers); @@ -160,6 +222,10 @@ pub struct PoolStats { pub hits: usize, /// Number of cache misses (new allocation) pub misses: usize, + /// Number of non-standard buffers replaced during return + pub replaced_nonstandard: usize, + /// Number of buffers dropped because the pool queue was full + pub dropped_pool_full: usize, } impl PoolStats { @@ -185,6 +251,7 @@ pub struct PooledBuffer { impl PooledBuffer { /// Take the inner buffer, preventing return to pool pub fn take(mut self) -> BytesMut { + self.pool.decrement_allocated(); self.buffer.take().unwrap() } @@ -364,6 +431,25 @@ mod tests { let stats = pool.stats(); assert_eq!(stats.pooled, 0); + assert_eq!(stats.allocated, 0); + } + + #[test] + fn test_pool_replaces_oversized_buffers() { + let pool = Arc::new(BufferPool::with_config(1024, 10)); + + { + let mut buf = pool.get(); + buf.reserve(8192); + assert!(buf.capacity() > 2048); + } + + let stats = pool.stats(); + assert_eq!(stats.replaced_nonstandard, 1); + assert_eq!(stats.pooled, 1); + + let buf = pool.get(); + assert!(buf.capacity() <= 2048); } #[test] diff --git a/src/tls_front/emulator.rs b/src/tls_front/emulator.rs index 80f2b1b..d6845a2 100644 --- a/src/tls_front/emulator.rs +++ b/src/tls_front/emulator.rs @@ -1,5 +1,6 @@ #![allow(clippy::too_many_arguments)] +use crc32fast::Hasher; use crate::crypto::{SecureRandom, sha256_hmac}; use crate::protocol::constants::{ MAX_TLS_CIPHERTEXT_SIZE, TLS_RECORD_APPLICATION, TLS_RECORD_CHANGE_CIPHER, @@ -98,6 +99,31 @@ fn build_compact_cert_info_payload(cert_info: &ParsedCertificateInfo) -> Option< Some(payload) } +fn hash_compact_cert_info_payload(cert_payload: Vec) -> Option> { + if cert_payload.is_empty() { + return None; + } + + let mut hashed = Vec::with_capacity(cert_payload.len()); + let mut seed_hasher = Hasher::new(); + seed_hasher.update(&cert_payload); + let mut state = seed_hasher.finalize(); + + while hashed.len() < cert_payload.len() { + let mut hasher = Hasher::new(); + hasher.update(&state.to_le_bytes()); + hasher.update(&cert_payload); + state = hasher.finalize(); + + let block = state.to_le_bytes(); + let remaining = cert_payload.len() - hashed.len(); + let copy_len = remaining.min(block.len()); + hashed.extend_from_slice(&block[..copy_len]); + } + + Some(hashed) +} + /// Build a ServerHello + CCS + ApplicationData sequence using cached TLS metadata. pub fn build_emulated_server_hello( secret: &[u8], @@ -190,7 +216,8 @@ pub fn build_emulated_server_hello( let compact_payload = cached .cert_info .as_ref() - .and_then(build_compact_cert_info_payload); + .and_then(build_compact_cert_info_payload) + .and_then(hash_compact_cert_info_payload); let selected_payload: Option<&[u8]> = if use_full_cert_payload { cached .cert_payload @@ -221,7 +248,6 @@ pub fn build_emulated_server_hello( marker.extend_from_slice(proto); marker }); - let mut payload_offset = 0usize; for (idx, size) in sizes.into_iter().enumerate() { let mut rec = Vec::with_capacity(5 + size); rec.push(TLS_RECORD_APPLICATION); @@ -231,11 +257,10 @@ pub fn build_emulated_server_hello( if let Some(payload) = selected_payload { if size > 17 { let body_len = size - 17; - let remaining = payload.len().saturating_sub(payload_offset); + let remaining = payload.len(); let copy_len = remaining.min(body_len); if copy_len > 0 { - rec.extend_from_slice(&payload[payload_offset..payload_offset + copy_len]); - payload_offset += copy_len; + rec.extend_from_slice(&payload[..copy_len]); } if body_len > copy_len { rec.extend_from_slice(&rng.bytes(body_len - copy_len)); @@ -317,7 +342,9 @@ mod tests { CachedTlsData, ParsedServerHello, TlsBehaviorProfile, TlsCertPayload, TlsProfileSource, }; - use super::build_emulated_server_hello; + use super::{ + build_compact_cert_info_payload, build_emulated_server_hello, hash_compact_cert_info_payload, + }; use crate::crypto::SecureRandom; use crate::protocol::constants::{ TLS_RECORD_APPLICATION, TLS_RECORD_CHANGE_CIPHER, TLS_RECORD_HANDSHAKE, @@ -432,7 +459,21 @@ mod tests { ); let payload = first_app_data_payload(&response); - assert!(payload.starts_with(b"CN=example.com")); + let expected_hashed_payload = build_compact_cert_info_payload( + cached + .cert_info + .as_ref() + .expect("test fixture must provide certificate info"), + ) + .and_then(hash_compact_cert_info_payload) + .expect("compact certificate info payload must be present for this test"); + let copied_prefix_len = expected_hashed_payload + .len() + .min(payload.len().saturating_sub(17)); + assert_eq!( + &payload[..copied_prefix_len], + &expected_hashed_payload[..copied_prefix_len] + ); } #[test] diff --git a/src/transport/middle_proxy/registry.rs b/src/transport/middle_proxy/registry.rs index ff4a68b..17fce47 100644 --- a/src/transport/middle_proxy/registry.rs +++ b/src/transport/middle_proxy/registry.rs @@ -55,6 +55,20 @@ struct RoutingTable { map: DashMap>, } +struct WriterTable { + map: DashMap>, +} + +#[derive(Clone)] +struct HotConnBinding { + writer_id: u64, + meta: ConnMeta, +} + +struct HotBindingTable { + map: DashMap, +} + struct BindingState { inner: Mutex, } @@ -83,6 +97,8 @@ impl BindingInner { pub struct ConnRegistry { routing: RoutingTable, + writers: WriterTable, + hot_binding: HotBindingTable, binding: BindingState, next_id: AtomicU64, route_channel_capacity: usize, @@ -105,6 +121,12 @@ impl ConnRegistry { routing: RoutingTable { map: DashMap::new(), }, + writers: WriterTable { + map: DashMap::new(), + }, + hot_binding: HotBindingTable { + map: DashMap::new(), + }, binding: BindingState { inner: Mutex::new(BindingInner::new()), }, @@ -149,16 +171,18 @@ impl ConnRegistry { pub async fn register_writer(&self, writer_id: u64, tx: mpsc::Sender) { let mut binding = self.binding.inner.lock().await; - binding.writers.insert(writer_id, tx); + binding.writers.insert(writer_id, tx.clone()); binding .conns_for_writer .entry(writer_id) .or_insert_with(HashSet::new); + self.writers.map.insert(writer_id, tx); } /// Unregister connection, returning associated writer_id if any. pub async fn unregister(&self, id: u64) -> Option { self.routing.map.remove(&id); + self.hot_binding.map.remove(&id); let mut binding = self.binding.inner.lock().await; binding.meta.remove(&id); if let Some(writer_id) = binding.writer_for_conn.remove(&id) { @@ -325,13 +349,20 @@ impl ConnRegistry { } binding.meta.insert(conn_id, meta.clone()); - binding.last_meta_for_writer.insert(writer_id, meta); + binding.last_meta_for_writer.insert(writer_id, meta.clone()); binding.writer_idle_since_epoch_secs.remove(&writer_id); binding .conns_for_writer .entry(writer_id) .or_insert_with(HashSet::new) .insert(conn_id); + self.hot_binding.map.insert( + conn_id, + HotConnBinding { + writer_id, + meta, + }, + ); true } @@ -392,39 +423,12 @@ impl ConnRegistry { } pub async fn get_writer(&self, conn_id: u64) -> Option { - let mut binding = self.binding.inner.lock().await; - // ROUTING IS THE SOURCE OF TRUTH: - // stale bindings are ignored and lazily cleaned when routing no longer - // contains the connection. if !self.routing.map.contains_key(&conn_id) { - binding.meta.remove(&conn_id); - if let Some(stale_writer_id) = binding.writer_for_conn.remove(&conn_id) - && let Some(conns) = binding.conns_for_writer.get_mut(&stale_writer_id) - { - conns.remove(&conn_id); - if conns.is_empty() { - binding - .writer_idle_since_epoch_secs - .insert(stale_writer_id, Self::now_epoch_secs()); - } - } return None; } - let writer_id = binding.writer_for_conn.get(&conn_id).copied()?; - let Some(writer) = binding.writers.get(&writer_id).cloned() else { - binding.writer_for_conn.remove(&conn_id); - binding.meta.remove(&conn_id); - if let Some(conns) = binding.conns_for_writer.get_mut(&writer_id) { - conns.remove(&conn_id); - if conns.is_empty() { - binding - .writer_idle_since_epoch_secs - .insert(writer_id, Self::now_epoch_secs()); - } - } - return None; - }; + let writer_id = self.hot_binding.map.get(&conn_id).map(|entry| entry.writer_id)?; + let writer = self.writers.map.get(&writer_id).map(|entry| entry.value().clone())?; Some(ConnWriter { writer_id, tx: writer, @@ -439,6 +443,7 @@ impl ConnRegistry { pub async fn writer_lost(&self, writer_id: u64) -> Vec { let mut binding = self.binding.inner.lock().await; binding.writers.remove(&writer_id); + self.writers.map.remove(&writer_id); binding.last_meta_for_writer.remove(&writer_id); binding.writer_idle_since_epoch_secs.remove(&writer_id); let conns = binding @@ -454,6 +459,15 @@ impl ConnRegistry { continue; } binding.writer_for_conn.remove(&conn_id); + let remove_hot = self + .hot_binding + .map + .get(&conn_id) + .map(|hot| hot.writer_id == writer_id) + .unwrap_or(false); + if remove_hot { + self.hot_binding.map.remove(&conn_id); + } if let Some(m) = binding.meta.get(&conn_id) { out.push(BoundConn { conn_id, @@ -466,8 +480,10 @@ impl ConnRegistry { #[allow(dead_code)] pub async fn get_meta(&self, conn_id: u64) -> Option { - let binding = self.binding.inner.lock().await; - binding.meta.get(&conn_id).cloned() + self.hot_binding + .map + .get(&conn_id) + .map(|entry| entry.meta.clone()) } pub async fn is_writer_empty(&self, writer_id: u64) -> bool { @@ -491,6 +507,7 @@ impl ConnRegistry { } binding.writers.remove(&writer_id); + self.writers.map.remove(&writer_id); binding.last_meta_for_writer.remove(&writer_id); binding.writer_idle_since_epoch_secs.remove(&writer_id); binding.conns_for_writer.remove(&writer_id);