Use token-bucket SYN limiter backends

Co-Authored-By: brekotis <93345790+brekotis@users.noreply.github.com>
This commit is contained in:
Alexey
2026-06-12 01:27:03 +03:00
parent 942882f9de
commit 9a3ff726b2
6 changed files with 200 additions and 67 deletions

View File

@@ -2220,6 +2220,9 @@ Note: This section also accepts the legacy alias `[server.admin_api]` (same sche
| [`port`](#port-serverlisteners) | `u16` | `server.port` | `` |
| [`client_mss`](#client_mss-serverlisteners) | `String` | `[server].client_mss` | `` |
| [`synlimit`](#synlimit-serverlisteners) | `false`, `"iptables"`, or `"nftables"` | `false` | `` |
| [`synlimit_seconds`](#synlimit_seconds-serverlisteners) | `u32` | `1` | `` |
| [`synlimit_hitcount`](#synlimit_hitcount-serverlisteners) | `u32` | `1` | `` |
| [`synlimit_burst`](#synlimit_burst-serverlisteners) | `u32` | `3` | `` |
| [`announce`](#announce) | `String` | — | `` |
| [`announce_ip`](#announce_ip) | `IpAddr` | — | `` |
| [`proxy_protocol`](#proxy_protocol) | `bool` | — | `` |
@@ -2257,7 +2260,7 @@ Note: This section also accepts the legacy alias `[server.admin_api]` (same sche
```
## synlimit (server.listeners)
- **Constraints / validation**: `false`, `"iptables"`, or `"nftables"`. Omitted or `false` disables SYN limiting for this listener.
- **Description**: Installs per-listener Linux netfilter SYN limiter rules for the listener port. `"iptables"` uses `iptables`/`ip6tables` filter rules with the `recent` match name `telemt`. `"nftables"` uses nftables dynamic timeout sets and auto-detects whether the host already uses `inet`, `ip`, or `ip6` table families before creating Telemt-owned tables. Rules are reconciled at runtime and removed during graceful Telemt shutdown; `SIGKILL` cannot be cleaned up by the process. Requires CAP_NET_ADMIN and listener restart/rebind for config changes.
- **Description**: Installs per-listener Linux netfilter SYN limiter rules for the listener port. `"iptables"` uses `iptables`/`ip6tables` filter rules with the `hashlimit` match as a per-source token bucket. `"nftables"` uses per-source `meter` rules with `limit rate over` and auto-detects whether the host already uses `inet`, `ip`, or `ip6` table families before creating Telemt-owned tables. The token-bucket rate is `synlimit_hitcount / synlimit_seconds`; `synlimit_burst` controls the burst size. Rules are reconciled at runtime and removed during graceful Telemt shutdown; `SIGKILL` cannot be cleaned up by the process. Requires CAP_NET_ADMIN and listener restart/rebind for config changes.
- **Example**:
```toml
@@ -2271,6 +2274,42 @@ Note: This section also accepts the legacy alias `[server.admin_api]` (same sche
port = 443
synlimit = "nftables"
```
## synlimit_seconds (server.listeners)
- **Constraints / validation**: `u32`, must be `> 0`. Default is `1`.
- **Description**: Token-bucket interval for both SYN limiter backends. The rate is `synlimit_hitcount / synlimit_seconds` and is rendered to native netfilter rate units (`second`, `minute`, `hour`, or `day`).
- **Example**:
```toml
[[server.listeners]]
ip = "0.0.0.0"
port = 443
synlimit = "iptables"
synlimit_seconds = 1
```
## synlimit_hitcount (server.listeners)
- **Constraints / validation**: `u32`, must be `> 0`. Default is `1`.
- **Description**: Token-bucket rate amount for both SYN limiter backends. Together with `synlimit_seconds`, it defines the allowed source-IP SYN rate before excess SYN packets are dropped.
- **Example**:
```toml
[[server.listeners]]
ip = "0.0.0.0"
port = 443
synlimit = "iptables"
synlimit_hitcount = 1
```
## synlimit_burst (server.listeners)
- **Constraints / validation**: `u32`, must be `> 0`. Default is `3`.
- **Description**: Token-bucket burst size for both SYN limiter backends. Higher values allow short connection bursts from the same source IP before the steady-state `synlimit_hitcount / synlimit_seconds` rate is enforced.
- **Example**:
```toml
[[server.listeners]]
ip = "0.0.0.0"
port = 443
synlimit = "iptables"
synlimit_burst = 3
```
## announce
- **Constraints / validation**: `String` (optional). Must not be empty when set.
- **Description**: Public IP/domain announced in proxy links for this listener. Takes precedence over `announce_ip`.

View File

@@ -2226,6 +2226,9 @@
| [`port`](#port-serverlisteners) | `u16` | `server.port` | `` |
| [`client_mss`](#client_mss-serverlisteners) | `String` | `[server].client_mss` | `` |
| [`synlimit`](#synlimit-serverlisteners) | `false`, `"iptables"` или `"nftables"` | `false` | `` |
| [`synlimit_seconds`](#synlimit_seconds-serverlisteners) | `u32` | `1` | `` |
| [`synlimit_hitcount`](#synlimit_hitcount-serverlisteners) | `u32` | `1` | `` |
| [`synlimit_burst`](#synlimit_burst-serverlisteners) | `u32` | `3` | `` |
| [`announce`](#announce) | `String` | — | `` |
| [`announce_ip`](#announce_ip) | `IpAddr` | — | `` |
| [`proxy_protocol`](#proxy_protocol) | `bool` | — | `` |
@@ -2263,7 +2266,7 @@
```
## synlimit (server.listeners)
- **Ограничения / валидация**: `false`, `"iptables"` или `"nftables"`. Если параметр не задан или задан как `false`, SYN limiter для этого listenerа выключен.
- **Описание**: Устанавливает per-listener Linux netfilter SYN limiter rules для порта listenerа. `"iptables"` использует `iptables`/`ip6tables` filter rules с `recent` name `telemt`. `"nftables"` использует nftables dynamic timeout sets и автоматически определяет, какие table families уже используются на хосте (`inet`, `ip`, `ip6`), перед созданием Telemt-owned tables. Rules reconciled at runtime и удаляются при graceful shutdown Telemt; `SIGKILL` процессом не очищается. Требует CAP_NET_ADMIN и restart/rebind listenerа для изменений конфигурации.
- **Описание**: Устанавливает per-listener Linux netfilter SYN limiter rules для порта listenerа. `"iptables"` использует `iptables`/`ip6tables` filter rules с `hashlimit` match как per-source token bucket. `"nftables"` использует per-source `meter` rules с `limit rate over` и автоматически определяет, какие table families уже используются на хосте (`inet`, `ip`, `ip6`), перед созданием Telemt-owned tables. Token-bucket rate равен `synlimit_hitcount / synlimit_seconds`; `synlimit_burst` управляет burst size. Rules reconciled at runtime и удаляются при graceful shutdown Telemt; `SIGKILL` процессом не очищается. Требует CAP_NET_ADMIN и restart/rebind listenerа для изменений конфигурации.
- **Пример**:
```toml
@@ -2277,6 +2280,42 @@
port = 443
synlimit = "nftables"
```
## synlimit_seconds (server.listeners)
- **Ограничения / валидация**: `u32`, должно быть `> 0`. Значение по умолчанию: `1`.
- **Описание**: Token-bucket interval для обоих SYN limiter backends. Rate равен `synlimit_hitcount / synlimit_seconds` и рендерится в native netfilter rate units (`second`, `minute`, `hour` или `day`).
- **Пример**:
```toml
[[server.listeners]]
ip = "0.0.0.0"
port = 443
synlimit = "iptables"
synlimit_seconds = 1
```
## synlimit_hitcount (server.listeners)
- **Ограничения / валидация**: `u32`, должно быть `> 0`. Значение по умолчанию: `1`.
- **Описание**: Token-bucket rate amount для обоих SYN limiter backends. Вместе с `synlimit_seconds` задает разрешенный source-IP SYN rate до того, как excess SYN packets начнут dropаться.
- **Пример**:
```toml
[[server.listeners]]
ip = "0.0.0.0"
port = 443
synlimit = "iptables"
synlimit_hitcount = 1
```
## synlimit_burst (server.listeners)
- **Ограничения / валидация**: `u32`, должно быть `> 0`. Значение по умолчанию: `3`.
- **Описание**: Token-bucket burst size для обоих SYN limiter backends. Более высокие значения разрешают short connection bursts с одного source IP перед применением steady-state rate `synlimit_hitcount / synlimit_seconds`.
- **Пример**:
```toml
[[server.listeners]]
ip = "0.0.0.0"
port = 443
synlimit = "iptables"
synlimit_burst = 3
```
## announce
- **Ограничения / валидация**: `String` (необязательный параметр). Не должен быть пустым, если задан.
- **Описание**: Публичный IP-адрес или домен, объявляемый в proxy-ссылках для данного listenerа. Имеет приоритет над `announce_ip`.

View File

@@ -56,6 +56,7 @@ const DEFAULT_CONNTRACK_PRESSURE_LOW_WATERMARK_PCT: u8 = 70;
const DEFAULT_CONNTRACK_DELETE_BUDGET_PER_SEC: u64 = 4096;
const DEFAULT_SYNLIMIT_SECONDS: u32 = 1;
const DEFAULT_SYNLIMIT_HITCOUNT: u32 = 1;
const DEFAULT_SYNLIMIT_BURST: u32 = 3;
const DEFAULT_UPSTREAM_CONNECT_RETRY_ATTEMPTS: u32 = 2;
const DEFAULT_UPSTREAM_UNHEALTHY_FAIL_THRESHOLD: u32 = 5;
const DEFAULT_UPSTREAM_CONNECT_BUDGET_MS: u64 = 3000;
@@ -253,6 +254,10 @@ pub(crate) fn default_synlimit_hitcount() -> u32 {
DEFAULT_SYNLIMIT_HITCOUNT
}
pub(crate) fn default_synlimit_burst() -> u32 {
DEFAULT_SYNLIMIT_BURST
}
pub(crate) fn default_prefer_4() -> u8 {
4
}

View File

@@ -349,6 +349,7 @@ const LISTENER_CONFIG_KEYS: &[&str] = &[
"synlimit",
"synlimit_seconds",
"synlimit_hitcount",
"synlimit_burst",
"announce",
"announce_ip",
"proxy_protocol",
@@ -1961,6 +1962,11 @@ impl ProxyConfig {
"server.listeners[{idx}].synlimit_hitcount must be > 0"
)));
}
if listener.synlimit_burst == 0 {
return Err(ProxyError::Config(format!(
"server.listeners[{idx}].synlimit_burst must be > 0"
)));
}
}
if config.server.accept_permit_timeout_ms > 60_000 {
@@ -2202,6 +2208,7 @@ impl ProxyConfig {
synlimit: SynLimitMode::default(),
synlimit_seconds: default_synlimit_seconds(),
synlimit_hitcount: default_synlimit_hitcount(),
synlimit_burst: default_synlimit_burst(),
announce: None,
announce_ip: None,
proxy_protocol: None,
@@ -2218,6 +2225,7 @@ impl ProxyConfig {
synlimit: SynLimitMode::default(),
synlimit_seconds: default_synlimit_seconds(),
synlimit_hitcount: default_synlimit_hitcount(),
synlimit_burst: default_synlimit_burst(),
announce: None,
announce_ip: None,
proxy_protocol: None,

View File

@@ -1375,9 +1375,9 @@ pub enum SynLimitMode {
/// Disable SYN limiting for this listener.
#[default]
Off,
/// Use iptables/ip6tables filter rules with the recent match.
/// Use iptables/ip6tables filter rules with the hashlimit match.
Iptables,
/// Use nftables rules with timeout-backed dynamic sets.
/// Use nftables rules with per-source token-bucket meters.
Nftables,
}
@@ -2176,12 +2176,15 @@ pub struct ListenerConfig {
/// Per-listener SYN limiter mode.
#[serde(default)]
pub synlimit: SynLimitMode,
/// Iptables recent-match interval for the per-listener SYN limiter.
/// Token-bucket rate interval for the per-listener SYN limiter.
#[serde(default = "default_synlimit_seconds")]
pub synlimit_seconds: u32,
/// Iptables recent-match hit count for the per-listener SYN limiter.
/// Token-bucket rate amount for the per-listener SYN limiter.
#[serde(default = "default_synlimit_hitcount")]
pub synlimit_hitcount: u32,
/// Token-bucket burst size for the per-listener SYN limiter.
#[serde(default = "default_synlimit_burst")]
pub synlimit_burst: u32,
/// IP address or hostname to announce in proxy links.
/// Takes precedence over `announce_ip` if both are set.
#[serde(default)]

View File

@@ -11,18 +11,17 @@ use tracing::warn;
use crate::config::{ProxyConfig, SynLimitMode};
const IPTABLES_CHAIN: &str = "TELEMT_SYNLIMIT";
const IPTABLES_RECENT_NAME: &str = "telemt";
const IPTABLES_HASHLIMIT_NAME: &str = "TELEMT-BUMPER";
const NFT_TABLE: &str = "telemt_synlimit";
const NFT_CHAIN: &str = "input";
const NFT_SET_V4: &str = "telemt_synlimit_v4";
const NFT_SET_V6: &str = "telemt_synlimit_v6";
type SynLimitTarget = (Option<IpAddr>, u16, u32, u32, u32);
#[derive(Default)]
struct SynLimitTargets {
iptables_v4: Vec<(Option<IpAddr>, u16, u32, u32)>,
iptables_v6: Vec<(Option<IpAddr>, u16, u32, u32)>,
nft_v4: Vec<(Option<IpAddr>, u16)>,
nft_v6: Vec<(Option<IpAddr>, u16)>,
iptables_v4: Vec<SynLimitTarget>,
iptables_v6: Vec<SynLimitTarget>,
nft_v4: Vec<SynLimitTarget>,
nft_v6: Vec<SynLimitTarget>,
}
#[derive(Clone, Copy)]
@@ -41,8 +40,8 @@ enum NftFamily {
struct NftApplyPlan<'a> {
family: NftFamily,
v4_targets: &'a [(Option<IpAddr>, u16)],
v6_targets: &'a [(Option<IpAddr>, u16)],
v4_targets: &'a [SynLimitTarget],
v6_targets: &'a [SynLimitTarget],
}
impl SynLimitTargets {
@@ -61,7 +60,6 @@ impl SynLimitTargets {
!self.nft_v4.is_empty() || !self.nft_v6.is_empty()
}
}
impl NftFamily {
fn as_str(self) -> &'static str {
match self {
@@ -146,19 +144,20 @@ fn synlimit_targets(cfg: &ProxyConfig) -> SynLimitTargets {
let ip = (!listener.ip.is_unspecified()).then_some(listener.ip);
let seconds = listener.synlimit_seconds;
let hitcount = listener.synlimit_hitcount;
let burst = listener.synlimit_burst;
match (backend, listener.ip.is_ipv4()) {
(SynLimitMode::Iptables, true) => {
iptables_v4.insert((ip, port, seconds, hitcount));
iptables_v4.insert((ip, port, seconds, hitcount, burst));
}
(SynLimitMode::Iptables, false) => {
iptables_v6.insert((ip, port, seconds, hitcount));
iptables_v6.insert((ip, port, seconds, hitcount, burst));
}
(SynLimitMode::Nftables, true) => {
nft_v4.insert((ip, port));
nft_v4.insert((ip, port, seconds, hitcount, burst));
}
(SynLimitMode::Nftables, false) => {
nft_v6.insert((ip, port));
nft_v6.insert((ip, port, seconds, hitcount, burst));
}
(SynLimitMode::Off, _) => {}
}
@@ -179,7 +178,7 @@ async fn apply_iptables_synlimit_rules(targets: &SynLimitTargets) -> Result<(),
async fn apply_iptables_synlimit_rules_for_binary(
binary: &str,
targets: &[(Option<IpAddr>, u16, u32, u32)],
targets: &[SynLimitTarget],
) -> Result<(), String> {
if targets.is_empty() {
return Ok(());
@@ -188,7 +187,7 @@ async fn apply_iptables_synlimit_rules_for_binary(
return Err(format!("{binary} is not available"));
}
run_command(binary, &["-t", "filter", "-N", IPTABLES_CHAIN], None).await?;
let _ = run_command(binary, &["-t", "filter", "-N", IPTABLES_CHAIN], None).await;
run_command(binary, &["-t", "filter", "-F", IPTABLES_CHAIN], None).await?;
if run_command(
binary,
@@ -206,28 +205,71 @@ async fn apply_iptables_synlimit_rules_for_binary(
.await?;
}
for (ip, port, seconds, hitcount) in targets {
let drop_args =
iptables_synlimit_rule_args(ip, *port, *seconds, *hitcount, "--rcheck", "DROP");
let accept_args =
iptables_synlimit_rule_args(ip, *port, *seconds, *hitcount, "--set", "ACCEPT");
for (idx, (ip, port, seconds, hitcount, burst)) in targets.iter().enumerate() {
let hashlimit_name = format!("{IPTABLES_HASHLIMIT_NAME}-{idx}");
let accept_args = iptables_hashlimit_accept_rule_args(
ip,
*port,
*seconds,
*hitcount,
*burst,
&hashlimit_name,
);
let drop_args = iptables_synlimit_drop_rule_args(ip, *port);
let drop_refs: Vec<&str> = drop_args.iter().map(String::as_str).collect();
let accept_refs: Vec<&str> = accept_args.iter().map(String::as_str).collect();
run_command(binary, &drop_refs, None).await?;
run_command(binary, &accept_refs, None).await?;
run_command(binary, &drop_refs, None).await?;
}
run_command(binary, &["-t", "filter", "-A", IPTABLES_CHAIN, "-j", "RETURN"], None).await?;
Ok(())
}
fn iptables_synlimit_rule_args(
fn iptables_hashlimit_accept_rule_args(
ip: &Option<IpAddr>,
port: u16,
seconds: u32,
hitcount: u32,
recent_op: &str,
verdict: &str,
burst: u32,
hashlimit_name: &str,
) -> Vec<String> {
let mut args = vec![
"-t".to_string(),
"filter".to_string(),
"-A".to_string(),
IPTABLES_CHAIN.to_string(),
"-p".to_string(),
"tcp".to_string(),
"--syn".to_string(),
];
if let Some(ip) = ip {
args.push("-d".to_string());
args.push(ip.to_string());
}
let rate = synlimit_rate_arg(seconds, hitcount);
args.extend([
"--dport".to_string(),
port.to_string(),
"-m".to_string(),
"hashlimit".to_string(),
"--hashlimit-name".to_string(),
hashlimit_name.to_string(),
"--hashlimit-mode".to_string(),
"srcip".to_string(),
"--hashlimit-upto".to_string(),
rate,
"--hashlimit-burst".to_string(),
burst.to_string(),
"--hashlimit-htable-expire".to_string(),
"15000".to_string(),
"-j".to_string(),
"ACCEPT".to_string(),
]);
args
}
fn iptables_synlimit_drop_rule_args(ip: &Option<IpAddr>, port: u16) -> Vec<String> {
let mut args = vec![
"-t".to_string(),
"filter".to_string(),
@@ -244,24 +286,33 @@ fn iptables_synlimit_rule_args(
args.extend([
"--dport".to_string(),
port.to_string(),
"-m".to_string(),
"recent".to_string(),
"--name".to_string(),
IPTABLES_RECENT_NAME.to_string(),
recent_op.to_string(),
"-j".to_string(),
"DROP".to_string(),
]);
if recent_op == "--rcheck" {
args.extend([
"--seconds".to_string(),
seconds.to_string(),
"--hitcount".to_string(),
hitcount.to_string(),
]);
}
args.extend(["-j".to_string(), verdict.to_string()]);
args
}
fn synlimit_rate_arg(seconds: u32, hitcount: u32) -> String {
let seconds = u64::from(seconds.max(1));
let hitcount = u64::from(hitcount.max(1));
for (unit_seconds, unit_name) in [
(1_u64, "second"),
(60_u64, "minute"),
(3_600_u64, "hour"),
(86_400_u64, "day"),
] {
let amount = hitcount.saturating_mul(unit_seconds);
if amount >= seconds && amount % seconds == 0 {
return format!("{}/{}", amount / seconds, unit_name);
}
}
let amount = hitcount
.saturating_mul(86_400)
.saturating_add(seconds - 1)
/ seconds;
format!("{}/day", amount.max(1))
}
async fn clear_iptables_synlimit_rules_for_binary(binary: &str) {
if !command_exists(binary) {
return;
@@ -324,11 +375,10 @@ async fn detect_nft_table_families() -> NftTableFamilies {
}
families
}
fn nft_apply_plan<'a>(
families: NftTableFamilies,
v4_targets: &'a [(Option<IpAddr>, u16)],
v6_targets: &'a [(Option<IpAddr>, u16)],
v4_targets: &'a [SynLimitTarget],
v6_targets: &'a [SynLimitTarget],
) -> Vec<NftApplyPlan<'a>> {
if !v4_targets.is_empty() && !v6_targets.is_empty() {
return vec![NftApplyPlan {
@@ -361,44 +411,33 @@ fn nft_apply_plan<'a>(
}
Vec::new()
}
fn nft_synlimit_script(plan: NftApplyPlan<'_>) -> String {
let mut script = String::new();
script.push_str(&format!("table {} {NFT_TABLE} {{\n", plan.family.as_str()));
if !plan.v4_targets.is_empty() {
script.push_str(&format!(" set {NFT_SET_V4} {{\n"));
script.push_str(" type ipv4_addr\n");
script.push_str(" flags timeout\n");
script.push_str(" }\n");
}
if !plan.v6_targets.is_empty() {
script.push_str(&format!(" set {NFT_SET_V6} {{\n"));
script.push_str(" type ipv6_addr\n");
script.push_str(" flags timeout\n");
script.push_str(" }\n");
}
script.push_str(&format!(" chain {NFT_CHAIN} {{\n"));
script.push_str(" type filter hook input priority filter; policy accept;\n");
for (ip, port) in plan.v4_targets {
for (idx, (ip, port, seconds, hitcount, burst)) in plan.v4_targets.iter().enumerate() {
let daddr = ip
.map(|ip| format!(" ip daddr {ip}"))
.unwrap_or_else(String::new);
let rate = synlimit_rate_arg(*seconds, *hitcount);
script.push_str(&format!(
" tcp flags & (fin|syn|rst|ack) == syn{daddr} tcp dport {port} ip saddr @{NFT_SET_V4} drop\n"
" tcp flags & (fin|syn|rst|ack) == syn{daddr} tcp dport {port} meter telemt_synlimit_v4_{idx} {{ ip saddr limit rate over {rate} burst {burst} packets }} drop\n"
));
script.push_str(&format!(
" tcp flags & (fin|syn|rst|ack) == syn{daddr} tcp dport {port} add @{NFT_SET_V4} {{ ip saddr timeout 1s }} accept\n"
" tcp flags & (fin|syn|rst|ack) == syn{daddr} tcp dport {port} accept\n"
));
}
for (ip, port) in plan.v6_targets {
for (idx, (ip, port, seconds, hitcount, burst)) in plan.v6_targets.iter().enumerate() {
let daddr = ip
.map(|ip| format!(" ip6 daddr {ip}"))
.unwrap_or_else(String::new);
let rate = synlimit_rate_arg(*seconds, *hitcount);
script.push_str(&format!(
" tcp flags & (fin|syn|rst|ack) == syn{daddr} tcp dport {port} ip6 saddr @{NFT_SET_V6} drop\n"
" tcp flags & (fin|syn|rst|ack) == syn{daddr} tcp dport {port} meter telemt_synlimit_v6_{idx} {{ ip6 saddr limit rate over {rate} burst {burst} packets }} drop\n"
));
script.push_str(&format!(
" tcp flags & (fin|syn|rst|ack) == syn{daddr} tcp dport {port} add @{NFT_SET_V6} {{ ip6 saddr timeout 1s }} accept\n"
" tcp flags & (fin|syn|rst|ack) == syn{daddr} tcp dport {port} accept\n"
));
}
script.push_str(" }\n");