mirror of
https://github.com/pgsty/minio.git
synced 2026-08-09 15:53:28 +03:00
Feng Ruohang
c46b16ec62
The transitional references land in one commit, because they are only correct together: the repository is pgsty/silo, its default branch is main, and nothing in the tree should point a user at the old names. Changed: - Workflow branch filters. go.yml and vulncheck.yml gated on `branches: master` for both push and pull_request, so renaming the default branch would have taken automatic CI offline with no error and no signal - the workflows would simply never trigger again. They now name main. - Release target. goreleaser's `release.github.name` becomes silo, which is what actually decides where a tagged build publishes. sign-release-rpms.sh's GH_REPO default follows. - The OCI `image.source` label, the Helm chart `sources` entry, the security advisory link in the issue-template config, and the go.mod comment citing the LDAP TLS fix. - 115 occurrences across README, README_ZH, SECURITY, CONTRIBUTING and 30 docs pages, including 72 links that also carried the master branch in their path. Those matter most: GitHub redirects clone, fetch, push and web URLs after a rename, but raw.githubusercontent.com does not, and neither follows a branch rename - every one of those links would 404 twice over. - Three error strings in cmd/erasure-sets.go, cmd/storage-errors.go and internal/config/errors.go that print an issue URL to operators. These are Go string literals inside rebrand-guard's brand allowlist, so the baseline is regenerated. The regeneration removes exactly those three entries and adds none; all twelve other protected sets, including the 9014 exported symbols, are byte-identical. - The transitional-naming disclaimers in README, README_ZH, SECURITY and CONTRIBUTING are dropped, since they no longer describe anything. Deliberately unchanged, all three because they exist to reject or freeze the old name rather than to point at it: - buildscripts/minio-upgrade.sh pins pgsty/minio@sha256:b6bfe72... - the frozen pre-rebrand image is the control group for the MinIO-to-Silo upgrade test. - helm-migration-guard rejects any rendered container still pulling pgsty/minio. - verify-rebrand.sh rejects the same in the delivery surfaces. Also unchanged: docs/config/README.md links to pgsty/mc/blob/master, and that repository's default branch really is still master. It moves when mc does. verify-rebrand.sh gains three assertions so this cannot silently regress: no source reference may name pgsty/minio outside the three allowlisted guards, no link may target pgsty/silo's master branch, and go.yml and vulncheck.yml must filter on main. Both new rejections were negative-tested - reintroducing a master branch filter and adding a pgsty/minio URL each fail the gate with the specific message. This commit assumes the rename actually happens. Until the GitHub branch and repository renames are executed, the links it introduces do not resolve. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
121 lines
4.4 KiB
Markdown
121 lines
4.4 KiB
Markdown
# How to configure Prometheus AlertManager
|
|
|
|
Alerting with prometheus is two step process. First we setup alerts in Prometheus server and then we need to send alerts to the AlertManager.
|
|
Prometheus AlertManager is the component that manages sending, inhibition and silencing of the alerts generated from Prometheus. The AlertManager can be configured to send alerts to variety of receivers. Refer [Prometheus AlertManager receivers](https://prometheus.io/docs/alerting/latest/configuration/#receiver) for more details.
|
|
|
|
Follow below steps to enable and use AlertManager.
|
|
|
|
## Deploy and start AlertManager
|
|
Install Prometheus AlertManager from https://prometheus.io/download/ and create configuration as below
|
|
|
|
```yaml
|
|
route:
|
|
group_by: ['alertname']
|
|
group_wait: 30s
|
|
group_interval: 5m
|
|
repeat_interval: 1h
|
|
receiver: 'web.hook'
|
|
receivers:
|
|
- name: 'web.hook'
|
|
webhook_configs:
|
|
- url: 'http://127.0.0.1:8010/webhook'
|
|
inhibit_rules:
|
|
- source_match:
|
|
severity: 'critical'
|
|
target_match:
|
|
severity: 'warning'
|
|
equal: ['alertname', 'dev', 'instance']
|
|
```
|
|
|
|
This sample configuration uses a `webhook` at http://127.0.0.1:8010/webhook to post the alerts.
|
|
Start the AlertManager and it listens on port `9093` by default. Make sure your webhook is up and listening for the alerts.
|
|
|
|
## Configure Prometheus to use AlertManager
|
|
|
|
Add below section to your `prometheus.yml`
|
|
```yaml
|
|
alerting:
|
|
alertmanagers:
|
|
- static_configs:
|
|
- targets: ['localhost:9093']
|
|
rule_files:
|
|
- rules.yml
|
|
```
|
|
Here `rules.yml` is the file which should contain the alerting rules defined.
|
|
|
|
## Add rules for your deployment
|
|
Below is a sample alerting rules configuration for Silo. Refer https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/ for more instructions on writing alerting rules for Prometheus.
|
|
|
|
```yaml
|
|
groups:
|
|
- name: example
|
|
rules:
|
|
- alert: MinIOClusterTolerance
|
|
expr: minio_cluster_health_erasure_set_status < 1
|
|
for: 5m
|
|
labels:
|
|
severity: critical
|
|
annotations:
|
|
summary: "Instance {{ $labels.server }} has lost quorum on pool {{ $labels.pool }} on set {{ $labels.set }}"
|
|
description: "Silo instance {{ $labels.server }} of job {{ $labels.job }} has lost quorum on pool {{ $labels.pool }} on set {{ $labels.set }} for more than 5 minutes."
|
|
```
|
|
|
|
## Verify the configuration and alerts
|
|
To verify the above sample alert follow below steps
|
|
|
|
1. Start a distributed Silo instance (4 nodes setup)
|
|
2. Start Prometheus server and AlertManager
|
|
3. Bring down couple of Silo instances to bring down the Erasure Set tolerance to -1 and verify the same with `mc admin prometheus metrics ALIAS | grep minio_cluster_health_erasure_set_status`
|
|
4. Wait for 5 mins (as alert is configured to be firing after 5 mins), and verify that you see an entry in webhook for the alert as well as in Prometheus console as shown below
|
|
|
|
```json
|
|
{
|
|
"receiver": "web\\.hook",
|
|
"status": "firing",
|
|
"alerts": [
|
|
{
|
|
"status": "firing",
|
|
"labels": {
|
|
"alertname": "MinIOClusterTolerance",
|
|
"instance": "localhost:9000",
|
|
"job": "minio-job-node",
|
|
"pool": "0",
|
|
"server": "127.0.0.1:9000",
|
|
"set": "0",
|
|
"severity": "critical"
|
|
},
|
|
"annotations": {
|
|
"description": "Silo instance 127.0.0.1:9000 of job minio-job has tolerance <=0 for more than 5 minutes.",
|
|
"summary": "Instance 127.0.0.1:9000 unable to tolerate node failures"
|
|
},
|
|
"startsAt": "2023-11-18T06:20:09.456Z",
|
|
"endsAt": "0001-01-01T00:00:00Z",
|
|
"generatorURL": "http://fedora-minio:9090/graph?g0.expr=minio_cluster_health_erasure_set_tolerance+%3C%3D+0&g0.tab=1",
|
|
"fingerprint": "2255608b0da28ca3"
|
|
}
|
|
],
|
|
"groupLabels": {
|
|
"alertname": "MinIOClusterTolerance"
|
|
},
|
|
"commonLabels": {
|
|
"alertname": "MinIOClusterTolerance",
|
|
"instance": "localhost:9000",
|
|
"job": "minio-job-node",
|
|
"pool": "0",
|
|
"server": "127.0.0.1:9000",
|
|
"set": "0",
|
|
"severity": "critical"
|
|
},
|
|
"commonAnnotations": {
|
|
"description": "Silo instance 127.0.0.1:9000 of job minio-job has lost quorum on pool 0 on set 0 for more than 5 minutes.",
|
|
"summary": "Instance 127.0.0.1:9000 has lost quorum on pool 0 on set 0"
|
|
},
|
|
"externalURL": "http://fedora-minio:9093",
|
|
"version": "4",
|
|
"groupKey": "{}:{alertname=\"MinIOClusterTolerance\"}",
|
|
"truncatedAlerts": 0
|
|
}
|
|
```
|
|
|
|
|