Feng Ruohang
2ff594f4bb
Add 'silo healthcheck [live|ready|cluster|cluster-read]', a thin anonymous HTTP client for the server's own /minio/health/* endpoints, so containers without a shell, curl, or mc can still run health checks. Design: silo.pgsty.com/compatibility/feature/healthcheck/ The check vocabulary maps 1:1 onto the health API paths; the probe target is derived from the server's own --address/MINIO_ADDRESS contract with HTTPS auto-detected from the certs directory, and can be overridden with --url. Exit codes are 0/1 only (Docker reserves 2); diagnostics (x-minio-server-status, quorum headers) go into a single output line for docker inspect. The request is strictly anonymous (a credentialed request would be rejected by the reserved-path guard), the transport bypasses HTTP_PROXY, and certificate verification is skipped to match kubelet HTTPS probe behavior. Cluster checks default to a 15s deadline so the server's 10s cluster_deadline can elapse. Compatibility notes: the preserved /minio/health/* path literals and the MINIO_ADDRESS env var are upstream wire/config surface, reused on purpose; the rebrand-guard baseline is regenerated for the new route literals (tests included) with zero new exported symbols. The docker entrypoint argv translation learns the new command name. Verified: unit tests, entrypoint tests, go vet, plus an end-to-end run against a live server covering all four checks, --maintenance (412), --json, usage errors, unreachable and timeout paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
3.9 KiB
Silo Healthcheck
Silo server exposes three un-authenticated, healthcheck endpoints liveness probe and a cluster probe at /minio/health/live and /minio/health/cluster respectively.
Native CLI probe
The silo binary can probe those endpoints itself, which makes health checking possible in containers that ship no shell, curl, or mc:
silo healthcheck [FLAGS] [live|ready|cluster|cluster-read]
The check name maps 1:1 onto /minio/health/<path>; live is the default. The exit code is 0 when healthy and 1 otherwise, and one diagnostic line (including the x-minio-server-status and quorum headers on failure) is printed for docker inspect to capture. The probe target is derived the same way the server derives its own listen address — --address / MINIO_ADDRESS, with HTTPS auto-detected from public.crt and private.key in --certs-dir — or overridden wholesale with --url. Certificate verification is skipped, matching the kubelet's behavior for HTTPS probes.
Use it as an image HEALTHCHECK (exec form, since there may be no shell):
HEALTHCHECK --interval=30s --timeout=5s --start-period=2m --start-interval=2s --retries=3 \
CMD ["/usr/bin/silo", "healthcheck", "ready"]
or as a Docker Compose healthcheck:
healthcheck:
test: ["CMD", "/usr/bin/silo", "healthcheck", "ready"]
interval: 5s
timeout: 5s
retries: 5
silo healthcheck --maintenance cluster answers the pre-drain question documented below: exit 0 when the node can be taken down safely, exit 1 (HTTP 412) when doing so would lose HA. Keep the cluster checks out of per-container liveness probes — they reflect cluster-wide quorum, not this process.
Liveness probe
This probe always responds with '200 OK'. Only fails if 'etcd' is configured and unreachable. When liveness probe fails, Kubernetes like platforms restart the container.
livenessProbe:
httpGet:
path: /minio/health/live
port: 9000
scheme: HTTP
initialDelaySeconds: 120
periodSeconds: 30
timeoutSeconds: 10
successThreshold: 1
failureThreshold: 3
Readiness probe
This probe always responds with '200 OK'. Only fails if 'etcd' is configured and unreachable. When readiness probe fails, Kubernetes like platforms turn-off routing to the container.
readinessProbe:
httpGet:
path: /minio/health/ready
port: 9000
scheme: HTTP
initialDelaySeconds: 120
periodSeconds: 15
timeoutSeconds: 10
successThreshold: 1
failureThreshold: 3
Cluster probe
Cluster-writeable probe
The reply is '200 OK' if cluster has write quorum if not it returns '503 Service Unavailable'.
curl http://silo1:9001/minio/health/cluster
HTTP/1.1 503 Service Unavailable
Accept-Ranges: bytes
Content-Length: 0
Server: Silo
Vary: Origin
X-Amz-Bucket-Region: us-east-1
X-Minio-Write-Quorum: 3
X-Amz-Request-Id: 16239D6AB80EBECF
X-Xss-Protection: 1; mode=block
Date: Tue, 21 Jul 2020 00:36:14 GMT
Cluster-readable probe
The reply is '200 OK' if cluster has read quorum if not it returns '503 Service Unavailable'.
curl http://silo1:9001/minio/health/cluster/read
HTTP/1.1 503 Service Unavailable
Accept-Ranges: bytes
Content-Length: 0
Server: Silo
Vary: Origin
X-Amz-Bucket-Region: us-east-1
X-Minio-Write-Quorum: 3
X-Amz-Request-Id: 16239D6AB80EBECF
X-Xss-Protection: 1; mode=block
Date: Tue, 21 Jul 2020 00:36:14 GMT
Checking cluster health for maintenance
You may query the cluster probe endpoint to check if the node which received the request can be taken down for maintenance, if the server replies back '412 Precondition Failed' this means you will lose HA. '200 OK' means you are okay to proceed.
curl http://silo1:9001/minio/health/cluster?maintenance=true
HTTP/1.1 412 Precondition Failed
Accept-Ranges: bytes
Content-Length: 0
Server: Silo
Vary: Origin
X-Amz-Bucket-Region: us-east-1
X-Amz-Request-Id: 16239D63820C6E76
X-Xss-Protection: 1; mode=block
X-Minio-Write-Quorum: 3
Date: Tue, 21 Jul 2020 00:35:43 GMT