Compare commits

..

22 Commits

Author SHA1 Message Date
Feng Ruohang 5c6e30d4f1 helm: harden Silo chart delivery 2026-07-28 11:28:06 +08:00
Wesley Schwengle 0ce4b0f14b docs: point documentation links to silo.pigsty.io (#41)
Replace upstream docs.min.io references with the SILO documentation domain and remove the upstream community path prefix.

Co-authored-by: waterkip <6317502+waterkip@users.noreply.github.com>
2026-07-20 19:43:45 +08:00
Feng Ruohang 3e61b1d3a5 chore: update Go module dependencies 2026-06-18 16:14:50 +08:00
Feng Ruohang df627ff896 fix: bump Go toolchain to 1.26.4
Update the module Go directive and release Docker build images from Go 1.26.2 to Go 1.26.4 so local, CI, hotfix, and release builds use the same patched toolchain.

Keep ordinary Go module requirements and replacements unchanged; this intentionally avoids a third-party dependency refresh while allowing container system packages to refresh through the newer golang Alpine base image and existing apk resolution.

Update the security advisory index to record the Go 1.26.4 toolchain bump alongside the earlier Go 1.26.2 security update.

Verified with go build ./..., go vet ./cmd/, and focused cmd tests. go mod tidy -diff was attempted as a read-only dependency drift check but could not complete because proxy.golang.org timed out while fetching uncached transitive test modules.

Co-authored-by: Codex <codex@openai.com>
2026-06-12 21:24:23 +08:00
Feng Ruohang 73ac524724 fix: CVE-2026-42600 remove ReadMultiple storage-REST API
The internode storage-REST ReadMultiple endpoint (/rmpl) joined
attacker-controlled Bucket/Prefix/Files into a filesystem path with no
validation, letting a peer with internode credentials read files outside
the drive root (GHSA-xh8f-g2qw-gcm7).

ReadMultiple has had no production caller since upstream #20390 removed
the last one (listParts) in Sep 2024; multipart now uses ReadParts (/rps).
Following the upstream fix, remove the whole API instead of validating
paths: route constant and registration, server handler, REST client
wrapper, the StorageAPI/xlStorage/xlStorageDiskIDCheck methods, the
storageMetricReadMultiple metric, and the ReadMultipleReq/Resp datatypes.
Regenerated the msgp and stringer outputs; storageRESTVersion stays at v63.

Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Code <claude-code@anthropic.com>
2026-06-12 21:08:59 +08:00
Feng Ruohang fd69c89d05 fix: complete CVE-2026-39414 S3 Select record limit enforcement
Route JSON Lines through the bounded PReader path so oversized records are rejected consistently instead of bypassing the limit on SIMD-capable CPUs.

Preserve S3 Select error codes in stream error events, wrap JSON parser errors as JSONParsingError, and flush completed records before returning a terminal error event. Add regression coverage for oversized JSON Lines input and error code preservation.

Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Code <claude-code@anthropic.com>
2026-06-12 21:08:59 +08:00
Feng Ruohang 5e40665acd fix: harden LDAP STS rate-limit source bucketing
Remove the per-username LDAP STS throttle bucket and keep the limiter keyed only by source IP. A username bucket is shared across all clients and lets one source keep a known account's bucket drained with bad-password attempts, locking the legitimate user out before LDAP bind.

For trusted proxies, stop trusting the left-most forwarded address. Resolve X-Forwarded-For right-to-left, skip trusted proxy hops, reject catch-all trusted-proxy CIDRs, and intentionally ignore RFC 7239 Forwarded for this security-sensitive bucket. Document that X-Real-IP is trusted verbatim and must be overwritten by the trusted proxy, not passed through from clients.

Update focused limiter, source-IP, trusted-proxy, and LDAP config tests to cover source-only buckets, spoofed appended XFF, multi-hop trusted proxies, Forwarded fallback, catch-all rejection, and the X-Real-IP deployment contract.

Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Code <claude-code@anthropic.com>
2026-06-12 21:08:59 +08:00
Feng Ruohang f48dbe777d docs: refresh security docs and fork references
Update the STS, security, select, and Docker documentation to reflect the recent hardening work, including LDAP STS throttling details, OIDC JWT verification changes, and the new pgsty-specific security policy and advisory index.

Rewrite repository and raw-document links that still pointed at minio/minio so the docs consistently reference pgsty/minio instead.

The core idea is to keep the documentation aligned with the fork's actual security behavior, ownership, and upgrade guidance without mixing in unrelated code changes.
2026-04-17 15:06:42 +08:00
Feng Ruohang f44110890b fix: tighten LDAP STS rate-limit accounting
Prevent LDAP STS reservation cancel paths from over-crediting rate-limit buckets by capping refill and refund capacity against in-flight reservations.

Add an explicit trusted-proxy allowlist for LDAP STS source bucketing, prefer clean X-Real-IP values on trusted peers, and extend tests/docs for the new behavior.
2026-04-16 23:22:13 +08:00
Feng Ruohang 9e10f6d9a0 fix: harden LDAP STS rate-limit source IP
Use the socket peer address for LDAP STS per-IP rate limiting instead of the generic forwarded-header-aware helper. This keeps the security-sensitive rate-limit key from trusting spoofable X-Forwarded-For, X-Real-IP, and Forwarded headers while leaving the rest of the source-IP behavior unchanged.

Add focused regression coverage for RemoteAddr parsing, header spoofing, and peer-address bucket selection.
2026-04-16 21:21:53 +08:00
Feng Ruohang 18b712d49a fix: preserve LDAP STS rate limits without penalizing success 2026-04-16 17:56:51 +08:00
Feng Ruohang db4c0fd5e3 fix: bump Go to 1.26.2 for upstream security fixes
Update go.mod and all golang build images from 1.26.1 to 1.26.2 to pick up the upstream 2026-04-07 security release.

This includes fixes for CVE-2026-32280 / CVE-2026-32281 in crypto/x509, CVE-2026-32283 in crypto/tls, and the related toolchain and standard library security fixes shipped in go1.26.2, without changing any unrelated dependencies.
2026-04-16 15:08:42 +08:00
Feng Ruohang efb6e5b00b fix: fake CVE-2026-40028 harden snowball unsigned-trailer auth
Track issue #28 / GHSA-9c4q-hq6p-c237 as fake CVE-2026-40028. Close the Snowball auto-extract auth gap in PutObjectExtractHandler by treating authTypeStreamingUnsignedTrailer the same as ordinary PUTs: honor X-Amz-Decoded-Content-Length, initialize newUnsignedV4ChunkedReader(), and verify the SigV4 request before any tar bytes reach untar(). This removes the forged-signature write primitive that let a single request fan out into arbitrary extracted object creation.

Add regression coverage for forged-signature Snowball unsigned-trailer writes, anonymous Snowball requests against non-public buckets, and legitimate signed Snowball extraction with trailing CRC32 trailers. Validate the new tests against the vulnerable parent and patched tree, and confirm with containerized before/after smoke runs that the exploit succeeds pre-fix, fails post-fix, and normal signed Snowball uploads still extract correctly.

Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Code <claude-code@anthropic.com>
2026-04-16 14:46:45 +08:00
Feng Ruohang f444b6f37e fix: fake CVE-2026-40027 block unsigned-trailer query auth bypass
Track issue #27 / GHSA-hv4r-mvr4-25vw as fake CVE-2026-40027. Close the unsigned-trailer trust flaw that let query-string credentials skip signature verification in PutObject and PutObjectPart by moving presigned rejection and SigV4 verification into newUnsignedV4ChunkedReader(), so authTypeStreamingUnsignedTrailer can no longer silently downgrade query auth into an anonymous body read.

Add focused regression coverage for forged query-string-only unsigned-trailer PUTs and multipart uploads, mixed header/query auth rejection, and anonymous unsigned-trailer writes that remain allowed only when bucket policy explicitly permits them. Validate the new tests against the vulnerable parent and confirm with before/after live-server runs that presigned unsigned-trailer attacks are rejected while legitimate header-authenticated and policy-driven flows still work.

Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Code <claude-code@anthropic.com>
2026-04-16 14:46:44 +08:00
Feng Ruohang 3252d5b7f3 fix: CVE-2026-39414 harden S3 Select oversized record handling
Enforce the 1 MiB maxCharsPerRecord limit while splitting CSV and line-delimited JSON input so oversized records are rejected before they can be buffered and parsed.

Return OverMaxRecordSize for these failures instead of collapsing them into InternalError, and preserve splitter errors in the JSON worker so oversized-record failures are not lost after successful partial decode.
2026-04-15 22:55:52 +08:00
Feng Ruohang 56fa63bfd1 fix: CVE-2026-34204 block replication metadata injection
Close the replication-header trust flaw that allowed ordinary PutObject and CopyObject requests to smuggle X-Minio-Replication-* headers into X-Minio-Internal-* SSE metadata and write objects into an unreadable state. Stop accepting replication-only metadata in the default extraction path, restore it only after a trusted replication write has passed ReplicateObjectAction, and tighten CopyObject by sanitizing replication-only request headers before metadata, precondition, and SSE-C source handling consume them. Also gate replica status writes on the same trusted replication path and restore replication SSE metadata in multipart and snowball upload flows so legitimate replication continues to work.

Add focused regression coverage for untrusted PUT and COPY header poisoning at the handler layer, plus helper tests for trusted vs untrusted metadata extraction and CopyObject header sanitization. Validate the new tests against both the patched tree and the vulnerable HEAD baseline, and confirm with live server before/after runs that malicious PUT/COPY requests no longer turn objects unreadable.

Co-authored-by: Codex <codex@openai.com>
Co-Authored-By: Claude <noreply@anthropic.com>
2026-04-15 18:36:49 +08:00
Feng Ruohang 3b950f8fa8 fix: CVE-2026-33419 harden LDAP STS auth
Prevent username enumeration in AssumeRoleWithLDAPIdentity by returning the same external STS error for unknown users and invalid passwords, while preserving LDAP infrastructure failures as upstream errors so they continue to surface as 500s and remain visible in server logs.

Add a small in-memory rate limiter for LDAP STS login attempts, keyed by source IP and normalized username, and add regression coverage for auth failure classification, throttling, and Docker-backed LDAP end-to-end flows.

Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Code <claude-code@anthropic.com>
2026-04-15 14:02:41 +08:00
Feng Ruohang d24f449e08 fix: CVE-2026-33322 harden OIDC JWT verification
Close the OIDC JWT algorithm confusion flaw in AssumeRoleWithWebIdentity by restoring a JWKS-only verification path. Stop injecting the client secret into the verifier keyring and restrict accepted signing methods to the asymmetric algorithms already supported by the existing JWKS flow.

Add regression coverage to verify HS256 tokens are rejected, RS256 tokens remain valid, and JWKS refresh and retry logic cannot bypass the method allowlist.

Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Code <claude-code@anthropic.com>
2026-04-15 14:02:41 +08:00
Feng Ruohang e4fa063942 Merge pull request #19 from mfredenhagen/mario/main/CVE-2026-39883-fix
Bump go.opentelemetry.io version to address CVE-2026-39883
2026-04-13 00:12:10 +08:00
Feng Ruohang ff58df9499 Merge pull request #18 from ZouhairCharef/master
fix: upgrade go-jose to v4.1.4 to patch CVE-2026-34986
2026-04-13 00:11:43 +08:00
Mario Fredenhagen 1869bd30b8 Bump go.opentelemetry.io version to address CVE-2026-39883 2026-04-10 14:33:39 +02:00
Zouhair EC-charef 68e0ba9971 fix: upgrade go-jose to v4.1.4 to patch CVE-2026-34986
Updates github.com/go-jose/go-jose/v4 from v4.1.3 to v4.1.4 to fix a high-severity denial of service vulnerability (CVE-2026-34986).
2026-04-06 20:34:51 +01:00
118 changed files with 3683 additions and 1861 deletions
+110
View File
@@ -0,0 +1,110 @@
name: Helm Chart
on:
workflow_dispatch:
pull_request:
paths:
- ".github/workflows/helm.yml"
- "helm/**"
- "helm-releases/minio-*.tgz"
- "helm-reindex.sh"
- "index.yaml"
push:
branches:
- master
paths:
- ".github/workflows/helm.yml"
- "helm/**"
- "helm-releases/minio-*.tgz"
- "helm-reindex.sh"
- "index.yaml"
permissions:
contents: read
jobs:
install-smoke:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up Helm
uses: azure/setup-helm@v4
with:
version: v3.19.0
- name: Verify packaged chart
run: |
set -euo pipefail
chart_version="$(awk '$1 == "version:" { print $2; exit }' helm/minio/Chart.yaml)"
chart_package="helm-releases/minio-${chart_version}.tgz"
test -f "${chart_package}"
echo "CHART_PACKAGE=${chart_package}" >> "${GITHUB_ENV}"
helm lint --strict "${chart_package}"
unpacked="$(mktemp -d)"
tar -xzf "${chart_package}" -C "${unpacked}"
diff -ru --exclude Chart.yaml helm/minio "${unpacked}/minio"
package_digest="$(sha256sum "${chart_package}" | awk '{ print $1 }')"
grep -F "digest: ${package_digest}" index.yaml
grep -F "https://raw.githubusercontent.com/pgsty/minio/master/${chart_package}" index.yaml
- name: Render and verify immutable images
run: |
set -euo pipefail
expected_image='pgsty/minio@sha256:dacff8306a6e0a734518533992dbdcca26bc1ca47f77cf47cb9945725f92b29b'
rendered="$(mktemp)"
helm template silo "${CHART_PACKAGE}" \
--namespace silo \
--set mode=standalone \
--set persistence.enabled=false > "${rendered}"
grep -F "image: \"${expected_image}\"" "${rendered}"
if grep -Eq 'image:.*(quay\.io/minio|minio/minio|minio/mc)' "${rendered}"; then
echo "Rendered chart still references an upstream MinIO image."
exit 1
fi
- name: Create Kind cluster
uses: helm/kind-action@v1
with:
version: v0.31.0
kubectl_version: v1.35.0
node_image: kindest/node:v1.35.0@sha256:452d707d4862f52530247495d180205e029056831160e22870e37e3f6c1ac31f
cluster_name: silo-chart
wait: 120s
- name: Install chart
run: |
set -euo pipefail
helm install silo "${CHART_PACKAGE}" \
--namespace silo \
--create-namespace \
--wait \
--wait-for-jobs \
--timeout 10m \
--set mode=standalone \
--set replicas=1 \
--set persistence.enabled=false \
--set resources.requests.memory=512Mi \
--set rootUser=siloadmin \
--set rootPassword=silo-smoke-password \
--set 'buckets[0].name=bootstrap' \
--set 'buckets[0].policy=none' \
--set 'buckets[0].purge=false'
kubectl rollout status deployment/silo-minio --namespace silo --timeout=2m
- name: Run S3 smoke test
run: |
set -euo pipefail
helm test silo --namespace silo --logs --timeout 5m
- name: Diagnostics
if: failure()
run: |
helm status silo --namespace silo || true
kubectl get all --namespace silo -o wide || true
kubectl describe pods --namespace silo || true
kubectl logs --namespace silo --all-containers=true --prefix=true --selector release=silo || true
+1 -1
View File
@@ -1,4 +1,4 @@
FROM golang:1.26.1-alpine AS build
FROM golang:1.26.4-alpine AS build
ARG TARGETARCH
+1 -1
View File
@@ -1,4 +1,4 @@
FROM golang:1.26.1-alpine as build
FROM golang:1.26.4-alpine as build
ARG TARGETARCH
ARG RELEASE
+1 -1
View File
@@ -1,4 +1,4 @@
FROM golang:1.26.1-alpine AS build
FROM golang:1.26.4-alpine AS build
ARG TARGETARCH
ARG RELEASE
+1 -1
View File
@@ -1,4 +1,4 @@
FROM golang:1.26.1-alpine AS build
FROM golang:1.26.4-alpine AS build
ARG TARGETARCH
ARG RELEASE
+12 -1
View File
@@ -8,7 +8,7 @@
[![Docker Image](https://img.shields.io/badge/Docker-pgsty/minio-%232496ED?style=flat&logo=docker&logoColor=white)](https://hub.docker.com/r/pgsty/minio)
> [!IMPORTANT]
> **This is a community-maintained fork of [minio/minio](https://github.com/minio/minio), maintained by [Pigsty](https://pigsty.io).**
> **This is a community-maintained fork of the upstream MinIO project, maintained by [Pigsty](https://pigsty.io).**
> This project is **NOT** affiliated with, endorsed by, or sponsored by MinIO, Inc.
> "MinIO" is a trademark of MinIO, Inc., used here solely to identify the upstream project.
>
@@ -30,3 +30,14 @@ Ansible Deployment: [https://pigsty.io/docs/minio](https://pigsty.io/docs/minio)
APT/YUM repo for `minio` and `mcli` binary: [https://pigsty.io/docs/infra](https://pigsty.io/docs/repo/infra/list/#object-storage)
## Kubernetes delivery and scope
This repository maintains the direct-deployment Helm chart in [`helm/minio`](helm/minio/README.md) for standalone and distributed Silo server deployments. Chart defaults use the version- and digest-pinned multi-architecture [`pgsty/minio`](https://hub.docker.com/r/pgsty/minio) image, and every chart change is checked by a real Kind install plus an S3 write/read/delete smoke test.
```bash
helm repo add silo https://raw.githubusercontent.com/pgsty/minio/master
helm repo update silo
helm install my-release silo/minio --namespace minio --create-namespace
```
The MinIO Operator, Tenant CRDs, and operator lifecycle are **not maintained by this repository**. There is no Pigsty/Silo operator fork here. Users who require an operator must select and validate a separate operator implementation; issues and pull requests for operator code should be filed with that implementation.
+10 -32
View File
@@ -1,42 +1,20 @@
# Security Policy
This repository is the `pgsty/minio` community fork of `minio/minio`. Upstream MinIO security contacts do not handle fork-specific fixes or release notes for this repository.
## Supported Versions
We always provide security updates for the [latest release](https://github.com/minio/minio/releases/latest).
Whenever there is a security update you just need to upgrade to the latest version.
Security fixes are tracked on the active `master` branch and summarized in [docs/security/advisories.md](docs/security/advisories.md).
## Reporting a Vulnerability
All security bugs in [minio/minio](https://github,com/minio/minio) (or other minio/* repositories)
should be reported by email to security@min.io. Your email will be acknowledged within 48 hours,
and you'll receive a more detailed response to your email within 72 hours indicating the next steps
in handling your report.
For vulnerabilities in this fork:
Please, provide a detailed explanation of the issue. In particular, outline the type of the security
issue (DoS, authentication bypass, information disclose, ...) and the assumptions you're making (e.g. do
you need access credentials for a successful exploit).
1. Follow the fork-specific expectations in [VULNERABILITY_REPORT.md](VULNERABILITY_REPORT.md).
2. Prefer the `pgsty/minio` repository's GitHub security reporting workflow when it is available.
3. If private reporting is not available, contact the maintainers through the `pgsty/minio` repository before publishing detailed exploit information.
4. If you confirm the issue also affects upstream `minio/minio`, report it upstream separately.
If you have not received a reply to your email within 48 hours or you have not heard from the security team
for the past five days please contact the security team directly:
## Disclosure Process
- Primary security coordinator: aead@min.io
- Secondary coordinator: harsha@min.io
- If you receive no response: dev@min.io
### Disclosure Process
MinIO uses the following disclosure process:
1. Once the security report is received one member of the security team tries to verify and reproduce
the issue and determines the impact it has.
2. A member of the security team will respond and either confirm or reject the security report.
If the report is rejected the response explains why.
3. Code is audited to find any potential similar problems.
4. Fixes are prepared for the latest release.
5. On the date that the fixes are applied a security advisory will be published on <https://blog.min.io>.
Please inform us in your report email whether MinIO should mention your contribution w.r.t. fixing
the security issue. By default MinIO will **not** publish this information to protect your privacy.
This process can take some time, especially when coordination is required with maintainers of other projects.
Every effort will be made to handle the bug in as timely a manner as possible, however it's important that we
follow the process described above to ensure that disclosures are handled consistently.
Fork-specific fixes and user-visible upgrade notes are published in [docs/security/advisories.md](docs/security/advisories.md). The fork-specific triage and remediation process is described in [VULNERABILITY_REPORT.md](VULNERABILITY_REPORT.md).
+19 -20
View File
@@ -1,38 +1,37 @@
# Vulnerability Management Policy
This document formally describes the process of addressing and managing a
reported vulnerability that has been found in the MinIO server code base,
any directly connected ecosystem component or a direct / indirect dependency
of the code base.
This document describes how the `pgsty/minio` maintainers investigate,
assess, and remediate reported vulnerabilities affecting this fork, any
directly shipped component, or a direct / indirect dependency used by this
repository.
## Scope
The vulnerability management policy described in this document covers the
process of investigating, assessing and resolving a vulnerability report
opened by a MinIO employee or an external third party.
This policy covers vulnerability reports opened by repository maintainers or
external third parties against `pgsty/minio` itself, its release artifacts, or
dependencies that materially affect this fork.
Therefore, it lists pre-conditions and actions that should be performed to
resolve and fix a reported vulnerability.
It defines the information needed for triage and the expected remediation
workflow for supported fixes.
## Vulnerability Management Process
The vulnerability management process requires that the vulnerability report
contains the following information:
A useful vulnerability report should contain the following information:
- The project / component that contains the reported vulnerability.
- A description of the vulnerability. In particular, the type of the
reported vulnerability and how it might be exploited. Alternatively,
a well-established vulnerability identifier, e.g. CVE number, can be
used instead.
reported vulnerability and how it might be exploited. Alternatively,
a well-established vulnerability identifier, such as a CVE or GHSA ID, can
be used instead.
Based on the description mentioned above, a MinIO engineer or security team
member investigates:
Based on the report, the `pgsty/minio` maintainers investigate:
- Whether the reported vulnerability exists.
- The conditions that are required such that the vulnerability can be exploited.
- Which releases, branches, or deployment paths are affected.
- The steps required to fix the vulnerability.
In general, if the vulnerability exists in one of the MinIO code bases
itself - not in a code dependency - then MinIO will, if possible, fix
the vulnerability or implement reasonable countermeasures such that the
vulnerability cannot be exploited anymore.
If the vulnerability exists in this fork itself, the maintainers will, when
feasible, fix the issue or implement reasonable countermeasures such that the
vulnerability can no longer be exploited. Fork-specific upgrade notes and
security advisories are published in `docs/security/advisories.md`.
+1 -1
View File
@@ -267,7 +267,7 @@ func (a adminAPIHandlers) AddServiceAccountLDAP(w http.ResponseWriter, r *http.R
lookupResult, targetGroups, err = globalIAMSys.LDAPConfig.LookupUserDN(targetUser)
if err != nil {
// if not found, check if DN
if strings.Contains(err.Error(), "User DN not found for:") {
if strings.Contains(strings.ToLower(err.Error()), "user dn not found for:") {
if isDN {
// warn user that DNs are not allowed
writeErrorResponseJSON(ctx, w, errorCodes.ToAPIErrWithErr(ErrAdminLDAPExpectedLoginName, err), r.URL)
+1 -1
View File
@@ -177,7 +177,7 @@ func formatGetBackendErasureVersion(b []byte) (string, error) {
return "", fmt.Errorf(`format.Version expected: %s, got: %s`, formatMetaVersionV1, meta.Version)
}
if meta.Format != formatBackendErasure && meta.Format != formatBackendErasureSingle {
return "", fmt.Errorf(`found backend type %s, expected %s or %s - to migrate to a supported backend visit https://docs.min.io/community/minio-object-store/operations/deployments/baremetal-migrate-fs-gateway.html`, meta.Format, formatBackendErasure, formatBackendErasureSingle)
return "", fmt.Errorf(`found backend type %s, expected %s or %s - to migrate to a supported backend visit https://silo.pigsty.io/operations/deployments/baremetal-migrate-fs-gateway.html`, meta.Format, formatBackendErasure, formatBackendErasureSingle)
}
// Erasure backend found, proceed to detect version.
format := &formatErasureVersionDetect{}
+13
View File
@@ -192,6 +192,16 @@ func extractMetadata(ctx context.Context, mimesHeader ...textproto.MIMEHeader) (
// extractMetadata extracts metadata from map values.
func extractMetadataFromMime(ctx context.Context, v textproto.MIMEHeader, m map[string]string) error {
return extractMetadataFromMimeWithReplication(ctx, v, m, false)
}
// extractReplicationMetadataFromMime restores replication-only metadata after the
// caller has validated that the request is a trusted replication write.
func extractReplicationMetadataFromMime(ctx context.Context, v textproto.MIMEHeader, m map[string]string) error {
return extractMetadataFromMimeWithReplication(ctx, v, m, true)
}
func extractMetadataFromMimeWithReplication(ctx context.Context, v textproto.MIMEHeader, m map[string]string, allowReplication bool) error {
if v == nil {
bugLogIf(ctx, errInvalidArgument)
return errInvalidArgument
@@ -208,6 +218,9 @@ func extractMetadataFromMime(ctx context.Context, v textproto.MIMEHeader, m map[
value, ok := nv[http.CanonicalHeaderKey(supportedHeader)]
if ok {
if v, ok := replicationToInternalHeaders[supportedHeader]; ok {
if !allowReplication {
continue
}
m[v] = strings.Join(value, ",")
} else {
m[supportedHeader] = strings.Join(value, ",")
+116
View File
@@ -24,11 +24,13 @@ import (
"io"
"net/http"
"net/textproto"
"net/url"
"os"
"reflect"
"testing"
"github.com/minio/minio/internal/config"
xhttp "github.com/minio/minio/internal/http"
)
// Tests validate bucket LocationConstraint.
@@ -152,6 +154,22 @@ func TestExtractMetadataHeaders(t *testing.T) {
},
shouldFail: false,
},
// Replication-only headers must not be accepted on ordinary requests.
{
header: http.Header{
"Content-Type": []string{"image/png"},
"X-Minio-Replication-Server-Side-Encryption-Sealed-Key": []string{"sealed-key"},
"X-Minio-Replication-Server-Side-Encryption-Seal-Algorithm": []string{"DAREv2-HMAC-SHA256"},
"X-Minio-Replication-Server-Side-Encryption-Iv": []string{"iv"},
"X-Minio-Replication-Encrypted-Multipart": []string{""},
"X-Minio-Replication-Actual-Object-Size": []string{"1"},
ReplicationSsecChecksumHeader: []string{"checksum"},
},
metadata: map[string]string{
"content-type": "image/png",
},
shouldFail: false,
},
// Empty header input returns empty metadata.
{
header: nil,
@@ -176,6 +194,104 @@ func TestExtractMetadataHeaders(t *testing.T) {
}
}
func TestExtractReplicationMetadataHeaders(t *testing.T) {
header := http.Header{
"X-Minio-Replication-Server-Side-Encryption-Sealed-Key": []string{"sealed-key"},
"X-Minio-Replication-Server-Side-Encryption-Seal-Algorithm": []string{"DAREv2-HMAC-SHA256"},
"X-Minio-Replication-Server-Side-Encryption-Iv": []string{"iv"},
"X-Minio-Replication-Encrypted-Multipart": []string{""},
"X-Minio-Replication-Actual-Object-Size": []string{"1"},
ReplicationSsecChecksumHeader: []string{"checksum"},
}
metadata := make(map[string]string)
if err := extractReplicationMetadataFromMime(t.Context(), textproto.MIMEHeader(header), metadata); err != nil {
t.Fatalf("failed to extract replication metadata: %v", err)
}
expected := map[string]string{
"X-Minio-Internal-Server-Side-Encryption-Sealed-Key": "sealed-key",
"X-Minio-Internal-Server-Side-Encryption-Seal-Algorithm": "DAREv2-HMAC-SHA256",
"X-Minio-Internal-Server-Side-Encryption-Iv": "iv",
"X-Minio-Internal-Encrypted-Multipart": "",
"X-Minio-Internal-Actual-Object-Size": "1",
ReplicationSsecChecksumHeader: "checksum",
}
if !reflect.DeepEqual(metadata, expected) {
t.Fatalf("unexpected replication metadata: expected %#v, got %#v", expected, metadata)
}
}
func TestGetCopyObjectMetadataFromHeaderReplication(t *testing.T) {
req, err := http.NewRequest(http.MethodPut, "http://localhost/test", nil)
if err != nil {
t.Fatal(err)
}
req.Form = make(url.Values)
req.Header.Set("X-Amz-Metadata-Directive", replaceDirective)
req.Header.Set("X-Minio-Replication-Server-Side-Encryption-Sealed-Key", "sealed-key")
metadata, err := getCpObjMetadataFromHeader(t.Context(), req, nil, false)
if err != nil {
t.Fatalf("copy metadata extraction failed: %v", err)
}
if _, ok := metadata["X-Minio-Internal-Server-Side-Encryption-Sealed-Key"]; ok {
t.Fatalf("unexpected replication metadata without validation: %#v", metadata)
}
metadata, err = getCpObjMetadataFromHeader(t.Context(), req, nil, true)
if err != nil {
t.Fatalf("copy metadata extraction with replication failed: %v", err)
}
if got := metadata["X-Minio-Internal-Server-Side-Encryption-Sealed-Key"]; got != "sealed-key" {
t.Fatalf("expected restored replication metadata, got %#v", metadata)
}
}
func TestCloneRequestWithoutCopyReplicationHeaders(t *testing.T) {
req, err := http.NewRequest(http.MethodPut, "http://localhost/test", nil)
if err != nil {
t.Fatal(err)
}
req.Header.Set(xhttp.MinIOSourceReplicationRequest, "true")
req.Header.Set(xhttp.MinIOSourceETag, "etag")
req.Header.Set(xhttp.MinIOSourceMTime, "2026-04-15T10:00:00Z")
req.Header.Set(xhttp.MinIOSourceTaggingTimestamp, "2026-04-15T10:00:00Z")
req.Header.Set(xhttp.MinIOSourceObjectRetentionTimestamp, "2026-04-15T10:00:00Z")
req.Header.Set(xhttp.MinIOSourceObjectLegalHoldTimestamp, "2026-04-15T10:00:00Z")
req.Header.Set(xhttp.MinIOReplicationActualObjectSize, "123")
req.Header.Set(ReplicationSsecChecksumHeader, "checksum")
req.Header.Set("Content-Type", "application/octet-stream")
clone := cloneRequestWithoutCopyReplicationHeaders(req)
if clone == req {
t.Fatal("expected cloned request")
}
for _, header := range []string{
xhttp.MinIOSourceReplicationRequest,
xhttp.MinIOSourceETag,
xhttp.MinIOSourceMTime,
xhttp.MinIOSourceTaggingTimestamp,
xhttp.MinIOSourceObjectRetentionTimestamp,
xhttp.MinIOSourceObjectLegalHoldTimestamp,
xhttp.MinIOReplicationActualObjectSize,
ReplicationSsecChecksumHeader,
} {
if got := clone.Header.Get(header); got != "" {
t.Fatalf("expected %s to be stripped, got %q", header, got)
}
if got := req.Header.Get(header); got == "" {
t.Fatalf("expected original request to preserve %s", header)
}
}
if got := clone.Header.Get("Content-Type"); got != "application/octet-stream" {
t.Fatalf("expected non-replication headers to be preserved, got %q", got)
}
}
// Test getResource()
func TestGetResource(t *testing.T) {
testCases := []struct {
-8
View File
@@ -324,14 +324,6 @@ func (d *naughtyDisk) StatInfoFile(ctx context.Context, volume, path string, glo
return d.disk.StatInfoFile(ctx, volume, path, glob)
}
func (d *naughtyDisk) ReadMultiple(ctx context.Context, req ReadMultipleReq, resp chan<- ReadMultipleResp) error {
if err := d.calcError(); err != nil {
close(resp)
return err
}
return d.disk.ReadMultiple(ctx, req, resp)
}
func (d *naughtyDisk) CleanAbandonedData(ctx context.Context, volume string, path string) error {
if err := d.calcError(); err != nil {
return err
+75 -11
View File
@@ -1036,7 +1036,7 @@ func (api objectAPIHandlers) HeadObjectHandler(w http.ResponseWriter, r *http.Re
// Extract metadata relevant for an CopyObject operation based on conditional
// header values specified in X-Amz-Metadata-Directive.
func getCpObjMetadataFromHeader(ctx context.Context, r *http.Request, userMeta map[string]string) (map[string]string, error) {
func getCpObjMetadataFromHeader(ctx context.Context, r *http.Request, userMeta map[string]string, allowReplication bool) (map[string]string, error) {
// Make a copy of the supplied metadata to avoid
// to change the original one.
defaultMeta := make(map[string]string, len(userMeta))
@@ -1067,6 +1067,11 @@ func getCpObjMetadataFromHeader(ctx context.Context, r *http.Request, userMeta m
if err != nil {
return nil, err
}
if allowReplication {
if err = extractReplicationMetadataFromMime(ctx, textproto.MIMEHeader(r.Header), emetadata); err != nil {
return nil, err
}
}
if sc != "" {
emetadata[xhttp.AmzStorageClass] = sc
}
@@ -1087,6 +1092,31 @@ func getCpObjMetadataFromHeader(ctx context.Context, r *http.Request, userMeta m
return defaultMeta, nil
}
func cloneRequestWithoutCopyReplicationHeaders(r *http.Request) *http.Request {
if r == nil {
return nil
}
clone := new(http.Request)
*clone = *r
clone.Header = r.Header.Clone()
for _, header := range []string{
xhttp.MinIOSourceReplicationRequest,
xhttp.MinIOSourceETag,
xhttp.MinIOSourceMTime,
xhttp.MinIOSourceTaggingTimestamp,
xhttp.MinIOSourceObjectRetentionTimestamp,
xhttp.MinIOSourceObjectLegalHoldTimestamp,
xhttp.MinIOReplicationActualObjectSize,
ReplicationSsecChecksumHeader,
} {
clone.Header.Del(header)
}
return clone
}
// getRemoteInstanceTransport contains a roundtripper for external (not peers) servers
var remoteInstanceTransport atomic.Value
@@ -1232,6 +1262,19 @@ func (api objectAPIHandlers) CopyObjectHandler(w http.ResponseWriter, r *http.Re
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(ErrInvalidStorageClass), r.URL)
return
}
allowReplicationMetadata := false
if r.Header.Get(xhttp.AmzBucketReplicationStatus) == replication.Replica.String() {
if s3Error := checkRequestAuthType(ctx, r, policy.ReplicateObjectAction, dstBucket, dstObject); s3Error != ErrNone {
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(s3Error), r.URL)
return
}
allowReplicationMetadata = true
}
trustedReplicationRequest := allowReplicationMetadata && r.Header.Get(xhttp.MinIOSourceReplicationRequest) == "true"
optsReq := r
if !trustedReplicationRequest {
optsReq = cloneRequestWithoutCopyReplicationHeaders(r)
}
// Check if bucket encryption is enabled
sseConfig, _ := globalBucketSSEConfigSys.Get(dstBucket)
@@ -1240,7 +1283,7 @@ func (api objectAPIHandlers) CopyObjectHandler(w http.ResponseWriter, r *http.Re
})
var srcOpts, dstOpts ObjectOptions
srcOpts, err = copySrcOpts(ctx, r, srcBucket, srcObject)
srcOpts, err = copySrcOpts(ctx, optsReq, srcBucket, srcObject)
if err != nil {
writeErrorResponse(ctx, w, toAPIError(ctx, err), r.URL)
return
@@ -1252,14 +1295,14 @@ func (api objectAPIHandlers) CopyObjectHandler(w http.ResponseWriter, r *http.Re
VersionID: srcOpts.VersionID,
Versioned: srcOpts.Versioned,
VersionSuspended: srcOpts.VersionSuspended,
ReplicationRequest: r.Header.Get(xhttp.MinIOSourceReplicationRequest) == "true",
ReplicationRequest: trustedReplicationRequest,
}
getSSE := encrypt.SSE(srcOpts.ServerSideEncryption)
if getSSE != srcOpts.ServerSideEncryption {
getOpts.ServerSideEncryption = getSSE
}
dstOpts, err = copyDstOpts(ctx, r, dstBucket, dstObject, nil)
dstOpts, err = copyDstOpts(ctx, optsReq, dstBucket, dstObject, nil)
if err != nil {
writeErrorResponse(ctx, w, toAPIError(ctx, err), r.URL)
return
@@ -1269,7 +1312,7 @@ func (api objectAPIHandlers) CopyObjectHandler(w http.ResponseWriter, r *http.Re
getObjectNInfo := objectAPI.GetObjectNInfo
checkCopyPrecondFn := func(o ObjectInfo) bool {
if _, err := DecryptObjectInfo(&o, r); err != nil {
if _, err := DecryptObjectInfo(&o, optsReq); err != nil {
writeErrorResponse(ctx, w, toAPIError(ctx, err), r.URL)
return true
}
@@ -1380,7 +1423,7 @@ func (api objectAPIHandlers) CopyObjectHandler(w http.ResponseWriter, r *http.Re
return
}
// Encryption parameters not present for this object.
if crypto.SSEC.IsEncrypted(srcInfo.UserDefined) && !crypto.SSECopy.IsRequested(r.Header) && r.Header.Get(xhttp.MinIOSourceReplicationRequest) != "true" {
if crypto.SSEC.IsEncrypted(srcInfo.UserDefined) && !crypto.SSECopy.IsRequested(r.Header) && !trustedReplicationRequest {
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(ErrInvalidSSECustomerAlgorithm), r.URL)
return
}
@@ -1546,7 +1589,7 @@ func (api objectAPIHandlers) CopyObjectHandler(w http.ResponseWriter, r *http.Re
srcInfo.PutObjReader = pReader
srcInfo.UserDefined, err = getCpObjMetadataFromHeader(ctx, r, srcInfo.UserDefined)
srcInfo.UserDefined, err = getCpObjMetadataFromHeader(ctx, r, srcInfo.UserDefined, allowReplicationMetadata)
if err != nil {
writeErrorResponse(ctx, w, toAPIError(ctx, err), r.URL)
return
@@ -1628,10 +1671,10 @@ func (api objectAPIHandlers) CopyObjectHandler(w http.ResponseWriter, r *http.Re
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(s3Err), r.URL)
return
}
if rs := r.Header.Get(xhttp.AmzBucketReplicationStatus); rs != "" {
if allowReplicationMetadata {
srcInfo.UserDefined[ReservedMetadataPrefixLower+ReplicaStatus] = replication.Replica.String()
srcInfo.UserDefined[ReservedMetadataPrefixLower+ReplicaTimestamp] = UTCNow().Format(time.RFC3339Nano)
srcInfo.UserDefined[xhttp.AmzBucketReplicationStatus] = rs
srcInfo.UserDefined[xhttp.AmzBucketReplicationStatus] = replication.Replica.String()
}
op := replication.ObjectReplicationType
@@ -1896,7 +1939,7 @@ func (api objectAPIHandlers) PutObjectHandler(w http.ResponseWriter, r *http.Req
}
case authTypeStreamingUnsignedTrailer:
// Initialize stream chunked reader with optional trailers.
rd, s3Err = newUnsignedV4ChunkedReader(r, true, r.Header.Get(xhttp.Authorization) != "")
rd, s3Err = newUnsignedV4ChunkedReader(r, true)
if s3Err != ErrNone {
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(s3Err), r.URL)
return
@@ -1933,6 +1976,10 @@ func (api objectAPIHandlers) PutObjectHandler(w http.ResponseWriter, r *http.Req
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(s3Err), r.URL)
return
}
if err = extractReplicationMetadataFromMime(ctx, textproto.MIMEHeader(r.Header), metadata); err != nil {
writeErrorResponse(ctx, w, toAPIError(ctx, err), r.URL)
return
}
metadata[ReservedMetadataPrefixLower+ReplicaStatus] = replication.Replica.String()
metadata[ReservedMetadataPrefixLower+ReplicaTimestamp] = UTCNow().Format(time.RFC3339Nano)
defer globalReplicationStats.Load().UpdateReplicaStat(bucket, size)
@@ -2242,7 +2289,7 @@ func (api objectAPIHandlers) PutObjectExtractHandler(w http.ResponseWriter, r *h
// if Content-Length is unknown/missing, deny the request
size := r.ContentLength
rAuthType := getRequestAuthType(r)
if rAuthType == authTypeStreamingSigned || rAuthType == authTypeStreamingSignedTrailer {
if rAuthType == authTypeStreamingSigned || rAuthType == authTypeStreamingSignedTrailer || rAuthType == authTypeStreamingUnsignedTrailer {
if sizeStr, ok := r.Header[xhttp.AmzDecodedContentLength]; ok {
if sizeStr[0] == "" {
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(ErrMissingContentLength), r.URL)
@@ -2296,6 +2343,13 @@ func (api objectAPIHandlers) PutObjectExtractHandler(w http.ResponseWriter, r *h
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(s3Err), r.URL)
return
}
case authTypeStreamingUnsignedTrailer:
// Initialize stream chunked reader with optional trailers.
reader, s3Err = newUnsignedV4ChunkedReader(r, true)
if s3Err != ErrNone {
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(s3Err), r.URL)
return
}
case authTypeSignedV2, authTypePresignedV2:
s3Err = isReqAuthenticatedV2(r)
if s3Err != ErrNone {
@@ -2388,10 +2442,15 @@ func (api objectAPIHandlers) PutObjectExtractHandler(w http.ResponseWriter, r *h
rawReader := hashReader
pReader := NewPutObjReader(rawReader)
allowReplicationMetadata := false
if r.Header.Get(xhttp.AmzBucketReplicationStatus) == replication.Replica.String() {
if s3Err = isPutActionAllowed(ctx, getRequestAuthType(r), bucket, object, r, policy.ReplicateObjectAction); s3Err != ErrNone {
return errors.New(errorCodes.ToAPIErr(s3Err).Code)
}
allowReplicationMetadata = true
if err = extractReplicationMetadataFromMime(ctx, textproto.MIMEHeader(r.Header), metadata); err != nil {
return err
}
metadata[ReservedMetadataPrefixLower+ReplicaStatus] = replication.Replica.String()
metadata[ReservedMetadataPrefixLower+ReplicaTimestamp] = UTCNow().Format(time.RFC3339Nano)
}
@@ -2417,6 +2476,11 @@ func (api objectAPIHandlers) PutObjectExtractHandler(w http.ResponseWriter, r *h
if err != nil {
return err
}
if allowReplicationMetadata {
if err = extractReplicationMetadataFromMime(ctx, textproto.MIMEHeader(hdrs), m); err != nil {
return err
}
}
maps.Copy(metadata, m)
} else {
versionID = r.Form.Get(xhttp.VersionID)
+157
View File
@@ -42,6 +42,7 @@ import (
"github.com/dustin/go-humanize"
"github.com/minio/minio/internal/auth"
"github.com/minio/minio/internal/crypto"
"github.com/minio/minio/internal/hash/sha256"
xhttp "github.com/minio/minio/internal/http"
ioutilx "github.com/minio/minio/internal/ioutil"
@@ -60,6 +61,47 @@ const (
MissingUploadID
)
func replicationSSEPoisonHeaders() map[string]string {
return map[string]string{
"X-Minio-Replication-Server-Side-Encryption-Sealed-Key": base64.StdEncoding.EncodeToString(make([]byte, 64)),
"X-Minio-Replication-Server-Side-Encryption-Seal-Algorithm": crypto.SealAlgorithm,
"X-Minio-Replication-Server-Side-Encryption-Iv": base64.StdEncoding.EncodeToString(make([]byte, 32)),
}
}
func assertObjectMetadataKeysAbsent(t *testing.T, metadata map[string]string, keys ...string) {
t.Helper()
for _, key := range keys {
if got, ok := metadata[key]; ok {
t.Fatalf("expected metadata %q to be absent, got %q", key, got)
}
}
}
func assertObjectMetadataValueNotEqual(t *testing.T, metadata map[string]string, key, unexpected string) {
t.Helper()
if got := metadata[key]; got == unexpected {
t.Fatalf("expected metadata %q to differ from %q", key, unexpected)
}
}
func assertObjectContents(t *testing.T, obj ObjectLayer, bucketName, objectName string, expected []byte) {
t.Helper()
reader, err := obj.GetObjectNInfo(context.Background(), bucketName, objectName, nil, nil, ObjectOptions{})
if err != nil {
t.Fatalf("failed to fetch object %s/%s: %v", bucketName, objectName, err)
}
defer reader.Close()
got, err := io.ReadAll(reader)
if err != nil {
t.Fatalf("failed to read object %s/%s: %v", bucketName, objectName, err)
}
if !bytes.Equal(got, expected) {
t.Fatalf("unexpected object contents: got %d bytes, expected %d bytes", len(got), len(expected))
}
}
// Wrapper for calling HeadObject API handler tests for both Erasure multiple disks and FS single drive setup.
func TestAPIHeadObjectHandler(t *testing.T) {
ExecObjectLayerAPITest(ExecObjectLayerAPITestArgs{t: t, objAPITest: testAPIHeadObjectHandler, endpoints: []string{"HeadObject"}})
@@ -1819,6 +1861,53 @@ func testAPICopyObjectPartHandlerSanity(obj ObjectLayer, instanceType, bucketNam
}
}
func TestAPIPutObjectReplicationHeaderPoisoning(t *testing.T) {
defer DetectTestLeak(t)()
ExecExtendedObjectLayerAPITest(t, testAPIPutObjectReplicationHeaderPoisoning, []string{"PutObject"})
}
func testAPIPutObjectReplicationHeaderPoisoning(obj ObjectLayer, instanceType, bucketName string, apiRouter http.Handler,
credentials auth.Credentials, t *testing.T,
) {
objectName := "replication-header-poison-put"
payload := []byte("replication-header-poison-put-payload")
headers := replicationSSEPoisonHeaders()
req, err := newTestSignedRequestV4(
http.MethodPut,
getPutObjectURL("", bucketName, objectName),
int64(len(payload)),
bytes.NewReader(payload),
credentials.AccessKey,
credentials.SecretKey,
headers,
)
if err != nil {
t.Fatalf("%s: failed to create signed put request: %v", instanceType, err)
}
rec := httptest.NewRecorder()
apiRouter.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("%s: expected put to succeed, got %d", instanceType, rec.Code)
}
objInfo, err := obj.GetObjectInfo(context.Background(), bucketName, objectName, ObjectOptions{})
if err != nil {
t.Fatalf("%s: failed to fetch object info: %v", instanceType, err)
}
assertObjectMetadataValueNotEqual(t, objInfo.UserDefined,
crypto.MetaSealedKeySSEC,
headers["X-Minio-Replication-Server-Side-Encryption-Sealed-Key"],
)
assertObjectMetadataValueNotEqual(t, objInfo.UserDefined,
crypto.MetaIV,
headers["X-Minio-Replication-Server-Side-Encryption-Iv"],
)
assertObjectContents(t, obj, bucketName, objectName, payload)
}
// Wrapper for calling Copy Object Part API handler tests for both Erasure multiple disks and single node setup.
func TestAPICopyObjectPartHandler(t *testing.T) {
defer DetectTestLeak(t)()
@@ -2860,6 +2949,74 @@ func testAPINewMultipartHandlerParallel(obj ObjectLayer, instanceType, bucketNam
}
}
func TestAPICopyObjectReplicationHeaderPoisoning(t *testing.T) {
defer DetectTestLeak(t)()
ExecExtendedObjectLayerAPITest(t, testAPICopyObjectReplicationHeaderPoisoning, []string{"CopyObject", "PutObject"})
}
func testAPICopyObjectReplicationHeaderPoisoning(obj ObjectLayer, instanceType, bucketName string, apiRouter http.Handler,
credentials auth.Credentials, t *testing.T,
) {
srcObject := "replication-header-poison-copy-src"
dstObject := "replication-header-poison-copy-dst"
payload := []byte("replication-header-poison-copy-payload")
if _, err := obj.PutObject(
context.Background(),
bucketName,
srcObject,
mustGetPutObjReader(t, bytes.NewReader(payload), int64(len(payload)), "", ""),
ObjectOptions{},
); err != nil {
t.Fatalf("%s: failed to create source object: %v", instanceType, err)
}
headers := replicationSSEPoisonHeaders()
headers[xhttp.AmzCopySource] = url.QueryEscape(SlashSeparator + bucketName + SlashSeparator + srcObject)
headers[xhttp.AmzMetadataDirective] = replaceDirective
headers[xhttp.AmzBucketReplicationStatus] = "PENDING"
headers["Content-Type"] = "application/octet-stream"
req, err := newTestSignedRequestV4(
http.MethodPut,
getCopyObjectURL("", bucketName, dstObject),
0,
nil,
credentials.AccessKey,
credentials.SecretKey,
headers,
)
if err != nil {
t.Fatalf("%s: failed to create signed copy request: %v", instanceType, err)
}
rec := httptest.NewRecorder()
apiRouter.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("%s: expected copy to succeed, got %d", instanceType, rec.Code)
}
objInfo, err := obj.GetObjectInfo(context.Background(), bucketName, dstObject, ObjectOptions{})
if err != nil {
t.Fatalf("%s: failed to fetch copied object info: %v", instanceType, err)
}
assertObjectMetadataValueNotEqual(t, objInfo.UserDefined,
crypto.MetaSealedKeySSEC,
headers["X-Minio-Replication-Server-Side-Encryption-Sealed-Key"],
)
assertObjectMetadataValueNotEqual(t, objInfo.UserDefined,
crypto.MetaIV,
headers["X-Minio-Replication-Server-Side-Encryption-Iv"],
)
assertObjectMetadataKeysAbsent(t, objInfo.UserDefined,
xhttp.AmzBucketReplicationStatus,
ReservedMetadataPrefixLower+ReplicaStatus,
ReservedMetadataPrefixLower+ReplicaTimestamp,
)
assertObjectContents(t, obj, bucketName, dstObject, payload)
}
// The UploadID from the response body is parsed and its existence is asserted with an attempt to ListParts using it.
func TestAPICompleteMultipartHandler(t *testing.T) {
defer DetectTestLeak(t)()
+11 -2
View File
@@ -24,6 +24,7 @@ import (
"io"
"maps"
"net/http"
"net/textproto"
"net/url"
"sort"
"strconv"
@@ -157,6 +158,14 @@ func (api objectAPIHandlers) NewMultipartUploadHandler(w http.ResponseWriter, r
metadata[xhttp.AmzObjectTagging] = objTags
}
if r.Header.Get(xhttp.AmzBucketReplicationStatus) == replication.Replica.String() {
if s3Err := isPutActionAllowed(ctx, getRequestAuthType(r), bucket, object, r, policy.ReplicateObjectAction); s3Err != ErrNone {
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(s3Err), r.URL)
return
}
if err = extractReplicationMetadataFromMime(ctx, textproto.MIMEHeader(r.Header), metadata); err != nil {
writeErrorResponse(ctx, w, toAPIError(ctx, err), r.URL)
return
}
metadata[ReservedMetadataPrefixLower+ReplicaStatus] = replication.Replica.String()
metadata[ReservedMetadataPrefixLower+ReplicaTimestamp] = UTCNow().Format(time.RFC3339Nano)
}
@@ -685,8 +694,8 @@ func (api objectAPIHandlers) PutObjectPartHandler(w http.ResponseWriter, r *http
return
}
case authTypeStreamingUnsignedTrailer:
// Initialize stream signature verifier.
reader, s3Error = newUnsignedV4ChunkedReader(r, true, r.Header.Get(xhttp.Authorization) != "")
// Initialize stream chunked reader with optional trailers.
reader, s3Error = newUnsignedV4ChunkedReader(r, true)
if s3Error != ErrNone {
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(s3Error), r.URL)
return
+2 -2
View File
@@ -144,7 +144,7 @@ func printServerCommonMsg(apiEndpoints []string) {
// Prints startup message for Object API access, prints link to our SDK documentation.
func printObjectAPIMsg() {
logger.Startup(color.Blue("\nDocs: ") + "https://docs.min.io")
logger.Startup(color.Blue("\nDocs: ") + "https://silo.pigsty.io")
}
func printLambdaTargets() {
@@ -184,7 +184,7 @@ func printCLIAccessMsg(endPoint string, alias string) {
// Get saved credentials.
cred := globalActiveCred
const mcQuickStartGuide = "https://docs.min.io/community/minio-object-store/reference/minio-mc.html#quickstart"
const mcQuickStartGuide = "https://silo.pigsty.io/reference/minio-mc.html#quickstart"
// Configure 'mc', following block prints platform specific information for minio client.
if color.IsTerminal() && (!globalServerCtxt.Anonymous && globalAPIConfig.permitRootAccess()) {
+380 -15
View File
@@ -18,10 +18,15 @@
package cmd
import (
"archive/tar"
"bytes"
"context"
"encoding/base64"
"encoding/binary"
"encoding/json"
"encoding/xml"
"fmt"
"hash/crc32"
"io"
"math/rand"
"net/http"
@@ -128,6 +133,13 @@ func runAllTests(suite *TestSuiteCommon, c *check) {
suite.TestBucketSQSNotificationWebHook(c)
suite.TestBucketSQSNotificationAMQP(c)
suite.TestUnsignedCVE(c)
suite.TestUnsignedQueryStringCVE(c)
suite.TestUnsignedQueryStringCVEMultipart(c)
suite.TestUnsignedTrailerRejectsMultipleAuthSources(c)
suite.TestUnsignedTrailerSnowballAnonymousDenied(c)
suite.TestUnsignedTrailerSnowballRequiresSignature(c)
suite.TestUnsignedTrailerSnowballExtract(c)
suite.TestAnonymousUnsignedTrailer(c)
suite.TearDownSuite(c)
}
@@ -374,24 +386,13 @@ func (s *TestSuiteCommon) TestUnsignedCVE(c *check) {
// assert the http response status code.
c.Assert(response.StatusCode, http.StatusOK)
req, err := http.NewRequest(http.MethodPut, getPutObjectURL(s.endPoint, bucketName, "test-cve-object.txt"), nil)
now := UTCNow()
req, err := newStreamingUnsignedTrailerRequest(http.MethodPut, getPutObjectURL(s.endPoint, bucketName, "test-cve-object.txt"), []byte("foobar!\n"), now)
c.Assert(err, nil)
req.Body = io.NopCloser(bytes.NewReader([]byte("foobar!\n")))
req.Trailer = http.Header{}
req.Trailer.Set("x-amz-checksum-crc32", "rK0DXg==")
now := UTCNow()
req = signer.StreamingUnsignedV4(req, "", 8, now)
maliciousHeaders := http.Header{
"Authorization": []string{fmt.Sprintf("AWS4-HMAC-SHA256 Credential=%s/%s/us-east-1/s3/aws4_request, SignedHeaders=invalidheader, Signature=deadbeefdeadbeefdeadbeeddeadbeeddeadbeefdeadbeefdeadbeefdeadbeef", s.accessKey, now.Format(yyyymmdd))},
"User-Agent": []string{"A malicious request"},
"X-Amz-Decoded-Content-Length": []string{"8"},
"Content-Encoding": []string{"aws-chunked"},
"X-Amz-Trailer": []string{"x-amz-checksum-crc32"},
"x-amz-content-sha256": []string{unsignedPayloadTrailer},
"Authorization": []string{fmt.Sprintf("AWS4-HMAC-SHA256 Credential=%s/%s/us-east-1/s3/aws4_request, SignedHeaders=invalidheader, Signature=deadbeefdeadbeefdeadbeeddeadbeeddeadbeefdeadbeefdeadbeefdeadbeef", s.accessKey, now.Format(yyyymmdd))},
"User-Agent": []string{"A malicious request"},
}
for k, v := range maliciousHeaders {
@@ -409,6 +410,370 @@ func (s *TestSuiteCommon) TestUnsignedCVE(c *check) {
c.Assert(response.StatusCode, http.StatusBadRequest)
}
func (s *TestSuiteCommon) TestUnsignedQueryStringCVE(c *check) {
c.Helper()
// generate a random bucket Name.
bucketName := getRandomBucketName()
// HTTP request to create the bucket.
request, err := newTestSignedRequest(http.MethodPut, getMakeBucketURL(s.endPoint, bucketName),
0, nil, s.accessKey, s.secretKey, s.signer)
c.Assert(err, nil)
// execute the request.
response, err := s.client.Do(request)
c.Assert(err, nil)
// assert the http response status code.
c.Assert(response.StatusCode, http.StatusOK)
now := UTCNow()
req, err := newStreamingUnsignedTrailerRequest(http.MethodPut, getPutObjectURL(s.endPoint, bucketName, "test-cve-presigned-object.txt"), []byte("foobar!\n"), now)
c.Assert(err, nil)
err = presignStreamingUnsignedTrailerRequest(req, s.accessKey, s.secretKey, 60, now)
c.Assert(err, nil)
// execute the request.
response, err = s.client.Do(req)
c.Assert(err, nil)
verifyError(c, response, "InvalidRequest", "The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.", http.StatusBadRequest)
}
func (s *TestSuiteCommon) TestUnsignedQueryStringCVEMultipart(c *check) {
c.Helper()
bucketName := getRandomBucketName()
request, err := newTestSignedRequest(http.MethodPut, getMakeBucketURL(s.endPoint, bucketName),
0, nil, s.accessKey, s.secretKey, s.signer)
c.Assert(err, nil)
response, err := s.client.Do(request)
c.Assert(err, nil)
c.Assert(response.StatusCode, http.StatusOK)
uploadID := s.mustStartMultipartUpload(c, bucketName, "test-cve-presigned-multipart.txt")
now := UTCNow()
req, err := newStreamingUnsignedTrailerRequest(http.MethodPut, getPartUploadURL(s.endPoint, bucketName, "test-cve-presigned-multipart.txt", uploadID, "1"), []byte("foobar!\n"), now)
c.Assert(err, nil)
err = presignStreamingUnsignedTrailerRequest(req, s.accessKey, s.secretKey, 60, now)
c.Assert(err, nil)
response, err = s.client.Do(req)
c.Assert(err, nil)
verifyError(c, response, "InvalidRequest", "The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.", http.StatusBadRequest)
}
func (s *TestSuiteCommon) TestUnsignedTrailerRejectsMultipleAuthSources(c *check) {
c.Helper()
bucketName := getRandomBucketName()
request, err := newTestSignedRequest(http.MethodPut, getMakeBucketURL(s.endPoint, bucketName),
0, nil, s.accessKey, s.secretKey, s.signer)
c.Assert(err, nil)
response, err := s.client.Do(request)
c.Assert(err, nil)
c.Assert(response.StatusCode, http.StatusOK)
now := UTCNow()
req, err := newStreamingUnsignedTrailerRequest(http.MethodPut, getPutObjectURL(s.endPoint, bucketName, "test-cve-mixed-auth.txt"), []byte("foobar!\n"), now)
c.Assert(err, nil)
err = presignStreamingUnsignedTrailerRequest(req, s.accessKey, s.secretKey, 60, now)
c.Assert(err, nil)
err = signRequestV4(req, s.accessKey, s.secretKey)
c.Assert(err, nil)
response, err = s.client.Do(req)
c.Assert(err, nil)
verifyError(c, response, "InvalidRequest", "Invalid Request (request has multiple authentication types, please use one)", http.StatusBadRequest)
}
func (s *TestSuiteCommon) TestUnsignedTrailerSnowballRequiresSignature(c *check) {
c.Helper()
bucketName := getRandomBucketName()
objectName := "snowball-upload.tar"
extractedObject := "payload.txt"
extractedData := []byte("snowball object\n")
request, err := newTestSignedRequest(http.MethodPut, getMakeBucketURL(s.endPoint, bucketName),
0, nil, s.accessKey, s.secretKey, s.signer)
c.Assert(err, nil)
response, err := s.client.Do(request)
c.Assert(err, nil)
c.Assert(response.StatusCode, http.StatusOK)
tarData, err := newTestTarArchive(extractedObject, extractedData)
c.Assert(err, nil)
// Deliberately send raw tar bytes: before the fix this path never
// initialized the unsigned-trailer reader, so Snowball accepted the
// body without chunked decoding or signature verification.
req, err := http.NewRequest(http.MethodPut, getPutObjectURL(s.endPoint, bucketName, objectName), bytes.NewReader(tarData))
c.Assert(err, nil)
req.ContentLength = int64(len(tarData))
req.Header.Set(xhttp.AmzSnowballExtract, "true")
req.Header.Set(xhttp.AmzContentSha256, unsignedPayloadTrailer)
err = signRequestV4(req, s.accessKey, s.secretKey)
c.Assert(err, nil)
req.Header.Set("Authorization",
regexp.MustCompile(`Signature=[0-9a-f]+`).ReplaceAllString(req.Header.Get("Authorization"), "Signature="+strings.Repeat("0", 64)))
response, err = s.client.Do(req)
c.Assert(err, nil)
verifyError(c, response, "SignatureDoesNotMatch", "The request signature we calculated does not match the signature you provided. Check your key and signing method.", http.StatusForbidden)
}
func (s *TestSuiteCommon) TestUnsignedTrailerSnowballAnonymousDenied(c *check) {
c.Helper()
bucketName := getRandomBucketName()
objectName := "snowball-upload.tar"
request, err := newTestSignedRequest(http.MethodPut, getMakeBucketURL(s.endPoint, bucketName),
0, nil, s.accessKey, s.secretKey, s.signer)
c.Assert(err, nil)
response, err := s.client.Do(request)
c.Assert(err, nil)
c.Assert(response.StatusCode, http.StatusOK)
tarData, err := newTestTarArchive("payload.txt", []byte("snowball object\n"))
c.Assert(err, nil)
req, err := http.NewRequest(http.MethodPut, getPutObjectURL(s.endPoint, bucketName, objectName), bytes.NewReader(tarData))
c.Assert(err, nil)
req.ContentLength = int64(len(tarData))
req.Header.Set(xhttp.AmzSnowballExtract, "true")
req.Header.Set(xhttp.AmzContentSha256, unsignedPayloadTrailer)
response, err = s.client.Do(req)
c.Assert(err, nil)
verifyError(c, response, "AccessDenied", "Access Denied.", http.StatusForbidden)
}
func (s *TestSuiteCommon) TestUnsignedTrailerSnowballExtract(c *check) {
c.Helper()
bucketName := getRandomBucketName()
objectName := "snowball-upload.tar"
extractedObject := "payload.txt"
extractedData := []byte("snowball object\n")
request, err := newTestSignedRequest(http.MethodPut, getMakeBucketURL(s.endPoint, bucketName),
0, nil, s.accessKey, s.secretKey, s.signer)
c.Assert(err, nil)
response, err := s.client.Do(request)
c.Assert(err, nil)
c.Assert(response.StatusCode, http.StatusOK)
tarData, err := newTestTarArchive(extractedObject, extractedData)
c.Assert(err, nil)
req, err := newStreamingUnsignedTrailerRequest(http.MethodPut, getPutObjectURL(s.endPoint, bucketName, objectName), tarData, UTCNow())
c.Assert(err, nil)
req.Header.Set(xhttp.AmzSnowballExtract, "true")
err = signRequestV4(req, s.accessKey, s.secretKey)
c.Assert(err, nil)
response, err = s.client.Do(req)
c.Assert(err, nil)
c.Assert(response.StatusCode, http.StatusOK)
request, err = newTestSignedRequest(http.MethodGet, getGetObjectURL(s.endPoint, bucketName, extractedObject),
0, nil, s.accessKey, s.secretKey, s.signer)
c.Assert(err, nil)
response, err = s.client.Do(request)
c.Assert(err, nil)
c.Assert(response.StatusCode, http.StatusOK)
got, err := io.ReadAll(response.Body)
c.Assert(err, nil)
c.Assert(bytes.Equal(got, extractedData), true)
}
func (s *TestSuiteCommon) TestAnonymousUnsignedTrailer(c *check) {
c.Helper()
bucketName := getRandomBucketName()
objectName := "test-anonymous-unsigned-trailer.txt"
objectData := []byte("foobar!\n")
request, err := newTestSignedRequest(http.MethodPut, getMakeBucketURL(s.endPoint, bucketName),
0, nil, s.accessKey, s.secretKey, s.signer)
c.Assert(err, nil)
response, err := s.client.Do(request)
c.Assert(err, nil)
c.Assert(response.StatusCode, http.StatusOK)
s.mustPutBucketPolicy(c, bucketName, getAnonWriteOnlyObjectPolicy(bucketName, objectName))
req, err := newStreamingUnsignedTrailerRequest(http.MethodPut, getPutObjectURL(s.endPoint, bucketName, objectName), objectData, UTCNow())
c.Assert(err, nil)
response, err = s.client.Do(req)
c.Assert(err, nil)
c.Assert(response.StatusCode, http.StatusOK)
request, err = newTestSignedRequest(http.MethodGet, getGetObjectURL(s.endPoint, bucketName, objectName),
0, nil, s.accessKey, s.secretKey, s.signer)
c.Assert(err, nil)
response, err = s.client.Do(request)
c.Assert(err, nil)
c.Assert(response.StatusCode, http.StatusOK)
got, err := io.ReadAll(response.Body)
c.Assert(err, nil)
c.Assert(bytes.Equal(got, objectData), true)
}
func newTestTarArchive(objectName string, objectData []byte) ([]byte, error) {
var buf bytes.Buffer
tw := tar.NewWriter(&buf)
err := tw.WriteHeader(&tar.Header{
Name: objectName,
Mode: 0o600,
Size: int64(len(objectData)),
})
if err != nil {
return nil, err
}
if _, err = tw.Write(objectData); err != nil {
return nil, err
}
if err = tw.Close(); err != nil {
return nil, err
}
return buf.Bytes(), nil
}
func newStreamingUnsignedTrailerRequest(method, targetURL string, data []byte, reqTime time.Time) (*http.Request, error) {
req, err := http.NewRequest(method, targetURL, nil)
if err != nil {
return nil, err
}
req.Body = io.NopCloser(bytes.NewReader(data))
req.Trailer = http.Header{}
req.Trailer.Set("x-amz-checksum-crc32", crc32Base64(data))
req = signer.StreamingUnsignedV4(req, "", int64(len(data)), reqTime)
req.Header.Set("X-Amz-Decoded-Content-Length", fmt.Sprintf("%d", len(data)))
req.Header.Set("Content-Encoding", "aws-chunked")
req.Header.Set("X-Amz-Trailer", "x-amz-checksum-crc32")
req.Header.Set(xhttp.AmzContentSha256, unsignedPayloadTrailer)
return req, nil
}
func crc32Base64(data []byte) string {
var crc [4]byte
binary.BigEndian.PutUint32(crc[:], crc32.ChecksumIEEE(data))
return base64.StdEncoding.EncodeToString(crc[:])
}
func presignStreamingUnsignedTrailerRequest(req *http.Request, accessKeyID, secretAccessKey string, expires int64, reqTime time.Time) error {
if accessKeyID == "" || secretAccessKey == "" {
return fmt.Errorf("presign cannot be generated without access and secret keys")
}
signedHeaders := []string{
"content-encoding",
"host",
"x-amz-content-sha256",
"x-amz-decoded-content-length",
"x-amz-trailer",
}
region := globalSite.Region()
scope := getScope(reqTime, region)
credential := fmt.Sprintf("%s/%s", accessKeyID, scope)
query := req.URL.Query()
query.Set(xhttp.AmzAlgorithm, signV4Algorithm)
query.Set(xhttp.AmzDate, reqTime.Format(iso8601Format))
query.Set(xhttp.AmzExpires, fmt.Sprintf("%d", expires))
query.Set(xhttp.AmzSignedHeaders, strings.Join(signedHeaders, ";"))
query.Set(xhttp.AmzCredential, credential)
query.Set(xhttp.AmzContentSha256, unsignedPayloadTrailer)
req.Form = query
extractedSignedHeaders, errCode := extractSignedHeaders(signedHeaders, req)
if errCode != ErrNone {
return fmt.Errorf("extractSignedHeaders failed: %v", errCode)
}
queryStr := strings.ReplaceAll(query.Encode(), "+", "%20")
canonicalRequest := getCanonicalRequest(extractedSignedHeaders, unsignedPayloadTrailer, queryStr, req.URL.Path, req.Method)
stringToSign := getStringToSign(canonicalRequest, reqTime, scope)
signingKey := getSigningKey(secretAccessKey, reqTime, region, serviceS3)
signature := getSignature(signingKey, stringToSign)
query.Set(xhttp.AmzSignature, signature)
req.URL.RawQuery = query.Encode()
req.Form = req.URL.Query()
return nil
}
func (s *TestSuiteCommon) mustPutBucketPolicy(c *check, bucketName string, bucketPolicy *policy.BucketPolicy) {
c.Helper()
policyBytes, err := json.Marshal(bucketPolicy)
c.Assert(err, nil)
request, err := newTestSignedRequest(http.MethodPut, getPutPolicyURL(s.endPoint, bucketName),
int64(len(policyBytes)), bytes.NewReader(policyBytes), s.accessKey, s.secretKey, s.signer)
c.Assert(err, nil)
response, err := s.client.Do(request)
c.Assert(err, nil)
c.Assert(response.StatusCode, http.StatusNoContent)
}
func (s *TestSuiteCommon) mustStartMultipartUpload(c *check, bucketName, objectName string) string {
c.Helper()
request, err := newTestSignedRequest(http.MethodPost, getNewMultipartURL(s.endPoint, bucketName, objectName),
0, nil, s.accessKey, s.secretKey, s.signer)
c.Assert(err, nil)
response, err := s.client.Do(request)
c.Assert(err, nil)
c.Assert(response.StatusCode, http.StatusOK)
decoder := xml.NewDecoder(response.Body)
newResponse := &InitiateMultipartUploadResponse{}
err = decoder.Decode(newResponse)
c.Assert(err, nil)
c.Assert(len(newResponse.UploadID) > 0, true)
return newResponse.UploadID
}
func (s *TestSuiteCommon) TestBucketSQSNotificationAMQP(c *check) {
// Sample bucket notification.
bucketNotificationBuf := `<NotificationConfiguration><QueueConfiguration><Event>s3:ObjectCreated:Put</Event><Filter><S3Key><FilterRule><Name>prefix</Name><Value>images/</Value></FilterRule></S3Key></Filter><Id>1</Id><Queue>arn:minio:sqs:us-east-1:444455556666:amqp</Queue></QueueConfiguration></NotificationConfiguration>`
-22
View File
@@ -396,28 +396,6 @@ func newFileInfo(object string, dataBlocks, parityBlocks int) (fi FileInfo) {
return fi
}
// ReadMultipleReq contains information of multiple files to read from disk.
type ReadMultipleReq struct {
Bucket string `msg:"bk"` // Bucket. Can be empty if multiple buckets.
Prefix string `msg:"pr,omitempty"` // Shared prefix of all files. Can be empty. Will be joined to filename without modification.
Files []string `msg:"fl"` // Individual files to read.
MaxSize int64 `msg:"ms"` // Return error if size is exceed.
MetadataOnly bool `msg:"mo"` // Read as XL meta and truncate data.
AbortOn404 bool `msg:"ab"` // Stop reading after first file not found.
MaxResults int `msg:"mr"` // Stop after this many successful results. <= 0 means all.
}
// ReadMultipleResp contains a single response from a ReadMultipleReq.
type ReadMultipleResp struct {
Bucket string `msg:"bk"` // Bucket as given by request.
Prefix string `msg:"pr,omitempty"` // Prefix as given by request.
File string `msg:"fl"` // File name as given in request.
Exists bool `msg:"ex"` // Returns whether the file existed on disk.
Error string `msg:"er,omitempty"` // Returns any error when reading.
Data []byte `msg:"d"` // Contains all data of file.
Modtime time.Time `msg:"m"` // Modtime of file on disk.
}
// DeleteVersionHandlerParams are parameters for DeleteVersionHandler
type DeleteVersionHandlerParams struct {
DiskID string `msg:"id"`
-666
View File
@@ -4234,672 +4234,6 @@ func (z ReadAllHandlerParams) Msgsize() (s int) {
return
}
// DecodeMsg implements msgp.Decodable
func (z *ReadMultipleReq) DecodeMsg(dc *msgp.Reader) (err error) {
var field []byte
_ = field
var zb0001 uint32
zb0001, err = dc.ReadMapHeader()
if err != nil {
err = msgp.WrapError(err)
return
}
var zb0001Mask uint8 /* 1 bits */
_ = zb0001Mask
for zb0001 > 0 {
zb0001--
field, err = dc.ReadMapKeyPtr()
if err != nil {
err = msgp.WrapError(err)
return
}
switch msgp.UnsafeString(field) {
case "bk":
z.Bucket, err = dc.ReadString()
if err != nil {
err = msgp.WrapError(err, "Bucket")
return
}
case "pr":
z.Prefix, err = dc.ReadString()
if err != nil {
err = msgp.WrapError(err, "Prefix")
return
}
zb0001Mask |= 0x1
case "fl":
var zb0002 uint32
zb0002, err = dc.ReadArrayHeader()
if err != nil {
err = msgp.WrapError(err, "Files")
return
}
if cap(z.Files) >= int(zb0002) {
z.Files = (z.Files)[:zb0002]
} else {
z.Files = make([]string, zb0002)
}
for za0001 := range z.Files {
z.Files[za0001], err = dc.ReadString()
if err != nil {
err = msgp.WrapError(err, "Files", za0001)
return
}
}
case "ms":
z.MaxSize, err = dc.ReadInt64()
if err != nil {
err = msgp.WrapError(err, "MaxSize")
return
}
case "mo":
z.MetadataOnly, err = dc.ReadBool()
if err != nil {
err = msgp.WrapError(err, "MetadataOnly")
return
}
case "ab":
z.AbortOn404, err = dc.ReadBool()
if err != nil {
err = msgp.WrapError(err, "AbortOn404")
return
}
case "mr":
z.MaxResults, err = dc.ReadInt()
if err != nil {
err = msgp.WrapError(err, "MaxResults")
return
}
default:
err = dc.Skip()
if err != nil {
err = msgp.WrapError(err)
return
}
}
}
// Clear omitted fields.
if (zb0001Mask & 0x1) == 0 {
z.Prefix = ""
}
return
}
// EncodeMsg implements msgp.Encodable
func (z *ReadMultipleReq) EncodeMsg(en *msgp.Writer) (err error) {
// check for omitted fields
zb0001Len := uint32(7)
var zb0001Mask uint8 /* 7 bits */
_ = zb0001Mask
if z.Prefix == "" {
zb0001Len--
zb0001Mask |= 0x2
}
// variable map header, size zb0001Len
err = en.Append(0x80 | uint8(zb0001Len))
if err != nil {
return
}
// skip if no fields are to be emitted
if zb0001Len != 0 {
// write "bk"
err = en.Append(0xa2, 0x62, 0x6b)
if err != nil {
return
}
err = en.WriteString(z.Bucket)
if err != nil {
err = msgp.WrapError(err, "Bucket")
return
}
if (zb0001Mask & 0x2) == 0 { // if not omitted
// write "pr"
err = en.Append(0xa2, 0x70, 0x72)
if err != nil {
return
}
err = en.WriteString(z.Prefix)
if err != nil {
err = msgp.WrapError(err, "Prefix")
return
}
}
// write "fl"
err = en.Append(0xa2, 0x66, 0x6c)
if err != nil {
return
}
err = en.WriteArrayHeader(uint32(len(z.Files)))
if err != nil {
err = msgp.WrapError(err, "Files")
return
}
for za0001 := range z.Files {
err = en.WriteString(z.Files[za0001])
if err != nil {
err = msgp.WrapError(err, "Files", za0001)
return
}
}
// write "ms"
err = en.Append(0xa2, 0x6d, 0x73)
if err != nil {
return
}
err = en.WriteInt64(z.MaxSize)
if err != nil {
err = msgp.WrapError(err, "MaxSize")
return
}
// write "mo"
err = en.Append(0xa2, 0x6d, 0x6f)
if err != nil {
return
}
err = en.WriteBool(z.MetadataOnly)
if err != nil {
err = msgp.WrapError(err, "MetadataOnly")
return
}
// write "ab"
err = en.Append(0xa2, 0x61, 0x62)
if err != nil {
return
}
err = en.WriteBool(z.AbortOn404)
if err != nil {
err = msgp.WrapError(err, "AbortOn404")
return
}
// write "mr"
err = en.Append(0xa2, 0x6d, 0x72)
if err != nil {
return
}
err = en.WriteInt(z.MaxResults)
if err != nil {
err = msgp.WrapError(err, "MaxResults")
return
}
}
return
}
// MarshalMsg implements msgp.Marshaler
func (z *ReadMultipleReq) MarshalMsg(b []byte) (o []byte, err error) {
o = msgp.Require(b, z.Msgsize())
// check for omitted fields
zb0001Len := uint32(7)
var zb0001Mask uint8 /* 7 bits */
_ = zb0001Mask
if z.Prefix == "" {
zb0001Len--
zb0001Mask |= 0x2
}
// variable map header, size zb0001Len
o = append(o, 0x80|uint8(zb0001Len))
// skip if no fields are to be emitted
if zb0001Len != 0 {
// string "bk"
o = append(o, 0xa2, 0x62, 0x6b)
o = msgp.AppendString(o, z.Bucket)
if (zb0001Mask & 0x2) == 0 { // if not omitted
// string "pr"
o = append(o, 0xa2, 0x70, 0x72)
o = msgp.AppendString(o, z.Prefix)
}
// string "fl"
o = append(o, 0xa2, 0x66, 0x6c)
o = msgp.AppendArrayHeader(o, uint32(len(z.Files)))
for za0001 := range z.Files {
o = msgp.AppendString(o, z.Files[za0001])
}
// string "ms"
o = append(o, 0xa2, 0x6d, 0x73)
o = msgp.AppendInt64(o, z.MaxSize)
// string "mo"
o = append(o, 0xa2, 0x6d, 0x6f)
o = msgp.AppendBool(o, z.MetadataOnly)
// string "ab"
o = append(o, 0xa2, 0x61, 0x62)
o = msgp.AppendBool(o, z.AbortOn404)
// string "mr"
o = append(o, 0xa2, 0x6d, 0x72)
o = msgp.AppendInt(o, z.MaxResults)
}
return
}
// UnmarshalMsg implements msgp.Unmarshaler
func (z *ReadMultipleReq) UnmarshalMsg(bts []byte) (o []byte, err error) {
var field []byte
_ = field
var zb0001 uint32
zb0001, bts, err = msgp.ReadMapHeaderBytes(bts)
if err != nil {
err = msgp.WrapError(err)
return
}
var zb0001Mask uint8 /* 1 bits */
_ = zb0001Mask
for zb0001 > 0 {
zb0001--
field, bts, err = msgp.ReadMapKeyZC(bts)
if err != nil {
err = msgp.WrapError(err)
return
}
switch msgp.UnsafeString(field) {
case "bk":
z.Bucket, bts, err = msgp.ReadStringBytes(bts)
if err != nil {
err = msgp.WrapError(err, "Bucket")
return
}
case "pr":
z.Prefix, bts, err = msgp.ReadStringBytes(bts)
if err != nil {
err = msgp.WrapError(err, "Prefix")
return
}
zb0001Mask |= 0x1
case "fl":
var zb0002 uint32
zb0002, bts, err = msgp.ReadArrayHeaderBytes(bts)
if err != nil {
err = msgp.WrapError(err, "Files")
return
}
if cap(z.Files) >= int(zb0002) {
z.Files = (z.Files)[:zb0002]
} else {
z.Files = make([]string, zb0002)
}
for za0001 := range z.Files {
z.Files[za0001], bts, err = msgp.ReadStringBytes(bts)
if err != nil {
err = msgp.WrapError(err, "Files", za0001)
return
}
}
case "ms":
z.MaxSize, bts, err = msgp.ReadInt64Bytes(bts)
if err != nil {
err = msgp.WrapError(err, "MaxSize")
return
}
case "mo":
z.MetadataOnly, bts, err = msgp.ReadBoolBytes(bts)
if err != nil {
err = msgp.WrapError(err, "MetadataOnly")
return
}
case "ab":
z.AbortOn404, bts, err = msgp.ReadBoolBytes(bts)
if err != nil {
err = msgp.WrapError(err, "AbortOn404")
return
}
case "mr":
z.MaxResults, bts, err = msgp.ReadIntBytes(bts)
if err != nil {
err = msgp.WrapError(err, "MaxResults")
return
}
default:
bts, err = msgp.Skip(bts)
if err != nil {
err = msgp.WrapError(err)
return
}
}
}
// Clear omitted fields.
if (zb0001Mask & 0x1) == 0 {
z.Prefix = ""
}
o = bts
return
}
// Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message
func (z *ReadMultipleReq) Msgsize() (s int) {
s = 1 + 3 + msgp.StringPrefixSize + len(z.Bucket) + 3 + msgp.StringPrefixSize + len(z.Prefix) + 3 + msgp.ArrayHeaderSize
for za0001 := range z.Files {
s += msgp.StringPrefixSize + len(z.Files[za0001])
}
s += 3 + msgp.Int64Size + 3 + msgp.BoolSize + 3 + msgp.BoolSize + 3 + msgp.IntSize
return
}
// DecodeMsg implements msgp.Decodable
func (z *ReadMultipleResp) DecodeMsg(dc *msgp.Reader) (err error) {
var field []byte
_ = field
var zb0001 uint32
zb0001, err = dc.ReadMapHeader()
if err != nil {
err = msgp.WrapError(err)
return
}
var zb0001Mask uint8 /* 2 bits */
_ = zb0001Mask
for zb0001 > 0 {
zb0001--
field, err = dc.ReadMapKeyPtr()
if err != nil {
err = msgp.WrapError(err)
return
}
switch msgp.UnsafeString(field) {
case "bk":
z.Bucket, err = dc.ReadString()
if err != nil {
err = msgp.WrapError(err, "Bucket")
return
}
case "pr":
z.Prefix, err = dc.ReadString()
if err != nil {
err = msgp.WrapError(err, "Prefix")
return
}
zb0001Mask |= 0x1
case "fl":
z.File, err = dc.ReadString()
if err != nil {
err = msgp.WrapError(err, "File")
return
}
case "ex":
z.Exists, err = dc.ReadBool()
if err != nil {
err = msgp.WrapError(err, "Exists")
return
}
case "er":
z.Error, err = dc.ReadString()
if err != nil {
err = msgp.WrapError(err, "Error")
return
}
zb0001Mask |= 0x2
case "d":
z.Data, err = dc.ReadBytes(z.Data)
if err != nil {
err = msgp.WrapError(err, "Data")
return
}
case "m":
z.Modtime, err = dc.ReadTime()
if err != nil {
err = msgp.WrapError(err, "Modtime")
return
}
default:
err = dc.Skip()
if err != nil {
err = msgp.WrapError(err)
return
}
}
}
// Clear omitted fields.
if zb0001Mask != 0x3 {
if (zb0001Mask & 0x1) == 0 {
z.Prefix = ""
}
if (zb0001Mask & 0x2) == 0 {
z.Error = ""
}
}
return
}
// EncodeMsg implements msgp.Encodable
func (z *ReadMultipleResp) EncodeMsg(en *msgp.Writer) (err error) {
// check for omitted fields
zb0001Len := uint32(7)
var zb0001Mask uint8 /* 7 bits */
_ = zb0001Mask
if z.Prefix == "" {
zb0001Len--
zb0001Mask |= 0x2
}
if z.Error == "" {
zb0001Len--
zb0001Mask |= 0x10
}
// variable map header, size zb0001Len
err = en.Append(0x80 | uint8(zb0001Len))
if err != nil {
return
}
// skip if no fields are to be emitted
if zb0001Len != 0 {
// write "bk"
err = en.Append(0xa2, 0x62, 0x6b)
if err != nil {
return
}
err = en.WriteString(z.Bucket)
if err != nil {
err = msgp.WrapError(err, "Bucket")
return
}
if (zb0001Mask & 0x2) == 0 { // if not omitted
// write "pr"
err = en.Append(0xa2, 0x70, 0x72)
if err != nil {
return
}
err = en.WriteString(z.Prefix)
if err != nil {
err = msgp.WrapError(err, "Prefix")
return
}
}
// write "fl"
err = en.Append(0xa2, 0x66, 0x6c)
if err != nil {
return
}
err = en.WriteString(z.File)
if err != nil {
err = msgp.WrapError(err, "File")
return
}
// write "ex"
err = en.Append(0xa2, 0x65, 0x78)
if err != nil {
return
}
err = en.WriteBool(z.Exists)
if err != nil {
err = msgp.WrapError(err, "Exists")
return
}
if (zb0001Mask & 0x10) == 0 { // if not omitted
// write "er"
err = en.Append(0xa2, 0x65, 0x72)
if err != nil {
return
}
err = en.WriteString(z.Error)
if err != nil {
err = msgp.WrapError(err, "Error")
return
}
}
// write "d"
err = en.Append(0xa1, 0x64)
if err != nil {
return
}
err = en.WriteBytes(z.Data)
if err != nil {
err = msgp.WrapError(err, "Data")
return
}
// write "m"
err = en.Append(0xa1, 0x6d)
if err != nil {
return
}
err = en.WriteTime(z.Modtime)
if err != nil {
err = msgp.WrapError(err, "Modtime")
return
}
}
return
}
// MarshalMsg implements msgp.Marshaler
func (z *ReadMultipleResp) MarshalMsg(b []byte) (o []byte, err error) {
o = msgp.Require(b, z.Msgsize())
// check for omitted fields
zb0001Len := uint32(7)
var zb0001Mask uint8 /* 7 bits */
_ = zb0001Mask
if z.Prefix == "" {
zb0001Len--
zb0001Mask |= 0x2
}
if z.Error == "" {
zb0001Len--
zb0001Mask |= 0x10
}
// variable map header, size zb0001Len
o = append(o, 0x80|uint8(zb0001Len))
// skip if no fields are to be emitted
if zb0001Len != 0 {
// string "bk"
o = append(o, 0xa2, 0x62, 0x6b)
o = msgp.AppendString(o, z.Bucket)
if (zb0001Mask & 0x2) == 0 { // if not omitted
// string "pr"
o = append(o, 0xa2, 0x70, 0x72)
o = msgp.AppendString(o, z.Prefix)
}
// string "fl"
o = append(o, 0xa2, 0x66, 0x6c)
o = msgp.AppendString(o, z.File)
// string "ex"
o = append(o, 0xa2, 0x65, 0x78)
o = msgp.AppendBool(o, z.Exists)
if (zb0001Mask & 0x10) == 0 { // if not omitted
// string "er"
o = append(o, 0xa2, 0x65, 0x72)
o = msgp.AppendString(o, z.Error)
}
// string "d"
o = append(o, 0xa1, 0x64)
o = msgp.AppendBytes(o, z.Data)
// string "m"
o = append(o, 0xa1, 0x6d)
o = msgp.AppendTime(o, z.Modtime)
}
return
}
// UnmarshalMsg implements msgp.Unmarshaler
func (z *ReadMultipleResp) UnmarshalMsg(bts []byte) (o []byte, err error) {
var field []byte
_ = field
var zb0001 uint32
zb0001, bts, err = msgp.ReadMapHeaderBytes(bts)
if err != nil {
err = msgp.WrapError(err)
return
}
var zb0001Mask uint8 /* 2 bits */
_ = zb0001Mask
for zb0001 > 0 {
zb0001--
field, bts, err = msgp.ReadMapKeyZC(bts)
if err != nil {
err = msgp.WrapError(err)
return
}
switch msgp.UnsafeString(field) {
case "bk":
z.Bucket, bts, err = msgp.ReadStringBytes(bts)
if err != nil {
err = msgp.WrapError(err, "Bucket")
return
}
case "pr":
z.Prefix, bts, err = msgp.ReadStringBytes(bts)
if err != nil {
err = msgp.WrapError(err, "Prefix")
return
}
zb0001Mask |= 0x1
case "fl":
z.File, bts, err = msgp.ReadStringBytes(bts)
if err != nil {
err = msgp.WrapError(err, "File")
return
}
case "ex":
z.Exists, bts, err = msgp.ReadBoolBytes(bts)
if err != nil {
err = msgp.WrapError(err, "Exists")
return
}
case "er":
z.Error, bts, err = msgp.ReadStringBytes(bts)
if err != nil {
err = msgp.WrapError(err, "Error")
return
}
zb0001Mask |= 0x2
case "d":
z.Data, bts, err = msgp.ReadBytesBytes(bts, z.Data)
if err != nil {
err = msgp.WrapError(err, "Data")
return
}
case "m":
z.Modtime, bts, err = msgp.ReadTimeBytes(bts)
if err != nil {
err = msgp.WrapError(err, "Modtime")
return
}
default:
bts, err = msgp.Skip(bts)
if err != nil {
err = msgp.WrapError(err)
return
}
}
}
// Clear omitted fields.
if zb0001Mask != 0x3 {
if (zb0001Mask & 0x1) == 0 {
z.Prefix = ""
}
if (zb0001Mask & 0x2) == 0 {
z.Error = ""
}
}
o = bts
return
}
// Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message
func (z *ReadMultipleResp) Msgsize() (s int) {
s = 1 + 3 + msgp.StringPrefixSize + len(z.Bucket) + 3 + msgp.StringPrefixSize + len(z.Prefix) + 3 + msgp.StringPrefixSize + len(z.File) + 3 + msgp.BoolSize + 3 + msgp.StringPrefixSize + len(z.Error) + 2 + msgp.BytesPrefixSize + len(z.Data) + 2 + msgp.TimeSize
return
}
// DecodeMsg implements msgp.Decodable
func (z *ReadPartsReq) DecodeMsg(dc *msgp.Reader) (err error) {
var field []byte
-226
View File
@@ -2156,232 +2156,6 @@ func BenchmarkDecodeReadAllHandlerParams(b *testing.B) {
}
}
func TestMarshalUnmarshalReadMultipleReq(t *testing.T) {
v := ReadMultipleReq{}
bts, err := v.MarshalMsg(nil)
if err != nil {
t.Fatal(err)
}
left, err := v.UnmarshalMsg(bts)
if err != nil {
t.Fatal(err)
}
if len(left) > 0 {
t.Errorf("%d bytes left over after UnmarshalMsg(): %q", len(left), left)
}
left, err = msgp.Skip(bts)
if err != nil {
t.Fatal(err)
}
if len(left) > 0 {
t.Errorf("%d bytes left over after Skip(): %q", len(left), left)
}
}
func BenchmarkMarshalMsgReadMultipleReq(b *testing.B) {
v := ReadMultipleReq{}
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
v.MarshalMsg(nil)
}
}
func BenchmarkAppendMsgReadMultipleReq(b *testing.B) {
v := ReadMultipleReq{}
bts := make([]byte, 0, v.Msgsize())
bts, _ = v.MarshalMsg(bts[0:0])
b.SetBytes(int64(len(bts)))
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
bts, _ = v.MarshalMsg(bts[0:0])
}
}
func BenchmarkUnmarshalReadMultipleReq(b *testing.B) {
v := ReadMultipleReq{}
bts, _ := v.MarshalMsg(nil)
b.ReportAllocs()
b.SetBytes(int64(len(bts)))
b.ResetTimer()
for i := 0; i < b.N; i++ {
_, err := v.UnmarshalMsg(bts)
if err != nil {
b.Fatal(err)
}
}
}
func TestEncodeDecodeReadMultipleReq(t *testing.T) {
v := ReadMultipleReq{}
var buf bytes.Buffer
msgp.Encode(&buf, &v)
m := v.Msgsize()
if buf.Len() > m {
t.Log("WARNING: TestEncodeDecodeReadMultipleReq Msgsize() is inaccurate")
}
vn := ReadMultipleReq{}
err := msgp.Decode(&buf, &vn)
if err != nil {
t.Error(err)
}
buf.Reset()
msgp.Encode(&buf, &v)
err = msgp.NewReader(&buf).Skip()
if err != nil {
t.Error(err)
}
}
func BenchmarkEncodeReadMultipleReq(b *testing.B) {
v := ReadMultipleReq{}
var buf bytes.Buffer
msgp.Encode(&buf, &v)
b.SetBytes(int64(buf.Len()))
en := msgp.NewWriter(msgp.Nowhere)
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
v.EncodeMsg(en)
}
en.Flush()
}
func BenchmarkDecodeReadMultipleReq(b *testing.B) {
v := ReadMultipleReq{}
var buf bytes.Buffer
msgp.Encode(&buf, &v)
b.SetBytes(int64(buf.Len()))
rd := msgp.NewEndlessReader(buf.Bytes(), b)
dc := msgp.NewReader(rd)
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
err := v.DecodeMsg(dc)
if err != nil {
b.Fatal(err)
}
}
}
func TestMarshalUnmarshalReadMultipleResp(t *testing.T) {
v := ReadMultipleResp{}
bts, err := v.MarshalMsg(nil)
if err != nil {
t.Fatal(err)
}
left, err := v.UnmarshalMsg(bts)
if err != nil {
t.Fatal(err)
}
if len(left) > 0 {
t.Errorf("%d bytes left over after UnmarshalMsg(): %q", len(left), left)
}
left, err = msgp.Skip(bts)
if err != nil {
t.Fatal(err)
}
if len(left) > 0 {
t.Errorf("%d bytes left over after Skip(): %q", len(left), left)
}
}
func BenchmarkMarshalMsgReadMultipleResp(b *testing.B) {
v := ReadMultipleResp{}
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
v.MarshalMsg(nil)
}
}
func BenchmarkAppendMsgReadMultipleResp(b *testing.B) {
v := ReadMultipleResp{}
bts := make([]byte, 0, v.Msgsize())
bts, _ = v.MarshalMsg(bts[0:0])
b.SetBytes(int64(len(bts)))
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
bts, _ = v.MarshalMsg(bts[0:0])
}
}
func BenchmarkUnmarshalReadMultipleResp(b *testing.B) {
v := ReadMultipleResp{}
bts, _ := v.MarshalMsg(nil)
b.ReportAllocs()
b.SetBytes(int64(len(bts)))
b.ResetTimer()
for i := 0; i < b.N; i++ {
_, err := v.UnmarshalMsg(bts)
if err != nil {
b.Fatal(err)
}
}
}
func TestEncodeDecodeReadMultipleResp(t *testing.T) {
v := ReadMultipleResp{}
var buf bytes.Buffer
msgp.Encode(&buf, &v)
m := v.Msgsize()
if buf.Len() > m {
t.Log("WARNING: TestEncodeDecodeReadMultipleResp Msgsize() is inaccurate")
}
vn := ReadMultipleResp{}
err := msgp.Decode(&buf, &vn)
if err != nil {
t.Error(err)
}
buf.Reset()
msgp.Encode(&buf, &v)
err = msgp.NewReader(&buf).Skip()
if err != nil {
t.Error(err)
}
}
func BenchmarkEncodeReadMultipleResp(b *testing.B) {
v := ReadMultipleResp{}
var buf bytes.Buffer
msgp.Encode(&buf, &v)
b.SetBytes(int64(buf.Len()))
en := msgp.NewWriter(msgp.Nowhere)
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
v.EncodeMsg(en)
}
en.Flush()
}
func BenchmarkDecodeReadMultipleResp(b *testing.B) {
v := ReadMultipleResp{}
var buf bytes.Buffer
msgp.Encode(&buf, &v)
b.SetBytes(int64(buf.Len()))
rd := msgp.NewEndlessReader(buf.Bytes(), b)
dc := msgp.NewReader(rd)
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
err := v.DecodeMsg(dc)
if err != nil {
b.Fatal(err)
}
}
}
func TestMarshalUnmarshalReadPartsReq(t *testing.T) {
v := ReadPartsReq{}
bts, err := v.MarshalMsg(nil)
-1
View File
@@ -101,7 +101,6 @@ type StorageAPI interface {
VerifyFile(ctx context.Context, volume, path string, fi FileInfo) (*CheckPartsResp, error)
StatInfoFile(ctx context.Context, volume, path string, glob bool) (stat []StatInfo, err error)
ReadParts(ctx context.Context, bucket string, partMetaPaths ...string) ([]*ObjectPartInfo, error)
ReadMultiple(ctx context.Context, req ReadMultipleReq, resp chan<- ReadMultipleResp) error
CleanAbandonedData(ctx context.Context, volume string, path string) error
// Write all data, syncs the data to disk.
-39
View File
@@ -912,45 +912,6 @@ func (client *storageRESTClient) StatInfoFile(ctx context.Context, volume, path
return stat, toStorageErr(err)
}
// ReadMultiple will read multiple files and send each back as response.
// Files are read and returned in the given order.
// The resp channel is closed before the call returns.
// Only a canceled context or network errors returns an error.
func (client *storageRESTClient) ReadMultiple(ctx context.Context, req ReadMultipleReq, resp chan<- ReadMultipleResp) error {
defer xioutil.SafeClose(resp)
body, err := req.MarshalMsg(nil)
if err != nil {
return err
}
respBody, err := client.call(ctx, storageRESTMethodReadMultiple, nil, bytes.NewReader(body), int64(len(body)))
if err != nil {
return err
}
defer xhttp.DrainBody(respBody)
pr, pw := io.Pipe()
go func() {
pw.CloseWithError(waitForHTTPStream(respBody, xioutil.NewDeadlineWriter(pw, globalDriveConfig.GetMaxTimeout())))
}()
mr := msgp.NewReader(pr)
defer readMsgpReaderPoolPut(mr)
for {
var file ReadMultipleResp
if err := file.DecodeMsg(mr); err != nil {
if errors.Is(err, io.EOF) {
err = nil
}
pr.CloseWithError(err)
return toStorageErr(err)
}
select {
case <-ctx.Done():
return ctx.Err()
case resp <- file:
}
}
}
// CleanAbandonedData will read metadata of the object on disk
// and delete any data directories and inline data that isn't referenced in metadata.
func (client *storageRESTClient) CleanAbandonedData(ctx context.Context, volume string, path string) error {
-1
View File
@@ -41,7 +41,6 @@ const (
storageRESTMethodRenameFile = "/rfile"
storageRESTMethodVerifyFile = "/vfile"
storageRESTMethodStatInfoFile = "/sfile"
storageRESTMethodReadMultiple = "/rmpl"
storageRESTMethodCleanAbandoned = "/cln"
storageRESTMethodDeleteBulk = "/dblk"
storageRESTMethodReadParts = "/rps"
-44
View File
@@ -28,7 +28,6 @@ import (
"net/http"
"os/user"
"path"
"runtime/debug"
"strconv"
"strings"
"sync"
@@ -1283,48 +1282,6 @@ func (s *storageRESTServer) DeleteBulkHandler(w http.ResponseWriter, r *http.Req
keepHTTPResponseAlive(w)(s.getStorage().DeleteBulk(r.Context(), volume, req.Paths...))
}
// ReadMultiple returns multiple files
func (s *storageRESTServer) ReadMultiple(w http.ResponseWriter, r *http.Request) {
if !s.IsValid(w, r) {
return
}
rw := streamHTTPResponse(w)
defer func() {
if r := recover(); r != nil {
debug.PrintStack()
rw.CloseWithError(fmt.Errorf("panic: %v", r))
}
}()
var req ReadMultipleReq
mr := msgpNewReader(r.Body)
defer readMsgpReaderPoolPut(mr)
err := req.DecodeMsg(mr)
if err != nil {
rw.CloseWithError(err)
return
}
mw := msgp.NewWriter(rw)
responses := make(chan ReadMultipleResp, len(req.Files))
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
for resp := range responses {
err := resp.EncodeMsg(mw)
if err != nil {
rw.CloseWithError(err)
return
}
mw.Flush()
}
}()
err = s.getStorage().ReadMultiple(r.Context(), req, responses)
wg.Wait()
rw.CloseWithError(err)
}
// globalLocalSetDrives is used for local drive as well as remote REST
// API caller for other nodes to talk to this node.
//
@@ -1363,7 +1320,6 @@ func registerStorageRESTHandlers(router *mux.Router, endpointServerPools Endpoin
subrouter.Methods(http.MethodPost).Path(storageRESTVersionPrefix + storageRESTMethodDeleteVersions).HandlerFunc(h(server.DeleteVersionsHandler))
subrouter.Methods(http.MethodPost).Path(storageRESTVersionPrefix + storageRESTMethodVerifyFile).HandlerFunc(h(server.VerifyFileHandler))
subrouter.Methods(http.MethodPost).Path(storageRESTVersionPrefix + storageRESTMethodStatInfoFile).HandlerFunc(h(server.StatInfoFile))
subrouter.Methods(http.MethodPost).Path(storageRESTVersionPrefix + storageRESTMethodReadMultiple).HandlerFunc(h(server.ReadMultiple))
subrouter.Methods(http.MethodPost).Path(storageRESTVersionPrefix + storageRESTMethodCleanAbandoned).HandlerFunc(h(server.CleanAbandonedDataHandler))
subrouter.Methods(http.MethodPost).Path(storageRESTVersionPrefix + storageRESTMethodDeleteBulk).HandlerFunc(h(server.DeleteBulkHandler))
subrouter.Methods(http.MethodPost).Path(storageRESTVersionPrefix + storageRESTMethodReadParts).HandlerFunc(h(server.ReadPartsHandler))
+8 -9
View File
@@ -33,18 +33,17 @@ func _() {
_ = x[storageMetricReadXL-22]
_ = x[storageMetricReadAll-23]
_ = x[storageMetricStatInfoFile-24]
_ = x[storageMetricReadMultiple-25]
_ = x[storageMetricDeleteAbandonedParts-26]
_ = x[storageMetricDiskInfo-27]
_ = x[storageMetricDeleteBulk-28]
_ = x[storageMetricRenamePart-29]
_ = x[storageMetricReadParts-30]
_ = x[storageMetricLast-31]
_ = x[storageMetricDeleteAbandonedParts-25]
_ = x[storageMetricDiskInfo-26]
_ = x[storageMetricDeleteBulk-27]
_ = x[storageMetricRenamePart-28]
_ = x[storageMetricReadParts-29]
_ = x[storageMetricLast-30]
}
const _storageMetric_name = "MakeVolBulkMakeVolListVolsStatVolDeleteVolWalkDirListDirReadFileAppendFileCreateFileReadFileStreamRenameFileRenameDataCheckPartsDeleteDeleteVersionsVerifyFileWriteAllDeleteVersionWriteMetadataUpdateMetadataReadVersionReadXLReadAllStatInfoFileReadMultipleDeleteAbandonedPartsDiskInfoDeleteBulkRenamePartReadPartsLast"
const _storageMetric_name = "MakeVolBulkMakeVolListVolsStatVolDeleteVolWalkDirListDirReadFileAppendFileCreateFileReadFileStreamRenameFileRenameDataCheckPartsDeleteDeleteVersionsVerifyFileWriteAllDeleteVersionWriteMetadataUpdateMetadataReadVersionReadXLReadAllStatInfoFileDeleteAbandonedPartsDiskInfoDeleteBulkRenamePartReadPartsLast"
var _storageMetric_index = [...]uint16{0, 11, 18, 26, 33, 42, 49, 56, 64, 74, 84, 98, 108, 118, 128, 134, 148, 158, 166, 179, 192, 206, 217, 223, 230, 242, 254, 274, 282, 292, 302, 311, 315}
var _storageMetric_index = [...]uint16{0, 11, 18, 26, 33, 42, 49, 56, 64, 74, 84, 98, 108, 118, 128, 134, 148, 158, 166, 179, 192, 206, 217, 223, 230, 242, 262, 270, 280, 290, 299, 303}
func (i storageMetric) String() string {
if i >= storageMetric(len(_storageMetric_index)-1) {
+17 -5
View File
@@ -26,12 +26,24 @@ import (
"strings"
)
// newUnsignedV4ChunkedReader returns a new s3UnsignedChunkedReader that translates the data read from r
// out of HTTP "chunked" format before returning it.
// newUnsignedV4ChunkedReader returns a new s3UnsignedChunkedReader that translates
// the data read from r out of HTTP "chunked" format before returning it.
//
// STREAMING-UNSIGNED-PAYLOAD-TRAILER requests cross the auth boundary here because
// the handler starts consuming the body immediately after this reader is created.
// Header-authenticated requests must therefore be validated before any body bytes
// are read, preserving the security boundary added in #21103, while presigned
// query-string auth is intentionally rejected for this streaming mode.
//
// The s3ChunkedReader returns io.EOF when the final 0-length chunk is read.
func newUnsignedV4ChunkedReader(req *http.Request, trailer bool, signature bool) (io.ReadCloser, APIErrorCode) {
if signature {
if errCode := doesSignatureMatch(unsignedPayloadTrailer, req, globalSite.Region(), serviceS3); errCode != ErrNone {
func newUnsignedV4ChunkedReader(req *http.Request, trailer bool) (io.ReadCloser, APIErrorCode) {
// Reject any request that claims presigned auth parameters are in use, even if
// the query is incomplete, to avoid silently downgrading it into the anonymous path.
if isRequestPresignedSignatureV4(req) {
return nil, ErrSignatureVersionNotSupported
}
if isRequestSignatureV4(req) {
if errCode := reqSignatureV4Verify(req, globalSite.Region(), serviceS3); errCode != ErrNone {
return nil, errCode
}
}
+332 -4
View File
@@ -25,14 +25,17 @@ import (
"encoding/json"
"errors"
"fmt"
"net"
"net/http"
"net/url"
"strconv"
"strings"
"sync"
"time"
"github.com/minio/madmin-go/v3"
"github.com/minio/minio/internal/auth"
idldap "github.com/minio/minio/internal/config/identity/ldap"
"github.com/minio/minio/internal/config/identity/openid"
"github.com/minio/minio/internal/hash/sha256"
xhttp "github.com/minio/minio/internal/http"
@@ -91,6 +94,15 @@ const (
// maximum supported STS session policy size
maxSTSSessionPolicySize = 2048
stsLDAPLoginBurst = 10
stsLDAPLoginEntryTTL = 15 * time.Minute
stsLDAPLoginRetryAfterSec = int((time.Minute / stsLDAPLoginBurst) / time.Second)
)
var (
errLDAPAuthenticationFailed = errors.New("LDAP authentication failed")
globalSTSLDAPLoginRateLimiter = newSTSLDAPLoginRateLimiter(time.Minute/time.Duration(stsLDAPLoginBurst), stsLDAPLoginBurst, stsLDAPLoginEntryTTL)
)
type stsClaims map[string]any
@@ -254,6 +266,308 @@ func getTokenSigningKey() (string, error) {
return secret, nil
}
// stsLDAPLoginRateLimiter throttles LDAP STS logins per source IP. It does not
// bucket by username on purpose: a username-keyed bucket is shared across all
// sources, so a single client sending bad-password attempts for a known account
// could keep that account's bucket drained and lock the legitimate user out.
//
// Scope of protection: the per-source bucket caps the attempt rate from any one
// source, and the uniform auth-failure response (not this limiter) is what hides
// whether a username exists. It does not stop attackers who spread across many
// sources (botnets, IPv6 address rotation) or the residual bind-timing side
// channel; those are accepted limitations of an in-memory, per-source control.
type stsLDAPLoginRateLimiter struct {
source *stsLDAPLoginKeyLimiterSet
}
type stsLDAPLoginReservation struct {
source *stsLDAPLoginKeyReservation
}
type stsLDAPLoginKeyLimiterSet struct {
mu sync.Mutex
refillEvery time.Duration
burst int
ttl time.Duration
lastCleanup time.Time
entries map[string]*stsLDAPLoginKeyLimiter
}
type stsLDAPLoginKeyLimiter struct {
tokens float64
lastRefill time.Time
lastSeen time.Time
inFlight int
}
type stsLDAPLoginKeyReservation struct {
set *stsLDAPLoginKeyLimiterSet
entry *stsLDAPLoginKeyLimiter
finalized bool
}
func newSTSLDAPLoginRateLimiter(refillEvery time.Duration, burst int, ttl time.Duration) *stsLDAPLoginRateLimiter {
return &stsLDAPLoginRateLimiter{
source: newSTSLDAPLoginKeyLimiterSet(refillEvery, burst, ttl),
}
}
func newSTSLDAPLoginKeyLimiterSet(refillEvery time.Duration, burst int, ttl time.Duration) *stsLDAPLoginKeyLimiterSet {
return &stsLDAPLoginKeyLimiterSet{
refillEvery: refillEvery,
burst: burst,
ttl: ttl,
entries: make(map[string]*stsLDAPLoginKeyLimiter),
}
}
func (l *stsLDAPLoginRateLimiter) Allow(sourceIP string) bool {
reservation := l.Reserve(sourceIP)
if reservation == nil {
return false
}
reservation.Commit()
return true
}
func (l *stsLDAPLoginRateLimiter) Reserve(sourceIP string) *stsLDAPLoginReservation {
// An empty source IP means we could not identify the peer; do not throttle
// rather than collapse every such request into one shared bucket.
if sourceIP == "" {
return &stsLDAPLoginReservation{}
}
source := l.source.Reserve(UTCNow(), sourceIP)
if source == nil {
return nil
}
return &stsLDAPLoginReservation{source: source}
}
func (r *stsLDAPLoginReservation) Commit() {
if r == nil || r.source == nil {
return
}
r.source.CommitAt(UTCNow())
r.source = nil
}
func (r *stsLDAPLoginReservation) Cancel() {
if r == nil || r.source == nil {
return
}
r.source.CancelAt(UTCNow())
r.source = nil
}
func (l *stsLDAPLoginKeyLimiterSet) Allow(now time.Time, key string) bool {
reservation := l.Reserve(now, key)
if reservation == nil {
return false
}
reservation.CommitAt(now)
return true
}
func (l *stsLDAPLoginKeyLimiterSet) Reserve(now time.Time, key string) *stsLDAPLoginKeyReservation {
l.mu.Lock()
defer l.mu.Unlock()
if l.lastCleanup.IsZero() || now.Sub(l.lastCleanup) >= l.ttl {
l.cleanup(now)
l.lastCleanup = now
}
entry := l.getOrCreateLocked(now, key)
l.refillLocked(now, entry)
if entry.tokens < 1 {
entry.lastSeen = now
return nil
}
entry.tokens--
entry.inFlight++
entry.lastSeen = now
return &stsLDAPLoginKeyReservation{set: l, entry: entry}
}
func (l *stsLDAPLoginKeyLimiterSet) cleanup(now time.Time) {
for key, entry := range l.entries {
if entry.inFlight == 0 && now.Sub(entry.lastSeen) > l.ttl {
delete(l.entries, key)
}
}
}
func (l *stsLDAPLoginKeyLimiterSet) getOrCreateLocked(now time.Time, key string) *stsLDAPLoginKeyLimiter {
entry, ok := l.entries[key]
if !ok {
entry = &stsLDAPLoginKeyLimiter{
tokens: float64(l.burst),
lastRefill: now,
lastSeen: now,
}
l.entries[key] = entry
}
return entry
}
func (l *stsLDAPLoginKeyLimiterSet) refillLocked(now time.Time, entry *stsLDAPLoginKeyLimiter) {
if entry.lastRefill.IsZero() {
entry.lastRefill = now
}
lastRefill := entry.lastRefill
if now.Before(lastRefill) {
lastRefill = now
}
if l.refillEvery <= 0 {
entry.lastRefill = now
entry.tokens = l.maxTokensLocked(entry)
return
}
entry.tokens += float64(now.Sub(lastRefill)) / float64(l.refillEvery)
if maxTokens := l.maxTokensLocked(entry); entry.tokens > maxTokens {
entry.tokens = maxTokens
}
entry.lastRefill = now
}
func (l *stsLDAPLoginKeyLimiterSet) maxTokensLocked(entry *stsLDAPLoginKeyLimiter) float64 {
maxTokens := l.burst - entry.inFlight
if maxTokens < 0 {
return 0
}
return float64(maxTokens)
}
func (r *stsLDAPLoginKeyReservation) CommitAt(now time.Time) {
r.finalize(now, false)
}
func (r *stsLDAPLoginKeyReservation) CancelAt(now time.Time) {
r.finalize(now, true)
}
func (r *stsLDAPLoginKeyReservation) finalize(now time.Time, refund bool) {
if r == nil {
return
}
r.set.mu.Lock()
defer r.set.mu.Unlock()
if r.finalized {
return
}
r.set.refillLocked(now, r.entry)
r.entry.lastSeen = now
if r.entry.inFlight > 0 {
r.entry.inFlight--
}
if refund {
r.entry.tokens++
if maxTokens := r.set.maxTokensLocked(r.entry); r.entry.tokens > maxTokens {
r.entry.tokens = maxTokens
}
}
r.finalized = true
}
func getSTSLDAPLoginSourceIP(r *http.Request) string {
peerIP := getSTSLDAPLoginCanonicalIP(r.RemoteAddr)
sourceIP := peerIP
if sourceIP == "" {
sourceIP = getSTSLDAPLoginPeerAddr(r.RemoteAddr)
}
if peerIP != "" && globalIAMSys != nil && globalIAMSys.LDAPConfig.IsSTSTrustedProxy(peerIP) {
if forwardedIP := getSTSLDAPTrustedProxySourceIP(r); forwardedIP != "" {
return forwardedIP
}
}
return sourceIP
}
// getSTSLDAPTrustedProxySourceIP resolves the client IP for a request whose peer
// is an allow-listed trusted proxy. A single clean X-Real-IP is preferred; for
// X-Forwarded-For we walk the chain right-to-left and skip trusted-proxy hops,
// returning the first untrusted address. The XFF result ignores any client-
// supplied (left-most) value unless the entire chain to its right is trusted,
// which an external client cannot forge.
//
// X-Real-IP, unlike XFF, is a single value with no chain, so it cannot be
// validated against the allowlist: it is trusted verbatim. The deployment
// contract is therefore that the trusted proxy MUST overwrite (not pass through)
// any client-supplied X-Real-IP; otherwise an attacker can vary it per request
// to evade per-source throttling. This is the standard reverse-proxy real-IP
// contract; reordering to prefer XFF would not remove the dependency, only move
// it (an X-Real-IP-only proxy would then be evaded via an injected XFF header).
//
// The RFC 7239 Forwarded header is intentionally not honored here; such
// deployments fall back to the safe peer-address bucket.
func getSTSLDAPTrustedProxySourceIP(r *http.Request) string {
if realIP := getSTSLDAPLoginCanonicalIP(r.Header.Get("X-Real-IP")); realIP != "" {
return realIP
}
forwarded := strings.Split(r.Header.Get("X-Forwarded-For"), ",")
for i := len(forwarded) - 1; i >= 0; i-- {
ip := getSTSLDAPLoginCanonicalIP(forwarded[i])
if ip == "" || globalIAMSys.LDAPConfig.IsSTSTrustedProxy(ip) {
continue
}
return ip
}
return ""
}
func getSTSLDAPLoginPeerAddr(remoteAddr string) string {
sourceIP, _, err := net.SplitHostPort(remoteAddr)
if err == nil {
return sourceIP
}
return remoteAddr
}
func getSTSLDAPLoginCanonicalIP(addr string) string {
addr = strings.TrimSpace(getSTSLDAPLoginPeerAddr(addr))
addr = strings.TrimPrefix(addr, "[")
addr = strings.TrimSuffix(addr, "]")
if ip := net.ParseIP(addr); ip != nil {
return ip.String()
}
return ""
}
// reserveSTSLDAPLogin acquires an immediate token from the per-source limiter
// before contacting LDAP. Call Commit on auth failures and Cancel when the
// attempt should not count as an authentication failure.
func reserveSTSLDAPLogin(r *http.Request) *stsLDAPLoginReservation {
return globalSTSLDAPLoginRateLimiter.Reserve(getSTSLDAPLoginSourceIP(r))
}
func ldapBindErrorToSTS(err error) (STSErrorCode, error) {
if idldap.IsAuthError(err) {
return ErrSTSInvalidParameterValue, errLDAPAuthenticationFailed
}
return ErrSTSUpstreamError, nil
}
func writeSTSThrottledResponse(w http.ResponseWriter) {
stsErrorResponse := STSErrorResponse{}
stsErrorResponse.Error.Code = "ThrottlingException"
stsErrorResponse.Error.Message = "Request throttled, please retry later."
stsErrorResponse.RequestID = w.Header().Get(xhttp.AmzRequestID)
w.Header().Set("Retry-After", strconv.Itoa(stsLDAPLoginRetryAfterSec))
encodedErrorResponse := encodeResponse(stsErrorResponse)
writeResponse(w, http.StatusTooManyRequests, encodedErrorResponse, mimeXML)
}
// AssumeRole - implementation of AWS STS API AssumeRole to get temporary
// credentials for regular users on Minio.
// https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html
@@ -690,12 +1004,26 @@ func (sts *stsAPIHandlers) AssumeRoleWithLDAPIdentity(w http.ResponseWriter, r *
return
}
lookupResult, groupDistNames, err := globalIAMSys.LDAPConfig.Bind(ldapUsername, ldapPassword)
if err != nil {
err = fmt.Errorf("LDAP server error: %w", err)
writeSTSErrorResponse(ctx, w, ErrSTSInvalidParameterValue, err)
loginReservation := reserveSTSLDAPLogin(r)
if loginReservation == nil {
writeSTSThrottledResponse(w)
return
}
defer loginReservation.Cancel()
lookupResult, groupDistNames, err := globalIAMSys.LDAPConfig.Bind(ldapUsername, ldapPassword)
if err != nil {
errCode, errResp := ldapBindErrorToSTS(err)
if errCode == ErrSTSUpstreamError {
loginReservation.Cancel()
stsLogIf(ctx, err, logger.ErrorKind)
} else {
loginReservation.Commit()
}
writeSTSErrorResponse(ctx, w, errCode, errResp)
return
}
loginReservation.Cancel()
ldapUserDN := lookupResult.NormDN
ldapActualUserDN := lookupResult.ActualDN
+918
View File
@@ -20,12 +20,19 @@ package cmd
import (
"bytes"
"context"
"encoding/xml"
"errors"
"fmt"
"io"
"net/http"
"net/http/httptest"
"net/url"
"os"
"reflect"
"slices"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
@@ -977,6 +984,345 @@ func TestIAMWithLDAPServerSuite(t *testing.T) {
}
}
type ldapSTSErrorResult struct {
StatusCode int
RetryAfter string
Code string
Message string
Body string
}
type ldapSTSHTTPResult struct {
StatusCode int
RetryAfter string
Body string
}
func withGlobalSTSLDAPLoginRateLimiterForTest(limiter *stsLDAPLoginRateLimiter, fn func()) {
previous := globalSTSLDAPLoginRateLimiter
globalSTSLDAPLoginRateLimiter = limiter
defer func() {
globalSTSLDAPLoginRateLimiter = previous
}()
fn()
}
func withLDAPSTSTrustedProxiesForTest(t *testing.T, trustedProxies string, fn func()) {
t.Helper()
previousIAMSys := globalIAMSys
if globalIAMSys == nil {
globalIAMSys = &IAMSys{}
}
previous := globalIAMSys.LDAPConfig.Clone()
if err := globalIAMSys.LDAPConfig.SetSTSTrustedProxies(trustedProxies); err != nil {
t.Fatalf("unable to set LDAP STS trusted proxies for test: %v", err)
}
defer func() {
globalIAMSys.LDAPConfig = previous
globalIAMSys = previousIAMSys
}()
fn()
}
func singleHeader(key, value string) http.Header {
header := make(http.Header)
if key != "" {
header.Set(key, value)
}
return header
}
func (s *TestSuiteIAM) postLDAPSTSWithHeaders(c *check, username, password string, headers http.Header) ldapSTSHTTPResult {
c.Helper()
ctx, cancel := context.WithTimeout(context.Background(), testDefaultTimeout)
defer cancel()
form := url.Values{}
form.Set("Action", ldapIdentity)
form.Set("Version", stsAPIVersion)
form.Set(stsLDAPUsername, username)
form.Set(stsLDAPPassword, password)
req, err := http.NewRequestWithContext(ctx, http.MethodPost, s.endPoint, strings.NewReader(form.Encode()))
if err != nil {
c.Fatalf("unexpected request creation error: %v", err)
}
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
for key, values := range headers {
for _, value := range values {
req.Header.Add(key, value)
}
}
resp, err := s.TestSuiteCommon.client.Do(req)
if err != nil {
c.Fatalf("unexpected LDAP STS request error: %v", err)
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
if err != nil {
c.Fatalf("unexpected LDAP STS response read error: %v", err)
}
return ldapSTSHTTPResult{
StatusCode: resp.StatusCode,
RetryAfter: resp.Header.Get("Retry-After"),
Body: string(body),
}
}
func (s *TestSuiteIAM) postLDAPSTS(c *check, username, password string) ldapSTSHTTPResult {
c.Helper()
return s.postLDAPSTSWithHeaders(c, username, password, nil)
}
func (s *TestSuiteIAM) postLDAPSTSForError(c *check, username, password string) ldapSTSErrorResult {
c.Helper()
result := s.postLDAPSTS(c, username, password)
if result.StatusCode == http.StatusOK {
c.Fatalf("expected LDAP STS request to fail, got success: %s", result.Body)
}
var stsErr STSErrorResponse
if err := xml.Unmarshal([]byte(result.Body), &stsErr); err != nil {
c.Fatalf("unexpected LDAP STS XML decode error: %v, body: %s", err, result.Body)
}
return ldapSTSErrorResult{
StatusCode: result.StatusCode,
RetryAfter: result.RetryAfter,
Code: stsErr.Error.Code,
Message: stsErr.Error.Message,
Body: result.Body,
}
}
func (s *TestSuiteIAM) TestLDAPSTSAuthFailureUniformResponse(c *check) {
withGlobalSTSLDAPLoginRateLimiterForTest(
newSTSLDAPLoginRateLimiter(time.Minute, stsLDAPLoginBurst, stsLDAPLoginEntryTTL),
func() {
missingUser := s.postLDAPSTSForError(c, "missing-user", "nottherightpassword")
wrongPassword := s.postLDAPSTSForError(c, "dillon", "nottherightpassword")
if missingUser.StatusCode != http.StatusBadRequest {
c.Fatalf("expected missing-user request status %d, got %d", http.StatusBadRequest, missingUser.StatusCode)
}
if wrongPassword.StatusCode != http.StatusBadRequest {
c.Fatalf("expected wrong-password request status %d, got %d", http.StatusBadRequest, wrongPassword.StatusCode)
}
if missingUser.Code != "InvalidParameterValue" || wrongPassword.Code != "InvalidParameterValue" {
c.Fatalf("expected InvalidParameterValue for both auth failures, got missing=%q wrong=%q", missingUser.Code, wrongPassword.Code)
}
if missingUser.Message != errLDAPAuthenticationFailed.Error() || wrongPassword.Message != errLDAPAuthenticationFailed.Error() {
c.Fatalf("expected uniform LDAP auth failure message, got missing=%q wrong=%q", missingUser.Message, wrongPassword.Message)
}
if strings.Contains(strings.ToLower(missingUser.Body), "unable to find user dn") {
c.Fatalf("missing-user response leaked lookup details: %s", missingUser.Body)
}
if strings.Contains(strings.ToLower(wrongPassword.Body), "ldap auth failed for dn") {
c.Fatalf("wrong-password response leaked bind details: %s", wrongPassword.Body)
}
},
)
}
func (s *TestSuiteIAM) TestLDAPSTSRateLimit(c *check) {
withGlobalSTSLDAPLoginRateLimiterForTest(
newSTSLDAPLoginRateLimiter(time.Hour, 2, stsLDAPLoginEntryTTL),
func() {
first := s.postLDAPSTSForError(c, "dillon", "nottherightpassword")
second := s.postLDAPSTSForError(c, "dillon", "nottherightpassword")
throttled := s.postLDAPSTSForError(c, "dillon", "nottherightpassword")
if first.StatusCode != http.StatusBadRequest || second.StatusCode != http.StatusBadRequest {
c.Fatalf("expected first two failed auth attempts to return %d, got %d and %d", http.StatusBadRequest, first.StatusCode, second.StatusCode)
}
if throttled.StatusCode != http.StatusTooManyRequests {
c.Fatalf("expected throttled request status %d, got %d", http.StatusTooManyRequests, throttled.StatusCode)
}
if throttled.Code != "ThrottlingException" {
c.Fatalf("expected throttled code %q, got %q", "ThrottlingException", throttled.Code)
}
if throttled.Message != "Request throttled, please retry later." {
c.Fatalf("expected throttled message %q, got %q", "Request throttled, please retry later.", throttled.Message)
}
if throttled.RetryAfter != fmt.Sprintf("%d", stsLDAPLoginRetryAfterSec) {
c.Fatalf("expected Retry-After %d, got %q", stsLDAPLoginRetryAfterSec, throttled.RetryAfter)
}
},
)
}
func (s *TestSuiteIAM) TestLDAPSTSSuccessDoesNotConsumeRateLimit(c *check) {
ctx, cancel := context.WithTimeout(context.Background(), testDefaultTimeout)
defer cancel()
userReq := madmin.PolicyAssociationReq{
Policies: []string{"consoleAdmin"},
User: "uid=dillon,ou=people,ou=swengg,dc=min,dc=io",
}
if _, err := s.adm.AttachPolicyLDAP(ctx, userReq); err != nil {
c.Fatalf("unable to attach LDAP policy for success rate-limit test: %v", err)
}
withGlobalSTSLDAPLoginRateLimiterForTest(
newSTSLDAPLoginRateLimiter(time.Hour, 2, stsLDAPLoginEntryTTL),
func() {
for attempt := 1; attempt <= 3; attempt++ {
success := s.postLDAPSTS(c, "dillon", "dillon")
if success.StatusCode != http.StatusOK {
c.Fatalf("expected successful LDAP STS login on attempt %d, got status %d body: %s", attempt, success.StatusCode, success.Body)
}
}
firstFailure := s.postLDAPSTSForError(c, "dillon", "nottherightpassword")
secondFailure := s.postLDAPSTSForError(c, "dillon", "nottherightpassword")
throttled := s.postLDAPSTSForError(c, "dillon", "nottherightpassword")
if firstFailure.StatusCode != http.StatusBadRequest || secondFailure.StatusCode != http.StatusBadRequest {
c.Fatalf("expected failed auth attempts after successful logins to return %d, got %d and %d", http.StatusBadRequest, firstFailure.StatusCode, secondFailure.StatusCode)
}
if throttled.StatusCode != http.StatusTooManyRequests {
c.Fatalf("expected third failed auth attempt after successful logins to be throttled with %d, got %d", http.StatusTooManyRequests, throttled.StatusCode)
}
},
)
}
func (s *TestSuiteIAM) TestLDAPSTSUpstreamFailure(c *check) {
original := globalIAMSys.LDAPConfig.Clone()
globalIAMSys.LDAPConfig.LDAP.ServerAddr = "127.0.0.1:1"
defer func() {
globalIAMSys.LDAPConfig = original
}()
withGlobalSTSLDAPLoginRateLimiterForTest(
newSTSLDAPLoginRateLimiter(time.Hour, 2, stsLDAPLoginEntryTTL),
func() {
for range 3 {
upstreamFailure := s.postLDAPSTSForError(c, "dillon", "dillon")
if upstreamFailure.StatusCode != http.StatusInternalServerError {
c.Fatalf("expected upstream failure status %d, got %d", http.StatusInternalServerError, upstreamFailure.StatusCode)
}
if upstreamFailure.Code != "InternalError" {
c.Fatalf("expected upstream failure code %q, got %q", "InternalError", upstreamFailure.Code)
}
if upstreamFailure.Message != stsErrCodes.ToSTSErr(ErrSTSUpstreamError).Description {
c.Fatalf("expected upstream failure message %q, got %q", stsErrCodes.ToSTSErr(ErrSTSUpstreamError).Description, upstreamFailure.Message)
}
if upstreamFailure.Message == errLDAPAuthenticationFailed.Error() {
c.Fatalf("expected upstream failure to stay distinct from auth failure, got %q", upstreamFailure.Message)
}
}
globalIAMSys.LDAPConfig = original
authFailure := s.postLDAPSTSForError(c, "dillon", "nottherightpassword")
if authFailure.StatusCode == http.StatusTooManyRequests {
c.Fatalf("expected upstream failures not to consume rate limit budget, got throttled response: %+v", authFailure)
}
if authFailure.StatusCode != http.StatusBadRequest {
c.Fatalf("expected auth failure after upstream recovery to return %d, got %d", http.StatusBadRequest, authFailure.StatusCode)
}
},
)
}
func (s *TestSuiteIAM) TestLDAPSTSTrustedProxyRateLimit(c *check) {
withLDAPSTSTrustedProxiesForTest(c.T, "127.0.0.0/8,::1/128", func() {
withGlobalSTSLDAPLoginRateLimiterForTest(
newSTSLDAPLoginRateLimiter(time.Hour, 1, stsLDAPLoginEntryTTL),
func() {
// These usernames intentionally do not exist. The test asserts that
// LDAP user-not-found stays classified as an auth error, so failed
// attempts still commit the reservation and hit the source bucket.
first := s.postLDAPSTSWithHeaders(c, "missing-user-a", "nottherightpassword", singleHeader("X-Real-IP", "203.0.113.10"))
second := s.postLDAPSTSWithHeaders(c, "missing-user-b", "nottherightpassword", singleHeader("X-Real-IP", "198.51.100.23"))
third := s.postLDAPSTSWithHeaders(c, "missing-user-c", "nottherightpassword", singleHeader("X-Real-IP", "203.0.113.10"))
if first.StatusCode != http.StatusBadRequest {
c.Fatalf("expected first trusted-proxy LDAP STS auth failure to return %d, got %d body: %s", http.StatusBadRequest, first.StatusCode, first.Body)
}
if second.StatusCode != http.StatusBadRequest {
c.Fatalf("expected a different forwarded client IP behind the same trusted proxy to avoid source throttling, got %d body: %s", second.StatusCode, second.Body)
}
if third.StatusCode != http.StatusTooManyRequests {
c.Fatalf("expected the same forwarded client IP behind the trusted proxy to be throttled with %d, got %d body: %s", http.StatusTooManyRequests, third.StatusCode, third.Body)
}
},
)
})
}
func TestIAMWithLDAPSecurityServerSuite(t *testing.T) {
tests := []struct {
name string
run func(*TestSuiteIAM, *check, string)
}{
{
name: "AuthFailureUniformResponse",
run: func(suite *TestSuiteIAM, c *check, ldapServer string) {
suite.TestLDAPSTSAuthFailureUniformResponse(c)
},
},
{
name: "RateLimit",
run: func(suite *TestSuiteIAM, c *check, ldapServer string) {
suite.TestLDAPSTSRateLimit(c)
},
},
{
name: "SuccessDoesNotConsumeRateLimit",
run: func(suite *TestSuiteIAM, c *check, ldapServer string) {
suite.TestLDAPSTSSuccessDoesNotConsumeRateLimit(c)
},
},
{
name: "UpstreamFailure",
run: func(suite *TestSuiteIAM, c *check, ldapServer string) {
suite.TestLDAPSTSUpstreamFailure(c)
},
},
{
name: "TrustedProxyRateLimit",
run: func(suite *TestSuiteIAM, c *check, ldapServer string) {
suite.TestLDAPSTSTrustedProxyRateLimit(c)
},
},
}
for i, testCase := range iamTestSuites {
t.Run(
fmt.Sprintf("Test: %d, ServerType: %s", i+1, testCase.ServerTypeDescription),
func(t *testing.T) {
ldapServer := os.Getenv(EnvTestLDAPServer)
if ldapServer == "" {
t.Skipf("Skipping LDAP security test as no LDAP server is provided via %s", EnvTestLDAPServer)
}
suite := testCase
for _, tc := range tests {
tc := tc
t.Run(tc.name, func(t *testing.T) {
c := &check{t, testCase.serverType}
suite.SetUpSuite(c)
suite.SetUpLDAP(c, ldapServer)
tc.run(suite, c, ldapServer)
suite.TearDownSuite(c)
})
}
},
)
}
}
// This test is for a fix added to handle non-normalized base DN values in the
// LDAP configuration. It runs the existing LDAP sub-tests with a non-normalized
// LDAP configuration.
@@ -1052,6 +1398,578 @@ func TestIAMExportImportWithLDAP(t *testing.T) {
}
}
type matchingAuthError struct{}
func (matchingAuthError) Error() string {
return "ldap auth failed"
}
func (matchingAuthError) Is(target error) bool {
return targetIsLDAPAuthFailure(target)
}
func targetIsLDAPAuthFailure(target error) bool {
return target != nil && target.Error() == "ldap authentication failed"
}
func TestSTSLDAPLoginRateLimiter(t *testing.T) {
limiter := newSTSLDAPLoginRateLimiter(time.Hour, 2, time.Minute)
if !limiter.Allow("192.0.2.10") {
t.Fatal("expected first attempt to be allowed")
}
if !limiter.Allow("192.0.2.10") {
t.Fatal("expected second attempt within burst to be allowed")
}
if limiter.Allow("192.0.2.10") {
t.Fatal("expected source IP bucket to be throttled after burst")
}
// A different source IP has its own independent bucket, so one client
// cannot exhaust another's budget (no per-username lockout dimension).
if !limiter.Allow("192.0.2.11") {
t.Fatal("expected a different source IP to be allowed")
}
// An empty source IP cannot be identified and must never be throttled,
// otherwise all such requests would collapse into one shared bucket.
if !limiter.Allow("") || !limiter.Allow("") {
t.Fatal("expected unidentified source to stay unthrottled")
}
}
func TestSTSLDAPLoginRateLimiterReserveCancel(t *testing.T) {
limiter := newSTSLDAPLoginRateLimiter(time.Hour, 1, time.Minute)
reservation := limiter.Reserve("192.0.2.10")
if reservation == nil {
t.Fatal("expected first reservation to succeed")
}
if limiter.Reserve("192.0.2.10") != nil {
t.Fatal("expected second reservation on the same source IP to be throttled before cancel")
}
reservation.Cancel()
reservation = limiter.Reserve("192.0.2.10")
if reservation == nil {
t.Fatal("expected canceled reservation to restore source-IP capacity")
}
reservation.Cancel()
reservation = limiter.Reserve("192.0.2.11")
if reservation == nil {
t.Fatal("expected a different source IP to have independent capacity")
}
reservation.Cancel()
}
func TestSTSLDAPLoginKeyLimiterCancelDoesNotOverCreditAfterRefill(t *testing.T) {
set := newSTSLDAPLoginKeyLimiterSet(10*time.Millisecond, 2, time.Minute)
start := time.Unix(0, 0)
first := set.Reserve(start, "192.0.2.10")
if first == nil {
t.Fatal("expected first reservation to succeed")
}
second := set.Reserve(start.Add(5*time.Millisecond), "192.0.2.10")
if second == nil {
t.Fatal("expected second reservation to succeed while one token remains available")
}
first.CancelAt(start.Add(10 * time.Millisecond))
third := set.Reserve(start.Add(10*time.Millisecond), "192.0.2.10")
if third == nil {
t.Fatal("expected canceled reservation to restore exactly one slot")
}
defer third.CancelAt(start.Add(10 * time.Millisecond))
if extra := set.Reserve(start.Add(10*time.Millisecond), "192.0.2.10"); extra != nil {
extra.CancelAt(start.Add(10 * time.Millisecond))
t.Fatal("expected only one slot to be restored after cancel; got over-credit from refill")
}
second.CancelAt(start.Add(10 * time.Millisecond))
}
func TestSTSLDAPLoginRateLimiterConcurrentReserveLifecycle(t *testing.T) {
limiter := newSTSLDAPLoginRateLimiter(time.Hour, 4, time.Minute)
const workers = 8
start := make(chan struct{})
finish := make(chan struct{})
var wg sync.WaitGroup
var reserveWG sync.WaitGroup
reservations := make([]*stsLDAPLoginReservation, workers)
var canceledReservations atomic.Int32
reserveWG.Add(workers)
for i := 0; i < workers; i++ {
wg.Add(1)
go func(worker int) {
defer wg.Done()
<-start
reservations[worker] = limiter.Reserve("192.0.2.10")
reserveWG.Done()
if reservations[worker] == nil {
return
}
<-finish
if worker%2 == 0 {
canceledReservations.Add(1)
reservations[worker].Cancel()
return
}
reservations[worker].Commit()
}(i)
}
close(start)
reserveWG.Wait()
successfulReservations := 0
for _, reservation := range reservations {
if reservation != nil {
successfulReservations++
}
}
if got := successfulReservations; got != 4 {
t.Fatalf("expected exactly 4 successful concurrent reservations, got %d", got)
}
close(finish)
wg.Wait()
remainingBudget := 0
for {
reservation := limiter.Reserve("192.0.2.10")
if reservation == nil {
break
}
remainingBudget++
reservation.Commit()
}
if want, got := int(canceledReservations.Load()), remainingBudget; got != want {
t.Fatalf("expected %d tokens to remain after concurrent commit/cancel mix, got %d", want, got)
}
}
func TestGetSTSLDAPLoginSourceIP(t *testing.T) {
tests := []struct {
name string
remoteAddr string
want string
}{
{name: "empty", remoteAddr: "", want: ""},
{name: "ipv4", remoteAddr: "192.0.2.10:9000", want: "192.0.2.10"},
{name: "ipv6", remoteAddr: "[2001:db8::10]:9000", want: "2001:db8::10"},
{name: "bare-host", remoteAddr: "192.0.2.10", want: "192.0.2.10"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
req := &http.Request{RemoteAddr: tt.remoteAddr}
if got := getSTSLDAPLoginSourceIP(req); got != tt.want {
t.Fatalf("expected %q, got %q", tt.want, got)
}
})
}
}
func TestGetSTSLDAPLoginSourceIPIgnoresSpoofedForwardingHeaders(t *testing.T) {
tests := []struct {
name string
headerKey string
headerValue string
}{
{
name: "x-forwarded-for",
headerKey: "X-Forwarded-For",
headerValue: "203.0.113.10, 198.51.100.24",
},
{
name: "x-real-ip",
headerKey: "X-Real-IP",
headerValue: "203.0.113.10",
},
{
name: "forwarded",
headerKey: "Forwarded",
headerValue: `for=203.0.113.10;proto=https`,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
req := &http.Request{
Header: singleHeader(tt.headerKey, tt.headerValue),
RemoteAddr: "192.0.2.10:9000",
}
if got := getSTSLDAPLoginSourceIP(req); got != "192.0.2.10" {
t.Fatalf("expected helper to ignore spoofed %s and return peer address, got %q", tt.headerKey, got)
}
})
}
}
func TestGetSTSLDAPLoginSourceIPUsesForwardedHeadersForTrustedProxy(t *testing.T) {
tests := []struct {
name string
headerKey string
headerValue string
want string
}{
{
name: "x-forwarded-for",
headerKey: "X-Forwarded-For",
headerValue: "203.0.113.10",
want: "203.0.113.10",
},
{
name: "x-real-ip",
headerKey: "X-Real-IP",
headerValue: "203.0.113.10",
want: "203.0.113.10",
},
}
withLDAPSTSTrustedProxiesForTest(t, "192.0.2.0/24", func() {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
req := &http.Request{
Header: singleHeader(tt.headerKey, tt.headerValue),
RemoteAddr: "192.0.2.10:9000",
}
if got := getSTSLDAPLoginSourceIP(req); got != tt.want {
t.Fatalf("expected trusted proxy header %s to resolve %q, got %q", tt.headerKey, tt.want, got)
}
})
}
})
}
// A client behind a trusted, appending proxy can prepend a spoofed left-most
// X-Forwarded-For value. The right-to-left walk must skip only trusted hops and
// return the real (right-most untrusted) client, ignoring the spoofed value.
func TestGetSTSLDAPLoginSourceIPTrustedProxyStripsSpoofedForwardedFor(t *testing.T) {
withLDAPSTSTrustedProxiesForTest(t, "192.0.2.0/24", func() {
req := &http.Request{
Header: singleHeader("X-Forwarded-For", "1.2.3.4, 198.51.100.50"),
RemoteAddr: "192.0.2.10:9000",
}
if got := getSTSLDAPLoginSourceIP(req); got != "198.51.100.50" {
t.Fatalf("expected spoofed left-most XFF entry to be ignored and real client returned, got %q", got)
}
})
}
// When several hops in the chain are trusted proxies, the walk skips all of
// them and resolves the left-most (real client) address.
func TestGetSTSLDAPLoginSourceIPTrustedProxyWalksMultipleTrustedHops(t *testing.T) {
withLDAPSTSTrustedProxiesForTest(t, "192.0.2.0/24", func() {
req := &http.Request{
Header: singleHeader("X-Forwarded-For", "203.0.113.10, 192.0.2.20, 192.0.2.21"),
RemoteAddr: "192.0.2.10:9000",
}
if got := getSTSLDAPLoginSourceIP(req); got != "203.0.113.10" {
t.Fatalf("expected walk to skip trusted hops and return real client, got %q", got)
}
})
}
// The RFC 7239 Forwarded header is not honored for trusted-proxy bucketing; such
// requests fall back to the safe peer-address bucket.
func TestGetSTSLDAPLoginSourceIPTrustedProxyIgnoresForwardedHeader(t *testing.T) {
withLDAPSTSTrustedProxiesForTest(t, "192.0.2.0/24", func() {
req := &http.Request{
Header: singleHeader("Forwarded", `for=203.0.113.10;proto=https`),
RemoteAddr: "192.0.2.10:9000",
}
if got := getSTSLDAPLoginSourceIP(req); got != "192.0.2.10" {
t.Fatalf("expected RFC 7239 Forwarded to be ignored and peer address used, got %q", got)
}
})
}
func TestGetSTSLDAPLoginSourceIPTrustedProxyPrefersXRealIPOverXForwardedFor(t *testing.T) {
withLDAPSTSTrustedProxiesForTest(t, "192.0.2.0/24", func() {
req := &http.Request{
Header: make(http.Header),
RemoteAddr: "192.0.2.10:9000",
}
req.Header.Set("X-Forwarded-For", "198.51.100.99, 203.0.113.10")
req.Header.Set("X-Real-IP", "203.0.113.10")
if got := getSTSLDAPLoginSourceIP(req); got != "203.0.113.10" {
t.Fatalf("expected trusted proxy path to prefer X-Real-IP over appended X-Forwarded-For, got %q", got)
}
})
}
// X-Real-IP is trusted verbatim (it cannot be chain-validated like X-Forwarded-For),
// so a client-supplied X-Real-IP that the proxy fails to overwrite wins even over a
// correctly appended X-Forwarded-For chain. This locks the documented deployment
// contract: the trusted proxy MUST overwrite X-Real-IP, otherwise it is a spoofing
// vector. If this assertion ever changes, the change must be deliberate.
func TestGetSTSLDAPLoginSourceIPTrustedProxyTrustsXRealIPVerbatim(t *testing.T) {
withLDAPSTSTrustedProxiesForTest(t, "192.0.2.0/24", func() {
req := &http.Request{
Header: make(http.Header),
RemoteAddr: "192.0.2.10:9000",
}
// Attacker spoofs X-Real-IP with a value that appears nowhere in the XFF
// chain; the trusted proxy still appends the real client (198.51.100.50).
// The spoofed X-Real-IP wins, so the result can only have come from it.
req.Header.Set("X-Real-IP", "10.10.10.10")
req.Header.Set("X-Forwarded-For", "1.1.1.1, 198.51.100.50")
if got := getSTSLDAPLoginSourceIP(req); got != "10.10.10.10" {
t.Fatalf("expected verbatim X-Real-IP trust (spoofable contract), got %q", got)
}
})
}
func TestGetSTSLDAPLoginSourceIPTrustedProxyFallsBackToPeerWithoutForwardingHeaders(t *testing.T) {
withLDAPSTSTrustedProxiesForTest(t, "192.0.2.0/24", func() {
req := &http.Request{RemoteAddr: "192.0.2.10:9000"}
if got := getSTSLDAPLoginSourceIP(req); got != "192.0.2.10" {
t.Fatalf("expected trusted proxy path without forwarding headers to fall back to peer address, got %q", got)
}
})
}
func TestReserveSTSLDAPLoginUsesPeerAddressBuckets(t *testing.T) {
tests := []struct {
name string
headerKey string
firstValue string
secondValue string
}{
{
name: "x-forwarded-for",
headerKey: "X-Forwarded-For",
firstValue: "203.0.113.10",
secondValue: "198.51.100.23, 198.51.100.24",
},
{
name: "x-real-ip",
headerKey: "X-Real-IP",
firstValue: "203.0.113.10",
secondValue: "198.51.100.23",
},
{
name: "forwarded",
headerKey: "Forwarded",
firstValue: `for=203.0.113.10;proto=https`,
secondValue: `for=198.51.100.23;proto=https`,
},
}
for _, tt := range tests {
t.Run(tt.name+"-same-peer", func(t *testing.T) {
limiter := newSTSLDAPLoginRateLimiter(time.Hour, 2, time.Minute)
withGlobalSTSLDAPLoginRateLimiterForTest(limiter, func() {
req1 := &http.Request{
Header: singleHeader(tt.headerKey, tt.firstValue),
RemoteAddr: "192.0.2.10:9000",
Form: url.Values{stsLDAPUsername: []string{"alice"}},
}
req2 := &http.Request{
Header: singleHeader(tt.headerKey, tt.secondValue),
RemoteAddr: "192.0.2.10:9001",
Form: url.Values{stsLDAPUsername: []string{"bob"}},
}
reservation1 := reserveSTSLDAPLogin(req1)
if reservation1 == nil {
t.Fatal("expected first reservation to succeed")
}
defer reservation1.Cancel()
reservation2 := reserveSTSLDAPLogin(req2)
if reservation2 == nil {
t.Fatal("expected second reservation to succeed with shared source bucket capacity")
}
defer reservation2.Cancel()
if got := len(limiter.source.entries); got != 1 {
t.Fatalf("expected requests from the same peer address to share one source bucket, got %d", got)
}
if _, ok := limiter.source.entries["192.0.2.10"]; !ok {
t.Fatalf("expected source bucket for peer address %q, got keys %v", "192.0.2.10", limiter.source.entries)
}
})
})
t.Run(tt.name+"-different-peers", func(t *testing.T) {
limiter := newSTSLDAPLoginRateLimiter(time.Hour, 1, time.Minute)
withGlobalSTSLDAPLoginRateLimiterForTest(limiter, func() {
req1 := &http.Request{
Header: singleHeader(tt.headerKey, tt.firstValue),
RemoteAddr: "192.0.2.10:9000",
Form: url.Values{stsLDAPUsername: []string{"alice"}},
}
req2 := &http.Request{
Header: singleHeader(tt.headerKey, tt.firstValue),
RemoteAddr: "192.0.2.11:9000",
Form: url.Values{stsLDAPUsername: []string{"bob"}},
}
reservation1 := reserveSTSLDAPLogin(req1)
if reservation1 == nil {
t.Fatal("expected first reservation to succeed")
}
defer reservation1.Cancel()
reservation2 := reserveSTSLDAPLogin(req2)
if reservation2 == nil {
t.Fatal("expected second reservation from a different peer address to succeed")
}
defer reservation2.Cancel()
if got := len(limiter.source.entries); got != 2 {
t.Fatalf("expected requests from different peer addresses to use different source buckets, got %d", got)
}
if _, ok := limiter.source.entries["192.0.2.10"]; !ok {
t.Fatalf("expected source bucket for peer address %q, got keys %v", "192.0.2.10", limiter.source.entries)
}
if _, ok := limiter.source.entries["192.0.2.11"]; !ok {
t.Fatalf("expected source bucket for peer address %q, got keys %v", "192.0.2.11", limiter.source.entries)
}
})
})
}
}
func TestReserveSTSLDAPLoginUsesForwardedBucketsForTrustedProxy(t *testing.T) {
limiter := newSTSLDAPLoginRateLimiter(time.Hour, 1, time.Minute)
withGlobalSTSLDAPLoginRateLimiterForTest(limiter, func() {
withLDAPSTSTrustedProxiesForTest(t, "192.0.2.0/24", func() {
req1 := &http.Request{
Header: singleHeader("X-Forwarded-For", "203.0.113.10"),
RemoteAddr: "192.0.2.10:9000",
Form: url.Values{stsLDAPUsername: []string{"alice"}},
}
req2 := &http.Request{
Header: singleHeader("X-Forwarded-For", "198.51.100.23"),
RemoteAddr: "192.0.2.10:9001",
Form: url.Values{stsLDAPUsername: []string{"bob"}},
}
reservation1 := reserveSTSLDAPLogin(req1)
if reservation1 == nil {
t.Fatal("expected first reservation through trusted proxy to succeed")
}
defer reservation1.Cancel()
reservation2 := reserveSTSLDAPLogin(req2)
if reservation2 == nil {
t.Fatal("expected forwarded client IPs behind the same trusted proxy to use distinct source buckets")
}
defer reservation2.Cancel()
if got := len(limiter.source.entries); got != 2 {
t.Fatalf("expected distinct forwarded client IPs to use two source buckets, got %d", got)
}
if _, ok := limiter.source.entries["203.0.113.10"]; !ok {
t.Fatalf("expected source bucket for forwarded client IP %q, got keys %v", "203.0.113.10", limiter.source.entries)
}
if _, ok := limiter.source.entries["198.51.100.23"]; !ok {
t.Fatalf("expected source bucket for forwarded client IP %q, got keys %v", "198.51.100.23", limiter.source.entries)
}
})
})
}
func TestLDAPBindErrorToSTS(t *testing.T) {
tests := []struct {
name string
err error
code STSErrorCode
message string
}{
{
name: "auth failure",
err: matchingAuthError{},
code: ErrSTSInvalidParameterValue,
message: errLDAPAuthenticationFailed.Error(),
},
{
name: "upstream failure",
err: errors.New("ldap server unavailable"),
code: ErrSTSUpstreamError,
message: "",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
code, err := ldapBindErrorToSTS(tt.err)
if code != tt.code {
t.Fatalf("expected code %v, got %v", tt.code, code)
}
if tt.message == "" {
if err != nil {
t.Fatalf("expected nil response error, got %v", err)
}
return
}
if err == nil || err.Error() != tt.message {
t.Fatalf("expected %q, got %v", tt.message, err)
}
})
}
}
func TestSTSLDAPLoginRateLimiterCleanup(t *testing.T) {
set := newSTSLDAPLoginKeyLimiterSet(time.Hour, 1, time.Minute)
start := time.Unix(0, 0)
if !set.Allow(start, "old-key") {
t.Fatal("expected initial key to be allowed")
}
if len(set.entries) != 1 {
t.Fatalf("expected one entry, got %d", len(set.entries))
}
if !set.Allow(start.Add(2*time.Minute), "new-key") {
t.Fatal("expected new key to be allowed after ttl expiry")
}
if _, ok := set.entries["old-key"]; ok {
t.Fatal("expected expired key to be cleaned up")
}
}
func TestWriteSTSThrottledResponse(t *testing.T) {
req := httptest.NewRequest(http.MethodPost, "http://minio.test", strings.NewReader(""))
rr := httptest.NewRecorder()
req = req.WithContext(newContext(req, rr, "test-throttle"))
writeSTSThrottledResponse(rr)
if rr.Code != http.StatusTooManyRequests {
t.Fatalf("expected status %d, got %d", http.StatusTooManyRequests, rr.Code)
}
if got := rr.Header().Get("Retry-After"); got != "6" {
t.Fatalf("expected Retry-After header %q, got %q", "6", got)
}
body := rr.Body.String()
if !strings.Contains(body, "<Code>ThrottlingException</Code>") {
t.Fatalf("expected throttling code in response, got %s", body)
}
if !strings.Contains(body, "<Message>Request throttled, please retry later.</Message>") {
t.Fatalf("expected throttling message in response, got %s", body)
}
}
func TestIAMImportAssetWithLDAP(t *testing.T) {
ctx, cancel := context.WithTimeout(t.Context(), testDefaultTimeout)
defer cancel()
+2 -2
View File
@@ -450,10 +450,10 @@ func getLatestReleaseTime(u *url.URL, timeout time.Duration, mode string) (sha25
const (
// Kubernetes deployment doc link.
kubernetesDeploymentDoc = "https://docs.min.io/community/minio-object-store/operations/deployments/kubernetes.html"
kubernetesDeploymentDoc = "https://silo.pigsty.io/operations/deployments/kubernetes.html"
// Mesos deployment doc link.
mesosDeploymentDoc = "https://docs.min.io/community/minio-object-store/operations/deployments/kubernetes.html"
mesosDeploymentDoc = "https://silo.pigsty.io/operations/deployments/kubernetes.html"
)
func getDownloadURL(releaseTag string) (downloadURL string) {
-16
View File
@@ -68,7 +68,6 @@ const (
storageMetricReadXL
storageMetricReadAll
storageMetricStatInfoFile
storageMetricReadMultiple
storageMetricDeleteAbandonedParts
storageMetricDiskInfo
storageMetricDeleteBulk
@@ -723,21 +722,6 @@ func (p *xlStorageDiskIDCheck) ReadParts(ctx context.Context, volume string, par
return p.storage.ReadParts(ctx, volume, partMetaPaths...)
}
// ReadMultiple will read multiple files and send each files as response.
// Files are read and returned in the given order.
// The resp channel is closed before the call returns.
// Only a canceled context will return an error.
func (p *xlStorageDiskIDCheck) ReadMultiple(ctx context.Context, req ReadMultipleReq, resp chan<- ReadMultipleResp) (err error) {
ctx, done, err := p.TrackDiskHealth(ctx, storageMetricReadMultiple, req.Bucket, req.Prefix)
if err != nil {
xioutil.SafeClose(resp)
return err
}
defer done(0, &err)
return p.storage.ReadMultiple(ctx, req, resp)
}
// CleanAbandonedData will read metadata of the object on disk
// and delete any data directories and inline data that isn't referenced in metadata.
func (p *xlStorageDiskIDCheck) CleanAbandonedData(ctx context.Context, volume string, path string) (err error) {
-75
View File
@@ -3187,81 +3187,6 @@ func (s *xlStorage) ReadParts(ctx context.Context, volume string, partMetaPaths
return parts, nil
}
// ReadMultiple will read multiple files and send each back as response.
// Files are read and returned in the given order.
// The resp channel is closed before the call returns.
// Only a canceled context will return an error.
func (s *xlStorage) ReadMultiple(ctx context.Context, req ReadMultipleReq, resp chan<- ReadMultipleResp) error {
defer xioutil.SafeClose(resp)
volumeDir := pathJoin(s.drivePath, req.Bucket)
found := 0
for _, f := range req.Files {
if contextCanceled(ctx) {
return ctx.Err()
}
r := ReadMultipleResp{
Bucket: req.Bucket,
Prefix: req.Prefix,
File: f,
}
var data []byte
var mt time.Time
fullPath := pathJoin(volumeDir, req.Prefix, f)
w := xioutil.NewDeadlineWorker(globalDriveConfig.GetMaxTimeout())
if err := w.Run(func() (err error) {
if req.MetadataOnly {
data, mt, err = s.readMetadataWithDMTime(ctx, fullPath)
} else {
data, mt, err = s.readAllDataWithDMTime(ctx, req.Bucket, volumeDir, fullPath)
}
return err
}); err != nil {
if !IsErr(err, errFileNotFound, errVolumeNotFound) {
r.Exists = true
r.Error = err.Error()
}
select {
case <-ctx.Done():
return ctx.Err()
case resp <- r:
}
if req.AbortOn404 && !r.Exists {
// We stop at first file not found.
// We have already reported the error, return nil.
return nil
}
continue
}
diskHealthCheckOK(ctx, nil)
if req.MaxSize > 0 && int64(len(data)) > req.MaxSize {
r.Exists = true
r.Error = fmt.Sprintf("max size (%d) exceeded: %d", req.MaxSize, len(data))
select {
case <-ctx.Done():
return ctx.Err()
case resp <- r:
continue
}
}
found++
r.Exists = true
r.Data = data
r.Modtime = mt
select {
case <-ctx.Done():
return ctx.Err()
case resp <- r:
}
if req.MaxResults > 0 && found >= req.MaxResults {
return nil
}
}
return nil
}
func (s *xlStorage) StatInfoFile(ctx context.Context, volume, path string, glob bool) (stat []StatInfo, err error) {
volumeDir, err := s.getVolDir(volume)
if err != nil {
+13 -13
View File
@@ -2,7 +2,7 @@
## **1. Cloud-native Architecture**
![cloud-native](https://github.com/minio/minio/blob/master/docs/bigdata/images/image1.png?raw=true "cloud native architecture")
![cloud-native](https://github.com/pgsty/minio/blob/master/docs/bigdata/images/image1.png?raw=true "cloud native architecture")
Kubernetes manages stateless Spark and Hive containers elastically on the compute nodes. Spark has native scheduler integration with Kubernetes. Hive, for legacy reasons, uses YARN scheduler on top of Kubernetes.
@@ -16,24 +16,24 @@ MinIO also supports multi-cluster, multi-site federation similar to AWS regions
- [Setup Ambari](https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.1.0/bk_ambari-installation/content/set_up_the_ambari_server.html) which automatically sets up YARN
- [Installing Spark](https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.1/installing-spark/content/installing_spark.html)
- Install MinIO Distributed Server using one of the guides below.
- [Deployment based on Kubernetes](https://docs.min.io/community/minio-object-store/operations/deployments/kubernetes.html)
- [Deployment based on Kubernetes](https://silo.pigsty.io/operations/deployments/kubernetes.html)
- [Deployment based on MinIO Helm Chart](https://github.com/helm/charts/tree/master/stable/minio)
## **3. Configure Hadoop, Spark, Hive to use MinIO**
After successful installation navigate to the Ambari UI `http://<ambari-server>:8080/` and login using the default credentials: [**_username: admin, password: admin_**]
![ambari-login](https://github.com/minio/minio/blob/master/docs/bigdata/images/image3.png?raw=true "ambari login")
![ambari-login](https://github.com/pgsty/minio/blob/master/docs/bigdata/images/image3.png?raw=true "ambari login")
### **3.1 Configure Hadoop**
Navigate to **Services** -> **HDFS** -> **CONFIGS** -> **ADVANCED** as shown below
![hdfs-configs](https://github.com/minio/minio/blob/master/docs/bigdata/images/image2.png?raw=true "hdfs advanced configs")
![hdfs-configs](https://github.com/pgsty/minio/blob/master/docs/bigdata/images/image2.png?raw=true "hdfs advanced configs")
Navigate to **Custom core-site** to configure MinIO parameters for `_s3a_` connector
![s3a-config](https://github.com/minio/minio/blob/master/docs/bigdata/images/image5.png?raw=true "custom core-site")
![s3a-config](https://github.com/pgsty/minio/blob/master/docs/bigdata/images/image5.png?raw=true "custom core-site")
```
sudo pip install yq
@@ -100,17 +100,17 @@ The rest of the other optimization options are discussed in the links below
Once the config changes are applied, proceed to restart **Hadoop** services.
![hdfs-services](https://github.com/minio/minio/blob/master/docs/bigdata/images/image7.png?raw=true "hdfs restart services")
![hdfs-services](https://github.com/pgsty/minio/blob/master/docs/bigdata/images/image7.png?raw=true "hdfs restart services")
### **3.2 Configure Spark2**
Navigate to **Services** -> **Spark2** -> **CONFIGS** as shown below
![spark-config](https://github.com/minio/minio/blob/master/docs/bigdata/images/image6.png?raw=true "spark config")
![spark-config](https://github.com/pgsty/minio/blob/master/docs/bigdata/images/image6.png?raw=true "spark config")
Navigate to “**Custom spark-defaults**” to configure MinIO parameters for `_s3a_` connector
![spark-config](https://github.com/minio/minio/blob/master/docs/bigdata/images/image9.png?raw=true "spark defaults")
![spark-config](https://github.com/pgsty/minio/blob/master/docs/bigdata/images/image9.png?raw=true "spark defaults")
Add the following optimal entries for _spark-defaults.conf_ to configure Spark with **MinIO**.
@@ -146,17 +146,17 @@ spark.hadoop.fs.s3a.threads.max 2048 # maximum number of threads for S3A
Once the config changes are applied, proceed to restart **Spark** services.
![spark-config](https://github.com/minio/minio/blob/master/docs/bigdata/images/image12.png?raw=true "spark restart services")
![spark-config](https://github.com/pgsty/minio/blob/master/docs/bigdata/images/image12.png?raw=true "spark restart services")
### **3.3 Configure Hive**
Navigate to **Services** -> **Hive** -> **CONFIGS**-> **ADVANCED** as shown below
![hive-config](https://github.com/minio/minio/blob/master/docs/bigdata/images/image10.png?raw=true "hive advanced config")
![hive-config](https://github.com/pgsty/minio/blob/master/docs/bigdata/images/image10.png?raw=true "hive advanced config")
Navigate to “**Custom hive-site**” to configure MinIO parameters for `_s3a_` connector
![hive-config](https://github.com/minio/minio/blob/master/docs/bigdata/images/image11.png?raw=true "hive advanced config")
![hive-config](https://github.com/pgsty/minio/blob/master/docs/bigdata/images/image11.png?raw=true "hive advanced config")
Add the following optimal entries for `hive-site.xml` to configure Hive with **MinIO**.
@@ -171,11 +171,11 @@ mapreduce.input.fileinputformat.list-status.num-threads=50
For more information about these options please visit [https://www.cloudera.com/documentation/enterprise/5-11-x/topics/admin_hive_on_s3_tuning.html](https://www.cloudera.com/documentation/enterprise/5-11-x/topics/admin_hive_on_s3_tuning.html)
![hive-config](https://github.com/minio/minio/blob/master/docs/bigdata/images/image13.png?raw=true "hive advanced custom config")
![hive-config](https://github.com/pgsty/minio/blob/master/docs/bigdata/images/image13.png?raw=true "hive advanced custom config")
Once the config changes are applied, proceed to restart all Hive services.
![hive-config](https://github.com/minio/minio/blob/master/docs/bigdata/images/image14.png?raw=true "restart hive services")
![hive-config](https://github.com/pgsty/minio/blob/master/docs/bigdata/images/image14.png?raw=true "restart hive services")
## **4. Run Sample Applications**
+2 -2
View File
@@ -1,6 +1,6 @@
# ILM Tiering Design [![slack](https://slack.min.io/slack?type=svg)](https://slack.min.io) [![Docker Pulls](https://img.shields.io/docker/pulls/minio/minio.svg?maxAge=604800)](https://hub.docker.com/r/minio/minio/)
Lifecycle transition functionality provided in [bucket lifecycle guide](https://github.com/minio/minio/master/docs/bucket/lifecycle/README.md) allows tiering of content from MinIO object store to public clouds or other MinIO clusters.
Lifecycle transition functionality provided in [bucket lifecycle guide](https://github.com/pgsty/minio/blob/master/docs/bucket/lifecycle/README.md) allows tiering of content from MinIO object store to public clouds or other MinIO clusters.
Transition tiers can be added to MinIO using `mc admin tier add` command to associate a `gcs`, `s3` or `azure` bucket or prefix path on a bucket to the tier name.
Lifecycle transition rules can be applied to buckets (both versioned and un-versioned) by specifying the tier name defined above as the transition storage class for the lifecycle rule.
@@ -51,5 +51,5 @@ Tiering and lifecycle transition are applicable only to erasure/distributed MinI
## Explore Further
- [MinIO | Golang Client API Reference](https://docs.min.io/community/minio-object-store/developers/go/API.html#SetBucketLifecycle)
- [MinIO | Golang Client API Reference](https://silo.pigsty.io/developers/go/API.html#SetBucketLifecycle)
- [Object Lifecycle Management](https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lifecycle-mgmt.html)
+4 -4
View File
@@ -4,8 +4,8 @@ Enable object lifecycle configuration on buckets to setup automatic deletion of
## 1. Prerequisites
- Install MinIO - [MinIO Quickstart Guide](https://docs.min.io/community/minio-object-store/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html).
- Install `mc` - [mc Quickstart Guide](https://docs.min.io/community/minio-object-store/reference/minio-mc.html#quickstart)
- Install MinIO - [MinIO Quickstart Guide](https://silo.pigsty.io/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html).
- Install `mc` - [mc Quickstart Guide](https://silo.pigsty.io/reference/minio-mc.html#quickstart)
## 2. Enable bucket lifecycle configuration
@@ -59,7 +59,7 @@ TempUploads | temp/ | ✓ | ✓ | 7 day(s) | ✗
## 3. Activate ILM versioning features
This will only work with a versioned bucket, take a look at [Bucket Versioning Guide](https://docs.min.io/community/minio-object-store/administration/object-management/object-versioning.html) for more understanding.
This will only work with a versioned bucket, take a look at [Bucket Versioning Guide](https://silo.pigsty.io/administration/object-management/object-versioning.html) for more understanding.
### 3.1 Automatic removal of non current objects versions
@@ -228,5 +228,5 @@ Note that transition event notification is a MinIO extension.
## Explore Further
- [MinIO | Golang Client API Reference](https://docs.min.io/community/minio-object-store/developers/go/API.html)
- [MinIO | Golang Client API Reference](https://silo.pigsty.io/developers/go/API.html)
- [Object Lifecycle Management](https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lifecycle-mgmt.html)
+3 -3
View File
@@ -30,7 +30,7 @@ Various event types supported by MinIO server are
| `s3:BucketCreated` |
| `s3:BucketRemoved` |
Use client tools like `mc` to set and listen for event notifications using the [`event` sub-command](https://docs.min.io/community/minio-object-store/reference/minio-mc/mc-event-add.html). MinIO SDK's [`BucketNotification` APIs](https://docs.min.io/community/minio-object-store/developers/go/API.html#setbucketnotification-ctx-context-context-bucketname-string-config-notification-configuration-error) can also be used. The notification message MinIO sends to publish an event is a JSON message with the following [structure](https://docs.aws.amazon.com/AmazonS3/latest/dev/notification-content-structure.html).
Use client tools like `mc` to set and listen for event notifications using the [`event` sub-command](https://silo.pigsty.io/reference/minio-mc/mc-event-add.html). MinIO SDK's [`BucketNotification` APIs](https://silo.pigsty.io/developers/go/API.html#setbucketnotification-ctx-context-context-bucketname-string-config-notification-configuration-error) can also be used. The notification message MinIO sends to publish an event is a JSON message with the following [structure](https://docs.aws.amazon.com/AmazonS3/latest/dev/notification-content-structure.html).
Bucket events can be published to the following targets:
@@ -43,8 +43,8 @@ Bucket events can be published to the following targets:
## Prerequisites
- Install and configure MinIO Server from [here](https://docs.min.io/community/minio-object-store/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html#procedure).
- Install and configure MinIO Client from [here](https://docs.min.io/community/minio-object-store/reference/minio-mc.html#quickstart).
- Install and configure MinIO Server from [here](https://silo.pigsty.io/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html#procedure).
- Install and configure MinIO Client from [here](https://silo.pigsty.io/reference/minio-mc.html#quickstart).
```
$ mc admin config get myminio | grep notify
+3 -3
View File
@@ -1,13 +1,13 @@
# Bucket Quota Configuration Quickstart Guide [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io) [![Docker Pulls](https://img.shields.io/docker/pulls/minio/minio.svg?maxAge=604800)](https://hub.docker.com/r/minio/minio/)
![quota](https://raw.githubusercontent.com/minio/minio/master/docs/bucket/quota/bucketquota.png)
![quota](https://raw.githubusercontent.com/pgsty/minio/master/docs/bucket/quota/bucketquota.png)
Buckets can be configured to have `Hard` quota - it disallows writes to the bucket after configured quota limit is reached.
## Prerequisites
- Install MinIO - [MinIO Quickstart Guide](https://docs.min.io/community/minio-object-store/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html#procedure).
- [Use `mc` with MinIO Server](https://docs.min.io/community/minio-object-store/reference/minio-mc.html#quickstart)
- Install MinIO - [MinIO Quickstart Guide](https://silo.pigsty.io/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html#procedure).
- [Use `mc` with MinIO Server](https://silo.pigsty.io/reference/minio-mc.html#quickstart)
## Set bucket quota configuration
+4 -4
View File
@@ -1,6 +1,6 @@
# Bucket Replication Design [![slack](https://slack.min.io/slack?type=svg)](https://slack.min.io) [![Docker Pulls](https://img.shields.io/docker/pulls/minio/minio.svg?maxAge=604800)](https://hub.docker.com/r/minio/minio/)
This document explains the design approach of server side bucket replication. If you're looking to get started with replication, we suggest you go through the [Bucket replication guide](https://github.com/minio/minio/blob/master/docs/bucket/replication/README.md) first.
This document explains the design approach of server side bucket replication. If you're looking to get started with replication, we suggest you go through the [Bucket replication guide](https://github.com/pgsty/minio/blob/master/docs/bucket/replication/README.md) first.
## Overview
@@ -59,7 +59,7 @@ If 3 or more targets are participating in active-active replication, the replica
### Internal metadata for replication
`xl.meta` that is in use for [versioning](https://github.com/minio/minio/blob/master/docs/bucket/versioning/DESIGN.md) has additional metadata for replication of objects,delete markers and versioned deletes.
`xl.meta` that is in use for [versioning](https://github.com/pgsty/minio/blob/master/docs/bucket/versioning/DESIGN.md) has additional metadata for replication of objects,delete markers and versioned deletes.
### Metadata for object replication - on source
@@ -156,5 +156,5 @@ If 3 or more targets are participating in active-active replication, the replica
## Explore Further
- [MinIO Bucket Versioning Implementation](https://docs.min.io/community/minio-object-store/administration/object-management/object-versioning.html)
- [MinIO Client Quickstart Guide](https://docs.min.io/community/minio-object-store/reference/minio-mc.html#quickstart)
- [MinIO Bucket Versioning Implementation](https://silo.pigsty.io/administration/object-management/object-versioning.html)
- [MinIO Client Quickstart Guide](https://silo.pigsty.io/reference/minio-mc.html#quickstart)
+11 -11
View File
@@ -2,9 +2,9 @@
Bucket replication is designed to replicate selected objects in a bucket to a destination bucket.
The contents of this page have been migrated to the new [MinIO Documentation: Bucket Replication](https://docs.min.io/community/minio-object-store/administration/bucket-replication.html) page. The [Bucket Replication](https://docs.min.io/community/minio-object-store/administration/bucket-replication/bucket-replication-requirements.html) page references dedicated tutorials for configuring one-way "Active-Passive" and two-way "Active-Active" bucket replication.
The contents of this page have been migrated to the new [MinIO Documentation: Bucket Replication](https://silo.pigsty.io/administration/bucket-replication.html) page. The [Bucket Replication](https://silo.pigsty.io/administration/bucket-replication/bucket-replication-requirements.html) page references dedicated tutorials for configuring one-way "Active-Passive" and two-way "Active-Active" bucket replication.
To replicate objects in a bucket to a destination bucket on a target site either in the same cluster or a different cluster, start by enabling [versioning](https://docs.min.io/community/minio-object-store/administration/object-management/object-versioning.html) for both source and destination buckets. Finally, the target site and the destination bucket need to be configured on the source MinIO server.
To replicate objects in a bucket to a destination bucket on a target site either in the same cluster or a different cluster, start by enabling [versioning](https://silo.pigsty.io/administration/object-management/object-versioning.html) for both source and destination buckets. Finally, the target site and the destination bucket need to be configured on the source MinIO server.
## Highlights
@@ -94,7 +94,7 @@ The access key provided for the replication *target* cluster should have these m
}
```
Please note that the permissions required by the admin user on the target cluster can be more fine grained to exclude permissions like "s3:ReplicateDelete", "s3:GetBucketObjectLockConfiguration" etc depending on whether delete replication rules are set up or if object locking is disabled on `destbucket`. The above policies assume that replication of objects, tags and delete marker replication are all enabled on object lock enabled buckets. A sample script to setup replication is provided [here](https://github.com/minio/minio/blob/master/docs/bucket/replication/setup_replication.sh)
Please note that the permissions required by the admin user on the target cluster can be more fine grained to exclude permissions like "s3:ReplicateDelete", "s3:GetBucketObjectLockConfiguration" etc depending on whether delete replication rules are set up or if object locking is disabled on `destbucket`. The above policies assume that replication of objects, tags and delete marker replication are all enabled on object lock enabled buckets. A sample script to setup replication is provided [here](https://github.com/pgsty/minio/blob/master/docs/bucket/replication/setup_replication.sh)
To set up replication from a source bucket `srcbucket` on myminio cluster to a bucket `destbucket` on the target minio cluster with endpoint https://replica-endpoint:9000, use:
```
@@ -155,15 +155,15 @@ The replication configuration generated has the following format and can be expo
The replication configuration follows [AWS S3 Spec](https://docs.aws.amazon.com/AmazonS3/latest/dev/replication-add-config.html). Any objects uploaded to the source bucket that meet replication criteria will now be automatically replicated by the MinIO server to the remote destination bucket. Replication can be disabled at any time by disabling specific rules in the configuration or deleting the replication configuration entirely.
When object locking is used in conjunction with replication, both source and destination buckets needs to have [object locking](https://docs.min.io/community/minio-object-store/administration/object-management/object-retention.html) enabled. Similarly objects encrypted on the server side, will be replicated if destination also supports encryption.
When object locking is used in conjunction with replication, both source and destination buckets needs to have [object locking](https://silo.pigsty.io/administration/object-management/object-retention.html) enabled. Similarly objects encrypted on the server side, will be replicated if destination also supports encryption.
Replication status can be seen in the metadata on the source and destination objects. On the source side, the `X-Amz-Replication-Status` changes from `PENDING` to `COMPLETED` or `FAILED` after replication attempt either succeeded or failed respectively. On the destination side, a `X-Amz-Replication-Status` status of `REPLICA` indicates that the object was replicated successfully. Any replication failures are automatically re-attempted during a periodic disk scanner cycle.
To perform bi-directional replication, repeat the above process on the target site - this time setting the source bucket as the replication target. It is recommended that replication be run in a system with at least two CPU's available to the process, so that replication can run in its own thread.
![put](https://raw.githubusercontent.com/minio/minio/master/docs/bucket/replication/PUT_bucket_replication.png)
![put](https://raw.githubusercontent.com/pgsty/minio/master/docs/bucket/replication/PUT_bucket_replication.png)
![head](https://raw.githubusercontent.com/minio/minio/master/docs/bucket/replication/HEAD_bucket_replication.png)
![head](https://raw.githubusercontent.com/pgsty/minio/master/docs/bucket/replication/HEAD_bucket_replication.png)
## Replica Modification sync
@@ -210,11 +210,11 @@ Also note that `mc` version `RELEASE.2021-09-02T09-21-27Z` or older supports onl
Status of delete marker replication can be viewed by doing a GET/HEAD on the object version - it will return a `X-Minio-Replication-DeleteMarker-Status` header and http response code of `405`. In the case of permanent deletes, if the delete replication is pending or failed to propagate to the target cluster, GET/HEAD will return additional `X-Minio-Replication-Delete-Status` header and a http response code of `405`.
![delete](https://raw.githubusercontent.com/minio/minio/master/docs/bucket/replication/DELETE_bucket_replication.png)
![delete](https://raw.githubusercontent.com/pgsty/minio/master/docs/bucket/replication/DELETE_bucket_replication.png)
The status of replication can be monitored by configuring event notifications on the source and target buckets using `mc event add`.On the source side, the `s3:PutObject`, `s3:Replication:OperationCompletedReplication` and `s3:Replication:OperationFailedReplication` events show the status of replication in the `X-Amz-Replication-Status` metadata.
On the target bucket, `s3:PutObject` event shows `X-Amz-Replication-Status` status of `REPLICA` in the metadata. Additional metrics to monitor backlog state for the purpose of bandwidth management and resource allocation are exposed via Prometheus - see <https://github.com/minio/minio/blob/master/docs/metrics/prometheus/list.md> for more details.
On the target bucket, `s3:PutObject` event shows `X-Amz-Replication-Status` status of `REPLICA` in the metadata. Additional metrics to monitor backlog state for the purpose of bandwidth management and resource allocation are exposed via Prometheus - see <https://github.com/pgsty/minio/blob/master/docs/metrics/prometheus/list.md> for more details.
### Sync/Async Replication
@@ -276,6 +276,6 @@ MinIO does not support SSE-C encrypted objects on replicated buckets, any applic
## Explore Further
- [MinIO Bucket Replication Design](https://github.com/minio/minio/blob/master/docs/bucket/replication/DESIGN.md)
- [MinIO Bucket Versioning Implementation](https://docs.min.io/community/minio-object-store/administration/object-management/object-retention.html)
- [MinIO Client Quickstart Guide](https://docs.min.io/community/minio-object-store/reference/minio-mc.html#quickstart)
- [MinIO Bucket Replication Design](https://github.com/pgsty/minio/blob/master/docs/bucket/replication/DESIGN.md)
- [MinIO Bucket Versioning Implementation](https://silo.pigsty.io/administration/object-management/object-retention.html)
- [MinIO Client Quickstart Guide](https://silo.pigsty.io/reference/minio-mc.html#quickstart)
+5 -5
View File
@@ -10,7 +10,7 @@ A default retention period and retention mode can be configured on a bucket to b
### 1. Prerequisites
- Install MinIO - [MinIO Quickstart Guide](https://docs.min.io/community/minio-object-store/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html)
- Install MinIO - [MinIO Quickstart Guide](https://silo.pigsty.io/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html)
- Install `awscli` - [Installing AWS Command Line Interface](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html)
### 2. Set bucket WORM configuration
@@ -53,7 +53,7 @@ See <https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lock-overview.html>
## Explore Further
- [Use `mc` with MinIO Server](https://docs.min.io/community/minio-object-store/reference/minio-mc.html#quickstart)
- [Use `aws-cli` with MinIO Server](https://docs.min.io/community/minio-object-store/integrations/aws-cli-with-minio.html)
- [Use `minio-go` SDK with MinIO Server](https://docs.min.io/community/minio-object-store/developers/go/minio-go.html)
- [The MinIO documentation website](https://docs.min.io/community/minio-object-store/index.html)
- [Use `mc` with MinIO Server](https://silo.pigsty.io/reference/minio-mc.html#quickstart)
- [Use `aws-cli` with MinIO Server](https://silo.pigsty.io/integrations/aws-cli-with-minio.html)
- [Use `minio-go` SDK with MinIO Server](https://silo.pigsty.io/developers/go/minio-go.html)
- [The MinIO documentation website](https://silo.pigsty.io/index.html)
+1 -1
View File
@@ -42,7 +42,7 @@ Therefore, the metadata is wrapped as a binary array for easy skipping.
- LegacyObjectType (preserves existing deployments and older xl.json format)
- DeleteMarker (a versionId to capture the DELETE sequences implemented primarily for AWS spec compatibility)
A sample msgpack-JSON `xl.meta`, you can debug the content inside `xl.meta` using [xl-meta.go](https://github.com/minio/minio/tree/master/docs/debugging#decoding-metadata) program.
A sample msgpack-JSON `xl.meta`, you can debug the content inside `xl.meta` using [xl-meta.go](https://github.com/pgsty/minio/tree/master/docs/debugging#decoding-metadata) program.
```json
{
+10 -10
View File
@@ -2,7 +2,7 @@
MinIO versioning is designed to keep multiple versions of an object in one bucket. For example, you could store `spark.csv` (version `ede336f2`) and `spark.csv` (version `fae684da`) in a single bucket. Versioning protects you from unintended overwrites, deletions, protect objects with retention policies.
To control data retention and storage usage, use object versioning with [object lifecycle management](https://github.com/minio/minio/blob/master/docs/bucket/lifecycle/README.md). If you have an object expiration lifecycle policy in your non-versioned bucket and you want to maintain the same permanent delete behavior when on versioning-enabled bucket, you must add a noncurrent expiration policy. The noncurrent expiration lifecycle policy will manage the deletes of the noncurrent object versions in the versioning-enabled bucket. (A version-enabled bucket maintains one current and zero or more noncurrent object versions.)
To control data retention and storage usage, use object versioning with [object lifecycle management](https://github.com/pgsty/minio/blob/master/docs/bucket/lifecycle/README.md). If you have an object expiration lifecycle policy in your non-versioned bucket and you want to maintain the same permanent delete behavior when on versioning-enabled bucket, you must add a noncurrent expiration policy. The noncurrent expiration lifecycle policy will manage the deletes of the noncurrent object versions in the versioning-enabled bucket. (A version-enabled bucket maintains one current and zero or more noncurrent object versions.)
Versioning must be explicitly enabled on a bucket, versioning is not enabled by default. Object locking enabled buckets have versioning enabled automatically. Enabling and suspending versioning is done at the bucket level.
@@ -10,25 +10,25 @@ Only MinIO generates version IDs, and they can't be edited. Version IDs are simp
When you PUT an object in a versioning-enabled bucket, the noncurrent version is not overwritten. The following figure shows that when a new version of `spark.csv` is PUT into a bucket that already contains an object with the same name, the original object (ID = `ede336f2`) remains in the bucket, MinIO generates a new version (ID = `fae684da`), and adds the newer version to the bucket.
![put](https://raw.githubusercontent.com/minio/minio/master/docs/bucket/versioning/versioning_PUT_versionEnabled.png)
![put](https://raw.githubusercontent.com/pgsty/minio/master/docs/bucket/versioning/versioning_PUT_versionEnabled.png)
This protects against accidental overwrites or deletes of objects, allows previous versions to be retrieved.
When you DELETE an object, all versions remain in the bucket and MinIO adds a delete marker, as shown below:
![delete](https://raw.githubusercontent.com/minio/minio/master/docs/bucket/versioning/versioning_DELETE_versionEnabled.png)
![delete](https://raw.githubusercontent.com/pgsty/minio/master/docs/bucket/versioning/versioning_DELETE_versionEnabled.png)
Now the delete marker becomes the current version of the object. GET requests by default always retrieve the latest stored version. So performing a simple GET object request when the current version is a delete marker would return `404` `The specified key does not exist` as shown below:
![get](https://raw.githubusercontent.com/minio/minio/master/docs/bucket/versioning/versioning_GET_versionEnabled.png)
![get](https://raw.githubusercontent.com/pgsty/minio/master/docs/bucket/versioning/versioning_GET_versionEnabled.png)
GET requests by specifying a version ID as shown below, you can retrieve the specific object version `fae684da`.
![get_version_id](https://raw.githubusercontent.com/minio/minio/master/docs/bucket/versioning/versioning_GET_versionEnabled_id.png)
![get_version_id](https://raw.githubusercontent.com/pgsty/minio/master/docs/bucket/versioning/versioning_GET_versionEnabled_id.png)
To permanently delete an object you need to specify the version you want to delete, only the user with appropriate permissions can permanently delete a version. As shown below DELETE request called with a specific version id permanently deletes an object from a bucket. Delete marker is not added for DELETE requests with version id.
![delete_version_id](https://raw.githubusercontent.com/minio/minio/master/docs/bucket/versioning/versioning_DELETE_versionEnabled_id.png)
![delete_version_id](https://raw.githubusercontent.com/pgsty/minio/master/docs/bucket/versioning/versioning_DELETE_versionEnabled_id.png)
## Concepts
@@ -211,7 +211,7 @@ public class IsVersioningEnabled {
## Explore Further
- [Use `minio-java` SDK with MinIO Server](https://docs.min.io/community/minio-object-store/developers/java/minio-java.html)
- [Object Lock and Immutability Guide](https://docs.min.io/community/minio-object-store/administration/object-management/object-retention.html)
- [MinIO Admin Complete Guide](https://docs.min.io/community/minio-object-store/reference/minio-mc-admin.html)
- [The MinIO documentation website](https://docs.min.io/community/minio-object-store/index.html)
- [Use `minio-java` SDK with MinIO Server](https://silo.pigsty.io/developers/java/minio-java.html)
- [Object Lock and Immutability Guide](https://silo.pigsty.io/administration/object-management/object-retention.html)
- [MinIO Admin Complete Guide](https://silo.pigsty.io/reference/minio-mc-admin.html)
- [The MinIO documentation website](https://silo.pigsty.io/index.html)
+5 -5
View File
@@ -51,8 +51,8 @@ Instance is now accessible on the host at port 9000, proceed to access the Web b
## Explore Further
- [MinIO Erasure Code Overview](https://docs.min.io/community/minio-object-store/operations/concepts/erasure-coding.html)
- [Use `mc` with MinIO Server](https://docs.min.io/community/minio-object-store/reference/minio-mc.html)
- [Use `aws-cli` with MinIO Server](https://docs.min.io/community/minio-object-store/integrations/aws-cli-with-minio.html)
- [Use `minio-go` SDK with MinIO Server](https://docs.min.io/community/minio-object-store/developers/go/minio-go.html)
- [The MinIO documentation website](https://docs.min.io/community/minio-object-store/index.html)
- [MinIO Erasure Code Overview](https://silo.pigsty.io/operations/concepts/erasure-coding.html)
- [Use `mc` with MinIO Server](https://silo.pigsty.io/reference/minio-mc.html)
- [Use `aws-cli` with MinIO Server](https://silo.pigsty.io/integrations/aws-cli-with-minio.html)
- [Use `minio-go` SDK with MinIO Server](https://silo.pigsty.io/developers/go/minio-go.html)
- [The MinIO documentation website](https://silo.pigsty.io/index.html)
+5 -5
View File
@@ -19,7 +19,7 @@ will increase speed when the content can be compressed.
### 1. Prerequisites
Install MinIO - [MinIO Quickstart Guide](https://docs.min.io/community/minio-object-store/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html).
Install MinIO - [MinIO Quickstart Guide](https://silo.pigsty.io/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html).
### 2. Run MinIO with compression
@@ -131,7 +131,7 @@ the data directory to view the size of the object.
## Explore Further
- [Use `mc` with MinIO Server](https://docs.min.io/community/minio-object-store/reference/minio-mc.html)
- [Use `aws-cli` with MinIO Server](https://docs.min.io/community/minio-object-store/integrations/aws-cli-with-minio.html)
- [Use `minio-go` SDK with MinIO Server](https://docs.min.io/community/minio-object-store/developers/go/minio-go.html)
- [The MinIO documentation website](https://docs.min.io/community/minio-object-store/index.html)
- [Use `mc` with MinIO Server](https://silo.pigsty.io/reference/minio-mc.html)
- [Use `aws-cli` with MinIO Server](https://silo.pigsty.io/integrations/aws-cli-with-minio.html)
- [Use `minio-go` SDK with MinIO Server](https://silo.pigsty.io/developers/go/minio-go.html)
- [The MinIO documentation website](https://silo.pigsty.io/index.html)
+8 -8
View File
@@ -2,11 +2,11 @@
## Configuration Directory
MinIO stores all its config as part of the server deployment, config is erasure coded on MinIO. On a fresh deployment MinIO automatically generates a new `config` and this config is available to be configured via `mc admin config` command. MinIO also encrypts all the config, IAM and policies content if KMS is configured. Please refer to how to encrypt your config and IAM credentials [here](https://github.com/minio/minio/blob/master/docs/kms/IAM.md).
MinIO stores all its config as part of the server deployment, config is erasure coded on MinIO. On a fresh deployment MinIO automatically generates a new `config` and this config is available to be configured via `mc admin config` command. MinIO also encrypts all the config, IAM and policies content if KMS is configured. Please refer to how to encrypt your config and IAM credentials [here](https://github.com/pgsty/minio/blob/master/docs/kms/IAM.md).
### Certificate Directory
TLS certificates by default are expected to be stored under ``${HOME}/.minio/certs`` directory. You need to place certificates here to enable `HTTPS` based access. Read more about [How to secure access to MinIO server with TLS](https://docs.min.io/community/minio-object-store/operations/network-encryption.html).
TLS certificates by default are expected to be stored under ``${HOME}/.minio/certs`` directory. You need to place certificates here to enable `HTTPS` based access. Read more about [How to secure access to MinIO server with TLS](https://silo.pigsty.io/operations/network-encryption.html).
Following is a sample directory structure for MinIO server with TLS certificates.
@@ -65,7 +65,7 @@ minio server /data
### Storage Class
By default, parity for objects with standard storage class is set to `N/2`, and parity for objects with reduced redundancy storage class objects is set to `2`. Read more about storage class support in MinIO server [here](https://github.com/minio/minio/blob/master/docs/erasure/storage-class/README.md).
By default, parity for objects with standard storage class is set to `N/2`, and parity for objects with reduced redundancy storage class objects is set to `2`. Read more about storage class support in MinIO server [here](https://github.com/pgsty/minio/blob/master/docs/erasure/storage-class/README.md).
```
KEY:
@@ -91,7 +91,7 @@ MINIO_STORAGE_CLASS_COMMENT (sentence) optionally add a comment to this setti
#### Etcd
MinIO supports storing encrypted IAM assets in etcd, if KMS is configured. Please refer to how to encrypt your config and IAM credentials [here](https://github.com/minio/minio/blob/master/docs/kms/IAM.md).
MinIO supports storing encrypted IAM assets in etcd, if KMS is configured. Please refer to how to encrypt your config and IAM credentials [here](https://github.com/pgsty/minio/blob/master/docs/kms/IAM.md).
> NOTE: if *path_prefix* is set then MinIO will not federate your buckets, namespaced IAM assets are assumed as isolated tenants, only buckets are considered globally unique but performing a lookup with a *bucket* which belongs to a different tenant will fail unlike federated setups where MinIO would port-forward and route the request to relevant cluster accordingly. This is a special feature, federated deployments should not need to set *path_prefix*.
@@ -125,7 +125,7 @@ MINIO_ETCD_COMMENT (sentence) optionally add a comment to this setting
### API
By default, there is no limitation on the number of concurrent requests that a server/cluster processes at the same time. However, it is possible to impose such limitation using the API subsystem. Read more about throttling limitation in MinIO server [here](https://github.com/minio/minio/blob/master/docs/throttle/README.md).
By default, there is no limitation on the number of concurrent requests that a server/cluster processes at the same time. However, it is possible to impose such limitation using the API subsystem. Read more about throttling limitation in MinIO server [here](https://github.com/pgsty/minio/blob/master/docs/throttle/README.md).
```
KEY:
@@ -172,7 +172,7 @@ MINIO_API_OBJECT_MAX_VERSIONS (number) set max allowed number of
#### Notifications
Notification targets supported by MinIO are in the following list. To configure individual targets please refer to more detailed documentation [here](https://docs.min.io/community/minio-object-store/administration/monitoring.html#bucket-notifications).
Notification targets supported by MinIO are in the following list. To configure individual targets please refer to more detailed documentation [here](https://silo.pigsty.io/administration/monitoring.html#bucket-notifications).
```
notify_webhook publish bucket notifications to webhook endpoints
@@ -336,5 +336,5 @@ minio server /data
## Explore Further
* [MinIO Quickstart Guide](https://docs.min.io/community/minio-object-store/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html)
* [Configure MinIO Server with TLS](https://docs.min.io/community/minio-object-store/operations/network-encryption.html)
* [MinIO Quickstart Guide](https://silo.pigsty.io/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html)
* [Configure MinIO Server with TLS](https://silo.pigsty.io/operations/network-encryption.html)
+1 -1
View File
@@ -2,7 +2,7 @@
## HTTP Trace
HTTP tracing can be enabled by using [`mc admin trace`](https://docs.min.io/community/minio-object-store/reference/minio-mc-admin/mc-admin-trace.html) command.
HTTP tracing can be enabled by using [`mc admin trace`](https://silo.pigsty.io/reference/minio-mc-admin/mc-admin-trace.html) command.
Example:
+1 -1
View File
@@ -115,7 +115,7 @@ In above example there are two server pools
> Notice the requirement of common SLA here original cluster had 1024 drives with 16 drives per erasure set with default parity of '4', second pool is expected to have a minimum of 8 drives per erasure set to match the original cluster SLA (parity count) of '4'. '12' drives stripe per erasure set in the second pool satisfies the original pool's parity count.
Refer to the sizing guide with details on the default parity count chosen for different erasure stripe sizes [here](https://github.com/minio/minio/blob/master/docs/distributed/SIZING.md)
Refer to the sizing guide with details on the default parity count chosen for different erasure stripe sizes [here](https://github.com/pgsty/minio/blob/master/docs/distributed/SIZING.md)
MinIO places new objects in server pools based on proportionate free space, per pool. Following pseudo code demonstrates this behavior.
+12 -12
View File
@@ -8,7 +8,7 @@ MinIO in distributed mode can help you setup a highly-available storage system w
### Data protection
Distributed MinIO provides protection against multiple node/drive failures and [bit rot](https://github.com/minio/minio/blob/master/docs/erasure/README.md#what-is-bit-rot-protection) using [erasure code](https://docs.min.io/community/minio-object-store/operations/concepts/erasure-coding.html). As the minimum drives required for distributed MinIO is 2 (same as minimum drives required for erasure coding), erasure code automatically kicks in as you launch distributed MinIO.
Distributed MinIO provides protection against multiple node/drive failures and [bit rot](https://github.com/pgsty/minio/blob/master/docs/erasure/README.md#what-is-bit-rot-protection) using [erasure code](https://silo.pigsty.io/operations/concepts/erasure-coding.html). As the minimum drives required for distributed MinIO is 2 (same as minimum drives required for erasure coding), erasure code automatically kicks in as you launch distributed MinIO.
If one or more drives are offline at the start of a PutObject or NewMultipartUpload operation the object will have additional data protection bits added automatically to provide additional safety for these objects.
@@ -18,7 +18,7 @@ A stand-alone MinIO server would go down if the server hosting the drives goes o
For example, an 16-server distributed setup with 200 drives per node would continue serving files, up to 4 servers can be offline in default configuration i.e around 800 drives down MinIO would continue to read and write objects.
Refer to sizing guide for more understanding on default values chosen depending on your erasure stripe size [here](https://github.com/minio/minio/blob/master/docs/distributed/SIZING.md). Parity settings can be changed using [storage classes](https://github.com/minio/minio/tree/master/docs/erasure/storage-class).
Refer to sizing guide for more understanding on default values chosen depending on your erasure stripe size [here](https://github.com/pgsty/minio/blob/master/docs/distributed/SIZING.md). Parity settings can be changed using [storage classes](https://github.com/pgsty/minio/tree/master/docs/erasure/storage-class).
### Consistency Guarantees
@@ -38,11 +38,11 @@ Install MinIO either on Kubernetes or Distributed Linux.
Install MinIO on Kubernetes:
- [MinIO Quickstart Guide for Kubernetes](https://docs.min.io/community/minio-object-store/operations/deployments/kubernetes.html).
- [Deploy a Tenant from the MinIO Operator](https://docs.min.io/community/minio-object-store/operations/deployments/k8s-deploy-minio-tenant-on-kubernetes.html)
- [MinIO Quickstart Guide for Kubernetes](https://silo.pigsty.io/operations/deployments/kubernetes.html).
- [Deploy a Tenant from the MinIO Operator](https://silo.pigsty.io/operations/deployments/k8s-deploy-minio-tenant-on-kubernetes.html)
Install Distributed MinIO on Linux:
- [Deploy Distributed MinIO on Linux](https://docs.min.io/community/minio-object-store/operations/install-deploy-manage/deploy-minio-multi-node-multi-drive.html#deploy-distributed-minio)
- [Deploy Distributed MinIO on Linux](https://silo.pigsty.io/operations/install-deploy-manage/deploy-minio-multi-node-multi-drive.html#deploy-distributed-minio)
### 2. Run distributed MinIO
@@ -63,7 +63,7 @@ To start a distributed MinIO instance, you just need to pass drive locations as
Example 1: Start distributed MinIO instance on n nodes with m drives each mounted at `/export1` to `/exportm` (pictured below), by running this command on all the n nodes:
![Distributed MinIO, n nodes with m drives each](https://github.com/minio/minio/blob/master/docs/screenshots/Architecture-diagram_distributed_nm.png?raw=true)
![Distributed MinIO, n nodes with m drives each](https://github.com/pgsty/minio/blob/master/docs/screenshots/Architecture-diagram_distributed_nm.png?raw=true)
### GNU/Linux and macOS
@@ -98,12 +98,12 @@ Now the server has expanded total storage by _(newly_added_servers\*m)_ more dri
## 3. Test your setup
To test this setup, access the MinIO server via browser or [`mc`](https://docs.min.io/community/minio-object-store/reference/minio-mc.html#quickstart).
To test this setup, access the MinIO server via browser or [`mc`](https://silo.pigsty.io/reference/minio-mc.html#quickstart).
## Explore Further
- [MinIO Erasure Code QuickStart Guide](https://docs.min.io/community/minio-object-store/operations/concepts/erasure-coding.html)
- [Use `mc` with MinIO Server](https://docs.min.io/community/minio-object-store/reference/minio-mc.html)
- [Use `aws-cli` with MinIO Server](https://docs.min.io/community/minio-object-store/integrations/aws-cli-with-minio.html)
- [Use `minio-go` SDK with MinIO Server](https://docs.min.io/community/minio-object-store/developers/go/minio-go.html)
- [The MinIO documentation website](https://docs.min.io/community/minio-object-store/index.html)
- [MinIO Erasure Code QuickStart Guide](https://silo.pigsty.io/operations/concepts/erasure-coding.html)
- [Use `mc` with MinIO Server](https://silo.pigsty.io/reference/minio-mc.html)
- [Use `aws-cli` with MinIO Server](https://silo.pigsty.io/integrations/aws-cli-with-minio.html)
- [Use `minio-go` SDK with MinIO Server](https://silo.pigsty.io/developers/go/minio-go.html)
- [The MinIO documentation website](https://silo.pigsty.io/index.html)
+6 -4
View File
@@ -2,13 +2,15 @@
See our web documentation on [Deploying MinIO in Standalone Mode](Deploy Standalone MinIO in a Container) for a more structured tutorial on deploying MinIO in a container.
For images built from this `pgsty/minio` fork, the container also bundles `mcli` and a compatibility `mc` symlink from `pgsty/mc`.
## Prerequisites
Docker installed on your machine. Download the relevant installer from [here](https://www.docker.com/community-edition#/download).
## Run Standalone MinIO on Docker
*Note*: Standalone MinIO is intended for early development and evaluation. For production clusters, deploy a [Distributed](https://docs.min.io/community/minio-object-store/operations/deployments/baremetal-deploy-minio-as-a-container.html) MinIO deployment.
*Note*: Standalone MinIO is intended for early development and evaluation. For production clusters, deploy a [Distributed](https://silo.pigsty.io/operations/deployments/baremetal-deploy-minio-as-a-container.html) MinIO deployment.
MinIO needs a persistent volume to store configuration and application data. For testing purposes, you can launch MinIO by simply passing a directory (`/data` in the example below). This directory gets created in the container filesystem at the time of container start. But all the data is lost after container exits.
@@ -57,7 +59,7 @@ docker run \
We recommend kubernetes based deployment for production level deployment <https://github.com/minio/operator>.
See the [Kubernetes documentation](https://docs.min.io/community/minio-object-store/operations/deployments/kubernetes.html) for more information.
See the [Kubernetes documentation](https://silo.pigsty.io/operations/deployments/kubernetes.html) for more information.
## MinIO Docker Tips
@@ -211,5 +213,5 @@ docker stats <container_id>
## Explore Further
* [MinIO in a Container Installation Guide](https://docs.min.io/community/minio-object-store/operations/deployments/baremetal-deploy-minio-as-a-container.html)
* [MinIO Erasure Code QuickStart Guide](https://docs.min.io/community/minio-object-store/operations/concepts/erasure-coding.html)
* [MinIO in a Container Installation Guide](https://silo.pigsty.io/operations/deployments/baremetal-deploy-minio-as-a-container.html)
* [MinIO Erasure Code QuickStart Guide](https://silo.pigsty.io/operations/concepts/erasure-coding.html)
+4 -4
View File
@@ -6,7 +6,7 @@ MinIO protects data against hardware failures and silent data corruption using e
Erasure code is a mathematical algorithm to reconstruct missing or corrupted data. MinIO uses Reed-Solomon code to shard objects into variable data and parity blocks. For example, in a 12 drive setup, an object can be sharded to a variable number of data and parity blocks across all the drives - ranging from six data and six parity blocks to ten data and two parity blocks.
By default, MinIO shards the objects across N/2 data and N/2 parity drives. Though, you can use [storage classes](https://github.com/minio/minio/tree/master/docs/erasure/storage-class) to use a custom configuration. We recommend N/2 data and parity blocks, as it ensures the best protection from drive failures.
By default, MinIO shards the objects across N/2 data and N/2 parity drives. Though, you can use [storage classes](https://github.com/pgsty/minio/tree/master/docs/erasure/storage-class) to use a custom configuration. We recommend N/2 data and parity blocks, as it ensures the best protection from drive failures.
In 12 drive example above, with MinIO server running in the default configuration, you can lose any of the six drives and still reconstruct the data reliably from the remaining drives.
@@ -14,7 +14,7 @@ In 12 drive example above, with MinIO server running in the default configuratio
Erasure code protects data from multiple drives failure, unlike RAID or replication. For example, RAID6 can protect against two drive failure whereas in MinIO erasure code you can lose as many as half of drives and still the data remains safe. Further, MinIO's erasure code is at the object level and can heal one object at a time. For RAID, healing can be done only at the volume level which translates into high downtime. As MinIO encodes each object individually, it can heal objects incrementally. Storage servers once deployed should not require drive replacement or healing for the lifetime of the server. MinIO's erasure coded backend is designed for operational efficiency and takes full advantage of hardware acceleration whenever available.
![Erasure](https://github.com/minio/minio/blob/master/docs/screenshots/erasure-code.jpg?raw=true)
![Erasure](https://github.com/pgsty/minio/blob/master/docs/screenshots/erasure-code.jpg?raw=true)
## What is Bit Rot protection?
@@ -26,7 +26,7 @@ MinIO's erasure coded backend uses high speed [HighwayHash](https://github.com/m
MinIO divides the drives you provide into erasure-coding sets of *2 to 16* drives. Therefore, the number of drives you present must be a multiple of one of these numbers. Each object is written to a single erasure-coding set.
Minio uses the largest possible EC set size which divides into the number of drives given. For example, *18 drives* are configured as *2 sets of 9 drives*, and *24 drives* are configured as *2 sets of 12 drives*. This is true for scenarios when running MinIO as a standalone erasure coded deployment. In [distributed setup however node (affinity) based](https://docs.min.io/community/minio-object-store/operations/install-deploy-manage/deploy-minio-multi-node-multi-drive.html) erasure stripe sizes are chosen.
Minio uses the largest possible EC set size which divides into the number of drives given. For example, *18 drives* are configured as *2 sets of 9 drives*, and *24 drives* are configured as *2 sets of 12 drives*. This is true for scenarios when running MinIO as a standalone erasure coded deployment. In [distributed setup however node (affinity) based](https://silo.pigsty.io/operations/install-deploy-manage/deploy-minio-multi-node-multi-drive.html) erasure stripe sizes are chosen.
The drives should all be of approximately the same size.
@@ -34,7 +34,7 @@ The drives should all be of approximately the same size.
### 1. Prerequisites
Install MinIO - [MinIO Quickstart Guide](https://docs.min.io/community/minio-object-store/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html)
Install MinIO - [MinIO Quickstart Guide](https://silo.pigsty.io/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html)
### 2. Run MinIO Server with Erasure Code
+3 -3
View File
@@ -2,7 +2,7 @@
MinIO server supports storage class in erasure coding mode. This allows configurable data and parity drives per object.
This page is intended as a summary of MinIO Erasure Coding. For a more complete explanation, see <https://docs.min.io/community/minio-object-store/operations/concepts/erasure-coding.html>.
This page is intended as a summary of MinIO Erasure Coding. For a more complete explanation, see <https://silo.pigsty.io/operations/concepts/erasure-coding.html>.
## Overview
@@ -53,7 +53,7 @@ The default value for the `STANDARD` storage class depends on the number of volu
| 6-7 | EC:3 |
| 8 or more | EC:4 |
For more complete documentation on Erasure Set sizing, see the [MinIO Documentation on Erasure Sets](https://docs.min.io/community/minio-object-store/operations/concepts/erasure-coding.html#erasure-sets).
For more complete documentation on Erasure Set sizing, see the [MinIO Documentation on Erasure Sets](https://silo.pigsty.io/operations/concepts/erasure-coding.html#erasure-sets).
### Allowed values for REDUCED_REDUNDANCY storage class
@@ -80,7 +80,7 @@ export MINIO_STORAGE_CLASS_STANDARD=EC:3
export MINIO_STORAGE_CLASS_RRS=EC:2
```
Storage class can also be set via `mc admin config` get/set commands to update the configuration. Refer [storage class](https://github.com/minio/minio/tree/master/docs/config#storage-class) for
Storage class can also be set via `mc admin config` get/set commands to update the configuration. Refer [storage class](https://github.com/pgsty/minio/tree/master/docs/config#storage-class) for
more details.
#### Note
+3 -3
View File
@@ -23,9 +23,9 @@ All properties except the file size are tied to the zip file. This means that mo
## Code Examples
[Using minio-go library](https://github.com/minio/minio/blob/master/docs/extensions/s3zip/examples/minio-go/main.go)
[Using AWS JS SDK v2](https://github.com/minio/minio/blob/master/docs/extensions/s3zip/examples/aws-js/main.js)
[Using boto3](https://github.com/minio/minio/blob/master/docs/extensions/s3zip/examples/boto3/main.py)
[Using minio-go library](https://github.com/pgsty/minio/blob/master/docs/extensions/s3zip/examples/minio-go/main.go)
[Using AWS JS SDK v2](https://github.com/pgsty/minio/blob/master/docs/extensions/s3zip/examples/aws-js/main.js)
[Using boto3](https://github.com/pgsty/minio/blob/master/docs/extensions/s3zip/examples/boto3/main.py)
## Requirements and limits
+7 -7
View File
@@ -6,7 +6,7 @@ This document explains how to configure MinIO with `Bucket lookup from DNS` styl
### 1. Prerequisites
Install MinIO - [MinIO Quickstart Guide](https://docs.min.io/community/minio-object-store/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html).
Install MinIO - [MinIO Quickstart Guide](https://silo.pigsty.io/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html).
### 2. Run MinIO in federated mode
@@ -17,7 +17,7 @@ Bucket lookup from DNS federation requires two dependencies
## Architecture
![bucket-lookup](https://github.com/minio/minio/blob/master/docs/federation/lookup/bucket-lookup.png?raw=true)
![bucket-lookup](https://github.com/pgsty/minio/blob/master/docs/federation/lookup/bucket-lookup.png?raw=true)
### Environment variables
@@ -76,11 +76,11 @@ it is randomized which cluster might provision the bucket.
### 3. Test your setup
To test this setup, access the MinIO server via browser or [`mc`](https://docs.min.io/community/minio-object-store/reference/minio-mc.html#quickstart). Youll see the uploaded files are accessible from the all the MinIO endpoints.
To test this setup, access the MinIO server via browser or [`mc`](https://silo.pigsty.io/reference/minio-mc.html#quickstart). Youll see the uploaded files are accessible from the all the MinIO endpoints.
## Explore Further
- [Use `mc` with MinIO Server](https://docs.min.io/community/minio-object-store/reference/minio-mc.html)
- [Use `aws-cli` with MinIO Server](https://docs.min.io/community/minio-object-store/integrations/aws-cli-with-minio.html)
- [Use `minio-go` SDK with MinIO Server](https://docs.min.io/community/minio-object-store/developers/go/minio-go.html)
- [The MinIO documentation website](https://docs.min.io/community/minio-object-store/index.html)
- [Use `mc` with MinIO Server](https://silo.pigsty.io/reference/minio-mc.html)
- [Use `aws-cli` with MinIO Server](https://silo.pigsty.io/integrations/aws-cli-with-minio.html)
- [Use `minio-go` SDK with MinIO Server](https://silo.pigsty.io/developers/go/minio-go.html)
- [The MinIO documentation website](https://silo.pigsty.io/index.html)
+1 -1
View File
@@ -64,7 +64,7 @@ Developer is advised to clone the MinIO source and checkout the MinIO release ta
```
Create a branch and proceed to push the branch **upstream**
> (upstream here points to git@github.com:minio/minio.git)
> (upstream here points to git@github.com:pgsty/minio.git)
```
λ git branch -m RELEASE.2021-04-22T15-44-28Z.hotfix
+8 -8
View File
@@ -5,10 +5,10 @@ When using Veeam Backup and Replication, you can use S3 compatible object storag
## Prerequisites
- One or both of Veeam Backup and Replication with support for S3 compatible object store (e.g. 9.5.4) and Veeam Backup for Office365 (VBO)
- MinIO object storage set up per <https://docs.min.io/community/minio-object-store/index.html>
- Veeam requires TLS connections to the object storage. This can be configured per <https://docs.min.io/community/minio-object-store/operations/network-encryption.html>
- MinIO object storage set up per <https://silo.pigsty.io/index.html>
- Veeam requires TLS connections to the object storage. This can be configured per <https://silo.pigsty.io/operations/network-encryption.html>
- The S3 bucket, Access Key and Secret Key have to be created before and outside of Veeam.
- Configure the minio client for the Veeam MinIO endpoint - <https://docs.min.io/community/minio-object-store/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html>
- Configure the minio client for the Veeam MinIO endpoint - <https://silo.pigsty.io/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html>
## Setting up an S3 compatible object store for Veeam Backup and Replication
@@ -26,7 +26,7 @@ mc mb myminio/veeambackup
mc mb -l myminio/veeambackup
```
> Object locking requires erasure coding enabled on the minio server. For more information see <https://docs.min.io/community/minio-object-store/operations/concepts/erasure-coding.html>.
> Object locking requires erasure coding enabled on the minio server. For more information see <https://silo.pigsty.io/operations/concepts/erasure-coding.html>.
### Add MinIO as an object store for Veeam
@@ -34,7 +34,7 @@ Follow the steps from the Veeam documentation for adding MinIO as an object stor
For Veeam Backup with Immutability, choose the amount of days you want to make backups immutable for
![Choose Immutability Days for Object Store](https://raw.githubusercontent.com/minio/minio/master/docs/integrations/veeam/screenshots/object_store_immutable_days.png)
![Choose Immutability Days for Object Store](https://raw.githubusercontent.com/pgsty/minio/master/docs/integrations/veeam/screenshots/object_store_immutable_days.png)
### Creating the Scale-out Backup Repository
@@ -58,7 +58,7 @@ For Veeam Backup with Immutability, choose the amount of days you want to make b
- For Veeam Backup with Immutability, you can choose a number of restore points or days to make backups immutable.
![Choose Immutability Options for Backups](https://raw.githubusercontent.com/minio/minio/master/docs/integrations/veeam/screenshots/backup_job_immutable_days.png)
![Choose Immutability Options for Backups](https://raw.githubusercontent.com/pgsty/minio/master/docs/integrations/veeam/screenshots/backup_job_immutable_days.png)
#### Backup Office 365 with VBO
@@ -70,7 +70,7 @@ mc mb -l myminio/vbo
- Under Backup Infrastructure, right click on Object Storage Repositories and choose "Add object storage"
![Adding Object Storage to VBO Step 1](https://raw.githubusercontent.com/minio/minio/master/docs/integrations/veeam/screenshots/1_add_object_store.png)
![Adding Object Storage to VBO Step 1](https://raw.githubusercontent.com/pgsty/minio/master/docs/integrations/veeam/screenshots/1_add_object_store.png)
- Follow through the wizard as above for Veeam Backup and Replication as the steps are the same between both products
@@ -78,7 +78,7 @@ mc mb -l myminio/vbo
- Follow the wizard. Under the "Object Storage Backup Repository" section, choose the MinIO object storage you created above
![Adding Object Storage to VBO Backup Repository](https://raw.githubusercontent.com/minio/minio/master/docs/integrations/veeam/screenshots/6_add_sobr_with_object_store.png)
![Adding Object Storage to VBO Backup Repository](https://raw.githubusercontent.com/pgsty/minio/master/docs/integrations/veeam/screenshots/6_add_sobr_with_object_store.png)
- When you create your backup job, choose the backup repository you created above.
+2 -2
View File
@@ -7,7 +7,7 @@ MinIO supports encrypting config, IAM assets with KMS provided keys. If the KMS
MinIO supports two ways of encrypting IAM and configuration data.
You can either use KES - together with an external KMS - or, much simpler,
set the env. variable `MINIO_KMS_SECRET_KEY` and start/restart the MinIO server. For more details about KES and how
to set it up refer to our [KMS Guide](https://github.com/minio/minio/blob/master/docs/kms/README.md).
to set it up refer to our [KMS Guide](https://github.com/pgsty/minio/blob/master/docs/kms/README.md).
Instead of configuring an external KMS you can start with a single key by
setting the env. variable `MINIO_KMS_SECRET_KEY`. It expects the following
@@ -45,7 +45,7 @@ kes key create my-minio-key OSMM+vkKUTCvQs9YL/CVMIMt43HFhkUpqJxTmGl6rYw=
- For instructions on setting up KES, see the [KES Getting Started guide](https://github.com/minio/kes/wiki/Getting-Started)
- For instructions on using KES for encrypting the MinIO backend, follow the [KMS Quick Start](https://github.com/minio/minio/tree/master/docs/kms). The SSE-S3 configuration setup also supports MinIO KMS backend encryption.
- For instructions on using KES for encrypting the MinIO backend, follow the [KMS Quick Start](https://github.com/pgsty/minio/tree/master/docs/kms). The SSE-S3 configuration setup also supports MinIO KMS backend encryption.
## FAQ
+6 -6
View File
@@ -4,7 +4,7 @@ MinIO uses a key-management-system (KMS) to support SSE-S3. If a client requests
## Quick Start
MinIO supports multiple KMS implementations via our [KES](https://github.com/minio/kes#kes) project. We run a KES instance at `https://play.min.io:7373` for you to experiment and quickly get started. To run MinIO with a KMS just fetch the root identity, set the following environment variables and then start your MinIO server. If you haven't installed MinIO, yet, then follow the MinIO [install instructions](https://docs.min.io/community/minio-object-store/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html) first.
MinIO supports multiple KMS implementations via our [KES](https://github.com/minio/kes#kes) project. We run a KES instance at `https://play.min.io:7373` for you to experiment and quickly get started. To run MinIO with a KMS just fetch the root identity, set the following environment variables and then start your MinIO server. If you haven't installed MinIO, yet, then follow the MinIO [install instructions](https://silo.pigsty.io/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html) first.
### 1. Fetch the root identity
@@ -67,7 +67,7 @@ The MinIO-KES configuration is always the same - regardless of the underlying KM
### Further references
- [Run MinIO with TLS / HTTPS](https://docs.min.io/community/minio-object-store/operations/network-encryption.html)
- [Run MinIO with TLS / HTTPS](https://silo.pigsty.io/operations/network-encryption.html)
- [Tweak the KES server configuration](https://github.com/minio/kes/wiki/Configuration)
- [Run a load balancer in front of KES](https://github.com/minio/kes/wiki/TLS-Proxy)
- [Understand the KES server concepts](https://github.com/minio/kes/wiki/Concepts)
@@ -137,7 +137,7 @@ Certificates are no secrets and sent in plaintext as part of the TLS handshake.
## Explore Further
- [Use `mc` with MinIO Server](https://docs.min.io/community/minio-object-store/reference/minio-mc.html)
- [Use `aws-cli` with MinIO Server](https://docs.min.io/community/minio-object-store/integrations/aws-cli-with-minio.html)
- [Use `minio-go` SDK with MinIO Server](https://docs.min.io/community/minio-object-store/developers/go/minio-go.html)
- [The MinIO documentation website](https://docs.min.io/community/minio-object-store/index.html)
- [Use `mc` with MinIO Server](https://silo.pigsty.io/reference/minio-mc.html)
- [Use `aws-cli` with MinIO Server](https://silo.pigsty.io/integrations/aws-cli-with-minio.html)
- [Use `minio-go` SDK with MinIO Server](https://silo.pigsty.io/developers/go/minio-go.html)
- [The MinIO documentation website](https://silo.pigsty.io/index.html)
+2 -2
View File
@@ -4,7 +4,7 @@ MinIO's Object Lambda implementation allows for transforming your data to serve
MinIO's Object Lambda, enables application developers to process data retrieved from MinIO before returning it to an application. You can register a Lambda Function target on MinIO, once successfully registered it can be used to transform the data for application GET requests on demand.
This document focuses on showing a working example on how to use Object Lambda with MinIO, you must have [MinIO deployed in your environment](https://docs.min.io/community/minio-object-store/operations/installation.html) before you can start using external lambda functions. You also must install Python version 3.8 or later for the lambda handlers to work.
This document focuses on showing a working example on how to use Object Lambda with MinIO, you must have [MinIO deployed in your environment](https://silo.pigsty.io/operations/installation.html) before you can start using external lambda functions. You also must install Python version 3.8 or later for the lambda handlers to work.
## Example Lambda handler
@@ -134,7 +134,7 @@ mc cp testobject myminio/functionbucket/
## Invoke Lambda transformation via PresignedGET
Following example shows how you can use [`minio-go` PresignedGetObject](https://docs.min.io/community/minio-object-store/developers/go/API.html#presignedgetobject-ctx-context-context-bucketname-objectname-string-expiry-time-duration-reqparams-url-values-url-url-error)
Following example shows how you can use [`minio-go` PresignedGetObject](https://silo.pigsty.io/developers/go/API.html#presignedgetobject-ctx-context-context-bucketname-objectname-string-expiry-time-duration-reqparams-url-values-url-url-error)
```go
package main
+4 -4
View File
@@ -17,7 +17,7 @@ Console target is on always and cannot be disabled.
HTTP target logs to a generic HTTP endpoint in JSON format and is not enabled by default. To enable HTTP target logging you would have to update your MinIO server configuration using `mc admin config set` command.
Assuming `mc` is already [configured](https://docs.min.io/community/minio-object-store/reference/minio-mc.html#quickstart)
Assuming `mc` is already [configured](https://silo.pigsty.io/reference/minio-mc.html#quickstart)
```
mc admin config get myminio/ logger_webhook
@@ -42,7 +42,7 @@ minio server /mnt/data
## Audit Targets
Assuming `mc` is already [configured](https://docs.min.io/community/minio-object-store/reference/minio-mc.html#quickstart)
Assuming `mc` is already [configured](https://silo.pigsty.io/reference/minio-mc.html#quickstart)
### Audit HTTP Target
@@ -224,5 +224,5 @@ NOTE:
## Explore Further
- [MinIO Quickstart Guide](https://docs.min.io/community/minio-object-store/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html)
- [Configure MinIO Server with TLS](https://docs.min.io/community/minio-object-store/operations/network-encryption.html)
- [MinIO Quickstart Guide](https://silo.pigsty.io/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html)
- [Configure MinIO Server with TLS](https://silo.pigsty.io/operations/network-encryption.html)
+2 -2
View File
@@ -9,7 +9,7 @@ MinIO server has two healthcheck related un-authenticated endpoints, a liveness
- Liveness probe available at `/minio/health/live`
- Cluster probe available at `/minio/health/cluster`
Read more on how to use these endpoints in [MinIO healthcheck guide](https://github.com/minio/minio/blob/master/docs/metrics/healthcheck/README.md).
Read more on how to use these endpoints in [MinIO healthcheck guide](https://github.com/pgsty/minio/blob/master/docs/metrics/healthcheck/README.md).
## Prometheus Probe
@@ -25,7 +25,7 @@ The additional bucket specific metrics which include additional go metrics or pr
The additional resource specific metrics which include additional go metrics or process metrics are exposed at
`<Address for MinIO Node>/minio/v2/metrics/resource`.
To use this endpoint, setup Prometheus to scrape data from this endpoint. Read more on how to configure and use Prometheus to monitor MinIO server in [How to monitor MinIO server with Prometheus](https://github.com/minio/minio/blob/master/docs/metrics/prometheus/README.md).
To use this endpoint, setup Prometheus to scrape data from this endpoint. Read more on how to configure and use Prometheus to monitor MinIO server in [How to monitor MinIO server with Prometheus](https://github.com/pgsty/minio/blob/master/docs/metrics/prometheus/README.md).
### **Deprecated metrics monitoring**
+5 -5
View File
@@ -9,7 +9,7 @@ This document explains how to setup Prometheus and configure it to scrape data f
## Prerequisites
To get started with MinIO, refer [MinIO QuickStart Document](https://docs.min.io/community/minio-object-store/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html).
To get started with MinIO, refer [MinIO QuickStart Document](https://silo.pigsty.io/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html).
Follow below steps to get started with MinIO monitoring using Prometheus.
### 1. Download Prometheus
@@ -49,7 +49,7 @@ minio server ~/test
> If MinIO is configured to expose metrics without authentication, you don't need to use `mc` to generate prometheus config. You can skip reading further and move to 3.2 section.
The Prometheus endpoint in MinIO requires authentication by default. Prometheus supports a bearer token approach to authenticate prometheus scrape requests, override the default Prometheus config with the one generated using mc. To generate a Prometheus config for an alias, use [mc](https://docs.min.io/community/minio-object-store/reference/minio-mc.html#quickstart) as follows `mc admin prometheus generate <alias> [METRIC-TYPE]`. The valid values for METRIC-TYPE are `cluster`, `node`, `bucket` and `resource` and if not mentioned, it defaults to `cluster`.
The Prometheus endpoint in MinIO requires authentication by default. Prometheus supports a bearer token approach to authenticate prometheus scrape requests, override the default Prometheus config with the one generated using mc. To generate a Prometheus config for an alias, use [mc](https://silo.pigsty.io/reference/minio-mc.html#quickstart) as follows `mc admin prometheus generate <alias> [METRIC-TYPE]`. The valid values for METRIC-TYPE are `cluster`, `node`, `bucket` and `resource` and if not mentioned, it defaults to `cluster`.
The command will generate the `scrape_configs` section of the prometheus.yml as follows:
@@ -171,7 +171,7 @@ Prometheus sets the `Host` header to `domain:port` as part of HTTP operations ag
### 6. Configure Grafana
After Prometheus is configured, you can use Grafana to visualize MinIO metrics. Refer the [document here to setup Grafana with MinIO prometheus metrics](https://github.com/minio/minio/blob/master/docs/metrics/prometheus/grafana/README.md).
After Prometheus is configured, you can use Grafana to visualize MinIO metrics. Refer the [document here to setup Grafana with MinIO prometheus metrics](https://github.com/pgsty/minio/blob/master/docs/metrics/prometheus/grafana/README.md).
## List of metrics exposed by MinIO
@@ -188,8 +188,8 @@ curl https://play.min.io/minio/v2/metrics/cluster
### List of metrics reported Cluster and Bucket level
[The list of metrics reported can be here](https://github.com/minio/minio/blob/master/docs/metrics/prometheus/list.md)
[The list of metrics reported can be here](https://github.com/pgsty/minio/blob/master/docs/metrics/prometheus/list.md)
### Configure Alerts for Prometheus
[The Prometheus AlertManager and alerts can be configured following this](https://github.com/minio/minio/blob/master/docs/metrics/prometheus/alerts.md)
[The Prometheus AlertManager and alerts can be configured following this](https://github.com/pgsty/minio/blob/master/docs/metrics/prometheus/alerts.md)
+1 -1
View File
@@ -117,4 +117,4 @@ To verify the above sample alert follow below steps
}
```
![Prometheus](https://raw.githubusercontent.com/minio/minio/master/docs/metrics/prometheus/minio-es-tolerance-alert.png)
![Prometheus](https://raw.githubusercontent.com/pgsty/minio/master/docs/metrics/prometheus/minio-es-tolerance-alert.png)
+11 -11
View File
@@ -4,31 +4,31 @@
## Prerequisites
- Prometheus and MinIO configured as explained in [document here](https://github.com/minio/minio/blob/master/docs/metrics/prometheus/README.md).
- Prometheus and MinIO configured as explained in [document here](https://github.com/pgsty/minio/blob/master/docs/metrics/prometheus/README.md).
- Grafana installed as explained [here](https://grafana.com/grafana/download).
## MinIO Grafana Dashboard
Visualize MinIO metrics with our official Grafana dashboard available on the [Grafana dashboard portal](https://grafana.com/grafana/dashboards/13502).
Refer to the dashboard [json file here](https://raw.githubusercontent.com/minio/minio/master/docs/metrics/prometheus/grafana/minio-dashboard.json).
Refer to the dashboard [json file here](https://raw.githubusercontent.com/pgsty/minio/master/docs/metrics/prometheus/grafana/minio-dashboard.json).
![Grafana](https://raw.githubusercontent.com/minio/minio/master/docs/metrics/prometheus/grafana/grafana-minio.png)
![Grafana](https://raw.githubusercontent.com/pgsty/minio/master/docs/metrics/prometheus/grafana/grafana-minio.png)
Node level Replication metrics can be viewed in the Grafana dashboard using [json file here](https://raw.githubusercontent.com/minio/minio/master/docs/metrics/prometheus/grafana/replication/minio-replication-node.json)
Node level Replication metrics can be viewed in the Grafana dashboard using [json file here](https://raw.githubusercontent.com/pgsty/minio/master/docs/metrics/prometheus/grafana/replication/minio-replication-node.json)
![Grafana](https://raw.githubusercontent.com/minio/minio/master/docs/metrics/prometheus/grafana/replication/grafana-replication-node.png)
![Grafana](https://raw.githubusercontent.com/pgsty/minio/master/docs/metrics/prometheus/grafana/replication/grafana-replication-node.png)
Cluster level Replication metrics can be viewed in the Grafana dashboard using [json file here](https://raw.githubusercontent.com/minio/minio/master/docs/metrics/prometheus/grafana/replication/minio-replication-cluster.json)
Cluster level Replication metrics can be viewed in the Grafana dashboard using [json file here](https://raw.githubusercontent.com/pgsty/minio/master/docs/metrics/prometheus/grafana/replication/minio-replication-cluster.json)
![Grafana](https://raw.githubusercontent.com/minio/minio/master/docs/metrics/prometheus/grafana/replication/grafana-replication-cluster.png)
![Grafana](https://raw.githubusercontent.com/pgsty/minio/master/docs/metrics/prometheus/grafana/replication/grafana-replication-cluster.png)
Bucket metrics can be viewed in the Grafana dashboard using [json file here](https://raw.githubusercontent.com/minio/minio/master/docs/metrics/prometheus/grafana/bucket/minio-bucket.json)
Bucket metrics can be viewed in the Grafana dashboard using [json file here](https://raw.githubusercontent.com/pgsty/minio/master/docs/metrics/prometheus/grafana/bucket/minio-bucket.json)
![Grafana](https://raw.githubusercontent.com/minio/minio/master/docs/metrics/prometheus/grafana/bucket/grafana-bucket.png)
![Grafana](https://raw.githubusercontent.com/pgsty/minio/master/docs/metrics/prometheus/grafana/bucket/grafana-bucket.png)
Node metrics can be viewed in the Grafana dashboard using [json file here](https://raw.githubusercontent.com/minio/minio/master/docs/metrics/prometheus/grafana/node/minio-node.json)
Node metrics can be viewed in the Grafana dashboard using [json file here](https://raw.githubusercontent.com/pgsty/minio/master/docs/metrics/prometheus/grafana/node/minio-node.json)
![Grafana](https://raw.githubusercontent.com/minio/minio/master/docs/metrics/prometheus/grafana/node/grafana-node.png)
![Grafana](https://raw.githubusercontent.com/pgsty/minio/master/docs/metrics/prometheus/grafana/node/grafana-node.png)
Note: All these dashboards are provided as an example and need basis they should be customized as well as new graphs should be added.
+6 -6
View File
@@ -78,8 +78,8 @@ For deployments behind a load balancer, use the load balancer hostname instead o
## Cluster Replication Metrics
Metrics marked as ``Site Replication Only`` only populate on deployments with [Site Replication](https://docs.min.io/community/minio-object-store/operations/install-deploy-manage/multi-site-replication.html) configurations.
For deployments with [bucket](https://docs.min.io/community/minio-object-store/administration/bucket-replication.html) or [batch](https://docs.min.io/community/minio-object-store/administration/batch-framework.html#replicate) configurations, these metrics populate instead under the [Bucket Metrics](#bucket-metrics) endpoint.
Metrics marked as ``Site Replication Only`` only populate on deployments with [Site Replication](https://silo.pigsty.io/operations/install-deploy-manage/multi-site-replication.html) configurations.
For deployments with [bucket](https://silo.pigsty.io/administration/bucket-replication.html) or [batch](https://silo.pigsty.io/administration/batch-framework.html#replicate) configurations, these metrics populate instead under the [Bucket Metrics](#bucket-metrics) endpoint.
| Name | Description
|:-----------------------------------------------------------|:---------------------------------------------------------------------------------------------------------|
@@ -108,8 +108,8 @@ For deployments with [bucket](https://docs.min.io/community/minio-object-store/a
## Node Replication Metrics
Metrics marked as ``Site Replication Only`` only populate on deployments with [Site Replication](https://docs.min.io/community/minio-object-store/operations/install-deploy-manage/multi-site-replication.html) configurations.
For deployments with [bucket](https://docs.min.io/community/minio-object-store/administration/bucket-replication.html) or [batch](https://docs.min.io/community/minio-object-store/administration/batch-framework.html#replicate) configurations, these metrics populate instead under the [Bucket Metrics](#bucket-metrics) endpoint.
Metrics marked as ``Site Replication Only`` only populate on deployments with [Site Replication](https://silo.pigsty.io/operations/install-deploy-manage/multi-site-replication.html) configurations.
For deployments with [bucket](https://silo.pigsty.io/administration/bucket-replication.html) or [batch](https://silo.pigsty.io/administration/batch-framework.html#replicate) configurations, these metrics populate instead under the [Bucket Metrics](#bucket-metrics) endpoint.
| Name | Description
|:-----------------------------------------------------------|:---------------------------------------------------------------------------------------------------------|
@@ -296,8 +296,8 @@ For deployments behind a load balancer, use the load balancer hostname instead o
## Replication Metrics
These metrics only populate on deployments with [Bucket Replication](https://docs.min.io/community/minio-object-store/administration/bucket-replication.html) or [Batch Replication](https://docs.min.io/community/minio-object-store/administration/batch-framework.html) configurations.
For deployments with [Site Replication](https://docs.min.io/community/minio-object-store/operations/install-deploy-manage/multi-site-replication.html) configured, select metrics populate under the [Cluster Metrics](#cluster-metrics) endpoint.
These metrics only populate on deployments with [Bucket Replication](https://silo.pigsty.io/administration/bucket-replication.html) or [Batch Replication](https://silo.pigsty.io/administration/batch-framework.html) configurations.
For deployments with [Site Replication](https://silo.pigsty.io/operations/install-deploy-manage/multi-site-replication.html) configured, select metrics populate under the [Cluster Metrics](#cluster-metrics) endpoint.
| Name | Description |
|:----------------------------------------------------|:---------------------------------------------------------------------------------|
+4 -4
View File
@@ -38,18 +38,18 @@ For optimal production setup MinIO recommends Linux kernel version 4.x and later
## List of Amazon S3 APIs not supported on MinIO
We found the following APIs to be redundant or less useful outside of AWS S3. If you have a different view on any of the APIs we missed, please consider opening a [GitHub issue](https://github.com/minio/minio/issues) with relevant details on why MinIO must implement them.
We found the following APIs to be redundant or less useful outside of AWS S3. If you have a different view on any of the APIs we missed, please consider opening a [GitHub issue](https://github.com/pgsty/minio/issues) with relevant details on why MinIO must implement them.
### List of Amazon S3 Bucket APIs not supported on MinIO
- BucketACL (Use [bucket policies](https://docs.min.io/community/minio-object-store/administration/identity-access-management/policy-based-access-control.html) instead)
- BucketACL (Use [bucket policies](https://silo.pigsty.io/administration/identity-access-management/policy-based-access-control.html) instead)
- BucketCORS (CORS enabled by default on all buckets for all HTTP verbs, you can optionally restrict the CORS domains)
- BucketWebsite (Use [`caddy`](https://github.com/caddyserver/caddy) or [`nginx`](https://www.nginx.com/resources/wiki/))
- BucketAnalytics, BucketMetrics, BucketLogging (Use [bucket notification](https://docs.min.io/community/minio-object-store/administration/monitoring/bucket-notifications.html) APIs)
- BucketAnalytics, BucketMetrics, BucketLogging (Use [bucket notification](https://silo.pigsty.io/administration/monitoring/bucket-notifications.html) APIs)
### List of Amazon S3 Object APIs not supported on MinIO
- ObjectACL (Use [bucket policies](https://docs.min.io/community/minio-object-store/administration/identity-access-management/policy-based-access-control.html) instead)
- ObjectACL (Use [bucket policies](https://silo.pigsty.io/administration/identity-access-management/policy-based-access-control.html) instead)
## Object name restrictions on MinIO
+4 -4
View File
@@ -20,7 +20,7 @@ minio server --address :9002 /data/tenant2
minio server --address :9003 /data/tenant3
```
![Example-1](https://github.com/minio/minio/blob/master/docs/screenshots/Example-1.jpg?raw=true)
![Example-1](https://github.com/pgsty/minio/blob/master/docs/screenshots/Example-1.jpg?raw=true)
### 1.2 Host Multiple Tenants on Multiple Drives (Erasure Code)
@@ -32,7 +32,7 @@ minio server --address :9002 /disk{1...4}/data/tenant2
minio server --address :9003 /disk{1...4}/data/tenant3
```
![Example-2](https://github.com/minio/minio/blob/master/docs/screenshots/Example-2.jpg?raw=true)
![Example-2](https://github.com/pgsty/minio/blob/master/docs/screenshots/Example-2.jpg?raw=true)
## 2. Distributed Deployment
@@ -58,10 +58,10 @@ minio server --address :9003 http://192.168.10.1{1...4}/data/tenant3
**Note:** Execute the commands on all 4 nodes.
![Example-3](https://github.com/minio/minio/blob/master/docs/screenshots/Example-3.jpg?raw=true)
![Example-3](https://github.com/pgsty/minio/blob/master/docs/screenshots/Example-3.jpg?raw=true)
**Note**: On distributed systems, root credentials are recommend to be defined by exporting the `MINIO_ROOT_USER` and `MINIO_ROOT_PASSWORD` environment variables. If no value is set MinIO setup will assume `minioadmin/minioadmin` as default credentials. If a domain is required, it must be specified by defining and exporting the `MINIO_DOMAIN` environment variable.
## Cloud Scale Deployment
A container orchestration platform (e.g. Kubernetes) is recommended for large-scale, multi-tenant MinIO deployments. See the [MinIO Deployment Quickstart Guide](https://docs.min.io/community/minio-object-store/operations/deployments/kubernetes.html) to get started with MinIO on orchestration platforms.
A container orchestration platform (e.g. Kubernetes) is recommended for large-scale, multi-tenant MinIO deployments. See the [MinIO Deployment Quickstart Guide](https://silo.pigsty.io/operations/deployments/kubernetes.html) to get started with MinIO on orchestration platforms.
+8 -8
View File
@@ -8,13 +8,13 @@ In this document we will explain in detail on how to configure multiple users.
### 1. Prerequisites
- Install mc - [MinIO Client Quickstart Guide](https://docs.min.io/community/minio-object-store/reference/minio-mc.html#quickstart)
- Install MinIO - [MinIO Quickstart Guide](https://docs.min.io/community/minio-object-store/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html)
- Configure etcd - [Etcd V3 Quickstart Guide](https://github.com/minio/minio/blob/master/docs/sts/etcd.md)
- Install mc - [MinIO Client Quickstart Guide](https://silo.pigsty.io/reference/minio-mc.html#quickstart)
- Install MinIO - [MinIO Quickstart Guide](https://silo.pigsty.io/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html)
- Configure etcd - [Etcd V3 Quickstart Guide](https://github.com/pgsty/minio/blob/master/docs/sts/etcd.md)
### 2. Create a new user with canned policy
Use [`mc admin policy`](https://docs.min.io/community/minio-object-store/reference/minio-mc-admin/mc-admin-policy.html) to create canned policies. Server provides a default set of canned policies namely `writeonly`, `readonly` and `readwrite` *(these policies apply to all resources on the server)*. These can be overridden by custom policies using `mc admin policy` command.
Use [`mc admin policy`](https://silo.pigsty.io/reference/minio-mc-admin/mc-admin-policy.html) to create canned policies. Server provides a default set of canned policies namely `writeonly`, `readonly` and `readwrite` *(these policies apply to all resources on the server)*. These can be overridden by custom policies using `mc admin policy` command.
Create new canned policy file `getonly.json`. This policy enables users to download all objects under `my-bucketname`.
@@ -272,7 +272,7 @@ Following example shows LDAP users full programmatic access to a LDAP user-speci
## Explore Further
- [MinIO Client Complete Guide](https://docs.min.io/community/minio-object-store/reference/minio-mc.html)
- [MinIO STS Quickstart Guide](https://docs.min.io/community/minio-object-store/developers/security-token-service.html)
- [MinIO Admin Complete Guide](https://docs.min.io/community/minio-object-store/reference/minio-mc-admin.html)
- [The MinIO documentation website](https://docs.min.io/community/minio-object-store/index.html)
- [MinIO Client Complete Guide](https://silo.pigsty.io/reference/minio-mc.html)
- [MinIO STS Quickstart Guide](https://silo.pigsty.io/developers/security-token-service.html)
- [MinIO Admin Complete Guide](https://silo.pigsty.io/reference/minio-mc-admin.html)
- [The MinIO documentation website](https://silo.pigsty.io/index.html)
+8 -8
View File
@@ -8,12 +8,12 @@ In this document we will explain in detail on how to configure admin users.
### 1. Prerequisites
- Install mc - [MinIO Client Quickstart Guide](https://docs.min.io/community/minio-object-store/reference/minio-mc.html#quickstart)
- Install MinIO - [MinIO Quickstart Guide](https://docs.min.io/community/minio-object-store/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html)
- Install mc - [MinIO Client Quickstart Guide](https://silo.pigsty.io/reference/minio-mc.html#quickstart)
- Install MinIO - [MinIO Quickstart Guide](https://silo.pigsty.io/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html)
### 2. Create a new admin user with CreateUser, DeleteUser and ConfigUpdate permissions
Use [`mc admin policy`](https://docs.min.io/community/minio-object-store/reference/minio-mc-admin/mc-admin-policy.html#command-mc.admin.policy) to create custom admin policies.
Use [`mc admin policy`](https://silo.pigsty.io/reference/minio-mc-admin/mc-admin-policy.html#command-mc.admin.policy) to create custom admin policies.
Create new canned policy file `adminManageUser.json`. This policy enables admin user to
manage other users.
@@ -162,11 +162,11 @@ mc admin policy attach myminio-admin1 user1policy --user=user1
### 5. Using an external IDP for admin users
Admin users can also be externally managed by an IDP by configuring admin policy with
special permissions listed above. Follow [MinIO STS Quickstart Guide](https://docs.min.io/community/minio-object-store/developers/security-token-service.html) to manage users with an IDP.
special permissions listed above. Follow [MinIO STS Quickstart Guide](https://silo.pigsty.io/developers/security-token-service.html) to manage users with an IDP.
## Explore Further
- [MinIO Client Complete Guide](https://docs.min.io/community/minio-object-store/reference/minio-mc.html)
- [MinIO STS Quickstart Guide](https://docs.min.io/community/minio-object-store/developers/security-token-service.html)
- [MinIO Admin Complete Guide](https://docs.min.io/community/minio-object-store/reference/minio-mc-admin.html)
- [The MinIO documentation website](https://docs.min.io/community/minio-object-store/index.html)
- [MinIO Client Complete Guide](https://silo.pigsty.io/reference/minio-mc.html)
- [MinIO STS Quickstart Guide](https://silo.pigsty.io/developers/security-token-service.html)
- [MinIO Admin Complete Guide](https://silo.pigsty.io/reference/minio-mc-admin.html)
- [The MinIO documentation website](https://silo.pigsty.io/index.html)
+2 -2
View File
@@ -4,7 +4,7 @@ MinIO is a cloud-native application designed to scale in a sustainable manner in
| Orchestration platforms |
|:---------------------------------------------------------------------------------------------------|
| [`Kubernetes`](https://docs.min.io/community/minio-object-store/operations/deployments/kubernetes.html) |
| [`Kubernetes`](https://silo.pigsty.io/operations/deployments/kubernetes.html) |
## Why is MinIO cloud-native?
@@ -18,4 +18,4 @@ MinIO is built ground up on the cloud-native premise. With features like erasure
In a typical modern infrastructure deployment, application, database, key-store, etc. already live in containers and are managed by orchestration platforms. MinIO brings robust, scalable, AWS S3 compatible object storage to the lot.
![Cloud-native](https://github.com/minio/minio/blob/master/docs/screenshots/Minio_Cloud_Native_Arch.jpg?raw=true)
![Cloud-native](https://github.com/pgsty/minio/blob/master/docs/screenshots/Minio_Cloud_Native_Arch.jpg?raw=true)
+4 -4
View File
@@ -11,7 +11,7 @@ With Compose, you use a Compose file to configure MinIO services. Then, using a
## 2. Run Distributed MinIO on Docker Compose
To deploy Distributed MinIO on Docker Compose, please download [docker-compose.yaml](https://github.com/minio/minio/blob/master/docs/orchestration/docker-compose/docker-compose.yaml?raw=true) and [nginx.conf](https://github.com/minio/minio/blob/master/docs/orchestration/docker-compose/nginx.conf?raw=true) to your current working directory. Note that Docker Compose pulls the MinIO Docker image, so there is no need to build MinIO from source when using Docker. For non-Docker deployments, MinIO community edition is now source-only and can be installed via `go install github.com/minio/minio@latest`. Then run one of the below commands
To deploy Distributed MinIO on Docker Compose, please download [docker-compose.yaml](https://github.com/pgsty/minio/blob/master/docs/orchestration/docker-compose/docker-compose.yaml?raw=true) and [nginx.conf](https://github.com/pgsty/minio/blob/master/docs/orchestration/docker-compose/nginx.conf?raw=true) to your current working directory. Note that Docker Compose pulls the MinIO Docker image, so there is no need to build MinIO from source when using Docker. For non-Docker deployments, MinIO community edition is now source-only and can be installed via `go install github.com/minio/minio@latest`. Then run one of the below commands
### GNU/Linux and macOS
@@ -50,10 +50,10 @@ Distributed instances are now accessible on the host using the Minio CLI on port
* Update the command section in each service.
* Add a new MinIO server instance to the upstream directive in the Nginx configuration file.
Read more about distributed MinIO [here](https://docs.min.io/community/minio-object-store/operations/deployments/baremetal-deploy-minio-as-a-container.html).
Read more about distributed MinIO [here](https://silo.pigsty.io/operations/deployments/baremetal-deploy-minio-as-a-container.html).
### Explore Further
* [Overview of Docker Compose](https://docs.docker.com/compose/overview/)
* [MinIO Docker Quickstart Guide](https://docs.min.io/community/minio-object-store/operations/deployments/baremetal-deploy-minio-as-a-container.html)
* [MinIO Erasure Code QuickStart Guide](https://docs.min.io/community/minio-object-store/operations/concepts/erasure-coding.html)
* [MinIO Docker Quickstart Guide](https://silo.pigsty.io/operations/deployments/baremetal-deploy-minio-as-a-container.html)
* [MinIO Erasure Code QuickStart Guide](https://silo.pigsty.io/operations/concepts/erasure-coding.html)
+2 -2
View File
@@ -8,7 +8,7 @@ There are multiple options to deploy MinIO on Kubernetes:
- MinIO-Operator: Operator offers seamless way to create and update highly available distributed MinIO clusters. Refer [MinIO Operator documentation](https://github.com/minio/minio-operator/blob/master/README.md) for more details.
- Helm Chart: MinIO Helm Chart offers customizable and easy MinIO deployment with a single command. Refer [MinIO Helm Chart documentation](https://github.com/minio/minio/tree/master/helm/minio) for more details.
- Helm Chart: MinIO Helm Chart offers customizable and easy MinIO deployment with a single command. Refer [MinIO Helm Chart documentation](https://github.com/pgsty/minio/tree/master/helm/minio) for more details.
## Monitoring MinIO in Kubernetes
@@ -16,6 +16,6 @@ MinIO server exposes un-authenticated liveness endpoints so Kubernetes can nativ
## Explore Further
- [MinIO Erasure Code QuickStart Guide](https://docs.min.io/community/minio-object-store/operations/concepts/erasure-coding.html)
- [MinIO Erasure Code QuickStart Guide](https://silo.pigsty.io/operations/concepts/erasure-coding.html)
- [Kubernetes Documentation](https://kubernetes.io/docs/home/)
- [Helm package manager for kubernetes](https://helm.sh/)
+3 -1
View File
@@ -1,5 +1,7 @@
# MinIO Security Overview [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io)
For fork-specific security advisories and upgrade notes in `pgsty/minio`, see [advisories.md](advisories.md).
## Server-Side Encryption
MinIO supports two different types of server-side encryption ([SSE](#sse)):
@@ -83,7 +85,7 @@ server only assumes that the KMS provides two services:
- `DecryptKey`: Takes a key ID and an encrypted data key and returns the plain data key - the decryption of the encrypted data key using the master key referenced by the key ID - on success or an error otherwise.
More details about supported KMS implementations and configuration can be found at the [KMS guide](https://github.com/minio/minio/blob/master/docs/kms/README.md).
More details about supported KMS implementations and configuration can be found at the [KMS guide](https://github.com/pgsty/minio/blob/master/docs/kms/README.md).
The MinIO server requests a new data key from the KMS for each uploaded object and uses that data key as EK. Additionally it stores the encrypted form of the data key and the master key ID as part of the object metadata. The plain data only resides in RAM during the en/decryption process. The MinIO server does not store any SSE-related key at the KMS. Instead the KMS is treated as trusted component that performs key sealing/unsealing operations to build a key hierarchy:
+28
View File
@@ -0,0 +1,28 @@
# pgsty/minio Security Advisories
This document summarizes fork-specific security fixes and closely related upgrade-impacting security notes in `pgsty/minio`. It is intentionally narrower than a full changelog and focuses on release-impacting security behavior.
## Advisories since `RELEASE.2026-03-21T00-00-00Z`
| ID | Fixed by | Affected area | Remote exploitability | Summary | Upgrade / workaround notes |
| :-- | :-- | :-- | :-- | :-- | :-- |
| `CVE-2026-33322` | `d24f449e0` | OIDC STS (`AssumeRoleWithWebIdentity`, `AssumeRoleWithClientGrants`) | Yes | Closes JWT algorithm confusion by removing HMAC/shared-secret verification and requiring JWKS-backed verifier keys | Breaking change: providers issuing `HS256`, `HS384`, or `HS512` tokens for these STS flows must switch to JWKS-backed RSA or ECDSA signing before upgrading. `PS256` and `EdDSA` are not currently supported. |
| `CVE-2026-33419` | `3b950f8fa` | LDAP STS authentication | Yes | Prevents username enumeration by unifying unknown-user and bad-password responses and adds in-memory login throttling | Unknown users and invalid passwords now both return `400 InvalidParameterValue`. Rate limiting is per-node, in-memory, and not currently configurable. Follow-up hardening landed in `18b712d49`, `9e10f6d9a`, and `f44110890`. |
| `CVE-2026-34204` | `56fa63bfd` | Replication metadata handling | Yes | Blocks untrusted `X-Minio-Replication-*` headers from being smuggled into internal replication metadata and leaving objects unreadable | Upgrade any server that accepts untrusted `PutObject` or `CopyObject` requests, which in practice means almost any production server that accepts writes. |
| `CVE-2026-39414` | `3252d5b7f` | S3 Select oversized record handling | Yes | Rejects oversized CSV records and non-`simdjson` line-delimited JSON records with `OverMaxRecordSize` instead of buffering them unchecked | Split oversized JSON lines client-side if you rely on `simdjson` input paths until the fast path gains the same pre-check. |
| `fake CVE-2026-40027` | `f444b6f37` | Unsigned-trailer PUT and multipart upload authentication | Yes | Closes the query-string authentication bypass in unsigned-trailer streaming requests | Upgrade if clients can reach object write endpoints using the `STREAMING-UNSIGNED-PAYLOAD-TRAILER` content-sha256 mode together with query-string SigV4 credentials. |
| `fake CVE-2026-40028` | `efb6e5b00` | Snowball auto-extract authentication | Yes | Verifies request authentication before tar extraction in Snowball unsigned-trailer flows | Upgrade if you use `PutObjectExtract` or Snowball uploads. |
## Dependency security updates
| ID | Fixed by | Summary |
| :-- | :-- | :-- |
| `CVE-2026-34986` | `68e0ba997` | Upgrades `go-jose` to `v4.1.4`. |
| `CVE-2026-39883` | `1869bd30b`, `e4fa06394` | Updates OpenTelemetry dependencies. |
| Upstream Go security fixes | `db4c0fd5e`, Go 1.26.4 update | Bumps the Go toolchain through `1.26.4`. |
## Operationally significant security-related fixes
| Change | Fixed by | Summary |
| :-- | :-- | :-- |
| LDAP TLS regression | `ce1c537eb` | Restores TLS configuration propagation for `ldaps://` `DialURL()` connections so `MINIO_IDENTITY_LDAP_TLS_SKIP_VERIFY` and custom root CAs work again. |
+8 -7
View File
@@ -12,7 +12,7 @@ You can use the Select API to query objects with following features:
Type inference and automatic conversion of values is performed based on the context when the value is un-typed (such as when reading CSV data). If present, the CAST function overrides automatic conversion.
The [mc sql](https://docs.min.io/community/minio-object-store/reference/minio-mc/mc-sql.html) command can be used for executing queries using the command line.
The [mc sql](https://silo.pigsty.io/reference/minio-mc/mc-sql.html) command can be used for executing queries using the command line.
(*) Parquet is disabled on the MinIO server by default. See below how to enable it.
@@ -27,7 +27,7 @@ To enable Parquet set the environment variable `MINIO_API_SELECT_PARQUET=on`.
### 1. Prerequisites
- Install MinIO Server from [here](https://docs.min.io/community/minio-object-store/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html#procedure).
- Install MinIO Server from [here](https://silo.pigsty.io/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html#procedure).
- Familiarity with AWS S3 API.
- Familiarity with Python and installing dependencies.
@@ -113,11 +113,11 @@ For a more detailed SELECT SQL reference, please see [here](https://docs.aws.ama
## 5. Explore Further
- [Use `mc` with MinIO Server](https://docs.min.io/community/minio-object-store/reference/minio-mc.html)
- [Use `mc sql` with MinIO Server](https://docs.min.io/community/minio-object-store/reference/minio-mc/mc-sql.html#command-mc.sql)
- [Use `minio-go` SDK with MinIO Server](https://docs.min.io/community/minio-object-store/developers/go/minio-go.html)
- [Use `aws-cli` with MinIO Server](https://docs.min.io/community/minio-object-store/integrations/aws-cli-with-minio.html)
- [The MinIO documentation website](https://docs.min.io/community/minio-object-store/index.html)
- [Use `mc` with MinIO Server](https://silo.pigsty.io/reference/minio-mc.html)
- [Use `mc sql` with MinIO Server](https://silo.pigsty.io/reference/minio-mc/mc-sql.html#command-mc.sql)
- [Use `minio-go` SDK with MinIO Server](https://silo.pigsty.io/developers/go/minio-go.html)
- [Use `aws-cli` with MinIO Server](https://silo.pigsty.io/integrations/aws-cli-with-minio.html)
- [The MinIO documentation website](https://silo.pigsty.io/index.html)
## 6. Implementation Status
@@ -128,4 +128,5 @@ For a more detailed SELECT SQL reference, please see [here](https://docs.aws.ama
- Large numbers (outside of the signed 64-bit range) are not yet supported.
- The Date [functions](https://docs.aws.amazon.com/AmazonS3/latest/dev/s3-glacier-select-sql-reference-date.html) `DATE_ADD`, `DATE_DIFF`, `EXTRACT` and `UTCNOW` along with type conversion using `CAST` to the `TIMESTAMP` data type are currently supported.
- AWS S3's [reserved keywords](https://docs.aws.amazon.com/AmazonS3/latest/dev/s3-glacier-select-sql-reference-keyword-list.html) list is not yet respected.
- CSV input records larger than 1 MiB are rejected with `OverMaxRecordSize`. For line-delimited JSON, the same limit is enforced on the non-`simdjson` fallback path; builds that use `simdjson` continue to rely on the parser's own streaming behavior.
- CSV input fields (even quoted) cannot contain newlines even if `RecordDelimiter` is something else.
+1 -1
View File
@@ -25,7 +25,7 @@ The following Bucket features will **not be replicated**, is designed to differ
- **Removing a site** is not allowed from a set of replicated sites once configured.
- All sites must be using the **same** external IDP(s) if any.
- For [SSE-S3 or SSE-KMS encryption via KMS](https://docs.min.io/community/minio-object-store/operations/server-side-encryption.html "MinIO KMS Guide"), all sites **must** have access to a central KMS deployment. This can be achieved via a central KES server or multiple KES servers (say one per site) connected via a central KMS (Vault) server.
- For [SSE-S3 or SSE-KMS encryption via KMS](https://silo.pigsty.io/operations/server-side-encryption.html "MinIO KMS Guide"), all sites **must** have access to a central KMS deployment. This can be achieved via a central KES server or multiple KES servers (say one per site) connected via a central KMS (Vault) server.
## Configuring Site Replication
+10 -10
View File
@@ -15,17 +15,17 @@ Following are advantages for using temporary credentials:
| AuthN | Description |
| :---------------------- | ------------------------------------------ |
| [**WebIdentity**](https://github.com/minio/minio/blob/master/docs/sts/web-identity.md) | Let users request temporary credentials using any OpenID(OIDC) compatible web identity providers such as KeyCloak, Dex, Facebook, Google etc. |
| [**AD/LDAP**](https://github.com/minio/minio/blob/master/docs/sts/ldap.md) | Let AD/LDAP users request temporary credentials using AD/LDAP username and password. |
| [**AssumeRole**](https://github.com/minio/minio/blob/master/docs/sts/assume-role.md) | Let MinIO users request temporary credentials using user access and secret keys. |
| [**WebIdentity**](https://github.com/pgsty/minio/blob/master/docs/sts/web-identity.md) | Let users request temporary credentials using any OpenID(OIDC) compatible web identity providers such as KeyCloak, Dex, Facebook, Google etc. |
| [**AD/LDAP**](https://github.com/pgsty/minio/blob/master/docs/sts/ldap.md) | Let AD/LDAP users request temporary credentials using AD/LDAP username and password. |
| [**AssumeRole**](https://github.com/pgsty/minio/blob/master/docs/sts/assume-role.md) | Let MinIO users request temporary credentials using user access and secret keys. |
### Understanding JWT Claims
> NOTE: JWT claims are only meant for WebIdentity and ClientGrants.
> AssumeRole or LDAP users can skip the entire portion and directly visit one of the links below.
>
> - [**AssumeRole**](https://github.com/minio/minio/blob/master/docs/sts/assume-role.md)
> - [**AD/LDAP**](https://github.com/minio/minio/blob/master/docs/sts/ldap.md)
> - [**AssumeRole**](https://github.com/pgsty/minio/blob/master/docs/sts/assume-role.md)
> - [**AD/LDAP**](https://github.com/pgsty/minio/blob/master/docs/sts/ldap.md)
The id_token received is a signed JSON Web Token (JWT). Use a JWT decoder to decode the id_token to access the payload of the token that includes following JWT claims, `policy` claim is mandatory and should be present as part of your JWT claim. Without this claim the generated credentials will not have access to any resources on the server, using these credentials application would receive 'Access Denied' errors.
@@ -37,12 +37,12 @@ The id_token received is a signed JSON Web Token (JWT). Use a JWT decoder to dec
In this document we will explain in detail on how to configure all the prerequisites.
> NOTE: If you are interested in AssumeRole API only, skip to [here](https://github.com/minio/minio/blob/master/docs/sts/assume-role.md)
> NOTE: If you are interested in AssumeRole API only, skip to [here](https://github.com/pgsty/minio/blob/master/docs/sts/assume-role.md)
### Prerequisites
- [Configuring keycloak](https://github.com/minio/minio/blob/master/docs/sts/keycloak.md) or [Configuring Casdoor](https://github.com/minio/minio/blob/master/docs/sts/casdoor.md)
- [Configuring etcd](https://github.com/minio/minio/blob/master/docs/sts/etcd.md)
- [Configuring keycloak](https://github.com/pgsty/minio/blob/master/docs/sts/keycloak.md) or [Configuring Casdoor](https://github.com/pgsty/minio/blob/master/docs/sts/casdoor.md)
- [Configuring etcd](https://github.com/pgsty/minio/blob/master/docs/sts/etcd.md)
### Setup MinIO with Identity Provider
@@ -106,5 +106,5 @@ These credentials can now be used to perform MinIO API operations.
## Explore Further
- [MinIO Admin Complete Guide](https://docs.min.io/community/minio-object-store/reference/minio-mc-admin.html)
- [The MinIO documentation website](https://docs.min.io/community/minio-object-store/index.html)
- [MinIO Admin Complete Guide](https://silo.pigsty.io/reference/minio-mc-admin.html)
- [The MinIO documentation website](https://silo.pigsty.io/index.html)
+3 -3
View File
@@ -89,7 +89,7 @@ export MINIO_ROOT_PASSWORD=minio123
minio server ~/test
```
Create new users following the multi-user guide [here](https://docs.min.io/community/minio-object-store/administration/identity-access-management.html)
Create new users following the multi-user guide [here](https://silo.pigsty.io/administration/identity-access-management.html)
### Testing an example with awscli tool
@@ -134,5 +134,5 @@ SessionToken: eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJhY2Nlc3NLZXkiOiIyN1lEUllFT
## Explore Further
- [MinIO Admin Complete Guide](https://docs.min.io/community/minio-object-store/reference/minio-mc-admin.html)
- [The MinIO documentation website](https://docs.min.io/community/minio-object-store/index.html)
- [MinIO Admin Complete Guide](https://silo.pigsty.io/reference/minio-mc-admin.html)
- [The MinIO documentation website](https://silo.pigsty.io/index.html)
+2 -2
View File
@@ -112,5 +112,5 @@ This will open the login page of Casdoor, upon successful login, STS credentials
## Explore Further
- [Casdoor MinIO Integration](https://casdoor.org/docs/integration/minio)
- [MinIO STS Quickstart Guide](https://docs.min.io/community/minio-object-store/developers/security-token-service.html)
- [The MinIO documentation website](https://docs.min.io/community/minio-object-store/index.html)
- [MinIO STS Quickstart Guide](https://silo.pigsty.io/developers/security-token-service.html)
- [The MinIO documentation website](https://silo.pigsty.io/index.html)
+5 -3
View File
@@ -6,6 +6,8 @@ Returns a set of temporary security credentials for applications/clients who hav
Calling AssumeRoleWithClientGrants does not require the use of MinIO default credentials. Therefore, client application can be distributed that requests temporary security credentials without including MinIO default credentials. Instead, the identity of the caller is validated by using a JWT access token from the identity provider. The temporary security credentials returned by this API consists of an access key, a secret key, and a security token. Applications can use these temporary security credentials to sign calls to MinIO API operations.
**Breaking change (`CVE-2026-33322`)**: releases after `RELEASE.2026-03-25T00-00-00Z` validate the JWT access token with the provider JWKS endpoint advertised by the configured OpenID discovery document. In practice, MinIO currently accepts the RSA PKCS#1 v1.5 and ECDSA families already implemented in the verifier (`RS256`, `RS384`, `RS512`, `ES256`, `ES384`, `ES512`, plus the existing `RS3*` and `ES3*` aliases). HMAC-signed tokens such as `HS256`, `HS384`, and `HS512` are rejected. Algorithms such as `PS256` and `EdDSA` are not currently supported. `MINIO_IDENTITY_OPENID_CLIENT_SECRET` is used for OpenID/OAuth client interactions only and is not used as a JWT verification key. If your provider previously issued HMAC-signed access tokens for this flow, reconfigure it to publish RSA or ECDSA signing keys through JWKS before upgrading.
By default, the temporary security credentials created by AssumeRoleWithClientGrants last for one hour. However, use the optional DurationSeconds parameter to specify the duration of the credentials. This value varies from 900 seconds (15 minutes) up to the maximum session duration of 365 days.
## API Request Parameters
@@ -95,7 +97,7 @@ minio server /mnt/export
```
Testing with an example
> Obtaining client ID and secrets follow [Keycloak configuring documentation](https://github.com/minio/minio/blob/master/docs/sts/keycloak.md)
> Obtaining client ID and secrets follow [Keycloak configuring documentation](https://github.com/pgsty/minio/blob/master/docs/sts/keycloak.md)
```
$ go run client-grants.go -cid PoEgXP6uVO45IsENRngDXj5Au5Ya -csec eKsw6z8CtOJVBtrOWvhRWL4TUCga
@@ -111,5 +113,5 @@ $ go run client-grants.go -cid PoEgXP6uVO45IsENRngDXj5Au5Ya -csec eKsw6z8CtOJVBt
## Explore Further
- [MinIO Admin Complete Guide](https://docs.min.io/community/minio-object-store/reference/minio-mc-admin.html)
- [The MinIO documentation website](https://docs.min.io/community/minio-object-store/index.html)
- [MinIO Admin Complete Guide](https://silo.pigsty.io/reference/minio-mc-admin.html)
- [The MinIO documentation website](https://silo.pigsty.io/index.html)
+2 -2
View File
@@ -99,5 +99,5 @@ and add relevant policies on MinIO using `mc admin policy create myminio/ <group
## Explore Further
- [MinIO STS Quickstart Guide](https://docs.min.io/community/minio-object-store/developers/security-token-service.html)
- [The MinIO documentation website](https://docs.min.io/community/minio-object-store/index.html)
- [MinIO STS Quickstart Guide](https://silo.pigsty.io/developers/security-token-service.html)
- [The MinIO documentation website](https://silo.pigsty.io/index.html)
+4 -4
View File
@@ -50,7 +50,7 @@ NOTE: If `etcd` is configured with `Client-to-server authentication with HTTPS c
Once etcd is configured, **any STS configuration** will work including Client Grants, Web Identity or AD/LDAP.
For example, you can configure STS with Client Grants (KeyCloak) using the guides at [MinIO STS Quickstart Guide](https://docs.min.io/community/minio-object-store/developers/security-token-service.html) and [KeyCloak Configuration Guide](https://github.com/minio/minio/blob/master/docs/sts/keycloak.md). Once this is done, STS credentials can be generated:
For example, you can configure STS with Client Grants (KeyCloak) using the guides at [MinIO STS Quickstart Guide](https://silo.pigsty.io/developers/security-token-service.html) and [KeyCloak Configuration Guide](https://github.com/pgsty/minio/blob/master/docs/sts/keycloak.md). Once this is done, STS credentials can be generated:
```
go run client-grants.go -cid PoEgXP6uVO45IsENRngDXj5Au5Ya -csec eKsw6z8CtOJVBtrOWvhRWL4TUCga
@@ -64,9 +64,9 @@ go run client-grants.go -cid PoEgXP6uVO45IsENRngDXj5Au5Ya -csec eKsw6z8CtOJVBtrO
}
```
These credentials can now be used to perform MinIO API operations, these credentials automatically expire in 1hr. To understand more about credential expiry duration and client grants STS API read further [here](https://github.com/minio/minio/blob/master/docs/sts/client-grants.md).
These credentials can now be used to perform MinIO API operations, these credentials automatically expire in 1hr. To understand more about credential expiry duration and client grants STS API read further [here](https://github.com/pgsty/minio/blob/master/docs/sts/client-grants.md).
## Explore Further
- [MinIO STS Quickstart Guide](https://docs.min.io/community/minio-object-store/developers/security-token-service.html)
- [The MinIO documentation website](https://docs.min.io/community/minio-object-store/index.html)
- [MinIO STS Quickstart Guide](https://silo.pigsty.io/developers/security-token-service.html)
- [The MinIO documentation website](https://silo.pigsty.io/index.html)
+2 -2
View File
@@ -172,5 +172,5 @@ These credentials can now be used to perform MinIO API operations.
## Explore Further
- [MinIO STS Quickstart Guide](https://docs.min.io/community/minio-object-store/developers/security-token-service.html)
- [The MinIO documentation website](https://docs.min.io/community/minio-object-store/index.html)
- [MinIO STS Quickstart Guide](https://silo.pigsty.io/developers/security-token-service.html)
- [The MinIO documentation website](https://silo.pigsty.io/index.html)
+49 -3
View File
@@ -46,6 +46,7 @@ MINIO_IDENTITY_LDAP_GROUP_SEARCH_BASE_DN (list) ";" separated list of gr
MINIO_IDENTITY_LDAP_TLS_SKIP_VERIFY (on|off) trust server TLS without verification (default: 'off')
MINIO_IDENTITY_LDAP_SERVER_INSECURE (on|off) allow plain text connection to AD/LDAP server (default: 'off')
MINIO_IDENTITY_LDAP_SERVER_STARTTLS (on|off) use StartTLS connection to AD/LDAP server (default: 'off')
MINIO_IDENTITY_LDAP_STS_TRUSTED_PROXIES (list) comma/semicolon/whitespace-separated list of trusted proxy IPs or CIDRs whose forwarded client IP headers may be used for LDAP STS rate limiting
MINIO_IDENTITY_LDAP_COMMENT (sentence) optionally add a comment to this setting
```
@@ -81,6 +82,43 @@ With the default (empty) value for `srv_record_name`, MinIO **will not** perform
The value of `srv_record_name` does not affect any TLS settings - they must be configured with their own parameters.
### LDAP STS rate limiting
LDAP STS rate limiting is enforced before each LDAP bind. Requests are tracked by source IP. A login attempt is throttled when that bucket is exhausted. Rate limiting is deliberately not keyed by username: a username-keyed bucket is shared across all sources, so an attacker could drain a known account's bucket with bad-password attempts and lock the legitimate user out. The per-source bucket caps the attempt rate from any single source, while the uniform auth-failure response is what conceals whether a username exists. This combination does not stop attackers who spread requests across many source IPs (botnets, IPv6 address rotation) or the residual bind-timing side channel; those are accepted limitations of an in-memory, per-source control.
By default, the source IP used for this key is the socket peer address. This is the safe default because MinIO does **not** trust `X-Forwarded-For`, `X-Real-IP`, or `Forwarded` headers for this security-sensitive rate-limit key unless you opt in explicitly.
Each login attempt reserves capacity for the duration of the LDAP bind. Successful logins and LDAP infrastructure failures refund that reservation. Only real authentication failures permanently consume tokens. Concurrent bursts above the burst capacity can still receive `429` responses even if those in-flight requests later succeed.
| Behavior | Value |
| :-- | :-- |
| Bucket key | Source IP |
| Burst capacity | 10 attempts per bucket |
| Refill rate | 1 token every 6 seconds, about 10 attempts per minute per bucket |
| Reservation lifetime | Held for the duration of the LDAP bind |
| Idle entry cleanup | Bucket state is removed after 15 minutes without activity |
| Throttled response | HTTP `429`, STS code `ThrottlingException`, `Retry-After: 6` |
| Scope | Per-node, in-memory, not cluster-wide |
| Configurability | Not currently configurable |
In a multi-node deployment behind a load balancer, each MinIO node tracks its own buckets. The effective aggregate budget therefore depends on how requests are distributed across nodes.
#### Trusted proxies
If MinIO is deployed behind a trusted reverse proxy, load balancer, or API gateway and you want LDAP STS throttling to bucket by the forwarded client IP instead of the proxy peer address, configure:
```
MINIO_IDENTITY_LDAP_STS_TRUSTED_PROXIES (list) comma/semicolon/whitespace-separated list of trusted proxy IPs or CIDRs
```
Only requests whose peer address matches this allowlist may supply forwarded client IP headers for LDAP STS rate limiting. Requests from all other peers continue to use the peer address directly. Catch-all ranges (`0.0.0.0/0`, `::/0`) are rejected, since they would trust forwarded headers from every peer.
`X-Forwarded-For` is parsed right-to-left, skipping addresses that match the trusted-proxy allowlist, and the first untrusted address is used. A client-supplied (left-most) value is therefore ignored unless the entire chain to its right is trusted — so even nginx's default appending `proxy_add_x_forwarded_for` is safe. List every proxy hop's address in the allowlist so intermediate hops are skipped.
`X-Real-IP` is preferred over `X-Forwarded-For` when present, but — unlike `X-Forwarded-For` — it is a single value that **cannot** be chain-validated against the allowlist, so it is trusted verbatim. **The trusted proxy must overwrite (not pass through) any client-supplied `X-Real-IP`.** If the proxy forwards a client-supplied value, an attacker can send a different `X-Real-IP` on each request to land in a fresh bucket and bypass per-source throttling. When in doubt, configure the proxy to set `X-Real-IP` from the connecting peer (e.g. nginx `proxy_set_header X-Real-IP $remote_addr`), or omit `X-Real-IP` and rely on the chain-validated `X-Forwarded-For`.
The RFC 7239 `Forwarded` header is not used for this bucket; deployments that only send `Forwarded` fall back to the peer-address bucket.
### Lookup-Bind
A low-privilege read-only LDAP service account is configured in the MinIO server by providing the account's Distinguished Name (DN) and password. This service account is used to perform directory lookups as needed.
@@ -261,6 +299,14 @@ XML response for this API is similar to [AWS STS AssumeRoleWithWebIdentity](http
XML error response for this API is similar to [AWS STS AssumeRoleWithWebIdentity](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRoleWithWebIdentity.html#API_AssumeRoleWithWebIdentity_Errors)
Authentication failures caused by unknown users and invalid passwords intentionally return the same external STS error response to prevent username enumeration via response differentiation (`CVE-2026-33419`).
| Condition | HTTP status | STS code | Notes |
| :-- | :-- | :-- | :-- |
| Unknown user or invalid password | `400` | `InvalidParameterValue` | Same response for both cases to prevent username enumeration |
| LDAP backend, network, or other infrastructure failure | `500` | `InternalError` | Returned as an upstream/internal failure and logged server-side |
| Rate limit exhausted | `429` | `ThrottlingException` | Includes a `Retry-After` header |
## Sample `POST` Request
```
@@ -303,7 +349,7 @@ export MINIO_IDENTITY_LDAP_GROUP_SEARCH_FILTER='(&(objectclass=groupOfNames)(mem
minio server ~/test
```
You can make sure it works appropriately using our [example program](https://raw.githubusercontent.com/minio/minio/master/docs/sts/ldap.go):
You can make sure it works appropriately using our [example program](https://raw.githubusercontent.com/pgsty/minio/master/docs/sts/ldap.go):
```
$ go run ldap.go -u foouser -p foopassword
@@ -319,5 +365,5 @@ $ go run ldap.go -u foouser -p foopassword
## Explore Further
- [MinIO Admin Complete Guide](https://docs.min.io/community/minio-object-store/reference/minio-mc-admin.html)
- [The MinIO documentation website](https://docs.min.io/community/minio-object-store/index.html)
- [MinIO Admin Complete Guide](https://silo.pigsty.io/reference/minio-mc-admin.html)
- [The MinIO documentation website](https://silo.pigsty.io/index.html)
+2 -2
View File
@@ -113,5 +113,5 @@ Further, the temp. S3 credentials will never out-live the client certificate. Fo
## Explore Further
- [MinIO Admin Complete Guide](https://docs.min.io/community/minio-object-store/reference/minio-mc-admin.html)
- [The MinIO documentation website](https://docs.min.io/community/minio-object-store/index.html)
- [MinIO Admin Complete Guide](https://silo.pigsty.io/reference/minio-mc-admin.html)
- [The MinIO documentation website](https://silo.pigsty.io/index.html)
+4 -2
View File
@@ -6,6 +6,8 @@ MinIO supports the standard AssumeRoleWithWebIdentity STS API to enable integrat
Calling AssumeRoleWithWebIdentity does not require the use of MinIO root or IAM credentials. Therefore, you can distribute an application (for example, on mobile devices) that requests temporary security credentials without including MinIO long lasting credentials in the application. Instead, the identity of the caller is validated by using a JWT id_token from the web identity provider. The temporary security credentials returned by this API consists of an access key, a secret key, and a security token. Applications can use these temporary security credentials to sign calls to MinIO API operations.
**Breaking change (`CVE-2026-33322`)**: releases after `RELEASE.2026-03-25T00-00-00Z` validate the `id_token` with the provider JWKS endpoint advertised by the OpenID discovery document at `MINIO_IDENTITY_OPENID_CONFIG_URL`. In practice, MinIO currently accepts the RSA PKCS#1 v1.5 and ECDSA families already implemented in the verifier (`RS256`, `RS384`, `RS512`, `ES256`, `ES384`, `ES512`, plus the existing `RS3*` and `ES3*` aliases). HMAC-signed tokens such as `HS256`, `HS384`, and `HS512` are rejected. Algorithms such as `PS256` and `EdDSA` are not currently supported. `MINIO_IDENTITY_OPENID_CLIENT_SECRET` is used for OpenID/OAuth client interactions only and is not used as a JWT verification key. If your provider previously issued HMAC-signed `id_token`s, reconfigure it to publish RSA or ECDSA signing keys through JWKS before upgrading.
By default, the temporary security credentials created by AssumeRoleWithWebIdentity last for one hour. However, the optional DurationSeconds parameter can be used to specify the validity duration of the generated credentials. This value varies from 900 seconds (15 minutes) up to the maximum session duration of 365 days.
## Configuring OpenID Identity Provider on MinIO
@@ -271,5 +273,5 @@ JWT token returned by the Identity Provider should include a custom claim for th
## Explore Further
- [MinIO Admin Complete Guide](https://docs.min.io/community/minio-object-store/reference/minio-mc-admin.html)
- [The MinIO documentation website](https://docs.min.io/community/minio-object-store/index.html)
- [MinIO Admin Complete Guide](https://silo.pigsty.io/reference/minio-mc-admin.html)
- [The MinIO documentation website](https://silo.pigsty.io/index.html)
+4 -4
View File
@@ -87,7 +87,7 @@ export MINIO_IDENTITY_OPENID_CLIENT_ID="843351d4-1080-11ea-aa20-271ecba3924a"
minio server /mnt/data
```
Assuming that MinIO server is configured to support STS API by following the doc [MinIO STS Quickstart Guide](https://docs.min.io/community/minio-object-store/developers/security-token-service.html), execute the following command to temporary credentials from MinIO server.
Assuming that MinIO server is configured to support STS API by following the doc [MinIO STS Quickstart Guide](https://silo.pigsty.io/developers/security-token-service.html), execute the following command to temporary credentials from MinIO server.
```
go run client-grants.go -cid PoEgXP6uVO45IsENRngDXj5Au5Ya -csec eKsw6z8CtOJVBtrOWvhRWL4TUCga
@@ -101,9 +101,9 @@ go run client-grants.go -cid PoEgXP6uVO45IsENRngDXj5Au5Ya -csec eKsw6z8CtOJVBtrO
}
```
These credentials can now be used to perform MinIO API operations, these credentials automatically expire in 1hr. To understand more about credential expiry duration and client grants STS API read further [here](https://github.com/minio/minio/blob/master/docs/sts/client-grants.md).
These credentials can now be used to perform MinIO API operations, these credentials automatically expire in 1hr. To understand more about credential expiry duration and client grants STS API read further [here](https://github.com/pgsty/minio/blob/master/docs/sts/client-grants.md).
## Explore Further
- [MinIO STS Quickstart Guide](https://docs.min.io/community/minio-object-store/developers/security-token-service.html)
- [The MinIO documentation website](https://docs.min.io/community/minio-object-store/index.html)
- [MinIO STS Quickstart Guide](https://silo.pigsty.io/developers/security-token-service.html)
- [The MinIO documentation website](https://silo.pigsty.io/index.html)
+7 -7
View File
@@ -9,11 +9,11 @@ This guide explains how to configure MinIO Server with TLS certificates on Linux
## 1. Install MinIO Server
Install MinIO Server using the instructions in the [MinIO Quickstart Guide](https://docs.min.io/community/minio-object-store/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html).
Install MinIO Server using the instructions in the [MinIO Quickstart Guide](https://silo.pigsty.io/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html).
## 2. Use an Existing Key and Certificate with MinIO
This section describes how to use a private key and public certificate that have been obtained from a certificate authority (CA). If these files have not been obtained, skip to [3. Generate Self-signed Certificates](#generate-use-self-signed-keys-certificates) or generate them with [Let's Encrypt](https://letsencrypt.org) using these instructions: [Generate Let's Encrypt certificate using Certbot for MinIO](https://docs.min.io/community/minio-object-store/integrations/generate-lets-encrypt-certificate-using-certbot-for-minio.html). For more about TLS and certificates in MinIO, see the [Network Encryption documentation](https://docs.min.io/community/minio-object-store/operations/network-encryption.html).
This section describes how to use a private key and public certificate that have been obtained from a certificate authority (CA). If these files have not been obtained, skip to [3. Generate Self-signed Certificates](#generate-use-self-signed-keys-certificates) or generate them with [Let's Encrypt](https://letsencrypt.org) using these instructions: [Generate Let's Encrypt certificate using Certbot for MinIO](https://silo.pigsty.io/integrations/generate-lets-encrypt-certificate-using-certbot-for-minio.html). For more about TLS and certificates in MinIO, see the [Network Encryption documentation](https://silo.pigsty.io/operations/network-encryption.html).
Copy the existing private key and public certificate to the `certs` directory. The default certs directory is:
@@ -237,8 +237,8 @@ MinIO can connect to other servers, including MinIO nodes or other server types
## Explore Further
* [TLS Configuration for MinIO server on Kubernetes](https://github.com/minio/minio/tree/master/docs/tls/kubernetes)
* [MinIO Client Complete Guide](https://docs.min.io/community/minio-object-store/reference/minio-mc.html)
* [MinIO Network Encryption Overview](https://docs.min.io/community/minio-object-store/operations/network-encryption.html)
* [Generate Let's Encrypt Certificate](https://docs.min.io/community/minio-object-store/integrations/generate-lets-encrypt-certificate-using-certbot-for-minio.html)
* [Setup nginx Proxy with MinIO Server](https://docs.min.io/community/minio-object-store/integrations/setup-nginx-proxy-with-minio.html)
* [TLS Configuration for MinIO server on Kubernetes](https://github.com/pgsty/minio/tree/master/docs/tls/kubernetes)
* [MinIO Client Complete Guide](https://silo.pigsty.io/reference/minio-mc.html)
* [MinIO Network Encryption Overview](https://silo.pigsty.io/operations/network-encryption.html)
* [Generate Let's Encrypt Certificate](https://silo.pigsty.io/integrations/generate-lets-encrypt-certificate-using-certbot-for-minio.html)
* [Setup nginx Proxy with MinIO Server](https://silo.pigsty.io/integrations/setup-nginx-proxy-with-minio.html)
+4 -4
View File
@@ -4,15 +4,15 @@ This document explains how to configure MinIO server with TLS certificates on Ku
## 1. Prerequisites
- Familiarity with [MinIO deployment process on Kubernetes](https://docs.min.io/community/minio-object-store/operations/deployments/kubernetes.html).
- Familiarity with [MinIO deployment process on Kubernetes](https://silo.pigsty.io/operations/deployments/kubernetes.html).
- Kubernetes cluster with `kubectl` configured.
- Acquire TLS certificates, either from a CA or [create self-signed certificates](https://docs.min.io/community/minio-object-store/operations/network-encryption.html).
- Acquire TLS certificates, either from a CA or [create self-signed certificates](https://silo.pigsty.io/operations/network-encryption.html).
For a [distributed MinIO setup](https://docs.min.io/community/minio-object-store/operations/deployments/kubernetes.html), where there are multiple pods with different domain names expected to run, you will either need wildcard certificates valid for all the domains or have specific certificates for each domain. If you are going to use specific certificates, make sure to create Kubernetes secrets accordingly.
For a [distributed MinIO setup](https://silo.pigsty.io/operations/deployments/kubernetes.html), where there are multiple pods with different domain names expected to run, you will either need wildcard certificates valid for all the domains or have specific certificates for each domain. If you are going to use specific certificates, make sure to create Kubernetes secrets accordingly.
For testing purposes, here is [how to create self-signed certificates](https://github.com/minio/minio/tree/master/docs/tls#3-generate-self-signed-certificates).
For testing purposes, here is [how to create self-signed certificates](https://github.com/pgsty/minio/tree/master/docs/tls#3-generate-self-signed-certificates).
## 2. Create Kubernetes secret
+1 -1
View File
@@ -11,7 +11,7 @@ Please make sure the following packages are already installed via `dnf` or `apt`
#### Step 1 - download `tuned.conf` from the referenced link
```
wget https://raw.githubusercontent.com/minio/minio/master/docs/tuning/tuned.conf
wget https://raw.githubusercontent.com/pgsty/minio/master/docs/tuning/tuned.conf
```
#### Step 2 - install tuned.conf as supported performance profile on all nodes
+39 -38
View File
@@ -1,6 +1,6 @@
module github.com/minio/minio
go 1.26.1
go 1.26.4
// Use Georg Mangold's maintained Console fork while keeping upstream import paths.
replace github.com/minio/console => github.com/georgmangold/console v1.9.1
@@ -19,7 +19,7 @@ tool (
require (
aead.dev/mtls v0.3.0
cloud.google.com/go/storage v1.61.3
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.21.0
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.21.1
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.13.1
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.6.4
github.com/IBM/sarama v1.45.1
@@ -47,7 +47,7 @@ require (
github.com/google/uuid v1.6.0
github.com/inconshreveable/mousetrap v1.1.0
github.com/json-iterator/go v1.1.12
github.com/klauspost/compress v1.18.5
github.com/klauspost/compress v1.18.6
github.com/klauspost/cpuid/v2 v2.3.0
github.com/klauspost/filepathx v1.1.1
github.com/klauspost/pgzip v1.2.6
@@ -61,7 +61,7 @@ require (
github.com/minio/csvparser v1.0.0
github.com/minio/dnscache v0.1.1
github.com/minio/dperf v0.7.1
github.com/minio/highwayhash v1.0.3
github.com/minio/highwayhash v1.0.4-0.20251030100505-070ab1a87a76
github.com/minio/kms-go/kes v0.3.1
github.com/minio/kms-go/kms v0.6.0
github.com/minio/madmin-go/v3 v3.0.110
@@ -74,7 +74,7 @@ require (
github.com/minio/xxml v0.0.3
github.com/minio/zipindex v0.5.0
github.com/mitchellh/go-homedir v1.1.0
github.com/nats-io/nats-server/v2 v2.11.1
github.com/nats-io/nats-server/v2 v2.11.15
github.com/nats-io/nats.go v1.49.0
github.com/nats-io/stan.go v0.10.4
github.com/ncw/directio v1.0.5
@@ -101,16 +101,16 @@ require (
go.etcd.io/etcd/api/v3 v3.6.8
go.etcd.io/etcd/client/v3 v3.6.8
go.uber.org/atomic v1.11.0
go.uber.org/zap v1.27.1
go.uber.org/zap v1.28.0
go.yaml.in/yaml/v3 v3.0.4
goftp.io/server/v2 v2.0.2
golang.org/x/crypto v0.49.0
golang.org/x/crypto v0.52.0
golang.org/x/oauth2 v0.36.0
golang.org/x/sync v0.20.0
golang.org/x/sys v0.42.0
golang.org/x/term v0.41.0
golang.org/x/sys v0.45.0
golang.org/x/term v0.43.0
golang.org/x/time v0.15.0
google.golang.org/api v0.272.0
google.golang.org/api v0.278.0
gopkg.in/yaml.v2 v2.4.0
)
@@ -119,21 +119,22 @@ require (
aead.dev/minisign v0.3.0 // indirect
cel.dev/expr v0.25.1 // indirect
cloud.google.com/go v0.123.0 // indirect
cloud.google.com/go/auth v0.18.2 // indirect
cloud.google.com/go/auth v0.20.0 // indirect
cloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect
cloud.google.com/go/compute/metadata v0.9.0 // indirect
cloud.google.com/go/iam v1.5.3 // indirect
cloud.google.com/go/monitoring v1.24.3 // indirect
filippo.io/edwards25519 v1.2.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/internal v1.11.2 // indirect
github.com/Azure/go-ntlmssp v0.1.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/internal v1.12.0 // indirect
github.com/Azure/go-ntlmssp v0.1.1 // indirect
github.com/AzureAD/microsoft-authentication-library-for-go v1.7.0 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.31.0 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.55.0 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.55.0 // indirect
github.com/VividCortex/ewma v1.2.0 // indirect
github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d // indirect
github.com/apache/thrift v0.22.0 // indirect
github.com/antithesishq/antithesis-sdk-go v0.6.0-default-no-op // indirect
github.com/apache/thrift v0.23.0 // indirect
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/charmbracelet/bubbles v1.0.0 // indirect
@@ -161,22 +162,22 @@ require (
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/go-asn1-ber/asn1-ber v1.5.8-0.20250403174932-29230038a667 // indirect
github.com/go-ini/ini v1.67.0 // indirect
github.com/go-jose/go-jose/v4 v4.1.3 // indirect
github.com/go-jose/go-jose/v4 v4.1.4 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-ole/go-ole v1.3.0 // indirect
github.com/go-openapi/analysis v0.25.0 // indirect
github.com/go-openapi/errors v0.22.7 // indirect
github.com/go-openapi/jsonpointer v0.22.5 // indirect
github.com/go-openapi/jsonpointer v0.23.1 // indirect
github.com/go-openapi/jsonreference v0.21.5 // indirect
github.com/go-openapi/runtime v0.29.3 // indirect
github.com/go-openapi/spec v0.22.4 // indirect
github.com/go-openapi/strfmt v0.26.1 // indirect
github.com/go-openapi/strfmt v0.26.2 // indirect
github.com/go-openapi/swag v0.25.5 // indirect
github.com/go-openapi/swag/cmdutils v0.25.5 // indirect
github.com/go-openapi/swag/conv v0.25.5 // indirect
github.com/go-openapi/swag/fileutils v0.25.5 // indirect
github.com/go-openapi/swag/jsonname v0.25.5 // indirect
github.com/go-openapi/swag/jsonname v0.26.0 // indirect
github.com/go-openapi/swag/jsonutils v0.25.5 // indirect
github.com/go-openapi/swag/loading v0.25.5 // indirect
github.com/go-openapi/swag/mangling v0.25.5 // indirect
@@ -193,16 +194,16 @@ require (
github.com/golang-jwt/jwt/v5 v5.3.1 // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/golang/snappy v1.0.0 // indirect
github.com/google/go-tpm v0.9.3 // indirect
github.com/google/pprof v0.0.0-20260302011040-a15ffb7f9dcc // indirect
github.com/google/go-tpm v0.9.8 // indirect
github.com/google/pprof v0.0.0-20260507013755-92041b743c96 // indirect
github.com/google/s2a-go v0.1.9 // indirect
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.14 // indirect
github.com/googleapis/gax-go/v2 v2.19.0 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.15 // indirect
github.com/googleapis/gax-go/v2 v2.22.0 // indirect
github.com/gorilla/mux v1.8.1 // indirect
github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674 // indirect
github.com/grafana/regexp v0.0.0-20250905093917-f7b3be9d1853 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.28.0 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.29.0 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/hashicorp/go-uuid v1.0.3 // indirect
@@ -247,7 +248,7 @@ require (
github.com/muesli/reflow v0.3.0 // indirect
github.com/muesli/termenv v0.16.0 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/nats-io/jwt/v2 v2.7.4 // indirect
github.com/nats-io/jwt/v2 v2.8.1 // indirect
github.com/nats-io/nats-streaming-server v0.24.6 // indirect
github.com/nats-io/nkeys v0.4.15 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
@@ -258,7 +259,7 @@ require (
github.com/posener/complete v1.2.3 // indirect
github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 // indirect
github.com/prometheus/prom2json v1.5.0 // indirect
github.com/prometheus/prometheus v0.310.0 // indirect
github.com/prometheus/prometheus v0.311.3 // indirect
github.com/rivo/uniseg v0.4.7 // indirect
github.com/rjeczalik/notify v0.9.3 // indirect
github.com/rs/xid v1.6.0 // indirect
@@ -281,21 +282,21 @@ require (
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/detectors/gcp v1.42.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.67.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.67.0 // indirect
go.opentelemetry.io/otel v1.42.0 // indirect
go.opentelemetry.io/otel/metric v1.42.0 // indirect
go.opentelemetry.io/otel/sdk v1.42.0 // indirect
go.opentelemetry.io/otel/sdk/metric v1.42.0 // indirect
go.opentelemetry.io/otel/trace v1.42.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.69.0 // indirect
go.opentelemetry.io/otel v1.44.0 // indirect
go.opentelemetry.io/otel/metric v1.44.0 // indirect
go.opentelemetry.io/otel/sdk v1.44.0 // indirect
go.opentelemetry.io/otel/sdk/metric v1.44.0 // indirect
go.opentelemetry.io/otel/trace v1.44.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.yaml.in/yaml/v2 v2.4.4 // indirect
golang.org/x/mod v0.34.0 // indirect
golang.org/x/net v0.52.0 // indirect
golang.org/x/text v0.35.0 // indirect
golang.org/x/tools v0.43.0 // indirect
golang.org/x/mod v0.36.0 // indirect
golang.org/x/net v0.55.0 // indirect
golang.org/x/text v0.37.0 // indirect
golang.org/x/tools v0.45.0 // indirect
google.golang.org/genproto v0.0.0-20260319201613-d00831a3d3e7 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20260319201613-d00831a3d3e7 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260319201613-d00831a3d3e7 // indirect
google.golang.org/grpc v1.79.3 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20260526163538-3dc84a4a5aaa // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260526163538-3dc84a4a5aaa // indirect
google.golang.org/grpc v1.81.1 // indirect
google.golang.org/protobuf v1.36.11 // indirect
)
+84 -82
View File
@@ -9,8 +9,8 @@ cel.dev/expr v0.25.1 h1:1KrZg61W6TWSxuNZ37Xy49ps13NUovb66QLprthtwi4=
cel.dev/expr v0.25.1/go.mod h1:hrXvqGP6G6gyx8UAHSHJ5RGk//1Oj5nXQ2NI02Nrsg4=
cloud.google.com/go v0.123.0 h1:2NAUJwPR47q+E35uaJeYoNhuNEM9kM8SjgRgdeOJUSE=
cloud.google.com/go v0.123.0/go.mod h1:xBoMV08QcqUGuPW65Qfm1o9Y4zKZBpGS+7bImXLTAZU=
cloud.google.com/go/auth v0.18.2 h1:+Nbt5Ev0xEqxlNjd6c+yYUeosQ5TtEUaNcN/3FozlaM=
cloud.google.com/go/auth v0.18.2/go.mod h1:xD+oY7gcahcu7G2SG2DsBerfFxgPAJz17zz2joOFF3M=
cloud.google.com/go/auth v0.20.0 h1:kXTssoVb4azsVDoUiF8KvxAqrsQcQtB53DcSgta74CA=
cloud.google.com/go/auth v0.20.0/go.mod h1:942/yi/itH1SsmpyrbnTMDgGfdy2BUqIKyd0cyYLc5Q=
cloud.google.com/go/auth/oauth2adapt v0.2.8 h1:keo8NaayQZ6wimpNSmW5OPc283g65QNIiLpZnkHRbnc=
cloud.google.com/go/auth/oauth2adapt v0.2.8/go.mod h1:XQ9y31RkqZCcwJWNSx2Xvric3RrU88hAYYbjDWYDL+c=
cloud.google.com/go/compute/metadata v0.9.0 h1:pDUj4QMoPejqq20dK0Pg2N4yG9zIkYGdBtwLoEkH9Zs=
@@ -29,20 +29,20 @@ cloud.google.com/go/trace v1.11.7 h1:kDNDX8JkaAG3R2nq1lIdkb7FCSi1rCmsEtKVsty7p+U
cloud.google.com/go/trace v1.11.7/go.mod h1:TNn9d5V3fQVf6s4SCveVMIBS2LJUqo73GACmq/Tky0s=
filippo.io/edwards25519 v1.2.0 h1:crnVqOiS4jqYleHd9vaKZ+HKtHfllngJIiOpNpoJsjo=
filippo.io/edwards25519 v1.2.0/go.mod h1:xzAOLCNug/yB62zG1bQ8uziwrIqIuxhctzJT18Q77mc=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.21.0 h1:fou+2+WFTib47nS+nz/ozhEBnvU96bKHy6LjRsY4E28=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.21.0/go.mod h1:t76Ruy8AHvUAC8GfMWJMa0ElSbuIcO03NLpynfbgsPA=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.21.1 h1:jHb/wfvRikGdxMXYV3QG/SzUOPYN9KEUUuC0Yd0/vC0=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.21.1/go.mod h1:pzBXCYn05zvYIrwLgtK8Ap8QcjRg+0i76tMQdWN6wOk=
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.13.1 h1:Hk5QBxZQC1jb2Fwj6mpzme37xbCDdNTxU7O9eb5+LB4=
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.13.1/go.mod h1:IYus9qsFobWIc2YVwe/WPjcnyCkPKtnHAqUYeebc8z0=
github.com/Azure/azure-sdk-for-go/sdk/azidentity/cache v0.3.2 h1:yz1bePFlP5Vws5+8ez6T3HWXPmwOK7Yvq8QxDBD3SKY=
github.com/Azure/azure-sdk-for-go/sdk/azidentity/cache v0.3.2/go.mod h1:Pa9ZNPuoNu/GztvBSKk9J1cDJW6vk/n0zLtV4mgd8N8=
github.com/Azure/azure-sdk-for-go/sdk/internal v1.11.2 h1:9iefClla7iYpfYWdzPCRDozdmndjTm8DXdpCzPajMgA=
github.com/Azure/azure-sdk-for-go/sdk/internal v1.11.2/go.mod h1:XtLgD3ZD34DAaVIIAyG3objl5DynM3CQ/vMcbBNJZGI=
github.com/Azure/azure-sdk-for-go/sdk/internal v1.12.0 h1:fhqpLE3UEXi9lPaBRpQ6XuRW0nU7hgg4zlmZZa+a9q4=
github.com/Azure/azure-sdk-for-go/sdk/internal v1.12.0/go.mod h1:7dCRMLwisfRH3dBupKeNCioWYUZ4SS09Z14H+7i8ZoY=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/storage/armstorage v1.8.1 h1:/Zt+cDPnpC3OVDm/JKLOs7M2DKmLRIIp3XIx9pHHiig=
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/storage/armstorage v1.8.1/go.mod h1:Ng3urmn6dYe8gnbCMoHHVl5APYz2txho3koEkV2o2HA=
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.6.4 h1:jWQK1GI+LeGGUKBADtcH2rRqPxYB1Ljwms5gFA2LqrM=
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.6.4/go.mod h1:8mwH4klAm9DUgR2EEHyEEAQlRDvLPyg5fQry3y+cDew=
github.com/Azure/go-ntlmssp v0.1.0 h1:DjFo6YtWzNqNvQdrwEyr/e4nhU3vRiwenz5QX7sFz+A=
github.com/Azure/go-ntlmssp v0.1.0/go.mod h1:NYqdhxd/8aAct/s4qSYZEerdPuH1liG2/X9DiVTbhpk=
github.com/Azure/go-ntlmssp v0.1.1 h1:l+FM/EEMb0U9QZE7mKNEDw5Mu3mFiaa2GKOoTSsNDPw=
github.com/Azure/go-ntlmssp v0.1.1/go.mod h1:NYqdhxd/8aAct/s4qSYZEerdPuH1liG2/X9DiVTbhpk=
github.com/AzureAD/microsoft-authentication-extensions-for-go/cache v0.1.1 h1:WJTmL004Abzc5wDB5VtZG2PJk5ndYDgVacGqfirKxjM=
github.com/AzureAD/microsoft-authentication-extensions-for-go/cache v0.1.1/go.mod h1:tCcJZ0uHAmvjsVYzEFivsRTN00oz5BEsRgQHu5JZ9WE=
github.com/AzureAD/microsoft-authentication-library-for-go v1.7.0 h1:4iB+IesclUXdP0ICgAabvq2FYLXrJWKx1fJQ+GxSo3Y=
@@ -68,11 +68,11 @@ github.com/alecthomas/participle v0.7.1/go.mod h1:HfdmEuwvr12HXQN44HPWXR0lHmVolV
github.com/alecthomas/repr v0.0.0-20181024024818-d37bc2a10ba1/go.mod h1:xTS7Pm1pD1mvyM075QCDSRqH6qRLXylzS24ZTpRiSzQ=
github.com/alexbrainman/sspi v0.0.0-20250919150558-7d374ff0d59e h1:4dAU9FXIyQktpoUAgOJK3OTFc/xug0PCXYCqU0FgDKI=
github.com/alexbrainman/sspi v0.0.0-20250919150558-7d374ff0d59e/go.mod h1:cEWa1LVoE5KvSD9ONXsZrj0z6KqySlCCNKHlLzbqAt4=
github.com/antithesishq/antithesis-sdk-go v0.4.3-default-no-op h1:+OSa/t11TFhqfrX0EOSqQBDJ0YlpmK0rDSiB19dg9M0=
github.com/antithesishq/antithesis-sdk-go v0.4.3-default-no-op/go.mod h1:IUpT2DPAKh6i/YhSbt6Gl3v2yvUZjmKncl7U91fup7E=
github.com/antithesishq/antithesis-sdk-go v0.6.0-default-no-op h1:kpBdlEPbRvff0mDD1gk7o9BhI16b9p5yYAXRlidpqJE=
github.com/antithesishq/antithesis-sdk-go v0.6.0-default-no-op/go.mod h1:IUpT2DPAKh6i/YhSbt6Gl3v2yvUZjmKncl7U91fup7E=
github.com/apache/thrift v0.16.0/go.mod h1:PHK3hniurgQaNMZYaCLEqXKsYK8upmhPbmdP2FXSqgU=
github.com/apache/thrift v0.22.0 h1:r7mTJdj51TMDe6RtcmNdQxgn9XcyfGDOzegMDRg47uc=
github.com/apache/thrift v0.22.0/go.mod h1:1e7J/O1Ae6ZQMTYdy9xa3w9k+XHWPfRvdPyJeynQ+/g=
github.com/apache/thrift v0.23.0 h1:wKR6YnefQSEnxpEfmgTPuJibNG4bF0p2TK34tHLWi3s=
github.com/apache/thrift v0.23.0/go.mod h1:zPt6WxgvTOM6hF92y8C+MkEM5LMxZuk4JcQOiU4Esvs=
github.com/araddon/dateparse v0.0.0-20210429162001-6b43995a97de/go.mod h1:DCaWoUhZrYW9p1lxo/cm8EmUOOzAPSEZNGF2DK1dJgw=
github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8=
github.com/armon/go-metrics v0.0.0-20190430140413-ec5e00d3c878/go.mod h1:3AMJUQhVx52RsWOnlkpikZr01T/yAVN2gn0861vByNg=
@@ -190,8 +190,8 @@ github.com/go-asn1-ber/asn1-ber v1.5.8-0.20250403174932-29230038a667 h1:BP4M0CvQ
github.com/go-asn1-ber/asn1-ber v1.5.8-0.20250403174932-29230038a667/go.mod h1:hEBeB/ic+5LoWskz+yKT7vGhhPYkProFKoKdwZRWMe0=
github.com/go-ini/ini v1.67.0 h1:z6ZrTEZqSWOTyH2FlglNbNgARyHG8oLW9gMELqKr06A=
github.com/go-ini/ini v1.67.0/go.mod h1:ByCAeIL28uOIIG0E3PJtZPDL8WnHpFKFOtgjp+3Ies8=
github.com/go-jose/go-jose/v4 v4.1.3 h1:CVLmWDhDVRa6Mi/IgCgaopNosCaHz7zrMeF9MlZRkrs=
github.com/go-jose/go-jose/v4 v4.1.3/go.mod h1:x4oUasVrzR7071A4TnHLGSPpNOm2a21K9Kf04k1rs08=
github.com/go-jose/go-jose/v4 v4.1.4 h1:moDMcTHmvE6Groj34emNPLs/qtYXRVcd6S7NHbHz3kA=
github.com/go-jose/go-jose/v4 v4.1.4/go.mod h1:x4oUasVrzR7071A4TnHLGSPpNOm2a21K9Kf04k1rs08=
github.com/go-ldap/ldap/v3 v3.4.12 h1:1b81mv7MagXZ7+1r7cLTWmyuTqVqdwbtJSjC0DAp9s4=
github.com/go-ldap/ldap/v3 v3.4.12/go.mod h1:+SPAGcTtOfmGsCb3h1RFiq4xpp4N636G75OEace8lNo=
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
@@ -206,8 +206,8 @@ github.com/go-openapi/analysis v0.25.0 h1:EnjAq1yO8wEO9HbPmY8vLPEIkdZuuFhCAKBPvC
github.com/go-openapi/analysis v0.25.0/go.mod h1:5WFTRE43WLkPG9r9OtlMfqkkvUTYLVVCIxLlEpyF8kE=
github.com/go-openapi/errors v0.22.7 h1:JLFBGC0Apwdzw3484MmBqspjPbwa2SHvpDm0u5aGhUA=
github.com/go-openapi/errors v0.22.7/go.mod h1://QW6SD9OsWtH6gHllUCddOXDL0tk0ZGNYHwsw4sW3w=
github.com/go-openapi/jsonpointer v0.22.5 h1:8on/0Yp4uTb9f4XvTrM2+1CPrV05QPZXu+rvu2o9jcA=
github.com/go-openapi/jsonpointer v0.22.5/go.mod h1:gyUR3sCvGSWchA2sUBJGluYMbe1zazrYWIkWPjjMUY0=
github.com/go-openapi/jsonpointer v0.23.1 h1:1HBACs7XIwR2RcmItfdSFlALhGbe6S92p0ry4d1GWg4=
github.com/go-openapi/jsonpointer v0.23.1/go.mod h1:iWRmZTrGn7XwYhtPt/fvdSFj1OfNBngqRT2UG3BxSqY=
github.com/go-openapi/jsonreference v0.21.5 h1:6uCGVXU/aNF13AQNggxfysJ+5ZcU4nEAe+pJyVWRdiE=
github.com/go-openapi/jsonreference v0.21.5/go.mod h1:u25Bw85sX4E2jzFodh1FOKMTZLcfifd1Q+iKKOUxExw=
github.com/go-openapi/loads v0.23.3 h1:g5Xap1JfwKkUnZdn+S0L3SzBDpcTIYzZ5Qaag0YDkKQ=
@@ -216,8 +216,8 @@ github.com/go-openapi/runtime v0.29.3 h1:h5twGaEqxtQg40ePiYm9vFFH1q06Czd7Ot6ufdK
github.com/go-openapi/runtime v0.29.3/go.mod h1:8A1W0/L5eyNJvKciqZtvIVQvYO66NlB7INMSZ9bw/oI=
github.com/go-openapi/spec v0.22.4 h1:4pxGjipMKu0FzFiu/DPwN3CTBRlVM2yLf/YTWorYfDQ=
github.com/go-openapi/spec v0.22.4/go.mod h1:WQ6Ai0VPWMZgMT4XySjlRIE6GP1bGQOtEThn3gcWLtQ=
github.com/go-openapi/strfmt v0.26.1 h1:7zGCHji7zSYDC2tCXIusoxYQz/48jAf2q+sF6wXTG+c=
github.com/go-openapi/strfmt v0.26.1/go.mod h1:Zslk5VZPOISLwmWTMBIS7oiVFem1o1EI6zULY8Uer7Y=
github.com/go-openapi/strfmt v0.26.2 h1:ysjheCh4i1rmFEo2LanhELDNucNzfWTZhUDKgWWPaFM=
github.com/go-openapi/strfmt v0.26.2/go.mod h1:fXh1e449cyUn2NYuz+wb3wARBUdMl7qPEZwX00nqivY=
github.com/go-openapi/swag v0.25.5 h1:pNkwbUEeGwMtcgxDr+2GBPAk4kT+kJ+AaB+TMKAg+TU=
github.com/go-openapi/swag v0.25.5/go.mod h1:B3RT6l8q7X803JRxa2e59tHOiZlX1t8viplOcs9CwTA=
github.com/go-openapi/swag/cmdutils v0.25.5 h1:yh5hHrpgsw4NwM9KAEtaDTXILYzdXh/I8Whhx9hKj7c=
@@ -226,8 +226,8 @@ github.com/go-openapi/swag/conv v0.25.5 h1:wAXBYEXJjoKwE5+vc9YHhpQOFj2JYBMF2DUi+
github.com/go-openapi/swag/conv v0.25.5/go.mod h1:CuJ1eWvh1c4ORKx7unQnFGyvBbNlRKbnRyAvDvzWA4k=
github.com/go-openapi/swag/fileutils v0.25.5 h1:B6JTdOcs2c0dBIs9HnkyTW+5gC+8NIhVBUwERkFhMWk=
github.com/go-openapi/swag/fileutils v0.25.5/go.mod h1:V3cT9UdMQIaH4WiTrUc9EPtVA4txS0TOmRURmhGF4kc=
github.com/go-openapi/swag/jsonname v0.25.5 h1:8p150i44rv/Drip4vWI3kGi9+4W9TdI3US3uUYSFhSo=
github.com/go-openapi/swag/jsonname v0.25.5/go.mod h1:jNqqikyiAK56uS7n8sLkdaNY/uq6+D2m2LANat09pKU=
github.com/go-openapi/swag/jsonname v0.26.0 h1:gV1NFX9M8avo0YSpmWogqfQISigCmpaiNci8cGECU5w=
github.com/go-openapi/swag/jsonname v0.26.0/go.mod h1:urBBR8bZNoDYGr653ynhIx+gTeIz0ARZxHkAPktJK2M=
github.com/go-openapi/swag/jsonutils v0.25.5 h1:XUZF8awQr75MXeC+/iaw5usY/iM7nXPDwdG3Jbl9vYo=
github.com/go-openapi/swag/jsonutils v0.25.5/go.mod h1:48FXUaz8YsDAA9s5AnaUvAmry1UcLcNVWUjY42XkrN4=
github.com/go-openapi/swag/jsonutils/fixtures_test v0.25.5 h1:SX6sE4FrGb4sEnnxbFL/25yZBb5Hcg1inLeErd86Y1U=
@@ -246,8 +246,8 @@ github.com/go-openapi/swag/yamlutils v0.25.5 h1:kASCIS+oIeoc55j28T4o8KwlV2S4ZLPT
github.com/go-openapi/swag/yamlutils v0.25.5/go.mod h1:Gek1/SjjfbYvM+Iq4QGwa/2lEXde9n2j4a3wI3pNuOQ=
github.com/go-openapi/testify/enable/yaml/v2 v2.4.1 h1:NZOrZmIb6PTv5LTFxr5/mKV/FjbUzGE7E6gLz7vFoOQ=
github.com/go-openapi/testify/enable/yaml/v2 v2.4.1/go.mod h1:r7dwsujEHawapMsxA69i+XMGZrQ5tRauhLAjV/sxg3Q=
github.com/go-openapi/testify/v2 v2.4.1 h1:zB34HDKj4tHwyUQHrUkpV0Q0iXQ6dUCOQtIqn8hE6Iw=
github.com/go-openapi/testify/v2 v2.4.1/go.mod h1:HCPmvFFnheKK2BuwSA0TbbdxJ3I16pjwMkYkP4Ywn54=
github.com/go-openapi/testify/v2 v2.4.2 h1:tiByHpvE9uHrrKjOszax7ZvKB7QOgizBWGBLuq0ePx4=
github.com/go-openapi/testify/v2 v2.4.2/go.mod h1:SgsVHtfooshd0tublTtJ50FPKhujf47YRqauXXOUxfw=
github.com/go-openapi/validate v0.25.2 h1:12NsfLAwGegqbGWr2CnvT65X/Q2USJipmJ9b7xDJZz0=
github.com/go-openapi/validate v0.25.2/go.mod h1:Pgl1LpPPGFnZ+ys4/hTlDiRYQdI1ocKypgE+8Q8BLfY=
github.com/go-sql-driver/mysql v1.6.0/go.mod h1:DCzpHaOWr8IXmIStZouvnhqoel9Qv2LBy8hT2VhHyBg=
@@ -292,24 +292,24 @@ github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/
github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/go-tpm v0.9.3 h1:+yx0/anQuGzi+ssRqeD6WpXjW2L/V0dItUayO0i9sRc=
github.com/google/go-tpm v0.9.3/go.mod h1:h9jEsEECg7gtLis0upRBQU+GhYVH6jMjrFxI8u6bVUY=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
github.com/google/go-tpm v0.9.8/go.mod h1:h9jEsEECg7gtLis0upRBQU+GhYVH6jMjrFxI8u6bVUY=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/martian/v3 v3.3.3 h1:DIhPTQrbPkgs2yJYdXU/eNACCG5DVQjySNRNlflZ9Fc=
github.com/google/martian/v3 v3.3.3/go.mod h1:iEPrYcgCF7jA9OtScMFQyAlZZ4YXTKEtJ1E6RWzmBA0=
github.com/google/pprof v0.0.0-20240227163752-401108e1b7e7/go.mod h1:czg5+yv1E0ZGTi6S6vVK1mke0fV+FaUhNGcd6VRS9Ik=
github.com/google/pprof v0.0.0-20260302011040-a15ffb7f9dcc h1:VBbFa1lDYWEeV5FZKUiYKYT0VxCp9twUmmaq9eb8sXw=
github.com/google/pprof v0.0.0-20260302011040-a15ffb7f9dcc/go.mod h1:MxpfABSjhmINe3F1It9d+8exIHFvUqtLIRCdOGNXqiI=
github.com/google/pprof v0.0.0-20260507013755-92041b743c96 h1:YDDnaZ9afWajDboPMt9Vikqca/yWAX7KAxVzb4lJU1M=
github.com/google/pprof v0.0.0-20260507013755-92041b743c96/go.mod h1:MxpfABSjhmINe3F1It9d+8exIHFvUqtLIRCdOGNXqiI=
github.com/google/s2a-go v0.1.9 h1:LGD7gtMgezd8a/Xak7mEWL0PjoTQFvpRudN895yqKW0=
github.com/google/s2a-go v0.1.9/go.mod h1:YA0Ei2ZQL3acow2O62kdp9UlnvMmU7kA6Eutn0dXayM=
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 h1:El6M4kTTCOh6aBiKaUGG7oYTSPP8MxqL4YI3kZKwcP4=
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510/go.mod h1:pupxD2MaaD3pAXIBCelhxNneeOaAeabZDe5s4K6zSpQ=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/enterprise-certificate-proxy v0.3.14 h1:yh8ncqsbUY4shRD5dA6RlzjJaT4hi3kII+zYw8wmLb8=
github.com/googleapis/enterprise-certificate-proxy v0.3.14/go.mod h1:vqVt9yG9480NtzREnTlmGSBmFrA+bzb0yl0TxoBQXOg=
github.com/googleapis/gax-go/v2 v2.19.0 h1:fYQaUOiGwll0cGj7jmHT/0nPlcrZDFPrZRhTsoCr8hE=
github.com/googleapis/gax-go/v2 v2.19.0/go.mod h1:w2ROXVdfGEVFXzmlciUU4EdjHgWvB5h2n6x/8XSTTJA=
github.com/googleapis/enterprise-certificate-proxy v0.3.15 h1:xolVQTEXusUcAA5UgtyRLjelpFFHWlPQ4XfWGc7MBas=
github.com/googleapis/enterprise-certificate-proxy v0.3.15/go.mod h1:vqVt9yG9480NtzREnTlmGSBmFrA+bzb0yl0TxoBQXOg=
github.com/googleapis/gax-go/v2 v2.22.0 h1:PjIWBpgGIVKGoCXuiCoP64altEJCj3/Ei+kSU5vlZD4=
github.com/googleapis/gax-go/v2 v2.22.0/go.mod h1:irWBbALSr0Sk3qlqb9SyJ1h68WjgeFuiOzI4Rqw5+aY=
github.com/gorilla/mux v1.8.1 h1:TuBL49tXwgrFYWhqrNgrUNEY92u81SPhu7sTdzQEiWY=
github.com/gorilla/mux v1.8.1/go.mod h1:AKf9I4AEqPTmMytcMc0KkNouC66V3BtZ4qD5fmWSiMQ=
github.com/gorilla/securecookie v1.1.1/go.mod h1:ra0sb63/xPlUeL+yeDciTfxMRAA+MP+HVt/4epWDjd4=
@@ -318,8 +318,8 @@ github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674 h1:JeSE6pjso5T
github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674/go.mod h1:r4w70xmWCQKmi1ONH4KIaBptdivuRPyosB9RmPlGEwA=
github.com/grafana/regexp v0.0.0-20250905093917-f7b3be9d1853 h1:cLN4IBkmkYZNnk7EAJ0BHIethd+J6LqxFNw5mSiI2bM=
github.com/grafana/regexp v0.0.0-20250905093917-f7b3be9d1853/go.mod h1:+JKpmjMGhpgPL+rXZ5nsZieVzvarn86asRlBg4uNGnk=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.28.0 h1:HWRh5R2+9EifMyIHV7ZV+MIZqgz+PMpZ14Jynv3O2Zs=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.28.0/go.mod h1:JfhWUomR1baixubs02l85lZYYOm7LV6om4ceouMv45c=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.29.0 h1:5VipnvEpbqr2gA2VbM+nYVbkIF28c5ZQfqCBQ5g2xfk=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.29.0/go.mod h1:Hyl3n6Twe1hvtd9XUXDec4pTvgMSEixRuQKPTMH2bNs=
github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY2I=
github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
@@ -378,8 +378,8 @@ github.com/keybase/go-keychain v0.0.1/go.mod h1:PdEILRW3i9D8JcdM+FmY6RwkHGnhHxXw
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/klauspost/compress v1.14.4/go.mod h1:/3/Vjq9QcHkK5uEr5lBEmyoZ1iFhe47etQ6QUkpK6sk=
github.com/klauspost/compress v1.18.5 h1:/h1gH5Ce+VWNLSWqPzOVn6XBO+vJbCNGvjoaGBFW2IE=
github.com/klauspost/compress v1.18.5/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/klauspost/compress v1.18.6 h1:2jupLlAwFm95+YDR+NwD2MEfFO9d4z4Prjl1XXDjuao=
github.com/klauspost/compress v1.18.6/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/klauspost/cpuid/v2 v2.0.1/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
@@ -470,8 +470,8 @@ github.com/minio/dperf v0.7.1/go.mod h1:hH1b9YthgQFNg9X36a8/rqTkntQBhqLgv9eZMBKx
github.com/minio/filepath v1.0.0 h1:fvkJu1+6X+ECRA6G3+JJETj4QeAYO9sV43I79H8ubDY=
github.com/minio/filepath v1.0.0/go.mod h1:/nRZA2ldl5z6jT9/KQuvZcQlxZIMQoFFQPvEXx9T/Bw=
github.com/minio/highwayhash v1.0.2/go.mod h1:BQskDq+xkJ12lmlUUi7U0M5Swg3EWR+dLTk+kldvVxY=
github.com/minio/highwayhash v1.0.3 h1:kbnuUMoHYyVl7szWjSxJnxw11k2U709jqFPPmIUyD6Q=
github.com/minio/highwayhash v1.0.3/go.mod h1:GGYsuwP/fPD6Y9hMiXuapVvlIUEhFhMTh0rxU3ik1LQ=
github.com/minio/highwayhash v1.0.4-0.20251030100505-070ab1a87a76 h1:KGuD/pM2JpL9FAYvBrnBBeENKZNh6eNtjqytV6TYjnk=
github.com/minio/highwayhash v1.0.4-0.20251030100505-070ab1a87a76/go.mod h1:GGYsuwP/fPD6Y9hMiXuapVvlIUEhFhMTh0rxU3ik1LQ=
github.com/minio/kms-go/kes v0.3.1 h1:K3sPFAvFbJx33XlCTUBnQo8JRmSZyDvT6T2/MQ2iC3A=
github.com/minio/kms-go/kes v0.3.1/go.mod h1:Q9Ct0KUAuN9dH0hSVa0eva45Jg99cahbZpPxeqR9rOQ=
github.com/minio/kms-go/kms v0.6.0 h1:oGdGUyjfCZwRIi7em0aj4wk+oOm7+4a0lzSZny7ZIDU=
@@ -518,11 +518,11 @@ github.com/muesli/termenv v0.16.0/go.mod h1:ZRfOIKPFDYQoDFF4Olj7/QJbW60Ol/kL1pU3
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/nats-io/jwt/v2 v2.2.1-0.20220330180145-442af02fd36a/go.mod h1:0tqz9Hlu6bCBFLWAASKhE5vUA4c24L9KPUUgvwumE/k=
github.com/nats-io/jwt/v2 v2.7.4 h1:jXFuDDxs/GQjGDZGhNgH4tXzSUK6WQi2rsj4xmsNOtI=
github.com/nats-io/jwt/v2 v2.7.4/go.mod h1:me11pOkwObtcBNR8AiMrUbtVOUGkqYjMQZ6jnSdVUIA=
github.com/nats-io/jwt/v2 v2.8.1 h1:V0xpGuD/N8Mi+fQNDynXohVvp7ZztevW5io8CUWlPmU=
github.com/nats-io/jwt/v2 v2.8.1/go.mod h1:nWnOEEiVMiKHQpnAy4eXlizVEtSfzacZ1Q43LIRavZg=
github.com/nats-io/nats-server/v2 v2.8.2/go.mod h1:vIdpKz3OG+DCg4q/xVPdXHoztEyKDWRtykQ4N7hd7C4=
github.com/nats-io/nats-server/v2 v2.11.1 h1:LwdauqMqMNhTxTN3+WFTX6wGDOKntHljgZ+7gL5HCnk=
github.com/nats-io/nats-server/v2 v2.11.1/go.mod h1:leXySghbdtXSUmWem8K9McnJ6xbJOb0t9+NQ5HTRZjI=
github.com/nats-io/nats-server/v2 v2.11.15 h1:StSf9TINInaZtr4oww2+kXmfwa9SkN//g/LwS19/UJ0=
github.com/nats-io/nats-server/v2 v2.11.15/go.mod h1:zwhv8Y0PE3KHyKgznJc/9Xoai638SaJd83zzJ5GJn74=
github.com/nats-io/nats-streaming-server v0.24.6 h1:iIZXuPSznnYkiy0P3L0AP9zEN9Etp+tITbbX1KKeq4Q=
github.com/nats-io/nats-streaming-server v0.24.6/go.mod h1:tdKXltY3XLeBJ21sHiZiaPl+j8sK3vcCKBWVyxeQs10=
github.com/nats-io/nats.go v1.13.0/go.mod h1:BPko4oXsySz4aSWeFgOHLZs3G4Jq4ZAyE6/zMCxRT6w=
@@ -590,8 +590,8 @@ github.com/prometheus/procfs v0.20.1 h1:XwbrGOIplXW/AU3YhIhLODXMJYyC1isLFfYCsTEy
github.com/prometheus/procfs v0.20.1/go.mod h1:o9EMBZGRyvDrSPH1RqdxhojkuXstoe4UlK79eF5TGGo=
github.com/prometheus/prom2json v1.5.0 h1:WIcAOjLE1x476W3dUlmTL6E/e98CgVGuwwYusl6MPP8=
github.com/prometheus/prom2json v1.5.0/go.mod h1:xPp6KDhCA30btxmqEfg/K3DAwgTIkp7TJKi4+7jaYd8=
github.com/prometheus/prometheus v0.310.0 h1:iS0Uul/dHjy8ifBnqo3YEOhRxlTOWantRoDWwmIowwA=
github.com/prometheus/prometheus v0.310.0/go.mod h1:rs6XoWKvgAStqxHxb2Twh1BR6rp7qw7fmUgW+gaXjbw=
github.com/prometheus/prometheus v0.311.3 h1:3IrVxQv6v5i/ZCGi6OrYeBhtCwaPTn6Z3DYruXoYm3M=
github.com/prometheus/prometheus v0.311.3/go.mod h1:gjsCxTKtHO1Q8T9333u1s+lUR1OjPyM7ruuGH8RvVyo=
github.com/puzpuzpuz/xsync/v3 v3.5.1 h1:GJYJZwO6IdxN/IKbneznS6yPkVC+c3zyY/j19c++5Fg=
github.com/puzpuzpuz/xsync/v3 v3.5.1/go.mod h1:VjzYrABPabuM4KyBh1Ftq6u8nhwY5tBPKP9jpmh0nnA=
github.com/rabbitmq/amqp091-go v1.10.0 h1:STpn5XsHlHGcecLmMFCtg7mqq0RnD+zFr4uzukfVhBw=
@@ -700,28 +700,30 @@ go.opentelemetry.io/contrib/detectors/gcp v1.42.0 h1:kpt2PEJuOuqYkPcktfJqWWDjTEd
go.opentelemetry.io/contrib/detectors/gcp v1.42.0/go.mod h1:W9zQ439utxymRrXsUOzZbFX4JhLxXU4+ZnCt8GG7yA8=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.67.0 h1:yI1/OhfEPy7J9eoa6Sj051C7n5dvpj0QX8g4sRchg04=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.67.0/go.mod h1:NoUCKYWK+3ecatC4HjkRktREheMeEtrXoQxrqYFeHSc=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.67.0 h1:OyrsyzuttWTSur2qN/Lm0m2a8yqyIjUVBZcxFPuXq2o=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.67.0/go.mod h1:C2NGBr+kAB4bk3xtMXfZ94gqFDtg/GkI7e9zqGh5Beg=
go.opentelemetry.io/otel v1.42.0 h1:lSQGzTgVR3+sgJDAU/7/ZMjN9Z+vUip7leaqBKy4sho=
go.opentelemetry.io/otel v1.42.0/go.mod h1:lJNsdRMxCUIWuMlVJWzecSMuNjE7dOYyWlqOXWkdqCc=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.69.0 h1:8tvICD4vSTOOsNrsI4Ljf6C+6UKvpTEH5XY3JMoyPoo=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.69.0/go.mod h1:z9+yiacE0IHRqM4qFfkbt/JYlmYXgss8GY/jXoNuPJI=
go.opentelemetry.io/otel v1.44.0 h1:JjwHmHpA4iZ3wBxluu2fbbE7j4kqlE8jXyAyPXH7HqU=
go.opentelemetry.io/otel v1.44.0/go.mod h1:BMgjTHL9WPRlRjL2oZCBTL4whCGtXch2H4BhOPIAyYc=
go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.40.0 h1:ZrPRak/kS4xI3AVXy8F7pipuDXmDsrO8Lg+yQjBLjw0=
go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.40.0/go.mod h1:3y6kQCWztq6hyW8Z9YxQDDm0Je9AJoFar2G0yDcmhRk=
go.opentelemetry.io/otel/metric v1.42.0 h1:2jXG+3oZLNXEPfNmnpxKDeZsFI5o4J+nz6xUlaFdF/4=
go.opentelemetry.io/otel/metric v1.42.0/go.mod h1:RlUN/7vTU7Ao/diDkEpQpnz3/92J9ko05BIwxYa2SSI=
go.opentelemetry.io/otel/sdk v1.42.0 h1:LyC8+jqk6UJwdrI/8VydAq/hvkFKNHZVIWuslJXYsDo=
go.opentelemetry.io/otel/sdk v1.42.0/go.mod h1:rGHCAxd9DAph0joO4W6OPwxjNTYWghRWmkHuGbayMts=
go.opentelemetry.io/otel/sdk/metric v1.42.0 h1:D/1QR46Clz6ajyZ3G8SgNlTJKBdGp84q9RKCAZ3YGuA=
go.opentelemetry.io/otel/sdk/metric v1.42.0/go.mod h1:Ua6AAlDKdZ7tdvaQKfSmnFTdHx37+J4ba8MwVCYM5hc=
go.opentelemetry.io/otel/trace v1.42.0 h1:OUCgIPt+mzOnaUTpOQcBiM/PLQ/Op7oq6g4LenLmOYY=
go.opentelemetry.io/otel/trace v1.42.0/go.mod h1:f3K9S+IFqnumBkKhRJMeaZeNk9epyhnCmQh/EysQCdc=
go.opentelemetry.io/otel/metric v1.44.0 h1:1w0gILTcHdr3YI+ixLyjemwrVnsMURbTZFrSYCdDdmc=
go.opentelemetry.io/otel/metric v1.44.0/go.mod h1:8O7hanEPBNgEMmybD3s2VBKcgWOCsA6tzHBPODAiquo=
go.opentelemetry.io/otel/metric/x v0.66.0 h1:YkCrx1zLOChi9ZcZ6euupOcsgzbVlec7D/xoEU1+cTA=
go.opentelemetry.io/otel/metric/x v0.66.0/go.mod h1:d1+BDj9t96do0/1LoU1ayfCv79ZgNE41qbhBvnMOBZk=
go.opentelemetry.io/otel/sdk v1.44.0 h1:nHYwb9lK+fJPU/dnT6s7W7Z8itMWyqrnVfbheVYrZ58=
go.opentelemetry.io/otel/sdk v1.44.0/go.mod h1:Osuydd3Se74nqjAKxid74N5eC+jfEqfTegHRnq58oK0=
go.opentelemetry.io/otel/sdk/metric v1.44.0 h1:3LlKgI+VjbVsjNRFZJZAJ30WjXC5VkNRks6si09iEfI=
go.opentelemetry.io/otel/sdk/metric v1.44.0/go.mod h1:5B5pMARnXxKhltooO4xUuCBorl65a4EpnTalObqOigA=
go.opentelemetry.io/otel/trace v1.44.0 h1:jxF5CsGYCe74MCRx2X4g7WsY/VBKRqqpNvXlX/6gtIk=
go.opentelemetry.io/otel/trace v1.44.0/go.mod h1:oLl1jrMQAVo6v3GAggN+1VH9VIz9iUSvW53sW1Q8PIE=
go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE=
go.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
go.uber.org/zap v1.27.1 h1:08RqriUEv8+ArZRYSTXy1LeBScaMpVSTBhCeaZYfMYc=
go.uber.org/zap v1.27.1/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=
go.uber.org/zap v1.28.0 h1:IZzaP1Fv73/T/pBMLk4VutPl36uNC+OSUh3JLG3FIjo=
go.uber.org/zap v1.28.0/go.mod h1:rDLpOi171uODNm/mxFcuYWxDsqWSAVkFdX4XojSKg/Q=
go.yaml.in/yaml/v2 v2.4.4 h1:tuyd0P+2Ont/d6e2rl3be67goVK4R6deVxCUX5vyPaQ=
go.yaml.in/yaml/v2 v2.4.4/go.mod h1:gMZqIpDtDqOfM0uNfy0SkpRhvUryYH0Z6wdMYcacYXQ=
go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
@@ -740,15 +742,15 @@ golang.org/x/crypto v0.0.0-20211209193657-4570a0811e8b/go.mod h1:IxCIyHEi3zRg3s0
golang.org/x/crypto v0.0.0-20220315160706-3147a52a75dd/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.5.0/go.mod h1:NK/OQwhpMQP3MwtdjgLlYHnH9ebylxKWv3e0fK+mkQU=
golang.org/x/crypto v0.6.0/go.mod h1:OFC/31mSvZgRz0V1QTNCzfAI1aIRzbiufJtkMIlEp58=
golang.org/x/crypto v0.49.0 h1:+Ng2ULVvLHnJ/ZFEq4KdcDd/cfjrrjjNSXNzxg0Y4U4=
golang.org/x/crypto v0.49.0/go.mod h1:ErX4dUh2UM+CFYiXZRTcMpEcN8b/1gxEuv3nODoYtCA=
golang.org/x/exp v0.0.0-20260112195511-716be5621a96 h1:Z/6YuSHTLOHfNFdb8zVZomZr7cqNgTJvA8+Qz75D8gU=
golang.org/x/exp v0.0.0-20260112195511-716be5621a96/go.mod h1:nzimsREAkjBCIEFtHiYkrJyT+2uy9YZJB7H1k68CXZU=
golang.org/x/crypto v0.52.0 h1:RMs7fP2rXdep0CftQlK8Uf+kibLm7qkCcradZWYz988=
golang.org/x/crypto v0.52.0/go.mod h1:1QgfPxDqh0T2M/elOJtp9RvuR95kVjir0e6/BvEmGbc=
golang.org/x/exp v0.0.0-20260218203240-3dfff04db8fa h1:Zt3DZoOFFYkKhDT3v7Lm9FDMEV06GpzjG2jrqW+QTE0=
golang.org/x/exp v0.0.0-20260218203240-3dfff04db8fa/go.mod h1:K79w1Vqn7PoiZn+TkNpx3BUWUQksGO3JcVX6qIjytmA=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.34.0 h1:xIHgNUUnW6sYkcM5Jleh05DvLOtwc6RitGHbDk4akRI=
golang.org/x/mod v0.34.0/go.mod h1:ykgH52iCZe79kzLLMhyCUzhMci+nQj+0XkbXpNYtVjY=
golang.org/x/mod v0.36.0 h1:JJjpVx6myfUsUdAzZuOSTTmRE0PfZeNWzzvKrP7amb4=
golang.org/x/mod v0.36.0/go.mod h1:moc6ELqsWcOw5Ef3xVprK5ul/MvtVvkIXLziUOICjUQ=
golang.org/x/net v0.0.0-20181201002055-351d144fa1fc/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
@@ -762,8 +764,8 @@ golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug
golang.org/x/net v0.5.0/go.mod h1:DivGGAXEgPSlEBzxGzZI+ZLohi+xUj054jfeKui00ws=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.52.0 h1:He/TN1l0e4mmR3QqHMT2Xab3Aj3L9qjbhRm78/6jrW0=
golang.org/x/net v0.52.0/go.mod h1:R1MAz7uMZxVMualyPXb+VaqGSa3LIaUqk0eEt3w36Sw=
golang.org/x/net v0.55.0 h1:bcvxaJn3e1U6InsFWt1JUq1aSjnRxLzT2rtD2KfkDF8=
golang.org/x/net v0.55.0/go.mod h1:L5U2KuzuOe1lY7Z+aWVIKK6qEeJXnXV9yzGA+WCHJww=
golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs=
golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -805,23 +807,23 @@ golang.org/x/sys v0.4.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.21.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.42.0 h1:omrd2nAlyT5ESRdCLYdm3+fMfNFE/+Rf4bDIQImRJeo=
golang.org/x/sys v0.42.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/sys v0.45.0 h1:dO4czNzziLiiXplLQgBCEpCvXQ3dnkn0SdaZSYdQ+FY=
golang.org/x/sys v0.45.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.4.0/go.mod h1:9P2UbLfCdcvo3p/nzKvsmas4TnlujnuoV9hGgYzW1lQ=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/term v0.41.0 h1:QCgPso/Q3RTJx2Th4bDLqML4W6iJiaXFq2/ftQF13YU=
golang.org/x/term v0.41.0/go.mod h1:3pfBgksrReYfZ5lvYM0kSO0LIkAl4Yl2bXOkKP7Ec2A=
golang.org/x/term v0.43.0 h1:S4RLU2sB31O/NCl+zFN9Aru9A/Cq2aqKpTZJ6B+DwT4=
golang.org/x/term v0.43.0/go.mod h1:lrhlHNdQJHO+1qVYiHfFKVuVioJIheAc3fBSMFYEIsk=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.6.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.35.0 h1:JOVx6vVDFokkpaq1AEptVzLTpDe9KGpj5tR4/X+ybL8=
golang.org/x/text v0.35.0/go.mod h1:khi/HExzZJ2pGnjenulevKNX1W67CUy0AsXcNubPGCA=
golang.org/x/text v0.37.0 h1:Cqjiwd9eSg8e0QAkyCaQTNHFIIzWtidPahFWR83rTrc=
golang.org/x/text v0.37.0/go.mod h1:a5sjxXGs9hsn/AJVwuElvCAo9v8QYLzvavO5z2PiM38=
golang.org/x/time v0.0.0-20211116232009-f0f3c7e86c11/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
@@ -831,24 +833,24 @@ golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtn
golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.43.0 h1:12BdW9CeB3Z+J/I/wj34VMl8X+fEXBxVR90JeMX5E7s=
golang.org/x/tools v0.43.0/go.mod h1:uHkMso649BX2cZK6+RpuIPXS3ho2hZo4FVwfoy1vIk0=
golang.org/x/tools v0.45.0 h1:18qN3FAooORvApf5XjCXgsuayZOEtXf6JK18I3+ONa8=
golang.org/x/tools v0.45.0/go.mod h1:LuUGqqaXcXMEFEruIVJVm5mgDD8vww/z/SR1gQ4uE/0=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gonum.org/v1/gonum v0.16.0 h1:5+ul4Swaf3ESvrOnidPp4GZbzf0mxVQpDCYUQE7OJfk=
gonum.org/v1/gonum v0.16.0/go.mod h1:fef3am4MQ93R2HHpKnLk4/Tbh/s0+wqD5nfa6Pnwy4E=
google.golang.org/api v0.272.0 h1:eLUQZGnAS3OHn31URRf9sAmRk3w2JjMx37d2k8AjJmA=
google.golang.org/api v0.272.0/go.mod h1:wKjowi5LNJc5qarNvDCvNQBn3rVK8nSy6jg2SwRwzIA=
gonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4=
gonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E=
google.golang.org/api v0.278.0 h1:W7jiRvRi53VYFfZ/HoZjQBtJk7gOFbHD8ot1RzVZU6E=
google.golang.org/api v0.278.0/go.mod h1:B9TqLBwJqVjp1mtt7WeoQwWRwvu/400y5lETOql+giQ=
google.golang.org/genproto v0.0.0-20260319201613-d00831a3d3e7 h1:XzmzkmB14QhVhgnawEVsOn6OFsnpyxNPRY9QV01dNB0=
google.golang.org/genproto v0.0.0-20260319201613-d00831a3d3e7/go.mod h1:L43LFes82YgSonw6iTXTxXUX1OlULt4AQtkik4ULL/I=
google.golang.org/genproto/googleapis/api v0.0.0-20260319201613-d00831a3d3e7 h1:41r6JMbpzBMen0R/4TZeeAmGXSJC7DftGINUodzTkPI=
google.golang.org/genproto/googleapis/api v0.0.0-20260319201613-d00831a3d3e7/go.mod h1:EIQZ5bFCfRQDV4MhRle7+OgjNtZ6P1PiZBgAKuxXu/Y=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260319201613-d00831a3d3e7 h1:ndE4FoJqsIceKP2oYSnUZqhTdYufCYYkqwtFzfrhI7w=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260319201613-d00831a3d3e7/go.mod h1:4Hqkh8ycfw05ld/3BWL7rJOSfebL2Q+DVDeRgYgxUU8=
google.golang.org/grpc v1.79.3 h1:sybAEdRIEtvcD68Gx7dmnwjZKlyfuc61Dyo9pGXXkKE=
google.golang.org/grpc v1.79.3/go.mod h1:KmT0Kjez+0dde/v2j9vzwoAScgEPx/Bw1CYChhHLrHQ=
google.golang.org/genproto/googleapis/api v0.0.0-20260526163538-3dc84a4a5aaa h1:Kjn0N0tCrDgiAFW+lGO4JZ3ck44CehvJQMAwj9QF0G8=
google.golang.org/genproto/googleapis/api v0.0.0-20260526163538-3dc84a4a5aaa/go.mod h1:q4lMZS6kskjT5HvCPrnnypcDPVJqT/f4nfxmkE7gryY=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260526163538-3dc84a4a5aaa h1:mZHHdPZl0dbGHCflZgAq/Q468DWVFcU2whhB2KAo8fk=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260526163538-3dc84a4a5aaa/go.mod h1:4Hqkh8ycfw05ld/3BWL7rJOSfebL2Q+DVDeRgYgxUU8=
google.golang.org/grpc v1.81.1 h1:VnnIIZ88UzOOKLukQi+ImGz8O1Wdp8nAGGnvOfEIWQQ=
google.golang.org/grpc v1.81.1/go.mod h1:xGH9GfzOyMTGIOXBJmXt+BX/V0kcdQbdcuwQ/zNw42I=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
+3 -2
View File
@@ -1,5 +1,6 @@
#!/bin/bash
#!/usr/bin/env bash
set -euo pipefail
helm package helm/minio -d helm-releases/
helm repo index --merge index.yaml --url https://charts.min.io .
helm repo index --merge index.yaml --url https://raw.githubusercontent.com/pgsty/minio/master .
Binary file not shown.
+9 -8
View File
@@ -1,18 +1,19 @@
apiVersion: v1
description: High Performance Object Storage
description: Silo, a community-maintained MinIO-compatible object store
name: minio
version: 5.4.0
appVersion: RELEASE.2024-12-18T13-15-44Z
version: 5.5.0
appVersion: RELEASE.2026-06-18T00-00-00Z
keywords:
- minio
- silo
- storage
- object-storage
- s3
- cluster
home: https://min.io
icon: https://min.io/resources/img/logo/MINIO_wordmark.png
home: https://silo.pigsty.io
icon: https://raw.githubusercontent.com/pgsty/minio/master/.github/logo.svg
sources:
- https://github.com/minio/minio
- https://github.com/pgsty/minio
maintainers:
- name: MinIO, Inc
email: dev@minio.io
- name: Pigsty
url: https://pigsty.io
+53 -25
View File
@@ -1,27 +1,42 @@
# MinIO Community Helm Chart
# Silo Community Helm Chart
[![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io) [![license](https://img.shields.io/badge/license-AGPL%20V3-blue)](https://github.com/minio/minio/blob/master/LICENSE)
[![license](https://img.shields.io/badge/license-AGPL%20V3-blue)](https://github.com/pgsty/minio/blob/master/LICENSE)
[![source](https://img.shields.io/badge/source-pgsty%2Fminio-blue?logo=github)](https://github.com/pgsty/minio)
[![image](https://img.shields.io/badge/image-pgsty%2Fminio-blue?logo=docker)](https://hub.docker.com/r/pgsty/minio)
MinIO is a High Performance Object Storage released under GNU Affero General Public License v3.0. It is API compatible with Amazon S3 cloud storage service. Use MinIO to build high performance infrastructure for machine learning, analytics and application data workloads.
Silo is a community-maintained, S3-compatible object store derived from MinIO. This chart deploys the Silo server directly on Kubernetes in standalone or distributed mode.
| IMPORTANT |
| -------------------------- |
| This Helm chart is community built, maintained, and supported. MinIO does not guarantee support for any given bug, feature request, or update referencing this chart. <br/><br/> MinIO publishes a separate [MinIO Kubernetes Operator and Tenant Helm Chart](https://github.com/minio/operator/tree/master/helm) that is officially maintained and supported. MinIO strongly recommends using the MinIO Kubernetes Operator for production deployments. See [Deploy Operator With Helm](https://docs.min.io/community/minio-object-store/operations/deployments/k8s-deploy-operator-helm-on-kubernetes.html?ref=github) for additional documentation. |
> [!IMPORTANT]
> This chart is maintained in [`pgsty/minio`](https://github.com/pgsty/minio). Chart changes are linted, installed into Kind, and verified with an S3 write/read/delete test.
>
> The MinIO Operator, Tenant CRDs, and operator lifecycle are **not maintained here**. This repository does not provide or plan to provide a Pigsty/Silo operator fork.
## Pinned images
The defaults are immutable multi-architecture references for `linux/amd64` and `linux/arm64`:
| Workload | Repository | Release | Manifest digest |
|:---------|:-----------|:--------|:----------------|
| Server | `pgsty/minio` | `RELEASE.2026-06-18T00-00-00Z` | `sha256:dacff8306a6e0a734518533992dbdcca26bc1ca47f77cf47cb9945725f92b29b` |
| Post-install jobs and tests | `pgsty/minio` | `RELEASE.2026-06-18T00-00-00Z` | `sha256:dacff8306a6e0a734518533992dbdcca26bc1ca47f77cf47cb9945725f92b29b` |
The Silo image bundles `mcli` and an `mc` compatibility alias. When an image `digest` is set it takes precedence over `tag`. To select another release, update both fields; alternatively set `digest: ""` to use the tag alone.
## Introduction
This chart bootstraps MinIO Cluster on [Kubernetes](http://kubernetes.io) using the [Helm](https://helm.sh) package manager.
This chart bootstraps a Silo cluster on [Kubernetes](https://kubernetes.io) using the [Helm](https://helm.sh) package manager.
## Prerequisites
- Helm cli with Kubernetes cluster configured.
- Helm 3 with a Kubernetes cluster configured.
- PV provisioner support in the underlying infrastructure. (We recommend using <https://github.com/minio/direct-csi>)
- Use Kubernetes version v1.19 and later for best experience.
- Kubernetes v1.25 or later is recommended. The current CI test version is recorded in [the Helm workflow](../../.github/workflows/helm.yml).
## Configure MinIO Helm repo
## Configure the Silo Helm repository
```bash
helm repo add minio https://charts.min.io/
helm repo add silo https://raw.githubusercontent.com/pgsty/minio/master
helm repo update silo
```
### Installing the Chart
@@ -29,7 +44,10 @@ helm repo add minio https://charts.min.io/
Install this chart using:
```bash
helm install --namespace minio --set rootUser=rootuser,rootPassword=rootpass123 --generate-name minio/minio
helm install my-release silo/minio \
--namespace minio \
--create-namespace \
--set rootUser=rootuser,rootPassword=rootpass123
```
The command deploys MinIO on the Kubernetes cluster in the default configuration. The [configuration](#configuration) section lists the parameters that can be configured during installation.
@@ -39,7 +57,17 @@ The command deploys MinIO on the Kubernetes cluster in the default configuration
Minimal toy setup for testing purposes can be deployed using:
```bash
helm install --set resources.requests.memory=512Mi --set replicas=1 --set persistence.enabled=false --set mode=standalone --set rootUser=rootuser,rootPassword=rootpass123 --generate-name minio/minio
helm install my-release silo/minio \
--namespace minio \
--create-namespace \
--set mode=standalone \
--set replicas=1 \
--set persistence.enabled=false \
--set resources.requests.memory=512Mi \
--set rootUser=rootuser \
--set rootPassword=rootpass123
helm test my-release --namespace minio --logs
```
### Upgrading the Chart
@@ -50,10 +78,10 @@ You can use Helm to update MinIO version in a live release. Assuming your releas
helm get values my-release > old_values.yaml
```
Then change the field `image.tag` in `old_values.yaml` file with MinIO image tag you want to use. Now update the chart using
Update `image.tag` and `image.digest` together for the server release. If post-install jobs are enabled, update `mcImage.tag` and `mcImage.digest` to the same tested artifact. Then upgrade the release:
```bash
helm upgrade -f old_values.yaml my-release minio/minio
helm upgrade -f old_values.yaml my-release silo/minio
```
Default upgrade strategies are specified in the `values.yaml` file. Update these fields if you'd like to use a different strategy.
@@ -65,7 +93,7 @@ Refer the [Values file](./values.yaml) for all the possible config fields.
You can specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. For example,
```bash
helm install --name my-release --set persistence.size=1Ti minio/minio
helm install my-release --set persistence.size=1Ti silo/minio
```
The above command deploys MinIO server with a 1Ti backing persistent volume.
@@ -73,7 +101,7 @@ The above command deploys MinIO server with a 1Ti backing persistent volume.
Alternately, you can provide a YAML file that specifies parameter values while installing the chart. For example,
```bash
helm install --name my-release -f values.yaml minio/minio
helm install my-release -f values.yaml silo/minio
```
### Persistence
@@ -81,7 +109,7 @@ helm install --name my-release -f values.yaml minio/minio
This chart provisions a PersistentVolumeClaim and mounts corresponding persistent volume to default location `/export`. You'll need physical storage available in the Kubernetes cluster for this to work. If you'd rather use `emptyDir`, disable PersistentVolumeClaim by:
```bash
helm install --set persistence.enabled=false minio/minio
helm install my-release --set persistence.enabled=false silo/minio
```
> *"An emptyDir volume is first created when a Pod is assigned to a Node, and exists as long as that Pod is running on that node. When a Pod is removed from a node for any reason, the data in the emptyDir is deleted forever."*
@@ -95,7 +123,7 @@ If a Persistent Volume Claim already exists, specify it during installation.
3. Install the chart
```bash
helm install --set persistence.existingClaim=PVC_NAME minio/minio
helm install my-release --set persistence.existingClaim=PVC_NAME silo/minio
```
### NetworkPolicy
@@ -134,7 +162,7 @@ kubectl create secret generic my-minio-secret --from-literal=rootUser=foobarbaz
Then install the chart, specifying that you want to use an existing secret:
```bash
helm install --set existingSecret=my-minio-secret minio/minio
helm install my-release --set existingSecret=my-minio-secret silo/minio
```
The following fields are expected in the secret:
@@ -157,7 +185,7 @@ kubectl create secret generic tls-ssl-minio --from-file=path/to/private.key --fr
Then install the chart, specifying that you want to use the TLS secret:
```bash
helm install --set tls.enabled=true,tls.certSecret=tls-ssl-minio minio/minio
helm install my-release --set tls.enabled=true,tls.certSecret=tls-ssl-minio silo/minio
```
### Installing certificates from third party CAs
@@ -191,7 +219,7 @@ or
Install the chart, specifying the buckets you want to create after install:
```bash
helm install --set buckets[0].name=bucket1,buckets[0].policy=none,buckets[0].purge=false minio/minio
helm install my-release --set buckets[0].name=bucket1,buckets[0].policy=none,buckets[0].purge=false silo/minio
```
Description of the configuration parameters used above -
@@ -205,7 +233,7 @@ Description of the configuration parameters used above -
Install the chart, specifying the policies you want to create after install:
```bash
helm install --set policies[0].name=mypolicy,policies[0].statements[0].resources[0]='arn:aws:s3:::bucket1',policies[0].statements[0].actions[0]='s3:ListBucket',policies[0].statements[0].actions[1]='s3:GetObject' minio/minio
helm install my-release --set policies[0].name=mypolicy,policies[0].statements[0].resources[0]='arn:aws:s3:::bucket1',policies[0].statements[0].actions[0]='s3:ListBucket',policies[0].statements[0].actions[1]='s3:GetObject' silo/minio
```
Description of the configuration parameters used above -
@@ -220,7 +248,7 @@ Description of the configuration parameters used above -
Install the chart, specifying the users you want to create after install:
```bash
helm install --set users[0].accessKey=accessKey,users[0].secretKey=secretKey,users[0].policy=none,users[1].accessKey=accessKey2,users[1].secretRef=existingSecret,users[1].secretKey=password,users[1].policy=none minio/minio
helm install my-release --set users[0].accessKey=accessKey,users[0].secretKey=secretKey,users[0].policy=none,users[1].accessKey=accessKey2,users[1].secretRef=existingSecret,users[1].secretKey=password,users[1].policy=none silo/minio
```
Description of the configuration parameters used above -
@@ -236,7 +264,7 @@ Description of the configuration parameters used above -
Install the chart, specifying the service accounts you want to create after install:
```bash
helm install --set svcaccts[0].accessKey=accessKey,svcaccts[0].secretKey=secretKey,svcaccts[0].user=parentUser,svcaccts[1].accessKey=accessKey2,svcaccts[1].secretRef=existingSecret,svcaccts[1].secretKey=password,svcaccts[1].user=parentUser2 minio/minio
helm install my-release --set svcaccts[0].accessKey=accessKey,svcaccts[0].secretKey=secretKey,svcaccts[0].user=parentUser,svcaccts[1].accessKey=accessKey2,svcaccts[1].secretRef=existingSecret,svcaccts[1].secretKey=password,svcaccts[1].user=parentUser2 silo/minio
```
Description of the configuration parameters used above -
+3 -3
View File
@@ -12,7 +12,7 @@ Read more about port forwarding here: http://kubernetes.io/docs/user-guide/kubec
You can now access MinIO server on http://localhost:9000. Follow the below steps to connect to MinIO server with mc client:
1. Download the MinIO mc client - https://docs.min.io/community/minio-object-store/reference/minio-mc.html#quickstart
1. Download the MinIO mc client - https://silo.pigsty.io/reference/minio-mc.html#quickstart
2. export MC_HOST_{{ template "minio.fullname" . }}_local=http://$(kubectl get secret --namespace {{ .Release.Namespace }} {{ template "minio.secretName" . }} -o jsonpath="{.data.rootUser}" | base64 --decode):$(kubectl get secret --namespace {{ .Release.Namespace }} {{ template "minio.secretName" . }} -o jsonpath="{.data.rootPassword}" | base64 --decode)@localhost:{{ .Values.service.port }}
@@ -27,13 +27,13 @@ Note that the public IP may take a couple of minutes to be available.
You can now access MinIO server on http://<External-IP>:9000. Follow the below steps to connect to MinIO server with mc client:
1. Download the MinIO mc client - https://docs.min.io/community/minio-object-store/reference/minio-mc.html#quickstart
1. Download the MinIO mc client - https://silo.pigsty.io/reference/minio-mc.html#quickstart
2. export MC_HOST_{{ template "minio.fullname" . }}_local=http://$(kubectl get secret {{ template "minio.secretName" . }} --namespace {{ .Release.Namespace }} -o jsonpath="{.data.rootUser}" | base64 --decode):$(kubectl get secret {{ template "minio.secretName" . }} -o jsonpath="{.data.rootPassword}" | base64 --decode)@<External-IP>:{{ .Values.service.port }}
3. mc ls {{ template "minio.fullname" . }}
Alternately, you can use your browser or the MinIO SDK to access the server - https://docs.min.io/community/minio-object-store/reference/minio-server/minio-server.html
Alternately, you can use your browser or the MinIO SDK to access the server - https://silo.pigsty.io/reference/minio-server/minio-server.html
{{- end }}
{{ if and (.Values.networkPolicy.enabled) (not .Values.networkPolicy.allowExternal) }}
+22
View File
@@ -31,6 +31,28 @@ Create chart name and version as used by the chart label.
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{/*
Build the immutable server image reference. Digest takes precedence over tag.
*/}}
{{- define "minio.image" -}}
{{- if .Values.image.digest -}}
{{- printf "%s@%s" .Values.image.repository .Values.image.digest -}}
{{- else -}}
{{- printf "%s:%s" .Values.image.repository .Values.image.tag -}}
{{- end -}}
{{- end -}}
{{/*
Build the immutable client/job image reference. Digest takes precedence over tag.
*/}}
{{- define "minio.mcImage" -}}
{{- if .Values.mcImage.digest -}}
{{- printf "%s@%s" .Values.mcImage.repository .Values.mcImage.digest -}}
{{- else -}}
{{- printf "%s:%s" .Values.mcImage.repository .Values.mcImage.tag -}}
{{- end -}}
{{- end -}}
{{/*
Return the appropriate apiVersion for networkpolicy.
*/}}
+1 -1
View File
@@ -62,7 +62,7 @@ spec:
{{- end }}
containers:
- name: {{ .Chart.Name }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
image: {{ include "minio.image" . | quote }}
imagePullPolicy: {{ .Values.image.pullPolicy }}
command:
- "/bin/sh"
+5 -5
View File
@@ -88,7 +88,7 @@ spec:
{{- if .Values.policies }}
initContainers:
- name: minio-make-policy
image: "{{ .Values.mcImage.repository }}:{{ .Values.mcImage.tag }}"
image: {{ include "minio.mcImage" . | quote }}
{{- if .Values.makePolicyJob.securityContext.enabled }}
{{- with .Values.makePolicyJob.containerSecurityContext }}
securityContext: {{ toYaml . | nindent 12 }}
@@ -122,7 +122,7 @@ spec:
containers:
{{- if .Values.buckets }}
- name: minio-make-bucket
image: "{{ .Values.mcImage.repository }}:{{ .Values.mcImage.tag }}"
image: {{ include "minio.mcImage" . | quote }}
{{- if .Values.makeBucketJob.securityContext.enabled }}
{{- with .Values.makeBucketJob.containerSecurityContext }}
securityContext: {{ toYaml . | nindent 12 }}
@@ -155,7 +155,7 @@ spec:
{{- end }}
{{- if .Values.users }}
- name: minio-make-user
image: "{{ .Values.mcImage.repository }}:{{ .Values.mcImage.tag }}"
image: {{ include "minio.mcImage" . | quote }}
{{- if .Values.makeUserJob.securityContext.enabled }}
{{- with .Values.makeUserJob.containerSecurityContext }}
securityContext: {{ toYaml . | nindent 12 }}
@@ -188,7 +188,7 @@ spec:
{{- end }}
{{- if .Values.customCommands }}
- name: minio-custom-command
image: "{{ .Values.mcImage.repository }}:{{ .Values.mcImage.tag }}"
image: {{ include "minio.mcImage" . | quote }}
{{- if .Values.customCommandJob.securityContext.enabled }}
{{- with .Values.customCommandJob.containerSecurityContext }}
securityContext: {{ toYaml . | nindent 12 }}
@@ -224,7 +224,7 @@ spec:
{{- end }}
{{- if .Values.svcaccts }}
- name: minio-make-svcacct
image: "{{ .Values.mcImage.repository }}:{{ .Values.mcImage.tag }}"
image: {{ include "minio.mcImage" . | quote }}
{{- if .Values.makeServiceAccountJob.securityContext.enabled }}
{{- with .Values.makeServiceAccountJob.containerSecurityContext }}
securityContext: {{ toYaml . | nindent 12 }}
+1 -1
View File
@@ -90,7 +90,7 @@ spec:
{{- end }}
containers:
- name: {{ .Chart.Name }}
image: {{ .Values.image.repository }}:{{ .Values.image.tag }}
image: {{ include "minio.image" . | quote }}
imagePullPolicy: {{ .Values.image.pullPolicy }}
command: [
"/bin/sh",

Some files were not shown because too many files have changed in this diff Show More