Compare commits

..

82 Commits

Author SHA1 Message Date
Feng Ruohang 5c6e30d4f1 helm: harden Silo chart delivery 2026-07-28 11:28:06 +08:00
Wesley Schwengle 0ce4b0f14b docs: point documentation links to silo.pigsty.io (#41)
Replace upstream docs.min.io references with the SILO documentation domain and remove the upstream community path prefix.

Co-authored-by: waterkip <6317502+waterkip@users.noreply.github.com>
2026-07-20 19:43:45 +08:00
Feng Ruohang 3e61b1d3a5 chore: update Go module dependencies 2026-06-18 16:14:50 +08:00
Feng Ruohang df627ff896 fix: bump Go toolchain to 1.26.4
Update the module Go directive and release Docker build images from Go 1.26.2 to Go 1.26.4 so local, CI, hotfix, and release builds use the same patched toolchain.

Keep ordinary Go module requirements and replacements unchanged; this intentionally avoids a third-party dependency refresh while allowing container system packages to refresh through the newer golang Alpine base image and existing apk resolution.

Update the security advisory index to record the Go 1.26.4 toolchain bump alongside the earlier Go 1.26.2 security update.

Verified with go build ./..., go vet ./cmd/, and focused cmd tests. go mod tidy -diff was attempted as a read-only dependency drift check but could not complete because proxy.golang.org timed out while fetching uncached transitive test modules.

Co-authored-by: Codex <codex@openai.com>
2026-06-12 21:24:23 +08:00
Feng Ruohang 73ac524724 fix: CVE-2026-42600 remove ReadMultiple storage-REST API
The internode storage-REST ReadMultiple endpoint (/rmpl) joined
attacker-controlled Bucket/Prefix/Files into a filesystem path with no
validation, letting a peer with internode credentials read files outside
the drive root (GHSA-xh8f-g2qw-gcm7).

ReadMultiple has had no production caller since upstream #20390 removed
the last one (listParts) in Sep 2024; multipart now uses ReadParts (/rps).
Following the upstream fix, remove the whole API instead of validating
paths: route constant and registration, server handler, REST client
wrapper, the StorageAPI/xlStorage/xlStorageDiskIDCheck methods, the
storageMetricReadMultiple metric, and the ReadMultipleReq/Resp datatypes.
Regenerated the msgp and stringer outputs; storageRESTVersion stays at v63.

Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Code <claude-code@anthropic.com>
2026-06-12 21:08:59 +08:00
Feng Ruohang fd69c89d05 fix: complete CVE-2026-39414 S3 Select record limit enforcement
Route JSON Lines through the bounded PReader path so oversized records are rejected consistently instead of bypassing the limit on SIMD-capable CPUs.

Preserve S3 Select error codes in stream error events, wrap JSON parser errors as JSONParsingError, and flush completed records before returning a terminal error event. Add regression coverage for oversized JSON Lines input and error code preservation.

Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Code <claude-code@anthropic.com>
2026-06-12 21:08:59 +08:00
Feng Ruohang 5e40665acd fix: harden LDAP STS rate-limit source bucketing
Remove the per-username LDAP STS throttle bucket and keep the limiter keyed only by source IP. A username bucket is shared across all clients and lets one source keep a known account's bucket drained with bad-password attempts, locking the legitimate user out before LDAP bind.

For trusted proxies, stop trusting the left-most forwarded address. Resolve X-Forwarded-For right-to-left, skip trusted proxy hops, reject catch-all trusted-proxy CIDRs, and intentionally ignore RFC 7239 Forwarded for this security-sensitive bucket. Document that X-Real-IP is trusted verbatim and must be overwritten by the trusted proxy, not passed through from clients.

Update focused limiter, source-IP, trusted-proxy, and LDAP config tests to cover source-only buckets, spoofed appended XFF, multi-hop trusted proxies, Forwarded fallback, catch-all rejection, and the X-Real-IP deployment contract.

Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Code <claude-code@anthropic.com>
2026-06-12 21:08:59 +08:00
Feng Ruohang f48dbe777d docs: refresh security docs and fork references
Update the STS, security, select, and Docker documentation to reflect the recent hardening work, including LDAP STS throttling details, OIDC JWT verification changes, and the new pgsty-specific security policy and advisory index.

Rewrite repository and raw-document links that still pointed at minio/minio so the docs consistently reference pgsty/minio instead.

The core idea is to keep the documentation aligned with the fork's actual security behavior, ownership, and upgrade guidance without mixing in unrelated code changes.
2026-04-17 15:06:42 +08:00
Feng Ruohang f44110890b fix: tighten LDAP STS rate-limit accounting
Prevent LDAP STS reservation cancel paths from over-crediting rate-limit buckets by capping refill and refund capacity against in-flight reservations.

Add an explicit trusted-proxy allowlist for LDAP STS source bucketing, prefer clean X-Real-IP values on trusted peers, and extend tests/docs for the new behavior.
2026-04-16 23:22:13 +08:00
Feng Ruohang 9e10f6d9a0 fix: harden LDAP STS rate-limit source IP
Use the socket peer address for LDAP STS per-IP rate limiting instead of the generic forwarded-header-aware helper. This keeps the security-sensitive rate-limit key from trusting spoofable X-Forwarded-For, X-Real-IP, and Forwarded headers while leaving the rest of the source-IP behavior unchanged.

Add focused regression coverage for RemoteAddr parsing, header spoofing, and peer-address bucket selection.
2026-04-16 21:21:53 +08:00
Feng Ruohang 18b712d49a fix: preserve LDAP STS rate limits without penalizing success 2026-04-16 17:56:51 +08:00
Feng Ruohang db4c0fd5e3 fix: bump Go to 1.26.2 for upstream security fixes
Update go.mod and all golang build images from 1.26.1 to 1.26.2 to pick up the upstream 2026-04-07 security release.

This includes fixes for CVE-2026-32280 / CVE-2026-32281 in crypto/x509, CVE-2026-32283 in crypto/tls, and the related toolchain and standard library security fixes shipped in go1.26.2, without changing any unrelated dependencies.
2026-04-16 15:08:42 +08:00
Feng Ruohang efb6e5b00b fix: fake CVE-2026-40028 harden snowball unsigned-trailer auth
Track issue #28 / GHSA-9c4q-hq6p-c237 as fake CVE-2026-40028. Close the Snowball auto-extract auth gap in PutObjectExtractHandler by treating authTypeStreamingUnsignedTrailer the same as ordinary PUTs: honor X-Amz-Decoded-Content-Length, initialize newUnsignedV4ChunkedReader(), and verify the SigV4 request before any tar bytes reach untar(). This removes the forged-signature write primitive that let a single request fan out into arbitrary extracted object creation.

Add regression coverage for forged-signature Snowball unsigned-trailer writes, anonymous Snowball requests against non-public buckets, and legitimate signed Snowball extraction with trailing CRC32 trailers. Validate the new tests against the vulnerable parent and patched tree, and confirm with containerized before/after smoke runs that the exploit succeeds pre-fix, fails post-fix, and normal signed Snowball uploads still extract correctly.

Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Code <claude-code@anthropic.com>
2026-04-16 14:46:45 +08:00
Feng Ruohang f444b6f37e fix: fake CVE-2026-40027 block unsigned-trailer query auth bypass
Track issue #27 / GHSA-hv4r-mvr4-25vw as fake CVE-2026-40027. Close the unsigned-trailer trust flaw that let query-string credentials skip signature verification in PutObject and PutObjectPart by moving presigned rejection and SigV4 verification into newUnsignedV4ChunkedReader(), so authTypeStreamingUnsignedTrailer can no longer silently downgrade query auth into an anonymous body read.

Add focused regression coverage for forged query-string-only unsigned-trailer PUTs and multipart uploads, mixed header/query auth rejection, and anonymous unsigned-trailer writes that remain allowed only when bucket policy explicitly permits them. Validate the new tests against the vulnerable parent and confirm with before/after live-server runs that presigned unsigned-trailer attacks are rejected while legitimate header-authenticated and policy-driven flows still work.

Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Code <claude-code@anthropic.com>
2026-04-16 14:46:44 +08:00
Feng Ruohang 3252d5b7f3 fix: CVE-2026-39414 harden S3 Select oversized record handling
Enforce the 1 MiB maxCharsPerRecord limit while splitting CSV and line-delimited JSON input so oversized records are rejected before they can be buffered and parsed.

Return OverMaxRecordSize for these failures instead of collapsing them into InternalError, and preserve splitter errors in the JSON worker so oversized-record failures are not lost after successful partial decode.
2026-04-15 22:55:52 +08:00
Feng Ruohang 56fa63bfd1 fix: CVE-2026-34204 block replication metadata injection
Close the replication-header trust flaw that allowed ordinary PutObject and CopyObject requests to smuggle X-Minio-Replication-* headers into X-Minio-Internal-* SSE metadata and write objects into an unreadable state. Stop accepting replication-only metadata in the default extraction path, restore it only after a trusted replication write has passed ReplicateObjectAction, and tighten CopyObject by sanitizing replication-only request headers before metadata, precondition, and SSE-C source handling consume them. Also gate replica status writes on the same trusted replication path and restore replication SSE metadata in multipart and snowball upload flows so legitimate replication continues to work.

Add focused regression coverage for untrusted PUT and COPY header poisoning at the handler layer, plus helper tests for trusted vs untrusted metadata extraction and CopyObject header sanitization. Validate the new tests against both the patched tree and the vulnerable HEAD baseline, and confirm with live server before/after runs that malicious PUT/COPY requests no longer turn objects unreadable.

Co-authored-by: Codex <codex@openai.com>
Co-Authored-By: Claude <noreply@anthropic.com>
2026-04-15 18:36:49 +08:00
Feng Ruohang 3b950f8fa8 fix: CVE-2026-33419 harden LDAP STS auth
Prevent username enumeration in AssumeRoleWithLDAPIdentity by returning the same external STS error for unknown users and invalid passwords, while preserving LDAP infrastructure failures as upstream errors so they continue to surface as 500s and remain visible in server logs.

Add a small in-memory rate limiter for LDAP STS login attempts, keyed by source IP and normalized username, and add regression coverage for auth failure classification, throttling, and Docker-backed LDAP end-to-end flows.

Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Code <claude-code@anthropic.com>
2026-04-15 14:02:41 +08:00
Feng Ruohang d24f449e08 fix: CVE-2026-33322 harden OIDC JWT verification
Close the OIDC JWT algorithm confusion flaw in AssumeRoleWithWebIdentity by restoring a JWKS-only verification path. Stop injecting the client secret into the verifier keyring and restrict accepted signing methods to the asymmetric algorithms already supported by the existing JWKS flow.

Add regression coverage to verify HS256 tokens are rejected, RS256 tokens remain valid, and JWKS refresh and retry logic cannot bypass the method allowlist.

Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Code <claude-code@anthropic.com>
2026-04-15 14:02:41 +08:00
Feng Ruohang e4fa063942 Merge pull request #19 from mfredenhagen/mario/main/CVE-2026-39883-fix
Bump go.opentelemetry.io version to address CVE-2026-39883
2026-04-13 00:12:10 +08:00
Feng Ruohang ff58df9499 Merge pull request #18 from ZouhairCharef/master
fix: upgrade go-jose to v4.1.4 to patch CVE-2026-34986
2026-04-13 00:11:43 +08:00
Mario Fredenhagen 1869bd30b8 Bump go.opentelemetry.io version to address CVE-2026-39883 2026-04-10 14:33:39 +02:00
Zouhair EC-charef 68e0ba9971 fix: upgrade go-jose to v4.1.4 to patch CVE-2026-34986
Updates github.com/go-jose/go-jose/v4 from v4.1.3 to v4.1.4 to fix a high-severity denial of service vulnerability (CVE-2026-34986).
2026-04-06 20:34:51 +01:00
Feng Ruohang ce1c537eb1 fix: pin deps with breaking changes and fix LDAP TLS regression (#15)
Replace minio/pkg/v3 with pgsty/minio-pkg/v3 v3.6.3 to fix LDAP TLS
regression where DialURL() was not passing TLS config for ldaps://
connections, causing InsecureSkipVerify and RootCAs to be silently
ignored (x509: certificate signed by unknown authority).

Pin four dependencies to avoid breaking changes introduced in 5abd9a80f:
- go-ldap/ldap/v3 v3.4.12: v3.4.13 rewrote GetLDAPError() internals
- IBM/sarama v1.45.1: v1.46.0 changed Kafka protocol version negotiation
- lib/pq v1.10.9: v1.11.0 treats nil []byte as NULL and drops PG <14
- etcd v3.6.8: stay on intermediate version per policy

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 12:24:00 +08:00
Feng Ruohang ee55e5391a remove upstream CI/CD workflows inherited from minio/minio
We maintain our own release pipeline (release.yml, test-release.yml)
and have no use for the upstream test/lint/integration workflows.
They reference infrastructure and secrets we don't have, and the
PR-triggered jobs never fire since we don't take external pull requests.

Closes #14

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 09:41:26 +08:00
Feng Ruohang f2f9a40dce add mcli/mc from pgsty/mc to Docker image
Rework Dockerfile.goreleaser to download the latest mcli binary from
pgsty/mc GitHub releases, verify its SHA-256 checksum, and install both
mcli and mc (symlink) into the final image alongside minio and curl.
Also add download-static-curl.sh to goreleaser extra_files and enable
workflow_dispatch for the release workflow.
2026-03-24 09:15:52 +08:00
Feng Ruohang 377fc616d9 fix: satisfy stricter Go 1.26.1 linter checks
Go 1.26.1 tightens a few toolchain checks that older builds tolerated.\n\nCast aliased replication status values back to their defining type before calling the generated msgp helpers, and replace Sprintf+WriteString pairs with direct Fprintf calls where needed.\n\nThese are compatibility-only source changes to keep the cmd package building cleanly under the newer linker/toolchain.
2026-03-21 13:49:36 +08:00
Feng Ruohang 5abd9a80f6 bump golang to 1.26.1 and update deps 2026-03-21 13:41:04 +08:00
Feng Ruohang 00f3cf74fc RELEASE.2026-03-14T12-00-00Z with go 1.26.0
Switch to community-maintained console fork (georgmangold/console v1.9.1)
and update dependencies accordingly. Fix go vet format directive in
grid_test.go and adapt test status code for Go 1.26 HTTP semantics.
2026-03-14 17:39:57 +08:00
Feng Ruohang 68521b37f2 add github ci/cd pipeline 2026-02-18 10:00:20 +08:00
Feng Ruohang 8630937e7d Restore embedded console and update README for community fork
- Revert console dependency from stripped v1.7.7-pre to v1.7.6,
  restoring the full embedded management console
- Rewrite README disclaimer with proper trademark attribution and
  nominative fair use language for AGPL compliance
- Update documentation links and Go module paths to this repository
- Restore docs removed upstream (hotfixes.md, metrics/v3.md)
- Restore feature request issue template
- Update Go version to 1.26.0

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 11:47:30 +08:00
Feng Ruohang d4cd4b4337 RELEASE.2025-12-03T12-00-00Z with go 1.25.5 2025-12-15 16:39:39 +08:00
Harshavardhana 27742d4694 update README.md maintenance mode 2025-12-03 00:13:11 -08:00
Krutika Dhananjay 58659f26f4 Drop v3 metrics from community docs (#21678) 2025-11-06 02:38:16 -08:00
dorman 3a0cc6c86e fix doc 404 (#21670) 2025-10-26 19:47:37 -07:00
yangw 10b0a234d2 fix: update metric descriptions to specify current MinIO server instance (#21638)
Signed-off-by: yangw <wuyangmuc@gmail.com>
2025-10-23 21:06:31 -07:00
Raul-Mircea Crivineanu 18f97e70b1 Updates for conditional put read quorum issue (#21653) 2025-10-23 21:05:31 -07:00
Menno Finlay-Smits 52eee5a2f1 fix(api): Don't send multiple responses for one request (#21651)
fix(api): Don't send responses twice.

In some cases multiple responses are being sent for one request, causing
the API server to incorrectly drop connections.

This change introduces a ResponseWriter which tracks whether a
response has already been sent. This is used to prevent a response being
sent if something already has (e.g. by a preconditions check function).

Fixes #21633.

Co-authored-by: Menno Finlay-Smits <hello@menno.io>
2025-10-23 21:05:19 -07:00
Rishabh Agrahari c6d3aac5c4 Fix typo in entrypoint script path in README (#21657) 2025-10-23 08:10:39 -07:00
M Alvee fa18589d1c fix: Tagging in PostPolicy upload does not enforce policy tags (#21656) 2025-10-23 08:10:12 -07:00
Harshavardhana 05e569960a update scripts pointing to internal registry for community releases 2025-10-19 01:22:05 -07:00
Harshavardhana 9e49d5e7a6 update README.md and other docs to point to source only releases 2025-10-15 10:29:55 -07:00
Aditya Manthramurthy c1a49490c7 fix: check sub-policy properly when present (#21642)
This fixes a security issue where sub-policy attached to a service
account or STS account is not properly validated under certain "own"
account operations (like creating new service accounts). This allowed a
service account to create new service accounts for the same user
bypassing the inline policy restriction.
2025-10-15 10:00:45 -07:00
Ravind Kumar 334c313da4 Change documentation link in README (#21636)
Updated documentation link to point to the GitHub project.
2025-10-10 12:00:53 -07:00
cduzer 1b8ac0af9f fix: allow trailing slash in AWS S3 POST policies (#21612) 2025-10-10 11:57:35 -07:00
Mark Theunissen ba3c0fd1c7 Bump Go version in toolchain directive to 1.24.8 (#21629) 2025-10-10 11:57:03 -07:00
Ravind Kumar d51a4a4ff6 Update README with Docker and Helm installation instructions (#21627)
Added instructions for building Docker image and using Helm charts.

This closes the loop on supported methods for deploying MinIO with latest changes.
2025-10-09 15:10:11 -07:00
Harshavardhana 62383dfbfe Fix formatting of features in README.md 2025-10-07 09:59:23 -07:00
Ravind Kumar bde0d5a291 Updating readme for MinIO docs (#21625) 2025-10-06 22:36:26 -07:00
yangw 534f4a9fb1 fix: timeN function return final closure not be called (#21615) 2025-09-30 23:06:01 -07:00
Klaus Post b8631cf531 Use new gofumpt (#21613)
Update tinylib. Should fix CI.

`gofumpt -w .&&go generate ./...`
2025-09-28 13:59:21 -07:00
jiuker 456d9462e5 fix: after saveRebalanceStats cancel will be empty (#21597) 2025-09-19 21:51:57 -07:00
jiuker 756f3c8142 fix: incorrect poolID when after decommission adding pools (#21590) 2025-09-18 04:47:48 -07:00
mosesdd 7a80ec1cce fix: LDAP TLS handshake fails with StartTLS and tls_skip_verify=off (#21582)
Fixes #21581
2025-09-17 00:58:27 -07:00
M Alvee ae71d76901 fix: remove unnecessary replication checks (#21569) 2025-09-08 10:43:13 -07:00
M Alvee 07c3a429bf fix: conditional checks write for multipart (#21567) 2025-09-07 09:13:09 -07:00
Minio Trusted 0cde982902 Update yaml files to latest version RELEASE.2025-09-06T17-38-46Z 2025-09-07 05:14:10 +00:00
Ian Roberts d0f50cdd9b fix: use correct dummy ARN for claim-based OIDC provider when listing access keys (#21549)
fix: use correct dummy ARN for claim-based OIDC provider

When listing OIDC access keys, use the correct ARN when looking up the provider configuration for the claim-based provider.  Without this it was impossible to list access keys for a claim-based provider, only for a role-policy-based provider.

Fixes minio/minio#21548
2025-09-06 10:38:46 -07:00
WGH da532ab93d Fix support for legacy compression env variables (#21533)
Commit b6eb8dff64 renamed compression
setting environment variables to follow consistent style.

Although it preserved backward compatibility for the most part (i.e. it
handled MINIO_COMPRESS_ALLOW_ENCRYPTION, MINIO_COMPRESS_EXTENSIONS, and
MINIO_COMPRESS_MIME_TYPES), MINIO_COMPRESS_ENABLE was left behind.

Additionally, due to incorrect fallback ordering, and DefaultKVS
containing enable=off allow_encryption=off (so kvs.Get should've been
tried last), that commit broke MINIO_COMPRESS_ALLOW_ENCRYPTION (even
though it appeared to be handled), and even older MINIO_COMPRESS, too.

The legacy MIME types and extensions variables take precedence over both
config and new variables, so they don't need fixing.
2025-09-06 10:37:10 -07:00
M Alvee 558fc1c09c fix: return error on conditional write for non existing object (#21550) 2025-09-06 10:34:38 -07:00
Alex 9fdbf6fe83 Updated object-browser to the latest version v2.0.4 (#21564)
Signed-off-by: Benjamin Perez <benjamin@bexsoft.net>
2025-09-06 10:33:19 -07:00
jiuker 5c87d4ae87 fix: when save the rebalanceStats not found the config file (#21547) 2025-09-04 13:47:24 -07:00
Klaus Post f0b91e5504 Run modernize (#21546)
`go run golang.org/x/tools/gopls/internal/analysis/modernize/cmd/modernize@latest -fix -test ./...` executed.

`go generate ./...` ran afterwards to keep generated.
2025-08-28 19:39:48 -07:00
Manuel Reis 3b7cb6512c Revert dns.msgUnPath, fixes #21541 (#21542)
* Add more tests to UnPath function
* Revert implementation on dns.msgUnPath. Fixes: #21541
2025-08-28 10:31:12 -07:00
Mark Theunissen 4ea6f3b06b fix: invalid checksum on site replication with conforming checksum types (#21535) 2025-08-22 07:15:21 -07:00
jiuker 86d9d9b55e fix: use amqp.ParseURL to parse amqp url (#21528) 2025-08-20 21:25:07 -07:00
Denis Peshkov 5a35585acd http/listener: fix bugs and simplify (#21514)
* Store `ctx.Done` channel in a struct instead of a `ctx`. See: https://go.dev/blog/context-and-structs
* Return from `handleListener` on `ctx` cancellation, preventing goroutine leaks
* Simplify `handleListener` by removing the `send` closure. The `handleListener` is inlined by the compiler
* Return the first error from `Close`
* Preallocate slice in `Addrs`
* Reduce duplication in handling `opts.Trace`
* http/listener: revert error propagation from Close()
* http/listener: preserve original listener address in Addr()
* Preserve the original address when calling Addr() with multiple listeners
* Remove unused listeners from the slice
2025-08-12 11:22:12 -07:00
Daryl White 0848e69602 Update docs links throughout (#21513) 2025-08-12 11:20:36 -07:00
M Alvee 02ba581ecf custom user-agent transport wrapper (#21483) 2025-08-08 10:51:53 -07:00
Ian Roberts b44b2a090c fix: when claim-based OIDC is configured, treat unknown roleArn as claim-based auth (#21512)
RoleARN is a required parameter in AssumeRoleWithWebIdentity, 
according to the standard AWS implementation, and the official 
AWS SDKs and CLI will not allow you to assume a role from a JWT 
without also specifying a RoleARN.  This meant that it was not 
possible to use the official SDKs for claim-based OIDC with Minio 
(minio/minio#21421), since Minio required you to _omit_ the RoleARN in this case.

minio/minio#21468 attempted to fix this by disabling the validation 
of the RoleARN when a claim-based provider was configured, but this had 
the side effect of making it impossible to have a mixture of claim-based 
and role-based OIDC providers configured at the same time - every 
authentication would be treated as claim-based, ignoring the RoleARN entirely.

This is an alternative fix, whereby:

- _if_ the `RoleARN` is one that Minio knows about, then use the associated role policy
- if the `RoleARN` is not recognised, but there is a claim-based provider configured, then ignore the role ARN and attempt authentication with the claim-based provider
- if the `RoleARN` is not recognised, and there is _no_ claim-based provider, then return an error.
2025-08-08 10:51:23 -07:00
dorman c7d6a9722d Modify permission verification type (#21505) 2025-08-08 02:47:37 -07:00
jiuker a8abdc797e fix: add name and description to ldap accesskey list (#21511) 2025-08-07 19:46:04 -07:00
M Alvee 0638ccc5f3 fix: claim based oidc for official aws libraries (#21468) 2025-08-07 19:42:38 -07:00
jiuker b1a34fd63f fix: errUploadIDNotFound will be ignored when err is from peer client (#21504) 2025-08-07 19:38:41 -07:00
Klaus Post ffcfa36b13 Check legalHoldPerm (#21508)
The provided parameter should be checked before accepting legal hold
2025-08-07 19:38:25 -07:00
Aditya Kotra 376fbd11a7 fix(helm): do not suspend versioning by default for buckets, only set versioning if specified(21349) (#21494)
Signed-off-by: Aditya Kotra <kaditya030@gmail.com>
2025-08-07 02:47:02 -07:00
dorman c76f209ccc Optimize outdated commands in the log (#21498) 2025-08-06 16:48:58 -07:00
M Alvee 7a6a2256b1 imagePullSecrets consistent types for global , local (#21500) 2025-08-06 16:48:24 -07:00
Johannes Horn d002beaee3 feat: add variable for datasource in grafana dashboards (#21470) 2025-08-03 18:46:49 -07:00
jiuker 71f293d9ab fix: record extral skippedEntry for listObject (#21484) 2025-08-01 08:53:35 -07:00
jiuker e3d183b6a4 bring more idempotent behavior to AbortMultipartUpload() (#21475)
fix #21456
2025-07-30 23:57:23 -07:00
Alex 752abc2e2c Update console to v2.0.3 (#21474)
Signed-off-by: Benjamin Perez <benjamin@bexsoft.net>
Co-authored-by: Benjamin Perez <benjamin@bexsoft.net>
2025-07-30 10:57:17 -07:00
Minio Trusted b9f0e8c712 Update yaml files to latest version RELEASE.2025-07-23T15-54-02Z 2025-07-23 18:28:46 +00:00
540 changed files with 7746 additions and 6972 deletions
-3
View File
@@ -7,9 +7,6 @@ assignees: ''
---
## NOTE
If this case is urgent, please subscribe to [Subnet](https://min.io/pricing) so that our 24/7 support team may help you faster.
<!--- Provide a general summary of the issue in the Title above -->
## Expected Behavior
+3 -3
View File
@@ -2,7 +2,7 @@ blank_issues_enabled: false
contact_links:
- name: MinIO Community Support
url: https://slack.min.io
about: Join here for Community Support
- name: MinIO SUBNET Support
about: Community support via Slack - for questions and discussions
- name: MinIO Enterprise Support (SUBNET)
url: https://min.io/pricing
about: Join here for Enterprise Support
about: Enterprise support with SLA - for production deployments and feature requests
+108
View File
@@ -0,0 +1,108 @@
version: 2
env:
- CGO_ENABLED=0
builds:
- id: minio
main: .
binary: minio
goos:
- linux
- darwin
- windows
goarch:
- amd64
- arm64
goamd64:
- v1
flags:
- -tags=kqueue
- -trimpath
ldflags:
- "{{ .Env.LDFLAGS }}"
archives:
- id: minio
ids:
- minio
name_template: "minio_{{ .Env.PKG_VERSION }}_{{ .Os }}_{{ .Arch }}"
dockers:
- id: minio-amd64
ids:
- minio
goos: linux
goarch: amd64
dockerfile: Dockerfile.goreleaser
use: buildx
image_templates:
- "pgsty/minio:{{ .Tag }}-amd64"
- "pgsty/minio:latest-amd64"
build_flag_templates:
- "--platform=linux/amd64"
- "--label=org.opencontainers.image.version={{ .Tag }}"
- "--label=org.opencontainers.image.created={{ .Date }}"
- "--label=org.opencontainers.image.revision={{ .FullCommit }}"
extra_files:
- dockerscripts/docker-entrypoint.sh
- dockerscripts/download-static-curl.sh
- LICENSE
- CREDITS
- id: minio-arm64
ids:
- minio
goos: linux
goarch: arm64
dockerfile: Dockerfile.goreleaser
use: buildx
image_templates:
- "pgsty/minio:{{ .Tag }}-arm64"
- "pgsty/minio:latest-arm64"
build_flag_templates:
- "--platform=linux/arm64"
- "--label=org.opencontainers.image.version={{ .Tag }}"
- "--label=org.opencontainers.image.created={{ .Date }}"
- "--label=org.opencontainers.image.revision={{ .FullCommit }}"
extra_files:
- dockerscripts/docker-entrypoint.sh
- dockerscripts/download-static-curl.sh
- LICENSE
- CREDITS
docker_manifests:
- name_template: "pgsty/minio:{{ .Tag }}"
image_templates:
- "pgsty/minio:{{ .Tag }}-amd64"
- "pgsty/minio:{{ .Tag }}-arm64"
- name_template: "pgsty/minio:latest"
image_templates:
- "pgsty/minio:latest-amd64"
- "pgsty/minio:latest-arm64"
checksum:
name_template: "minio_{{ .Env.PKG_VERSION }}_checksums.txt"
algorithm: sha256
release:
github:
owner: pgsty
name: minio
draft: false
prerelease: false
mode: replace
replace_existing_artifacts: true
name_template: "{{ .Tag }}"
changelog:
sort: asc
filters:
exclude:
- "^docs:"
- "^test:"
- "Merge pull request"
- "Merge branch"
announce:
skip: true
-14
View File
@@ -1,14 +0,0 @@
name: 'Dependency Review'
on: [pull_request]
permissions:
contents: read
jobs:
dependency-review:
runs-on: ubuntu-latest
steps:
- name: 'Checkout Repository'
uses: actions/checkout@v4
- name: 'Dependency Review'
uses: actions/dependency-review-action@v4
-39
View File
@@ -1,39 +0,0 @@
name: Crosscompile
on:
pull_request:
branches:
- master
# This ensures that previous jobs for the PR are canceled when the PR is
# updated.
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
build:
name: Build Tests with Go ${{ matrix.go-version }} on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
go-version: [1.24.x]
os: [ubuntu-latest]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go-version }}
check-latest: true
- name: Build on ${{ matrix.os }}
if: matrix.os == 'ubuntu-latest'
env:
CGO_ENABLED: 0
GO111MODULE: on
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make crosscompile
-44
View File
@@ -1,44 +0,0 @@
name: Healing Functional Tests
on:
pull_request:
branches:
- master
# This ensures that previous jobs for the PR are canceled when the PR is
# updated.
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
build:
name: Go ${{ matrix.go-version }} on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
go-version: [1.24.x]
os: [ubuntu-latest]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go-version }}
check-latest: true
- name: Build on ${{ matrix.os }}
if: matrix.os == 'ubuntu-latest'
env:
CGO_ENABLED: 0
GO111MODULE: on
MINIO_KMS_SECRET_KEY: "my-minio-key:oyArl7zlPECEduNbB1KXgdzDn2Bdpvvw0l8VO51HQnY="
MINIO_KMS_AUTO_ENCRYPTION: on
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make verify-healing
make verify-healing-inconsistent-versions
make verify-healing-with-root-disks
make verify-healing-with-rewrite
-42
View File
@@ -1,42 +0,0 @@
name: Linters and Tests
on:
pull_request:
branches:
- master
# This ensures that previous jobs for the PR are canceled when the PR is
# updated.
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
build:
name: Go ${{ matrix.go-version }} on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
go-version: [1.24.x]
os: [ubuntu-latest]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go-version }}
check-latest: true
- name: Build on ${{ matrix.os }}
if: matrix.os == 'ubuntu-latest'
env:
CGO_ENABLED: 0
GO111MODULE: on
run: |
sudo apt install jq -y
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make
make test
make test-race
-39
View File
@@ -1,39 +0,0 @@
name: Resiliency Functional Tests
on:
pull_request:
branches:
- master
# This ensures that previous jobs for the PR are canceled when the PR is
# updated.
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
build:
name: Go ${{ matrix.go-version }} on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
go-version: [1.24.x]
os: [ubuntu-latest]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go-version }}
check-latest: true
- name: Build on ${{ matrix.os }}
if: matrix.os == 'ubuntu-latest'
env:
CGO_ENABLED: 0
GO111MODULE: on
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make test-resiliency
-42
View File
@@ -1,42 +0,0 @@
name: Functional Tests
on:
pull_request:
branches:
- master
# This ensures that previous jobs for the PR are canceled when the PR is
# updated.
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
build:
name: Go ${{ matrix.go-version }} on ${{ matrix.os }} - healing
runs-on: ${{ matrix.os }}
strategy:
matrix:
go-version: [1.24.x]
os: [ubuntu-latest]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go-version }}
check-latest: true
- name: Build on ${{ matrix.os }}
if: matrix.os == 'ubuntu-latest'
env:
CGO_ENABLED: 0
GO111MODULE: on
MINIO_KMS_SECRET_KEY: "my-minio-key:OSMM+vkKUTCvQs9YL/CVMIMt43HFhkUpqJxTmGl6rYw="
MINIO_KMS_AUTO_ENCRYPTION: on
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make verify
make test-timeout
-30
View File
@@ -1,30 +0,0 @@
name: Helm Chart linting
on:
pull_request:
branches:
- master
# This ensures that previous jobs for the PR are canceled when the PR is
# updated.
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
release:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Install Helm
uses: azure/setup-helm@v4
- name: Run helm lint
run: |
cd helm/minio
helm lint .
+110
View File
@@ -0,0 +1,110 @@
name: Helm Chart
on:
workflow_dispatch:
pull_request:
paths:
- ".github/workflows/helm.yml"
- "helm/**"
- "helm-releases/minio-*.tgz"
- "helm-reindex.sh"
- "index.yaml"
push:
branches:
- master
paths:
- ".github/workflows/helm.yml"
- "helm/**"
- "helm-releases/minio-*.tgz"
- "helm-reindex.sh"
- "index.yaml"
permissions:
contents: read
jobs:
install-smoke:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up Helm
uses: azure/setup-helm@v4
with:
version: v3.19.0
- name: Verify packaged chart
run: |
set -euo pipefail
chart_version="$(awk '$1 == "version:" { print $2; exit }' helm/minio/Chart.yaml)"
chart_package="helm-releases/minio-${chart_version}.tgz"
test -f "${chart_package}"
echo "CHART_PACKAGE=${chart_package}" >> "${GITHUB_ENV}"
helm lint --strict "${chart_package}"
unpacked="$(mktemp -d)"
tar -xzf "${chart_package}" -C "${unpacked}"
diff -ru --exclude Chart.yaml helm/minio "${unpacked}/minio"
package_digest="$(sha256sum "${chart_package}" | awk '{ print $1 }')"
grep -F "digest: ${package_digest}" index.yaml
grep -F "https://raw.githubusercontent.com/pgsty/minio/master/${chart_package}" index.yaml
- name: Render and verify immutable images
run: |
set -euo pipefail
expected_image='pgsty/minio@sha256:dacff8306a6e0a734518533992dbdcca26bc1ca47f77cf47cb9945725f92b29b'
rendered="$(mktemp)"
helm template silo "${CHART_PACKAGE}" \
--namespace silo \
--set mode=standalone \
--set persistence.enabled=false > "${rendered}"
grep -F "image: \"${expected_image}\"" "${rendered}"
if grep -Eq 'image:.*(quay\.io/minio|minio/minio|minio/mc)' "${rendered}"; then
echo "Rendered chart still references an upstream MinIO image."
exit 1
fi
- name: Create Kind cluster
uses: helm/kind-action@v1
with:
version: v0.31.0
kubectl_version: v1.35.0
node_image: kindest/node:v1.35.0@sha256:452d707d4862f52530247495d180205e029056831160e22870e37e3f6c1ac31f
cluster_name: silo-chart
wait: 120s
- name: Install chart
run: |
set -euo pipefail
helm install silo "${CHART_PACKAGE}" \
--namespace silo \
--create-namespace \
--wait \
--wait-for-jobs \
--timeout 10m \
--set mode=standalone \
--set replicas=1 \
--set persistence.enabled=false \
--set resources.requests.memory=512Mi \
--set rootUser=siloadmin \
--set rootPassword=silo-smoke-password \
--set 'buckets[0].name=bootstrap' \
--set 'buckets[0].policy=none' \
--set 'buckets[0].purge=false'
kubectl rollout status deployment/silo-minio --namespace silo --timeout=2m
- name: Run S3 smoke test
run: |
set -euo pipefail
helm test silo --namespace silo --logs --timeout 5m
- name: Diagnostics
if: failure()
run: |
helm status silo --namespace silo || true
kubectl get all --namespace silo -o wide || true
kubectl describe pods --namespace silo || true
kubectl logs --namespace silo --all-containers=true --prefix=true --selector release=silo || true
-161
View File
@@ -1,161 +0,0 @@
name: IAM integration
on:
pull_request:
branches:
- master
# This ensures that previous jobs for the PR are canceled when the PR is
# updated.
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
iam-matrix-test:
name: "[Go=${{ matrix.go-version }}|ldap=${{ matrix.ldap }}|etcd=${{ matrix.etcd }}|openid=${{ matrix.openid }}]"
runs-on: ubuntu-latest
services:
openldap:
image: quay.io/minio/openldap
ports:
- "389:389"
- "636:636"
env:
LDAP_ORGANIZATION: "MinIO Inc"
LDAP_DOMAIN: "min.io"
LDAP_ADMIN_PASSWORD: "admin"
etcd:
image: "quay.io/coreos/etcd:v3.5.1"
env:
ETCD_LISTEN_CLIENT_URLS: "http://0.0.0.0:2379"
ETCD_ADVERTISE_CLIENT_URLS: "http://0.0.0.0:2379"
ports:
- "2379:2379"
options: >-
--health-cmd "etcdctl endpoint health"
--health-interval 10s
--health-timeout 5s
--health-retries 5
openid:
image: quay.io/minio/dex
ports:
- "5556:5556"
env:
DEX_LDAP_SERVER: "openldap:389"
openid2:
image: quay.io/minio/dex
ports:
- "5557:5557"
env:
DEX_LDAP_SERVER: "openldap:389"
DEX_ISSUER: "http://127.0.0.1:5557/dex"
DEX_WEB_HTTP: "0.0.0.0:5557"
strategy:
# When ldap, etcd or openid vars are empty below, those external servers
# are turned off - i.e. if ldap="", then ldap server is not enabled for
# the tests.
matrix:
go-version: [1.24.x]
ldap: ["", "localhost:389"]
etcd: ["", "http://localhost:2379"]
openid: ["", "http://127.0.0.1:5556/dex"]
exclude:
# exclude combos where all are empty.
- ldap: ""
etcd: ""
openid: ""
# exclude combos where both ldap and openid IDPs are specified.
- ldap: "localhost:389"
openid: "http://127.0.0.1:5556/dex"
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go-version }}
check-latest: true
- name: Test LDAP/OpenID/Etcd combo
env:
_MINIO_LDAP_TEST_SERVER: ${{ matrix.ldap }}
_MINIO_ETCD_TEST_SERVER: ${{ matrix.etcd }}
_MINIO_OPENID_TEST_SERVER: ${{ matrix.openid }}
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make test-iam
- name: Test with multiple OpenID providers
if: matrix.openid == 'http://127.0.0.1:5556/dex'
env:
_MINIO_LDAP_TEST_SERVER: ${{ matrix.ldap }}
_MINIO_ETCD_TEST_SERVER: ${{ matrix.etcd }}
_MINIO_OPENID_TEST_SERVER: ${{ matrix.openid }}
_MINIO_OPENID_TEST_SERVER_2: "http://127.0.0.1:5557/dex"
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make test-iam
- name: Test with Access Management Plugin enabled
env:
_MINIO_LDAP_TEST_SERVER: ${{ matrix.ldap }}
_MINIO_ETCD_TEST_SERVER: ${{ matrix.etcd }}
_MINIO_OPENID_TEST_SERVER: ${{ matrix.openid }}
_MINIO_POLICY_PLUGIN_TEST_ENDPOINT: "http://127.0.0.1:8080"
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
go run docs/iam/access-manager-plugin.go &
make test-iam
- name: Test MinIO Old Version data to IAM import current version
if: matrix.ldap == 'ldaphost:389'
env:
_MINIO_LDAP_TEST_SERVER: ${{ matrix.ldap }}
run: |
make test-iam-ldap-upgrade-import
- name: Test LDAP for automatic site replication
if: matrix.ldap == 'localhost:389'
run: |
make test-site-replication-ldap
- name: Test OIDC for automatic site replication
if: matrix.openid == 'http://127.0.0.1:5556/dex'
run: |
make test-site-replication-oidc
iam-import-with-missing-entities:
name: Test IAM import in new cluster with missing entities
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go-version }}
check-latest: true
- name: Checkout minio-iam-testing
uses: actions/checkout@v4
with:
repository: minio/minio-iam-testing
path: minio-iam-testing
- name: Test import of IAM artifacts when in fresh cluster there are missing groups etc
run: |
make test-iam-import-with-missing-entities
iam-import-with-openid:
name: Test IAM import in new cluster with opendid configurations
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go-version }}
check-latest: true
- name: Checkout minio-iam-testing
uses: actions/checkout@v4
with:
repository: minio/minio-iam-testing
path: minio-iam-testing
- name: Test import of IAM artifacts when in fresh cluster with openid configurations
run: |
make test-iam-import-with-openid
-18
View File
@@ -1,18 +0,0 @@
# @format
name: Issue Workflow
on:
issues:
types:
- opened
jobs:
add-to-project:
name: Add issue to project
runs-on: ubuntu-latest
steps:
- uses: actions/add-to-project@v0.5.0
with:
project-url: https://github.com/orgs/miniohq/projects/2
github-token: ${{ secrets.BOT_PAT }}
-24
View File
@@ -1,24 +0,0 @@
name: 'Lock Threads'
on:
schedule:
- cron: '0 0 * * *'
workflow_dispatch:
permissions:
issues: write
concurrency:
group: lock
jobs:
action:
runs-on: ubuntu-latest
steps:
- uses: dessant/lock-threads@v3
with:
github-token: ${{ github.token }}
issue-inactive-days: '365'
exclude-any-issue-labels: 'do-not-close'
issue-lock-reason: 'resolved'
log-output: true
-81
View File
@@ -1,81 +0,0 @@
name: Mint Tests
on:
pull_request:
branches:
- master
# This ensures that previous jobs for the PR are canceled when the PR is
# updated.
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
mint-test:
runs-on: mint
timeout-minutes: 120
steps:
- name: cleanup #https://github.com/actions/checkout/issues/273
run: |
sudo -S rm -rf ${GITHUB_WORKSPACE}
mkdir ${GITHUB_WORKSPACE}
- name: checkout-step
uses: actions/checkout@v4
- name: setup-go-step
uses: actions/setup-go@v5
with:
go-version: 1.24.x
- name: github sha short
id: vars
run: echo "sha_short=$(git rev-parse --short HEAD)" >> $GITHUB_OUTPUT
- name: build-minio
run: |
TAG="quay.io/minio/minio:${{ steps.vars.outputs.sha_short }}" make docker
- name: multipart uploads test
run: |
${GITHUB_WORKSPACE}/.github/workflows/multipart/migrate.sh "${{ steps.vars.outputs.sha_short }}"
- name: compress and encrypt
run: |
${GITHUB_WORKSPACE}/.github/workflows/run-mint.sh "compress-encrypt" "minio" "minio123" "${{ steps.vars.outputs.sha_short }}"
- name: multiple pools
run: |
${GITHUB_WORKSPACE}/.github/workflows/run-mint.sh "pools" "minio" "minio123" "${{ steps.vars.outputs.sha_short }}"
- name: standalone erasure
run: |
${GITHUB_WORKSPACE}/.github/workflows/run-mint.sh "erasure" "minio" "minio123" "${{ steps.vars.outputs.sha_short }}"
# FIXME: renable this back when we have a valid way to add deadlines for PUT()s (internode CreateFile)
# - name: resiliency
# run: |
# ${GITHUB_WORKSPACE}/.github/workflows/run-mint.sh "resiliency" "minio" "minio123" "${{ steps.vars.outputs.sha_short }}"
- name: The job must cleanup
if: ${{ always() }}
run: |
export JOB_NAME=${{ steps.vars.outputs.sha_short }}
for mode in $(echo compress-encrypt pools erasure); do
docker-compose -f ${GITHUB_WORKSPACE}/.github/workflows/mint/minio-${mode}.yaml down || true
docker-compose -f ${GITHUB_WORKSPACE}/.github/workflows/mint/minio-${mode}.yaml rm || true
done
docker-compose -f ${GITHUB_WORKSPACE}/.github/workflows/multipart/docker-compose-site1.yaml rm -s -f || true
docker-compose -f ${GITHUB_WORKSPACE}/.github/workflows/multipart/docker-compose-site2.yaml rm -s -f || true
for volume in $(docker volume ls -q | grep minio); do
docker volume rm ${volume} || true
done
docker rmi -f quay.io/minio/minio:${{ steps.vars.outputs.sha_short }}
docker system prune -f || true
docker volume prune -f || true
docker volume rm $(docker volume ls -q -f dangling=true) || true
@@ -1,80 +0,0 @@
version: '3.7'
# Settings and configurations that are common for all containers
x-minio-common: &minio-common
image: quay.io/minio/minio:${JOB_NAME}
command: server --console-address ":9001" http://minio{1...4}/cdata{1...2}
expose:
- "9000"
- "9001"
environment:
MINIO_CI_CD: "on"
MINIO_ROOT_USER: "minio"
MINIO_ROOT_PASSWORD: "minio123"
MINIO_COMPRESSION_ENABLE: "on"
MINIO_COMPRESSION_MIME_TYPES: "*"
MINIO_COMPRESSION_ALLOW_ENCRYPTION: "on"
MINIO_KMS_SECRET_KEY: "my-minio-key:OSMM+vkKUTCvQs9YL/CVMIMt43HFhkUpqJxTmGl6rYw="
healthcheck:
test: ["CMD", "mc", "ready", "local"]
interval: 5s
timeout: 5s
retries: 5
# starts 4 docker containers running minio server instances.
# using nginx reverse proxy, load balancing, you can access
# it through port 9000.
services:
minio1:
<<: *minio-common
hostname: minio1
volumes:
- cdata1-1:/cdata1
- cdata1-2:/cdata2
minio2:
<<: *minio-common
hostname: minio2
volumes:
- cdata2-1:/cdata1
- cdata2-2:/cdata2
minio3:
<<: *minio-common
hostname: minio3
volumes:
- cdata3-1:/cdata1
- cdata3-2:/cdata2
minio4:
<<: *minio-common
hostname: minio4
volumes:
- cdata4-1:/cdata1
- cdata4-2:/cdata2
nginx:
image: nginx:1.19.2-alpine
hostname: nginx
volumes:
- ./nginx-4-node.conf:/etc/nginx/nginx.conf:ro
ports:
- "9000:9000"
- "9001:9001"
depends_on:
- minio1
- minio2
- minio3
- minio4
## By default this config uses default local driver,
## For custom volumes replace with volume driver configuration.
volumes:
cdata1-1:
cdata1-2:
cdata2-1:
cdata2-2:
cdata3-1:
cdata3-2:
cdata4-1:
cdata4-2:
-51
View File
@@ -1,51 +0,0 @@
version: '3.7'
# Settings and configurations that are common for all containers
x-minio-common: &minio-common
image: quay.io/minio/minio:${JOB_NAME}
command: server --console-address ":9001" edata{1...4}
expose:
- "9000"
- "9001"
environment:
MINIO_CI_CD: "on"
MINIO_ROOT_USER: "minio"
MINIO_ROOT_PASSWORD: "minio123"
MINIO_KMS_SECRET_KEY: "my-minio-key:OSMM+vkKUTCvQs9YL/CVMIMt43HFhkUpqJxTmGl6rYw="
healthcheck:
test: ["CMD", "mc", "ready", "local"]
interval: 5s
timeout: 5s
retries: 5
# starts 4 docker containers running minio server instances.
# using nginx reverse proxy, load balancing, you can access
# it through port 9000.
services:
minio1:
<<: *minio-common
hostname: minio1
volumes:
- edata1-1:/edata1
- edata1-2:/edata2
- edata1-3:/edata3
- edata1-4:/edata4
nginx:
image: nginx:1.19.2-alpine
hostname: nginx
volumes:
- ./nginx-1-node.conf:/etc/nginx/nginx.conf:ro
ports:
- "9000:9000"
- "9001:9001"
depends_on:
- minio1
## By default this config uses default local driver,
## For custom volumes replace with volume driver configuration.
volumes:
edata1-1:
edata1-2:
edata1-3:
edata1-4:
-117
View File
@@ -1,117 +0,0 @@
version: '3.7'
# Settings and configurations that are common for all containers
x-minio-common: &minio-common
image: quay.io/minio/minio:${JOB_NAME}
command: server --console-address ":9001" http://minio{1...4}/pdata{1...2} http://minio{5...8}/pdata{1...2}
expose:
- "9000"
- "9001"
environment:
MINIO_CI_CD: "on"
MINIO_ROOT_USER: "minio"
MINIO_ROOT_PASSWORD: "minio123"
MINIO_KMS_SECRET_KEY: "my-minio-key:OSMM+vkKUTCvQs9YL/CVMIMt43HFhkUpqJxTmGl6rYw="
healthcheck:
test: ["CMD", "mc", "ready", "local"]
interval: 5s
timeout: 5s
retries: 5
# starts 4 docker containers running minio server instances.
# using nginx reverse proxy, load balancing, you can access
# it through port 9000.
services:
minio1:
<<: *minio-common
hostname: minio1
volumes:
- pdata1-1:/pdata1
- pdata1-2:/pdata2
minio2:
<<: *minio-common
hostname: minio2
volumes:
- pdata2-1:/pdata1
- pdata2-2:/pdata2
minio3:
<<: *minio-common
hostname: minio3
volumes:
- pdata3-1:/pdata1
- pdata3-2:/pdata2
minio4:
<<: *minio-common
hostname: minio4
volumes:
- pdata4-1:/pdata1
- pdata4-2:/pdata2
minio5:
<<: *minio-common
hostname: minio5
volumes:
- pdata5-1:/pdata1
- pdata5-2:/pdata2
minio6:
<<: *minio-common
hostname: minio6
volumes:
- pdata6-1:/pdata1
- pdata6-2:/pdata2
minio7:
<<: *minio-common
hostname: minio7
volumes:
- pdata7-1:/pdata1
- pdata7-2:/pdata2
minio8:
<<: *minio-common
hostname: minio8
volumes:
- pdata8-1:/pdata1
- pdata8-2:/pdata2
nginx:
image: nginx:1.19.2-alpine
hostname: nginx
volumes:
- ./nginx-8-node.conf:/etc/nginx/nginx.conf:ro
ports:
- "9000:9000"
- "9001:9001"
depends_on:
- minio1
- minio2
- minio3
- minio4
- minio5
- minio6
- minio7
- minio8
## By default this config uses default local driver,
## For custom volumes replace with volume driver configuration.
volumes:
pdata1-1:
pdata1-2:
pdata2-1:
pdata2-2:
pdata3-1:
pdata3-2:
pdata4-1:
pdata4-2:
pdata5-1:
pdata5-2:
pdata6-1:
pdata6-2:
pdata7-1:
pdata7-2:
pdata8-1:
pdata8-2:
@@ -1,78 +0,0 @@
version: '3.7'
# Settings and configurations that are common for all containers
x-minio-common: &minio-common
image: quay.io/minio/minio:${JOB_NAME}
command: server --console-address ":9001" http://minio{1...4}/rdata{1...2}
expose:
- "9000"
- "9001"
environment:
MINIO_CI_CD: "on"
MINIO_ROOT_USER: "minio"
MINIO_ROOT_PASSWORD: "minio123"
MINIO_KMS_SECRET_KEY: "my-minio-key:OSMM+vkKUTCvQs9YL/CVMIMt43HFhkUpqJxTmGl6rYw="
MINIO_DRIVE_MAX_TIMEOUT: "5s"
healthcheck:
test: ["CMD", "mc", "ready", "local"]
interval: 5s
timeout: 5s
retries: 5
# starts 4 docker containers running minio server instances.
# using nginx reverse proxy, load balancing, you can access
# it through port 9000.
services:
minio1:
<<: *minio-common
hostname: minio1
volumes:
- rdata1-1:/rdata1
- rdata1-2:/rdata2
minio2:
<<: *minio-common
hostname: minio2
volumes:
- rdata2-1:/rdata1
- rdata2-2:/rdata2
minio3:
<<: *minio-common
hostname: minio3
volumes:
- rdata3-1:/rdata1
- rdata3-2:/rdata2
minio4:
<<: *minio-common
hostname: minio4
volumes:
- rdata4-1:/rdata1
- rdata4-2:/rdata2
nginx:
image: nginx:1.19.2-alpine
hostname: nginx
volumes:
- ./nginx-4-node.conf:/etc/nginx/nginx.conf:ro
ports:
- "9000:9000"
- "9001:9001"
depends_on:
- minio1
- minio2
- minio3
- minio4
## By default this config uses default local driver,
## For custom volumes replace with volume driver configuration.
volumes:
rdata1-1:
rdata1-2:
rdata2-1:
rdata2-2:
rdata3-1:
rdata3-2:
rdata4-1:
rdata4-2:
-100
View File
@@ -1,100 +0,0 @@
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 4096;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
keepalive_timeout 65;
# include /etc/nginx/conf.d/*.conf;
upstream minio {
server minio1:9000;
}
upstream console {
ip_hash;
server minio1:9001;
}
server {
listen 9000;
listen [::]:9000;
server_name localhost;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
proxy_request_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 300;
# Default is HTTP/1, keepalive is only enabled in HTTP/1.1
proxy_http_version 1.1;
proxy_set_header Connection "";
chunked_transfer_encoding off;
proxy_pass http://minio;
}
}
server {
listen 9001;
listen [::]:9001;
server_name localhost;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
proxy_request_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-NginX-Proxy true;
# This is necessary to pass the correct IP to be hashed
real_ip_header X-Real-IP;
proxy_connect_timeout 300;
# To support websocket
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
chunked_transfer_encoding off;
proxy_pass http://console;
}
}
}
-105
View File
@@ -1,105 +0,0 @@
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 4096;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
keepalive_timeout 65;
# include /etc/nginx/conf.d/*.conf;
upstream minio {
server minio1:9000 max_fails=1 fail_timeout=10s;
server minio2:9000 max_fails=1 fail_timeout=10s;
server minio3:9000 max_fails=1 fail_timeout=10s;
}
upstream console {
ip_hash;
server minio1:9001;
server minio2:9001;
server minio3:9001;
server minio4:9001;
}
server {
listen 9000;
listen [::]:9000;
server_name localhost;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
proxy_request_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 300;
# Default is HTTP/1, keepalive is only enabled in HTTP/1.1
proxy_http_version 1.1;
proxy_set_header Connection "";
chunked_transfer_encoding off;
proxy_pass http://minio;
}
}
server {
listen 9001;
listen [::]:9001;
server_name localhost;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
proxy_request_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-NginX-Proxy true;
# This is necessary to pass the correct IP to be hashed
real_ip_header X-Real-IP;
proxy_connect_timeout 300;
# To support websocket
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
chunked_transfer_encoding off;
proxy_pass http://console;
}
}
}
-114
View File
@@ -1,114 +0,0 @@
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 4096;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
keepalive_timeout 65;
# include /etc/nginx/conf.d/*.conf;
upstream minio {
server minio1:9000 max_fails=1 fail_timeout=10s;
server minio2:9000 max_fails=1 fail_timeout=10s;
server minio3:9000 max_fails=1 fail_timeout=10s;
server minio4:9000 max_fails=1 fail_timeout=10s;
server minio5:9000 max_fails=1 fail_timeout=10s;
server minio6:9000 max_fails=1 fail_timeout=10s;
server minio7:9000 max_fails=1 fail_timeout=10s;
server minio8:9000 max_fails=1 fail_timeout=10s;
}
upstream console {
ip_hash;
server minio1:9001;
server minio2:9001;
server minio3:9001;
server minio4:9001;
server minio5:9001;
server minio6:9001;
server minio7:9001;
server minio8:9001;
}
server {
listen 9000;
listen [::]:9000;
server_name localhost;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
proxy_request_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 300;
# Default is HTTP/1, keepalive is only enabled in HTTP/1.1
proxy_http_version 1.1;
proxy_set_header Connection "";
chunked_transfer_encoding off;
proxy_pass http://minio;
}
}
server {
listen 9001;
listen [::]:9001;
server_name localhost;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
proxy_request_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-NginX-Proxy true;
# This is necessary to pass the correct IP to be hashed
real_ip_header X-Real-IP;
proxy_connect_timeout 300;
# To support websocket
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
chunked_transfer_encoding off;
proxy_pass http://console;
}
}
}
-106
View File
@@ -1,106 +0,0 @@
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 4096;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
keepalive_timeout 65;
# include /etc/nginx/conf.d/*.conf;
upstream minio {
server minio1:9000 max_fails=1 fail_timeout=10s;
server minio2:9000 max_fails=1 fail_timeout=10s;
server minio3:9000 max_fails=1 fail_timeout=10s;
server minio4:9000 max_fails=1 fail_timeout=10s;
}
upstream console {
ip_hash;
server minio1:9001;
server minio2:9001;
server minio3:9001;
server minio4:9001;
}
server {
listen 9000;
listen [::]:9000;
server_name localhost;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
proxy_request_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 300;
# Default is HTTP/1, keepalive is only enabled in HTTP/1.1
proxy_http_version 1.1;
proxy_set_header Connection "";
chunked_transfer_encoding off;
proxy_pass http://minio;
}
}
server {
listen 9001;
listen [::]:9001;
server_name localhost;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
proxy_request_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-NginX-Proxy true;
# This is necessary to pass the correct IP to be hashed
real_ip_header X-Real-IP;
proxy_connect_timeout 300;
# To support websocket
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
chunked_transfer_encoding off;
proxy_pass http://console;
}
}
}
@@ -1,66 +0,0 @@
version: '3.7'
# Settings and configurations that are common for all containers
x-minio-common: &minio-common
image: quay.io/minio/minio:${RELEASE}
command: server http://site1-minio{1...4}/data{1...2}
environment:
- MINIO_PROMETHEUS_AUTH_TYPE=public
- CI=true
# starts 4 docker containers running minio server instances.
# using nginx reverse proxy, load balancing, you can access
# it through port 9000.
services:
site1-minio1:
<<: *minio-common
hostname: site1-minio1
volumes:
- site1-data1-1:/data1
- site1-data1-2:/data2
site1-minio2:
<<: *minio-common
hostname: site1-minio2
volumes:
- site1-data2-1:/data1
- site1-data2-2:/data2
site1-minio3:
<<: *minio-common
hostname: site1-minio3
volumes:
- site1-data3-1:/data1
- site1-data3-2:/data2
site1-minio4:
<<: *minio-common
hostname: site1-minio4
volumes:
- site1-data4-1:/data1
- site1-data4-2:/data2
site1-nginx:
image: nginx:1.19.2-alpine
hostname: site1-nginx
volumes:
- ./nginx-site1.conf:/etc/nginx/nginx.conf:ro
ports:
- "9001:9001"
depends_on:
- site1-minio1
- site1-minio2
- site1-minio3
- site1-minio4
## By default this config uses default local driver,
## For custom volumes replace with volume driver configuration.
volumes:
site1-data1-1:
site1-data1-2:
site1-data2-1:
site1-data2-2:
site1-data3-1:
site1-data3-2:
site1-data4-1:
site1-data4-2:
@@ -1,66 +0,0 @@
version: '3.7'
# Settings and configurations that are common for all containers
x-minio-common: &minio-common
image: quay.io/minio/minio:${RELEASE}
command: server http://site2-minio{1...4}/data{1...2}
environment:
- MINIO_PROMETHEUS_AUTH_TYPE=public
- CI=true
# starts 4 docker containers running minio server instances.
# using nginx reverse proxy, load balancing, you can access
# it through port 9000.
services:
site2-minio1:
<<: *minio-common
hostname: site2-minio1
volumes:
- site2-data1-1:/data1
- site2-data1-2:/data2
site2-minio2:
<<: *minio-common
hostname: site2-minio2
volumes:
- site2-data2-1:/data1
- site2-data2-2:/data2
site2-minio3:
<<: *minio-common
hostname: site2-minio3
volumes:
- site2-data3-1:/data1
- site2-data3-2:/data2
site2-minio4:
<<: *minio-common
hostname: site2-minio4
volumes:
- site2-data4-1:/data1
- site2-data4-2:/data2
site2-nginx:
image: nginx:1.19.2-alpine
hostname: site2-nginx
volumes:
- ./nginx-site2.conf:/etc/nginx/nginx.conf:ro
ports:
- "9002:9002"
depends_on:
- site2-minio1
- site2-minio2
- site2-minio3
- site2-minio4
## By default this config uses default local driver,
## For custom volumes replace with volume driver configuration.
volumes:
site2-data1-1:
site2-data1-2:
site2-data2-1:
site2-data2-2:
site2-data3-1:
site2-data3-2:
site2-data4-1:
site2-data4-2:
-147
View File
@@ -1,147 +0,0 @@
#!/bin/bash
set -x
## change working directory
cd .github/workflows/multipart/
function cleanup() {
docker-compose -f docker-compose-site1.yaml rm -s -f || true
docker-compose -f docker-compose-site2.yaml rm -s -f || true
for volume in $(docker volume ls -q | grep minio); do
docker volume rm ${volume} || true
done
docker system prune -f || true
docker volume prune -f || true
docker volume rm $(docker volume ls -q -f dangling=true) || true
}
cleanup
if [ ! -f ./mc ]; then
wget --quiet -O mc https://dl.minio.io/client/mc/release/linux-amd64/mc &&
chmod +x mc
fi
export RELEASE=RELEASE.2023-08-29T23-07-35Z
docker-compose -f docker-compose-site1.yaml up -d
docker-compose -f docker-compose-site2.yaml up -d
sleep 30s
./mc alias set site1 http://site1-nginx:9001 minioadmin minioadmin --api s3v4
./mc alias set site2 http://site2-nginx:9002 minioadmin minioadmin --api s3v4
./mc ready site1/
./mc ready site2/
./mc admin replicate add site1 site2
./mc mb site1/testbucket/
./mc cp -r --quiet /usr/bin site1/testbucket/
sleep 5
./s3-check-md5 -h
failed_count_site1=$(./s3-check-md5 -versions -access-key minioadmin -secret-key minioadmin -endpoint http://site1-nginx:9001 -bucket testbucket 2>&1 | grep FAILED | wc -l)
failed_count_site2=$(./s3-check-md5 -versions -access-key minioadmin -secret-key minioadmin -endpoint http://site2-nginx:9002 -bucket testbucket 2>&1 | grep FAILED | wc -l)
if [ $failed_count_site1 -ne 0 ]; then
echo "failed with multipart on site1 uploads"
exit 1
fi
if [ $failed_count_site2 -ne 0 ]; then
echo "failed with multipart on site2 uploads"
exit 1
fi
./mc cp -r --quiet /usr/bin site1/testbucket/
sleep 5
failed_count_site1=$(./s3-check-md5 -versions -access-key minioadmin -secret-key minioadmin -endpoint http://site1-nginx:9001 -bucket testbucket 2>&1 | grep FAILED | wc -l)
failed_count_site2=$(./s3-check-md5 -versions -access-key minioadmin -secret-key minioadmin -endpoint http://site2-nginx:9002 -bucket testbucket 2>&1 | grep FAILED | wc -l)
## we do not need to fail here, since we are going to test
## upgrading to master, healing and being able to recover
## the last version.
if [ $failed_count_site1 -ne 0 ]; then
echo "failed with multipart on site1 uploads ${failed_count_site1}"
fi
if [ $failed_count_site2 -ne 0 ]; then
echo "failed with multipart on site2 uploads ${failed_count_site2}"
fi
export RELEASE=${1}
docker-compose -f docker-compose-site1.yaml up -d
docker-compose -f docker-compose-site2.yaml up -d
./mc ready site1/
./mc ready site2/
for i in $(seq 1 10); do
# mc admin heal -r --remove when used against a LB endpoint
# behaves flaky, let this run 10 times before giving up
./mc admin heal -r --remove --json site1/ 2>&1 >/dev/null
./mc admin heal -r --remove --json site2/ 2>&1 >/dev/null
done
failed_count_site1=$(./s3-check-md5 -versions -access-key minioadmin -secret-key minioadmin -endpoint http://site1-nginx:9001 -bucket testbucket 2>&1 | grep FAILED | wc -l)
failed_count_site2=$(./s3-check-md5 -versions -access-key minioadmin -secret-key minioadmin -endpoint http://site2-nginx:9002 -bucket testbucket 2>&1 | grep FAILED | wc -l)
if [ $failed_count_site1 -ne 0 ]; then
echo "failed with multipart on site1 uploads"
exit 1
fi
if [ $failed_count_site2 -ne 0 ]; then
echo "failed with multipart on site2 uploads"
exit 1
fi
# Add user group test
./mc admin user add site1 site-replication-issue-user site-replication-issue-password
./mc admin group add site1 site-replication-issue-group site-replication-issue-user
max_wait_attempts=30
wait_interval=5
attempt=1
while true; do
diff <(./mc admin group info site1 site-replication-issue-group) <(./mc admin group info site2 site-replication-issue-group)
if [[ $? -eq 0 ]]; then
echo "Outputs are consistent."
break
fi
remaining_attempts=$((max_wait_attempts - attempt))
if ((attempt >= max_wait_attempts)); then
echo "Outputs remain inconsistent after $max_wait_attempts attempts. Exiting with error."
exit 1
else
echo "Outputs are inconsistent. Waiting for $wait_interval seconds (attempt $attempt/$max_wait_attempts)."
sleep $wait_interval
fi
((attempt++))
done
status=$(./mc admin group info site1 site-replication-issue-group --json | jq .groupStatus | tr -d '"')
if [[ $status == "enabled" ]]; then
echo "Success"
else
echo "Expected status: enabled, actual status: $status"
exit 1
fi
cleanup
## change working directory
cd ../../../
@@ -1,61 +0,0 @@
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 4096;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
keepalive_timeout 65;
# include /etc/nginx/conf.d/*.conf;
upstream minio {
server site1-minio1:9000;
server site1-minio2:9000;
server site1-minio3:9000;
server site1-minio4:9000;
}
server {
listen 9001;
listen [::]:9001;
server_name localhost;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
proxy_request_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 300;
# Default is HTTP/1, keepalive is only enabled in HTTP/1.1
proxy_http_version 1.1;
proxy_set_header Connection "";
chunked_transfer_encoding off;
proxy_pass http://minio;
}
}
}
@@ -1,61 +0,0 @@
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 4096;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
keepalive_timeout 65;
# include /etc/nginx/conf.d/*.conf;
upstream minio {
server site2-minio1:9000;
server site2-minio2:9000;
server site2-minio3:9000;
server site2-minio4:9000;
}
server {
listen 9002;
listen [::]:9002;
server_name localhost;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
proxy_request_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 300;
# Default is HTTP/1, keepalive is only enabled in HTTP/1.1
proxy_http_version 1.1;
proxy_set_header Connection "";
chunked_transfer_encoding off;
proxy_pass http://minio;
}
}
}
+141
View File
@@ -0,0 +1,141 @@
name: Release
on:
push:
tags:
- "RELEASE.*"
workflow_dispatch:
inputs:
tag:
description: "Release tag (e.g. RELEASE.2026-03-24T12-00-00Z)"
required: true
permissions:
contents: write
packages: write
jobs:
release:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version-file: go.mod
cache: true
- name: Compute release variables
run: |
set -euo pipefail
TAG="${{ github.event.inputs.tag || github.ref_name }}"
VERSION_HYPHEN="${TAG#RELEASE.}"
PKG_VERSION="$(echo "${VERSION_HYPHEN}" | sed -E 's/^([0-9]{4})-([0-9]{2})-([0-9]{2})T([0-9]{2})-([0-9]{2})-([0-9]{2})Z$/\1\2\3\4\5\6.0.0/')"
if [ "${PKG_VERSION}" = "${VERSION_HYPHEN}" ]; then
echo "Invalid release tag format: ${TAG}"
exit 1
fi
VERSION_COLON="$(echo "${VERSION_HYPHEN}" | sed -E 's/T([0-9]{2})-([0-9]{2})-([0-9]{2})Z$/T\1:\2:\3Z/')"
LDFLAGS="$(MINIO_RELEASE=RELEASE go run buildscripts/gen-ldflags.go "${VERSION_COLON}")"
{
echo "RELEASE_TAG=${TAG}"
echo "PKG_VERSION=${PKG_VERSION}"
echo "LDFLAGS=${LDFLAGS}"
} >> "${GITHUB_ENV}"
echo "Release tag: ${TAG}"
echo "Package version: ${PKG_VERSION}"
echo "LDFLAGS: ${LDFLAGS}"
- name: Validate Docker Hub credentials
run: |
set -euo pipefail
if [ -z "${{ secrets.DOCKERHUB_USERNAME }}" ] || [ -z "${{ secrets.DOCKERHUB_TOKEN }}" ]; then
echo "Missing Docker Hub credentials. Set DOCKERHUB_USERNAME and DOCKERHUB_TOKEN repository secrets."
exit 1
fi
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
with:
platforms: arm64
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and publish with GoReleaser
uses: goreleaser/goreleaser-action@v6
with:
version: "~> v2"
args: release --clean --skip=validate --config .github/goreleaser.yml
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
LDFLAGS: ${{ env.LDFLAGS }}
PKG_VERSION: ${{ env.PKG_VERSION }}
- name: Install pkger
run: |
go install github.com/minio/pkger/v2@v2.6.18
echo "$(go env GOPATH)/bin" >> "${GITHUB_PATH}"
- name: Prepare package layout
run: |
set -euo pipefail
copy_binary() {
local arch="$1"
local pattern="$2"
local src
src="$(find dist -maxdepth 2 -type f -path "dist/${pattern}/minio" | head -n1 || true)"
if [ -z "${src}" ]; then
echo "Missing GoReleaser binary for ${arch} (${pattern})"
exit 1
fi
mkdir -p "dist/linux-${arch}"
cp "${src}" "dist/linux-${arch}/minio.${RELEASE_TAG}"
}
copy_binary amd64 "minio_linux_amd64*"
copy_binary arm64 "minio_linux_arm64*"
- name: Build standard pkger packages
run: |
set -euo pipefail
pkger -r "${RELEASE_TAG}" --appName minio --releaseDir dist --ignore
# Keep only full package files; drop convenience symlinks (minio.rpm/minio.deb/minio.apk)
find dist/linux-* -maxdepth 1 -type l \
\( -name 'minio.rpm' -o -name 'minio.deb' -o -name 'minio.apk' \) -delete
find dist -maxdepth 2 -type f \
\( -name '*.rpm' -o -name '*.deb' -o -name '*.apk' -o -name '*.sha256sum' -o -name 'downloads-minio.json' \) | sort
- name: Upload pkger artifacts to GitHub release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
mapfile -t files < <(find dist -maxdepth 2 -type f \
\( -name '*.rpm' -o -name '*.deb' -o -name '*.apk' -o -name '*.sha256sum' -o -name 'downloads-minio.json' \) | sort)
if [ "${#files[@]}" -eq 0 ]; then
echo "No packages were generated."
exit 1
fi
gh release upload "${RELEASE_TAG}" "${files[@]}" --clobber
- name: Upload dist artifact
if: always()
uses: actions/upload-artifact@v4
with:
name: dist
path: dist/
-79
View File
@@ -1,79 +0,0 @@
name: MinIO advanced tests
on:
pull_request:
branches:
- master
# This ensures that previous jobs for the PR are canceled when the PR is
# updated.
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
replication-test:
name: Advanced Tests with Go ${{ matrix.go-version }}
runs-on: ubuntu-latest
strategy:
matrix:
go-version: [1.24.x]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go-version }}
check-latest: true
- name: Test Decom
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make test-decom
- name: Test ILM
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make test-ilm
make test-ilm-transition
- name: Test PBAC
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make test-pbac
- name: Test Config File
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make test-configfile
- name: Test Replication
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make test-replication
- name: Test MinIO IDP for automatic site replication
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make test-site-replication-minio
- name: Test Versioning
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make test-versioning
- name: Test Multipart upload with failures
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make test-multipart
-34
View File
@@ -1,34 +0,0 @@
name: Root lockdown tests
on:
pull_request:
branches:
- master
# This ensures that previous jobs for the PR are canceled when the PR is
# updated.
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
build:
name: Go ${{ matrix.go-version }} on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
go-version: [1.24.x]
os: [ubuntu-latest]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go-version }}
check-latest: true
- name: Start root lockdown tests
run: |
make test-root-disable
-9
View File
@@ -1,9 +0,0 @@
-----BEGIN CERTIFICATE-----
MIIBKDCB26ADAgECAhB6vebGMUfKnmBKyqoApRSOMAUGAytlcDAbMRkwFwYDVQQD
DBByb290QHBsYXkubWluLmlvMB4XDTIwMDQzMDE1MjIyNVoXDTI1MDQyOTE1MjIy
NVowGzEZMBcGA1UEAwwQcm9vdEBwbGF5Lm1pbi5pbzAqMAUGAytlcAMhALzn735W
fmSH/ghKs+4iPWziZMmWdiWr/sqvqeW+WwSxozUwMzAOBgNVHQ8BAf8EBAMCB4Aw
EwYDVR0lBAwwCgYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAFBgMrZXADQQDZOrGK
b2ATkDlu2pTcP3LyhSBDpYh7V4TvjRkBTRgjkacCzwFLm+mh+7US8V4dBpIDsJ4u
uWoF0y6vbLVGIlkG
-----END CERTIFICATE-----
-3
View File
@@ -1,3 +0,0 @@
-----BEGIN PRIVATE KEY-----
MC4CAQAwBQYDK2VwBCIEID9E7FSYWrMD+VjhI6q545cYT9YOyFxZb7UnjEepYDRc
-----END PRIVATE KEY-----
-64
View File
@@ -1,64 +0,0 @@
#!/bin/bash
set -ex
export MODE="$1"
export ACCESS_KEY="$2"
export SECRET_KEY="$3"
export JOB_NAME="$4"
export MINT_MODE="full"
docker system prune -f || true
docker volume prune -f || true
docker volume rm $(docker volume ls -f dangling=true) || true
## change working directory
cd .github/workflows/mint
## always pull latest
docker pull docker.io/minio/mint:edge
docker-compose -f minio-${MODE}.yaml up -d
sleep 1m
docker system prune -f || true
docker volume prune -f || true
docker volume rm $(docker volume ls -q -f dangling=true) || true
# Stop two nodes, one of each pool, to check that all S3 calls work while quorum is still there
[ "${MODE}" == "pools" ] && docker-compose -f minio-${MODE}.yaml stop minio2
[ "${MODE}" == "pools" ] && docker-compose -f minio-${MODE}.yaml stop minio6
# Pause one node, to check that all S3 calls work while one node goes wrong
[ "${MODE}" == "resiliency" ] && docker-compose -f minio-${MODE}.yaml pause minio4
docker run --rm --net=mint_default \
--name="mint-${MODE}-${JOB_NAME}" \
-e SERVER_ENDPOINT="nginx:9000" \
-e ACCESS_KEY="${ACCESS_KEY}" \
-e SECRET_KEY="${SECRET_KEY}" \
-e ENABLE_HTTPS=0 \
-e MINT_MODE="${MINT_MODE}" \
docker.io/minio/mint:edge
# FIXME: enable this after fixing aws-sdk-java-v2 tests
# # unpause the node, to check that all S3 calls work while one node goes wrong
# [ "${MODE}" == "resiliency" ] && docker-compose -f minio-${MODE}.yaml unpause minio4
# [ "${MODE}" == "resiliency" ] && docker run --rm --net=mint_default \
# --name="mint-${MODE}-${JOB_NAME}" \
# -e SERVER_ENDPOINT="nginx:9000" \
# -e ACCESS_KEY="${ACCESS_KEY}" \
# -e SECRET_KEY="${SECRET_KEY}" \
# -e ENABLE_HTTPS=0 \
# -e MINT_MODE="${MINT_MODE}" \
# docker.io/minio/mint:edge
docker-compose -f minio-${MODE}.yaml down || true
sleep 10s
docker system prune -f || true
docker volume prune -f || true
docker volume rm $(docker volume ls -q -f dangling=true) || true
## change working directory
cd ../../../
-22
View File
@@ -1,22 +0,0 @@
name: Shell formatting checks
on:
pull_request:
branches:
- master
permissions:
contents: read
jobs:
build:
name: runner / shfmt
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: luizm/action-sh-checker@master
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SHFMT_OPTS: "-s"
with:
sh_checker_shellcheck_disable: true # disable for now
+89
View File
@@ -0,0 +1,89 @@
name: Test Release Pipeline
on:
workflow_dispatch:
pull_request:
paths:
- ".github/goreleaser.yml"
- "Dockerfile.goreleaser"
- "minio.service"
- ".github/workflows/release.yml"
- ".github/workflows/test-release.yml"
permissions:
contents: read
jobs:
validate:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version-file: go.mod
cache: true
- name: Compute test variables
run: |
set -euo pipefail
RELEASE_TAG="RELEASE.2026-02-14T12-00-00Z"
VERSION_COLON="2026-02-14T12:00:00Z"
PKG_VERSION="20260214120000.0.0"
LDFLAGS="$(MINIO_RELEASE=RELEASE go run buildscripts/gen-ldflags.go "${VERSION_COLON}")"
echo "RELEASE_TAG=${RELEASE_TAG}" >> "${GITHUB_ENV}"
echo "PKG_VERSION=${PKG_VERSION}" >> "${GITHUB_ENV}"
echo "LDFLAGS=${LDFLAGS}" >> "${GITHUB_ENV}"
echo "PKG_VERSION: ${PKG_VERSION}"
echo "LDFLAGS: ${LDFLAGS}"
- name: GoReleaser config check
uses: goreleaser/goreleaser-action@v6
with:
version: "~> v2"
args: check --config .github/goreleaser.yml
- name: Build snapshot artifacts
uses: goreleaser/goreleaser-action@v6
with:
version: "~> v2"
args: release --snapshot --clean --skip=publish,docker --config .github/goreleaser.yml
env:
LDFLAGS: ${{ env.LDFLAGS }}
PKG_VERSION: ${{ env.PKG_VERSION }}
- name: Install pkger
run: |
go install github.com/minio/pkger/v2@v2.6.18
echo "$(go env GOPATH)/bin" >> "${GITHUB_PATH}"
- name: Package snapshot binaries with pkger
run: |
set -euo pipefail
copy_binary() {
local arch="$1"
local pattern="$2"
local src
src="$(find dist -maxdepth 2 -type f -path "dist/${pattern}/minio" | head -n1 || true)"
if [ -z "${src}" ]; then
echo "Missing GoReleaser binary for ${arch} (${pattern})"
exit 1
fi
mkdir -p "dist/linux-${arch}"
cp "${src}" "dist/linux-${arch}/minio.${RELEASE_TAG}"
}
copy_binary amd64 "minio_linux_amd64*"
copy_binary arm64 "minio_linux_arm64*"
pkger -r "${RELEASE_TAG}" --appName minio --releaseDir dist --ignore
# Keep only full package files; drop convenience symlinks (minio.rpm/minio.deb/minio.apk)
find dist/linux-* -maxdepth 1 -type l \
\( -name 'minio.rpm' -o -name 'minio.deb' -o -name 'minio.apk' \) -delete
find dist -maxdepth 2 -type f \
\( -name '*.rpm' -o -name '*.deb' -o -name '*.apk' -o -name '*.sha256sum' -o -name 'downloads-minio.json' \) | sort
-15
View File
@@ -1,15 +0,0 @@
---
name: Spelling
on: [pull_request]
jobs:
run:
name: Spell Check with Typos
runs-on: ubuntu-latest
steps:
- name: Checkout Actions Repository
uses: actions/checkout@v4
- name: Check spelling of repo
uses: crate-ci/typos@master
-34
View File
@@ -1,34 +0,0 @@
name: Upgrade old version tests
on:
pull_request:
branches:
- master
# This ensures that previous jobs for the PR are canceled when the PR is
# updated.
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
build:
name: Go ${{ matrix.go-version }} on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
go-version: [1.24.x]
os: [ubuntu-latest]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go-version }}
check-latest: true
- name: Start upgrade tests
run: |
make test-upgrade
-31
View File
@@ -1,31 +0,0 @@
name: VulnCheck
on:
pull_request:
branches:
- master
push:
branches:
- master
permissions:
contents: read # to fetch code (actions/checkout)
jobs:
vulncheck:
name: Analysis
runs-on: ubuntu-latest
steps:
- name: Check out code into the Go module directory
uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: 1.24.x
cached: false
- name: Get official govulncheck
run: go install golang.org/x/vuln/cmd/govulncheck@latest
shell: bash
- name: Run govulncheck
run: govulncheck -show verbose ./...
shell: bash
+11
View File
@@ -53,3 +53,14 @@ s3-check-md5
s3-verify
xattr
xl-meta
.gitignore
.goreleaser.yml
dist/
.claude/
.codex/
_bmad/
_bmad-output/
+7 -1
View File
@@ -1,8 +1,14 @@
FROM minio/minio:latest
ARG TARGETARCH
ARG RELEASE
RUN chmod -R 777 /usr/bin
COPY ./minio /usr/bin/minio
COPY ./minio-${TARGETARCH}.${RELEASE} /usr/bin/minio
COPY ./minio-${TARGETARCH}.${RELEASE}.minisig /usr/bin/minio.minisig
COPY ./minio-${TARGETARCH}.${RELEASE}.sha256sum /usr/bin/minio.sha256sum
COPY dockerscripts/docker-entrypoint.sh /usr/bin/docker-entrypoint.sh
ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"]
+96
View File
@@ -0,0 +1,96 @@
FROM golang:1.26.4-alpine AS build
ARG TARGETARCH
ENV GOPATH=/go
ENV CGO_ENABLED=0
ARG MC_REPO=pgsty/mc
ARG MC_VERSION=latest
RUN apk add -U --no-cache \
ca-certificates \
bash \
curl \
jq && \
case "${TARGETARCH}" in \
amd64) MC_ARCH=amd64 ;; \
arm64) MC_ARCH=arm64 ;; \
*) echo "Unsupported TARGETARCH=${TARGETARCH}"; exit 1 ;; \
esac && \
if [ "${MC_VERSION}" = "latest" ]; then \
MC_RELEASE_URL="https://api.github.com/repos/${MC_REPO}/releases/latest"; \
else \
MC_RELEASE_URL="https://api.github.com/repos/${MC_REPO}/releases/tags/${MC_VERSION}"; \
fi && \
curl -fsSL "${MC_RELEASE_URL}" -o /tmp/mc-release.json && \
MC_ARCHIVE_URL=$(jq -r --arg arch "${MC_ARCH}" \
'.assets[] | select(.name | endswith("_linux_" + $arch + ".tar.gz")) | .browser_download_url' \
/tmp/mc-release.json | head -n 1) && \
MC_CHECKSUM_URL=$(jq -r \
'.assets[] | select(.name | endswith("_checksums.txt")) | .browser_download_url' \
/tmp/mc-release.json | head -n 1) && \
[ -n "${MC_ARCHIVE_URL}" ] || { echo "Cannot find mcli archive for linux/${MC_ARCH}"; exit 1; } && \
[ -n "${MC_CHECKSUM_URL}" ] || { echo "Cannot find mcli checksums file"; exit 1; } && \
ARCHIVE_NAME=$(basename "${MC_ARCHIVE_URL}") && \
echo "Downloading ${ARCHIVE_NAME} ..." && \
curl -fsSL "${MC_ARCHIVE_URL}" -o /tmp/mcli.tar.gz && \
curl -fsSL "${MC_CHECKSUM_URL}" -o /tmp/mcli_checksums.txt && \
EXPECTED=$(grep " ${ARCHIVE_NAME}$" /tmp/mcli_checksums.txt | awk '{print $1}') && \
ACTUAL=$(sha256sum /tmp/mcli.tar.gz | awk '{print $1}') && \
[ -n "${EXPECTED}" ] || { echo "Checksum entry not found for ${ARCHIVE_NAME}"; exit 1; } && \
[ "${EXPECTED}" = "${ACTUAL}" ] || { echo "Checksum mismatch: expected ${EXPECTED}, got ${ACTUAL}"; exit 1; } && \
echo "Checksum OK: ${ACTUAL}" && \
mkdir -p /tmp/mcli-extract && \
tar -xzf /tmp/mcli.tar.gz -C /tmp/mcli-extract/ && \
if [ -f /tmp/mcli-extract/mcli ]; then \
cp /tmp/mcli-extract/mcli /go/bin/mcli; \
elif [ -f /tmp/mcli-extract/mc ]; then \
cp /tmp/mcli-extract/mc /go/bin/mcli; \
else \
echo "No mc or mcli binary found in archive:"; ls -la /tmp/mcli-extract/; exit 1; \
fi && \
chmod +x /go/bin/mcli && \
ln -sf mcli /go/bin/mc
COPY dockerscripts/download-static-curl.sh /build/download-static-curl
RUN chmod +x /build/download-static-curl && \
/build/download-static-curl
FROM registry.access.redhat.com/ubi9/ubi:latest AS certs
RUN dnf -y install ca-certificates && \
update-ca-trust && \
cp /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem /tmp/ca-certificates.crt && \
dnf clean all && \
rm -rf /var/cache/dnf
FROM registry.access.redhat.com/ubi9/ubi-micro:latest
LABEL maintainer="pgsty <https://github.com/pgsty/minio>" \
description="MinIO community fork, build by pgsty"
ENV MINIO_ACCESS_KEY_FILE=access_key \
MINIO_SECRET_KEY_FILE=secret_key \
MINIO_ROOT_USER_FILE=access_key \
MINIO_ROOT_PASSWORD_FILE=secret_key \
MINIO_KMS_SECRET_KEY_FILE=kms_master_key \
MINIO_UPDATE_MINISIGN_PUBKEY="RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav" \
MINIO_CONFIG_ENV_FILE=config.env \
MC_CONFIG_DIR=/tmp/.mc
COPY --from=certs /tmp/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
COPY minio /usr/bin/minio
COPY --from=build /go/bin/mcli /usr/bin/mcli
COPY --from=build /go/bin/curl* /usr/bin/
COPY dockerscripts/docker-entrypoint.sh /usr/bin/docker-entrypoint.sh
COPY LICENSE /licenses/LICENSE
COPY CREDITS /licenses/CREDITS
RUN chmod +x /usr/bin/minio /usr/bin/mcli /usr/bin/docker-entrypoint.sh && \
ln -sf mcli /usr/bin/mc
EXPOSE 9000
VOLUME ["/data"]
ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"]
CMD ["minio"]
+1 -1
View File
@@ -1,4 +1,4 @@
FROM golang:1.24-alpine as build
FROM golang:1.26.4-alpine as build
ARG TARGETARCH
ARG RELEASE
+1 -1
View File
@@ -1,4 +1,4 @@
FROM golang:1.24-alpine AS build
FROM golang:1.26.4-alpine AS build
ARG TARGETARCH
ARG RELEASE
+1 -1
View File
@@ -1,4 +1,4 @@
FROM golang:1.24-alpine AS build
FROM golang:1.26.4-alpine AS build
ARG TARGETARCH
ARG RELEASE
+1 -1
View File
@@ -32,7 +32,7 @@ verifiers: lint check-gen
check-gen: ## check for updated autogenerated files
@go generate ./... >/dev/null
@go mod tidy -compat=1.21
@go mod tidy -compat=1.26
@(! git diff --name-only | grep '_gen.go$$') || (echo "Non-committed changes in auto-generated code is detected, please commit them to proceed." && false)
@(! git diff --name-only | grep 'go.sum') || (echo "Non-committed changes in auto-generated go.sum is detected, please commit them to proceed." && false)
+30 -261
View File
@@ -1,274 +1,43 @@
# MinIO Quickstart Guide
# Silo (Community maintained fork of MinIO)
[![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io) [![Docker Pulls](https://img.shields.io/docker/pulls/minio/minio.svg?maxAge=604800)](https://hub.docker.com/r/minio/minio/) [![license](https://img.shields.io/badge/license-AGPL%20V3-blue)](https://github.com/minio/minio/blob/master/LICENSE)
[![Website: silo.pigsty.io](https://img.shields.io/badge/website-silo.pigsty.io-slategray?style=flat&logo=cilium&logoColor=white)](https://silo.pigsty.io)
[![CN: silo.pigsty.cc](https://img.shields.io/badge/网站-silo.pigsty.cc-slategray?style=flat&logo=cilium&logoColor=white)](https://silo.pigsty.cc)
[![github: pgsty/minio](https://img.shields.io/badge/Repo-pgsty/minio-slategray?style=flat&logo=github&logoColor=white)](https://github.com/pgsty/minio)
[![github: pgsty/mc](https://img.shields.io/badge/Repo-pgsty/mc-slategray?style=flat&logo=github&logoColor=white)](https://github.com/pgsty/mc)
[![github: pgsty/minio-docs](https://img.shields.io/badge/Repo-pgsty/minio--docs-slategray?style=flat&logo=github&logoColor=white)](https://github.com/pgsty/minio-docs)
[![Docker Image](https://img.shields.io/badge/Docker-pgsty/minio-%232496ED?style=flat&logo=docker&logoColor=white)](https://hub.docker.com/r/pgsty/minio)
[![MinIO](https://raw.githubusercontent.com/minio/minio/master/.github/logo.svg?sanitize=true)](https://min.io)
> [!IMPORTANT]
> **This is a community-maintained fork of the upstream MinIO project, maintained by [Pigsty](https://pigsty.io).**
> This project is **NOT** affiliated with, endorsed by, or sponsored by MinIO, Inc.
> "MinIO" is a trademark of MinIO, Inc., used here solely to identify the upstream project.
>
> Changes from upstream are minimal:
> - Restored the embedded management console version reference
> - Updated documentation links and Go module paths to point to this repository
>
> Distributed under the original [GNU AGPLv3](LICENSE) license.
MinIO is a high-performance, S3-compatible object storage solution released under the GNU AGPL v3.0 license. Designed for speed and scalability, it powers AI/ML, analytics, and data-intensive workloads with industry-leading performance.
Documentation: [English](https://silo.pigsty.io) | [简体中文](https://silo.pigsty.cc)
🔹 S3 API Compatible Seamless integration with existing S3 tools
🔹 Built for AI & Analytics Optimized for large-scale data pipelines
🔹 High Performance Ideal for demanding storage workloads.
Docker Hub: [https://hub.docker.com/r/pgsty/minio](https://hub.docker.com/r/pgsty/minio) / [https://hub.docker.com/r/pgsty/mc](https://hub.docker.com/r/pgsty/mc)
AI storage documentation (https://min.io/solutions/object-storage-for-ai).
Client Repo: [`pgsty/mc`](https://github.com/pgsty/mc) CLI.
This README provides quickstart instructions on running MinIO on bare metal hardware, including container-based installations. For Kubernetes environments, use the [MinIO Kubernetes Operator](https://github.com/minio/operator/blob/master/README.md).
Console: [`georgmangold/console`](https://github.com/georgmangold/console/), a community-maintained fork of restored console.
## Container Installation
Ansible Deployment: [https://pigsty.io/docs/minio](https://pigsty.io/docs/minio)
Use the following commands to run a standalone MinIO server as a container.
APT/YUM repo for `minio` and `mcli` binary: [https://pigsty.io/docs/infra](https://pigsty.io/docs/repo/infra/list/#object-storage)
Standalone MinIO servers are best suited for early development and evaluation. Certain features such as versioning, object locking, and bucket replication
require distributed deploying MinIO with Erasure Coding. For extended development and production, deploy MinIO with Erasure Coding enabled - specifically,
with a *minimum* of 4 drives per MinIO server. See [MinIO Erasure Code Overview](https://min.io/docs/minio/linux/operations/concepts/erasure-coding.html)
for more complete documentation.
## Kubernetes delivery and scope
### Stable
This repository maintains the direct-deployment Helm chart in [`helm/minio`](helm/minio/README.md) for standalone and distributed Silo server deployments. Chart defaults use the version- and digest-pinned multi-architecture [`pgsty/minio`](https://hub.docker.com/r/pgsty/minio) image, and every chart change is checked by a real Kind install plus an S3 write/read/delete smoke test.
Run the following command to run the latest stable image of MinIO as a container using an ephemeral data volume:
```sh
podman run -p 9000:9000 -p 9001:9001 \
quay.io/minio/minio server /data --console-address ":9001"
```bash
helm repo add silo https://raw.githubusercontent.com/pgsty/minio/master
helm repo update silo
helm install my-release silo/minio --namespace minio --create-namespace
```
The MinIO deployment starts using default root credentials `minioadmin:minioadmin`. You can test the deployment using the MinIO Console, an embedded
object browser built into MinIO Server. Point a web browser running on the host machine to <http://127.0.0.1:9000> and log in with the
root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.
You can also connect using any S3-compatible tool, such as the MinIO Client `mc` commandline tool. See
[Test using MinIO Client `mc`](#test-using-minio-client-mc) for more information on using the `mc` commandline tool. For application developers,
see <https://min.io/docs/minio/linux/developers/minio-drivers.html> to view MinIO SDKs for supported languages.
> [!NOTE]
> To deploy MinIO on with persistent storage, you must map local persistent directories from the host OS to the container using the `podman -v` option.
> For example, `-v /mnt/data:/data` maps the host OS drive at `/mnt/data` to `/data` on the container.
## macOS
Use the following commands to run a standalone MinIO server on macOS.
Standalone MinIO servers are best suited for early development and evaluation. Certain features such as versioning, object locking, and bucket replication require distributed deploying MinIO with Erasure Coding. For extended development and production, deploy MinIO with Erasure Coding enabled - specifically, with a *minimum* of 4 drives per MinIO server. See [MinIO Erasure Code Overview](https://min.io/docs/minio/linux/operations/concepts/erasure-coding.html) for more complete documentation.
### Homebrew (recommended)
Run the following command to install the latest stable MinIO package using [Homebrew](https://brew.sh/). Replace ``/data`` with the path to the drive or directory in which you want MinIO to store data.
```sh
brew install minio/stable/minio
minio server /data
```
> [!NOTE]
> If you previously installed minio using `brew install minio` then it is recommended that you reinstall minio from `minio/stable/minio` official repo instead.
```sh
brew uninstall minio
brew install minio/stable/minio
```
The MinIO deployment starts using default root credentials `minioadmin:minioadmin`. You can test the deployment using the MinIO Console, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to <http://127.0.0.1:9000> and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.
You can also connect using any S3-compatible tool, such as the MinIO Client `mc` commandline tool. See [Test using MinIO Client `mc`](#test-using-minio-client-mc) for more information on using the `mc` commandline tool. For application developers, see <https://min.io/docs/minio/linux/developers/minio-drivers.html/> to view MinIO SDKs for supported languages.
### Binary Download
Use the following command to download and run a standalone MinIO server on macOS. Replace ``/data`` with the path to the drive or directory in which you want MinIO to store data.
```sh
wget https://dl.min.io/server/minio/release/darwin-amd64/minio
chmod +x minio
./minio server /data
```
The MinIO deployment starts using default root credentials `minioadmin:minioadmin`. You can test the deployment using the MinIO Console, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to <http://127.0.0.1:9000> and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.
You can also connect using any S3-compatible tool, such as the MinIO Client `mc` commandline tool. See [Test using MinIO Client `mc`](#test-using-minio-client-mc) for more information on using the `mc` commandline tool. For application developers, see <https://min.io/docs/minio/linux/developers/minio-drivers.html> to view MinIO SDKs for supported languages.
## GNU/Linux
Use the following command to run a standalone MinIO server on Linux hosts running 64-bit Intel/AMD architectures. Replace ``/data`` with the path to the drive or directory in which you want MinIO to store data.
```sh
wget https://dl.min.io/server/minio/release/linux-amd64/minio
chmod +x minio
./minio server /data
```
The following table lists supported architectures. Replace the `wget` URL with the architecture for your Linux host.
| Architecture | URL |
| -------- | ------ |
| 64-bit Intel/AMD | <https://dl.min.io/server/minio/release/linux-amd64/minio> |
| 64-bit ARM | <https://dl.min.io/server/minio/release/linux-arm64/minio> |
| 64-bit PowerPC LE (ppc64le) | <https://dl.min.io/server/minio/release/linux-ppc64le/minio> |
The MinIO deployment starts using default root credentials `minioadmin:minioadmin`. You can test the deployment using the MinIO Console, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to <http://127.0.0.1:9000> and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.
You can also connect using any S3-compatible tool, such as the MinIO Client `mc` commandline tool. See [Test using MinIO Client `mc`](#test-using-minio-client-mc) for more information on using the `mc` commandline tool. For application developers, see <https://min.io/docs/minio/linux/developers/minio-drivers.html> to view MinIO SDKs for supported languages.
> [!NOTE]
> Standalone MinIO servers are best suited for early development and evaluation. Certain features such as versioning, object locking, and bucket replication require distributed deploying MinIO with Erasure Coding. For extended development and production, deploy MinIO with Erasure Coding enabled - specifically, with a *minimum* of 4 drives per MinIO server. See [MinIO Erasure Code Overview](https://min.io/docs/minio/linux/operations/concepts/erasure-coding.html#) for more complete documentation.
## Microsoft Windows
To run MinIO on 64-bit Windows hosts, download the MinIO executable from the following URL:
```sh
https://dl.min.io/server/minio/release/windows-amd64/minio.exe
```
Use the following command to run a standalone MinIO server on the Windows host. Replace ``D:\`` with the path to the drive or directory in which you want MinIO to store data. You must change the terminal or powershell directory to the location of the ``minio.exe`` executable, *or* add the path to that directory to the system ``$PATH``:
```sh
minio.exe server D:\
```
The MinIO deployment starts using default root credentials `minioadmin:minioadmin`. You can test the deployment using the MinIO Console, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to <http://127.0.0.1:9000> and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.
You can also connect using any S3-compatible tool, such as the MinIO Client `mc` commandline tool. See [Test using MinIO Client `mc`](#test-using-minio-client-mc) for more information on using the `mc` commandline tool. For application developers, see <https://min.io/docs/minio/linux/developers/minio-drivers.html> to view MinIO SDKs for supported languages.
> [!NOTE]
> Standalone MinIO servers are best suited for early development and evaluation. Certain features such as versioning, object locking, and bucket replication require distributed deploying MinIO with Erasure Coding. For extended development and production, deploy MinIO with Erasure Coding enabled - specifically, with a *minimum* of 4 drives per MinIO server. See [MinIO Erasure Code Overview](https://min.io/docs/minio/linux/operations/concepts/erasure-coding.html#) for more complete documentation.
## Install from Source
Use the following commands to compile and run a standalone MinIO server from source. Source installation is only intended for developers and advanced users. If you do not have a working Golang environment, please follow [How to install Golang](https://golang.org/doc/install). Minimum version required is [go1.24](https://golang.org/dl/#stable)
```sh
go install github.com/minio/minio@latest
```
The MinIO deployment starts using default root credentials `minioadmin:minioadmin`. You can test the deployment using the MinIO Console, an embedded web-based object browser built into MinIO Server. Point a web browser running on the host machine to <http://127.0.0.1:9000> and log in with the root credentials. You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.
You can also connect using any S3-compatible tool, such as the MinIO Client `mc` commandline tool. See [Test using MinIO Client `mc`](#test-using-minio-client-mc) for more information on using the `mc` commandline tool. For application developers, see <https://min.io/docs/minio/linux/developers/minio-drivers.html> to view MinIO SDKs for supported languages.
> [!NOTE]
> Standalone MinIO servers are best suited for early development and evaluation. Certain features such as versioning, object locking, and bucket replication require distributed deploying MinIO with Erasure Coding. For extended development and production, deploy MinIO with Erasure Coding enabled - specifically, with a *minimum* of 4 drives per MinIO server. See [MinIO Erasure Code Overview](https://min.io/docs/minio/linux/operations/concepts/erasure-coding.html) for more complete documentation.
MinIO strongly recommends *against* using compiled-from-source MinIO servers for production environments.
## Deployment Recommendations
### Allow port access for Firewalls
By default MinIO uses the port 9000 to listen for incoming connections. If your platform blocks the port by default, you may need to enable access to the port.
### ufw
For hosts with ufw enabled (Debian based distros), you can use `ufw` command to allow traffic to specific ports. Use below command to allow access to port 9000
```sh
ufw allow 9000
```
Below command enables all incoming traffic to ports ranging from 9000 to 9010.
```sh
ufw allow 9000:9010/tcp
```
### firewall-cmd
For hosts with firewall-cmd enabled (CentOS), you can use `firewall-cmd` command to allow traffic to specific ports. Use below commands to allow access to port 9000
```sh
firewall-cmd --get-active-zones
```
This command gets the active zone(s). Now, apply port rules to the relevant zones returned above. For example if the zone is `public`, use
```sh
firewall-cmd --zone=public --add-port=9000/tcp --permanent
```
> [!NOTE]
> `permanent` makes sure the rules are persistent across firewall start, restart or reload. Finally reload the firewall for changes to take effect.
```sh
firewall-cmd --reload
```
### iptables
For hosts with iptables enabled (RHEL, CentOS, etc), you can use `iptables` command to enable all traffic coming to specific ports. Use below command to allow
access to port 9000
```sh
iptables -A INPUT -p tcp --dport 9000 -j ACCEPT
service iptables restart
```
Below command enables all incoming traffic to ports ranging from 9000 to 9010.
```sh
iptables -A INPUT -p tcp --dport 9000:9010 -j ACCEPT
service iptables restart
```
## Test MinIO Connectivity
### Test using MinIO Console
MinIO Server comes with an embedded web based object browser. Point your web browser to <http://127.0.0.1:9000> to ensure your server has started successfully.
> [!NOTE]
> MinIO runs console on random port by default, if you wish to choose a specific port use `--console-address` to pick a specific interface and port.
### Things to consider
MinIO redirects browser access requests to the configured server port (i.e. `127.0.0.1:9000`) to the configured Console port. MinIO uses the hostname or IP address specified in the request when building the redirect URL. The URL and port *must* be accessible by the client for the redirection to work.
For deployments behind a load balancer, proxy, or ingress rule where the MinIO host IP address or port is not public, use the `MINIO_BROWSER_REDIRECT_URL` environment variable to specify the external hostname for the redirect. The LB/Proxy must have rules for directing traffic to the Console port specifically.
For example, consider a MinIO deployment behind a proxy `https://minio.example.net`, `https://console.minio.example.net` with rules for forwarding traffic on port :9000 and :9001 to MinIO and the MinIO Console respectively on the internal network. Set `MINIO_BROWSER_REDIRECT_URL` to `https://console.minio.example.net` to ensure the browser receives a valid reachable URL.
| Dashboard | Creating a bucket |
| ------------- | ------------- |
| ![Dashboard](https://github.com/minio/minio/blob/master/docs/screenshots/pic1.png?raw=true) | ![Dashboard](https://github.com/minio/minio/blob/master/docs/screenshots/pic2.png?raw=true) |
## Test using MinIO Client `mc`
`mc` provides a modern alternative to UNIX commands like ls, cat, cp, mirror, diff etc. It supports filesystems and Amazon S3 compatible cloud storage services. Follow the MinIO Client [Quickstart Guide](https://min.io/docs/minio/linux/reference/minio-mc.html#quickstart) for further instructions.
## Upgrading MinIO
Upgrades require zero downtime in MinIO, all upgrades are non-disruptive, all transactions on MinIO are atomic. So upgrading all the servers simultaneously is the recommended way to upgrade MinIO.
> [!NOTE]
> requires internet access to update directly from <https://dl.min.io>, optionally you can host any mirrors at <https://my-artifactory.example.com/minio/>
- For deployments that installed the MinIO server binary by hand, use [`mc admin update`](https://min.io/docs/minio/linux/reference/minio-mc-admin/mc-admin-update.html)
```sh
mc admin update <minio alias, e.g., myminio>
```
- For deployments without external internet access (e.g. airgapped environments), download the binary from <https://dl.min.io> and replace the existing MinIO binary let's say for example `/opt/bin/minio`, apply executable permissions `chmod +x /opt/bin/minio` and proceed to perform `mc admin service restart alias/`.
- For installations using Systemd MinIO service, upgrade via RPM/DEB packages **parallelly** on all servers or replace the binary lets say `/opt/bin/minio` on all nodes, apply executable permissions `chmod +x /opt/bin/minio` and process to perform `mc admin service restart alias/`.
### Upgrade Checklist
- Test all upgrades in a lower environment (DEV, QA, UAT) before applying to production. Performing blind upgrades in production environments carries significant risk.
- Read the release notes for MinIO *before* performing any upgrade, there is no forced requirement to upgrade to latest release upon every release. Some release may not be relevant to your setup, avoid upgrading production environments unnecessarily.
- If you plan to use `mc admin update`, MinIO process must have write access to the parent directory where the binary is present on the host system.
- `mc admin update` is not supported and should be avoided in kubernetes/container environments, please upgrade containers by upgrading relevant container images.
- **We do not recommend upgrading one MinIO server at a time, the product is designed to support parallel upgrades please follow our recommended guidelines.**
## Explore Further
- [MinIO Erasure Code Overview](https://min.io/docs/minio/linux/operations/concepts/erasure-coding.html)
- [Use `mc` with MinIO Server](https://min.io/docs/minio/linux/reference/minio-mc.html)
- [Use `minio-go` SDK with MinIO Server](https://min.io/docs/minio/linux/developers/go/minio-go.html)
- [The MinIO documentation website](https://min.io/docs/minio/linux/index.html)
## Contribute to MinIO Project
Please follow MinIO [Contributor's Guide](https://github.com/minio/minio/blob/master/CONTRIBUTING.md)
## License
- MinIO source is licensed under the [GNU AGPLv3](https://github.com/minio/minio/blob/master/LICENSE).
- MinIO [documentation](https://github.com/minio/minio/tree/master/docs) is licensed under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/).
- [License Compliance](https://github.com/minio/minio/blob/master/COMPLIANCE.md)
The MinIO Operator, Tenant CRDs, and operator lifecycle are **not maintained by this repository**. There is no Pigsty/Silo operator fork here. Users who require an operator must select and validate a separate operator implementation; issues and pull requests for operator code should be filed with that implementation.
+10 -32
View File
@@ -1,42 +1,20 @@
# Security Policy
This repository is the `pgsty/minio` community fork of `minio/minio`. Upstream MinIO security contacts do not handle fork-specific fixes or release notes for this repository.
## Supported Versions
We always provide security updates for the [latest release](https://github.com/minio/minio/releases/latest).
Whenever there is a security update you just need to upgrade to the latest version.
Security fixes are tracked on the active `master` branch and summarized in [docs/security/advisories.md](docs/security/advisories.md).
## Reporting a Vulnerability
All security bugs in [minio/minio](https://github,com/minio/minio) (or other minio/* repositories)
should be reported by email to security@min.io. Your email will be acknowledged within 48 hours,
and you'll receive a more detailed response to your email within 72 hours indicating the next steps
in handling your report.
For vulnerabilities in this fork:
Please, provide a detailed explanation of the issue. In particular, outline the type of the security
issue (DoS, authentication bypass, information disclose, ...) and the assumptions you're making (e.g. do
you need access credentials for a successful exploit).
1. Follow the fork-specific expectations in [VULNERABILITY_REPORT.md](VULNERABILITY_REPORT.md).
2. Prefer the `pgsty/minio` repository's GitHub security reporting workflow when it is available.
3. If private reporting is not available, contact the maintainers through the `pgsty/minio` repository before publishing detailed exploit information.
4. If you confirm the issue also affects upstream `minio/minio`, report it upstream separately.
If you have not received a reply to your email within 48 hours or you have not heard from the security team
for the past five days please contact the security team directly:
## Disclosure Process
- Primary security coordinator: aead@min.io
- Secondary coordinator: harsha@min.io
- If you receive no response: dev@min.io
### Disclosure Process
MinIO uses the following disclosure process:
1. Once the security report is received one member of the security team tries to verify and reproduce
the issue and determines the impact it has.
2. A member of the security team will respond and either confirm or reject the security report.
If the report is rejected the response explains why.
3. Code is audited to find any potential similar problems.
4. Fixes are prepared for the latest release.
5. On the date that the fixes are applied a security advisory will be published on <https://blog.min.io>.
Please inform us in your report email whether MinIO should mention your contribution w.r.t. fixing
the security issue. By default MinIO will **not** publish this information to protect your privacy.
This process can take some time, especially when coordination is required with maintainers of other projects.
Every effort will be made to handle the bug in as timely a manner as possible, however it's important that we
follow the process described above to ensure that disclosures are handled consistently.
Fork-specific fixes and user-visible upgrade notes are published in [docs/security/advisories.md](docs/security/advisories.md). The fork-specific triage and remediation process is described in [VULNERABILITY_REPORT.md](VULNERABILITY_REPORT.md).
+19 -20
View File
@@ -1,38 +1,37 @@
# Vulnerability Management Policy
This document formally describes the process of addressing and managing a
reported vulnerability that has been found in the MinIO server code base,
any directly connected ecosystem component or a direct / indirect dependency
of the code base.
This document describes how the `pgsty/minio` maintainers investigate,
assess, and remediate reported vulnerabilities affecting this fork, any
directly shipped component, or a direct / indirect dependency used by this
repository.
## Scope
The vulnerability management policy described in this document covers the
process of investigating, assessing and resolving a vulnerability report
opened by a MinIO employee or an external third party.
This policy covers vulnerability reports opened by repository maintainers or
external third parties against `pgsty/minio` itself, its release artifacts, or
dependencies that materially affect this fork.
Therefore, it lists pre-conditions and actions that should be performed to
resolve and fix a reported vulnerability.
It defines the information needed for triage and the expected remediation
workflow for supported fixes.
## Vulnerability Management Process
The vulnerability management process requires that the vulnerability report
contains the following information:
A useful vulnerability report should contain the following information:
- The project / component that contains the reported vulnerability.
- A description of the vulnerability. In particular, the type of the
reported vulnerability and how it might be exploited. Alternatively,
a well-established vulnerability identifier, e.g. CVE number, can be
used instead.
reported vulnerability and how it might be exploited. Alternatively,
a well-established vulnerability identifier, such as a CVE or GHSA ID, can
be used instead.
Based on the description mentioned above, a MinIO engineer or security team
member investigates:
Based on the report, the `pgsty/minio` maintainers investigate:
- Whether the reported vulnerability exists.
- The conditions that are required such that the vulnerability can be exploited.
- Which releases, branches, or deployment paths are affected.
- The steps required to fix the vulnerability.
In general, if the vulnerability exists in one of the MinIO code bases
itself - not in a code dependency - then MinIO will, if possible, fix
the vulnerability or implement reasonable countermeasures such that the
vulnerability cannot be exploited anymore.
If the vulnerability exists in this fork itself, the maintainers will, when
feasible, fix the issue or implement reasonable countermeasures such that the
vulnerability can no longer be exploited. Fork-specific upgrade notes and
security advisories are published in `docs/security/advisories.md`.
+7 -7
View File
@@ -193,27 +193,27 @@ func (a adminAPIHandlers) SetConfigKVHandler(w http.ResponseWriter, r *http.Requ
func setConfigKV(ctx context.Context, objectAPI ObjectLayer, kvBytes []byte) (result setConfigResult, err error) {
result.Cfg, err = readServerConfig(ctx, objectAPI, nil)
if err != nil {
return
return result, err
}
result.Dynamic, err = result.Cfg.ReadConfig(bytes.NewReader(kvBytes))
if err != nil {
return
return result, err
}
result.SubSys, _, _, err = config.GetSubSys(string(kvBytes))
if err != nil {
return
return result, err
}
tgts, err := config.ParseConfigTargetID(bytes.NewReader(kvBytes))
if err != nil {
return
return result, err
}
ctx = context.WithValue(ctx, config.ContextKeyForTargetFromConfig, tgts)
if verr := validateConfig(ctx, result.Cfg, result.SubSys); verr != nil {
err = badConfigErr{Err: verr}
return
return result, err
}
// Check if subnet proxy being set and if so set the same value to proxy of subnet
@@ -222,12 +222,12 @@ func setConfigKV(ctx context.Context, objectAPI ObjectLayer, kvBytes []byte) (re
// Update the actual server config on disk.
if err = saveServerConfig(ctx, objectAPI, result.Cfg); err != nil {
return
return result, err
}
// Write the config input KV to history.
err = saveServerConfigHistory(ctx, objectAPI, kvBytes)
return
return result, err
}
// GetConfigKVHandler - GET /minio/admin/v3/get-config-kv?key={key}
+9 -5
View File
@@ -267,7 +267,7 @@ func (a adminAPIHandlers) AddServiceAccountLDAP(w http.ResponseWriter, r *http.R
lookupResult, targetGroups, err = globalIAMSys.LDAPConfig.LookupUserDN(targetUser)
if err != nil {
// if not found, check if DN
if strings.Contains(err.Error(), "User DN not found for:") {
if strings.Contains(strings.ToLower(err.Error()), "user dn not found for:") {
if isDN {
// warn user that DNs are not allowed
writeErrorResponseJSON(ctx, w, errorCodes.ToAPIErrWithErr(ErrAdminLDAPExpectedLoginName, err), r.URL)
@@ -445,8 +445,10 @@ func (a adminAPIHandlers) ListAccessKeysLDAP(w http.ResponseWriter, r *http.Requ
for _, svc := range serviceAccounts {
expiryTime := svc.Expiration
serviceAccountList = append(serviceAccountList, madmin.ServiceAccountInfo{
AccessKey: svc.AccessKey,
Expiration: &expiryTime,
AccessKey: svc.AccessKey,
Expiration: &expiryTime,
Name: svc.Name,
Description: svc.Description,
})
}
for _, sts := range stsKeys {
@@ -625,8 +627,10 @@ func (a adminAPIHandlers) ListAccessKeysLDAPBulk(w http.ResponseWriter, r *http.
}
for _, svc := range serviceAccounts {
accessKeys.ServiceAccounts = append(accessKeys.ServiceAccounts, madmin.ServiceAccountInfo{
AccessKey: svc.AccessKey,
Expiration: &svc.Expiration,
AccessKey: svc.AccessKey,
Expiration: &svc.Expiration,
Name: svc.Name,
Description: svc.Description,
})
}
// if only service accounts, skip if user has no service accounts
+2
View File
@@ -173,6 +173,8 @@ func (a adminAPIHandlers) ListAccessKeysOpenIDBulk(w http.ResponseWriter, r *htt
if _, ok := accessKey.Claims[iamPolicyClaimNameOpenID()]; !ok {
continue // skip if no roleArn and no policy claim
}
// claim-based provider is in the roleArnMap under dummy ARN
arn = dummyRoleARN
}
matchingCfgName, ok := roleArnMap[arn]
if !ok {
+4 -4
View File
@@ -61,7 +61,7 @@ func (a adminAPIHandlers) StartDecommission(w http.ResponseWriter, r *http.Reque
return
}
if z.IsRebalanceStarted() {
if z.IsRebalanceStarted(ctx) {
writeErrorResponseJSON(ctx, w, errorCodes.ToAPIErr(ErrAdminRebalanceAlreadyStarted), r.URL)
return
}
@@ -277,7 +277,7 @@ func (a adminAPIHandlers) RebalanceStart(w http.ResponseWriter, r *http.Request)
return
}
if pools.IsRebalanceStarted() {
if pools.IsRebalanceStarted(ctx) {
writeErrorResponseJSON(ctx, w, errorCodes.ToAPIErr(ErrAdminRebalanceAlreadyStarted), r.URL)
return
}
@@ -380,7 +380,7 @@ func (a adminAPIHandlers) RebalanceStop(w http.ResponseWriter, r *http.Request)
func proxyDecommissionRequest(ctx context.Context, defaultEndPoint Endpoint, w http.ResponseWriter, r *http.Request) (proxy bool) {
host := env.Get("_MINIO_DECOM_ENDPOINT_HOST", defaultEndPoint.Host)
if host == "" {
return
return proxy
}
for nodeIdx, proxyEp := range globalProxyEndpoints {
if proxyEp.Host == host && !proxyEp.IsLocal {
@@ -389,5 +389,5 @@ func proxyDecommissionRequest(ctx context.Context, defaultEndPoint Endpoint, w h
}
}
}
return
return proxy
}
+4 -4
View File
@@ -70,7 +70,7 @@ func (a adminAPIHandlers) SiteReplicationAdd(w http.ResponseWriter, r *http.Requ
func getSRAddOptions(r *http.Request) (opts madmin.SRAddOptions) {
opts.ReplicateILMExpiry = r.Form.Get("replicateILMExpiry") == "true"
return
return opts
}
// SRPeerJoin - PUT /minio/admin/v3/site-replication/join
@@ -304,7 +304,7 @@ func (a adminAPIHandlers) SRPeerGetIDPSettings(w http.ResponseWriter, r *http.Re
}
}
func parseJSONBody(ctx context.Context, body io.Reader, v interface{}, encryptionKey string) error {
func parseJSONBody(ctx context.Context, body io.Reader, v any, encryptionKey string) error {
data, err := io.ReadAll(body)
if err != nil {
return SRError{
@@ -422,7 +422,7 @@ func (a adminAPIHandlers) SiteReplicationEdit(w http.ResponseWriter, r *http.Req
func getSREditOptions(r *http.Request) (opts madmin.SREditOptions) {
opts.DisableILMExpiryReplication = r.Form.Get("disableILMExpiryReplication") == "true"
opts.EnableILMExpiryReplication = r.Form.Get("enableILMExpiryReplication") == "true"
return
return opts
}
// SRPeerEdit - PUT /minio/admin/v3/site-replication/peer/edit
@@ -484,7 +484,7 @@ func getSRStatusOptions(r *http.Request) (opts madmin.SRStatusOptions) {
opts.EntityValue = q.Get("entityvalue")
opts.ShowDeleted = q.Get("showDeleted") == "true"
opts.Metrics = q.Get("metrics") == "true"
return
return opts
}
// SiteReplicationRemove - PUT /minio/admin/v3/site-replication/remove
+4 -4
View File
@@ -89,7 +89,7 @@ func (s *TestSuiteIAM) TestDeleteUserRace(c *check) {
// Create a policy policy
policy := "mypolicy"
policyBytes := []byte(fmt.Sprintf(`{
policyBytes := fmt.Appendf(nil, `{
"Version": "2012-10-17",
"Statement": [
{
@@ -104,7 +104,7 @@ func (s *TestSuiteIAM) TestDeleteUserRace(c *check) {
]
}
]
}`, bucket))
}`, bucket)
err = s.adm.AddCannedPolicy(ctx, policy, policyBytes)
if err != nil {
c.Fatalf("policy add error: %v", err)
@@ -113,7 +113,7 @@ func (s *TestSuiteIAM) TestDeleteUserRace(c *check) {
userCount := 50
accessKeys := make([]string, userCount)
secretKeys := make([]string, userCount)
for i := 0; i < userCount; i++ {
for i := range userCount {
accessKey, secretKey := mustGenerateCredentials(c)
err = s.adm.SetUser(ctx, accessKey, secretKey, madmin.AccountEnabled)
if err != nil {
@@ -133,7 +133,7 @@ func (s *TestSuiteIAM) TestDeleteUserRace(c *check) {
}
g := errgroup.Group{}
for i := 0; i < userCount; i++ {
for i := range userCount {
g.Go(func(i int) func() error {
return func() error {
uClient := s.getUserClient(c, accessKeys[i], secretKeys[i], "")
+3 -4
View File
@@ -24,6 +24,7 @@ import (
"errors"
"fmt"
"io"
"maps"
"net/http"
"os"
"slices"
@@ -157,9 +158,7 @@ func (a adminAPIHandlers) ListUsers(w http.ResponseWriter, r *http.Request) {
writeErrorResponseJSON(ctx, w, toAdminAPIErr(ctx, err), r.URL)
return
}
for k, v := range ldapUsers {
allCredentials[k] = v
}
maps.Copy(allCredentials, ldapUsers)
// Marshal the response
data, err := json.Marshal(allCredentials)
@@ -2949,7 +2948,7 @@ func commonAddServiceAccount(r *http.Request, ldap bool) (context.Context, auth.
name: createReq.Name,
description: description,
expiration: createReq.Expiration,
claims: make(map[string]interface{}),
claims: make(map[string]any),
}
condValues := getConditionValues(r, "", cred)
+125 -21
View File
@@ -208,6 +208,8 @@ func TestIAMInternalIDPServerSuite(t *testing.T) {
suite.TestGroupAddRemove(c)
suite.TestServiceAccountOpsByAdmin(c)
suite.TestServiceAccountPrivilegeEscalationBug(c)
suite.TestServiceAccountPrivilegeEscalationBug2_2025_10_15(c, true)
suite.TestServiceAccountPrivilegeEscalationBug2_2025_10_15(c, false)
suite.TestServiceAccountOpsByUser(c)
suite.TestServiceAccountDurationSecondsCondition(c)
suite.TestAddServiceAccountPerms(c)
@@ -332,7 +334,7 @@ func (s *TestSuiteIAM) TestUserPolicyEscalationBug(c *check) {
// 2.2 create and associate policy to user
policy := "mypolicy-test-user-update"
policyBytes := []byte(fmt.Sprintf(`{
policyBytes := fmt.Appendf(nil, `{
"Version": "2012-10-17",
"Statement": [
{
@@ -355,7 +357,7 @@ func (s *TestSuiteIAM) TestUserPolicyEscalationBug(c *check) {
]
}
]
}`, bucket, bucket))
}`, bucket, bucket)
err = s.adm.AddCannedPolicy(ctx, policy, policyBytes)
if err != nil {
c.Fatalf("policy add error: %v", err)
@@ -562,7 +564,7 @@ func (s *TestSuiteIAM) TestPolicyCreate(c *check) {
// 1. Create a policy
policy := "mypolicy"
policyBytes := []byte(fmt.Sprintf(`{
policyBytes := fmt.Appendf(nil, `{
"Version": "2012-10-17",
"Statement": [
{
@@ -585,7 +587,7 @@ func (s *TestSuiteIAM) TestPolicyCreate(c *check) {
]
}
]
}`, bucket, bucket))
}`, bucket, bucket)
err = s.adm.AddCannedPolicy(ctx, policy, policyBytes)
if err != nil {
c.Fatalf("policy add error: %v", err)
@@ -680,7 +682,7 @@ func (s *TestSuiteIAM) TestCannedPolicies(c *check) {
c.Fatalf("bucket creat error: %v", err)
}
policyBytes := []byte(fmt.Sprintf(`{
policyBytes := fmt.Appendf(nil, `{
"Version": "2012-10-17",
"Statement": [
{
@@ -703,7 +705,7 @@ func (s *TestSuiteIAM) TestCannedPolicies(c *check) {
]
}
]
}`, bucket, bucket))
}`, bucket, bucket)
// Check that default policies can be overwritten.
err = s.adm.AddCannedPolicy(ctx, "readwrite", policyBytes)
@@ -739,7 +741,7 @@ func (s *TestSuiteIAM) TestGroupAddRemove(c *check) {
}
policy := "mypolicy"
policyBytes := []byte(fmt.Sprintf(`{
policyBytes := fmt.Appendf(nil, `{
"Version": "2012-10-17",
"Statement": [
{
@@ -762,7 +764,7 @@ func (s *TestSuiteIAM) TestGroupAddRemove(c *check) {
]
}
]
}`, bucket, bucket))
}`, bucket, bucket)
err = s.adm.AddCannedPolicy(ctx, policy, policyBytes)
if err != nil {
c.Fatalf("policy add error: %v", err)
@@ -911,7 +913,7 @@ func (s *TestSuiteIAM) TestServiceAccountOpsByUser(c *check) {
// Create policy, user and associate policy
policy := "mypolicy"
policyBytes := []byte(fmt.Sprintf(`{
policyBytes := fmt.Appendf(nil, `{
"Version": "2012-10-17",
"Statement": [
{
@@ -934,7 +936,7 @@ func (s *TestSuiteIAM) TestServiceAccountOpsByUser(c *check) {
]
}
]
}`, bucket, bucket))
}`, bucket, bucket)
err = s.adm.AddCannedPolicy(ctx, policy, policyBytes)
if err != nil {
c.Fatalf("policy add error: %v", err)
@@ -995,7 +997,7 @@ func (s *TestSuiteIAM) TestServiceAccountDurationSecondsCondition(c *check) {
// Create policy, user and associate policy
policy := "mypolicy"
policyBytes := []byte(fmt.Sprintf(`{
policyBytes := fmt.Appendf(nil, `{
"Version": "2012-10-17",
"Statement": [
{
@@ -1026,7 +1028,7 @@ func (s *TestSuiteIAM) TestServiceAccountDurationSecondsCondition(c *check) {
]
}
]
}`, bucket, bucket))
}`, bucket, bucket)
err = s.adm.AddCannedPolicy(ctx, policy, policyBytes)
if err != nil {
c.Fatalf("policy add error: %v", err)
@@ -1093,7 +1095,7 @@ func (s *TestSuiteIAM) TestServiceAccountOpsByAdmin(c *check) {
// Create policy, user and associate policy
policy := "mypolicy"
policyBytes := []byte(fmt.Sprintf(`{
policyBytes := fmt.Appendf(nil, `{
"Version": "2012-10-17",
"Statement": [
{
@@ -1116,7 +1118,7 @@ func (s *TestSuiteIAM) TestServiceAccountOpsByAdmin(c *check) {
]
}
]
}`, bucket, bucket))
}`, bucket, bucket)
err = s.adm.AddCannedPolicy(ctx, policy, policyBytes)
if err != nil {
c.Fatalf("policy add error: %v", err)
@@ -1249,6 +1251,108 @@ func (s *TestSuiteIAM) TestServiceAccountPrivilegeEscalationBug(c *check) {
}
}
func (s *TestSuiteIAM) TestServiceAccountPrivilegeEscalationBug2_2025_10_15(c *check, forRoot bool) {
ctx, cancel := context.WithTimeout(context.Background(), testDefaultTimeout)
defer cancel()
for i := range 3 {
err := s.client.MakeBucket(ctx, fmt.Sprintf("bucket%d", i+1), minio.MakeBucketOptions{})
if err != nil {
c.Fatalf("bucket create error: %v", err)
}
defer func(i int) {
_ = s.client.RemoveBucket(ctx, fmt.Sprintf("bucket%d", i+1))
}(i)
}
allow2BucketsPolicyBytes := []byte(`{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ListBucket1AndBucket2",
"Effect": "Allow",
"Action": ["s3:ListBucket"],
"Resource": ["arn:aws:s3:::bucket1", "arn:aws:s3:::bucket2"]
},
{
"Sid": "ReadWriteBucket1AndBucket2Objects",
"Effect": "Allow",
"Action": [
"s3:DeleteObject",
"s3:DeleteObjectVersion",
"s3:GetObject",
"s3:GetObjectVersion",
"s3:PutObject"
],
"Resource": ["arn:aws:s3:::bucket1/*", "arn:aws:s3:::bucket2/*"]
}
]
}`)
if forRoot {
// Create a service account for the root user.
_, err := s.adm.AddServiceAccount(ctx, madmin.AddServiceAccountReq{
Policy: allow2BucketsPolicyBytes,
AccessKey: "restricted",
SecretKey: "restricted123",
})
if err != nil {
c.Fatalf("could not create service account")
}
defer func() {
_ = s.adm.DeleteServiceAccount(ctx, "restricted")
}()
} else {
// Create a regular user and attach consoleAdmin policy
err := s.adm.AddUser(ctx, "foobar", "foobar123")
if err != nil {
c.Fatalf("could not create user")
}
_, err = s.adm.AttachPolicy(ctx, madmin.PolicyAssociationReq{
Policies: []string{"consoleAdmin"},
User: "foobar",
})
if err != nil {
c.Fatalf("could not attach policy")
}
// Create a service account for the regular user.
_, err = s.adm.AddServiceAccount(ctx, madmin.AddServiceAccountReq{
Policy: allow2BucketsPolicyBytes,
TargetUser: "foobar",
AccessKey: "restricted",
SecretKey: "restricted123",
})
if err != nil {
c.Fatalf("could not create service account: %v", err)
}
defer func() {
_ = s.adm.DeleteServiceAccount(ctx, "restricted")
_ = s.adm.RemoveUser(ctx, "foobar")
}()
}
restrictedClient := s.getUserClient(c, "restricted", "restricted123", "")
buckets, err := restrictedClient.ListBuckets(ctx)
if err != nil {
c.Fatalf("err fetching buckets %s", err)
}
if len(buckets) != 2 || buckets[0].Name != "bucket1" || buckets[1].Name != "bucket2" {
c.Fatalf("restricted service account should only have access to bucket1 and bucket2")
}
// Try to escalate privileges
restrictedAdmClient := s.getAdminClient(c, "restricted", "restricted123", "")
_, err = restrictedAdmClient.AddServiceAccount(ctx, madmin.AddServiceAccountReq{
AccessKey: "newroot",
SecretKey: "newroot123",
})
if err == nil {
c.Fatalf("restricted service account was able to create service account bypassing sub-policy!")
}
}
func (s *TestSuiteIAM) SetUpAccMgmtPlugin(c *check) {
ctx, cancel := context.WithTimeout(context.Background(), testDefaultTimeout)
defer cancel()
@@ -1367,7 +1471,7 @@ func (s *TestSuiteIAM) TestAccMgmtPlugin(c *check) {
svcAK, svcSK := mustGenerateCredentials(c)
// This policy does not allow listing objects.
policyBytes := []byte(fmt.Sprintf(`{
policyBytes := fmt.Appendf(nil, `{
"Version": "2012-10-17",
"Statement": [
{
@@ -1381,7 +1485,7 @@ func (s *TestSuiteIAM) TestAccMgmtPlugin(c *check) {
]
}
]
}`, bucket))
}`, bucket)
cr, err := userAdmClient.AddServiceAccount(ctx, madmin.AddServiceAccountReq{
Policy: policyBytes,
TargetUser: accessKey,
@@ -1558,7 +1662,7 @@ func (c *check) mustDownload(ctx context.Context, client *minio.Client, bucket s
func (c *check) mustUploadReturnVersions(ctx context.Context, client *minio.Client, bucket string) []string {
c.Helper()
versions := []string{}
for i := 0; i < 5; i++ {
for range 5 {
ui, err := client.PutObject(ctx, bucket, "some-object", bytes.NewBuffer([]byte("stuff")), 5, minio.PutObjectOptions{})
if err != nil {
c.Fatalf("upload did not succeed got %#v", err)
@@ -1627,7 +1731,7 @@ func (c *check) assertSvcAccSessionPolicyUpdate(ctx context.Context, s *TestSuit
svcAK, svcSK := mustGenerateCredentials(c)
// This policy does not allow listing objects.
policyBytes := []byte(fmt.Sprintf(`{
policyBytes := fmt.Appendf(nil, `{
"Version": "2012-10-17",
"Statement": [
{
@@ -1641,7 +1745,7 @@ func (c *check) assertSvcAccSessionPolicyUpdate(ctx context.Context, s *TestSuit
]
}
]
}`, bucket))
}`, bucket)
cr, err := madmClient.AddServiceAccount(ctx, madmin.AddServiceAccountReq{
Policy: policyBytes,
TargetUser: accessKey,
@@ -1655,7 +1759,7 @@ func (c *check) assertSvcAccSessionPolicyUpdate(ctx context.Context, s *TestSuit
c.mustNotListObjects(ctx, svcClient, bucket)
// This policy allows listing objects.
newPolicyBytes := []byte(fmt.Sprintf(`{
newPolicyBytes := fmt.Appendf(nil, `{
"Version": "2012-10-17",
"Statement": [
{
@@ -1668,7 +1772,7 @@ func (c *check) assertSvcAccSessionPolicyUpdate(ctx context.Context, s *TestSuit
]
}
]
}`, bucket))
}`, bucket)
err = madmClient.UpdateServiceAccount(ctx, svcAK, madmin.UpdateServiceAccountReq{
NewPolicy: newPolicyBytes,
})
+10 -10
View File
@@ -954,7 +954,7 @@ func (a adminAPIHandlers) ForceUnlockHandler(w http.ResponseWriter, r *http.Requ
var args dsync.LockArgs
var lockers []dsync.NetLocker
for _, path := range strings.Split(vars["paths"], ",") {
for path := range strings.SplitSeq(vars["paths"], ",") {
if path == "" {
continue
}
@@ -1193,7 +1193,7 @@ type dummyFileInfo struct {
mode os.FileMode
modTime time.Time
isDir bool
sys interface{}
sys any
}
func (f dummyFileInfo) Name() string { return f.name }
@@ -1201,7 +1201,7 @@ func (f dummyFileInfo) Size() int64 { return f.size }
func (f dummyFileInfo) Mode() os.FileMode { return f.mode }
func (f dummyFileInfo) ModTime() time.Time { return f.modTime }
func (f dummyFileInfo) IsDir() bool { return f.isDir }
func (f dummyFileInfo) Sys() interface{} { return f.sys }
func (f dummyFileInfo) Sys() any { return f.sys }
// DownloadProfilingHandler - POST /minio/admin/v3/profiling/download
// ----------
@@ -1243,17 +1243,17 @@ func extractHealInitParams(vars map[string]string, qParams url.Values, r io.Read
if hip.objPrefix != "" {
// Bucket is required if object-prefix is given
err = ErrHealMissingBucket
return
return hip, err
}
} else if isReservedOrInvalidBucket(hip.bucket, false) {
err = ErrInvalidBucketName
return
return hip, err
}
// empty prefix is valid.
if !IsValidObjectPrefix(hip.objPrefix) {
err = ErrInvalidObjectName
return
return hip, err
}
if len(qParams[mgmtClientToken]) > 0 {
@@ -1275,7 +1275,7 @@ func extractHealInitParams(vars map[string]string, qParams url.Values, r io.Read
if (hip.forceStart && hip.forceStop) ||
(hip.clientToken != "" && (hip.forceStart || hip.forceStop)) {
err = ErrInvalidRequest
return
return hip, err
}
// ignore body if clientToken is provided
@@ -1284,12 +1284,12 @@ func extractHealInitParams(vars map[string]string, qParams url.Values, r io.Read
if jerr != nil {
adminLogIf(GlobalContext, jerr, logger.ErrorKind)
err = ErrRequestBodyParse
return
return hip, err
}
}
err = ErrNone
return
return hip, err
}
// HealHandler - POST /minio/admin/v3/heal/
@@ -2022,7 +2022,7 @@ func extractTraceOptions(r *http.Request) (opts madmin.ServiceTraceOpts, err err
opts.OS = true
// Older mc - cannot deal with more types...
}
return
return opts, err
}
// TraceHandler - POST /minio/admin/v3/trace
+3 -3
View File
@@ -402,7 +402,7 @@ func (b byResourceUID) Less(i, j int) bool {
func TestTopLockEntries(t *testing.T) {
locksHeld := make(map[string][]lockRequesterInfo)
var owners []string
for i := 0; i < 4; i++ {
for i := range 4 {
owners = append(owners, fmt.Sprintf("node-%d", i))
}
@@ -410,7 +410,7 @@ func TestTopLockEntries(t *testing.T) {
// request UID, but 10 different resource names associated with it.
var lris []lockRequesterInfo
uuid := mustGetUUID()
for i := 0; i < 10; i++ {
for i := range 10 {
resource := fmt.Sprintf("bucket/delete-object-%d", i)
lri := lockRequesterInfo{
Name: resource,
@@ -425,7 +425,7 @@ func TestTopLockEntries(t *testing.T) {
}
// Add a few concurrent read locks to the mix
for i := 0; i < 50; i++ {
for i := range 50 {
resource := fmt.Sprintf("bucket/get-object-%d", i)
lri := lockRequesterInfo{
Name: resource,
+4 -9
View File
@@ -22,6 +22,7 @@ import (
"encoding/json"
"errors"
"fmt"
"maps"
"net/http"
"sort"
"sync"
@@ -520,9 +521,7 @@ func (h *healSequence) getScannedItemsMap() map[madmin.HealItemType]int64 {
// Make a copy before returning the value
retMap := make(map[madmin.HealItemType]int64, len(h.scannedItemsMap))
for k, v := range h.scannedItemsMap {
retMap[k] = v
}
maps.Copy(retMap, h.scannedItemsMap)
return retMap
}
@@ -534,9 +533,7 @@ func (h *healSequence) getHealedItemsMap() map[madmin.HealItemType]int64 {
// Make a copy before returning the value
retMap := make(map[madmin.HealItemType]int64, len(h.healedItemsMap))
for k, v := range h.healedItemsMap {
retMap[k] = v
}
maps.Copy(retMap, h.healedItemsMap)
return retMap
}
@@ -549,9 +546,7 @@ func (h *healSequence) getHealFailedItemsMap() map[madmin.HealItemType]int64 {
// Make a copy before returning the value
retMap := make(map[madmin.HealItemType]int64, len(h.healFailedItemsMap))
for k, v := range h.healFailedItemsMap {
retMap[k] = v
}
maps.Copy(retMap, h.healFailedItemsMap)
return retMap
}
+1 -1
View File
@@ -296,7 +296,7 @@ func registerAdminRouter(router *mux.Router, enableConfigOps bool) {
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/import-iam").HandlerFunc(adminMiddleware(adminAPI.ImportIAM, noGZFlag))
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/import-iam-v2").HandlerFunc(adminMiddleware(adminAPI.ImportIAMV2, noGZFlag))
// IDentity Provider configuration APIs
// Identity Provider configuration APIs
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/idp-config/{type}/{name}").HandlerFunc(adminMiddleware(adminAPI.AddIdentityProviderCfg))
adminRouter.Methods(http.MethodPost).Path(adminVersion + "/idp-config/{type}/{name}").HandlerFunc(adminMiddleware(adminAPI.UpdateIdentityProviderCfg))
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/idp-config/{type}").HandlerFunc(adminMiddleware(adminAPI.ListIdentityProviderCfg))
+3 -3
View File
@@ -65,7 +65,7 @@ func setCommonHeaders(w http.ResponseWriter) {
}
// Encodes the response headers into XML format.
func encodeResponse(response interface{}) []byte {
func encodeResponse(response any) []byte {
var buf bytes.Buffer
buf.WriteString(xml.Header)
if err := xml.NewEncoder(&buf).Encode(response); err != nil {
@@ -83,7 +83,7 @@ func encodeResponse(response interface{}) []byte {
// Do not use this function for anything other than ListObjects()
// variants, please open a github discussion if you wish to use
// this in other places.
func encodeResponseList(response interface{}) []byte {
func encodeResponseList(response any) []byte {
var buf bytes.Buffer
buf.WriteString(xxml.Header)
if err := xxml.NewEncoder(&buf).Encode(response); err != nil {
@@ -94,7 +94,7 @@ func encodeResponseList(response interface{}) []byte {
}
// Encodes the response headers into JSON format.
func encodeResponseJSON(response interface{}) []byte {
func encodeResponseJSON(response any) []byte {
var bytesBuffer bytes.Buffer
e := json.NewEncoder(&bytesBuffer)
e.Encode(response)
+13 -13
View File
@@ -31,7 +31,7 @@ func getListObjectsV1Args(values url.Values) (prefix, marker, delimiter string,
var err error
if maxkeys, err = strconv.Atoi(values.Get("max-keys")); err != nil {
errCode = ErrInvalidMaxKeys
return
return prefix, marker, delimiter, maxkeys, encodingType, errCode
}
} else {
maxkeys = maxObjectList
@@ -41,7 +41,7 @@ func getListObjectsV1Args(values url.Values) (prefix, marker, delimiter string,
marker = values.Get("marker")
delimiter = values.Get("delimiter")
encodingType = values.Get("encoding-type")
return
return prefix, marker, delimiter, maxkeys, encodingType, errCode
}
func getListBucketObjectVersionsArgs(values url.Values) (prefix, marker, delimiter string, maxkeys int, encodingType, versionIDMarker string, errCode APIErrorCode) {
@@ -51,7 +51,7 @@ func getListBucketObjectVersionsArgs(values url.Values) (prefix, marker, delimit
var err error
if maxkeys, err = strconv.Atoi(values.Get("max-keys")); err != nil {
errCode = ErrInvalidMaxKeys
return
return prefix, marker, delimiter, maxkeys, encodingType, versionIDMarker, errCode
}
} else {
maxkeys = maxObjectList
@@ -62,7 +62,7 @@ func getListBucketObjectVersionsArgs(values url.Values) (prefix, marker, delimit
delimiter = values.Get("delimiter")
encodingType = values.Get("encoding-type")
versionIDMarker = values.Get("version-id-marker")
return
return prefix, marker, delimiter, maxkeys, encodingType, versionIDMarker, errCode
}
// Parse bucket url queries for ListObjects V2.
@@ -73,7 +73,7 @@ func getListObjectsV2Args(values url.Values) (prefix, token, startAfter, delimit
if val, ok := values["continuation-token"]; ok {
if len(val[0]) == 0 {
errCode = ErrIncorrectContinuationToken
return
return prefix, token, startAfter, delimiter, fetchOwner, maxkeys, encodingType, errCode
}
}
@@ -81,7 +81,7 @@ func getListObjectsV2Args(values url.Values) (prefix, token, startAfter, delimit
var err error
if maxkeys, err = strconv.Atoi(values.Get("max-keys")); err != nil {
errCode = ErrInvalidMaxKeys
return
return prefix, token, startAfter, delimiter, fetchOwner, maxkeys, encodingType, errCode
}
} else {
maxkeys = maxObjectList
@@ -97,11 +97,11 @@ func getListObjectsV2Args(values url.Values) (prefix, token, startAfter, delimit
decodedToken, err := base64.StdEncoding.DecodeString(token)
if err != nil {
errCode = ErrIncorrectContinuationToken
return
return prefix, token, startAfter, delimiter, fetchOwner, maxkeys, encodingType, errCode
}
token = string(decodedToken)
}
return
return prefix, token, startAfter, delimiter, fetchOwner, maxkeys, encodingType, errCode
}
// Parse bucket url queries for ?uploads
@@ -112,7 +112,7 @@ func getBucketMultipartResources(values url.Values) (prefix, keyMarker, uploadID
var err error
if maxUploads, err = strconv.Atoi(values.Get("max-uploads")); err != nil {
errCode = ErrInvalidMaxUploads
return
return prefix, keyMarker, uploadIDMarker, delimiter, maxUploads, encodingType, errCode
}
} else {
maxUploads = maxUploadsList
@@ -123,7 +123,7 @@ func getBucketMultipartResources(values url.Values) (prefix, keyMarker, uploadID
uploadIDMarker = values.Get("upload-id-marker")
delimiter = values.Get("delimiter")
encodingType = values.Get("encoding-type")
return
return prefix, keyMarker, uploadIDMarker, delimiter, maxUploads, encodingType, errCode
}
// Parse object url queries
@@ -134,7 +134,7 @@ func getObjectResources(values url.Values) (uploadID string, partNumberMarker, m
if values.Get("max-parts") != "" {
if maxParts, err = strconv.Atoi(values.Get("max-parts")); err != nil {
errCode = ErrInvalidMaxParts
return
return uploadID, partNumberMarker, maxParts, encodingType, errCode
}
} else {
maxParts = maxPartsList
@@ -143,11 +143,11 @@ func getObjectResources(values url.Values) (uploadID string, partNumberMarker, m
if values.Get("part-number-marker") != "" {
if partNumberMarker, err = strconv.Atoi(values.Get("part-number-marker")); err != nil {
errCode = ErrInvalidPartNumberMarker
return
return uploadID, partNumberMarker, maxParts, encodingType, errCode
}
}
uploadID = values.Get("uploadId")
encodingType = values.Get("encoding-type")
return
return uploadID, partNumberMarker, maxParts, encodingType, errCode
}
+48
View File
@@ -889,6 +889,12 @@ func generateMultiDeleteResponse(quiet bool, deletedObjects []DeletedObject, err
}
func writeResponse(w http.ResponseWriter, statusCode int, response []byte, mType mimeType) {
// Don't write a response if one has already been written.
// Fixes https://github.com/minio/minio/issues/21633
if headersAlreadyWritten(w) {
return
}
if statusCode == 0 {
statusCode = 200
}
@@ -1015,3 +1021,45 @@ func writeCustomErrorResponseJSON(ctx context.Context, w http.ResponseWriter, er
encodedErrorResponse := encodeResponseJSON(errorResponse)
writeResponse(w, err.HTTPStatusCode, encodedErrorResponse, mimeJSON)
}
type unwrapper interface {
Unwrap() http.ResponseWriter
}
// headersAlreadyWritten returns true if the headers have already been written
// to this response writer. It will unwrap the ResponseWriter if possible to try
// and find a trackingResponseWriter.
func headersAlreadyWritten(w http.ResponseWriter) bool {
for {
if trw, ok := w.(*trackingResponseWriter); ok {
return trw.headerWritten
} else if uw, ok := w.(unwrapper); ok {
w = uw.Unwrap()
} else {
return false
}
}
}
// trackingResponseWriter wraps a ResponseWriter and notes when WriterHeader has
// been called. This allows high level request handlers to check if something
// has already sent the header.
type trackingResponseWriter struct {
http.ResponseWriter
headerWritten bool
}
func (w *trackingResponseWriter) WriteHeader(statusCode int) {
if !w.headerWritten {
w.headerWritten = true
w.ResponseWriter.WriteHeader(statusCode)
}
}
func (w *trackingResponseWriter) Write(b []byte) (int, error) {
return w.ResponseWriter.Write(b)
}
func (w *trackingResponseWriter) Unwrap() http.ResponseWriter {
return w.ResponseWriter
}
+90 -1
View File
@@ -18,8 +18,12 @@
package cmd
import (
"io"
"net/http"
"net/http/httptest"
"testing"
"github.com/klauspost/compress/gzhttp"
)
// Tests object location.
@@ -100,7 +104,6 @@ func TestObjectLocation(t *testing.T) {
},
}
for _, testCase := range testCases {
testCase := testCase
t.Run("", func(t *testing.T) {
gotLocation := getObjectLocation(testCase.request, testCase.domains, testCase.bucket, testCase.object)
if testCase.expectedLocation != gotLocation {
@@ -123,3 +126,89 @@ func TestGetURLScheme(t *testing.T) {
t.Errorf("Expected %s, got %s", httpsScheme, gotScheme)
}
}
func TestTrackingResponseWriter(t *testing.T) {
rw := httptest.NewRecorder()
trw := &trackingResponseWriter{ResponseWriter: rw}
trw.WriteHeader(299)
if !trw.headerWritten {
t.Fatal("headerWritten was not set by WriteHeader call")
}
_, err := trw.Write([]byte("hello"))
if err != nil {
t.Fatalf("Write unexpectedly failed: %v", err)
}
// Check that WriteHeader and Write were called on the underlying response writer
resp := rw.Result()
if resp.StatusCode != 299 {
t.Fatalf("unexpected status: %v", resp.StatusCode)
}
body, err := io.ReadAll(resp.Body)
if err != nil {
t.Fatalf("reading response body failed: %v", err)
}
if string(body) != "hello" {
t.Fatalf("response body incorrect: %v", string(body))
}
// Check that Unwrap works
if trw.Unwrap() != rw {
t.Fatalf("Unwrap returned wrong result: %v", trw.Unwrap())
}
}
func TestHeadersAlreadyWritten(t *testing.T) {
rw := httptest.NewRecorder()
trw := &trackingResponseWriter{ResponseWriter: rw}
if headersAlreadyWritten(trw) {
t.Fatal("headers have not been written yet")
}
trw.WriteHeader(123)
if !headersAlreadyWritten(trw) {
t.Fatal("headers were written")
}
}
func TestHeadersAlreadyWrittenWrapped(t *testing.T) {
rw := httptest.NewRecorder()
trw := &trackingResponseWriter{ResponseWriter: rw}
wrap1 := &gzhttp.NoGzipResponseWriter{ResponseWriter: trw}
wrap2 := &gzhttp.NoGzipResponseWriter{ResponseWriter: wrap1}
if headersAlreadyWritten(wrap2) {
t.Fatal("headers have not been written yet")
}
wrap2.WriteHeader(123)
if !headersAlreadyWritten(wrap2) {
t.Fatal("headers were written")
}
}
func TestWriteResponseHeadersNotWritten(t *testing.T) {
rw := httptest.NewRecorder()
trw := &trackingResponseWriter{ResponseWriter: rw}
writeResponse(trw, 299, []byte("hello"), "application/foo")
resp := rw.Result()
if resp.StatusCode != 299 {
t.Fatal("response wasn't written")
}
}
func TestWriteResponseHeadersWritten(t *testing.T) {
rw := httptest.NewRecorder()
rw.Code = -1
trw := &trackingResponseWriter{ResponseWriter: rw, headerWritten: true}
writeResponse(trw, 200, []byte("hello"), "application/foo")
if rw.Code != -1 {
t.Fatalf("response was written when it shouldn't have been (Code=%v)", rw.Code)
}
}
+2
View File
@@ -218,6 +218,8 @@ func s3APIMiddleware(f http.HandlerFunc, flags ...s3HFlag) http.HandlerFunc {
handlerName := getHandlerName(f, "objectAPIHandlers")
var handler http.HandlerFunc = func(w http.ResponseWriter, r *http.Request) {
w = &trackingResponseWriter{ResponseWriter: w}
// Wrap the actual handler with the appropriate tracing middleware.
var tracedHandler http.HandlerFunc
if handlerFlags.has(traceHdrsS3HFlag) {
+3 -3
View File
@@ -216,7 +216,7 @@ func getSessionToken(r *http.Request) (token string) {
// Fetch claims in the security token returned by the client, doesn't return
// errors - upon errors the returned claims map will be empty.
func mustGetClaimsFromToken(r *http.Request) map[string]interface{} {
func mustGetClaimsFromToken(r *http.Request) map[string]any {
claims, _ := getClaimsFromToken(getSessionToken(r))
return claims
}
@@ -266,7 +266,7 @@ func getClaimsFromTokenWithSecret(token, secret string) (*xjwt.MapClaims, error)
}
// Fetch claims in the security token returned by the client.
func getClaimsFromToken(token string) (map[string]interface{}, error) {
func getClaimsFromToken(token string) (map[string]any, error) {
jwtClaims, err := getClaimsFromTokenWithSecret(token, globalActiveCred.SecretKey)
if err != nil {
return nil, err
@@ -275,7 +275,7 @@ func getClaimsFromToken(token string) (map[string]interface{}, error) {
}
// Fetch claims in the security token returned by the client and validate the token.
func checkClaimsFromToken(r *http.Request, cred auth.Credentials) (map[string]interface{}, APIErrorCode) {
func checkClaimsFromToken(r *http.Request, cred auth.Credentials) (map[string]any, APIErrorCode) {
token := getSessionToken(r)
if token != "" && cred.AccessKey == "" {
// x-amz-security-token is not allowed for anonymous access.
+2 -6
View File
@@ -24,6 +24,7 @@ import (
"fmt"
"io"
"os"
"slices"
"sort"
"strings"
"sync"
@@ -269,12 +270,7 @@ func (h *healingTracker) delete(ctx context.Context) error {
func (h *healingTracker) isHealed(bucket string) bool {
h.mu.RLock()
defer h.mu.RUnlock()
for _, v := range h.HealedBuckets {
if v == bucket {
return true
}
}
return false
return slices.Contains(h.HealedBuckets, bucket)
}
// resume will reset progress to the numbers at the start of the bucket.
+2 -2
View File
@@ -1,7 +1,7 @@
package cmd
// Code generated by github.com/tinylib/msgp DO NOT EDIT.
package cmd
import (
"github.com/tinylib/msgp/msgp"
)
+2 -2
View File
@@ -1,7 +1,7 @@
package cmd
// Code generated by github.com/tinylib/msgp DO NOT EDIT.
package cmd
import (
"bytes"
"testing"
+1 -1
View File
@@ -37,7 +37,7 @@ import (
"github.com/minio/pkg/v3/wildcard"
"github.com/minio/pkg/v3/workers"
"github.com/minio/pkg/v3/xtime"
"gopkg.in/yaml.v3"
"go.yaml.in/yaml/v3"
)
// expire: # Expire objects that match a condition
+2 -2
View File
@@ -1,7 +1,7 @@
package cmd
// Code generated by github.com/tinylib/msgp DO NOT EDIT.
package cmd
import (
"time"
+2 -2
View File
@@ -1,7 +1,7 @@
package cmd
// Code generated by github.com/tinylib/msgp DO NOT EDIT.
package cmd
import (
"bytes"
"testing"
+1 -1
View File
@@ -21,7 +21,7 @@ import (
"slices"
"testing"
"gopkg.in/yaml.v3"
"go.yaml.in/yaml/v3"
)
func TestParseBatchJobExpire(t *testing.T) {
+3 -4
View File
@@ -25,6 +25,7 @@ import (
"errors"
"fmt"
"io"
"maps"
"math/rand"
"net/http"
"net/url"
@@ -51,7 +52,7 @@ import (
"github.com/minio/pkg/v3/env"
"github.com/minio/pkg/v3/policy"
"github.com/minio/pkg/v3/workers"
"gopkg.in/yaml.v3"
"go.yaml.in/yaml/v3"
)
var globalBatchConfig batch.Config
@@ -574,9 +575,7 @@ func toObjectInfo(bucket, object string, objInfo minio.ObjectInfo) ObjectInfo {
oi.UserDefined[xhttp.AmzStorageClass] = objInfo.StorageClass
}
for k, v := range objInfo.UserMetadata {
oi.UserDefined[k] = v
}
maps.Copy(oi.UserDefined, objInfo.UserMetadata)
return oi
}
+2 -2
View File
@@ -1,7 +1,7 @@
package cmd
// Code generated by github.com/tinylib/msgp DO NOT EDIT.
package cmd
import (
"github.com/tinylib/msgp/msgp"
)
+2 -2
View File
@@ -1,7 +1,7 @@
package cmd
// Code generated by github.com/tinylib/msgp DO NOT EDIT.
package cmd
import (
"bytes"
"testing"
+1 -1
View File
@@ -21,7 +21,7 @@ import (
"slices"
"testing"
"gopkg.in/yaml.v3"
"go.yaml.in/yaml/v3"
)
func TestBatchJobPrefix_UnmarshalYAML(t *testing.T) {
+2 -2
View File
@@ -24,7 +24,7 @@ import (
"github.com/dustin/go-humanize"
"github.com/minio/pkg/v3/wildcard"
"gopkg.in/yaml.v3"
"go.yaml.in/yaml/v3"
)
//go:generate msgp -file $GOFILE
@@ -275,7 +275,7 @@ func (sf BatchJobSizeFilter) Validate() error {
type BatchJobSize int64
// UnmarshalYAML to parse humanized byte values
func (s *BatchJobSize) UnmarshalYAML(unmarshal func(interface{}) error) error {
func (s *BatchJobSize) UnmarshalYAML(unmarshal func(any) error) error {
var batchExpireSz string
err := unmarshal(&batchExpireSz)
if err != nil {
+2 -2
View File
@@ -1,7 +1,7 @@
package cmd
// Code generated by github.com/tinylib/msgp DO NOT EDIT.
package cmd
import (
"github.com/tinylib/msgp/msgp"
)
+2 -2
View File
@@ -1,7 +1,7 @@
package cmd
// Code generated by github.com/tinylib/msgp DO NOT EDIT.
package cmd
import (
"bytes"
"testing"
+2 -2
View File
@@ -1,7 +1,7 @@
package cmd
// Code generated by github.com/tinylib/msgp DO NOT EDIT.
package cmd
import (
"github.com/tinylib/msgp/msgp"
)
+2 -2
View File
@@ -1,7 +1,7 @@
package cmd
// Code generated by github.com/tinylib/msgp DO NOT EDIT.
package cmd
import (
"bytes"
"testing"
+1 -1
View File
@@ -21,7 +21,7 @@ import (
"slices"
"testing"
"gopkg.in/yaml.v3"
"go.yaml.in/yaml/v3"
)
func TestParseBatchJobReplicate(t *testing.T) {
+3 -6
View File
@@ -21,6 +21,7 @@ import (
"context"
"encoding/base64"
"fmt"
"maps"
"math/rand"
"net/http"
"runtime"
@@ -110,9 +111,7 @@ func (e BatchJobKeyRotateEncryption) Validate() error {
}
}
e.kmsContext = kms.Context{}
for k, v := range ctx {
e.kmsContext[k] = v
}
maps.Copy(e.kmsContext, ctx)
ctx["MinIO batch API"] = "batchrotate" // Context for a test key operation
if _, err := GlobalKMS.GenerateKey(GlobalContext, &kms.GenerateKeyRequest{Name: e.Key, AssociatedData: ctx}); err != nil {
return err
@@ -225,9 +224,7 @@ func (r *BatchJobKeyRotateV1) KeyRotate(ctx context.Context, api ObjectLayer, ob
// Since we are rotating the keys, make sure to update the metadata.
oi.metadataOnly = true
oi.keyRotation = true
for k, v := range encMetadata {
oi.UserDefined[k] = v
}
maps.Copy(oi.UserDefined, encMetadata)
if _, err := api.CopyObject(ctx, r.Bucket, oi.Name, r.Bucket, oi.Name, oi, ObjectOptions{
VersionID: oi.VersionID,
}, ObjectOptions{
+2 -2
View File
@@ -1,7 +1,7 @@
package cmd
// Code generated by github.com/tinylib/msgp DO NOT EDIT.
package cmd
import (
"github.com/tinylib/msgp/msgp"
)
+2 -2
View File
@@ -1,7 +1,7 @@
package cmd
// Code generated by github.com/tinylib/msgp DO NOT EDIT.
package cmd
import (
"bytes"
"testing"
+5 -5
View File
@@ -51,8 +51,8 @@ func runPutObjectBenchmark(b *testing.B, obj ObjectLayer, objSize int) {
// benchmark utility which helps obtain number of allocations and bytes allocated per ops.
b.ReportAllocs()
// the actual benchmark for PutObject starts here. Reset the benchmark timer.
b.ResetTimer()
for i := 0; i < b.N; i++ {
for i := 0; b.Loop(); i++ {
// insert the object.
objInfo, err := obj.PutObject(b.Context(), bucket, "object"+strconv.Itoa(i),
mustGetPutObjReader(b, bytes.NewReader(textData), int64(len(textData)), md5hex, sha256hex), ObjectOptions{})
@@ -101,11 +101,11 @@ func runPutObjectPartBenchmark(b *testing.B, obj ObjectLayer, partSize int) {
// benchmark utility which helps obtain number of allocations and bytes allocated per ops.
b.ReportAllocs()
// the actual benchmark for PutObjectPart starts here. Reset the benchmark timer.
b.ResetTimer()
for i := 0; i < b.N; i++ {
for i := 0; b.Loop(); i++ {
// insert the object.
totalPartsNR := int(math.Ceil(float64(objSize) / float64(partSize)))
for j := 0; j < totalPartsNR; j++ {
for j := range totalPartsNR {
if j < totalPartsNR-1 {
textPartData = textData[j*partSize : (j+1)*partSize-1]
} else {
+1 -1
View File
@@ -99,7 +99,7 @@ func BitrotAlgorithmFromString(s string) (a BitrotAlgorithm) {
return alg
}
}
return
return a
}
func newBitrotWriter(disk StorageAPI, origvolume, volume, filePath string, length int64, algo BitrotAlgorithm, shardSize int64) io.Writer {
+6 -10
View File
@@ -1,7 +1,7 @@
package cmd
// Code generated by github.com/tinylib/msgp DO NOT EDIT.
package cmd
import (
"github.com/tinylib/msgp/msgp"
)
@@ -59,19 +59,17 @@ func (z *ServerSystemConfig) DecodeMsg(dc *msgp.Reader) (err error) {
if z.MinioEnv == nil {
z.MinioEnv = make(map[string]string, zb0003)
} else if len(z.MinioEnv) > 0 {
for key := range z.MinioEnv {
delete(z.MinioEnv, key)
}
clear(z.MinioEnv)
}
for zb0003 > 0 {
zb0003--
var za0002 string
var za0003 string
za0002, err = dc.ReadString()
if err != nil {
err = msgp.WrapError(err, "MinioEnv")
return
}
var za0003 string
za0003, err = dc.ReadString()
if err != nil {
err = msgp.WrapError(err, "MinioEnv", za0002)
@@ -240,14 +238,12 @@ func (z *ServerSystemConfig) UnmarshalMsg(bts []byte) (o []byte, err error) {
if z.MinioEnv == nil {
z.MinioEnv = make(map[string]string, zb0003)
} else if len(z.MinioEnv) > 0 {
for key := range z.MinioEnv {
delete(z.MinioEnv, key)
}
clear(z.MinioEnv)
}
for zb0003 > 0 {
var za0002 string
var za0003 string
zb0003--
var za0002 string
za0002, bts, err = msgp.ReadStringBytes(bts)
if err != nil {
err = msgp.WrapError(err, "MinioEnv")
+2 -2
View File
@@ -1,7 +1,7 @@
package cmd
// Code generated by github.com/tinylib/msgp DO NOT EDIT.
package cmd
import (
"bytes"
"testing"
+7 -9
View File
@@ -154,7 +154,6 @@ func initFederatorBackend(buckets []string, objLayer ObjectLayer) {
g := errgroup.WithNErrs(len(bucketsToBeUpdatedSlice)).WithConcurrency(50)
for index := range bucketsToBeUpdatedSlice {
index := index
g.Go(func() error {
return globalDNSConfig.Put(bucketsToBeUpdatedSlice[index])
}, index)
@@ -593,7 +592,7 @@ func (api objectAPIHandlers) DeleteMultipleObjectsHandler(w http.ResponseWriter,
output[idx] = obj
idx++
}
return
return output
}
// Disable timeouts and cancellation
@@ -1387,10 +1386,7 @@ func (api objectAPIHandlers) PostPolicyBucketHandler(w http.ResponseWriter, r *h
// Set the correct hex md5sum for the fan-out stream.
fanOutOpts.MD5Hex = hex.EncodeToString(md5w.Sum(nil))
concurrentSize := 100
if runtime.GOMAXPROCS(0) < concurrentSize {
concurrentSize = runtime.GOMAXPROCS(0)
}
concurrentSize := min(runtime.GOMAXPROCS(0), 100)
fanOutResp := make([]minio.PutObjectFanOutResponse, 0, len(fanOutEntries))
eventArgsList := make([]eventArgs, 0, len(fanOutEntries))
@@ -1661,9 +1657,11 @@ func (api objectAPIHandlers) HeadBucketHandler(w http.ResponseWriter, r *http.Re
return
}
if s3Error := checkRequestAuthType(ctx, r, policy.ListBucketAction, bucket, ""); s3Error != ErrNone {
writeErrorResponseHeadersOnly(w, errorCodes.ToAPIErr(s3Error))
return
if s3Error := checkRequestAuthType(ctx, r, policy.HeadBucketAction, bucket, ""); s3Error != ErrNone {
if s3Error := checkRequestAuthType(ctx, r, policy.ListBucketAction, bucket, ""); s3Error != ErrNone {
writeErrorResponseHeadersOnly(w, errorCodes.ToAPIErr(s3Error))
return
}
}
getBucketInfo := objectAPI.GetBucketInfo
+2 -2
View File
@@ -657,7 +657,7 @@ func testAPIDeleteMultipleObjectsHandler(obj ObjectLayer, instanceType, bucketNa
sha256sum := ""
var objectNames []string
for i := 0; i < 10; i++ {
for i := range 10 {
contentBytes := []byte("hello")
objectName := "test-object-" + strconv.Itoa(i)
if i == 0 {
@@ -687,7 +687,7 @@ func testAPIDeleteMultipleObjectsHandler(obj ObjectLayer, instanceType, bucketNa
// The following block will create a bucket policy with delete object to 'public/*'. This is
// to test a mixed response of a successful & failure while deleting objects in a single request
policyBytes := []byte(fmt.Sprintf(`{"Id": "Policy1637752602639", "Version": "2012-10-17", "Statement": [{"Sid": "Stmt1637752600730", "Action": "s3:DeleteObject", "Effect": "Allow", "Resource": "arn:aws:s3:::%s/public/*", "Principal": "*"}]}`, bucketName))
policyBytes := fmt.Appendf(nil, `{"Id": "Policy1637752602639", "Version": "2012-10-17", "Statement": [{"Sid": "Stmt1637752600730", "Action": "s3:DeleteObject", "Effect": "Allow", "Resource": "arn:aws:s3:::%s/public/*", "Principal": "*"}]}`, bucketName)
rec := httptest.NewRecorder()
req, err := newTestSignedRequestV4(http.MethodPut, getPutPolicyURL("", bucketName), int64(len(policyBytes)), bytes.NewReader(policyBytes),
credentials.AccessKey, credentials.SecretKey, nil)
+2 -3
View File
@@ -23,6 +23,7 @@ import (
"errors"
"fmt"
"io"
"maps"
"net/http"
"strconv"
"strings"
@@ -959,9 +960,7 @@ func putRestoreOpts(bucket, object string, rreq *RestoreObjectRequest, objInfo O
UserDefined: meta,
}
}
for k, v := range objInfo.UserDefined {
meta[k] = v
}
maps.Copy(meta, objInfo.UserDefined)
if len(objInfo.UserTags) != 0 {
meta[xhttp.AmzObjectTagging] = objInfo.UserTags
}
+4 -4
View File
@@ -248,19 +248,19 @@ func proxyRequestByToken(ctx context.Context, w http.ResponseWriter, r *http.Req
if subToken, nodeIndex = parseRequestToken(token); nodeIndex >= 0 {
proxied, success = proxyRequestByNodeIndex(ctx, w, r, nodeIndex, returnErr)
}
return
return subToken, proxied, success
}
func proxyRequestByNodeIndex(ctx context.Context, w http.ResponseWriter, r *http.Request, index int, returnErr bool) (proxied, success bool) {
if len(globalProxyEndpoints) == 0 {
return
return proxied, success
}
if index < 0 || index >= len(globalProxyEndpoints) {
return
return proxied, success
}
ep := globalProxyEndpoints[index]
if ep.IsLocal {
return
return proxied, success
}
return true, proxyRequest(ctx, w, r, ep, returnErr)
}
+1 -2
View File
@@ -472,7 +472,7 @@ func (sys *BucketMetadataSys) GetConfig(ctx context.Context, bucket string) (met
return meta, reloaded, nil
}
val, err, _ := sys.group.Do(bucket, func() (val interface{}, err error) {
val, err, _ := sys.group.Do(bucket, func() (val any, err error) {
meta, err = loadBucketMetadata(ctx, objAPI, bucket)
if err != nil {
if !sys.Initialized() {
@@ -511,7 +511,6 @@ func (sys *BucketMetadataSys) concurrentLoad(ctx context.Context, buckets []stri
g := errgroup.WithNErrs(len(buckets))
bucketMetas := make([]BucketMetadata, len(buckets))
for index := range buckets {
index := index
g.Go(func() error {
// Sleep and stagger to avoid blocked CPU and thundering
// herd upon start up sequence.
+4 -4
View File
@@ -161,7 +161,7 @@ func (b BucketMetadata) lastUpdate() (t time.Time) {
t = b.BucketTargetsConfigMetaUpdatedAt
}
return
return t
}
// Versioning returns true if versioning is enabled
@@ -542,13 +542,13 @@ func (b *BucketMetadata) migrateTargetConfig(ctx context.Context, objectAPI Obje
func encryptBucketMetadata(ctx context.Context, bucket string, input []byte, kmsContext kms.Context) (output, metabytes []byte, err error) {
if GlobalKMS == nil {
output = input
return
return output, metabytes, err
}
metadata := make(map[string]string)
key, err := GlobalKMS.GenerateKey(ctx, &kms.GenerateKeyRequest{AssociatedData: kmsContext})
if err != nil {
return
return output, metabytes, err
}
outbuf := bytes.NewBuffer(nil)
@@ -561,7 +561,7 @@ func encryptBucketMetadata(ctx context.Context, bucket string, input []byte, kms
}
metabytes, err = json.Marshal(metadata)
if err != nil {
return
return output, metabytes, err
}
return outbuf.Bytes(), metabytes, nil
}
+2 -2
View File
@@ -1,7 +1,7 @@
package cmd
// Code generated by github.com/tinylib/msgp DO NOT EDIT.
package cmd
import (
"github.com/tinylib/msgp/msgp"
)
+2 -2
View File
@@ -1,7 +1,7 @@
package cmd
// Code generated by github.com/tinylib/msgp DO NOT EDIT.
package cmd
import (
"bytes"
"testing"
+3
View File
@@ -297,6 +297,9 @@ func checkPutObjectLockAllowed(ctx context.Context, rq *http.Request, bucket, ob
if legalHold, lerr = objectlock.ParseObjectLockLegalHoldHeaders(rq.Header); lerr != nil {
return mode, retainDate, legalHold, toAPIErrorCode(ctx, lerr)
}
if legalHoldPermErr != ErrNone {
return mode, retainDate, legalHold, legalHoldPermErr
}
}
if retentionRequested {

Some files were not shown because too many files have changed in this diff Show More