Commit Graph

580 Commits

Author SHA1 Message Date
Feng Ruohang 76195f1c68 fix: keep streaming trailers visible after replication headers are stripped
A request that does not earn replication trust continues with a clone whose
internal replication headers are removed. r.Clone copies the Trailer map, but
the streaming body reader created from the original request fills the
original map, so a trailing checksum was never seen by the hash reader and
PutObject and UploadPart with STREAMING-UNSIGNED-PAYLOAD-TRAILER failed with
XAmzContentChecksumMismatch whenever an untrusted X-Minio-Source-* header was
present. Share the trailer map with the clone, as the Snowball path already
does for its per-entry requests, and cover both handlers with a test.

The marker evaluation that was copied into six handlers now lives in
evaluateReplicationTrust so the rule (a declared replica without the
replication permission is rejected; trust needs the exact marker plus the
permission) is defined once.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 14:06:56 +08:00
Feng Ruohang 75a6734e49 fix: authorize explicit version deletes with DeleteObjectVersion
Select DeleteObject or DeleteObjectVersion from each request's effective version ID for single and multi-delete. Authenticate multi-delete once, bind version conditions to each XML entry, and keep the established DeleteObject plus ReplicateDelete target contract with explicit version denies honored only on trusted replication.\n\nRefs: #58

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:08:02 +08:00
Feng Ruohang 04b097fd9f chore: refresh compatibility and lint baselines
Accept the new CORS test routes, resident getter, and replication header literals in the rebrand guard. Apply gofumpt, context-first helper ordering, and spelling fixes required by CI.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 02:31:20 +08:00
Feng Ruohang ab3ae99ca3 fix: preserve Snowball request defaults across workers
Snapshot per-entry requests after applying bucket encryption defaults but before streaming trailers are consumed. Keep authorization failures fatal while retaining Snowball ignore-errors behavior for object-lock failures.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 00:20:06 +08:00
Feng Ruohang ff44527a3c fix: isolate Snowball replication trust per entry
Evaluate PutObject and ReplicateObject permissions with immutable per-entry request snapshots during concurrent Snowball extraction. Preserve the first API error without sharing mutable handler state, and cover prefix-scoped trust under the race detector.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 00:02:24 +08:00
Feng Ruohang c9ad746732 fix: verify replication permissions in validity probes
Evaluate ReplicateObject or ReplicateDelete before returning the no-op validation response, so underprivileged target credentials fail during replication setup instead of at runtime.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-01 23:16:09 +08:00
Feng Ruohang 938603458d fix: harden CORS and replication request trust
Keep pre-authentication CORS lookups resident-only so attacker-controlled path segments cannot trigger metadata I/O or grow the metadata cache. Preserve fail-closed behavior for startup, load failures, invalid metadata, and the internal namespace.

Centralize replication request trust after authentication, distinguish general replication from replica-only privileges, and gate SSE-C ciphertext handling, source metadata, object-lock bypasses, event suppression, delete semantics, and replica status on the appropriate permission. Add least-privilege, multipart, PostPolicy, CORS amplification, and compatibility regressions.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-01 20:50:08 +08:00
Feng Ruohang 74c97d005d fix: authorize SSE-C attribute reads by replication permission
GetObjectAttributes lets a replication peer read SSE-C attributes without
presenting the customer key. That carve-out was keyed on the
X-Minio-Source-Replication-Request header alone, which any client can set,
so a caller holding only s3:GetObject could read an SSE-C object's ETag,
plaintext size, part list, and checksums without the key -- the response was
byte-identical to one made with the correct key.

Gate the carve-out on s3:ReplicateObject for the target object, mirroring
CopyObjectHandler's existing replication check.

The pre-existing test asserted the carve-out with root credentials, which
hold every action and therefore cannot tell a gated check apart from an
ungated one. Add least-privilege cases that do: a reader without
s3:ReplicateObject is now refused, a caller that holds it keeps the
carve-out, and the ordinary key-bearing path is unchanged.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-30 15:50:24 +08:00
Feng Ruohang f2ba439022 Merge remote-tracking branch 'origin/main' into codex/get-object-attributes-ssec 2026-08-29 19:04:14 +08:00
Feng Ruohang 21870fa2e7 fix: preserve replicated object attributes reads
Keep the existing trusted replication carve-out while authenticating ordinary SSE-C GetObjectAttributes requests.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 18:18:01 +08:00
Feng Ruohang 474cd5801e fix: authenticate SSE-C for object attributes
Unseal the supplied customer key after request preconditions before returning object size, ETag, checksum, or part attributes. Cover zero-byte and non-empty objects with correct, wrong, and missing keys.\n\nRefs #84.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 17:26:37 +08:00
Feng Ruohang ffb70eb373 fix: re-encrypt a key rotation the object layer has to rewrite
A key rotation rewraps the object key held in metadata; it never re-encrypts
the stored bytes. CopyObjectHandler took that shortcut whenever the request
looked like a same-object SSE-C rotation, on the assumption that the object
layer would then leave the stored bytes alone. That is the same assumption
copyRewritesObjectData() was added to stop making.

When the source is a null version on a bucket that gained versioning after the
object was written, the object layer cannot reference that version and falls
back to PutObject. The reader at that point holds plaintext decrypted with the
old key and no EncryptFn is set, so the destination ends up storing plaintext
under metadata that claims the object is SSE-C encrypted. A subsequent GET
failed with "sio: unsupported version".

Gate the rotation shortcut on the same prediction the compression metadata
already uses. When the object layer stores new object data the rotation falls
through to the regular re-encrypting copy, which decrypts with the old key and
re-encrypts with the new one. The source version selection moves next to the
gate because both decisions need it.

That fallback authenticates the source key through the source decryptor, which
GetObjectNInfo does not build for a zero byte object. Check the key explicitly
before the destination is written, so the gate cannot turn a rotation that the
shortcut rejected with AccessDenied into one that succeeds. The re-encrypting
copy regenerates the encrypted ETag, unlike an in-place rotation; the test
records that difference.

The other three object layer CopyObject callers that set metadataOnly -
PostRestoreObjectHandler, updateRestoreMetadata and batchKeyRotate - address
the same version on both sides and never set Versioned, so they only reach the
two in-place cases already covered by the copyRewritesObjectData table.

Signed-off-by: Feng Ruohang <rh@vonng.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fk3PAD7DHCYzcyegYWqAmt
2026-08-29 16:00:21 +08:00
Feng Ruohang 0b0ae2423a fix: keep copy metadata consistent with a rewritten null version
CopyObjectHandler recorded the source compression metadata whenever the copy
was metadata-only, on the assumption that the object layer would then leave
the stored bytes alone. That assumption does not hold. Both
erasureServerPools.CopyObject and erasureSets.CopyObject only skip a data
rewrite in three cases, and otherwise fall back to a full PutObject.

The reachable gap is a copy whose source is a null version on a bucket that
gained versioning after the object was written. Neither version ID is set, so
the self-referential version branch is skipped, the data is rewritten as
plaintext, and the preserved compression metadata then described bytes that
no longer exist. A subsequent GET failed with "s2: corrupt input".

Mirror the object layer's decision in copyRewritesObjectData and record the
compression metadata from it, so the metadata always describes whichever
bytes are finally stored. The source version selection that lets a versioned
metadata-only copy add a self-referential version moves next to the same
decision, since both depend on the effective metadata-only value.

Signed-off-by: Feng Ruohang <rh@vonng.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 16:00:21 +08:00
Feng Ruohang e73436c99d fix: decrypt CopyObject checksums with destination key
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 16:00:21 +08:00
Feng Ruohang 8d76a255c4 fix: return the remote part checksum to federated UploadPartCopy (#72)
The legacy etcd federation branch of CopyObjectPartHandler forwards copied
bytes with minio-go Core.PutObjectPart, which can only recover a checksum
from response headers. After the server-side part checksum work, the remote
computes and persists the checksum, but an AWS-compatible UploadPart response
correctly omits a checksum the request did not supply, so the proxy had
nothing to put in CopyPartResult.

The destination now returns the non-empty checksum fields of the PartInfo
produced by that exact write, but only when the request carries the
minio-federated application token that getRemoteInstanceClient already
attaches. Ordinary UploadPart responses are unchanged, and the checksum type
is deliberately not returned because UploadPart does not carry it. The
User-Agent is a response-shape hint only: it never gates authorization,
visibility or validation, and it can expose nothing beyond the checksum of
the body the caller just uploaded.

Reading the checksum from the same PartInfo that produced the response ETag
also keeps the pair bound to one write, so a concurrent overwrite of the same
part number cannot publish another writer's checksum.

Tests cover the application token gating matrix including lookalike tokens,
the real minio-go response parser, concurrent overwrites of one part number,
and an in-process two-deployment probe that drives the federation branch
through the real getRemoteInstanceClient into a real PutObjectPartHandler for
both FULL_OBJECT and COMPOSITE uploads.

Fixes #64

Signed-off-by: Feng Ruohang <rh@vonng.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 01:06:50 +08:00
Feng Ruohang f2520f3346 fix: return checksums from CopyObject
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-24 20:59:27 +08:00
Feng Ruohang 05df6e70d7 fix: preserve transform state on metadata-only copies
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-24 20:46:31 +08:00
Feng Ruohang c0e7159771 fix: checksum CopyObject data before compression
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-24 20:33:54 +08:00
Feng Ruohang 2f55347f78 fix(iam): bind policy conditions to effective request values
Policy evaluation mixed server-derived identity and transport values with raw headers and query parameters. A client could therefore shadow internal condition keys, synthesize LDAP or JWT resource variables, substitute request tags for stored tags, or make a condition observe a value different from the one the handler actually used.

Partition condition sources, reserve internal names, adopt exact-name lookup from silo-pkg, and bind authorization to the effective request state. Preserve compatible query forms for storage class and upload tags with explicit header precedence, while restricting signature age and existing-object tags to authenticated or server-resolved values.

Tests sweep every supported key across header and query routes and exercise LDAP/OIDC variables, object-lock spelling, STS tags, metadata extraction, and end-to-end policy decisions.

Co-authored-by: ChatGPT <noreply@openai.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-08-04 23:00:30 +08:00
Feng Ruohang efb6e5b00b fix: fake CVE-2026-40028 harden snowball unsigned-trailer auth
Track issue #28 / GHSA-9c4q-hq6p-c237 as fake CVE-2026-40028. Close the Snowball auto-extract auth gap in PutObjectExtractHandler by treating authTypeStreamingUnsignedTrailer the same as ordinary PUTs: honor X-Amz-Decoded-Content-Length, initialize newUnsignedV4ChunkedReader(), and verify the SigV4 request before any tar bytes reach untar(). This removes the forged-signature write primitive that let a single request fan out into arbitrary extracted object creation.

Add regression coverage for forged-signature Snowball unsigned-trailer writes, anonymous Snowball requests against non-public buckets, and legitimate signed Snowball extraction with trailing CRC32 trailers. Validate the new tests against the vulnerable parent and patched tree, and confirm with containerized before/after smoke runs that the exploit succeeds pre-fix, fails post-fix, and normal signed Snowball uploads still extract correctly.

Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Code <claude-code@anthropic.com>
2026-04-16 14:46:45 +08:00
Feng Ruohang f444b6f37e fix: fake CVE-2026-40027 block unsigned-trailer query auth bypass
Track issue #27 / GHSA-hv4r-mvr4-25vw as fake CVE-2026-40027. Close the unsigned-trailer trust flaw that let query-string credentials skip signature verification in PutObject and PutObjectPart by moving presigned rejection and SigV4 verification into newUnsignedV4ChunkedReader(), so authTypeStreamingUnsignedTrailer can no longer silently downgrade query auth into an anonymous body read.

Add focused regression coverage for forged query-string-only unsigned-trailer PUTs and multipart uploads, mixed header/query auth rejection, and anonymous unsigned-trailer writes that remain allowed only when bucket policy explicitly permits them. Validate the new tests against the vulnerable parent and confirm with before/after live-server runs that presigned unsigned-trailer attacks are rejected while legitimate header-authenticated and policy-driven flows still work.

Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Code <claude-code@anthropic.com>
2026-04-16 14:46:44 +08:00
Feng Ruohang 56fa63bfd1 fix: CVE-2026-34204 block replication metadata injection
Close the replication-header trust flaw that allowed ordinary PutObject and CopyObject requests to smuggle X-Minio-Replication-* headers into X-Minio-Internal-* SSE metadata and write objects into an unreadable state. Stop accepting replication-only metadata in the default extraction path, restore it only after a trusted replication write has passed ReplicateObjectAction, and tighten CopyObject by sanitizing replication-only request headers before metadata, precondition, and SSE-C source handling consume them. Also gate replica status writes on the same trusted replication path and restore replication SSE metadata in multipart and snowball upload flows so legitimate replication continues to work.

Add focused regression coverage for untrusted PUT and COPY header poisoning at the handler layer, plus helper tests for trusted vs untrusted metadata extraction and CopyObject header sanitization. Validate the new tests against both the patched tree and the vulnerable HEAD baseline, and confirm with live server before/after runs that malicious PUT/COPY requests no longer turn objects unreadable.

Co-authored-by: Codex <codex@openai.com>
Co-Authored-By: Claude <noreply@anthropic.com>
2026-04-15 18:36:49 +08:00
Klaus Post b8631cf531 Use new gofumpt (#21613)
Update tinylib. Should fix CI.

`gofumpt -w .&&go generate ./...`
2025-09-28 13:59:21 -07:00
M Alvee 07c3a429bf fix: conditional checks write for multipart (#21567) 2025-09-07 09:13:09 -07:00
Klaus Post f0b91e5504 Run modernize (#21546)
`go run golang.org/x/tools/gopls/internal/analysis/modernize/cmd/modernize@latest -fix -test ./...` executed.

`go generate ./...` ran afterwards to keep generated.
2025-08-28 19:39:48 -07:00
Mark Theunissen 2718d9a430 CopyObject must preserve checksums and encrypt them if required (#21399) 2025-06-25 08:08:54 -07:00
Harshavardhana 8c70975283 make sure to validate signature unsigned trailer stream (#21103)
This is a security incident fix, it would seem like since
the implementation of unsigned payload trailer on PUTs,
we do not validate the signature of the incoming request.

The signature can be invalid and is totally being ignored,
this in-turn allows any arbitrary secret to upload objects
given the user has "WRITE" permissions on the bucket, since
acces-key is a public information in general exposes these
potential users with WRITE on the bucket to be used by any
arbitrary client to make a fake request to MinIO the signature
under Authorization: header is totally ignored.

A test has been added to cover this scenario and fail
appropriately.
2025-04-03 07:55:52 -07:00
Krishnan Parthasarathi 01447d2438 Fix evaluation of NewerNoncurrentVersions (#21096)
- Move VersionPurgeStatus into replication package
- ilm: Evaluate policy w/ obj retention/replication
- lifecycle: Use Evaluator to enforce ILM in scanner
- Unit tests covering ILM, replication and retention
- Simplify NewEvaluator constructor
2025-04-02 23:45:06 -07:00
Taran Pelkey e88d494775 Migrate golanglint-ci config to V2 (#21081) 2025-03-29 17:56:02 -07:00
Klaus Post 90f5e1e5f6 tests: Do not allow forced type asserts (#20905) 2025-02-18 08:25:55 -08:00
Ramon de Klein 437dd4e32a Fix missing authorization check for PutObjectRetentionHandler (#20929) 2025-02-12 08:08:13 -08:00
Manuel Reis 60f8423157 Quick patch for Snowball AutoExtract: #20883 (#20885)
* Checking allowance on empty prefix or Snowball-prefix - fixes #20883
* Check the policy for each object during Snowball auto-extraction
2025-02-10 15:52:59 -08:00
Klaus Post b8dde47d4e fix: multipart replication with single part objects (#20895)
x-amz-checksum-algorithm is not set, causing all multipart single-part objects
to fail to replicate going via sftp/FTP uploads.
2025-02-05 15:06:02 -08:00
Klaus Post 827004cd6d Add Full Object Checksums and CRC64-NVME (#20855)
Backport of AIStor PR 247.

Add support for full object checksums as described here:

https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity.html

New checksum types are fully supported. Mint tests from https://github.com/minio/minio-go/pull/2026 are now passing.

Includes fixes from https://github.com/minio/minio/pull/20743 for mint tests.

Add using checksums as validation for object content. Fixes #20845 #20849

Fixes checksum replication (downstream PR 250)
2025-01-20 06:49:07 -08:00
Mark Theunissen 01e520eb23 s3: Sanitize the source object name in CopyObject handler (#20774) 2024-12-17 07:01:07 -08:00
Klaus Post 5bb31e4883 Disable mint full object tests (#20743)
Remove expected failures from https://github.com/minio/minio-go/pull/2026
2024-12-09 18:59:22 -08:00
Mark Theunissen d202fdd022 Add the policy name to the audit logs tags when doing policy-based API calls. Add retention settings to tags (#20638)
* Add the policy name to the audit log tags when doing policy-based API calls

* Audit log the retention settings requested in the API call

* Audit log of retention on PutObjectRetention API path too
2024-11-25 09:17:12 -08:00
Poorna 1bc6681176 fix tagging overwrite during resync (#20525) 2024-10-04 22:16:15 -07:00
Klaus Post 05a6c170bf Fix PutObject Trailing checksum (#20456)
PutObject would verify trailing checksums, but not store them.

Fixes #20455
2024-09-19 05:59:07 -07:00
Anis Eleuch 398ffb1136 Enable compression with encryption in CopyObject API (#20411)
When the encryption and compression are both enabled, the
the server will avoid compressing the data for no apparent reason

This commit will enable it and update unit tests.
2024-09-12 13:10:44 -07:00
Harshavardhana c2e318dd40 remove mincache EOS related feature from upstream (#20375) 2024-09-03 11:23:41 -07:00
Klaus Post f1302c40fe Fix uninitialized replication stats (#20260)
Services are unfrozen before `initBackgroundReplication` is finished. This means that 
the globalReplicationStats write is racy. Switch to an atomic pointer.

Provide the `ReplicationPool` with the stats, so it doesn't have to be grabbed 
from the atomic pointer on every use.

All other loads and checks are nil, and calls return empty values when stats 
still haven't been initialized.
2024-08-15 05:04:40 -07:00
Klaus Post 49055658a9 Fix missing hash in GetObjectAttributes (#20231)
SHA256/SHA1 were mixed up.

Simplify code as well.
2024-08-08 13:19:41 -07:00
Poorna 3535197f99 replication: proxy only on missing object or read quorum err (#20101) 2024-07-16 16:46:41 -07:00
Harshavardhana be84a4fd68 do not proxy invalid object names (#20031) 2024-07-02 14:28:55 -07:00
Harshavardhana c91d1ec2e3 fix: avoid metadata cache without data for all callers (#19935) 2024-06-14 06:28:35 -07:00
jiuker 62e6dc950d fix: do not update metadata cache upon headObject() (#19929) 2024-06-13 08:42:02 -07:00
Klaus Post ad04afe381 Fix SSEC multipart checksum replication (#19915)
* Multipart SSEC checksums were not transferred.
* Remove key mismatch logging. This key is user-controlled with SSEC.
* If the source is SSEC and the destination reports ErrSSEEncryptedObject, 
  assume replication is good.
2024-06-12 23:56:12 -07:00
jiuker 0662c90b5c fix: copyObject restore with a specific version, update test cases (#19895) 2024-06-10 08:50:49 -07:00
Klaus Post a2cab02554 Fix SSE-C checksums (#19896)
Compression will be disabled by default if SSE-C is specified. So we can still honor SSE-C.
2024-06-10 08:31:51 -07:00