Commit Graph

6620 Commits

Author SHA1 Message Date
Feng Ruohang 5594d284fc chore: remove dead code left by earlier fixes
checkSSECCopySourceKey duplicated the source-key authentication that both
CopyObject read paths already perform; the DeleteObjects signature-error
branch became unreachable once signatures are verified once per request; the
discrete PostgreSQL and MySQL notification environment constants have had no
reader since DSNs became mandatory.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 14:06:56 +08:00
Feng Ruohang 632eb4729a refactor: serve non-resident buckets with the global CORS policy
The pre-authentication CORS lookup stays resident-only, so client-supplied
path segments still cause no metadata I/O and no cache growth. The
fail-closed states for startup, load failures, and the internal namespace
are gone: CORS is a browser response policy rather than an authorization
boundary, and failing closed only denied browser clients CORS headers while
bucket metadata was still loading. A bucket whose stored CORS document does
not parse still gets no CORS headers. This removes the loadFailed bookkeeping
and the unused GetCorsConfig, HasAllowedOrigin, and generic Update path for
CORS; tests use the CORS-specific writer.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 14:06:56 +08:00
Feng Ruohang 3b5de82f5a fix: keep plain Enabled versioning when Object Lock is enabled on a bucket
Site adoption and ForceCreate preserved a suspended or prefix-excluded
versioning configuration while bootstrapping Object Lock, persisting a state
that PutBucketVersioning itself rejects: objects under an excluded prefix in
a WORM bucket were not versioned and escaped retention. enablePeerBucketVersioning
now takes the lock intent and replaces such configurations with plain Enabled
versioning, and metadata loading ignores prefix exclusions on a locked bucket
as it ignored suspension before. The adoption tests assert the normalized
state and keep the timestamp-preservation checks on valid documents.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 14:06:56 +08:00
Feng Ruohang 76195f1c68 fix: keep streaming trailers visible after replication headers are stripped
A request that does not earn replication trust continues with a clone whose
internal replication headers are removed. r.Clone copies the Trailer map, but
the streaming body reader created from the original request fills the
original map, so a trailing checksum was never seen by the hash reader and
PutObject and UploadPart with STREAMING-UNSIGNED-PAYLOAD-TRAILER failed with
XAmzContentChecksumMismatch whenever an untrusted X-Minio-Source-* header was
present. Share the trailer map with the clone, as the Snowball path already
does for its per-entry requests, and cover both handlers with a test.

The marker evaluation that was copied into six handlers now lives in
evaluateReplicationTrust so the rule (a declared replica without the
replication permission is rejected; trust needs the exact marker plus the
permission) is defined once.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 14:06:56 +08:00
Feng Ruohang 53b09f4e25 style: apply gofumpt to migration globals
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:20:53 +08:00
Feng Ruohang 59812446ad fix: surface metadata migration failures after locking
Degrade to the loaded in-memory configuration only when the short migration lock cannot be acquired. Preserve parse, I/O, KMS, and persistence errors once the lock is held.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:20:53 +08:00
Feng Ruohang b3a6d5dbf8 fix: bound bucket metadata migration locking
Use a short dedicated migration lock timeout and fall back to the already loaded legacy or target configuration in memory when persistence is contended. Never fall back to an unlocked metadata save, and keep peer fan-out contexts free of stale lock markers.\n\nRefs: #102

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:20:53 +08:00
Feng Ruohang 11de51ec26 fix: limit imported metadata replication to changed fields
Release metadata.lock with defer and dispatch only the configuration types actually present in the import. Avoid stamping unrelated peer metadata with the import timestamp.\n\nRefs: #102

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:20:53 +08:00
Feng Ruohang 10f3a8590b fix: harden bucket metadata creation edge cases
Preserve existing records only for ForceCreate, reject ghost metadata on genuine creation, keep object-lock versioning invariants, and complete metadata saves after caller cancellation. Expand deterministic coverage for peer bulk, lifecycle delete, ghost creation, and cancellation.\n\nRefs: #102

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:20:53 +08:00
Feng Ruohang 32a1b81e4c fix: rebase imports and migrations under metadata.lock
Apply only validated import fields to a fresh locked record, block ForceCreate after real read errors, and route legacy or target-config migration saves through the shared lock. Compute lifecycle deletion state from the locked record.\n\nRefs: #102

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:20:53 +08:00
Feng Ruohang 62d8c649fe fix: preserve metadata during bucket creation and adoption
After storage bucket creation, merge required versioning and lock defaults into the latest on-disk metadata under metadata.lock. Avoid ForceCreate and site-adoption rewrites that replaced existing bucket configuration.\n\nRefs: #102

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:20:53 +08:00
Feng Ruohang 312397739e test: reproduce ForceCreate bucket metadata clobber
Create policy and CORS state, force-create the existing bucket, and require the original Created time and both metadata fields to survive.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:20:53 +08:00
Feng Ruohang 7ade0c045b fix: serialize bucket metadata updates across config types
Use one per-bucket metadata.lock for ordinary updates, CORS transitions, and legacy bulk replication. Persist and update the local cache while locked, then release before peer metadata reload fan-out.\n\nRefs: #102

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:20:53 +08:00
Feng Ruohang f9f9fa6c90 test: reproduce cross-type bucket metadata lost updates
Pause one whole-record writer at the metadata PutObject boundary and let a different config writer commit from the same stale snapshot. Assert that policy+CORS and tagging+SSE both survive on disk and in the resident cache.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:20:53 +08:00
Feng Ruohang dee2c3a02b fix: preserve multi-delete authentication and audit context
Authenticate DeleteObjects before validating entry count, remove the obsolete per-version auth helper, and pin the marker-only request to the ordinary authorization path.\n\nRefs: #58

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:08:02 +08:00
Feng Ruohang 75a6734e49 fix: authorize explicit version deletes with DeleteObjectVersion
Select DeleteObject or DeleteObjectVersion from each request's effective version ID for single and multi-delete. Authenticate multi-delete once, bind version conditions to each XML entry, and keep the established DeleteObject plus ReplicateDelete target contract with explicit version denies honored only on trusted replication.\n\nRefs: #58

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:08:02 +08:00
Feng Ruohang f8b598f1d3 test: reproduce explicit-version delete authorization mismatch
Prove that DeleteObjectVersion-only principals cannot delete named UUID or null versions while DeleteObject-only principals can, contrary to the S3 action mapping in issue #58.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:08:02 +08:00
Feng Ruohang 04b097fd9f chore: refresh compatibility and lint baselines
Accept the new CORS test routes, resident getter, and replication header literals in the rebrand guard. Apply gofumpt, context-first helper ordering, and spelling fixes required by CI.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 02:31:20 +08:00
Feng Ruohang ab3ae99ca3 fix: preserve Snowball request defaults across workers
Snapshot per-entry requests after applying bucket encryption defaults but before streaming trailers are consumed. Keep authorization failures fatal while retaining Snowball ignore-errors behavior for object-lock failures.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 00:20:06 +08:00
Feng Ruohang 5db7be4ee4 fix: validate replication within the rule prefix
Place synthetic permission-check objects under each enabled rule's effective prefix, so least-privilege target policies are validated against the namespace they will actually replicate.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 00:03:33 +08:00
Feng Ruohang ff44527a3c fix: isolate Snowball replication trust per entry
Evaluate PutObject and ReplicateObject permissions with immutable per-entry request snapshots during concurrent Snowball extraction. Preserve the first API error without sharing mutable handler state, and cover prefix-scoped trust under the race detector.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 00:02:24 +08:00
Feng Ruohang c9ad746732 fix: verify replication permissions in validity probes
Evaluate ReplicateObject or ReplicateDelete before returning the no-op validation response, so underprivileged target credentials fail during replication setup instead of at runtime.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-01 23:16:09 +08:00
Feng Ruohang f3438b2602 fix: validate CORS state in replication status
Count only valid live CORS states in per-site summaries. Treat baselines and tombstones as absent, and diagnose malformed payloads or missing source timestamps.\n\nRefs: #77

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-01 23:16:04 +08:00
Feng Ruohang 938603458d fix: harden CORS and replication request trust
Keep pre-authentication CORS lookups resident-only so attacker-controlled path segments cannot trigger metadata I/O or grow the metadata cache. Preserve fail-closed behavior for startup, load failures, invalid metadata, and the internal namespace.

Centralize replication request trust after authentication, distinguish general replication from replica-only privileges, and gate SSE-C ciphertext handling, source metadata, object-lock bypasses, event suppression, delete semantics, and replica status on the appropriate permission. Add least-privilege, multipart, PostPolicy, CORS amplification, and compatibility regressions.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-01 20:50:08 +08:00
Feng Ruohang 74c97d005d fix: authorize SSE-C attribute reads by replication permission
GetObjectAttributes lets a replication peer read SSE-C attributes without
presenting the customer key. That carve-out was keyed on the
X-Minio-Source-Replication-Request header alone, which any client can set,
so a caller holding only s3:GetObject could read an SSE-C object's ETag,
plaintext size, part list, and checksums without the key -- the response was
byte-identical to one made with the correct key.

Gate the carve-out on s3:ReplicateObject for the target object, mirroring
CopyObjectHandler's existing replication check.

The pre-existing test asserted the carve-out with root credentials, which
hold every action and therefore cannot tell a gated check apart from an
ungated one. Add least-privilege cases that do: a reader without
s3:ReplicateObject is now refused, a caller that holds it keeps the
carve-out, and the ordinary key-bearing path is unchanged.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-30 15:50:24 +08:00
Feng Ruohang 0507c3d56a Merge remote-tracking branch 'origin/main' into codex/cors-hotpath 2026-08-29 20:02:51 +08:00
Feng Ruohang ca1f11b708 Merge remote-tracking branch 'origin/main' into codex/issue-77-status-accounting 2026-08-29 19:50:27 +08:00
Feng Ruohang f8106cd693 Merge remote-tracking branch 'origin/main' into codex/issue-78-preserve-bucket-config 2026-08-29 19:39:50 +08:00
Feng Ruohang b1ba685ac5 Merge pull request #89 from pgsty/codex/issue-76-object-lock-wire
fix: replicate object lock config in its own field
2026-08-29 19:39:22 +08:00
Feng Ruohang f377b5c95b Merge remote-tracking branch 'origin/main' into codex/issue-50-after-checksum-contract
# Conflicts:
#	cmd/erasure-multipart-fullobject_test.go
2026-08-29 19:28:06 +08:00
Feng Ruohang 20aaefa89c Merge remote-tracking branch 'origin/main' into codex/reject-unsupported-checksums 2026-08-29 19:14:51 +08:00
Feng Ruohang f2ba439022 Merge remote-tracking branch 'origin/main' into codex/get-object-attributes-ssec 2026-08-29 19:04:14 +08:00
Feng Ruohang 2ba1f38503 Merge pull request #87 from pgsty/codex/issue-82-after-83
fix: authenticate SSE-C keys on zero-byte reads
2026-08-29 19:03:57 +08:00
Feng Ruohang 8d58343263 Merge pull request #96 from pgsty/codex/crc64-completion-after-foundation
fix: reject composite CRC64NVME completion
2026-08-29 18:51:09 +08:00
Feng Ruohang 32b2aa49f1 fix: reject composite CRC64NVME completion
Remove the remaining type-only canonicalization at CompleteMultipartUpload while preserving legacy uploads stored as FULL_OBJECT.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 18:36:02 +08:00
Feng Ruohang 8448512a1f docs: clarify layered SSE-C key authentication
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 18:21:13 +08:00
Feng Ruohang 150e7b5f9e fix: preserve global CORS response semantics
Guard only the per-bucket metadata lookup, then retain the global handler Vary and originless preflight behavior for non-CORS traffic.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 18:19:57 +08:00
Feng Ruohang 21870fa2e7 fix: preserve replicated object attributes reads
Keep the existing trusted replication carve-out while authenticating ordinary SSE-C GetObjectAttributes requests.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 18:18:01 +08:00
Feng Ruohang d4c8da162b fix: reject composite CRC64NVME trailers
Apply the full-object-only rule to declared streaming checksum trailers and cover the HTTP mutation path.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 18:15:51 +08:00
Feng Ruohang fc7bf7b295 test: keep CORS hot-path coverage route-neutral
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 17:58:30 +08:00
Feng Ruohang d28885d0e5 fix: reject composite CRC64NVME checksums
Return InvalidArgument for CRC64NVME with COMPOSITE at multipart initiation and PutObject instead of silently canonicalizing the request to FULL_OBJECT.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 17:32:36 +08:00
Feng Ruohang c4fd97d0bf test: align zero-byte rotation key errors
After zero-byte reads authenticate SSE-C at the object layer, equal invalid rotation keys match non-empty reads and return AccessDenied before the rotation-specific distinction.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 17:32:25 +08:00
Feng Ruohang b73581b05d fix: authenticate SSE-C keys on zero-byte reads
Unseal supplied SSE-C and copy-source keys after range and request preconditions when a zero-byte read cannot build a decryptor. Preserve internal no-decryption, replication, restore, and absent-header reads.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 17:32:25 +08:00
Feng Ruohang 474cd5801e fix: authenticate SSE-C for object attributes
Unseal the supplied customer key after request preconditions before returning object size, ETag, checksum, or part attributes. Cover zero-byte and non-empty objects with correct, wrong, and missing keys.\n\nRefs #84.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 17:26:37 +08:00
Feng Ruohang fb406fdc94 fix: report site replication metadata per site
Count only each site own valid bucket metadata, populate quota totals, and keep malformed fields from suppressing unrelated bucket statistics. Emit bounded diagnostics for invalid payloads.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 17:25:11 +08:00
Feng Ruohang dd3bdb8086 fix: preserve bucket configs during site adoption
Keep existing Object Lock and enabled versioning documents and timestamps when adopting a same-name bucket. Bootstrap missing configs, enable suspended versioning, and retain custom excluded-prefix settings.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 17:24:27 +08:00
Feng Ruohang 7c103389f5 fix: reject unsupported checksum assertions
Reject unimplemented x-amz-checksum value and trailer names instead of accepting uploads without verification. Apply the same contract to PutObject, multipart initiation and parts, CopyObject, and UploadPartCopy while preserving the five supported algorithms.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 17:23:26 +08:00
Feng Ruohang 3861f33cba fix: replicate object lock config in its own field
Populate ObjectLockConfig for live, initial-sync, and heal events, while accepting the legacy Tags field during rolling upgrades. Exercise signed admin dispatch and remote-heal transport on both object-layer backends.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 16:54:00 +08:00
Feng Ruohang 07d92db523 fix: skip bucket CORS lookup without an origin
Bypass per-bucket metadata work for non-CORS traffic, including admin and Console routes. Keep operational metadata errors fail-closed and pin the existing global fallback for missing buckets.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 16:34:51 +08:00
Feng Ruohang 5732930102 test: cover null-version copy rewrite directions
Exercise the silent compression-on-copy path, compressed SSE-C re-encryption, and the equal-invalid-key error contract on both object-layer backends.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 16:28:42 +08:00