Commit Graph

12834 Commits

Author SHA1 Message Date
Feng Ruohang e7d0e62f16 Merge pull request #135 from pgsty/fix/attributes-encrypted-parts-test-reconcile
test: reconcile encrypted-parts attributes test with #119 write validation
2026-09-06 09:59:54 +08:00
Feng Ruohang aee290fc34 test: reconcile encrypted-parts attributes test with #119 write validation
TestAPIGetObjectAttributesEncryptedPartLengths (from #128) built its
fixtures by PutObjectPart-ing plaintext bodies under encrypted-object
metadata with per-part sizes 5245473 and 1. Since #119, PutObjectPart
always derives an encrypted part's plaintext length from the bytes
written and rejects a part that cannot be a valid sio stream, so those
fixtures can no longer be created through a normal write and both
variants failed at write time.

Such an on-disk shape now only exists as pre-#119 data or from an old
peer, which is exactly the state the GetObjectAttributes per-part
tamper check (#128) defends. Inject that ObjectInfo directly through a
stub object layer (the setObjectLayer pattern used by the #110 tamper
test) and exercise the handler, which is what this test pins. The
handler path, the crafted part sizes, and both assertions
(separately-encrypted-parts -> ErrObjectTampered; legacy-single-stream
-> stored fragment sizes) are unchanged. Test-only; reconciles two
already-merged correct changes (#119 and #128).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-06 09:51:09 +08:00
Feng Ruohang 62cce2b152 Merge pull request #121 from pgsty/fix/issue-110-tampered-status
fix: return 500 for unreadable objects instead of 206
2026-09-06 09:26:28 +08:00
Feng Ruohang 65d4806a7b Merge pull request #127 from pgsty/fix/issue-112-bucket-metadata-cors
fix: include per-bucket CORS in bucket metadata export and import
2026-09-06 09:26:21 +08:00
Feng Ruohang 765757473a Merge pull request #126 from pgsty/fix/issue-118-no-compressed-ssec
fix: exclude SSE-C objects from compression
2026-09-06 09:26:12 +08:00
Feng Ruohang 425bd7fff1 Merge pull request #124 from pgsty/fix/issue-119-ssec-part-actual-size
fix: record plaintext part sizes for replicated SSE-C multipart parts
2026-09-06 09:26:04 +08:00
Feng Ruohang e12e739a53 Merge pull request #128 from pgsty/fix/issue-114-115-object-attributes-parts
fix: report logical part sizes and end pagination correctly in GetObjectAttributes
2026-09-06 09:25:28 +08:00
Feng Ruohang 8b736dee34 Merge pull request #123 from pgsty/fix/issue-113-rotation-checksum-algorithm
fix: honor a requested checksum algorithm on SSE-C key rotation
2026-09-06 09:25:02 +08:00
Feng Ruohang 32e75c27bf Merge pull request #122 from pgsty/fix/issue-109-raw-ssec-replica
fix: store raw SSE-C replicas verbatim on the destination
2026-09-06 09:24:26 +08:00
Feng Ruohang 885ca604a1 Merge pull request #129 from pgsty/fix/issue-111-object-lock-replica-ordering
fix: order value-less replicated Object Lock updates by timestamp
2026-09-06 09:23:34 +08:00
Feng Ruohang d10382d0dc build: refresh rebrand compatibility baseline for the SSE-C replica helper
The raw SSE-C seal headers moved from an inline slice in
cmd/object-multipart-handlers.go to the shared isRawSSECReplica helper
in cmd/replication-trust.go (already recorded), so regenerate the
compat-baseline allowlist to drop the three stale object-multipart
entries. No behaviour change; make rebrand-guard is green.

Refs pgsty/silo#109

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-06 09:08:22 +08:00
Feng Ruohang b2dca43fda fix: order value-less replicated Object Lock updates by timestamp
CopyObjectHandler rebuilt the destination metadata with the public Object Lock keys stripped (cmd/object-handlers.go:1708) and then restored a value only inside retentionMode.Valid() and legalHold.Status.Valid() (cmd/object-handlers.go:1715 and :1732), so a replica update that carried no retention or legal-hold value never reached the ordering comparison and silently erased whatever the destination held, however new it was; a retention removal that did win recorded no ordering timestamp either, so cmd/bucket-object-lock.go:370 later read an unparseable stored timestamp and let an older retained value back in. Each replica field is now decided on its source timestamp first and its incoming value second, and both restore helpers write the stored timestamp back before returning early on an empty stored value, which is the only way a removal timestamp survives the REPLACE metadata directive. Legal hold stays deliberately asymmetric: S3 has no legal-hold removal, an explicitly empty status is already rejected as invalid, and an absent status conveys no change even when an orphaned timestamp arrives with it, so only a valid ON or OFF can win.

Three inherited defects would have defeated that ordering, so they are fixed here too. The SSE-KMS branch of putOptsFromHeaders built its own ObjectOptions and dropped the parsed lock timestamps, leaving every replicated lock update unordered on a bucket with default KMS encryption; it now carries them. The in-place SSE-C key rotation snapshots the stored reserved metadata into encMetadata before the lock decision exists and merges it back afterwards to preserve the encryption headers, reinstating the ordering timestamp the decision had just replaced; the snapshot is now reconciled with the decision for a trusted replica. Finally, the value-less handling applies only to an actual replica: a trusted peer that sends the replication marker without REPLICA status keeps the previous behaviour, so a REPLACE copy carrying no lock headers still writes a version with no retention and no hold.

Tests: TestAPICopyObjectReplicaAbsentLockFieldsPreserveNewerState, TestAPICopyObjectReplicaRetentionRemovalKeepsOrderingTimestamp, TestAPICopyObjectReplicaObjectLockOrdering, TestAPICopyObjectReplicaRetentionRemovalUnderBucketKMS, TestAPICopyObjectReplicaLockTimestampSurvivesSSECKeyRotation and TestAPICopyObjectMarkerOnlyLeavesObjectLockUnchanged, all on ErasureSD and Erasure. Compatibility: no API, wire or stored-field change, and a field arriving with no source timestamp is unordered and now preserves destination state, so an un-upgraded 0806 peer keeps replicating safely while it still runs the old erasing receiver.

Fixes pgsty/silo#111

Signed-off-by: Feng Ruohang <rh@vonng.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-05 16:58:11 +08:00
Feng Ruohang 33a91d972f fix: carry per-bucket CORS through bucket metadata export/import
The admin bucket-metadata handlers enumerate every bucket config by name, and
per-bucket CORS was never added to that enumeration: export omitted cors.xml
(cmd/admin-bucket-handlers.go:414 cfgFiles) and import ignored the entry
outright, with no case in applyImportedBucketMetadata (:598) or SetStatus
(:629), so a CORS-only archive reported 0/0 buckets imported and a restored
bucket silently lost its configuration. Export now writes the stored document
verbatim and import validates it with the same parser and validator as
PutBucketCorsHandler, merging it under the existing bucket metadata lock and
announcing it through the dedicated SRBucketMetaTypeCorsConfig event; the local
CORS timestamp rule is extracted into localCORSUpdatedAt and reused so an
imported document always lands strictly above bucket creation, which matters
because the import stamps its fields before creating any missing bucket and a
CORS event below Created is dropped as an older bucket incarnation.

The import reads one byte past the declared entry size so archive/zip reaches
EOF and verifies the entry checksum, otherwise a corrupt or over-long entry
carrying well formed XML would overwrite the stored document; and the CORS
event is sent even when the shared bucket metadata hook failed, so an
unreachable peer cannot withhold an already committed CORS document from the
reachable ones.

Tests: TestAdminBucketMetadataCORSRoundTrip and
TestAdminBucketMetadataCORSImportReplicatesPastPeerFailure (new, ErasureSD and
Erasure).
Compatibility: the ZIP gains one entry, older archives stay importable and
leave CORS untouched; no mcli or madmin-go change is needed because
madmin.BucketStatus already carries Cors and mcli copies the export ZIP
verbatim.

Fixes pgsty/silo#112

Signed-off-by: Feng Ruohang <rh@vonng.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
2026-09-05 16:21:12 +08:00
Feng Ruohang 2cbd48a3c3 fix: end GetObjectAttributes part pagination correctly
GetObjectAttributes decided truncation by comparing the last returned
part number with the part count (cmd/object-handlers.go:720), which is
only a coincidence of contiguous numbering. Sparse parts 1/3 reported a
complete page as truncated with a marker that loops, and parts 1/3/5
with max-parts=1 stopped after part 3 and silently dropped part 5. Set
IsTruncated in the break that proves an eligible part was left
unreturned, and zero NextPartNumberMarker when the listing is complete,
as ListObjectParts already does. Also reject negative x-amz-max-parts
and x-amz-part-number-marker in getAndValidateAttributesOpts with the
same API errors ListObjectParts uses, instead of answering an invalid
request with an empty parts listing; an absent or zero max-parts still
means the default page size.

Tests: TestAPIGetObjectAttributesPartsPagination (sparse 1/3/5 and
contiguous 1/2 walks on ErasureSD and Erasure),
TestGetAndValidateAttributesOptsPartsRange, and the sparse variants of
TestAPIGetObjectAttributesMultipartLogicalPartSize. Compatibility: no
field is added or removed; IsTruncated and NextPartNumberMarker change
only where they were wrong, and negative pagination values that no SDK
sends now fail fast.

Fixes pgsty/silo#115

Signed-off-by: Feng Ruohang <rh@vonng.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
2026-09-05 16:17:39 +08:00
Feng Ruohang b5409ca112 fix: report logical part sizes in GetObjectAttributes
GetObjectAttributes filled ObjectPart.Size from the on-disk part length
(cmd/object-handlers.go:715), so every compressed or encrypted multipart
object reported transformed sizes that do not sum to the logical
ObjectSize the same response returns from objInfo.GetActualSize().
Report each part's uploaded plaintext length instead: a compressed part
uses its recorded ActualSize, and a separately encrypted part derives the
plaintext length with sio.DecryptedSize, because ActualSize is the
ciphertext length for a replicated SSE-C part and zero for parts written
before actualSize existed.

Only parts of an encrypted multipart object are streams of their own. A
legacy encrypted object carries no multipart marker and is one continuous
stream that the erasure writer split into storage fragments, so those
fragments keep their stored size. Where a part is a stream, one whose
length cannot be a valid encrypted stream has no logical length, and the
request now fails with XMinioObjectTampered rather than reporting the
ciphertext length; DecryptObjectInfo does not catch that case, because
ObjectInfo.isMultipart gives up on the first bad part and only the object
total is then validated.

Tests: TestAPIGetObjectAttributesMultipartLogicalPartSize (plain,
compressed, SSE-C and compressed+SSE-C, consecutive and sparse part
numbers), TestAPIGetObjectAttributesCompressedEmptyTrailingPart,
TestAPIGetObjectAttributesEncryptedPartLengths, and a part-size assertion
added to TestAPISSECMultipartReplicationTrust. Compatibility: the XML
shape is unchanged and nothing is written to disk, only the value of the
existing Size element is corrected.

Fixes pgsty/silo#114

Signed-off-by: Feng Ruohang <rh@vonng.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
2026-09-05 16:17:39 +08:00
Feng Ruohang 35bd75948a fix: exclude SSE-C objects from compression
With compression allow_encryption=on an SSE-C object is stored as
encrypt(s2(plaintext)), while replication reads it raw (NoDecryption at
cmd/erasure-object.go:257) and putReplicationOpts drops the internal
compression and actual-size headers (cmd/bucket-replication.go:786). The
replica keeps the source seal with no compression marker, so a GET with the
correct customer key returns HTTP 200 and the raw S2 stream instead of the
object, and the source records the transfer as COMPLETED.

Widen the one condition in excludeForCompression (cmd/object-api-utils.go:613)
so SSE-C is never compressed, whatever allow_encryption says. This covers all
four producers at once, PutObject, NewMultipartUpload, CopyObject and
PutObjectExtract, plus any future caller of isCompressible.
crypto.SSEC.IsRequested ignores copy-source headers, so a copy is judged on its
destination key only, and a raw SSE-C replica write is unaffected because it
carries no public SSE-C headers. allow_encryption keeps its meaning for SSE-S3
and SSE-KMS, where the server owns the key and decompresses before replicating.

Tests: TestAPISSECCompressionReplicaStaysReadable (single PUT and multipart),
TestAPISSECCompressionProducerMatrix, TestAPISSECCompressionSkippedOnCopyObject,
TestAPISSECCompressionSkippedOnSnowballExtract and the control
TestSSECBatchReplicationCannotRead in cmd/compression-ssec_test.go. Two existing
expectations pinned the removed shape and are updated:
TestAPICopyObjectSSECKeyRotationNullVersionCompressesRewrite is renamed
TestAPICopyObjectSSECKeyRotationNullVersionSkipsCompression and now expects an
uncompressed rewrite, keeping its body, checksum, version and ETag assertions;
the SSE-C compressed-encrypted variant of
TestAPICopyObjectServerSideChecksumEncryption becomes compressible-extension and
expects an uncompressed destination, its SSE-S3 sibling keeping the compressed
coverage.

Compatibility: a deliberate behaviour change. Deployments with
allow_encryption=on no longer store new or rewritten SSE-C data compressed, so
those writes cost more space; objects already stored compressed keep working on
the source and are the concern of pgsty/silo#109, which rejects them at
replication time. Multipart uploads initiated before this change keep
compressing their parts from the metadata saved at initiation. Upstream
468a9fae8 refused this combination at PUT time and a2cab0255 removed the guard;
upstream master is still unguarded, so this is a deliberate divergence.

Fixes pgsty/silo#118

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-05 15:51:24 +08:00
Feng Ruohang 0c8d74205b fix: record plaintext part sizes for replicated SSE-C multipart parts
Trusted SSE-C replication uploads parts as raw ciphertext with the
ciphertext length as Content-Length, and erasureObjects.PutObjectPart only
derived the plaintext length when the caller passed a negative size, so
each replicated part persisted the ciphertext length as ActualSize (the
field defined as the uploaded size without encryption bytes). On the
replica, partNumberToRangeSpec turned those lengths into a plaintext range,
so GET/HEAD ?partNumber=N returned the wrong bytes and shifted
Content-Range (2560, 2560 and 5120 bytes for 5 MiB, 5 MiB and 1 MiB
parts), and a later decommission or rebalance re-uploaded the parts with
the stale value and recomputed the object-level actual-size from their sum,
after which a whole-object GET advertised a Content-Length larger than the
body it wrote.

Derive the plaintext length of an encrypted, uncompressed part from the
bytes actually written (sio.DecryptedSize) in PutObjectPart, the single
place a part is persisted, rejecting a length that cannot be a valid
stream before the part is committed; and derive part lengths from
part.Size in partNumberToRangeSpec for encrypted, uncompressed objects,
returning an error instead of a nil range, so replicas already on disk
read correctly without a resync. Compressed parts keep ActualSize.

Tests: TestAPISSECReplicaPartNumberReads (three-part SSE-C replica,
?partNumber=N bytes, Content-Length and Content-Range equal the source)
and TestSSECReplicaPartActualSizeDataMovement (replay through the data
movement path leaves part and object sizes at plaintext values) fail on
main and pass with the fix on ErasureSD and Erasure;
TestAPIGetObjectWithPartNumberHandler, TestAPISSECMultipartReplicationTrust
and TestAPIListObjectPartsHandler stay green. Compatibility: no wire or
API change; objects an unfixed server already moved carry a poisoned
object-level actual-size and need a rewrite or resync.

Fixes pgsty/silo#119

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-05 15:33:23 +08:00
Feng Ruohang fcc4d77895 fix: honor a requested checksum algorithm on SSE-C key rotation
An in-place SSE-C key rotation takes the fast path at cmd/object-handlers.go:1523
that only rewraps the object key, while every line that turns
x-amz-checksum-algorithm into a stored checksum lives in the re-encrypting else
branch at 1571-1609, so a requested algorithm was silently dropped and the stale
source checksum was kept and reported. Extend the canRotateKeyInPlace guard so a
client request carrying the header falls through to the copy that recomputes,
stores and reports it. Replica-trusted requests keep the fast path: getOpts
leaves their source reader encrypted, so a rewrite would hash ciphertext, and a
replica has to keep the checksum its source assigned.

Tests: TestAPICopyObjectSSECKeyRotationChecksumAlgorithm (new, red before the
guard), TestAPICopyObjectSSECKeyRotationKeepsChecksumAbsence (new, pins the
accepted limitation that a headerless rotation preserves the stored checksum
state including absence, gaining no default CRC64NVME) and
TestAPICopyObjectSSECKeyRotationReplicaKeepsFastPath (new, pins the replica
carve-out on a non-empty and on a zero byte source).
Compatibility: no API or wire change; a rotation without the header and every
replica-trusted rotation are unchanged, while a client rotation carrying the
header now rewrites the object data, so the ETag changes, a multipart source
collapses to a single part object, the copy replicates as an object rather than
as metadata, and the rewritten bytes are compressed if compression is enabled for
that object, as AWS CopyObject documents. Upstream MinIO carries the same
defect from 2718d9a43 (minio/minio#21399); this is a deliberate divergence.

Fixes pgsty/silo#113

Signed-off-by: Feng Ruohang <rh@vonng.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
2026-09-05 15:25:08 +08:00
Feng Ruohang c52acc1a5d fix: store raw SSE-C replicas verbatim on the destination
A raw SSE-C replica write carries the source ciphertext and the source seal
in X-Minio-Replication-Server-Side-Encryption-* headers but no public SSE-C
request headers, so crypto.Requested() was false and PutObjectHandler and
NewMultipartUploadHandler applied the destination's default encryption and
compression to bytes that were already ciphertext (upstream 468a9fae8,
"Enable replication of SSE-C objects", never exempted the raw path). With
destination default SSE-S3 the replica's IV and seal were overwritten and
GET returned 400; with destination compression the replica stored
compress(ciphertext) and GET failed, while the source reported COMPLETED.

Recognize a validated raw SSE-C replica (replicaTrusted plus a seal header,
shared helper isRawSSECReplica) and skip bucket default encryption,
compression and the encryption branch on the single PUT path, and default
encryption plus compression on the multipart initiation path, which already
skipped key generation. On the sender, reject replication of an object that
is both compressed and SSE-C, since the wire carries no compression state
and the destination would otherwise store an undetectable S2 stream, and
make replicateObject/replicateAll report a putReplicationOpts failure as
Failed instead of Completed.

Tests: TestAPISSECReplicaSkipsDestinationTransforms (single PUT and
multipart under destination default SSE-S3, compression and an explicit SSE
header, plus an untrusted control), TestAPISSECMultipartReplicaRoundTripWith
Compression, and TestPutReplicationOptsRejectsCompressedSSEC fail on main
and pass with the fix on ErasureSD and Erasure; the replication-trust,
multipart and PutObject suites stay green. Compatibility: no wire, API or
metadata change; the destination change applies only to trusted replica
writes carrying a source seal; replicas already transformed must be
rewritten from an intact source (see pgsty/silo#120 for why a resync does
not do that yet).

Fixes pgsty/silo#109

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-05 15:07:22 +08:00
Feng Ruohang 5703426b3c fix: return 500 for unreadable objects instead of 206
ErrObjectTampered was mapped to http.StatusPartialContent since upstream
ca6b4773e (2017), so a GET or HEAD of an object the server cannot decode
(invalid encrypted size, malformed actual-size, bad multipart ETag shape)
answered with a success status and an XML error document that SDKs handed
back as object content; boto3 returned the XML as Body and a zero-length
HEAD as success. Every origin of errObjectTampered is a stored-state
defect, not caller input, so map the entry to 500 Internal Server Error
and keep the XMinioObjectTampered code and message. The comment records
the deliberate divergence from upstream.

Tests: TestObjectTamperedGETHEADStatus (signed GET and HEAD, ErasureSD and
Erasure) fails with 206 on main and passes with 500; TestAPIErrCode,
TestAPIErrCodeDefinition, TestAPIHeadObjectHandler,
TestAPIHeadObjectHandlerWithEncryption and TestAPIGetObjectHandler stay
green. Compatibility: only the status line of one MinIO-specific error
changes; clients now retry damaged-object reads per their 5xx policy.

Fixes pgsty/silo#110

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-05 14:50:00 +08:00
Feng Ruohang f0bd164b92 docs: refresh community contributors
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-04 19:21:26 +08:00
Feng Ruohang 9936a69d89 ci: recover container publication from verified component pins
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-04 15:10:43 +08:00
Feng Ruohang ce2326c946 build: pin the published mcli 20260903 archives
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-04 14:38:54 +08:00
Feng Ruohang 4c164907f5 build: align Helm defaults with the published release tag
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-04 14:36:12 +08:00
Feng Ruohang 9b11dc9469 build: converge the final 20260903 dependency graph
Adopt silo-pkg v3.13.2, the 20260903 mcli release, and the latest validated SILO Console commit. Move maintained source imports to the pgsty/silo-pkg module path, refresh the resulting dependency closure and credits, and keep only documented legacy minio/pkg transitive residue.

Signed-off-by: Feng Ruohang <rh@vonng.com>
RELEASE.2026-09-03T13-18-01Z
2026-09-03 20:54:40 +08:00
Feng Ruohang 1eccc6908b Merge branch 'codex/ci-fixes': CI repairs for the 20260903 release
Flush S3 Select records queued before an error message (an inherited select
race that the CVE-2026-39414 regression test caught under the race detector
in CI), update golang.org/x/crypto to v0.56.0 for GO-2026-6354/6355 reachable
through the SFTP server, move the toolchain to Go 1.27.1, and extend the
message writer tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QwirCh4nsrJCQp2WaXoVLK
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-03 07:09:11 +08:00
Feng Ruohang 202371afbd docs: record the x/crypto SSH fixes and Go 1.27.1 in the advisory ledger
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QwirCh4nsrJCQp2WaXoVLK
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-03 06:53:09 +08:00
Feng Ruohang 41ef4411d9 build: move to Go 1.27.1
Go 1.27.1 (2026-09-01) carries fixes to the runtime, compiler, net/http,
encoding/json, os and database/sql. The go directive, the dependency check
script and the image build stage follow it; CI reads the version from
go.mod. The remaining golang.org/x modules the server requires are already
at their latest releases.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QwirCh4nsrJCQp2WaXoVLK
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-03 06:52:33 +08:00
Feng Ruohang 035aa6c201 test: cover the S3 Select message writer ordering
Exercise the writer directly and decode the event stream with the client
parser: a buffered plus a queued record before an error, an error with
nothing queued, a record larger than the staging buffer on both the success
and the error path, and the unchanged success ordering of records, Stats and
End.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QwirCh4nsrJCQp2WaXoVLK
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-03 06:52:33 +08:00
Feng Ruohang edf36bcbfa deps: update golang.org/x/crypto to v0.56.0
GO-2026-6354 and GO-2026-6355 (denial of service on deadlocked SSH channels)
are reachable through the SFTP server, which listens with x/crypto/ssh.
v0.56.0 carries the fixes; govulncheck reports no reachable vulnerability.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QwirCh4nsrJCQp2WaXoVLK
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-03 06:43:49 +08:00
Feng Ruohang 3598c4305d fix: flush records queued before an S3 Select error message
The writer goroutine selects between the record channel and the error
channel at random. When Evaluate had just queued a record and then reported
an error, the error case could win first; it flushed only the staging
buffer, so the queued record was dropped in the exit drain and the client
saw the error without the records that preceded it. Stage whatever is queued
before flushing and writing the error. The CVE-2026-39414 regression test
asserted this contract and failed once under the race detector in CI; a new
unit test exercises the ordering directly.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QwirCh4nsrJCQp2WaXoVLK
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-03 06:43:49 +08:00
Feng Ruohang 94fbb6df6d fix: never record a resident bucket as a CORS load failure
A bucket that is resident keeps its last loaded metadata through a failed
refresh, exactly like every other bucket configuration, so the load-failure
set only ever holds buckets that were never loaded. This makes the CORS
lookup's precedence explicit: resident document first, fail-closed only for
a bucket that has no loaded document.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-03 00:45:02 +08:00
Feng Ruohang f4c1286c9d fix: apply replicated Object Lock updates only when newer than the stored state
A replicated CopyObject uses the REPLACE metadata directive, so the map the
handler compared replication timestamps against had already been rebuilt
from the request and filtered of Object Lock keys: the stored retention and
legal-hold timestamps were never seen, every replica update was applied
regardless of order, and the legal-hold timestamp was written under the
retention key. A stale replica could turn a newer legal hold off or shorten
a newer retention.

Capture the stored Object Lock state before the metadata is rebuilt, apply a
replica update only when its source timestamp is newer, put the stored state
back when the update is stale, and keep each timestamp under its own key.
Inherited from upstream; recorded in the advisory ledger.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-03 00:45:02 +08:00
Feng Ruohang 6e112d1856 build: point Helm defaults at the 20260903 release
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 23:11:32 +08:00
Feng Ruohang e62dc6e023 Merge branch 'codex/prerelease-cleanup': pre-release cleanup and dependency pins
Fifteen reviewed commits: streaming-trailer trust fix and a single
replication-trust evaluator, Object Lock versioning invariant, resident-only
CORS lookup with fail-closed startup and load-failure states, dead-code
removal, compatibility baseline without exported symbols, upstream minio-go
with Console 43f8447fd and mcli 20260901, advisory ledger SN-2026-006..010,
contributor records, verification scripts with a valid root password, and a
deterministic dynamic-timeout test.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 21:34:30 +08:00
Feng Ruohang 84e1580a47 fix: clear the load-failure bit when GetConfig reloads a bucket on demand
A successful on-demand load makes the bucket resident, so the failure
recorded by an earlier startup or refresh no longer applies. Clearing it here
keeps the set's invariant exact instead of waiting for the next refresh.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 21:23:30 +08:00
Feng Ruohang ebac0ca73b test: give the dynamic timeout tests a private random source
TestDynamicTimeoutAdjustExponential and TestDynamicTimeoutAdjustNormal
seeded the global generator and then drew from it while other tests in the
package may use the same generator, so the sample was not the one the seed
promised and the exponential case failed once in a full race run. A private
source makes both tests deterministic.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 20:04:57 +08:00
Feng Ruohang 3f9c79e919 fix: keep load-failed buckets fail-closed in the CORS lookup
Restore the load-failure set that the cleanup removed. A presigned URL is
authenticated by its signature, so for such requests the bucket's CORS
document is the only origin boundary a browser enforces; a real bucket whose
metadata failed to load must therefore not be answered with the global
policy, and without this bit it is indistinguishable from a name that is not
a bucket. Two helpers own the set's lifecycle; the resident-only lookup and
the removal of the internal-namespace special case stay.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 19:51:54 +08:00
Feng Ruohang ec2979ca48 fix: keep the CORS lookup fail-closed until bucket metadata is loaded
Restore the startup guard removed by the previous cleanup: while bucket
metadata is still loading, a non-resident name may be a bucket with a
restrictive CORS document, so the request gets no CORS answer instead of the
global policy. After startup a non-resident name still falls back to the
global policy without any metadata I/O; the separate load-failure set stays
removed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 18:49:17 +08:00
Feng Ruohang 21646eebd2 fix: derive the Object Lock versioning rule from the parsed configuration
The load-time normalization compared the stored lock document with the
canonical enabled document byte for byte, so a lock configuration that also
carries a default retention rule kept a suspended or prefix-excluded
versioning document. Decide from the parsed configuration instead, after it
is parsed, so every writer that goes through Save, including the site
replication versioning and heal paths, ends with plain Enabled versioning on
a locked bucket. Receiving a lock configuration on a bucket created without
lock now enables versioning as well; the test that asserted the opposite is
updated, and a new test covers a rule-bearing lock document with suspended
and prefix-excluded versioning through Update, Get, and reload.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 18:49:16 +08:00
Feng Ruohang 5711996231 build: use an eight-character root password in the verification scripts
The rebrand shortened the scripts' root password from minio123 to silo123.
The server requires at least eight characters, so every script that starts
a server with it failed at startup and verify-build.sh then waited forever
on mc ready. No workflow runs these scripts, which is why it went unnoticed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 15:53:08 +08:00
Feng Ruohang 0079723d35 deps: return to upstream minio-go and embed Console 43f8447fd
The silo-go fork is retired: silo-pkg v3.13.0, Console v2.3.0, and mcli
20260901 all require upstream github.com/minio/minio-go/v7 again. Drop the
replacement and require the same upstream pre-release they use
(v7.3.1-0.20260828014306-0e78d3f18efe, one commit past the fork's base).

Console moves from e07ef01 (v2.2.1 plus pins) to 43f8447fd, the last commit
of the v2.3.0 line before Console adopted the github.com/pgsty/silo-pkg/v3
module path. It carries the six v2.3.0 security fixes (forwarding-header
trust, outbound TLS verification, credential redaction, WebSocket session and
connection caps), the IAM wildcard and session-identity fixes, and their
regenerated assets, while still consuming silo-pkg through the existing
replacement. mc stays on the last commit before its own path migration;
silo-pkg stays on the last commit that declares the github.com/minio/pkg/v3
path, which differs from v3.13.0 only by that path change. CREDITS follows
the module set.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 14:51:10 +08:00
Feng Ruohang bc3b35f975 build: track only served routes in the compatibility baseline
The baseline recorded 9,051 exported symbols of the main and internal
packages, which nothing outside this module can import, and 112 request
paths that exist only in test fixtures. Both changed with almost every
functional commit and protected no compatibility promise. The guard now
records routes from non-test files only, ignores untracked files, and drops
the symbol set; the baseline shrinks from 522 KB to 46 KB. CONTRIBUTING
explains when to refresh it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 14:08:11 +08:00
Feng Ruohang 00d864ed0e docs: record advisories SN-2026-006 to 010 and refresh contributors
Ledger entries for the zero-byte SSE-C key check, GetObjectAttributes
authentication, replication request trust, user and group status
authorization, and DeleteObjectVersion authorization, plus the Go 1.27
toolchain refresh. The ledger names pgsty/silo, CONTRIBUTORS lists the
per-bucket CORS, ChecksumType, and NoSuchBucket contributors, and the CORS
design record states its merged status without the review logs.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 14:08:11 +08:00
Feng Ruohang 8f2a30d9af build: bundle mcli 20260901 and describe the component pins
mcli RELEASE.2026-09-01 carries the credential redaction fixes for --debug
output. The go.mod comments now state what the Console and silo-pkg pins are:
the last commits that consume silo-pkg through the github.com/minio/pkg/v3
replacement, since silo-pkg v3.13.0 and Console v2.3.0 moved to the
pgsty/silo-pkg module path.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 14:06:56 +08:00
Feng Ruohang d9766d7378 chore: drop the wait_pipe lint exclusion and use gomodguard_v2
Assigning the two pipe halves before returning them removes the gofumpt and
gofmt disagreement that needed a permanent formatter exclusion. gomodguard is
deprecated in golangci-lint v2.12; the v2 linter takes the same configuration.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 14:06:56 +08:00
Feng Ruohang 5594d284fc chore: remove dead code left by earlier fixes
checkSSECCopySourceKey duplicated the source-key authentication that both
CopyObject read paths already perform; the DeleteObjects signature-error
branch became unreachable once signatures are verified once per request; the
discrete PostgreSQL and MySQL notification environment constants have had no
reader since DSNs became mandatory.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 14:06:56 +08:00
Feng Ruohang 632eb4729a refactor: serve non-resident buckets with the global CORS policy
The pre-authentication CORS lookup stays resident-only, so client-supplied
path segments still cause no metadata I/O and no cache growth. The
fail-closed states for startup, load failures, and the internal namespace
are gone: CORS is a browser response policy rather than an authorization
boundary, and failing closed only denied browser clients CORS headers while
bucket metadata was still loading. A bucket whose stored CORS document does
not parse still gets no CORS headers. This removes the loadFailed bookkeeping
and the unused GetCorsConfig, HasAllowedOrigin, and generic Update path for
CORS; tests use the CORS-specific writer.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 14:06:56 +08:00
Feng Ruohang 3b5de82f5a fix: keep plain Enabled versioning when Object Lock is enabled on a bucket
Site adoption and ForceCreate preserved a suspended or prefix-excluded
versioning configuration while bootstrapping Object Lock, persisting a state
that PutBucketVersioning itself rejects: objects under an excluded prefix in
a WORM bucket were not versioned and escaped retention. enablePeerBucketVersioning
now takes the lock intent and replaces such configurations with plain Enabled
versioning, and metadata loading ignores prefix exclusions on a locked bucket
as it ignored suspension before. The adoption tests assert the normalized
state and keep the timestamp-preservation checks on valid documents.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 14:06:56 +08:00
Feng Ruohang 76195f1c68 fix: keep streaming trailers visible after replication headers are stripped
A request that does not earn replication trust continues with a clone whose
internal replication headers are removed. r.Clone copies the Trailer map, but
the streaming body reader created from the original request fills the
original map, so a trailing checksum was never seen by the hash reader and
PutObject and UploadPart with STREAMING-UNSIGNED-PAYLOAD-TRAILER failed with
XAmzContentChecksumMismatch whenever an untrusted X-Minio-Source-* header was
present. Share the trailer map with the clone, as the Snowball path already
does for its per-entry requests, and cover both handlers with a test.

The marker evaluation that was copied into six handlers now lives in
evaluateReplicationTrust so the rule (a declared replica without the
replication permission is rejected; trust needs the exact marker plus the
permission) is defined once.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 14:06:56 +08:00