fix: retransmit existing SSE-C replicas instead of metadata-copying them

The replication sender's target HEAD carries no SSE-C customer key, so for
an SSE-C object the target answers 400 and replicateAll fell into a
metadata-only CopyObject that fails on any non-empty SSE-C object (the
undecryptable source checksum makes the target recompute one and rewrite
the data with a plaintext-sized reader). Once a non-empty SSE-C replica
existed, tag, retention and legal-hold changes never reached it, a heal
never retransmitted, and a resync neither repaired the replica nor
counted it correctly. Forcing a full retransmit alone was not enough:
checkPreconditionsPUT rejects a write whose PreserveETag and VersionID
match the stored version, only the single-part sealed ETag is truncated
before that comparison, so a multipart SSE-C retransmit answered 412,
which the sender turns into success. Inherited from upstream ad04afe38.

Select replicateAll when the SSE-C HEAD cannot answer (the two previous
assignments were dead: rAction still forced the metadata path), exempt an
authenticated replica write that carries an SSE-C seal from the duplicate
version and ETag rejection (the predicate is the incoming write's
restored SSE-C metadata, not what the destination holds), and send the
internal replication marker on the resync accounting HEAD for SSE-C
objects so a peer answers with the replica metadata instead of 400.

Tests: TestAPISSECReplicaRetransmitOverExistingVersion (multipart replica
initiation over the same version and ETag answered 412 on main, now 200
with parts sent and plaintext readback; single-part and zero-byte writes
unchanged), TestAPISSECReplicaWriteExemptionIsKeyedOnTheIncomingWrite
(plaintext replica over an SSE-C version still 412; SSE-C replica over a
plaintext version exempted and readable), and
TestAPISSECReplicationTargetHead (keyless HEAD 400, missing key 404,
marked HEAD 200 with metadata, metadata CopyObject ExcessData on a
non-empty object) on ErasureSD and Erasure. Compatibility: every update
of an SSE-C object now retransmits its bytes; a peer that rejects the
internal marker fails the accounting HEAD as before; the #109 destination
fix must be deployed first or a retransmitted replica is transformed
again.

Fixes pgsty/silo#120

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
This commit is contained in:
Feng Ruohang
2026-09-05 15:23:49 +08:00
parent c201148738
commit 87746913fc
3 changed files with 821 additions and 9 deletions
+17 -8
View File
@@ -872,19 +872,25 @@ func putReplicationOpts(ctx context.Context, sc string, objInfo ObjectInfo) (put
if cc, ok := lkMap.Lookup(xhttp.CacheControl); ok {
putOpts.CacheControl = cc
}
if mode, ok := lkMap.Lookup(xhttp.AmzObjectLockMode); ok {
rmode := minio.RetentionMode(mode)
putOpts.Mode = rmode
mode, hasMode := lkMap.Lookup(xhttp.AmzObjectLockMode)
retainDateStr, hasRetainDate := lkMap.Lookup(xhttp.AmzObjectLockRetainUntilDate)
if hasMode {
putOpts.Mode = minio.RetentionMode(mode)
}
if retainDateStr, ok := lkMap.Lookup(xhttp.AmzObjectLockRetainUntilDate); ok {
// A removed retention is stored as an empty mode and date; it is sent as
// a value-less update that still carries its ordering timestamp.
if hasRetainDate && retainDateStr != "" {
rdate, err := amztime.ISO8601Parse(retainDateStr)
if err != nil {
return putOpts, false, err
}
putOpts.RetainUntilDate = rdate
}
if hasMode || hasRetainDate {
// set retention timestamp in opts
retTimestamp := objInfo.ModTime
if retainTmstampStr, ok := objInfo.UserDefined[ReservedMetadataPrefixLower+ObjectLockRetentionTimestamp]; ok {
var err error
retTimestamp, err = time.Parse(time.RFC3339Nano, retainTmstampStr)
if err != nil {
return putOpts, false, err
@@ -1520,11 +1526,14 @@ func (ri ReplicateObjectInfo) replicateAll(ctx context.Context, objectAPI Object
return rinfo
}
} else {
// SSEC objects will refuse HeadObject without the decryption key.
// Ignore the error, since we know the object exists and versioning prevents overwriting existing versions.
// The sender holds no customer key, so the target refuses HeadObject on
// an SSE-C object and the replica cannot be compared. The metadata-only
// CopyObject that a replicateMetadata action would run then fails on any
// non-empty object, because the undecryptable source checksum makes the
// target recompute one and rewrite the data. A full retransmit is the
// only action that completes.
if isSSEC && strings.Contains(cerr.Error(), errorCodes[ErrSSEEncryptedObject].Description) {
rinfo.ReplicationStatus = replication.Completed
rinfo.ReplicationAction = replicateNone
rAction = replicateAll
goto applyAction
}
// if target returns error other than NoSuchKey, defer replication attempt