fix: record plaintext part sizes for replicated SSE-C multipart parts

Trusted SSE-C replication uploads parts as raw ciphertext with the
ciphertext length as Content-Length, and erasureObjects.PutObjectPart only
derived the plaintext length when the caller passed a negative size, so
each replicated part persisted the ciphertext length as ActualSize (the
field defined as the uploaded size without encryption bytes). On the
replica, partNumberToRangeSpec turned those lengths into a plaintext range,
so GET/HEAD ?partNumber=N returned the wrong bytes and shifted
Content-Range (2560, 2560 and 5120 bytes for 5 MiB, 5 MiB and 1 MiB
parts), and a later decommission or rebalance re-uploaded the parts with
the stale value and recomputed the object-level actual-size from their sum,
after which a whole-object GET advertised a Content-Length larger than the
body it wrote.

Derive the plaintext length of an encrypted, uncompressed part from the
bytes actually written (sio.DecryptedSize) in PutObjectPart, the single
place a part is persisted, rejecting a length that cannot be a valid
stream before the part is committed; and derive part lengths from
part.Size in partNumberToRangeSpec for encrypted, uncompressed objects,
returning an error instead of a nil range, so replicas already on disk
read correctly without a resync. Compressed parts keep ActualSize.

Tests: TestAPISSECReplicaPartNumberReads (three-part SSE-C replica,
?partNumber=N bytes, Content-Length and Content-Range equal the source)
and TestSSECReplicaPartActualSizeDataMovement (replay through the data
movement path leaves part and object sizes at plaintext values) fail on
main and pass with the fix on ErasureSD and Erasure;
TestAPIGetObjectWithPartNumberHandler, TestAPISSECMultipartReplicationTrust
and TestAPIListObjectPartsHandler stay green. Compatibility: no wire or
API change; objects an unfixed server already moved carry a poisoned
object-level actual-size and need a rewrite or resync.

Fixes pgsty/silo#119

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
This commit is contained in:
Feng Ruohang
2026-09-05 15:03:43 +08:00
parent f0bd164b92
commit 0c8d74205b
5 changed files with 843 additions and 21 deletions
+25 -5
View File
@@ -49,6 +49,7 @@ import (
xhttp "github.com/minio/minio/internal/http"
xioutil "github.com/minio/minio/internal/ioutil"
"github.com/minio/minio/internal/logger"
"github.com/minio/sio"
"github.com/pgsty/silo-pkg/v3/trie"
"github.com/pgsty/silo-pkg/v3/wildcard"
"github.com/valyala/bytebufferpool"
@@ -675,19 +676,35 @@ func getPartFile(entriesTrie *trie.Trie, partNumber int, etag string) (partFile
return partFile
}
func partNumberToRangeSpec(oi ObjectInfo, partNumber int) *HTTPRangeSpec {
func partNumberToRangeSpec(oi ObjectInfo, partNumber int) (*HTTPRangeSpec, error) {
if oi.Size == 0 || len(oi.Parts) == 0 {
return nil
return nil, nil
}
// For an encrypted, uncompressed object derive each part's plaintext length
// from the stored ciphertext length instead of trusting ActualSize: parts
// written before this was normalised record the ciphertext length there.
// The range returned here is consumed in the plaintext domain, where
// GetDecryptedRange and DecryptedSize both use exactly this arithmetic.
_, isEncrypted := crypto.IsEncrypted(oi.UserDefined)
deriveFromSize := isEncrypted && !oi.IsCompressed()
var start int64
end := int64(-1)
for i := 0; i < len(oi.Parts) && i < partNumber; i++ {
partSize := oi.Parts[i].ActualSize
if deriveFromSize {
decrypted, err := sio.DecryptedSize(uint64(oi.Parts[i].Size))
if err != nil {
return nil, errObjectTampered
}
partSize = int64(decrypted)
}
start = end + 1
end = start + oi.Parts[i].ActualSize - 1
end = start + partSize - 1
}
return &HTTPRangeSpec{Start: start, End: end}
return &HTTPRangeSpec{Start: start, End: end}, nil
}
// Returns the compressed offset which should be skipped.
@@ -806,7 +823,10 @@ func NewGetObjectReader(rs *HTTPRangeSpec, oi ObjectInfo, opts ObjectOptions, h
}
if rs == nil && opts.PartNumber > 0 {
rs = partNumberToRangeSpec(oi, opts.PartNumber)
rs, err = partNumberToRangeSpec(oi, opts.PartNumber)
if err != nil {
return nil, 0, 0, err
}
}
_, isEncrypted := crypto.IsEncrypted(oi.UserDefined)