astelm/minio
1
0
Fork 0
mirror of https://github.com/pgsty/minio.git synced 2026-09-25 20:35:58 +03:00
Code Issues Packages Projects Releases Wiki Activity

fix: exclude SSE-C objects from compression

Browse Source 
With compression allow_encryption=on an SSE-C object is stored as
encrypt(s2(plaintext)), while replication reads it raw (NoDecryption at
cmd/erasure-object.go:257) and putReplicationOpts drops the internal
compression and actual-size headers (cmd/bucket-replication.go:786). The
replica keeps the source seal with no compression marker, so a GET with the
correct customer key returns HTTP 200 and the raw S2 stream instead of the
object, and the source records the transfer as COMPLETED.

Widen the one condition in excludeForCompression (cmd/object-api-utils.go:613)
so SSE-C is never compressed, whatever allow_encryption says. This covers all
four producers at once, PutObject, NewMultipartUpload, CopyObject and
PutObjectExtract, plus any future caller of isCompressible.
crypto.SSEC.IsRequested ignores copy-source headers, so a copy is judged on its
destination key only, and a raw SSE-C replica write is unaffected because it
carries no public SSE-C headers. allow_encryption keeps its meaning for SSE-S3
and SSE-KMS, where the server owns the key and decompresses before replicating.

Tests: TestAPISSECCompressionReplicaStaysReadable (single PUT and multipart),
TestAPISSECCompressionProducerMatrix, TestAPISSECCompressionSkippedOnCopyObject,
TestAPISSECCompressionSkippedOnSnowballExtract and the control
TestSSECBatchReplicationCannotRead in cmd/compression-ssec_test.go. Two existing
expectations pinned the removed shape and are updated:
TestAPICopyObjectSSECKeyRotationNullVersionCompressesRewrite is renamed
TestAPICopyObjectSSECKeyRotationNullVersionSkipsCompression and now expects an
uncompressed rewrite, keeping its body, checksum, version and ETag assertions;
the SSE-C compressed-encrypted variant of
TestAPICopyObjectServerSideChecksumEncryption becomes compressible-extension and
expects an uncompressed destination, its SSE-S3 sibling keeping the compressed
coverage.

Compatibility: a deliberate behaviour change. Deployments with
allow_encryption=on no longer store new or rewritten SSE-C data compressed, so
those writes cost more space; objects already stored compressed keep working on
the source and are the concern of pgsty/silo#109, which rejects them at
replication time. Multipart uploads initiated before this change keep
compressing their parts from the metadata saved at initiation. Upstream
468a9fae8 refused this combination at PUT time and a2cab0255 removed the guard;
upstream master is still unguarded, so this is a deliberate divergence.

Fixes pgsty/silo#118

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
This commit is contained in:
Feng Ruohang
2026-09-05 15:21:22 +08:00
parent f0bd164b92
commit 35bd75948a
6 changed files with 825 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/compression/README.md
+7
View File
@@ -86,6 +86,13 @@ To enable compression+encryption use:
Or alternatively through the environment variable `MINIO_COMPRESSION_ALLOW_ENCRYPTION=on`.
SSE-C objects are excluded from compression even with `allow_encryption=on`.
Replication ships an SSE-C object as raw ciphertext, because the server never holds the
customer key, and the compression metadata is not carried over the wire. A compressed
SSE-C object would therefore replicate to a replica that decrypts to a compressed stream.
`allow_encryption` still applies to SSE-S3 and SSE-KMS, where the server owns the key and
decompresses before replicating.
### 4. Excluded Types
- Already compressed objects are not fit for compression since they do not have compressible patterns.
docs/site-replication/run-ssec-object-replication-with-compression.sh
+64 -11
View File
@@ -62,13 +62,14 @@ echo "Hello world" >/tmp/data/plainfile
echo "Hello from encrypted world" >/tmp/data/encrypted
touch /tmp/data/defpartsize
shred -s 500M /tmp/data/defpartsize
touch /tmp/data/mpartobj.txt
shred -s 500M /tmp/data/mpartobj.txt
# Compressible, and large enough for a multipart upload, so the object would be
# stored compressed if SSE-C were not excluded from compression.
yes "silo compression and sse-c replication payload" | head -c 100000000 >/tmp/data/mpartobj.txt
echo "done"
# Enable compression for site silo1
./mc admin config set silo1 compression enable=on extensions=".txt" --insecure
./mc admin config set silo1 compression allow_encryption=off --insecure
./mc admin config set silo1 compression enable=on extensions=".txt" --insecure || exit_1
./mc admin config set silo1 compression allow_encryption=on --insecure || exit_1
# Create bucket in source cluster
echo "Create bucket in source Silo instance"
@@ -80,13 +81,13 @@ echo "Loading objects to source Silo instance"
./mc cp /tmp/data/encrypted silo1/test-bucket/encrypted --enc-c "silo1/test-bucket/encrypted=${TEST_MINIO_ENC_KEY}" --insecure
./mc cp /tmp/data/defpartsize silo1/test-bucket/defpartsize --enc-c "silo1/test-bucket/defpartsize=${TEST_MINIO_ENC_KEY}" --insecure
# Below should fail as compression and SSEC used at the same time
# DISABLED: We must check the response header to see if compression was actually applied
#RESULT=$({ ./mc put /tmp/data/mpartobj.txt silo1/test-bucket/mpartobj.txt --enc-c "silo1/test-bucket/mpartobj.txt=${TEST_MINIO_ENC_KEY}" --insecure; } 2>&1)
#if [[ ${RESULT} != *"Server side encryption specified with SSE-C with compression not allowed"* ]]; then
# echo "BUG: Loading an SSE-C object to site with compression should fail. Succeeded though."
# exit_1
#fi
# A compressible .txt object written with SSE-C while allow_encryption=on. SSE-C
# is excluded from compression whatever allow_encryption says, because
# replication ships SSE-C objects as raw ciphertext and the wire cannot carry the
# compression metadata. Were the object stored compressed, the replica would hold
# the compressed bytes with no compression marker and decrypt to a raw S2 stream,
# which the size and content checks below detect.
./mc cp /tmp/data/mpartobj.txt silo1/test-bucket/mpartobj.txt --enc-c "silo1/test-bucket/mpartobj.txt=${TEST_MINIO_ENC_KEY}" --insecure
# Add replication site
./mc admin replicate add silo1 silo2 --insecure
@@ -111,6 +112,11 @@ if [ "${count3}" -ne 1 ]; then
echo "BUG: object silo1/test-bucket/defpartsize not found"
exit_1
fi
count4=$(./mc ls silo1/test-bucket/mpartobj.txt --insecure | wc -l)
if [ "${count4}" -ne 1 ]; then
echo "BUG: object silo1/test-bucket/mpartobj.txt not found"
exit_1
fi
sleep 120
# List the objects from replicated site
@@ -131,6 +137,11 @@ if [ "${repcount3}" -ne 1 ]; then
echo "BUG: object test-bucket/defpartsize not replicated"
exit_1
fi
repcount4=$(./mc ls silo2/test-bucket/mpartobj.txt --insecure | wc -l)
if [ "${repcount4}" -ne 1 ]; then
echo "BUG: object test-bucket/mpartobj.txt not replicated"
exit_1
fi
# Stat the SSEC objects from source site
echo "Stat silo1/test-bucket/encrypted"
@@ -145,6 +156,25 @@ stat_out2=$(./mc stat --no-list silo1/test-bucket/defpartsize --enc-c "silo1/tes
src_obj2_etag=$(echo "${stat_out2}" | jq '.etag')
src_obj2_size=$(echo "${stat_out2}" | jq '.size')
src_obj2_md5=$(echo "${stat_out2}" | jq '.metadata."X-Amz-Server-Side-Encryption-Customer-Key-Md5"')
echo "Stat silo1/test-bucket/mpartobj.txt"
./mc stat --no-list silo1/test-bucket/mpartobj.txt --enc-c "silo1/test-bucket/mpartobj.txt=${TEST_MINIO_ENC_KEY}" --insecure --json
# The compression marker reaches the client only as the X-Minio-Compressed
# response header, which the SDK filters out of `mc stat --json`, so read the
# raw HTTP trace instead. The sentinel check keeps the assertion from passing
# vacuously if the trace format ever changes.
stat_trace=$(./mc --debug stat --no-list silo1/test-bucket/mpartobj.txt --enc-c "silo1/test-bucket/mpartobj.txt=${TEST_MINIO_ENC_KEY}" --insecure 2>&1) || exit_1
if ! grep -qi "X-Amz-Request-Id" <<<"${stat_trace}"; then
echo "BUG: 'mc --debug stat' printed no response headers, so the compression check below proves nothing"
exit_1
fi
if grep -qi "X-Minio-Compressed" <<<"${stat_trace}"; then
echo "BUG: SSE-C object 'silo1/test-bucket/mpartobj.txt' was stored compressed despite the SSE-C compression exclusion"
exit_1
fi
stat_out3=$(./mc stat --no-list silo1/test-bucket/mpartobj.txt --enc-c "silo1/test-bucket/mpartobj.txt=${TEST_MINIO_ENC_KEY}" --insecure --json)
src_obj3_etag=$(echo "${stat_out3}" | jq '.etag')
src_obj3_size=$(echo "${stat_out3}" | jq '.size')
src_obj3_md5=$(echo "${stat_out3}" | jq '.metadata."X-Amz-Server-Side-Encryption-Customer-Key-Md5"')
# Stat the SSEC objects from replicated site
echo "Stat silo2/test-bucket/encrypted"
@@ -159,6 +189,12 @@ stat_out2_rep=$(./mc stat --no-list silo2/test-bucket/defpartsize --enc-c "silo2
rep_obj2_etag=$(echo "${stat_out2_rep}" | jq '.etag')
rep_obj2_size=$(echo "${stat_out2_rep}" | jq '.size')
rep_obj2_md5=$(echo "${stat_out2_rep}" | jq '.metadata."X-Amz-Server-Side-Encryption-Customer-Key-Md5"')
echo "Stat silo2/test-bucket/mpartobj.txt"
./mc stat --no-list silo2/test-bucket/mpartobj.txt --enc-c "silo2/test-bucket/mpartobj.txt=${TEST_MINIO_ENC_KEY}" --insecure --json
stat_out3_rep=$(./mc stat --no-list silo2/test-bucket/mpartobj.txt --enc-c "silo2/test-bucket/mpartobj.txt=${TEST_MINIO_ENC_KEY}" --insecure --json)
rep_obj3_etag=$(echo "${stat_out3_rep}" | jq '.etag')
rep_obj3_size=$(echo "${stat_out3_rep}" | jq '.size')
rep_obj3_md5=$(echo "${stat_out3_rep}" | jq '.metadata."X-Amz-Server-Side-Encryption-Customer-Key-Md5"')
# Check the etag and size of replicated SSEC objects
if [ "${rep_obj1_etag}" != "${src_obj1_etag}" ]; then
@@ -177,10 +213,23 @@ if [ "${rep_obj2_size}" != "${src_obj2_size}" ]; then
echo "BUG: Size: '${rep_obj2_size}' of replicated object: 'silo2/test-bucket/defpartsize' doesn't match with source value: '${src_obj2_size}'"
exit_1
fi
if [ "${rep_obj3_etag}" != "${src_obj3_etag}" ]; then
echo "BUG: Etag: '${rep_obj3_etag}' of replicated object: 'silo2/test-bucket/mpartobj.txt' doesn't match with source value: '${src_obj3_etag}'"
exit_1
fi
if [ "${rep_obj3_size}" != "${src_obj3_size}" ]; then
echo "BUG: Size: '${rep_obj3_size}' of replicated object: 'silo2/test-bucket/mpartobj.txt' doesn't match with source value: '${src_obj3_size}'"
exit_1
fi
# Check content of replicated SSEC objects
./mc cat silo2/test-bucket/encrypted --enc-c "silo2/test-bucket/encrypted=${TEST_MINIO_ENC_KEY}" --insecure
./mc cat silo2/test-bucket/defpartsize --enc-c "silo2/test-bucket/defpartsize=${TEST_MINIO_ENC_KEY}" --insecure >/dev/null || exit_1
./mc cat silo2/test-bucket/mpartobj.txt --enc-c "silo2/test-bucket/mpartobj.txt=${TEST_MINIO_ENC_KEY}" --insecure >/tmp/data/mpartobj.replica || exit_1
if ! cmp -s /tmp/data/mpartobj.txt /tmp/data/mpartobj.replica; then
echo "BUG: replicated object 'silo2/test-bucket/mpartobj.txt' does not match the source; a compressed SSE-C object decrypts to a raw S2 stream on the replica"
exit_1
fi
# Check the MD5 checksums of encrypted objects from source and target
if [ "${src_obj1_md5}" != "${rep_obj1_md5}" ]; then
@@ -191,5 +240,9 @@ if [ "${src_obj2_md5}" != "${rep_obj2_md5}" ]; then
echo "BUG: MD5 checksum of object 'silo2/test-bucket/defpartsize' doesn't match with source. Expected: '${src_obj2_md5}', Found: '${rep_obj2_md5}'"
exit_1
fi
if [ "${src_obj3_md5}" != "${rep_obj3_md5}" ]; then
echo "BUG: MD5 checksum of object 'silo2/test-bucket/mpartobj.txt' doesn't match with source. Expected: '${src_obj3_md5}', Found: '${rep_obj3_md5}'"
exit_1
fi
cleanup