astelm/minio
1
0
Fork 0
mirror of https://github.com/pgsty/minio.git synced 2026-07-24 06:26:17 +03:00
Code Issues Packages Projects Releases Wiki Activity

Add Full Object Checksums and CRC64-NVME (#20855)

Browse Source 
Backport of AIStor PR 247.

Add support for full object checksums as described here:

https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity.html

New checksum types are fully supported. Mint tests from https://github.com/minio/minio-go/pull/2026 are now passing.

Includes fixes from https://github.com/minio/minio/pull/20743 for mint tests.

Add using checksums as validation for object content. Fixes #20845 #20849

Fixes checksum replication (downstream PR 250)
This commit is contained in:
Klaus Post
2025-01-20 06:49:07 -08:00
committed by GitHub
parent 779ec8f0d4
commit 827004cd6d
18 changed files with 665 additions and 168 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/hash/reader.go
-20
View File
@@ -257,26 +257,6 @@ func (r *Reader) Read(p []byte) (int, error) {
r.contentHasher.Write(p[:n])
}
// If we have reached our expected size,
// do one more read to ensure we are at EOF
// and that any trailers have been read.
attempts := 0
for err == nil && r.size >= 0 && r.bytesRead >= r.size {
attempts++
if r.bytesRead > r.size {
return 0, SizeTooLarge{Want: r.size, Got: r.bytesRead}
}
var tmp [1]byte
var n2 int
n2, err = r.src.Read(tmp[:])
if n2 > 0 {
return 0, SizeTooLarge{Want: r.size, Got: r.bytesRead}
}
if attempts == 100 {
return 0, io.ErrNoProgress
}
}
if err == io.EOF { // Verify content SHA256, if set.
if r.expectedMin > 0 {
if r.bytesRead < r.expectedMin {