Add new ReadFileWithVerify storage-layer API (#4349)

This is an enhancement to the XL/distributed-XL mode. FS mode is
unaffected.

The ReadFileWithVerify storage-layer call is similar to ReadFile with
the additional functionality of performing bit-rot checking. It
accepts additional parameters for a hashing algorithm to use and the
expected hex-encoded hash string.

This patch provides significant performance improvement because:

1. combines the step of reading the file (during
erasure-decoding/reconstruction) with bit-rot verification;

2. limits the number of file-reads; and

3. avoids transferring the file over the network for bit-rot
verification.

ReadFile API is implemented as ReadFileWithVerify with empty hashing
arguments.

Credits to AB and Harsha for the algorithmic improvement.

Fixes #4236.

cmd/retry-storage_test.go

@@ -18,6 +18,8 @@ package cmd
 
 import (
 	"bytes"
+	"crypto/sha256"
+	"encoding/hex"
 	"reflect"
 	"testing"
 	"time"
@@ -290,6 +292,31 @@ func TestRetryStorage(t *testing.T) {
 		if n, err = disk.ReadFile("existent", "path", 7, buf2); err != nil {
 			t.Fatal(err)
 		}
+		if err != nil {
+			t.Error("Error in ReadFile", err)
+		}
+		if n != 5 {
+			t.Fatalf("Expected 5, got %d", n)
+		}
+		if !bytes.Equal(buf2, []byte("World")) {
+			t.Fatalf("Expected `World`, got %s", string(buf2))
+		}
+	}
+
+	sha256Hash := func(s string) string {
+		k := sha256.Sum256([]byte(s))
+		return hex.EncodeToString(k[:])
+	}
+	for _, disk := range storageDisks {
+		var buf2 = make([]byte, 5)
+		var n int64
+		if n, err = disk.ReadFileWithVerify("existent", "path", 7, buf2,
+			HashSha256, sha256Hash("Hello, World")); err != nil {
+			t.Fatal(err)
+		}
+		if err != nil {
+			t.Error("Error in ReadFileWithVerify", err)
+		}
 		if n != 5 {
 			t.Fatalf("Expected 5, got %d", n)
 		}