implement a safer completeMultipart implementation (#20227)

- optimize writing part.N.meta by writing both part.N
  and its meta in sequence without network component.

- remove part.N.meta, part.N which were partially success
  ful, in quorum loss situations during renamePart()

- allow for strict read quorum check arbitrated via ETag
  for the given part number, this makes it double safer
  upon final commit.

- return an appropriate error when read quorum is missing,
  instead of returning InvalidPart{}, which is non-retryable
  error. This kind of situation can happen when many
  nodes are going offline in rotation, an example of such
  a restart() behavior is statefulset updates in k8s.

fixes #20091
This commit is contained in:
Harshavardhana
2024-08-12 01:38:15 -07:00
committed by GitHub
parent 909b169593
commit 2e0fd2cba9
19 changed files with 1487 additions and 275 deletions
+2
View File
@@ -113,6 +113,7 @@ const (
HandlerRenameDataInline
HandlerRenameData2
HandlerCheckParts2
HandlerRenamePart
// Add more above here ^^^
// If all handlers are used, the type of Handler can be changed.
@@ -194,6 +195,7 @@ var handlerPrefixes = [handlerLast]string{
HandlerRenameDataInline: storagePrefix,
HandlerRenameData2: storagePrefix,
HandlerCheckParts2: storagePrefix,
HandlerRenamePart: storagePrefix,
}
const (