A node that reaches only some drives of a set, for example right after
it restarts and before it reconnects to its peers' drives, could answer
404 NoSuchKey for an object that exists. It happened when the copies it
could see were split between two revisions and neither reached read
quorum: a restore, a transition or an unversioned overwrite committed on
a write quorum but not on every drive. S3 clients and caches treat 404
as final; the honest answer from such a view is a retryable 503.
Two paths turned that view into "not found":
- getObjectFileInfo converted InsufficientReadQuorum{RQInconsistentMeta}
to not found. Upstream added this for the case where every drive
answered, but never checked it.
- When the version merge found no version in quorum,
pickLatestQuorumFilesInfo marked every drive that answered as not
found, including drives that hold the object.
Both now require every drive to have answered: metadata, or a definite
file, version or volume not found, or corrupt metadata. Otherwise the
read keeps its read-quorum error. A key absent from every drive that
answered still gets 404, and so do inconsistent copies when every drive
answered. The dangling-object purge is unchanged.
Deletes, copies, metadata updates, decommission and rebalance that read
through such a view now fail and retry instead of treating the object
as absent. The change is local to the serving node: no storage RPC,
on-disk format or configuration changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
Feng Ruohang