The xl.meta rename at the end of RenameData creates the object directory
the same way the data rename does, and its failure branch left that
directory behind for writes without a data directory, such as inline
and zero-byte objects. Remove it and its empty parents there too, with
the same trailing-slash call that PR #286 adds to the data branch for
issue #285.
Making the cleanup safe against concurrent writes needs two more
changes:
- deleteFile now walks up with a trailing slash, so each parent step
removes only an empty directory. Between two steps of a walk, another
walk can remove the rest of the chain and a writer can commit an
object whose xl.meta sits at the path the first walk removes next,
for example the object "parent" under the key "parent/xl.meta/child".
The walk used to delete that file. Ordinary deletes could already do
this; the new cleanup made it reachable with failed writes.
- The two renames of RenameData retry when the directory creation or
the rename reports "not found" while the source is still in place,
up to once per path level. A late, failed RenameData can now remove
the directory another write of the same key has just created, and
that write used to fail on the drive although its data was intact.
The retry decides on the error of the attempt itself, because another
write can recreate the directories in between. renameAll's error
mapping moves unchanged into renameAllErr so that the retry can see
the raw error.
The stagedDataGoneDisks test helper now swaps the drives behind the
set's disk list under the lock that the list takes, instead of
replacing the getDisks callback that background tasks read unlocked.
Tests:
- TestXLStorageRenameDataCommitMetaMissingRemovesEmptyDirs: zero-byte
and inline commit failures leave no empty directories, a sibling and
a moved data directory stay.
- TestXLStorageWalkUpKeepsCommittedObject: a committed object at a
parent path survives the walk, through DeleteVersion and both
cleanup branches.
- TestRenameDataSurvivesConcurrentEmptyDirCleanup: PutObject (large
and inline) and CompleteMultipartUpload succeed when a late failed
RenameData of the same key removes their fresh directory on all 16
drives.
- TestXLStorageRenameDataRetriesWhileCleanupWalksUp: a write survives
a cleanup that removes each level of its path in turn, also when a
nested-key write recreates the directories in between.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
Merge maintainer follow-up for lifecycle-aware pool migration.
Preserve replication, tier cleanup, lifecycle-generation, and whole-object completion semantics on top of contributor fix#266.
Merge the contributor's lifecycle migration fix after forward-porting it onto current main.
The contributor commits are retained in the merge commit. Maintainer follow-up work will address the remaining replication, tier cleanup, lifecycle configuration, and DeleteAll completion-fence cases.
Availability-optimized writes can leave a readable null version across separate EC groups when a disk is unavailable. Recount ordinary null versions for all non-strict reads and cover version listing and complete pool removal with end-to-end tests.
Signed-off-by: Feng Ruohang <rh@vonng.com>
Replace process-wide hash-slot mutexes with reference-counted locks keyed
by drive and path. Retain entries before waiting, acquire unique paths in
order, and reclaim entries only after all holders and waiters finish.
Add real storage API regressions for unrelated colliding objects and drives,
plus coverage for cancellation, multi-path order and waiter lifetimes.
Signed-off-by: Feng Ruohang <rh@vonng.com>
Feng Ruohang