mirror of
https://github.com/pgsty/minio.git
synced 2026-11-10 09:35:31 +03:00
fix(ilm): purge only the expired key's own versions in delete-all
An expiry of all versions (ExpiredObjectAllVersions, DelMarkerExpiration, and batch expiration with the default purge) asked the pools layer for a prefix delete. The pools layer rebuilt the options, so the request ran as a recursive delete of the key's directory in every erasure set of every pool. Object a/b is stored inside the directory of a, so every key under a/ was removed with it, however new, with one audit entry and one notification for a alone. The rebuilt options also skipped the lifecycle recheck, so on a single pool a version written after the decision was deleted too, and tiered versions lost their metadata without free versions. The expiry worker, the migration expiry and batch expiration now ask for a purge with the in-memory ObjectOptions.DeleteAllVersions: - DeleteObject hands it to the purge under the object lock, before the all-pools check. ILM decides with the unchanged lifecycleEventsAllPools, on a single pool too; batch needs the version its walk matched to still be the latest. A second read of each pool's authoritative history records every copy, older writes of the same ID included, and must equal the approval. Anything else deletes nothing. - Before deleting, the purge reads every drive of each set that holds a copy. It refuses the key when a drive cannot be read, when any revision of an approved version there is transitioned, or, with Object Lock, when retention or a legal hold protects one. It then deletes on exactly the drive handles it checked. - Versions are deleted by ID with the drives' version delete, which only removes the object directory once it is empty: oldest first, for each version older writes before the copy reads choose, and the latest last, only once every older version is gone from every pool. No marker, replication state or free version is written. The purge stops at the first error, repeats a quorum-successful delete on the drives that missed it while the lock is held, and stops before the latest version at a budget of 1000 versions or one minute, reads included. - The ILM audit entry records how many versions were removed. A purge that failed after it began to delete is audited as partial and sends no removal event. Batch counts come from the purge; a key whose latest version changed is not expired, and a failed delete marker fails the job. The option is rejected with conflicting options and by every layer below the pools. Force delete and internal prefix deletes keep DeletePrefix. Delete-all of a transitioned latest version keeps its current path, and a key with tiered older versions is refused. On a single pool the purge relies on the lock-domain fix for writer exclusion. Reviewed adversarially in two design rounds and four patch rounds; the last approved with nits, which are applied. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Feng Ruohang <rh@vonng.com>
This commit is contained in:
@@ -1234,6 +1234,11 @@ func (z *erasureServerPools) DeleteObject(ctx context.Context, bucket string, ob
|
||||
if err = checkDelObjArgs(ctx, bucket, object); err != nil {
|
||||
return objInfo, err
|
||||
}
|
||||
if opts.DeleteAllVersions {
|
||||
if err := opts.deleteAllInvalid(); err != nil {
|
||||
return objInfo, InvalidArgument{Bucket: bucket, Object: object, Err: err}
|
||||
}
|
||||
}
|
||||
|
||||
if !opts.DeletePrefix { // DeletePrefix handles dir object encoding differently.
|
||||
object = encodeDirObject(object)
|
||||
@@ -1248,6 +1253,12 @@ func (z *erasureServerPools) DeleteObject(ctx context.Context, bucket string, ob
|
||||
ctx = lkctx.Context()
|
||||
defer lk.Unlock(lkctx)
|
||||
|
||||
// A purge of all versions approves and deletes on the history it reads
|
||||
// itself. The checks below decide by other options and would discard it.
|
||||
if opts.DeleteAllVersions {
|
||||
return z.deleteAllVersions(ctx, bucket, object, opts)
|
||||
}
|
||||
|
||||
// An ILM expiry is decided on the versions in one pool. Under the lock,
|
||||
// check it with the versions in all pools; the sets then take it as
|
||||
// decided, without a recheck by the versions each one holds. They only
|
||||
@@ -1496,6 +1507,12 @@ func deletesData(obj ObjectToDelete, opts ObjectOptions) bool {
|
||||
func (z *erasureServerPools) DeleteObjects(ctx context.Context, bucket string, objects []ObjectToDelete, opts ObjectOptions) ([]DeletedObject, []error) {
|
||||
derrs := make([]error, len(objects))
|
||||
dobjects := make([]DeletedObject, len(objects))
|
||||
if opts.DeleteAllVersions {
|
||||
for i := range derrs {
|
||||
derrs[i] = errDeleteAllUnsupported
|
||||
}
|
||||
return dobjects, derrs
|
||||
}
|
||||
objSets := set.NewStringSet()
|
||||
for i := range derrs {
|
||||
objects[i].ObjectName = encodeDirObject(objects[i].ObjectName)
|
||||
|
||||
Reference in New Issue
Block a user