A batch expiration that keeps no versions authorized the versions its
walk saw, and the purge then deleted the versions it read under its
lock. A version written after the walk, or tags changed on one since,
was deleted without being authorized for the submitter.
The purge takes an authorization that a batch purge must set and no
other delete may. Under the object lock, once it has approved the
history and checked every drive, and before its first delete, it
hands the writes it is about to delete to the job: every copy of every
approved version and every revision of one that a drive holds, one per
distinct version ID and tags, as they are then. The job evaluates the
expire actions on each for the submitter. A denial deletes nothing and
ends the job as refused; the time it takes counts against the purge's
budget. A locked version still refuses the whole key, counted as a
refused key. The checks on the walk's snapshot are removed.
Reviewed adversarially; the review's regressions are included.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
An expiry of all versions (ExpiredObjectAllVersions, DelMarkerExpiration,
and batch expiration with the default purge) asked the pools layer for a
prefix delete. The pools layer rebuilt the options, so the request ran as
a recursive delete of the key's directory in every erasure set of every
pool. Object a/b is stored inside the directory of a, so every key under
a/ was removed with it, however new, with one audit entry and one
notification for a alone. The rebuilt options also skipped the lifecycle
recheck, so on a single pool a version written after the decision was
deleted too, and tiered versions lost their metadata without free
versions.
The expiry worker, the migration expiry and batch expiration now ask for
a purge with the in-memory ObjectOptions.DeleteAllVersions:
- DeleteObject hands it to the purge under the object lock, before the
all-pools check. ILM decides with the unchanged lifecycleEventsAllPools,
on a single pool too; batch needs the version its walk matched to still
be the latest. A second read of each pool's authoritative history
records every copy, older writes of the same ID included, and must
equal the approval. Anything else deletes nothing.
- Before deleting, the purge reads every drive of each set that holds a
copy. It refuses the key when a drive cannot be read, when any revision
of an approved version there is transitioned, or, with Object Lock,
when retention or a legal hold protects one. It then deletes on exactly
the drive handles it checked.
- Versions are deleted by ID with the drives' version delete, which only
removes the object directory once it is empty: oldest first, for each
version older writes before the copy reads choose, and the latest last,
only once every older version is gone from every pool. No marker,
replication state or free version is written. The purge stops at the
first error, repeats a quorum-successful delete on the drives that
missed it while the lock is held, and stops before the latest version
at a budget of 1000 versions or one minute, reads included.
- The ILM audit entry records how many versions were removed. A purge
that failed after it began to delete is audited as partial and sends
no removal event. Batch counts come from the purge; a key whose latest
version changed is not expired, and a failed delete marker fails the
job.
The option is rejected with conflicting options and by every layer below
the pools. Force delete and internal prefix deletes keep DeletePrefix.
Delete-all of a transitioned latest version keeps its current path, and
a key with tiered older versions is refused. On a single pool the purge
relies on the lock-domain fix for writer exclusion.
Reviewed adversarially in two design rounds and four patch rounds; the
last approved with nits, which are applied.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
Feng Ruohang