An ILM expiry that is refused, because the rule no longer applies when
it is checked again under the object lock or Object Lock protects the
version, or that fails, for example for lack of quorum, left almost no
trace. The bulk noncurrent purge skipped such entries without audit,
trace, log or count; the scanner traced a refusal exactly like a
successful expiry; and nothing counted either. Operators could not see
why a version they expected gone was still there.
- Count every such expiry by reason in expiry_incomplete_total
{reason="not_applicable"|"locked"|"history_unavailable"|"error"}, in
metrics v2 and v3; the counting unit is one expiry attempt, and no
bucket or object is a label.
- Trace it with result=refused or result=failed and the reason. A
failure is also audited and logged once per object; a refusal changes
nothing and is not audited, since a split key is refused on every scan.
- Never announce it: no ObjectRemoved notification for a version that is
still there.
- Name a completed expiry by what its delete did. An expiry that
addressed no version and whose result is a delete marker created that
marker, also with the null version ID of the version it replaced; any
other removed the version it addressed, also a delete marker. The
scanner and the transitioned expiry took the identity from the queued
object instead: the expiry of a latest transitioned version in a
versioned bucket announced the removal of the version it kept, and the
removal of an expired delete marker was announced as a marker created.
Tests: classification of each error; a bulk purge with a refused, a
failed and a removed entry counts and traces the first two, audits only
the failure and announces only the removal; the expiry events name the
created marker for a latest version (versioned, suspended null,
transitioned) and the removed version for a noncurrent version and an
expired delete marker; both metrics versions export the counter by
reason. The #278 side-effect tests now tell removal traces from
refusal and failure traces.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
The migration follow-up for #265 (PR #276) made lifecycle-created
deletes replication-aware: the scanner, the expiry of a transitioned
version and the bulk noncurrent purge evaluated the delete replication
decision and queued a delete marker or a version purge for the targets.
That is the content of the still-open PR #240. It changes behavior for
every bucket whose replication rule replicates deletes or delete
markers, the mc default: a target with longer retention loses versions
on the source's schedule, and a purge keeps its space as a pending
tombstone until the target confirms. SILO, like S3, does not replicate
lifecycle deletes; an opt-in can come later. The release did not
replicate them.
Remove exactly the replication parts: the decision callbacks and the
post-delete scheduling in applyExpiryOnNonTransitionedObjects and
expireTransitionedObject, the scheduling in deleteObjectVersions, and
ilmPurgeToDelete; the scanner builds noncurrent entries as before. The
rest of #276 stays: the hot-tier delete before the remote one, the
free-version retry record, the lifecycle-change fence and the migration
completion checks. API deletes still replicate, and the replication heal
still handles persisted pending markers and purges. As in the release,
the replication heal also resyncs an empty-state marker when
existing-object replication applies to it.
The delete-all variant of the scheduling, which queued a task with an
empty decision, goes with it.
Tests: the scanner's noncurrent purge, run with replication configured,
deletes the version at once and queues nothing; an ILM marker carries
no replication state and the heal leaves it unless existing-object
replication applies, in which case it is resynced as before; an API
version delete still queues a purge. The cross-pool batch, dispatch and
decommission tests now assert no replication, and a decommission whose
expiry used to wait for the target completes in one run. Adding back the
scanner's decision makes the first two fail.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
The expiry worker now asks DeleteObjects to recheck ILM against all
pools. A rejected entry keeps its version and returns
errLifecycleNotApplicable, but the worker still wrote an ILMExpiry
audit record, an ILM trace and an s3:ObjectRemoved:Delete event for it.
Event delivery drops the error element, so consumers saw an ordinary
removal, repeated on every scan while the versions stay split across
pools.
Skip rejected entries; eligible entries of the same batch are still
reported.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
Merge lock and tag revisions for duplicate versions before all-pool
lifecycle evaluation using the existing metadata reconciliation rules.
Distinguish ordinary lifecycle refusal from object absence and clear
rejected batch replication state so a late policy withdrawal cannot
schedule a purge. Preserve restored-copy cleanup and real absence.
Cover single and batch protections, legitimate metadata removals, and
actual replication queue and HTTP target behavior.
Signed-off-by: Feng Ruohang <rh@vonng.com>
Feng Ruohang