A version can have copies in two pools, for example after an interrupted
decommission or rebalance. DeleteObject left ILM expiries out of the
reconciled delete of an addressed version, so a lifecycle version expiry
deleted only the copy that latest-key routing chose. The other copy stayed
readable; for a tiered version, expireTransitionedObject then deleted the
remote object that copy still referenced; and a migration of the pool
holding it left the expired version behind and failed its completion
check. With delete replication on, the replication completion's own
delete had reconciled the copies and hidden this.
Admit an ILM version expiry that the pools layer approved under the lock
with the versions in all pools (CheckedAllPools, and not a restored-copy
expiry) to deleteObjectReconciled. It removes the secondary copies before
the primary and stops at the first failure, so the remote object is only
deleted after every copy is gone. Each set still guards the protection
the pools approved. Marker creation, restored-copy expiry, free-version
cleanup and delete-all keep their own paths; the bulk noncurrent purge
already reconciled.
Tests: with copies in two pools, a hot and a tiered version expiry remove
both copies, and the tiered one deletes its remote object once; when the
secondary copy cannot be deleted the expiry fails, both copies and the
remote object stay; with replication configured the tiered expiry queues
nothing; decommissioning the pool that holds the second copy completes
and leaves no expired version. All fail without the change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
The migration follow-up for #265 (PR #276) made lifecycle-created
deletes replication-aware: the scanner, the expiry of a transitioned
version and the bulk noncurrent purge evaluated the delete replication
decision and queued a delete marker or a version purge for the targets.
That is the content of the still-open PR #240. It changes behavior for
every bucket whose replication rule replicates deletes or delete
markers, the mc default: a target with longer retention loses versions
on the source's schedule, and a purge keeps its space as a pending
tombstone until the target confirms. SILO, like S3, does not replicate
lifecycle deletes; an opt-in can come later. The release did not
replicate them.
Remove exactly the replication parts: the decision callbacks and the
post-delete scheduling in applyExpiryOnNonTransitionedObjects and
expireTransitionedObject, the scheduling in deleteObjectVersions, and
ilmPurgeToDelete; the scanner builds noncurrent entries as before. The
rest of #276 stays: the hot-tier delete before the remote one, the
free-version retry record, the lifecycle-change fence and the migration
completion checks. API deletes still replicate, and the replication heal
still handles persisted pending markers and purges. As in the release,
the replication heal also resyncs an empty-state marker when
existing-object replication applies to it.
The delete-all variant of the scheduling, which queued a task with an
empty decision, goes with it.
Tests: the scanner's noncurrent purge, run with replication configured,
deletes the version at once and queues nothing; an ILM marker carries
no replication state and the heal leaves it unless existing-object
replication applies, in which case it is resynced as before; an API
version delete still queues a purge. The cross-pool batch, dispatch and
decommission tests now assert no replication, and a decommission whose
expiry used to wait for the target completes in one run. Adding back the
scanner's decision makes the first two fail.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
The merged all-pool check from #279 decided on the versions that a read
quorum of each set merged non-strictly. A drive that missed one metadata
update could therefore supply the revision used for the decision, and a
partially unreadable set could drop a committed version from the history.
Both let an expiry delete data the rule did not cover, including a version
under legal hold.
- objectVersions accepts a revision only when max(n/2+1, the version's
metadata read quorum) drives hold it, identifies revisions by the header
plus a sha256 over a canonical, length-prefixed encoding of the
normalized metadata, filters non-accepted revisions before the strict
merge, and fails closed on omitted versions, too many unreadable drives,
or cancellation.
- Lock fields are read with the evaluator's key precedence. Cross-pool
copies that disagree on hold or retention must be strictly ordered by
valid timestamps; copies with different ModTime are separate writes and
an older protected write refuses the object. The set safeguard re-checks
protection and pending purges on its fresh read for checked version
deletes.
- Duplicate copies with semantically different replication or purge state
refuse the object.
- Batch expiry announces and audits only completed or already-absent
removals.
Reviewed adversarially over seven rounds; the final round approved with no
remaining findings in scope. All-versions expiry through deletePrefix and
tiered delete-all dispatch are tracked separately.
Signed-off-by: Feng Ruohang <rh@vonng.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merge lock and tag revisions for duplicate versions before all-pool
lifecycle evaluation using the existing metadata reconciliation rules.
Distinguish ordinary lifecycle refusal from object absence and clear
rejected batch replication state so a late policy withdrawal cannot
schedule a purge. Preserve restored-copy cleanup and real absence.
Cover single and batch protections, legitimate metadata removals, and
actual replication queue and HTTP target behavior.
Signed-off-by: Feng Ruohang <rh@vonng.com>
Feng Ruohang