Commit Graph

6874 Commits

Author SHA1 Message Date
Feng Ruohang f83d02831c Integrate complete R3 encoding and safe-read repairs
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-03 22:46:04 +08:00
Feng Ruohang 496eab4b76 fix(restore): preserve safe read context through nested lookups
Carry restore context through POST status reads, metadata copy routing, null-version Object Lock checks and multipart completion. A failed read must preserve the target and must not enqueue its remote object for garbage collection. Cover initial/status POST failures and metadata loss during single and multipart rehydration on real disks.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-03 22:46:03 +08:00
Feng Ruohang ea483e453e Integrate R2 restore cleanup boundary repairs
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-03 22:45:14 +08:00
Feng Ruohang ce60cc0c84 Integrate R1 adversarial review repairs
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-03 22:45:14 +08:00
Feng Ruohang 7c9b6386c6 Merge R2 restore safety repair into R3
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-03 22:40:08 +08:00
Feng Ruohang ba0b59d0b2 fix(ilm): preserve duplicate metadata and reject unapplied purges
Merge lock and tag revisions for duplicate versions before all-pool
lifecycle evaluation using the existing metadata reconciliation rules.

Distinguish ordinary lifecycle refusal from object absence and clear
rejected batch replication state so a late policy withdrawal cannot
schedule a purge. Preserve restored-copy cleanup and real absence.

Cover single and batch protections, legitimate metadata removals, and
actual replication queue and HTTP target behavior.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-03 22:38:21 +08:00
Feng Ruohang d9837fb319 fix(restore): retain legacy part-size compatibility
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-03 22:32:55 +08:00
Feng Ruohang 8579466557 fix(restore): preserve versions across cleanup and failed reads
Reject replacement markers and legacy objects before restored-copy cleanup
can mutate their type or data. Prevent restore lookups and failure cleanup
from purging dangling versions, preserve Transition on metadata rereads,
and return without writing when the initial restore read fails.

Add real-disk regressions for stale null tasks, minority metadata, initial
and cleanup read failures, and stored marker/legacy types. Keep ordinary
dangling cleanup and marker purge as positive controls.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-03 22:32:33 +08:00
Feng Ruohang 4f72d0ea94 fix(restore): preserve encoded object identity and part layout
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-03 22:16:47 +08:00
Feng Ruohang 3c4980eb80 Integrate multipart error-case test lint correction
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-03 21:40:59 +08:00
Feng Ruohang 50a234d6c3 test(restore): clarify invalid-upload error cases
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-03 21:40:59 +08:00
Feng Ruohang a02ce222f0 Integrate complete R3 input-error regression fix
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-03 21:39:36 +08:00
Feng Ruohang 8ae19de7a8 fix(restore): avoid duplicate completion locks for empty upload IDs
Keep the storage layer error precedence for missing buckets and uploads while retaining every real upload lock. Cover the alias with short deadlines and race-check the original complete-multipart suite in CI.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-03 21:39:36 +08:00
Feng Ruohang 7a4f0b5e69 Integrate native restore test lint correction
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-03 21:21:46 +08:00
Feng Ruohang 5ac27f7533 test(restore): follow context-first helper convention
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-03 21:21:46 +08:00
Feng Ruohang fa42a09826 fix(restore): rehydrate the selected version under its object lock
Preserve handler version context, resolve the requested version's pool, and
return on the initial read failure without writing metadata. Propagate owned
NoLock to single-part writes, multipart completion and failure cleanup in
the same patch. Preserve the original modification time on cleanup.

Route the restore status copy by version and allow internal multipart
rehydration only for a matching completed transition. Single-pool restores
own the metadata snapshot through download and cleanup. Keep public
completion in object-before-upload order, using the target set's object
lock for single-pool calls, so concurrent completion cannot invert locks.

Cover real signed POSTs, null/UUID and cross-pool identity, download failures,
multipart overwrite guards, Object Lock updates and concurrent completion.
This package depends on the restored-copy cleanup safety delivered in R2.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-03 21:20:24 +08:00
Feng Ruohang fcbf2c7d49 Integrate complete native restore repair after R2 cleanup safety
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-03 21:20:24 +08:00
Feng Ruohang 18ba0ef32a Merge R4 version-specific read expiry rechecks after R1
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-03 21:11:41 +08:00
Feng Ruohang 65c59dd860 fix(ilm): recheck read expiry for the resolved version
Recheck only local ordinary-delete candidates that need an all-pool lifecycle decision. Match UUID/null identity to the version actually read and queue its global metadata and event; retain a successful read on recheck failure, absence, or refusal.

Keep GET reader cleanup, authorization, proxy reads, single-pool reads, restored-copy cleanup, and expiry response headers on their existing paths. Cover signed GET/HEAD, queue contents, metadata faults, missing matches, no-scan boundaries, and reader-lock release.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-03 21:07:55 +08:00
Feng Ruohang e7103d77cb fix(ilm): constrain restored-copy cleanup to verified tiered versions
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-03 20:55:22 +08:00
Feng Ruohang 167b95dfd8 Integrate R2 restored-copy cleanup safety with R1
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-03 20:55:22 +08:00
Feng Ruohang 3a1e132f16 fix(ilm): preserve all-pool approval for tiered batch expiry
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-03 20:40:04 +08:00
Feng Ruohang 30588eb830 Merge PR #282 restore-copy version targeting for R2
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-03 20:37:15 +08:00
Feng Ruohang 393a84076f Merge commit '8c1473a618f89fda3b89462c69c2a3c1a320d4c6' into codex/silo-lifecycle-r1-r4
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-03 20:36:07 +08:00
Denis Babchuk a21f420de3 fix(sts): keep "not found" in the LDAP auth failure so the console can log in root
Signed-off-by: Denis Babchuk <babchukdenis@icloud.com>
2026-11-03 08:52:08 +08:00
Denis Babchuk dcad2ae3e7 fix(iam): decide unstorable IAM record names before asking the drives
Signed-off-by: Denis Babchuk <babchukdenis@icloud.com>
2026-11-03 08:38:24 +08:00
Denis Babchuk 525ab5fa40 fix(pools): do not panic when a rebalance stop finds no rebalance metadata
Signed-off-by: Denis Babchuk <babchukdenis@icloud.com>
2026-11-03 08:23:59 +08:00
Denis Babchuk 001078e9ca fix(ilm): expire a restored copy by the version it belongs to
Signed-off-by: Denis Babchuk <babchukdenis@icloud.com>
2026-11-02 21:41:37 +03:00
Denis Babchuk 8c1473a618 fix(pools): move versions whose ILM expiry does not hold with the versions in other pools
Signed-off-by: Denis Babchuk <babchukdenis@icloud.com>
2026-11-02 19:59:13 +03:00
Denis Babchuk 347d9763fd fix(ilm): recheck expiries decided in one pool against the versions of the object in all pools
Signed-off-by: Denis Babchuk <babchukdenis@icloud.com>
2026-11-02 19:58:46 +03:00
Feng Ruohang a47bacbc0f test: satisfy lint for lifecycle migration follow-up
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-02 14:37:04 +08:00
Feng Ruohang 0a546c7a76 fix(pools): preserve ILM cleanup during migration
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-02 14:20:17 +08:00
Feng Ruohang 4cc3bc1de6 Merge remote-tracking branch 'origin/main' into HEAD
Signed-off-by: Feng Ruohang <rh@vonng.com>

# Conflicts:
#	cmd/erasure-server-pool-decom.go
2026-11-02 12:29:08 +08:00
Feng Ruohang 0cfca29cb2 Merge pull request #264 from botdeniska/fix/free-versions-not-versions
fix(ilm): stop taking the free versions of deleted tiered versions for versions
2026-11-01 14:44:22 +08:00
Feng Ruohang a7310d59fe Fix cross-parity version listing after pool removal (#267)
Availability-optimized writes can leave a readable null version across separate EC groups when a disk is unavailable. Recount ordinary null versions for all non-strict reads and cover version listing and complete pool removal with end-to-end tests.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-01 14:08:09 +08:00
Denis Babchuk a995921e57 fix(pools): move ILM-expired current versions during rebalance and remove moved copies
Signed-off-by: Denis Babchuk <babchukdenis@icloud.com>
2026-10-29 22:43:26 +03:00
Denis Babchuk 19910230aa fix(pools): move ILM-expired current versions during decommission
Signed-off-by: Denis Babchuk <babchukdenis@icloud.com>
2026-10-29 22:42:17 +03:00
Denis Babchuk 7ce7fc8d1e fix(pools): do not list or move free versions as delete markers
Signed-off-by: Denis Babchuk <babchukdenis@icloud.com>
2026-10-29 22:32:44 +03:00
Denis Babchuk fb1a619ee1 fix(listing): skip free versions when looking for a latest delete marker
Signed-off-by: Denis Babchuk <babchukdenis@icloud.com>
2026-10-29 22:08:34 +03:00
Denis Babchuk 3f13c26eed fix(ilm): do not let a free version make the latest version noncurrent
Signed-off-by: Denis Babchuk <babchukdenis@icloud.com>
2026-10-29 22:07:56 +03:00
Feng Ruohang dcc2f20ba8 Preserve maintenance timeout behavior and prepare SILO 20261026
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-26 22:43:51 +08:00
Feng Ruohang 39278e068e fix: isolate storage mutation guards by exact resource
Replace process-wide hash-slot mutexes with reference-counted locks keyed
by drive and path. Retain entries before waiting, acquire unique paths in
order, and reclaim entries only after all holders and waiters finish.

Add real storage API regressions for unrelated colliding objects and drives,
plus coverage for cancellation, multi-path order and waiter lifetimes.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-26 21:11:49 +08:00
Feng Ruohang 641eb51a40 test: pin the rollback fixture to its required erasure layout
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 23:47:55 +08:00
Feng Ruohang 6921f8c186 fix: return current bucket inventory from account info
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 21:01:01 +08:00
Feng Ruohang 25d928c9df test: pin failed-overwrite compatibility fixture to 12+4
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 18:48:39 +08:00
Feng Ruohang ee3e7ea716 test: keep deferred read diagnostic helpers with their build tag
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 18:40:40 +08:00
Feng Ruohang 2a04d5f485 fix: retain maintenance read compatibility with write safety fixes
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 18:39:31 +08:00
Feng Ruohang 9449e6a4a8 test: preserve reads across late commits and rollback
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 18:23:14 +08:00
Feng Ruohang 9c4bb39571 fix: preserve versioning context when verifying recovery
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 16:05:28 +08:00
Feng Ruohang 86ce8d6012 test: cover versioned and encrypted multi-pool recovery
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 16:03:13 +08:00