Commit Graph

6831 Commits

Author SHA1 Message Date
Feng Ruohang 6921f8c186 fix: return current bucket inventory from account info
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 21:01:01 +08:00
Feng Ruohang 25d928c9df test: pin failed-overwrite compatibility fixture to 12+4
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 18:48:39 +08:00
Feng Ruohang ee3e7ea716 test: keep deferred read diagnostic helpers with their build tag
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 18:40:40 +08:00
Feng Ruohang 2a04d5f485 fix: retain maintenance read compatibility with write safety fixes
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 18:39:31 +08:00
Feng Ruohang 9449e6a4a8 test: preserve reads across late commits and rollback
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 18:23:14 +08:00
Feng Ruohang 9c4bb39571 fix: preserve versioning context when verifying recovery
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 16:05:28 +08:00
Feng Ruohang 86ce8d6012 test: cover versioned and encrypted multi-pool recovery
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 16:03:13 +08:00
Feng Ruohang b15aa97c94 test: follow recovery regression lint conventions
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 15:59:45 +08:00
Feng Ruohang f54c04846e fix: recover inconsistent multi-pool copies before acknowledging PUT
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 15:56:28 +08:00
Feng Ruohang 6fdea3b8d9 test: reject recovery ACK when another pool still wins reads
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 15:55:10 +08:00
Feng Ruohang 380fcbfe47 test: require safe refusal when pool recovery cannot be confirmed
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 15:49:29 +08:00
Feng Ruohang a02fa9644c test: pin multi-pool overwrite recovery after a minority commit
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 15:48:33 +08:00
Feng Ruohang 652a1c09d3 fix: protect unfinished commits from abandoned data cleanup
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 13:23:20 +08:00
Feng Ruohang c86df14f4d test: pin abandoned cleanup races with commit and undo
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 13:22:27 +08:00
Feng Ruohang 03875d61e3 fix: reuse validated latest state without rescanning version history
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 13:02:23 +08:00
Feng Ruohang 1865985aa1 fix: separate rollback ownership from readable metadata identity
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 12:41:54 +08:00
Feng Ruohang 076a13f9d2 test: preserve reads after partial no-op metadata updates
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 12:40:00 +08:00
Feng Ruohang 9a4e089f56 fix: reuse unanimous latest metadata for healthy version histories
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 12:33:49 +08:00
Feng Ruohang 9255f443a6 test: guard healthy history reads and same-version divergence
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 12:31:10 +08:00
Feng Ruohang ea7c794a94 fix: preserve version state across late rollback and nested mutations
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 12:22:47 +08:00
Feng Ruohang 16c21428d9 test: pin metadata rollback and nested mutation lifetime regressions
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 12:07:50 +08:00
Feng Ruohang 7e83f9950a merge: restore built-in IAM policies after Reset
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 11:02:53 +08:00
Feng Ruohang 1e3ab56b06 iam: restore native policy immediately after resetting overrides
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 03:32:53 +08:00
Feng Ruohang 5ad1f344b7 fix: keep admitted mutation RPCs alive until their result arrives
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 01:12:58 +08:00
Feng Ruohang 8a4097309f fix: preserve shared-version metadata during overwrite rollback
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 01:11:27 +08:00
Feng Ruohang 0501a198e3 fix: reject COPY-to-self from unresolved object generations
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 01:11:27 +08:00
Feng Ruohang d2eb0ade91 fix: assign PUT generations while holding the commit lock
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 01:08:36 +08:00
Feng Ruohang 5c074efd1d test: preserve external shards in generation conflict fixtures
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 01:03:14 +08:00
Feng Ruohang 03fe5e9c21 test: isolate endpoint locality fixtures from public DNS
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 00:50:41 +08:00
Feng Ruohang 0b1c21cb4b fix: retain unresolved generations during fast reads and healing
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 00:48:16 +08:00
Feng Ruohang b55495aef0 fix: preserve remote rollback options and reject stale read quorums
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 00:37:21 +08:00
Feng Ruohang d8b77e5f3b test: require read-quorum errors for split listing generations
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 00:24:28 +08:00
Feng Ruohang cd1f6cec3c fix: drain canceled storage commits before releasing object locks
Wait for admitted data and metadata mutations, preserve overwrite backups during rollback, and reject rollback of a different generation. Report conflicting or unreadable pool metadata as read-quorum failures instead of exposing stale copies. Add barrier-driven local/RPC and multi-pool regressions for cancellation, self-copy, rollback and old-value reads.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-25 00:19:42 +08:00
Feng Ruohang 4f17a23e33 test: isolate release acceptance fixtures across backends
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-24 22:39:29 +08:00
Feng Ruohang d16f5ed9c8 fix(ilm): keep tier data when expiry only adds a delete marker (#253)
Preserve transitioned version data when lifecycle expiration only creates a delete marker, and apply suspended-versioning semantics consistently.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-24 17:46:10 +08:00
Feng Ruohang 6d890fc860 Merge pull request #255: preserve FULL_OBJECT checksum type in replication
Forward FULL_OBJECT checksum type for multipart CRC replication so the target validates the whole-object checksum correctly. Restore the existing single-PUT replication path for server-side copies retaining these checksums.

Fixes #254

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-24 17:39:38 +08:00
Feng Ruohang a2073d2514 fix(pools): preserve shared tier references during failed deletes
Reuse deleteObjectReconciled for tiered bulk entries and unversioned single deletes. Its reference-aware cleanup avoids scheduling remote contents for garbage collection while another copy still needs them, including restored copies. Keep the normal local-data bulk path unchanged.

Extend existing tier GC regressions to bulk and unversioned deletes and include them in the multi-pool race CI step.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-24 15:47:51 +08:00
Feng Ruohang 3c82748e14 fix(pools): retain authoritative copies until bulk cleanup succeeds
Resolve each addressed version under the shared write lock, keep bulk I/O parallel within cleanup and authoritative-deletion stages, and preserve replication response identity on errors. Treat reliable absence as successful physical cleanup on retries without accepting it for marker or replication-state writes.

Cover execution-time faults, partial cleanup recovery, mixed authoritative pools, response identity, null marker ordering, metadata uncertainty, cancellation, and concurrent PUT. Run the delete consistency regressions under race in CI.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-24 15:36:07 +08:00
Feng Ruohang 1e94c5a30d Merge remote-tracking branch 'origin/main' into codex/pr257-delete-safety
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-24 15:13:21 +08:00
Feng Ruohang 945dc16060 Update SFTP and Go dependencies with object-operation regression coverage
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-24 11:50:39 +08:00
Denis Babchuk a30270d9ba fix(pools): put a delete marker into a pool that can take it
Signed-off-by: Denis Babchuk <babchukdenis@icloud.com>
2026-10-23 22:29:49 +03:00
Denis Babchuk 1841fce4c4 fix(pools): keep the latest copy when a delete fails in one pool
Signed-off-by: Denis Babchuk <babchukdenis@icloud.com>
2026-10-23 21:28:46 +03:00
Denis Babchuk f2b8d4b9cb fix(replication): forward FULL_OBJECT checksum type for multipart replicas
Signed-off-by: Denis Babchuk <babchukdenis@icloud.com>
2026-10-23 17:52:58 +03:00
Denis Babchuk 8ed5c90dcd fix(ilm): keep tier data when expiry only adds a delete marker
Signed-off-by: Denis Babchuk <babchukdenis@icloud.com>
2026-10-23 17:36:21 +03:00
Feng Ruohang 1b5477f3df fix(iam): isolate policy reloads and propagate KMS cancellation
Release the authentication-cache lock while retaining IAM write ordering. Carry request cancellation through IAM and SSE-S3 list decryption, and publish policy replacements only after a complete successful load.

Preserve caller deadlines for slow policy queries and keep the existing etcd read budget scoped to its RPC. Cover concurrent refresh, updates, revocation, cancellation, and deadline compatibility.

Validation: 36 focused and 23 adversarial scenarios each passed three race runs; real 32-second object/etcd probes passed. Full suite passed with the Go DNS resolver after reproducing native DNS failures on the baseline; existing etcd suites, make verifiers, and make build also passed.
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-23 15:50:12 +08:00
Feng Ruohang 625c11e7e1 fix(iam): preserve revision errors for storable record names
Only classify a valid IAM key as absent when a length error accompanies an ASCII path component longer than 255 bytes. Normalized LDAP DNs satisfy this condition, while total-path errors and Unicode names can refer to existing revocation records.

Cover component boundaries, wrapped and storage errors, and real revocation records moved beneath a deep Darwin drive path.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-23 10:35:31 +08:00
GL 9faeb7a114 fix(iam): treat IAM record names too long for the drive as absent
Since RELEASE.2026-09-16T00-00-00Z, LDAP users whose normalized DN is
longer than 255 bytes cannot log in. NormDN hex-escapes every non-ASCII
byte (go-ldap DN.String()), so one Cyrillic letter takes six characters.
IAM looks up records named after the DN: the parent revision check added
with cmd/iam-revision.go reads config/iam/users/<DN>/identity.json, and
PolicyDBGet reads config/iam/policydb/sts-users/<DN>.json when the
mapping is not cached.

A drive rejects such an element in one of two ways. checkPathLength
refuses more than 255 counted characters, but outside Windows it never
counts a backslash, so an escaped DN usually passes it. The filesystem
then fails with ENAMETOOLONG once the parent directory exists.
toObjectErr reports both as ObjectNameInvalid, and the callers treat
only errConfigNotFound as "no record". As a result,
AssumeRoleWithLDAPIdentity, access-key creation and STS loading fail.

Such a record cannot have been stored. When the object layer's own name
validation (checkBucketAndObjectNames) accepts the path,
ObjectNameInvalid can only mean a length rejection, so report it as
errConfigNotFound. Invalid names and storage errors are returned
unchanged, and saving such a record still fails. The etcd store has no
such limit and is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: GL <g@lopatin.info>
2026-10-19 18:49:20 +03:00
Feng Ruohang 0cd388c666 fix(iam): persist and propagate scoped STS revocation
Enumerate durable STS identities for parent/type revocation, retain the
operation cutoff, and notify sibling caches for each committed deletion.
Repair missed notifications during periodic object/etcd IAM refresh.

Carry immutable STS tombstones and expiry through the existing IAM revision
protocol and deletion healing. Delayed creation cannot undo revocation,
while unrelated and newly issued keys stay valid. Keep partial failures
visible while propagating the successfully committed deletions.

Cover uncached credentials, warm sibling caches, dropped notifications,
object/etcd backends, reordered replay, concurrent issuance and partial
persistence failure. No new dependencies or authentication-path I/O.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-18 22:39:12 +08:00
Feng Ruohang 1748f62c7e fix(pools): preserve delete markers and recheck migration cleanup
Move solitary and null delete markers instead of dropping them, including
idempotent UUID marker retries. Check that a reader's source version still
exists before moving it, and preserve the established read winner for
conflicting timestamp ties.

Retire only the versions that were successfully moved while holding the
pooled object lock. Recheck the source and retained copies before cleanup,
so concurrent deletes and uncertain destination reads cannot revive data.
Count remaining markers when verifying decommission completion.

Add two/three-pool walker, history, purge, concurrent-delete, read-fault,
and timestamp-tie regressions.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-18 22:39:12 +08:00
Denis Babchuk 60c7179ddb fix(pools): do not let data movement overwrite a newer copy
Apply PR #248 from bca40a4ac9.

Signed-off-by: Denis Babchuk <babchukdenis@icloud.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-18 22:39:07 +08:00