70914759e refused an edit whose stored record no longer matched the
caller's view, but left the stale copy in the serving server's cache in
two cases: a service-account edit bound to a parent whose account was
deleted and created again for another parent, and a replicated edit older
than what is stored, which site replication acknowledges. That server
kept authenticating the old credential until a notification or the
periodic load.
Both refusals now drop the cache entry of the addressed kind, as a
refusal for an absent or deleted identity does: the service-account
closure reports such a record as gone (callers still get
errNoSuchServiceAccount), and editStoredIdentity also drops the entry on
errIAMStaleUpdate. The LDAP group refresh again treats a stale update as
a refusal that writes nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
UpdateServiceAccount, SetUserStatus and UpdateUserSecretKey read the
identity from the serving server's cache, changed it and wrote it back
whole. On a server that missed a notification, an ordinary edit (a
description, a status, a secret) undid a rotation made through another
server and brought back a user or service account deleted meanwhile, with
its old secret.
Edits now go through iamIdentityEdit, inside the identity's cross-node
revision lock: the stored record is loaded and normalized as the loaders
do (access key from the path, claims from the token, a deprecated comment
for an empty description), the edit applies to it, and an absent or
deleted identity is refused and dropped from the cache of the server
that tried. The LDAP group refresh of a0de92fcb uses the same mechanism.
- The admin handler binds a service-account edit to the parent it
authorized: an account deleted and created again for another parent
meanwhile is not edited (parentUser in updateServiceAccountOpts;
replication does not set it).
- A replicated edit older than a deletion this site applied is answered
as stale, so site replication acknowledges it instead of retrying.
- SetUserStatus refuses a service or temporary credential as before, and
a refused edit drops only a cache entry of the addressed kind.
- A session-policy edit stores the claims of the token it signs; the
token is signed once.
- UpdateUserSecretKey addressed to a service-account key finds no user
instead of writing the account to the regular-user path.
Tests use a second IAM store over the same storage whose cache missed the
change: rotation, deletion, status and secret edits of users, parent
binding, legacy records, replicated edits around a deletion, and status
calls on a service account. Against the cache-based writers every case
fails.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
The hourly LDAP group refresh, which every server runs, reads the STS and
service-account credentials from its own cache, queries the directory,
and wrote each credential whose groups changed back whole through
UpdateUserIdentity. A secret rotation or an edit made meanwhile, on this
server or one whose notification it missed, was undone, and a credential
deleted meanwhile came back enabled with its old secret.
RefreshUserGroups replaces that writer, whose only caller was the
refresh. Under the identity's cross-node revision lock it loads the
stored record, refuses one that is absent, deleted, expired or now has
another parent (whose groups the lookup did not compute), and otherwise
changes only the groups of the stored record. A record without its
access key gets it from the path, as the loaders do. A refused refresh
drops the stale entry from the cache, which loads what is stored on next
use; a written or unchanged one caches the stored record. An STS record
keeps the lease of its token's remaining lifetime.
Tests cover rotation and edit, deletion, a sibling server whose cache did
not hear of a rotation or a deletion, recreation under the same and under
another parent, unchanged groups, and a stored record without its access
key. Written back as the old snapshot write, every case fails.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
Batch authorization evaluates the submitter's mapped policies from one IAM
cache revision. It took the cache write lock for the whole evaluation and
performed the lazy loads it needed under that lock: the mapping of a user
whose policies come from a group is never cached, so every authorization
of such a submitter (and of an LDAP or STS parent user) read storage twice
with the write lock held, and an evicted policy document was read under
it too. While a read lasted, every reader of the cache waited: request
authentication (GetUser, CheckKey) and PolicyDBGet on the whole node. An
expire job authorizes each version several times, so a large job with a
group-mapped submitter serialized node authentication behind IAM storage
latency, and a mapping load had no deadline at all.
batchPolicySnapshot now holds the store's writer mutex and the cache read
lock, and releases the read lock around each storage read, as
MergePolicies releases the cache lock around a document load. The writer
mutex still excludes cache writers, so the mapping and the documents of
one evaluation still come from one revision; the read lock is enough
because the batch path writes nothing to the cache except lazily loaded
mappings, which the request path stores under the read lock as well, and
taking the write lock back after each read would have made readers queue
behind it. Each read is bounded by the job's context and the same 5s
MergePolicies grants a document load. Concurrent reads of the group
fallback release the lock once and take it back after the last.
policyDBGet and policyDBGetGroups take the loads they need through the
iamMappingLoader interface, which *IAMStoreSys satisfies unchanged.
Unreadable policies still end the attempt as unavailable, and the error
classification is unchanged.
TestBatchAuthorizeSlowLoadKeepsCachedAuthorizationAvailable holds the
lazy load of a group-mapped user's mapping, and of an evicted document,
and requires that GetUser and IsAllowed of a cached user return within a
second and that a load ends with the job's context. Before this change
both cases timed out on the blocked cache.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
Release the authentication-cache lock while retaining IAM write ordering. Carry request cancellation through IAM and SSE-S3 list decryption, and publish policy replacements only after a complete successful load.
Preserve caller deadlines for slow policy queries and keep the existing etcd read budget scoped to its RPC. Cover concurrent refresh, updates, revocation, cancellation, and deadline compatibility.
Validation: 36 focused and 23 adversarial scenarios each passed three race runs; real 32-second object/etcd probes passed. Full suite passed with the Go DNS resolver after reproducing native DNS failures on the baseline; existing etcd suites, make verifiers, and make build also passed.
Signed-off-by: Feng Ruohang <rh@vonng.com>
Enumerate durable STS identities for parent/type revocation, retain the
operation cutoff, and notify sibling caches for each committed deletion.
Repair missed notifications during periodic object/etcd IAM refresh.
Carry immutable STS tombstones and expiry through the existing IAM revision
protocol and deletion healing. Delayed creation cannot undo revocation,
while unrelated and newly issued keys stay valid. Keep partial failures
visible while propagating the successfully committed deletions.
Cover uncached credentials, warm sibling caches, dropped notifications,
object/etcd backends, reordered replay, concurrent issuance and partial
persistence failure. No new dependencies or authentication-path I/O.
Signed-off-by: Feng Ruohang <rh@vonng.com>
Retain source-ordered tombstones and parent grant boundaries across both IAM backends, cache reloads, and deliberate identity recreation. Reconcile deletions through a versioned, bounded replication protocol with restart-aware acknowledgements.
Cover inherited group grants, STS retention, same-key service recreation, absolute expiration, and failures after the durable commit. Document coordinated upgrades and the remaining consistency boundaries.
Signed-off-by: Feng Ruohang <rh@vonng.com>
Adopt silo-pkg v3.13.2, the 20260903 mcli release, and the latest validated SILO Console commit. Move maintained source imports to the pgsty/silo-pkg module path, refresh the resulting dependency closure and credits, and keep only documented legacy minio/pkg transitive residue.
Signed-off-by: Feng Ruohang <rh@vonng.com>
`go run golang.org/x/tools/gopls/internal/analysis/modernize/cmd/modernize@latest -fix -test ./...` executed.
`go generate ./...` ran afterwards to keep generated.
When Keycloak vendor is set, the code will start to clean up service
accounts that parents do not exist anymore. However, the code will also
look for the parent user of site-replicator-0, MINIO_ROOT_USER, which
obviously does not exist in Keycloak. Therefore, the site-replicator-0
will be removed automatically.
This commit will avoid cleaning up service accounts generated from
the root user.
Tests if imported service accounts have
required access to buckets and objects.
Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
Co-authored-by: Harshavardhana <harsha@minio.io>
if a user policy is found, avoid reading from the drives
for missing group mappings, group mappings are not mandatory
and conditional.
This PR restores the older behavior while making sure that
if a direct user policy is not found, we would still attempt
to load from the group from the drives.
If the site replication is enabled and the code tries to extract jwt
claims while the site replication service account credentials are still
not loaded yet, the code will enter an infinite loop, causing in a
high CPU usage.
Another possibility of the infinite loop is having some service accounts
created by an old deployment version where the service account JWT was
signed by the root credentials, but not anymore.
This commit will remove the possibility of the infinite loop in the code
and add root credential fallback to extract claims from old service
accounts.
Hot load a policy document when during account authorization evaluation
to avoid returning 403 during server startup, when not all policies are
already loaded.
Add this support for group policies as well.
avoid concurrent callers for LoadUser() to even initiate
object read() requests, if an on-going operation is in progress.
this avoids many callers hitting the drives causing I/O
spikes, also allows for loading credentials faster.
the reason for this is to avoid STS mappings to be
purged without a successful load of other policies,
and all the credentials only loaded successfully
are properly handled.
This also avoids unnecessary cache store which was
implemented earlier for optimization.
fix: authenticate LDAP via actual DN instead of normalized DN
Normalized DN is only for internal representation, not for
external communication, any communication to LDAP must be
based on actual user DN. LDAP servers do not understand
normalized DN.
fixes#19757
This change uses the updated ldap library in minio/pkg (bumped
up to v3). A new config parameter is added for LDAP configuration to
specify extra user attributes to load from the LDAP server and to store
them as additional claims for the user.
A test is added in sts_handlers.go that shows how to access the LDAP
attributes as a claim.
This is in preparation for adding SSH pubkey authentication to MinIO's SFTP
integration.
Create new code paths for multiple subsystems in the code. This will
make maintaing this easier later.
Also introduce bugLogIf() for errors that should not happen in the first
place.
IAM loading is a lazy operation, allow these
fallbacks to be in place when we cannot find
in-memory state().
this allows us to honor the request even if pay
a small price for lookup and populating the data.
This change is to decouple need for root credentials to match between
site replication deployments.
Also ensuring site replication config initialization is re-tried until
it succeeds, this deoendency is critical to STS flow in site replication
scenario.
Feng Ruohang