Compare commits

..

16 Commits

Author SHA1 Message Date
Feng Ruohang 2616d43fbc fix(iam): enforce admin credential authorization boundaries
Honor self-service authorization denials, prevent session-limited credential delegation and updates, and protect root credential mutations under built-in IAM. Preserve empty-policy inheritance and damaged-account cleanup, normalize new service policy claims at issuance, and add signed Admin API and LDAP regressions.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-16 21:49:59 +08:00
Feng Ruohang 00a375c4e2 Merge pull request #238 from pgsty/codex/checksum-error-followup
Embed the Console versioning error fix at e5b165e4c0ee and add regression coverage for the algorithm-specific checksum messages merged in #229. Go CI, vulnerability checks, DCO and release-pipeline validation passed; the actual embedded Console and S3 checksum responses were also verified locally.

Fixes #231.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-15 21:47:26 +08:00
Feng Ruohang ab4c97e607 fix: embed Console versioning error handling and cover checksum messages
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-15 21:33:02 +08:00
Feng Ruohang 721dbfd1b7 Merge pull request #229 from yupoet/fix/putobject-checksum-error-message-226
Preserve the checksum rejection status and code while naming the checksum algorithm in PutObject and UploadPart error messages.

Validated on the branch updated to main: all six Go CI jobs and VulnCheck passed. Additional local coverage checked all five algorithms, wrapped errors, trailing checksums, multipart count mismatches and the generic fallback.

The maintainer explicitly accepted this contribution without the original author's DCO trailer as a one-time exception. Original authorship and the original commit are retained; no sign-off has been added on the author's behalf.

Fixes #226

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-15 21:22:45 +08:00
Feng Ruohang 2926474688 Merge branch 'main' into fix/putobject-checksum-error-message-226 2026-10-15 21:10:14 +08:00
Feng Ruohang a7370aa9ac Merge pull request #236 from clown15/fix-cleanup-stale-uploads-modtime
Parse native multipart directory timestamps before the Base64 compatibility path. Cover ambiguous plaintext IDs, cleanup metadata reads, expiration, and conservative legacy fallback.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-15 18:31:35 +08:00
Feng Ruohang a592d69f45 Merge pull request #232 from julienlau/julienlau/fix/commit-metadata-missing-503
Classify missing final-commit metadata as a retryable write failure, preserving storage RPC and quorum behavior. Cover real staged-metadata loss, API responses, successful quorums, and rollback boundaries.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-15 18:30:01 +08:00
Feng Ruohang bfc7906064 fix: prefer native multipart timestamps during cleanup
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-15 18:19:37 +08:00
Feng Ruohang eeff112491 test: verify commit metadata loss and write quorum boundaries
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-15 18:18:45 +08:00
Feng Ruohang a70156e45b Merge remote-tracking branch 'origin/main' into codex/pr236-maintainer-completion
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-15 18:12:09 +08:00
Feng Ruohang 99245e3ef8 Merge remote-tracking branch 'origin/main' into codex/pr232-maintainer-completion
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-15 18:12:09 +08:00
Feng Ruohang 7a3ca0f821 build: gate releases on Object Lock regressions (#237)
Run the existing Object Lock authorization and overwrite race regressions before release state changes, builds, packages, and uploads.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-10-15 13:20:15 +08:00
clown15 2d57b975d1 cmd: fix fast-path modtime parsing in cleanupStaleUploadsOnDisk
Upload ID directories on disk are named as '<UUID>x<UnixNano>'.
Attempting to decode them as Base64 fails unconditionally, causing
the cleanup routine to bypass the timestamp check and fall back to
disk.ReadVersion on every active upload.

Directly parse the timestamp suffix from the plaintext directory
name to restore O(1) stale upload detection, while preserving Base64
decoding and ReadVersion fallback for legacy uploads.

Signed-off-by: clown15 <thsdhrhd15@naver.com>
2026-10-14 08:55:51 +00:00
Julien Laurenceau cb85461c42 test: cover renameData commit-metadata-missing conversion
Drive renameData() with stub disks to assert that an in-quorum
errCommitMetaMissing is converted to errErasureWriteQuorum (HTTP 503)
and that an in-quorum errFileNotFound still surfaces unchanged so
PutObject() keeps returning IncompleteBody for client disconnects.

Signed-off-by: Julien Laurenceau <julien.laurenceau@pepitedata.com>
2026-10-12 17:24:40 +02:00
Julien Laurenceau f480086b6b fix: return 503 instead of 404 when commit metadata is missing
When the staged xl.meta disappears between the data rename and the
meta-file commit inside RenameData(), the affected drives return
errFileNotFound. An in-quorum errFileNotFound is reduced by
reduceWriteQuorumErrs() and previously surfaced as:

- IncompleteBody (400) from PutObject(), misattributing the failure
  to a client disconnect, or
- ObjectNotFound (404 NoSuchKey) from CompleteMultipartUpload(),
  telling clients the object is absent so they skip retries.

Map errFileNotFound at the commit step to a dedicated internal
errCommitMetaMissing, convert it to errErasureWriteQuorum in
renameData(), and map it to InsufficientWriteQuorum (HTTP 503) in
toObjectErr() so clients retry. Below-quorum outcomes already return
InsufficientWriteQuorum and are unchanged, as is the IncompleteBody
mapping for staging failures of client-disconnected uploads.

Signed-off-by: Julien Laurenceau <julien.laurenceau@pepitedata.com>
2026-10-12 16:11:06 +02:00
Paris Yu 968c30238c fix: name the checksum algorithm in PutObject/UploadPart mismatch errors
hash.ChecksumMismatch now carries the requested algorithm (CRC32, CRC32C,
SHA1, SHA256, CRC64NVME), and toAPIError renders it S3-style:

  The CRC32 you specified did not match the calculated checksum.

instead of the generic "The provided 'x-amz-checksum' header does not match
what was computed.", for all four mismatch construction sites (streaming
reader, trailing-checksum decode, and both Checksum.Matches paths). Mirrors
the wording already shipped for CompleteMultipartUpload in #225.

Fixes #226
2026-09-26 20:37:20 +08:00
23 changed files with 1257 additions and 113 deletions
+3 -1
View File
@@ -249,8 +249,10 @@ func (a adminAPIHandlers) AddServiceAccountLDAP(w http.ResponseWriter, r *http.R
// In case of LDAP/OIDC we need to set `opts.claims` to ensure
// it is associated with the LDAP/OIDC user properly.
// The new credential's policy comes from the request, not the caller.
for k, v := range cred.Claims {
if k == expClaim {
switch k {
case expClaim, policy.SessionPolicyName, sessionPolicyNameExtracted, iamPolicyClaimNameSA():
continue
}
opts.claims[k] = v
+507
View File
@@ -0,0 +1,507 @@
// Copyright (c) 2026 PGSTY
// SPDX-License-Identifier: AGPL-3.0-or-later
package cmd
import (
"bytes"
"context"
"encoding/base64"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"strings"
"sync/atomic"
"testing"
"time"
"github.com/minio/madmin-go/v3"
"github.com/minio/minio/internal/auth"
polplugin "github.com/minio/minio/internal/config/policy/plugin"
"github.com/minio/mux"
xnet "github.com/pgsty/silo-pkg/v3/net"
"github.com/pgsty/silo-pkg/v3/policy"
)
func iamAdminFixture(t *testing.T) (context.Context, *IAMSys) {
t.Helper()
ctx, sys, obj := prepareIAMRevisionFixture(t)
globalObjLayerMutex.Lock()
globalObjectAPI = obj
globalObjLayerMutex.Unlock()
return ctx, sys
}
func iamAdminUser(ctx context.Context, t *testing.T, sys *IAMSys, name, policyName string) auth.Credentials {
t.Helper()
cred := auth.Credentials{AccessKey: name, SecretKey: "review-user-valid-secret"}
_, err := sys.CreateUser(ctx, name, madmin.AddOrUpdateUserReq{SecretKey: cred.SecretKey, Status: madmin.AccountEnabled})
mustIAM(t, err)
if policyName != "" {
_, err = sys.PolicyDBSet(ctx, name, policyName, regUser, false)
mustIAM(t, err)
}
return cred
}
func iamAdminServiceAccount(ctx context.Context, t *testing.T, sys *IAMSys, parent, key, document string) auth.Credentials {
t.Helper()
var sp *policy.Policy
var err error
if document != "" {
sp, err = policy.ParseConfig(strings.NewReader(document))
mustIAM(t, err)
}
cred, _, err := sys.NewServiceAccount(ctx, parent, nil, newServiceAccountOpts{accessKey: key, secretKey: "review-service-valid-secret", sessionPolicy: sp})
mustIAM(t, err)
return cred
}
func iamAdminRequest(t *testing.T, cred auth.Credentials, method, target string, payload any) *http.Request {
t.Helper()
var body []byte
if payload != nil {
raw, err := json.Marshal(payload)
mustIAM(t, err)
body, err = madmin.EncryptData(cred.SecretKey, raw)
mustIAM(t, err)
}
headers := map[string]string{}
if cred.IsTemp() && !cred.IsServiceAccount() {
headers["X-Amz-Security-Token"] = cred.SessionToken
}
req, err := newTestSignedRequestV4(method, "http://localhost"+target, int64(len(body)), bytes.NewReader(body), cred.AccessKey, cred.SecretKey, headers)
mustIAM(t, err)
mustIAM(t, req.ParseForm())
return req
}
func iamAdminCall(t *testing.T, cred auth.Credentials, method, target string, payload any, vars map[string]string, handler http.HandlerFunc) *httptest.ResponseRecorder {
t.Helper()
req := iamAdminRequest(t, cred, method, target, payload)
req = mux.SetURLVars(req, vars)
rec := httptest.NewRecorder()
req = req.WithContext(newContext(req, rec, "IAMAdminAuthorization"))
handler(rec, req)
return rec
}
func assertIAMAdminStatus(t *testing.T, operation string, rec *httptest.ResponseRecorder, want int) {
t.Helper()
if rec.Code != want {
t.Errorf("%s status=%d want=%d body=%s", operation, rec.Code, want, rec.Body.String())
return
}
if want == http.StatusForbidden || want == http.StatusNotFound {
var response APIErrorResponse
mustIAM(t, json.Unmarshal(rec.Body.Bytes(), &response))
wantCode := errorCodes.ToAPIErr(ErrAccessDenied).Code
if want == http.StatusNotFound {
wantCode = errorCodes.ToAPIErr(ErrAdminServiceAccountNotFound).Code
}
if response.Code != wantCode {
t.Errorf("%s error code=%s want=%s", operation, response.Code, wantCode)
}
}
}
func iamAdminPlugin(t *testing.T, allow bool) {
t.Helper()
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_ = json.NewEncoder(w).Encode(map[string]bool{"result": allow})
}))
t.Cleanup(server.Close)
u, err := xnet.ParseHTTPURL(server.URL)
mustIAM(t, err)
previous := newGlobalAuthZPluginFn()
setGlobalAuthZPlugin(polplugin.New(polplugin.Args{URL: u, CloseRespFn: func(r io.ReadCloser) { r.Close() }}))
t.Cleanup(func() { setGlobalAuthZPlugin(previous) })
}
func TestIAMAdminRootEmptyPolicy(t *testing.T) {
for _, doc := range []string{"", "null", "{}", `{"Statement":null}`} {
t.Run("policy="+doc, func(t *testing.T) {
ctx, sys := iamAdminFixture(t)
caller := iamAdminServiceAccount(ctx, t, sys, globalActiveCred.AccessKey, "review-root-empty", doc)
sibling := iamAdminServiceAccount(ctx, t, sys, globalActiveCred.AccessKey, "review-root-sibling", "")
req := iamAdminRequest(t, caller, "GET", "/minio/admin/v3/list-service-accounts", nil)
checked, owner, code := checkKeyValid(req, caller.AccessKey)
if code != ErrNone {
t.Fatalf("checkKeyValid: %v", code)
}
active, _ := isAllowedBySessionPolicyForServiceAccount(policy.Args{Claims: checked.Claims})
t.Logf("owner=%v parsed_active_session_policy=%v", owner, active)
api := adminAPIHandlers{}
assertIAMAdminStatus(t, "create", iamAdminCall(t, caller, "PUT", "/minio/admin/v3/add-service-account", madmin.AddServiceAccountReq{AccessKey: "review-root-new", SecretKey: "review-service-valid-secret"}, nil, api.AddServiceAccount), 200)
assertIAMAdminStatus(t, "update", iamAdminCall(t, caller, "POST", "/minio/admin/v3/update-service-account?accessKey="+sibling.AccessKey, madmin.UpdateServiceAccountReq{NewName: "review"}, map[string]string{"accessKey": sibling.AccessKey}, api.UpdateServiceAccount), 204)
assertIAMAdminStatus(t, "delete", iamAdminCall(t, caller, "DELETE", "/minio/admin/v3/delete-service-account?accessKey="+sibling.AccessKey, nil, map[string]string{"accessKey": sibling.AccessKey}, api.DeleteServiceAccount), 204)
assertIAMAdminStatus(t, "revoke", iamAdminCall(t, caller, "POST", "/minio/admin/v3/revoke-tokens/builtin?fullRevoke=true", nil, map[string]string{"userProvider": madmin.BuiltinProvider}, api.RevokeTokens), 204)
})
}
}
func TestIAMAdminEmptyPolicyDelegation(t *testing.T) {
for _, parentType := range []string{"root", "user"} {
for _, document := range []string{"null", "{}", `{"Statement":null}`} {
t.Run(parentType+"/"+document, func(t *testing.T) {
ctx, sys := iamAdminFixture(t)
parent := globalActiveCred
if parentType == "user" {
parent = iamAdminUser(ctx, t, sys, "empty-policy-parent", "readwrite")
}
sp, err := policy.ParseConfig(strings.NewReader(document))
mustIAM(t, err)
caller, _, err := sys.NewServiceAccount(ctx, parent.AccessKey, nil, newServiceAccountOpts{
accessKey: "empty-policy-caller", secretKey: "empty-policy-caller-secret", sessionPolicy: sp,
claims: map[string]any{"identity-attribute": "preserved"},
})
mustIAM(t, err)
const limitedPolicy = `{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":["admin:ListServiceAccounts"]}]}`
for _, tc := range []struct {
policy string
listStatus int
allowObject bool
}{
{"", http.StatusOK, true},
{"null", http.StatusOK, true},
{"{}", http.StatusOK, true},
{`{"Statement":null}`, http.StatusOK, true},
{limitedPolicy, http.StatusOK, false},
{`{"Version":"2012-10-17"}`, http.StatusForbidden, false},
{`{"Statement":[]}`, http.StatusForbidden, false},
} {
expires := UTCNow().Add(time.Hour)
req := madmin.AddServiceAccountReq{Expiration: &expires}
if tc.policy != "" {
req.Policy = json.RawMessage(tc.policy)
}
api := adminAPIHandlers{}
rec := iamAdminCall(t, caller, "PUT", "/minio/admin/v3/add-service-account", req, nil, api.AddServiceAccount)
assertIAMAdminStatus(t, "create expiring child", rec, http.StatusOK)
data, err := madmin.DecryptData(caller.SecretKey, rec.Body)
mustIAM(t, err)
var response madmin.AddServiceAccountResp
mustIAM(t, json.Unmarshal(data, &response))
child, ok := sys.store.GetUser(response.Credentials.AccessKey)
if !ok {
t.Fatal("new service account missing")
}
// Exercise the issued key through authentication and authorization.
assertIAMAdminStatus(t, "use expiring child policy="+tc.policy, iamAdminCall(t, child.Credentials, "GET", "/minio/admin/v3/list-service-accounts", nil, nil, api.ListServiceAccounts), tc.listStatus)
signed := iamAdminRequest(t, child.Credentials, "GET", "/example/object", nil)
checked, owner, code := checkKeyValid(signed, child.Credentials.AccessKey)
if code != ErrNone {
t.Fatalf("child authentication: %v", code)
}
if checked.Claims["identity-attribute"] != "preserved" {
t.Fatal("creation lost the caller's identity context")
}
allowed := sys.IsAllowed(policy.Args{AccountName: checked.AccessKey, Claims: checked.Claims, IsOwner: owner, Action: policy.GetObjectAction, BucketName: "example", ObjectName: "object"})
if allowed != tc.allowObject {
t.Errorf("GetObject allowed=%v with requested policy=%s", allowed, tc.policy)
}
}
})
}
}
}
func TestIAMAdminCorruptServiceCleanup(t *testing.T) {
for _, status := range []string{auth.AccountOn, auth.AccountOff, "expired"} {
t.Run(status, func(t *testing.T) {
ctx, sys := iamAdminFixture(t)
caller := iamAdminUser(ctx, t, sys, "review-admin", "consoleAdmin")
parent := iamAdminUser(ctx, t, sys, "review-parent", "readwrite")
svc := iamAdminServiceAccount(ctx, t, sys, parent.AccessKey, "rv-corrupt", "")
if status == auth.AccountOff {
_, err := sys.UpdateServiceAccount(ctx, svc.AccessKey, updateServiceAccountOpts{status: status})
mustIAM(t, err)
}
cache := sys.store.lock()
u := cache.iamUsersMap[svc.AccessKey]
u.Credentials.SessionToken = "broken-session-token"
if status == "expired" {
u.Credentials.Expiration = UTCNow().Add(-time.Minute)
}
cache.iamUsersMap[svc.AccessKey] = u
sys.store.unlock()
_, _, err := sys.GetServiceAccount(ctx, svc.AccessKey)
if err == nil {
t.Fatal("fault injection failed")
}
raw, exists := sys.store.GetUser(svc.AccessKey)
_, loginOK := sys.GetUser(ctx, svc.AccessKey)
t.Logf("metadata_exists=%v parent_known=%v GetUser_ok=%v", exists, raw.Credentials.ParentUser == parent.AccessKey, loginOK)
api := adminAPIHandlers{}
outsider := iamAdminUser(ctx, t, sys, "cleanup-outsider", "readwrite")
assertIAMAdminStatus(t, "unauthorized cleanup", iamAdminCall(t, outsider, "DELETE", "/minio/admin/v3/delete-service-account?accessKey="+svc.AccessKey, nil, map[string]string{"accessKey": svc.AccessKey}, api.DeleteServiceAccount), 404)
assertIAMAdminStatus(t, "admin cleanup", iamAdminCall(t, caller, "DELETE", "/minio/admin/v3/delete-service-account?accessKey="+svc.AccessKey, nil, map[string]string{"accessKey": svc.AccessKey}, api.DeleteServiceAccount), 204)
if _, ok := sys.store.GetUser(svc.AccessKey); ok {
t.Fatal("deleted service account remains in the IAM cache")
}
})
}
}
func TestIAMAdminPolicylessSelf(t *testing.T) {
ctx, sys := iamAdminFixture(t)
caller := iamAdminUser(ctx, t, sys, "review-policyless", "")
svc := iamAdminServiceAccount(ctx, t, sys, caller.AccessKey, "rv-policyless-child", "")
api := adminAPIHandlers{}
assertIAMAdminStatus(t, "list", iamAdminCall(t, caller, "GET", "/minio/admin/v3/list-service-accounts", nil, nil, api.ListServiceAccounts), 200)
assertIAMAdminStatus(t, "info service", iamAdminCall(t, caller, "GET", "/minio/admin/v3/info-service-account?accessKey="+svc.AccessKey, nil, map[string]string{"accessKey": svc.AccessKey}, api.InfoServiceAccount), 200)
assertIAMAdminStatus(t, "info access key", iamAdminCall(t, caller, "GET", "/minio/admin/v3/info-access-key?accessKey="+svc.AccessKey, nil, map[string]string{"accessKey": svc.AccessKey}, api.InfoAccessKey), 200)
assertIAMAdminStatus(t, "delete", iamAdminCall(t, caller, "DELETE", "/minio/admin/v3/delete-service-account?accessKey="+svc.AccessKey, nil, map[string]string{"accessKey": svc.AccessKey}, api.DeleteServiceAccount), 204)
assertIAMAdminStatus(t, "revoke", iamAdminCall(t, caller, "POST", "/minio/admin/v3/revoke-tokens/builtin?fullRevoke=true", nil, map[string]string{"userProvider": madmin.BuiltinProvider}, api.RevokeTokens), 204)
assertIAMAdminStatus(t, "list another user", iamAdminCall(t, caller, "GET", "/minio/admin/v3/list-service-accounts?user="+globalActiveCred.AccessKey, nil, nil, api.ListServiceAccounts), 403)
assertIAMAdminStatus(t, "revoke another user", iamAdminCall(t, caller, "POST", "/minio/admin/v3/revoke-tokens/builtin?fullRevoke=true&user="+globalActiveCred.AccessKey, nil, map[string]string{"userProvider": madmin.BuiltinProvider}, api.RevokeTokens), 403)
}
func TestIAMAdminExpiringInheritedPolicy(t *testing.T) {
ctx, sys := iamAdminFixture(t)
parent := iamAdminUser(ctx, t, sys, "review-legacy-parent", "consoleAdmin")
document := `{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":["admin:CreateServiceAccount","admin:UpdateServiceAccount"]}]}`
// Model a credential persisted by the old handler: it copied its caller's
// policy claims but set the newly created service's type to inherited.
expires := UTCNow().Add(time.Hour)
caller, _, err := sys.NewServiceAccount(ctx, parent.AccessKey, nil, newServiceAccountOpts{
accessKey: "rv-exp-inherited", secretKey: "review-service-valid-secret", expiration: &expires,
claims: map[string]any{policy.SessionPolicyName: base64.StdEncoding.EncodeToString([]byte(document))},
})
mustIAM(t, err)
req := iamAdminRequest(t, caller, "GET", "/minio/admin/v3/list-service-accounts", nil)
checked, owner, code := checkKeyValid(req, caller.AccessKey)
if code != ErrNone {
t.Fatalf("checkKeyValid: %v", code)
}
args := policy.Args{AccountName: caller.AccessKey, Claims: checked.Claims, IsOwner: owner, Action: policy.CreateServiceAccountAdminAction, DenyOnly: true}
if !sys.IsAllowed(args) {
t.Fatal("fixture cannot create")
}
args.Action = policy.CreateUserAdminAction
args.DenyOnly = false
if sys.IsAllowed(args) {
t.Fatal("fixture not constrained")
}
t.Logf("service=%v temp=%v inherited=%v; create allowed, create-user denied", checked.IsServiceAccount(), checked.IsTemp(), checked.Claims[iamPolicyClaimNameSA()] == inheritedPolicyType)
api := adminAPIHandlers{}
rec := iamAdminCall(t, caller, "PUT", "/minio/admin/v3/add-service-account", madmin.AddServiceAccountReq{AccessKey: "review-escaped", SecretKey: "review-service-valid-secret"}, nil, api.AddServiceAccount)
if rec.Code == 200 {
child, ok := sys.GetUser(ctx, "review-escaped")
if !ok {
t.Fatal("new key missing")
}
childReq := iamAdminRequest(t, child.Credentials, "GET", "/minio/admin/v3/list-service-accounts", nil)
childChecked, childOwner, childCode := checkKeyValid(childReq, child.Credentials.AccessKey)
if childCode != ErrNone {
t.Fatalf("child auth=%v", childCode)
}
allowed := sys.IsAllowed(policy.Args{AccountName: childChecked.AccessKey, Claims: childChecked.Claims, IsOwner: childOwner, Action: policy.CreateUserAdminAction})
t.Errorf("constrained inherited service created a new key; new key CreateUser allowed=%v", allowed)
} else {
assertIAMAdminStatus(t, "reject derived creation", rec, 403)
}
updated := iamAdminCall(t, caller, "POST", "/minio/admin/v3/update-service-account?accessKey="+caller.AccessKey, madmin.UpdateServiceAccountReq{NewName: "metadata-only"}, map[string]string{"accessKey": caller.AccessKey}, api.UpdateServiceAccount)
if updated.Code == 204 {
after, ok := sys.GetUser(ctx, caller.AccessKey)
if !ok {
t.Fatal("updated key missing")
}
afterReq := iamAdminRequest(t, after.Credentials, "GET", "/minio/admin/v3/list-service-accounts", nil)
afterChecked, afterOwner, afterCode := checkKeyValid(afterReq, caller.AccessKey)
if afterCode != ErrNone {
t.Fatalf("updated auth=%v", afterCode)
}
allowed := sys.IsAllowed(policy.Args{AccountName: caller.AccessKey, Claims: afterChecked.Claims, IsOwner: afterOwner, Action: policy.CreateUserAdminAction})
t.Errorf("constrained inherited service updated itself; CreateUser allowed after update=%v", allowed)
} else {
assertIAMAdminStatus(t, "reject inherited self update", updated, 403)
}
}
func TestIAMAdminSessionPolicyCannotDelegate(t *testing.T) {
for _, temp := range []bool{false, true} {
t.Run(map[bool]string{false: "service", true: "sts"}[temp], func(t *testing.T) {
ctx, sys := iamAdminFixture(t)
parent := iamAdminUser(ctx, t, sys, "rv-parent", "consoleAdmin")
doc := `{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":["admin:CreateServiceAccount","admin:UpdateServiceAccount"]}]}`
var caller auth.Credentials
if !temp {
caller = iamAdminServiceAccount(ctx, t, sys, parent.AccessKey, "review-restricted", doc)
} else {
var err error
caller, err = auth.GetNewCredentialsWithMetadata(map[string]any{parentClaim: parent.AccessKey, "exp": UTCNow().Add(time.Hour).Unix(), policy.SessionPolicyName: base64.StdEncoding.EncodeToString([]byte(doc))}, globalActiveCred.SecretKey)
mustIAM(t, err)
caller.ParentUser = parent.AccessKey
_, err = sys.SetTempUser(ctx, caller.AccessKey, caller, "")
mustIAM(t, err)
}
sibling := iamAdminServiceAccount(ctx, t, sys, parent.AccessKey, "rv-sibling", "")
api := adminAPIHandlers{}
assertIAMAdminStatus(t, "restrict create", iamAdminCall(t, caller, "PUT", "/minio/admin/v3/add-service-account", madmin.AddServiceAccountReq{AccessKey: "review-bad-new", SecretKey: "review-service-valid-secret"}, nil, api.AddServiceAccount), 403)
assertIAMAdminStatus(t, "restrict LDAP create", iamAdminCall(t, caller, "PUT", "/minio/admin/v3/idp/ldap/add-service-account", madmin.AddServiceAccountReq{AccessKey: "review-bad-new", SecretKey: "review-service-valid-secret"}, nil, api.AddServiceAccountLDAP), 403)
assertIAMAdminStatus(t, "restrict update", iamAdminCall(t, caller, "POST", "/minio/admin/v3/update-service-account?accessKey="+sibling.AccessKey, madmin.UpdateServiceAccountReq{NewSecretKey: "review-known-new-secret"}, map[string]string{"accessKey": sibling.AccessKey}, api.UpdateServiceAccount), 403)
if _, ok := sys.store.GetUser("review-bad-new"); ok {
t.Fatal("denied creation persisted a credential")
}
u, ok := sys.store.GetUser(sibling.AccessKey)
if !ok || u.Credentials.SecretKey != sibling.SecretKey {
t.Fatal("denied update changed the target credential")
}
})
}
}
func TestIAMAdminSelfDenials(t *testing.T) {
for _, mode := range []string{"parent_deny", "group_deny", "session_implicit_deny", "session_explicit_deny", "plugin_deny"} {
t.Run(mode, func(t *testing.T) {
ctx, sys := iamAdminFixture(t)
parent := iamAdminUser(ctx, t, sys, "rv-self-parent", "readwrite")
target := iamAdminServiceAccount(ctx, t, sys, parent.AccessKey, "rv-self-target", "")
caller := parent
doc := `{"Version":"2012-10-17","Statement":[{"Effect":"Deny","Action":["admin:ListServiceAccounts","admin:RemoveServiceAccount"]}]}`
switch mode {
case "parent_deny", "group_deny":
p, err := policy.ParseConfig(strings.NewReader(doc))
mustIAM(t, err)
_, err = sys.SetPolicy(ctx, "review-self-deny", *p)
mustIAM(t, err)
if mode == "group_deny" {
_, err = sys.AddUsersToGroup(ctx, "self-deny-group", []string{parent.AccessKey})
mustIAM(t, err)
_, err = sys.PolicyDBSet(ctx, "self-deny-group", "review-self-deny", regUser, true)
} else {
_, err = sys.PolicyDBSet(ctx, parent.AccessKey, "readwrite,review-self-deny", regUser, false)
}
mustIAM(t, err)
case "session_implicit_deny":
caller = iamAdminServiceAccount(ctx, t, sys, parent.AccessKey, "rv-self-caller", `{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":["s3:GetObject"],"Resource":["arn:aws:s3:::example/*"]}]}`)
case "session_explicit_deny":
caller = iamAdminServiceAccount(ctx, t, sys, parent.AccessKey, "rv-self-caller", doc)
case "plugin_deny":
iamAdminPlugin(t, false)
}
api := adminAPIHandlers{}
assertIAMAdminStatus(t, "list", iamAdminCall(t, caller, "GET", "/minio/admin/v3/list-service-accounts", nil, nil, api.ListServiceAccounts), 403)
assertIAMAdminStatus(t, "info service", iamAdminCall(t, caller, "GET", "/minio/admin/v3/info-service-account?accessKey="+target.AccessKey, nil, map[string]string{"accessKey": target.AccessKey}, api.InfoServiceAccount), 403)
assertIAMAdminStatus(t, "info access key", iamAdminCall(t, caller, "GET", "/minio/admin/v3/info-access-key?accessKey="+target.AccessKey, nil, map[string]string{"accessKey": target.AccessKey}, api.InfoAccessKey), 403)
assertIAMAdminStatus(t, "delete", iamAdminCall(t, caller, "DELETE", "/minio/admin/v3/delete-service-account?accessKey="+target.AccessKey, nil, map[string]string{"accessKey": target.AccessKey}, api.DeleteServiceAccount), 403)
assertIAMAdminStatus(t, "revoke", iamAdminCall(t, caller, "POST", "/minio/admin/v3/revoke-tokens/builtin?fullRevoke=true", nil, map[string]string{"userProvider": madmin.BuiltinProvider}, api.RevokeTokens), 403)
})
}
}
func TestIAMAdminNonRootCannotManageRoot(t *testing.T) {
ctx, sys := iamAdminFixture(t)
caller := iamAdminUser(ctx, t, sys, "rv-console-admin", "consoleAdmin")
target := iamAdminServiceAccount(ctx, t, sys, globalActiveCred.AccessKey, "rv-root-target", "")
api := adminAPIHandlers{}
assertIAMAdminStatus(t, "create root", iamAdminCall(t, caller, "PUT", "/minio/admin/v3/add-service-account", madmin.AddServiceAccountReq{AccessKey: "rv-new-root", SecretKey: "review-service-valid-secret", TargetUser: globalActiveCred.AccessKey}, nil, api.AddServiceAccount), 403)
assertIAMAdminStatus(t, "update root", iamAdminCall(t, caller, "POST", "/minio/admin/v3/update-service-account?accessKey="+target.AccessKey, madmin.UpdateServiceAccountReq{NewSecretKey: "review-known-new-secret"}, map[string]string{"accessKey": target.AccessKey}, api.UpdateServiceAccount), 403)
assertIAMAdminStatus(t, "delete root", iamAdminCall(t, caller, "DELETE", "/minio/admin/v3/delete-service-account?accessKey="+target.AccessKey, nil, map[string]string{"accessKey": target.AccessKey}, api.DeleteServiceAccount), 404)
assertIAMAdminStatus(t, "revoke root", iamAdminCall(t, caller, "POST", "/minio/admin/v3/revoke-tokens/builtin?fullRevoke=true&user="+globalActiveCred.AccessKey, nil, map[string]string{"userProvider": madmin.BuiltinProvider}, api.RevokeTokens), 403)
// Restricted root-derived credentials and the site replicator are not root
// administrators, even when their effective policy allows these actions.
limited := iamAdminServiceAccount(ctx, t, sys, globalActiveCred.AccessKey, "limited-root", `{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":["admin:*"]}]}`)
assertIAMAdminStatus(t, "limited root delete", iamAdminCall(t, limited, "DELETE", "/minio/admin/v3/delete-service-account?accessKey="+target.AccessKey, nil, map[string]string{"accessKey": target.AccessKey}, api.DeleteServiceAccount), 404)
assertIAMAdminStatus(t, "limited root revoke", iamAdminCall(t, limited, "POST", "/minio/admin/v3/revoke-tokens/builtin?fullRevoke=true", nil, map[string]string{"userProvider": madmin.BuiltinProvider}, api.RevokeTokens), 403)
replicator, _, err := sys.NewServiceAccount(ctx, globalActiveCred.AccessKey, nil, newServiceAccountOpts{accessKey: siteReplicatorSvcAcc, secretKey: "replicator-valid-secret", allowSiteReplicatorAccount: true})
mustIAM(t, err)
assertIAMAdminStatus(t, "replicator create root", iamAdminCall(t, replicator, "PUT", "/minio/admin/v3/add-service-account", madmin.AddServiceAccountReq{}, nil, api.AddServiceAccount), 403)
// An external plugin remains authoritative, including for root targets.
iamAdminPlugin(t, true)
assertIAMAdminStatus(t, "plugin update root", iamAdminCall(t, limited, "POST", "/minio/admin/v3/update-service-account?accessKey="+target.AccessKey, madmin.UpdateServiceAccountReq{NewName: "plugin-authorized"}, map[string]string{"accessKey": target.AccessKey}, api.UpdateServiceAccount), 204)
assertIAMAdminStatus(t, "plugin create root", iamAdminCall(t, limited, "PUT", "/minio/admin/v3/add-service-account", madmin.AddServiceAccountReq{}, nil, api.AddServiceAccount), 200)
}
func TestIAMAdminDeleteCorruptServiceReplicates(t *testing.T) {
ctx, sys := iamAdminFixture(t)
caller := iamAdminUser(ctx, t, sys, "cleanup-admin", "consoleAdmin")
parent := iamAdminUser(ctx, t, sys, "cleanup-parent", "readwrite")
target := iamAdminServiceAccount(ctx, t, sys, parent.AccessKey, "cleanup-target", "")
cache := sys.store.lock()
u := cache.iamUsersMap[target.AccessKey]
u.Credentials.SessionToken = "broken-session-token"
u.Credentials.Status = auth.AccountOff
cache.iamUsersMap[target.AccessKey] = u
sys.store.unlock()
var deletions atomic.Int32
peer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/minio/health/live" {
return
}
var batch iamRevisionBatch
if err := json.NewDecoder(r.Body).Decode(&batch); err != nil {
t.Error(err)
w.WriteHeader(http.StatusBadRequest)
return
}
if len(batch.Items) != 1 || batch.Items[0].SvcAccChange == nil || batch.Items[0].SvcAccChange.Delete == nil || batch.Items[0].SvcAccChange.Delete.AccessKey != target.AccessKey {
t.Error("replication did not carry the service account deletion")
} else {
deletions.Add(1)
}
_ = json.NewEncoder(w).Encode(iamRevisionResponse{iamRevisionStatus: iamRevisionStatus{Version: iamRevisionProtocol, Node: "peer", Instance: "peer-boot", Digest: "deleted"}})
}))
t.Cleanup(peer.Close)
globalSiteReplicatorCred.RLock()
previousSecret := globalSiteReplicatorCred.secretKey
globalSiteReplicatorCred.RUnlock()
globalSiteReplicatorCred.Set(caller.SecretKey)
t.Cleanup(func() { globalSiteReplicatorCred.Set(previousSecret) })
globalSiteReplicationSys.Lock()
globalSiteReplicationSys.enabled = true
globalSiteReplicationSys.state = srState{ServiceAccountAccessKey: caller.AccessKey, Peers: map[string]madmin.PeerInfo{
globalDeploymentID(): {DeploymentID: globalDeploymentID(), Name: "local"},
"peer": {DeploymentID: "peer", Name: "peer", Endpoint: peer.URL},
}}
globalSiteReplicationSys.Unlock()
t.Cleanup(func() {
globalSiteReplicationSys.Lock()
globalSiteReplicationSys.enabled = false
globalSiteReplicationSys.Unlock()
})
api := adminAPIHandlers{}
assertIAMAdminStatus(t, "replicated cleanup", iamAdminCall(t, caller, "DELETE", "/minio/admin/v3/delete-service-account?accessKey="+target.AccessKey, nil, map[string]string{"accessKey": target.AccessKey}, api.DeleteServiceAccount), 204)
if deletions.Load() != 1 {
t.Fatalf("expected one immediate deletion notification, got %d", deletions.Load())
}
}
func TestIAMAdminListSelfAliases(t *testing.T) {
for _, kind := range []string{"service", "sts"} {
t.Run(kind, func(t *testing.T) {
ctx, sys := iamAdminFixture(t)
parent := iamAdminUser(ctx, t, sys, "alias-parent", "readwrite")
target := iamAdminServiceAccount(ctx, t, sys, parent.AccessKey, "alias-target", "")
caller := iamAdminServiceAccount(ctx, t, sys, parent.AccessKey, "alias-caller", "")
if kind == "sts" {
var err error
caller, err = auth.GetNewCredentialsWithMetadata(map[string]any{parentClaim: parent.AccessKey, "exp": UTCNow().Add(time.Hour).Unix()}, globalActiveCred.SecretKey)
mustIAM(t, err)
caller.ParentUser = parent.AccessKey
_, err = sys.SetTempUser(ctx, caller.AccessKey, caller, "")
mustIAM(t, err)
}
api := adminAPIHandlers{}
for _, alias := range []string{"", caller.AccessKey, parent.AccessKey} {
rec := iamAdminCall(t, caller, "GET", "/minio/admin/v3/list-service-accounts?user="+alias, nil, nil, api.ListServiceAccounts)
assertIAMAdminStatus(t, "list self alias", rec, 200)
data, err := madmin.DecryptData(caller.SecretKey, rec.Body)
mustIAM(t, err)
var response madmin.ListServiceAccountsResp
mustIAM(t, json.Unmarshal(data, &response))
found := false
for _, account := range response.Accounts {
found = found || account.AccessKey == target.AccessKey
}
if !found {
t.Fatalf("self alias %q did not list the parent's service accounts", alias)
}
}
})
}
}
+139 -82
View File
@@ -714,8 +714,10 @@ func (a adminAPIHandlers) AddServiceAccount(w http.ResponseWriter, r *http.Reque
// In case of LDAP/OIDC we need to set `opts.claims` to ensure
// it is associated with the LDAP/OIDC user properly.
// The new credential's policy comes from the request, not the caller.
for k, v := range cred.Claims {
if k == expClaim {
switch k {
case expClaim, policy.SessionPolicyName, sessionPolicyNameExtracted, iamPolicyClaimNameSA():
continue
}
opts.claims[k] = v
@@ -855,10 +857,9 @@ func (a adminAPIHandlers) UpdateServiceAccount(w http.ResponseWriter, r *http.Re
// Permission checks:
//
// 1. Any type of account (i.e. access keys (previously/still called service
// accounts), STS accounts, internal IDP accounts, etc) with the
// policy.UpdateServiceAccountAdminAction permission can update any service
// account.
// 1. Updating requires policy.UpdateServiceAccountAdminAction. Under built-in
// IAM, session-limited callers cannot update credentials, and only unrestricted
// root credentials can manage root's service accounts.
//
// 2. We would like to let a user update their own access keys, however it
// is currently blocked pending a re-design. Users are still able to delete
@@ -874,6 +875,11 @@ func (a adminAPIHandlers) UpdateServiceAccount(w http.ResponseWriter, r *http.Re
writeErrorResponseJSON(ctx, w, errorCodes.ToAPIErr(ErrAccessDenied), r.URL)
return
}
if newGlobalAuthZPluginFn() == nil && (hasActiveSessionPolicy(cred) ||
(svcAccount.ParentUser == globalActiveCred.AccessKey && !canManageRootCredentials(cred))) {
writeErrorResponseJSON(ctx, w, errorCodes.ToAPIErr(ErrAccessDenied), r.URL)
return
}
var sp *policy.Policy
if len(updateReq.NewPolicy) > 0 {
@@ -951,23 +957,21 @@ func (a adminAPIHandlers) InfoServiceAccount(w http.ResponseWriter, r *http.Requ
return
}
if !globalIAMSys.IsAllowed(policy.Args{
requestUser := cred.AccessKey
if cred.ParentUser != "" {
requestUser = cred.ParentUser
}
if !isAllowedServiceAccountAction(cred, policy.Args{
AccountName: cred.AccessKey,
Groups: cred.Groups,
Action: policy.ListServiceAccountsAdminAction,
ConditionValues: getConditionValues(r, "", cred),
IsOwner: owner,
Claims: cred.Claims,
DenyOnly: requestUser == svcAccount.ParentUser,
}) {
requestUser := cred.AccessKey
if cred.ParentUser != "" {
requestUser = cred.ParentUser
}
if requestUser != svcAccount.ParentUser {
writeErrorResponseJSON(ctx, w, errorCodes.ToAPIErr(ErrAccessDenied), r.URL)
return
}
writeErrorResponseJSON(ctx, w, errorCodes.ToAPIErr(ErrAccessDenied), r.URL)
return
}
// if session policy is nil or empty, then it is implied policy
@@ -1039,29 +1043,27 @@ func (a adminAPIHandlers) ListServiceAccounts(w http.ResponseWriter, r *http.Req
return
}
var targetAccount string
// If listing is requested for a specific user (who is not the request
// sender), check that the user has permissions.
selfUser := cred.AccessKey
if cred.ParentUser != "" {
selfUser = cred.ParentUser
}
user := r.Form.Get("user")
if user != "" && user != cred.AccessKey {
if !globalIAMSys.IsAllowed(policy.Args{
AccountName: cred.AccessKey,
Groups: cred.Groups,
Action: policy.ListServiceAccountsAdminAction,
ConditionValues: getConditionValues(r, "", cred),
IsOwner: owner,
Claims: cred.Claims,
}) {
writeErrorResponseJSON(ctx, w, errorCodes.ToAPIErr(ErrAccessDenied), r.URL)
return
}
targetAccount = user
} else {
targetAccount = cred.AccessKey
if cred.ParentUser != "" {
targetAccount = cred.ParentUser
}
selfOnly := user == "" || user == cred.AccessKey || user == selfUser
if !isAllowedServiceAccountAction(cred, policy.Args{
AccountName: cred.AccessKey,
Groups: cred.Groups,
Action: policy.ListServiceAccountsAdminAction,
ConditionValues: getConditionValues(r, "", cred),
IsOwner: owner,
Claims: cred.Claims,
DenyOnly: selfOnly,
}) {
writeErrorResponseJSON(ctx, w, errorCodes.ToAPIErr(ErrAccessDenied), r.URL)
return
}
targetAccount := user
if selfOnly {
targetAccount = selfUser
}
serviceAccounts, err := globalIAMSys.ListServiceAccounts(ctx, targetAccount)
@@ -1110,7 +1112,7 @@ func (a adminAPIHandlers) DeleteServiceAccount(w http.ResponseWriter, r *http.Re
// Get current object layer instance.
objectAPI := newObjectLayerFn()
if objectAPI == nil || globalNotificationSys == nil {
if objectAPI == nil || globalNotificationSys == nil || !globalIAMSys.Initialized() {
writeErrorResponseJSON(ctx, w, errorCodes.ToAPIErr(ErrServerNotInitialized), r.URL)
return
}
@@ -1131,35 +1133,41 @@ func (a adminAPIHandlers) DeleteServiceAccount(w http.ResponseWriter, r *http.Re
writeErrorResponseJSON(ctx, w, errorCodes.ToAPIErr(ErrInvalidArgument), r.URL)
return
}
// We do not care if service account is readable or not at this point,
// since this is a delete call we shall allow it to be deleted if possible.
svcAccount, _, err := globalIAMSys.GetServiceAccount(ctx, serviceAccount)
if errors.Is(err, errNoSuchServiceAccount) {
// Deletion needs the stored identity, not a valid token or session policy.
// Keep disabled, expired and damaged service accounts removable, and retain
// their parent for authorization and the site replication hook below.
u, ok := globalIAMSys.store.GetUser(serviceAccount)
if !ok || !u.Credentials.IsServiceAccount() {
writeErrorResponseJSON(ctx, w, errorCodes.ToAPIErr(ErrAdminServiceAccountNotFound), r.URL)
return
}
svcAccount := u.Credentials
adminPrivilege := globalIAMSys.IsAllowed(policy.Args{
parentUser := cred.AccessKey
if cred.ParentUser != "" {
parentUser = cred.ParentUser
}
isSelf := parentUser == svcAccount.ParentUser
if !isAllowedServiceAccountAction(cred, policy.Args{
AccountName: cred.AccessKey,
Groups: cred.Groups,
Action: policy.RemoveServiceAccountAdminAction,
ConditionValues: getConditionValues(r, "", cred),
IsOwner: owner,
Claims: cred.Claims,
})
if !adminPrivilege {
parentUser := cred.AccessKey
if cred.ParentUser != "" {
parentUser = cred.ParentUser
}
if svcAccount.ParentUser != "" && parentUser != svcAccount.ParentUser {
// The service account belongs to another user but return not
// found error to mitigate brute force attacks. or the
// serviceAccount doesn't exist.
DenyOnly: isSelf,
}) {
if !isSelf {
// Hide the existence of another user's service account.
writeErrorResponseJSON(ctx, w, errorCodes.ToAPIErr(ErrAdminServiceAccountNotFound), r.URL)
return
}
writeErrorResponseJSON(ctx, w, errorCodes.ToAPIErr(ErrAccessDenied), r.URL)
return
}
if newGlobalAuthZPluginFn() == nil && svcAccount.ParentUser == globalActiveCred.AccessKey && !canManageRootCredentials(cred) {
writeErrorResponseJSON(ctx, w, errorCodes.ToAPIErr(ErrAdminServiceAccountNotFound), r.URL)
return
}
if err := globalIAMSys.DeleteServiceAccount(ctx, serviceAccount, true); err != nil {
@@ -2046,26 +2054,30 @@ func (a adminAPIHandlers) RevokeTokens(w http.ResponseWriter, r *http.Request) {
return
}
adminPrivilege := globalIAMSys.IsAllowed(policy.Args{
parentUser := cred.AccessKey
if cred.ParentUser != "" {
parentUser = cred.ParentUser
}
isSelfTarget := isTokenSelfRevoke || user == parentUser
if !isAllowedServiceAccountAction(cred, policy.Args{
AccountName: cred.AccessKey,
Groups: cred.Groups,
Action: policy.RemoveServiceAccountAdminAction,
ConditionValues: getConditionValues(r, "", cred),
IsOwner: owner,
Claims: cred.Claims,
})
if !adminPrivilege || isTokenSelfRevoke {
parentUser := cred.AccessKey
if cred.ParentUser != "" {
parentUser = cred.ParentUser
}
if !isTokenSelfRevoke && user != parentUser {
writeErrorResponseJSON(ctx, w, errorCodes.ToAPIErr(ErrAccessDenied), r.URL)
return
}
DenyOnly: isSelfTarget,
}) {
writeErrorResponseJSON(ctx, w, errorCodes.ToAPIErr(ErrAccessDenied), r.URL)
return
}
if isSelfTarget {
user = parentUser
}
if newGlobalAuthZPluginFn() == nil && user == globalActiveCred.AccessKey && !canManageRootCredentials(cred) {
writeErrorResponseJSON(ctx, w, errorCodes.ToAPIErr(ErrAccessDenied), r.URL)
return
}
// Infer token revoke type from the request if requestor is STS.
if isTokenSelfRevoke && tokenRevokeType == "" && !fullRevoke {
@@ -2112,29 +2124,22 @@ func (a adminAPIHandlers) InfoAccessKey(w http.ResponseWriter, r *http.Request)
u, ok := globalIAMSys.GetUser(ctx, accessKey)
targetCred := u.Credentials
if !globalIAMSys.IsAllowed(policy.Args{
requestUser := cred.AccessKey
if cred.ParentUser != "" {
requestUser = cred.ParentUser
}
isSelf := ok && targetCred.ParentUser != "" && requestUser == targetCred.ParentUser
if !isAllowedServiceAccountAction(cred, policy.Args{
AccountName: cred.AccessKey,
Groups: cred.Groups,
Action: policy.ListServiceAccountsAdminAction,
ConditionValues: getConditionValues(r, "", cred),
IsOwner: owner,
Claims: cred.Claims,
DenyOnly: isSelf,
}) {
// If requested user does not exist and requestor is not allowed to list service accounts, return access denied.
if !ok {
writeErrorResponseJSON(ctx, w, errorCodes.ToAPIErr(ErrAccessDenied), r.URL)
return
}
requestUser := cred.AccessKey
if cred.ParentUser != "" {
requestUser = cred.ParentUser
}
if requestUser != targetCred.ParentUser {
writeErrorResponseJSON(ctx, w, errorCodes.ToAPIErr(ErrAccessDenied), r.URL)
return
}
writeErrorResponseJSON(ctx, w, errorCodes.ToAPIErr(ErrAccessDenied), r.URL)
return
}
if !ok {
@@ -2908,6 +2913,44 @@ func addExpirationToCondValues(exp *time.Time, condValues map[string][]string) e
return nil
}
// hasActiveSessionPolicy follows the same credential dispatch as IAMSys.IsAllowed.
// Expiring service accounts also satisfy IsTemp and are evaluated as STS there.
// Callers use this only after authorization under built-in IAM.
func hasActiveSessionPolicy(cred auth.Credentials) bool {
args := policy.Args{Claims: cred.Claims}
if cred.IsTemp() {
hasSP, _ := isAllowedBySessionPolicy(args)
return hasSP
}
if cred.IsServiceAccount() {
if cred.Claims[iamPolicyClaimNameSA()] == inheritedPolicyType {
return false
}
hasSP, _ := isAllowedBySessionPolicyForServiceAccount(args)
return hasSP
}
return false
}
// canManageRootCredentials is an additional guard after built-in IAM authorization.
// The signature's owner flag is insufficient: historical empty session policies
// clear it even though the service account still inherits root's permissions.
func canManageRootCredentials(cred auth.Credentials) bool {
return cred.AccessKey == globalActiveCred.AccessKey ||
(cred.ParentUser == globalActiveCred.AccessKey && cred.AccessKey != siteReplicatorSvcAcc && !hasActiveSessionPolicy(cred))
}
// isAllowedServiceAccountAction preserves regular users' self inspection and
// cleanup even without a policy attachment, while enforcing explicit denies.
// Derived credentials and external authorization plugins use full IAM evaluation.
func isAllowedServiceAccountAction(cred auth.Credentials, args policy.Args) bool {
if args.DenyOnly && !args.IsOwner && !cred.IsTemp() && !cred.IsServiceAccount() && newGlobalAuthZPluginFn() == nil {
policies, err := globalIAMSys.PolicyDBGet(args.AccountName, args.Groups...)
return err == nil && globalIAMSys.GetCombinedPolicy(policies...).IsAllowed(args)
}
return globalIAMSys.IsAllowed(args)
}
func commonAddServiceAccount(r *http.Request, ldap bool) (context.Context, auth.Credentials, newServiceAccountOpts, madmin.AddServiceAccountReq, string, APIError) {
ctx := r.Context()
@@ -2996,6 +3039,14 @@ func commonAddServiceAccount(r *http.Request, ldap bool) (context.Context, auth.
}) {
return ctx, auth.Credentials{}, newServiceAccountOpts{}, madmin.AddServiceAccountReq{}, "", errorCodes.ToAPIErr(ErrAccessDenied)
}
effectiveTarget := targetUser
if (targetUser == cred.AccessKey || targetUser == cred.ParentUser) && cred.ParentUser != "" {
effectiveTarget = cred.ParentUser
}
if newGlobalAuthZPluginFn() == nil && (hasActiveSessionPolicy(cred) ||
(effectiveTarget == globalActiveCred.AccessKey && !canManageRootCredentials(cred))) {
return ctx, auth.Credentials{}, newServiceAccountOpts{}, madmin.AddServiceAccountReq{}, "", errorCodes.ToAPIErr(ErrAccessDenied)
}
var sp *policy.Policy
if len(createReq.Policy) > 0 {
@@ -3003,6 +3054,12 @@ func commonAddServiceAccount(r *http.Request, ldap bool) (context.Context, auth.
if err != nil {
return ctx, auth.Credentials{}, newServiceAccountOpts{}, madmin.AddServiceAccountReq{}, "", toAdminAPIErr(ctx, err)
}
// Empty service policies mean inheritance, including for expiring keys.
// Normalize the replication request as well as the local credential.
if sp.Version == "" && sp.Statements == nil && sp.ID == "" {
sp = nil
createReq.Policy = nil
}
}
opts.sessionPolicy = sp
+45
View File
@@ -0,0 +1,45 @@
// Copyright (c) 2026 PGSTY
//
// This file is part of SILO Object Storage.
// SPDX-License-Identifier: AGPL-3.0-or-later
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package cmd
import (
"fmt"
"net/http"
"testing"
"github.com/minio/minio/internal/hash"
)
func TestChecksumMismatchAPIMessage(t *testing.T) {
for _, algorithm := range []string{"CRC32", "CRC32C", "SHA1", "SHA256", "CRC64NVME", ""} {
t.Run(algorithm, func(t *testing.T) {
mismatch := hash.ChecksumMismatch{Algorithm: algorithm, Want: "expected", Got: "actual"}
want := errorCodes.ToAPIErr(ErrContentChecksumMismatch).Description
if algorithm != "" {
want = fmt.Sprintf("The %s you specified did not match the calculated checksum.", algorithm)
}
for _, err := range []error{mismatch, fmt.Errorf("upload: %w", mismatch)} {
got := toAPIError(t.Context(), err)
if got.HTTPStatusCode != http.StatusBadRequest || got.Code != "XAmzContentChecksumMismatch" || got.Description != want {
t.Fatalf("toAPIError(%v) = %+v, want 400 XAmzContentChecksumMismatch: %s", err, got, want)
}
}
})
}
}
+8
View File
@@ -2505,6 +2505,14 @@ func toAPIError(ctx context.Context, err error) APIError {
case errors.Is(err, errMissingPartChecksum):
apiErr.Description = strings.TrimPrefix(err.Error(), errMissingPartChecksum.Error()+": ")
}
// PutObject/UploadPart checksum mismatches name the algorithm in the
// message, matching S3 (e.g. "The CRC32 you specified did not match the
// calculated checksum."). Falls through to the generic table description
// when the algorithm is unknown.
var checksumMismatch hash.ChecksumMismatch
if errors.As(err, &checksumMismatch) && checksumMismatch.Algorithm != "" {
apiErr.Description = fmt.Sprintf("The %s you specified did not match the calculated checksum.", checksumMismatch.Algorithm)
}
switch apiErr.Code {
case "NotImplemented":
apiErr = APIError{
+91
View File
@@ -20,6 +20,7 @@ package cmd
import (
"context"
"encoding/hex"
"errors"
"fmt"
"math/rand"
"reflect"
@@ -110,6 +111,96 @@ func TestReduceErrs(t *testing.T) {
}
}
// TestReduceWriteQuorumErrsCommitMetaMissing verifies that an in-quorum
// errCommitMetaMissing reduces to the sentinel, which renameData() converts
// to errErasureWriteQuorum, while below-quorum reduces directly to
// errErasureWriteQuorum.
func TestReduceWriteQuorumErrsCommitMetaMissing(t *testing.T) {
errs := make([]error, 16)
for i := range 9 {
errs[i] = errCommitMetaMissing
}
if err := reduceWriteQuorumErrs(t.Context(), errs, objectOpIgnoredErrs, 9); !errors.Is(err, errCommitMetaMissing) {
t.Errorf("expected %s, got %s", errCommitMetaMissing, err)
}
// The sentinel must be distinguishable from errFileNotFound so that
// commit failures are never mapped to ObjectNotFound (404).
if errors.Is(errCommitMetaMissing, errFileNotFound) {
t.Error("errCommitMetaMissing must not match errFileNotFound")
}
// Below quorum the reduction must yield a write-quorum failure (503);
// eight sentinels against eight committed drives is not a quorum.
belowQuorum := make([]error, 16)
for i := range 8 {
belowQuorum[i] = errCommitMetaMissing
}
if err := reduceWriteQuorumErrs(t.Context(), belowQuorum, objectOpIgnoredErrs, 9); !errors.Is(err, errErasureWriteQuorum) {
t.Errorf("expected %s, got %s", errErasureWriteQuorum, err)
}
}
// renameDataFailingDisk is a StorageAPI stub returning a fixed error
// from RenameData, enough to drive renameData() in unit tests.
type renameDataFailingDisk struct {
StorageAPI
err error
undoWrite bool
}
func (d *renameDataFailingDisk) RenameData(context.Context, string, string, FileInfo, string, string, RenameOptions) (RenameDataResp, error) {
return RenameDataResp{}, d.err
}
func (d *renameDataFailingDisk) DeleteVersion(_ context.Context, _, _ string, _ FileInfo, _ bool, opts DeleteOptions) error {
d.undoWrite = opts.UndoWrite
return nil
}
// TestRenameDataCommitMetaMissing verifies that renameData() converts an
// in-quorum errCommitMetaMissing into errErasureWriteQuorum (HTTP 503),
// while an in-quorum errFileNotFound keeps its meaning so that
// PutObject() still reports IncompleteBody for client disconnects.
func TestRenameDataCommitMetaMissing(t *testing.T) {
for _, tc := range []struct {
name string
failures int
diskErr error
wantErr error
}{
{"all_commit_metadata_missing", 16, errCommitMetaMissing, errErasureWriteQuorum},
{"quorum_commit_metadata_missing", 9, errCommitMetaMissing, errErasureWriteQuorum},
{"neither_side_has_quorum", 8, errCommitMetaMissing, errErasureWriteQuorum},
{"successful_write_quorum", 7, errCommitMetaMissing, nil},
{"staging_failure", 16, errFileNotFound, errFileNotFound},
} {
t.Run(tc.name, func(t *testing.T) {
metadata := make([]FileInfo, 16)
disks := make([]StorageAPI, 16)
for i := range disks {
metadata[i] = FileInfo{Deleted: true} // valid without erasure info
disk := &renameDataFailingDisk{}
if i < tc.failures {
disk.err = tc.diskErr
}
disks[i] = disk
}
_, _, _, err := renameData(t.Context(), disks, minioMetaTmpBucket, "tmp", metadata, "bucket", "object", 9)
if !errors.Is(err, tc.wantErr) {
t.Fatalf("expected %v, got %v", tc.wantErr, err)
}
for i, disk := range disks {
// Failed writes undo only the drives that committed. A
// successful quorum must never be rolled back.
wantUndo := tc.wantErr != nil && i >= tc.failures
if got := disk.(*renameDataFailingDisk).undoWrite; got != wantUndo {
t.Errorf("disk %d: UndoWrite = %v, want %v", i, got, wantUndo)
}
}
})
}
}
// TestHashOrder - test order of ints in array
func TestHashOrder(t *testing.T) {
testCases := []struct {
+177
View File
@@ -0,0 +1,177 @@
// Copyright (c) 2026 clown15
//
// This file is part of Silo Object Storage stack
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package cmd
import (
"context"
"encoding/base64"
"fmt"
"net/url"
"os"
"path/filepath"
"testing"
"time"
)
func TestExtractModTimeFromUploadIDDir(t *testing.T) {
now := time.Unix(1792056747, 123456789).UTC()
nano := now.UnixNano()
uuid := "d16ad42a-6fe5-4a3b-8092-69983142238f"
testCases := []struct {
name string
dirName string
expectedMod time.Time
wantZero bool
}{
{
name: "Modern Plaintext Format (<UUID>x<UnixNano>)",
dirName: fmt.Sprintf("%sx%d", uuid, nano),
expectedMod: time.Unix(0, nano),
wantZero: false,
},
{
// This valid UUID decodes to bytes containing 'x'. Checking
// for that byte before parsing plaintext loses the timestamp.
name: "Plaintext Also Decodes To Base64 Containing x",
dirName: fmt.Sprintf("4eacd477-e733-4851-b821-6fd42785ff87x%d", nano),
expectedMod: now,
},
{
name: "Legacy Format Without Timestamp (<UUID>)",
dirName: uuid,
expectedMod: time.Time{},
wantZero: true,
},
{
name: "Base64 Encoded Format",
dirName: base64.RawURLEncoding.EncodeToString([]byte(fmt.Sprintf("%sx%d", uuid, nano))),
expectedMod: time.Unix(0, nano),
wantZero: false,
},
{
name: "Encoded Format With Deployment Prefix",
dirName: base64.RawURLEncoding.EncodeToString([]byte(fmt.Sprintf("%s.%sx%d", uuid, uuid, nano))),
expectedMod: now,
},
{
name: "Malformed Suffix (<UUID>xinvalid)",
dirName: fmt.Sprintf("%sxnotanumber", uuid),
expectedMod: time.Time{},
wantZero: true,
},
{
name: "Timestamp Overflow",
dirName: uuid + "x9223372036854775808",
wantZero: true,
},
{
name: "Malformed Plaintext UUID",
dirName: fmt.Sprintf("not-a-uuidx%d", nano),
wantZero: true,
},
{
name: "Empty Directory Name",
dirName: "",
expectedMod: time.Time{},
wantZero: true,
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
got := extractModTimeFromUploadIDDir(tc.dirName)
if tc.wantZero {
if !got.IsZero() {
t.Fatalf("expected zero time for %q, got %v", tc.dirName, got)
}
return
}
if !got.Equal(tc.expectedMod) {
t.Fatalf("extractModTimeFromUploadIDDir(%q) = %v; want %v", tc.dirName, got, tc.expectedMod)
}
})
}
}
type multipartCleanupDisk struct {
StorageAPI
root string
modTime time.Time
readErr error
reads int
}
func (d *multipartCleanupDisk) Endpoint() Endpoint {
return Endpoint{URL: &url.URL{Path: d.root}}
}
func (d *multipartCleanupDisk) ReadVersion(context.Context, string, string, string, string, ReadOptions) (FileInfo, error) {
d.reads++
return FileInfo{ModTime: d.modTime}, d.readErr
}
func (d *multipartCleanupDisk) StatVol(context.Context, string) (VolInfo, error) {
return VolInfo{Created: time.Now()}, nil
}
func TestCleanupStaleUploadsOnDisk(t *testing.T) {
const uuid = "d16ad42a-6fe5-4a3b-8092-69983142238f"
expiry := globalAPIConfig.getStaleUploadsExpiry()
for _, tc := range []struct {
name, prefix string
age time.Duration
legacy bool
readErr error
wantReads int
wantDeleted bool
}{
{name: "recent", prefix: uuid, age: expiry / 2, readErr: errFileNotFound},
{name: "plaintext_base64_collision", prefix: "4eacd477-e733-4851-b821-6fd42785ff87", age: expiry / 2, readErr: errFileNotFound},
{name: "expired", prefix: uuid, age: 2 * expiry, readErr: errFileNotFound, wantDeleted: true},
{name: "legacy_recent", prefix: uuid, age: expiry / 2, legacy: true, wantReads: 1},
{name: "legacy_expired", prefix: uuid, age: 2 * expiry, legacy: true, wantReads: 1, wantDeleted: true},
{name: "legacy_metadata_missing", prefix: uuid, age: 2 * expiry, legacy: true, readErr: errFileNotFound, wantReads: 1},
{name: "malformed_metadata_missing", prefix: "not-a-uuid", age: 2 * expiry, readErr: errFileNotFound, wantReads: 1},
} {
t.Run(tc.name, func(t *testing.T) {
modTime := time.Now().Add(-tc.age)
id := tc.prefix
if !tc.legacy {
id = fmt.Sprintf("%sx%d", tc.prefix, modTime.UnixNano())
}
disk := &multipartCleanupDisk{root: t.TempDir(), modTime: modTime, readErr: tc.readErr}
uploadDir := filepath.Join(disk.root, minioMetaMultipartBucket, "sha", id)
if err := os.MkdirAll(uploadDir, 0o755); err != nil {
t.Fatal(err)
}
erasureObjects{}.cleanupStaleUploadsOnDisk(t.Context(), disk)
if disk.reads != tc.wantReads {
t.Errorf("ReadVersion calls = %d, want %d", disk.reads, tc.wantReads)
}
_, err := os.Stat(uploadDir)
if tc.wantDeleted {
if !os.IsNotExist(err) {
t.Errorf("expired upload must be removed: %v", err)
}
} else if err != nil {
t.Errorf("upload must be preserved: %v", err)
}
})
}
}
+19 -10
View File
@@ -174,6 +174,24 @@ func (er erasureObjects) deleteAll(ctx context.Context, bucket, prefix string) {
wg.Wait()
}
// extractModTimeFromUploadIDDir extracts creation time from the upload ID directory on disk.
// On-disk directory names are of the form <UUID>x<UnixNano>.
// Parse the native format first: plaintext UUIDs can also be valid Base64.
// Preserve the previous encoded-format parser for compatibility.
func extractModTimeFromUploadIDDir(uploadIDDir string) time.Time {
if modTime, ok := multipartUploadTime(uploadIDDir); ok {
return modTime
}
if b64, err := base64.RawURLEncoding.DecodeString(uploadIDDir); err == nil {
if split := strings.Split(string(b64), "x"); len(split) == 2 {
if t, err := strconv.ParseInt(split[1], 10, 64); err == nil {
return time.Unix(0, t)
}
}
}
return time.Time{}
}
// Remove the old multipart uploads on the given disk.
func (er erasureObjects) cleanupStaleUploadsOnDisk(ctx context.Context, disk StorageAPI) {
drivePath := disk.Endpoint().Path
@@ -181,16 +199,7 @@ func (er erasureObjects) cleanupStaleUploadsOnDisk(ctx context.Context, disk Sto
readDirFn(pathJoin(drivePath, minioMetaMultipartBucket), func(shaDir string, typ os.FileMode) error {
readDirFn(pathJoin(drivePath, minioMetaMultipartBucket, shaDir), func(uploadIDDir string, typ os.FileMode) error {
uploadIDPath := pathJoin(shaDir, uploadIDDir)
var modTime time.Time
// Upload IDs are of the form base64_url(<UUID>x<UnixNano>), we can extract the time from the UUID.
if b64, err := base64.RawURLEncoding.DecodeString(uploadIDDir); err == nil {
if split := strings.Split(string(b64), "x"); len(split) == 2 {
t, err := strconv.ParseInt(split[1], 10, 64)
if err == nil {
modTime = time.Unix(0, t)
}
}
}
modTime := extractModTimeFromUploadIDDir(uploadIDDir)
// Fallback for older uploads without time in the ID.
if modTime.IsZero() {
wait := deleteMultipartCleanupSleeper.Timer(ctx)
+6
View File
@@ -1083,6 +1083,12 @@ func renameData(ctx context.Context, disks []StorageAPI, srcBucket, srcEntry str
errs := g.Wait()
err := reduceWriteQuorumErrs(ctx, errs, objectOpIgnoredErrs, writeQuorum)
if errors.Is(err, errCommitMetaMissing) {
// Commit metadata went missing on a quorum of drives; the write
// did not commit. Return a 503 class error so clients retry
// instead of treating the object as absent (404).
err = errErasureWriteQuorum
}
if err != nil {
dg := errgroup.WithNErrs(len(disks))
for index, nerr := range errs {
+2
View File
@@ -24,6 +24,8 @@ func TestIAMPeerDeleteNotificationReloadsCommittedState(t *testing.T) {
resetTestGlobals()
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
// Both caches use object storage, independent of earlier etcd suites.
globalEtcdClient = nil
obj, disk, err := prepareFS(ctx)
if err != nil {
t.Fatal(err)
+1 -1
View File
@@ -162,7 +162,7 @@ func toObjectErr(oerr error, params ...string) error {
apiErr.Type = v.Type
}
return apiErr
case errErasureWriteQuorum.Error():
case errErasureWriteQuorum.Error(), errCommitMetaMissing.Error():
apiErr := InsufficientWriteQuorum{}
if len(params) >= 1 {
apiErr.Bucket = params[0]
+9 -4
View File
@@ -62,12 +62,13 @@ func testAPIPutObjectChunkedChecksum(obj ObjectLayer, instanceType, bucketName s
// A well-formed 4-byte value that does not match the content.
wrongCRC := base64.StdEncoding.EncodeToString([]byte{0x01, 0x02, 0x03, 0x04})
apiCode := func(rec *httptest.ResponseRecorder) string {
apiErrOf := func(rec *httptest.ResponseRecorder) APIErrorResponse {
var apiErr APIErrorResponse
b, _ := io.ReadAll(rec.Body)
_ = xml.Unmarshal(b, &apiErr)
return apiErr.Code
return apiErr
}
apiCode := func(rec *httptest.ResponseRecorder) string { return apiErrOf(rec).Code }
// newChunkedJavaForm builds a non-trailer signed chunked PutObject request that
// mirrors the AWS Java SDK v2 wire form: the checksum value sits in the header
@@ -131,8 +132,12 @@ func testAPIPutObjectChunkedChecksum(obj ObjectLayer, instanceType, bucketName s
if rec.Code != http.StatusBadRequest {
t.Fatalf("%s: chunked+wrong CRC32: expected 400, got %d", instanceType, rec.Code)
}
if code := apiCode(rec); code != "XAmzContentChecksumMismatch" {
t.Fatalf("%s: chunked+wrong CRC32: want XAmzContentChecksumMismatch, got %q", instanceType, code)
gotErr := apiErrOf(rec)
if gotErr.Code != "XAmzContentChecksumMismatch" {
t.Fatalf("%s: chunked+wrong CRC32: want XAmzContentChecksumMismatch, got %q", instanceType, gotErr.Code)
}
if gotErr.Message != "The CRC32 you specified did not match the calculated checksum." {
t.Fatalf("%s: chunked+wrong CRC32: message = %q, want S3 algorithm-specific wording", instanceType, gotErr.Message)
}
}
}
+5
View File
@@ -70,6 +70,11 @@ var errDiskAccessDenied = StorageErr("drive access denied")
// errFileNotFound - cannot find the file.
var errFileNotFound = StorageErr("file not found")
// errCommitMetaMissing - the staged xl.meta disappeared between the
// data rename and the meta-file commit; a write-quorum failure, not
// a missing object.
var errCommitMetaMissing = StorageErr("commit metadata is missing")
// errFileNotFound - cannot find requested file version.
var errFileVersionNotFound = StorageErr("file version not found")
+2
View File
@@ -106,6 +106,8 @@ func toStorageErr(err error) error {
return errVolumeExists
case errFileNotFound.Error():
return errFileNotFound
case errCommitMetaMissing.Error():
return errCommitMetaMissing
case errFileVersionNotFound.Error():
return errFileVersionNotFound
case errFileNameTooLong.Error():
+20 -2
View File
@@ -44,6 +44,7 @@ import (
"github.com/minio/minio-go/v7/pkg/tags"
xhttp "github.com/minio/minio/internal/http"
"github.com/pgsty/silo-pkg/v3/ldap"
"github.com/pgsty/silo-pkg/v3/policy"
)
func runAllIAMSTSTests(suite *TestSuiteIAM, c *check) {
@@ -2384,8 +2385,10 @@ func (s *TestSuiteIAM) TestIAMImport(c *check, exportedContent []byte, caseNum i
if err != nil {
c.Fatalf("import %d: Unable to list policies: %v", caseNum, err)
}
defaultCannedPolicies := set.CreateStringSet("consoleAdmin", "readwrite", "readonly",
"diagnostics", "writeonly")
defaultCannedPolicies := set.CreateStringSet()
for _, p := range policy.DefaultPolicies {
defaultCannedPolicies.Add(p.Name)
}
for policy, policyBytes := range policyContentMap {
if defaultCannedPolicies.Contains(policy) {
continue
@@ -2991,6 +2994,21 @@ func (s *TestSuiteIAM) TestLDAPSTSServiceAccounts(c *check) {
// 6. Check that service account cannot be created for some other user.
c.mustNotCreateSvcAccount(ctx, globalActiveCred.AccessKey, userAdmClient)
// An empty policy must not contaminate an expiring LDAP child credential.
emptyPolicyCred, err := userAdmClient.AddServiceAccountLDAP(ctx, madmin.AddServiceAccountReq{Policy: []byte("null")})
if err != nil {
c.Fatalf("Unable to create empty-policy LDAP service account: %v", err)
}
emptyPolicyAdmin := s.getAdminClient(c, emptyPolicyCred.AccessKey, emptyPolicyCred.SecretKey, "")
expires := UTCNow().Add(time.Hour)
for _, childPolicy := range [][]byte{nil, []byte("null")} {
child, err := emptyPolicyAdmin.AddServiceAccountLDAP(ctx, madmin.AddServiceAccountReq{Expiration: &expires, Policy: childPolicy})
if err != nil {
c.Fatalf("Unable to create expiring LDAP child: %v", err)
}
c.assertSvcAccS3Access(ctx, s, child, bucket)
}
// Detach the policy from the user
if _, err = s.adm.DetachPolicyLDAP(ctx, userReq); err != nil {
c.Fatalf("Unable to detach user policy: %v", err)
+123
View File
@@ -18,10 +18,16 @@
package cmd
import (
"context"
"errors"
"net/http"
"os"
"path/filepath"
"runtime"
"sync"
"syscall"
"testing"
"time"
)
func TestSysErrors(t *testing.T) {
@@ -56,3 +62,120 @@ func TestSysErrors(t *testing.T) {
}
}
}
// TestCommitMetaMissingMapping verifies the RPC and object-layer mapping
// of errCommitMetaMissing, returned when the staged xl.meta disappears
// between the data rename and the meta-file commit in RenameData().
func TestCommitMetaMissingMapping(t *testing.T) {
// The sentinel must survive a storage REST/grid round-trip; errors
// cross the wire as strings and are remapped by toStorageErr().
if got := toStorageErr(errors.New(errCommitMetaMissing.Error())); !errors.Is(got, errCommitMetaMissing) {
t.Fatalf("expected %s, got %s", errCommitMetaMissing, got)
}
if got := toStorageErr(errCommitMetaMissing); got != errCommitMetaMissing {
t.Fatalf("expected %s, got %s", errCommitMetaMissing, got)
}
// toObjectErr() must map the sentinel to InsufficientWriteQuorum
// (503), never to ObjectNotFound (404).
err := toObjectErr(errCommitMetaMissing, "bucket", "object")
qerr, ok := err.(InsufficientWriteQuorum)
if !ok {
t.Fatalf("expected InsufficientWriteQuorum, got %s (%T)", err, err)
}
if qerr.Bucket != "bucket" || qerr.Object != "object" {
t.Fatalf("unexpected bucket/object %s/%s", qerr.Bucket, qerr.Object)
}
apiErr := toAPIError(t.Context(), err)
if apiErr.Code != "SlowDownWrite" || apiErr.HTTPStatusCode != http.StatusServiceUnavailable {
t.Fatalf("expected SlowDownWrite/503, got %+v", apiErr)
}
}
// missingCommitMetaContext removes staged metadata at an existing cancellation
// check, after WriteAll and (for non-inline objects) the data rename succeeded.
// This deterministically models concurrent cleanup without changing production
// code or relying on a filesystem race. The context itself remains live.
type missingCommitMetaContext struct {
context.Context
mu sync.Mutex
meta, dataDest string
removed bool
removeErr error
}
func (c *missingCommitMetaContext) Done() <-chan struct{} {
c.mu.Lock()
defer c.mu.Unlock()
if !c.removed {
if _, err := os.Stat(c.meta); err == nil {
dataMoved := c.dataDest == ""
if !dataMoved {
_, err = os.Stat(c.dataDest)
dataMoved = err == nil
}
if dataMoved {
c.removeErr = os.Remove(c.meta)
c.removed = c.removeErr == nil
}
}
}
return c.Context.Done()
}
func TestXLStorageRenameDataCommitMetaMissing(t *testing.T) {
for _, tc := range []struct {
name string
dataFile bool
}{
{"empty_object", false},
{"data_directory", true},
} {
t.Run(tc.name, func(t *testing.T) {
disk, root, err := newXLStorageTestSetup(t)
if err != nil {
t.Fatal(err)
}
defer disk.Close()
s := disk.storage
if err := s.MakeVol(t.Context(), "bucket"); err != nil {
t.Fatal(err)
}
fi := newFileInfo("object", 2, 2)
fi.DataDir, fi.VersionID = mustGetUUID(), mustGetUUID()
fi.Erasure.Index = 1
fi.ModTime = time.Now().UTC()
fi.Fresh = true
ctx := &missingCommitMetaContext{
Context: t.Context(),
meta: filepath.Join(root, minioMetaTmpBucket, "staged", xlStorageFormatFile),
}
if tc.dataFile {
fi.Size = 1
fi.AddObjectPart(1, "", 1, 1, fi.ModTime, nil, nil)
part := filepath.Join(root, minioMetaTmpBucket, "staged", fi.DataDir, "part.1")
if err := os.MkdirAll(filepath.Dir(part), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(part, []byte("a"), 0o644); err != nil {
t.Fatal(err)
}
ctx.dataDest = filepath.Join(root, "bucket", "object", fi.DataDir)
}
_, err = s.RenameData(ctx, minioMetaTmpBucket, "staged", fi, "bucket", "object", RenameOptions{})
if !ctx.removed {
t.Fatalf("metadata removal did not execute: removal error = %v, RenameData error = %v", ctx.removeErr, err)
}
if !errors.Is(err, errCommitMetaMissing) {
t.Fatalf("expected commit metadata missing, got %v", err)
}
apiErr := toAPIError(t.Context(), toObjectErr(err, "bucket", "object"))
if apiErr.Code != "SlowDownWrite" || apiErr.HTTPStatusCode != http.StatusServiceUnavailable {
t.Fatalf("expected SlowDownWrite/503, got %+v", apiErr)
}
if _, err := os.Stat(filepath.Join(root, "bucket", "object", xlStorageFormatFile)); !os.IsNotExist(err) {
t.Fatalf("failed commit must not produce destination metadata: %v", err)
}
})
}
}
+9 -1
View File
@@ -2925,7 +2925,15 @@ func (s *xlStorage) RenameData(ctx context.Context, srcVolume, srcPath string, f
// if its a partial rename() do not attempt to delete recursively.
// this can be healed since all parts are available.
s.deleteFile(dstVolumeDir, dstDataPath, false, false)
return res, osErrToFileErr(err)
err = osErrToFileErr(err)
if errors.Is(err, errFileNotFound) {
// The staged xl.meta disappeared between staging and commit.
// This is a write-quorum failure, not a missing object; use a
// dedicated error so the object layer returns 503 instead of
// mapping it to ObjectNotFound (404).
err = errCommitMetaMissing
}
return res, err
}
if srcVolume != minioMetaMultipartBucket {
+1 -1
View File
@@ -4,7 +4,7 @@ go 1.27.2
// Console and MC retain their historical module paths for best-effort upstream
// compatibility. Pin the maintained PGSTY implementations used by SILO.
replace github.com/minio/console => github.com/pgsty/silo-console v0.0.0-20260916075814-1360e26d976d
replace github.com/minio/console => github.com/pgsty/silo-console v0.0.0-20261015132217-e5b165e4c0ee
replace github.com/minio/mc => github.com/pgsty/mc v0.0.0-20260916070421-e952aa78f10a
+2 -2
View File
@@ -547,8 +547,8 @@ github.com/pborman/getopt v0.0.0-20170112200414-7148bc3a4c30/go.mod h1:85jBQOZwp
github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic=
github.com/pgsty/mc v0.0.0-20260916070421-e952aa78f10a h1:FY1vcTQT67HGBe23ozlhOiQnWCzUCGmeSN5x6DT/kFs=
github.com/pgsty/mc v0.0.0-20260916070421-e952aa78f10a/go.mod h1:6z9lX8kOvaWz2vLlP7BjjF3JWHnIH86XDA2HwcmpZH8=
github.com/pgsty/silo-console v0.0.0-20260916075814-1360e26d976d h1:A9ou1GkUckGZd3BOgEXeZ393nbYST967SKcI4Lgr/Vs=
github.com/pgsty/silo-console v0.0.0-20260916075814-1360e26d976d/go.mod h1:JS165CF2yoTBIx1X8k1IQ6vunJcZFG3pRZlWHJ8UT8o=
github.com/pgsty/silo-console v0.0.0-20261015132217-e5b165e4c0ee h1:e51AnrGe0Zq+nHTYaGfv/GkehQMO9Mekuw/ew6zYwOM=
github.com/pgsty/silo-console v0.0.0-20261015132217-e5b165e4c0ee/go.mod h1:jpt39bx4wjhXoFB+BTOa/bx/pmU3pXzp2cT7PnT21Lo=
github.com/pgsty/silo-pkg/v3 v3.14.1 h1:8MONT3Hky9EOnsuPk29590pEBE6MILaOnJtaejPPi7M=
github.com/pgsty/silo-pkg/v3 v3.14.1/go.mod h1:jxKWxi52jSsDaDhXXPqjOyPDYdxx2iqdBXyLfM/xSIw=
github.com/philhofer/fwd v1.2.0 h1:e6DnBTl7vGY+Gz322/ASL4Gyp1FspeMvx1RNDoToZuM=
+73
View File
@@ -0,0 +1,73 @@
// Copyright (c) 2026 PGSTY
//
// This file is part of SILO Object Storage.
// SPDX-License-Identifier: AGPL-3.0-or-later
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package hash
import (
"bytes"
"errors"
"io"
"net/http"
"net/http/httptest"
"testing"
)
func TestChecksumMismatchAlgorithm(t *testing.T) {
content := []byte("actual content")
for _, typ := range []ChecksumType{ChecksumCRC32, ChecksumCRC32C, ChecksumSHA1, ChecksumSHA256, ChecksumCRC64NVME} {
t.Run(typ.String(), func(t *testing.T) {
wrong := NewChecksumFromData(typ, []byte("wrong content"))
check := func(t *testing.T, err error) {
t.Helper()
var mismatch ChecksumMismatch
if !errors.As(err, &mismatch) || mismatch.Algorithm != typ.String() {
t.Fatalf("error=%+v, want %s checksum mismatch", err, typ)
}
}
check(t, wrong.Matches(content, 0))
parts := NewChecksumFromData(typ, content)
parts.WantParts = 2
check(t, parts.Matches(content, 1))
for _, mode := range []string{"header", "trailer", "malformed-trailer"} {
t.Run(mode, func(t *testing.T) {
reader, err := NewReader(t.Context(), bytes.NewReader(content), int64(len(content)), "", "", int64(len(content)))
if err != nil {
t.Fatal(err)
}
req := httptest.NewRequest(http.MethodPut, "http://localhost/bucket/object", nil)
if mode == "header" {
req.Header.Set(typ.Key(), wrong.Encoded)
} else {
req.Header.Set("x-amz-trailer", typ.Key())
req.Trailer = make(http.Header)
value := wrong.Encoded
if mode == "malformed-trailer" {
value = "!invalid-base64!"
}
req.Trailer.Set(typ.Key(), value)
}
if err = reader.AddChecksum(req, false); err != nil {
t.Fatal(err)
}
_, err = io.ReadAll(reader)
check(t, err)
})
}
})
}
}
+6 -4
View File
@@ -584,15 +584,17 @@ func (c Checksum) Matches(content []byte, parts int) error {
sum := hasher.Sum(nil)
if c.WantParts > 0 && c.WantParts != parts {
return ChecksumMismatch{
Want: fmt.Sprintf("%s-%d", c.Encoded, c.WantParts),
Got: fmt.Sprintf("%s-%d", base64.StdEncoding.EncodeToString(sum), parts),
Algorithm: c.Type.String(),
Want: fmt.Sprintf("%s-%d", c.Encoded, c.WantParts),
Got: fmt.Sprintf("%s-%d", base64.StdEncoding.EncodeToString(sum), parts),
}
}
if !bytes.Equal(sum, c.Raw) {
return ChecksumMismatch{
Want: c.Encoded,
Got: base64.StdEncoding.EncodeToString(sum),
Algorithm: c.Type.String(),
Want: c.Encoded,
Got: base64.StdEncoding.EncodeToString(sum),
}
}
return nil
+5 -2
View File
@@ -74,8 +74,11 @@ func (e SizeMismatch) Error() string {
// ChecksumMismatch - when content checksum does not match with what was sent from client.
type ChecksumMismatch struct {
Want string
Got string
// Algorithm carries the requested checksum type (e.g. "CRC32") so API
// error rendering can name it, matching S3's wording.
Algorithm string
Want string
Got string
}
func (e ChecksumMismatch) Error() string {
+4 -3
View File
@@ -302,13 +302,14 @@ func (r *Reader) Read(p []byte) (int, error) {
r.contentHash.Encoded = r.trailer.Get(r.contentHash.Type.Key())
r.contentHash.Raw, err = base64.StdEncoding.DecodeString(r.contentHash.Encoded)
if err != nil || len(r.contentHash.Raw) == 0 {
return 0, ChecksumMismatch{Got: r.contentHash.Encoded}
return 0, ChecksumMismatch{Algorithm: r.contentHash.Type.String(), Got: r.contentHash.Encoded}
}
}
if sum := r.contentHasher.Sum(nil); !bytes.Equal(r.contentHash.Raw, sum) {
err := ChecksumMismatch{
Want: r.contentHash.Encoded,
Got: base64.StdEncoding.EncodeToString(sum),
Algorithm: r.contentHash.Type.String(),
Want: r.contentHash.Encoded,
Got: base64.StdEncoding.EncodeToString(sum),
}
return n, err
}