mirror of
https://github.com/pgsty/minio.git
synced 2026-11-11 01:55:30 +03:00
Compare commits
19 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 2f93e0c6b3 | |||
| f3b42b0eb8 | |||
| 2b62df876a | |||
| 518915d241 | |||
| ce5d191077 | |||
| b404f78764 | |||
| d138996983 | |||
| d52ba0290a | |||
| 8f3657dc52 | |||
| 7efdedd5ba | |||
| a18293d0c3 | |||
| 032ac49061 | |||
| 244de7ecc1 | |||
| c11bfda8f4 | |||
| 2c06b28dbb | |||
| a2ff3c2ecd | |||
| 67bc32f401 | |||
| 3c4a6296f9 | |||
| afa35d16e9 |
+40
-6
@@ -1,12 +1,45 @@
|
|||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
|
## Unreleased
|
||||||
|
|
||||||
|
- An edit of a service account or access key that names no policy keeps its
|
||||||
|
inline policy. `mcli admin user svcacct disable`, `enable` and `edit` without
|
||||||
|
`--policy`, the same `idp ldap accesskey` commands, and any other
|
||||||
|
`update-service-account` request without `newPolicy` removed the inline
|
||||||
|
policy at the site that took the request and, for an account of another user
|
||||||
|
than root, at its replication peers, and the account then had all its parent
|
||||||
|
is allowed. The inline policy is removed
|
||||||
|
only on request, by an empty policy document such as `{"Statement":[]}`
|
||||||
|
(`mcli admin user svcacct edit --policy`); a client that left the policy out
|
||||||
|
to remove it must send that document. A JSON `null` policy counts as none.
|
||||||
|
In the Console, clearing the policy editor of an access key no longer removes
|
||||||
|
the policy; enter `{}` to remove it. The fix does not bring back a policy
|
||||||
|
that was removed, and a site of a replicated deployment that does not run the
|
||||||
|
fix still removes the policy when such an edit reaches it, so sites can
|
||||||
|
differ. Upgrade every site, then review on each site the accounts that
|
||||||
|
`svcacct info` reports with `Policy: implied`, and set the policy again on
|
||||||
|
those that were meant to be restricted.
|
||||||
|
|
||||||
|
- S3 `DeleteObjects` on a single pool stops at an entry whose name fails the
|
||||||
|
object name check: the entry returns `XMinioInvalidObjectName` and is not
|
||||||
|
locked or sent to a drive, as with a single-object DELETE and with more than
|
||||||
|
one pool. Until now such an entry still reached the drives of the node, where
|
||||||
|
a name with a `..` segment was joined to the request bucket. A caller allowed
|
||||||
|
to delete in one bucket could remove a version in another bucket, including
|
||||||
|
one under Object Lock retention, and from a versioned bucket could put a
|
||||||
|
delete marker on an object of another bucket or on stored system metadata
|
||||||
|
such as a bucket's configuration. The other entries of a request are deleted
|
||||||
|
as before.
|
||||||
|
|
||||||
## RELEASE.2026-11-09T00-00-00Z
|
## RELEASE.2026-11-09T00-00-00Z
|
||||||
|
|
||||||
Package version: `20261109000000.0.0`. Publication status and artifacts are
|
Package version: `20261109000000.0.0`; published from source `d998292387cc`.
|
||||||
listed in [GitHub Releases](https://github.com/pgsty/silo/releases). This
|
Artifacts are listed in the [GitHub release](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-11-09T00-00-00Z).
|
||||||
section covers the changes after the published 20261026 source `c3ad836aed0d`.
|
The coordinated [English release notes](https://silo.pgsty.com/blog/release/silo-20261109/)
|
||||||
Most fixes come from reports and patches by
|
and [Chinese release notes](https://silo.pgsty.com/zh/blog/release/silo-20261109/)
|
||||||
[@botdeniska](https://github.com/botdeniska); see [CONTRIBUTORS.md](CONTRIBUTORS.md).
|
cover all components, upgrade guidance, and validation scope since the
|
||||||
|
published 20261026 source `c3ad836aed0d`. Most fixes come from reports and
|
||||||
|
patches by [@botdeniska](https://github.com/botdeniska); see [CONTRIBUTORS.md](CONTRIBUTORS.md).
|
||||||
|
|
||||||
Lifecycle, multi-pool and tiering:
|
Lifecycle, multi-pool and tiering:
|
||||||
|
|
||||||
@@ -21,7 +54,8 @@ Lifecycle, multi-pool and tiering:
|
|||||||
`retainVersions: 0`) on the expired key instead of the keys below it; a key
|
`retainVersions: 0`) on the expired key instead of the keys below it; a key
|
||||||
with a tiered or Object Lock protected version is refused
|
with a tiered or Object Lock protected version is refused
|
||||||
([#297](https://github.com/pgsty/silo/issues/297)).
|
([#297](https://github.com/pgsty/silo/issues/297)).
|
||||||
- Stop taking the free version of a deleted tiered version for a version
|
- Exclude the internal free-version marker of a deleted tiered version from
|
||||||
|
visible version history and lifecycle decisions
|
||||||
([#263](https://github.com/pgsty/silo/issues/263)), apply rules whose only
|
([#263](https://github.com/pgsty/silo/issues/263)), apply rules whose only
|
||||||
action is `DelMarkerExpiration` ([#268](https://github.com/pgsty/silo/issues/268)),
|
action is `DelMarkerExpiration` ([#268](https://github.com/pgsty/silo/issues/268)),
|
||||||
and report and account only the removals that happened
|
and report and account only the removals that happened
|
||||||
|
|||||||
+5
-2
@@ -165,8 +165,11 @@ license grant to SILO or upstream MinIO maintainers is required.
|
|||||||
|
|
||||||
### How does Silo manage dependencies?
|
### How does Silo manage dependencies?
|
||||||
|
|
||||||
Silo uses Go modules. Preserve the compatibility module and import paths in
|
Silo uses Go modules. Shared packages in `github.com/pgsty/silo-pkg/v3` are
|
||||||
`go.mod`; downstream forks are selected with explicit `replace` directives.
|
required and imported directly under their own module path, upstream
|
||||||
|
`github.com/minio/minio-go/v7` is pinned to a verified commit, and Console
|
||||||
|
(`github.com/minio/console`) and MC (`github.com/minio/mc`) are selected with
|
||||||
|
explicit `replace` directives in `go.mod`.
|
||||||
|
|
||||||
- Run `go get foo/bar` in the source folder to add the dependency to `go.mod` file.
|
- Run `go get foo/bar` in the source folder to add the dependency to `go.mod` file.
|
||||||
|
|
||||||
|
|||||||
+35
-23
File diff suppressed because one or more lines are too long
@@ -42,7 +42,8 @@ PRs must flow smoothly through review to reach production. Large PRs should be s
|
|||||||
- Document testing steps in the PR summary if manual testing was performed.
|
- Document testing steps in the PR summary if manual testing was performed.
|
||||||
|
|
||||||
4. **Before Submitting**:
|
4. **Before Submitting**:
|
||||||
- Run `make verify` to check formatting, linting, and tests.
|
- Sign off every commit under the Developer Certificate of Origin (`git commit -s`).
|
||||||
|
- Run `make verifiers` to check formatting, linting, code generation, and `rebrand-guard`.
|
||||||
- Reference related issues (e.g., “Closes #1234”).
|
- Reference related issues (e.g., “Closes #1234”).
|
||||||
- Notify team members via GitHub `@mentions` if urgent or complex.
|
- Notify team members via GitHub `@mentions` if urgent or complex.
|
||||||
|
|
||||||
|
|||||||
@@ -36,7 +36,7 @@
|
|||||||
|
|
||||||
PGSTY SILO keeps one maintained release line of the open-source MinIO server alive after upstream ended community distribution: builds, packages, multi-arch images, security fixes, and the full web console. Pigsty runs it in production as its PostgreSQL backup repository.
|
PGSTY SILO keeps one maintained release line of the open-source MinIO server alive after upstream ended community distribution: builds, packages, multi-arch images, security fixes, and the full web console. Pigsty runs it in production as its PostgreSQL backup repository.
|
||||||
|
|
||||||
It follows one rule — **the product and its delivery surfaces are renamed; the protocol and your data are not.** Everything else lives on [silo.pgsty.com](https://silo.pgsty.com/).
|
It follows one rule: **the product and its delivery surfaces are renamed; the protocol and your data are not.** Full documentation is published at [silo.pgsty.com](https://silo.pgsty.com/).
|
||||||
|
|
||||||
**Related:** [`pgsty/mc`](https://github.com/pgsty/mc) · [`pgsty/silo-console`](https://github.com/pgsty/silo-console) · [`pgsty/silo-pkg`](https://github.com/pgsty/silo-pkg) · [`pgsty/kes`](https://github.com/pgsty/kes) · [`pgsty/pigsty`](https://github.com/pgsty/pigsty)
|
**Related:** [`pgsty/mc`](https://github.com/pgsty/mc) · [`pgsty/silo-console`](https://github.com/pgsty/silo-console) · [`pgsty/silo-pkg`](https://github.com/pgsty/silo-pkg) · [`pgsty/kes`](https://github.com/pgsty/kes) · [`pgsty/pigsty`](https://github.com/pgsty/pigsty)
|
||||||
|
|
||||||
@@ -44,10 +44,10 @@ It follows one rule — **the product and its delivery surfaces are renamed; the
|
|||||||
<img src="https://silo.pgsty.com/images/silo-console/console-metrics-simple.webp" alt="Silo Console">
|
<img src="https://silo.pgsty.com/images/silo-console/console-metrics-simple.webp" alt="Silo Console">
|
||||||
</p>
|
</p>
|
||||||
|
|
||||||
#### SILO 20261109
|
### SILO 20261109
|
||||||
|
|
||||||
> [!WARNING]
|
> [!WARNING]
|
||||||
> [SILO 20261109](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-11-09T00-00-00Z) fixes bugs and security vulnerabilities. We strongly recommend scheduling an upgrade to reduce security risks.
|
> [SILO 20261109](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-11-09T00-00-00Z) (`20261109000000.0.0`, source `d998292387cc`) fixes storage, lifecycle, native restore, Object Lock, batch authorization, and IAM defects, embedding Console `v2.7.0` with `mcli 20261109`, `silo-pkg v3.16.0`, and Go `1.27.3`. Review the [release notes and upgrade requirements](https://silo.pgsty.com/blog/release/silo-20261109/) before upgrading.
|
||||||
|
|
||||||
> [!NOTE]
|
> [!NOTE]
|
||||||
> On 2026-08-06, `pgsty/minio` became `pgsty/silo`, and the default branch changed from `master` to `main`. The last MinIO-branded release is [`20260804`](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-08-04T00-00-00Z) on the [`minio`](https://github.com/pgsty/silo/tree/minio) branch.
|
> On 2026-08-06, `pgsty/minio` became `pgsty/silo`, and the default branch changed from `master` to `main`. The last MinIO-branded release is [`20260804`](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-08-04T00-00-00Z) on the [`minio`](https://github.com/pgsty/silo/tree/minio) branch.
|
||||||
@@ -86,7 +86,7 @@ docker exec silo mcli mb local/demo && docker exec silo mcli ls local
|
|||||||
| Kubernetes | Helm chart, see [Download & Install](https://silo.pgsty.com/download/) |
|
| Kubernetes | Helm chart, see [Download & Install](https://silo.pgsty.com/download/) |
|
||||||
| Source | `go build -o silo . && ./silo --version` |
|
| Source | `go build -o silo . && ./silo --version` |
|
||||||
|
|
||||||
Every release ships checksums, SPDX SBOMs, Sigstore-signed manifests, and GitHub build attestations. Installation methods and verification commands are documented at [Download & Install](https://silo.pgsty.com/download/); migrating from upstream MinIO — taking over an existing `minio.service` and its `/etc/default/minio`, and keeping data ownership stable with a `/etc/systemd/system/silo.service.d/10-legacy-user.conf` drop-in — is covered by the [migration guide](https://silo.pgsty.com/compatibility/migration/) and the [binary & service notes](https://silo.pgsty.com/compatibility/binary/).
|
Every release ships checksums, SPDX SBOMs, Sigstore-signed manifests, and GitHub build attestations. Installation methods and verification commands are documented at [Download & Install](https://silo.pgsty.com/download/). To migrate from upstream MinIO, take over `minio.service` and `/etc/default/minio`, and preserve data ownership with a `/etc/systemd/system/silo.service.d/10-legacy-user.conf` drop-in. See the [migration guide](https://silo.pgsty.com/compatibility/migration/) and [binary & service notes](https://silo.pgsty.com/compatibility/binary/).
|
||||||
|
|
||||||
## Compatibility
|
## Compatibility
|
||||||
|
|
||||||
@@ -98,7 +98,7 @@ Every divergence from upstream is listed in the code-verified [compatibility aud
|
|||||||
|
|
||||||
### TLS and Go upgrades
|
### TLS and Go upgrades
|
||||||
|
|
||||||
Server 20260916 restored Go's default key exchange; Server 20261018 retains it with Go 1.27.2.
|
Server 20260916 restored Go's default key exchange; Server 20261109 retains it with Go 1.27.3.
|
||||||
TLS key exchange follows Go's defaults across the S3 listener, node links,
|
TLS key exchange follows Go's defaults across the S3 listener, node links,
|
||||||
replication, identity providers, etcd, and external HTTP services. If an endpoint
|
replication, identity providers, etcd, and external HTTP services. If an endpoint
|
||||||
cannot accept ML-KEM, `GODEBUG=tlsmlkem=0` disables the default hybrid exchanges
|
cannot accept ML-KEM, `GODEBUG=tlsmlkem=0` disables the default hybrid exchanges
|
||||||
|
|||||||
+6
-6
@@ -36,7 +36,7 @@
|
|||||||
|
|
||||||
上游停止社区发行后,Silo 为开源 MinIO 服务端维护一条持续可用的版本线:构建、软件包、多架构镜像、安全修复与完整 Web 控制台。Pigsty 在生产环境中用它承载 PostgreSQL 备份存储。
|
上游停止社区发行后,Silo 为开源 MinIO 服务端维护一条持续可用的版本线:构建、软件包、多架构镜像、安全修复与完整 Web 控制台。Pigsty 在生产环境中用它承载 PostgreSQL 备份存储。
|
||||||
|
|
||||||
它只遵循一条原则:**改名的是产品与交付物,不是协议与你的数据。** 其余内容都在 [silo.pgsty.com](https://silo.pgsty.com/zh/)。
|
它只遵循一条原则:**改名的是产品与交付物,不是协议与你的数据。** 完整文档发布在 [silo.pgsty.com](https://silo.pgsty.com/zh/)。
|
||||||
|
|
||||||
**相关项目:**[`pgsty/mc`](https://github.com/pgsty/mc) · [`pgsty/silo-console`](https://github.com/pgsty/silo-console) · [`pgsty/silo-pkg`](https://github.com/pgsty/silo-pkg) · [`pgsty/kes`](https://github.com/pgsty/kes) · [`pgsty/pigsty`](https://github.com/pgsty/pigsty)
|
**相关项目:**[`pgsty/mc`](https://github.com/pgsty/mc) · [`pgsty/silo-console`](https://github.com/pgsty/silo-console) · [`pgsty/silo-pkg`](https://github.com/pgsty/silo-pkg) · [`pgsty/kes`](https://github.com/pgsty/kes) · [`pgsty/pigsty`](https://github.com/pgsty/pigsty)
|
||||||
|
|
||||||
@@ -44,10 +44,10 @@
|
|||||||
<img src="https://silo.pgsty.com/images/silo-console/console-metrics-simple.webp" alt="Silo 控制台">
|
<img src="https://silo.pgsty.com/images/silo-console/console-metrics-simple.webp" alt="Silo 控制台">
|
||||||
</p>
|
</p>
|
||||||
|
|
||||||
#### SILO 20261109
|
### SILO 20261109
|
||||||
|
|
||||||
> [!WARNING]
|
> [!WARNING]
|
||||||
> [SILO 20261109](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-11-09T00-00-00Z) 修复了缺陷与安全漏洞,强烈建议尽快安排升级,以降低安全风险。
|
> [SILO 20261109](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-11-09T00-00-00Z)(包版本 `20261109000000.0.0`,源码 `d998292387cc`)修复了多池生命周期、原生恢复、Object Lock、批处理授权与 IAM 等缺陷,内嵌 Console `v2.7.0` 并配套 `mcli 20261109`、`silo-pkg v3.16.0` 与 Go `1.27.3`。升级前请阅读[发布说明与升级要求](https://silo.pgsty.com/zh/blog/release/silo-20261109/)。
|
||||||
|
|
||||||
> [!NOTE]
|
> [!NOTE]
|
||||||
> 2026-08-06,`pgsty/minio` 更名为 `pgsty/silo`,默认分支由 `master` 改为 `main`。最后一个 MinIO 品牌发布是 [`minio`](https://github.com/pgsty/silo/tree/minio) 分支上的 [`20260804`](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-08-04T00-00-00Z)。
|
> 2026-08-06,`pgsty/minio` 更名为 `pgsty/silo`,默认分支由 `master` 改为 `main`。最后一个 MinIO 品牌发布是 [`minio`](https://github.com/pgsty/silo/tree/minio) 分支上的 [`20260804`](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-08-04T00-00-00Z)。
|
||||||
@@ -98,9 +98,9 @@ Silo 保留 S3 与存储格式兼容性,包括既有 `MINIO_*` 环境变量、
|
|||||||
|
|
||||||
### TLS 与 Go 升级
|
### TLS 与 Go 升级
|
||||||
|
|
||||||
Server 20260916 已恢复 Go 默认密钥交换策略,20261018 在 Go 1.27.2 下保留该行为。
|
Server 20260916 恢复了 Go 默认密钥交换策略,Server 20261109 在 Go 1.27.3 下保留该行为。S3 监听端口、节点间连接、站点与桶复制、身份提供方、etcd 及外部 HTTP 服务均使用 Go 默认 TLS 密钥交换。若对端无法接受 ML-KEM,设置 `GODEBUG=tlsmlkem=0` 可在当前进程禁用默认混合密钥交换(证书校验保持开启);该选项不会禁用 ML-DSA 签名,也不能解决所有 TLS 连接重置问题,建议在升级不兼容对端后移除该临时配置。若仅新增的 SecP 混合组引发问题,可设置 `GODEBUG=tlssecpmlkem=0` 仅禁用这些组并保留 `X25519MLKEM768`。
|
||||||
`GODEBUG=tlsmlkem=0`、`tlssecpmlkem=0` 的适用范围、macOS 根证书来源变化,
|
|
||||||
以及 OIDC discovery 的诊断方法见 [Go 1.27 TLS 与 OIDC 指南](https://silo.pgsty.com/zh/blog/design/go127-tls-oidc-discovery/)。
|
在 Go 1.27 构建中,macOS 上设置 `SSL_CERT_FILE` 或 `SSL_CERT_DIR` 会以磁盘根证书和 Go 验证器替代系统钥匙串信任;陈旧或不完整的 CA 路径可能导致原先受信任的连接失败,清除继承的环境变量即可恢复钥匙串信任。配置的 `CAs` 目录中的显式证书仍会叠加加入所选根证书池。Go 1.27 二进制要求 macOS 13 或更高版本。详见 [Go 1.27 发行说明](https://go.dev/doc/go1.27)与 [Go 1.27 TLS 与 OIDC 发现指南](https://silo.pgsty.com/zh/blog/design/go127-tls-oidc-discovery/)。
|
||||||
|
|
||||||
## 文档归属
|
## 文档归属
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -38,7 +38,7 @@ For vulnerabilities in this fork:
|
|||||||
|
|
||||||
1. Follow the fork-specific expectations in [VULNERABILITY_REPORT.md](VULNERABILITY_REPORT.md).
|
1. Follow the fork-specific expectations in [VULNERABILITY_REPORT.md](VULNERABILITY_REPORT.md).
|
||||||
2. Prefer this repository's [private GitHub security advisory](https://github.com/pgsty/silo/security/advisories/new) workflow.
|
2. Prefer this repository's [private GitHub security advisory](https://github.com/pgsty/silo/security/advisories/new) workflow.
|
||||||
3. If you cannot use GitHub private reporting, email [ron@pgsty.com](mailto:ron@pgsty.com) or rh@vonng.com
|
3. If you cannot use GitHub private reporting, email [ron@pgsty.com](mailto:ron@pgsty.com) or [rh@vonng.com](mailto:rh@vonng.com)
|
||||||
directly. Do not publish vulnerability details or exploit code in public
|
directly. Do not publish vulnerability details or exploit code in public
|
||||||
issues or pull requests.
|
issues or pull requests.
|
||||||
4. If you confirm the issue also affects upstream `minio/minio`, report it upstream separately.
|
4. If you confirm the issue also affects upstream `minio/minio`, report it upstream separately.
|
||||||
|
|||||||
@@ -885,16 +885,16 @@ func (a adminAPIHandlers) UpdateServiceAccount(w http.ResponseWriter, r *http.Re
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// A request without a policy, or with a JSON null for one, keeps the
|
||||||
|
// policy the account has. An empty policy document is passed on as
|
||||||
|
// such: it is the request to remove the embedded policy.
|
||||||
var sp *policy.Policy
|
var sp *policy.Policy
|
||||||
if len(updateReq.NewPolicy) > 0 {
|
if len(updateReq.NewPolicy) > 0 && !bytes.Equal(updateReq.NewPolicy, []byte("null")) {
|
||||||
sp, err = policy.ParseConfigStrict(bytes.NewReader(updateReq.NewPolicy))
|
sp, err = policy.ParseConfigStrict(bytes.NewReader(updateReq.NewPolicy))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
writeErrorResponseJSON(ctx, w, toAdminAPIErr(ctx, err), r.URL)
|
writeErrorResponseJSON(ctx, w, toAdminAPIErr(ctx, err), r.URL)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
if sp.Version == "" && len(sp.Statements) == 0 {
|
|
||||||
sp = nil
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
opts := updateServiceAccountOpts{
|
opts := updateServiceAccountOpts{
|
||||||
secretKey: updateReq.NewSecretKey,
|
secretKey: updateReq.NewSecretKey,
|
||||||
|
|||||||
@@ -0,0 +1,289 @@
|
|||||||
|
// Copyright (c) 2026 Denis Babchuk
|
||||||
|
//
|
||||||
|
// This file is part of Silo Object Storage stack
|
||||||
|
//
|
||||||
|
// This program is free software: you can redistribute it and/or modify
|
||||||
|
// it under the terms of the GNU Affero General Public License as published by
|
||||||
|
// the Free Software Foundation, either version 3 of the License, or
|
||||||
|
// (at your option) any later version.
|
||||||
|
//
|
||||||
|
// This program is distributed in the hope that it will be useful,
|
||||||
|
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||||
|
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||||
|
// GNU Affero General Public License for more details.
|
||||||
|
//
|
||||||
|
// You should have received a copy of the GNU Affero General Public License
|
||||||
|
// along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||||
|
|
||||||
|
package cmd
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"maps"
|
||||||
|
"net/http"
|
||||||
|
"slices"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"github.com/minio/minio-go/v7/pkg/set"
|
||||||
|
"github.com/minio/minio-go/v7/pkg/tags"
|
||||||
|
"github.com/minio/minio/internal/auth"
|
||||||
|
"github.com/minio/minio/internal/logger"
|
||||||
|
"github.com/pgsty/silo-pkg/v3/policy"
|
||||||
|
"github.com/pgsty/silo-pkg/v3/policy/condition"
|
||||||
|
)
|
||||||
|
|
||||||
|
// authorizeRequestWithUnreadTags authorizes action on an object whose stored
|
||||||
|
// tags could not be read because reading the object failed with readErr.
|
||||||
|
//
|
||||||
|
// Without the tags, an Allow statement that tests s3:ExistingObjectTag cannot
|
||||||
|
// match, so a caller allowed only by such a statement would get AccessDenied,
|
||||||
|
// which clients do not retry, for a transient server error such as lost read
|
||||||
|
// quorum. For a server error the request is therefore also authorized as if
|
||||||
|
// the object carried tags that the policies ask for: a caller that some tags
|
||||||
|
// let through gets the error, a caller that no tags let through still gets
|
||||||
|
// AccessDenied. For any other error, NoSuchKey among them, the tags stay empty.
|
||||||
|
func authorizeRequestWithUnreadTags(ctx context.Context, r *http.Request, action policy.Action, readErr error) APIErrorCode {
|
||||||
|
s3Err := authorizeRequest(ctx, r, action)
|
||||||
|
if s3Err == ErrNone || toAPIError(ctx, readErr).HTTPStatusCode < http.StatusInternalServerError {
|
||||||
|
return s3Err
|
||||||
|
}
|
||||||
|
for _, existingTags := range policyExistingObjectTags(ctx, r, action) {
|
||||||
|
if authorizeRequestWithExistingTags(ctx, r, action, existingTags) == ErrNone {
|
||||||
|
return ErrNone
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return s3Err
|
||||||
|
}
|
||||||
|
|
||||||
|
// maxExistingTagCandidates bounds the tag sets tried for one request; each
|
||||||
|
// one costs an authorization.
|
||||||
|
const maxExistingTagCandidates = 64
|
||||||
|
|
||||||
|
// policyExistingObjectTags returns tag sets under which the policies that
|
||||||
|
// authorize the request may let action through on its object: the bucket
|
||||||
|
// policy for an anonymous request, else the cached IAM policies and, as a
|
||||||
|
// second policy set, the session policy of the credential. A tag set passes
|
||||||
|
// the s3:ExistingObjectTag tests of one Allow statement of each policy set and,
|
||||||
|
// as far as the same tags allow, keeps the Deny statements that test tags from
|
||||||
|
// matching. The tag sets are only candidates; the regular authorization
|
||||||
|
// decides what they allow.
|
||||||
|
func policyExistingObjectTags(ctx context.Context, r *http.Request, action policy.Action) []string {
|
||||||
|
reqInfo := logger.GetReqInfo(ctx)
|
||||||
|
if reqInfo == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
bucket, object := reqInfo.BucketName, reqInfo.ObjectName
|
||||||
|
var (
|
||||||
|
allows [][]map[string]string // per policy set, the tags its Allow statements ask for
|
||||||
|
denies []map[string]string // tags that keep a Deny statement from matching
|
||||||
|
)
|
||||||
|
// add files the tags of a statement that applies to action on the
|
||||||
|
// object, whatever its conditions.
|
||||||
|
add := func(effect policy.Effect, conditions condition.Functions, values map[string][]string, fromSet *[]map[string]string) {
|
||||||
|
if tagMap, ok := existingObjectTagsFor(effect, conditions, values); ok {
|
||||||
|
if effect == policy.Allow {
|
||||||
|
*fromSet = append(*fromSet, tagMap)
|
||||||
|
} else {
|
||||||
|
denies = append(denies, tagMap)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
switch {
|
||||||
|
case reqInfo.Cred.AccessKey == "":
|
||||||
|
p, err := globalPolicySys.Get(bucket)
|
||||||
|
if err != nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
values := getConditionValues(r, reqInfo.Region, auth.AnonymousCredentials)
|
||||||
|
var fromSet []map[string]string
|
||||||
|
for _, st := range p.Statements {
|
||||||
|
match := st
|
||||||
|
match.Effect, match.Conditions = policy.Allow, nil
|
||||||
|
if match.IsAllowed(policy.BucketPolicyArgs{Action: action, BucketName: bucket, ObjectName: object, ConditionValues: values}) {
|
||||||
|
add(st.Effect, st.Conditions, values, &fromSet)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
allows = append(allows, fromSet)
|
||||||
|
case newGlobalAuthZPluginFn() == nil && globalIAMSys.Initialized():
|
||||||
|
values := getConditionValues(r, "", reqInfo.Cred)
|
||||||
|
// From the cache only: reading the drives may be what just failed.
|
||||||
|
docs, _ := globalIAMSys.store.listPolicyDocs(ctx, "")
|
||||||
|
policySets := [][]policy.Policy{nil}
|
||||||
|
for _, name := range slices.Sorted(maps.Keys(docs)) {
|
||||||
|
policySets[0] = append(policySets[0], docs[name].Policy)
|
||||||
|
}
|
||||||
|
if sp, ok := reqInfo.Cred.Claims[sessionPolicyNameExtracted].(string); ok {
|
||||||
|
if p, err := policy.ParseConfig(strings.NewReader(sp)); err == nil {
|
||||||
|
policySets = append(policySets, []policy.Policy{*p})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, policies := range policySets {
|
||||||
|
var fromSet []map[string]string
|
||||||
|
for _, p := range policies {
|
||||||
|
for _, st := range p.Statements {
|
||||||
|
match := st
|
||||||
|
match.Effect, match.Conditions = policy.Allow, nil
|
||||||
|
if match.IsAllowed(policy.Args{Action: action, BucketName: bucket, ObjectName: object, ConditionValues: values}) {
|
||||||
|
add(st.Effect, st.Conditions, values, &fromSet)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
allows = append(allows, fromSet)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return combineExistingObjectTags(allows, denies)
|
||||||
|
}
|
||||||
|
|
||||||
|
// combineExistingObjectTags merges the tags of one Allow statement of each
|
||||||
|
// policy set, or of none for a set that may allow without tags, and tries each
|
||||||
|
// result also with the tags that keep the Deny statements from matching. The
|
||||||
|
// first tags win a conflict; the authorization of the result settles it.
|
||||||
|
func combineExistingObjectTags(allows [][]map[string]string, denies []map[string]string) []string {
|
||||||
|
merge := func(parts ...map[string]string) map[string]string {
|
||||||
|
merged := make(map[string]string)
|
||||||
|
for _, part := range parts {
|
||||||
|
for k, v := range part {
|
||||||
|
if _, ok := merged[k]; !ok {
|
||||||
|
merged[k] = v
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return merged
|
||||||
|
}
|
||||||
|
combos := [][]map[string]string{nil}
|
||||||
|
for _, fromSet := range allows {
|
||||||
|
// Many statements ask for the same tags.
|
||||||
|
seen := set.NewStringSet()
|
||||||
|
fromSet = slices.DeleteFunc(slices.Clone(fromSet), func(tagMap map[string]string) bool {
|
||||||
|
t, err := tags.MapToObjectTags(tagMap)
|
||||||
|
if err != nil || seen.Contains(t.String()) {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
seen.Add(t.String())
|
||||||
|
return false
|
||||||
|
})
|
||||||
|
next := make([][]map[string]string, 0, len(combos)*(len(fromSet)+1))
|
||||||
|
for _, combo := range combos {
|
||||||
|
next = append(next, combo)
|
||||||
|
for _, tagMap := range fromSet {
|
||||||
|
next = append(next, append(slices.Clip(combo), tagMap))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
combos = next[:min(len(next), maxExistingTagCandidates)]
|
||||||
|
}
|
||||||
|
deny := merge(denies...)
|
||||||
|
found := set.NewStringSet()
|
||||||
|
var candidates []string
|
||||||
|
for _, combo := range combos {
|
||||||
|
for _, tagMap := range []map[string]string{merge(combo...), merge(append(slices.Clip(combo), deny)...)} {
|
||||||
|
t, err := tags.MapToObjectTags(tagMap)
|
||||||
|
if len(tagMap) == 0 || err != nil || found.Contains(t.String()) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
found.Add(t.String())
|
||||||
|
if candidates = append(candidates, t.String()); len(candidates) == maxExistingTagCandidates {
|
||||||
|
return candidates
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return candidates
|
||||||
|
}
|
||||||
|
|
||||||
|
// existingObjectTagsFor returns tags under which the s3:ExistingObjectTag tests
|
||||||
|
// of conditions all pass, for an Allow statement, or one of them fails, for a
|
||||||
|
// Deny statement; a tag left out is absent. It returns false if the conditions
|
||||||
|
// test no tag, or if no such tags were found among those tried for each tag:
|
||||||
|
// none, the values the tests name, with policy variables taken from values and
|
||||||
|
// wildcards filled, and the fillers "" and "_". The policy engine itself
|
||||||
|
// evaluates the tests for each of them.
|
||||||
|
func existingObjectTagsFor(effect policy.Effect, conditions condition.Functions, values map[string][]string) (map[string]string, bool) {
|
||||||
|
// Each function tests one key, so the tests of one tag pass or fail
|
||||||
|
// whatever the other tags are.
|
||||||
|
byKey := make(map[string]condition.Functions)
|
||||||
|
for _, f := range conditions {
|
||||||
|
for key := range condition.NewFunctions(f).Keys() {
|
||||||
|
if key.Is(condition.ExistingObjectTag) {
|
||||||
|
byKey[key.Name()] = append(byKey[key.Name()], f)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(byKey) == 0 {
|
||||||
|
return nil, false
|
||||||
|
}
|
||||||
|
tagMap := make(map[string]string)
|
||||||
|
for _, key := range slices.Sorted(maps.Keys(byKey)) {
|
||||||
|
tagKey := strings.TrimPrefix(key, condition.ExistingObjectTag.Name()+"/")
|
||||||
|
value, present, found := existingObjectTagValue(key, tagKey, byKey[key], values, effect == policy.Allow)
|
||||||
|
switch {
|
||||||
|
case effect == policy.Deny && found:
|
||||||
|
if present {
|
||||||
|
tagMap[tagKey] = value
|
||||||
|
}
|
||||||
|
return tagMap, true
|
||||||
|
case effect == policy.Allow && !found:
|
||||||
|
return nil, false
|
||||||
|
case present:
|
||||||
|
tagMap[tagKey] = value
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if effect == policy.Deny {
|
||||||
|
return nil, false
|
||||||
|
}
|
||||||
|
return tagMap, true
|
||||||
|
}
|
||||||
|
|
||||||
|
// existingObjectTagValue looks for a value of tag tagKey, or its absence,
|
||||||
|
// under which tests, the functions that test the condition key key, evaluate
|
||||||
|
// to want.
|
||||||
|
func existingObjectTagValue(key, tagKey string, tests condition.Functions, values map[string][]string, want bool) (value string, present, found bool) {
|
||||||
|
defer delete(values, key)
|
||||||
|
delete(values, key)
|
||||||
|
if tests.Evaluate(values) == want {
|
||||||
|
return "", false, true
|
||||||
|
}
|
||||||
|
// The functions keep their values unexported; JSON exposes them.
|
||||||
|
buf, err := json.Marshal(tests)
|
||||||
|
if err != nil {
|
||||||
|
return "", false, false
|
||||||
|
}
|
||||||
|
var byName map[string]map[string]condition.ValueSet
|
||||||
|
if err = json.Unmarshal(buf, &byName); err != nil {
|
||||||
|
return "", false, false
|
||||||
|
}
|
||||||
|
var tried []string
|
||||||
|
for name, byKey := range byName {
|
||||||
|
if name == "Null" {
|
||||||
|
continue // its operand says whether the tag is there, not a value
|
||||||
|
}
|
||||||
|
for _, valueSet := range byKey {
|
||||||
|
for _, v := range valueSet.ToSlice() {
|
||||||
|
s := substituteConditionVariables(v.String(), values)
|
||||||
|
tried = append(tried, s, strings.NewReplacer("*", "", "?", "_").Replace(s))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
slices.Sort(tried)
|
||||||
|
for _, v := range append(slices.Compact(tried), "", "_") {
|
||||||
|
if _, err := tags.MapToObjectTags(map[string]string{tagKey: v}); err != nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
values[key] = []string{v}
|
||||||
|
if tests.Evaluate(values) == want {
|
||||||
|
return v, true, true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return "", false, false
|
||||||
|
}
|
||||||
|
|
||||||
|
// substituteConditionVariables resolves the policy variables in v from the
|
||||||
|
// condition values as the policy engine does for the values of a condition;
|
||||||
|
// silo-pkg keeps its own function for this unexported.
|
||||||
|
func substituteConditionVariables(v string, values map[string][]string) string {
|
||||||
|
for _, key := range condition.CommonKeys {
|
||||||
|
if rvalues, ok := values[key.Name()]; ok && len(rvalues) > 0 && rvalues[0] != "" {
|
||||||
|
v = strings.ReplaceAll(v, key.VarName(), rvalues[0])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return v
|
||||||
|
}
|
||||||
@@ -0,0 +1,305 @@
|
|||||||
|
// Copyright (c) 2026 Denis Babchuk
|
||||||
|
//
|
||||||
|
// This file is part of Silo Object Storage stack
|
||||||
|
//
|
||||||
|
// This program is free software: you can redistribute it and/or modify
|
||||||
|
// it under the terms of the GNU Affero General Public License as published by
|
||||||
|
// the Free Software Foundation, either version 3 of the License, or
|
||||||
|
// (at your option) any later version.
|
||||||
|
//
|
||||||
|
// This program is distributed in the hope that it will be useful,
|
||||||
|
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||||
|
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||||
|
// GNU Affero General Public License for more details.
|
||||||
|
//
|
||||||
|
// You should have received a copy of the GNU Affero General Public License
|
||||||
|
// along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||||
|
|
||||||
|
package cmd
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"encoding/xml"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"net/url"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/minio/minio-go/v7/pkg/tags"
|
||||||
|
"github.com/minio/minio/internal/auth"
|
||||||
|
xhttp "github.com/minio/minio/internal/http"
|
||||||
|
"github.com/pgsty/silo-pkg/v3/policy"
|
||||||
|
"github.com/pgsty/silo-pkg/v3/policy/condition"
|
||||||
|
)
|
||||||
|
|
||||||
|
// objectReadFault takes the first n drives of the set that holds object
|
||||||
|
// offline for the metadata reads of that object only, so that IAM and bucket
|
||||||
|
// metadata stay readable. It returns a function that restores the drives.
|
||||||
|
func objectReadFault(z *erasureServerPools, bucket, object string, n int) func() {
|
||||||
|
pool := z.serverPools[0]
|
||||||
|
setIndex := pool.getHashedSetIndex(object)
|
||||||
|
pool.erasureDisksMu.Lock()
|
||||||
|
defer pool.erasureDisksMu.Unlock()
|
||||||
|
previous := pool.erasureDisks[setIndex]
|
||||||
|
disks := append([]StorageAPI(nil), previous...)
|
||||||
|
for i := range n {
|
||||||
|
disks[i] = consistencyReadFaultDisk{StorageAPI: disks[i], bucket: bucket, object: object}
|
||||||
|
}
|
||||||
|
pool.erasureDisks[setIndex] = disks
|
||||||
|
return func() {
|
||||||
|
pool.erasureDisksMu.Lock()
|
||||||
|
pool.erasureDisks[setIndex] = previous
|
||||||
|
pool.erasureDisksMu.Unlock()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A caller allowed to read an object only through s3:ExistingObjectTag must
|
||||||
|
// get the storage error, which clients retry, and not AccessDenied, when the
|
||||||
|
// object's tags cannot be read because read quorum is lost. A caller that no
|
||||||
|
// tags would let through must still get AccessDenied, and nothing changes
|
||||||
|
// while the tags can be read.
|
||||||
|
func TestExistingObjectTagAuthReadQuorumLost(t *testing.T) {
|
||||||
|
ctx, cancel := context.WithCancel(t.Context())
|
||||||
|
defer cancel()
|
||||||
|
obj, dirs, err := prepareErasure16(ctx)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
z := obj.(*erasureServerPools)
|
||||||
|
previous := newObjectLayerFn()
|
||||||
|
setObjectLayer(z)
|
||||||
|
defer func() {
|
||||||
|
z.Shutdown(context.Background())
|
||||||
|
removeRoots(dirs)
|
||||||
|
setObjectLayer(previous)
|
||||||
|
}()
|
||||||
|
bucket, router, err := initAPIHandlerTest(ctx, z, nil, MakeBucketOptions{})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
statement := func(effect, resource, conditions string) string {
|
||||||
|
st := fmt.Sprintf(`{"Effect": %q, "Action": ["s3:GetObject", "s3:PutObjectTagging"], "Resource": ["arn:aws:s3:::%s/%s"]`,
|
||||||
|
effect, bucket, resource)
|
||||||
|
if conditions != "" {
|
||||||
|
st += `, "Condition": ` + conditions
|
||||||
|
}
|
||||||
|
return st + "}"
|
||||||
|
}
|
||||||
|
tagStatement := func(effect, resource, value string) string {
|
||||||
|
return statement(effect, resource, fmt.Sprintf(`{"StringEquals": {"s3:ExistingObjectTag/k": %q}}`, value))
|
||||||
|
}
|
||||||
|
newUser := func(statements ...string) auth.Credentials {
|
||||||
|
return newDeleteAuthzPolicyUser(t, "", bucket, "["+strings.Join(statements, ",")+"]")
|
||||||
|
}
|
||||||
|
newServiceAccount := func(parent auth.Credentials, statements string) auth.Credentials {
|
||||||
|
sessionPolicy, err := policy.ParseConfig(strings.NewReader(`{"Version": "2012-10-17", "Statement": [` + statements + `]}`))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
cred, _, err := globalIAMSys.NewServiceAccount(ctx, parent.AccessKey, nil, newServiceAccountOpts{sessionPolicy: sessionPolicy})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return cred
|
||||||
|
}
|
||||||
|
|
||||||
|
type user struct {
|
||||||
|
name string
|
||||||
|
creds auth.Credentials
|
||||||
|
// The status with the object's tags readable and with read quorum
|
||||||
|
// lost: 200 and 503 for a caller that some tags let through, 403
|
||||||
|
// and 403 for a caller that none do.
|
||||||
|
healthy, quorumLost int
|
||||||
|
}
|
||||||
|
allowedByTags := func(name string, creds auth.Credentials) user {
|
||||||
|
return user{name, creds, http.StatusOK, http.StatusServiceUnavailable}
|
||||||
|
}
|
||||||
|
deniedByAnyTags := func(name string, creds auth.Credentials) user {
|
||||||
|
return user{name, creds, http.StatusForbidden, http.StatusForbidden}
|
||||||
|
}
|
||||||
|
// Each case below that tests a tag key of its own uses a key no other
|
||||||
|
// policy asks for.
|
||||||
|
ownerCondition := `{"StringEquals": {"s3:ExistingObjectTag/owner": "${aws:username}"}}`
|
||||||
|
owner := newUser(statement("Allow", "*", ownerCondition))
|
||||||
|
users := []user{
|
||||||
|
// Allowed only while the object carries k=v.
|
||||||
|
allowedByTags("tagged", newUser(tagStatement("Allow", "*", "v"))),
|
||||||
|
// Allowed while the object carries a tag np, whatever its value.
|
||||||
|
allowedByTags("nullFalse", newUser(statement("Allow", "*", `{"Null": {"s3:ExistingObjectTag/np": "false"}}`))),
|
||||||
|
// Allowed while a tag ne is there and is not w.
|
||||||
|
allowedByTags("notEquals", newUser(statement("Allow", "*",
|
||||||
|
`{"Null": {"s3:ExistingObjectTag/ne": "false"}, "StringNotEquals": {"s3:ExistingObjectTag/ne": "w"}}`))),
|
||||||
|
// Allowed while every value of tag fa differs from w; that needs a tag fa.
|
||||||
|
allowedByTags("forAllNotEquals", newUser(statement("Allow", "*", `{"ForAllValues:StringNotEquals": {"s3:ExistingObjectTag/fa": "w"}}`))),
|
||||||
|
// Allowed while the object is tagged with the caller's name.
|
||||||
|
allowedByTags("owner", owner),
|
||||||
|
// Allowed, but denied unless the object carries du=1.
|
||||||
|
allowedByTags("denyUnlessTagged", newUser(statement("Allow", "*", ""),
|
||||||
|
statement("Deny", "*", `{"StringNotEquals": {"s3:ExistingObjectTag/du": "1"}}`))),
|
||||||
|
// A service account whose parent asks for k=v and whose own policy
|
||||||
|
// asks for j=w: only an object carrying both lets it through.
|
||||||
|
allowedByTags("serviceAccount", newServiceAccount(newUser(tagStatement("Allow", "*", "v")),
|
||||||
|
statement("Allow", "*", `{"StringEquals": {"s3:ExistingObjectTag/j": "w"}}`))),
|
||||||
|
|
||||||
|
// Allowed only for k=w: the decision still depends on the tags.
|
||||||
|
{"otherValue", newUser(tagStatement("Allow", "*", "w")), http.StatusForbidden, http.StatusServiceUnavailable},
|
||||||
|
|
||||||
|
// Allowed for k=v under another prefix only.
|
||||||
|
deniedByAnyTags("otherPrefix", newUser(tagStatement("Allow", "other/*", "v"))),
|
||||||
|
// Allowed and denied for k=v alike.
|
||||||
|
deniedByAnyTags("allowDeny", newUser(tagStatement("Allow", "*", "v"), tagStatement("Deny", "*", "v"))),
|
||||||
|
// Allowed only without a tag nc that is 1.
|
||||||
|
deniedByAnyTags("nullConflict", newUser(statement("Allow", "*",
|
||||||
|
`{"Null": {"s3:ExistingObjectTag/nc": "true"}, "StringEquals": {"s3:ExistingObjectTag/nc": "1"}}`))),
|
||||||
|
// Tagged with the caller's name, under another prefix only.
|
||||||
|
deniedByAnyTags("ownerPrefix", newUser(statement("Allow", "other/*", ownerCondition))),
|
||||||
|
// Denied unless the object carries db=1, and denied if it does.
|
||||||
|
deniedByAnyTags("denyBoth", newUser(statement("Allow", "*", ""),
|
||||||
|
statement("Deny", "*", `{"StringNotEquals": {"s3:ExistingObjectTag/db": "1"}}`),
|
||||||
|
statement("Deny", "*", `{"StringEquals": {"s3:ExistingObjectTag/db": "1"}}`))),
|
||||||
|
// The parent asks for k=v and the service account for k=x.
|
||||||
|
deniedByAnyTags("serviceAccountConflict", newServiceAccount(newUser(tagStatement("Allow", "*", "v")),
|
||||||
|
tagStatement("Allow", "*", "x"))),
|
||||||
|
}
|
||||||
|
|
||||||
|
const object, missing = "tagged", "missing"
|
||||||
|
if _, err := z.PutObject(ctx, bucket, object, mustGetPutObjReader(t, bytes.NewReader([]byte("data")), 4, "", ""),
|
||||||
|
ObjectOptions{UserDefined: map[string]string{xhttp.AmzObjectTagging: "k=v&j=w&np=1&ne=1&fa=1&du=1&owner=" + owner.AccessKey}}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
// Anonymous reads are allowed by the bucket policy while the object carries k=v.
|
||||||
|
if _, err := globalBucketMetadataSys.Update(ctx, bucket, bucketPolicyConfig, []byte(`{"Version": "2012-10-17", "Statement": [
|
||||||
|
{"Effect": "Allow", "Principal": {"AWS": ["*"]}, "Action": ["s3:GetObject"], "Resource": ["arn:aws:s3:::`+bucket+`/*"],
|
||||||
|
"Condition": {"StringEquals": {"s3:ExistingObjectTag/k": "v"}}}]}`)); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
type request struct {
|
||||||
|
name string
|
||||||
|
method, target string
|
||||||
|
header map[string]string
|
||||||
|
body []byte
|
||||||
|
}
|
||||||
|
objectURL := getGetObjectURL("", bucket, object)
|
||||||
|
get := request{"GET", http.MethodGet, objectURL, nil, nil}
|
||||||
|
head := request{"HEAD", http.MethodHead, objectURL, nil, nil}
|
||||||
|
taggingURL := makeTestTargetURL("", bucket, object, url.Values{"tagging": []string{""}})
|
||||||
|
taggingBody := []byte(`<Tagging><TagSet><Tag><Key>k</Key><Value>v</Value></Tag></TagSet></Tagging>`)
|
||||||
|
putTagging := request{"PutObjectTagging", http.MethodPut, taggingURL, nil, taggingBody}
|
||||||
|
check := func(stage string, creds auth.Credentials, name string, req request, want int) {
|
||||||
|
t.Helper()
|
||||||
|
var rec *httptest.ResponseRecorder
|
||||||
|
if creds.AccessKey == "" {
|
||||||
|
anonymous, err := newTestRequest(req.method, req.target, 0, nil)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
rec = httptest.NewRecorder()
|
||||||
|
router.ServeHTTP(rec, anonymous)
|
||||||
|
} else {
|
||||||
|
var body io.ReadSeeker
|
||||||
|
if req.body != nil {
|
||||||
|
body = bytes.NewReader(req.body)
|
||||||
|
}
|
||||||
|
signed, err := newTestSignedRequestV4(req.method, req.target, int64(len(req.body)), body, creds.AccessKey, creds.SecretKey, req.header)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
rec = httptest.NewRecorder()
|
||||||
|
router.ServeHTTP(rec, signed)
|
||||||
|
}
|
||||||
|
if rec.Code != want {
|
||||||
|
var apiErr APIErrorResponse
|
||||||
|
_ = xml.Unmarshal(rec.Body.Bytes(), &apiErr)
|
||||||
|
t.Errorf("%s: %s by %s: status %d %s, want %d", stage, req.name, name, rec.Code, apiErr.Code, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
var anonymous auth.Credentials
|
||||||
|
for _, u := range users {
|
||||||
|
check("healthy", u.creds, u.name, get, u.healthy)
|
||||||
|
}
|
||||||
|
check("healthy", anonymous, "anonymous", get, http.StatusOK)
|
||||||
|
// The tags were read and allow the request: the range error is the answer.
|
||||||
|
check("healthy", users[0].creds, users[0].name,
|
||||||
|
request{"GET bytes=100-200", http.MethodGet, objectURL, map[string]string{xhttp.Range: "bytes=100-200"}, nil},
|
||||||
|
http.StatusRequestedRangeNotSatisfiable)
|
||||||
|
// Without an object there are no tags: AccessDenied, as before.
|
||||||
|
missingURL := getGetObjectURL("", bucket, missing)
|
||||||
|
check("missing key", users[0].creds, users[0].name, request{"GET", http.MethodGet, missingURL, nil, nil}, http.StatusForbidden)
|
||||||
|
check("missing key", users[0].creds, users[0].name, request{"HEAD", http.MethodHead, missingURL, nil, nil}, http.StatusForbidden)
|
||||||
|
|
||||||
|
defer objectReadFault(z, bucket, object, 10)()
|
||||||
|
if _, err := z.GetObjectInfo(ctx, bucket, object, ObjectOptions{}); !isErrReadQuorum(err) {
|
||||||
|
t.Fatalf("GetObjectInfo with 10 of 16 drives failing: %v, want InsufficientReadQuorum", err)
|
||||||
|
}
|
||||||
|
for _, req := range []request{get, head, putTagging} {
|
||||||
|
for _, u := range users {
|
||||||
|
check("read quorum lost", u.creds, u.name, req, u.quorumLost)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
check("read quorum lost", anonymous, "anonymous", get, http.StatusServiceUnavailable)
|
||||||
|
check("read quorum lost", anonymous, "anonymous", head, http.StatusServiceUnavailable)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExistingObjectTagsFor(t *testing.T) {
|
||||||
|
values := map[string][]string{"username": {"alice"}, "sub": {"u1"}}
|
||||||
|
for _, tc := range []struct {
|
||||||
|
effect policy.Effect
|
||||||
|
conditions string
|
||||||
|
want string // the tags to set, encoded as in x-amz-tagging
|
||||||
|
ok bool // false: no tags for this statement
|
||||||
|
}{
|
||||||
|
{policy.Allow, `{"StringEquals": {"s3:ExistingObjectTag/k": "v"}}`, "k=v", true},
|
||||||
|
{policy.Allow, `{"StringEqualsIgnoreCase": {"s3:ExistingObjectTag/k": "V"}, "IpAddress": {"aws:SourceIp": "10.0.0.0/8"}}`, "k=V", true},
|
||||||
|
{policy.Allow, `{"ForAnyValue:StringEquals": {"s3:ExistingObjectTag/k": ["b", "a"]}}`, "k=a", true},
|
||||||
|
{policy.Allow, `{"StringLike": {"s3:ExistingObjectTag/k": "a?c*"}}`, "k=a_c", true},
|
||||||
|
{policy.Allow, `{"StringEquals": {"s3:ExistingObjectTag/b": "2", "s3:ExistingObjectTag/a": "1"}}`, "a=1&b=2", true},
|
||||||
|
// Passed without the tag.
|
||||||
|
{policy.Allow, `{"StringNotEquals": {"s3:ExistingObjectTag/k": "v"}}`, "", true},
|
||||||
|
{policy.Allow, `{"Null": {"s3:ExistingObjectTag/k": "true"}}`, "", true},
|
||||||
|
// Null "false" asks for the tag, with any value.
|
||||||
|
{policy.Allow, `{"Null": {"s3:ExistingObjectTag/k": "false"}}`, "k=", true},
|
||||||
|
{policy.Allow, `{"Null": {"s3:ExistingObjectTag/k": "false"}, "StringNotEquals": {"s3:ExistingObjectTag/k": ["", "w"]}}`, "k=_", true},
|
||||||
|
{policy.Allow, `{"Null": {"s3:ExistingObjectTag/k": "false"}, "StringNotLike": {"s3:ExistingObjectTag/k": "*w"}}`, "k=", true},
|
||||||
|
{policy.Allow, `{"ForAllValues:StringNotEquals": {"s3:ExistingObjectTag/k": "w"}}`, "k=", true},
|
||||||
|
{policy.Allow, `{"Null": {"s3:ExistingObjectTag/k": "true"}, "StringEquals": {"s3:ExistingObjectTag/k": "v"}}`, "", false},
|
||||||
|
// Policy variables take the values of the request.
|
||||||
|
{policy.Allow, `{"StringEquals": {"s3:ExistingObjectTag/owner": "${aws:username}"}}`, "owner=alice", true},
|
||||||
|
{policy.Allow, `{"StringLike": {"s3:ExistingObjectTag/owner": "${jwt:sub}-*"}}`, "owner=u1-", true},
|
||||||
|
{policy.Allow, `{"StringEquals": {"s3:ExistingObjectTag/owner": "${ldap:user}"}}`, "", false},
|
||||||
|
// A Deny is kept from matching.
|
||||||
|
{policy.Deny, `{"StringNotEquals": {"s3:ExistingObjectTag/k": "v"}}`, "k=v", true},
|
||||||
|
{policy.Deny, `{"Null": {"s3:ExistingObjectTag/k": "true"}}`, "k=", true},
|
||||||
|
{policy.Deny, `{"StringEquals": {"s3:ExistingObjectTag/k": "v"}}`, "", true},
|
||||||
|
{policy.Deny, `{"StringEquals": {"s3:prefix": "v"}}`, "", false},
|
||||||
|
// No tag tests.
|
||||||
|
{policy.Allow, `{"StringEquals": {"s3:prefix": "v"}}`, "", false},
|
||||||
|
{policy.Allow, "", "", false},
|
||||||
|
} {
|
||||||
|
var conditions condition.Functions // nil: a statement without a Condition
|
||||||
|
if tc.conditions != "" {
|
||||||
|
if err := json.Unmarshal([]byte(tc.conditions), &conditions); err != nil {
|
||||||
|
t.Fatalf("%s: %v", tc.conditions, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
tagMap, ok := existingObjectTagsFor(tc.effect, conditions, values)
|
||||||
|
got := ""
|
||||||
|
if len(tagMap) > 0 {
|
||||||
|
encoded, err := tags.MapToObjectTags(tagMap)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("%s: %v", tc.conditions, err)
|
||||||
|
}
|
||||||
|
got = encoded.String()
|
||||||
|
}
|
||||||
|
if got != tc.want || ok != tc.ok {
|
||||||
|
t.Errorf("%s %s: got %q %v, want %q %v", tc.effect, tc.conditions, got, ok, tc.want, tc.ok)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -425,6 +425,19 @@ func authorizeRequest(ctx context.Context, r *http.Request, action policy.Action
|
|||||||
return authorizeRequestWithExistingTags(ctx, r, action, "")
|
return authorizeRequestWithExistingTags(ctx, r, action, "")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// authorizeBucketRequest authorizes a bucket action, such as s3:ListBucket,
|
||||||
|
// on the bucket of a request made for one of its objects.
|
||||||
|
func authorizeBucketRequest(ctx context.Context, r *http.Request, action policy.Action) APIErrorCode {
|
||||||
|
reqInfo := logger.GetReqInfo(ctx)
|
||||||
|
if reqInfo == nil {
|
||||||
|
return ErrAccessDenied
|
||||||
|
}
|
||||||
|
objectName := reqInfo.ObjectName
|
||||||
|
reqInfo.ObjectName = ""
|
||||||
|
defer func() { reqInfo.ObjectName = objectName }()
|
||||||
|
return authorizeRequest(ctx, r, action)
|
||||||
|
}
|
||||||
|
|
||||||
func deleteObjectAction(versionID string) policy.Action {
|
func deleteObjectAction(versionID string) policy.Action {
|
||||||
if versionID != "" {
|
if versionID != "" {
|
||||||
return policy.DeleteObjectVersionAction
|
return policy.DeleteObjectVersionAction
|
||||||
|
|||||||
@@ -330,7 +330,7 @@ func TestBatchReviewRotationKeepsMigrationCleanup(t *testing.T) {
|
|||||||
t.Errorf("rotation changed a draining source after its version was copied (rotation error: %v)", rotationErr)
|
t.Errorf("rotation changed a draining source after its version was copied (rotation error: %v)", rotationErr)
|
||||||
}
|
}
|
||||||
snapshot := []FileInfo{{VersionID: before.VersionID, ModTime: beforeInfo.ModTime}}
|
snapshot := []FileInfo{{VersionID: before.VersionID, ModTime: beforeInfo.ModTime}}
|
||||||
if err := z.removeMovedVersions(t.Context(), bucket, object, 0, snapshot, nil); err != nil {
|
if _, err := z.removeMovedVersions(t.Context(), bucket, object, 0, snapshot, nil); err != nil {
|
||||||
t.Fatalf("rotation prevented migration cleanup: %v", err)
|
t.Fatalf("rotation prevented migration cleanup: %v", err)
|
||||||
}
|
}
|
||||||
if _, err := z.serverPools[0].GetObjectInfo(t.Context(), bucket, object, ObjectOptions{VersionID: before.VersionID}); !isErrObjectNotFound(err) && !isErrVersionNotFound(err) {
|
if _, err := z.serverPools[0].GetObjectInfo(t.Context(), bucket, object, ObjectOptions{VersionID: before.VersionID}); !isErrObjectNotFound(err) && !isErrVersionNotFound(err) {
|
||||||
|
|||||||
@@ -0,0 +1,156 @@
|
|||||||
|
// Copyright (c) 2026 PGSTY
|
||||||
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
|
||||||
|
package cmd
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"encoding/xml"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"slices"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/minio/minio/internal/auth"
|
||||||
|
)
|
||||||
|
|
||||||
|
// A bulk delete joins each name to the request bucket on the drives, so a name
|
||||||
|
// with a ".." segment names a path in another bucket or in the meta bucket.
|
||||||
|
// Such a name is the error of its own entry and reaches no drive, and the
|
||||||
|
// other entries of the request are deleted as usual. This holds for the one
|
||||||
|
// bulk lock of a single node, for a pool that locks per set, and for more than
|
||||||
|
// one pool.
|
||||||
|
func TestPoolsBulkDeleteInvalidNames(t *testing.T) {
|
||||||
|
for _, topology := range []struct {
|
||||||
|
name string
|
||||||
|
new func(t *testing.T) (*erasureServerPools, string)
|
||||||
|
}{
|
||||||
|
{"single node", func(t *testing.T) (*erasureServerPools, string) { return bulkDeletePools(t, 1) }},
|
||||||
|
{"set locks", func(t *testing.T) (*erasureServerPools, string) { return ldPool(t, ldLocal) }},
|
||||||
|
{"two pools", func(t *testing.T) (*erasureServerPools, string) { return bulkDeletePools(t, 2) }},
|
||||||
|
} {
|
||||||
|
t.Run(topology.name, func(t *testing.T) {
|
||||||
|
z, bucket := topology.new(t)
|
||||||
|
ctx := t.Context()
|
||||||
|
other := "bulk-delete-other"
|
||||||
|
if err := z.MakeBucket(ctx, other, MakeBucketOptions{VersioningEnabled: true}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
old := batchPutVersion(t, z, other, "kept", nil)
|
||||||
|
latest := batchPutVersion(t, z, other, "kept", nil)
|
||||||
|
batchPutVersion(t, z, other, "visible", nil)
|
||||||
|
metaKey := pathJoin(bucketMetaPrefix, other, bucketMetadataFile)
|
||||||
|
if !batchVersionExists(t, z, minioMetaBucket, metaKey, "") {
|
||||||
|
t.Fatal("the bucket metadata is not an object of the meta bucket")
|
||||||
|
}
|
||||||
|
for _, key := range []string{"own", "dir/"} {
|
||||||
|
if _, err := z.PutObject(ctx, bucket, key, mustGetPutObjReader(t, bytes.NewReader(nil), 0, "", ""), ObjectOptions{}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
names := []ObjectToDelete{
|
||||||
|
{ObjectV: ObjectV{ObjectName: "../" + other + "/kept", VersionID: old.VersionID}}, // purge in another bucket
|
||||||
|
{ObjectV: ObjectV{ObjectName: "own"}},
|
||||||
|
{ObjectV: ObjectV{ObjectName: "a/../../" + other + "/visible"}}, // marker in another bucket
|
||||||
|
{ObjectV: ObjectV{ObjectName: "../" + minioMetaBucket + "/" + metaKey}}, // marker on bucket metadata
|
||||||
|
{ObjectV: ObjectV{ObjectName: "dir/"}},
|
||||||
|
{ObjectV: ObjectV{ObjectName: ""}},
|
||||||
|
}
|
||||||
|
invalid := map[int]bool{0: true, 2: true, 3: true, 5: true}
|
||||||
|
// Without a version ID these options ask for a delete marker, the
|
||||||
|
// one write a bulk delete makes.
|
||||||
|
deleted, errs := z.DeleteObjects(ctx, bucket, slices.Clone(names), ObjectOptions{Versioned: true})
|
||||||
|
for i, name := range names {
|
||||||
|
_, rejected := errs[i].(ObjectNameInvalid)
|
||||||
|
if rejected != invalid[i] || !invalid[i] && errs[i] != nil {
|
||||||
|
t.Errorf("%d %q: error = %v, rejected as invalid = %v, want %v", i, name.ObjectName, errs[i], rejected, invalid[i])
|
||||||
|
}
|
||||||
|
if deleted[i].ObjectName != name.ObjectName || deleted[i].found == invalid[i] {
|
||||||
|
t.Errorf("%d %q: result = %+v", i, name.ObjectName, deleted[i])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, version := range []string{old.VersionID, latest.VersionID} {
|
||||||
|
if !batchVersionExists(t, z, other, "kept", version) {
|
||||||
|
t.Errorf("version %q of the object in the other bucket was removed", version)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !batchVersionExists(t, z, other, "visible", "") {
|
||||||
|
t.Error("the object of the other bucket is hidden by a marker")
|
||||||
|
}
|
||||||
|
if !batchVersionExists(t, z, minioMetaBucket, metaKey, "") {
|
||||||
|
t.Error("the metadata of the other bucket is hidden by a marker")
|
||||||
|
}
|
||||||
|
for _, key := range []string{"own", "dir/"} {
|
||||||
|
if batchVersionExists(t, z, bucket, key, "") {
|
||||||
|
t.Errorf("%q of the request bucket was not deleted", key)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The names of an S3 DeleteObjects request come from its body, which the path
|
||||||
|
// checks of the request URL do not see. A user allowed to delete in one bucket
|
||||||
|
// reaches nothing outside it through such a name: a version under COMPLIANCE
|
||||||
|
// retention in another bucket stays, and no delete marker lands on that object
|
||||||
|
// or on the stored metadata of its bucket.
|
||||||
|
func TestBulkDeleteObjectLockNameOutsideBucket(t *testing.T) {
|
||||||
|
defer DetectTestLeak(t)()
|
||||||
|
ExecObjectLayerAPITest(ExecObjectLayerAPITestArgs{
|
||||||
|
t: t, endpoints: []string{"DeleteMultipleObjects"},
|
||||||
|
makeBucketOptions: MakeBucketOptions{VersioningEnabled: true},
|
||||||
|
objAPITest: func(obj ObjectLayer, instance, bucket string, router http.Handler, _ auth.Credentials, t *testing.T) {
|
||||||
|
ctx := t.Context()
|
||||||
|
user := newObjectAttributesAuthzUser(t, instance, bucket, `"s3:DeleteObject","s3:DeleteObjectVersion"`)
|
||||||
|
locked := getRandomBucketName()
|
||||||
|
if err := obj.MakeBucket(ctx, locked, MakeBucketOptions{VersioningEnabled: true, LockEnabled: true}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
enableBucketObjectLock(t, locked)
|
||||||
|
worm := batchPutVersion(t, obj, locked, "secret", batchLockMeta("COMPLIANCE"))
|
||||||
|
batchPutVersion(t, obj, bucket, "own", nil)
|
||||||
|
metaKey := pathJoin(bucketMetaPrefix, locked, bucketMetadataFile)
|
||||||
|
|
||||||
|
outside := []ObjectToDelete{
|
||||||
|
{ObjectV: ObjectV{ObjectName: "../" + locked + "/secret", VersionID: worm.VersionID}},
|
||||||
|
{ObjectV: ObjectV{ObjectName: "../" + locked + "/secret"}},
|
||||||
|
{ObjectV: ObjectV{ObjectName: "../" + minioMetaBucket + "/" + metaKey}},
|
||||||
|
}
|
||||||
|
body := encodeResponse(DeleteObjectsRequest{Objects: append(slices.Clone(outside), ObjectToDelete{ObjectV: ObjectV{ObjectName: "own"}})})
|
||||||
|
req, err := newTestSignedRequestV4(http.MethodPost, getDeleteMultipleObjectsURL("", bucket), int64(len(body)), bytes.NewReader(body),
|
||||||
|
user.AccessKey, user.SecretKey, nil)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
router.ServeHTTP(rec, req)
|
||||||
|
var resp DeleteObjectsResponse
|
||||||
|
if rec.Code != http.StatusOK || xml.Unmarshal(rec.Body.Bytes(), &resp) != nil {
|
||||||
|
t.Fatalf("%s: status %d: %s", instance, rec.Code, rec.Body.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(resp.DeletedObjects) != 1 || resp.DeletedObjects[0].ObjectName != "own" || len(resp.Errors) != len(outside) {
|
||||||
|
t.Errorf("%s: want the three names outside the bucket refused and \"own\" deleted: %s", instance, rec.Body.String())
|
||||||
|
}
|
||||||
|
for _, e := range resp.Errors {
|
||||||
|
if e.Code != errorCodes[ErrInvalidObjectName].Code {
|
||||||
|
t.Errorf("%s: %q: code %s, want %s", instance, e.Key, e.Code, errorCodes[ErrInvalidObjectName].Code)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !batchVersionExists(t, obj, locked, "secret", worm.VersionID) {
|
||||||
|
t.Errorf("%s: the version under COMPLIANCE retention in the other bucket was removed", instance)
|
||||||
|
}
|
||||||
|
if !batchVersionExists(t, obj, locked, "secret", "") {
|
||||||
|
t.Errorf("%s: a delete marker hides the object of the other bucket", instance)
|
||||||
|
}
|
||||||
|
if !batchVersionExists(t, obj, minioMetaBucket, metaKey, "") {
|
||||||
|
t.Errorf("%s: a delete marker hides the stored metadata of the other bucket", instance)
|
||||||
|
}
|
||||||
|
if batchVersionExists(t, obj, bucket, "own", "") {
|
||||||
|
t.Errorf("%s: the object of the request bucket was not deleted", instance)
|
||||||
|
}
|
||||||
|
},
|
||||||
|
})
|
||||||
|
}
|
||||||
@@ -271,16 +271,17 @@ func (z *erasureServerPools) applyMigrationExpiry(ctx context.Context, bucket, o
|
|||||||
// in the source until the ordinary expiry path has deleted them: that path
|
// in the source until the ordinary expiry path has deleted them: that path
|
||||||
// owns replication state and tier cleanup, and it re-evaluates the current
|
// owns replication state and tier cleanup, and it re-evaluates the current
|
||||||
// lifecycle rule. A migration cleanup must never turn an asynchronous ILM
|
// lifecycle rule. A migration cleanup must never turn an asynchronous ILM
|
||||||
// decision into an unconditional physical purge.
|
// decision into an unconditional physical purge. It returns how many source
|
||||||
func (z *erasureServerPools) removeMovedVersions(ctx context.Context, bucket, object string, source int, versions []FileInfo, expired []migrationExpiredVersion) error {
|
// versions it deleted.
|
||||||
|
func (z *erasureServerPools) removeMovedVersions(ctx context.Context, bucket, object string, source int, versions []FileInfo, expired []migrationExpiredVersion) (removed int, err error) {
|
||||||
if err := z.applyMigrationExpiry(ctx, bucket, object, source, expired); err != nil {
|
if err := z.applyMigrationExpiry(ctx, bucket, object, source, expired); err != nil {
|
||||||
return err
|
return 0, err
|
||||||
}
|
}
|
||||||
object = encodeDirObject(object)
|
object = encodeDirObject(object)
|
||||||
lk := z.NewNSLock(bucket, object)
|
lk := z.NewNSLock(bucket, object)
|
||||||
lkctx, err := lk.GetLock(ctx, globalOperationTimeout)
|
lkctx, err := lk.GetLock(ctx, globalOperationTimeout)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return 0, err
|
||||||
}
|
}
|
||||||
ctx = lkctx.Context()
|
ctx = lkctx.Context()
|
||||||
defer lk.Unlock(lkctx)
|
defer lk.Unlock(lkctx)
|
||||||
@@ -294,7 +295,7 @@ func (z *erasureServerPools) removeMovedVersions(ctx context.Context, bucket, ob
|
|||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return removed, err
|
||||||
}
|
}
|
||||||
var src *ObjectInfo
|
var src *ObjectInfo
|
||||||
var retained []PoolObjInfo
|
var retained []PoolObjInfo
|
||||||
@@ -309,7 +310,7 @@ func (z *erasureServerPools) removeMovedVersions(ctx context.Context, bucket, ob
|
|||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
if !src.ModTime.Equal(version.ModTime) || src.DeleteMarker != version.Deleted {
|
if !src.ModTime.Equal(version.ModTime) || src.DeleteMarker != version.Deleted {
|
||||||
return PreConditionFailed{}
|
return removed, PreConditionFailed{}
|
||||||
}
|
}
|
||||||
if isExpiredVersion(expired, version) {
|
if isExpiredVersion(expired, version) {
|
||||||
// The logical expiry path owns this version. Keeping the source is
|
// The logical expiry path owns this version. Keeping the source is
|
||||||
@@ -319,17 +320,20 @@ func (z *erasureServerPools) removeMovedVersions(ctx context.Context, bucket, ob
|
|||||||
// Successful movement is not enough if the destination changed again.
|
// Successful movement is not enough if the destination changed again.
|
||||||
// Keep the source unless another copy still preserves its read result.
|
// Keep the source unless another copy still preserves its read result.
|
||||||
if len(retained) == 0 || retained[0].ObjInfo.ModTime.Before(src.ModTime) {
|
if len(retained) == 0 || retained[0].ObjInfo.ModTime.Before(src.ModTime) {
|
||||||
return PreConditionFailed{}
|
return removed, PreConditionFailed{}
|
||||||
}
|
}
|
||||||
_, err = z.serverPools[source].DeleteObject(ctx, bucket, object, ObjectOptions{
|
_, err = z.serverPools[source].DeleteObject(ctx, bucket, object, ObjectOptions{
|
||||||
VersionID: vid, NoLock: true, NoAuditLog: true,
|
VersionID: vid, NoLock: true, NoAuditLog: true,
|
||||||
SkipFreeVersion: true, // preserve the old prefix cleanup's tier-GC behavior
|
SkipFreeVersion: true, // preserve the old prefix cleanup's tier-GC behavior
|
||||||
})
|
})
|
||||||
if err != nil && !isErrObjectNotFound(err) && !isErrVersionNotFound(err) {
|
if err != nil && !isErrObjectNotFound(err) && !isErrVersionNotFound(err) {
|
||||||
return err
|
return removed, err
|
||||||
|
}
|
||||||
|
if err == nil {
|
||||||
|
removed++
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return nil
|
return removed, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// ilmDeletesVersionOnly reports whether the ILM action queued for a version
|
// ilmDeletesVersionOnly reports whether the ILM action queued for a version
|
||||||
|
|||||||
@@ -0,0 +1,66 @@
|
|||||||
|
// Copyright (c) 2026 Denis Babchuk
|
||||||
|
//
|
||||||
|
// This file is part of MinIO Object Storage stack
|
||||||
|
//
|
||||||
|
// This program is free software: you can redistribute it and/or modify
|
||||||
|
// it under the terms of the GNU Affero General Public License as published by
|
||||||
|
// the Free Software Foundation, either version 3 of the License, or
|
||||||
|
// (at your option) any later version.
|
||||||
|
//
|
||||||
|
// This program is distributed in the hope that it will be useful
|
||||||
|
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||||
|
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||||
|
// GNU Affero General Public License for more details.
|
||||||
|
//
|
||||||
|
// You should have received a copy of the GNU Affero General Public License
|
||||||
|
// along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||||
|
|
||||||
|
package cmd
|
||||||
|
|
||||||
|
import "testing"
|
||||||
|
|
||||||
|
// A key below another key ("k/b" under "k") is a directory inside the
|
||||||
|
// directory of that key. The listing treats the directory of an object as a
|
||||||
|
// leaf, so a decommission walk does not see "k/b" while "k" is still in the
|
||||||
|
// pool. The walk has to be repeated once the keys above are gone, or the
|
||||||
|
// check after the bucket finds them and the decommission fails.
|
||||||
|
|
||||||
|
func keysBelowDecommission(t *testing.T, z *erasureServerPools) PoolDecommissionInfo {
|
||||||
|
t.Helper()
|
||||||
|
if err := z.StartDecommission(t.Context(), 0); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
z.doDecommissionInRoutine(t.Context(), 0)
|
||||||
|
z.poolMetaMutex.RLock()
|
||||||
|
defer z.poolMetaMutex.RUnlock()
|
||||||
|
return *z.poolMeta.Pools[0].Decommission
|
||||||
|
}
|
||||||
|
|
||||||
|
func keysBelowPools(t *testing.T) (*erasureServerPools, string, []string) {
|
||||||
|
t.Helper()
|
||||||
|
z, _ := migrationTestPools(t)
|
||||||
|
const bucket = "keys-below"
|
||||||
|
if err := z.MakeBucket(t.Context(), bucket, MakeBucketOptions{VersioningEnabled: true}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
keys := []string{"k", "k/b", "k/b/c"}
|
||||||
|
for _, key := range keys {
|
||||||
|
putConsistencyObject(t, z, bucket, key, 0, key, ObjectOptions{Versioned: true})
|
||||||
|
}
|
||||||
|
return z, bucket, keys
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDecommissionMovesKeysBelowAKey(t *testing.T) {
|
||||||
|
z, bucket, keys := keysBelowPools(t)
|
||||||
|
if info := keysBelowDecommission(t, z); !info.Complete || info.Failed {
|
||||||
|
t.Errorf("decommission: complete %v, failed %v; want complete", info.Complete, info.Failed)
|
||||||
|
}
|
||||||
|
for _, key := range keys {
|
||||||
|
if _, err := z.serverPools[0].GetObjectInfo(t.Context(), bucket, key, ObjectOptions{}); !isErrObjectNotFound(err) {
|
||||||
|
t.Errorf("%s still in pool 0: %v", key, err)
|
||||||
|
}
|
||||||
|
if body := migrationTestBody(t, z.GetObjectNInfo, bucket, key, ""); body != key {
|
||||||
|
t.Errorf("%s reads %q", key, body)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -28,6 +28,7 @@ import (
|
|||||||
"slices"
|
"slices"
|
||||||
"sort"
|
"sort"
|
||||||
"strings"
|
"strings"
|
||||||
|
"sync/atomic"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/dustin/go-humanize"
|
"github.com/dustin/go-humanize"
|
||||||
@@ -752,7 +753,25 @@ func (set *erasureObjects) listObjectsToDecommission(ctx context.Context, bi dec
|
|||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// decommissionPool moves bucket bi out of pool idx. A key below another key
|
||||||
|
// ("k/b" under "k") is a directory inside the directory of that key, and the
|
||||||
|
// listing does not descend into the directory of an object: a walk sees "k/b"
|
||||||
|
// only once "k" has left the pool. The walk is repeated while it removes
|
||||||
|
// source versions, so each pass moves the keys that the previous one
|
||||||
|
// uncovered. Each repeat needs a version removed from the pool, which takes
|
||||||
|
// no new writes, so the repeats end.
|
||||||
func (z *erasureServerPools) decommissionPool(ctx context.Context, idx int, pool *erasureSets, bi decomBucketInfo) error {
|
func (z *erasureServerPools) decommissionPool(ctx context.Context, idx int, pool *erasureSets, bi decomBucketInfo) error {
|
||||||
|
for {
|
||||||
|
removed, err := z.decommissionPoolWalk(ctx, idx, pool, bi)
|
||||||
|
if err != nil || removed == 0 || contextCanceled(ctx) {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// decommissionPoolWalk walks bucket bi of pool idx once and returns the number
|
||||||
|
// of keys whose source versions it removed.
|
||||||
|
func (z *erasureServerPools) decommissionPoolWalk(ctx context.Context, idx int, pool *erasureSets, bi decomBucketInfo) (int64, error) {
|
||||||
ctx = logger.SetReqInfo(ctx, &logger.ReqInfo{})
|
ctx = logger.SetReqInfo(ctx, &logger.ReqInfo{})
|
||||||
|
|
||||||
const envDecomWorkers = "_MINIO_DECOMMISSION_WORKERS"
|
const envDecomWorkers = "_MINIO_DECOMMISSION_WORKERS"
|
||||||
@@ -766,9 +785,11 @@ func (z *erasureServerPools) decommissionPool(ctx context.Context, idx int, pool
|
|||||||
// add that many extra workers.
|
// add that many extra workers.
|
||||||
workerSize += len(pool.sets)
|
workerSize += len(pool.sets)
|
||||||
|
|
||||||
|
var removed atomic.Int64
|
||||||
|
|
||||||
wk, err := workers.New(workerSize)
|
wk, err := workers.New(workerSize)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return 0, err
|
||||||
}
|
}
|
||||||
|
|
||||||
var vc *versioning.Versioning
|
var vc *versioning.Versioning
|
||||||
@@ -779,24 +800,24 @@ func (z *erasureServerPools) decommissionPool(ctx context.Context, idx int, pool
|
|||||||
if bi.Name != minioMetaBucket {
|
if bi.Name != minioMetaBucket {
|
||||||
vc, err = globalBucketVersioningSys.Get(bi.Name)
|
vc, err = globalBucketVersioningSys.Get(bi.Name)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return 0, err
|
||||||
}
|
}
|
||||||
|
|
||||||
// Check if the current bucket has a configured lifecycle policy
|
// Check if the current bucket has a configured lifecycle policy
|
||||||
lc, lifecycleUpdatedAt, err = globalBucketMetadataSys.GetLifecycleConfig(bi.Name)
|
lc, lifecycleUpdatedAt, err = globalBucketMetadataSys.GetLifecycleConfig(bi.Name)
|
||||||
if err != nil && !errors.Is(err, BucketLifecycleNotFound{Bucket: bi.Name}) {
|
if err != nil && !errors.Is(err, BucketLifecycleNotFound{Bucket: bi.Name}) {
|
||||||
return err
|
return 0, err
|
||||||
}
|
}
|
||||||
|
|
||||||
// Check if bucket is object locked.
|
// Check if bucket is object locked.
|
||||||
lr, err = globalBucketObjectLockSys.Get(bi.Name)
|
lr, err = globalBucketObjectLockSys.Get(bi.Name)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return 0, err
|
||||||
}
|
}
|
||||||
|
|
||||||
rcfg, err = getReplicationConfig(ctx, bi.Name)
|
rcfg, err = getReplicationConfig(ctx, bi.Name)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return 0, err
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1019,7 +1040,10 @@ func (z *erasureServerPools) decommissionPool(ctx context.Context, idx int, pool
|
|||||||
// if all versions were decommissioned, then we can delete the object versions.
|
// if all versions were decommissioned, then we can delete the object versions.
|
||||||
if decommissioned == len(fivs.Versions) {
|
if decommissioned == len(fivs.Versions) {
|
||||||
stopFn := globalDecommissionMetrics.log(decomMetricDecommissionRemoveObject, idx, bi.Name, entry.name)
|
stopFn := globalDecommissionMetrics.log(decomMetricDecommissionRemoveObject, idx, bi.Name, entry.name)
|
||||||
err := z.removeMovedVersions(ctx, bi.Name, entry.name, idx, fivs.Versions, expired)
|
n, err := z.removeMovedVersions(ctx, bi.Name, entry.name, idx, fivs.Versions, expired)
|
||||||
|
if n > 0 {
|
||||||
|
removed.Add(1)
|
||||||
|
}
|
||||||
if err == nil {
|
if err == nil {
|
||||||
err = z.removeFreeVersions(ctx, bi.Name, entry.name, idx, fivs.FreeVersions)
|
err = z.removeFreeVersions(ctx, bi.Name, entry.name, idx, fivs.FreeVersions)
|
||||||
}
|
}
|
||||||
@@ -1062,7 +1086,7 @@ func (z *erasureServerPools) decommissionPool(ctx context.Context, idx int, pool
|
|||||||
}(setIdx)
|
}(setIdx)
|
||||||
}
|
}
|
||||||
wk.Wait()
|
wk.Wait()
|
||||||
return nil
|
return removed.Load(), nil
|
||||||
}
|
}
|
||||||
|
|
||||||
//msgp:ignore decomMetrics
|
//msgp:ignore decomMetrics
|
||||||
|
|||||||
@@ -1666,7 +1666,7 @@ func TestPoolsMigrationLifecycleRefusedExpiryKeepsSource(t *testing.T) {
|
|||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
}
|
}
|
||||||
version := FileInfo{Name: key, VersionID: id, ModTime: source.ModTime}
|
version := FileInfo{Name: key, VersionID: id, ModTime: source.ModTime}
|
||||||
err = s.z.removeMovedVersions(ctx, s.bucket, key, 0, []FileInfo{version}, []migrationExpiredVersion{{
|
_, err = s.z.removeMovedVersions(ctx, s.bucket, key, 0, []FileInfo{version}, []migrationExpiredVersion{{
|
||||||
version: version,
|
version: version,
|
||||||
event: lifecycle.Event{Action: lifecycle.DeleteAction},
|
event: lifecycle.Event{Action: lifecycle.DeleteAction},
|
||||||
lifecycleUpdatedAt: updatedAt,
|
lifecycleUpdatedAt: updatedAt,
|
||||||
|
|||||||
@@ -132,7 +132,7 @@ func TestPoolsMigrationCleanupRechecksSource(t *testing.T) {
|
|||||||
_, err := z.serverPools[1].DeleteObject(t.Context(), bucket, oi.Name, ObjectOptions{VersionID: nullVersionID})
|
_, err := z.serverPools[1].DeleteObject(t.Context(), bucket, oi.Name, ObjectOptions{VersionID: nullVersionID})
|
||||||
mustIAM(t, err)
|
mustIAM(t, err)
|
||||||
}
|
}
|
||||||
err := z.removeMovedVersions(t.Context(), bucket, oi.Name, 0, snapshot, nil)
|
_, err := z.removeMovedVersions(t.Context(), bucket, oi.Name, 0, snapshot, nil)
|
||||||
if change != "delete-uuid" && err == nil {
|
if change != "delete-uuid" && err == nil {
|
||||||
t.Fatal("cleanup accepted changed or uncertain state")
|
t.Fatal("cleanup accepted changed or uncertain state")
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -779,7 +779,7 @@ func (z *erasureServerPools) rebalanceBucket(ctx context.Context, bucket string,
|
|||||||
// if all versions were rebalanced, we can delete the object versions.
|
// if all versions were rebalanced, we can delete the object versions.
|
||||||
if rebalanced == len(fivs.Versions) {
|
if rebalanced == len(fivs.Versions) {
|
||||||
stopFn := globalRebalanceMetrics.log(rebalanceMetricRebalanceRemoveObject, poolIdx, bucket, entry.name)
|
stopFn := globalRebalanceMetrics.log(rebalanceMetricRebalanceRemoveObject, poolIdx, bucket, entry.name)
|
||||||
err := z.removeMovedVersions(ctx, bucket, entry.name, poolIdx, fivs.Versions, expired)
|
_, err := z.removeMovedVersions(ctx, bucket, entry.name, poolIdx, fivs.Versions, expired)
|
||||||
stopFn(0, err)
|
stopFn(0, err)
|
||||||
auditLogRebalance(ctx, "Rebalance:DeleteObject", bucket, entry.name, "", err)
|
auditLogRebalance(ctx, "Rebalance:DeleteObject", bucket, entry.name, "", err)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
@@ -26,6 +26,7 @@ import (
|
|||||||
"math/rand"
|
"math/rand"
|
||||||
"net/http"
|
"net/http"
|
||||||
"path"
|
"path"
|
||||||
|
"slices"
|
||||||
"sort"
|
"sort"
|
||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
@@ -1543,6 +1544,27 @@ func (z *erasureServerPools) DeleteObjects(ctx context.Context, bucket string, o
|
|||||||
objSets.Add(objects[i].ObjectName)
|
objSets.Add(objects[i].ObjectName)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// A name that fails the check keeps its error and goes no further. Joined
|
||||||
|
// to the bucket it can name a path in another bucket or in the meta
|
||||||
|
// bucket, so it is not locked, looked up or sent to a drive. The other
|
||||||
|
// names are deleted as a request of their own: everything below sees only
|
||||||
|
// names that passed.
|
||||||
|
if slices.ContainsFunc(derrs, func(err error) bool { return err != nil }) {
|
||||||
|
valid, at := make([]ObjectToDelete, 0, len(objects)), make([]int, 0, len(objects))
|
||||||
|
for i, obj := range objects {
|
||||||
|
if derrs[i] == nil {
|
||||||
|
valid, at = append(valid, obj), append(at, i)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(valid) > 0 {
|
||||||
|
deleted, errs := z.DeleteObjects(ctx, bucket, valid, opts)
|
||||||
|
for j, i := range at {
|
||||||
|
dobjects[i], derrs[i] = deleted[j], errs[j]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return dobjects, derrs
|
||||||
|
}
|
||||||
|
|
||||||
// A single pool locks each name in the set its writers lock it in, one
|
// A single pool locks each name in the set its writers lock it in, one
|
||||||
// set at a time. A single node keeps one lock table for all sets, so it
|
// set at a time. A single node keeps one lock table for all sets, so it
|
||||||
// keeps the one bulk lock below: its local lock takes the names one by
|
// keeps the one bulk lock below: its local lock takes the names one by
|
||||||
|
|||||||
+6
-5
@@ -2930,11 +2930,12 @@ func updatedServiceAccountCredentials(ctx context.Context, accessKey string, cr
|
|||||||
// at this point.
|
// at this point.
|
||||||
m.Delete(sessionPolicyNameExtracted)
|
m.Delete(sessionPolicyNameExtracted)
|
||||||
|
|
||||||
nosp := opts.sessionPolicy == nil || opts.sessionPolicy.Version == "" && len(opts.sessionPolicy.Statements) == 0
|
// A nil sessionPolicy is an edit that does not concern the policy, and
|
||||||
|
// the embedded one stays. Only a policy that was given, and is empty,
|
||||||
// sessionPolicy is nil and there is embedded policy attached we remove
|
// removes it: the account then has all its parent is allowed, which is
|
||||||
// embedded policy at that point.
|
// usually more.
|
||||||
if _, ok := m.Lookup(policy.SessionPolicyName); ok && nosp {
|
reset := opts.sessionPolicy != nil && opts.sessionPolicy.Version == "" && len(opts.sessionPolicy.Statements) == 0
|
||||||
|
if _, ok := m.Lookup(policy.SessionPolicyName); ok && reset {
|
||||||
m.Delete(policy.SessionPolicyName)
|
m.Delete(policy.SessionPolicyName)
|
||||||
m.Set(iamPolicyClaimNameSA(), inheritedPolicyType)
|
m.Set(iamPolicyClaimNameSA(), inheritedPolicyType)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,227 @@
|
|||||||
|
// Copyright (c) 2026 PGSTY
|
||||||
|
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||||
|
|
||||||
|
package cmd
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"runtime"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/minio/madmin-go/v3"
|
||||||
|
minio "github.com/minio/minio-go/v7"
|
||||||
|
"github.com/minio/minio/internal/auth"
|
||||||
|
"github.com/pgsty/silo-pkg/v3/policy"
|
||||||
|
)
|
||||||
|
|
||||||
|
// An edit of a service account that names no policy keeps the embedded
|
||||||
|
// policy: the account must not come out of a status change, a key rotation
|
||||||
|
// or a rename with all its parent is allowed. Only a policy that is given
|
||||||
|
// and empty removes the embedded one.
|
||||||
|
|
||||||
|
const svcAccEditPolicy = `{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":["s3:GetObject"],"Resource":["arn:aws:s3:::edit/*"]}]}`
|
||||||
|
|
||||||
|
func TestIAMServiceAccountEditKeepsPolicy(t *testing.T) {
|
||||||
|
f := newGroupRefreshFixture(t)
|
||||||
|
want, err := policy.ParseConfig(strings.NewReader(svcAccEditPolicy))
|
||||||
|
mustIAM(t, err)
|
||||||
|
|
||||||
|
// restricted creates a service account with the embedded policy.
|
||||||
|
restricted := func(t *testing.T, key, status string) {
|
||||||
|
t.Helper()
|
||||||
|
_, _, err := f.sys.NewServiceAccount(f.ctx, f.parent.AccessKey, nil, newServiceAccountOpts{accessKey: key, secretKey: "edit-original-secret", status: status, sessionPolicy: want})
|
||||||
|
mustIAM(t, err)
|
||||||
|
}
|
||||||
|
// embedded reports whether this server and storage have the policy, and
|
||||||
|
// returns the stored credential.
|
||||||
|
embedded := func(t *testing.T, key string) (cached, stored bool, cr auth.Credentials) {
|
||||||
|
t.Helper()
|
||||||
|
_, got, err := f.sys.GetServiceAccount(f.ctx, key)
|
||||||
|
mustIAM(t, err)
|
||||||
|
if got != nil && !got.Equals(*want) {
|
||||||
|
t.Fatalf("the embedded policy changed: %+v", got)
|
||||||
|
}
|
||||||
|
u, ok := f.stored(t, key)
|
||||||
|
if !ok {
|
||||||
|
t.Fatal("service account missing in storage")
|
||||||
|
}
|
||||||
|
claims, err := getClaimsFromTokenWithSecret(u.Credentials.SessionToken, u.Credentials.SecretKey)
|
||||||
|
mustIAM(t, err)
|
||||||
|
_, hasPolicy := claims.Lookup(policy.SessionPolicyName)
|
||||||
|
return got != nil, hasPolicy && claims.MapClaims[iamPolicyClaimNameSA()] == embeddedPolicyType, u.Credentials
|
||||||
|
}
|
||||||
|
// check wants the edit applied, and the policy kept or removed.
|
||||||
|
check := func(t *testing.T, key string, keep bool, took func(auth.Credentials) bool) {
|
||||||
|
t.Helper()
|
||||||
|
cached, stored, cr := embedded(t, key)
|
||||||
|
if cached != keep || stored != keep {
|
||||||
|
t.Errorf("embedded policy: cached=%v stored=%v, want %v", cached, stored, keep)
|
||||||
|
}
|
||||||
|
if took != nil && !took(cr) {
|
||||||
|
t.Errorf("the edit did not apply: status=%q name=%q description=%q expiration=%v", cr.Status, cr.Name, cr.Description, cr.Expiration)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
later := UTCNow().Add(time.Hour)
|
||||||
|
for _, edit := range []struct {
|
||||||
|
name, start string
|
||||||
|
opts updateServiceAccountOpts
|
||||||
|
took func(auth.Credentials) bool
|
||||||
|
}{
|
||||||
|
{"disable", auth.AccountOn, updateServiceAccountOpts{status: string(madmin.AccountDisabled)}, func(c auth.Credentials) bool { return c.Status == auth.AccountOff }},
|
||||||
|
{"enable", auth.AccountOff, updateServiceAccountOpts{status: string(madmin.AccountEnabled)}, func(c auth.Credentials) bool { return c.Status == auth.AccountOn }},
|
||||||
|
{"status-off", auth.AccountOn, updateServiceAccountOpts{status: auth.AccountOff}, func(c auth.Credentials) bool { return c.Status == auth.AccountOff }},
|
||||||
|
{"status-on", auth.AccountOff, updateServiceAccountOpts{status: auth.AccountOn}, func(c auth.Credentials) bool { return c.Status == auth.AccountOn }},
|
||||||
|
{"secret-key", auth.AccountOn, updateServiceAccountOpts{secretKey: "edit-rotated-secret"}, func(c auth.Credentials) bool { return c.SecretKey == "edit-rotated-secret" }},
|
||||||
|
{"name", auth.AccountOn, updateServiceAccountOpts{name: "renamed"}, func(c auth.Credentials) bool { return c.Name == "renamed" }},
|
||||||
|
{"description", auth.AccountOn, updateServiceAccountOpts{description: "described"}, func(c auth.Credentials) bool { return c.Description == "described" }},
|
||||||
|
{"expiration", auth.AccountOn, updateServiceAccountOpts{expiration: &later}, func(c auth.Credentials) bool { return c.Expiration.Equal(later) }},
|
||||||
|
} {
|
||||||
|
t.Run(edit.name, func(t *testing.T) {
|
||||||
|
key := "edit-" + edit.name
|
||||||
|
restricted(t, key, edit.start)
|
||||||
|
_, err := f.sys.UpdateServiceAccount(f.ctx, key, edit.opts)
|
||||||
|
mustIAM(t, err)
|
||||||
|
check(t, key, true, edit.took)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
t.Run("empty-policy", func(t *testing.T) {
|
||||||
|
const key = "edit-empty-policy"
|
||||||
|
restricted(t, key, auth.AccountOn)
|
||||||
|
_, err := f.sys.UpdateServiceAccount(f.ctx, key, updateServiceAccountOpts{sessionPolicy: &policy.Policy{}})
|
||||||
|
mustIAM(t, err)
|
||||||
|
check(t, key, false, nil)
|
||||||
|
})
|
||||||
|
|
||||||
|
// A peer applies the edit as the site that took it did. The edit goes
|
||||||
|
// through the encoding of the wire, which is where no policy and a null
|
||||||
|
// one become the same.
|
||||||
|
for _, edit := range []struct {
|
||||||
|
name string
|
||||||
|
update madmin.SRSvcAccUpdate
|
||||||
|
keep bool
|
||||||
|
took func(auth.Credentials) bool
|
||||||
|
}{
|
||||||
|
{"peer-status", madmin.SRSvcAccUpdate{Status: auth.AccountOff}, true, func(c auth.Credentials) bool { return c.Status == auth.AccountOff }},
|
||||||
|
{"peer-null", madmin.SRSvcAccUpdate{Description: "described", SessionPolicy: madmin.SRSessionPolicy("null")}, true, func(c auth.Credentials) bool { return c.Description == "described" }},
|
||||||
|
{"peer-empty", madmin.SRSvcAccUpdate{SessionPolicy: madmin.SRSessionPolicy(`{"Statement":[]}`)}, false, nil},
|
||||||
|
} {
|
||||||
|
t.Run(edit.name, func(t *testing.T) {
|
||||||
|
key := "edit-" + edit.name
|
||||||
|
restricted(t, key, auth.AccountOn)
|
||||||
|
edit.update.AccessKey = key
|
||||||
|
sent, err := json.Marshal(iamReplicationItem{SRIAMItem: madmin.SRIAMItem{
|
||||||
|
Type: madmin.SRIAMItemSvcAcc,
|
||||||
|
SvcAccChange: &madmin.SRSvcAccChange{Update: &edit.update},
|
||||||
|
UpdatedAt: UTCNow(),
|
||||||
|
}})
|
||||||
|
mustIAM(t, err)
|
||||||
|
var got iamReplicationItem
|
||||||
|
mustIAM(t, json.Unmarshal(sent, &got))
|
||||||
|
mustIAM(t, applyIAMReplicationItem(f.ctx, got))
|
||||||
|
check(t, key, edit.keep, edit.took)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The same over the admin API, with what a client can send for no policy.
|
||||||
|
func TestServiceAccountUpdateKeepsPolicy(t *testing.T) {
|
||||||
|
if runtime.GOOS == globalWindowsOSName {
|
||||||
|
t.Skip("IAM integration harness is disabled on Windows")
|
||||||
|
}
|
||||||
|
s := newTestSuiteIAM(TestSuiteCommon{serverType: "ErasureSD", signer: signerV4}, false)
|
||||||
|
c := &check{t, s.serverType}
|
||||||
|
s.SetUpSuite(c)
|
||||||
|
defer s.TearDownSuite(c)
|
||||||
|
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), testDefaultTimeout)
|
||||||
|
defer cancel()
|
||||||
|
|
||||||
|
bucket := getRandomBucketName()
|
||||||
|
if err := s.client.MakeBucket(ctx, bucket, minio.MakeBucketOptions{}); err != nil {
|
||||||
|
c.Fatalf("bucket create error: %v", err)
|
||||||
|
}
|
||||||
|
if err := s.adm.AddCannedPolicy(ctx, "keeps-policy-parent", fmt.Appendf(nil, `{"Version":"2012-10-17","Statement":[
|
||||||
|
{"Effect":"Allow","Action":["s3:ListBucket"],"Resource":["arn:aws:s3:::%s"]},
|
||||||
|
{"Effect":"Allow","Action":["s3:PutObject","s3:GetObject"],"Resource":["arn:aws:s3:::%s/*"]}]}`, bucket, bucket)); err != nil {
|
||||||
|
c.Fatalf("policy add error: %v", err)
|
||||||
|
}
|
||||||
|
parentAK, parentSK := mustGenerateCredentials(c)
|
||||||
|
if err := s.adm.SetUser(ctx, parentAK, parentSK, madmin.AccountEnabled); err != nil {
|
||||||
|
c.Fatalf("unable to set user: %v", err)
|
||||||
|
}
|
||||||
|
if _, err := s.adm.AttachPolicy(ctx, madmin.PolicyAssociationReq{Policies: []string{"keeps-policy-parent"}, User: parentAK}); err != nil {
|
||||||
|
c.Fatalf("unable to attach policy: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The account may read objects; listing is its parent's to do.
|
||||||
|
svcAK, svcSK := mustGenerateCredentials(c)
|
||||||
|
cr, err := s.adm.AddServiceAccount(ctx, madmin.AddServiceAccountReq{
|
||||||
|
Policy: fmt.Appendf(nil, `{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":["s3:GetObject"],"Resource":["arn:aws:s3:::%s/*"]}]}`, bucket),
|
||||||
|
TargetUser: parentAK,
|
||||||
|
AccessKey: svcAK,
|
||||||
|
SecretKey: svcSK,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
c.Fatalf("unable to create svc acc: %v", err)
|
||||||
|
}
|
||||||
|
c.mustUpload(ctx, s.client, bucket)
|
||||||
|
svcClient := s.getUserClient(c, cr.AccessKey, cr.SecretKey, "")
|
||||||
|
c.mustDownload(ctx, svcClient, bucket)
|
||||||
|
c.mustNotListObjects(ctx, svcClient, bucket)
|
||||||
|
|
||||||
|
info := func() madmin.InfoServiceAccountResp {
|
||||||
|
info, err := s.adm.InfoServiceAccount(ctx, svcAK)
|
||||||
|
if err != nil {
|
||||||
|
c.Fatalf("unable to get svc acc info: %v", err)
|
||||||
|
}
|
||||||
|
return info
|
||||||
|
}
|
||||||
|
|
||||||
|
_, rotatedSK := mustGenerateCredentials(c)
|
||||||
|
later := UTCNow().Add(time.Hour).Truncate(time.Second)
|
||||||
|
for _, edit := range []struct {
|
||||||
|
name string
|
||||||
|
req madmin.UpdateServiceAccountReq
|
||||||
|
took func(madmin.InfoServiceAccountResp) bool
|
||||||
|
}{
|
||||||
|
{"disable", madmin.UpdateServiceAccountReq{NewStatus: "off"}, func(i madmin.InfoServiceAccountResp) bool { return i.AccountStatus == auth.AccountOff }},
|
||||||
|
{"enable", madmin.UpdateServiceAccountReq{NewStatus: "on"}, func(i madmin.InfoServiceAccountResp) bool { return i.AccountStatus == auth.AccountOn }},
|
||||||
|
{"description", madmin.UpdateServiceAccountReq{NewDescription: "described"}, func(i madmin.InfoServiceAccountResp) bool { return i.Description == "described" }},
|
||||||
|
{"name", madmin.UpdateServiceAccountReq{NewName: "renamed"}, func(i madmin.InfoServiceAccountResp) bool { return i.Name == "renamed" }},
|
||||||
|
{"expiry", madmin.UpdateServiceAccountReq{NewExpiration: &later}, func(i madmin.InfoServiceAccountResp) bool { return i.Expiration != nil && i.Expiration.Equal(later) }},
|
||||||
|
{"secret-key", madmin.UpdateServiceAccountReq{NewSecretKey: rotatedSK}, func(madmin.InfoServiceAccountResp) bool {
|
||||||
|
// The new key signs a request the embedded policy allows.
|
||||||
|
svcClient = s.getUserClient(c, svcAK, rotatedSK, "")
|
||||||
|
_, err := svcClient.StatObject(ctx, bucket, "some-object", minio.StatObjectOptions{})
|
||||||
|
return err == nil
|
||||||
|
}},
|
||||||
|
{"null-policy", madmin.UpdateServiceAccountReq{NewPolicy: json.RawMessage("null"), NewDescription: "null"}, func(i madmin.InfoServiceAccountResp) bool { return i.Description == "null" }},
|
||||||
|
} {
|
||||||
|
if err := s.adm.UpdateServiceAccount(ctx, svcAK, edit.req); err != nil {
|
||||||
|
c.Fatalf("%s: unable to update svc acc: %v", edit.name, err)
|
||||||
|
}
|
||||||
|
got := info()
|
||||||
|
if got.ImpliedPolicy {
|
||||||
|
c.Fatalf("%s: the account lost its embedded policy", edit.name)
|
||||||
|
}
|
||||||
|
if !edit.took(got) {
|
||||||
|
c.Fatalf("%s: the edit did not apply: %+v", edit.name, got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
c.mustNotListObjects(ctx, svcClient, bucket)
|
||||||
|
|
||||||
|
// An empty policy document asks for the parent's policies.
|
||||||
|
if err := s.adm.UpdateServiceAccount(ctx, svcAK, madmin.UpdateServiceAccountReq{NewPolicy: []byte(`{"Statement":[]}`)}); err != nil {
|
||||||
|
c.Fatalf("unable to remove the policy of svc acc: %v", err)
|
||||||
|
}
|
||||||
|
if !info().ImpliedPolicy {
|
||||||
|
c.Fatalf("an empty policy document must remove the embedded policy")
|
||||||
|
}
|
||||||
|
c.mustListObjects(ctx, svcClient, bucket)
|
||||||
|
}
|
||||||
@@ -0,0 +1,91 @@
|
|||||||
|
// Copyright (c) 2026 Denis Babchuk
|
||||||
|
//
|
||||||
|
// This file is part of Silo Object Storage stack
|
||||||
|
//
|
||||||
|
// This program is free software: you can redistribute it and/or modify
|
||||||
|
// it under the terms of the GNU Affero General Public License as published by
|
||||||
|
// the Free Software Foundation, either version 3 of the License, or
|
||||||
|
// (at your option) any later version.
|
||||||
|
//
|
||||||
|
// This program is distributed in the hope that it will be useful,
|
||||||
|
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||||
|
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||||
|
// GNU Affero General Public License for more details.
|
||||||
|
//
|
||||||
|
// You should have received a copy of the GNU Affero General Public License
|
||||||
|
// along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||||
|
|
||||||
|
package cmd
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/xml"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"net/url"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/minio/minio/internal/auth"
|
||||||
|
xhttp "github.com/minio/minio/internal/http"
|
||||||
|
)
|
||||||
|
|
||||||
|
// GetObjectAttributes answers NoSuchKey with AccessDenied to a caller without
|
||||||
|
// s3:ListBucket on the bucket, as S3 does, so that a missing object cannot be
|
||||||
|
// told apart from a forbidden one. A server error, such as lost read quorum,
|
||||||
|
// must reach a caller allowed to read the object as the error it is.
|
||||||
|
func TestGetObjectAttributesReadQuorumLost(t *testing.T) {
|
||||||
|
ctx, cancel := context.WithCancel(t.Context())
|
||||||
|
defer cancel()
|
||||||
|
obj, dirs, err := prepareErasure16(ctx)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
z := obj.(*erasureServerPools)
|
||||||
|
previous := newObjectLayerFn()
|
||||||
|
setObjectLayer(z)
|
||||||
|
defer func() {
|
||||||
|
z.Shutdown(context.Background())
|
||||||
|
removeRoots(dirs)
|
||||||
|
setObjectLayer(previous)
|
||||||
|
}()
|
||||||
|
bucket, router, err := initAPIHandlerTest(ctx, z, nil, MakeBucketOptions{})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
const object, missing = "object", "missing"
|
||||||
|
if _, err := z.PutObject(ctx, bucket, object, mustGetPutObjReader(t, bytes.NewReader([]byte("data")), 4, "", ""), ObjectOptions{}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
read := `{"Effect": "Allow", "Action": ["s3:GetObject", "s3:GetObjectAttributes"], "Resource": ["arn:aws:s3:::` + bucket + `/*"]}`
|
||||||
|
withoutList := newDeleteAuthzPolicyUser(t, "", bucket, "["+read+"]")
|
||||||
|
withList := newDeleteAuthzPolicyUser(t, "", bucket, "["+read+
|
||||||
|
`, {"Effect": "Allow", "Action": ["s3:ListBucket"], "Resource": ["arn:aws:s3:::`+bucket+`"]}]`)
|
||||||
|
|
||||||
|
check := func(stage, user string, creds auth.Credentials, key string, want int) {
|
||||||
|
t.Helper()
|
||||||
|
target := makeTestTargetURL("", bucket, key, url.Values{"attributes": []string{""}})
|
||||||
|
req, err := newTestSignedRequestV4(http.MethodGet, target, 0, nil, creds.AccessKey, creds.SecretKey,
|
||||||
|
map[string]string{xhttp.AmzObjectAttributes: "ETag"})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
router.ServeHTTP(rec, req)
|
||||||
|
if rec.Code != want {
|
||||||
|
var apiErr APIErrorResponse
|
||||||
|
_ = xml.Unmarshal(rec.Body.Bytes(), &apiErr)
|
||||||
|
t.Errorf("%s: GetObjectAttributes %s by %s: status %d %s, want %d", stage, key, user, rec.Code, apiErr.Code, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
check("healthy", "withoutList", withoutList, object, http.StatusOK)
|
||||||
|
check("healthy", "withList", withList, object, http.StatusOK)
|
||||||
|
check("healthy", "withoutList", withoutList, missing, http.StatusForbidden)
|
||||||
|
check("healthy", "withList", withList, missing, http.StatusNotFound)
|
||||||
|
|
||||||
|
defer objectReadFault(z, bucket, object, 10)()
|
||||||
|
check("read quorum lost", "withoutList", withoutList, object, http.StatusServiceUnavailable)
|
||||||
|
check("read quorum lost", "withList", withList, object, http.StatusServiceUnavailable)
|
||||||
|
}
|
||||||
+32
-12
@@ -395,12 +395,14 @@ func (api objectAPIHandlers) getObjectHandler(ctx context.Context, objectAPI Obj
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Validate pre-conditions if any.
|
// Validate pre-conditions if any.
|
||||||
|
var storedTags *string // set once the object has been read far enough
|
||||||
opts.CheckPrecondFn = func(oi ObjectInfo) bool {
|
opts.CheckPrecondFn = func(oi ObjectInfo) bool {
|
||||||
if _, err := DecryptObjectInfo(&oi, r); err != nil {
|
if _, err := DecryptObjectInfo(&oi, r); err != nil {
|
||||||
writeErrorResponse(ctx, w, toAPIError(ctx, err), r.URL)
|
writeErrorResponse(ctx, w, toAPIError(ctx, err), r.URL)
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
|
|
||||||
|
storedTags = &oi.UserTags
|
||||||
if s3Error := authorizeRequestWithExistingTags(ctx, r, policy.GetObjectAction, oi.UserTags); s3Error != ErrNone {
|
if s3Error := authorizeRequestWithExistingTags(ctx, r, policy.GetObjectAction, oi.UserTags); s3Error != ErrNone {
|
||||||
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(s3Error), r.URL)
|
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(s3Error), r.URL)
|
||||||
return true
|
return true
|
||||||
@@ -445,8 +447,14 @@ func (api objectAPIHandlers) getObjectHandler(ctx context.Context, objectAPI Obj
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
// validate if the request indeed was authorized, if it wasn't we need to return "ErrAccessDenied"
|
// validate if the request indeed was authorized, if it wasn't we need to return "ErrAccessDenied"
|
||||||
// instead of any namespace related error.
|
// instead of any namespace related error. Use the stored tags if the callback read them.
|
||||||
if s3Error := authorizeRequest(ctx, r, policy.GetObjectAction); s3Error != ErrNone {
|
var s3Error APIErrorCode
|
||||||
|
if storedTags != nil {
|
||||||
|
s3Error = authorizeRequestWithExistingTags(ctx, r, policy.GetObjectAction, *storedTags)
|
||||||
|
} else {
|
||||||
|
s3Error = authorizeRequestWithUnreadTags(ctx, r, policy.GetObjectAction, err)
|
||||||
|
}
|
||||||
|
if s3Error != ErrNone {
|
||||||
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(s3Error), r.URL)
|
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(s3Error), r.URL)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
@@ -623,12 +631,15 @@ func (api objectAPIHandlers) getObjectAttributesHandler(ctx context.Context, obj
|
|||||||
|
|
||||||
objInfo, err := objectAPI.GetObjectInfo(ctx, bucket, object, opts)
|
objInfo, err := objectAPI.GetObjectInfo(ctx, bucket, object, opts)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
s3Error = authorizeRequest(ctx, r, policy.ListBucketAction)
|
// As with S3, a caller without s3:ListBucket on the bucket gets
|
||||||
if s3Error == ErrNone {
|
// AccessDenied in place of NoSuchKey. A server error, such as lost
|
||||||
writeErrorResponse(ctx, w, toAPIError(ctx, err), r.URL)
|
// read quorum, tells nothing about the object and reaches the caller,
|
||||||
return
|
// who may read the object, as it is.
|
||||||
|
apiErr := toAPIError(ctx, err)
|
||||||
|
if apiErr.HTTPStatusCode < http.StatusInternalServerError && authorizeBucketRequest(ctx, r, policy.ListBucketAction) != ErrNone {
|
||||||
|
apiErr = errorCodes.ToAPIErr(ErrAccessDenied)
|
||||||
}
|
}
|
||||||
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(ErrAccessDenied), r.URL)
|
writeErrorResponse(ctx, w, apiErr, r.URL)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -925,7 +936,14 @@ func (api objectAPIHandlers) headObjectHandler(ctx context.Context, objectAPI Ob
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if s3Error := authorizeRequestWithExistingTags(ctx, r, policy.GetObjectAction, objInfo.UserTags); s3Error != ErrNone {
|
// objInfo carries the stored tags unless reading the object failed.
|
||||||
|
var s3Error APIErrorCode
|
||||||
|
if err != nil && !proxy.Proxy {
|
||||||
|
s3Error = authorizeRequestWithUnreadTags(ctx, r, policy.GetObjectAction, err)
|
||||||
|
} else {
|
||||||
|
s3Error = authorizeRequestWithExistingTags(ctx, r, policy.GetObjectAction, objInfo.UserTags)
|
||||||
|
}
|
||||||
|
if s3Error != ErrNone {
|
||||||
writeErrorResponseHeadersOnly(w, errorCodes.ToAPIErr(s3Error))
|
writeErrorResponseHeadersOnly(w, errorCodes.ToAPIErr(s3Error))
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
@@ -3783,11 +3801,13 @@ func (api objectAPIHandlers) PutObjectTaggingHandler(w http.ResponseWriter, r *h
|
|||||||
}
|
}
|
||||||
|
|
||||||
objInfo, err := objAPI.GetObjectInfo(ctx, bucket, object, opts)
|
objInfo, err := objAPI.GetObjectInfo(ctx, bucket, object, opts)
|
||||||
existingTags := ""
|
var s3Error APIErrorCode
|
||||||
if err == nil {
|
if err != nil {
|
||||||
existingTags = objInfo.UserTags
|
s3Error = authorizeRequestWithUnreadTags(ctx, r, policy.PutObjectTaggingAction, err)
|
||||||
|
} else {
|
||||||
|
s3Error = authorizeRequestWithExistingTags(ctx, r, policy.PutObjectTaggingAction, objInfo.UserTags)
|
||||||
}
|
}
|
||||||
if s3Error := authorizeRequestWithExistingTags(ctx, r, policy.PutObjectTaggingAction, existingTags); s3Error != ErrNone {
|
if s3Error != ErrNone {
|
||||||
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(s3Error), r.URL)
|
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(s3Error), r.URL)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# Silo Healthcheck
|
# Silo Healthcheck
|
||||||
|
|
||||||
Silo server exposes three un-authenticated, healthcheck endpoints liveness probe and a cluster probe at `/minio/health/live` and `/minio/health/cluster` respectively.
|
Silo server exposes four unauthenticated healthcheck endpoints: `/minio/health/live`, `/minio/health/ready`, `/minio/health/cluster`, and `/minio/health/cluster/read`.
|
||||||
|
|
||||||
## Native CLI probe
|
## Native CLI probe
|
||||||
|
|
||||||
@@ -76,7 +76,7 @@ readinessProbe:
|
|||||||
The reply is '200 OK' if cluster has write quorum if not it returns '503 Service Unavailable'.
|
The reply is '200 OK' if cluster has write quorum if not it returns '503 Service Unavailable'.
|
||||||
|
|
||||||
```
|
```
|
||||||
curl http://silo1:9001/minio/health/cluster
|
curl http://silo1:9000/minio/health/cluster
|
||||||
HTTP/1.1 503 Service Unavailable
|
HTTP/1.1 503 Service Unavailable
|
||||||
Accept-Ranges: bytes
|
Accept-Ranges: bytes
|
||||||
Content-Length: 0
|
Content-Length: 0
|
||||||
@@ -94,7 +94,7 @@ Date: Tue, 21 Jul 2020 00:36:14 GMT
|
|||||||
The reply is '200 OK' if cluster has read quorum if not it returns '503 Service Unavailable'.
|
The reply is '200 OK' if cluster has read quorum if not it returns '503 Service Unavailable'.
|
||||||
|
|
||||||
```
|
```
|
||||||
curl http://silo1:9001/minio/health/cluster/read
|
curl http://silo1:9000/minio/health/cluster/read
|
||||||
HTTP/1.1 503 Service Unavailable
|
HTTP/1.1 503 Service Unavailable
|
||||||
Accept-Ranges: bytes
|
Accept-Ranges: bytes
|
||||||
Content-Length: 0
|
Content-Length: 0
|
||||||
@@ -112,7 +112,7 @@ Date: Tue, 21 Jul 2020 00:36:14 GMT
|
|||||||
You may query the cluster probe endpoint to check if the node which received the request can be taken down for maintenance, if the server replies back '412 Precondition Failed' this means you will lose HA. '200 OK' means you are okay to proceed.
|
You may query the cluster probe endpoint to check if the node which received the request can be taken down for maintenance, if the server replies back '412 Precondition Failed' this means you will lose HA. '200 OK' means you are okay to proceed.
|
||||||
|
|
||||||
```
|
```
|
||||||
curl http://silo1:9001/minio/health/cluster?maintenance=true
|
curl http://silo1:9000/minio/health/cluster?maintenance=true
|
||||||
HTTP/1.1 412 Precondition Failed
|
HTTP/1.1 412 Precondition Failed
|
||||||
Accept-Ranges: bytes
|
Accept-Ranges: bytes
|
||||||
Content-Length: 0
|
Content-Length: 0
|
||||||
|
|||||||
+17
-3
@@ -34,9 +34,10 @@ helm install silo ./helm/silo \
|
|||||||
--set rootPassword='replace-with-a-long-random-secret'
|
--set rootPassword='replace-with-a-long-random-secret'
|
||||||
```
|
```
|
||||||
|
|
||||||
The default server and post-install image is `docker.io/pgsty/silo`. The image
|
The default server image is `docker.io/pgsty/silo` (which contains the `silo`
|
||||||
contains the `silo` server, `mcli`, and an `mc` compatibility symlink. It does
|
server, `mcli`, and an `mc` client compatibility symlink, without a `minio`
|
||||||
not contain a `minio` server binary.
|
server binary). Post-install resource jobs use `mcImage` (`docker.io/pgsty/mc`)
|
||||||
|
by default.
|
||||||
|
|
||||||
## Compatibility contract
|
## Compatibility contract
|
||||||
|
|
||||||
@@ -100,6 +101,19 @@ new chart invokes `silo server`; an old image does not provide that executable.
|
|||||||
The new image accepts a legacy first argv token of `minio` only to support old
|
The new image accepts a legacy first argv token of `minio` only to support old
|
||||||
chart commands during the forward migration.
|
chart commands during the forward migration.
|
||||||
|
|
||||||
|
When upgrading an existing Silo release to `RELEASE.2026-11-09T00-00-00Z` (chart
|
||||||
|
`7.0.4`):
|
||||||
|
|
||||||
|
- `helm upgrade --reuse-values` keeps the previously saved `image.tag` and
|
||||||
|
`mcImage.tag`; pass `--set-string image.tag=RELEASE.2026-11-09T00-00-00Z` and
|
||||||
|
`--set-string mcImage.tag=RELEASE.2026-11-09T00-00-00Z` explicitly.
|
||||||
|
- Drain or cancel active batch jobs before upgrading; `20261026` cannot resume
|
||||||
|
new submitter-bound batch manifests on downgrade.
|
||||||
|
- Single-pool clusters whose erasure sets do not all share the same pods/hosts
|
||||||
|
require a drained stop-all upgrade rather than a rolling pod restart, and
|
||||||
|
`POST ?restore` returns `503` while mixed versions are serving. See the
|
||||||
|
[20261109 release notes](https://silo.pgsty.com/blog/release/silo-20261109/#upgrade).
|
||||||
|
|
||||||
## Persistence and TLS
|
## Persistence and TLS
|
||||||
|
|
||||||
The default PersistentVolumeClaim mounts at `/export`. Use an existing claim
|
The default PersistentVolumeClaim mounts at `/export`. Use an existing claim
|
||||||
|
|||||||
@@ -1,43 +1,44 @@
|
|||||||
|
{{- $scheme := ternary "https" "http" .Values.tls.enabled -}}
|
||||||
{{- if eq .Values.service.type "ClusterIP" "NodePort" }}
|
{{- if eq .Values.service.type "ClusterIP" "NodePort" }}
|
||||||
Silo can be accessed via port {{ .Values.service.port }} on the following DNS name from within your cluster:
|
Silo can be accessed via port {{ .Values.service.port }} on the following DNS name from within your cluster:
|
||||||
{{ template "silo.fullname" . }}.{{ .Release.Namespace }}.{{ .Values.clusterDomain }}
|
{{ template "silo.fullname" . }}.{{ .Release.Namespace }}.svc.{{ .Values.clusterDomain }}
|
||||||
|
|
||||||
To access Silo from localhost, run the below commands:
|
To access Silo from localhost, run the commands below:
|
||||||
|
|
||||||
1. export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "release={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")
|
1. export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "release={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")
|
||||||
|
|
||||||
2. kubectl port-forward $POD_NAME 9000 --namespace {{ .Release.Namespace }}
|
2. kubectl port-forward $POD_NAME {{ .Values.service.port }}:{{ .Values.minioAPIPort | default 9000 }} --namespace {{ .Release.Namespace }}
|
||||||
|
|
||||||
Read more about port forwarding here: http://kubernetes.io/docs/user-guide/kubectl/kubectl_port-forward/
|
Read more about port forwarding here: https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/
|
||||||
|
|
||||||
You can now access Silo server on http://localhost:9000. Follow the below steps to connect to Silo server with mc client:
|
You can now access the Silo server on {{ $scheme }}://localhost:{{ .Values.service.port }}. Connect with the mcli client:
|
||||||
|
|
||||||
1. Download the Silo mc client - https://silo.pgsty.com/reference/minio-mc/#quickstart
|
1. Download mcli: https://silo.pgsty.com/download/#client
|
||||||
|
|
||||||
2. export MC_HOST_{{ template "silo.fullname" . }}_local=http://$(kubectl get secret --namespace {{ .Release.Namespace }} {{ template "silo.secretName" . }} -o jsonpath="{.data.rootUser}" | base64 --decode):$(kubectl get secret --namespace {{ .Release.Namespace }} {{ template "silo.secretName" . }} -o jsonpath="{.data.rootPassword}" | base64 --decode)@localhost:{{ .Values.service.port }}
|
2. mcli alias set local {{ $scheme }}://localhost:{{ .Values.service.port }} "$(kubectl get secret --namespace {{ .Release.Namespace }} {{ template "silo.secretName" . }} -o jsonpath="{.data.rootUser}" | base64 --decode)" "$(kubectl get secret --namespace {{ .Release.Namespace }} {{ template "silo.secretName" . }} -o jsonpath="{.data.rootPassword}" | base64 --decode)"
|
||||||
|
|
||||||
3. mc ls {{ template "silo.fullname" . }}_local
|
3. mcli ls local
|
||||||
|
|
||||||
{{- end }}
|
{{- end }}
|
||||||
{{- if eq .Values.service.type "LoadBalancer" }}
|
{{- if eq .Values.service.type "LoadBalancer" }}
|
||||||
Silo can be accessed via port {{ .Values.service.port }} on an external IP address. Get the service external IP address by:
|
Silo can be accessed via port {{ .Values.service.port }} on an external IP address. Get the service external IP address by:
|
||||||
kubectl get svc --namespace {{ .Release.Namespace }} -l app={{ template "silo.fullname" . }}
|
kubectl get svc --namespace {{ .Release.Namespace }} -l "app={{ template "silo.name" . }},release={{ .Release.Name }}"
|
||||||
|
|
||||||
Note that the public IP may take a couple of minutes to be available.
|
Note that the public IP may take a couple of minutes to become available.
|
||||||
|
|
||||||
You can now access Silo server on http://<External-IP>:9000. Follow the below steps to connect to Silo server with mc client:
|
You can now access the Silo server on {{ $scheme }}://<External-IP>:{{ .Values.service.port }}. Connect with the mcli client:
|
||||||
|
|
||||||
1. Download the Silo mc client - https://silo.pgsty.com/reference/minio-mc/#quickstart
|
1. Download mcli: https://silo.pgsty.com/download/#client
|
||||||
|
|
||||||
2. export MC_HOST_{{ template "silo.fullname" . }}_local=http://$(kubectl get secret {{ template "silo.secretName" . }} --namespace {{ .Release.Namespace }} -o jsonpath="{.data.rootUser}" | base64 --decode):$(kubectl get secret {{ template "silo.secretName" . }} -o jsonpath="{.data.rootPassword}" | base64 --decode)@<External-IP>:{{ .Values.service.port }}
|
2. mcli alias set external {{ $scheme }}://<External-IP>:{{ .Values.service.port }} "$(kubectl get secret {{ template "silo.secretName" . }} --namespace {{ .Release.Namespace }} -o jsonpath="{.data.rootUser}" | base64 --decode)" "$(kubectl get secret {{ template "silo.secretName" . }} --namespace {{ .Release.Namespace }} -o jsonpath="{.data.rootPassword}" | base64 --decode)"
|
||||||
|
|
||||||
3. mc ls {{ template "silo.fullname" . }}
|
3. mcli ls external
|
||||||
|
|
||||||
Alternately, use a browser, an S3 SDK, or a MinIO-compatible SDK to access Silo - https://silo.pgsty.com/reference/minio-server/minio-server/
|
Alternately, use a browser, an S3 SDK, or a MinIO-compatible SDK to access Silo: https://silo.pgsty.com/reference/minio-server/
|
||||||
{{- end }}
|
{{- end }}
|
||||||
|
|
||||||
{{ if and (.Values.networkPolicy.enabled) (not .Values.networkPolicy.allowExternal) }}
|
{{ if and (.Values.networkPolicy.enabled) (not .Values.networkPolicy.allowExternal) }}
|
||||||
Note: Since NetworkPolicy is enabled, only pods with label
|
Note: Since NetworkPolicy is enabled, only pods with label
|
||||||
{{ template "silo.fullname" . }}-client=true"
|
{{ template "silo.fullname" . }}-client=true
|
||||||
will be able to connect to this Silo cluster.
|
will be able to connect to this Silo cluster.
|
||||||
{{- end }}
|
{{- end }}
|
||||||
|
|||||||
@@ -11,7 +11,7 @@ fullnameOverride: ""
|
|||||||
##
|
##
|
||||||
clusterDomain: cluster.local
|
clusterDomain: cluster.local
|
||||||
|
|
||||||
## Set default image, imageTag, and imagePullPolicy. mode is used to indicate the
|
## Set default Silo server image repository, tag, and pull policy.
|
||||||
##
|
##
|
||||||
image:
|
image:
|
||||||
repository: pgsty/silo
|
repository: pgsty/silo
|
||||||
@@ -517,7 +517,7 @@ postJob:
|
|||||||
## Add environment variables for Silo. The MINIO_* names remain the stable
|
## Add environment variables for Silo. The MINIO_* names remain the stable
|
||||||
## configuration interface and are passed to the Silo container(s).
|
## configuration interface and are passed to the Silo container(s).
|
||||||
environment:
|
environment:
|
||||||
## Please refer for comprehensive list https://silo.pgsty.com/reference/minio-server/minio-server/
|
## See https://silo.pgsty.com/reference/minio-server/settings/ for available settings.
|
||||||
## MINIO_BROWSER: "off"
|
## MINIO_BROWSER: "off"
|
||||||
|
|
||||||
## The name of a secret in the same kubernetes namespace which contain secret values
|
## The name of a secret in the same kubernetes namespace which contain secret values
|
||||||
|
|||||||
Reference in New Issue
Block a user