Compare commits

..

19 Commits

Author SHA1 Message Date
Feng Ruohang 2f93e0c6b3 Merge pull request #294 from botdeniska/fix/tag-auth-server-error
Preserve server errors for reads authorized by object tags and GetObjectAttributes.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-10 23:30:24 +08:00
Feng Ruohang f3b42b0eb8 Merge pull request #335 from botdeniska/fix/decom-keys-below-a-key
fix(decom): repeat the walk of a bucket while it removes source versions

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-10 21:30:18 +08:00
Feng Ruohang 2b62df876a docs(contributors): cite the real CVE-2026-39414 follow-up commit
Regenerated from the contributor data on silo.pgsty.com. One note named
5dc705936, which is not a commit in any repository; the June follow-up
is fd69c89d0.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-10 17:24:28 +08:00
Feng Ruohang 518915d241 docs(contributors): credit upstream advisory researchers and fix a commit link
Regenerated from the reviewed contributor data on silo.pgsty.com. The roll
grows from 76 to 81 people: it now names the researchers credited on the
upstream advisories for inherited CVEs. The link to the Helm chart 7.0.4
commit carried a full hash that matches no commit; it now points at
47275042f.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-10 16:57:15 +08:00
Denis Babchuk ce5d191077 fix(decom): repeat the walk of a bucket while it removes source versions
Signed-off-by: Denis Babchuk <babchukdenis@icloud.com>
2026-11-10 11:48:42 +03:00
Denis Babchuk b404f78764 test(decom): cover keys below a key that the walk does not see
Signed-off-by: Denis Babchuk <babchukdenis@icloud.com>
2026-11-10 11:48:42 +03:00
Feng Ruohang d138996983 docs(helm): fix NOTES connection commands and document 20261109 upgrade gates
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-10 14:41:49 +08:00
Feng Ruohang d52ba0290a docs: align release, TLS, healthcheck, and contributor docs with 20261109
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-10 14:41:49 +08:00
Feng Ruohang 8f3657dc52 Merge pull request #333 from pgsty/claude/cross-bucket-path-traversal-delete-5e1fc2
fix(pools): keep rejected names out of a single-pool bulk delete
2026-11-10 14:39:36 +08:00
Feng Ruohang 7efdedd5ba Merge pull request #329 from pgsty/deps/console-2.7.0
deps: pin SILO Console 2.7.0
2026-11-10 14:30:02 +08:00
Feng Ruohang a18293d0c3 fix(pools): keep rejected names out of a single-pool bulk delete
DeleteObjects checks every name with checkDelObjArgs, but a single pool
still sent all entries to the drives and cleared the error of an entry
that a drive reported as found. Local drives join the name to the bucket
directory, so a name with a ".." segment reached another bucket or the
meta bucket. An S3 DeleteObjects caller allowed to delete in one bucket
could remove a version elsewhere, including one under Object Lock
retention, and from a versioned bucket could write a delete marker on an
object of another bucket or on stored bucket metadata. The bulk lock was
taken on the normalized name as well, that is on the other object.

Entries that fail the check now keep their error, and the rest of the
request runs as a request of its own, before any lock or lookup. A
request whose names are all valid does not enter the new code. More than
one pool already skipped such entries.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-10 14:24:35 +08:00
Feng Ruohang 032ac49061 Merge pull request #332 from pgsty/claude/multi-pool-ilm-version-dup-210154
fix(ilm): count a version under old and healed headers as one revision
2026-11-10 14:14:32 +08:00
Feng Ruohang 244de7ecc1 Merge pull request #331 from pgsty/claude/svcacct-inline-policy-bug-961eaa
fix(iam): keep a service account's inline policy on an edit that names none
2026-11-10 14:12:36 +08:00
Feng Ruohang c11bfda8f4 fix(iam): keep a service account's inline policy on an edit that names none
An update-service-account request without a policy removed the inline
policy of the account: the store took a nil session policy for an empty
one, and the handler turned an empty one into nil, so "not given" and
"given, empty" were one case. A status change, a secret key rotation, a
rename or a new expiry (mcli admin user svcacct disable, enable, and edit
without --policy, and the idp ldap accesskey equivalents) signed the token
again as inherited-policy, and the account then had all its parent is
allowed. A replication peer did the same with the forwarded edit.

A nil session policy now leaves the policy as it is. The handler passes an
empty policy document on as such, and only that removes the inline policy,
as mcli admin user svcacct edit --policy with {"Statement":[]} already
asks for. A JSON null policy counts as none. The peer handler passes nil
for an edit without a policy and needs no change.

What an update does with any other policy document is unchanged.

The behaviour came with upstream f0462322f (minio/minio#14847).

Signed-off-by: Feng Ruohang <rh@vonng.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 11:55:28 +08:00
Feng Ruohang 2c06b28dbb deps: pin SILO Console 2.7.0
Pin the Console module to the v2.7.0 tag commit (4e91ae506e70), published
on 2026-11-09. It adds ten UI languages with Arabic right-to-left layout,
fixes usage-worker TLS, scheduling and shutdown, serves folder sizes over
plain HTTP, and fixes the container CA bundle and /tmp handling.

Console 2.7.0 requires silo-pkg v3.16.0 and builds with the mcli
RELEASE.2026-11-09T00-00-00Z source (pgsty/mc e1d4cc5abfe0). Go ignores
replacements declared by dependencies, so copy Console's single mc
replacement here and raise the silo-pkg requirement to match. silo-pkg
v3.16.0 changes only a policy comment and dependency floors.

Minimal version selection also raises klauspost/cpuid/v2 to v2.4.2,
go.yaml.in/yaml/v3 to v3.0.6, go-jose/v4 to v4.1.9, go-openapi/runtime to
v0.33.4 and tidwall/gjson to v1.20.1. minio-go stays at 320af050c445.
CREDITS is unchanged after regeneration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-11-09 20:49:43 +08:00
Denis Babchuk a2ff3c2ecd fix(api): keep GetObjectAttributes server errors and check s3:ListBucket on the bucket
Signed-off-by: Denis Babchuk <babchukdenis@icloud.com>
2026-11-07 11:43:31 +03:00
Denis Babchuk 67bc32f401 test(api): cover GetObjectAttributes errors with and without s3:ListBucket
Signed-off-by: Denis Babchuk <babchukdenis@icloud.com>
2026-11-07 11:43:31 +03:00
Denis Babchuk 3c4a6296f9 fix(auth): keep the server error when a read allowed by object tags cannot read the tags
Signed-off-by: Denis Babchuk <babchukdenis@icloud.com>
2026-11-07 11:43:30 +03:00
Denis Babchuk afa35d16e9 test(auth): cover reads allowed by s3:ExistingObjectTag when the object cannot be read
Signed-off-by: Denis Babchuk <babchukdenis@icloud.com>
2026-11-07 11:43:29 +03:00
28 changed files with 1393 additions and 110 deletions
+40 -6
View File
@@ -1,12 +1,45 @@
# Changelog # Changelog
## Unreleased
- An edit of a service account or access key that names no policy keeps its
inline policy. `mcli admin user svcacct disable`, `enable` and `edit` without
`--policy`, the same `idp ldap accesskey` commands, and any other
`update-service-account` request without `newPolicy` removed the inline
policy at the site that took the request and, for an account of another user
than root, at its replication peers, and the account then had all its parent
is allowed. The inline policy is removed
only on request, by an empty policy document such as `{"Statement":[]}`
(`mcli admin user svcacct edit --policy`); a client that left the policy out
to remove it must send that document. A JSON `null` policy counts as none.
In the Console, clearing the policy editor of an access key no longer removes
the policy; enter `{}` to remove it. The fix does not bring back a policy
that was removed, and a site of a replicated deployment that does not run the
fix still removes the policy when such an edit reaches it, so sites can
differ. Upgrade every site, then review on each site the accounts that
`svcacct info` reports with `Policy: implied`, and set the policy again on
those that were meant to be restricted.
- S3 `DeleteObjects` on a single pool stops at an entry whose name fails the
object name check: the entry returns `XMinioInvalidObjectName` and is not
locked or sent to a drive, as with a single-object DELETE and with more than
one pool. Until now such an entry still reached the drives of the node, where
a name with a `..` segment was joined to the request bucket. A caller allowed
to delete in one bucket could remove a version in another bucket, including
one under Object Lock retention, and from a versioned bucket could put a
delete marker on an object of another bucket or on stored system metadata
such as a bucket's configuration. The other entries of a request are deleted
as before.
## RELEASE.2026-11-09T00-00-00Z ## RELEASE.2026-11-09T00-00-00Z
Package version: `20261109000000.0.0`. Publication status and artifacts are Package version: `20261109000000.0.0`; published from source `d998292387cc`.
listed in [GitHub Releases](https://github.com/pgsty/silo/releases). This Artifacts are listed in the [GitHub release](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-11-09T00-00-00Z).
section covers the changes after the published 20261026 source `c3ad836aed0d`. The coordinated [English release notes](https://silo.pgsty.com/blog/release/silo-20261109/)
Most fixes come from reports and patches by and [Chinese release notes](https://silo.pgsty.com/zh/blog/release/silo-20261109/)
[@botdeniska](https://github.com/botdeniska); see [CONTRIBUTORS.md](CONTRIBUTORS.md). cover all components, upgrade guidance, and validation scope since the
published 20261026 source `c3ad836aed0d`. Most fixes come from reports and
patches by [@botdeniska](https://github.com/botdeniska); see [CONTRIBUTORS.md](CONTRIBUTORS.md).
Lifecycle, multi-pool and tiering: Lifecycle, multi-pool and tiering:
@@ -21,7 +54,8 @@ Lifecycle, multi-pool and tiering:
`retainVersions: 0`) on the expired key instead of the keys below it; a key `retainVersions: 0`) on the expired key instead of the keys below it; a key
with a tiered or Object Lock protected version is refused with a tiered or Object Lock protected version is refused
([#297](https://github.com/pgsty/silo/issues/297)). ([#297](https://github.com/pgsty/silo/issues/297)).
- Stop taking the free version of a deleted tiered version for a version - Exclude the internal free-version marker of a deleted tiered version from
visible version history and lifecycle decisions
([#263](https://github.com/pgsty/silo/issues/263)), apply rules whose only ([#263](https://github.com/pgsty/silo/issues/263)), apply rules whose only
action is `DelMarkerExpiration` ([#268](https://github.com/pgsty/silo/issues/268)), action is `DelMarkerExpiration` ([#268](https://github.com/pgsty/silo/issues/268)),
and report and account only the removals that happened and report and account only the removals that happened
+5 -2
View File
@@ -165,8 +165,11 @@ license grant to SILO or upstream MinIO maintainers is required.
### How does Silo manage dependencies? ### How does Silo manage dependencies?
Silo uses Go modules. Preserve the compatibility module and import paths in Silo uses Go modules. Shared packages in `github.com/pgsty/silo-pkg/v3` are
`go.mod`; downstream forks are selected with explicit `replace` directives. required and imported directly under their own module path, upstream
`github.com/minio/minio-go/v7` is pinned to a verified commit, and Console
(`github.com/minio/console`) and MC (`github.com/minio/mc`) are selected with
explicit `replace` directives in `go.mod`.
- Run `go get foo/bar` in the source folder to add the dependency to `go.mod` file. - Run `go get foo/bar` in the source folder to add the dependency to `go.mod` file.
+35 -23
View File
File diff suppressed because one or more lines are too long
+2 -1
View File
@@ -42,7 +42,8 @@ PRs must flow smoothly through review to reach production. Large PRs should be s
- Document testing steps in the PR summary if manual testing was performed. - Document testing steps in the PR summary if manual testing was performed.
4. **Before Submitting**: 4. **Before Submitting**:
- Run `make verify` to check formatting, linting, and tests. - Sign off every commit under the Developer Certificate of Origin (`git commit -s`).
- Run `make verifiers` to check formatting, linting, code generation, and `rebrand-guard`.
- Reference related issues (e.g., “Closes #1234”). - Reference related issues (e.g., “Closes #1234”).
- Notify team members via GitHub `@mentions` if urgent or complex. - Notify team members via GitHub `@mentions` if urgent or complex.
+5 -5
View File
@@ -36,7 +36,7 @@
PGSTY SILO keeps one maintained release line of the open-source MinIO server alive after upstream ended community distribution: builds, packages, multi-arch images, security fixes, and the full web console. Pigsty runs it in production as its PostgreSQL backup repository. PGSTY SILO keeps one maintained release line of the open-source MinIO server alive after upstream ended community distribution: builds, packages, multi-arch images, security fixes, and the full web console. Pigsty runs it in production as its PostgreSQL backup repository.
It follows one rule — **the product and its delivery surfaces are renamed; the protocol and your data are not.** Everything else lives on [silo.pgsty.com](https://silo.pgsty.com/). It follows one rule: **the product and its delivery surfaces are renamed; the protocol and your data are not.** Full documentation is published at [silo.pgsty.com](https://silo.pgsty.com/).
**Related:** [`pgsty/mc`](https://github.com/pgsty/mc) · [`pgsty/silo-console`](https://github.com/pgsty/silo-console) · [`pgsty/silo-pkg`](https://github.com/pgsty/silo-pkg) · [`pgsty/kes`](https://github.com/pgsty/kes) · [`pgsty/pigsty`](https://github.com/pgsty/pigsty) **Related:** [`pgsty/mc`](https://github.com/pgsty/mc) · [`pgsty/silo-console`](https://github.com/pgsty/silo-console) · [`pgsty/silo-pkg`](https://github.com/pgsty/silo-pkg) · [`pgsty/kes`](https://github.com/pgsty/kes) · [`pgsty/pigsty`](https://github.com/pgsty/pigsty)
@@ -44,10 +44,10 @@ It follows one rule — **the product and its delivery surfaces are renamed; the
<img src="https://silo.pgsty.com/images/silo-console/console-metrics-simple.webp" alt="Silo Console"> <img src="https://silo.pgsty.com/images/silo-console/console-metrics-simple.webp" alt="Silo Console">
</p> </p>
#### SILO 20261109 ### SILO 20261109
> [!WARNING] > [!WARNING]
> [SILO 20261109](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-11-09T00-00-00Z) fixes bugs and security vulnerabilities. We strongly recommend scheduling an upgrade to reduce security risks. > [SILO 20261109](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-11-09T00-00-00Z) (`20261109000000.0.0`, source `d998292387cc`) fixes storage, lifecycle, native restore, Object Lock, batch authorization, and IAM defects, embedding Console `v2.7.0` with `mcli 20261109`, `silo-pkg v3.16.0`, and Go `1.27.3`. Review the [release notes and upgrade requirements](https://silo.pgsty.com/blog/release/silo-20261109/) before upgrading.
> [!NOTE] > [!NOTE]
> On 2026-08-06, `pgsty/minio` became `pgsty/silo`, and the default branch changed from `master` to `main`. The last MinIO-branded release is [`20260804`](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-08-04T00-00-00Z) on the [`minio`](https://github.com/pgsty/silo/tree/minio) branch. > On 2026-08-06, `pgsty/minio` became `pgsty/silo`, and the default branch changed from `master` to `main`. The last MinIO-branded release is [`20260804`](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-08-04T00-00-00Z) on the [`minio`](https://github.com/pgsty/silo/tree/minio) branch.
@@ -86,7 +86,7 @@ docker exec silo mcli mb local/demo && docker exec silo mcli ls local
| Kubernetes | Helm chart, see [Download & Install](https://silo.pgsty.com/download/) | | Kubernetes | Helm chart, see [Download & Install](https://silo.pgsty.com/download/) |
| Source | `go build -o silo . && ./silo --version` | | Source | `go build -o silo . && ./silo --version` |
Every release ships checksums, SPDX SBOMs, Sigstore-signed manifests, and GitHub build attestations. Installation methods and verification commands are documented at [Download & Install](https://silo.pgsty.com/download/); migrating from upstream MinIO — taking over an existing `minio.service` and its `/etc/default/minio`, and keeping data ownership stable with a `/etc/systemd/system/silo.service.d/10-legacy-user.conf` drop-in — is covered by the [migration guide](https://silo.pgsty.com/compatibility/migration/) and the [binary & service notes](https://silo.pgsty.com/compatibility/binary/). Every release ships checksums, SPDX SBOMs, Sigstore-signed manifests, and GitHub build attestations. Installation methods and verification commands are documented at [Download & Install](https://silo.pgsty.com/download/). To migrate from upstream MinIO, take over `minio.service` and `/etc/default/minio`, and preserve data ownership with a `/etc/systemd/system/silo.service.d/10-legacy-user.conf` drop-in. See the [migration guide](https://silo.pgsty.com/compatibility/migration/) and [binary & service notes](https://silo.pgsty.com/compatibility/binary/).
## Compatibility ## Compatibility
@@ -98,7 +98,7 @@ Every divergence from upstream is listed in the code-verified [compatibility aud
### TLS and Go upgrades ### TLS and Go upgrades
Server 20260916 restored Go's default key exchange; Server 20261018 retains it with Go 1.27.2. Server 20260916 restored Go's default key exchange; Server 20261109 retains it with Go 1.27.3.
TLS key exchange follows Go's defaults across the S3 listener, node links, TLS key exchange follows Go's defaults across the S3 listener, node links,
replication, identity providers, etcd, and external HTTP services. If an endpoint replication, identity providers, etcd, and external HTTP services. If an endpoint
cannot accept ML-KEM, `GODEBUG=tlsmlkem=0` disables the default hybrid exchanges cannot accept ML-KEM, `GODEBUG=tlsmlkem=0` disables the default hybrid exchanges
+6 -6
View File
@@ -36,7 +36,7 @@
上游停止社区发行后,Silo 为开源 MinIO 服务端维护一条持续可用的版本线:构建、软件包、多架构镜像、安全修复与完整 Web 控制台。Pigsty 在生产环境中用它承载 PostgreSQL 备份存储。 上游停止社区发行后,Silo 为开源 MinIO 服务端维护一条持续可用的版本线:构建、软件包、多架构镜像、安全修复与完整 Web 控制台。Pigsty 在生产环境中用它承载 PostgreSQL 备份存储。
它只遵循一条原则:**改名的是产品与交付物,不是协议与你的数据。** 其余内容都在 [silo.pgsty.com](https://silo.pgsty.com/zh/)。 它只遵循一条原则:**改名的是产品与交付物,不是协议与你的数据。** 完整文档发布在 [silo.pgsty.com](https://silo.pgsty.com/zh/)。
**相关项目:**[`pgsty/mc`](https://github.com/pgsty/mc) · [`pgsty/silo-console`](https://github.com/pgsty/silo-console) · [`pgsty/silo-pkg`](https://github.com/pgsty/silo-pkg) · [`pgsty/kes`](https://github.com/pgsty/kes) · [`pgsty/pigsty`](https://github.com/pgsty/pigsty) **相关项目:**[`pgsty/mc`](https://github.com/pgsty/mc) · [`pgsty/silo-console`](https://github.com/pgsty/silo-console) · [`pgsty/silo-pkg`](https://github.com/pgsty/silo-pkg) · [`pgsty/kes`](https://github.com/pgsty/kes) · [`pgsty/pigsty`](https://github.com/pgsty/pigsty)
@@ -44,10 +44,10 @@
<img src="https://silo.pgsty.com/images/silo-console/console-metrics-simple.webp" alt="Silo 控制台"> <img src="https://silo.pgsty.com/images/silo-console/console-metrics-simple.webp" alt="Silo 控制台">
</p> </p>
#### SILO 20261109 ### SILO 20261109
> [!WARNING] > [!WARNING]
> [SILO 20261109](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-11-09T00-00-00Z) 修复了缺陷与安全漏洞,强烈建议尽快安排升级,以降低安全风险。 > [SILO 20261109](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-11-09T00-00-00Z)(包版本 `20261109000000.0.0`,源码 `d998292387cc`)修复了多池生命周期、原生恢复、Object Lock、批处理授权与 IAM 等缺陷,内嵌 Console `v2.7.0` 并配套 `mcli 20261109`、`silo-pkg v3.16.0` 与 Go `1.27.3`。升级前请阅读[发布说明与升级要求](https://silo.pgsty.com/zh/blog/release/silo-20261109/)。
> [!NOTE] > [!NOTE]
> 2026-08-06,`pgsty/minio` 更名为 `pgsty/silo`,默认分支由 `master` 改为 `main`。最后一个 MinIO 品牌发布是 [`minio`](https://github.com/pgsty/silo/tree/minio) 分支上的 [`20260804`](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-08-04T00-00-00Z)。 > 2026-08-06,`pgsty/minio` 更名为 `pgsty/silo`,默认分支由 `master` 改为 `main`。最后一个 MinIO 品牌发布是 [`minio`](https://github.com/pgsty/silo/tree/minio) 分支上的 [`20260804`](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-08-04T00-00-00Z)。
@@ -98,9 +98,9 @@ Silo 保留 S3 与存储格式兼容性,包括既有 `MINIO_*` 环境变量、
### TLS 与 Go 升级 ### TLS 与 Go 升级
Server 20260916 已恢复 Go 默认密钥交换策略,20261018 在 Go 1.27.2 下保留该行为。 Server 20260916 恢复了 Go 默认密钥交换策略,Server 20261109 在 Go 1.27.3 下保留该行为。S3 监听端口、节点间连接、站点与桶复制、身份提供方、etcd 及外部 HTTP 服务均使用 Go 默认 TLS 密钥交换。若对端无法接受 ML-KEM,设置 `GODEBUG=tlsmlkem=0` 可在当前进程禁用默认混合密钥交换(证书校验保持开启);该选项不会禁用 ML-DSA 签名,也不能解决所有 TLS 连接重置问题,建议在升级不兼容对端后移除该临时配置。若仅新增的 SecP 混合组引发问题,可设置 `GODEBUG=tlssecpmlkem=0` 仅禁用这些组并保留 `X25519MLKEM768`。
`GODEBUG=tlsmlkem=0`、`tlssecpmlkem=0` 的适用范围、macOS 根证书来源变化,
以及 OIDC discovery 的诊断方法见 [Go 1.27 TLS 与 OIDC 指南](https://silo.pgsty.com/zh/blog/design/go127-tls-oidc-discovery/)。 在 Go 1.27 构建中,macOS 上设置 `SSL_CERT_FILE` 或 `SSL_CERT_DIR` 会以磁盘根证书和 Go 验证器替代系统钥匙串信任;陈旧或不完整的 CA 路径可能导致原先受信任的连接失败,清除继承的环境变量即可恢复钥匙串信任。配置的 `CAs` 目录中的显式证书仍会叠加加入所选根证书池。Go 1.27 二进制要求 macOS 13 或更高版本。详见 [Go 1.27 发行说明](https://go.dev/doc/go1.27)与 [Go 1.27 TLS 与 OIDC 发现指南](https://silo.pgsty.com/zh/blog/design/go127-tls-oidc-discovery/)。
## 文档归属 ## 文档归属
+1 -1
View File
@@ -38,7 +38,7 @@ For vulnerabilities in this fork:
1. Follow the fork-specific expectations in [VULNERABILITY_REPORT.md](VULNERABILITY_REPORT.md). 1. Follow the fork-specific expectations in [VULNERABILITY_REPORT.md](VULNERABILITY_REPORT.md).
2. Prefer this repository's [private GitHub security advisory](https://github.com/pgsty/silo/security/advisories/new) workflow. 2. Prefer this repository's [private GitHub security advisory](https://github.com/pgsty/silo/security/advisories/new) workflow.
3. If you cannot use GitHub private reporting, email [ron@pgsty.com](mailto:ron@pgsty.com) or rh@vonng.com 3. If you cannot use GitHub private reporting, email [ron@pgsty.com](mailto:ron@pgsty.com) or [rh@vonng.com](mailto:rh@vonng.com)
directly. Do not publish vulnerability details or exploit code in public directly. Do not publish vulnerability details or exploit code in public
issues or pull requests. issues or pull requests.
4. If you confirm the issue also affects upstream `minio/minio`, report it upstream separately. 4. If you confirm the issue also affects upstream `minio/minio`, report it upstream separately.
+4 -4
View File
@@ -885,16 +885,16 @@ func (a adminAPIHandlers) UpdateServiceAccount(w http.ResponseWriter, r *http.Re
return return
} }
// A request without a policy, or with a JSON null for one, keeps the
// policy the account has. An empty policy document is passed on as
// such: it is the request to remove the embedded policy.
var sp *policy.Policy var sp *policy.Policy
if len(updateReq.NewPolicy) > 0 { if len(updateReq.NewPolicy) > 0 && !bytes.Equal(updateReq.NewPolicy, []byte("null")) {
sp, err = policy.ParseConfigStrict(bytes.NewReader(updateReq.NewPolicy)) sp, err = policy.ParseConfigStrict(bytes.NewReader(updateReq.NewPolicy))
if err != nil { if err != nil {
writeErrorResponseJSON(ctx, w, toAdminAPIErr(ctx, err), r.URL) writeErrorResponseJSON(ctx, w, toAdminAPIErr(ctx, err), r.URL)
return return
} }
if sp.Version == "" && len(sp.Statements) == 0 {
sp = nil
}
} }
opts := updateServiceAccountOpts{ opts := updateServiceAccountOpts{
secretKey: updateReq.NewSecretKey, secretKey: updateReq.NewSecretKey,
+289
View File
@@ -0,0 +1,289 @@
// Copyright (c) 2026 Denis Babchuk
//
// This file is part of Silo Object Storage stack
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package cmd
import (
"context"
"encoding/json"
"maps"
"net/http"
"slices"
"strings"
"github.com/minio/minio-go/v7/pkg/set"
"github.com/minio/minio-go/v7/pkg/tags"
"github.com/minio/minio/internal/auth"
"github.com/minio/minio/internal/logger"
"github.com/pgsty/silo-pkg/v3/policy"
"github.com/pgsty/silo-pkg/v3/policy/condition"
)
// authorizeRequestWithUnreadTags authorizes action on an object whose stored
// tags could not be read because reading the object failed with readErr.
//
// Without the tags, an Allow statement that tests s3:ExistingObjectTag cannot
// match, so a caller allowed only by such a statement would get AccessDenied,
// which clients do not retry, for a transient server error such as lost read
// quorum. For a server error the request is therefore also authorized as if
// the object carried tags that the policies ask for: a caller that some tags
// let through gets the error, a caller that no tags let through still gets
// AccessDenied. For any other error, NoSuchKey among them, the tags stay empty.
func authorizeRequestWithUnreadTags(ctx context.Context, r *http.Request, action policy.Action, readErr error) APIErrorCode {
s3Err := authorizeRequest(ctx, r, action)
if s3Err == ErrNone || toAPIError(ctx, readErr).HTTPStatusCode < http.StatusInternalServerError {
return s3Err
}
for _, existingTags := range policyExistingObjectTags(ctx, r, action) {
if authorizeRequestWithExistingTags(ctx, r, action, existingTags) == ErrNone {
return ErrNone
}
}
return s3Err
}
// maxExistingTagCandidates bounds the tag sets tried for one request; each
// one costs an authorization.
const maxExistingTagCandidates = 64
// policyExistingObjectTags returns tag sets under which the policies that
// authorize the request may let action through on its object: the bucket
// policy for an anonymous request, else the cached IAM policies and, as a
// second policy set, the session policy of the credential. A tag set passes
// the s3:ExistingObjectTag tests of one Allow statement of each policy set and,
// as far as the same tags allow, keeps the Deny statements that test tags from
// matching. The tag sets are only candidates; the regular authorization
// decides what they allow.
func policyExistingObjectTags(ctx context.Context, r *http.Request, action policy.Action) []string {
reqInfo := logger.GetReqInfo(ctx)
if reqInfo == nil {
return nil
}
bucket, object := reqInfo.BucketName, reqInfo.ObjectName
var (
allows [][]map[string]string // per policy set, the tags its Allow statements ask for
denies []map[string]string // tags that keep a Deny statement from matching
)
// add files the tags of a statement that applies to action on the
// object, whatever its conditions.
add := func(effect policy.Effect, conditions condition.Functions, values map[string][]string, fromSet *[]map[string]string) {
if tagMap, ok := existingObjectTagsFor(effect, conditions, values); ok {
if effect == policy.Allow {
*fromSet = append(*fromSet, tagMap)
} else {
denies = append(denies, tagMap)
}
}
}
switch {
case reqInfo.Cred.AccessKey == "":
p, err := globalPolicySys.Get(bucket)
if err != nil {
return nil
}
values := getConditionValues(r, reqInfo.Region, auth.AnonymousCredentials)
var fromSet []map[string]string
for _, st := range p.Statements {
match := st
match.Effect, match.Conditions = policy.Allow, nil
if match.IsAllowed(policy.BucketPolicyArgs{Action: action, BucketName: bucket, ObjectName: object, ConditionValues: values}) {
add(st.Effect, st.Conditions, values, &fromSet)
}
}
allows = append(allows, fromSet)
case newGlobalAuthZPluginFn() == nil && globalIAMSys.Initialized():
values := getConditionValues(r, "", reqInfo.Cred)
// From the cache only: reading the drives may be what just failed.
docs, _ := globalIAMSys.store.listPolicyDocs(ctx, "")
policySets := [][]policy.Policy{nil}
for _, name := range slices.Sorted(maps.Keys(docs)) {
policySets[0] = append(policySets[0], docs[name].Policy)
}
if sp, ok := reqInfo.Cred.Claims[sessionPolicyNameExtracted].(string); ok {
if p, err := policy.ParseConfig(strings.NewReader(sp)); err == nil {
policySets = append(policySets, []policy.Policy{*p})
}
}
for _, policies := range policySets {
var fromSet []map[string]string
for _, p := range policies {
for _, st := range p.Statements {
match := st
match.Effect, match.Conditions = policy.Allow, nil
if match.IsAllowed(policy.Args{Action: action, BucketName: bucket, ObjectName: object, ConditionValues: values}) {
add(st.Effect, st.Conditions, values, &fromSet)
}
}
}
allows = append(allows, fromSet)
}
}
return combineExistingObjectTags(allows, denies)
}
// combineExistingObjectTags merges the tags of one Allow statement of each
// policy set, or of none for a set that may allow without tags, and tries each
// result also with the tags that keep the Deny statements from matching. The
// first tags win a conflict; the authorization of the result settles it.
func combineExistingObjectTags(allows [][]map[string]string, denies []map[string]string) []string {
merge := func(parts ...map[string]string) map[string]string {
merged := make(map[string]string)
for _, part := range parts {
for k, v := range part {
if _, ok := merged[k]; !ok {
merged[k] = v
}
}
}
return merged
}
combos := [][]map[string]string{nil}
for _, fromSet := range allows {
// Many statements ask for the same tags.
seen := set.NewStringSet()
fromSet = slices.DeleteFunc(slices.Clone(fromSet), func(tagMap map[string]string) bool {
t, err := tags.MapToObjectTags(tagMap)
if err != nil || seen.Contains(t.String()) {
return true
}
seen.Add(t.String())
return false
})
next := make([][]map[string]string, 0, len(combos)*(len(fromSet)+1))
for _, combo := range combos {
next = append(next, combo)
for _, tagMap := range fromSet {
next = append(next, append(slices.Clip(combo), tagMap))
}
}
combos = next[:min(len(next), maxExistingTagCandidates)]
}
deny := merge(denies...)
found := set.NewStringSet()
var candidates []string
for _, combo := range combos {
for _, tagMap := range []map[string]string{merge(combo...), merge(append(slices.Clip(combo), deny)...)} {
t, err := tags.MapToObjectTags(tagMap)
if len(tagMap) == 0 || err != nil || found.Contains(t.String()) {
continue
}
found.Add(t.String())
if candidates = append(candidates, t.String()); len(candidates) == maxExistingTagCandidates {
return candidates
}
}
}
return candidates
}
// existingObjectTagsFor returns tags under which the s3:ExistingObjectTag tests
// of conditions all pass, for an Allow statement, or one of them fails, for a
// Deny statement; a tag left out is absent. It returns false if the conditions
// test no tag, or if no such tags were found among those tried for each tag:
// none, the values the tests name, with policy variables taken from values and
// wildcards filled, and the fillers "" and "_". The policy engine itself
// evaluates the tests for each of them.
func existingObjectTagsFor(effect policy.Effect, conditions condition.Functions, values map[string][]string) (map[string]string, bool) {
// Each function tests one key, so the tests of one tag pass or fail
// whatever the other tags are.
byKey := make(map[string]condition.Functions)
for _, f := range conditions {
for key := range condition.NewFunctions(f).Keys() {
if key.Is(condition.ExistingObjectTag) {
byKey[key.Name()] = append(byKey[key.Name()], f)
}
}
}
if len(byKey) == 0 {
return nil, false
}
tagMap := make(map[string]string)
for _, key := range slices.Sorted(maps.Keys(byKey)) {
tagKey := strings.TrimPrefix(key, condition.ExistingObjectTag.Name()+"/")
value, present, found := existingObjectTagValue(key, tagKey, byKey[key], values, effect == policy.Allow)
switch {
case effect == policy.Deny && found:
if present {
tagMap[tagKey] = value
}
return tagMap, true
case effect == policy.Allow && !found:
return nil, false
case present:
tagMap[tagKey] = value
}
}
if effect == policy.Deny {
return nil, false
}
return tagMap, true
}
// existingObjectTagValue looks for a value of tag tagKey, or its absence,
// under which tests, the functions that test the condition key key, evaluate
// to want.
func existingObjectTagValue(key, tagKey string, tests condition.Functions, values map[string][]string, want bool) (value string, present, found bool) {
defer delete(values, key)
delete(values, key)
if tests.Evaluate(values) == want {
return "", false, true
}
// The functions keep their values unexported; JSON exposes them.
buf, err := json.Marshal(tests)
if err != nil {
return "", false, false
}
var byName map[string]map[string]condition.ValueSet
if err = json.Unmarshal(buf, &byName); err != nil {
return "", false, false
}
var tried []string
for name, byKey := range byName {
if name == "Null" {
continue // its operand says whether the tag is there, not a value
}
for _, valueSet := range byKey {
for _, v := range valueSet.ToSlice() {
s := substituteConditionVariables(v.String(), values)
tried = append(tried, s, strings.NewReplacer("*", "", "?", "_").Replace(s))
}
}
}
slices.Sort(tried)
for _, v := range append(slices.Compact(tried), "", "_") {
if _, err := tags.MapToObjectTags(map[string]string{tagKey: v}); err != nil {
continue
}
values[key] = []string{v}
if tests.Evaluate(values) == want {
return v, true, true
}
}
return "", false, false
}
// substituteConditionVariables resolves the policy variables in v from the
// condition values as the policy engine does for the values of a condition;
// silo-pkg keeps its own function for this unexported.
func substituteConditionVariables(v string, values map[string][]string) string {
for _, key := range condition.CommonKeys {
if rvalues, ok := values[key.Name()]; ok && len(rvalues) > 0 && rvalues[0] != "" {
v = strings.ReplaceAll(v, key.VarName(), rvalues[0])
}
}
return v
}
+305
View File
@@ -0,0 +1,305 @@
// Copyright (c) 2026 Denis Babchuk
//
// This file is part of Silo Object Storage stack
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package cmd
import (
"bytes"
"context"
"encoding/json"
"encoding/xml"
"fmt"
"io"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"testing"
"github.com/minio/minio-go/v7/pkg/tags"
"github.com/minio/minio/internal/auth"
xhttp "github.com/minio/minio/internal/http"
"github.com/pgsty/silo-pkg/v3/policy"
"github.com/pgsty/silo-pkg/v3/policy/condition"
)
// objectReadFault takes the first n drives of the set that holds object
// offline for the metadata reads of that object only, so that IAM and bucket
// metadata stay readable. It returns a function that restores the drives.
func objectReadFault(z *erasureServerPools, bucket, object string, n int) func() {
pool := z.serverPools[0]
setIndex := pool.getHashedSetIndex(object)
pool.erasureDisksMu.Lock()
defer pool.erasureDisksMu.Unlock()
previous := pool.erasureDisks[setIndex]
disks := append([]StorageAPI(nil), previous...)
for i := range n {
disks[i] = consistencyReadFaultDisk{StorageAPI: disks[i], bucket: bucket, object: object}
}
pool.erasureDisks[setIndex] = disks
return func() {
pool.erasureDisksMu.Lock()
pool.erasureDisks[setIndex] = previous
pool.erasureDisksMu.Unlock()
}
}
// A caller allowed to read an object only through s3:ExistingObjectTag must
// get the storage error, which clients retry, and not AccessDenied, when the
// object's tags cannot be read because read quorum is lost. A caller that no
// tags would let through must still get AccessDenied, and nothing changes
// while the tags can be read.
func TestExistingObjectTagAuthReadQuorumLost(t *testing.T) {
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
obj, dirs, err := prepareErasure16(ctx)
if err != nil {
t.Fatal(err)
}
z := obj.(*erasureServerPools)
previous := newObjectLayerFn()
setObjectLayer(z)
defer func() {
z.Shutdown(context.Background())
removeRoots(dirs)
setObjectLayer(previous)
}()
bucket, router, err := initAPIHandlerTest(ctx, z, nil, MakeBucketOptions{})
if err != nil {
t.Fatal(err)
}
statement := func(effect, resource, conditions string) string {
st := fmt.Sprintf(`{"Effect": %q, "Action": ["s3:GetObject", "s3:PutObjectTagging"], "Resource": ["arn:aws:s3:::%s/%s"]`,
effect, bucket, resource)
if conditions != "" {
st += `, "Condition": ` + conditions
}
return st + "}"
}
tagStatement := func(effect, resource, value string) string {
return statement(effect, resource, fmt.Sprintf(`{"StringEquals": {"s3:ExistingObjectTag/k": %q}}`, value))
}
newUser := func(statements ...string) auth.Credentials {
return newDeleteAuthzPolicyUser(t, "", bucket, "["+strings.Join(statements, ",")+"]")
}
newServiceAccount := func(parent auth.Credentials, statements string) auth.Credentials {
sessionPolicy, err := policy.ParseConfig(strings.NewReader(`{"Version": "2012-10-17", "Statement": [` + statements + `]}`))
if err != nil {
t.Fatal(err)
}
cred, _, err := globalIAMSys.NewServiceAccount(ctx, parent.AccessKey, nil, newServiceAccountOpts{sessionPolicy: sessionPolicy})
if err != nil {
t.Fatal(err)
}
return cred
}
type user struct {
name string
creds auth.Credentials
// The status with the object's tags readable and with read quorum
// lost: 200 and 503 for a caller that some tags let through, 403
// and 403 for a caller that none do.
healthy, quorumLost int
}
allowedByTags := func(name string, creds auth.Credentials) user {
return user{name, creds, http.StatusOK, http.StatusServiceUnavailable}
}
deniedByAnyTags := func(name string, creds auth.Credentials) user {
return user{name, creds, http.StatusForbidden, http.StatusForbidden}
}
// Each case below that tests a tag key of its own uses a key no other
// policy asks for.
ownerCondition := `{"StringEquals": {"s3:ExistingObjectTag/owner": "${aws:username}"}}`
owner := newUser(statement("Allow", "*", ownerCondition))
users := []user{
// Allowed only while the object carries k=v.
allowedByTags("tagged", newUser(tagStatement("Allow", "*", "v"))),
// Allowed while the object carries a tag np, whatever its value.
allowedByTags("nullFalse", newUser(statement("Allow", "*", `{"Null": {"s3:ExistingObjectTag/np": "false"}}`))),
// Allowed while a tag ne is there and is not w.
allowedByTags("notEquals", newUser(statement("Allow", "*",
`{"Null": {"s3:ExistingObjectTag/ne": "false"}, "StringNotEquals": {"s3:ExistingObjectTag/ne": "w"}}`))),
// Allowed while every value of tag fa differs from w; that needs a tag fa.
allowedByTags("forAllNotEquals", newUser(statement("Allow", "*", `{"ForAllValues:StringNotEquals": {"s3:ExistingObjectTag/fa": "w"}}`))),
// Allowed while the object is tagged with the caller's name.
allowedByTags("owner", owner),
// Allowed, but denied unless the object carries du=1.
allowedByTags("denyUnlessTagged", newUser(statement("Allow", "*", ""),
statement("Deny", "*", `{"StringNotEquals": {"s3:ExistingObjectTag/du": "1"}}`))),
// A service account whose parent asks for k=v and whose own policy
// asks for j=w: only an object carrying both lets it through.
allowedByTags("serviceAccount", newServiceAccount(newUser(tagStatement("Allow", "*", "v")),
statement("Allow", "*", `{"StringEquals": {"s3:ExistingObjectTag/j": "w"}}`))),
// Allowed only for k=w: the decision still depends on the tags.
{"otherValue", newUser(tagStatement("Allow", "*", "w")), http.StatusForbidden, http.StatusServiceUnavailable},
// Allowed for k=v under another prefix only.
deniedByAnyTags("otherPrefix", newUser(tagStatement("Allow", "other/*", "v"))),
// Allowed and denied for k=v alike.
deniedByAnyTags("allowDeny", newUser(tagStatement("Allow", "*", "v"), tagStatement("Deny", "*", "v"))),
// Allowed only without a tag nc that is 1.
deniedByAnyTags("nullConflict", newUser(statement("Allow", "*",
`{"Null": {"s3:ExistingObjectTag/nc": "true"}, "StringEquals": {"s3:ExistingObjectTag/nc": "1"}}`))),
// Tagged with the caller's name, under another prefix only.
deniedByAnyTags("ownerPrefix", newUser(statement("Allow", "other/*", ownerCondition))),
// Denied unless the object carries db=1, and denied if it does.
deniedByAnyTags("denyBoth", newUser(statement("Allow", "*", ""),
statement("Deny", "*", `{"StringNotEquals": {"s3:ExistingObjectTag/db": "1"}}`),
statement("Deny", "*", `{"StringEquals": {"s3:ExistingObjectTag/db": "1"}}`))),
// The parent asks for k=v and the service account for k=x.
deniedByAnyTags("serviceAccountConflict", newServiceAccount(newUser(tagStatement("Allow", "*", "v")),
tagStatement("Allow", "*", "x"))),
}
const object, missing = "tagged", "missing"
if _, err := z.PutObject(ctx, bucket, object, mustGetPutObjReader(t, bytes.NewReader([]byte("data")), 4, "", ""),
ObjectOptions{UserDefined: map[string]string{xhttp.AmzObjectTagging: "k=v&j=w&np=1&ne=1&fa=1&du=1&owner=" + owner.AccessKey}}); err != nil {
t.Fatal(err)
}
// Anonymous reads are allowed by the bucket policy while the object carries k=v.
if _, err := globalBucketMetadataSys.Update(ctx, bucket, bucketPolicyConfig, []byte(`{"Version": "2012-10-17", "Statement": [
{"Effect": "Allow", "Principal": {"AWS": ["*"]}, "Action": ["s3:GetObject"], "Resource": ["arn:aws:s3:::`+bucket+`/*"],
"Condition": {"StringEquals": {"s3:ExistingObjectTag/k": "v"}}}]}`)); err != nil {
t.Fatal(err)
}
type request struct {
name string
method, target string
header map[string]string
body []byte
}
objectURL := getGetObjectURL("", bucket, object)
get := request{"GET", http.MethodGet, objectURL, nil, nil}
head := request{"HEAD", http.MethodHead, objectURL, nil, nil}
taggingURL := makeTestTargetURL("", bucket, object, url.Values{"tagging": []string{""}})
taggingBody := []byte(`<Tagging><TagSet><Tag><Key>k</Key><Value>v</Value></Tag></TagSet></Tagging>`)
putTagging := request{"PutObjectTagging", http.MethodPut, taggingURL, nil, taggingBody}
check := func(stage string, creds auth.Credentials, name string, req request, want int) {
t.Helper()
var rec *httptest.ResponseRecorder
if creds.AccessKey == "" {
anonymous, err := newTestRequest(req.method, req.target, 0, nil)
if err != nil {
t.Fatal(err)
}
rec = httptest.NewRecorder()
router.ServeHTTP(rec, anonymous)
} else {
var body io.ReadSeeker
if req.body != nil {
body = bytes.NewReader(req.body)
}
signed, err := newTestSignedRequestV4(req.method, req.target, int64(len(req.body)), body, creds.AccessKey, creds.SecretKey, req.header)
if err != nil {
t.Fatal(err)
}
rec = httptest.NewRecorder()
router.ServeHTTP(rec, signed)
}
if rec.Code != want {
var apiErr APIErrorResponse
_ = xml.Unmarshal(rec.Body.Bytes(), &apiErr)
t.Errorf("%s: %s by %s: status %d %s, want %d", stage, req.name, name, rec.Code, apiErr.Code, want)
}
}
var anonymous auth.Credentials
for _, u := range users {
check("healthy", u.creds, u.name, get, u.healthy)
}
check("healthy", anonymous, "anonymous", get, http.StatusOK)
// The tags were read and allow the request: the range error is the answer.
check("healthy", users[0].creds, users[0].name,
request{"GET bytes=100-200", http.MethodGet, objectURL, map[string]string{xhttp.Range: "bytes=100-200"}, nil},
http.StatusRequestedRangeNotSatisfiable)
// Without an object there are no tags: AccessDenied, as before.
missingURL := getGetObjectURL("", bucket, missing)
check("missing key", users[0].creds, users[0].name, request{"GET", http.MethodGet, missingURL, nil, nil}, http.StatusForbidden)
check("missing key", users[0].creds, users[0].name, request{"HEAD", http.MethodHead, missingURL, nil, nil}, http.StatusForbidden)
defer objectReadFault(z, bucket, object, 10)()
if _, err := z.GetObjectInfo(ctx, bucket, object, ObjectOptions{}); !isErrReadQuorum(err) {
t.Fatalf("GetObjectInfo with 10 of 16 drives failing: %v, want InsufficientReadQuorum", err)
}
for _, req := range []request{get, head, putTagging} {
for _, u := range users {
check("read quorum lost", u.creds, u.name, req, u.quorumLost)
}
}
check("read quorum lost", anonymous, "anonymous", get, http.StatusServiceUnavailable)
check("read quorum lost", anonymous, "anonymous", head, http.StatusServiceUnavailable)
}
func TestExistingObjectTagsFor(t *testing.T) {
values := map[string][]string{"username": {"alice"}, "sub": {"u1"}}
for _, tc := range []struct {
effect policy.Effect
conditions string
want string // the tags to set, encoded as in x-amz-tagging
ok bool // false: no tags for this statement
}{
{policy.Allow, `{"StringEquals": {"s3:ExistingObjectTag/k": "v"}}`, "k=v", true},
{policy.Allow, `{"StringEqualsIgnoreCase": {"s3:ExistingObjectTag/k": "V"}, "IpAddress": {"aws:SourceIp": "10.0.0.0/8"}}`, "k=V", true},
{policy.Allow, `{"ForAnyValue:StringEquals": {"s3:ExistingObjectTag/k": ["b", "a"]}}`, "k=a", true},
{policy.Allow, `{"StringLike": {"s3:ExistingObjectTag/k": "a?c*"}}`, "k=a_c", true},
{policy.Allow, `{"StringEquals": {"s3:ExistingObjectTag/b": "2", "s3:ExistingObjectTag/a": "1"}}`, "a=1&b=2", true},
// Passed without the tag.
{policy.Allow, `{"StringNotEquals": {"s3:ExistingObjectTag/k": "v"}}`, "", true},
{policy.Allow, `{"Null": {"s3:ExistingObjectTag/k": "true"}}`, "", true},
// Null "false" asks for the tag, with any value.
{policy.Allow, `{"Null": {"s3:ExistingObjectTag/k": "false"}}`, "k=", true},
{policy.Allow, `{"Null": {"s3:ExistingObjectTag/k": "false"}, "StringNotEquals": {"s3:ExistingObjectTag/k": ["", "w"]}}`, "k=_", true},
{policy.Allow, `{"Null": {"s3:ExistingObjectTag/k": "false"}, "StringNotLike": {"s3:ExistingObjectTag/k": "*w"}}`, "k=", true},
{policy.Allow, `{"ForAllValues:StringNotEquals": {"s3:ExistingObjectTag/k": "w"}}`, "k=", true},
{policy.Allow, `{"Null": {"s3:ExistingObjectTag/k": "true"}, "StringEquals": {"s3:ExistingObjectTag/k": "v"}}`, "", false},
// Policy variables take the values of the request.
{policy.Allow, `{"StringEquals": {"s3:ExistingObjectTag/owner": "${aws:username}"}}`, "owner=alice", true},
{policy.Allow, `{"StringLike": {"s3:ExistingObjectTag/owner": "${jwt:sub}-*"}}`, "owner=u1-", true},
{policy.Allow, `{"StringEquals": {"s3:ExistingObjectTag/owner": "${ldap:user}"}}`, "", false},
// A Deny is kept from matching.
{policy.Deny, `{"StringNotEquals": {"s3:ExistingObjectTag/k": "v"}}`, "k=v", true},
{policy.Deny, `{"Null": {"s3:ExistingObjectTag/k": "true"}}`, "k=", true},
{policy.Deny, `{"StringEquals": {"s3:ExistingObjectTag/k": "v"}}`, "", true},
{policy.Deny, `{"StringEquals": {"s3:prefix": "v"}}`, "", false},
// No tag tests.
{policy.Allow, `{"StringEquals": {"s3:prefix": "v"}}`, "", false},
{policy.Allow, "", "", false},
} {
var conditions condition.Functions // nil: a statement without a Condition
if tc.conditions != "" {
if err := json.Unmarshal([]byte(tc.conditions), &conditions); err != nil {
t.Fatalf("%s: %v", tc.conditions, err)
}
}
tagMap, ok := existingObjectTagsFor(tc.effect, conditions, values)
got := ""
if len(tagMap) > 0 {
encoded, err := tags.MapToObjectTags(tagMap)
if err != nil {
t.Fatalf("%s: %v", tc.conditions, err)
}
got = encoded.String()
}
if got != tc.want || ok != tc.ok {
t.Errorf("%s %s: got %q %v, want %q %v", tc.effect, tc.conditions, got, ok, tc.want, tc.ok)
}
}
}
+13
View File
@@ -425,6 +425,19 @@ func authorizeRequest(ctx context.Context, r *http.Request, action policy.Action
return authorizeRequestWithExistingTags(ctx, r, action, "") return authorizeRequestWithExistingTags(ctx, r, action, "")
} }
// authorizeBucketRequest authorizes a bucket action, such as s3:ListBucket,
// on the bucket of a request made for one of its objects.
func authorizeBucketRequest(ctx context.Context, r *http.Request, action policy.Action) APIErrorCode {
reqInfo := logger.GetReqInfo(ctx)
if reqInfo == nil {
return ErrAccessDenied
}
objectName := reqInfo.ObjectName
reqInfo.ObjectName = ""
defer func() { reqInfo.ObjectName = objectName }()
return authorizeRequest(ctx, r, action)
}
func deleteObjectAction(versionID string) policy.Action { func deleteObjectAction(versionID string) policy.Action {
if versionID != "" { if versionID != "" {
return policy.DeleteObjectVersionAction return policy.DeleteObjectVersionAction
+1 -1
View File
@@ -330,7 +330,7 @@ func TestBatchReviewRotationKeepsMigrationCleanup(t *testing.T) {
t.Errorf("rotation changed a draining source after its version was copied (rotation error: %v)", rotationErr) t.Errorf("rotation changed a draining source after its version was copied (rotation error: %v)", rotationErr)
} }
snapshot := []FileInfo{{VersionID: before.VersionID, ModTime: beforeInfo.ModTime}} snapshot := []FileInfo{{VersionID: before.VersionID, ModTime: beforeInfo.ModTime}}
if err := z.removeMovedVersions(t.Context(), bucket, object, 0, snapshot, nil); err != nil { if _, err := z.removeMovedVersions(t.Context(), bucket, object, 0, snapshot, nil); err != nil {
t.Fatalf("rotation prevented migration cleanup: %v", err) t.Fatalf("rotation prevented migration cleanup: %v", err)
} }
if _, err := z.serverPools[0].GetObjectInfo(t.Context(), bucket, object, ObjectOptions{VersionID: before.VersionID}); !isErrObjectNotFound(err) && !isErrVersionNotFound(err) { if _, err := z.serverPools[0].GetObjectInfo(t.Context(), bucket, object, ObjectOptions{VersionID: before.VersionID}); !isErrObjectNotFound(err) && !isErrVersionNotFound(err) {
+156
View File
@@ -0,0 +1,156 @@
// Copyright (c) 2026 PGSTY
// SPDX-License-Identifier: AGPL-3.0-or-later
package cmd
import (
"bytes"
"encoding/xml"
"net/http"
"net/http/httptest"
"slices"
"testing"
"github.com/minio/minio/internal/auth"
)
// A bulk delete joins each name to the request bucket on the drives, so a name
// with a ".." segment names a path in another bucket or in the meta bucket.
// Such a name is the error of its own entry and reaches no drive, and the
// other entries of the request are deleted as usual. This holds for the one
// bulk lock of a single node, for a pool that locks per set, and for more than
// one pool.
func TestPoolsBulkDeleteInvalidNames(t *testing.T) {
for _, topology := range []struct {
name string
new func(t *testing.T) (*erasureServerPools, string)
}{
{"single node", func(t *testing.T) (*erasureServerPools, string) { return bulkDeletePools(t, 1) }},
{"set locks", func(t *testing.T) (*erasureServerPools, string) { return ldPool(t, ldLocal) }},
{"two pools", func(t *testing.T) (*erasureServerPools, string) { return bulkDeletePools(t, 2) }},
} {
t.Run(topology.name, func(t *testing.T) {
z, bucket := topology.new(t)
ctx := t.Context()
other := "bulk-delete-other"
if err := z.MakeBucket(ctx, other, MakeBucketOptions{VersioningEnabled: true}); err != nil {
t.Fatal(err)
}
old := batchPutVersion(t, z, other, "kept", nil)
latest := batchPutVersion(t, z, other, "kept", nil)
batchPutVersion(t, z, other, "visible", nil)
metaKey := pathJoin(bucketMetaPrefix, other, bucketMetadataFile)
if !batchVersionExists(t, z, minioMetaBucket, metaKey, "") {
t.Fatal("the bucket metadata is not an object of the meta bucket")
}
for _, key := range []string{"own", "dir/"} {
if _, err := z.PutObject(ctx, bucket, key, mustGetPutObjReader(t, bytes.NewReader(nil), 0, "", ""), ObjectOptions{}); err != nil {
t.Fatal(err)
}
}
names := []ObjectToDelete{
{ObjectV: ObjectV{ObjectName: "../" + other + "/kept", VersionID: old.VersionID}}, // purge in another bucket
{ObjectV: ObjectV{ObjectName: "own"}},
{ObjectV: ObjectV{ObjectName: "a/../../" + other + "/visible"}}, // marker in another bucket
{ObjectV: ObjectV{ObjectName: "../" + minioMetaBucket + "/" + metaKey}}, // marker on bucket metadata
{ObjectV: ObjectV{ObjectName: "dir/"}},
{ObjectV: ObjectV{ObjectName: ""}},
}
invalid := map[int]bool{0: true, 2: true, 3: true, 5: true}
// Without a version ID these options ask for a delete marker, the
// one write a bulk delete makes.
deleted, errs := z.DeleteObjects(ctx, bucket, slices.Clone(names), ObjectOptions{Versioned: true})
for i, name := range names {
_, rejected := errs[i].(ObjectNameInvalid)
if rejected != invalid[i] || !invalid[i] && errs[i] != nil {
t.Errorf("%d %q: error = %v, rejected as invalid = %v, want %v", i, name.ObjectName, errs[i], rejected, invalid[i])
}
if deleted[i].ObjectName != name.ObjectName || deleted[i].found == invalid[i] {
t.Errorf("%d %q: result = %+v", i, name.ObjectName, deleted[i])
}
}
for _, version := range []string{old.VersionID, latest.VersionID} {
if !batchVersionExists(t, z, other, "kept", version) {
t.Errorf("version %q of the object in the other bucket was removed", version)
}
}
if !batchVersionExists(t, z, other, "visible", "") {
t.Error("the object of the other bucket is hidden by a marker")
}
if !batchVersionExists(t, z, minioMetaBucket, metaKey, "") {
t.Error("the metadata of the other bucket is hidden by a marker")
}
for _, key := range []string{"own", "dir/"} {
if batchVersionExists(t, z, bucket, key, "") {
t.Errorf("%q of the request bucket was not deleted", key)
}
}
})
}
}
// The names of an S3 DeleteObjects request come from its body, which the path
// checks of the request URL do not see. A user allowed to delete in one bucket
// reaches nothing outside it through such a name: a version under COMPLIANCE
// retention in another bucket stays, and no delete marker lands on that object
// or on the stored metadata of its bucket.
func TestBulkDeleteObjectLockNameOutsideBucket(t *testing.T) {
defer DetectTestLeak(t)()
ExecObjectLayerAPITest(ExecObjectLayerAPITestArgs{
t: t, endpoints: []string{"DeleteMultipleObjects"},
makeBucketOptions: MakeBucketOptions{VersioningEnabled: true},
objAPITest: func(obj ObjectLayer, instance, bucket string, router http.Handler, _ auth.Credentials, t *testing.T) {
ctx := t.Context()
user := newObjectAttributesAuthzUser(t, instance, bucket, `"s3:DeleteObject","s3:DeleteObjectVersion"`)
locked := getRandomBucketName()
if err := obj.MakeBucket(ctx, locked, MakeBucketOptions{VersioningEnabled: true, LockEnabled: true}); err != nil {
t.Fatal(err)
}
enableBucketObjectLock(t, locked)
worm := batchPutVersion(t, obj, locked, "secret", batchLockMeta("COMPLIANCE"))
batchPutVersion(t, obj, bucket, "own", nil)
metaKey := pathJoin(bucketMetaPrefix, locked, bucketMetadataFile)
outside := []ObjectToDelete{
{ObjectV: ObjectV{ObjectName: "../" + locked + "/secret", VersionID: worm.VersionID}},
{ObjectV: ObjectV{ObjectName: "../" + locked + "/secret"}},
{ObjectV: ObjectV{ObjectName: "../" + minioMetaBucket + "/" + metaKey}},
}
body := encodeResponse(DeleteObjectsRequest{Objects: append(slices.Clone(outside), ObjectToDelete{ObjectV: ObjectV{ObjectName: "own"}})})
req, err := newTestSignedRequestV4(http.MethodPost, getDeleteMultipleObjectsURL("", bucket), int64(len(body)), bytes.NewReader(body),
user.AccessKey, user.SecretKey, nil)
if err != nil {
t.Fatal(err)
}
rec := httptest.NewRecorder()
router.ServeHTTP(rec, req)
var resp DeleteObjectsResponse
if rec.Code != http.StatusOK || xml.Unmarshal(rec.Body.Bytes(), &resp) != nil {
t.Fatalf("%s: status %d: %s", instance, rec.Code, rec.Body.String())
}
if len(resp.DeletedObjects) != 1 || resp.DeletedObjects[0].ObjectName != "own" || len(resp.Errors) != len(outside) {
t.Errorf("%s: want the three names outside the bucket refused and \"own\" deleted: %s", instance, rec.Body.String())
}
for _, e := range resp.Errors {
if e.Code != errorCodes[ErrInvalidObjectName].Code {
t.Errorf("%s: %q: code %s, want %s", instance, e.Key, e.Code, errorCodes[ErrInvalidObjectName].Code)
}
}
if !batchVersionExists(t, obj, locked, "secret", worm.VersionID) {
t.Errorf("%s: the version under COMPLIANCE retention in the other bucket was removed", instance)
}
if !batchVersionExists(t, obj, locked, "secret", "") {
t.Errorf("%s: a delete marker hides the object of the other bucket", instance)
}
if !batchVersionExists(t, obj, minioMetaBucket, metaKey, "") {
t.Errorf("%s: a delete marker hides the stored metadata of the other bucket", instance)
}
if batchVersionExists(t, obj, bucket, "own", "") {
t.Errorf("%s: the object of the request bucket was not deleted", instance)
}
},
})
}
+13 -9
View File
@@ -271,16 +271,17 @@ func (z *erasureServerPools) applyMigrationExpiry(ctx context.Context, bucket, o
// in the source until the ordinary expiry path has deleted them: that path // in the source until the ordinary expiry path has deleted them: that path
// owns replication state and tier cleanup, and it re-evaluates the current // owns replication state and tier cleanup, and it re-evaluates the current
// lifecycle rule. A migration cleanup must never turn an asynchronous ILM // lifecycle rule. A migration cleanup must never turn an asynchronous ILM
// decision into an unconditional physical purge. // decision into an unconditional physical purge. It returns how many source
func (z *erasureServerPools) removeMovedVersions(ctx context.Context, bucket, object string, source int, versions []FileInfo, expired []migrationExpiredVersion) error { // versions it deleted.
func (z *erasureServerPools) removeMovedVersions(ctx context.Context, bucket, object string, source int, versions []FileInfo, expired []migrationExpiredVersion) (removed int, err error) {
if err := z.applyMigrationExpiry(ctx, bucket, object, source, expired); err != nil { if err := z.applyMigrationExpiry(ctx, bucket, object, source, expired); err != nil {
return err return 0, err
} }
object = encodeDirObject(object) object = encodeDirObject(object)
lk := z.NewNSLock(bucket, object) lk := z.NewNSLock(bucket, object)
lkctx, err := lk.GetLock(ctx, globalOperationTimeout) lkctx, err := lk.GetLock(ctx, globalOperationTimeout)
if err != nil { if err != nil {
return err return 0, err
} }
ctx = lkctx.Context() ctx = lkctx.Context()
defer lk.Unlock(lkctx) defer lk.Unlock(lkctx)
@@ -294,7 +295,7 @@ func (z *erasureServerPools) removeMovedVersions(ctx context.Context, bucket, ob
continue continue
} }
if err != nil { if err != nil {
return err return removed, err
} }
var src *ObjectInfo var src *ObjectInfo
var retained []PoolObjInfo var retained []PoolObjInfo
@@ -309,7 +310,7 @@ func (z *erasureServerPools) removeMovedVersions(ctx context.Context, bucket, ob
continue continue
} }
if !src.ModTime.Equal(version.ModTime) || src.DeleteMarker != version.Deleted { if !src.ModTime.Equal(version.ModTime) || src.DeleteMarker != version.Deleted {
return PreConditionFailed{} return removed, PreConditionFailed{}
} }
if isExpiredVersion(expired, version) { if isExpiredVersion(expired, version) {
// The logical expiry path owns this version. Keeping the source is // The logical expiry path owns this version. Keeping the source is
@@ -319,17 +320,20 @@ func (z *erasureServerPools) removeMovedVersions(ctx context.Context, bucket, ob
// Successful movement is not enough if the destination changed again. // Successful movement is not enough if the destination changed again.
// Keep the source unless another copy still preserves its read result. // Keep the source unless another copy still preserves its read result.
if len(retained) == 0 || retained[0].ObjInfo.ModTime.Before(src.ModTime) { if len(retained) == 0 || retained[0].ObjInfo.ModTime.Before(src.ModTime) {
return PreConditionFailed{} return removed, PreConditionFailed{}
} }
_, err = z.serverPools[source].DeleteObject(ctx, bucket, object, ObjectOptions{ _, err = z.serverPools[source].DeleteObject(ctx, bucket, object, ObjectOptions{
VersionID: vid, NoLock: true, NoAuditLog: true, VersionID: vid, NoLock: true, NoAuditLog: true,
SkipFreeVersion: true, // preserve the old prefix cleanup's tier-GC behavior SkipFreeVersion: true, // preserve the old prefix cleanup's tier-GC behavior
}) })
if err != nil && !isErrObjectNotFound(err) && !isErrVersionNotFound(err) { if err != nil && !isErrObjectNotFound(err) && !isErrVersionNotFound(err) {
return err return removed, err
}
if err == nil {
removed++
} }
} }
return nil return removed, nil
} }
// ilmDeletesVersionOnly reports whether the ILM action queued for a version // ilmDeletesVersionOnly reports whether the ILM action queued for a version
@@ -0,0 +1,66 @@
// Copyright (c) 2026 Denis Babchuk
//
// This file is part of MinIO Object Storage stack
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package cmd
import "testing"
// A key below another key ("k/b" under "k") is a directory inside the
// directory of that key. The listing treats the directory of an object as a
// leaf, so a decommission walk does not see "k/b" while "k" is still in the
// pool. The walk has to be repeated once the keys above are gone, or the
// check after the bucket finds them and the decommission fails.
func keysBelowDecommission(t *testing.T, z *erasureServerPools) PoolDecommissionInfo {
t.Helper()
if err := z.StartDecommission(t.Context(), 0); err != nil {
t.Fatal(err)
}
z.doDecommissionInRoutine(t.Context(), 0)
z.poolMetaMutex.RLock()
defer z.poolMetaMutex.RUnlock()
return *z.poolMeta.Pools[0].Decommission
}
func keysBelowPools(t *testing.T) (*erasureServerPools, string, []string) {
t.Helper()
z, _ := migrationTestPools(t)
const bucket = "keys-below"
if err := z.MakeBucket(t.Context(), bucket, MakeBucketOptions{VersioningEnabled: true}); err != nil {
t.Fatal(err)
}
keys := []string{"k", "k/b", "k/b/c"}
for _, key := range keys {
putConsistencyObject(t, z, bucket, key, 0, key, ObjectOptions{Versioned: true})
}
return z, bucket, keys
}
func TestDecommissionMovesKeysBelowAKey(t *testing.T) {
z, bucket, keys := keysBelowPools(t)
if info := keysBelowDecommission(t, z); !info.Complete || info.Failed {
t.Errorf("decommission: complete %v, failed %v; want complete", info.Complete, info.Failed)
}
for _, key := range keys {
if _, err := z.serverPools[0].GetObjectInfo(t.Context(), bucket, key, ObjectOptions{}); !isErrObjectNotFound(err) {
t.Errorf("%s still in pool 0: %v", key, err)
}
if body := migrationTestBody(t, z.GetObjectNInfo, bucket, key, ""); body != key {
t.Errorf("%s reads %q", key, body)
}
}
}
+31 -7
View File
@@ -28,6 +28,7 @@ import (
"slices" "slices"
"sort" "sort"
"strings" "strings"
"sync/atomic"
"time" "time"
"github.com/dustin/go-humanize" "github.com/dustin/go-humanize"
@@ -752,7 +753,25 @@ func (set *erasureObjects) listObjectsToDecommission(ctx context.Context, bi dec
return err return err
} }
// decommissionPool moves bucket bi out of pool idx. A key below another key
// ("k/b" under "k") is a directory inside the directory of that key, and the
// listing does not descend into the directory of an object: a walk sees "k/b"
// only once "k" has left the pool. The walk is repeated while it removes
// source versions, so each pass moves the keys that the previous one
// uncovered. Each repeat needs a version removed from the pool, which takes
// no new writes, so the repeats end.
func (z *erasureServerPools) decommissionPool(ctx context.Context, idx int, pool *erasureSets, bi decomBucketInfo) error { func (z *erasureServerPools) decommissionPool(ctx context.Context, idx int, pool *erasureSets, bi decomBucketInfo) error {
for {
removed, err := z.decommissionPoolWalk(ctx, idx, pool, bi)
if err != nil || removed == 0 || contextCanceled(ctx) {
return err
}
}
}
// decommissionPoolWalk walks bucket bi of pool idx once and returns the number
// of keys whose source versions it removed.
func (z *erasureServerPools) decommissionPoolWalk(ctx context.Context, idx int, pool *erasureSets, bi decomBucketInfo) (int64, error) {
ctx = logger.SetReqInfo(ctx, &logger.ReqInfo{}) ctx = logger.SetReqInfo(ctx, &logger.ReqInfo{})
const envDecomWorkers = "_MINIO_DECOMMISSION_WORKERS" const envDecomWorkers = "_MINIO_DECOMMISSION_WORKERS"
@@ -766,9 +785,11 @@ func (z *erasureServerPools) decommissionPool(ctx context.Context, idx int, pool
// add that many extra workers. // add that many extra workers.
workerSize += len(pool.sets) workerSize += len(pool.sets)
var removed atomic.Int64
wk, err := workers.New(workerSize) wk, err := workers.New(workerSize)
if err != nil { if err != nil {
return err return 0, err
} }
var vc *versioning.Versioning var vc *versioning.Versioning
@@ -779,24 +800,24 @@ func (z *erasureServerPools) decommissionPool(ctx context.Context, idx int, pool
if bi.Name != minioMetaBucket { if bi.Name != minioMetaBucket {
vc, err = globalBucketVersioningSys.Get(bi.Name) vc, err = globalBucketVersioningSys.Get(bi.Name)
if err != nil { if err != nil {
return err return 0, err
} }
// Check if the current bucket has a configured lifecycle policy // Check if the current bucket has a configured lifecycle policy
lc, lifecycleUpdatedAt, err = globalBucketMetadataSys.GetLifecycleConfig(bi.Name) lc, lifecycleUpdatedAt, err = globalBucketMetadataSys.GetLifecycleConfig(bi.Name)
if err != nil && !errors.Is(err, BucketLifecycleNotFound{Bucket: bi.Name}) { if err != nil && !errors.Is(err, BucketLifecycleNotFound{Bucket: bi.Name}) {
return err return 0, err
} }
// Check if bucket is object locked. // Check if bucket is object locked.
lr, err = globalBucketObjectLockSys.Get(bi.Name) lr, err = globalBucketObjectLockSys.Get(bi.Name)
if err != nil { if err != nil {
return err return 0, err
} }
rcfg, err = getReplicationConfig(ctx, bi.Name) rcfg, err = getReplicationConfig(ctx, bi.Name)
if err != nil { if err != nil {
return err return 0, err
} }
} }
@@ -1019,7 +1040,10 @@ func (z *erasureServerPools) decommissionPool(ctx context.Context, idx int, pool
// if all versions were decommissioned, then we can delete the object versions. // if all versions were decommissioned, then we can delete the object versions.
if decommissioned == len(fivs.Versions) { if decommissioned == len(fivs.Versions) {
stopFn := globalDecommissionMetrics.log(decomMetricDecommissionRemoveObject, idx, bi.Name, entry.name) stopFn := globalDecommissionMetrics.log(decomMetricDecommissionRemoveObject, idx, bi.Name, entry.name)
err := z.removeMovedVersions(ctx, bi.Name, entry.name, idx, fivs.Versions, expired) n, err := z.removeMovedVersions(ctx, bi.Name, entry.name, idx, fivs.Versions, expired)
if n > 0 {
removed.Add(1)
}
if err == nil { if err == nil {
err = z.removeFreeVersions(ctx, bi.Name, entry.name, idx, fivs.FreeVersions) err = z.removeFreeVersions(ctx, bi.Name, entry.name, idx, fivs.FreeVersions)
} }
@@ -1062,7 +1086,7 @@ func (z *erasureServerPools) decommissionPool(ctx context.Context, idx int, pool
}(setIdx) }(setIdx)
} }
wk.Wait() wk.Wait()
return nil return removed.Load(), nil
} }
//msgp:ignore decomMetrics //msgp:ignore decomMetrics
@@ -1666,7 +1666,7 @@ func TestPoolsMigrationLifecycleRefusedExpiryKeepsSource(t *testing.T) {
t.Fatal(err) t.Fatal(err)
} }
version := FileInfo{Name: key, VersionID: id, ModTime: source.ModTime} version := FileInfo{Name: key, VersionID: id, ModTime: source.ModTime}
err = s.z.removeMovedVersions(ctx, s.bucket, key, 0, []FileInfo{version}, []migrationExpiredVersion{{ _, err = s.z.removeMovedVersions(ctx, s.bucket, key, 0, []FileInfo{version}, []migrationExpiredVersion{{
version: version, version: version,
event: lifecycle.Event{Action: lifecycle.DeleteAction}, event: lifecycle.Event{Action: lifecycle.DeleteAction},
lifecycleUpdatedAt: updatedAt, lifecycleUpdatedAt: updatedAt,
@@ -132,7 +132,7 @@ func TestPoolsMigrationCleanupRechecksSource(t *testing.T) {
_, err := z.serverPools[1].DeleteObject(t.Context(), bucket, oi.Name, ObjectOptions{VersionID: nullVersionID}) _, err := z.serverPools[1].DeleteObject(t.Context(), bucket, oi.Name, ObjectOptions{VersionID: nullVersionID})
mustIAM(t, err) mustIAM(t, err)
} }
err := z.removeMovedVersions(t.Context(), bucket, oi.Name, 0, snapshot, nil) _, err := z.removeMovedVersions(t.Context(), bucket, oi.Name, 0, snapshot, nil)
if change != "delete-uuid" && err == nil { if change != "delete-uuid" && err == nil {
t.Fatal("cleanup accepted changed or uncertain state") t.Fatal("cleanup accepted changed or uncertain state")
} }
+1 -1
View File
@@ -779,7 +779,7 @@ func (z *erasureServerPools) rebalanceBucket(ctx context.Context, bucket string,
// if all versions were rebalanced, we can delete the object versions. // if all versions were rebalanced, we can delete the object versions.
if rebalanced == len(fivs.Versions) { if rebalanced == len(fivs.Versions) {
stopFn := globalRebalanceMetrics.log(rebalanceMetricRebalanceRemoveObject, poolIdx, bucket, entry.name) stopFn := globalRebalanceMetrics.log(rebalanceMetricRebalanceRemoveObject, poolIdx, bucket, entry.name)
err := z.removeMovedVersions(ctx, bucket, entry.name, poolIdx, fivs.Versions, expired) _, err := z.removeMovedVersions(ctx, bucket, entry.name, poolIdx, fivs.Versions, expired)
stopFn(0, err) stopFn(0, err)
auditLogRebalance(ctx, "Rebalance:DeleteObject", bucket, entry.name, "", err) auditLogRebalance(ctx, "Rebalance:DeleteObject", bucket, entry.name, "", err)
if err != nil { if err != nil {
+22
View File
@@ -26,6 +26,7 @@ import (
"math/rand" "math/rand"
"net/http" "net/http"
"path" "path"
"slices"
"sort" "sort"
"strconv" "strconv"
"strings" "strings"
@@ -1543,6 +1544,27 @@ func (z *erasureServerPools) DeleteObjects(ctx context.Context, bucket string, o
objSets.Add(objects[i].ObjectName) objSets.Add(objects[i].ObjectName)
} }
// A name that fails the check keeps its error and goes no further. Joined
// to the bucket it can name a path in another bucket or in the meta
// bucket, so it is not locked, looked up or sent to a drive. The other
// names are deleted as a request of their own: everything below sees only
// names that passed.
if slices.ContainsFunc(derrs, func(err error) bool { return err != nil }) {
valid, at := make([]ObjectToDelete, 0, len(objects)), make([]int, 0, len(objects))
for i, obj := range objects {
if derrs[i] == nil {
valid, at = append(valid, obj), append(at, i)
}
}
if len(valid) > 0 {
deleted, errs := z.DeleteObjects(ctx, bucket, valid, opts)
for j, i := range at {
dobjects[i], derrs[i] = deleted[j], errs[j]
}
}
return dobjects, derrs
}
// A single pool locks each name in the set its writers lock it in, one // A single pool locks each name in the set its writers lock it in, one
// set at a time. A single node keeps one lock table for all sets, so it // set at a time. A single node keeps one lock table for all sets, so it
// keeps the one bulk lock below: its local lock takes the names one by // keeps the one bulk lock below: its local lock takes the names one by
+6 -5
View File
@@ -2930,11 +2930,12 @@ func updatedServiceAccountCredentials(ctx context.Context, accessKey string, cr
// at this point. // at this point.
m.Delete(sessionPolicyNameExtracted) m.Delete(sessionPolicyNameExtracted)
nosp := opts.sessionPolicy == nil || opts.sessionPolicy.Version == "" && len(opts.sessionPolicy.Statements) == 0 // A nil sessionPolicy is an edit that does not concern the policy, and
// the embedded one stays. Only a policy that was given, and is empty,
// sessionPolicy is nil and there is embedded policy attached we remove // removes it: the account then has all its parent is allowed, which is
// embedded policy at that point. // usually more.
if _, ok := m.Lookup(policy.SessionPolicyName); ok && nosp { reset := opts.sessionPolicy != nil && opts.sessionPolicy.Version == "" && len(opts.sessionPolicy.Statements) == 0
if _, ok := m.Lookup(policy.SessionPolicyName); ok && reset {
m.Delete(policy.SessionPolicyName) m.Delete(policy.SessionPolicyName)
m.Set(iamPolicyClaimNameSA(), inheritedPolicyType) m.Set(iamPolicyClaimNameSA(), inheritedPolicyType)
} }
+227
View File
@@ -0,0 +1,227 @@
// Copyright (c) 2026 PGSTY
// SPDX-License-Identifier: AGPL-3.0-or-later
package cmd
import (
"context"
"encoding/json"
"fmt"
"runtime"
"strings"
"testing"
"time"
"github.com/minio/madmin-go/v3"
minio "github.com/minio/minio-go/v7"
"github.com/minio/minio/internal/auth"
"github.com/pgsty/silo-pkg/v3/policy"
)
// An edit of a service account that names no policy keeps the embedded
// policy: the account must not come out of a status change, a key rotation
// or a rename with all its parent is allowed. Only a policy that is given
// and empty removes the embedded one.
const svcAccEditPolicy = `{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":["s3:GetObject"],"Resource":["arn:aws:s3:::edit/*"]}]}`
func TestIAMServiceAccountEditKeepsPolicy(t *testing.T) {
f := newGroupRefreshFixture(t)
want, err := policy.ParseConfig(strings.NewReader(svcAccEditPolicy))
mustIAM(t, err)
// restricted creates a service account with the embedded policy.
restricted := func(t *testing.T, key, status string) {
t.Helper()
_, _, err := f.sys.NewServiceAccount(f.ctx, f.parent.AccessKey, nil, newServiceAccountOpts{accessKey: key, secretKey: "edit-original-secret", status: status, sessionPolicy: want})
mustIAM(t, err)
}
// embedded reports whether this server and storage have the policy, and
// returns the stored credential.
embedded := func(t *testing.T, key string) (cached, stored bool, cr auth.Credentials) {
t.Helper()
_, got, err := f.sys.GetServiceAccount(f.ctx, key)
mustIAM(t, err)
if got != nil && !got.Equals(*want) {
t.Fatalf("the embedded policy changed: %+v", got)
}
u, ok := f.stored(t, key)
if !ok {
t.Fatal("service account missing in storage")
}
claims, err := getClaimsFromTokenWithSecret(u.Credentials.SessionToken, u.Credentials.SecretKey)
mustIAM(t, err)
_, hasPolicy := claims.Lookup(policy.SessionPolicyName)
return got != nil, hasPolicy && claims.MapClaims[iamPolicyClaimNameSA()] == embeddedPolicyType, u.Credentials
}
// check wants the edit applied, and the policy kept or removed.
check := func(t *testing.T, key string, keep bool, took func(auth.Credentials) bool) {
t.Helper()
cached, stored, cr := embedded(t, key)
if cached != keep || stored != keep {
t.Errorf("embedded policy: cached=%v stored=%v, want %v", cached, stored, keep)
}
if took != nil && !took(cr) {
t.Errorf("the edit did not apply: status=%q name=%q description=%q expiration=%v", cr.Status, cr.Name, cr.Description, cr.Expiration)
}
}
later := UTCNow().Add(time.Hour)
for _, edit := range []struct {
name, start string
opts updateServiceAccountOpts
took func(auth.Credentials) bool
}{
{"disable", auth.AccountOn, updateServiceAccountOpts{status: string(madmin.AccountDisabled)}, func(c auth.Credentials) bool { return c.Status == auth.AccountOff }},
{"enable", auth.AccountOff, updateServiceAccountOpts{status: string(madmin.AccountEnabled)}, func(c auth.Credentials) bool { return c.Status == auth.AccountOn }},
{"status-off", auth.AccountOn, updateServiceAccountOpts{status: auth.AccountOff}, func(c auth.Credentials) bool { return c.Status == auth.AccountOff }},
{"status-on", auth.AccountOff, updateServiceAccountOpts{status: auth.AccountOn}, func(c auth.Credentials) bool { return c.Status == auth.AccountOn }},
{"secret-key", auth.AccountOn, updateServiceAccountOpts{secretKey: "edit-rotated-secret"}, func(c auth.Credentials) bool { return c.SecretKey == "edit-rotated-secret" }},
{"name", auth.AccountOn, updateServiceAccountOpts{name: "renamed"}, func(c auth.Credentials) bool { return c.Name == "renamed" }},
{"description", auth.AccountOn, updateServiceAccountOpts{description: "described"}, func(c auth.Credentials) bool { return c.Description == "described" }},
{"expiration", auth.AccountOn, updateServiceAccountOpts{expiration: &later}, func(c auth.Credentials) bool { return c.Expiration.Equal(later) }},
} {
t.Run(edit.name, func(t *testing.T) {
key := "edit-" + edit.name
restricted(t, key, edit.start)
_, err := f.sys.UpdateServiceAccount(f.ctx, key, edit.opts)
mustIAM(t, err)
check(t, key, true, edit.took)
})
}
t.Run("empty-policy", func(t *testing.T) {
const key = "edit-empty-policy"
restricted(t, key, auth.AccountOn)
_, err := f.sys.UpdateServiceAccount(f.ctx, key, updateServiceAccountOpts{sessionPolicy: &policy.Policy{}})
mustIAM(t, err)
check(t, key, false, nil)
})
// A peer applies the edit as the site that took it did. The edit goes
// through the encoding of the wire, which is where no policy and a null
// one become the same.
for _, edit := range []struct {
name string
update madmin.SRSvcAccUpdate
keep bool
took func(auth.Credentials) bool
}{
{"peer-status", madmin.SRSvcAccUpdate{Status: auth.AccountOff}, true, func(c auth.Credentials) bool { return c.Status == auth.AccountOff }},
{"peer-null", madmin.SRSvcAccUpdate{Description: "described", SessionPolicy: madmin.SRSessionPolicy("null")}, true, func(c auth.Credentials) bool { return c.Description == "described" }},
{"peer-empty", madmin.SRSvcAccUpdate{SessionPolicy: madmin.SRSessionPolicy(`{"Statement":[]}`)}, false, nil},
} {
t.Run(edit.name, func(t *testing.T) {
key := "edit-" + edit.name
restricted(t, key, auth.AccountOn)
edit.update.AccessKey = key
sent, err := json.Marshal(iamReplicationItem{SRIAMItem: madmin.SRIAMItem{
Type: madmin.SRIAMItemSvcAcc,
SvcAccChange: &madmin.SRSvcAccChange{Update: &edit.update},
UpdatedAt: UTCNow(),
}})
mustIAM(t, err)
var got iamReplicationItem
mustIAM(t, json.Unmarshal(sent, &got))
mustIAM(t, applyIAMReplicationItem(f.ctx, got))
check(t, key, edit.keep, edit.took)
})
}
}
// The same over the admin API, with what a client can send for no policy.
func TestServiceAccountUpdateKeepsPolicy(t *testing.T) {
if runtime.GOOS == globalWindowsOSName {
t.Skip("IAM integration harness is disabled on Windows")
}
s := newTestSuiteIAM(TestSuiteCommon{serverType: "ErasureSD", signer: signerV4}, false)
c := &check{t, s.serverType}
s.SetUpSuite(c)
defer s.TearDownSuite(c)
ctx, cancel := context.WithTimeout(context.Background(), testDefaultTimeout)
defer cancel()
bucket := getRandomBucketName()
if err := s.client.MakeBucket(ctx, bucket, minio.MakeBucketOptions{}); err != nil {
c.Fatalf("bucket create error: %v", err)
}
if err := s.adm.AddCannedPolicy(ctx, "keeps-policy-parent", fmt.Appendf(nil, `{"Version":"2012-10-17","Statement":[
{"Effect":"Allow","Action":["s3:ListBucket"],"Resource":["arn:aws:s3:::%s"]},
{"Effect":"Allow","Action":["s3:PutObject","s3:GetObject"],"Resource":["arn:aws:s3:::%s/*"]}]}`, bucket, bucket)); err != nil {
c.Fatalf("policy add error: %v", err)
}
parentAK, parentSK := mustGenerateCredentials(c)
if err := s.adm.SetUser(ctx, parentAK, parentSK, madmin.AccountEnabled); err != nil {
c.Fatalf("unable to set user: %v", err)
}
if _, err := s.adm.AttachPolicy(ctx, madmin.PolicyAssociationReq{Policies: []string{"keeps-policy-parent"}, User: parentAK}); err != nil {
c.Fatalf("unable to attach policy: %v", err)
}
// The account may read objects; listing is its parent's to do.
svcAK, svcSK := mustGenerateCredentials(c)
cr, err := s.adm.AddServiceAccount(ctx, madmin.AddServiceAccountReq{
Policy: fmt.Appendf(nil, `{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":["s3:GetObject"],"Resource":["arn:aws:s3:::%s/*"]}]}`, bucket),
TargetUser: parentAK,
AccessKey: svcAK,
SecretKey: svcSK,
})
if err != nil {
c.Fatalf("unable to create svc acc: %v", err)
}
c.mustUpload(ctx, s.client, bucket)
svcClient := s.getUserClient(c, cr.AccessKey, cr.SecretKey, "")
c.mustDownload(ctx, svcClient, bucket)
c.mustNotListObjects(ctx, svcClient, bucket)
info := func() madmin.InfoServiceAccountResp {
info, err := s.adm.InfoServiceAccount(ctx, svcAK)
if err != nil {
c.Fatalf("unable to get svc acc info: %v", err)
}
return info
}
_, rotatedSK := mustGenerateCredentials(c)
later := UTCNow().Add(time.Hour).Truncate(time.Second)
for _, edit := range []struct {
name string
req madmin.UpdateServiceAccountReq
took func(madmin.InfoServiceAccountResp) bool
}{
{"disable", madmin.UpdateServiceAccountReq{NewStatus: "off"}, func(i madmin.InfoServiceAccountResp) bool { return i.AccountStatus == auth.AccountOff }},
{"enable", madmin.UpdateServiceAccountReq{NewStatus: "on"}, func(i madmin.InfoServiceAccountResp) bool { return i.AccountStatus == auth.AccountOn }},
{"description", madmin.UpdateServiceAccountReq{NewDescription: "described"}, func(i madmin.InfoServiceAccountResp) bool { return i.Description == "described" }},
{"name", madmin.UpdateServiceAccountReq{NewName: "renamed"}, func(i madmin.InfoServiceAccountResp) bool { return i.Name == "renamed" }},
{"expiry", madmin.UpdateServiceAccountReq{NewExpiration: &later}, func(i madmin.InfoServiceAccountResp) bool { return i.Expiration != nil && i.Expiration.Equal(later) }},
{"secret-key", madmin.UpdateServiceAccountReq{NewSecretKey: rotatedSK}, func(madmin.InfoServiceAccountResp) bool {
// The new key signs a request the embedded policy allows.
svcClient = s.getUserClient(c, svcAK, rotatedSK, "")
_, err := svcClient.StatObject(ctx, bucket, "some-object", minio.StatObjectOptions{})
return err == nil
}},
{"null-policy", madmin.UpdateServiceAccountReq{NewPolicy: json.RawMessage("null"), NewDescription: "null"}, func(i madmin.InfoServiceAccountResp) bool { return i.Description == "null" }},
} {
if err := s.adm.UpdateServiceAccount(ctx, svcAK, edit.req); err != nil {
c.Fatalf("%s: unable to update svc acc: %v", edit.name, err)
}
got := info()
if got.ImpliedPolicy {
c.Fatalf("%s: the account lost its embedded policy", edit.name)
}
if !edit.took(got) {
c.Fatalf("%s: the edit did not apply: %+v", edit.name, got)
}
}
c.mustNotListObjects(ctx, svcClient, bucket)
// An empty policy document asks for the parent's policies.
if err := s.adm.UpdateServiceAccount(ctx, svcAK, madmin.UpdateServiceAccountReq{NewPolicy: []byte(`{"Statement":[]}`)}); err != nil {
c.Fatalf("unable to remove the policy of svc acc: %v", err)
}
if !info().ImpliedPolicy {
c.Fatalf("an empty policy document must remove the embedded policy")
}
c.mustListObjects(ctx, svcClient, bucket)
}
+91
View File
@@ -0,0 +1,91 @@
// Copyright (c) 2026 Denis Babchuk
//
// This file is part of Silo Object Storage stack
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package cmd
import (
"bytes"
"context"
"encoding/xml"
"net/http"
"net/http/httptest"
"net/url"
"testing"
"github.com/minio/minio/internal/auth"
xhttp "github.com/minio/minio/internal/http"
)
// GetObjectAttributes answers NoSuchKey with AccessDenied to a caller without
// s3:ListBucket on the bucket, as S3 does, so that a missing object cannot be
// told apart from a forbidden one. A server error, such as lost read quorum,
// must reach a caller allowed to read the object as the error it is.
func TestGetObjectAttributesReadQuorumLost(t *testing.T) {
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
obj, dirs, err := prepareErasure16(ctx)
if err != nil {
t.Fatal(err)
}
z := obj.(*erasureServerPools)
previous := newObjectLayerFn()
setObjectLayer(z)
defer func() {
z.Shutdown(context.Background())
removeRoots(dirs)
setObjectLayer(previous)
}()
bucket, router, err := initAPIHandlerTest(ctx, z, nil, MakeBucketOptions{})
if err != nil {
t.Fatal(err)
}
const object, missing = "object", "missing"
if _, err := z.PutObject(ctx, bucket, object, mustGetPutObjReader(t, bytes.NewReader([]byte("data")), 4, "", ""), ObjectOptions{}); err != nil {
t.Fatal(err)
}
read := `{"Effect": "Allow", "Action": ["s3:GetObject", "s3:GetObjectAttributes"], "Resource": ["arn:aws:s3:::` + bucket + `/*"]}`
withoutList := newDeleteAuthzPolicyUser(t, "", bucket, "["+read+"]")
withList := newDeleteAuthzPolicyUser(t, "", bucket, "["+read+
`, {"Effect": "Allow", "Action": ["s3:ListBucket"], "Resource": ["arn:aws:s3:::`+bucket+`"]}]`)
check := func(stage, user string, creds auth.Credentials, key string, want int) {
t.Helper()
target := makeTestTargetURL("", bucket, key, url.Values{"attributes": []string{""}})
req, err := newTestSignedRequestV4(http.MethodGet, target, 0, nil, creds.AccessKey, creds.SecretKey,
map[string]string{xhttp.AmzObjectAttributes: "ETag"})
if err != nil {
t.Fatal(err)
}
rec := httptest.NewRecorder()
router.ServeHTTP(rec, req)
if rec.Code != want {
var apiErr APIErrorResponse
_ = xml.Unmarshal(rec.Body.Bytes(), &apiErr)
t.Errorf("%s: GetObjectAttributes %s by %s: status %d %s, want %d", stage, key, user, rec.Code, apiErr.Code, want)
}
}
check("healthy", "withoutList", withoutList, object, http.StatusOK)
check("healthy", "withList", withList, object, http.StatusOK)
check("healthy", "withoutList", withoutList, missing, http.StatusForbidden)
check("healthy", "withList", withList, missing, http.StatusNotFound)
defer objectReadFault(z, bucket, object, 10)()
check("read quorum lost", "withoutList", withoutList, object, http.StatusServiceUnavailable)
check("read quorum lost", "withList", withList, object, http.StatusServiceUnavailable)
}
+32 -12
View File
@@ -395,12 +395,14 @@ func (api objectAPIHandlers) getObjectHandler(ctx context.Context, objectAPI Obj
} }
// Validate pre-conditions if any. // Validate pre-conditions if any.
var storedTags *string // set once the object has been read far enough
opts.CheckPrecondFn = func(oi ObjectInfo) bool { opts.CheckPrecondFn = func(oi ObjectInfo) bool {
if _, err := DecryptObjectInfo(&oi, r); err != nil { if _, err := DecryptObjectInfo(&oi, r); err != nil {
writeErrorResponse(ctx, w, toAPIError(ctx, err), r.URL) writeErrorResponse(ctx, w, toAPIError(ctx, err), r.URL)
return true return true
} }
storedTags = &oi.UserTags
if s3Error := authorizeRequestWithExistingTags(ctx, r, policy.GetObjectAction, oi.UserTags); s3Error != ErrNone { if s3Error := authorizeRequestWithExistingTags(ctx, r, policy.GetObjectAction, oi.UserTags); s3Error != ErrNone {
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(s3Error), r.URL) writeErrorResponse(ctx, w, errorCodes.ToAPIErr(s3Error), r.URL)
return true return true
@@ -445,8 +447,14 @@ func (api objectAPIHandlers) getObjectHandler(ctx context.Context, objectAPI Obj
return return
} }
// validate if the request indeed was authorized, if it wasn't we need to return "ErrAccessDenied" // validate if the request indeed was authorized, if it wasn't we need to return "ErrAccessDenied"
// instead of any namespace related error. // instead of any namespace related error. Use the stored tags if the callback read them.
if s3Error := authorizeRequest(ctx, r, policy.GetObjectAction); s3Error != ErrNone { var s3Error APIErrorCode
if storedTags != nil {
s3Error = authorizeRequestWithExistingTags(ctx, r, policy.GetObjectAction, *storedTags)
} else {
s3Error = authorizeRequestWithUnreadTags(ctx, r, policy.GetObjectAction, err)
}
if s3Error != ErrNone {
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(s3Error), r.URL) writeErrorResponse(ctx, w, errorCodes.ToAPIErr(s3Error), r.URL)
return return
} }
@@ -623,12 +631,15 @@ func (api objectAPIHandlers) getObjectAttributesHandler(ctx context.Context, obj
objInfo, err := objectAPI.GetObjectInfo(ctx, bucket, object, opts) objInfo, err := objectAPI.GetObjectInfo(ctx, bucket, object, opts)
if err != nil { if err != nil {
s3Error = authorizeRequest(ctx, r, policy.ListBucketAction) // As with S3, a caller without s3:ListBucket on the bucket gets
if s3Error == ErrNone { // AccessDenied in place of NoSuchKey. A server error, such as lost
writeErrorResponse(ctx, w, toAPIError(ctx, err), r.URL) // read quorum, tells nothing about the object and reaches the caller,
return // who may read the object, as it is.
apiErr := toAPIError(ctx, err)
if apiErr.HTTPStatusCode < http.StatusInternalServerError && authorizeBucketRequest(ctx, r, policy.ListBucketAction) != ErrNone {
apiErr = errorCodes.ToAPIErr(ErrAccessDenied)
} }
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(ErrAccessDenied), r.URL) writeErrorResponse(ctx, w, apiErr, r.URL)
return return
} }
@@ -925,7 +936,14 @@ func (api objectAPIHandlers) headObjectHandler(ctx context.Context, objectAPI Ob
} }
} }
if s3Error := authorizeRequestWithExistingTags(ctx, r, policy.GetObjectAction, objInfo.UserTags); s3Error != ErrNone { // objInfo carries the stored tags unless reading the object failed.
var s3Error APIErrorCode
if err != nil && !proxy.Proxy {
s3Error = authorizeRequestWithUnreadTags(ctx, r, policy.GetObjectAction, err)
} else {
s3Error = authorizeRequestWithExistingTags(ctx, r, policy.GetObjectAction, objInfo.UserTags)
}
if s3Error != ErrNone {
writeErrorResponseHeadersOnly(w, errorCodes.ToAPIErr(s3Error)) writeErrorResponseHeadersOnly(w, errorCodes.ToAPIErr(s3Error))
return return
} }
@@ -3783,11 +3801,13 @@ func (api objectAPIHandlers) PutObjectTaggingHandler(w http.ResponseWriter, r *h
} }
objInfo, err := objAPI.GetObjectInfo(ctx, bucket, object, opts) objInfo, err := objAPI.GetObjectInfo(ctx, bucket, object, opts)
existingTags := "" var s3Error APIErrorCode
if err == nil { if err != nil {
existingTags = objInfo.UserTags s3Error = authorizeRequestWithUnreadTags(ctx, r, policy.PutObjectTaggingAction, err)
} else {
s3Error = authorizeRequestWithExistingTags(ctx, r, policy.PutObjectTaggingAction, objInfo.UserTags)
} }
if s3Error := authorizeRequestWithExistingTags(ctx, r, policy.PutObjectTaggingAction, existingTags); s3Error != ErrNone { if s3Error != ErrNone {
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(s3Error), r.URL) writeErrorResponse(ctx, w, errorCodes.ToAPIErr(s3Error), r.URL)
return return
} }
+4 -4
View File
@@ -1,6 +1,6 @@
# Silo Healthcheck # Silo Healthcheck
Silo server exposes three un-authenticated, healthcheck endpoints liveness probe and a cluster probe at `/minio/health/live` and `/minio/health/cluster` respectively. Silo server exposes four unauthenticated healthcheck endpoints: `/minio/health/live`, `/minio/health/ready`, `/minio/health/cluster`, and `/minio/health/cluster/read`.
## Native CLI probe ## Native CLI probe
@@ -76,7 +76,7 @@ readinessProbe:
The reply is '200 OK' if cluster has write quorum if not it returns '503 Service Unavailable'. The reply is '200 OK' if cluster has write quorum if not it returns '503 Service Unavailable'.
``` ```
curl http://silo1:9001/minio/health/cluster curl http://silo1:9000/minio/health/cluster
HTTP/1.1 503 Service Unavailable HTTP/1.1 503 Service Unavailable
Accept-Ranges: bytes Accept-Ranges: bytes
Content-Length: 0 Content-Length: 0
@@ -94,7 +94,7 @@ Date: Tue, 21 Jul 2020 00:36:14 GMT
The reply is '200 OK' if cluster has read quorum if not it returns '503 Service Unavailable'. The reply is '200 OK' if cluster has read quorum if not it returns '503 Service Unavailable'.
``` ```
curl http://silo1:9001/minio/health/cluster/read curl http://silo1:9000/minio/health/cluster/read
HTTP/1.1 503 Service Unavailable HTTP/1.1 503 Service Unavailable
Accept-Ranges: bytes Accept-Ranges: bytes
Content-Length: 0 Content-Length: 0
@@ -112,7 +112,7 @@ Date: Tue, 21 Jul 2020 00:36:14 GMT
You may query the cluster probe endpoint to check if the node which received the request can be taken down for maintenance, if the server replies back '412 Precondition Failed' this means you will lose HA. '200 OK' means you are okay to proceed. You may query the cluster probe endpoint to check if the node which received the request can be taken down for maintenance, if the server replies back '412 Precondition Failed' this means you will lose HA. '200 OK' means you are okay to proceed.
``` ```
curl http://silo1:9001/minio/health/cluster?maintenance=true curl http://silo1:9000/minio/health/cluster?maintenance=true
HTTP/1.1 412 Precondition Failed HTTP/1.1 412 Precondition Failed
Accept-Ranges: bytes Accept-Ranges: bytes
Content-Length: 0 Content-Length: 0
+17 -3
View File
@@ -34,9 +34,10 @@ helm install silo ./helm/silo \
--set rootPassword='replace-with-a-long-random-secret' --set rootPassword='replace-with-a-long-random-secret'
``` ```
The default server and post-install image is `docker.io/pgsty/silo`. The image The default server image is `docker.io/pgsty/silo` (which contains the `silo`
contains the `silo` server, `mcli`, and an `mc` compatibility symlink. It does server, `mcli`, and an `mc` client compatibility symlink, without a `minio`
not contain a `minio` server binary. server binary). Post-install resource jobs use `mcImage` (`docker.io/pgsty/mc`)
by default.
## Compatibility contract ## Compatibility contract
@@ -100,6 +101,19 @@ new chart invokes `silo server`; an old image does not provide that executable.
The new image accepts a legacy first argv token of `minio` only to support old The new image accepts a legacy first argv token of `minio` only to support old
chart commands during the forward migration. chart commands during the forward migration.
When upgrading an existing Silo release to `RELEASE.2026-11-09T00-00-00Z` (chart
`7.0.4`):
- `helm upgrade --reuse-values` keeps the previously saved `image.tag` and
`mcImage.tag`; pass `--set-string image.tag=RELEASE.2026-11-09T00-00-00Z` and
`--set-string mcImage.tag=RELEASE.2026-11-09T00-00-00Z` explicitly.
- Drain or cancel active batch jobs before upgrading; `20261026` cannot resume
new submitter-bound batch manifests on downgrade.
- Single-pool clusters whose erasure sets do not all share the same pods/hosts
require a drained stop-all upgrade rather than a rolling pod restart, and
`POST ?restore` returns `503` while mixed versions are serving. See the
[20261109 release notes](https://silo.pgsty.com/blog/release/silo-20261109/#upgrade).
## Persistence and TLS ## Persistence and TLS
The default PersistentVolumeClaim mounts at `/export`. Use an existing claim The default PersistentVolumeClaim mounts at `/export`. Use an existing claim
+17 -16
View File
@@ -1,43 +1,44 @@
{{- $scheme := ternary "https" "http" .Values.tls.enabled -}}
{{- if eq .Values.service.type "ClusterIP" "NodePort" }} {{- if eq .Values.service.type "ClusterIP" "NodePort" }}
Silo can be accessed via port {{ .Values.service.port }} on the following DNS name from within your cluster: Silo can be accessed via port {{ .Values.service.port }} on the following DNS name from within your cluster:
{{ template "silo.fullname" . }}.{{ .Release.Namespace }}.{{ .Values.clusterDomain }} {{ template "silo.fullname" . }}.{{ .Release.Namespace }}.svc.{{ .Values.clusterDomain }}
To access Silo from localhost, run the below commands: To access Silo from localhost, run the commands below:
1. export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "release={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}") 1. export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "release={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")
2. kubectl port-forward $POD_NAME 9000 --namespace {{ .Release.Namespace }} 2. kubectl port-forward $POD_NAME {{ .Values.service.port }}:{{ .Values.minioAPIPort | default 9000 }} --namespace {{ .Release.Namespace }}
Read more about port forwarding here: http://kubernetes.io/docs/user-guide/kubectl/kubectl_port-forward/ Read more about port forwarding here: https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/
You can now access Silo server on http://localhost:9000. Follow the below steps to connect to Silo server with mc client: You can now access the Silo server on {{ $scheme }}://localhost:{{ .Values.service.port }}. Connect with the mcli client:
1. Download the Silo mc client - https://silo.pgsty.com/reference/minio-mc/#quickstart 1. Download mcli: https://silo.pgsty.com/download/#client
2. export MC_HOST_{{ template "silo.fullname" . }}_local=http://$(kubectl get secret --namespace {{ .Release.Namespace }} {{ template "silo.secretName" . }} -o jsonpath="{.data.rootUser}" | base64 --decode):$(kubectl get secret --namespace {{ .Release.Namespace }} {{ template "silo.secretName" . }} -o jsonpath="{.data.rootPassword}" | base64 --decode)@localhost:{{ .Values.service.port }} 2. mcli alias set local {{ $scheme }}://localhost:{{ .Values.service.port }} "$(kubectl get secret --namespace {{ .Release.Namespace }} {{ template "silo.secretName" . }} -o jsonpath="{.data.rootUser}" | base64 --decode)" "$(kubectl get secret --namespace {{ .Release.Namespace }} {{ template "silo.secretName" . }} -o jsonpath="{.data.rootPassword}" | base64 --decode)"
3. mc ls {{ template "silo.fullname" . }}_local 3. mcli ls local
{{- end }} {{- end }}
{{- if eq .Values.service.type "LoadBalancer" }} {{- if eq .Values.service.type "LoadBalancer" }}
Silo can be accessed via port {{ .Values.service.port }} on an external IP address. Get the service external IP address by: Silo can be accessed via port {{ .Values.service.port }} on an external IP address. Get the service external IP address by:
kubectl get svc --namespace {{ .Release.Namespace }} -l app={{ template "silo.fullname" . }} kubectl get svc --namespace {{ .Release.Namespace }} -l "app={{ template "silo.name" . }},release={{ .Release.Name }}"
Note that the public IP may take a couple of minutes to be available. Note that the public IP may take a couple of minutes to become available.
You can now access Silo server on http://<External-IP>:9000. Follow the below steps to connect to Silo server with mc client: You can now access the Silo server on {{ $scheme }}://<External-IP>:{{ .Values.service.port }}. Connect with the mcli client:
1. Download the Silo mc client - https://silo.pgsty.com/reference/minio-mc/#quickstart 1. Download mcli: https://silo.pgsty.com/download/#client
2. export MC_HOST_{{ template "silo.fullname" . }}_local=http://$(kubectl get secret {{ template "silo.secretName" . }} --namespace {{ .Release.Namespace }} -o jsonpath="{.data.rootUser}" | base64 --decode):$(kubectl get secret {{ template "silo.secretName" . }} -o jsonpath="{.data.rootPassword}" | base64 --decode)@<External-IP>:{{ .Values.service.port }} 2. mcli alias set external {{ $scheme }}://<External-IP>:{{ .Values.service.port }} "$(kubectl get secret {{ template "silo.secretName" . }} --namespace {{ .Release.Namespace }} -o jsonpath="{.data.rootUser}" | base64 --decode)" "$(kubectl get secret {{ template "silo.secretName" . }} --namespace {{ .Release.Namespace }} -o jsonpath="{.data.rootPassword}" | base64 --decode)"
3. mc ls {{ template "silo.fullname" . }} 3. mcli ls external
Alternately, use a browser, an S3 SDK, or a MinIO-compatible SDK to access Silo - https://silo.pgsty.com/reference/minio-server/minio-server/ Alternately, use a browser, an S3 SDK, or a MinIO-compatible SDK to access Silo: https://silo.pgsty.com/reference/minio-server/
{{- end }} {{- end }}
{{ if and (.Values.networkPolicy.enabled) (not .Values.networkPolicy.allowExternal) }} {{ if and (.Values.networkPolicy.enabled) (not .Values.networkPolicy.allowExternal) }}
Note: Since NetworkPolicy is enabled, only pods with label Note: Since NetworkPolicy is enabled, only pods with label
{{ template "silo.fullname" . }}-client=true" {{ template "silo.fullname" . }}-client=true
will be able to connect to this Silo cluster. will be able to connect to this Silo cluster.
{{- end }} {{- end }}
+2 -2
View File
@@ -11,7 +11,7 @@ fullnameOverride: ""
## ##
clusterDomain: cluster.local clusterDomain: cluster.local
## Set default image, imageTag, and imagePullPolicy. mode is used to indicate the ## Set default Silo server image repository, tag, and pull policy.
## ##
image: image:
repository: pgsty/silo repository: pgsty/silo
@@ -517,7 +517,7 @@ postJob:
## Add environment variables for Silo. The MINIO_* names remain the stable ## Add environment variables for Silo. The MINIO_* names remain the stable
## configuration interface and are passed to the Silo container(s). ## configuration interface and are passed to the Silo container(s).
environment: environment:
## Please refer for comprehensive list https://silo.pgsty.com/reference/minio-server/minio-server/ ## See https://silo.pgsty.com/reference/minio-server/settings/ for available settings.
## MINIO_BROWSER: "off" ## MINIO_BROWSER: "off"
## The name of a secret in the same kubernetes namespace which contain secret values ## The name of a secret in the same kubernetes namespace which contain secret values