Compare commits

..

1 Commits

Author SHA1 Message Date
Feng Ruohang 96794aff81 docs: mark this branch as the archived MinIO-identity state
This branch is the last state of the project under the MinIO identity. It is
kept so the artifacts released in that form stay traceable to the source that
produced them, and so anyone landing here from an old link is not left reading
current-looking documentation for a line that has moved.

Both READMEs now open with an archive notice: development continues on main as
Silo, the repository is pgsty/silo, the last release cut from here is
RELEASE.2026-08-04T00-00-00Z, and its assets and container image keep the minio
names and are not moved or re-signed.

The notice also states what did not change, because that is the question an
operator arriving here will actually have: MINIO_* variables, minio_* metrics,
x-minio-* headers, /minio/* routes, the .minio.sys layout, IAM and ARN values,
and the github.com/minio/minio module path are all preserved on main, and the
packages install side by side so migration and rollback stay explicit.

Only the two README files change. The tree is otherwise the exact code of
RELEASE.2026-08-04T00-00-00Z, which the tag continues to mark.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 09:45:23 +08:00
484 changed files with 16109 additions and 35788 deletions
+5 -8
View File
@@ -8,12 +8,9 @@ assignees: ''
--- ---
## NOTE ## NOTE
All GitHub issues are addressed on a best-effort basis at MinIO's sole discretion. There are no Service Level Agreements (SLA) or Objectives (SLO). Remember our [Code of Conduct](https://github.com/minio/minio/blob/master/code_of_conduct.md) when engaging with MinIO Engineers and the larger community.
Silo issues are handled by community maintainers on a best-effort basis. There For urgent issues (e.g. production down, etc.), subscribe to [SUBNET](https://min.io/pricing?jmp=github) for direct to engineering support.
is no SLA, SLO, or emergency production-support channel. Follow the local
[Code of Conduct](../code_of_conduct.md) when participating. Report suspected
vulnerabilities through the private process in [SECURITY.md](../SECURITY.md),
not in a public issue.
<!--- Provide a general summary of the issue in the Title above --> <!--- Provide a general summary of the issue in the Title above -->
@@ -32,7 +29,7 @@ not in a public issue.
## Steps to Reproduce (for bugs) ## Steps to Reproduce (for bugs)
<!--- Provide a link to a live example, or an unambiguous set of steps to --> <!--- Provide a link to a live example, or an unambiguous set of steps to -->
<!--- reproduce this bug. Include code to reproduce, if relevant --> <!--- reproduce this bug. Include code to reproduce, if relevant -->
<!--- and include relevant Silo logs with secrets and credentials removed --> <!--- and make sure you have followed https://github.com/minio/minio/tree/release/docs/debugging to capture relevant logs -->
1. 1.
2. 2.
@@ -45,10 +42,10 @@ not in a public issue.
## Regression ## Regression
<!-- Is this issue a regression? (Yes / No) --> <!-- Is this issue a regression? (Yes / No) -->
<!-- If Yes, optionally include the Silo version, commit id, or PR that caused this regression. --> <!-- If Yes, optionally please include minio version or commit id or PR# that caused this regression, if you have these details. -->
## Your Environment ## Your Environment
<!--- Include as many relevant details about the environment you experienced the bug in --> <!--- Include as many relevant details about the environment you experienced the bug in -->
* Version used (`silo --version`): * Version used (`minio --version`):
* Server setup and configuration: * Server setup and configuration:
* Operating System and version (`uname -a`): * Operating System and version (`uname -a`):
+3 -3
View File
@@ -24,7 +24,7 @@ assignees: ''
## Steps to Reproduce (for bugs) ## Steps to Reproduce (for bugs)
<!--- Provide a link to a live example, or an unambiguous set of steps to --> <!--- Provide a link to a live example, or an unambiguous set of steps to -->
<!--- reproduce this bug. Include code to reproduce, if relevant --> <!--- reproduce this bug. Include code to reproduce, if relevant -->
<!--- and include the relevant Silo logs with secrets and credentials removed --> <!--- and make sure you have followed https://github.com/minio/minio/tree/release/docs/debugging to capture relevant logs -->
1. 1.
2. 2.
@@ -37,10 +37,10 @@ assignees: ''
## Regression ## Regression
<!-- Is this issue a regression? (Yes / No) --> <!-- Is this issue a regression? (Yes / No) -->
<!-- If Yes, optionally include the Silo version, commit id, or PR that caused the regression. --> <!-- If Yes, optionally please include minio version or commit id or PR# that caused this regression, if you have these details. -->
## Your Environment ## Your Environment
<!--- Include as many relevant details about the environment you experienced the bug in --> <!--- Include as many relevant details about the environment you experienced the bug in -->
* Version used (`silo --version`): * Version used (`minio --version`):
* Server setup and configuration: * Server setup and configuration:
* Operating System and version (`uname -a`): * Operating System and version (`uname -a`):
+6 -6
View File
@@ -1,8 +1,8 @@
blank_issues_enabled: false blank_issues_enabled: false
contact_links: contact_links:
- name: Silo Documentation - name: MinIO Community Support
url: https://silo.pgsty.com/docs/ url: https://slack.min.io
about: Installation, configuration, operations, and compatibility guidance about: Community support via Slack - for questions and discussions
- name: Private Security Report - name: MinIO Enterprise Support (SUBNET)
url: https://github.com/pgsty/silo/security/advisories/new url: https://min.io/pricing
about: Privately report a suspected vulnerability in Silo about: Enterprise support with SLA - for production deployments and feature requests
+6 -21
View File
@@ -1,13 +1,8 @@
## Contribution Licensing (no CLA, inbound=outbound, DCO required) ## Community Contribution License
All community contributions in this pull request are licensed to the project maintainers
This project does not use a CLA; contributions are accepted inbound=outbound. under the terms of the [Apache 2 license](https://www.apache.org/licenses/LICENSE-2.0).
By submitting this pull request I represent that I have the right to contribute By creating this pull request I represent that I have the right to license the
the changes, which are licensed under this repository's contributions to the project maintainers under the Apache 2 license.
[GNU Affero General Public License v3.0 or later](https://www.gnu.org/licenses/agpl-3.0.html)
and remain my copyright. Every commit must carry a DCO `Signed-off-by` trailer
(`git commit -s`) certifying the
[Developer Certificate of Origin](https://developercertificate.org/) — see
[CONTRIBUTING.md](https://github.com/pgsty/silo/blob/main/CONTRIBUTING.md).
## Description ## Description
@@ -18,12 +13,6 @@ and remain my copyright. Every commit must carry a DCO `Signed-off-by` trailer
## How to test this PR? ## How to test this PR?
## Compatibility impact
<!-- Note effects on APIs, clients, MINIO_* configuration, metrics, headers,
routes, storage metadata, module/import paths, upgrades, or rollback. -->
## Types of changes ## Types of changes
- [ ] Bug fix (non-breaking change which fixes an issue) - [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality) - [ ] New feature (non-breaking change which adds functionality)
@@ -31,11 +20,7 @@ routes, storage metadata, module/import paths, upgrades, or rollback. -->
- [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Checklist: ## Checklist:
- [ ] All commits are signed off (`git commit -s`) per the [DCO](https://developercertificate.org/)
- [ ] Fixes a regression (If yes, please add `commit-id` or `PR #` here) - [ ] Fixes a regression (If yes, please add `commit-id` or `PR #` here)
- [ ] Unit tests added/updated - [ ] Unit tests added/updated
- [ ] `make verifiers` passes
- [ ] Relevant package tests and `make build` pass
- [ ] Compatibility and rollback impact documented
- [ ] Internal documentation updated - [ ] Internal documentation updated
- [ ] Public documentation update opened in `pgsty/silo.pgsty.com`, if needed - [ ] Create a documentation update request [here](https://github.com/minio/docs/issues/new?label=doc-change,title=Doc+Updated+Needed+For+PR+github.com%2fminio%2fminio%2fpull%2fNNNNN)
+8 -49
View File
@@ -4,9 +4,9 @@ env:
- CGO_ENABLED=0 - CGO_ENABLED=0
builds: builds:
- id: silo - id: minio
main: . main: .
binary: silo binary: minio
goos: goos:
- linux - linux
- darwin - darwin
@@ -23,64 +23,23 @@ builds:
- "{{ .Env.LDFLAGS }}" - "{{ .Env.LDFLAGS }}"
archives: archives:
- id: silo - id: minio
ids: ids:
- silo - minio
name_template: "silo_{{ .Env.PKG_VERSION }}_{{ .Os }}_{{ .Arch }}" name_template: "minio_{{ .Env.PKG_VERSION }}_{{ .Os }}_{{ .Arch }}"
# Explicit so the license materials cannot silently drop out of the
# binary archives: GoReleaser's default file globs would miss NOTICE.
# CREDITS stays out deliberately -- at 1.8MB it would dominate the
# archive; it remains available in the repository and the OCI image.
files:
- README.md
- LICENSE
- NOTICE
checksum: checksum:
name_template: "silo_{{ .Env.PKG_VERSION }}_checksums.txt" name_template: "minio_{{ .Env.PKG_VERSION }}_checksums.txt"
algorithm: sha256 algorithm: sha256
# Generate one SPDX JSON document per platform archive. SBOMs are created
# before the checksum stage, so the signed checksum manifest covers both the
# archives and their corresponding software bills of materials.
sboms:
- id: silo-archives
artifacts: archive
# Avoid network-backed package enrichment: the release SBOM must be
# reproducible from the artifact alone and the PR gate must work offline.
args:
- "$artifact"
- "--output"
- "spdx-json=$document"
env:
- SYFT_FILE_METADATA_CATALOGER_ENABLED=true
- SYFT_CHECK_FOR_APP_UPDATE=false
# A keyless Sigstore bundle is the detached signature for the checksum
# manifest. Consumers can verify the whole archive/SBOM set without trusting a
# long-lived project key copied into the repository.
signs:
- id: silo-checksums
cmd: cosign
signature: "${artifact}.sigstore.json"
args:
- sign-blob
- "--bundle=${signature}"
- "${artifact}"
- --yes
artifacts: checksum
output: true
release: release:
github: github:
owner: pgsty owner: pgsty
name: silo name: minio
draft: true draft: true
prerelease: false prerelease: false
replace_existing_draft: true mode: append
replace_existing_artifacts: false replace_existing_artifacts: false
mode: replace
# Draft replacement matches the release name; keep it identical to the tag.
name_template: "{{ .Tag }}" name_template: "{{ .Tag }}"
changelog: changelog:
+6
View File
@@ -0,0 +1,6 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="255 293 539 140" fill="none">
<path d="M255.222 385.861 304.199 383.375c1.061 6.446 3.22 11.358 6.478 14.734 5.303 5.465 12.879 8.197 22.728 8.197 7.348 0 13.011-1.397 16.988-4.19 3.977-2.794 5.966-6.032 5.966-9.716 0-3.5-1.894-6.631-5.682-9.394-3.788-2.763-12.576-5.372-26.364-7.828-22.576-4.113-38.674-9.577-48.295-16.392-9.697-6.815-14.546-15.502-14.546-26.062 0-6.938 2.481-13.491 7.443-19.662 4.962-6.17 12.424-11.02 22.386-14.55 9.962-3.53 23.617-5.295 40.966-5.295 21.288 0 37.519 3.208 48.693 9.624 11.174 6.416 17.822 16.622 19.943 30.621l-48.523 2.302c-1.288-6.078-3.996-10.498-8.125-13.261-4.129-2.763-9.83-4.144-17.103-4.144-5.984 0-10.492 1.028-13.522 3.085-3.03 2.056-4.545 4.558-4.545 7.505 0 2.149 1.25 4.083 3.75 5.802 2.424 1.78 8.182 3.438 17.273 4.973 22.5 3.93 38.617 7.905 48.352 11.926 9.735 4.022 16.818 9.01 21.25 14.965 4.432 5.956 6.648 12.617 6.648 19.984 0 8.657-2.954 16.638-8.863 23.944-5.909 7.306-14.167 12.847-24.773 16.623-10.606 3.776-23.977 5.664-40.113 5.664-28.333 0-47.954-4.42-58.863-13.261-10.909-8.841-17.083-20.076-18.522-33.706Z" fill="#287CAB"/>
<path d="M434.578 295.519h43.304v135.007h-43.304z" fill="#287CAB"/>
<path d="M517.881 295.519h43.209v101.762h67.436v33.245H517.881Z" fill="#287CAB"/>
<path d="M639.4 363.115c0-22.041 6.359-39.201 19.077-51.48 12.718-12.28 30.427-18.419 53.129-18.419 23.274 0 41.206 6.032 53.796 18.096 12.591 12.064 18.886 28.963 18.886 50.697 0 15.778-2.75 28.717-8.251 38.817-5.5 10.099-13.449 17.958-23.846 23.575-10.397 5.618-23.353 8.427-38.869 8.427-15.77 0-28.822-2.425-39.155-7.275-10.333-4.85-18.711-12.525-25.133-23.023-6.423-10.499-9.634-23.637-9.634-39.416Zm43.209.184c0 13.63 2.623 23.422 7.869 29.377 5.246 5.956 12.384 8.934 21.413 8.934 9.284 0 16.47-2.916 21.557-8.749 5.087-5.833 7.631-16.301 7.631-31.403 0-12.709-2.655-21.995-7.965-27.858-5.31-5.863-12.511-8.795-21.604-8.795-8.712 0-15.707 2.978-20.985 8.933-5.278 5.956-7.916 15.81-7.916 29.561Z" fill="#287CAB"/>
</svg>

After

Width:  |  Height:  |  Size: 2.0 KiB

+4 -77
View File
@@ -1,4 +1,4 @@
name: silo name: minio
arch: ${NFPM_ARCH} arch: ${NFPM_ARCH}
platform: linux platform: linux
version: ${PKG_VERSION} version: ${PKG_VERSION}
@@ -7,99 +7,26 @@ release: ${NFPM_RELEASE}
section: utils section: utils
priority: optional priority: optional
maintainer: "Ruohang Feng (@Vonng) <rh@vonng.com>" maintainer: "Ruohang Feng (@Vonng) <rh@vonng.com>"
description: S3-Interface Libre Object Storage, a community-maintained S3-compatible server. description: S3-Interface Libre Object Storage, Community-maintained MinIO server fork.
vendor: PGSTY vendor: PGSTY
homepage: https://silo.pgsty.com homepage: https://silo.pgsty.com
license: AGPL-3.0-or-later license: AGPL-3.0-or-later
contents: contents:
- src: ${NFPM_SOURCE} - src: ${NFPM_SOURCE}
dst: /usr/bin/silo dst: /usr/local/bin/minio
expand: true expand: true
file_info: file_info:
mode: 0755 mode: 0755
owner: root owner: root
group: root group: root
- src: ${NFPM_UNIT} - src: ${NFPM_UNIT}
dst: /usr/lib/systemd/system/silo.service dst: /usr/lib/systemd/system/minio.service
expand: true expand: true
file_info: file_info:
mode: 0644 mode: 0644
owner: root owner: root
group: root group: root
- src: ${NFPM_DEFAULTS}
dst: /etc/default/silo
type: config|noreplace
expand: true
file_info:
mode: 0644
owner: root
group: root
- src: ${NFPM_SYSUSERS}
dst: /usr/lib/sysusers.d/silo.conf
expand: true
file_info:
mode: 0644
owner: root
group: root
# The license materials are declared once per packager: nfpm only honors
# type: license on rpm and silently drops such entries from deb and apk, so
# the rpm keeps its %license flag while deb and apk carry plain files at the
# same path.
- src: ${NFPM_LICENSE}
dst: /usr/share/doc/silo/LICENSE
type: license
packager: rpm
expand: true
file_info:
mode: 0644
owner: root
group: root
- src: ${NFPM_NOTICE}
dst: /usr/share/doc/silo/NOTICE
type: license
packager: rpm
expand: true
file_info:
mode: 0644
owner: root
group: root
- src: ${NFPM_LICENSE}
dst: /usr/share/doc/silo/LICENSE
packager: deb
expand: true
file_info:
mode: 0644
owner: root
group: root
- src: ${NFPM_NOTICE}
dst: /usr/share/doc/silo/NOTICE
packager: deb
expand: true
file_info:
mode: 0644
owner: root
group: root
- src: ${NFPM_LICENSE}
dst: /usr/share/doc/silo/LICENSE
packager: apk
expand: true
file_info:
mode: 0644
owner: root
group: root
- src: ${NFPM_NOTICE}
dst: /usr/share/doc/silo/NOTICE
packager: apk
expand: true
file_info:
mode: 0644
owner: root
group: root
scripts:
postinstall: buildscripts/package/postinstall.sh
preremove: buildscripts/package/preremove.sh
rpm: rpm:
group: Applications/File group: Applications/File
-105
View File
@@ -1,105 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg"
width="1500" height="570" viewBox="0 0 1500 570"
preserveAspectRatio="xMidYMid meet"
role="img" aria-labelledby="silo-logo-title silo-logo-desc"
shape-rendering="geometricPrecision">
<title id="silo-logo-title">SILO logo</title>
<desc id="silo-logo-desc">The SILO horizontal lockup: the circular silo emblem on the left, the SILO wordmark on the right.</desc>
<defs>
<!-- Emblem gradient, in the emblem's local coordinates; the group transform maps it. -->
<linearGradient id="silo-logo-mark-color" x1="276" y1="720" x2="742" y2="286" gradientUnits="userSpaceOnUse">
<stop offset="0" stop-color="#064A83"/>
<stop offset="0.5" stop-color="#007FA8"/>
<stop offset="1" stop-color="#22C7C9"/>
</linearGradient>
<!-- Wordmark gradient, in the wordmark's local coordinates. -->
<linearGradient id="silo-logo-word-color" x1="44.0" y1="-94.4" x2="1950.0" y2="794.4" gradientUnits="userSpaceOnUse">
<stop class="silo-logo-wm-a" offset="0.06" stop-color="#1d588c"/>
<stop class="silo-logo-wm-b" offset="0.94" stop-color="#b4762e"/>
</linearGradient>
<style>
/* Light theme values of --pg-strong / --copper; dark theme swaps in its own pair. */
@media (prefers-color-scheme: dark) {
.silo-logo-wm-a { stop-color: #7fb8e8; }
.silo-logo-wm-b { stop-color: #e0a35c; }
}
</style>
</defs>
<!-- Circular emblem, shifted from its native viewBox (230 213 570 570) to the 0..570 square. -->
<g id="silo-logo-mark" transform="translate(-230 -213)" fill="url(#silo-logo-mark-color)">
<!-- Outer circular band, intentionally opened at the lower-right plinth. -->
<path d="
M 734 676
A 283.5 278.5 0 1 0 310 692
L 359 692
A 247 248.5 0 1 1 688 676
Z"/>
<!-- Flowing left wall; its upper tangent matches the inner ellipse. -->
<path d="
M 300.57 375
C 292 390 300 430 328 450
C 343 461 357 472 374 481
C 410 501 426 517 426 544
L 426 676
L 336 676
C 308 647 286 608 274 565
C 262 522 263 479 272 439
C 278 413 286 389 300.57 375
Z"/>
<!-- Right column with tangent-continuous upper shoulder. -->
<path d="
M 602 373
Q 602 368 607 370
C 619 374 634 381 634 389
L 634 647
Q 634 649 636 649
L 708 649
L 734 676
L 602 676
Z"/>
<!-- Lower circular cap. -->
<path d="
M 310 692
L 714 692
C 668 743 596 774 512 774
C 428 774 355 743 310 692
Z"/>
<!-- Main silo body, with a tangent-continuous right shoulder. -->
<path d="
M 389 389
C 389 374 447 351 512 351
C 540 351 565 354 580 359
Q 583 360 583 364
L 583 676
L 443 676
L 443 541
C 443 509 426 490 389 470
Z"/>
<!-- Peaked roof with softly tapered, burr-free tips. -->
<path d="
M 512 274
L 647 365
Q 649 370 647 376
C 609 350 563 337 512 337
C 460 337 414 350 377 376
Q 375 370 377 365
Z"/>
</g>
<!-- Wordmark, scaled 3/7 to a 300-unit cap height and centered on the emblem's axis. -->
<g id="silo-logo-word" transform="translate(645.429 135) scale(0.428571)"
fill="url(#silo-logo-word-color)" fill-rule="nonzero">
<path id="silo-logo-s" d="M0 592V492H134V551L167 584H374L408 550V434L375 401H110L2 293V108L110 0H426L534 108V209H400V149L367 116H169L136 149V252L169 285H434L542 393V590L432 700H108Z"/>
<path id="silo-logo-i" d="M637 0H773V700H637Z"/>
<path id="silo-logo-l" d="M888 0H1024V585H1374V700H888Z"/>
<path id="silo-logo-o" d="M1404 585V115L1519 0H1879L1994 115V585L1879 700H1519ZM1807 584 1858 533V167L1807 116H1591L1540 167V533L1591 584Z"/>
</g>
</svg>

Before

Width:  |  Height:  |  Size: 3.8 KiB

-43
View File
@@ -1,43 +0,0 @@
name: DCO
on:
pull_request:
branches:
- main
permissions:
contents: read
jobs:
check:
name: Verify DCO sign-off
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
# Every non-merge commit in the pull request must carry a Signed-off-by
# trailer matching the commit author's email, certifying the Developer
# Certificate of Origin 1.1 (https://developercertificate.org/).
# Only commits authored from a GitHub-issued bot address are exempt; a
# display name is attacker-controlled and must never grant the exemption.
- name: Check Signed-off-by trailers
env:
BASE_SHA: ${{ github.event.pull_request.base.sha }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
fail=0
while read -r sha; do
author_name="$(git log -1 --format='%an' "${sha}")"
author_email="$(git log -1 --format='%ae' "${sha}")"
case "${author_email}" in
*"[bot]@users.noreply.github.com") continue ;;
esac
if ! git log -1 --format='%(trailers:key=Signed-off-by,valueonly)' "${sha}" |
grep -qiF "<${author_email}>"; then
echo "::error::commit ${sha} by ${author_name} <${author_email}> lacks a matching Signed-off-by trailer; sign with 'git commit -s', repair with 'git rebase --signoff'"
fail=1
fi
done < <(git rev-list --no-merges "${BASE_SHA}..${HEAD_SHA}")
exit "${fail}"
+28 -223
View File
@@ -4,15 +4,12 @@ on:
workflow_dispatch: workflow_dispatch:
inputs: inputs:
tag: tag:
description: "Published RELEASE.* tag to package as pgsty/silo" description: "Published RELEASE.* tag to package as pgsty/minio"
required: true required: true
type: string type: string
permissions: permissions:
contents: read contents: read
id-token: write
attestations: write
artifact-metadata: write
concurrency: concurrency:
group: docker-release group: docker-release
@@ -69,20 +66,11 @@ jobs:
fi fi
- name: Checkout release tag - name: Checkout release tag
uses: actions/checkout@v7 uses: actions/checkout@v4
with: with:
ref: ${{ inputs.tag }} ref: ${{ inputs.tag }}
fetch-depth: 0 fetch-depth: 0
- name: Verify workflow identity matches release source
run: |
set -euo pipefail
CHECKED_OUT_REVISION="$(git rev-parse HEAD)"
if [ "${CHECKED_OUT_REVISION}" != "${GITHUB_SHA}" ]; then
echo "Checked out ${CHECKED_OUT_REVISION}, but workflow identity is ${GITHUB_SHA}. Dispatch this workflow from ${RELEASE_TAG}." >&2
exit 1
fi
- name: Prepare verified Docker contexts - name: Prepare verified Docker contexts
env: env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@@ -91,9 +79,9 @@ jobs:
assets_dir="docker-release/assets" assets_dir="docker-release/assets"
mkdir -p "${assets_dir}" mkdir -p "${assets_dir}"
amd64_archive="silo_${PKG_VERSION}_linux_amd64.tar.gz" amd64_archive="minio_${PKG_VERSION}_linux_amd64.tar.gz"
arm64_archive="silo_${PKG_VERSION}_linux_arm64.tar.gz" arm64_archive="minio_${PKG_VERSION}_linux_arm64.tar.gz"
checksums="silo_${PKG_VERSION}_checksums.txt" checksums="minio_${PKG_VERSION}_checksums.txt"
gh release download "${RELEASE_TAG}" --repo "${GITHUB_REPOSITORY}" \ gh release download "${RELEASE_TAG}" --repo "${GITHUB_REPOSITORY}" \
--dir "${assets_dir}" \ --dir "${assets_dir}" \
--pattern "${amd64_archive}" \ --pattern "${amd64_archive}" \
@@ -102,22 +90,10 @@ jobs:
# The binaries going into the images are the published ones, checked # The binaries going into the images are the published ones, checked
# against the published checksums, not a rebuild that merely ought to # against the published checksums, not a rebuild that merely ought to
# match them. awk matches the manifest filename column exactly: a # match them.
# substring grep would also pull in the archive's .sbom.json line,
# whose file is deliberately not downloaded in this lane.
cd "${assets_dir}" cd "${assets_dir}"
awk -v name="${amd64_archive}" '$2 == name' "${checksums}" | sha256sum --check grep -F " ${amd64_archive}" "${checksums}" | sha256sum --check
awk -v name="${arm64_archive}" '$2 == name' "${checksums}" | sha256sum --check grep -F " ${arm64_archive}" "${checksums}" | sha256sum --check
# The checksum manifest and both archives must have provenance from
# this repository's release workflow at the exact checked-out tag.
for artifact in "${checksums}" "${amd64_archive}" "${arm64_archive}"; do
gh attestation verify "${artifact}" \
--repo "${GITHUB_REPOSITORY}" \
--signer-workflow "${GITHUB_REPOSITORY}/.github/workflows/release.yml" \
--source-digest "$(git -C "${GITHUB_WORKSPACE}" rev-parse HEAD)" \
--source-ref "refs/tags/${RELEASE_TAG}" >/dev/null
done
cd "${GITHUB_WORKSPACE}" cd "${GITHUB_WORKSPACE}"
# Dockerfile.goreleaser expects the binary at the context root and # Dockerfile.goreleaser expects the binary at the context root and
@@ -125,10 +101,10 @@ jobs:
# GoReleaser used to assemble via extra_files. # GoReleaser used to assemble via extra_files.
for arch in amd64 arm64; do for arch in amd64 arm64; do
context="docker-release/${arch}" context="docker-release/${arch}"
archive="${assets_dir}/silo_${PKG_VERSION}_linux_${arch}.tar.gz" archive="${assets_dir}/minio_${PKG_VERSION}_linux_${arch}.tar.gz"
mkdir -p "${context}/dockerscripts" mkdir -p "${context}/dockerscripts"
tar -xzf "${archive}" -C "${context}" silo tar -xzf "${archive}" -C "${context}" minio
cp Dockerfile.goreleaser Dockerfile.distroless LICENSE NOTICE CREDITS "${context}/" cp Dockerfile.goreleaser LICENSE CREDITS "${context}/"
cp dockerscripts/docker-entrypoint.sh dockerscripts/download-static-curl.sh \ cp dockerscripts/docker-entrypoint.sh dockerscripts/download-static-curl.sh \
"${context}/dockerscripts/" "${context}/dockerscripts/"
done done
@@ -136,230 +112,59 @@ jobs:
echo "RELEASE_REVISION=$(git rev-parse HEAD)" >> "${GITHUB_ENV}" echo "RELEASE_REVISION=$(git rev-parse HEAD)" >> "${GITHUB_ENV}"
- name: Set up QEMU - name: Set up QEMU
uses: docker/setup-qemu-action@v4 uses: docker/setup-qemu-action@v3
with: with:
platforms: arm64 platforms: arm64
- name: Set up Docker Buildx - name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4 uses: docker/setup-buildx-action@v3
- name: Login to Docker Hub - name: Login to Docker Hub
uses: docker/login-action@v4 uses: docker/login-action@v3
with: with:
username: ${{ secrets.DOCKERHUB_USERNAME }} username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }} password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Install Syft
uses: anchore/sbom-action/download-syft@e22c389904149dbc22b58101806040fa8d37a610 # v0.24.0
with:
syft-version: v1.50.0
- name: Build and push amd64 image - name: Build and push amd64 image
id: build-amd64 uses: docker/build-push-action@v6
uses: docker/build-push-action@v7
with: with:
context: docker-release/amd64 context: docker-release/amd64
file: docker-release/amd64/Dockerfile.goreleaser file: docker-release/amd64/Dockerfile.goreleaser
platforms: linux/amd64 platforms: linux/amd64
push: true push: true
tags: | tags: |
pgsty/silo:${{ env.RELEASE_TAG }}-amd64 pgsty/minio:${{ env.RELEASE_TAG }}-amd64
pgsty/silo:latest-amd64 pgsty/minio:latest-amd64
labels: | labels: |
org.opencontainers.image.version=${{ env.RELEASE_TAG }} org.opencontainers.image.version=${{ env.RELEASE_TAG }}
org.opencontainers.image.created=${{ env.PUBLISHED_AT }} org.opencontainers.image.created=${{ env.PUBLISHED_AT }}
org.opencontainers.image.revision=${{ env.RELEASE_REVISION }} org.opencontainers.image.revision=${{ env.RELEASE_REVISION }}
- name: Build and push arm64 image - name: Build and push arm64 image
id: build-arm64 uses: docker/build-push-action@v6
uses: docker/build-push-action@v7
with: with:
context: docker-release/arm64 context: docker-release/arm64
file: docker-release/arm64/Dockerfile.goreleaser file: docker-release/arm64/Dockerfile.goreleaser
platforms: linux/arm64 platforms: linux/arm64
push: true push: true
tags: | tags: |
pgsty/silo:${{ env.RELEASE_TAG }}-arm64 pgsty/minio:${{ env.RELEASE_TAG }}-arm64
pgsty/silo:latest-arm64 pgsty/minio:latest-arm64
labels: | labels: |
org.opencontainers.image.version=${{ env.RELEASE_TAG }} org.opencontainers.image.version=${{ env.RELEASE_TAG }}
org.opencontainers.image.created=${{ env.PUBLISHED_AT }} org.opencontainers.image.created=${{ env.PUBLISHED_AT }}
org.opencontainers.image.revision=${{ env.RELEASE_REVISION }} org.opencontainers.image.revision=${{ env.RELEASE_REVISION }}
# The distroless variant is a pilot published alongside the classic
# image; it ships the silo binary alone and relies on the native
# `silo healthcheck` subcommand for container health.
# Design: https://silo.pgsty.com/compatibility/feature/healthcheck/
- name: Build and push amd64 distroless image
id: build-amd64-distroless
uses: docker/build-push-action@v7
with:
context: docker-release/amd64
file: docker-release/amd64/Dockerfile.distroless
platforms: linux/amd64
push: true
tags: |
pgsty/silo:${{ env.RELEASE_TAG }}-distroless-amd64
pgsty/silo:distroless-amd64
labels: |
org.opencontainers.image.version=${{ env.RELEASE_TAG }}
org.opencontainers.image.created=${{ env.PUBLISHED_AT }}
org.opencontainers.image.revision=${{ env.RELEASE_REVISION }}
- name: Build and push arm64 distroless image
id: build-arm64-distroless
uses: docker/build-push-action@v7
with:
context: docker-release/arm64
file: docker-release/arm64/Dockerfile.distroless
platforms: linux/arm64
push: true
tags: |
pgsty/silo:${{ env.RELEASE_TAG }}-distroless-arm64
pgsty/silo:distroless-arm64
labels: |
org.opencontainers.image.version=${{ env.RELEASE_TAG }}
org.opencontainers.image.created=${{ env.PUBLISHED_AT }}
org.opencontainers.image.revision=${{ env.RELEASE_REVISION }}
- name: Verify HEALTHCHECK survived the distroless push
run: |
set -euo pipefail
# HEALTHCHECK is a Docker extension absent from the OCI image
# spec, and a publish path can drop it silently. Check the pushed
# architecture image now, before the versioned and rolling
# multi-arch manifests are created, so a broken health config
# stops their promotion. (The architecture-suffixed tags above
# are already public by this point - full staging-then-promote
# would be a workflow-wide redesign shared with the classic
# image lanes.)
docker pull "pgsty/silo:${RELEASE_TAG}-distroless-amd64" >/dev/null
test "$(docker inspect -f '{{json .Config.Healthcheck.Test}}' "pgsty/silo:${RELEASE_TAG}-distroless-amd64")" \
= '["CMD","/usr/bin/silo","healthcheck","ready"]'
- name: Publish multi-architecture manifests - name: Publish multi-architecture manifests
run: | run: |
set -euo pipefail set -euo pipefail
mkdir -p docker-release/metadata
docker buildx imagetools create \ docker buildx imagetools create \
--tag "pgsty/silo:${RELEASE_TAG}" \ --tag "pgsty/minio:${RELEASE_TAG}" \
--metadata-file docker-release/metadata/release.json \ "pgsty/minio:${RELEASE_TAG}-amd64" \
"pgsty/silo:${RELEASE_TAG}-amd64" \ "pgsty/minio:${RELEASE_TAG}-arm64"
"pgsty/silo:${RELEASE_TAG}-arm64"
docker buildx imagetools create \ docker buildx imagetools create \
--tag "pgsty/silo:latest" \ --tag "pgsty/minio:latest" \
--metadata-file docker-release/metadata/latest.json \ "pgsty/minio:latest-amd64" \
"pgsty/silo:latest-amd64" \ "pgsty/minio:latest-arm64"
"pgsty/silo:latest-arm64" docker buildx imagetools inspect "pgsty/minio:${RELEASE_TAG}"
docker buildx imagetools inspect "pgsty/silo:${RELEASE_TAG}" docker buildx imagetools inspect "pgsty/minio:latest"
docker buildx imagetools inspect "pgsty/silo:latest"
docker buildx imagetools create \
--tag "pgsty/silo:${RELEASE_TAG}-distroless" \
--metadata-file docker-release/metadata/release-distroless.json \
"pgsty/silo:${RELEASE_TAG}-distroless-amd64" \
"pgsty/silo:${RELEASE_TAG}-distroless-arm64"
docker buildx imagetools create \
--tag "pgsty/silo:distroless" \
--metadata-file docker-release/metadata/distroless.json \
"pgsty/silo:distroless-amd64" \
"pgsty/silo:distroless-arm64"
docker buildx imagetools inspect "pgsty/silo:${RELEASE_TAG}-distroless"
docker buildx imagetools inspect "pgsty/silo:distroless"
RELEASE_DIGEST="$(jq -r '."containerimage.descriptor".digest' docker-release/metadata/release.json)"
LATEST_DIGEST="$(jq -r '."containerimage.descriptor".digest' docker-release/metadata/latest.json)"
if ! [[ "${RELEASE_DIGEST}" =~ ^sha256:[0-9a-f]{64}$ ]]; then
echo "Invalid release manifest digest: ${RELEASE_DIGEST}" >&2
exit 1
fi
if [ "${RELEASE_DIGEST}" != "${LATEST_DIGEST}" ]; then
echo "Release and latest tags resolved to different manifests" >&2
exit 1
fi
echo "SILO_IMAGE_DIGEST=${RELEASE_DIGEST}" >> "${GITHUB_ENV}"
DISTROLESS_RELEASE_DIGEST="$(jq -r '."containerimage.descriptor".digest' docker-release/metadata/release-distroless.json)"
DISTROLESS_ROLLING_DIGEST="$(jq -r '."containerimage.descriptor".digest' docker-release/metadata/distroless.json)"
if ! [[ "${DISTROLESS_RELEASE_DIGEST}" =~ ^sha256:[0-9a-f]{64}$ ]]; then
echo "Invalid distroless manifest digest: ${DISTROLESS_RELEASE_DIGEST}" >&2
exit 1
fi
if [ "${DISTROLESS_RELEASE_DIGEST}" != "${DISTROLESS_ROLLING_DIGEST}" ]; then
echo "Distroless release and rolling tags resolved to different manifests" >&2
exit 1
fi
echo "SILO_DISTROLESS_DIGEST=${DISTROLESS_RELEASE_DIGEST}" >> "${GITHUB_ENV}"
- name: Generate architecture image SBOMs
env:
AMD64_DIGEST: ${{ steps.build-amd64.outputs.digest }}
ARM64_DIGEST: ${{ steps.build-arm64.outputs.digest }}
DISTROLESS_AMD64_DIGEST: ${{ steps.build-amd64-distroless.outputs.digest }}
DISTROLESS_ARM64_DIGEST: ${{ steps.build-arm64-distroless.outputs.digest }}
SYFT_CHECK_FOR_APP_UPDATE: "false"
run: |
set -euo pipefail
mkdir -p docker-release/sbom
# Each per-architecture digest names an OCI index (image plus the
# provenance attestation buildx attaches), and Syft's platform
# default on an index follows the amd64 runner - an arm64-only
# index would fail outright. Select the platform explicitly.
syft "registry:index.docker.io/pgsty/silo@${AMD64_DIGEST}" \
--platform linux/amd64 \
--output "spdx-json=docker-release/sbom/linux-amd64.spdx.json"
syft "registry:index.docker.io/pgsty/silo@${ARM64_DIGEST}" \
--platform linux/arm64 \
--output "spdx-json=docker-release/sbom/linux-arm64.spdx.json"
syft "registry:index.docker.io/pgsty/silo@${DISTROLESS_AMD64_DIGEST}" \
--platform linux/amd64 \
--output "spdx-json=docker-release/sbom/linux-amd64-distroless.spdx.json"
syft "registry:index.docker.io/pgsty/silo@${DISTROLESS_ARM64_DIGEST}" \
--platform linux/arm64 \
--output "spdx-json=docker-release/sbom/linux-arm64-distroless.spdx.json"
- name: Attest amd64 image SBOM
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-name: index.docker.io/pgsty/silo
subject-digest: ${{ steps.build-amd64.outputs.digest }}
sbom-path: docker-release/sbom/linux-amd64.spdx.json
push-to-registry: true
- name: Attest arm64 image SBOM
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-name: index.docker.io/pgsty/silo
subject-digest: ${{ steps.build-arm64.outputs.digest }}
sbom-path: docker-release/sbom/linux-arm64.spdx.json
push-to-registry: true
- name: Attest amd64 distroless image SBOM
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-name: index.docker.io/pgsty/silo
subject-digest: ${{ steps.build-amd64-distroless.outputs.digest }}
sbom-path: docker-release/sbom/linux-amd64-distroless.spdx.json
push-to-registry: true
- name: Attest arm64 distroless image SBOM
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-name: index.docker.io/pgsty/silo
subject-digest: ${{ steps.build-arm64-distroless.outputs.digest }}
sbom-path: docker-release/sbom/linux-arm64-distroless.spdx.json
push-to-registry: true
- name: Attest multi-architecture image provenance
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-name: index.docker.io/pgsty/silo
subject-digest: ${{ env.SILO_IMAGE_DIGEST }}
push-to-registry: true
- name: Attest multi-architecture distroless image provenance
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-name: index.docker.io/pgsty/silo
subject-digest: ${{ env.SILO_DISTROLESS_DIGEST }}
push-to-registry: true
-234
View File
@@ -1,234 +0,0 @@
name: Finalize Release Packages
# Manual-only lane that runs AFTER the maintainer has GPG-signed the RPMs in a
# Draft release (buildscripts/sign-release-rpms.sh --upload) and BEFORE the
# release is published. GPG signing rewrites the RPM bytes, which strands the
# SBOMs, the packages checksum manifest, and the attestations that release.yml
# generated from the as-built packages. This lane regenerates those materials
# from the published (signed) bytes under the workflow identity, so the
# sigstore layer describes exactly what the release ships.
on:
workflow_dispatch:
inputs:
tag:
description: "Draft RELEASE.* tag whose signed RPMs need refreshed SBOMs and checksums"
required: true
type: string
permissions:
contents: write
id-token: write
attestations: write
artifact-metadata: write
concurrency:
group: release-${{ inputs.tag }}
cancel-in-progress: false
jobs:
finalize:
runs-on: ubuntu-latest
steps:
- name: Checkout release tag
uses: actions/checkout@v7
with:
ref: ${{ inputs.tag }}
fetch-depth: 0
- name: Verify workflow identity matches release source
run: |
set -euo pipefail
CHECKED_OUT_REVISION="$(git rev-parse HEAD)"
if [ "${CHECKED_OUT_REVISION}" != "${GITHUB_SHA}" ]; then
echo "Checked out ${CHECKED_OUT_REVISION}, but workflow identity is ${GITHUB_SHA}. Dispatch from the release tag." >&2
exit 1
fi
- name: Compute release variables
env:
# Environment passthrough keeps the dispatch input out of the script
# source, mirroring release.yml.
INPUT_TAG: ${{ inputs.tag }}
run: |
set -euo pipefail
TAG="${INPUT_TAG}"
if [[ ! "${TAG}" =~ ^RELEASE\.[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}-[0-9]{2}-[0-9]{2}Z$ ]]; then
echo "Invalid release tag format: ${TAG}" >&2
exit 1
fi
VERSION_HYPHEN="${TAG#RELEASE.}"
PKG_VERSION="$(echo "${VERSION_HYPHEN}" | sed -E 's/^([0-9]{4})-([0-9]{2})-([0-9]{2})T([0-9]{2})-([0-9]{2})-([0-9]{2})Z$/\1\2\3\4\5\6.0.0/')"
{
echo "RELEASE_TAG=${TAG}"
echo "PKG_VERSION=${PKG_VERSION}"
} >> "${GITHUB_ENV}"
- name: Refuse to touch a published release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
if [ "$(gh release view "${RELEASE_TAG}" --repo "${GITHUB_REPOSITORY}" --json isDraft --jq .isDraft)" != "true" ]; then
echo "${RELEASE_TAG} is not a Draft release; finalize runs only before publishing." >&2
exit 1
fi
- name: Install Syft
uses: anchore/sbom-action/download-syft@e22c389904149dbc22b58101806040fa8d37a610 # v0.24.0
with:
syft-version: v1.50.0
- name: Install Cosign
uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2
with:
cosign-release: v3.1.2
- name: Download and verify the package set
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
# sign-release-rpms.sh owns the package identity; import its values
# the same way test-release.yml does so the lanes cannot drift.
eval "$(grep -E '^expected_(release|fingerprint)=' buildscripts/sign-release-rpms.sh)"
test -n "${expected_release}"
test -n "${expected_fingerprint}"
packages_dir="finalize/packages"
sidecar_dir="finalize/sidecars"
mkdir -p "${packages_dir}" "${sidecar_dir}"
manifest="silo_${PKG_VERSION}_packages_checksums.txt"
# Exactly the twelve manifest subjects land in packages_dir; the
# tarball SBOMs in the release root do not match these patterns.
gh release download "${RELEASE_TAG}" --repo "${GITHUB_REPOSITORY}" --dir "${packages_dir}" \
--pattern '*.rpm' --pattern '*.deb' --pattern '*.apk' \
--pattern '*.rpm.sbom.json' --pattern '*.deb.sbom.json' --pattern '*.apk.sbom.json'
gh release download "${RELEASE_TAG}" --repo "${GITHUB_REPOSITORY}" --dir "${sidecar_dir}" \
--pattern '*.rpm.sha256sum' --pattern "${manifest}"
cd "${packages_dir}"
subject_count="$(find . -maxdepth 1 -type f | wc -l | tr -d ' ')"
if [ "${subject_count}" -ne 12 ]; then
echo "Expected twelve package subjects, found ${subject_count}" >&2
find . -maxdepth 1 -type f >&2
exit 1
fi
# The signed RPMs must match the .sha256sum sidecars the signing
# script regenerated and uploaded alongside them.
for rpm_file in "silo-${PKG_VERSION}-${expected_release}.x86_64.rpm" \
"silo-${PKG_VERSION}-${expected_release}.aarch64.rpm"; do
test -s "${rpm_file}"
actual="$(sha256sum "${rpm_file}" | awk '{print $1}')"
recorded="$(awk '{print $1}' "../sidecars/${rpm_file}.sha256sum")"
if [ "${actual}" != "${recorded}" ]; then
echo "Digest mismatch for ${rpm_file}: sidecar ${recorded}, asset ${actual}" >&2
exit 1
fi
done
# Everything the maintainer did not re-sign must still match the
# manifest release.yml generated: this lane refreshes RPM materials,
# it does not accept drift anywhere else.
for package_file in *.deb *.apk *.deb.sbom.json *.apk.sbom.json; do
awk -v name="${package_file}" '$2 == name' "../sidecars/${manifest}" | sha256sum --check
done
- name: Verify RPM GPG signatures
run: |
set -euo pipefail
eval "$(grep -E '^expected_fingerprint=' buildscripts/sign-release-rpms.sh)"
sudo apt-get update
sudo apt-get install --yes rpm
sudo rpmkeys --import buildscripts/pgsty-rpm-signing-key.asc
key_id="$(printf '%s' "${expected_fingerprint}" | tail -c 8 | tr '[:upper:]' '[:lower:]')"
for rpm_file in finalize/packages/*.rpm; do
signature_output="$(sudo rpmkeys --checksig --verbose "${rpm_file}")"
printf '%s\n' "${signature_output}"
if ! printf '%s\n' "${signature_output}" | tr '[:upper:]' '[:lower:]' | grep -q "key id ${key_id}: ok"; then
echo "Signature verification failed for ${rpm_file}" >&2
exit 1
fi
done
- name: Regenerate RPM SBOMs and the packages checksum manifest
env:
SYFT_CHECK_FOR_APP_UPDATE: "false"
run: |
set -euo pipefail
cd finalize/packages
for rpm_file in *.rpm; do
rm -f "${rpm_file}.sbom.json"
syft "${rpm_file}" --output "spdx-json=${rpm_file}.sbom.json"
done
manifest="silo_${PKG_VERSION}_packages_checksums.txt"
mapfile -t subjects < <(find . -maxdepth 1 -type f \
\( -name '*.rpm' -o -name '*.deb' -o -name '*.apk' -o -name '*.sbom.json' \) | sort)
if [ "${#subjects[@]}" -ne 12 ]; then
echo "Expected six packages and six SBOMs, found ${#subjects[@]} subjects" >&2
exit 1
fi
sha256sum "${subjects[@]}" | sed 's# \./# #' > "${manifest}"
cosign sign-blob --bundle="${manifest}.sigstore.json" "${manifest}" --yes
- name: Attest the finalized package artifacts
id: attest-finalize
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-path: |
finalize/packages/*.rpm
finalize/packages/*.rpm.sbom.json
finalize/packages/*_checksums.txt
finalize/packages/*_checksums.txt.sigstore.json
finalize/sidecars/*.rpm.sha256sum
- name: Preserve finalize provenance bundle as a release asset
env:
BUNDLE_PATH: ${{ steps.attest-finalize.outputs.bundle-path }}
run: |
set -euo pipefail
test -s "${BUNDLE_PATH}"
cp "${BUNDLE_PATH}" "finalize/packages/silo_${PKG_VERSION}_packages_provenance.sigstore.json"
- name: Upload finalized assets to the Draft release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
eval "$(grep -E '^expected_release=' buildscripts/sign-release-rpms.sh)"
manifest="silo_${PKG_VERSION}_packages_checksums.txt"
cd finalize/packages
files=(
"silo-${PKG_VERSION}-${expected_release}.x86_64.rpm.sbom.json"
"silo-${PKG_VERSION}-${expected_release}.aarch64.rpm.sbom.json"
"${manifest}"
"${manifest}.sigstore.json"
"silo_${PKG_VERSION}_packages_provenance.sigstore.json"
)
gh release upload "${RELEASE_TAG}" --repo "${GITHUB_REPOSITORY}" --clobber "${files[@]}"
for asset in "${files[@]}"; do
local_digest="sha256:$(sha256sum "${asset}" | awk '{print $1}')"
remote_digest=""
for attempt in 1 2 3 4 5; do
remote_digest="$(gh release view "${RELEASE_TAG}" --repo "${GITHUB_REPOSITORY}" --json assets \
--jq ".assets[] | select(.name == \"${asset}\") | .digest")"
if [ "${local_digest}" = "${remote_digest}" ]; then
break
fi
if [ "${attempt}" -lt 5 ]; then
sleep 2
fi
done
if [ "${local_digest}" != "${remote_digest}" ]; then
echo "GitHub asset digest mismatch for ${asset}" >&2
echo "Local: ${local_digest}" >&2
echo "Remote: ${remote_digest}" >&2
exit 1
fi
echo "Verified GitHub asset: ${asset} ${remote_digest}"
done
+19 -26
View File
@@ -3,13 +3,13 @@ name: Go CI
on: on:
pull_request: pull_request:
branches: branches:
- main - master
push: push:
branches: branches:
- main - master
workflow_dispatch: workflow_dispatch:
# Cancel superseded runs for the same PR; never cancel main push runs. # Cancel superseded runs for the same PR; never cancel master push runs.
# Keyed on PR number (not head_ref) so fork PRs sharing a branch name # Keyed on PR number (not head_ref) so fork PRs sharing a branch name
# do not collide. # do not collide.
concurrency: concurrency:
@@ -25,21 +25,20 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: 20 timeout-minutes: 20
steps: steps:
- uses: actions/checkout@v7 - uses: actions/checkout@v4
- uses: actions/setup-go@v7 - uses: actions/setup-go@v5
with: with:
go-version-file: go.mod go-version-file: go.mod
cache: true cache: true
- name: Check gofmt - name: Check gofmt
run: | run: |
mapfile -t unformatted < <(gofmt -l main.go cmd internal \ unformatted=$(gofmt -l main.go cmd internal)
buildscripts/rebrand-guard buildscripts/helm-migration-guard) if [ -n "${unformatted}" ]; then
if [ "${#unformatted[@]}" -ne 0 ]; then
echo "The following files are not gofmt-formatted:" echo "The following files are not gofmt-formatted:"
printf '%s\n' "${unformatted[@]}" echo "${unformatted}"
gofmt -d "${unformatted[@]}" gofmt -d ${unformatted}
exit 1 exit 1
fi fi
@@ -51,20 +50,14 @@ jobs:
- name: Vet - name: Vet
run: go vet ./... run: go vet ./...
- name: Verify rebrand compatibility contracts
run: |
go run ./buildscripts/rebrand-guard
buildscripts/verify-rebrand.sh
dockerscripts/docker-entrypoint_test.sh
quality: quality:
name: Lint, Generated Files name: Lint, Generated Files
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: 25 timeout-minutes: 25
steps: steps:
- uses: actions/checkout@v7 - uses: actions/checkout@v4
- uses: actions/setup-go@v7 - uses: actions/setup-go@v5
with: with:
go-version-file: go.mod go-version-file: go.mod
cache: true cache: true
@@ -80,9 +73,9 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: 15 timeout-minutes: 15
steps: steps:
- uses: actions/checkout@v7 - uses: actions/checkout@v4
- uses: actions/setup-go@v7 - uses: actions/setup-go@v5
with: with:
go-version-file: go.mod go-version-file: go.mod
cache: true cache: true
@@ -95,9 +88,9 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: 20 timeout-minutes: 20
steps: steps:
- uses: actions/checkout@v7 - uses: actions/checkout@v4
- uses: actions/setup-go@v7 - uses: actions/setup-go@v5
with: with:
go-version-file: go.mod go-version-file: go.mod
cache: true cache: true
@@ -110,9 +103,9 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: 25 timeout-minutes: 25
steps: steps:
- uses: actions/checkout@v7 - uses: actions/checkout@v4
- uses: actions/setup-go@v7 - uses: actions/setup-go@v5
with: with:
go-version-file: go.mod go-version-file: go.mod
cache: true cache: true
@@ -134,9 +127,9 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: 35 timeout-minutes: 35
steps: steps:
- uses: actions/checkout@v7 - uses: actions/checkout@v4
- uses: actions/setup-go@v7 - uses: actions/setup-go@v5
with: with:
go-version-file: go.mod go-version-file: go.mod
cache: true cache: true
+6 -131
View File
@@ -1,10 +1,5 @@
name: Release name: Release
# Retry contract: an absent or single unfinalized Draft may be rebuilt from
# scratch; a published release or a Draft carrying finalize's GPG-derived
# provenance marker is terminal for this lane. The per-tag lock serializes
# workflows, but a maintainer must not publish the Draft while this job runs.
on: on:
push: push:
tags: tags:
@@ -17,20 +12,13 @@ on:
permissions: permissions:
contents: write contents: write
id-token: write
attestations: write
artifact-metadata: write
concurrency:
group: release-${{ github.event_name == 'workflow_dispatch' && inputs.tag || github.ref_name }}
cancel-in-progress: false
jobs: jobs:
release: release:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@v7 uses: actions/checkout@v4
with: with:
fetch-depth: 0 fetch-depth: 0
# Build the code at the tag being released, not whatever branch the # Build the code at the tag being released, not whatever branch the
@@ -40,7 +28,7 @@ jobs:
ref: ${{ github.event.inputs.tag || github.ref }} ref: ${{ github.event.inputs.tag || github.ref }}
- name: Set up Go - name: Set up Go
uses: actions/setup-go@v7 uses: actions/setup-go@v5
with: with:
go-version-file: go.mod go-version-file: go.mod
cache: true cache: true
@@ -48,28 +36,12 @@ jobs:
- name: Verify clean checkout - name: Verify clean checkout
run: | run: |
set -euo pipefail set -euo pipefail
# GitHub's OIDC certificate records GITHUB_SHA, not the ref passed to
# actions/checkout. A manual dispatch must therefore be launched from
# the release tag itself; otherwise the provenance identity would
# describe different source from the bytes being published.
CHECKED_OUT_REVISION="$(git rev-parse HEAD)"
if [ "${CHECKED_OUT_REVISION}" != "${GITHUB_SHA}" ]; then
echo "Checked out ${CHECKED_OUT_REVISION}, but workflow identity is ${GITHUB_SHA}. Dispatch from the release tag." >&2
exit 1
fi
if [ -n "$(git status --porcelain)" ]; then if [ -n "$(git status --porcelain)" ]; then
echo "Refusing to release from a dirty working tree:" >&2 echo "Refusing to release from a dirty working tree:" >&2
git status --porcelain >&2 git status --porcelain >&2
exit 1 exit 1
fi fi
- name: Verify rebrand compatibility contracts
run: |
set -euo pipefail
go run ./buildscripts/rebrand-guard
buildscripts/verify-rebrand.sh
dockerscripts/docker-entrypoint_test.sh
- name: Compute release variables - name: Compute release variables
env: env:
# Passed through the environment, never interpolated into the script # Passed through the environment, never interpolated into the script
@@ -87,15 +59,6 @@ jobs:
echo "Invalid release tag format: ${TAG}" >&2 echo "Invalid release tag format: ${TAG}" >&2
exit 1 exit 1
fi fi
if ! TAG_COMMIT="$(git rev-parse "${TAG}^{commit}" 2>/dev/null)"; then
echo "Release tag ${TAG} does not resolve to a commit" >&2
exit 1
fi
HEAD_COMMIT="$(git rev-parse HEAD)"
if [ "${TAG_COMMIT}" != "${HEAD_COMMIT}" ]; then
echo "Release tag ${TAG} resolves to ${TAG_COMMIT}, checkout is ${HEAD_COMMIT}" >&2
exit 1
fi
VERSION_HYPHEN="${TAG#RELEASE.}" VERSION_HYPHEN="${TAG#RELEASE.}"
PKG_VERSION="$(echo "${VERSION_HYPHEN}" | sed -E 's/^([0-9]{4})-([0-9]{2})-([0-9]{2})T([0-9]{2})-([0-9]{2})-([0-9]{2})Z$/\1\2\3\4\5\6.0.0/')" PKG_VERSION="$(echo "${VERSION_HYPHEN}" | sed -E 's/^([0-9]{4})-([0-9]{2})-([0-9]{2})T([0-9]{2})-([0-9]{2})-([0-9]{2})Z$/\1\2\3\4\5\6.0.0/')"
VERSION_COLON="$(echo "${VERSION_HYPHEN}" | sed -E 's/T([0-9]{2})-([0-9]{2})-([0-9]{2})Z$/T\1:\2:\3Z/')" VERSION_COLON="$(echo "${VERSION_HYPHEN}" | sed -E 's/T([0-9]{2})-([0-9]{2})-([0-9]{2})Z$/T\1:\2:\3Z/')"
@@ -111,34 +74,13 @@ jobs:
echo "Package version: ${PKG_VERSION}" echo "Package version: ${PKG_VERSION}"
echo "LDFLAGS: ${LDFLAGS}" echo "LDFLAGS: ${LDFLAGS}"
- name: Check existing release state
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
buildscripts/check-release-state.sh "${RELEASE_TAG}"
# Both installer actions are pinned to immutable commits. The explicit
# tool versions keep the release format reproducible across workflow
# reruns while the installers verify the downloaded executables.
- name: Install Syft
uses: anchore/sbom-action/download-syft@e22c389904149dbc22b58101806040fa8d37a610 # v0.24.0
with:
syft-version: v1.50.0
- name: Install Cosign
uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2
with:
cosign-release: v3.1.2
- name: Build Draft release with GoReleaser - name: Build Draft release with GoReleaser
uses: goreleaser/goreleaser-action@v7 uses: goreleaser/goreleaser-action@v6
with: with:
version: "~> v2" version: "~> v2"
args: release --clean --skip=validate --config .github/goreleaser.yml args: release --clean --skip=validate --config .github/goreleaser.yml
env: env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GORELEASER_CURRENT_TAG: ${{ env.RELEASE_TAG }}
LDFLAGS: ${{ env.LDFLAGS }} LDFLAGS: ${{ env.LDFLAGS }}
PKG_VERSION: ${{ env.PKG_VERSION }} PKG_VERSION: ${{ env.PKG_VERSION }}
@@ -158,89 +100,22 @@ jobs:
set -euo pipefail set -euo pipefail
buildscripts/package-release.sh buildscripts/package-release.sh
- name: Generate package SBOMs and signed checksum manifest
env:
SYFT_CHECK_FOR_APP_UPDATE: "false"
run: |
set -euo pipefail
packages_dir="dist/packages"
mapfile -t packages < <(find "${packages_dir}" -maxdepth 1 -type f \
\( -name '*.rpm' -o -name '*.deb' -o -name '*.apk' \) | sort)
if [ "${#packages[@]}" -ne 6 ]; then
echo "Expected six Linux packages, found ${#packages[@]}" >&2
printf '%s\n' "${packages[@]}" >&2
exit 1
fi
for package in "${packages[@]}"; do
syft "${package}" --output "spdx-json=${package}.sbom.json"
done
manifest="${packages_dir}/silo_${PKG_VERSION}_packages_checksums.txt"
(
cd "${packages_dir}"
mapfile -t subjects < <(find . -maxdepth 1 -type f \
\( -name '*.rpm' -o -name '*.deb' -o -name '*.apk' -o -name '*.sbom.json' \) | sort)
if [ "${#subjects[@]}" -ne 12 ]; then
echo "Expected six packages and six SBOMs, found ${#subjects[@]} subjects" >&2
exit 1
fi
sha256sum "${subjects[@]}" | sed 's# \./# #' > "$(basename "${manifest}")"
)
cosign sign-blob --bundle="${manifest}.sigstore.json" "${manifest}" --yes
- name: Attest downloadable release artifacts
id: attest-release
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-path: |
dist/*.tar.gz
dist/*.zip
dist/*.sbom.json
dist/*_checksums.txt
dist/*.sigstore.json
dist/packages/*.rpm
dist/packages/*.deb
dist/packages/*.apk
dist/packages/*.sha256sum
dist/packages/*.sbom.json
dist/packages/*_checksums.txt
dist/packages/*.sigstore.json
- name: Preserve provenance bundle as a release asset
env:
BUNDLE_PATH: ${{ steps.attest-release.outputs.bundle-path }}
run: |
set -euo pipefail
test -s "${BUNDLE_PATH}"
cp "${BUNDLE_PATH}" "dist/silo_${PKG_VERSION}_provenance.sigstore.json"
- name: Confirm unfinalized Draft release state
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REQUIRE_DRAFT: "true"
run: |
set -euo pipefail
buildscripts/check-release-state.sh "${RELEASE_TAG}"
- name: Upload nFPM packages to Draft release - name: Upload nFPM packages to Draft release
env: env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: | run: |
set -euo pipefail set -euo pipefail
mapfile -t files < <(find dist/packages -maxdepth 1 -type f \ mapfile -t files < <(find dist/packages -maxdepth 1 -type f \
\( -name '*.rpm' -o -name '*.deb' -o -name '*.apk' -o -name '*.sha256sum' \ \( -name '*.rpm' -o -name '*.deb' -o -name '*.apk' -o -name '*.sha256sum' \) | sort)
-o -name '*.sbom.json' -o -name '*_checksums.txt' -o -name '*.sigstore.json' \) | sort)
if [ "${#files[@]}" -eq 0 ]; then if [ "${#files[@]}" -eq 0 ]; then
echo "No packages were generated." echo "No packages were generated."
exit 1 exit 1
fi fi
gh release upload "${RELEASE_TAG}" "${files[@]}" \ gh release upload "${RELEASE_TAG}" "${files[@]}"
"dist/silo_${PKG_VERSION}_provenance.sigstore.json"
- name: Upload dist artifact - name: Upload dist artifact
if: always() if: always()
uses: actions/upload-artifact@v7 uses: actions/upload-artifact@v4
with: with:
name: dist name: dist
path: dist/ path: dist/
+69 -300
View File
@@ -7,33 +7,13 @@ on:
- ".github/goreleaser.yml" - ".github/goreleaser.yml"
- ".github/nfpm.yml" - ".github/nfpm.yml"
- "Dockerfile.goreleaser" - "Dockerfile.goreleaser"
- "Dockerfile.distroless"
- "cmd/healthcheck-main.go"
- "cmd/main.go"
- "dockerscripts/download-static-curl.sh"
- "dockerscripts/docker-entrypoint.sh" - "dockerscripts/docker-entrypoint.sh"
- "dockerscripts/docker-entrypoint_test.sh" - "minio.service"
- "silo.service"
- "silo.env"
- "silo.sysusers"
- "buildscripts/package-release.sh" - "buildscripts/package-release.sh"
- "buildscripts/package/postinstall.sh"
- "buildscripts/package/preremove.sh"
- "buildscripts/package/lifecycle_test.sh"
- "buildscripts/minio-upgrade.sh"
- "buildscripts/sign-release-rpms.sh" - "buildscripts/sign-release-rpms.sh"
- "buildscripts/verify-build-provenance.sh" - "buildscripts/verify-build-provenance.sh"
- "buildscripts/check-release-state.sh"
- "buildscripts/check-release-state_test.sh"
- "buildscripts/verify-rebrand.sh"
- "buildscripts/verify-helm-migration.sh"
- "buildscripts/helm-migration-guard/**"
- "helm/silo/**"
- "buildscripts/rebrand-guard/**"
- "buildscripts/gen-ldflags.go" - "buildscripts/gen-ldflags.go"
- ".github/workflows/release.yml" - ".github/workflows/release.yml"
- ".github/workflows/docker-release.yml"
- ".github/workflows/finalize-release.yml"
- ".github/workflows/test-release.yml" - ".github/workflows/test-release.yml"
- ".gitignore" - ".gitignore"
@@ -45,12 +25,12 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@v7 uses: actions/checkout@v4
with: with:
fetch-depth: 0 fetch-depth: 0
- name: Set up Go - name: Set up Go
uses: actions/setup-go@v7 uses: actions/setup-go@v5
with: with:
go-version-file: go.mod go-version-file: go.mod
cache: true cache: true
@@ -71,46 +51,20 @@ jobs:
echo "LDFLAGS: ${LDFLAGS}" echo "LDFLAGS: ${LDFLAGS}"
- name: GoReleaser config check - name: GoReleaser config check
uses: goreleaser/goreleaser-action@v7 uses: goreleaser/goreleaser-action@v6
with: with:
version: "~> v2" version: "~> v2"
args: check --config .github/goreleaser.yml args: check --config .github/goreleaser.yml
- name: Validate Helm chart and legacy upgrade identity
run: buildscripts/verify-helm-migration.sh
- name: Install Syft
uses: anchore/sbom-action/download-syft@e22c389904149dbc22b58101806040fa8d37a610 # v0.24.0
with:
syft-version: v1.50.0
- name: Build snapshot artifacts - name: Build snapshot artifacts
uses: goreleaser/goreleaser-action@v7 uses: goreleaser/goreleaser-action@v6
with: with:
version: "~> v2" version: "~> v2"
# A pull-request snapshot has no trusted release identity. Exercise args: release --snapshot --clean --skip=publish,docker --config .github/goreleaser.yml
# the SBOM/checksum pipeline here, and reserve keyless signing for
# the tag-triggered release workflow with GitHub OIDC.
args: release --snapshot --clean --skip=publish,docker,sign --config .github/goreleaser.yml
env: env:
LDFLAGS: ${{ env.LDFLAGS }} LDFLAGS: ${{ env.LDFLAGS }}
PKG_VERSION: ${{ env.PKG_VERSION }} PKG_VERSION: ${{ env.PKG_VERSION }}
- name: Verify archive SBOM and checksum coverage
run: |
set -euo pipefail
mapfile -t archives < <(find dist -maxdepth 1 -type f -name 'silo_*.tar.gz' | sort)
mapfile -t sboms < <(find dist -maxdepth 1 -type f -name 'silo_*.tar.gz.sbom.json' | sort)
test "${#archives[@]}" -eq 6
test "${#sboms[@]}" -eq 6
manifest="dist/silo_${PKG_VERSION}_checksums.txt"
test -s "${manifest}"
(
cd dist
sha256sum --check "$(basename "${manifest}")"
)
test "$(wc -l < "${manifest}" | tr -d ' ')" -eq 12
- name: Verify binary provenance stamps - name: Verify binary provenance stamps
run: | run: |
set -euo pipefail set -euo pipefail
@@ -134,38 +88,15 @@ jobs:
set -euo pipefail set -euo pipefail
cd dist/packages cd dist/packages
# The signing script asserts these same values, but it runs on the
# maintainer's machine after the release workflow has already built
# and uploaded. Take its expectations as the single source of truth
# so nfpm.yml and the signing script cannot drift apart without
# failing here first, while a fix is still cheap.
#
# This grep is deliberately limited to the eight identity variables,
# all of which are single-line. That is what makes the eval safe:
# should one ever become multi-line, the grep captures an
# unterminated quote and the eval aborts on a syntax error under
# set -e rather than quietly binding an empty value and comparing
# against nothing. expected_payload is multi-line by design and must
# stay out of this set for the same reason.
eval "$(grep -E '^expected_(release|vendor|packager|url|summary|description|license|group)=' \
../../buildscripts/sign-release-rpms.sh)"
for value in "${expected_release}" "${expected_vendor}" "${expected_packager}" \
"${expected_url}" "${expected_summary}" "${expected_description}" \
"${expected_license}" "${expected_group}"; do
test -n "${value}"
done
# These are the public download names; a drift here breaks every # These are the public download names; a drift here breaks every
# script that fetches packages by URL. RPM and DEB carry the PGSTY # script that fetches packages by URL.
# release segment; APK cannot (Alpine pkgrel admits only -r<integer>),
# so it stays bare. package-release.sh builds the same three shapes.
expected=( expected=(
"silo-${PKG_VERSION}-${expected_release}.aarch64.rpm" "minio-${PKG_VERSION}-1.aarch64.rpm"
"silo-${PKG_VERSION}-${expected_release}.x86_64.rpm" "minio-${PKG_VERSION}-1.x86_64.rpm"
"silo_${PKG_VERSION}-${expected_release}_amd64.deb" "minio_${PKG_VERSION}_aarch64.apk"
"silo_${PKG_VERSION}-${expected_release}_arm64.deb" "minio_${PKG_VERSION}_amd64.deb"
"silo_${PKG_VERSION}_aarch64.apk" "minio_${PKG_VERSION}_arm64.deb"
"silo_${PKG_VERSION}_x86_64.apk" "minio_${PKG_VERSION}_x86_64.apk"
) )
for package in "${expected[@]}"; do for package in "${expected[@]}"; do
@@ -176,14 +107,30 @@ jobs:
test "$(find . -maxdepth 1 -type f \( -name '*.rpm' -o -name '*.deb' -o -name '*.apk' \) | wc -l)" -eq 6 test "$(find . -maxdepth 1 -type f \( -name '*.rpm' -o -name '*.deb' -o -name '*.apk' \) | wc -l)" -eq 6
service_sha="$(sha256sum ../../silo.service | awk '{print $1}')" # The signing script asserts these same values, but it runs on the
defaults_sha="$(sha256sum ../../silo.env | awk '{print $1}')" # maintainer's machine after the release workflow has already built
sysusers_sha="$(sha256sum ../../silo.sysusers | awk '{print $1}')" # and uploaded. Take its expectations as the single source of truth
license_sha="$(sha256sum ../../LICENSE | awk '{print $1}')" # so nfpm.yml and the signing script cannot drift apart without
notice_sha="$(sha256sum ../../NOTICE | awk '{print $1}')" # failing here first, while a fix is still cheap.
#
# This grep is deliberately limited to the seven identity variables,
# all of which are single-line. That is what makes the eval safe:
# should one ever become multi-line, the grep captures an
# unterminated quote and the eval aborts on a syntax error under
# set -e rather than quietly binding an empty value and comparing
# against nothing. expected_payload is multi-line by design and must
# stay out of this set for the same reason.
eval "$(grep -E '^expected_(vendor|packager|url|summary|description|license|group)=' \
../../buildscripts/sign-release-rpms.sh)"
for value in "${expected_vendor}" "${expected_packager}" "${expected_url}" \
"${expected_summary}" "${expected_description}" \
"${expected_license}" "${expected_group}"; do
test -n "${value}"
done
rpm_file="silo-${PKG_VERSION}-${expected_release}.x86_64.rpm" service_sha="$(sha256sum ../../minio.service | awk '{print $1}')"
test "$(rpm -qp --queryformat '%{RELEASE}' "${rpm_file}")" = "${expected_release}"
rpm_file="minio-${PKG_VERSION}-1.x86_64.rpm"
test "$(rpm -qp --queryformat '%{VENDOR}' "${rpm_file}")" = "${expected_vendor}" test "$(rpm -qp --queryformat '%{VENDOR}' "${rpm_file}")" = "${expected_vendor}"
test "$(rpm -qp --queryformat '%{PACKAGER}' "${rpm_file}")" = "${expected_packager}" test "$(rpm -qp --queryformat '%{PACKAGER}' "${rpm_file}")" = "${expected_packager}"
test "$(rpm -qp --queryformat '%{URL}' "${rpm_file}")" = "${expected_url}" test "$(rpm -qp --queryformat '%{URL}' "${rpm_file}")" = "${expected_url}"
@@ -191,68 +138,31 @@ jobs:
test "$(rpm -qp --queryformat '%{DESCRIPTION}' "${rpm_file}")" = "${expected_description}" test "$(rpm -qp --queryformat '%{DESCRIPTION}' "${rpm_file}")" = "${expected_description}"
test "$(rpm -qp --queryformat '%{LICENSE}' "${rpm_file}")" = "${expected_license}" test "$(rpm -qp --queryformat '%{LICENSE}' "${rpm_file}")" = "${expected_license}"
test "$(rpm -qp --queryformat '%{GROUP}' "${rpm_file}")" = "${expected_group}" test "$(rpm -qp --queryformat '%{GROUP}' "${rpm_file}")" = "${expected_group}"
rpm -qpl "${rpm_file}" | grep -Fx '/usr/bin/silo' # Both payload entries: the unit file is as much a part of the
rpm -qpl "${rpm_file}" | grep -Fx '/usr/lib/systemd/system/silo.service' # package as the binary, and losing it would install a server with
rpm -qpl "${rpm_file}" | grep -Fx '/etc/default/silo' # nothing to start it.
rpm -qpl "${rpm_file}" | grep -Fx '/usr/lib/sysusers.d/silo.conf' rpm -qpl "${rpm_file}" | grep -Fx '/usr/local/bin/minio'
rpm -qpl "${rpm_file}" | grep -Fx '/usr/share/doc/silo/LICENSE' rpm -qpl "${rpm_file}" | grep -Fx '/usr/lib/systemd/system/minio.service'
rpm -qpl "${rpm_file}" | grep -Fx '/usr/share/doc/silo/NOTICE' test "$(rpm -qpl "${rpm_file}" | wc -l)" -eq 2
test "$(rpm -qpl "${rpm_file}" | wc -l)" -eq 6
# nfpm only honors type: license on rpm, which is why nfpm.yml
# declares the license materials once per packager. Pin the rpm
# %license flag so that split cannot silently regress.
rpm -qp --queryformat '[%{FILEFLAGS:fflags} %{FILENAMES}\n]' "${rpm_file}" \
| grep -Fx 'l /usr/share/doc/silo/LICENSE'
rpm -qp --queryformat '[%{FILEFLAGS:fflags} %{FILENAMES}\n]' "${rpm_file}" \
| grep -Fx 'l /usr/share/doc/silo/NOTICE'
if rpm -qp --conflicts "${rpm_file}" | grep -qi minio; then
echo "RPM must not declare a cross-name conflict with MinIO" >&2
exit 1
fi
if rpm -qp --obsoletes "${rpm_file}" | grep -qi minio; then
echo "RPM must not obsolete a MinIO package" >&2
exit 1
fi
if rpm -qp --provides "${rpm_file}" | grep -qi minio; then
echo "RPM must not provide a MinIO package alias" >&2
exit 1
fi
deb_file="silo_${PKG_VERSION}-${expected_release}_amd64.deb" deb_file="minio_${PKG_VERSION}_amd64.deb"
test "$(dpkg-deb --field "${deb_file}" Maintainer)" = "${expected_packager}" test "$(dpkg-deb --field "${deb_file}" Maintainer)" = "${expected_packager}"
test "$(dpkg-deb --field "${deb_file}" Version)" = "${PKG_VERSION}-${expected_release}" test "$(dpkg-deb --field "${deb_file}" Version)" = "${PKG_VERSION}"
test "$(dpkg-deb --field "${deb_file}" License)" = "${expected_license}" test "$(dpkg-deb --field "${deb_file}" License)" = "${expected_license}"
test "$(dpkg-deb --field "${deb_file}" Section)" = "utils" test "$(dpkg-deb --field "${deb_file}" Section)" = "utils"
test "$(dpkg-deb --field "${deb_file}" Homepage)" = "${expected_url}" test "$(dpkg-deb --field "${deb_file}" Homepage)" = "${expected_url}"
test "$(dpkg-deb --field "${deb_file}" Description)" = "${expected_description}" test "$(dpkg-deb --field "${deb_file}" Description)" = "${expected_description}"
dpkg-deb --contents "${deb_file}" | grep -E 'usr/bin/silo$' dpkg-deb --contents "${deb_file}" | grep -E 'usr/local/bin/minio$'
dpkg-deb --contents "${deb_file}" | grep -E 'usr/lib/systemd/system/silo\.service$' dpkg-deb --contents "${deb_file}" | grep -E 'usr/lib/systemd/system/minio\.service$'
dpkg-deb --contents "${deb_file}" | grep -E 'etc/default/silo$'
dpkg-deb --contents "${deb_file}" | grep -E 'usr/lib/sysusers\.d/silo\.conf$'
dpkg-deb --contents "${deb_file}" | grep -E 'usr/share/doc/silo/LICENSE$'
dpkg-deb --contents "${deb_file}" | grep -E 'usr/share/doc/silo/NOTICE$'
test "$(dpkg-deb --contents "${deb_file}" | awk '$1 !~ /^d/ { count++ } END { print count + 0 }')" -eq 6
test -z "$(dpkg-deb --field "${deb_file}" Conflicts)"
test -z "$(dpkg-deb --field "${deb_file}" Replaces)"
test -z "$(dpkg-deb --field "${deb_file}" Provides)"
apk_info="$(tar -xOzf "silo_${PKG_VERSION}_x86_64.apk" .PKGINFO)" apk_info="$(tar -xOzf "minio_${PKG_VERSION}_x86_64.apk" .PKGINFO)"
grep -Fx "pkgver = ${PKG_VERSION}" <<< "${apk_info}" grep -Fx "pkgver = ${PKG_VERSION}" <<< "${apk_info}"
grep -Fx "url = ${expected_url}" <<< "${apk_info}" grep -Fx "url = ${expected_url}" <<< "${apk_info}"
grep -Fx "maintainer = ${expected_packager}" <<< "${apk_info}" grep -Fx "maintainer = ${expected_packager}" <<< "${apk_info}"
grep -Fx "license = ${expected_license}" <<< "${apk_info}" grep -Fx "license = ${expected_license}" <<< "${apk_info}"
grep -Fx "pkgdesc = ${expected_description}" <<< "${apk_info}" grep -Fx "pkgdesc = ${expected_description}" <<< "${apk_info}"
tar -tzf "silo_${PKG_VERSION}_x86_64.apk" | grep -Fx 'usr/bin/silo' tar -tzf "minio_${PKG_VERSION}_x86_64.apk" | grep -Fx 'usr/local/bin/minio'
tar -tzf "silo_${PKG_VERSION}_x86_64.apk" | grep -Fx 'usr/lib/systemd/system/silo.service' tar -tzf "minio_${PKG_VERSION}_x86_64.apk" | grep -Fx 'usr/lib/systemd/system/minio.service'
tar -tzf "silo_${PKG_VERSION}_x86_64.apk" | grep -Fx 'etc/default/silo'
tar -tzf "silo_${PKG_VERSION}_x86_64.apk" | grep -Fx 'usr/lib/sysusers.d/silo.conf'
tar -tzf "silo_${PKG_VERSION}_x86_64.apk" | grep -Fx 'usr/share/doc/silo/LICENSE'
tar -tzf "silo_${PKG_VERSION}_x86_64.apk" | grep -Fx 'usr/share/doc/silo/NOTICE'
test "$(tar -tzf "silo_${PKG_VERSION}_x86_64.apk" | awk '$0 !~ /^\./ && $0 !~ /\/$/ { count++ } END { print count + 0 }')" -eq 6
if grep -Ei '^provides = .*minio' <<< "${apk_info}"; then
echo "APK must not provide a MinIO package alias" >&2
exit 1
fi
for arch in amd64 arm64; do for arch in amd64 arm64; do
if [ "${arch}" = amd64 ]; then if [ "${arch}" = amd64 ]; then
@@ -265,9 +175,9 @@ jobs:
apk_arch=aarch64 apk_arch=aarch64
fi fi
test "$(rpm -qp --queryformat '%{ARCH}' "silo-${PKG_VERSION}-${expected_release}.${rpm_arch}.rpm")" = "${rpm_arch}" test "$(rpm -qp --queryformat '%{ARCH}' "minio-${PKG_VERSION}-1.${rpm_arch}.rpm")" = "${rpm_arch}"
test "$(dpkg-deb --field "silo_${PKG_VERSION}-${expected_release}_${deb_arch}.deb" Architecture)" = "${deb_arch}" test "$(dpkg-deb --field "minio_${PKG_VERSION}_${deb_arch}.deb" Architecture)" = "${deb_arch}"
grep -Fx "arch = ${apk_arch}" <<< "$(tar -xOzf "silo_${PKG_VERSION}_${apk_arch}.apk" .PKGINFO)" grep -Fx "arch = ${apk_arch}" <<< "$(tar -xOzf "minio_${PKG_VERSION}_${apk_arch}.apk" .PKGINFO)"
# Accepted weakness: this takes the first match, unsorted, where # Accepted weakness: this takes the first match, unsorted, where
# find_binary in package-release.sh demands exactly one. It cannot # find_binary in package-release.sh demands exactly one. It cannot
@@ -276,7 +186,7 @@ jobs:
# goamd64 gains a second level, or if find_binary's exactly-one # goamd64 gains a second level, or if find_binary's exactly-one
# contract is ever relaxed -- at that point this weak copy would be # contract is ever relaxed -- at that point this weak copy would be
# the only one left choosing silently. # the only one left choosing silently.
source_binary="$(find .. -maxdepth 2 -type f -path "../silo_linux_${arch}*/silo" | head -n 1)" source_binary="$(find .. -maxdepth 2 -type f -path "../minio_linux_${arch}*/minio" | head -n 1)"
source_sha="$(sha256sum "${source_binary}" | awk '{print $1}')" source_sha="$(sha256sum "${source_binary}" | awk '{print $1}')"
# Do not pipe rpm2cpio here: Debian's build exits non-zero even when # Do not pipe rpm2cpio here: Debian's build exits non-zero even when
@@ -284,31 +194,15 @@ jobs:
# rpm's own digests instead -- -K checks the payload against the # rpm's own digests instead -- -K checks the payload against the
# header, and FILEDIGESTS is the sha256 rpm itself verifies on # header, and FILEDIGESTS is the sha256 rpm itself verifies on
# install. # install.
rpm -K "silo-${PKG_VERSION}-${expected_release}.${rpm_arch}.rpm" rpm -K "minio-${PKG_VERSION}-1.${rpm_arch}.rpm"
rpm_sha="$(rpm -qp --queryformat '[%{FILENAMES} %{FILEDIGESTS}\n]' \ rpm_sha="$(rpm -qp --queryformat '[%{FILENAMES} %{FILEDIGESTS}\n]' \
"silo-${PKG_VERSION}-${expected_release}.${rpm_arch}.rpm" | awk '$1 == "/usr/bin/silo" { print $2 }')" "minio-${PKG_VERSION}-1.${rpm_arch}.rpm" | awk '$1 == "/usr/local/bin/minio" { print $2 }')"
rpm_service_sha="$(rpm -qp --queryformat '[%{FILENAMES} %{FILEDIGESTS}\n]' \ rpm_service_sha="$(rpm -qp --queryformat '[%{FILENAMES} %{FILEDIGESTS}\n]' \
"silo-${PKG_VERSION}-${expected_release}.${rpm_arch}.rpm" | awk '$1 == "/usr/lib/systemd/system/silo.service" { print $2 }')" "minio-${PKG_VERSION}-1.${rpm_arch}.rpm" | awk '$1 == "/usr/lib/systemd/system/minio.service" { print $2 }')"
rpm_defaults_sha="$(rpm -qp --queryformat '[%{FILENAMES} %{FILEDIGESTS}\n]' \ deb_sha="$(ar p "minio_${PKG_VERSION}_${deb_arch}.deb" data.tar.gz | tar -xzOf - ./usr/local/bin/minio | sha256sum | awk '{print $1}')"
"silo-${PKG_VERSION}-${expected_release}.${rpm_arch}.rpm" | awk '$1 == "/etc/default/silo" { print $2 }')" deb_service_sha="$(ar p "minio_${PKG_VERSION}_${deb_arch}.deb" data.tar.gz | tar -xzOf - ./usr/lib/systemd/system/minio.service | sha256sum | awk '{print $1}')"
rpm_sysusers_sha="$(rpm -qp --queryformat '[%{FILENAMES} %{FILEDIGESTS}\n]' \ apk_sha="$(tar -xzOf "minio_${PKG_VERSION}_${apk_arch}.apk" usr/local/bin/minio | sha256sum | awk '{print $1}')"
"silo-${PKG_VERSION}-${expected_release}.${rpm_arch}.rpm" | awk '$1 == "/usr/lib/sysusers.d/silo.conf" { print $2 }')" apk_service_sha="$(tar -xzOf "minio_${PKG_VERSION}_${apk_arch}.apk" usr/lib/systemd/system/minio.service | sha256sum | awk '{print $1}')"
rpm_license_sha="$(rpm -qp --queryformat '[%{FILENAMES} %{FILEDIGESTS}\n]' \
"silo-${PKG_VERSION}-${expected_release}.${rpm_arch}.rpm" | awk '$1 == "/usr/share/doc/silo/LICENSE" { print $2 }')"
rpm_notice_sha="$(rpm -qp --queryformat '[%{FILENAMES} %{FILEDIGESTS}\n]' \
"silo-${PKG_VERSION}-${expected_release}.${rpm_arch}.rpm" | awk '$1 == "/usr/share/doc/silo/NOTICE" { print $2 }')"
deb_sha="$(ar p "silo_${PKG_VERSION}-${expected_release}_${deb_arch}.deb" data.tar.gz | tar -xzOf - ./usr/bin/silo | sha256sum | awk '{print $1}')"
deb_service_sha="$(ar p "silo_${PKG_VERSION}-${expected_release}_${deb_arch}.deb" data.tar.gz | tar -xzOf - ./usr/lib/systemd/system/silo.service | sha256sum | awk '{print $1}')"
deb_defaults_sha="$(ar p "silo_${PKG_VERSION}-${expected_release}_${deb_arch}.deb" data.tar.gz | tar -xzOf - ./etc/default/silo | sha256sum | awk '{print $1}')"
deb_sysusers_sha="$(ar p "silo_${PKG_VERSION}-${expected_release}_${deb_arch}.deb" data.tar.gz | tar -xzOf - ./usr/lib/sysusers.d/silo.conf | sha256sum | awk '{print $1}')"
deb_license_sha="$(ar p "silo_${PKG_VERSION}-${expected_release}_${deb_arch}.deb" data.tar.gz | tar -xzOf - ./usr/share/doc/silo/LICENSE | sha256sum | awk '{print $1}')"
deb_notice_sha="$(ar p "silo_${PKG_VERSION}-${expected_release}_${deb_arch}.deb" data.tar.gz | tar -xzOf - ./usr/share/doc/silo/NOTICE | sha256sum | awk '{print $1}')"
apk_sha="$(tar -xzOf "silo_${PKG_VERSION}_${apk_arch}.apk" usr/bin/silo | sha256sum | awk '{print $1}')"
apk_service_sha="$(tar -xzOf "silo_${PKG_VERSION}_${apk_arch}.apk" usr/lib/systemd/system/silo.service | sha256sum | awk '{print $1}')"
apk_defaults_sha="$(tar -xzOf "silo_${PKG_VERSION}_${apk_arch}.apk" etc/default/silo | sha256sum | awk '{print $1}')"
apk_sysusers_sha="$(tar -xzOf "silo_${PKG_VERSION}_${apk_arch}.apk" usr/lib/sysusers.d/silo.conf | sha256sum | awk '{print $1}')"
apk_license_sha="$(tar -xzOf "silo_${PKG_VERSION}_${apk_arch}.apk" usr/share/doc/silo/LICENSE | sha256sum | awk '{print $1}')"
apk_notice_sha="$(tar -xzOf "silo_${PKG_VERSION}_${apk_arch}.apk" usr/share/doc/silo/NOTICE | sha256sum | awk '{print $1}')"
test "${source_sha}" = "${rpm_sha}" test "${source_sha}" = "${rpm_sha}"
test "${source_sha}" = "${deb_sha}" test "${source_sha}" = "${deb_sha}"
@@ -316,18 +210,6 @@ jobs:
test "${service_sha}" = "${rpm_service_sha}" test "${service_sha}" = "${rpm_service_sha}"
test "${service_sha}" = "${deb_service_sha}" test "${service_sha}" = "${deb_service_sha}"
test "${service_sha}" = "${apk_service_sha}" test "${service_sha}" = "${apk_service_sha}"
test "${defaults_sha}" = "${rpm_defaults_sha}"
test "${defaults_sha}" = "${deb_defaults_sha}"
test "${defaults_sha}" = "${apk_defaults_sha}"
test "${sysusers_sha}" = "${rpm_sysusers_sha}"
test "${sysusers_sha}" = "${deb_sysusers_sha}"
test "${sysusers_sha}" = "${apk_sysusers_sha}"
test "${license_sha}" = "${rpm_license_sha}"
test "${license_sha}" = "${deb_license_sha}"
test "${license_sha}" = "${apk_license_sha}"
test "${notice_sha}" = "${rpm_notice_sha}"
test "${notice_sha}" = "${deb_notice_sha}"
test "${notice_sha}" = "${apk_notice_sha}"
done done
find . -maxdepth 1 -type f | sort find . -maxdepth 1 -type f | sort
@@ -341,20 +223,19 @@ jobs:
# goreleaser already produced; the mcli-download build stage is skipped # goreleaser already produced; the mcli-download build stage is skipped
# on purpose to keep this gate offline and deterministic. # on purpose to keep this gate offline and deterministic.
ctx="$(mktemp -d)" ctx="$(mktemp -d)"
tar -xzf "dist/silo_${PKG_VERSION}_linux_amd64.tar.gz" -C "${ctx}" silo tar -xzf "dist/minio_${PKG_VERSION}_linux_amd64.tar.gz" -C "${ctx}" minio
cp dockerscripts/docker-entrypoint.sh "${ctx}/docker-entrypoint.sh" cp dockerscripts/docker-entrypoint.sh "${ctx}/docker-entrypoint.sh"
{ {
echo "FROM registry.access.redhat.com/ubi9/ubi-micro:latest" echo "FROM registry.access.redhat.com/ubi9/ubi-micro:latest"
echo "COPY silo /usr/bin/silo" echo "COPY minio /usr/bin/minio"
echo "COPY docker-entrypoint.sh /usr/bin/docker-entrypoint.sh" echo "COPY docker-entrypoint.sh /usr/bin/docker-entrypoint.sh"
echo "RUN mkdir -p /data && chmod 0777 /data && chmod +x /usr/bin/silo /usr/bin/docker-entrypoint.sh" echo "RUN mkdir -p /data && chmod 0777 /data && chmod +x /usr/bin/minio /usr/bin/docker-entrypoint.sh"
echo 'ENV HOME=/tmp'
echo 'ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"]' echo 'ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"]'
echo 'CMD ["silo"]' echo 'CMD ["minio"]'
} > "${ctx}/Dockerfile" } > "${ctx}/Dockerfile"
docker build -t silo-runtime-test:snapshot "${ctx}" docker build -t minio-runtime-test:snapshot "${ctx}"
# PID 1 must be silo, not the entry shell, on every privilege path, so # PID 1 must be minio, not the entry shell, on every privilege path, so
# a SIGTERM from docker stop reaches the server and it exits gracefully # a SIGTERM from docker stop reaches the server and it exits gracefully
# instead of being killed at the stop timeout. Regression guard for the # instead of being killed at the stop timeout. Regression guard for the
# exec-into-chroot entrypoint fix. # exec-into-chroot entrypoint fix.
@@ -363,7 +244,7 @@ jobs:
docker rm -f "${name}" >/dev/null 2>&1 || true docker rm -f "${name}" >/dev/null 2>&1 || true
docker run -d --name "${name}" \ docker run -d --name "${name}" \
-e MINIO_CI_CD=1 -e MINIO_ROOT_USER=ciadmin -e MINIO_ROOT_PASSWORD=ciadmin-secret-123 \ -e MINIO_CI_CD=1 -e MINIO_ROOT_USER=ciadmin -e MINIO_ROOT_PASSWORD=ciadmin-secret-123 \
"$@" silo-runtime-test:snapshot silo server /data --address :9000 >/dev/null "$@" minio-runtime-test:snapshot minio server /data --address :9000 >/dev/null
up="" up=""
for _ in $(seq 1 60); do for _ in $(seq 1 60); do
if docker logs "${name}" 2>&1 | grep -q "API:"; then up=1; break; fi if docker logs "${name}" 2>&1 | grep -q "API:"; then up=1; break; fi
@@ -382,127 +263,15 @@ jobs:
[ "${code}" = "0" ] || { echo "non-zero exit (${name}): ${code}"; exit 1; } [ "${code}" = "0" ] || { echo "non-zero exit (${name}): ${code}"; exit 1; }
[ "${elapsed}" -lt 10 ] || { echo "shutdown too slow (${name}): ${elapsed}s - signal not forwarded"; exit 1; } [ "${elapsed}" -lt 10 ] || { echo "shutdown too slow (${name}): ${elapsed}s - signal not forwarded"; exit 1; }
} }
assert_graceful silo-rt-default assert_graceful minio-rt-default
assert_graceful silo-rt-dropuser -e MINIO_USERNAME=silo-user -e MINIO_GROUPNAME=silo-group assert_graceful minio-rt-dropuser -e MINIO_USERNAME=minio-user -e MINIO_GROUPNAME=minio-group
assert_graceful silo-rt-rootless --user 1001:1001
# The compatibility shim translates only the legacy first argv token;
# the image contains no /usr/bin/minio file.
docker run --rm silo-runtime-test:snapshot sh -c 'test ! -e /usr/bin/minio'
docker run --rm -d --name silo-rt-legacy \
-e MINIO_CI_CD=1 -e MINIO_ROOT_USER=ciadmin -e MINIO_ROOT_PASSWORD=ciadmin-secret-123 \
silo-runtime-test:snapshot minio server /data --address :9000 >/dev/null
sleep 2
test "$(docker exec silo-rt-legacy cat /proc/1/comm)" = silo
docker rm -f silo-rt-legacy >/dev/null
- name: Build distroless runtime image and verify native healthcheck
run: |
set -euo pipefail
# Unlike the classic image, Dockerfile.distroless has no release
# download stages, so the real shipped file can be built and gated
# here. It must keep working with nothing in it but the silo
# binary: no shell, no mc, no entrypoint script.
ctx="$(mktemp -d)"
tar -xzf "dist/silo_${PKG_VERSION}_linux_amd64.tar.gz" -C "${ctx}" silo
cp Dockerfile.distroless LICENSE NOTICE CREDITS "${ctx}/"
docker build -t silo-distroless-test:snapshot -f "${ctx}/Dockerfile.distroless" "${ctx}"
# HEALTHCHECK is a Docker extension absent from the OCI image
# spec; assert the exact probe command survived into the image
# config, not merely a substring of it.
test "$(docker inspect -f '{{json .Config.Healthcheck.Test}}' silo-distroless-test:snapshot)" \
= '["CMD","/usr/bin/silo","healthcheck","ready"]'
# /data ships in the image layer world-writable (issue #55):
# there is no entrypoint left to repair ownership at runtime.
# Export the rootfs once, then assert each required and each
# forbidden entry individually: tar's member-argument mode exits
# non-zero on any missing name, which under pipefail masks a
# found forbidden file, and -tv prints symlinks as 'name ->
# target' which defeats $-anchored greps.
probe="$(docker create silo-distroless-test:snapshot server /data)"
docker export "${probe}" -o "${ctx}/rootfs.tar"
docker rm "${probe}" >/dev/null
tar -tf "${ctx}/rootfs.tar" > "${ctx}/names.txt"
tar -tvf "${ctx}/rootfs.tar" > "${ctx}/verbose.txt"
grep -E '^drwxrwxrwx.* data/$' "${ctx}/verbose.txt" >/dev/null
for want in usr/bin/silo licenses/LICENSE licenses/NOTICE licenses/CREDITS; do
grep -Fxq "${want}" "${ctx}/names.txt" || { echo "missing ${want}"; exit 1; }
done
for forbid in bin/sh usr/bin/sh busybox/sh usr/bin/minio usr/bin/mc usr/bin/mcli; do
if grep -Fxq "${forbid}" "${ctx}/names.txt"; then
echo "distroless image unexpectedly contains ${forbid}"
exit 1
fi
done
# The baked-in healthcheck must drive Docker's health state on its
# own, the probe binary must be directly exec-able without any
# shell, and SIGTERM must still reach PID 1 (the server binary is
# the entrypoint) for a graceful stop.
assert_distroless() {
name="$1"; shift
docker rm -f "${name}" >/dev/null 2>&1 || true
docker run -d --name "${name}" \
-e MINIO_CI_CD=1 -e MINIO_ROOT_USER=ciadmin -e MINIO_ROOT_PASSWORD=ciadmin-secret-123 \
"$@" silo-distroless-test:snapshot server /data --address :9000 >/dev/null
status=""
for _ in $(seq 1 90); do
status="$(docker inspect -f '{{.State.Health.Status}}' "${name}" 2>/dev/null || echo '?')"
if [ "${status}" = "healthy" ]; then break; fi
if [ "$(docker inspect -f '{{.State.Running}}' "${name}")" != "true" ]; then break; fi
sleep 1
done
if [ "${status}" != "healthy" ]; then
echo "container never became healthy (${name}): status=${status}"
docker logs "${name}" 2>&1 | tail -5
exit 1
fi
docker exec "${name}" /usr/bin/silo healthcheck ready
docker exec "${name}" /usr/bin/silo healthcheck cluster
start="$(date +%s)"; docker stop -t 15 "${name}" >/dev/null; end="$(date +%s)"
code="$(docker inspect -f '{{.State.ExitCode}}' "${name}")"
elapsed=$((end - start))
graceful=0; docker logs "${name}" 2>&1 | grep -q "Exiting on signal" && graceful=1
echo "${name}: health=${status} stop=${elapsed}s exit=${code}"
docker rm -f "${name}" >/dev/null 2>&1 || true
[ "${graceful}" = "1" ] || { echo "no graceful-shutdown log (${name}) - signal not forwarded"; exit 1; }
[ "${code}" = "0" ] || { echo "non-zero exit (${name}): ${code}"; exit 1; }
[ "${elapsed}" -lt 10 ] || { echo "shutdown too slow (${name}): ${elapsed}s - signal not forwarded"; exit 1; }
}
assert_distroless silo-dl-default
assert_distroless silo-dl-rootless --user 1001:1001
- name: Validate release scripts - name: Validate release scripts
run: | run: |
set -euo pipefail set -euo pipefail
bash -n buildscripts/package-release.sh bash -n buildscripts/package-release.sh
bash -n buildscripts/minio-upgrade.sh
bash -n buildscripts/sign-release-rpms.sh bash -n buildscripts/sign-release-rpms.sh
bash -n buildscripts/verify-build-provenance.sh bash -n buildscripts/verify-build-provenance.sh
bash -n buildscripts/check-release-state.sh
bash -n buildscripts/check-release-state_test.sh
bash -n buildscripts/verify-rebrand.sh
bash -n buildscripts/verify-helm-migration.sh
sh -n buildscripts/package/postinstall.sh
sh -n buildscripts/package/preremove.sh
bash -n buildscripts/package/lifecycle_test.sh
buildscripts/package/lifecycle_test.sh
bash -n dockerscripts/docker-entrypoint_test.sh
bash -n dockerscripts/download-static-curl.sh
dockerscripts/docker-entrypoint_test.sh
go run ./buildscripts/rebrand-guard
buildscripts/verify-rebrand.sh
test -x buildscripts/package-release.sh test -x buildscripts/package-release.sh
test -x buildscripts/sign-release-rpms.sh test -x buildscripts/sign-release-rpms.sh
test -x buildscripts/verify-build-provenance.sh test -x buildscripts/verify-build-provenance.sh
test -x buildscripts/check-release-state.sh
test -x buildscripts/check-release-state_test.sh
test -x buildscripts/verify-rebrand.sh
test -x buildscripts/verify-helm-migration.sh
test -x buildscripts/package/postinstall.sh
test -x buildscripts/package/preremove.sh
test -x buildscripts/package/lifecycle_test.sh
test -x dockerscripts/docker-entrypoint_test.sh
buildscripts/check-release-state_test.sh
+5 -5
View File
@@ -3,10 +3,10 @@ name: VulnCheck
on: on:
pull_request: pull_request:
branches: branches:
- main - master
push: push:
branches: branches:
- main - master
workflow_dispatch: workflow_dispatch:
permissions: permissions:
@@ -19,17 +19,17 @@ jobs:
timeout-minutes: 20 timeout-minutes: 20
steps: steps:
- name: Check out code - name: Check out code
uses: actions/checkout@v7 uses: actions/checkout@v4
- name: Set up Go - name: Set up Go
uses: actions/setup-go@v7 uses: actions/setup-go@v5
with: with:
go-version-file: go.mod go-version-file: go.mod
cache: true cache: true
- name: Install govulncheck - name: Install govulncheck
run: | run: |
go install golang.org/x/vuln/cmd/govulncheck@v1.7.0 go install golang.org/x/vuln/cmd/govulncheck@v1.6.0
echo "$(go env GOPATH)/bin" >> "${GITHUB_PATH}" echo "$(go env GOPATH)/bin" >> "${GITHUB_PATH}"
- name: Run govulncheck - name: Run govulncheck
-3
View File
@@ -2,7 +2,6 @@
cover.out cover.out
*~ *~
minio minio
silo
!*/ !*/
site/ site/
**/*.test **/*.test
@@ -67,5 +66,3 @@ CLAUDE.md
_bmad/ _bmad/
_bmad-output/ _bmad-output/
docs/security/ docs/security/
docs/rebranding.md
.release-sign/
-3
View File
@@ -59,9 +59,6 @@ formatters:
exclusions: exclusions:
generated: lax generated: lax
paths: paths:
# gofumpt v0.11.0 and Go 1.27's gofmt disagree on the indentation of
# multiple composite literals returned from a single statement.
- internal/ioutil/wait_pipe\.go$
- third_party$ - third_party$
- builtin$ - builtin$
- examples$ - examples$
+1
View File
@@ -0,0 +1 @@
charts.min.io
+3 -13
View File
@@ -1,17 +1,7 @@
# AGPLv3 Compliance # AGPLv3 Compliance
Silo is distributed under the [GNU Affero General Public License v3.0](LICENSE). We have designed MinIO as an Open Source software for the Open Source software community. This requires applications to consider whether their usage of MinIO is in compliance with the GNU AGPLv3 [license](https://github.com/minio/minio/blob/master/LICENSE).
It incorporates source code from the MinIO project and preserves the original
copyright, license, and attribution notices in [`NOTICE`](NOTICE),
[`CREDITS`](CREDITS), and source-file headers.
You are responsible for determining how the AGPLv3 applies to your use, MinIO cannot make the determination as to whether your application's usage of MinIO is in compliance with the AGPLv3 license requirements. You should instead rely on your own legal counsel or licensing specialists to audit and ensure your application is in compliance with the licenses of MinIO and all other open-source projects with which your application integrates or interacts. We understand that AGPLv3 licensing is complex and nuanced. It is for that reason we strongly encourage using experts in licensing to make any such determinations around compliance instead of relying on apocryphal or anecdotal advice.
modification, deployment, and distribution of Silo and its dependencies. The
Silo maintainers cannot provide legal advice or determine whether a particular
application or service satisfies the license. Consult qualified counsel when
the obligations are material to your deployment.
If you convey modified binaries or provide network access to a modified [MinIO Commercial Licensing](https://min.io/pricing) is the best option for applications that trigger AGPLv3 obligations (e.g. open sourcing your application). Applications using MinIO - or any other OSS-licensed code - without validating their usage do so at their own risk.
version, review the complete AGPLv3 text and ensure that the corresponding
source and notices are made available as required. Dependency licenses and
separate notices continue to apply independently.
+30 -95
View File
@@ -1,61 +1,55 @@
# Contributing to Silo # MinIO Contribution Guide [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io) [![Docker Pulls](https://img.shields.io/docker/pulls/minio/minio.svg?maxAge=604800)](https://hub.docker.com/r/minio/minio/)
Silo welcomes focused contributions that improve security, reliability, ``MinIO`` community welcomes your contribution. To make the process as seamless as possible, we recommend you read this contribution guide.
compatibility, packaging, tests, or maintainability. This repository preserves
MinIO-compatible interfaces and storage formats, so changes must identify and
test any compatibility impact.
## Development Workflow ## Development Workflow
Fork the current Silo source repository, create a topic branch, and submit a Start by forking the MinIO GitHub repository, make changes in a branch and then send a pull request. We encourage pull requests to discuss code changes. Here are the steps in details:
pull request. Discuss broad or compatibility-sensitive changes in an issue
before implementation.
### Set up a checkout ### Setup your MinIO GitHub Repository
Fork [MinIO upstream](https://github.com/minio/minio/fork) source repository to your own personal repository. Copy the URL of your MinIO fork (you will need it for the `git clone` command below).
```sh ```sh
git clone https://github.com/pgsty/silo git clone https://github.com/minio/minio
cd silo cd minio
go build -o silo . go install -v
./silo --version ls $(go env GOPATH)/bin/minio
``` ```
### Keep the lineage remote separate ### Set up git remote as ``upstream``
```sh ```sh
git remote add lineage https://github.com/minio/minio $ cd minio
git fetch lineage $ git remote add upstream https://github.com/minio/minio
$ git fetch upstream
$ git merge upstream/master
...
``` ```
Do not merge an upstream branch into a pull request unless the maintainers have
agreed on the scope. Silo intentionally carries a small downstream delta.
### Create your feature branch ### Create your feature branch
Create a separate branch before making code changes: Before making code changes, make sure you create a separate branch for these changes
``` ```
git checkout -b my-new-feature git checkout -b my-new-feature
``` ```
### Test Silo server changes ### Test MinIO server changes
Before opening a pull request: After your code changes, make sure
- Add or update tests for changed behavior. - To add test cases for the new code. If you have questions about how to do it, please ask on our [Slack](https://slack.min.io) channel.
- Run `make verifiers`. - To run `make verifiers`
- Run the smallest relevant package tests, then `make test` when practical. - To squash your commits into a single commit. `git rebase -i`. It's okay to force update your pull request.
- Run `make build` and confirm the generated executable is `silo`. - To run `make test` and `make build` completes.
- Explain any preserved `MINIO_*`, `minio_*`, `x-minio-*`, `/minio/*`,
`.minio.sys`, ARN, module/import-path, or serialized compatibility name.
### Commit changes ### Commit changes
After verification, commit your changes with a concise message and a DCO After verification, commit your changes. This is a [great post](https://chris.beams.io/posts/git-commit/) on how to write useful commit messages
sign-off (see [Licensing of Contributions](#licensing-of-contributions)):
``` ```
git commit -s -am 'Fix object replication retry handling' git commit -am 'Add some feature'
``` ```
### Push to the branch ### Push to the branch
@@ -68,71 +62,13 @@ git push origin my-new-feature
### Create a Pull Request ### Create a Pull Request
Pull requests should include motivation, reproduction steps where applicable, Pull requests can be created via GitHub. Refer to [this document](https://help.github.com/articles/creating-a-pull-request/) for detailed steps on how to create a pull request. After a Pull Request gets peer reviewed and approved, it will be merged.
test evidence, compatibility notes, and documentation impact. Public product
documentation is owned by the separate
[`pgsty/silo.pgsty.com`](https://github.com/pgsty/silo.pgsty.com) repository.
## Licensing of Contributions
Silo is licensed under the [GNU AGPL v3.0 or later](LICENSE). Its core is
Copyright (c) MinIO, Inc.; the combined work can never be relicensed, and this
fork does not try to.
* **No CLA.** We do not ask you to sign a Contributor License Agreement and we
do not take your copyright. Contributions are accepted inbound=outbound: you
keep the copyright to your changes and license them under the same
AGPL-3.0-or-later as the project itself. The maintainers receive no rights
beyond the project license.
* **DCO sign-off required.** Every commit must carry a
`Signed-off-by: Your Name <you@example.com>` trailer certifying the
[Developer Certificate of Origin 1.1](https://developercertificate.org/) —
your statement that you have the right to submit the code under the project
license. Sign each commit with:
```
git commit -s
```
Forgot some? Repair your branch with `git rebase --signoff` and force-push.
CI rejects pull requests containing unsigned commits; the sign-off email
must match the commit author email. (Lowercase `-s` is the plain-text DCO
sign-off; cryptographic `-S`/GPG signing is welcome but independent.)
* **Provenance.** Only submit code you are entitled to submit. This matters
more here than in most projects: Silo carries a downstream delta over an
upstream code base, and cherry-picks from the lineage remote or other forks
are routine. When relaying a patch written by someone else, preserve original
authorship (`git cherry-pick -x`, keep the author field and any existing
`Signed-off-by` trailers) and add your own sign-off as the person passing it
along. Never import code from a proprietary distribution.
* **File headers.** Files derived from upstream keep the original MinIO
copyright header unchanged. New files added by this fork use the dual
header, followed by the standard AGPL boilerplate:
```
// Copyright (c) 2015-2025 MinIO, Inc.
// Copyright (c) 2025-2026 PGSTY
```
* **Squash merges** must keep the `Signed-off-by:` trailers in the resulting
commit message.
* **Authorship and tooling.** The human contributor is the author of the commit
and the sole signatory of its DCO sign-off. Attribution trailers for
assistive tooling (for example `Co-Authored-By:` naming an AI assistant) are
informational only: they record which tools were used, and do not create
authorship, co-authorship, or any copyright claim. Whoever signs off remains
responsible for the content of the commit, whatever produced it.
## FAQs ## FAQs
### How does Silo manage dependencies? ### How does ``MinIO`` manage dependencies?
Silo uses Go modules. Preserve the compatibility module and import paths in ``MinIO`` uses `go mod` to manage its dependencies.
`go.mod`; downstream forks are selected with explicit `replace` directives.
- Run `go get foo/bar` in the source folder to add the dependency to `go.mod` file. - Run `go get foo/bar` in the source folder to add the dependency to `go.mod` file.
@@ -141,7 +77,6 @@ To remove a dependency
- Edit your code and remove the import reference. - Edit your code and remove the import reference.
- Run `go mod tidy` in the source folder to remove dependency from `go.mod` file. - Run `go mod tidy` in the source folder to remove dependency from `go.mod` file.
### What are the coding guidelines? ### What are the coding guidelines for MinIO?
Follow the existing Go style, run `gofmt` on changed Go files, and keep changes ``MinIO`` is fully conformant with Golang style. Refer: [Effective Go](https://github.com/golang/go/wiki/CodeReviewComments) article from Golang project. If you observe offending code, please feel free to send a pull request or ping us on [Slack](https://slack.min.io).
compact. See the Go project's [code review comments](https://go.dev/wiki/CodeReviewComments).
-77
View File
@@ -1,77 +0,0 @@
# Contributors
Silo is maintained by the Pigsty community. This file records the people who have contributed to
this fork since it was established in 2026 — merged code, proposed changes, and the bug reports and
compatibility findings that shaped the releases.
It is maintained by hand and updated at each release. GitHub's own contributor graph is unavailable
here because `pgsty/silo` is a fork, so this file — not that page — is the project's attribution
record. Contributors are listed by GitHub handle. Authorship of every merged commit is preserved in
the Git history and can be verified with `git log --format='%an <%ae>'`.
Upstream MinIO authorship is recorded separately: this fork derives from
[`minio/minio`](https://github.com/minio/minio), [`NOTICE`](NOTICE) retains the upstream product
notice, and the Git history carries the full upstream commit record.
## Code
Contributors whose changes are merged into `main`.
| Contributor | Change | Pull request | Commit |
| :-- | :-- | :-- | :-- |
| [@ZouhairCharef](https://github.com/ZouhairCharef) | Upgraded `go-jose` to v4.1.4 to patch CVE-2026-34986 | [#18](https://github.com/pgsty/silo/pull/18) | [`68e0ba9`](https://github.com/pgsty/silo/commit/68e0ba997) |
| [@mfredenhagen](https://github.com/mfredenhagen) | Bumped `go.opentelemetry.io` to address CVE-2026-39883 | [#19](https://github.com/pgsty/silo/pull/19) | [`1869bd3`](https://github.com/pgsty/silo/commit/1869bd30b) |
| [@pinginfo](https://github.com/pinginfo) | Implemented `Flush` on `trackingResponseWriter`, repairing bucket notification streaming | [#34](https://github.com/pgsty/silo/pull/34) | [`65795ee`](https://github.com/pgsty/silo/commit/65795ee1f) |
| [@waterkip](https://github.com/waterkip) | Repointed documentation links from the upstream domain to the Silo portal | [#41](https://github.com/pgsty/silo/pull/41) | [`d495d30`](https://github.com/pgsty/silo/commit/d495d30d5) |
## Proposed changes
Pull requests that are open for review, or that were closed after informing work that shipped
differently.
| Contributor | Change | Pull request | Status |
| :-- | :-- | :-- | :-- |
| [@magicxor](https://github.com/magicxor) | `DELETE` precondition checks for the `If-Match` header | [#12](https://github.com/pgsty/silo/pull/12) | Open, queued for review |
| [@ycjlin](https://github.com/ycjlin) | `ListObjects` should return `NoSuchBucket` for a prefix on a missing bucket | [#37](https://github.com/pgsty/silo/pull/37) | Open, queued for review |
| [@davinkevin](https://github.com/davinkevin) | Distroless-based Docker image variant | [#21](https://github.com/pgsty/silo/pull/21) | Superseded by the distroless variant shipped in RELEASE.2026-08-06, which the PR anticipated by four months |
| [@lem21h](https://github.com/lem21h) | Assorted fixes and improvements | [#36](https://github.com/pgsty/silo/pull/36) | Closed |
| [@sulin37392](https://github.com/sulin37392) | Dependency updates against the fork | [#8](https://github.com/pgsty/silo/pull/8) | Closed |
## Reports
Bug reports, compatibility findings, and proposals filed against this fork. Several shipped fixes
trace directly back to these: the bundled-client guarantee (#4, #9), the LDAP-over-TLS repair (#15),
the completed native package payload (#33), GPG-signed RPMs (#43), and the upstream migration guide
(#42).
| Contributor | Reported |
| :-- | :-- |
| [@mosesdd](https://github.com/mosesdd) | [#1](https://github.com/pgsty/silo/issues/1) Helm chart availability |
| [@Xavier-777](https://github.com/Xavier-777) | [#2](https://github.com/pgsty/silo/issues/2) Console bucket lifecycle management · [#17](https://github.com/pgsty/silo/issues/17) Log and XML file preview |
| [@jiadzh](https://github.com/jiadzh) | [#3](https://github.com/pgsty/silo/issues/3) Windows build guidance |
| [@TLINDEN](https://github.com/TLINDEN) | [#4](https://github.com/pgsty/silo/issues/4) `mc` missing from released tarballs |
| [@AntonOfTheWoods](https://github.com/AntonOfTheWoods) | [#5](https://github.com/pgsty/silo/issues/5) Upstream Helm chart and operator options |
| [@zylpsrs](https://github.com/zylpsrs) | [#6](https://github.com/pgsty/silo/issues/6) Console missing Tiering and Site Replication |
| [@nsanitate](https://github.com/nsanitate) | [#7](https://github.com/pgsty/silo/issues/7) CNCF Sandbox governance proposal |
| [@makinikm](https://github.com/makinikm) | [#9](https://github.com/pgsty/silo/issues/9) `mc` missing from the Docker image |
| [@magicxor](https://github.com/magicxor) | [#10](https://github.com/pgsty/silo/issues/10) `DeleteObject` ignores the `If-Match` header |
| [@spaceg00se-r](https://github.com/spaceg00se-r) | [#11](https://github.com/pgsty/silo/issues/11) `cpuv1` support · [#14](https://github.com/pgsty/silo/issues/14) Project workflow token failure |
| [@heroes1412](https://github.com/heroes1412) | [#13](https://github.com/pgsty/silo/issues/13) Profile option unusable |
| [@vampywiz17](https://github.com/vampywiz17) | [#15](https://github.com/pgsty/silo/issues/15) LDAP TLS regression breaking Console login on Kubernetes |
| [@davinkevin](https://github.com/davinkevin) | [#20](https://github.com/pgsty/silo/issues/20) Renovate for automated dependency updates |
| [@chalukyaj](https://github.com/chalukyaj) | [#30](https://github.com/pgsty/silo/issues/30) Silo Operator discoverability |
| [@cbornet](https://github.com/cbornet) | [#31](https://github.com/pgsty/silo/issues/31) Multipart uploads with `FULL_OBJECT` CRC32 · [#32](https://github.com/pgsty/silo/issues/32) `ListObjects` bucket-existence semantics |
| [@jvasile](https://github.com/jvasile) | [#33](https://github.com/pgsty/silo/issues/33) `.deb` missing user, group, and default files |
| [@Kesavaambati](https://github.com/Kesavaambati) | [#35](https://github.com/pgsty/silo/issues/35) Community support for the Docker images |
| [@redfoxfox](https://github.com/redfoxfox) | [#38](https://github.com/pgsty/silo/issues/38) Chinese documentation site unreachable |
| [@kuldeep-link11](https://github.com/kuldeep-link11) | [#39](https://github.com/pgsty/silo/issues/39) `notify_nats` rejects JWT credentials files · [#40](https://github.com/pgsty/silo/issues/40) `notify_nats` target changes require a restart |
| [@meesudzu](https://github.com/meesudzu) | [#42](https://github.com/pgsty/silo/issues/42) Migration guide from upstream MinIO |
| [@pmezhuev](https://github.com/pmezhuev) | [#43](https://github.com/pgsty/silo/issues/43) RPM package missing its GPG signature |
| [@kh0mka](https://github.com/kh0mka) | [#51](https://github.com/pgsty/silo/issues/51) Inter-node I/O timeout in `ReadFileStreamHandler` |
## Adding yourself
Contributions are accepted inbound=outbound under AGPL-3.0-or-later with no CLA; see
[`CONTRIBUTING.md`](CONTRIBUTING.md). Merged pull requests are added here at the next release. If a
contribution is missing or recorded incorrectly, open an issue or say so on the pull request and it
will be fixed.
+6120 -4101
View File
File diff suppressed because it is too large Load Diff
+18
View File
@@ -0,0 +1,18 @@
FROM minio/minio:latest
ARG TARGETARCH
ARG RELEASE
RUN chmod -R 777 /usr/bin
COPY ./minio-${TARGETARCH}.${RELEASE} /usr/bin/minio
COPY ./minio-${TARGETARCH}.${RELEASE}.minisig /usr/bin/minio.minisig
COPY ./minio-${TARGETARCH}.${RELEASE}.sha256sum /usr/bin/minio.sha256sum
COPY dockerscripts/docker-entrypoint.sh /usr/bin/docker-entrypoint.sh
ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"]
VOLUME ["/data"]
CMD ["minio"]
+3
View File
@@ -0,0 +1,3 @@
FROM minio/minio:edge
CMD ["minio", "server", "/data"]
-50
View File
@@ -1,50 +0,0 @@
# The distroless variant ships exactly one program: the silo binary.
# No shell, no mc, no curl, no entrypoint script; health checking is
# provided by the binary itself (`silo healthcheck`).
# Design note: https://silo.pgsty.com/compatibility/feature/healthcheck/
# A distroless final stage cannot RUN anything, so /data is prepared in a
# throwaway stage. It ships world-writable (see pgsty/silo#55): Docker
# seeds fresh volumes from the image-layer mountpoint, no entrypoint
# exists to repair ownership at runtime, and 0777 is what keeps every
# privilege mode working, --user included.
FROM busybox:1.37.0 AS prep
RUN mkdir -p /prep/data && chmod 0777 /prep/data
FROM gcr.io/distroless/static-debian12:latest
LABEL org.opencontainers.image.title="Silo" \
org.opencontainers.image.description="S3-Interface Libre Object Storage (distroless)" \
org.opencontainers.image.url="https://silo.pgsty.com" \
org.opencontainers.image.source="https://github.com/pgsty/silo" \
org.opencontainers.image.licenses="AGPL-3.0-or-later" \
maintainer="PGSTY <https://silo.pgsty.com>"
ENV MINIO_ACCESS_KEY_FILE=access_key \
MINIO_SECRET_KEY_FILE=secret_key \
MINIO_ROOT_USER_FILE=access_key \
MINIO_ROOT_PASSWORD_FILE=secret_key \
MINIO_KMS_SECRET_KEY_FILE=kms_master_key \
MINIO_CONFIG_ENV_FILE=config.env \
HOME=/tmp
COPY --chmod=0755 silo /usr/bin/silo
# COPY of a directory copies its contents, not the directory entry, so an
# empty /prep/data would arrive as a default root:0755 /data and non-root
# runs would fail storage init. Copying the parent makes data/ itself a
# copied entry, which --chmod then actually applies to.
COPY --from=prep --chmod=0777 /prep/ /
COPY LICENSE NOTICE CREDITS /licenses/
EXPOSE 9000
VOLUME ["/data"]
# Exec form is mandatory: there is no /bin/sh in this image. `ready`
# rather than `live` because Docker health feeds start-order gating
# (readiness semantics); the two are identical unless KMS/etcd are used.
# The outer timeout stays above the probe's own 5s deadline so the
# probe can report its diagnostic line instead of being SIGKILLed.
HEALTHCHECK --interval=30s --timeout=10s --start-period=2m --start-interval=2s --retries=3 \
CMD ["/usr/bin/silo", "healthcheck", "ready"]
ENTRYPOINT ["/usr/bin/silo"]
+11 -19
View File
@@ -1,4 +1,4 @@
FROM golang:1.27.0-alpine AS build FROM golang:1.26.5-alpine AS build
ARG TARGETARCH ARG TARGETARCH
@@ -6,9 +6,7 @@ ENV GOPATH=/go
ENV CGO_ENABLED=0 ENV CGO_ENABLED=0
ARG MC_REPO=pgsty/mc ARG MC_REPO=pgsty/mc
ARG MC_VERSION=RELEASE.2026-08-06T00-00-00Z ARG MC_VERSION=latest
ARG MC_AMD64_SHA256=4b488bd30af54ad4214e5b654746677c79cd93dc6cad4be3aa2d09dbb48370ff
ARG MC_ARM64_SHA256=83f6fedb16ed9c1e8efa8aea6776203dff132bc474214543d5c0767ed2066c2f
RUN apk add -U --no-cache \ RUN apk add -U --no-cache \
ca-certificates \ ca-certificates \
@@ -16,8 +14,8 @@ RUN apk add -U --no-cache \
curl \ curl \
jq && \ jq && \
case "${TARGETARCH}" in \ case "${TARGETARCH}" in \
amd64) MC_ARCH=amd64; MC_PINNED_SHA256="${MC_AMD64_SHA256}" ;; \ amd64) MC_ARCH=amd64 ;; \
arm64) MC_ARCH=arm64; MC_PINNED_SHA256="${MC_ARM64_SHA256}" ;; \ arm64) MC_ARCH=arm64 ;; \
*) echo "Unsupported TARGETARCH=${TARGETARCH}"; exit 1 ;; \ *) echo "Unsupported TARGETARCH=${TARGETARCH}"; exit 1 ;; \
esac && \ esac && \
if [ "${MC_VERSION}" = "latest" ]; then \ if [ "${MC_VERSION}" = "latest" ]; then \
@@ -41,8 +39,7 @@ RUN apk add -U --no-cache \
EXPECTED=$(grep " ${ARCHIVE_NAME}$" /tmp/mcli_checksums.txt | awk '{print $1}') && \ EXPECTED=$(grep " ${ARCHIVE_NAME}$" /tmp/mcli_checksums.txt | awk '{print $1}') && \
ACTUAL=$(sha256sum /tmp/mcli.tar.gz | awk '{print $1}') && \ ACTUAL=$(sha256sum /tmp/mcli.tar.gz | awk '{print $1}') && \
[ -n "${EXPECTED}" ] || { echo "Checksum entry not found for ${ARCHIVE_NAME}"; exit 1; } && \ [ -n "${EXPECTED}" ] || { echo "Checksum entry not found for ${ARCHIVE_NAME}"; exit 1; } && \
[ "${EXPECTED}" = "${MC_PINNED_SHA256}" ] || { echo "Published checksum drift for ${ARCHIVE_NAME}"; exit 1; } && \ [ "${EXPECTED}" = "${ACTUAL}" ] || { echo "Checksum mismatch: expected ${EXPECTED}, got ${ACTUAL}"; exit 1; } && \
[ "${MC_PINNED_SHA256}" = "${ACTUAL}" ] || { echo "Checksum mismatch: expected ${MC_PINNED_SHA256}, got ${ACTUAL}"; exit 1; } && \
echo "Checksum OK: ${ACTUAL}" && \ echo "Checksum OK: ${ACTUAL}" && \
mkdir -p /tmp/mcli-extract && \ mkdir -p /tmp/mcli-extract && \
tar -xzf /tmp/mcli.tar.gz -C /tmp/mcli-extract/ && \ tar -xzf /tmp/mcli.tar.gz -C /tmp/mcli-extract/ && \
@@ -69,36 +66,31 @@ RUN dnf -y install ca-certificates && \
FROM registry.access.redhat.com/ubi9/ubi-micro:latest FROM registry.access.redhat.com/ubi9/ubi-micro:latest
LABEL org.opencontainers.image.title="Silo" \ LABEL maintainer="pgsty <https://github.com/pgsty/minio>" \
org.opencontainers.image.description="S3-Interface Libre Object Storage" \ description="MinIO community fork, build by pgsty"
org.opencontainers.image.url="https://silo.pgsty.com" \
org.opencontainers.image.source="https://github.com/pgsty/silo" \
org.opencontainers.image.licenses="AGPL-3.0-or-later" \
maintainer="PGSTY <https://silo.pgsty.com>"
ENV MINIO_ACCESS_KEY_FILE=access_key \ ENV MINIO_ACCESS_KEY_FILE=access_key \
MINIO_SECRET_KEY_FILE=secret_key \ MINIO_SECRET_KEY_FILE=secret_key \
MINIO_ROOT_USER_FILE=access_key \ MINIO_ROOT_USER_FILE=access_key \
MINIO_ROOT_PASSWORD_FILE=secret_key \ MINIO_ROOT_PASSWORD_FILE=secret_key \
MINIO_KMS_SECRET_KEY_FILE=kms_master_key \ MINIO_KMS_SECRET_KEY_FILE=kms_master_key \
MINIO_UPDATE_MINISIGN_PUBKEY="RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav" \
MINIO_CONFIG_ENV_FILE=config.env \ MINIO_CONFIG_ENV_FILE=config.env \
HOME=/tmp \
MC_CONFIG_DIR=/tmp/.mc MC_CONFIG_DIR=/tmp/.mc
COPY --from=certs /tmp/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt COPY --from=certs /tmp/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
COPY silo /usr/bin/silo COPY minio /usr/bin/minio
COPY --from=build /go/bin/mcli /usr/bin/mcli COPY --from=build /go/bin/mcli /usr/bin/mcli
COPY --from=build /go/bin/curl* /usr/bin/ COPY --from=build /go/bin/curl* /usr/bin/
COPY dockerscripts/docker-entrypoint.sh /usr/bin/docker-entrypoint.sh COPY dockerscripts/docker-entrypoint.sh /usr/bin/docker-entrypoint.sh
COPY LICENSE /licenses/LICENSE COPY LICENSE /licenses/LICENSE
COPY NOTICE /licenses/NOTICE
COPY CREDITS /licenses/CREDITS COPY CREDITS /licenses/CREDITS
RUN chmod +x /usr/bin/silo /usr/bin/mcli /usr/bin/docker-entrypoint.sh && \ RUN chmod +x /usr/bin/minio /usr/bin/mcli /usr/bin/docker-entrypoint.sh && \
ln -sf mcli /usr/bin/mc ln -sf mcli /usr/bin/mc
EXPOSE 9000 EXPOSE 9000
VOLUME ["/data"] VOLUME ["/data"]
ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"] ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"]
CMD ["silo"] CMD ["minio"]
+71
View File
@@ -0,0 +1,71 @@
FROM golang:1.26.5-alpine as build
ARG TARGETARCH
ARG RELEASE
ENV GOPATH=/go
ENV CGO_ENABLED=0
# Install curl and minisign
RUN apk add -U --no-cache ca-certificates && \
apk add -U --no-cache curl && \
go install aead.dev/minisign/cmd/minisign@v0.2.1
# Download minio binary and signature files
RUN curl -s -q https://dl.min.io/server/minio/hotfixes/linux-${TARGETARCH}/archive/minio.${RELEASE} -o /go/bin/minio && \
curl -s -q https://dl.min.io/server/minio/hotfixes/linux-${TARGETARCH}/archive/minio.${RELEASE}.minisig -o /go/bin/minio.minisig && \
curl -s -q https://dl.min.io/server/minio/hotfixes/linux-${TARGETARCH}/archive/minio.${RELEASE}.sha256sum -o /go/bin/minio.sha256sum && \
chmod +x /go/bin/minio
# Download mc binary and signature files
RUN curl -s -q https://dl.min.io/client/mc/release/linux-${TARGETARCH}/mc -o /go/bin/mc && \
curl -s -q https://dl.min.io/client/mc/release/linux-${TARGETARCH}/mc.minisig -o /go/bin/mc.minisig && \
curl -s -q https://dl.min.io/client/mc/release/linux-${TARGETARCH}/mc.sha256sum -o /go/bin/mc.sha256sum && \
chmod +x /go/bin/mc
RUN if [ "$TARGETARCH" = "amd64" ]; then \
curl -L -s -q https://github.com/moparisthebest/static-curl/releases/latest/download/curl-${TARGETARCH} -o /go/bin/curl; \
chmod +x /go/bin/curl; \
fi
# Verify binary signature using public key "RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGavRUN"
RUN minisign -Vqm /go/bin/minio -x /go/bin/minio.minisig -P RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav && \
minisign -Vqm /go/bin/mc -x /go/bin/mc.minisig -P RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav
FROM registry.access.redhat.com/ubi9/ubi-micro:latest
ARG RELEASE
LABEL name="MinIO" \
vendor="MinIO Inc <dev@min.io>" \
maintainer="MinIO Inc <dev@min.io>" \
version="${RELEASE}" \
release="${RELEASE}" \
summary="MinIO is a High Performance Object Storage, API compatible with Amazon S3 cloud storage service." \
description="MinIO object storage is fundamentally different. Designed for performance and the S3 API, it is 100% open-source. MinIO is ideal for large, private cloud environments with stringent security requirements and delivers mission-critical availability across a diverse range of workloads."
ENV MINIO_ACCESS_KEY_FILE=access_key \
MINIO_SECRET_KEY_FILE=secret_key \
MINIO_ROOT_USER_FILE=access_key \
MINIO_ROOT_PASSWORD_FILE=secret_key \
MINIO_KMS_SECRET_KEY_FILE=kms_master_key \
MINIO_UPDATE_MINISIGN_PUBKEY="RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav" \
MINIO_CONFIG_ENV_FILE=config.env \
MC_CONFIG_DIR=/tmp/.mc
RUN chmod -R 777 /usr/bin
COPY --from=build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
COPY --from=build /go/bin/minio* /usr/bin/
COPY --from=build /go/bin/mc* /usr/bin/
COPY --from=build /go/bin/cur* /usr/bin/
COPY CREDITS /licenses/CREDITS
COPY LICENSE /licenses/LICENSE
COPY dockerscripts/docker-entrypoint.sh /usr/bin/docker-entrypoint.sh
EXPOSE 9000
VOLUME ["/data"]
ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"]
CMD ["minio"]
+73
View File
@@ -0,0 +1,73 @@
FROM golang:1.26.5-alpine AS build
ARG TARGETARCH
ARG RELEASE
ENV GOPATH=/go
ENV CGO_ENABLED=0
WORKDIR /build
# Install curl and minisign
RUN apk add -U --no-cache ca-certificates && \
apk add -U --no-cache curl && \
apk add -U --no-cache bash && \
go install aead.dev/minisign/cmd/minisign@v0.2.1
# Download minio binary and signature files
RUN curl -s -q https://dl.min.io/server/minio/release/linux-${TARGETARCH}/archive/minio.${RELEASE} -o /go/bin/minio && \
curl -s -q https://dl.min.io/server/minio/release/linux-${TARGETARCH}/archive/minio.${RELEASE}.minisig -o /go/bin/minio.minisig && \
curl -s -q https://dl.min.io/server/minio/release/linux-${TARGETARCH}/archive/minio.${RELEASE}.sha256sum -o /go/bin/minio.sha256sum && \
chmod +x /go/bin/minio
# Download mc binary and signature files
RUN curl -s -q https://dl.min.io/client/mc/release/linux-${TARGETARCH}/mc -o /go/bin/mc && \
curl -s -q https://dl.min.io/client/mc/release/linux-${TARGETARCH}/mc.minisig -o /go/bin/mc.minisig && \
curl -s -q https://dl.min.io/client/mc/release/linux-${TARGETARCH}/mc.sha256sum -o /go/bin/mc.sha256sum && \
chmod +x /go/bin/mc
# Verify binary signature using public key "RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGavRUN"
RUN minisign -Vqm /go/bin/minio -x /go/bin/minio.minisig -P RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav && \
minisign -Vqm /go/bin/mc -x /go/bin/mc.minisig -P RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav
COPY dockerscripts/download-static-curl.sh /build/download-static-curl
RUN chmod +x /build/download-static-curl && \
/build/download-static-curl
FROM registry.access.redhat.com/ubi9/ubi-micro:latest
ARG RELEASE
LABEL name="MinIO" \
vendor="MinIO Inc <dev@min.io>" \
maintainer="MinIO Inc <dev@min.io>" \
version="${RELEASE}" \
release="${RELEASE}" \
summary="MinIO is a High Performance Object Storage, API compatible with Amazon S3 cloud storage service." \
description="MinIO object storage is fundamentally different. Designed for performance and the S3 API, it is 100% open-source. MinIO is ideal for large, private cloud environments with stringent security requirements and delivers mission-critical availability across a diverse range of workloads."
ENV MINIO_ACCESS_KEY_FILE=access_key \
MINIO_SECRET_KEY_FILE=secret_key \
MINIO_ROOT_USER_FILE=access_key \
MINIO_ROOT_PASSWORD_FILE=secret_key \
MINIO_KMS_SECRET_KEY_FILE=kms_master_key \
MINIO_UPDATE_MINISIGN_PUBKEY="RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav" \
MINIO_CONFIG_ENV_FILE=config.env \
MC_CONFIG_DIR=/tmp/.mc
RUN chmod -R 777 /usr/bin
COPY --from=build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
COPY --from=build /go/bin/minio* /usr/bin/
COPY --from=build /go/bin/mc* /usr/bin/
COPY --from=build /go/bin/curl* /usr/bin/
COPY CREDITS /licenses/CREDITS
COPY LICENSE /licenses/LICENSE
COPY dockerscripts/docker-entrypoint.sh /usr/bin/docker-entrypoint.sh
EXPOSE 9000
VOLUME ["/data"]
ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"]
CMD ["minio"]
+71
View File
@@ -0,0 +1,71 @@
FROM golang:1.26.5-alpine AS build
ARG TARGETARCH
ARG RELEASE
ENV GOPATH=/go
ENV CGO_ENABLED=0
# Install curl and minisign
RUN apk add -U --no-cache ca-certificates && \
apk add -U --no-cache curl && \
go install aead.dev/minisign/cmd/minisign@v0.2.1
# Download minio binary and signature files
RUN curl -s -q https://dl.min.io/server/minio/release/linux-${TARGETARCH}/archive/minio.${RELEASE} -o /go/bin/minio && \
curl -s -q https://dl.min.io/server/minio/release/linux-${TARGETARCH}/archive/minio.${RELEASE}.minisig -o /go/bin/minio.minisig && \
curl -s -q https://dl.min.io/server/minio/release/linux-${TARGETARCH}/archive/minio.${RELEASE}.sha256sum -o /go/bin/minio.sha256sum && \
chmod +x /go/bin/minio
# Download mc binary and signature files
RUN curl -s -q https://dl.min.io/client/mc/release/linux-${TARGETARCH}/mc -o /go/bin/mc && \
curl -s -q https://dl.min.io/client/mc/release/linux-${TARGETARCH}/mc.minisig -o /go/bin/mc.minisig && \
curl -s -q https://dl.min.io/client/mc/release/linux-${TARGETARCH}/mc.sha256sum -o /go/bin/mc.sha256sum && \
chmod +x /go/bin/mc
RUN if [ "$TARGETARCH" = "amd64" ]; then \
curl -L -s -q https://github.com/moparisthebest/static-curl/releases/latest/download/curl-${TARGETARCH} -o /go/bin/curl; \
chmod +x /go/bin/curl; \
fi
# Verify binary signature using public key "RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGavRUN"
RUN minisign -Vqm /go/bin/minio -x /go/bin/minio.minisig -P RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav && \
minisign -Vqm /go/bin/mc -x /go/bin/mc.minisig -P RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav
FROM registry.access.redhat.com/ubi8/ubi-micro:latest
ARG RELEASE
LABEL name="MinIO" \
vendor="MinIO Inc <dev@min.io>" \
maintainer="MinIO Inc <dev@min.io>" \
version="${RELEASE}" \
release="${RELEASE}" \
summary="MinIO is a High Performance Object Storage, API compatible with Amazon S3 cloud storage service." \
description="MinIO object storage is fundamentally different. Designed for performance and the S3 API, it is 100% open-source. MinIO is ideal for large, private cloud environments with stringent security requirements and delivers mission-critical availability across a diverse range of workloads."
ENV MINIO_ACCESS_KEY_FILE=access_key \
MINIO_SECRET_KEY_FILE=secret_key \
MINIO_ROOT_USER_FILE=access_key \
MINIO_ROOT_PASSWORD_FILE=secret_key \
MINIO_KMS_SECRET_KEY_FILE=kms_master_key \
MINIO_UPDATE_MINISIGN_PUBKEY="RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav" \
MINIO_CONFIG_ENV_FILE=config.env \
MC_CONFIG_DIR=/tmp/.mc
RUN chmod -R 777 /usr/bin
COPY --from=build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
COPY --from=build /go/bin/minio* /usr/bin/
COPY --from=build /go/bin/mc* /usr/bin/
COPY --from=build /go/bin/cur* /usr/bin/
COPY CREDITS /licenses/CREDITS
COPY LICENSE /licenses/LICENSE
COPY dockerscripts/docker-entrypoint.sh /usr/bin/docker-entrypoint.sh
EXPOSE 9000
VOLUME ["/data"]
ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"]
CMD ["minio"]
+5
View File
@@ -0,0 +1,5 @@
FROM scratch
COPY minio /minio
CMD ["/minio"]
+67 -64
View File
@@ -4,11 +4,11 @@ LDFLAGS := $(shell go run buildscripts/gen-ldflags.go)
GOOS ?= $(shell go env GOOS) GOOS ?= $(shell go env GOOS)
GOARCH ?= $(shell go env GOARCH) GOARCH ?= $(shell go env GOARCH)
GOLANGCI_VERSION ?= v2.13.1 GOLANGCI_VERSION ?= v2.11.3
VERSION ?= $(shell git describe --tags) VERSION ?= $(shell git describe --tags)
REPO ?= docker.io/pgsty REPO ?= quay.io/minio
TAG ?= $(REPO)/silo:$(VERSION) TAG ?= $(REPO)/minio:$(VERSION)
GOLANGCI_DIR = .bin/golangci/$(GOLANGCI_VERSION) GOLANGCI_DIR = .bin/golangci/$(GOLANGCI_VERSION)
GOLANGCI = $(GOLANGCI_DIR)/golangci-lint GOLANGCI = $(GOLANGCI_DIR)/golangci-lint
@@ -33,24 +33,15 @@ getdeps: ## fetch necessary dependencies
sh "$$script" -b $(GOLANGCI_DIR) $(GOLANGCI_VERSION); \ sh "$$script" -b $(GOLANGCI_DIR) $(GOLANGCI_VERSION); \
fi fi
crosscompile: ## cross compile Silo crosscompile: ## cross compile minio
@(env bash $(PWD)/buildscripts/cross-compile.sh) @(env bash $(PWD)/buildscripts/cross-compile.sh)
verifiers: lint check-gen rebrand-guard verifiers: lint check-gen
rebrand-guard: ## verify Silo branding and protected compatibility identifiers
@go run ./buildscripts/rebrand-guard
@env bash $(PWD)/buildscripts/verify-rebrand.sh
@env bash $(PWD)/dockerscripts/docker-entrypoint_test.sh
credits: ## regenerate CREDITS from the licenses of Go modules linked into the binary
@env bash $(PWD)/buildscripts/gen-credits.sh
check-gen: ## check for updated autogenerated files check-gen: ## check for updated autogenerated files
@go generate ./... >/dev/null @go generate ./... >/dev/null
@go mod tidy -compat=1.27 @go mod tidy -compat=1.26
@env bash $(PWD)/buildscripts/gen-credits.sh @changed=$$(git diff --name-only -- '*_gen.go' '*_gen_test.go' '*_msgp_test.go' '*_string.go' go.mod go.sum); \
@changed=$$(git diff --name-only -- '*_gen.go' '*_gen_test.go' '*_msgp_test.go' '*_string.go' go.mod go.sum CREDITS); \
if [ -n "$$changed" ]; then \ if [ -n "$$changed" ]; then \
echo "Non-committed generated changes detected:"; \ echo "Non-committed generated changes detected:"; \
echo "$$changed"; \ echo "$$changed"; \
@@ -73,12 +64,12 @@ lint-fix: getdeps ## runs golangci-lint suite of linters with automatic fixes
@$(GOLANGCI) run --build-tags kqueue --timeout=10m --config ./.golangci.yml --fix @$(GOLANGCI) run --build-tags kqueue --timeout=10m --config ./.golangci.yml --fix
check: test check: test
test: verifiers build ## builds Silo, runs linters, tests test: verifiers build ## builds minio, runs linters, tests
@echo "Running unit tests" @echo "Running unit tests"
@MINIO_API_REQUESTS_MAX=10000 CGO_ENABLED=0 go test -v -tags kqueue,dev ./... @MINIO_API_REQUESTS_MAX=10000 CGO_ENABLED=0 go test -v -tags kqueue,dev ./...
test-root-disable: install-race test-root-disable: install-race
@echo "Running Silo root lockdown tests" @echo "Running minio root lockdown tests"
@env bash $(PWD)/buildscripts/disable-root.sh @env bash $(PWD)/buildscripts/disable-root.sh
test-ilm: install-race test-ilm: install-race
@@ -94,7 +85,7 @@ test-pbac: install-race
@env bash $(PWD)/docs/iam/policies/pbac-tests.sh @env bash $(PWD)/docs/iam/policies/pbac-tests.sh
test-decom: install-race test-decom: install-race
@echo "Running Silo decom tests" @echo "Running minio decom tests"
@env bash $(PWD)/docs/distributed/decom.sh @env bash $(PWD)/docs/distributed/decom.sh
@env bash $(PWD)/docs/distributed/decom-encrypted.sh @env bash $(PWD)/docs/distributed/decom-encrypted.sh
@env bash $(PWD)/docs/distributed/decom-encrypted-sse-s3.sh @env bash $(PWD)/docs/distributed/decom-encrypted-sse-s3.sh
@@ -102,17 +93,17 @@ test-decom: install-race
@env bash $(PWD)/docs/distributed/decom-encrypted-kes.sh @env bash $(PWD)/docs/distributed/decom-encrypted-kes.sh
test-versioning: install-race test-versioning: install-race
@echo "Running Silo versioning tests" @echo "Running minio versioning tests"
@env bash $(PWD)/docs/bucket/versioning/versioning-tests.sh @env bash $(PWD)/docs/bucket/versioning/versioning-tests.sh
test-configfile: install-race test-configfile: install-race
@env bash $(PWD)/docs/distributed/distributed-from-config-file.sh @env bash $(PWD)/docs/distributed/distributed-from-config-file.sh
test-upgrade: test-upgrade: install-race
@echo "Running MinIO-to-Silo upgrade tests" @echo "Running minio upgrade tests"
@(env bash $(PWD)/buildscripts/minio-upgrade.sh) @(env bash $(PWD)/buildscripts/minio-upgrade.sh)
test-race: verifiers build ## builds Silo, runs linters, tests (race) test-race: verifiers build ## builds minio, runs linters, tests (race)
@echo "Running unit tests under -race" @echo "Running unit tests under -race"
@(env bash $(PWD)/buildscripts/race.sh) @(env bash $(PWD)/buildscripts/race.sh)
@@ -160,9 +151,9 @@ test-site-replication-oidc: install-race ## verify automatic site replication
@echo "Running tests for automatic site replication of IAM (with OIDC)" @echo "Running tests for automatic site replication of IAM (with OIDC)"
@(env bash $(PWD)/docs/site-replication/run-multi-site-oidc.sh) @(env bash $(PWD)/docs/site-replication/run-multi-site-oidc.sh)
test-site-replication-silo: install-race ## verify automatic site replication test-site-replication-minio: install-race ## verify automatic site replication
@echo "Running tests for automatic site replication of IAM (with Silo IDP)" @echo "Running tests for automatic site replication of IAM (with MinIO IDP)"
@(env bash $(PWD)/docs/site-replication/run-multi-site-silo-idp.sh) @(env bash $(PWD)/docs/site-replication/run-multi-site-minio-idp.sh)
@echo "Running tests for automatic site replication of SSE-C objects" @echo "Running tests for automatic site replication of SSE-C objects"
@(env bash $(PWD)/docs/site-replication/run-ssec-object-replication.sh) @(env bash $(PWD)/docs/site-replication/run-ssec-object-replication.sh)
@echo "Running tests for automatic site replication of SSE-C objects with SSE-KMS enabled for bucket" @echo "Running tests for automatic site replication of SSE-C objects with SSE-KMS enabled for bucket"
@@ -178,11 +169,11 @@ test-timeout: install-race ## test multipart
@echo "Test server timeout" @echo "Test server timeout"
@(env bash $(PWD)/buildscripts/test-timeout.sh) @(env bash $(PWD)/buildscripts/test-timeout.sh)
verify: install-race ## verify Silo in various setups verify: install-race ## verify minio various setups
@echo "Verifying build with race" @echo "Verifying build with race"
@(env bash $(PWD)/buildscripts/verify-build.sh) @(env bash $(PWD)/buildscripts/verify-build.sh)
verify-healing: install-race ## verify healing and replacing disks with the Silo binary verify-healing: install-race ## verify healing and replacing disks with minio binary
@echo "Verify healing build with race" @echo "Verify healing build with race"
@(env bash $(PWD)/buildscripts/verify-healing.sh) @(env bash $(PWD)/buildscripts/verify-healing.sh)
@(env bash $(PWD)/buildscripts/verify-healing-empty-erasure-set.sh) @(env bash $(PWD)/buildscripts/verify-healing-empty-erasure-set.sh)
@@ -203,49 +194,59 @@ verify-healing-inconsistent-versions: install-race ## verify resolving inconsist
build-debugging: build-debugging:
@(env bash $(PWD)/docs/debugging/build.sh) @(env bash $(PWD)/docs/debugging/build.sh)
build: checks build-debugging ## builds Silo to $(PWD) build: checks build-debugging ## builds minio to $(PWD)
@echo "Building Silo binary to './silo'" @echo "Building minio binary to './minio'"
@CGO_ENABLED=0 GOOS=$(GOOS) GOARCH=$(GOARCH) go build -tags kqueue -trimpath --ldflags "$(LDFLAGS)" -o $(PWD)/silo 1>/dev/null @CGO_ENABLED=0 GOOS=$(GOOS) GOARCH=$(GOARCH) go build -tags kqueue -trimpath --ldflags "$(LDFLAGS)" -o $(PWD)/minio 1>/dev/null
docker: checks build-debugging ## builds the local Linux Silo container image hotfix-vars:
@echo "Building Silo container image '$(TAG)'" $(eval LDFLAGS := $(shell MINIO_RELEASE="RELEASE" MINIO_HOTFIX="hotfix.$(shell git rev-parse --short HEAD)" go run buildscripts/gen-ldflags.go $(shell git describe --tags --abbrev=0 | \
@set -e; \ sed 's#RELEASE\.\([0-9]\+\)-\([0-9]\+\)-\([0-9]\+\)T\([0-9]\+\)-\([0-9]\+\)-\([0-9]\+\)Z#\1-\2-\3T\4:\5:\6Z#')))
context=$$(mktemp -d); \ $(eval VERSION := $(shell git describe --tags --abbrev=0).hotfix.$(shell git rev-parse --short HEAD))
trap 'rm -rf "$$context"' EXIT; \
CGO_ENABLED=0 GOOS=linux GOARCH=$(GOARCH) go build -tags kqueue -trimpath \
--ldflags "$(LDFLAGS)" -o "$$context/silo"; \
mkdir -p "$$context/dockerscripts"; \
cp Dockerfile.goreleaser LICENSE NOTICE CREDITS "$$context/"; \
cp dockerscripts/docker-entrypoint.sh dockerscripts/download-static-curl.sh \
"$$context/dockerscripts/"; \
docker build -q --no-cache --platform linux/$(GOARCH) -t $(TAG) --build-arg TARGETARCH=$(GOARCH) \
-f "$$context/Dockerfile.goreleaser" "$$context"
docker-distroless: checks build-debugging ## builds the local Linux Silo distroless container image hotfix: hotfix-vars clean install ## builds minio binary with hotfix tags
@echo "Building Silo distroless container image '$(TAG)-distroless'" @wget -q -c https://github.com/minio/pkger/releases/download/v2.3.11/pkger_2.3.11_linux_amd64.deb
@set -e; \ @wget -q -c https://raw.githubusercontent.com/minio/minio-service/v1.1.1/linux-systemd/distributed/minio.service
context=$$(mktemp -d); \ @sudo apt install ./pkger_2.3.11_linux_amd64.deb --yes
trap 'rm -rf "$$context"' EXIT; \ @mkdir -p minio-release/$(GOOS)-$(GOARCH)/archive
CGO_ENABLED=0 GOOS=linux GOARCH=$(GOARCH) go build -tags kqueue -trimpath \ @cp -af ./minio minio-release/$(GOOS)-$(GOARCH)/minio
--ldflags "$(LDFLAGS)" -o "$$context/silo"; \ @cp -af ./minio minio-release/$(GOOS)-$(GOARCH)/minio.$(VERSION)
cp Dockerfile.distroless LICENSE NOTICE CREDITS "$$context/"; \ @minisign -qQSm minio-release/$(GOOS)-$(GOARCH)/minio.$(VERSION) -s "${CRED_DIR}/minisign.key" < "${CRED_DIR}/minisign-passphrase"
docker build -q --no-cache --platform linux/$(GOARCH) -t $(TAG)-distroless \ @sha256sum < minio-release/$(GOOS)-$(GOARCH)/minio.$(VERSION) | sed 's, -,minio.$(VERSION),g' > minio-release/$(GOOS)-$(GOARCH)/minio.$(VERSION).sha256sum
-f "$$context/Dockerfile.distroless" "$$context" @cp -af minio-release/$(GOOS)-$(GOARCH)/minio.$(VERSION)* minio-release/$(GOOS)-$(GOARCH)/archive/
@pkger -r $(VERSION) --ignore
hotfix-push: hotfix
@scp -q -r minio-release/$(GOOS)-$(GOARCH)/* minio@dl-0.minio.io:~/releases/server/minio/hotfixes/linux-$(GOOS)/
@scp -q -r minio-release/$(GOOS)-$(GOARCH)/* minio@dl-0.minio.io:~/releases/server/minio/hotfixes/linux-$(GOOS)/archive
@scp -q -r minio-release/$(GOOS)-$(GOARCH)/* minio@dl-1.minio.io:~/releases/server/minio/hotfixes/linux-$(GOOS)/
@scp -q -r minio-release/$(GOOS)-$(GOARCH)/* minio@dl-1.minio.io:~/releases/server/minio/hotfixes/linux-$(GOOS)/archive
@echo "Published new hotfix binaries at https://dl.min.io/server/minio/hotfixes/linux-$(GOOS)/archive/minio.$(VERSION)"
docker-hotfix-push: docker-hotfix
@docker push -q $(TAG) && echo "Published new container $(TAG)"
docker-hotfix: hotfix-push checks ## builds minio docker container with hotfix tags
@echo "Building minio docker image '$(TAG)'"
@docker build -q --no-cache -t $(TAG) --build-arg RELEASE=$(VERSION) . -f Dockerfile.hotfix
docker: build ## builds minio docker container
@echo "Building minio docker image '$(TAG)'"
@docker build -q --no-cache -t $(TAG) . -f Dockerfile
test-resiliency: build test-resiliency: build
@echo "Running resiliency tests" @echo "Running resiliency tests"
@(DOCKER_COMPOSE_FILE=$(PWD)/docs/resiliency/docker-compose.yaml env bash $(PWD)/docs/resiliency/resiliency-tests.sh) @(DOCKER_COMPOSE_FILE=$(PWD)/docs/resiliency/docker-compose.yaml env bash $(PWD)/docs/resiliency/resiliency-tests.sh)
install-race: checks build-debugging ## builds Silo to $(PWD) install-race: checks build-debugging ## builds minio to $(PWD)
@echo "Building Silo binary with -race to './silo'" @echo "Building minio binary with -race to './minio'"
@GORACE=history_size=7 CGO_ENABLED=1 go build -tags kqueue,dev -race -trimpath --ldflags "$(LDFLAGS)" -o $(PWD)/silo 1>/dev/null @GORACE=history_size=7 CGO_ENABLED=1 go build -tags kqueue,dev -race -trimpath --ldflags "$(LDFLAGS)" -o $(PWD)/minio 1>/dev/null
@echo "Installing Silo binary with -race to '$(GOPATH)/bin/silo'" @echo "Installing minio binary with -race to '$(GOPATH)/bin/minio'"
@mkdir -p $(GOPATH)/bin && cp -af $(PWD)/silo $(GOPATH)/bin/silo @mkdir -p $(GOPATH)/bin && cp -af $(PWD)/minio $(GOPATH)/bin/minio
install: build ## builds Silo and installs it to $GOPATH/bin. install: build ## builds minio and installs it to $GOPATH/bin.
@echo "Installing Silo binary to '$(GOPATH)/bin/silo'" @echo "Installing minio binary to '$(GOPATH)/bin/minio'"
@mkdir -p $(GOPATH)/bin && cp -af $(PWD)/silo $(GOPATH)/bin/silo @mkdir -p $(GOPATH)/bin && cp -af $(PWD)/minio $(GOPATH)/bin/minio
@echo "Installation successful. To learn more, try \"silo --help\"." @echo "Installation successful. To learn more, try \"minio --help\"."
clean: ## cleanup all generated assets clean: ## cleanup all generated assets
@echo "Cleaning up all the generated files" @echo "Cleaning up all the generated files"
@@ -253,8 +254,10 @@ clean: ## cleanup all generated assets
@find . -name '*~' | xargs rm -fv @find . -name '*~' | xargs rm -fv
@find . -name '.#*#' | xargs rm -fv @find . -name '.#*#' | xargs rm -fv
@find . -name '#*#' | xargs rm -fv @find . -name '#*#' | xargs rm -fv
@rm -rvf silo @rm -rvf minio
@rm -rvf build @rm -rvf build
@rm -rvf release @rm -rvf release
@rm -rvf .verify* @rm -rvf .verify*
@rm -rvf minio-release
@rm -rvf minio.RELEASE*.hotfix.*
@rm -rvf pkger_*.deb @rm -rvf pkger_*.deb
+1 -7
View File
@@ -1,4 +1,4 @@
MinIO Project, (C) 2015-2025 MinIO, Inc. MinIO Project, (C) 2015-2023 MinIO, Inc.
This product includes software developed at MinIO, Inc. This product includes software developed at MinIO, Inc.
(https://min.io/). (https://min.io/).
@@ -7,9 +7,3 @@ The MinIO project contains unmodified/modified subcomponents too with
separate copyright notices and license terms. Your use of the source separate copyright notices and license terms. Your use of the source
code for these subcomponents is subject to the terms and conditions code for these subcomponents is subject to the terms and conditions
of GNU Affero General Public License 3.0. of GNU Affero General Public License 3.0.
Silo Project modifications, (C) 2025-2026 PGSTY.
Silo is an independent community-maintained project incorporating MinIO
source code. It is not affiliated with or endorsed by MinIO, Inc. Modified
source and Silo release artifacts are maintained by the Silo project.
+12 -12
View File
@@ -1,12 +1,12 @@
# Silo Pull Request Guidelines # MinIO Pull Request Guidelines
These guidelines ensure high-quality commits in Silo's GitHub repositories, maintaining These guidelines ensure high-quality commits in MinIO’s GitHub repositories, maintaining
a clear, valuable commit history for our open-source projects. They apply to all contributors, a clear, valuable commit history for our open-source projects. They apply to all contributors,
fostering efficient reviews and robust code. fostering efficient reviews and robust code.
## Why Pull Requests? ## Why Pull Requests?
Pull Requests (PRs) drive quality in Silo's codebase by: Pull Requests (PRs) drive quality in MinIO’s codebase by:
- Enabling peer review without pair programming. - Enabling peer review without pair programming.
- Documenting changes for future reference. - Documenting changes for future reference.
- Ensuring commits tell a clear story of development. - Ensuring commits tell a clear story of development.
@@ -15,12 +15,12 @@ Pull Requests (PRs) drive quality in Silo's codebase by:
## Crafting a Quality PR ## Crafting a Quality PR
A strong Silo PR: A strong MinIO PR:
- Delivers a complete, valuable change (feature, bug fix, or improvement). - Delivers a complete, valuable change (feature, bug fix, or improvement).
- Has a concise title (e.g., `[S3] Fix bucket policy parsing #1234`) and a summary with context, referencing issues (e.g., `#1234`). - Has a concise title (e.g., `[S3] Fix bucket policy parsing #1234`) and a summary with context, referencing issues (e.g., `#1234`).
- Contains well-written, logical commits explaining *why* changes were made (e.g., “Add S3 bucket tagging support so that users can organize resources efficiently”). - Contains well-written, logical commits explaining *why* changes were made (e.g., “Add S3 bucket tagging support so that users can organize resources efficiently”).
- Is small, focused, and easy to review—ideally one commit, unless multiple commits better narrate complex work. - Is small, focused, and easy to review—ideally one commit, unless multiple commits better narrate complex work.
- Adheres to Silo's coding standards (e.g., Go style, error handling, testing). - Adheres to MinIO’s coding standards (e.g., Go style, error handling, testing).
PRs must flow smoothly through review to reach production. Large PRs should be split into smaller, manageable ones. PRs must flow smoothly through review to reach production. Large PRs should be split into smaller, manageable ones.
@@ -48,14 +48,14 @@ PRs must flow smoothly through review to reach production. Large PRs should be s
## Reviewing PRs ## Reviewing PRs
Reviewers ensure Silo's commit history remains a clear, reliable record. Responsibilities include: Reviewers ensure MinIO’s commit history remains a clear, reliable record. Responsibilities include:
1. **Commit Quality**: 1. **Commit Quality**:
- Verify each commit explains *why* the change was made (e.g., “So that…”). - Verify each commit explains *why* the change was made (e.g., “So that…”).
- Request rebasing if commits are unclear, redundant, or lack context (e.g., “Please squash typo fixes into the parent commit”). - Request rebasing if commits are unclear, redundant, or lack context (e.g., “Please squash typo fixes into the parent commit”).
2. **Code Quality**: 2. **Code Quality**:
- Check adherence to Silo's Go standards (e.g., error handling, documentation). - Check adherence to MinIO’s Go standards (e.g., error handling, documentation).
- Ensure tests cover new code and pass CI. - Ensure tests cover new code and pass CI.
- Flag bugs or critical issues for immediate fixes; suggest non-blocking improvements as follow-up issues. - Flag bugs or critical issues for immediate fixes; suggest non-blocking improvements as follow-up issues.
@@ -65,7 +65,7 @@ Reviewers ensure Silo's commit history remains a clear, reliable record. Respons
- If unable to complete the review, tag another reviewer (e.g., `@username please take over`). - If unable to complete the review, tag another reviewer (e.g., `@username please take over`).
4. **Shared Responsibility**: 4. **Shared Responsibility**:
- All Silo contributors are reviewers. The first commenter on a PR owns the review unless they delegate. - All MinIO contributors are reviewers. The first commenter on a PR owns the review unless they delegate.
- Multiple reviewers are encouraged for complex PRs. - Multiple reviewers are encouraged for complex PRs.
5. **No Self-Edits**: 5. **No Self-Edits**:
@@ -80,14 +80,14 @@ Reviewers ensure Silo's commit history remains a clear, reliable record. Respons
- **Small PRs**: Easier to review, faster to merge. Split large changes logically. - **Small PRs**: Easier to review, faster to merge. Split large changes logically.
- **Clear Commits**: Use `git rebase -i` to refine history before submitting. - **Clear Commits**: Use `git rebase -i` to refine history before submitting.
- **Engage Early**: Discuss complex changes in a GitHub issue before coding. - **Engage Early**: Discuss complex changes in issues or Slack (https://slack.min.io) before coding.
- **Be Responsive**: Address reviewer feedback promptly to keep PRs moving. - **Be Responsive**: Address reviewer feedback promptly to keep PRs moving.
- **Learn from Reviews**: Use feedback to improve future contributions. - **Learn from Reviews**: Use feedback to improve future contributions.
## Resources ## Resources
- [Silo Contribution Guide](CONTRIBUTING.md) - [MinIO Coding Standards](https://github.com/minio/minio/blob/master/CONTRIBUTING.md)
- [Effective Commit Messages](https://mislav.net/2014/02/hidden-documentation/) - [Effective Commit Messages](https://mislav.net/2014/02/hidden-documentation/)
- [GitHub PR Tips](https://github.com/blog/1943-how-to-write-the-perfect-pull-request) - [GitHub PR Tips](https://github.com/blog/1943-how-to-write-the-perfect-pull-request)
By following these guidelines, we ensure Silo's codebase remains high-quality, maintainable, and a joy to contribute to. Happy coding! By following these guidelines, we ensure MinIO’s codebase remains high-quality, maintainable, and a joy to contribute to. Happy coding!
+122 -118
View File
@@ -1,161 +1,165 @@
> [!WARNING]
> **This branch is archived and receives no further changes.**
>
> `minio` holds the final state of this project under the MinIO identity. Development continues on **[`main`](https://github.com/pgsty/silo/tree/main)**, where the project is named **Silo** and the binary, packages, systemd unit, container image, and Helm chart are all named `silo`. On 2026-08-06 the repository was renamed `pgsty/minio` → **[`pgsty/silo`](https://github.com/pgsty/silo)** and its default branch `master` → `main`.
>
> The last release cut from this branch is **[`RELEASE.2026-08-04T00-00-00Z`](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-08-04T00-00-00Z)** (2026-08-04). Its 19 assets carry the `minio` artifact names, and the matching container image is `docker.io/pgsty/minio:RELEASE.2026-08-04T00-00-00Z`. Those artifacts stay published and unmodified — no tag is moved, re-signed, or removed. If you need the artifacts maintained under the original MinIO identity, this branch and the releases up to that tag are where they live. Later releases carry the `silo` names.
>
> **Nothing on the wire changed with the rename.** `MINIO_*` environment variables, `minio_*` Prometheus metrics, `x-minio-*` headers, `/minio/*` routes, the `.minio.sys` on-disk layout, IAM and ARN values, and the `github.com/minio/minio` Go module path are all preserved on `main`. A Silo server reads data written by this release, and the packages install side by side, so migrating or rolling back stays an explicit administrator action.
<h1 align="center"> <h1 align="center">
<a href="https://silo.pgsty.com/"> <img src=".github/silo-word.svg" alt="SILO" height="80">
<img src=".github/silo-logo.svg" alt="Silo" width="160">
</a>
</h1> </h1>
<p align="center"> <p align="center">
<strong>S3-compatible object storage — a MinIO fork maintained by PGSTY</strong> <strong>A conservatively maintained MinIO fork</strong><br>
Security maintenance, versioned release artifacts, and operational continuity for existing deployments.
</p> </p>
<p align="center"> <p align="center">
<a href="https://silo.pgsty.com/">Website</a> · <a href="https://silo.pgsty.com/">Website</a> ·
<a href="https://silo.pgsty.com/docs/">Documentation</a> · <a href="https://silo.pgsty.com/docs/">Documentation</a> ·
<a href="https://silo.pgsty.com/download/">Download</a> · <a href="https://silo.pgsty.com/download/">Download</a> ·
<a href="https://silo.pgsty.com/tags/silo/">Release Notes</a> · <a href="https://silo.pgsty.com/blog/">Blog</a> ·
<a href="https://silo.pgsty.com/compatibility/server/">Compatibility</a> · <a href="https://github.com/pgsty/minio/releases">Releases</a> ·
<a href="https://silo.pgsty.com/about/manifesto/">Manifesto</a> ·
<a href="SECURITY.md">Security</a> · <a href="SECURITY.md">Security</a> ·
<a href="README_ZH.md">中文</a> <a href="README_ZH.md">中文</a>
</p> </p>
<p align="center"> <p align="center">
<a href="https://silo.pgsty.com/"><img alt="Website" src="https://img.shields.io/badge/Website-silo.pgsty.com-1d588c"></a> <a href="https://github.com/pgsty/minio/releases"><img alt="GitHub Release" src="https://img.shields.io/github/v/release/pgsty/minio?include_prereleases&label=release&logo=github"></a>
<a href="https://github.com/pgsty/silo/releases"><img alt="GitHub Release" src="https://img.shields.io/github/v/release/pgsty/silo?include_prereleases&label=release&logo=github"></a> <a href="https://hub.docker.com/r/pgsty/minio"><img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/pgsty/minio?logo=docker"></a>
<a href="https://hub.docker.com/r/pgsty/silo"><img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/pgsty/minio?logo=docker"></a> <a href="go.mod"><img alt="Go Version" src="https://img.shields.io/github/go-mod/go-version/pgsty/minio?logo=go"></a>
<a href="go.mod"><img alt="Go Version" src="https://img.shields.io/github/go-mod/go-version/pgsty/silo?logo=go"></a>
<a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/license-AGPLv3-blue"></a> <a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/license-AGPLv3-blue"></a>
</p> </p>
> [!IMPORTANT] > [!IMPORTANT]
> **PGSTY Silo** (hereinafter “Silo”) is an independent, community-maintained fork of the open-source MinIO server, published by [Pigsty](https://pigsty.io) from [`pgsty/silo`](https://github.com/pgsty/silo). It is not affiliated with, endorsed by, or sponsored by MinIO, Inc. “MinIO” is used only to identify the upstream project and compatibility lineage. > Silo is an independent, community-maintained fork of the open-source MinIO server, published by [Pigsty](https://pigsty.io) from [`pgsty/minio`](https://github.com/pgsty/minio). It is not affiliated with, endorsed by, or sponsored by MinIO, Inc. “MinIO” is used only to identify the upstream project and compatibility lineage.
> [!NOTE]
> Renamed from `pgsty/minio` to `pgsty/silo`, default branch `master` → `main`, on 2026-08-06. Artifacts under the original MinIO identity stay published on the archived [`minio`](https://github.com/pgsty/silo/tree/minio) branch and in releases up to [`RELEASE.2026-08-04T00-00-00Z`](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-08-04T00-00-00Z).
## Overview ## Overview
PGSTY SILO keeps one maintained release line of the open-source MinIO server alive after upstream ended community distribution: builds, packages, multi-arch images, security fixes, and the full web console. Pigsty runs it in production as its PostgreSQL backup repository. Silo maintains one downstream release line based on MinIO [`RELEASE.2025-12-03T12-00-00Z`](https://github.com/minio/minio/releases/tag/RELEASE.2025-12-03T12-00-00Z). It provides maintained builds and release artifacts for existing MinIO-compatible deployments after upstream community distribution ended. Pigsty uses this fork for object storage as an optional PG backup repo.
It follows one rule — **the product and its delivery surfaces are renamed; the protocol and your data are not.** Everything else lives on [silo.pgsty.com](https://silo.pgsty.com/). The official project portal is [silo.pgsty.com](https://silo.pgsty.com/). It brings documentation, downloads, release and security notes, and project background together. English is served at the site root; Chinese is available under [/zh/](https://silo.pgsty.com/zh/).
**Related:** [`pgsty/mc`](https://github.com/pgsty/mc) client (shipped as `mcli`) · [`pgsty/silo-console`](https://github.com/pgsty/silo-console) · [`pgsty/silo-pkg`](https://github.com/pgsty/silo-pkg) · [`pgsty/pigsty`](https://github.com/pgsty/pigsty) ## Find the Right Resource
<p align="center"> | Looking for | Canonical location |
<img src="https://silo.pgsty.com/images/silo-console/console-metrics-simple.webp" alt="Silo Console">
</p>
## Quick Start
```bash
docker run -d --name silo -p 9000:9000 -p 9001:9001 \
-e MINIO_ROOT_USER=minioadmin \
-e MINIO_ROOT_PASSWORD=change-me-long-password \
-v "$PWD/data:/data" \
docker.io/pgsty/silo:latest server /data --console-address ":9001"
```
<p align="center">
<img src="https://silo.pgsty.com/images/silo-console/console-login.webp" alt="Silo Console">
</p>
Console on <http://localhost:9001>, S3 API on <http://localhost:9000>. The image bundles the client as `mcli`:
```bash
docker exec silo mcli alias set local http://127.0.0.1:9000 minioadmin change-me-long-password
docker exec silo mcli mb local/demo && docker exec silo mcli ls local
```
> [!WARNING]
> For production, pin a release, use unique credentials and TLS, monitor the service, keep independent backups, and test recovery. Start from the [documentation](https://silo.pgsty.com/docs/).
## Install
| Method | Where |
| :-- | :-- | | :-- | :-- |
| Container | [`pgsty/silo`](https://hub.docker.com/r/pgsty/silo), multi-arch for `linux/amd64` and `linux/arm64` | | Project overview and navigation | [Silo Website](https://silo.pgsty.com/) ([中文](https://silo.pgsty.com/zh/)) |
| Binaries | [GitHub Releases](https://github.com/pgsty/silo/releases) — Linux, macOS, Windows on `amd64` and `arm64` | | Installation methods and downloads | [Download & Install](https://silo.pgsty.com/download/) ([中文](https://silo.pgsty.com/zh/download/)) |
| Packages | RPM, DEB, and APK, also via the [Pigsty repository](https://pigsty.io/docs/repo/) | | Operations, administration, development, and reference | [Documentation](https://silo.pgsty.com/docs/) ([中文](https://silo.pgsty.com/zh/docs/)) |
| Kubernetes | Helm chart, see [Download & Install](https://silo.pgsty.com/download/) | | Project news, release notes, and security notes | [Blog](https://silo.pgsty.com/blog/), including [releases](https://silo.pgsty.com/blog/release/) and [security](https://silo.pgsty.com/blog/security/) |
| Source | `go build -o silo . && ./silo --version` | | Versioned binaries, checksums, and source archives | [GitHub Releases](https://github.com/pgsty/minio/releases) |
| Bug reports and feature discussions | [GitHub Issues](https://github.com/pgsty/minio/issues) |
| License, attribution, and trademark information | [License](https://silo.pgsty.com/about/license/), [Attribution](https://silo.pgsty.com/about/attribution/), and [Trademark](https://silo.pgsty.com/about/trademark/) |
Every release ships checksums, SPDX SBOMs, Sigstore-signed manifests, and GitHub build attestations. Installation methods and verification commands are documented at [Download & Install](https://silo.pgsty.com/download/); migrating from upstream MinIO — taking over an existing `minio.service` and its `/etc/default/minio`, and keeping data ownership stable with a `/etc/systemd/system/silo.service.d/10-legacy-user.conf` drop-in — is covered by the [migration guide](https://silo.pgsty.com/compatibility/migration/) and the [binary & service notes](https://silo.pgsty.com/compatibility/binary/). ## Maintenance Policy
The active release line covers:
- build and dependency maintenance;
- applicable security fixes and advisories;
- focused fixes for reproducible defects;
- versioned binaries, packages, checksums, and multi-architecture images;
- the web console, client, documentation, and Pigsty integration.
Changes are kept narrow and tested where practical. Maintenance is best effort; no response, remediation, or release schedule is guaranteed.
### Out of scope
- a separate product roadmap, new storage engine, or speculative S3 features;
- broad rewrites or changes that materially expand the downstream delta;
- historical releases or multiple support branches;
- commercial support, SLAs, 24×7 coverage, or SUBNET access;
- deployment design, access control, monitoring, backup, or recovery.
## Compatibility ## Compatibility
The S3 API, `MINIO_*` variables, `minio_*` metrics, `x-minio-*` headers, `/minio/*` routes, the `github.com/minio/*` import paths, and the on-disk format (including `.minio.sys`) are preserved and held in place by a CI compatibility check. Only Silo-owned delivery surfaces change: the `silo` executable, package, service, Helm chart, and container image — no `minio` binary alias is installed. Silo aims to preserve:
Every divergence from upstream is listed in the code-verified [compatibility audit](https://silo.pgsty.com/compatibility/server/). Treat each release as a downstream upgrade: pin versions, read the [release notes](https://silo.pgsty.com/tags/silo/), and keep a rollback path. - MinIO-compatible S3 APIs, configuration, environment variables, and CLI conventions;
- `RELEASE.YYYY-MM-DDTHH-MM-SSZ` tags, container entrypoints, and common deployment workflows.
## Security & Contributing Compatibility is the default constraint. Silo preserves existing wire, client, configuration, and operational behavior whenever doing so remains safe. Compatibility is broken only when necessary to close a major security issue, and the release notes must identify the affected behavior and migration path. Treat each release as a downstream upgrade: pin versions, review [release notes](https://silo.pgsty.com/blog/release/) and [security advisories](docs/security/advisories.md), keep a rollback path, and test before production use.
Report vulnerabilities privately as described in [`SECURITY.md`](SECURITY.md); every fix ships with a public [advisory](https://silo.pgsty.com/blog/security/). Contributions are accepted inbound=outbound under AGPL-3.0-or-later with no CLA — only DCO sign-off (`git commit -s`) is required; see [`CONTRIBUTING.md`](CONTRIBUTING.md). ## Downloads and Release Artifacts
## Contributors Use [Download & Install](https://silo.pgsty.com/download/) to choose an installation method. GitHub Releases remains the source for versioned server binaries, checksums, and source archives.
<table> | Artifact | Location |
<tr> | :-- | :-- |
<td align="center" width="150"> | Source | [`github.com/pgsty/minio`](https://github.com/pgsty/minio) |
<a href="https://github.com/ZouhairCharef"><img src="https://github.com/ZouhairCharef.png?size=100" width="72" alt="ZouhairCharef"><br><sub><b>@ZouhairCharef</b></sub></a><br><sub>CVE-2026-34986</sub> | Container image | [`pgsty/minio`](https://hub.docker.com/r/pgsty/minio), multi-arch for `linux/amd64` and `linux/arm64` |
</td> | Server binaries and checksums | [GitHub Releases](https://github.com/pgsty/minio/releases) for Linux, macOS, and Windows on `amd64` and `arm64` |
<td align="center" width="150"> | Linux packages | RPM, DEB, and APK artifacts, also distributed through the [Pigsty repository](https://pigsty.io/docs/repo/) |
<a href="https://github.com/mfredenhagen"><img src="https://github.com/mfredenhagen.png?size=100" width="72" alt="mfredenhagen"><br><sub><b>@mfredenhagen</b></sub></a><br><sub>CVE-2026-39883</sub> | Client | [`pgsty/mc`](https://github.com/pgsty/mc), bundled in the container as `mcli` with an `mc` compatibility alias |
</td> | Console | Maintained [`georgmangold/console`](https://github.com/georgmangold/console) fork, embedded in the server build |
<td align="center" width="150"> | Shared library | [`pgsty/silo-pkg`](https://github.com/pgsty/silo-pkg) v3.7.0, consumed through a `replace` directive while preserving `github.com/minio/pkg/v3` import paths ([release notes](https://silo.pgsty.com/blog/release/pkg-3.7.0/)) |
<a href="https://github.com/pinginfo"><img src="https://github.com/pinginfo.png?size=100" width="72" alt="pinginfo"><br><sub><b>@pinginfo</b></sub></a><br><sub>Notification streaming</sub>
</td>
<td align="center" width="150">
<a href="https://github.com/waterkip"><img src="https://github.com/waterkip.png?size=100" width="72" alt="waterkip"><br><sub><b>@waterkip</b></sub></a><br><sub>Documentation links</sub>
</td>
</tr>
</table>
<p>
<a href="https://github.com/magicxor"><img src="https://github.com/magicxor.png?size=64" width="44" alt="magicxor" title="@magicxor"></a>
<a href="https://github.com/ycjlin"><img src="https://github.com/ycjlin.png?size=64" width="44" alt="ycjlin" title="@ycjlin"></a>
<a href="https://github.com/davinkevin"><img src="https://github.com/davinkevin.png?size=64" width="44" alt="davinkevin" title="@davinkevin"></a>
<a href="https://github.com/lem21h"><img src="https://github.com/lem21h.png?size=64" width="44" alt="lem21h" title="@lem21h"></a>
<a href="https://github.com/sulin37392"><img src="https://github.com/sulin37392.png?size=64" width="44" alt="sulin37392" title="@sulin37392"></a>
<a href="https://github.com/mosesdd"><img src="https://github.com/mosesdd.png?size=64" width="44" alt="mosesdd" title="@mosesdd"></a>
<a href="https://github.com/Xavier-777"><img src="https://github.com/Xavier-777.png?size=64" width="44" alt="Xavier-777" title="@Xavier-777"></a>
<a href="https://github.com/jiadzh"><img src="https://github.com/jiadzh.png?size=64" width="44" alt="jiadzh" title="@jiadzh"></a>
<a href="https://github.com/TLINDEN"><img src="https://github.com/TLINDEN.png?size=64" width="44" alt="TLINDEN" title="@TLINDEN"></a>
<a href="https://github.com/AntonOfTheWoods"><img src="https://github.com/AntonOfTheWoods.png?size=64" width="44" alt="AntonOfTheWoods" title="@AntonOfTheWoods"></a>
<a href="https://github.com/zylpsrs"><img src="https://github.com/zylpsrs.png?size=64" width="44" alt="zylpsrs" title="@zylpsrs"></a>
<a href="https://github.com/nsanitate"><img src="https://github.com/nsanitate.png?size=64" width="44" alt="nsanitate" title="@nsanitate"></a>
<a href="https://github.com/makinikm"><img src="https://github.com/makinikm.png?size=64" width="44" alt="makinikm" title="@makinikm"></a>
<a href="https://github.com/spaceg00se-r"><img src="https://github.com/spaceg00se-r.png?size=64" width="44" alt="spaceg00se-r" title="@spaceg00se-r"></a>
<a href="https://github.com/heroes1412"><img src="https://github.com/heroes1412.png?size=64" width="44" alt="heroes1412" title="@heroes1412"></a>
<a href="https://github.com/vampywiz17"><img src="https://github.com/vampywiz17.png?size=64" width="44" alt="vampywiz17" title="@vampywiz17"></a>
<a href="https://github.com/chalukyaj"><img src="https://github.com/chalukyaj.png?size=64" width="44" alt="chalukyaj" title="@chalukyaj"></a>
<a href="https://github.com/cbornet"><img src="https://github.com/cbornet.png?size=64" width="44" alt="cbornet" title="@cbornet"></a>
<a href="https://github.com/jvasile"><img src="https://github.com/jvasile.png?size=64" width="44" alt="jvasile" title="@jvasile"></a>
<a href="https://github.com/Kesavaambati"><img src="https://github.com/Kesavaambati.png?size=64" width="44" alt="Kesavaambati" title="@Kesavaambati"></a>
<a href="https://github.com/redfoxfox"><img src="https://github.com/redfoxfox.png?size=64" width="44" alt="redfoxfox" title="@redfoxfox"></a>
<a href="https://github.com/kuldeep-link11"><img src="https://github.com/kuldeep-link11.png?size=64" width="44" alt="kuldeep-link11" title="@kuldeep-link11"></a>
<a href="https://github.com/meesudzu"><img src="https://github.com/meesudzu.png?size=64" width="44" alt="meesudzu" title="@meesudzu"></a>
<a href="https://github.com/pmezhuev"><img src="https://github.com/pmezhuev.png?size=64" width="44" alt="pmezhuev" title="@pmezhuev"></a>
<a href="https://github.com/kh0mka"><img src="https://github.com/kh0mka.png?size=64" width="44" alt="kh0mka" title="@kh0mka"></a>
</p>
GitHub does not generate a contributor graph for forks, so [`CONTRIBUTORS.md`](CONTRIBUTORS.md) — not the Insights page — is this project's attribution record. It names everyone alongside the change or report they contributed. ## Quick Start
For local evaluation:
```bash
mkdir -p data
export MINIO_ROOT_USER=minioadmin
export MINIO_ROOT_PASSWORD=change-me-long-password
docker run -d --name silo \
-p 9000:9000 \
-p 9001:9001 \
-e MINIO_ROOT_USER \
-e MINIO_ROOT_PASSWORD \
-v "$PWD/data:/data" \
pgsty/minio:latest server /data --console-address ":9001"
```
Open the console at <http://localhost:9001>; the S3 API listens on <http://localhost:9000>.
The image includes the compatible client as `mcli`:
```bash
docker exec silo mcli alias set local http://127.0.0.1:9000 \
"$MINIO_ROOT_USER" "$MINIO_ROOT_PASSWORD"
docker exec silo mcli mb local/demo
docker exec silo mcli ls local
```
> [!WARNING]
> For production, pin a release, use unique credentials and TLS, monitor the service, keep independent backups, and test recovery.
Build the server from source:
```bash
go build -o minio .
./minio --version
```
For other installation paths—including native packages, binaries, Podman, Kubernetes, source, and Pigsty Ansible—use [Download & Install](https://silo.pgsty.com/download/). For production deployment and administration, start with the [Silo documentation](https://silo.pgsty.com/docs/). Pigsty users can also use the [Pigsty MinIO module](https://pigsty.io/docs/minio/).
## Security
Security fixes target the active `master` branch and are recorded in the [advisory log](docs/security/advisories.md) and the portal's [security notes](https://silo.pgsty.com/blog/security/). Report vulnerabilities privately as described in [`SECURITY.md`](SECURITY.md) and [`VULNERABILITY_REPORT.md`](VULNERABILITY_REPORT.md). Report issues that also affect upstream MinIO there as well.
## Contributing
Useful contributions include security and dependency updates, reproducible bug fixes, tests, release automation, packaging, and documentation.
Issues and pull requests should include the affected version, reproduction steps, impact, expected behavior, tests, and compatibility notes. Discuss large changes in an issue first.
## Background ## Background
Upstream wound down its community edition: the web console was cut back to a stub, prebuilt community binaries stopped, and the community repository was archived. Silo exists to keep those deployments running. The fork is a means, not an identity — if upstream restores its community edition, we will narrow our scope and offer the fixes back. This project was created in response to changes in the upstream community distribution and maintenance model. The maintainer’s analysis, alternatives considered, and early maintenance record are documented below:
The [**Manifesto**](https://silo.pgsty.com/about/manifesto/) is the project's public commitment in eleven articles, under one discipline: every article is either something already done with public evidence, or something explicitly refused. In short: | Essay | Subject |
| :-- | :-- |
| [MinIO Is Dead](https://silo.pgsty.com/blog/post/minio-is-dead/) | Changes to the upstream project and distribution model |
| [MinIO Is Dead, Long Live MinIO](https://silo.pgsty.com/blog/post/minio-resurrect/) | Establishing the fork and its release pipeline |
| [Two months into maintaining a MinIO fork](https://silo.pgsty.com/blog/post/minio-promise-kept/) | Initial security and maintenance work |
- **Compatibility contract** — the protocol and your data do not change, and every release documents its tested rollback target and path. ## License and Trademark
- **The license cannot change** — AGPLv3, no CLA, no copyright aggregation; nobody here, ourselves included, holds enough copyright to relicense on everyone else's behalf.
- **The never list**, append-only — no paywalling existing features, no registration wall on downloads, no telemetry (upstream's phone-home paths are removed outright), no CLA, no license change, no trademark enforcement against normal use.
- **Security and release discipline** — a public advisory for every fix, and a release every one to two months, at most a quarter apart. Judge both against the public record.
Essays: [MinIO Is Dead](https://silo.pgsty.com/blog/post/minio-is-dead/) · [Who Takes Over?](https://silo.pgsty.com/blog/post/minio-alternative/) · [Long Live MinIO](https://silo.pgsty.com/blog/post/minio-resurrect/) · [Promise Kept](https://silo.pgsty.com/blog/post/minio-promise-kept/) The server remains licensed under the [GNU Affero General Public License v3.0](LICENSE). See [`CREDITS`](CREDITS) for upstream authorship and attribution. MinIO is a trademark of MinIO, Inc. Silo and `pgsty/minio` are independent community efforts and are not affiliated with or endorsed by MinIO, Inc.
## License & Trademark
Silo is [AGPL-3.0-or-later](LICENSE), derived from [`minio/minio`](https://github.com/minio/minio) with upstream copyright and third-party notices preserved in [`NOTICE`](NOTICE) and [`CREDITS`](CREDITS). MinIO is a trademark of MinIO, Inc.; the name is used here only to identify the upstream project and compatibility lineage.
Details: [license](https://silo.pgsty.com/about/license/) · [attribution](https://silo.pgsty.com/about/attribution/) · [trademark](https://silo.pgsty.com/about/trademark/)
+119 -108
View File
@@ -1,161 +1,172 @@
> [!WARNING]
> **本分支已归档,不再接受任何修改。**
>
> `minio` 分支保留本项目以 MinIO 身份存在的最后状态。开发已迁移到 **[`main`](https://github.com/pgsty/silo/tree/main)** 分支,项目在那里以 **Silo** 的身份继续维护,二进制、软件包、systemd unit、容器镜像与 Helm chart 全部更名为 `silo`。2026-08-06,仓库由 `pgsty/minio` 更名为 **[`pgsty/silo`](https://github.com/pgsty/silo)**,默认分支由 `master` 更名为 `main`。
>
> 本分支上切出的最后一个版本是 **[`RELEASE.2026-08-04T00-00-00Z`](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-08-04T00-00-00Z)**(2026-08-04)。它的 19 个资产使用 `minio` 命名,对应容器镜像为 `docker.io/pgsty/minio:RELEASE.2026-08-04T00-00-00Z`。这些产物保持已发布状态且不做改动 —— 不移动、不重新签名、不删除任何 tag。如果你需要以原本 MinIO 形态维持的归档构件,就在本分支以及截止到该 tag 的历次发布中。此后的版本使用 `silo` 命名。
>
> **更名没有改变任何对外接口。** `MINIO_*` 环境变量、`minio_*` 指标、`x-minio-*` 头、`/minio/*` 路由、`.minio.sys` 磁盘布局、IAM 与 ARN 取值,以及 Go 模块路径 `github.com/minio/minio`,在 `main` 上全部原样保留。Silo 服务端可直接读取本版本写入的数据;新旧软件包并存安装,因此迁移与回滚始终是管理员显式触发的动作。
<h1 align="center"> <h1 align="center">
<a href="https://silo.pgsty.com/zh/"> <img src=".github/silo.svg" alt="" height="80">
<img src=".github/silo-logo.svg" alt="Silo" width="160"> <img src=".github/silo-word.svg" alt="SILO" height="80">
</a>
</h1> </h1>
<p align="center"> <p align="center">
<strong>S3 兼容对象存储 —— 由 PGSTY 维护的 MinIO 社区分支</strong> <strong>审慎维护的 MinIO 社区分支</strong><br>
为现有部署提供安全维护、带版本的发行产物与持续运维支持。
</p> </p>
<p align="center"> <p align="center">
<a href="https://silo.pgsty.com/zh/">官网</a> · <a href="https://silo.pgsty.com/zh/">官网</a> ·
<a href="https://silo.pgsty.com/zh/docs/">文档</a> · <a href="https://silo.pgsty.com/zh/docs/">文档</a> ·
<a href="https://silo.pgsty.com/zh/download/">下载</a> · <a href="https://silo.pgsty.com/zh/download/">下载</a> ·
<a href="https://silo.pgsty.com/zh/tags/silo/">版本说明</a> · <a href="https://silo.pgsty.com/zh/blog/">博客</a> ·
<a href="https://silo.pgsty.com/zh/compatibility/server/">兼容性</a> · <a href="https://github.com/pgsty/minio/releases">版本发布</a> ·
<a href="https://silo.pgsty.com/zh/about/manifesto/">宣言</a> ·
<a href="SECURITY.md">安全策略</a> · <a href="SECURITY.md">安全策略</a> ·
<a href="README.md">English</a> <a href="README.md">English</a>
</p> </p>
<p align="center"> <p align="center">
<a href="https://silo.pgsty.com/zh/"><img alt="官网" src="https://img.shields.io/badge/%E5%AE%98%E7%BD%91-silo.pgsty.com%2Fzh-1d588c"></a> <a href="https://github.com/pgsty/minio/releases"><img alt="GitHub Release" src="https://img.shields.io/github/v/release/pgsty/minio?include_prereleases&label=release&logo=github"></a>
<a href="https://github.com/pgsty/silo/releases"><img alt="GitHub Release" src="https://img.shields.io/github/v/release/pgsty/silo?include_prereleases&label=release&logo=github"></a> <a href="https://hub.docker.com/r/pgsty/minio"><img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/pgsty/minio?logo=docker"></a>
<a href="https://hub.docker.com/r/pgsty/silo"><img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/pgsty/minio?logo=docker"></a> <a href="go.mod"><img alt="Go Version" src="https://img.shields.io/github/go-mod/go-version/pgsty/minio?logo=go"></a>
<a href="go.mod"><img alt="Go Version" src="https://img.shields.io/github/go-mod/go-version/pgsty/silo?logo=go"></a>
<a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/license-AGPLv3-blue"></a> <a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/license-AGPLv3-blue"></a>
</p> </p>
> [!IMPORTANT] > [!IMPORTANT]
> **PGSTY Silo**(以下简称 Silo)是由 [Pigsty](https://pigsty.cc) 独立维护、从 [`pgsty/silo`](https://github.com/pgsty/silo) 发布的开源 MinIO 社区分支。本项目与 MinIO, Inc. 不存在隶属、背书或赞助关系;文中使用 “MinIO” 仅用于说明上游项目及兼容谱系。 > Silo 是由 [Pigsty](https://pigsty.cc) 独立维护、从 [`pgsty/minio`](https://github.com/pgsty/minio) 发布的开源 MinIO 社区分支。本项目与 MinIO, Inc. 不存在隶属、背书或赞助关系;文中使用 “MinIO” 仅用于说明上游项目及兼容谱系。
> [!NOTE]
> 2026-08-06,本仓库由 `pgsty/minio` 更名为 `pgsty/silo`,默认分支由 `master` 更名为 `main`。以原 MinIO 形态维持的归档构件仍位于归档的 [`minio`](https://github.com/pgsty/silo/tree/minio) 分支,以及截止 [`RELEASE.2026-08-04T00-00-00Z`](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-08-04T00-00-00Z) 的历次发布中。
## 概述 ## 概述
上游停止社区发行后,Silo 为开源 MinIO 服务端维护一条持续可用的版本线:构建、软件包、多架构镜像、安全修复与完整 Web 控制台。Pigsty 在生产环境中用它承载 PostgreSQL 备份存储。 Silo 维护一条基于 MinIO [`RELEASE.2025-12-03T12-00-00Z`](https://github.com/minio/minio/releases/tag/RELEASE.2025-12-03T12-00-00Z) 的下游版本线,为上游停止社区发行后仍在运行 MinIO 兼容部署的用户提供持续构建与发行产物。
它只遵循一条原则:**改名的是产品与交付物,不是协议与你的数据。** 其余内容都在 [silo.pgsty.com](https://silo.pgsty.com/zh/)。 Pigsty 使用本分支提供对象存储,包括 PostgreSQL 备份存储。
**相关项目:**[`pgsty/mc`](https://github.com/pgsty/mc) 客户端(以 `mcli` 发行) · [`pgsty/silo-console`](https://github.com/pgsty/silo-console) · [`pgsty/silo-pkg`](https://github.com/pgsty/silo-pkg) · [`pgsty/pigsty`](https://github.com/pgsty/pigsty) 项目统一门户为 [silo.pgsty.com](https://silo.pgsty.com/zh/),集中提供文档、下载安装、版本与安全动态及项目背景。中文内容位于 `/zh/`,英文内容位于站点根路径。
<p align="center"> ## 按需求选择入口
<img src="https://silo.pgsty.com/images/silo-console/console-metrics-simple.webp" alt="Silo 控制台">
</p>
## 快速上手 | 需求 | 权威入口 |
| :-- | :-- |
| 项目概览与全站导航 | [Silo 中文门户](https://silo.pgsty.com/zh/)([English](https://silo.pgsty.com/)) |
| 安装方式与软件下载 | [下载与安装](https://silo.pgsty.com/zh/download/)([English](https://silo.pgsty.com/download/)) |
| 运维、管理、开发与参考指南 | [中文文档](https://silo.pgsty.com/zh/docs/)([English](https://silo.pgsty.com/docs/)) |
| 项目动态、版本说明与安全通告 | [博客](https://silo.pgsty.com/zh/blog/),包括[版本发布](https://silo.pgsty.com/zh/blog/release/)与[安全通告](https://silo.pgsty.com/zh/blog/security/) |
| 带版本的二进制、校验和与源码归档 | [GitHub Releases](https://github.com/pgsty/minio/releases) |
| 缺陷报告与功能讨论 | [GitHub Issues](https://github.com/pgsty/minio/issues) |
| 私密漏洞报告 | [`SECURITY.md`](SECURITY.md) 与 [`VULNERABILITY_REPORT.md`](VULNERABILITY_REPORT.md) |
| 许可证、署名与商标信息 | [许可证](https://silo.pgsty.com/zh/about/license/)、[署名归属](https://silo.pgsty.com/zh/about/attribution/)与[商标政策](https://silo.pgsty.com/zh/about/trademark/) |
## 维护政策
活跃版本线的维护范围包括:
- 构建与依赖项维护;
- 适用的安全修复与公告;
- 针对可复现缺陷的范围明确的修复;
- 带版本的二进制、软件包、校验和与多架构镜像;
- Web Console、客户端、文档与 Pigsty 集成。
改动保持克制,并在可行时提供测试。所有维护均为尽力而为,不承诺固定的响应、修复或发布时间。
### 范围之外
- 独立产品路线图、新存储引擎或假设性的 S3 新特性;
- 大规模重写或显著扩大下游差异的改动;
- 历史版本或多条支持分支;
- 商业支持、SLA、7×24 服务或 SUBNET 服务;
- 部署设计、访问控制、监控、备份与恢复。
## 兼容策略
Silo 尽量保留:
- `minio` 可执行文件与 `github.com/minio/minio` module path;
- MinIO 兼容的 S3 API、配置、环境变量与命令行约定;
- `RELEASE.YYYY-MM-DDTHH-MM-SSZ` 标签、容器入口与常见部署方式。
兼容性是默认约束。只要不会留下安全问题,Silo 就保留既有的协议、客户端、配置与运维行为;只有在修复重大安全问题确有必要时才会打破兼容,并在版本说明中明确受影响行为与迁移方式。每个版本仍应视为下游升级:锁定版本,阅读[版本说明](https://silo.pgsty.com/zh/blog/release/)与[安全公告](docs/security/advisories.md),保留回滚路径,并在生产使用前完成测试。
## 下载与发行产物
请先在[下载与安装](https://silo.pgsty.com/zh/download/)页面选择合适的安装方式;GitHub Releases 仍是带版本服务端二进制、校验和与源码归档的获取位置。
| 产物 | 位置 |
| :-- | :-- |
| 源码 | [`github.com/pgsty/minio`](https://github.com/pgsty/minio) |
| 容器镜像 | [`pgsty/minio`](https://hub.docker.com/r/pgsty/minio),支持 `linux/amd64` 与 `linux/arm64` 多架构清单 |
| 服务端二进制与校验和 | [GitHub Releases](https://github.com/pgsty/minio/releases),覆盖 Linux、macOS、Windows 的 `amd64` 与 `arm64` |
| Linux 软件包 | RPM、DEB、APK,并通过 [Pigsty 软件仓库](https://pigsty.cc/docs/repo/) 分发 |
| 客户端 | [`pgsty/mc`](https://github.com/pgsty/mc),容器内以 `mcli` 提供,并保留 `mc` 兼容别名 |
| 管理控制台 | 社区维护的 [`georgmangold/console`](https://github.com/georgmangold/console),嵌入服务端构建 |
| 共享库 | [`pgsty/silo-pkg`](https://github.com/pgsty/silo-pkg) v3.7.0,通过 `replace` 指令使用,同时保留 `github.com/minio/pkg/v3` 导入路径([版本说明](https://silo.pgsty.com/zh/blog/release/pkg-3.7.0/)) |
## 快速开始
本地体验:
```bash ```bash
docker run -d --name silo -p 9000:9000 -p 9001:9001 \ mkdir -p data
-e MINIO_ROOT_USER=minioadmin \
-e MINIO_ROOT_PASSWORD=change-me-long-password \ export MINIO_ROOT_USER=minioadmin
export MINIO_ROOT_PASSWORD=change-me-long-password
docker run -d --name silo \
-p 9000:9000 \
-p 9001:9001 \
-e MINIO_ROOT_USER \
-e MINIO_ROOT_PASSWORD \
-v "$PWD/data:/data" \ -v "$PWD/data:/data" \
docker.io/pgsty/silo:latest server /data --console-address ":9001" pgsty/minio:latest server /data --console-address ":9001"
``` ```
<p align="center"> 管理控制台位于 <http://localhost:9001>,S3 API 位于 <http://localhost:9000>。
<img src="https://silo.pgsty.com/images/silo-console/console-login.webp" alt="Silo 控制台">
</p>
控制台位于 <http://localhost:9001>,S3 API 位于 <http://localhost:9000>。镜像内置客户端 `mcli`: 镜像内置兼容客户端 `mcli`:
```bash ```bash
docker exec silo mcli alias set local http://127.0.0.1:9000 minioadmin change-me-long-password docker exec silo mcli alias set local http://127.0.0.1:9000 \
docker exec silo mcli mb local/demo && docker exec silo mcli ls local "$MINIO_ROOT_USER" "$MINIO_ROOT_PASSWORD"
docker exec silo mcli mb local/demo
docker exec silo mcli ls local
``` ```
> [!WARNING] > [!WARNING]
> 生产环境应锁定版本,使用独立凭据与 TLS,配置监控,保留独立备份,并验证恢复流程。请从[文档](https://silo.pgsty.com/zh/docs/)开始。 > 生产环境应锁定版本,使用独立凭据与 TLS,配置监控,保留独立备份,并验证恢复流程。
## 安装 从源码构建服务端:
| 方式 | 位置 | ```bash
| :-- | :-- | go build -o minio .
| 容器镜像 | [`pgsty/silo`](https://hub.docker.com/r/pgsty/silo),支持 `linux/amd64` 与 `linux/arm64` | ./minio --version
| 二进制 | [GitHub Releases](https://github.com/pgsty/silo/releases),覆盖 Linux、macOS、Windows 的 `amd64` 与 `arm64` | ```
| 软件包 | RPM、DEB、APK,也可通过 [Pigsty 软件仓库](https://pigsty.cc/docs/repo/) 安装 |
| Kubernetes | Helm Chart,参见[下载与安装](https://silo.pgsty.com/zh/download/) |
| 源码构建 | `go build -o silo . && ./silo --version` |
每个版本都附带校验和、SPDX SBOM、Sigstore 签名清单与 GitHub 构建证明。完整安装方式与验证命令见[下载与安装](https://silo.pgsty.com/zh/download/);从上游 MinIO 迁移 —— 接管既有 `minio.service` 与 `/etc/default/minio`,并用 `/etc/systemd/system/silo.service.d/10-legacy-user.conf` drop-in 保持数据属主不变 —— 见[迁移指南](https://silo.pgsty.com/zh/compatibility/migration/)与[二进制与服务说明](https://silo.pgsty.com/zh/compatibility/binary/)。 其他安装方式(包括原生软件包、二进制、Podman、Kubernetes、源码构建与 Pigsty Ansible)请前往[下载与安装](https://silo.pgsty.com/zh/download/);生产部署与管理请从 [Silo 中文文档](https://silo.pgsty.com/zh/docs/)开始。Pigsty 用户也可以直接使用 [Pigsty MinIO 模块](https://pigsty.cc/docs/minio/)。
## 兼容性 ## 安全
S3 API、`MINIO_*` 环境变量、`minio_*` 指标、`x-minio-*` 头、`/minio/*` 路由、`github.com/minio/*` 导入路径与磁盘格式(含 `.minio.sys`)原样保留,并由 CI 兼容性门禁冻结。只有 Silo 自有交付面改名:`silo` 可执行文件、软件包、服务、Helm Chart 与容器镜像 —— 原生交付物不会安装 `minio` 二进制别名。 安全修复面向活跃的 `master` 分支,并记录在仓库[安全公告](docs/security/advisories.md)与门户[安全通告](https://silo.pgsty.com/zh/blog/security/)中。请按照 [`SECURITY.md`](SECURITY.md) 与 [`VULNERABILITY_REPORT.md`](VULNERABILITY_REPORT.md) 私密报告漏洞;同时影响上游 MinIO 的问题也应向上游报告。
与上游的全部分歧,以逐项核验代码的[兼容性审计](https://silo.pgsty.com/zh/compatibility/server/)形式维护。每个版本仍应视为下游升级:锁定版本,阅读[版本说明](https://silo.pgsty.com/zh/tags/silo/),并保留回滚路径。 ## 参与贡献
## 安全与贡献 欢迎安全与依赖项更新、可复现缺陷修复、测试、发布自动化、打包与文档改进。
请按照 [`SECURITY.md`](SECURITY.md) 私密报告漏洞;每项修复都会发布公开[安全公告](https://silo.pgsty.com/zh/blog/security/)。本项目不要求签署 CLA:贡献按 AGPL-3.0-or-later(inbound=outbound)接收,只需 DCO 签署(`git commit -s`),详见 [`CONTRIBUTING.md`](CONTRIBUTING.md)。 Issue 与 Pull Request 应说明受影响版本、复现步骤、影响、预期行为、测试与兼容性说明。大型改动请先提交 Issue 讨论。
## 贡献者
<table>
<tr>
<td align="center" width="150">
<a href="https://github.com/ZouhairCharef"><img src="https://github.com/ZouhairCharef.png?size=100" width="72" alt="ZouhairCharef"><br><sub><b>@ZouhairCharef</b></sub></a><br><sub>CVE-2026-34986</sub>
</td>
<td align="center" width="150">
<a href="https://github.com/mfredenhagen"><img src="https://github.com/mfredenhagen.png?size=100" width="72" alt="mfredenhagen"><br><sub><b>@mfredenhagen</b></sub></a><br><sub>CVE-2026-39883</sub>
</td>
<td align="center" width="150">
<a href="https://github.com/pinginfo"><img src="https://github.com/pinginfo.png?size=100" width="72" alt="pinginfo"><br><sub><b>@pinginfo</b></sub></a><br><sub>桶通知流式输出</sub>
</td>
<td align="center" width="150">
<a href="https://github.com/waterkip"><img src="https://github.com/waterkip.png?size=100" width="72" alt="waterkip"><br><sub><b>@waterkip</b></sub></a><br><sub>文档链接修正</sub>
</td>
</tr>
</table>
<p>
<a href="https://github.com/magicxor"><img src="https://github.com/magicxor.png?size=64" width="44" alt="magicxor" title="@magicxor"></a>
<a href="https://github.com/ycjlin"><img src="https://github.com/ycjlin.png?size=64" width="44" alt="ycjlin" title="@ycjlin"></a>
<a href="https://github.com/davinkevin"><img src="https://github.com/davinkevin.png?size=64" width="44" alt="davinkevin" title="@davinkevin"></a>
<a href="https://github.com/lem21h"><img src="https://github.com/lem21h.png?size=64" width="44" alt="lem21h" title="@lem21h"></a>
<a href="https://github.com/sulin37392"><img src="https://github.com/sulin37392.png?size=64" width="44" alt="sulin37392" title="@sulin37392"></a>
<a href="https://github.com/mosesdd"><img src="https://github.com/mosesdd.png?size=64" width="44" alt="mosesdd" title="@mosesdd"></a>
<a href="https://github.com/Xavier-777"><img src="https://github.com/Xavier-777.png?size=64" width="44" alt="Xavier-777" title="@Xavier-777"></a>
<a href="https://github.com/jiadzh"><img src="https://github.com/jiadzh.png?size=64" width="44" alt="jiadzh" title="@jiadzh"></a>
<a href="https://github.com/TLINDEN"><img src="https://github.com/TLINDEN.png?size=64" width="44" alt="TLINDEN" title="@TLINDEN"></a>
<a href="https://github.com/AntonOfTheWoods"><img src="https://github.com/AntonOfTheWoods.png?size=64" width="44" alt="AntonOfTheWoods" title="@AntonOfTheWoods"></a>
<a href="https://github.com/zylpsrs"><img src="https://github.com/zylpsrs.png?size=64" width="44" alt="zylpsrs" title="@zylpsrs"></a>
<a href="https://github.com/nsanitate"><img src="https://github.com/nsanitate.png?size=64" width="44" alt="nsanitate" title="@nsanitate"></a>
<a href="https://github.com/makinikm"><img src="https://github.com/makinikm.png?size=64" width="44" alt="makinikm" title="@makinikm"></a>
<a href="https://github.com/spaceg00se-r"><img src="https://github.com/spaceg00se-r.png?size=64" width="44" alt="spaceg00se-r" title="@spaceg00se-r"></a>
<a href="https://github.com/heroes1412"><img src="https://github.com/heroes1412.png?size=64" width="44" alt="heroes1412" title="@heroes1412"></a>
<a href="https://github.com/vampywiz17"><img src="https://github.com/vampywiz17.png?size=64" width="44" alt="vampywiz17" title="@vampywiz17"></a>
<a href="https://github.com/chalukyaj"><img src="https://github.com/chalukyaj.png?size=64" width="44" alt="chalukyaj" title="@chalukyaj"></a>
<a href="https://github.com/cbornet"><img src="https://github.com/cbornet.png?size=64" width="44" alt="cbornet" title="@cbornet"></a>
<a href="https://github.com/jvasile"><img src="https://github.com/jvasile.png?size=64" width="44" alt="jvasile" title="@jvasile"></a>
<a href="https://github.com/Kesavaambati"><img src="https://github.com/Kesavaambati.png?size=64" width="44" alt="Kesavaambati" title="@Kesavaambati"></a>
<a href="https://github.com/redfoxfox"><img src="https://github.com/redfoxfox.png?size=64" width="44" alt="redfoxfox" title="@redfoxfox"></a>
<a href="https://github.com/kuldeep-link11"><img src="https://github.com/kuldeep-link11.png?size=64" width="44" alt="kuldeep-link11" title="@kuldeep-link11"></a>
<a href="https://github.com/meesudzu"><img src="https://github.com/meesudzu.png?size=64" width="44" alt="meesudzu" title="@meesudzu"></a>
<a href="https://github.com/pmezhuev"><img src="https://github.com/pmezhuev.png?size=64" width="44" alt="pmezhuev" title="@pmezhuev"></a>
<a href="https://github.com/kh0mka"><img src="https://github.com/kh0mka.png?size=64" width="44" alt="kh0mka" title="@kh0mka"></a>
</p>
GitHub 不为 fork 仓库生成贡献者图表,因此 [`CONTRIBUTORS.md`](CONTRIBUTORS.md)(而非 Insights 页面)才是本项目的署名记录,其中逐一记录了每个人对应的改动或报告。
## 背景 ## 背景
本项目因上游收缩社区版而生:Web 控制台被削减为残桩、社区预编译制品停发、社区仓库被归档。Silo 的存在就是让这些部署继续跑下去。Fork 是手段,不是身份 —— 若上游恢复社区版承诺,我们乐意收缩范围,并把修复回馈上游。 本项目源于上游社区发行与维护模式的变化。维护者对相关变化的分析、替代方案评估与早期维护记录见以下文章:
[**宣言**](https://silo.pgsty.com/zh/about/manifesto/)是项目的公开承诺,共十一条,通篇遵循一项纪律:**每一条,要么是已经在做且有公开证据的事实,要么是刻意拒绝的承诺。** 摘要: | 文章 | 主题 |
| :-- | :-- |
- **兼容性合同** —— 协议与数据不改,每个版本都标注经过测试的回滚目标与路径。 | [MinIO已死](https://silo.pgsty.com/zh/blog/post/minio-is-dead/) | 上游项目与发行模式的变化 |
- **许可证无法变更** —— AGPLv3、无 CLA、不做版权聚合;包括我们自己在内,没有人握有足够版权代表所有贡献者重新授权。 | [MinIO已死,谁能接盘?](https://silo.pgsty.com/zh/blog/post/minio-alternative/) | 可选替代方案评估 |
- **永不清单**(只增不减)—— 永不将既有功能移入付费墙、永不给下载设注册墙、永不加入遥测(上游回连路径已整体移除)、永不引入 CLA、永不变更许可证、永不以商标追究正常使用。 | [MinIO 已死,MinIO 复生](https://silo.pgsty.com/zh/blog/post/minio-resurrect/) | 建立分支及其发行流水线 |
- **安全与发布纪律** —— 每项安全修复配一篇公开公告;通常每一到两个月发布一版,最长不超过一个季度。请拿公开记录检验这两条。 | [续命 MinIO:承诺兑现](https://silo.pgsty.com/zh/blog/post/minio-promise-kept/) | 初期安全与维护工作 |
延伸阅读:[MinIO已死](https://silo.pgsty.com/zh/blog/post/minio-is-dead/) · [谁能接盘?](https://silo.pgsty.com/zh/blog/post/minio-alternative/) · [MinIO 复生](https://silo.pgsty.com/zh/blog/post/minio-resurrect/) · [承诺兑现](https://silo.pgsty.com/zh/blog/post/minio-promise-kept/)
## 许可证与商标 ## 许可证与商标
Silo 采用 [AGPL-3.0-or-later](LICENSE),衍生自 [`minio/minio`](https://github.com/minio/minio),上游版权与第三方声明完整保留于 [`NOTICE`](NOTICE) 与 [`CREDITS`](CREDITS)。MinIO 是 MinIO, Inc. 的商标,此处使用仅为标识上游项目与兼容谱系。 服务端继续采用 [GNU Affero General Public License v3.0](LICENSE) 发布。上游作者与署名信息见 [`CREDITS`](CREDITS)。
详见:[许可证](https://silo.pgsty.com/zh/about/license/) · [署名归属](https://silo.pgsty.com/zh/about/attribution/) · [商标声明](https://silo.pgsty.com/zh/about/trademark/) MinIO 是 MinIO, Inc. 的商标。Silo、Pigsty 与 `pgsty/minio` 均为独立社区项目,与 MinIO, Inc. 不存在隶属或背书关系。
+4 -29
View File
@@ -1,43 +1,18 @@
# Security Policy # Security Policy
Silo is an independent, community-maintained object-storage server derived from This repository is the `pgsty/minio` community fork of `minio/minio`. Upstream MinIO security contacts do not handle fork-specific fixes or release notes for this repository.
the open-source MinIO server. Upstream MinIO security contacts do not handle
Silo-specific fixes or release notes.
## Supported Versions ## Supported Versions
Security fixes are tracked on the active development branch and summarized in Security fixes are tracked on the active `master` branch and summarized in [docs/security/advisories.md](docs/security/advisories.md).
[docs/security/advisories.md](docs/security/advisories.md). Only the current
Silo release line is supported unless an advisory says otherwise.
## Inherited Fix Evidence
The canonical ledger also records security fixes inherited from upstream when
they are part of the Silo release baseline. Source and fork commits are linked
separately even when the fork preserves the original commit object and SHA.
- [CVE-2025-62506](https://github.com/advisories/GHSA-jjjj-jwhf-8rgr):
upstream [PR #21642](https://github.com/minio/minio/pull/21642) merged as
[`minio/minio@c1a49490`](https://github.com/minio/minio/commit/c1a49490c78e9c3ebcad86ba0662319138ace190),
inherited unchanged as
[`pgsty/silo@c1a49490`](https://github.com/pgsty/silo/commit/c1a49490c78e9c3ebcad86ba0662319138ace190),
and is present in every Silo community release beginning with
[`RELEASE.2025-12-03T12-00-00Z`](https://github.com/pgsty/silo/releases/tag/RELEASE.2025-12-03T12-00-00Z).
The inherited [service-account](https://github.com/pgsty/silo/blob/c1a49490c78e9c3ebcad86ba0662319138ace190/cmd/admin-handlers-users_test.go#L211-L212)
and [STS](https://github.com/pgsty/silo/blob/c1a49490c78e9c3ebcad86ba0662319138ace190/cmd/sts-handlers_test.go#L45-L46)
regression groups remain part of `go test ./cmd`; see the
[canonical ledger](docs/security/advisories.md#inherited-upstream-advisory-baseline)
for the operator-facing record.
## Reporting a Vulnerability ## Reporting a Vulnerability
For vulnerabilities in this fork: For vulnerabilities in this fork:
1. Follow the fork-specific expectations in [VULNERABILITY_REPORT.md](VULNERABILITY_REPORT.md). 1. Follow the fork-specific expectations in [VULNERABILITY_REPORT.md](VULNERABILITY_REPORT.md).
2. Prefer this repository's [private GitHub security advisory](https://github.com/pgsty/silo/security/advisories/new) workflow. 2. Prefer the `pgsty/minio` repository's GitHub security reporting workflow when it is available.
3. If private reporting is unavailable, contact the maintainers through the 3. If private reporting is not available, contact the maintainers through the `pgsty/minio` repository before publishing detailed exploit information.
repository without publishing exploit details until a private channel is
established.
4. If you confirm the issue also affects upstream `minio/minio`, report it upstream separately. 4. If you confirm the issue also affects upstream `minio/minio`, report it upstream separately.
## Disclosure Process ## Disclosure Process
+3 -3
View File
@@ -1,6 +1,6 @@
# Vulnerability Management Policy # Vulnerability Management Policy
This document describes how the Silo maintainers investigate, This document describes how the `pgsty/minio` maintainers investigate,
assess, and remediate reported vulnerabilities affecting this fork, any assess, and remediate reported vulnerabilities affecting this fork, any
directly shipped component, or a direct / indirect dependency used by this directly shipped component, or a direct / indirect dependency used by this
repository. repository.
@@ -8,7 +8,7 @@ repository.
## Scope ## Scope
This policy covers vulnerability reports opened by repository maintainers or This policy covers vulnerability reports opened by repository maintainers or
external third parties against Silo itself, its release artifacts, or external third parties against `pgsty/minio` itself, its release artifacts, or
dependencies that materially affect this fork. dependencies that materially affect this fork.
It defines the information needed for triage and the expected remediation It defines the information needed for triage and the expected remediation
@@ -24,7 +24,7 @@ A useful vulnerability report should contain the following information:
a well-established vulnerability identifier, such as a CVE or GHSA ID, can a well-established vulnerability identifier, such as a CVE or GHSA ID, can
be used instead. be used instead.
Based on the report, the Silo maintainers investigate: Based on the report, the `pgsty/minio` maintainers investigate:
- Whether the reported vulnerability exists. - Whether the reported vulnerability exists.
- The conditions that are required such that the vulnerability can be exploited. - The conditions that are required such that the vulnerability can be exploited.
+1
View File
@@ -0,0 +1 @@
theme: jekyll-theme-minimal
-76
View File
@@ -1,76 +0,0 @@
#!/usr/bin/env bash
# Fail closed before a release job can replace published or finalized assets.
# An ordinary Draft is retry state; a finalized Draft contains GPG-derived
# materials and must never be replaced by the build lane.
set -euo pipefail
release_tag="${1:-}"
fixture="${2:-}"
repository="${GITHUB_REPOSITORY:-pgsty/silo}"
require_draft="${REQUIRE_DRAFT:-false}"
if ! command -v jq >/dev/null 2>&1; then
echo "jq is required to inspect GitHub release state" >&2
exit 1
fi
if [[ ! "${release_tag}" =~ ^RELEASE\.[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}-[0-9]{2}-[0-9]{2}Z$ ]]; then
echo "Invalid release tag format: ${release_tag:-<empty>}" >&2
exit 1
fi
if [ -n "${fixture}" ]; then
release_json="$(<"${fixture}")"
else
error_file="$(mktemp)"
trap 'rm -f "${error_file}"' EXIT
if ! release_json="$(
gh api --paginate "repos/${repository}/releases?per_page=100" --jq '.[]' 2>"${error_file}" |
jq --arg tag "${release_tag}" -s '[.[] | select(.tag_name == $tag)]'
)"; then
cat "${error_file}" >&2
exit 1
fi
fi
if ! jq -e 'type == "array" and all(.[]; type == "object" and (.tag_name | type == "string") and (.draft | type == "boolean"))' \
<<<"${release_json}" >/dev/null 2>&1; then
echo "Invalid release state response for ${release_tag}" >&2
exit 1
fi
if ! jq -e --arg tag "${release_tag}" 'all(.[]; .tag_name == $tag)' \
<<<"${release_json}" >/dev/null 2>&1; then
echo "Release state returned a tag other than ${release_tag}" >&2
exit 1
fi
release_count="$(jq 'length' <<<"${release_json}")"
if [ "${release_count}" -eq 0 ]; then
if [ "${require_draft}" = "true" ]; then
echo "Expected one Draft release for ${release_tag}, found none" >&2
exit 1
fi
echo "No existing release for ${release_tag}."
exit 0
fi
if [ "${release_count}" -ne 1 ]; then
echo "Refusing to choose among ${release_count} releases for ${release_tag}; clean duplicate Drafts first" >&2
exit 1
fi
if [ "$(jq -r '.[0].draft' <<<"${release_json}")" != "true" ]; then
echo "Refusing to overwrite published release ${release_tag}" >&2
exit 1
fi
finalize_markers="$(jq '[.[0].assets[]? | select(.name | endswith("_packages_provenance.sigstore.json"))] | length' <<<"${release_json}")"
if [ "${finalize_markers}" -ne 0 ]; then
echo "Refusing to replace finalized Draft ${release_tag}" >&2
exit 1
fi
echo "Existing unfinalized Draft ${release_tag} will be replaced from scratch."
-68
View File
@@ -1,68 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
checker="${script_dir}/check-release-state.sh"
tag="RELEASE.2026-08-29T00-00-00Z"
fixture="$(mktemp)"
stdout_file="$(mktemp)"
stderr_file="$(mktemp)"
trap 'rm -f "${fixture}" "${stdout_file}" "${stderr_file}"' EXIT
expect_success() {
if ! "${checker}" "$@" >"${stdout_file}" 2>"${stderr_file}"; then
cat "${stderr_file}" >&2
return 1
fi
}
expect_failure() {
if "${checker}" "$@" >"${stdout_file}" 2>"${stderr_file}"; then
echo "Expected release-state check to fail: $*" >&2
return 1
fi
}
printf '[]\n' >"${fixture}"
expect_success "${tag}" "${fixture}"
grep -qF "No existing release for ${tag}." "${stdout_file}"
if REQUIRE_DRAFT=true "${checker}" "${tag}" "${fixture}" >"${stdout_file}" 2>"${stderr_file}"; then
echo "Expected required-Draft check to fail when no release exists" >&2
exit 1
fi
grep -qF "Expected one Draft release for ${tag}, found none" "${stderr_file}"
printf '[{"tag_name":"%s","draft":true,"assets":[]}]\n' "${tag}" >"${fixture}"
expect_success "${tag}" "${fixture}"
grep -qF "Existing unfinalized Draft ${tag} will be replaced from scratch." "${stdout_file}"
if ! REQUIRE_DRAFT=true "${checker}" "${tag}" "${fixture}" >"${stdout_file}" 2>"${stderr_file}"; then
cat "${stderr_file}" >&2
exit 1
fi
printf '[{"tag_name":"%s","draft":true,"assets":[{"name":"silo_20260829000000.0.0_packages_provenance.sigstore.json"}]}]\n' "${tag}" >"${fixture}"
expect_failure "${tag}" "${fixture}"
grep -qF "Refusing to replace finalized Draft ${tag}" "${stderr_file}"
printf '[{"tag_name":"%s","draft":false}]\n' "${tag}" >"${fixture}"
expect_failure "${tag}" "${fixture}"
grep -qF "Refusing to overwrite published release ${tag}" "${stderr_file}"
printf '[{"tag_name":"%s","draft":true},{"tag_name":"%s","draft":true}]\n' "${tag}" "${tag}" >"${fixture}"
expect_failure "${tag}" "${fixture}"
grep -qF "Refusing to choose among 2 releases" "${stderr_file}"
printf '[{"tag_name":"RELEASE.2026-08-28T00-00-00Z","draft":true}]\n' >"${fixture}"
expect_failure "${tag}" "${fixture}"
grep -qF "other than ${tag}" "${stderr_file}"
printf '{not-json}\n' >"${fixture}"
expect_failure "${tag}" "${fixture}"
grep -qF "Invalid release state response for ${tag}" "${stderr_file}"
expect_failure "not-a-release-tag" "${fixture}"
grep -qF "Invalid release tag format" "${stderr_file}"
echo "release-state decision tests passed"
+1 -1
View File
@@ -7,7 +7,7 @@ _init() {
## Minimum required versions for build dependencies ## Minimum required versions for build dependencies
GIT_VERSION="1.0" GIT_VERSION="1.0"
GO_VERSION="1.27.0" GO_VERSION="1.16"
OSX_VERSION="10.8" OSX_VERSION="10.8"
KNAME=$(uname -s) KNAME=$(uname -s)
ARCH=$(uname -m) ARCH=$(uname -m)
+23 -22
View File
@@ -3,7 +3,7 @@
set -x set -x
export MINIO_CI_CD=1 export MINIO_CI_CD=1
killall -9 silo killall -9 minio
rm -rf ${HOME}/tmp/dist rm -rf ${HOME}/tmp/dist
@@ -19,27 +19,28 @@ done
echo $args echo $args
for ((i = 0; i < $((nr_servers)); i++)); do for ((i = 0; i < $((nr_servers)); i++)); do
(silo server --address ":$((9100 + i))" $args 2>&1 >/tmp/log$i.txt) & (minio server --address ":$((9100 + i))" $args 2>&1 >/tmp/log$i.txt) &
done done
sleep 10s sleep 10s
if [ ! -f ./mc ]; then if [ ! -f ./mc ]; then
"$(git rev-parse --show-toplevel)/buildscripts/install-mcli.sh" ./mc wget --quiet -O ./mc https://dl.minio.io/client/mc/release/linux-amd64/./mc &&
chmod +x mc
fi fi
set +e set +e
export MC_HOST_siloadm=http://minioadmin:minioadmin@localhost:9100/ export MC_HOST_minioadm=http://minioadmin:minioadmin@localhost:9100/
./mc ready siloadm ./mc ready minioadm
./mc ls siloadm/ ./mc ls minioadm/
./mc admin config set siloadm/ api root_access=off ./mc admin config set minioadm/ api root_access=off
sleep 3s # let things settle a little sleep 3s # let things settle a little
./mc ls siloadm/ ./mc ls minioadm/
if [ $? -eq 0 ]; then if [ $? -eq 0 ]; then
echo "listing succeeded, 'minioadmin' was not disabled" echo "listing succeeded, 'minioadmin' was not disabled"
exit 1 exit 1
@@ -47,38 +48,38 @@ fi
set -e set -e
killall -9 silo killall -9 minio
export MINIO_API_ROOT_ACCESS=on export MINIO_API_ROOT_ACCESS=on
for ((i = 0; i < $((nr_servers)); i++)); do for ((i = 0; i < $((nr_servers)); i++)); do
(silo server --address ":$((9100 + i))" $args 2>&1 >/tmp/log$i.txt) & (minio server --address ":$((9100 + i))" $args 2>&1 >/tmp/log$i.txt) &
done done
set +e set +e
./mc ready siloadm/ ./mc ready minioadm/
./mc ls siloadm/ ./mc ls minioadm/
if [ $? -ne 0 ]; then if [ $? -ne 0 ]; then
echo "listing failed, 'minioadmin' should be enabled" echo "listing failed, 'minioadmin' should be enabled"
exit 1 exit 1
fi fi
killall -9 silo killall -9 minio
rm -rf /tmp/multisitea/ rm -rf /tmp/multisitea/
rm -rf /tmp/multisiteb/ rm -rf /tmp/multisiteb/
echo "Setup site-replication and then disable root credentials" echo "Setup site-replication and then disable root credentials"
silo server --address 127.0.0.1:9001 "http://127.0.0.1:9001/tmp/multisitea/data/disterasure/xl{1...4}" \ minio server --address 127.0.0.1:9001 "http://127.0.0.1:9001/tmp/multisitea/data/disterasure/xl{1...4}" \
"http://127.0.0.1:9002/tmp/multisitea/data/disterasure/xl{5...8}" >/tmp/sitea_1.log 2>&1 & "http://127.0.0.1:9002/tmp/multisitea/data/disterasure/xl{5...8}" >/tmp/sitea_1.log 2>&1 &
silo server --address 127.0.0.1:9002 "http://127.0.0.1:9001/tmp/multisitea/data/disterasure/xl{1...4}" \ minio server --address 127.0.0.1:9002 "http://127.0.0.1:9001/tmp/multisitea/data/disterasure/xl{1...4}" \
"http://127.0.0.1:9002/tmp/multisitea/data/disterasure/xl{5...8}" >/tmp/sitea_2.log 2>&1 & "http://127.0.0.1:9002/tmp/multisitea/data/disterasure/xl{5...8}" >/tmp/sitea_2.log 2>&1 &
silo server --address 127.0.0.1:9003 "http://127.0.0.1:9003/tmp/multisiteb/data/disterasure/xl{1...4}" \ minio server --address 127.0.0.1:9003 "http://127.0.0.1:9003/tmp/multisiteb/data/disterasure/xl{1...4}" \
"http://127.0.0.1:9004/tmp/multisiteb/data/disterasure/xl{5...8}" >/tmp/siteb_1.log 2>&1 & "http://127.0.0.1:9004/tmp/multisiteb/data/disterasure/xl{5...8}" >/tmp/siteb_1.log 2>&1 &
silo server --address 127.0.0.1:9004 "http://127.0.0.1:9003/tmp/multisiteb/data/disterasure/xl{1...4}" \ minio server --address 127.0.0.1:9004 "http://127.0.0.1:9003/tmp/multisiteb/data/disterasure/xl{1...4}" \
"http://127.0.0.1:9004/tmp/multisiteb/data/disterasure/xl{5...8}" >/tmp/siteb_2.log 2>&1 & "http://127.0.0.1:9004/tmp/multisiteb/data/disterasure/xl{5...8}" >/tmp/siteb_2.log 2>&1 &
export MC_HOST_sitea=http://minioadmin:minioadmin@127.0.0.1:9001 export MC_HOST_sitea=http://minioadmin:minioadmin@127.0.0.1:9001
@@ -95,19 +96,19 @@ export MC_HOST_siteb=http://minioadmin:minioadmin@127.0.0.1:9004
./mc admin user info siteb foobar ./mc admin user info siteb foobar
killall -9 silo killall -9 minio
echo "turning off root access, however site replication must continue" echo "turning off root access, however site replication must continue"
export MINIO_API_ROOT_ACCESS=off export MINIO_API_ROOT_ACCESS=off
silo server --address 127.0.0.1:9001 "http://127.0.0.1:9001/tmp/multisitea/data/disterasure/xl{1...4}" \ minio server --address 127.0.0.1:9001 "http://127.0.0.1:9001/tmp/multisitea/data/disterasure/xl{1...4}" \
"http://127.0.0.1:9002/tmp/multisitea/data/disterasure/xl{5...8}" >/tmp/sitea_1.log 2>&1 & "http://127.0.0.1:9002/tmp/multisitea/data/disterasure/xl{5...8}" >/tmp/sitea_1.log 2>&1 &
silo server --address 127.0.0.1:9002 "http://127.0.0.1:9001/tmp/multisitea/data/disterasure/xl{1...4}" \ minio server --address 127.0.0.1:9002 "http://127.0.0.1:9001/tmp/multisitea/data/disterasure/xl{1...4}" \
"http://127.0.0.1:9002/tmp/multisitea/data/disterasure/xl{5...8}" >/tmp/sitea_2.log 2>&1 & "http://127.0.0.1:9002/tmp/multisitea/data/disterasure/xl{5...8}" >/tmp/sitea_2.log 2>&1 &
silo server --address 127.0.0.1:9003 "http://127.0.0.1:9003/tmp/multisiteb/data/disterasure/xl{1...4}" \ minio server --address 127.0.0.1:9003 "http://127.0.0.1:9003/tmp/multisiteb/data/disterasure/xl{1...4}" \
"http://127.0.0.1:9004/tmp/multisiteb/data/disterasure/xl{5...8}" >/tmp/siteb_1.log 2>&1 & "http://127.0.0.1:9004/tmp/multisiteb/data/disterasure/xl{5...8}" >/tmp/siteb_1.log 2>&1 &
silo server --address 127.0.0.1:9004 "http://127.0.0.1:9003/tmp/multisiteb/data/disterasure/xl{1...4}" \ minio server --address 127.0.0.1:9004 "http://127.0.0.1:9003/tmp/multisiteb/data/disterasure/xl{1...4}" \
"http://127.0.0.1:9004/tmp/multisiteb/data/disterasure/xl{5...8}" >/tmp/siteb_2.log 2>&1 & "http://127.0.0.1:9004/tmp/multisiteb/data/disterasure/xl{5...8}" >/tmp/siteb_2.log 2>&1 &
export MC_HOST_sitea=http://foobar:foo12345@127.0.0.1:9001 export MC_HOST_sitea=http://foobar:foo12345@127.0.0.1:9001
-125
View File
@@ -1,125 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
# Regenerates CREDITS from the license text of every Go module linked into the
# silo binary. The module set is what `go list -deps` reports for the main
# package, so test-only and tool dependencies stay out: CREDITS documents what
# a shipped binary actually contains. check-gen runs this and fails on a diff,
# which keeps CREDITS from drifting when go.mod changes.
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
repo_dir="$(cd "${script_dir}/.." && pwd)"
cd "${repo_dir}"
out_file="${1:-${repo_dir}/CREDITS}"
tmp_file="${out_file}.tmp"
trap 'rm -f "${tmp_file}"' EXIT
rule_dash='----------------------------------------------------------------'
rule_equal='================================================================'
# These modules repackage Go standard library code and publish no license
# file; their source files carry the Go Authors' BSD-style header pointing at
# the Go project license, so that text is reproduced for them.
stdlib_derived='github.com/minio/colorjson github.com/minio/csvparser github.com/minio/filepath'
is_stdlib_derived() {
case " ${stdlib_derived} " in
*" $1 "*) return 0 ;;
*) return 1 ;;
esac
}
# Command substitution strips trailing newlines; printf adds exactly one back,
# so every entry ends the same way regardless of how the license file ends.
emit_text() {
printf '%s\n' "$(cat "$1")"
}
# The module cache must hold every dependency before .Dir can resolve.
go mod download
# The Go project license text is taken from the pinned golang.org/x/sys module
# rather than GOROOT: Homebrew's Go does not ship GOROOT/LICENSE, and the
# module copy is version-locked by go.mod, so the output cannot vary with the
# machine's toolchain packaging.
go_license="$(go list -m -f '{{.Dir}}' golang.org/x/sys)/LICENSE"
if [ ! -f "${go_license}" ]; then
echo "Missing Go license text: ${go_license}" >&2
exit 1
fi
{
printf '%s\n' \
'Silo bundles third-party software under the licenses reproduced below.' \
'Generated by buildscripts/gen-credits.sh (make credits) from the Go' \
'modules linked into the silo binary. Do not edit by hand.' \
'' \
"${rule_equal}" \
''
printf '%s\n%s\n%s\n' 'Go (the standard library)' 'https://golang.org/' "${rule_dash}"
emit_text "${go_license}"
printf '\n%s\n\n' "${rule_equal}"
# The dependency closure is GOOS/GOARCH-specific: platform-only modules
# (darwin's go-m1cpu, windows' wmi, ...) enter and leave it with the host.
# Pin the primary shipped target and the release build tags so regenerating
# CREDITS produces identical output on every machine, including CI.
GOOS=linux GOARCH=amd64 go list -deps -tags kqueue \
-f '{{if and (not .Standard) .Module}}{{.Module.Path}}{{end}}' . \
| LC_ALL=C sort -u \
| grep -vx 'github.com/minio/minio' \
| xargs go list -m -f '{{.Path}}|{{with .Replace}}{{.Path}}{{end}}|{{.Dir}}' \
| while IFS='|' read -r path replacement dir; do
if [ -z "${dir}" ] || [ ! -d "${dir}" ]; then
echo "Module cache directory missing for ${path}" >&2
exit 1
fi
name="${path}"
url="https://${path}"
# A same-path replace only pins a version; the annotation is for
# dependencies actually served from a different repository.
if [ -n "${replacement}" ] && [ "${replacement}" != "${path}" ]; then
name="${path} (replaced by ${replacement})"
url="https://${replacement}"
fi
printf '%s\n%s\n%s\n' "${name}" "${url}" "${rule_dash}"
if is_stdlib_derived "${path}"; then
printf '%s\n%s\n\n' \
'This module repackages Go standard library code and publishes no' \
'license file; the Go project license below applies per its file headers.'
emit_text "${go_license}"
else
license_file=''
for candidate in LICENSE LICENSE.txt LICENSE.md COPYING COPYING.txt LICENCE UNLICENSE; do
if [ -f "${dir}/${candidate}" ]; then
license_file="${dir}/${candidate}"
break
fi
done
if [ -z "${license_file}" ]; then
echo "No license file found for ${path} in ${dir}" >&2
exit 1
fi
emit_text "${license_file}"
fi
# Apache License 2.0 section 4(d) requires redistributing the NOTICE
# file contents alongside the licensed work.
for notice in NOTICE NOTICE.txt; do
if [ -f "${dir}/${notice}" ]; then
printf '\n%s\n\n' 'Bundled NOTICE file:'
emit_text "${dir}/${notice}"
break
fi
done
printf '\n%s\n\n' "${rule_equal}"
done
} > "${tmp_file}"
mv "${tmp_file}" "${out_file}"
+30 -19
View File
@@ -5,34 +5,45 @@ set -o pipefail
set -x set -x
WORK_DIR="$PWD/.verify-$RANDOM" WORK_DIR="$PWD/.verify-$RANDOM"
SILO_CONFIG_DIR="$WORK_DIR/.silo" MINIO_CONFIG_DIR="$WORK_DIR/.minio"
SILO=("$PWD/silo" --config-dir "$SILO_CONFIG_DIR" server) MINIO=("$PWD/minio" --config-dir "$MINIO_CONFIG_DIR" server)
if [ ! -x "$PWD/silo" ]; then if [ ! -x "$PWD/minio" ]; then
echo "Silo executable binary not found in current directory" echo "minio executable binary not found in current directory"
exit 1 exit 1
fi fi
if [ ! -x "$PWD/silo" ]; then if [ ! -x "$PWD/minio" ]; then
echo "Silo executable binary not found in current directory" echo "minio executable binary not found in current directory"
exit 1 exit 1
fi fi
function start_silo_4drive() { function start_minio_4drive() {
start_port=$1 start_port=$1
export MINIO_ROOT_USER=silo export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=silo123 export MINIO_ROOT_PASSWORD=minio123
export MC_HOST_silo="http://silo:silo123@127.0.0.1:${start_port}/" export MC_HOST_minio="http://minio:minio123@127.0.0.1:${start_port}/"
unset MINIO_KMS_AUTO_ENCRYPTION # do not auto-encrypt objects unset MINIO_KMS_AUTO_ENCRYPTION # do not auto-encrypt objects
export MINIO_CI_CD=1 export MINIO_CI_CD=1
mkdir ${WORK_DIR} mkdir ${WORK_DIR}
C_PWD=${PWD}
if [ ! -x "$PWD/mc" ]; then if [ ! -x "$PWD/mc" ]; then
"$(git rev-parse --show-toplevel)/buildscripts/install-mcli.sh" "$PWD/mc" MC_BUILD_DIR="mc-$RANDOM"
if ! git clone --quiet https://github.com/minio/mc "$MC_BUILD_DIR"; then
echo "failed to download https://github.com/minio/mc"
purge "${MC_BUILD_DIR}"
exit 1
fi
(cd "${MC_BUILD_DIR}" && go build -o "$C_PWD/mc")
# remove mc source.
purge "${MC_BUILD_DIR}"
fi fi
"${SILO[@]}" --address ":$start_port" "${WORK_DIR}/disk{1...4}" >"${WORK_DIR}/server1.log" 2>&1 & "${MINIO[@]}" --address ":$start_port" "${WORK_DIR}/disk{1...4}" >"${WORK_DIR}/server1.log" 2>&1 &
pid=$! pid=$!
disown $pid disown $pid
sleep 5 sleep 5
@@ -45,30 +56,30 @@ function start_silo_4drive() {
exit 1 exit 1
fi fi
"${PWD}/mc" mb --with-versioning silo/bucket "${PWD}/mc" mb --with-versioning minio/bucket
for i in $(seq 1 4); do for i in $(seq 1 4); do
"${PWD}/mc" cp /etc/hosts silo/bucket/testobj "${PWD}/mc" cp /etc/hosts minio/bucket/testobj
sudo chown -R root. "${WORK_DIR}/disk${i}" sudo chown -R root. "${WORK_DIR}/disk${i}"
"${PWD}/mc" cp /etc/hosts silo/bucket/testobj "${PWD}/mc" cp /etc/hosts minio/bucket/testobj
sudo chown -R ${USER}. "${WORK_DIR}/disk${i}" sudo chown -R ${USER}. "${WORK_DIR}/disk${i}"
done done
for vid in $("${PWD}/mc" ls --json --versions silo/bucket/testobj | jq -r .versionId); do for vid in $("${PWD}/mc" ls --json --versions minio/bucket/testobj | jq -r .versionId); do
"${PWD}/mc" cat --vid "${vid}" silo/bucket/testobj | md5sum "${PWD}/mc" cat --vid "${vid}" minio/bucket/testobj | md5sum
done done
pkill silo pkill minio
sleep 3 sleep 3
} }
function main() { function main() {
start_port=$(shuf -i 10000-65000 -n 1) start_port=$(shuf -i 10000-65000 -n 1)
start_silo_4drive ${start_port} start_minio_4drive ${start_port}
} }
function purge() { function purge() {
-331
View File
@@ -1,331 +0,0 @@
// Copyright 2026 PGSTY contributors.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// helm-migration-guard compares a rendered legacy MinIO chart with the Silo
// upgrade candidate. Product labels, images, and commands may change; resource
// identity, selectors, PVCs, storage mounts, ports, secrets, and service-account
// references must not.
package main
import (
"encoding/json"
"errors"
"fmt"
"io"
"os"
"sort"
"strings"
"go.yaml.in/yaml/v3"
)
type resource struct {
key string
doc map[string]any
}
func main() {
if len(os.Args) != 3 {
fatal(errors.New("usage: helm-migration-guard OLD_RENDER NEW_RENDER"))
}
oldResources, err := readResources(os.Args[1])
if err != nil {
fatal(err)
}
newResources, err := readResources(os.Args[2])
if err != nil {
fatal(err)
}
if err := compare(oldResources, newResources); err != nil {
fatal(err)
}
fmt.Printf("Silo Helm migration identity is stable across %d rendered resources\n", len(oldResources))
}
func readResources(path string) (map[string]resource, error) {
file, err := os.Open(path)
if err != nil {
return nil, fmt.Errorf("open %s: %w", path, err)
}
defer file.Close()
resources := make(map[string]resource)
decoder := yaml.NewDecoder(file)
for document := 1; ; document++ {
var doc map[string]any
err = decoder.Decode(&doc)
if errors.Is(err, io.EOF) {
break
}
if err != nil {
return nil, fmt.Errorf("decode %s document %d: %w", path, document, err)
}
if len(doc) == 0 || text(doc["kind"]) == "" {
continue
}
metadata := object(doc["metadata"])
key := strings.Join([]string{text(doc["kind"]), text(metadata["namespace"]), text(metadata["name"])}, "/")
if _, exists := resources[key]; exists {
return nil, fmt.Errorf("%s contains duplicate resource %s", path, key)
}
resources[key] = resource{key: key, doc: doc}
}
return resources, nil
}
func compare(oldResources, newResources map[string]resource) error {
for key := range oldResources {
if _, ok := newResources[key]; !ok {
return fmt.Errorf("legacy resource would be removed or renamed: %s", key)
}
}
for key := range newResources {
if _, ok := oldResources[key]; !ok {
return fmt.Errorf("upgrade candidate unexpectedly adds a resource: %s", key)
}
}
keys := make([]string, 0, len(oldResources))
for key := range oldResources {
keys = append(keys, key)
}
sort.Strings(keys)
for _, key := range keys {
oldDoc := oldResources[key].doc
newDoc := newResources[key].doc
kind := text(oldDoc["kind"])
switch kind {
case "Service":
if err := same(key, "Service selector", at(oldDoc, "spec", "selector"), at(newDoc, "spec", "selector")); err != nil {
return err
}
if err := same(key, "Service ports", at(oldDoc, "spec", "ports"), at(newDoc, "spec", "ports")); err != nil {
return err
}
case "Deployment", "StatefulSet":
if err := same(key, "workload selector", at(oldDoc, "spec", "selector"), at(newDoc, "spec", "selector")); err != nil {
return err
}
}
if kind == "StatefulSet" {
if err := same(key, "StatefulSet serviceName", at(oldDoc, "spec", "serviceName"), at(newDoc, "spec", "serviceName")); err != nil {
return err
}
if err := same(key, "volume claim templates", claimTemplates(oldDoc), claimTemplates(newDoc)); err != nil {
return err
}
}
if kind == "PersistentVolumeClaim" {
if err := same(key, "PVC specification", at(oldDoc, "spec"), at(newDoc, "spec")); err != nil {
return err
}
}
if kind == "Secret" {
if err := same(key, "Secret keys", secretKeys(oldDoc), secretKeys(newDoc)); err != nil {
return err
}
}
if kind == "Deployment" || kind == "StatefulSet" || kind == "Job" {
if err := comparePod(key, kind, oldDoc, newDoc); err != nil {
return err
}
}
}
return nil
}
func comparePod(key, kind string, oldDoc, newDoc map[string]any) error {
oldPod := object(at(oldDoc, "spec", "template", "spec"))
newPod := object(at(newDoc, "spec", "template", "spec"))
if err := same(key, "service account", oldPod["serviceAccountName"], newPod["serviceAccountName"]); err != nil {
return err
}
if err := same(key, "referenced volume sources", volumeSources(oldPod), volumeSources(newPod)); err != nil {
return err
}
oldContainers := containers(oldPod)
newContainers := containers(newPod)
if err := same(key, "container identities", sortedKeys(oldContainers), sortedKeys(newContainers)); err != nil {
return err
}
for _, name := range sortedKeys(oldContainers) {
oldContainer := oldContainers[name]
newContainer := newContainers[name]
if err := same(key, name+" ports", oldContainer["ports"], newContainer["ports"]); err != nil {
return err
}
if err := same(key, name+" environment", oldContainer["env"], newContainer["env"]); err != nil {
return err
}
if err := same(key, name+" envFrom", oldContainer["envFrom"], newContainer["envFrom"]); err != nil {
return err
}
if err := same(key, name+" storage mounts", normalizedMounts(oldContainer, oldPod), normalizedMounts(newContainer, newPod)); err != nil {
return err
}
image := text(newContainer["image"])
if strings.HasPrefix(image, "pgsty/minio:") || strings.HasPrefix(image, "docker.io/pgsty/minio:") {
return fmt.Errorf("%s container %s still uses frozen image %s", key, name, image)
}
command := commandText(newContainer)
if strings.Contains(command, "/usr/bin/minio") {
return fmt.Errorf("%s container %s still invokes /usr/bin/minio", key, name)
}
if (kind == "Deployment" || kind == "StatefulSet") && strings.Contains(image, "pgsty/silo:") {
if !strings.Contains(command, "silo") || !strings.Contains(command, "server") {
return fmt.Errorf("%s container %s does not invoke the Silo server: %q", key, name, command)
}
}
}
return nil
}
func claimTemplates(doc map[string]any) []string {
var result []string
for _, raw := range list(at(doc, "spec", "volumeClaimTemplates")) {
claim := object(raw)
metadata := object(claim["metadata"])
result = append(result, text(metadata["name"])+"="+canonical(claim["spec"]))
}
sort.Strings(result)
return result
}
func secretKeys(doc map[string]any) []string {
var result []string
for _, section := range []string{"data", "stringData"} {
for key := range object(doc[section]) {
result = append(result, section+":"+key)
}
}
sort.Strings(result)
return result
}
func volumeSources(pod map[string]any) []string {
var result []string
for _, raw := range list(pod["volumes"]) {
volume := cloneObject(object(raw))
delete(volume, "name")
result = append(result, canonical(volume))
}
sort.Strings(result)
return result
}
func volumeSourceByName(pod map[string]any) map[string]string {
result := make(map[string]string)
for _, raw := range list(pod["volumes"]) {
volume := cloneObject(object(raw))
name := text(volume["name"])
delete(volume, "name")
result[name] = canonical(volume)
}
return result
}
func normalizedMounts(container, pod map[string]any) []string {
sources := volumeSourceByName(pod)
var result []string
for _, raw := range list(container["volumeMounts"]) {
mount := cloneObject(object(raw))
name := text(mount["name"])
delete(mount, "name")
mount["source"] = sources[name]
result = append(result, canonical(mount))
}
sort.Strings(result)
return result
}
func containers(pod map[string]any) map[string]map[string]any {
result := make(map[string]map[string]any)
for _, section := range []string{"initContainers", "containers"} {
for _, raw := range list(pod[section]) {
container := object(raw)
result[section+":"+text(container["name"])] = container
}
}
return result
}
func commandText(container map[string]any) string {
var parts []string
for _, field := range []string{"command", "args"} {
for _, value := range list(container[field]) {
parts = append(parts, text(value))
}
}
return strings.Join(parts, " ")
}
func at(root map[string]any, path ...string) any {
var current any = root
for _, part := range path {
current = object(current)[part]
}
return current
}
func object(value any) map[string]any {
if value == nil {
return map[string]any{}
}
result, _ := value.(map[string]any)
return result
}
func cloneObject(value map[string]any) map[string]any {
result := make(map[string]any, len(value))
for key, item := range value {
result[key] = item
}
return result
}
func list(value any) []any {
result, _ := value.([]any)
return result
}
func text(value any) string {
result, _ := value.(string)
return result
}
func sortedKeys[T any](values map[string]T) []string {
result := make([]string, 0, len(values))
for key := range values {
result = append(result, key)
}
sort.Strings(result)
return result
}
func same(resourceKey, field string, oldValue, newValue any) error {
oldCanonical := canonical(oldValue)
newCanonical := canonical(newValue)
if oldCanonical != newCanonical {
return fmt.Errorf("%s changes %s\nold: %s\nnew: %s", resourceKey, field, oldCanonical, newCanonical)
}
return nil
}
func canonical(value any) string {
data, err := json.Marshal(value)
if err != nil {
return fmt.Sprintf("<unmarshalable %T: %v>", value, err)
}
return string(data)
}
func fatal(err error) {
fmt.Fprintf(os.Stderr, "Silo Helm migration check failed: %v\n", err)
os.Exit(1)
}
-91
View File
@@ -1,91 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
if [ "$#" -ne 1 ]; then
echo "usage: $0 TARGET" >&2
exit 2
fi
target=$1
target_dir=$(dirname "${target}")
if [ ! -d "${target_dir}" ]; then
echo "target directory does not exist: ${target_dir}" >&2
exit 1
fi
sha256_file() {
if command -v sha256sum >/dev/null 2>&1; then
sha256sum "$1" | awk '{print $1}'
else
shasum -a 256 "$1" | awk '{print $1}'
fi
}
if [ -n "${MCLI_BIN:-}" ]; then
if [ ! -f "${MCLI_BIN}" ]; then
echo "MCLI_BIN is not a regular file: ${MCLI_BIN}" >&2
exit 1
fi
if ! printf '%s\n' "${MCLI_SHA256:-}" | grep -Eq '^[0-9a-fA-F]{64}$'; then
echo "MCLI_SHA256 must contain the expected SHA-256 for MCLI_BIN" >&2
exit 1
fi
actual=$(sha256_file "${MCLI_BIN}")
if [ "${actual}" != "${MCLI_SHA256,,}" ]; then
echo "MCLI_BIN checksum mismatch: expected ${MCLI_SHA256,,}, got ${actual}" >&2
exit 1
fi
install -m 0755 "${MCLI_BIN}" "${target}"
exit 0
fi
release=${MCLI_RELEASE:-RELEASE.2026-08-06T00-00-00Z}
version_hyphen=${release#RELEASE.}
package_version=$(printf '%s\n' "${version_hyphen}" | sed -E 's/^([0-9]{4})-([0-9]{2})-([0-9]{2})T([0-9]{2})-([0-9]{2})-([0-9]{2})Z$/\1\2\3\4\5\6.0.0/')
if [ "${package_version}" = "${version_hyphen}" ]; then
echo "invalid MCLI_RELEASE: ${release}" >&2
exit 1
fi
case $(uname -s) in
Linux) os=linux ;;
Darwin) os=darwin ;;
*) echo "unsupported mcli host OS: $(uname -s)" >&2; exit 1 ;;
esac
case $(uname -m) in
x86_64 | amd64) arch=amd64 ;;
aarch64 | arm64) arch=arm64 ;;
*) echo "unsupported mcli host architecture: $(uname -m)" >&2; exit 1 ;;
esac
archive="mcli_${package_version}_${os}_${arch}.tar.gz"
checksums="mcli_${package_version}_checksums.txt"
base_url="https://github.com/pgsty/mc/releases/download/${release}"
tmp_dir=$(mktemp -d "${TMPDIR:-/tmp}/silo-mcli.XXXXXX")
trap 'rm -rf "${tmp_dir}"' EXIT
curl --fail --location --retry 3 --silent --show-error \
"${base_url}/${checksums}" --output "${tmp_dir}/${checksums}"
curl --fail --location --retry 3 --silent --show-error \
"${base_url}/${archive}" --output "${tmp_dir}/${archive}"
expected=$(awk -v asset="${archive}" '
{
name=$2
sub(/^\*/, "", name)
if (name == asset && $1 ~ /^[0-9a-fA-F]{64}$/) print tolower($1)
}
' "${tmp_dir}/${checksums}")
if ! printf '%s\n' "${expected}" | grep -Eq '^[0-9a-f]{64}$'; then
echo "checksum manifest does not contain exactly one valid entry for ${archive}" >&2
exit 1
fi
actual=$(sha256_file "${tmp_dir}/${archive}")
if [ "${actual}" != "${expected}" ]; then
echo "downloaded ${archive} checksum mismatch" >&2
exit 1
fi
tar -xzf "${tmp_dir}/${archive}" -C "${tmp_dir}" mcli
install -m 0755 "${tmp_dir}/mcli" "${target}"
-48
View File
@@ -1,48 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
if [ "$#" -ne 3 ]; then
echo "usage: $0 SOURCE SHA256 TARGET" >&2
exit 2
fi
source_ref=$1
expected=${2,,}
target=$3
if ! printf '%s\n' "${expected}" | grep -Eq '^[0-9a-f]{64}$'; then
echo "expected checksum must be a lowercase SHA-256 digest" >&2
exit 1
fi
if [ ! -d "$(dirname "${target}")" ]; then
echo "target directory does not exist: $(dirname "${target}")" >&2
exit 1
fi
tmp_file=$(mktemp "${TMPDIR:-/tmp}/silo-fixture.XXXXXX")
trap 'rm -f "${tmp_file}"' EXIT
case ${source_ref} in
https://*)
curl --fail --location --retry 3 --silent --show-error \
"${source_ref}" --output "${tmp_file}"
;;
*)
if [ ! -f "${source_ref}" ]; then
echo "fixture is not a regular file: ${source_ref}" >&2
exit 1
fi
cp "${source_ref}" "${tmp_file}"
;;
esac
if command -v sha256sum >/dev/null 2>&1; then
actual=$(sha256sum "${tmp_file}" | awk '{print $1}')
else
actual=$(shasum -a 256 "${tmp_file}" | awk '{print $1}')
fi
if [ "${actual}" != "${expected}" ]; then
echo "fixture checksum mismatch: expected ${expected}, got ${actual}" >&2
exit 1
fi
install -m 0755 "${tmp_file}" "${target}"
@@ -1,7 +1,7 @@
#!/bin/bash #!/bin/bash
# This script tests IAM migration from an old MinIO compatibility fixture into # This script is used to test the migration of IAM content from old minio
# the current Silo server. # instance to new minio instance.
# #
# To run it locally, start the LDAP server in github.com/minio/minio-iam-testing # To run it locally, start the LDAP server in github.com/minio/minio-iam-testing
# repo (e.g. make podman-run), and then run this script. # repo (e.g. make podman-run), and then run this script.
@@ -15,7 +15,6 @@
OLD_VERSION=RELEASE.2024-03-26T22-10-45Z OLD_VERSION=RELEASE.2024-03-26T22-10-45Z
OLD_BINARY_LINK=https://dl.min.io/server/minio/release/linux-amd64/archive/minio.${OLD_VERSION} OLD_BINARY_LINK=https://dl.min.io/server/minio/release/linux-amd64/archive/minio.${OLD_VERSION}
OLD_BINARY_SHA256=2050199d89e3057571620a1d453118fed5bd2de9d4f3b266b11365fdf984d676
__init__() { __init__() {
if which curl &>/dev/null; then if which curl &>/dev/null; then
@@ -28,16 +27,17 @@ __init__() {
export GOPATH=/tmp/gopath export GOPATH=/tmp/gopath
export PATH="${PATH}":"${GOPATH}"/bin export PATH="${PATH}":"${GOPATH}"/bin
if [ ! -x "${GOPATH}/bin/mc" ]; then if which mc &>/dev/null; then
echo "Installing verified compatible client fixture" echo "mc is already installed"
mkdir -p "${GOPATH}/bin" else
"$(git rev-parse --show-toplevel)/buildscripts/install-mcli.sh" "${GOPATH}/bin/mc" echo "Installing mc:"
go install github.com/minio/mc@latest
fi fi
if [ ! -x ./minio.${OLD_VERSION} ]; then if [ ! -x ./minio.${OLD_VERSION} ]; then
echo "Installing verified upstream compatibility fixture minio.${OLD_VERSION}" echo "Downloading minio.${OLD_VERSION} binary"
"$(git rev-parse --show-toplevel)/buildscripts/install-verified-fixture.sh" \ curl -o minio.${OLD_VERSION} ${OLD_BINARY_LINK}
"${OLD_BINARY_LINK}" "${OLD_BINARY_SHA256}" "minio.${OLD_VERSION}" chmod +x minio.${OLD_VERSION}
fi fi
if [ -z "$_MINIO_LDAP_TEST_SERVER" ]; then if [ -z "$_MINIO_LDAP_TEST_SERVER" ]; then
@@ -49,7 +49,7 @@ __init__() {
} }
create_iam_content_in_old_minio() { create_iam_content_in_old_minio() {
echo "Creating IAM content in the old MinIO compatibility fixture." echo "Creating IAM content in old minio instance."
MINIO_CI_CD=1 ./minio.${OLD_VERSION} server /tmp/data/{1...4} & MINIO_CI_CD=1 ./minio.${OLD_VERSION} server /tmp/data/{1...4} &
sleep 5 sleep 5
@@ -80,9 +80,9 @@ create_iam_content_in_old_minio() {
} }
import_iam_content_in_new_minio() { import_iam_content_in_new_minio() {
echo "Importing IAM content into the current Silo instance." echo "Importing IAM content in new minio instance."
# Assume the current Silo binary exists. # Assume current minio binary exists.
MINIO_CI_CD=1 ./silo server /tmp/data/{1...4} & MINIO_CI_CD=1 ./minio server /tmp/data/{1...4} &
sleep 5 sleep 5
set -x set -x
+95 -106
View File
@@ -1,124 +1,113 @@
#!/usr/bin/env bash #!/bin/bash
set -euo pipefail trap 'cleanup $LINENO' ERR
# Exercise both directions of the on-disk compatibility contract using an
# immutable pre-rebrand image and a container built from the current checkout.
# Every Docker resource is uniquely named and removed explicitly; this test
# never prunes unrelated images, containers, networks, or volumes.
repo_dir="$(git rev-parse --show-toplevel)"
old_image="${OLD_IMAGE:-docker.io/pgsty/minio@sha256:b6bfe7239bfc83fb90d31612d9704d86039dd714f7904b3f1ad68f211e602372}"
new_image="${NEW_IMAGE:-silo-upgrade-test:dev}"
suffix="$(date +%s)-$$"
network="silo-upgrade-net-${suffix}"
volume="silo-upgrade-data-${suffix}"
old_container="silo-upgrade-old-${suffix}"
new_container="silo-upgrade-new-${suffix}"
rollback_container="silo-upgrade-rollback-${suffix}"
root_user=silo-upgrade-admin
root_password=silo-upgrade-secret-123
# shellcheck disable=SC2120
cleanup() { cleanup() {
status=$? MINIO_VERSION=dev /tmp/gopath/bin/docker-compose \
trap - EXIT -f "buildscripts/upgrade-tests/compose.yml" \
if [ "${status}" -ne 0 ]; then down || true
for name in "${old_container}" "${new_container}" "${rollback_container}"; do
docker logs "${name}" 2>/dev/null | tail -n 80 >&2 || true
done
fi
if [ "${KEEP_UPGRADE_TEST_RESOURCES:-0}" = 1 ]; then
printf 'Retained Docker resources for inspection: %s %s\n' "${network}" "${volume}" >&2
exit "${status}"
fi
docker rm -f "${old_container}" "${new_container}" "${rollback_container}" >/dev/null 2>&1 || true
docker network rm "${network}" >/dev/null 2>&1 || true
docker volume rm "${volume}" >/dev/null 2>&1 || true
exit "${status}"
}
trap cleanup EXIT
trap 'exit 130' INT TERM
wait_ready() { MINIO_VERSION=dev /tmp/gopath/bin/docker-compose \
name="$1" -f "buildscripts/upgrade-tests/compose.yml" \
ready="" rm || true
for _ in $(seq 1 90); do
if docker logs "${name}" 2>&1 | grep -q 'API:'; then for volume in $(docker volume ls -q | grep upgrade); do
ready=1 docker volume rm ${volume} || true
break
fi
if [ "$(docker inspect -f '{{.State.Running}}' "${name}")" != true ]; then
break
fi
sleep 1
done done
if [ -z "${ready}" ]; then
echo "Server did not become ready: ${name}" >&2 docker volume prune -f
return 1 docker system prune -f || true
docker volume prune -f || true
docker volume rm $(docker volume ls -q -f dangling=true) || true
}
verify_checksum_after_heal() {
local sum1
sum1=$(curl -s "$2" | sha256sum)
mc admin heal --json -r "$1" >/dev/null # test after healing
local sum1_heal
sum1_heal=$(curl -s "$2" | sha256sum)
if [ "${sum1_heal}" != "${sum1}" ]; then
echo "mismatch expected ${sum1_heal}, got ${sum1}"
exit 1
fi fi
} }
start_server() { verify_checksum_mc() {
name="$1" local expected
image="$2" expected=$(mc cat "$1" | sha256sum)
shift 2 local got
docker run -d --name "${name}" --network "${network}" \ got=$(mc cat "$2" | sha256sum)
--mount "source=${volume},target=/data" \
-e MINIO_CI_CD=1 -e MINIO_ROOT_USER="${root_user}" -e MINIO_ROOT_PASSWORD="${root_password}" \ if [ "${expected}" != "${got}" ]; then
"${image}" "$@" >/dev/null echo "mismatch - expected ${expected}, got ${got}"
wait_ready "${name}" exit 1
docker exec "${name}" mcli alias set local http://127.0.0.1:9000 "${root_user}" "${root_password}" >/dev/null fi
echo "matches - ${expected}, got ${got}"
} }
stop_server() { add_alias() {
name="$1" for i in $(seq 1 4); do
docker stop -t 20 "${name}" >/dev/null echo "... attempting to add alias $i"
test "$(docker inspect -f '{{.State.ExitCode}}' "${name}")" = 0 until (mc alias set minio http://127.0.0.1:9000 minioadmin minioadmin); do
docker logs "${name}" 2>&1 | grep -q 'Exiting on signal' echo "...waiting... for 5secs" && sleep 5
docker rm "${name}" >/dev/null done
done
echo "Sleeping for nginx"
sleep 20
} }
command -v docker >/dev/null __init__() {
docker info >/dev/null sudo apt install curl -y
if ! docker image inspect "${old_image}" >/dev/null 2>&1; then export GOPATH=/tmp/gopath
docker pull "${old_image}" export PATH=${PATH}:${GOPATH}/bin
fi
if [ "${SILO_UPGRADE_SKIP_BUILD:-0}" != 1 ]; then go install github.com/minio/mc@latest
make -C "${repo_dir}" docker TAG="${new_image}"
fi
docker image inspect "${new_image}" >/dev/null
docker network create "${network}" >/dev/null ## this is needed because github actions don't have
docker volume create "${volume}" >/dev/null ## docker-compose on all runners
COMPOSE_VERSION=v2.35.1
mkdir -p /tmp/gopath/bin/
wget -O /tmp/gopath/bin/docker-compose https://github.com/docker/compose/releases/download/${COMPOSE_VERSION}/docker-compose-linux-x86_64
chmod +x /tmp/gopath/bin/docker-compose
start_server "${old_container}" "${old_image}" minio server /data --address :9000 cleanup
docker exec "${old_container}" mcli mb local/compat >/dev/null
docker exec "${old_container}" mcli version enable local/compat >/dev/null
printf 'old-version-1\n' | docker exec -i "${old_container}" mcli pipe local/compat/versioned.txt >/dev/null
printf 'old-version-2\n' | docker exec -i "${old_container}" mcli pipe local/compat/versioned.txt >/dev/null
test "$(docker exec "${old_container}" mcli ls --versions local/compat/versioned.txt | grep -c 'versioned.txt')" -ge 2
docker exec "${old_container}" mcli mb --with-lock local/locked >/dev/null
printf 'locked-by-old\n' | docker exec -i "${old_container}" mcli pipe local/locked/object.txt >/dev/null
dd if=/dev/zero bs=1048576 count=70 2>/dev/null | docker exec -i "${old_container}" mcli pipe local/compat/multipart.bin >/dev/null
test "$(docker exec "${old_container}" mcli stat --json local/compat/multipart.bin | jq -r '.size')" = 73400320
docker exec "${old_container}" mcli admin user add local migration-user migration-secret-123 >/dev/null
docker exec "${old_container}" mcli admin policy attach local readwrite --user migration-user >/dev/null
stop_server "${old_container}"
start_server "${new_container}" "${new_image}" silo server /data --address :9000 TAG=minio/minio:dev make docker
test "$(docker exec "${new_container}" mcli cat local/compat/versioned.txt)" = old-version-2
test "$(docker exec "${new_container}" mcli cat local/locked/object.txt)" = locked-by-old
test "$(docker exec "${new_container}" mcli stat --json local/compat/multipart.bin | jq -r '.size')" = 73400320
docker exec "${new_container}" mcli admin user info local migration-user >/dev/null
docker exec "${new_container}" mcli alias set migrated http://127.0.0.1:9000 migration-user migration-secret-123 >/dev/null
printf 'written-by-silo\n' | docker exec -i "${new_container}" mcli pipe migrated/compat/silo.txt >/dev/null
stop_server "${new_container}"
start_server "${rollback_container}" "${old_image}" minio server /data --address :9000 MINIO_VERSION=RELEASE.2019-12-19T22-52-26Z docker-compose \
test "$(docker exec "${rollback_container}" mcli cat local/compat/versioned.txt)" = old-version-2 -f "buildscripts/upgrade-tests/compose.yml" \
test "$(docker exec "${rollback_container}" mcli cat local/compat/silo.txt)" = written-by-silo up -d --build
test "$(docker exec "${rollback_container}" mcli stat --json local/compat/multipart.bin | jq -r '.size')" = 73400320
stop_server "${rollback_container}"
echo "MinIO-to-Silo data upgrade and rollback checks passed" add_alias
mc mb minio/minio-test/
mc cp ./minio minio/minio-test/to-read/
mc cp /etc/hosts minio/minio-test/to-read/hosts
mc anonymous set download minio/minio-test
verify_checksum_mc ./minio minio/minio-test/to-read/minio
curl -s http://127.0.0.1:9000/minio-test/to-read/hosts | sha256sum
MINIO_VERSION=dev /tmp/gopath/bin/docker-compose -f "buildscripts/upgrade-tests/compose.yml" stop
}
main() {
MINIO_VERSION=dev /tmp/gopath/bin/docker-compose -f "buildscripts/upgrade-tests/compose.yml" up -d --build
add_alias
verify_checksum_after_heal minio/minio-test http://127.0.0.1:9000/minio-test/to-read/hosts
verify_checksum_mc ./minio minio/minio-test/to-read/minio
verify_checksum_mc /etc/hosts minio/minio-test/to-read/hosts
cleanup
}
(__init__ "$@" && main "$@")
+30 -19
View File
@@ -5,16 +5,16 @@ if [ -n "$TEST_DEBUG" ]; then
fi fi
WORK_DIR="$PWD/.verify-$RANDOM" WORK_DIR="$PWD/.verify-$RANDOM"
SILO_CONFIG_DIR="$WORK_DIR/.silo" MINIO_CONFIG_DIR="$WORK_DIR/.minio"
SILO=("$PWD/silo" --config-dir "$SILO_CONFIG_DIR" server) MINIO=("$PWD/minio" --config-dir "$MINIO_CONFIG_DIR" server)
if [ ! -x "$PWD/silo" ]; then if [ ! -x "$PWD/minio" ]; then
echo "Silo executable binary not found in current directory" echo "minio executable binary not found in current directory"
exit 1 exit 1
fi fi
if [ ! -x "$PWD/silo" ]; then if [ ! -x "$PWD/minio" ]; then
echo "Silo executable binary not found in current directory" echo "minio executable binary not found in current directory"
exit 1 exit 1
fi fi
@@ -30,9 +30,9 @@ catch() {
echo "error on line $1" echo "error on line $1"
fi fi
echo "Cleaning up instances of Silo" echo "Cleaning up instances of MinIO"
pkill silo || true pkill minio || true
pkill -9 silo || true pkill -9 minio || true
purge "$WORK_DIR" purge "$WORK_DIR"
if [ $# -ne 0 ]; then if [ $# -ne 0 ]; then
exit $# exit $#
@@ -41,21 +41,32 @@ catch() {
catch catch
function start_silo_10drive() { function start_minio_10drive() {
start_port=$1 start_port=$1
export MINIO_ROOT_USER=silo export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=silo123 export MINIO_ROOT_PASSWORD=minio123
export MC_HOST_silo="http://silo:silo123@127.0.0.1:${start_port}/" export MC_HOST_minio="http://minio:minio123@127.0.0.1:${start_port}/"
unset MINIO_KMS_AUTO_ENCRYPTION # do not auto-encrypt objects unset MINIO_KMS_AUTO_ENCRYPTION # do not auto-encrypt objects
export MINIO_CI_CD=1 export MINIO_CI_CD=1
mkdir ${WORK_DIR} mkdir ${WORK_DIR}
C_PWD=${PWD}
if [ ! -x "$PWD/mc" ]; then if [ ! -x "$PWD/mc" ]; then
"$(git rev-parse --show-toplevel)/buildscripts/install-mcli.sh" "$PWD/mc" MC_BUILD_DIR="mc-$RANDOM"
if ! git clone --quiet https://github.com/minio/mc "$MC_BUILD_DIR"; then
echo "failed to download https://github.com/minio/mc"
purge "${MC_BUILD_DIR}"
exit 1
fi
(cd "${MC_BUILD_DIR}" && go build -o "$C_PWD/mc")
# remove mc source.
purge "${MC_BUILD_DIR}"
fi fi
"${SILO[@]}" --address ":$start_port" "${WORK_DIR}/disk{1...10}" >"${WORK_DIR}/server1.log" 2>&1 & "${MINIO[@]}" --address ":$start_port" "${WORK_DIR}/disk{1...10}" >"${WORK_DIR}/server1.log" 2>&1 &
pid=$! pid=$!
disown $pid disown $pid
sleep 5 sleep 5
@@ -68,10 +79,10 @@ function start_silo_10drive() {
exit 1 exit 1
fi fi
"${PWD}/mc" mb --with-versioning silo/bucket "${PWD}/mc" mb --with-versioning minio/bucket
export AWS_ACCESS_KEY_ID=silo export AWS_ACCESS_KEY_ID=minio
export AWS_SECRET_ACCESS_KEY=silo123 export AWS_SECRET_ACCESS_KEY=minio123
aws --endpoint-url http://localhost:"$start_port" s3api create-multipart-upload --bucket bucket --key obj-1 >upload-id.json aws --endpoint-url http://localhost:"$start_port" s3api create-multipart-upload --bucket bucket --key obj-1 >upload-id.json
uploadId=$(jq -r '.UploadId' upload-id.json) uploadId=$(jq -r '.UploadId' upload-id.json)
@@ -109,7 +120,7 @@ EOF
function main() { function main() {
start_port=$(shuf -i 10000-65000 -n 1) start_port=$(shuf -i 10000-65000 -n 1)
start_silo_10drive ${start_port} start_minio_10drive ${start_port}
} }
main "$@" main "$@"
+13 -50
View File
@@ -20,10 +20,6 @@ if ! [[ "${PKG_VERSION}" =~ ^[0-9]{14}\.0\.0$ ]]; then
exit 1 exit 1
fi fi
# The PGSTY release segment, PGDG-style. sign-release-rpms.sh declares the
# same value as expected_release, and test-release.yml asserts the two agree.
PKG_RELEASE="1PGSTY"
if ! command -v nfpm >/dev/null 2>&1; then if ! command -v nfpm >/dev/null 2>&1; then
echo "nfpm is required" >&2 echo "nfpm is required" >&2
exit 1 exit 1
@@ -38,37 +34,11 @@ fi
# against the config file, so the unit path is passed in absolute. Otherwise # against the config file, so the unit path is passed in absolute. Otherwise
# this only works when invoked from the repository root and fails elsewhere on # this only works when invoked from the repository root and fails elsewhere on
# a message that names the file rather than the cause. # a message that names the file rather than the cause.
unit_file="${repo_dir}/silo.service" unit_file="${repo_dir}/minio.service"
defaults_file="${repo_dir}/silo.env"
sysusers_file="${repo_dir}/silo.sysusers"
license_file="${repo_dir}/LICENSE"
notice_file="${repo_dir}/NOTICE"
postinstall_file="${repo_dir}/buildscripts/package/postinstall.sh"
preremove_file="${repo_dir}/buildscripts/package/preremove.sh"
if [ ! -f "${unit_file}" ]; then if [ ! -f "${unit_file}" ]; then
echo "Missing systemd unit: ${unit_file}" >&2 echo "Missing systemd unit: ${unit_file}" >&2
exit 1 exit 1
fi fi
if [ ! -f "${defaults_file}" ]; then
echo "Missing defaults file: ${defaults_file}" >&2
exit 1
fi
if [ ! -f "${sysusers_file}" ]; then
echo "Missing sysusers file: ${sysusers_file}" >&2
exit 1
fi
for distributed_doc in "${license_file}" "${notice_file}"; do
if [ ! -s "${distributed_doc}" ]; then
echo "Missing license material: ${distributed_doc}" >&2
exit 1
fi
done
for lifecycle_script in "${postinstall_file}" "${preremove_file}"; do
if [ ! -x "${lifecycle_script}" ]; then
echo "Missing executable package lifecycle script: ${lifecycle_script}" >&2
exit 1
fi
done
packages_dir="${dist_dir}/packages" packages_dir="${dist_dir}/packages"
mkdir -p "${packages_dir}" mkdir -p "${packages_dir}"
@@ -98,7 +68,7 @@ find_binary() {
# variants (an added goamd64 level, a stale dist entry) would silently ship a # variants (an added goamd64 level, a stale dist entry) would silently ship a
# package whose contents do not match its name. # package whose contents do not match its name.
matches="$(find "${dist_dir}" -maxdepth 2 -type f \ matches="$(find "${dist_dir}" -maxdepth 2 -type f \
-path "${dist_dir}/silo_linux_${goarch}*/silo" | sort)" -path "${dist_dir}/minio_linux_${goarch}*/minio" | sort)"
count="$(printf '%s' "${matches}" | grep -c . || true)" count="$(printf '%s' "${matches}" | grep -c . || true)"
if [ "${count}" -eq 0 ]; then if [ "${count}" -eq 0 ]; then
@@ -126,25 +96,18 @@ build_arch() {
source="$(find_binary "${goarch}")" source="$(find_binary "${goarch}")"
# These names are the public download names and must not drift; RPM and DEB # These names are the public download names and must not drift; RPM carries a
# carry the PGSTY release number (nfpm renders it as the RPM Release tag and # release number, DEB and APK do not, matching what pkger produced.
# as the Debian revision after a dash). APK stays bare: Alpine pkgrel only rpm_file="${packages_dir}/minio-${PKG_VERSION}-1.${rpm_arch}.rpm"
# admits -r<integer>, so a lettered release cannot ride along there. deb_file="${packages_dir}/minio_${PKG_VERSION}_${deb_arch}.deb"
rpm_file="${packages_dir}/silo-${PKG_VERSION}-${PKG_RELEASE}.${rpm_arch}.rpm" apk_file="${packages_dir}/minio_${PKG_VERSION}_${apk_arch}.apk"
deb_file="${packages_dir}/silo_${PKG_VERSION}-${PKG_RELEASE}_${deb_arch}.deb"
apk_file="${packages_dir}/silo_${PKG_VERSION}_${apk_arch}.apk"
( PKG_VERSION="${PKG_VERSION}" NFPM_RELEASE=1 NFPM_ARCH="${goarch}" NFPM_SOURCE="${source}" NFPM_UNIT="${unit_file}" \
cd "${repo_dir}" nfpm package --config "${nfpm_config}" --packager rpm --target "${rpm_file}"
export NFPM_UNIT="${unit_file}" NFPM_DEFAULTS="${defaults_file}" NFPM_SYSUSERS="${sysusers_file}" \ PKG_VERSION="${PKG_VERSION}" NFPM_RELEASE='' NFPM_ARCH="${goarch}" NFPM_SOURCE="${source}" NFPM_UNIT="${unit_file}" \
NFPM_LICENSE="${license_file}" NFPM_NOTICE="${notice_file}" nfpm package --config "${nfpm_config}" --packager deb --target "${deb_file}"
PKG_VERSION="${PKG_VERSION}" NFPM_RELEASE="${PKG_RELEASE}" NFPM_ARCH="${goarch}" NFPM_SOURCE="${source}" \ PKG_VERSION="${PKG_VERSION}" NFPM_RELEASE='' NFPM_ARCH="${goarch}" NFPM_SOURCE="${source}" NFPM_UNIT="${unit_file}" \
nfpm package --config "${nfpm_config}" --packager rpm --target "${rpm_file}" nfpm package --config "${nfpm_config}" --packager apk --target "${apk_file}"
PKG_VERSION="${PKG_VERSION}" NFPM_RELEASE="${PKG_RELEASE}" NFPM_ARCH="${goarch}" NFPM_SOURCE="${source}" \
nfpm package --config "${nfpm_config}" --packager deb --target "${deb_file}"
PKG_VERSION="${PKG_VERSION}" NFPM_RELEASE='' NFPM_ARCH="${goarch}" NFPM_SOURCE="${source}" \
nfpm package --config "${nfpm_config}" --packager apk --target "${apk_file}"
)
sha256_file "${rpm_file}" sha256_file "${rpm_file}"
sha256_file "${deb_file}" sha256_file "${deb_file}"
-172
View File
@@ -1,172 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
repo_dir="$(cd "${script_dir}/../.." && pwd)"
postinstall="${script_dir}/postinstall.sh"
preremove="${script_dir}/preremove.sh"
test_dir="$(mktemp -d)"
fakebin="${test_dir}/bin"
log_file="${test_dir}/calls.log"
useradd_shell=/usr/sbin/nologin
[ -x "${useradd_shell}" ] || useradd_shell=/sbin/nologin
busybox_shell=/sbin/nologin
[ -x "${busybox_shell}" ] || busybox_shell=/bin/false
cleanup() {
rm -rf "${test_dir}"
}
trap cleanup EXIT
mkdir -p "${fakebin}"
touch "${log_file}"
# One dispatcher represents every external command used by the lifecycle
# scripts. The tested scripts run with no host utilities in PATH, so a green
# result cannot create a real account or touch the host service manager.
cat > "${fakebin}/fake-command" <<'EOF'
#!/bin/sh
set -eu
command_name=${0##*/}
case "${command_name}" in
id)
[ "${PACKAGE_TEST_USER_EXISTS:-0}" = 1 ]
;;
getent)
[ "${PACKAGE_TEST_GROUP_EXISTS:-0}" = 1 ]
;;
systemd-sysusers|useradd|addgroup|adduser|systemctl)
{
printf '%s' "${command_name}"
for argument in "$@"; do
printf ' %s' "${argument}"
done
printf '\n'
} >> "${PACKAGE_TEST_LOG}"
;;
*)
echo "unexpected fake command: ${command_name}" >&2
exit 1
;;
esac
EOF
chmod +x "${fakebin}/fake-command"
link_command() {
ln -sf fake-command "${fakebin}/$1"
}
unlink_optional_commands() {
rm -f \
"${fakebin}/systemd-sysusers" \
"${fakebin}/useradd" \
"${fakebin}/adduser" \
"${fakebin}/addgroup"
}
reset_log() {
: > "${log_file}"
}
run_postinstall() {
PACKAGE_TEST_LOG="${log_file}" \
PACKAGE_TEST_USER_EXISTS="${1}" \
PACKAGE_TEST_GROUP_EXISTS="${2}" \
PATH="${fakebin}" \
/bin/sh "${postinstall}"
}
run_preremove() {
PACKAGE_TEST_LOG="${log_file}" PATH="${fakebin}" \
/bin/sh "${preremove}" "$@"
}
assert_log_line() {
grep -Fx -- "$1" "${log_file}" >/dev/null
}
reject_log_text() {
if grep -F -- "$1" "${log_file}" >/dev/null; then
echo "unexpected lifecycle call containing '$1':" >&2
cat "${log_file}" >&2
exit 1
fi
}
link_command id
link_command getent
link_command systemctl
# Clean install through systemd-sysusers. Side-by-side safety is represented
# by the fact that the only service-manager operation is daemon-reload: no old
# service is stopped, disabled, enabled, masked, or restarted.
unlink_optional_commands
link_command systemd-sysusers
reset_log
run_postinstall 0 0
assert_log_line "systemd-sysusers /usr/lib/sysusers.d/silo.conf"
assert_log_line "systemctl daemon-reload"
test "$(wc -l < "${log_file}" | tr -d ' ')" -eq 2
# An existing service account is preserved without modification.
reset_log
run_postinstall 1 0
test "$(cat "${log_file}")" = "systemctl daemon-reload"
# useradd creates a private group only when one does not already exist. An
# administrator may pre-create group silo with the legacy GID; that group must
# be reused rather than causing installation to fail.
unlink_optional_commands
link_command useradd
reset_log
run_postinstall 0 0
assert_log_line "useradd --system --user-group --no-create-home --shell ${useradd_shell} --comment Silo object storage service silo"
reject_log_text "--gid silo"
reset_log
run_postinstall 0 1
assert_log_line "useradd --system --gid silo --no-create-home --shell ${useradd_shell} --comment Silo object storage service silo"
reject_log_text "--user-group"
# BusyBox follows the same existing-group contract.
unlink_optional_commands
link_command adduser
link_command addgroup
reset_log
run_postinstall 0 0
assert_log_line "addgroup -S silo"
assert_log_line "adduser -S -D -H -G silo -s ${busybox_shell} silo"
reset_log
run_postinstall 0 1
reject_log_text "addgroup"
assert_log_line "adduser -S -D -H -G silo -s ${busybox_shell} silo"
# Debian remove, RPM erase, and Alpine deinstall stop the Silo unit. Upgrade
# arguments must leave the running service alone.
for removal_argument in remove 0 20260214120000.0.0-r0; do
reset_log
run_preremove "${removal_argument}"
test "$(cat "${log_file}")" = "systemctl disable --now silo.service"
done
for upgrade_argument in upgrade 1; do
reset_log
run_preremove "${upgrade_argument}"
test ! -s "${log_file}"
done
# The package deliberately leaves legacy ownership changes to an explicit
# systemd drop-in. Lifecycle scripts must never rewrite ownership or touch the
# old unit, and the base unit must expose overridable User/Group directives.
grep -Fx 'User=silo' "${repo_dir}/silo.service" >/dev/null
grep -Fx 'Group=silo' "${repo_dir}/silo.service" >/dev/null
if grep -Ein '\b(chown|chgrp|usermod|groupmod)\b|minio\.service' \
"${postinstall}" "${preremove}"; then
echo "package lifecycle scripts must not mutate data ownership or the legacy service" >&2
exit 1
fi
echo "Silo package lifecycle checks passed"
-44
View File
@@ -1,44 +0,0 @@
#!/bin/sh
set -eu
sysusers_file=/usr/lib/sysusers.d/silo.conf
group_exists() {
if command -v getent >/dev/null 2>&1; then
getent group silo >/dev/null 2>&1
return
fi
[ -r /etc/group ] || return 1
while IFS=: read -r group_name _; do
[ "${group_name}" = silo ] && return 0
done < /etc/group
return 1
}
if ! id -u silo >/dev/null 2>&1; then
if command -v systemd-sysusers >/dev/null 2>&1; then
systemd-sysusers "${sysusers_file}"
elif command -v useradd >/dev/null 2>&1; then
nologin_shell=/usr/sbin/nologin
[ -x "${nologin_shell}" ] || nologin_shell=/sbin/nologin
if group_exists; then
useradd --system --gid silo --no-create-home --shell "${nologin_shell}" --comment "Silo object storage service" silo
else
useradd --system --user-group --no-create-home --shell "${nologin_shell}" --comment "Silo object storage service" silo
fi
elif command -v adduser >/dev/null 2>&1 && command -v addgroup >/dev/null 2>&1; then
nologin_shell=/sbin/nologin
[ -x "${nologin_shell}" ] || nologin_shell=/bin/false
group_exists || addgroup -S silo
adduser -S -D -H -G silo -s "${nologin_shell}" silo
else
echo "Unable to create the silo system account: systemd-sysusers, useradd, or BusyBox adduser is required" >&2
exit 1
fi
fi
if command -v systemctl >/dev/null 2>&1; then
systemctl daemon-reload >/dev/null 2>&1 || true
fi
-14
View File
@@ -1,14 +0,0 @@
#!/bin/sh
set -eu
# Debian passes "remove" for an actual removal, RPM passes 0 to %preun, and
# Alpine runs pre-deinstall only for removal and passes the old dotted version.
# Upgrade paths deliberately leave the running service untouched.
case "${1:-}" in
remove|0|*.*)
if command -v systemctl >/dev/null 2>&1; then
systemctl disable --now silo.service >/dev/null 2>&1 || true
fi
;;
esac
-52
View File
@@ -1,52 +0,0 @@
-----BEGIN PGP PUBLIC KEY BLOCK-----
mQINBGaV5PwBEACbErI+7yOrsXTT3mR83O6Fw9WyHJqozhyNPF3dA1gAtWpfWqd4
S9x6vBjVwUbIRn21jYgov0hDiaLABNQhRzifvVr0r1IjBW8lhA8zJGaO42Uz0aBW
YIkajOklsXgYMX+gSmy5WXzM31sDQVMnzptHh9dwW067hMM5pJKDslu2pLMwSb9K
QgIFcYsaR0taBkcDg4dNu1gncriD/GcdXIS0/V4R82DIYeIqj2S0lt0jDTACbUz3
C6esrTw2XerCeHKHb9c/V+KMhqvLJOOpy/aJWLrTGBoaH7xw6v0qg32OYiBxlUj9
VEzoQbDfbRkR+jlxiuYP3scUs/ziKrSh+0mshVbeuLRSNfuHLa7C4xTEnATcgD1J
MZeMaJXIcDt+DN+1aHVQjY5YNvr5wA3ykxW51uReZf7/odgqVW3+1rhW5pd8NQKQ
qoVUHOtIrC9KaiGfrczEtJTNUxcNZV9eBgcKHYDXB2hmR2pIf7WvydgXTs/qIsXg
SIzfKjisi795Dd5GrvdLYXVnu9YzylWlkJ5rjod1wnSxkI/CcCJaoPLnXZA9KV7A
cpMWWaUEXP/XBIwIU+vxDd1taBIaPIOv1KIdzvG7QqAQtf5Lphi5HfaGvBud/CVt
mvWhRPJMr1J0ER2xAgU2iZR7dN0vSF6zDqc0W09RAoC0nDS3tupDX2BrOwARAQAB
tCRSdW9oYW5nIEZlbmcgKFBpZ3N0eSkgPHJoQHZvbm5nLmNvbT6JAlEEEwEIADsW
IQSVkqe8emguczM3bgnnk12Nub2LIAUCZpXk/AIbAwULCQgHAgIiAgYVCgkICwIE
FgIDAQIeBwIXgAAKCRDnk12Nub2LIOMuEACBLVc09O4icFwc45R3KMvOMu14Egpn
UkpmBKhErjup0TIunzI0zZH6HG8LGuf6XEdH4ItCJeLg5349UE00BUHNmxk2coo2
u4Wtu28LPqmxb6sqpuRAaefedU6vqfs7YN6WWp52pVF1KdOHkIOcgAQ9z3ZHdosM
I/Y/UxO2t4pjdCAfJHOmGPrbgLcHSMpoLLxjuf3YIwS5NSfjNDd0Y8sKFUcMGLCF
5P0lv5feLLdZvh2Una34UmHKhZlXC5E3vlY9bf/LgsRzXRFQosD0RsCXbz3Tk+zF
+j/eP3WhUvJshqIDuY6eJYCzMjiA8sM5gety+htVJuD0mewp+qAhjxE0d4bIr4qO
BKQzBt9tT2ackCPdgW42VPS+IZymm1oMET0hgZfKiVpwsKO6qxeWn4RW2jJ0zkUJ
MsrrxOPFdZQAtuFcLwa5PUAHHs6XQT2vzxDpeE9lInQ14lshofU5ZKIeb9sbvb/w
P+xnDqvZ1pcotEIBvDK0S0jHbHHqtioIUdDFvdCBlBlYP1TQRNPlJ7TJDBBvhj8i
fmjQsYSV1u36aHOJVGYNHv+SyJpVd3nHCZn97ADM9qHnDm7xljyHXPzIx4FMmBGJ
UTiLH5yxa1xhWr42Iv3TykaQJVbpydmBuegFR8WbWitAvVqI3HvRG+FalLsjJruc
8YDAf7gHdj/937kCDQRmleT8ARAAmJxscC76NZzqFBiaeq2+aJxOt1HGPqKb4pbz
jLKRX9sFkeXuzhfZaNDljnr2yrnQ75rit9Aah/loEhbSHanNUDCNmvOeSEISr9yA
yfOnqlcVOtcwWQK57n6MvlCSM8Js3jdoSmCFHVtdFFwxejE5ok0dk1VFYDIg6DRk
ZBMuxGO7ZJW7TzCxhK4AL+NNYA2wX6b+IVMn6CA9kwNwCNrrnGHR1sblSxZp7lPo
+GsqzYY0LXGR2eEicgKd4lk38gaO8Q4d1mlpX95vgdhGKxR+CM26y9QU0qrO1hXP
Fw6lX9HfIUkVNrqAa1mzgneYXivnLvcj8gc7bFAdweX4MyBHsmiPm32WqjUJFAmw
kcKYaiyfDJ+1wusa/b+7RCnshWc8B9udYbXfvcpOGgphpUuvomKT8at3ToJfEWmR
BzToYYTsgAAX8diY/X53BHCE/+MhLccglEUYNZyBRkTwDLrS9QgNkhrADaTwxsv1
8PwnVKve/ZxwOU0QGf4ZOhA2YQOE5hkRDR5uY2OHsOS5vHsd9Y6kNNnO8EBy99d1
QiBJOW3AP0nr4Cj1/NhdigAujsYRKiCAuPT7dgqART58VU4bZ3PgonMlziLe7+ht
YYxV+wyP6LVqicDd0MLLvG7r/JOiWuABOUxsFFaRecehoPJjeAEQxnWJjedokXKL
HVOFaEkAEQEAAYkCNgQYAQgAIBYhBJWSp7x6aC5zMzduCeeTXY25vYsgBQJmleT8
AhsMAAoJEOeTXY25vYsgG8sP/3UdsWuiwTsf/x4BTW82K+Uk9YwZDnUNH+4dUMED
bKT1C6CbuSZ7Mnbi2rVsmGzOMs9MehIx6Ko8/iCR2OCeWi8Q+wM+iffAfWuT1GK6
7f/VIfoYBUWEa+kvDcPgEbd5Tu7ZdUO/jROVBSlXRSjzK9LpIj7GozBTJ8Vqy5x7
oqbWPPEYtGDVHime8o6f5/wfhNgL3mFnoq6srK7KhwACwfTXlNqAlGiXGa30Yj+b
Cj6IvmxoII49E67/ovMEmzDCb3RXiaL6OATy25P+HQJvWvAam7Qq5Xn+bZg65Mup
vXq3zoX0a7EKXc5vsJVNtTlXO1ATdYszKP5uNzkHrNAN52VRYaowq1vPy/MVMbSI
rL/hTFKr7ZNhmC7jmS3OuJyCYQsfEerubtBUuc/W6JDc2oTI3xOG1S2Zj8f4PxLl
H7vMG4E+p6eOrUGw6VQXjFsH9GtwhkPh/ZGMKENb2+JztJ02674Cok4s5c/lZFKz
mmRUcNjX2bm2K0GfGG5/hAog/CHCeUZvwIh4hZLkdeJ1QsIYpN8xbvY7QP6yh4VB
XrL18+2sontZ45MsGResrRibB35x7IrCrxZsVtRJZthHqshiORPatgy+AiWcAtEv
UWEnnC1xBSasNebw4fSE8AJg9JMCRw+3GAetlotOeW9q7PN6yrXD9rGuV/QquQNd
/c7w
=4rRi
-----END PGP PUBLIC KEY BLOCK-----
File diff suppressed because it is too large Load Diff
-480
View File
@@ -1,480 +0,0 @@
// Copyright 2026 PGSTY contributors.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// rebrand-guard records the compatibility identifiers that a product rebrand
// must not accidentally rename. It intentionally excludes product branding and
// delivery names, which are validated by buildscripts/verify-rebrand.sh.
package main
import (
"bytes"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"flag"
"fmt"
"go/ast"
"go/parser"
"go/token"
"os"
"os/exec"
"path/filepath"
"regexp"
"sort"
"strconv"
"strings"
)
const manifestVersion = 3
var (
minioImportRE = regexp.MustCompile(`github\.com/minio/[A-Za-z0-9_./-]+`)
envRE = regexp.MustCompile(`\b_?MINIO_[A-Z0-9_]+\b`)
metricRE = regexp.MustCompile(`\bminio_[A-Za-z0-9_]+\b`)
headerRE = regexp.MustCompile(`(?i)\bx-minio-[a-z0-9_-]+\b`)
routeRE = regexp.MustCompile(`^/[A-Za-z0-9._~!$&'()*+,;=:@%/?{}=-]*`)
storageRE = regexp.MustCompile(`\.minio\.sys(?:/[A-Za-z0-9._${}-]+)*`)
policyRE = regexp.MustCompile(`(?:arn:minio|minio:s3)[A-Za-z0-9_:/.*${}-]*`)
brandRE = regexp.MustCompile(`(?i)(^|[^a-z0-9_])minio([^a-z0-9_]|$)`)
)
type manifest struct {
Version int `json:"version"`
ModulePath string `json:"module_path"`
MinioImports []string `json:"minio_imports"`
Environment []string `json:"environment"`
Metrics []string `json:"metrics"`
Headers []string `json:"headers"`
Routes []string `json:"routes"`
RouteRoots []string `json:"route_roots"`
GridRoutes []string `json:"grid_routes"`
StorageMarkers []string `json:"storage_markers"`
PolicyValues []string `json:"policy_values"`
ExportedSymbols []string `json:"exported_symbols"`
BrandAllowlist []string `json:"brand_allowlist"`
}
func main() {
write := flag.Bool("write", false, "replace the checked-in compatibility baseline")
flag.Parse()
repo, err := gitOutput("rev-parse", "--show-toplevel")
if err != nil {
fatal(err)
}
repo = strings.TrimSpace(repo)
baselinePath := filepath.Join(repo, "buildscripts", "rebrand-guard", "compat-baseline.json")
current, err := collect(repo)
if err != nil {
fatal(err)
}
if *write {
if err := writeManifest(baselinePath, current); err != nil {
fatal(err)
}
fmt.Printf("wrote %s\n", baselinePath)
printSummary(current)
return
}
want, err := readManifest(baselinePath)
if err != nil {
fatal(err)
}
if err := compare(want, current); err != nil {
fatal(err)
}
printSummary(current)
fmt.Println("Silo rebrand compatibility baseline is unchanged")
}
func collect(repo string) (manifest, error) {
files, err := trackedFiles(repo)
if err != nil {
return manifest{}, err
}
sets := map[string]map[string]struct{}{
"imports": {},
"env": {},
"metrics": {},
"headers": {},
"routes": {},
"roots": {},
"grid": {},
"storage": {},
"policy": {},
"exported": {},
"brand": {},
}
modulePath := ""
fset := token.NewFileSet()
for _, rel := range files {
if rel == "SILO_REBRANDING_MIGRATION.md" ||
strings.HasPrefix(rel, "buildscripts/rebrand-guard/") ||
strings.HasPrefix(rel, "buildscripts/helm-migration-guard/") {
continue
}
path := filepath.Join(repo, filepath.FromSlash(rel))
data, err := os.ReadFile(path)
if errors.Is(err, os.ErrNotExist) {
continue
}
if err != nil {
return manifest{}, fmt.Errorf("read %s: %w", rel, err)
}
if bytes.IndexByte(data, 0) >= 0 {
continue
}
text := string(data)
addMatches(sets["env"], envRE, text, false)
addMatches(sets["headers"], headerRE, text, true)
addMatches(sets["storage"], storageRE, text, false)
addMatches(sets["policy"], policyRE, text, false)
if strings.HasSuffix(rel, ".go") && (strings.HasPrefix(rel, "cmd/") || strings.HasPrefix(rel, "internal/")) {
addMatches(sets["metrics"], metricRE, text, false)
}
if rel == "go.mod" {
addMatches(sets["imports"], minioImportRE, text, false)
for _, line := range strings.Split(text, "\n") {
fields := strings.Fields(line)
if len(fields) == 2 && fields[0] == "module" {
modulePath = fields[1]
break
}
}
}
if strings.HasSuffix(rel, ".go") {
file, err := parser.ParseFile(fset, path, data, parser.SkipObjectResolution)
if err != nil {
return manifest{}, fmt.Errorf("parse %s: %w", rel, err)
}
for _, spec := range file.Imports {
value, err := strconv.Unquote(spec.Path.Value)
if err == nil && strings.HasPrefix(value, "github.com/minio/") {
sets["imports"][value] = struct{}{}
}
}
collectStringMatches(sets["routes"], routeRE, file)
collectNamedStringValues(sets["roots"], rel, file, "minioReservedBucket")
if rel == "internal/grid/manager.go" {
collectStringMatches(sets["grid"], routeRE, file)
}
if !strings.HasSuffix(rel, "_test.go") {
collectExported(sets["exported"], filepath.ToSlash(filepath.Dir(rel)), file)
if strings.HasPrefix(rel, "cmd/") || strings.HasPrefix(rel, "internal/") {
collectBrandStrings(sets["brand"], rel, file)
}
}
}
}
// This was a shell-local PID variable in the generated inspect script,
// never a supported environment setting.
delete(sets["env"], "MINIO_SRVR_PID")
if modulePath == "" {
return manifest{}, errors.New("go.mod module path was not found")
}
return manifest{
Version: manifestVersion,
ModulePath: modulePath,
MinioImports: sorted(sets["imports"]),
Environment: sorted(sets["env"]),
Metrics: sorted(sets["metrics"]),
Headers: sorted(sets["headers"]),
Routes: sorted(sets["routes"]),
RouteRoots: sorted(sets["roots"]),
GridRoutes: sorted(sets["grid"]),
StorageMarkers: sorted(sets["storage"]),
PolicyValues: sorted(sets["policy"]),
ExportedSymbols: sorted(sets["exported"]),
BrandAllowlist: sorted(sets["brand"]),
}, nil
}
func collectBrandStrings(dst map[string]struct{}, rel string, file *ast.File) {
ast.Inspect(file, func(node ast.Node) bool {
literal, ok := node.(*ast.BasicLit)
if !ok || literal.Kind != token.STRING {
return true
}
value, err := strconv.Unquote(literal.Value)
if err != nil || !brandRE.MatchString(value) || strings.HasPrefix(value, "github.com/minio/") {
return true
}
dst[filepath.ToSlash(rel)+"="+strconv.Quote(value)] = struct{}{}
return true
})
}
func collectNamedStringValues(dst map[string]struct{}, rel string, file *ast.File, names ...string) {
wanted := make(map[string]struct{}, len(names))
for _, name := range names {
wanted[name] = struct{}{}
}
for _, decl := range file.Decls {
gen, ok := decl.(*ast.GenDecl)
if !ok {
continue
}
for _, rawSpec := range gen.Specs {
spec, ok := rawSpec.(*ast.ValueSpec)
if !ok {
continue
}
for i, name := range spec.Names {
if _, ok := wanted[name.Name]; !ok || i >= len(spec.Values) {
continue
}
literal, ok := spec.Values[i].(*ast.BasicLit)
if !ok || literal.Kind != token.STRING {
continue
}
value, err := strconv.Unquote(literal.Value)
if err == nil {
dst[filepath.ToSlash(rel)+":"+name.Name+"="+value] = struct{}{}
}
}
}
}
}
func collectStringMatches(dst map[string]struct{}, re *regexp.Regexp, file *ast.File) {
ast.Inspect(file, func(node ast.Node) bool {
literal, ok := node.(*ast.BasicLit)
if !ok || literal.Kind != token.STRING {
return true
}
value, err := strconv.Unquote(literal.Value)
if err == nil {
addMatches(dst, re, value, false)
}
return true
})
}
func trackedFiles(repo string) ([]string, error) {
cmd := exec.Command("git", "-C", repo, "ls-files", "--cached", "--others", "--exclude-standard", "-z")
out, err := cmd.Output()
if err != nil {
return nil, fmt.Errorf("git ls-files: %w", err)
}
parts := bytes.Split(out, []byte{0})
files := make([]string, 0, len(parts))
for _, part := range parts {
if len(part) > 0 {
files = append(files, string(part))
}
}
return files, nil
}
func addMatches(dst map[string]struct{}, re *regexp.Regexp, text string, lower bool) {
for _, match := range re.FindAllString(text, -1) {
if lower {
match = strings.ToLower(match)
}
dst[match] = struct{}{}
}
}
func collectExported(dst map[string]struct{}, dir string, file *ast.File) {
prefix := dir + ":" + file.Name.Name + ":"
for _, decl := range file.Decls {
switch decl := decl.(type) {
case *ast.FuncDecl:
if !ast.IsExported(decl.Name.Name) {
continue
}
if decl.Recv == nil {
dst[prefix+"func:"+decl.Name.Name] = struct{}{}
continue
}
receiver := receiverName(decl.Recv.List[0].Type)
dst[prefix+"method:"+receiver+"."+decl.Name.Name] = struct{}{}
case *ast.GenDecl:
for _, spec := range decl.Specs {
switch spec := spec.(type) {
case *ast.TypeSpec:
if !ast.IsExported(spec.Name.Name) {
continue
}
dst[prefix+"type:"+spec.Name.Name] = struct{}{}
collectExportedFields(dst, prefix, spec.Name.Name, spec.Type)
case *ast.ValueSpec:
kind := strings.ToLower(decl.Tok.String())
for _, name := range spec.Names {
if ast.IsExported(name.Name) {
dst[prefix+kind+":"+name.Name] = struct{}{}
}
}
}
}
}
}
}
func collectExportedFields(dst map[string]struct{}, prefix, typeName string, expr ast.Expr) {
var fields *ast.FieldList
switch typed := expr.(type) {
case *ast.StructType:
fields = typed.Fields
case *ast.InterfaceType:
fields = typed.Methods
default:
return
}
for _, field := range fields.List {
for _, name := range field.Names {
if ast.IsExported(name.Name) {
dst[prefix+"field:"+typeName+"."+name.Name] = struct{}{}
}
}
}
}
func receiverName(expr ast.Expr) string {
switch expr := expr.(type) {
case *ast.Ident:
return expr.Name
case *ast.StarExpr:
return receiverName(expr.X)
case *ast.IndexExpr:
return receiverName(expr.X)
case *ast.IndexListExpr:
return receiverName(expr.X)
case *ast.SelectorExpr:
return receiverName(expr.X) + "." + expr.Sel.Name
default:
return fmt.Sprintf("%T", expr)
}
}
func sorted(set map[string]struct{}) []string {
values := make([]string, 0, len(set))
for value := range set {
values = append(values, value)
}
sort.Strings(values)
return values
}
func writeManifest(path string, value manifest) error {
data, err := json.MarshalIndent(value, "", " ")
if err != nil {
return err
}
data = append(data, '\n')
return os.WriteFile(path, data, 0o644)
}
func readManifest(path string) (manifest, error) {
data, err := os.ReadFile(path)
if err != nil {
return manifest{}, fmt.Errorf("read compatibility baseline (run go run ./buildscripts/rebrand-guard --write once): %w", err)
}
var value manifest
if err := json.Unmarshal(data, &value); err != nil {
return manifest{}, err
}
if value.Version != manifestVersion {
return manifest{}, fmt.Errorf("unsupported compatibility baseline version %d", value.Version)
}
return value, nil
}
func compare(want, got manifest) error {
var failures []string
if want.ModulePath != got.ModulePath {
failures = append(failures, fmt.Sprintf("module_path: want %q, got %q", want.ModulePath, got.ModulePath))
}
checks := []struct {
name string
want, got []string
}{
{"minio_imports", want.MinioImports, got.MinioImports},
{"environment", want.Environment, got.Environment},
{"metrics", want.Metrics, got.Metrics},
{"headers", want.Headers, got.Headers},
{"routes", want.Routes, got.Routes},
{"route_roots", want.RouteRoots, got.RouteRoots},
{"grid_routes", want.GridRoutes, got.GridRoutes},
{"storage_markers", want.StorageMarkers, got.StorageMarkers},
{"policy_values", want.PolicyValues, got.PolicyValues},
{"exported_symbols", want.ExportedSymbols, got.ExportedSymbols},
{"brand_allowlist", want.BrandAllowlist, got.BrandAllowlist},
}
for _, check := range checks {
if missing, added := setDiff(check.want, check.got); len(missing) > 0 || len(added) > 0 {
var b strings.Builder
fmt.Fprintf(&b, "%s compatibility set changed", check.name)
for _, value := range missing {
fmt.Fprintf(&b, "\n - %s", value)
}
for _, value := range added {
fmt.Fprintf(&b, "\n + %s", value)
}
failures = append(failures, b.String())
}
}
if len(failures) > 0 {
return errors.New(strings.Join(failures, "\n"))
}
return nil
}
func setDiff(want, got []string) (missing, added []string) {
wantSet := make(map[string]struct{}, len(want))
gotSet := make(map[string]struct{}, len(got))
for _, value := range want {
wantSet[value] = struct{}{}
}
for _, value := range got {
gotSet[value] = struct{}{}
}
for _, value := range want {
if _, ok := gotSet[value]; !ok {
missing = append(missing, value)
}
}
for _, value := range got {
if _, ok := wantSet[value]; !ok {
added = append(added, value)
}
}
return missing, added
}
func printSummary(value manifest) {
fmt.Printf("compatibility manifest: imports=%d env=%d metrics=%d headers=%d routes=%d roots=%d grid=%d storage=%d policy=%d exported=%d brand=%d sha256=%s\n",
len(value.MinioImports), len(value.Environment), len(value.Metrics), len(value.Headers),
len(value.Routes), len(value.RouteRoots), len(value.GridRoutes), len(value.StorageMarkers), len(value.PolicyValues),
len(value.ExportedSymbols), len(value.BrandAllowlist), manifestDigest(value))
}
func manifestDigest(value manifest) string {
data, _ := json.Marshal(value)
sum := sha256.Sum256(data)
return hex.EncodeToString(sum[:])
}
func gitOutput(args ...string) (string, error) {
out, err := exec.Command("git", args...).CombinedOutput()
if err != nil {
return "", fmt.Errorf("git %s: %s: %w", strings.Join(args, " "), strings.TrimSpace(string(out)), err)
}
return string(out), nil
}
func fatal(err error) {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
+23 -14
View File
@@ -6,29 +6,38 @@ set -x
set -e set -e
WORK_DIR="$PWD/.verify-$RANDOM" WORK_DIR="$PWD/.verify-$RANDOM"
SILO_CONFIG_DIR="$WORK_DIR/.silo" MINIO_CONFIG_DIR="$WORK_DIR/.minio"
SILO=("$PWD/silo" --config-dir "$SILO_CONFIG_DIR" server) MINIO=("$PWD/minio" --config-dir "$MINIO_CONFIG_DIR" server)
if [ ! -x "$PWD/silo" ]; then if [ ! -x "$PWD/minio" ]; then
echo "Silo executable binary not found in current directory" echo "minio executable binary not found in current directory"
exit 1 exit 1
fi fi
function start_silo_5drive() { function start_minio_5drive() {
start_port=$1 start_port=$1
export MINIO_ROOT_USER=silo export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=silo123 export MINIO_ROOT_PASSWORD=minio123
export MC_HOST_silo="http://silo:silo123@127.0.0.1:${start_port}/" export MC_HOST_minio="http://minio:minio123@127.0.0.1:${start_port}/"
unset MINIO_KMS_AUTO_ENCRYPTION # do not auto-encrypt objects unset MINIO_KMS_AUTO_ENCRYPTION # do not auto-encrypt objects
export MINIO_CI_CD=1 export MINIO_CI_CD=1
mkdir -p "${WORK_DIR}" MC_BUILD_DIR="mc-$RANDOM"
"$(git rev-parse --show-toplevel)/buildscripts/install-mcli.sh" "${WORK_DIR}/mc" if ! git clone --quiet https://github.com/minio/mc "$MC_BUILD_DIR"; then
echo "failed to download https://github.com/minio/mc"
purge "${MC_BUILD_DIR}"
exit 1
fi
(cd "${MC_BUILD_DIR}" && go build -o "$WORK_DIR/mc")
# remove mc source.
purge "${MC_BUILD_DIR}"
"${WORK_DIR}/mc" cp --quiet -r "buildscripts/cicd-corpus/" "${WORK_DIR}/cicd-corpus/" "${WORK_DIR}/mc" cp --quiet -r "buildscripts/cicd-corpus/" "${WORK_DIR}/cicd-corpus/"
"${SILO[@]}" --address ":$start_port" "${WORK_DIR}/cicd-corpus/disk{1...5}" >"${WORK_DIR}/server1.log" 2>&1 & "${MINIO[@]}" --address ":$start_port" "${WORK_DIR}/cicd-corpus/disk{1...5}" >"${WORK_DIR}/server1.log" 2>&1 &
pid=$! pid=$!
disown $pid disown $pid
sleep 5 sleep 5
@@ -41,16 +50,16 @@ function start_silo_5drive() {
exit 1 exit 1
fi fi
"${WORK_DIR}/mc" stat silo/bucket/testobj "${WORK_DIR}/mc" stat minio/bucket/testobj
pkill silo pkill minio
sleep 3 sleep 3
} }
function main() { function main() {
start_port=$(shuf -i 10000-65000 -n 1) start_port=$(shuf -i 10000-65000 -n 1)
start_silo_5drive ${start_port} start_minio_5drive ${start_port}
} }
function purge() { function purge() {
+36 -30
View File
@@ -5,42 +5,48 @@ set -o pipefail
set -x set -x
WORK_DIR="$PWD/.verify-$RANDOM" WORK_DIR="$PWD/.verify-$RANDOM"
SILO_CONFIG_DIR="$WORK_DIR/.silo" MINIO_CONFIG_DIR="$WORK_DIR/.minio"
MINIO_OLD=("$PWD/minio.RELEASE.2020-10-28T08-16-50Z" --config-dir "$SILO_CONFIG_DIR" server) MINIO_OLD=("$PWD/minio.RELEASE.2020-10-28T08-16-50Z" --config-dir "$MINIO_CONFIG_DIR" server)
SILO=("$PWD/silo" --config-dir "$SILO_CONFIG_DIR" server) MINIO=("$PWD/minio" --config-dir "$MINIO_CONFIG_DIR" server)
if [ ! -x "$PWD/silo" ]; then if [ ! -x "$PWD/minio" ]; then
echo "Silo executable binary not found in current directory" echo "minio executable binary not found in current directory"
exit 1 exit 1
fi fi
function download_old_release() { function download_old_release() {
if [ ! -x minio.RELEASE.2020-10-28T08-16-50Z ]; then if [ ! -f minio.RELEASE.2020-10-28T08-16-50Z ]; then
: "${SILO_LEGACY_FIXTURE_2020:?set SILO_LEGACY_FIXTURE_2020 to the audited legacy binary}" curl --silent -O https://dl.minio.io/server/minio/release/linux-amd64/archive/minio.RELEASE.2020-10-28T08-16-50Z
: "${SILO_LEGACY_SHA256_2020:?set SILO_LEGACY_SHA256_2020 to its audited SHA-256}" chmod a+x minio.RELEASE.2020-10-28T08-16-50Z
"$(git rev-parse --show-toplevel)/buildscripts/install-verified-fixture.sh" \
"${SILO_LEGACY_FIXTURE_2020}" "${SILO_LEGACY_SHA256_2020}" \
"minio.RELEASE.2020-10-28T08-16-50Z"
fi fi
} }
function verify_rewrite() { function verify_rewrite() {
start_port=$1 start_port=$1
export MINIO_ACCESS_KEY=silo export MINIO_ACCESS_KEY=minio
export MINIO_SECRET_KEY=silo123 export MINIO_SECRET_KEY=minio123
export MC_HOST_silo="http://silo:silo123@127.0.0.1:${start_port}/" export MC_HOST_minio="http://minio:minio123@127.0.0.1:${start_port}/"
unset MINIO_KMS_AUTO_ENCRYPTION # do not auto-encrypt objects unset MINIO_KMS_AUTO_ENCRYPTION # do not auto-encrypt objects
export MINIO_CI_CD=1 export MINIO_CI_CD=1
mkdir -p "${WORK_DIR}" MC_BUILD_DIR="mc-$RANDOM"
"$(git rev-parse --show-toplevel)/buildscripts/install-mcli.sh" "${WORK_DIR}/mc" if ! git clone --quiet https://github.com/minio/mc "$MC_BUILD_DIR"; then
echo "failed to download https://github.com/minio/mc"
purge "${MC_BUILD_DIR}"
exit 1
fi
(cd "${MC_BUILD_DIR}" && go build -o "$WORK_DIR/mc")
# remove mc source.
purge "${MC_BUILD_DIR}"
"${MINIO_OLD[@]}" --address ":$start_port" "${WORK_DIR}/xl{1...16}" >"${WORK_DIR}/server1.log" 2>&1 & "${MINIO_OLD[@]}" --address ":$start_port" "${WORK_DIR}/xl{1...16}" >"${WORK_DIR}/server1.log" 2>&1 &
pid=$! pid=$!
disown $pid disown $pid
"${WORK_DIR}/mc" ready silo/ "${WORK_DIR}/mc" ready minio/
if ! ps -p ${pid} 1>&2 >/dev/null; then if ! ps -p ${pid} 1>&2 >/dev/null; then
echo "server1 log:" echo "server1 log:"
@@ -50,30 +56,30 @@ function verify_rewrite() {
exit 1 exit 1
fi fi
"${WORK_DIR}/mc" mb silo/healing-rewrite-bucket --quiet --with-lock "${WORK_DIR}/mc" mb minio/healing-rewrite-bucket --quiet --with-lock
"${WORK_DIR}/mc" cp \ "${WORK_DIR}/mc" cp \
buildscripts/verify-build.sh \ buildscripts/verify-build.sh \
silo/healing-rewrite-bucket/ \ minio/healing-rewrite-bucket/ \
--disable-multipart --quiet --disable-multipart --quiet
"${WORK_DIR}/mc" cp \ "${WORK_DIR}/mc" cp \
buildscripts/verify-build.sh \ buildscripts/verify-build.sh \
silo/healing-rewrite-bucket/ \ minio/healing-rewrite-bucket/ \
--disable-multipart --quiet --disable-multipart --quiet
"${WORK_DIR}/mc" cp \ "${WORK_DIR}/mc" cp \
buildscripts/verify-build.sh \ buildscripts/verify-build.sh \
silo/healing-rewrite-bucket/ \ minio/healing-rewrite-bucket/ \
--disable-multipart --quiet --disable-multipart --quiet
kill ${pid} kill ${pid}
sleep 3 sleep 3
"${SILO[@]}" --address ":$start_port" "${WORK_DIR}/xl{1...16}" >"${WORK_DIR}/server1.log" 2>&1 & "${MINIO[@]}" --address ":$start_port" "${WORK_DIR}/xl{1...16}" >"${WORK_DIR}/server1.log" 2>&1 &
pid=$! pid=$!
disown $pid disown $pid
"${WORK_DIR}/mc" ready silo/ "${WORK_DIR}/mc" ready minio/
if ! ps -p ${pid} 1>&2 >/dev/null; then if ! ps -p ${pid} 1>&2 >/dev/null; then
echo "server1 log:" echo "server1 log:"
@@ -86,8 +92,8 @@ function verify_rewrite() {
if ! ./s3-check-md5 \ if ! ./s3-check-md5 \
-debug \ -debug \
-versions \ -versions \
-access-key silo \ -access-key minio \
-secret-key silo123 \ -secret-key minio123 \
-endpoint "http://127.0.0.1:${start_port}/" 2>&1 | grep INTACT; then -endpoint "http://127.0.0.1:${start_port}/" 2>&1 | grep INTACT; then
echo "server1 log:" echo "server1 log:"
cat "${WORK_DIR}/server1.log" cat "${WORK_DIR}/server1.log"
@@ -95,7 +101,7 @@ function verify_rewrite() {
mkdir -p inspects mkdir -p inspects
( (
cd inspects cd inspects
"${WORK_DIR}/mc" admin inspect silo/healing-rewrite-bucket/verify-build.sh/** "${WORK_DIR}/mc" admin inspect minio/healing-rewrite-bucket/verify-build.sh/**
) )
"${WORK_DIR}/mc" mb play/inspects "${WORK_DIR}/mc" mb play/inspects
@@ -105,14 +111,14 @@ function verify_rewrite() {
exit 1 exit 1
fi fi
go run ./buildscripts/heal-manual.go "127.0.0.1:${start_port}" "silo" "silo123" go run ./buildscripts/heal-manual.go "127.0.0.1:${start_port}" "minio" "minio123"
sleep 1 sleep 1
if ! ./s3-check-md5 \ if ! ./s3-check-md5 \
-debug \ -debug \
-versions \ -versions \
-access-key silo \ -access-key minio \
-secret-key silo123 \ -secret-key minio123 \
-endpoint http://127.0.0.1:${start_port}/ 2>&1 | grep INTACT; then -endpoint http://127.0.0.1:${start_port}/ 2>&1 | grep INTACT; then
echo "server1 log:" echo "server1 log:"
cat "${WORK_DIR}/server1.log" cat "${WORK_DIR}/server1.log"
@@ -120,7 +126,7 @@ function verify_rewrite() {
mkdir -p inspects mkdir -p inspects
( (
cd inspects cd inspects
"${WORK_DIR}/mc" admin inspect silo/healing-rewrite-bucket/verify-build.sh/** "${WORK_DIR}/mc" admin inspect minio/healing-rewrite-bucket/verify-build.sh/**
) )
"${WORK_DIR}/mc" mb play/inspects "${WORK_DIR}/mc" mb play/inspects
+10 -15
View File
@@ -7,21 +7,16 @@ set -euo pipefail
# other way round would only surface here, on the maintainer's machine, after # other way round would only surface here, on the maintainer's machine, after
# the build has already run and uploaded. # the build has already run and uploaded.
expected_fingerprint="9592A7BC7A682E7333376E09E7935D8DB9BD8B20" expected_fingerprint="9592A7BC7A682E7333376E09E7935D8DB9BD8B20"
expected_release="1PGSTY"
expected_vendor="PGSTY" expected_vendor="PGSTY"
expected_packager="Ruohang Feng (@Vonng) <rh@vonng.com>" expected_packager="Ruohang Feng (@Vonng) <rh@vonng.com>"
expected_url="https://silo.pgsty.com" expected_url="https://silo.pgsty.com"
expected_summary="S3-Interface Libre Object Storage, a community-maintained S3-compatible server." expected_summary="S3-Interface Libre Object Storage, Community-maintained MinIO server fork."
expected_description="S3-Interface Libre Object Storage, a community-maintained S3-compatible server." expected_description="S3-Interface Libre Object Storage, Community-maintained MinIO server fork."
expected_license="AGPL-3.0-or-later" expected_license="AGPL-3.0-or-later"
expected_group="Applications/File" expected_group="Applications/File"
expected_payload="/etc/default/silo expected_payload="/usr/lib/systemd/system/minio.service
/usr/bin/silo /usr/local/bin/minio"
/usr/lib/systemd/system/silo.service repository="${GH_REPO:-pgsty/minio}"
/usr/lib/sysusers.d/silo.conf
/usr/share/doc/silo/LICENSE
/usr/share/doc/silo/NOTICE"
repository="${GH_REPO:-pgsty/silo}"
container="${DNFUPDATE_CONTAINER:-dnfupdate}" container="${DNFUPDATE_CONTAINER:-dnfupdate}"
upload=false upload=false
release_tag="" release_tag=""
@@ -127,8 +122,8 @@ mkdir -p "${unsigned_dir}" "${signed_dir}"
chmod 700 "${work_dir}" "${unsigned_dir}" "${signed_dir}" chmod 700 "${work_dir}" "${unsigned_dir}" "${signed_dir}"
rpm_files=( rpm_files=(
"silo-${package_version}-${expected_release}.x86_64.rpm" "minio-${package_version}-1.x86_64.rpm"
"silo-${package_version}-${expected_release}.aarch64.rpm" "minio-${package_version}-1.aarch64.rpm"
) )
download_patterns=() download_patterns=()
@@ -164,7 +159,7 @@ for rpm_file in "${rpm_files[@]}"; do
done done
safe_tag="$(printf '%s' "${release_tag}" | tr -c 'A-Za-z0-9._-' '_')" safe_tag="$(printf '%s' "${release_tag}" | tr -c 'A-Za-z0-9._-' '_')"
container_dir="/tmp/silo-sign-${safe_tag}-$$" container_dir="/tmp/minio-sign-${safe_tag}-$$"
docker exec "${container}" mkdir -p "${container_dir}" docker exec "${container}" mkdir -p "${container_dir}"
cleanup_container() { cleanup_container() {
@@ -208,9 +203,9 @@ for rpm_file in "${rpm_files[@]}"; do
docker cp "${unsigned_dir}/${rpm_file}" "${container}:${container_dir}/${rpm_file}" >/dev/null docker cp "${unsigned_dir}/${rpm_file}" "${container}:${container_dir}/${rpm_file}" >/dev/null
container_rpm="${container_dir}/${rpm_file}" container_rpm="${container_dir}/${rpm_file}"
assert_rpm_tag "${container_rpm}" NAME silo assert_rpm_tag "${container_rpm}" NAME minio
assert_rpm_tag "${container_rpm}" VERSION "${package_version}" assert_rpm_tag "${container_rpm}" VERSION "${package_version}"
assert_rpm_tag "${container_rpm}" RELEASE "${expected_release}" assert_rpm_tag "${container_rpm}" RELEASE 1
assert_rpm_tag "${container_rpm}" ARCH "${expected_arch}" assert_rpm_tag "${container_rpm}" ARCH "${expected_arch}"
assert_rpm_tag "${container_rpm}" VENDOR "${expected_vendor}" assert_rpm_tag "${container_rpm}" VENDOR "${expected_vendor}"
assert_rpm_tag "${container_rpm}" PACKAGER "${expected_packager}" assert_rpm_tag "${container_rpm}" PACKAGER "${expected_packager}"
+32 -21
View File
@@ -5,16 +5,16 @@ if [ -n "$TEST_DEBUG" ]; then
fi fi
WORK_DIR="$PWD/.verify-$RANDOM" WORK_DIR="$PWD/.verify-$RANDOM"
SILO_CONFIG_DIR="$WORK_DIR/.silo" MINIO_CONFIG_DIR="$WORK_DIR/.minio"
SILO=("$PWD/silo" --config-dir "$SILO_CONFIG_DIR" server) MINIO=("$PWD/minio" --config-dir "$MINIO_CONFIG_DIR" server)
if [ ! -x "$PWD/silo" ]; then if [ ! -x "$PWD/minio" ]; then
echo "Silo executable binary not found in current directory" echo "minio executable binary not found in current directory"
exit 1 exit 1
fi fi
if [ ! -x "$PWD/silo" ]; then if [ ! -x "$PWD/minio" ]; then
echo "Silo executable binary not found in current directory" echo "minio executable binary not found in current directory"
exit 1 exit 1
fi fi
@@ -30,9 +30,9 @@ catch() {
echo "error on line $1" echo "error on line $1"
fi fi
echo "Cleaning up instances of Silo" echo "Cleaning up instances of MinIO"
pkill silo || true pkill minio || true
pkill -9 silo || true pkill -9 minio || true
purge "$WORK_DIR" purge "$WORK_DIR"
if [ $# -ne 0 ]; then if [ $# -ne 0 ]; then
exit $# exit $#
@@ -70,20 +70,31 @@ function send_put_object_request() {
return 0 return 0
} }
function test_silo_with_timeout() { function test_minio_with_timeout() {
start_port=$1 start_port=$1
export MINIO_ROOT_USER=silo export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=silo123 export MINIO_ROOT_PASSWORD=minio123
export MC_HOST_silo="http://silo:silo123@127.0.0.1:${start_port}/" export MC_HOST_minio="http://minio:minio123@127.0.0.1:${start_port}/"
export MINIO_CI_CD=1 export MINIO_CI_CD=1
mkdir ${WORK_DIR} mkdir ${WORK_DIR}
C_PWD=${PWD}
if [ ! -x "$PWD/mc" ]; then if [ ! -x "$PWD/mc" ]; then
"$(git rev-parse --show-toplevel)/buildscripts/install-mcli.sh" "$PWD/mc" MC_BUILD_DIR="mc-$RANDOM"
if ! git clone --quiet https://github.com/minio/mc "$MC_BUILD_DIR"; then
echo "failed to download https://github.com/minio/mc"
purge "${MC_BUILD_DIR}"
exit 1
fi
(cd "${MC_BUILD_DIR}" && go build -o "$C_PWD/mc")
# remove mc source.
purge "${MC_BUILD_DIR}"
fi fi
"${SILO[@]}" --address ":$start_port" --read-header-timeout ${srv_hdr_timeout}s --idle-timeout ${srv_idle_timeout}s "${WORK_DIR}/disk/" >"${WORK_DIR}/server1.log" 2>&1 & "${MINIO[@]}" --address ":$start_port" --read-header-timeout ${srv_hdr_timeout}s --idle-timeout ${srv_idle_timeout}s "${WORK_DIR}/disk/" >"${WORK_DIR}/server1.log" 2>&1 &
pid=$! pid=$!
disown $pid disown $pid
sleep 1 sleep 1
@@ -98,20 +109,20 @@ function test_silo_with_timeout() {
set -e set -e
"${PWD}/mc" mb silo/testbucket "${PWD}/mc" mb minio/testbucket
"${PWD}/mc" anonymous set public silo/testbucket "${PWD}/mc" anonymous set public minio/testbucket
# slow header writing # slow header writing
send_put_object_request 20 0 && exit -1 send_put_object_request 20 0 && exit -1
"${PWD}/mc" stat silo/testbucket/testobject && exit -1 "${PWD}/mc" stat minio/testbucket/testobject && exit -1
# quick header write and slow bodywrite # quick header write and slow bodywrite
send_put_object_request 0 40 && exit -1 send_put_object_request 0 40 && exit -1
"${PWD}/mc" stat silo/testbucket/testobject && exit -1 "${PWD}/mc" stat minio/testbucket/testobject && exit -1
# quick header and body write # quick header and body write
send_put_object_request 1 1 || exit -1 send_put_object_request 1 1 || exit -1
"${PWD}/mc" stat silo/testbucket/testobject || exit -1 "${PWD}/mc" stat minio/testbucket/testobject || exit -1
} }
function main() { function main() {
@@ -120,7 +131,7 @@ function main() {
export srv_idle_timeout=5 export srv_idle_timeout=5
export -f gen_put_request export -f gen_put_request
test_silo_with_timeout ${start_port} test_minio_with_timeout ${start_port}
} }
main "$@" main "$@"
+74
View File
@@ -0,0 +1,74 @@
# Settings and configurations that are common for all containers
x-minio-common: &minio-common
image: minio/minio:${MINIO_VERSION}
command: server http://minio{1...4}/data{1...3}
env_file:
- ./minio.env
expose:
- "9000"
- "9001"
# starts 4 docker containers running minio server instances.
# using nginx reverse proxy, load balancing, you can access
# it through port 9000.
services:
minio1:
<<: *minio-common
hostname: minio1
volumes:
- data1-1:/data1
- data1-2:/data2
- data1-3:/data3
minio2:
<<: *minio-common
hostname: minio2
volumes:
- data2-1:/data1
- data2-2:/data2
- data2-3:/data3
minio3:
<<: *minio-common
hostname: minio3
volumes:
- data3-1:/data1
- data3-2:/data2
- data3-3:/data3
minio4:
<<: *minio-common
hostname: minio4
volumes:
- data4-1:/data1
- data4-2:/data2
- data4-3:/data3
nginx:
image: nginx:1.19.2-alpine
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf:ro
ports:
- "9000:9000"
- "9001:9001"
depends_on:
- minio1
- minio2
- minio3
- minio4
## By default this config uses default local driver,
## For custom volumes replace with volume driver configuration.
volumes:
data1-1:
data1-2:
data1-3:
data2-1:
data2-2:
data2-3:
data3-1:
data3-2:
data3-3:
data4-1:
data4-2:
data4-3:
+3
View File
@@ -0,0 +1,3 @@
MINIO_ACCESS_KEY=minioadmin
MINIO_SECRET_KEY=minioadmin
MINIO_BROWSER=off
+68
View File
@@ -0,0 +1,68 @@
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
# include /etc/nginx/conf.d/*.conf;
upstream minio {
server minio1:9000;
server minio2:9000;
server minio3:9000;
server minio4:9000;
}
# main minio
server {
listen 9000;
listen [::]:9000;
server_name localhost;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 300;
# Default is HTTP/1, keepalive is only enabled in HTTP/1.1
proxy_http_version 1.1;
proxy_set_header Connection "";
chunked_transfer_encoding off;
proxy_pass http://minio;
}
}
}
+1 -1
View File
@@ -41,7 +41,7 @@ while IFS= read -r binary; do
grep -F 'vcs.' <<< "${info}" >&2 || true grep -F 'vcs.' <<< "${info}" >&2 || true
exit 1 exit 1
fi fi
done < <(find "${dist_dir}" -maxdepth 2 -type f \( -name 'silo' -o -name 'silo.exe' \) | sort) done < <(find "${dist_dir}" -maxdepth 2 -type f \( -name 'minio' -o -name 'minio.exe' \) | sort)
if [ "${count}" -ne "${expected_count}" ]; then if [ "${count}" -ne "${expected_count}" ]; then
echo "Expected ${expected_count} release binaries, found ${count}" >&2 echo "Expected ${expected_count} release binaries, found ${count}" >&2
+66 -56
View File
@@ -5,8 +5,8 @@ set -e
set -E set -E
set -o pipefail set -o pipefail
if [ ! -x "$PWD/silo" ]; then if [ ! -x "$PWD/minio" ]; then
echo "Silo executable binary not found in current directory" echo "minio executable binary not found in current directory"
exit 1 exit 1
fi fi
@@ -15,196 +15,196 @@ WORK_DIR="$PWD/.verify-$RANDOM"
export MINT_MODE=core export MINT_MODE=core
export MINT_DATA_DIR="$WORK_DIR/data" export MINT_DATA_DIR="$WORK_DIR/data"
export SERVER_ENDPOINT="127.0.0.1:9000" export SERVER_ENDPOINT="127.0.0.1:9000"
export MC_HOST_verify="http://silo:silo123@${SERVER_ENDPOINT}/" export MC_HOST_verify="http://minio:minio123@${SERVER_ENDPOINT}/"
export MC_HOST_verify_ipv6="http://silo:silo123@[::1]:9000/" export MC_HOST_verify_ipv6="http://minio:minio123@[::1]:9000/"
export ACCESS_KEY="silo" export ACCESS_KEY="minio"
export SECRET_KEY="silo123" export SECRET_KEY="minio123"
export ENABLE_HTTPS=0 export ENABLE_HTTPS=0
export GO111MODULE=on export GO111MODULE=on
export GOGC=25 export GOGC=25
export ENABLE_ADMIN=1 export ENABLE_ADMIN=1
export MINIO_CI_CD=1 export MINIO_CI_CD=1
SILO_CONFIG_DIR="$WORK_DIR/.silo" MINIO_CONFIG_DIR="$WORK_DIR/.minio"
SILO=("$PWD/silo" --config-dir "$SILO_CONFIG_DIR") MINIO=("$PWD/minio" --config-dir "$MINIO_CONFIG_DIR")
FILE_1_MB="$MINT_DATA_DIR/datafile-1-MB" FILE_1_MB="$MINT_DATA_DIR/datafile-1-MB"
FILE_65_MB="$MINT_DATA_DIR/datafile-65-MB" FILE_65_MB="$MINT_DATA_DIR/datafile-65-MB"
FUNCTIONAL_TESTS="$WORK_DIR/functional-tests.sh" FUNCTIONAL_TESTS="$WORK_DIR/functional-tests.sh"
function start_silo_fs() { function start_minio_fs() {
export MINIO_ROOT_USER=$ACCESS_KEY export MINIO_ROOT_USER=$ACCESS_KEY
export MINIO_ROOT_PASSWORD=$SECRET_KEY export MINIO_ROOT_PASSWORD=$SECRET_KEY
"${SILO[@]}" server "${WORK_DIR}/fs-disk" >"$WORK_DIR/fs-silo.log" 2>&1 & "${MINIO[@]}" server "${WORK_DIR}/fs-disk" >"$WORK_DIR/fs-minio.log" 2>&1 &
"${WORK_DIR}/mc" ready verify "${WORK_DIR}/mc" ready verify
} }
function start_silo_erasure() { function start_minio_erasure() {
"${SILO[@]}" server "${WORK_DIR}/erasure-disk1" "${WORK_DIR}/erasure-disk2" "${WORK_DIR}/erasure-disk3" "${WORK_DIR}/erasure-disk4" >"$WORK_DIR/erasure-silo.log" 2>&1 & "${MINIO[@]}" server "${WORK_DIR}/erasure-disk1" "${WORK_DIR}/erasure-disk2" "${WORK_DIR}/erasure-disk3" "${WORK_DIR}/erasure-disk4" >"$WORK_DIR/erasure-minio.log" 2>&1 &
"${WORK_DIR}/mc" ready verify "${WORK_DIR}/mc" ready verify
} }
function start_silo_erasure_sets() { function start_minio_erasure_sets() {
export MINIO_ENDPOINTS="${WORK_DIR}/erasure-disk-sets{1...32}" export MINIO_ENDPOINTS="${WORK_DIR}/erasure-disk-sets{1...32}"
"${SILO[@]}" server >"$WORK_DIR/erasure-silo-sets.log" 2>&1 & "${MINIO[@]}" server >"$WORK_DIR/erasure-minio-sets.log" 2>&1 &
"${WORK_DIR}/mc" ready verify "${WORK_DIR}/mc" ready verify
} }
function start_silo_pool_erasure_sets() { function start_minio_pool_erasure_sets() {
export MINIO_ROOT_USER=$ACCESS_KEY export MINIO_ROOT_USER=$ACCESS_KEY
export MINIO_ROOT_PASSWORD=$SECRET_KEY export MINIO_ROOT_PASSWORD=$SECRET_KEY
export MINIO_ENDPOINTS="http://127.0.0.1:9000${WORK_DIR}/pool-disk-sets{1...4} http://127.0.0.1:9001${WORK_DIR}/pool-disk-sets{5...8}" export MINIO_ENDPOINTS="http://127.0.0.1:9000${WORK_DIR}/pool-disk-sets{1...4} http://127.0.0.1:9001${WORK_DIR}/pool-disk-sets{5...8}"
"${SILO[@]}" server --address ":9000" >"$WORK_DIR/pool-silo-9000.log" 2>&1 & "${MINIO[@]}" server --address ":9000" >"$WORK_DIR/pool-minio-9000.log" 2>&1 &
"${SILO[@]}" server --address ":9001" >"$WORK_DIR/pool-silo-9001.log" 2>&1 & "${MINIO[@]}" server --address ":9001" >"$WORK_DIR/pool-minio-9001.log" 2>&1 &
"${WORK_DIR}/mc" ready verify "${WORK_DIR}/mc" ready verify
} }
function start_silo_pool_erasure_sets_ipv6() { function start_minio_pool_erasure_sets_ipv6() {
export MINIO_ROOT_USER=$ACCESS_KEY export MINIO_ROOT_USER=$ACCESS_KEY
export MINIO_ROOT_PASSWORD=$SECRET_KEY export MINIO_ROOT_PASSWORD=$SECRET_KEY
export MINIO_ENDPOINTS="http://[::1]:9000${WORK_DIR}/pool-disk-sets-ipv6{1...4} http://[::1]:9001${WORK_DIR}/pool-disk-sets-ipv6{5...8}" export MINIO_ENDPOINTS="http://[::1]:9000${WORK_DIR}/pool-disk-sets-ipv6{1...4} http://[::1]:9001${WORK_DIR}/pool-disk-sets-ipv6{5...8}"
"${SILO[@]}" server --address="[::1]:9000" >"$WORK_DIR/pool-silo-ipv6-9000.log" 2>&1 & "${MINIO[@]}" server --address="[::1]:9000" >"$WORK_DIR/pool-minio-ipv6-9000.log" 2>&1 &
"${SILO[@]}" server --address="[::1]:9001" >"$WORK_DIR/pool-silo-ipv6-9001.log" 2>&1 & "${MINIO[@]}" server --address="[::1]:9001" >"$WORK_DIR/pool-minio-ipv6-9001.log" 2>&1 &
"${WORK_DIR}/mc" ready verify_ipv6 "${WORK_DIR}/mc" ready verify_ipv6
} }
function start_silo_dist_erasure() { function start_minio_dist_erasure() {
export MINIO_ROOT_USER=$ACCESS_KEY export MINIO_ROOT_USER=$ACCESS_KEY
export MINIO_ROOT_PASSWORD=$SECRET_KEY export MINIO_ROOT_PASSWORD=$SECRET_KEY
export MINIO_ENDPOINTS="http://127.0.0.1:9000${WORK_DIR}/dist-disk1 http://127.0.0.1:9001${WORK_DIR}/dist-disk2 http://127.0.0.1:9002${WORK_DIR}/dist-disk3 http://127.0.0.1:9003${WORK_DIR}/dist-disk4" export MINIO_ENDPOINTS="http://127.0.0.1:9000${WORK_DIR}/dist-disk1 http://127.0.0.1:9001${WORK_DIR}/dist-disk2 http://127.0.0.1:9002${WORK_DIR}/dist-disk3 http://127.0.0.1:9003${WORK_DIR}/dist-disk4"
for i in $(seq 0 3); do for i in $(seq 0 3); do
"${SILO[@]}" server --address ":900${i}" >"$WORK_DIR/dist-silo-900${i}.log" 2>&1 & "${MINIO[@]}" server --address ":900${i}" >"$WORK_DIR/dist-minio-900${i}.log" 2>&1 &
done done
"${WORK_DIR}/mc" ready verify "${WORK_DIR}/mc" ready verify
} }
function run_test_fs() { function run_test_fs() {
start_silo_fs start_minio_fs
(cd "$WORK_DIR" && "$FUNCTIONAL_TESTS") (cd "$WORK_DIR" && "$FUNCTIONAL_TESTS")
rv=$? rv=$?
pkill silo pkill minio
sleep 3 sleep 3
if [ "$rv" -ne 0 ]; then if [ "$rv" -ne 0 ]; then
cat "$WORK_DIR/fs-silo.log" cat "$WORK_DIR/fs-minio.log"
fi fi
rm -f "$WORK_DIR/fs-silo.log" rm -f "$WORK_DIR/fs-minio.log"
return "$rv" return "$rv"
} }
function run_test_erasure_sets() { function run_test_erasure_sets() {
start_silo_erasure_sets start_minio_erasure_sets
(cd "$WORK_DIR" && "$FUNCTIONAL_TESTS") (cd "$WORK_DIR" && "$FUNCTIONAL_TESTS")
rv=$? rv=$?
pkill silo pkill minio
sleep 3 sleep 3
if [ "$rv" -ne 0 ]; then if [ "$rv" -ne 0 ]; then
cat "$WORK_DIR/erasure-silo-sets.log" cat "$WORK_DIR/erasure-minio-sets.log"
fi fi
rm -f "$WORK_DIR/erasure-silo-sets.log" rm -f "$WORK_DIR/erasure-minio-sets.log"
return "$rv" return "$rv"
} }
function run_test_pool_erasure_sets() { function run_test_pool_erasure_sets() {
start_silo_pool_erasure_sets start_minio_pool_erasure_sets
(cd "$WORK_DIR" && "$FUNCTIONAL_TESTS") (cd "$WORK_DIR" && "$FUNCTIONAL_TESTS")
rv=$? rv=$?
pkill silo pkill minio
sleep 3 sleep 3
if [ "$rv" -ne 0 ]; then if [ "$rv" -ne 0 ]; then
for i in $(seq 0 1); do for i in $(seq 0 1); do
echo "server$i log:" echo "server$i log:"
cat "$WORK_DIR/pool-silo-900$i.log" cat "$WORK_DIR/pool-minio-900$i.log"
done done
fi fi
for i in $(seq 0 1); do for i in $(seq 0 1); do
rm -f "$WORK_DIR/pool-silo-900$i.log" rm -f "$WORK_DIR/pool-minio-900$i.log"
done done
return "$rv" return "$rv"
} }
function run_test_pool_erasure_sets_ipv6() { function run_test_pool_erasure_sets_ipv6() {
start_silo_pool_erasure_sets_ipv6 start_minio_pool_erasure_sets_ipv6
export SERVER_ENDPOINT="[::1]:9000" export SERVER_ENDPOINT="[::1]:9000"
(cd "$WORK_DIR" && "$FUNCTIONAL_TESTS") (cd "$WORK_DIR" && "$FUNCTIONAL_TESTS")
rv=$? rv=$?
pkill silo pkill minio
sleep 3 sleep 3
if [ "$rv" -ne 0 ]; then if [ "$rv" -ne 0 ]; then
for i in $(seq 0 1); do for i in $(seq 0 1); do
echo "server$i log:" echo "server$i log:"
cat "$WORK_DIR/pool-silo-ipv6-900$i.log" cat "$WORK_DIR/pool-minio-ipv6-900$i.log"
done done
fi fi
for i in $(seq 0 1); do for i in $(seq 0 1); do
rm -f "$WORK_DIR/pool-silo-ipv6-900$i.log" rm -f "$WORK_DIR/pool-minio-ipv6-900$i.log"
done done
return "$rv" return "$rv"
} }
function run_test_erasure() { function run_test_erasure() {
start_silo_erasure start_minio_erasure
(cd "$WORK_DIR" && "$FUNCTIONAL_TESTS") (cd "$WORK_DIR" && "$FUNCTIONAL_TESTS")
rv=$? rv=$?
pkill silo pkill minio
sleep 3 sleep 3
if [ "$rv" -ne 0 ]; then if [ "$rv" -ne 0 ]; then
cat "$WORK_DIR/erasure-silo.log" cat "$WORK_DIR/erasure-minio.log"
fi fi
rm -f "$WORK_DIR/erasure-silo.log" rm -f "$WORK_DIR/erasure-minio.log"
return "$rv" return "$rv"
} }
function run_test_dist_erasure() { function run_test_dist_erasure() {
start_silo_dist_erasure start_minio_dist_erasure
(cd "$WORK_DIR" && "$FUNCTIONAL_TESTS") (cd "$WORK_DIR" && "$FUNCTIONAL_TESTS")
rv=$? rv=$?
pkill silo pkill minio
sleep 3 sleep 3
if [ "$rv" -ne 0 ]; then if [ "$rv" -ne 0 ]; then
echo "server1 log:" echo "server1 log:"
cat "$WORK_DIR/dist-silo-9000.log" cat "$WORK_DIR/dist-minio-9000.log"
echo "server2 log:" echo "server2 log:"
cat "$WORK_DIR/dist-silo-9001.log" cat "$WORK_DIR/dist-minio-9001.log"
echo "server3 log:" echo "server3 log:"
cat "$WORK_DIR/dist-silo-9002.log" cat "$WORK_DIR/dist-minio-9002.log"
echo "server4 log:" echo "server4 log:"
cat "$WORK_DIR/dist-silo-9003.log" cat "$WORK_DIR/dist-minio-9003.log"
fi fi
rm -f "$WORK_DIR/dist-silo-9000.log" "$WORK_DIR/dist-silo-9001.log" "$WORK_DIR/dist-silo-9002.log" "$WORK_DIR/dist-silo-9003.log" rm -f "$WORK_DIR/dist-minio-9000.log" "$WORK_DIR/dist-minio-9001.log" "$WORK_DIR/dist-minio-9002.log" "$WORK_DIR/dist-minio-9003.log"
return "$rv" return "$rv"
} }
@@ -216,21 +216,31 @@ function purge() {
function __init__() { function __init__() {
echo "Initializing environment" echo "Initializing environment"
mkdir -p "$WORK_DIR" mkdir -p "$WORK_DIR"
mkdir -p "$SILO_CONFIG_DIR" mkdir -p "$MINIO_CONFIG_DIR"
mkdir -p "$MINT_DATA_DIR" mkdir -p "$MINT_DATA_DIR"
"$(git rev-parse --show-toplevel)/buildscripts/install-mcli.sh" "${WORK_DIR}/mc" MC_BUILD_DIR="mc-$RANDOM"
if ! git clone --quiet https://github.com/minio/mc "$MC_BUILD_DIR"; then
echo "failed to download https://github.com/minio/mc"
purge "${MC_BUILD_DIR}"
exit 1
fi
(cd "${MC_BUILD_DIR}" && go build -o "${WORK_DIR}/mc")
# remove mc source.
purge "${MC_BUILD_DIR}"
shred -n 1 -s 1M - 1>"$FILE_1_MB" 2>/dev/null shred -n 1 -s 1M - 1>"$FILE_1_MB" 2>/dev/null
shred -n 1 -s 65M - 1>"$FILE_65_MB" 2>/dev/null shred -n 1 -s 65M - 1>"$FILE_65_MB" 2>/dev/null
## version is purposefully set to '3' for minio to migrate configuration file ## version is purposefully set to '3' for minio to migrate configuration file
echo '{"version": "3", "credential": {"accessKey": "silo", "secretKey": "silo123"}, "region": "us-east-1"}' >"$SILO_CONFIG_DIR/config.json" echo '{"version": "3", "credential": {"accessKey": "minio", "secretKey": "minio123"}, "region": "us-east-1"}' >"$MINIO_CONFIG_DIR/config.json"
"$(git rev-parse --show-toplevel)/buildscripts/install-verified-fixture.sh" \ if ! wget -q -O "$FUNCTIONAL_TESTS" https://raw.githubusercontent.com/minio/mc/master/functional-tests.sh; then
https://raw.githubusercontent.com/pgsty/mc/4c4dcc4b55baf238cd0c81030d77945b3828f157/functional-tests.sh \ echo "failed to download https://raw.githubusercontent.com/minio/mc/master/functional-tests.sh"
9b98c8152b294d567b9bc732869226dd4f65d0f4d84092dc066a900a66a9e22c \ exit 1
"$FUNCTIONAL_TESTS" fi
sed -i 's|-sS|-sSg|g' "$FUNCTIONAL_TESTS" sed -i 's|-sS|-sSg|g' "$FUNCTIONAL_TESTS"
chmod a+x "$FUNCTIONAL_TESTS" chmod a+x "$FUNCTIONAL_TESTS"
@@ -4,18 +4,18 @@
set -E set -E
set -o pipefail set -o pipefail
if [ ! -x "$PWD/silo" ]; then if [ ! -x "$PWD/minio" ]; then
echo "Silo executable binary not found in current directory" echo "minio executable binary not found in current directory"
exit 1 exit 1
fi fi
WORK_DIR="$PWD/.verify-$RANDOM" WORK_DIR="$PWD/.verify-$RANDOM"
SILO_CONFIG_DIR="$WORK_DIR/.silo" MINIO_CONFIG_DIR="$WORK_DIR/.minio"
SILO=("$PWD/silo" --config-dir "$SILO_CONFIG_DIR" server) MINIO=("$PWD/minio" --config-dir "$MINIO_CONFIG_DIR" server)
function start_silo_3_node() { function start_minio_3_node() {
export MINIO_ROOT_USER=silo export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=silo123 export MINIO_ROOT_PASSWORD=minio123
export MINIO_ERASURE_SET_DRIVE_COUNT=6 export MINIO_ERASURE_SET_DRIVE_COUNT=6
export MINIO_CI_CD=1 export MINIO_CI_CD=1
@@ -25,26 +25,26 @@ function start_silo_3_node() {
args="$args http://127.0.0.1:$((start_port + i))${WORK_DIR}/$i/1/ http://127.0.0.1:$((start_port + i))${WORK_DIR}/$i/2/ http://127.0.0.1:$((start_port + i))${WORK_DIR}/$i/3/ http://127.0.0.1:$((start_port + i))${WORK_DIR}/$i/4/ http://127.0.0.1:$((start_port + i))${WORK_DIR}/$i/5/ http://127.0.0.1:$((start_port + i))${WORK_DIR}/$i/6/" args="$args http://127.0.0.1:$((start_port + i))${WORK_DIR}/$i/1/ http://127.0.0.1:$((start_port + i))${WORK_DIR}/$i/2/ http://127.0.0.1:$((start_port + i))${WORK_DIR}/$i/3/ http://127.0.0.1:$((start_port + i))${WORK_DIR}/$i/4/ http://127.0.0.1:$((start_port + i))${WORK_DIR}/$i/5/ http://127.0.0.1:$((start_port + i))${WORK_DIR}/$i/6/"
done done
"${SILO[@]}" --address ":$((start_port + 1))" $args >"${WORK_DIR}/dist-silo-server1.log" 2>&1 & "${MINIO[@]}" --address ":$((start_port + 1))" $args >"${WORK_DIR}/dist-minio-server1.log" 2>&1 &
pid1=$! pid1=$!
disown ${pid1} disown ${pid1}
"${SILO[@]}" --address ":$((start_port + 2))" $args >"${WORK_DIR}/dist-silo-server2.log" 2>&1 & "${MINIO[@]}" --address ":$((start_port + 2))" $args >"${WORK_DIR}/dist-minio-server2.log" 2>&1 &
pid2=$! pid2=$!
disown $pid2 disown $pid2
"${SILO[@]}" --address ":$((start_port + 3))" $args >"${WORK_DIR}/dist-silo-server3.log" 2>&1 & "${MINIO[@]}" --address ":$((start_port + 3))" $args >"${WORK_DIR}/dist-minio-server3.log" 2>&1 &
pid3=$! pid3=$!
disown $pid3 disown $pid3
export MC_HOST_mysilo="http://silo:silo123@127.0.0.1:$((start_port + 1))" export MC_HOST_myminio="http://minio:minio123@127.0.0.1:$((start_port + 1))"
timeout 15m /tmp/mc ready mysilo || fail timeout 15m /tmp/mc ready myminio || fail
# Wait for all drives to be online and formatted # Wait for all drives to be online and formatted
while [ $(/tmp/mc admin info --json mysilo | jq '.info.servers[].drives[].state | select(. != "ok")' | wc -l) -gt 0 ]; do sleep 1; done while [ $(/tmp/mc admin info --json myminio | jq '.info.servers[].drives[].state | select(. != "ok")' | wc -l) -gt 0 ]; do sleep 1; done
# Wait for all drives to be healed # Wait for all drives to be healed
while [ $(/tmp/mc admin info --json mysilo | jq '.info.servers[].drives[].healing | select(. != null) | select(. == true)' | wc -l) -gt 0 ]; do sleep 1; done while [ $(/tmp/mc admin info --json myminio | jq '.info.servers[].drives[].healing | select(. != null) | select(. == true)' | wc -l) -gt 0 ]; do sleep 1; done
# Wait for Status: in MinIO output # Wait for Status: in MinIO output
while true; do while true; do
@@ -66,26 +66,26 @@ function start_silo_3_node() {
done done
if ! ps -p $pid1 1>&2 >/dev/null; then if ! ps -p $pid1 1>&2 >/dev/null; then
echo "silo-server-1 is not running." && fail echo "minio-server-1 is not running." && fail
fi fi
if ! ps -p $pid2 1>&2 >/dev/null; then if ! ps -p $pid2 1>&2 >/dev/null; then
echo "silo-server-2 is not running." && fail echo "minio-server-2 is not running." && fail
fi fi
if ! ps -p $pid3 1>&2 >/dev/null; then if ! ps -p $pid3 1>&2 >/dev/null; then
echo "silo-server-3 is not running." && fail echo "minio-server-3 is not running." && fail
fi fi
if ! pkill silo; then if ! pkill minio; then
fail fail
fi fi
sleep 1 sleep 1
if pgrep silo; then if pgrep minio; then
# forcibly killing, to proceed further properly. # forcibly killing, to proceed further properly.
if ! pkill -9 silo; then if ! pkill -9 minio; then
echo "no Silo process running anymore, proceed." echo "no minio process running anymore, proceed."
fi fi
fi fi
} }
@@ -93,7 +93,7 @@ function start_silo_3_node() {
function fail() { function fail() {
for i in $(seq 1 3); do for i in $(seq 1 3); do
echo "server$i log:" echo "server$i log:"
cat "${WORK_DIR}/dist-silo-server$i.log" cat "${WORK_DIR}/dist-minio-server$i.log"
done done
echo "FAILED" echo "FAILED"
purge "$WORK_DIR" purge "$WORK_DIR"
@@ -101,7 +101,7 @@ function fail() {
} }
function check_online() { function check_online() {
if ! grep -q 'API:' ${WORK_DIR}/dist-silo-*.log; then if ! grep -q 'API:' ${WORK_DIR}/dist-minio-*.log; then
echo "1" echo "1"
fi fi
} }
@@ -113,18 +113,19 @@ function purge() {
function __init__() { function __init__() {
echo "Initializing environment" echo "Initializing environment"
mkdir -p "$WORK_DIR" mkdir -p "$WORK_DIR"
mkdir -p "$SILO_CONFIG_DIR" mkdir -p "$MINIO_CONFIG_DIR"
## version is purposefully set to '3' for minio to migrate configuration file ## version is purposefully set to '3' for minio to migrate configuration file
echo '{"version": "3", "credential": {"accessKey": "silo", "secretKey": "silo123"}, "region": "us-east-1"}' >"$SILO_CONFIG_DIR/config.json" echo '{"version": "3", "credential": {"accessKey": "minio", "secretKey": "minio123"}, "region": "us-east-1"}' >"$MINIO_CONFIG_DIR/config.json"
if [ ! -f /tmp/mc ]; then if [ ! -f /tmp/mc ]; then
"$(git rev-parse --show-toplevel)/buildscripts/install-mcli.sh" /tmp/mc wget --quiet -O /tmp/mc https://dl.minio.io/client/mc/release/linux-amd64/mc &&
chmod +x /tmp/mc
fi fi
} }
function perform_test() { function perform_test() {
start_silo_3_node $2 start_minio_3_node $2
echo "Testing Distributed Erasure setup healing of drives" echo "Testing Distributed Erasure setup healing of drives"
echo "Remove the contents of the disks belonging to '${1}' erasure set" echo "Remove the contents of the disks belonging to '${1}' erasure set"
@@ -132,7 +133,7 @@ function perform_test() {
rm -rf ${WORK_DIR}/${1}/*/ rm -rf ${WORK_DIR}/${1}/*/
set -x set -x
start_silo_3_node $2 start_minio_3_node $2
} }
function main() { function main() {
+10 -10
View File
@@ -4,20 +4,20 @@ set -E
set -o pipefail set -o pipefail
set -x set -x
if [ ! -x "$PWD/silo" ]; then if [ ! -x "$PWD/minio" ]; then
echo "Silo executable binary not found in current directory" echo "minio executable binary not found in current directory"
exit 1 exit 1
fi fi
WORK_DIR="$(mktemp -d)" WORK_DIR="$(mktemp -d)"
SILO_CONFIG_DIR="$WORK_DIR/.silo" MINIO_CONFIG_DIR="$WORK_DIR/.minio"
SILO=("$PWD/silo" --config-dir "$SILO_CONFIG_DIR" server) MINIO=("$PWD/minio" --config-dir "$MINIO_CONFIG_DIR" server)
function start_silo() { function start_minio() {
start_port=$1 start_port=$1
export MINIO_ROOT_USER=silo export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=silo123 export MINIO_ROOT_PASSWORD=minio123
unset MINIO_KMS_AUTO_ENCRYPTION # do not auto-encrypt objects unset MINIO_KMS_AUTO_ENCRYPTION # do not auto-encrypt objects
unset MINIO_CI_CD unset MINIO_CI_CD
unset CI unset CI
@@ -28,7 +28,7 @@ function start_silo() {
done done
for i in $(seq 1 4); do for i in $(seq 1 4); do
"${SILO[@]}" --address ":$((start_port + i))" ${args[@]} 2>&1 >"${WORK_DIR}/server$i.log" & "${MINIO[@]}" --address ":$((start_port + i))" ${args[@]} 2>&1 >"${WORK_DIR}/server$i.log" &
done done
# Wait until all nodes return 403 # Wait until all nodes return 403
@@ -60,7 +60,7 @@ function prepare_block_devices() {
# Start a distributed MinIO setup, unmount one disk and check if it is formatted # Start a distributed MinIO setup, unmount one disk and check if it is formatted
function main() { function main() {
start_port=$(shuf -i 10000-65000 -n 1) start_port=$(shuf -i 10000-65000 -n 1)
start_silo ${start_port} start_minio ${start_port}
# Unmount the disk, after the unmount the device id # Unmount the disk, after the unmount the device id
# /tmp/xxx/mnt/disk4 will be the same as '/' and it # /tmp/xxx/mnt/disk4 will be the same as '/' and it
@@ -82,7 +82,7 @@ function main() {
} }
function cleanup() { function cleanup() {
pkill silo pkill minio
sudo umount ${WORK_DIR}/mnt/disk{1..3}/ sudo umount ${WORK_DIR}/mnt/disk{1..3}/
sudo rm /dev/minio-loopdisk* sudo rm /dev/minio-loopdisk*
rm -rf "$WORK_DIR" rm -rf "$WORK_DIR"
+31 -30
View File
@@ -4,23 +4,23 @@
set -E set -E
set -o pipefail set -o pipefail
if [ ! -x "$PWD/silo" ]; then if [ ! -x "$PWD/minio" ]; then
echo "Silo executable binary not found in current directory" echo "minio executable binary not found in current directory"
exit 1 exit 1
fi fi
WORK_DIR="$PWD/.verify-$RANDOM" WORK_DIR="$PWD/.verify-$RANDOM"
SILO_CONFIG_DIR="$WORK_DIR/.silo" MINIO_CONFIG_DIR="$WORK_DIR/.minio"
SILO=("$PWD/silo" --config-dir "$SILO_CONFIG_DIR" server) MINIO=("$PWD/minio" --config-dir "$MINIO_CONFIG_DIR" server)
GOPATH=/tmp/gopath GOPATH=/tmp/gopath
function start_silo_3_node() { function start_minio_3_node() {
for i in $(seq 1 3); do for i in $(seq 1 3); do
rm "${WORK_DIR}/dist-silo-server$i.log" rm "${WORK_DIR}/dist-minio-server$i.log"
done done
export MINIO_ROOT_USER=silo export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=silo123 export MINIO_ROOT_PASSWORD=minio123
export MINIO_ERASURE_SET_DRIVE_COUNT=6 export MINIO_ERASURE_SET_DRIVE_COUNT=6
export MINIO_CI_CD=1 export MINIO_CI_CD=1
@@ -34,51 +34,51 @@ function start_silo_3_node() {
args="$args http://127.0.0.1:$((start_port + 1))${WORK_DIR}/1/${d}/ http://127.0.0.1:$((start_port + 2))${WORK_DIR}/2/${d}/ http://127.0.0.1:$((start_port + 3))${WORK_DIR}/3/${d}/ " args="$args http://127.0.0.1:$((start_port + 1))${WORK_DIR}/1/${d}/ http://127.0.0.1:$((start_port + 2))${WORK_DIR}/2/${d}/ http://127.0.0.1:$((start_port + 3))${WORK_DIR}/3/${d}/ "
done done
"${SILO[@]}" --address ":$((start_port + 1))" $args >"${WORK_DIR}/dist-silo-server1.log" 2>&1 & "${MINIO[@]}" --address ":$((start_port + 1))" $args >"${WORK_DIR}/dist-minio-server1.log" 2>&1 &
pid1=$! pid1=$!
disown ${pid1} disown ${pid1}
"${SILO[@]}" --address ":$((start_port + 2))" $args >"${WORK_DIR}/dist-silo-server2.log" 2>&1 & "${MINIO[@]}" --address ":$((start_port + 2))" $args >"${WORK_DIR}/dist-minio-server2.log" 2>&1 &
pid2=$! pid2=$!
disown $pid2 disown $pid2
"${SILO[@]}" --address ":$((start_port + 3))" $args >"${WORK_DIR}/dist-silo-server3.log" 2>&1 & "${MINIO[@]}" --address ":$((start_port + 3))" $args >"${WORK_DIR}/dist-minio-server3.log" 2>&1 &
pid3=$! pid3=$!
disown $pid3 disown $pid3
export MC_HOST_mysilo="http://silo:silo123@127.0.0.1:$((start_port + 1))" export MC_HOST_myminio="http://minio:minio123@127.0.0.1:$((start_port + 1))"
timeout 15m /tmp/mc ready mysilo || fail timeout 15m /tmp/mc ready myminio || fail
[ ${first_time} -eq 0 ] && upload_objects [ ${first_time} -eq 0 ] && upload_objects
[ ${first_time} -ne 0 ] && sleep 120 [ ${first_time} -ne 0 ] && sleep 120
if ! ps -p $pid1 1>&2 >/dev/null; then if ! ps -p $pid1 1>&2 >/dev/null; then
echo "silo server 1 is not running" && fail echo "minio server 1 is not running" && fail
fi fi
if ! ps -p $pid2 1>&2 >/dev/null; then if ! ps -p $pid2 1>&2 >/dev/null; then
echo "silo server 2 is not running" && fail echo "minio server 2 is not running" && fail
fi fi
if ! ps -p $pid3 1>&2 >/dev/null; then if ! ps -p $pid3 1>&2 >/dev/null; then
echo "silo server 3 is not running" && fail echo "minio server 3 is not running" && fail
fi fi
if ! pkill silo; then if ! pkill minio; then
fail fail
fi fi
sleep 1 sleep 1
if pgrep silo; then if pgrep minio; then
# forcibly killing, to proceed further properly. # forcibly killing, to proceed further properly.
if ! pkill -9 silo; then if ! pkill -9 minio; then
echo "no Silo process running anymore, proceed." echo "no minio process running anymore, proceed."
fi fi
fi fi
} }
function check_heal() { function check_heal() {
if ! grep -q 'API:' ${WORK_DIR}/dist-silo-*.log; then if ! grep -q 'API:' ${WORK_DIR}/dist-minio-*.log; then
return 1 return 1
fi fi
@@ -103,9 +103,9 @@ function purge() {
function fail() { function fail() {
for i in $(seq 1 3); do for i in $(seq 1 3); do
echo "server$i log:" echo "server$i log:"
cat "${WORK_DIR}/dist-silo-server$i.log" cat "${WORK_DIR}/dist-minio-server$i.log"
done done
pkill -9 silo pkill -9 minio
echo "FAILED" echo "FAILED"
purge "$WORK_DIR" purge "$WORK_DIR"
exit 1 exit 1
@@ -114,27 +114,28 @@ function fail() {
function __init__() { function __init__() {
echo "Initializing environment" echo "Initializing environment"
mkdir -p "$WORK_DIR" mkdir -p "$WORK_DIR"
mkdir -p "$SILO_CONFIG_DIR" mkdir -p "$MINIO_CONFIG_DIR"
## version is purposefully set to '3' for minio to migrate configuration file ## version is purposefully set to '3' for minio to migrate configuration file
echo '{"version": "3", "credential": {"accessKey": "silo", "secretKey": "silo123"}, "region": "us-east-1"}' >"$SILO_CONFIG_DIR/config.json" echo '{"version": "3", "credential": {"accessKey": "minio", "secretKey": "minio123"}, "region": "us-east-1"}' >"$MINIO_CONFIG_DIR/config.json"
if [ ! -f /tmp/mc ]; then if [ ! -f /tmp/mc ]; then
"$(git rev-parse --show-toplevel)/buildscripts/install-mcli.sh" /tmp/mc wget --quiet -O /tmp/mc https://dl.minio.io/client/mc/release/linux-amd64/mc &&
chmod +x /tmp/mc
fi fi
} }
function upload_objects() { function upload_objects() {
/tmp/mc mb mysilo/testbucket/ /tmp/mc mb myminio/testbucket/
for ((i = 0; i < 20; i++)); do for ((i = 0; i < 20; i++)); do
echo "my content" | /tmp/mc pipe mysilo/testbucket/file-$i echo "my content" | /tmp/mc pipe myminio/testbucket/file-$i
done done
} }
function perform_test() { function perform_test() {
start_port=$2 start_port=$2
start_silo_3_node $start_port start_minio_3_node $start_port
echo "Testing Distributed Erasure setup healing of drives" echo "Testing Distributed Erasure setup healing of drives"
echo "Remove the contents of the disks belonging to '${1}' node" echo "Remove the contents of the disks belonging to '${1}' node"
@@ -142,7 +143,7 @@ function perform_test() {
rm -rf ${WORK_DIR}/${1}/*/ rm -rf ${WORK_DIR}/${1}/*/
set -x set -x
start_silo_3_node $start_port start_minio_3_node $start_port
check_heal ${1} check_heal ${1}
rv=$? rv=$?
-122
View File
@@ -1,122 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
repo_dir="$(cd "${script_dir}/.." && pwd)"
baseline_commit="${HELM_LEGACY_COMMIT:-d88f46cce}"
helm_image="${HELM_IMAGE:-alpine/helm:3.18.6@sha256:c6d8088ddb279625a2e1ca3b08b22c18c946d1f65c8b810f28f1597435a1134c}"
work_dir="$(mktemp -d "${TMPDIR:-/tmp}/silo-helm.XXXXXX")"
cleanup() {
rm -rf "${work_dir}"
}
trap cleanup EXIT
cd "${repo_dir}"
git cat-file -e "${baseline_commit}^{commit}"
git archive "${baseline_commit}" helm/minio | tar -x -C "${work_dir}"
if command -v helm >/dev/null 2>&1; then
new_chart="${repo_dir}/helm/silo"
old_chart="${work_dir}/helm/minio"
output_dir="${work_dir}"
helm_run() {
helm "$@"
}
else
command -v docker >/dev/null 2>&1 || {
echo "helm or docker is required" >&2
exit 1
}
new_chart=/repo/helm/silo
old_chart=/check/helm/minio
output_dir=/check
helm_run() {
docker run --rm \
-v "${repo_dir}:/repo:ro" \
-v "${work_dir}:/check" \
"${helm_image}" "$@"
}
fi
helm_run lint "${new_chart}"
helm_run template silo "${new_chart}" \
--namespace silo \
--set rootUser=silo-admin \
--set rootPassword=test-password-123456 >/dev/null
helm_run template silo "${new_chart}" \
--namespace silo \
--set mode=standalone \
--set replicas=1 \
--set persistence.enabled=false \
--set rootUser=silo-admin \
--set rootPassword=test-password-123456 >/dev/null
# Exercise optional templates that the default render leaves dormant.
helm_run template silo-all "${new_chart}" \
--namespace silo \
--set rootUser=silo-admin \
--set rootPassword=test-password-123456 \
--set tls.enabled=true \
--set tls.certSecret=silo-tls \
--set trustedCertsSecret=silo-trusted-ca \
--set ingress.enabled=true \
--set consoleIngress.enabled=true \
--set networkPolicy.enabled=true \
--set podDisruptionBudget.enabled=true \
--set metrics.serviceMonitor.enabled=true \
--set metrics.serviceMonitor.includeNode=true \
--set 'buckets[0].name=chart-test' \
--set 'buckets[0].policy=none' \
--set 'buckets[0].purge=false' >/dev/null
# Existing values commonly address the historical myminio target. Render the
# custom-command path explicitly so both the new and compatibility aliases are
# protected by the release gate rather than only by a source-text assertion.
custom_render="${work_dir}/custom-command.yaml"
helm_run template silo-custom "${new_chart}" \
--namespace silo \
--set rootUser=silo-admin \
--set rootPassword=test-password-123456 \
--set-string 'customCommands[0].command=admin info myminio' \
--show-only templates/configmap.yaml >"${custom_render}"
for expected in \
'alias set mysilo' \
'alias set myminio' \
'runCommand admin info myminio'; do
grep -F -- "${expected}" "${custom_render}" >/dev/null || {
echo "rendered custom command is missing: ${expected}" >&2
exit 1
}
done
old_render="${work_dir}/legacy.yaml"
new_render="${work_dir}/candidate.yaml"
helm_run template my-release "${old_chart}" \
--namespace my-namespace \
--set rootUser=legacy-admin \
--set rootPassword=legacy-password-123456 >"${old_render}"
helm_run template my-release "${new_chart}" \
--namespace my-namespace \
-f "${old_chart}/values.yaml" \
--set rootUser=legacy-admin \
--set rootPassword=legacy-password-123456 \
--set nameOverride=minio \
--set fullnameOverride=my-release-minio \
--set serviceAccount.name=minio-sa \
--set image.repository=pgsty/silo \
--set mcImage.repository=pgsty/silo \
--set-string image.tag=RELEASE.2026-08-04T00-00-00Z \
--set-string mcImage.tag=RELEASE.2026-08-04T00-00-00Z >"${new_render}"
go run ./buildscripts/helm-migration-guard "${old_render}" "${new_render}"
helm_run package "${new_chart}" --destination "${output_dir}" >/dev/null
test -s "${work_dir}/silo-7.0.1.tgz"
if find "${work_dir}" -maxdepth 1 -type f -name 'minio-*.tgz' | grep -q .; then
echo "Helm packaging emitted a legacy MinIO chart name" >&2
exit 1
fi
echo "Silo Helm lint, render, legacy-upgrade, and package checks passed"
-218
View File
@@ -1,218 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
repo_dir="$(cd "${script_dir}/.." && pwd)"
cd "${repo_dir}"
fail() {
echo "Silo rebrand verification failed: $*" >&2
exit 1
}
require_file() {
[ -f "$1" ] || fail "missing required file: $1"
}
require_text() {
local file="$1"
local text="$2"
grep -Fq -- "${text}" "${file}" || fail "${file} does not contain: ${text}"
}
reject_text() {
local file="$1"
local text="$2"
if grep -Fq -- "${text}" "${file}"; then
fail "${file} still contains forbidden delivery text: ${text}"
fi
}
for file in \
.github/goreleaser.yml \
.github/nfpm.yml \
buildscripts/package/lifecycle_test.sh \
buildscripts/verify-helm-migration.sh \
Dockerfile.goreleaser \
Dockerfile.distroless \
dockerscripts/download-static-curl.sh \
dockerscripts/docker-entrypoint.sh \
helm/silo/Chart.yaml \
helm/silo/values.yaml \
silo.service \
silo.env \
silo.sysusers; do
require_file "${file}"
done
for retired_path in \
CNAME _config.yml index.yaml helm-reindex.sh helm/minio helm-releases \
Dockerfile Dockerfile.cicd Dockerfile.hotfix Dockerfile.release \
Dockerfile.release.old_cpu Dockerfile.scratch docker-buildx.sh \
minio.service cmd/callhome.go buildscripts/upgrade-tests .github/logo.svg \
docs/federation/lookup/bucket-lookup.png \
docs/screenshots/Minio_Cloud_Native_Arch.jpg \
docs/screenshots/Minio_Cloud_Native_Arch.png \
docs/screenshots/Minio_Cloud_Native_Arch.svg \
docs/screenshots/Architecture-diagram_distributed_8.jpg \
docs/screenshots/Architecture-diagram_distributed_8.png \
docs/screenshots/Architecture-diagram_distributed_8.svg \
docs/screenshots/Architecture-diagram_distributed_16.jpg \
docs/screenshots/Architecture-diagram_distributed_16.png \
docs/screenshots/Architecture-diagram_distributed_16.svg \
docs/screenshots/Architecture-diagram_distributed_nm.png \
docs/screenshots/Example-1.jpg docs/screenshots/Example-1.png \
docs/screenshots/Example-2.jpg docs/screenshots/Example-2.png \
docs/screenshots/Example-3.jpg docs/screenshots/Example-3.png \
docs/screenshots/pic1.png docs/screenshots/pic2.png \
docs/metrics/prometheus/grafana/grafana-minio.png \
docs/metrics/prometheus/grafana/bucket/grafana-bucket.png \
docs/metrics/prometheus/grafana/node/grafana-node.png \
docs/metrics/prometheus/grafana/replication/grafana-replication-cluster.png \
docs/metrics/prometheus/grafana/replication/grafana-replication-node.png; do
[ ! -e "${retired_path}" ] || fail "retired upstream delivery path remains: ${retired_path}"
done
require_text .github/goreleaser.yml "binary: silo"
require_text .github/goreleaser.yml 'name_template: "silo_{{ .Env.PKG_VERSION }}_{{ .Os }}_{{ .Arch }}"'
require_text .github/goreleaser.yml "sboms:"
require_text .github/goreleaser.yml "artifacts: archive"
require_text .github/goreleaser.yml "cmd: cosign"
# shellcheck disable=SC2016 # Match the literal GoReleaser template variable.
require_text .github/goreleaser.yml 'signature: "${artifact}.sigstore.json"'
require_text .github/nfpm.yml "name: silo"
require_text .github/nfpm.yml "dst: /usr/bin/silo"
require_text .github/nfpm.yml "dst: /etc/default/silo"
require_text .github/nfpm.yml "dst: /usr/lib/sysusers.d/silo.conf"
require_text buildscripts/package/lifecycle_test.sh "Silo package lifecycle checks passed"
require_text buildscripts/verify-helm-migration.sh "Silo Helm lint, render, legacy-upgrade, and package checks passed"
require_text silo.service "Conflicts=minio.service"
require_text silo.service "EnvironmentFile=-/etc/default/minio"
require_text silo.service "EnvironmentFile=-/etc/default/silo"
# shellcheck disable=SC2016 # Match the literal service environment variables.
require_text silo.service 'ExecStart=/usr/bin/silo server $MINIO_OPTS $MINIO_VOLUMES'
require_text README.md "/etc/systemd/system/silo.service.d/10-legacy-user.conf"
require_text README_ZH.md "/etc/systemd/system/silo.service.d/10-legacy-user.conf"
require_text Dockerfile.goreleaser "COPY silo /usr/bin/silo"
require_text Dockerfile.goreleaser 'CMD ["silo"]'
require_text Dockerfile.goreleaser "MC_AMD64_SHA256="
require_text Dockerfile.goreleaser "Published checksum drift"
require_text Dockerfile.distroless 'COPY --chmod=0755 silo /usr/bin/silo'
require_text Dockerfile.distroless 'ENTRYPOINT ["/usr/bin/silo"]'
require_text Dockerfile.distroless '"/usr/bin/silo", "healthcheck", "ready"'
require_text dockerscripts/download-static-curl.sh "sha256sum -c"
require_text helm/silo/Chart.yaml "name: silo"
require_text helm/silo/values.yaml "repository: pgsty/silo"
require_text helm/silo/templates/deployment.yaml "/usr/bin/docker-entrypoint.sh silo server"
require_text helm/silo/templates/statefulset.yaml "/usr/bin/docker-entrypoint.sh silo server"
require_text docs/orchestration/docker-compose/docker-compose.yaml 'http://silo{1...4}/data{1...2}'
require_text docs/resiliency/docker-compose.yaml 'http://silo{1...4}/data{1...8}'
require_text docs/distributed/DECOMMISSION.md 'systemctl restart silo'
# shellcheck disable=SC2016 # Match the literal shell variable.
require_text docs/resiliency/resiliency-tests.sh 'docker exec resiliency-silo$NODE-1'
require_text .github/workflows/release.yml "Attest downloadable release artifacts"
require_text .github/workflows/release.yml "packages_checksums.txt"
require_text .github/workflows/docker-release.yml "Attest multi-architecture image provenance"
require_text .github/workflows/docker-release.yml "index.docker.io/pgsty/silo"
# Copyright notices credit both parties with fixed terms: upstream MinIO
# development ends at its own last year, and the fork's own term starts when
# the fork did. Deriving the upstream end year from the clock would extend
# MinIO's copyright term every January.
require_text cmd/build-constants.go 'upstreamCopyrightEndYear = "2025"'
require_text cmd/build-constants.go 'forkCopyrightStartYear = "2025"'
require_text cmd/main.go 'upstreamCopyrightEndYear'
reject_text cmd/main.go 'CopyrightYear = strconv.Itoa(time.Now().Year())'
require_text NOTICE 'MinIO Project, (C) 2015-2025 MinIO, Inc.'
require_text NOTICE 'Silo Project modifications, (C) 2025-2026 PGSTY.'
# Contribution policy: no CLA, inbound=outbound, DCO sign-off enforced in CI.
require_file .github/workflows/dco.yml
require_text .github/workflows/dco.yml "Signed-off-by"
require_text CONTRIBUTING.md "developercertificate.org"
require_text CONTRIBUTING.md "No CLA"
for file in .github/nfpm.yml Dockerfile.goreleaser silo.service; do
reject_text "${file}" "/usr/bin/minio"
reject_text "${file}" "/usr/local/bin/minio"
done
reject_text Dockerfile.goreleaser "MINIO_UPDATE_MINISIGN_PUBKEY"
reject_text buildscripts/minio-upgrade.sh "docker system prune"
reject_text buildscripts/minio-upgrade.sh "docker volume prune"
reject_text docs/orchestration/docker-compose/docker-compose.yaml 'http://minio{1...4}'
reject_text docs/resiliency/docker-compose.yaml 'http://minio{1...4}'
reject_text docs/distributed/DECOMMISSION.md 'systemctl restart minio'
reject_text docs/resiliency/resiliency-tests.sh 'resiliency-minio'
reject_text docs/resiliency/resiliency-tests.sh 'docker system prune'
reject_text docs/resiliency/resiliency-tests.sh 'docker image prune'
reject_text docs/resiliency/resiliency-tests.sh 'docker ps -q'
if grep -Ev '^[[:space:]]*(#|$)' silo.env | grep -q '='; then
fail "silo.env must not contain active assignments that shadow /etc/default/minio"
fi
if rg -n 'pgsty/minio:' .github/workflows Dockerfile.goreleaser helm/silo; then
fail "an active delivery surface still publishes the frozen pgsty/minio image"
fi
# The repository and its default branch are pgsty/silo and main. The invariant
# is that the old name is never a live target, not that it is never spoken: the
# READMEs have to name it to explain the rename and to point at the archived
# artifacts, which is the opposite of stranding a reader on it.
#
# So two rules. First, no live URL may resolve to the old repository anywhere,
# READMEs included.
stale_repo_url="$(rg -n -e 'github\.com/pgsty/minio' -e 'hub\.docker\.com/r/pgsty/minio' \
--glob '!.git/**' --glob '!dist/**' \
--glob '!SILO_REBRANDING_MIGRATION.md' \
--glob '!buildscripts/rebrand-guard/compat-baseline.json' . |
sed 's#^\./##' | grep -v '^buildscripts/verify-rebrand\.sh:' || true)"
if [ -n "${stale_repo_url}" ]; then
printf '%s\n' "${stale_repo_url}" >&2
fail "a link still resolves to the pre-rename pgsty/minio repository"
fi
# Second, the bare name may only appear where it is deliberate: the pinned
# pre-rebrand image digest in the upgrade test, the two guards that refuse a
# legacy image, and the two READMEs that document the rename and the archived
# minio branch.
repo_guard_allowlist='^(buildscripts/minio-upgrade\.sh|buildscripts/verify-rebrand\.sh|buildscripts/helm-migration-guard/main\.go|README\.md|README_ZH\.md):'
stale_repo="$(rg -n 'pgsty/minio' --glob '!.git/**' --glob '!dist/**' \
--glob '!SILO_REBRANDING_MIGRATION.md' \
--glob '!buildscripts/rebrand-guard/compat-baseline.json' . |
sed 's#^\./##' | grep -Ev "${repo_guard_allowlist}" || true)"
if [ -n "${stale_repo}" ]; then
printf '%s\n' "${stale_repo}" >&2
fail "a source reference still names the pre-rename pgsty/minio repository"
fi
stale_branch="$(rg -n 'pgsty/silo/(blob/|tree/|raw/)?master' \
--glob '!.git/**' --glob '!dist/**' . || true)"
if [ -n "${stale_branch}" ]; then
printf '%s\n' "${stale_branch}" >&2
fail "a link still targets the retired master branch; raw and Actions URLs do not follow a branch rename"
fi
for workflow in .github/workflows/go.yml .github/workflows/vulncheck.yml; do
if rg -q '^\s+- master$' "${workflow}"; then
fail "${workflow} still filters on master and would go silently dormant on main"
fi
require_text "${workflow}" " - main"
done
network_hits="$(rg -n --glob '*.go' --glob '!**/*_test.go' \
'https?://[^"`[:space:]]*(dl\.min\.io|subnet\.min\.io|api\.min\.io|slack\.min\.io|play\.min\.io)' \
cmd internal || true)"
network_hits="$(printf '%s\n' "${network_hits}" | grep -Ev '^[^:]+:[0-9]+:[[:space:]]*//' || true)"
if [ -n "${network_hits}" ]; then
printf '%s\n' "${network_hits}" >&2
fail "runtime code still contains an upstream MinIO service endpoint"
fi
require_text cmd/build-constants.go 'MinioReleaseBaseURL = ""'
require_text cmd/globals.go "globalInplaceUpdateDisabled = true"
reject_text cmd/globals.go "subnetAdminPublicKey"
reject_text cmd/admin-handlers.go "getSubnetAdminPublicKey"
echo "Silo delivery and runtime rebrand checks passed"
+1 -3
View File
@@ -255,11 +255,9 @@ func (a adminAPIHandlers) SRPeerReplicateBucketItem(w http.ResponseWriter, r *ht
case madmin.SRBucketMetaTypeTags: case madmin.SRBucketMetaTypeTags:
err = globalSiteReplicationSys.PeerBucketTaggingHandler(ctx, item.Bucket, item.Tags, item.UpdatedAt) err = globalSiteReplicationSys.PeerBucketTaggingHandler(ctx, item.Bucket, item.Tags, item.UpdatedAt)
case madmin.SRBucketMetaTypeObjectLockConfig: case madmin.SRBucketMetaTypeObjectLockConfig:
err = globalSiteReplicationSys.peerBucketObjectLockConfigItem(ctx, item) err = globalSiteReplicationSys.PeerBucketObjectLockConfigHandler(ctx, item.Bucket, item.ObjectLockConfig, item.UpdatedAt)
case madmin.SRBucketMetaTypeSSEConfig: case madmin.SRBucketMetaTypeSSEConfig:
err = globalSiteReplicationSys.PeerBucketSSEConfigHandler(ctx, item.Bucket, item.SSEConfig, item.UpdatedAt) err = globalSiteReplicationSys.PeerBucketSSEConfigHandler(ctx, item.Bucket, item.SSEConfig, item.UpdatedAt)
case madmin.SRBucketMetaTypeCorsConfig:
err = globalSiteReplicationSys.PeerBucketCorsConfigHandler(ctx, item.Bucket, item.Cors, item.UpdatedAt)
case madmin.SRBucketMetaLCConfig: case madmin.SRBucketMetaLCConfig:
err = globalSiteReplicationSys.PeerBucketLCConfigHandler(ctx, item.Bucket, item.ExpiryLCConfig, item.UpdatedAt) err = globalSiteReplicationSys.PeerBucketLCConfigHandler(ctx, item.Bucket, item.ExpiryLCConfig, item.UpdatedAt)
} }
+13 -27
View File
@@ -355,25 +355,18 @@ func (a adminAPIHandlers) ListGroups(w http.ResponseWriter, r *http.Request) {
} }
// SetGroupStatus - PUT /minio/admin/v3/set-group-status?group=mygroup1&status=enabled // SetGroupStatus - PUT /minio/admin/v3/set-group-status?group=mygroup1&status=enabled
func setGroupStatusAdminAction(status string) policy.AdminAction {
if madmin.GroupStatus(status) == madmin.GroupDisabled {
return policy.DisableGroupAdminAction
}
return policy.EnableGroupAdminAction
}
func (a adminAPIHandlers) SetGroupStatus(w http.ResponseWriter, r *http.Request) { func (a adminAPIHandlers) SetGroupStatus(w http.ResponseWriter, r *http.Request) {
ctx := r.Context() ctx := r.Context()
objectAPI, _ := validateAdminReq(ctx, w, r, policy.EnableGroupAdminAction)
if objectAPI == nil {
return
}
vars := mux.Vars(r) vars := mux.Vars(r)
group := vars["group"] group := vars["group"]
status := vars["status"] status := vars["status"]
objectAPI, _ := validateAdminReq(ctx, w, r, setGroupStatusAdminAction(status))
if objectAPI == nil {
return
}
var ( var (
err error err error
updatedAt time.Time updatedAt time.Time
@@ -405,25 +398,18 @@ func (a adminAPIHandlers) SetGroupStatus(w http.ResponseWriter, r *http.Request)
} }
// SetUserStatus - PUT /minio/admin/v3/set-user-status?accessKey=<access_key>&status=[enabled|disabled] // SetUserStatus - PUT /minio/admin/v3/set-user-status?accessKey=<access_key>&status=[enabled|disabled]
func setUserStatusAdminAction(status string) policy.AdminAction {
if madmin.AccountStatus(status) == madmin.AccountDisabled {
return policy.DisableUserAdminAction
}
return policy.EnableUserAdminAction
}
func (a adminAPIHandlers) SetUserStatus(w http.ResponseWriter, r *http.Request) { func (a adminAPIHandlers) SetUserStatus(w http.ResponseWriter, r *http.Request) {
ctx := r.Context() ctx := r.Context()
objectAPI, creds := validateAdminReq(ctx, w, r, policy.EnableUserAdminAction)
if objectAPI == nil {
return
}
vars := mux.Vars(r) vars := mux.Vars(r)
accessKey := vars["accessKey"] accessKey := vars["accessKey"]
status := vars["status"] status := vars["status"]
objectAPI, creds := validateAdminReq(ctx, w, r, setUserStatusAdminAction(status))
if objectAPI == nil {
return
}
// you cannot enable or disable yourself. // you cannot enable or disable yourself.
if accessKey == creds.AccessKey { if accessKey == creds.AccessKey {
writeErrorResponseJSON(ctx, w, toAdminAPIErr(ctx, errInvalidArgument), r.URL) writeErrorResponseJSON(ctx, w, toAdminAPIErr(ctx, errInvalidArgument), r.URL)
@@ -873,7 +859,7 @@ func (a adminAPIHandlers) UpdateServiceAccount(w http.ResponseWriter, r *http.Re
var sp *policy.Policy var sp *policy.Policy
if len(updateReq.NewPolicy) > 0 { if len(updateReq.NewPolicy) > 0 {
sp, err = policy.ParseConfigStrict(bytes.NewReader(updateReq.NewPolicy)) sp, err = policy.ParseConfig(bytes.NewReader(updateReq.NewPolicy))
if err != nil { if err != nil {
writeErrorResponseJSON(ctx, w, toAdminAPIErr(ctx, err), r.URL) writeErrorResponseJSON(ctx, w, toAdminAPIErr(ctx, err), r.URL)
return return
@@ -1743,7 +1729,7 @@ func (a adminAPIHandlers) AddCannedPolicy(w http.ResponseWriter, r *http.Request
return return
} }
iamPolicy, err := policy.ParseConfigStrict(bytes.NewReader(iamPolicyBytes)) iamPolicy, err := policy.ParseConfig(bytes.NewReader(iamPolicyBytes))
if err != nil { if err != nil {
writeErrorResponseJSON(ctx, w, toAdminAPIErr(ctx, err), r.URL) writeErrorResponseJSON(ctx, w, toAdminAPIErr(ctx, err), r.URL)
return return
@@ -2995,7 +2981,7 @@ func commonAddServiceAccount(r *http.Request, ldap bool) (context.Context, auth.
var sp *policy.Policy var sp *policy.Policy
if len(createReq.Policy) > 0 { if len(createReq.Policy) > 0 {
sp, err = policy.ParseConfigStrict(bytes.NewReader(createReq.Policy)) sp, err = policy.ParseConfig(bytes.NewReader(createReq.Policy))
if err != nil { if err != nil {
return ctx, auth.Credentials{}, newServiceAccountOpts{}, madmin.AddServiceAccountReq{}, "", toAdminAPIErr(ctx, err) return ctx, auth.Credentials{}, newServiceAccountOpts{}, madmin.AddServiceAccountReq{}, "", toAdminAPIErr(ctx, err)
} }
-281
View File
@@ -41,53 +41,12 @@ import (
"github.com/minio/minio-go/v7/pkg/signer" "github.com/minio/minio-go/v7/pkg/signer"
"github.com/minio/minio/internal/auth" "github.com/minio/minio/internal/auth"
"github.com/minio/pkg/v3/env" "github.com/minio/pkg/v3/env"
"github.com/minio/pkg/v3/policy"
) )
const ( const (
testDefaultTimeout = 30 * time.Second testDefaultTimeout = 30 * time.Second
) )
func TestSetUserStatusAdminAction(t *testing.T) {
tests := []struct {
name string
status string
want policy.AdminAction
}{
{name: "enable", status: string(madmin.AccountEnabled), want: policy.EnableUserAdminAction},
{name: "disable", status: string(madmin.AccountDisabled), want: policy.DisableUserAdminAction},
{name: "invalid preserves authenticated default", status: "invalid", want: policy.EnableUserAdminAction},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := setUserStatusAdminAction(tt.status); got != tt.want {
t.Fatalf("setUserStatusAdminAction(%q) = %q, want %q", tt.status, got, tt.want)
}
})
}
}
func TestSetGroupStatusAdminAction(t *testing.T) {
tests := []struct {
name string
status string
want policy.AdminAction
}{
{name: "enable", status: string(madmin.GroupEnabled), want: policy.EnableGroupAdminAction},
{name: "disable", status: string(madmin.GroupDisabled), want: policy.DisableGroupAdminAction},
{name: "invalid preserves authenticated default", status: "invalid", want: policy.EnableGroupAdminAction},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := setGroupStatusAdminAction(tt.status); got != tt.want {
t.Fatalf("setGroupStatusAdminAction(%q) = %q, want %q", tt.status, got, tt.want)
}
})
}
}
// API suite container for IAM // API suite container for IAM
type TestSuiteIAM struct { type TestSuiteIAM struct {
TestSuiteCommon TestSuiteCommon
@@ -243,11 +202,8 @@ func TestIAMInternalIDPServerSuite(t *testing.T) {
suite.SetUpSuite(c) suite.SetUpSuite(c)
suite.TestUserCreate(c) suite.TestUserCreate(c)
suite.TestUserStatusActionAuthorization(c)
suite.TestGroupStatusActionAuthorization(c)
suite.TestUserPolicyEscalationBug(c) suite.TestUserPolicyEscalationBug(c)
suite.TestPolicyCreate(c) suite.TestPolicyCreate(c)
suite.TestServiceAccountBareARNPolicyRejected(c)
suite.TestCannedPolicies(c) suite.TestCannedPolicies(c)
suite.TestGroupAddRemove(c) suite.TestGroupAddRemove(c)
suite.TestServiceAccountOpsByAdmin(c) suite.TestServiceAccountOpsByAdmin(c)
@@ -356,184 +312,6 @@ func (s *TestSuiteIAM) TestUserCreate(c *check) {
} }
} }
func (s *TestSuiteIAM) TestUserStatusActionAuthorization(c *check) {
ctx, cancel := context.WithTimeout(context.Background(), testDefaultTimeout)
defer cancel()
var createdUsers []string
var createdPolicies []string
defer func() {
for _, user := range createdUsers {
if err := s.adm.RemoveUser(ctx, user); err != nil {
c.Errorf("unable to remove test user %s: %v", user, err)
}
}
for _, policyName := range createdPolicies {
if err := s.adm.RemoveCannedPolicy(ctx, policyName); err != nil {
c.Errorf("unable to remove test policy %s: %v", policyName, err)
}
}
}()
createUser := func() (string, string) {
accessKey, secretKey := mustGenerateCredentials(c)
if err := s.adm.SetUser(ctx, accessKey, secretKey, madmin.AccountEnabled); err != nil {
c.Fatalf("unable to create test user: %v", err)
}
createdUsers = append(createdUsers, accessKey)
return accessKey, secretKey
}
createStatusClient := func(action policy.AdminAction) *madmin.AdminClient {
accessKey, secretKey := createUser()
policyName := getRandomBucketName()
policyBytes := fmt.Appendf(nil, `{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": ["%s"]
}]
}`, action)
if err := s.adm.AddCannedPolicy(ctx, policyName, policyBytes); err != nil {
c.Fatalf("unable to add status policy: %v", err)
}
createdPolicies = append(createdPolicies, policyName)
if _, err := s.adm.AttachPolicy(ctx, madmin.PolicyAssociationReq{
Policies: []string{policyName},
User: accessKey,
}); err != nil {
c.Fatalf("unable to attach status policy: %v", err)
}
client, err := madmin.NewWithOptions(s.endpoint, &madmin.Options{
Creds: credentials.NewStaticV4(accessKey, secretKey, ""),
Secure: s.secure,
})
if err != nil {
c.Fatalf("unable to create status admin client: %v", err)
}
client.SetCustomTransport(s.TestSuiteCommon.client.Transport)
return client
}
targetAccessKey, _ := createUser()
disableClient := createStatusClient(policy.DisableUserAdminAction)
if err := disableClient.SetUserStatus(ctx, targetAccessKey, madmin.AccountDisabled); err != nil {
c.Fatalf("DisableUser-only client could not disable a user: %v", err)
}
if err := disableClient.SetUserStatus(ctx, targetAccessKey, madmin.AccountEnabled); err == nil || madmin.ToErrorResponse(err).Code != "AccessDenied" {
c.Fatalf("DisableUser-only client unexpectedly enabled a user: %v", err)
}
enableClient := createStatusClient(policy.EnableUserAdminAction)
if err := enableClient.SetUserStatus(ctx, targetAccessKey, madmin.AccountEnabled); err != nil {
c.Fatalf("EnableUser-only client could not enable a user: %v", err)
}
if err := enableClient.SetUserStatus(ctx, targetAccessKey, madmin.AccountDisabled); err == nil || madmin.ToErrorResponse(err).Code != "AccessDenied" {
c.Fatalf("EnableUser-only client unexpectedly disabled a user: %v", err)
}
}
func (s *TestSuiteIAM) TestGroupStatusActionAuthorization(c *check) {
ctx, cancel := context.WithTimeout(context.Background(), testDefaultTimeout)
defer cancel()
var createdUsers []string
var createdPolicies []string
group := getRandomBucketName()
var groupCreated bool
defer func() {
if groupCreated {
if err := s.adm.UpdateGroupMembers(ctx, madmin.GroupAddRemove{
Group: group,
Members: createdUsers[:1],
IsRemove: true,
}); err != nil {
c.Errorf("unable to remove group member: %v", err)
}
if err := s.adm.UpdateGroupMembers(ctx, madmin.GroupAddRemove{Group: group, IsRemove: true}); err != nil {
c.Errorf("unable to remove test group: %v", err)
}
}
for _, user := range createdUsers {
if err := s.adm.RemoveUser(ctx, user); err != nil {
c.Errorf("unable to remove test user %s: %v", user, err)
}
}
for _, policyName := range createdPolicies {
if err := s.adm.RemoveCannedPolicy(ctx, policyName); err != nil {
c.Errorf("unable to remove test policy %s: %v", policyName, err)
}
}
}()
createUser := func() (string, string) {
accessKey, secretKey := mustGenerateCredentials(c)
if err := s.adm.SetUser(ctx, accessKey, secretKey, madmin.AccountEnabled); err != nil {
c.Fatalf("unable to create test user: %v", err)
}
createdUsers = append(createdUsers, accessKey)
return accessKey, secretKey
}
targetAccessKey, _ := createUser()
if err := s.adm.UpdateGroupMembers(ctx, madmin.GroupAddRemove{
Group: group,
Members: []string{targetAccessKey},
}); err != nil {
c.Fatalf("unable to create test group: %v", err)
}
groupCreated = true
createStatusClient := func(action policy.AdminAction) *madmin.AdminClient {
accessKey, secretKey := createUser()
policyName := getRandomBucketName()
policyBytes := fmt.Appendf(nil, `{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": ["%s"]
}]
}`, action)
if err := s.adm.AddCannedPolicy(ctx, policyName, policyBytes); err != nil {
c.Fatalf("unable to add group status policy: %v", err)
}
createdPolicies = append(createdPolicies, policyName)
if _, err := s.adm.AttachPolicy(ctx, madmin.PolicyAssociationReq{
Policies: []string{policyName},
User: accessKey,
}); err != nil {
c.Fatalf("unable to attach group status policy: %v", err)
}
client, err := madmin.NewWithOptions(s.endpoint, &madmin.Options{
Creds: credentials.NewStaticV4(accessKey, secretKey, ""),
Secure: s.secure,
})
if err != nil {
c.Fatalf("unable to create group status admin client: %v", err)
}
client.SetCustomTransport(s.TestSuiteCommon.client.Transport)
return client
}
disableClient := createStatusClient(policy.DisableGroupAdminAction)
if err := disableClient.SetGroupStatus(ctx, group, madmin.GroupDisabled); err != nil {
c.Fatalf("DisableGroup-only client could not disable a group: %v", err)
}
if err := disableClient.SetGroupStatus(ctx, group, madmin.GroupEnabled); err == nil || madmin.ToErrorResponse(err).Code != "AccessDenied" {
c.Fatalf("DisableGroup-only client unexpectedly enabled a group: %v", err)
}
enableClient := createStatusClient(policy.EnableGroupAdminAction)
if err := enableClient.SetGroupStatus(ctx, group, madmin.GroupEnabled); err != nil {
c.Fatalf("EnableGroup-only client could not enable a group: %v", err)
}
if err := enableClient.SetGroupStatus(ctx, group, madmin.GroupDisabled); err == nil || madmin.ToErrorResponse(err).Code != "AccessDenied" {
c.Fatalf("EnableGroup-only client unexpectedly disabled a group: %v", err)
}
}
func (s *TestSuiteIAM) TestUserPolicyEscalationBug(c *check) { func (s *TestSuiteIAM) TestUserPolicyEscalationBug(c *check) {
ctx, cancel := context.WithTimeout(context.Background(), testDefaultTimeout) ctx, cancel := context.WithTimeout(context.Background(), testDefaultTimeout)
defer cancel() defer cancel()
@@ -822,20 +600,6 @@ func (s *TestSuiteIAM) TestPolicyCreate(c *check) {
c.Fatalf("invalid policy creation success") c.Fatalf("invalid policy creation success")
} }
for i, resource := range []string{"arn:aws:s3:::", "*arn:aws:s3:::"} {
barePolicyBytes := fmt.Appendf(nil, `{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Deny",
"Action": ["s3:GetObject"],
"Resource": ["%s"]
}]
}`, resource)
if err = s.adm.AddCannedPolicy(ctx, fmt.Sprintf("%s-bare-%d", policy, i), barePolicyBytes); err == nil {
c.Fatalf("bare ARN policy creation succeeded for %q", resource)
}
}
// 3. Create a user, associate policy and verify access // 3. Create a user, associate policy and verify access
accessKey, secretKey := mustGenerateCredentials(c) accessKey, secretKey := mustGenerateCredentials(c)
err = s.adm.SetUser(ctx, accessKey, secretKey, madmin.AccountEnabled) err = s.adm.SetUser(ctx, accessKey, secretKey, madmin.AccountEnabled)
@@ -889,51 +653,6 @@ func (s *TestSuiteIAM) TestPolicyCreate(c *check) {
} }
} }
func (s *TestSuiteIAM) TestServiceAccountBareARNPolicyRejected(c *check) {
ctx, cancel := context.WithTimeout(context.Background(), testDefaultTimeout)
defer cancel()
barePolicy := []byte(`{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": ["s3:GetObject"],
"NotResource": ["arn:aws:s3:::"]
}]
}`)
if _, err := s.adm.AddServiceAccount(ctx, madmin.AddServiceAccountReq{
TargetUser: globalActiveCred.AccessKey,
Policy: barePolicy,
}); err == nil {
c.Fatal("service account creation accepted a bare ARN policy")
}
validPolicy := []byte(`{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": ["s3:GetObject"],
"Resource": ["arn:aws:s3:::*"]
}]
}`)
credentials, err := s.adm.AddServiceAccount(ctx, madmin.AddServiceAccountReq{
TargetUser: globalActiveCred.AccessKey,
Policy: validPolicy,
})
if err != nil {
c.Fatalf("service account creation rejected an explicit resource: %v", err)
}
defer func() {
_ = s.adm.DeleteServiceAccount(ctx, credentials.AccessKey)
}()
if err = s.adm.UpdateServiceAccount(ctx, credentials.AccessKey, madmin.UpdateServiceAccountReq{
NewPolicy: barePolicy,
}); err == nil {
c.Fatal("service account update accepted a bare ARN policy")
}
}
func (s *TestSuiteIAM) TestCannedPolicies(c *check) { func (s *TestSuiteIAM) TestCannedPolicies(c *check) {
ctx, cancel := context.WithTimeout(context.Background(), testDefaultTimeout) ctx, cancel := context.WithTimeout(context.Background(), testDefaultTimeout)
defer cancel() defer cancel()
+41 -33
View File
@@ -84,7 +84,7 @@ const (
// ServerUpdateV2Handler - POST /minio/admin/v3/update?updateURL={updateURL}&type=2 // ServerUpdateV2Handler - POST /minio/admin/v3/update?updateURL={updateURL}&type=2
// ---------- // ----------
// Retained for Admin API compatibility. Silo always returns MethodNotAllowed. // updates all minio servers and restarts them gracefully.
func (a adminAPIHandlers) ServerUpdateV2Handler(w http.ResponseWriter, r *http.Request) { func (a adminAPIHandlers) ServerUpdateV2Handler(w http.ResponseWriter, r *http.Request) {
ctx := r.Context() ctx := r.Context()
@@ -320,7 +320,7 @@ func (a adminAPIHandlers) ServerUpdateV2Handler(w http.ResponseWriter, r *http.R
// ServerUpdateHandler - POST /minio/admin/v3/update?updateURL={updateURL} // ServerUpdateHandler - POST /minio/admin/v3/update?updateURL={updateURL}
// ---------- // ----------
// Retained for Admin API compatibility. Silo always returns MethodNotAllowed. // updates all minio servers and restarts them gracefully.
func (a adminAPIHandlers) ServerUpdateHandler(w http.ResponseWriter, r *http.Request) { func (a adminAPIHandlers) ServerUpdateHandler(w http.ResponseWriter, r *http.Request) {
ctx := r.Context() ctx := r.Context()
@@ -330,7 +330,7 @@ func (a adminAPIHandlers) ServerUpdateHandler(w http.ResponseWriter, r *http.Req
} }
if globalInplaceUpdateDisabled || currentReleaseTime.IsZero() { if globalInplaceUpdateDisabled || currentReleaseTime.IsZero() {
// MINIO_UPDATE is retained, but Silo permanently disables in-place updates. // if MINIO_UPDATE=off - inplace update is disabled, mostly in containers.
writeErrorResponseJSON(ctx, w, errorCodes.ToAPIErr(ErrMethodNotAllowed), r.URL) writeErrorResponseJSON(ctx, w, errorCodes.ToAPIErr(ErrMethodNotAllowed), r.URL)
return return
} }
@@ -2708,10 +2708,10 @@ func fetchHealthInfo(healthCtx context.Context, objectAPI ObjectLayer, query *ur
} }
// Server start command regex groups: // Server start command regex groups:
// 1 - silo server (or the legacy minio command) // 1 - minio server
// 2 - flags e.g. `--address :9000 --certs-dir /etc/silo/certs` // 2 - flags e.g. `--address :9000 --certs-dir /etc/minio/certs`
// 3 - pool args e.g. `https://node{01...16}.domain/data/disk{001...204} https://node{17...32}.domain/data/disk{001...204}` // 3 - pool args e.g. `https://node{01...16}.domain/data/disk{001...204} https://node{17...32}.domain/data/disk{001...204}`
re := regexp.MustCompile(`^(.*silo\s+server\s+|.*minio\s+server\s+)(--[^\s]+\s+[^\s]+\s+)*(.*)`) re := regexp.MustCompile(`^(.*minio\s+server\s+)(--[^\s]+\s+[^\s]+\s+)*(.*)`)
// stays unchanged in the anonymized version // stays unchanged in the anonymized version
cmdLineWithoutPools := re.ReplaceAllString(cmdLine, `$1$2`) cmdLineWithoutPools := re.ReplaceAllString(cmdLine, `$1$2`)
@@ -3282,7 +3282,28 @@ func (a adminAPIHandlers) InspectDataHandler(w http.ResponseWriter, r *http.Requ
stream := estream.NewWriter(w) stream := estream.NewWriter(w)
defer stream.Close() defer stream.Close()
if err := addInspectDataKey(stream, publicKey, getClusterMetaInfo(ctx)); err != nil { clusterKey, err := bytesToPublicKey(getSubnetAdminPublicKey())
if err != nil {
bugLogIf(ctx, stream.AddError(err.Error()))
return
}
err = stream.AddKeyEncrypted(clusterKey)
if err != nil {
bugLogIf(ctx, stream.AddError(err.Error()))
return
}
if b := getClusterMetaInfo(ctx); len(b) > 0 {
w, err := stream.AddEncryptedStream("cluster.info", nil)
if err != nil {
bugLogIf(ctx, err)
return
}
w.Write(b)
w.Close()
}
// Add new key for inspect data.
if err := stream.AddKeyEncrypted(publicKey); err != nil {
bugLogIf(ctx, stream.AddError(err.Error())) bugLogIf(ctx, stream.AddError(err.Error()))
return return
} }
@@ -3411,7 +3432,7 @@ func (a adminAPIHandlers) InspectDataHandler(w http.ResponseWriter, r *http.Requ
scheme = "http" scheme = "http"
} }
// Save a Silo start script to inspect command. // save MinIO start script to inspect command
var scrb bytes.Buffer var scrb bytes.Buffer
fmt.Fprintf(&scrb, `#!/usr/bin/env bash fmt.Fprintf(&scrb, `#!/usr/bin/env bash
@@ -3422,43 +3443,30 @@ function main() {
done done
# Read content of inspect-input.txt # Read content of inspect-input.txt
SILO_OPTS=$(grep "Server command line args" <./inspect-input.txt | sed "s/Server command line args: //g" | sed -r "s#%s:\/\/#\.\/#g") MINIO_OPTS=$(grep "Server command line args" <./inspect-input.txt | sed "s/Server command line args: //g" | sed -r "s#%s:\/\/#\.\/#g")
# Start Silo using the options # Start MinIO instance using the options
START_CMD="CI=on _MINIO_AUTO_DRIVE_HEALING=off silo server ${SILO_OPTS} &" START_CMD="CI=on _MINIO_AUTO_DRIVE_HEALING=off minio server ${MINIO_OPTS} &"
echo echo
echo "Starting Silo: ${START_CMD}" echo "Starting MinIO instance: ${START_CMD}"
echo echo
eval "$START_CMD" eval "$START_CMD"
SILO_SRVR_PID="$!" MINIO_SRVR_PID="$!"
echo "Silo Server PID: ${SILO_SRVR_PID}" echo "MinIO Server PID: ${MINIO_SRVR_PID}"
echo echo
echo "Waiting for Silo to get ready!" echo "Waiting for MinIO instance to get ready!"
sleep 10 sleep 10
} }
main "$@"`, scheme) main "$@"`, scheme)
adminLogIf(ctx, embedFileInZip(inspectZipW, "start-silo.sh", scrb.Bytes(), 0o755)) adminLogIf(ctx, embedFileInZip(inspectZipW, "start-minio.sh", scrb.Bytes(), 0o755))
} }
// addInspectDataKey makes the requester the only recipient of encrypted func getSubnetAdminPublicKey() []byte {
// diagnostic data. Silo has no built-in vendor or support-service recipient. if globalIsCICD {
func addInspectDataKey(stream *estream.Writer, publicKey *rsa.PublicKey, clusterInfo []byte) error { return subnetAdminPublicKeyDev
if err := stream.AddKeyEncrypted(publicKey); err != nil {
return err
} }
if len(clusterInfo) == 0 { return subnetAdminPublicKey
return nil
}
w, err := stream.AddEncryptedStream("cluster.info", nil)
if err != nil {
return err
}
if _, err = w.Write(clusterInfo); err != nil {
_ = w.Close()
return err
}
return w.Close()
} }
func createHostAnonymizerForFSMode() map[string]string { func createHostAnonymizerForFSMode() map[string]string {
+1 -1
View File
@@ -64,7 +64,7 @@ func prepareAdminErasureTestBed(ctx context.Context) (*adminErasureTestBed, erro
return nil, xlErr return nil, xlErr
} }
// Initialize Silo server config. // Initialize minio server config.
if err := newTestConfig(globalMinioDefaultRegion, objLayer); err != nil { if err := newTestConfig(globalMinioDefaultRegion, objLayer); err != nil {
cancel() cancel()
return nil, err return nil, err
+1 -13
View File
@@ -1154,7 +1154,7 @@ var errorCodes = errorCodeMap{
}, },
ErrUnsupportedNotification: { ErrUnsupportedNotification: {
Code: "UnsupportedNotification", Code: "UnsupportedNotification",
Description: "Silo does not support Topic or Cloud Function based notifications.", Description: "MinIO server does not support Topic or Cloud Function based notifications.",
HTTPStatusCode: http.StatusBadRequest, HTTPStatusCode: http.StatusBadRequest,
}, },
ErrInvalidCopyPartRange: { ErrInvalidCopyPartRange: {
@@ -2169,10 +2169,6 @@ func toAPIErrorCode(ctx context.Context, err error) (apiErr APIErrorCode) {
err = unwrapAll(err) err = unwrapAll(err)
switch err { switch err {
case errCompleteMultipartChecksumMismatch, errCompleteMultipartChecksumTypeMismatch:
apiErr = ErrBadDigest
case errMissingPartChecksum:
apiErr = ErrInvalidRequest
case errInvalidArgument: case errInvalidArgument:
apiErr = ErrAdminInvalidArgument apiErr = ErrAdminInvalidArgument
case errNoSuchPolicy: case errNoSuchPolicy:
@@ -2469,14 +2465,6 @@ func toAPIError(ctx context.Context, err error) APIError {
} }
apiErr := errorCodes.ToAPIErr(toAPIErrorCode(ctx, err)) apiErr := errorCodes.ToAPIErr(toAPIErrorCode(ctx, err))
switch {
case errors.Is(err, errCompleteMultipartChecksumMismatch):
apiErr.Description = strings.TrimPrefix(err.Error(), errCompleteMultipartChecksumMismatch.Error()+": ")
case errors.Is(err, errCompleteMultipartChecksumTypeMismatch):
apiErr.Description = strings.TrimPrefix(err.Error(), errCompleteMultipartChecksumTypeMismatch.Error()+": ")
case errors.Is(err, errMissingPartChecksum):
apiErr.Description = strings.TrimPrefix(err.Error(), errMissingPartChecksum.Error()+": ")
}
switch apiErr.Code { switch apiErr.Code {
case "NotImplemented": case "NotImplemented":
apiErr = APIError{ apiErr = APIError{
-4
View File
@@ -39,10 +39,6 @@ var toAPIErrorTests = []struct {
{err: ObjectNameInvalid{}, errCode: ErrInvalidObjectName}, {err: ObjectNameInvalid{}, errCode: ErrInvalidObjectName},
{err: InvalidUploadID{}, errCode: ErrNoSuchUpload}, {err: InvalidUploadID{}, errCode: ErrNoSuchUpload},
{err: InvalidPart{}, errCode: ErrInvalidPart}, {err: InvalidPart{}, errCode: ErrInvalidPart},
{err: errCompleteMultipartChecksumMismatch, errCode: ErrBadDigest},
{err: errCompleteMultipartChecksumTypeMismatch, errCode: ErrBadDigest},
{err: errMissingPartChecksum, errCode: ErrInvalidRequest},
{err: hash.ChecksumMismatch{}, errCode: ErrContentChecksumMismatch},
{err: InsufficientReadQuorum{}, errCode: ErrSlowDownRead}, {err: InsufficientReadQuorum{}, errCode: ErrSlowDownRead},
{err: InsufficientWriteQuorum{}, errCode: ErrSlowDownWrite}, {err: InsufficientWriteQuorum{}, errCode: ErrSlowDownWrite},
{err: InvalidUploadIDKeyCombination{}, errCode: ErrNotImplemented}, {err: InvalidUploadIDKeyCombination{}, errCode: ErrNotImplemented},
-9
View File
@@ -18,18 +18,9 @@
package cmd package cmd
import ( import (
"net/http/httptest"
"testing" "testing"
) )
func TestCommonHeadersUseSiloProductName(t *testing.T) {
recorder := httptest.NewRecorder()
setCommonHeaders(recorder)
if got := recorder.Header().Get("Server"); got != "Silo" {
t.Fatalf("Server header = %q, want Silo", got)
}
}
func TestNewRequestID(t *testing.T) { func TestNewRequestID(t *testing.T) {
// Ensure that it returns an alphanumeric result of length 16. // Ensure that it returns an alphanumeric result of length 16.
id := mustGetRequestID(UTCNow()) id := mustGetRequestID(UTCNow())
+9 -34
View File
@@ -27,6 +27,7 @@ import (
"path" "path"
"strconv" "strconv"
"strings" "strings"
"time"
"github.com/minio/minio/internal/amztime" "github.com/minio/minio/internal/amztime"
"github.com/minio/minio/internal/crypto" "github.com/minio/minio/internal/crypto"
@@ -379,13 +380,6 @@ type CopyObjectResponse struct {
XMLName xml.Name `xml:"http://s3.amazonaws.com/doc/2006-03-01/ CopyObjectResult" json:"-"` XMLName xml.Name `xml:"http://s3.amazonaws.com/doc/2006-03-01/ CopyObjectResult" json:"-"`
LastModified string // time string of format "2006-01-02T15:04:05.000Z" LastModified string // time string of format "2006-01-02T15:04:05.000Z"
ETag string // md5sum of the copied object. ETag string // md5sum of the copied object.
ChecksumCRC32 string `xml:",omitempty"`
ChecksumCRC32C string `xml:",omitempty"`
ChecksumSHA1 string `xml:",omitempty"`
ChecksumSHA256 string `xml:",omitempty"`
ChecksumCRC64NVME string `xml:",omitempty"`
ChecksumType string `xml:",omitempty"`
} }
// CopyObjectPartResponse container returns ETag and LastModified of the successfully copied object // CopyObjectPartResponse container returns ETag and LastModified of the successfully copied object
@@ -393,12 +387,6 @@ type CopyObjectPartResponse struct {
XMLName xml.Name `xml:"http://s3.amazonaws.com/doc/2006-03-01/ CopyPartResult" json:"-"` XMLName xml.Name `xml:"http://s3.amazonaws.com/doc/2006-03-01/ CopyPartResult" json:"-"`
LastModified string // time string of format "2006-01-02T15:04:05.000Z" LastModified string // time string of format "2006-01-02T15:04:05.000Z"
ETag string // md5sum of the copied object part. ETag string // md5sum of the copied object part.
ChecksumCRC32 string `xml:",omitempty"`
ChecksumCRC32C string `xml:",omitempty"`
ChecksumSHA1 string `xml:",omitempty"`
ChecksumSHA256 string `xml:",omitempty"`
ChecksumCRC64NVME string `xml:",omitempty"`
} }
// Initiator inherit from Owner struct, fields are same // Initiator inherit from Owner struct, fields are same
@@ -428,7 +416,6 @@ type CompleteMultipartUploadResponse struct {
Key string Key string
ETag string ETag string
ChecksumType string `xml:"ChecksumType,omitempty"`
ChecksumCRC32 string `xml:"ChecksumCRC32,omitempty"` ChecksumCRC32 string `xml:"ChecksumCRC32,omitempty"`
ChecksumCRC32C string `xml:"ChecksumCRC32C,omitempty"` ChecksumCRC32C string `xml:"ChecksumCRC32C,omitempty"`
ChecksumSHA1 string `xml:"ChecksumSHA1,omitempty"` ChecksumSHA1 string `xml:"ChecksumSHA1,omitempty"`
@@ -776,30 +763,19 @@ func generateListObjectsV2Response(ctx context.Context, bucket, prefix, token, n
type metaCheckFn = func(name string, action policy.Action) (s3Err APIErrorCode) type metaCheckFn = func(name string, action policy.Action) (s3Err APIErrorCode)
// generates CopyObjectResponse from the committed object information. // generates CopyObjectResponse from etag and lastModified time.
func generateCopyObjectResponse(oi ObjectInfo, cs map[string]string) CopyObjectResponse { func generateCopyObjectResponse(etag string, lastModified time.Time) CopyObjectResponse {
return CopyObjectResponse{ return CopyObjectResponse{
ETag: "\"" + oi.ETag + "\"", ETag: "\"" + etag + "\"",
LastModified: amztime.ISO8601Format(oi.ModTime.UTC()), LastModified: amztime.ISO8601Format(lastModified.UTC()),
ChecksumCRC32: cs[hash.ChecksumCRC32.String()],
ChecksumCRC32C: cs[hash.ChecksumCRC32C.String()],
ChecksumSHA1: cs[hash.ChecksumSHA1.String()],
ChecksumSHA256: cs[hash.ChecksumSHA256.String()],
ChecksumCRC64NVME: cs[hash.ChecksumCRC64NVME.String()],
ChecksumType: cs[xhttp.AmzChecksumType],
} }
} }
// generates CopyObjectPartResponse from the uploaded part information. // generates CopyObjectPartResponse from etag and lastModified time.
func generateCopyObjectPartResponse(partInfo PartInfo) CopyObjectPartResponse { func generateCopyObjectPartResponse(etag string, lastModified time.Time) CopyObjectPartResponse {
return CopyObjectPartResponse{ return CopyObjectPartResponse{
ETag: "\"" + partInfo.ETag + "\"", ETag: "\"" + etag + "\"",
LastModified: amztime.ISO8601Format(partInfo.LastModified.UTC()), LastModified: amztime.ISO8601Format(lastModified.UTC()),
ChecksumCRC32: partInfo.ChecksumCRC32,
ChecksumCRC32C: partInfo.ChecksumCRC32C,
ChecksumSHA1: partInfo.ChecksumSHA1,
ChecksumSHA256: partInfo.ChecksumSHA256,
ChecksumCRC64NVME: partInfo.ChecksumCRC64NVME,
} }
} }
@@ -821,7 +797,6 @@ func generateCompleteMultipartUploadResponse(bucket, key, location string, oi Ob
Key: key, Key: key,
// AWS S3 quotes the ETag in XML, make sure we are compatible here. // AWS S3 quotes the ETag in XML, make sure we are compatible here.
ETag: "\"" + oi.ETag + "\"", ETag: "\"" + oi.ETag + "\"",
ChecksumType: cs[xhttp.AmzChecksumType],
ChecksumSHA1: cs[hash.ChecksumSHA1.String()], ChecksumSHA1: cs[hash.ChecksumSHA1.String()],
ChecksumSHA256: cs[hash.ChecksumSHA256.String()], ChecksumSHA256: cs[hash.ChecksumSHA256.String()],
ChecksumCRC32: cs[hash.ChecksumCRC32.String()], ChecksumCRC32: cs[hash.ChecksumCRC32.String()],
+6 -125
View File
@@ -18,23 +18,16 @@
package cmd package cmd
import ( import (
"context"
"errors"
"net" "net"
"net/http" "net/http"
"strconv"
"strings"
consoleapi "github.com/minio/console/api" consoleapi "github.com/minio/console/api"
bktcors "github.com/minio/minio/internal/bucket/cors"
xhttp "github.com/minio/minio/internal/http" xhttp "github.com/minio/minio/internal/http"
"github.com/minio/mux" "github.com/minio/mux"
"github.com/minio/pkg/v3/wildcard" "github.com/minio/pkg/v3/wildcard"
"github.com/rs/cors" "github.com/rs/cors"
) )
type bucketCorsAppliedKey struct{}
func newHTTPServerFn() *xhttp.Server { func newHTTPServerFn() *xhttp.Server {
globalObjLayerMutex.RLock() globalObjLayerMutex.RLock()
defer globalObjLayerMutex.RUnlock() defer globalObjLayerMutex.RUnlock()
@@ -118,6 +111,11 @@ var rejectedBucketAPIs = []rejectedAPI{
methods: []string{http.MethodGet, http.MethodPut, http.MethodDelete}, methods: []string{http.MethodGet, http.MethodPut, http.MethodDelete},
queries: []string{"inventory", ""}, queries: []string{"inventory", ""},
}, },
{
api: "cors",
methods: []string{http.MethodPut, http.MethodDelete},
queries: []string{"cors", ""},
},
{ {
api: "metrics", api: "metrics",
methods: []string{http.MethodGet, http.MethodPut, http.MethodDelete}, methods: []string{http.MethodGet, http.MethodPut, http.MethodDelete},
@@ -650,94 +648,6 @@ func registerAPIRouter(router *mux.Router) {
apiRouter.MethodNotAllowedHandler = collectAPIStats("methodnotallowed", httpTraceAll(methodNotAllowedHandler("S3"))) apiRouter.MethodNotAllowedHandler = collectAPIStats("methodnotallowed", httpTraceAll(methodNotAllowedHandler("S3")))
} }
// applyBucketCors applies a bucket's CORS configuration to the request.
// For an OPTIONS preflight it writes the full CORS response and returns true
// (request is complete). For an actual request it adds the applicable
// Access-Control-* response headers and returns false so the request
// continues down the handler chain. If no rule matches a preflight it writes
// 403 and returns true. A matched actual request is marked in its context so
// inner legacy middleware does not rewrite an explicitly allowed null origin.
func applyBucketCors(w http.ResponseWriter, r *http.Request, cfg *bktcors.Config) (handled bool) {
origin := r.Header.Get("Origin")
if origin == "" {
return false // not a CORS request
}
h := w.Header()
h.Add("Vary", "Origin")
isPreflight := r.Method == http.MethodOptions &&
r.Header.Get("Access-Control-Request-Method") != ""
if isPreflight {
method := r.Header.Get("Access-Control-Request-Method")
reqHeaders := splitAndTrim(r.Header.Get("Access-Control-Request-Headers"))
// A preflight response depends on all three request headers that
// determine the outcome, including when the request is rejected.
h.Add("Vary", "Access-Control-Request-Method")
h.Add("Vary", "Access-Control-Request-Headers")
rule, allowedOrigin, allowedHeaders, maxAgeSeconds, ok := cfg.MatchPreflight(origin, method, reqHeaders)
if !ok {
writeResponse(w, http.StatusForbidden, nil, mimeNone)
return true
}
setBucketCorsOriginHeaders(h, allowedOrigin, origin)
h.Set("Access-Control-Allow-Methods", strings.Join(rule.AllowedMethods, ", "))
if len(allowedHeaders) > 0 {
h.Set("Access-Control-Allow-Headers", strings.Join(allowedHeaders, ", "))
}
if len(rule.ExposeHeaders) > 0 {
h.Set("Access-Control-Expose-Headers", strings.Join(rule.ExposeHeaders, ", "))
}
if maxAgeSeconds != nil {
h.Set("Access-Control-Max-Age", strconv.Itoa(*maxAgeSeconds))
}
writeResponse(w, http.StatusOK, nil, mimeNone)
return true
}
// Actual request: attach headers if the origin+method match.
rule, allowedOrigin, ok := cfg.MatchRule(origin, r.Method)
if !ok {
return false // no matching rule → no CORS headers, continue normally
}
*r = *r.WithContext(context.WithValue(r.Context(), bucketCorsAppliedKey{}, struct{}{}))
setBucketCorsOriginHeaders(h, allowedOrigin, origin)
if len(rule.ExposeHeaders) > 0 {
h.Set("Access-Control-Expose-Headers", strings.Join(rule.ExposeHeaders, ", "))
}
return false
}
func bucketCorsWasApplied(r *http.Request) bool {
_, ok := r.Context().Value(bucketCorsAppliedKey{}).(struct{})
return ok
}
func setBucketCorsOriginHeaders(h http.Header, allowedOrigin, requestOrigin string) {
if allowedOrigin == "*" {
h.Set("Access-Control-Allow-Origin", "*")
h.Del("Access-Control-Allow-Credentials")
return
}
h.Set("Access-Control-Allow-Origin", requestOrigin)
h.Set("Access-Control-Allow-Credentials", "true")
}
// splitAndTrim splits a comma-separated header list into trimmed, non-empty values.
func splitAndTrim(s string) []string {
if s == "" {
return nil
}
parts := strings.Split(s, ",")
out := parts[:0]
for _, p := range parts {
if p = strings.TrimSpace(p); p != "" {
out = append(out, p)
}
}
return out
}
// corsHandler handler for CORS (Cross Origin Resource Sharing) // corsHandler handler for CORS (Cross Origin Resource Sharing)
func corsHandler(handler http.Handler) http.Handler { func corsHandler(handler http.Handler) http.Handler {
commonS3Headers := []string{ commonS3Headers := []string{
@@ -783,34 +693,5 @@ func corsHandler(handler http.Handler) http.Handler {
ExposedHeaders: commonS3Headers, ExposedHeaders: commonS3Headers,
AllowCredentials: true, AllowCredentials: true,
} }
globalCors := cors.New(opts).Handler(handler) return cors.New(opts).Handler(handler)
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("Origin") != "" {
if bucket, _ := request2BucketObjectName(r); bucket != "" && globalBucketMetadataSys != nil {
// Resident-only lookup: this runs pre-auth for every
// Origin-bearing request using a client-supplied path segment as
// the bucket name. It must never load or cache metadata for
// arbitrary names (see GetResidentCorsConfig). GetResidentCorsConfig
// is the single decision point: it returns errInvalidArgument for
// the internal .minio.sys namespace (fail closed), a config for a
// resident bucket, errBucketMetadataNotInitialized for a real but
// unloaded bucket (fail closed), and errConfigNotFound otherwise
// (fall back to the global policy below).
cfg, _, err := globalBucketMetadataSys.GetResidentCorsConfig(bucket)
if err == nil && cfg != nil {
if applyBucketCors(w, r, cfg) {
return
}
handler.ServeHTTP(w, r)
return
}
if err != nil && !errors.Is(err, errConfigNotFound) {
internalLogOnceIf(r.Context(), err, "bucket-cors-metadata")
handler.ServeHTTP(w, r)
return
}
}
}
globalCors.ServeHTTP(w, r)
})
} }
+30 -65
View File
@@ -363,6 +363,17 @@ func checkRequestAuthTypeWithRequestTags(ctx context.Context, r *http.Request, a
return authorizeRequestWithTags(ctx, r, action, "", requestTags) return authorizeRequestWithTags(ctx, r, action, "", requestTags)
} }
// checkRequestAuthTypeWithVID is similar to checkRequestAuthType
// passes versionID additionally.
func checkRequestAuthTypeWithVID(ctx context.Context, r *http.Request, action policy.Action, bucketName, objectName, versionID string) (s3Err APIErrorCode) {
logger.GetReqInfo(ctx).BucketName = bucketName
logger.GetReqInfo(ctx).ObjectName = objectName
logger.GetReqInfo(ctx).VersionID = versionID
_, _, s3Err = checkRequestAuthTypeCredential(ctx, r, action)
return s3Err
}
func authenticateRequest(ctx context.Context, r *http.Request, action policy.Action) (s3Err APIErrorCode) { func authenticateRequest(ctx context.Context, r *http.Request, action policy.Action) (s3Err APIErrorCode) {
if logger.GetReqInfo(ctx) == nil { if logger.GetReqInfo(ctx) == nil {
bugLogIf(ctx, errors.New("unexpected context.Context does not have a logger.ReqInfo"), logger.ErrorKind) bugLogIf(ctx, errors.New("unexpected context.Context does not have a logger.ReqInfo"), logger.ErrorKind)
@@ -428,23 +439,6 @@ func authorizeRequest(ctx context.Context, r *http.Request, action policy.Action
return authorizeRequestWithExistingTags(ctx, r, action, "") return authorizeRequestWithExistingTags(ctx, r, action, "")
} }
func deleteObjectAction(versionID string) policy.Action {
if versionID != "" {
return policy.DeleteObjectVersionAction
}
return policy.DeleteObjectAction
}
func actionUsesObjectVersion(action policy.Action) bool {
switch action {
case policy.DeleteObjectAction, policy.DeleteObjectVersionAction,
policy.ReplicateDeleteAction, policy.BypassGovernanceRetentionAction:
return true
default:
return false
}
}
func authorizeRequestWithExistingTags(ctx context.Context, r *http.Request, action policy.Action, existingTags string) (s3Err APIErrorCode) { func authorizeRequestWithExistingTags(ctx context.Context, r *http.Request, action policy.Action, existingTags string) (s3Err APIErrorCode) {
return authorizeRequestWithTags(ctx, r, action, existingTags, nil) return authorizeRequestWithTags(ctx, r, action, existingTags, nil)
} }
@@ -463,7 +457,7 @@ func authorizeRequestWithTags(ctx context.Context, r *http.Request, action polic
versionID := reqInfo.VersionID versionID := reqInfo.VersionID
conditionValuesForAuth := func(locationConstraint string, credentials auth.Credentials) map[string][]string { conditionValuesForAuth := func(locationConstraint string, credentials auth.Credentials) map[string][]string {
values := getConditionValuesWithTags(r, locationConstraint, credentials, existingTags, requestTags) values := getConditionValuesWithTags(r, locationConstraint, credentials, existingTags, requestTags)
if actionUsesObjectVersion(action) { if action == policy.DeleteObjectAction {
// DeleteObjects carries the effective version ID in each XML object, // DeleteObjects carries the effective version ID in each XML object,
// not in the request query. Keep authorization scoped to that entry. // not in the request query. Keep authorization scoped to that entry.
if versionID == "" { if versionID == "" {
@@ -509,6 +503,21 @@ func authorizeRequestWithTags(ctx context.Context, r *http.Request, action polic
return ErrAccessDenied return ErrAccessDenied
} }
if action == policy.DeleteObjectAction && versionID != "" {
if !globalIAMSys.IsAllowed(policy.Args{
AccountName: cred.AccessKey,
Groups: cred.Groups,
Action: policy.Action(policy.DeleteObjectVersionAction),
BucketName: bucket,
ConditionValues: conditionValuesForAuth("", cred),
ObjectName: object,
IsOwner: owner,
Claims: cred.Claims,
DenyOnly: true,
}) { // Request is not allowed if Deny action on DeleteObjectVersionAction
return ErrAccessDenied
}
}
if globalIAMSys.IsAllowed(policy.Args{ if globalIAMSys.IsAllowed(policy.Args{
AccountName: cred.AccessKey, AccountName: cred.AccessKey,
Groups: cred.Groups, Groups: cred.Groups,
@@ -544,40 +553,6 @@ func authorizeRequestWithTags(ctx context.Context, r *http.Request, action polic
return ErrAccessDenied return ErrAccessDenied
} }
// authorizeReplicationDelete preserves the established target-credential
// contract for trusted replication: DeleteObject and ReplicateDelete must be
// allowed, while an explicit DeleteObjectVersion deny still blocks a named
// version. Ordinary S3 requests never use this compatibility path.
func authorizeReplicationDelete(ctx context.Context, r *http.Request) APIErrorCode {
if s3Err := authorizeRequest(ctx, r, policy.DeleteObjectAction); s3Err != ErrNone {
return s3Err
}
reqInfo := logger.GetReqInfo(ctx)
if reqInfo == nil {
return ErrAccessDenied
}
if reqInfo.VersionID == "" {
return ErrNone
}
cred := reqInfo.Cred
values := getConditionValuesWithTags(r, "", cred, "", nil)
values["versionid"] = []string{reqInfo.VersionID}
if !globalIAMSys.IsAllowed(policy.Args{
AccountName: cred.AccessKey,
Groups: cred.Groups,
Action: policy.DeleteObjectVersionAction,
BucketName: reqInfo.BucketName,
ConditionValues: values,
ObjectName: reqInfo.ObjectName,
IsOwner: reqInfo.Owner,
Claims: cred.Claims,
DenyOnly: true,
}) {
return ErrAccessDenied
}
return ErrNone
}
// Check request auth type verifies the incoming http request // Check request auth type verifies the incoming http request
// - validates the request signature // - validates the request signature
// - validates the policy action if anonymous tests bucket policies if any, // - validates the policy action if anonymous tests bucket policies if any,
@@ -811,20 +786,10 @@ func isPutActionAllowedWithRequestTags(ctx context.Context, atype authType, buck
return s3Err return s3Err
} }
reqInfo := logger.GetReqInfo(ctx) logger.GetReqInfo(ctx).Cred = cred
if reqInfo == nil { logger.GetReqInfo(ctx).Owner = owner
return ErrAccessDenied logger.GetReqInfo(ctx).Region = region
}
reqInfo.Lock()
reqInfo.Cred = cred
reqInfo.Owner = owner
reqInfo.Region = region
reqInfo.Unlock()
return isPutActionAllowedWithCred(bucketName, objectName, r, action, requestTags, cred, owner)
}
func isPutActionAllowedWithCred(bucketName, objectName string, r *http.Request, action policy.Action, requestTags *string, cred auth.Credentials, owner bool) APIErrorCode {
// Do not check for PutObjectRetentionAction permission, // Do not check for PutObjectRetentionAction permission,
// if mode and retain until date are not set. // if mode and retain until date are not set.
// Can happen when bucket has default lock config set // Can happen when bucket has default lock config set
+3 -3
View File
@@ -391,7 +391,7 @@ func (r *BatchJobReplicateV1) StartFromSource(ctx context.Context, api ObjectLay
return err return err
} }
c.SetAppInfo("silo-"+batchJobPrefix, r.APIVersion+" "+job.ID) c.SetAppInfo("minio-"+batchJobPrefix, r.APIVersion+" "+job.ID)
core := &minio.Core{Client: c} core := &minio.Core{Client: c}
workerSize, err := strconv.Atoi(env.Get("_MINIO_BATCH_REPLICATION_WORKERS", strconv.Itoa(runtime.GOMAXPROCS(0)/2))) workerSize, err := strconv.Atoi(env.Get("_MINIO_BATCH_REPLICATION_WORKERS", strconv.Itoa(runtime.GOMAXPROCS(0)/2)))
@@ -1155,7 +1155,7 @@ func (r *BatchJobReplicateV1) Start(ctx context.Context, api ObjectLayer, job Ba
return err return err
} }
c.SetAppInfo("silo-"+batchJobPrefix, r.APIVersion+" "+job.ID) c.SetAppInfo("minio-"+batchJobPrefix, r.APIVersion+" "+job.ID)
retry := false retry := false
for attempts := 1; attempts <= retryAttempts; attempts++ { for attempts := 1; attempts <= retryAttempts; attempts++ {
@@ -1477,7 +1477,7 @@ func (r *BatchJobReplicateV1) Validate(ctx context.Context, job BatchJobRequest,
if err != nil { if err != nil {
return err return err
} }
c.SetAppInfo("silo-"+batchJobPrefix, r.APIVersion+" "+job.ID) c.SetAppInfo("minio-"+batchJobPrefix, r.APIVersion+" "+job.ID)
vcfg, err := c.GetBucketVersioning(ctx, remoteBkt) vcfg, err := c.GetBucketVersioning(ctx, remoteBkt)
if err != nil { if err != nil {
+1 -1
View File
@@ -53,7 +53,7 @@ type ServerSystemConfig struct {
// Diff - returns error on first difference found in two configs. // Diff - returns error on first difference found in two configs.
func (s1 *ServerSystemConfig) Diff(s2 *ServerSystemConfig) error { func (s1 *ServerSystemConfig) Diff(s2 *ServerSystemConfig) error {
if s1.Checksum != s2.Checksum { if s1.Checksum != s2.Checksum {
return fmt.Errorf("Expected Silo binary checksum: %s, seen: %s", s1.Checksum, s2.Checksum) return fmt.Errorf("Expected MinIO binary checksum: %s, seen: %s", s1.Checksum, s2.Checksum)
} }
ns1 := s1.NEndpoints ns1 := s1.NEndpoints
-235
View File
@@ -1,235 +0,0 @@
// Copyright (c) 2015-2021 MinIO, Inc.
//
// This file is part of MinIO Object Storage stack
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
package cmd
import (
"bytes"
"encoding/base64"
"encoding/binary"
"hash/crc32"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/minio/minio/internal/auth"
)
func TestPutBucketCorsWireValidation(t *testing.T) {
ExecObjectLayerAPITest(ExecObjectLayerAPITestArgs{
t: t,
objAPITest: testPutBucketCorsWireValidation,
endpoints: []string{"PutBucketCors"},
})
}
func testPutBucketCorsWireValidation(_ ObjectLayer, _ string, bucketName string, apiRouter http.Handler,
creds auth.Credentials, t *testing.T,
) {
valid := `<CORSConfiguration><CORSRule><AllowedOrigin>*</AllowedOrigin><AllowedMethod>GET</AllowedMethod></CORSRule></CORSConfiguration>`
rule := `<CORSRule><AllowedOrigin>*</AllowedOrigin><AllowedMethod>GET</AllowedMethod></CORSRule>`
tests := []struct {
name string
body string
want int
wantCode string
}{
{
name: "second XML root",
body: valid + `<Extra/>`,
want: http.StatusBadRequest,
wantCode: "MalformedXML",
},
{
name: "255 Unicode character ID",
body: `<CORSConfiguration><CORSRule><ID>` + strings.Repeat("界", 255) + `</ID><AllowedOrigin>*</AllowedOrigin><AllowedMethod>GET</AllowedMethod></CORSRule></CORSConfiguration>`,
want: http.StatusOK,
},
{
name: "256 Unicode character ID",
body: `<CORSConfiguration><CORSRule><ID>` + strings.Repeat("界", 256) + `</ID><AllowedOrigin>*</AllowedOrigin><AllowedMethod>GET</AllowedMethod></CORSRule></CORSConfiguration>`,
want: http.StatusBadRequest,
wantCode: "MalformedXML",
},
{
name: "lowercase method",
body: `<CORSConfiguration><CORSRule><AllowedOrigin>*</AllowedOrigin><AllowedMethod>get</AllowedMethod></CORSRule></CORSConfiguration>`,
want: http.StatusBadRequest,
wantCode: "MalformedXML",
},
{
name: "empty origin",
body: `<CORSConfiguration><CORSRule><AllowedOrigin/><AllowedMethod>GET</AllowedMethod></CORSRule></CORSConfiguration>`,
want: http.StatusBadRequest,
wantCode: "MalformedXML",
},
{
name: "question mark origin wildcard",
body: `<CORSConfiguration><CORSRule><AllowedOrigin>https://?.example.com</AllowedOrigin><AllowedMethod>GET</AllowedMethod></CORSRule></CORSConfiguration>`,
want: http.StatusBadRequest,
wantCode: "MalformedXML",
},
{
name: "question mark header wildcard",
body: `<CORSConfiguration><CORSRule><AllowedOrigin>*</AllowedOrigin><AllowedMethod>GET</AllowedMethod><AllowedHeader>x-amz-?</AllowedHeader></CORSRule></CORSConfiguration>`,
want: http.StatusBadRequest,
wantCode: "MalformedXML",
},
{
name: "unknown element",
body: `<CORSConfiguration><CORSRule><AllowedOrigin>*</AllowedOrigin><AllowedMethod>GET</AllowedMethod><Unknown/></CORSRule></CORSConfiguration>`,
want: http.StatusBadRequest,
wantCode: "MalformedXML",
},
{
name: "empty max age",
body: `<CORSConfiguration><CORSRule><AllowedOrigin>*</AllowedOrigin><AllowedMethod>GET</AllowedMethod><MaxAgeSeconds/></CORSRule></CORSConfiguration>`,
want: http.StatusBadRequest,
wantCode: "MalformedXML",
},
{
name: "zero max age",
body: `<CORSConfiguration><CORSRule><AllowedOrigin>*</AllowedOrigin><AllowedMethod>GET</AllowedMethod><MaxAgeSeconds>0</MaxAgeSeconds></CORSRule></CORSConfiguration>`,
want: http.StatusOK,
},
{
name: "max age int32 overflow",
body: `<CORSConfiguration><CORSRule><AllowedOrigin>*</AllowedOrigin><AllowedMethod>GET</AllowedMethod><MaxAgeSeconds>2147483648</MaxAgeSeconds></CORSRule></CORSConfiguration>`,
want: http.StatusBadRequest,
wantCode: "MalformedXML",
},
{
name: "100 rules",
body: `<CORSConfiguration>` + strings.Repeat(rule, 100) + `</CORSConfiguration>`,
want: http.StatusOK,
},
{
name: "101 rules",
body: `<CORSConfiguration>` + strings.Repeat(rule, 101) + `</CORSConfiguration>`,
want: http.StatusBadRequest,
wantCode: "MalformedXML",
},
{
name: "exactly 64 KiB",
body: sizedCORSConfig(maxBucketCorsSize),
want: http.StatusOK,
},
{
name: "over 64 KiB",
body: sizedCORSConfig(maxBucketCorsSize + 1),
want: http.StatusBadRequest,
wantCode: "EntityTooLarge",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
req, err := newTestSignedRequestV4(http.MethodPut, getBucketCorsURL("", bucketName),
int64(len(tt.body)), bytes.NewReader([]byte(tt.body)), creds.AccessKey, creds.SecretKey, nil)
if err != nil {
t.Fatal(err)
}
rec := httptest.NewRecorder()
apiRouter.ServeHTTP(rec, req)
if rec.Code != tt.want {
t.Fatalf("expected status %d, got %d: %s", tt.want, rec.Code, rec.Body.String())
}
if tt.wantCode != "" && !bytes.Contains(rec.Body.Bytes(), []byte("<Code>"+tt.wantCode+"</Code>")) {
t.Fatalf("expected error code %s, got: %s", tt.wantCode, rec.Body.String())
}
})
}
}
func sizedCORSConfig(size int) string {
prefix := `<CORSConfiguration><CORSRule><AllowedOrigin>*</AllowedOrigin><AllowedMethod>GET</AllowedMethod>`
suffix := `</CORSRule></CORSConfiguration>`
return prefix + strings.Repeat(" ", size-len(prefix)-len(suffix)) + suffix
}
func TestPutBucketCorsChecksumValidation(t *testing.T) {
ExecObjectLayerAPITest(ExecObjectLayerAPITestArgs{
t: t,
objAPITest: testPutBucketCorsChecksumValidation,
endpoints: []string{"PutBucketCors"},
})
}
func testPutBucketCorsChecksumValidation(_ ObjectLayer, _ string, bucketName string, apiRouter http.Handler,
creds auth.Credentials, t *testing.T,
) {
body := []byte(`<CORSConfiguration><CORSRule><AllowedOrigin>*</AllowedOrigin><AllowedMethod>GET</AllowedMethod></CORSRule></CORSConfiguration>`)
tests := []struct {
name string
configure func(*http.Request)
want int
wantCode string
}{
{
name: "missing checksum",
configure: func(req *http.Request) {
req.Header.Del("Content-Md5")
},
want: http.StatusBadRequest,
wantCode: "MissingContentMD5",
},
{
name: "bad content md5",
configure: func(req *http.Request) {
req.Header.Set("Content-Md5", getMD5HashBase64([]byte("different body")))
},
want: http.StatusBadRequest,
wantCode: "BadDigest",
},
{
name: "valid sdk crc32",
configure: func(req *http.Request) {
req.Header.Del("Content-Md5")
req.Header.Set("X-Amz-Sdk-Checksum-Algorithm", "CRC32")
req.Header.Set("X-Amz-Checksum-Crc32", corsCRC32Base64(body))
},
want: http.StatusOK,
},
{
name: "bad sdk crc32",
configure: func(req *http.Request) {
req.Header.Del("Content-Md5")
req.Header.Set("X-Amz-Sdk-Checksum-Algorithm", "CRC32")
req.Header.Set("X-Amz-Checksum-Crc32", corsCRC32Base64([]byte("different body")))
},
want: http.StatusBadRequest,
wantCode: "BadDigest",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
req, err := newTestRequest(http.MethodPut, getBucketCorsURL("", bucketName), int64(len(body)), bytes.NewReader(body))
if err != nil {
t.Fatal(err)
}
tt.configure(req)
if err = signRequestV4(req, creds.AccessKey, creds.SecretKey); err != nil {
t.Fatal(err)
}
rec := httptest.NewRecorder()
apiRouter.ServeHTTP(rec, req)
if rec.Code != tt.want || (tt.wantCode != "" && !bytes.Contains(rec.Body.Bytes(), []byte("<Code>"+tt.wantCode+"</Code>"))) {
t.Fatalf("expected status %d and code %s, got %d: %s", tt.want, tt.wantCode, rec.Code, rec.Body.String())
}
})
}
}
func corsCRC32Base64(data []byte) string {
var checksum [4]byte
binary.BigEndian.PutUint32(checksum[:], crc32.ChecksumIEEE(data))
return base64.StdEncoding.EncodeToString(checksum[:])
}
-203
View File
@@ -1,203 +0,0 @@
// Copyright (c) 2015-2021 MinIO, Inc.
//
// This file is part of MinIO Object Storage stack
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package cmd
import (
"bytes"
"encoding/base64"
"errors"
"io"
"net/http"
humanize "github.com/dustin/go-humanize"
"github.com/minio/madmin-go/v3"
"github.com/minio/minio/internal/bucket/cors"
hashpkg "github.com/minio/minio/internal/hash"
"github.com/minio/minio/internal/logger"
"github.com/minio/mux"
"github.com/minio/pkg/v3/policy"
)
// maxBucketCorsSize is the maximum allowed size of a CORS configuration document.
const maxBucketCorsSize = 64 * humanize.KiByte
// PutBucketCorsHandler - PUT bucket cors.
func (api objectAPIHandlers) PutBucketCorsHandler(w http.ResponseWriter, r *http.Request) {
ctx := newContext(r, w, "PutBucketCors")
defer logger.AuditLog(ctx, w, r, mustGetClaimsFromToken(r))
objAPI := api.ObjectAPI()
if objAPI == nil {
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(ErrServerNotInitialized), r.URL)
return
}
vars := mux.Vars(r)
bucket := vars["bucket"]
if s3Error := checkRequestAuthType(ctx, r, policy.PutBucketCorsAction, bucket, ""); s3Error != ErrNone {
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(s3Error), r.URL)
return
}
if _, err := objAPI.GetBucketInfo(ctx, bucket, BucketOptions{}); err != nil {
writeErrorResponse(ctx, w, toAPIError(ctx, err), r.URL)
return
}
if r.ContentLength <= 0 {
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(ErrMissingContentLength), r.URL)
return
}
if r.ContentLength > maxBucketCorsSize {
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(ErrEntityTooLarge), r.URL)
return
}
// PutBucketCors requires a Content-Md5 or a supported full-header
// checksum. validateLengthAndChecksum wraps r.Body so the supplied digest
// is verified as the body is read below.
if !validateLengthAndChecksum(r) {
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(ErrMissingContentMD5), r.URL)
return
}
corsBytes, err := io.ReadAll(r.Body)
if err != nil {
if errors.Is(err, hashpkg.ErrInvalidChecksum) {
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(ErrBadDigest), r.URL)
return
}
writeErrorResponse(ctx, w, toAPIError(ctx, err), r.URL)
return
}
corsCfg, err := cors.ParseBucketCorsConfig(bytes.NewReader(corsBytes))
if err != nil {
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(ErrMalformedXML), r.URL)
return
}
if err := corsCfg.Validate(); err != nil {
writeErrorResponse(ctx, w, APIError{
Code: "MalformedXML",
HTTPStatusCode: http.StatusBadRequest,
Description: err.Error(),
}, r.URL)
return
}
updatedAt, err := updateLocalBucketCORSMetadata(ctx, objAPI, bucket, corsBytes)
if err != nil {
writeErrorResponse(ctx, w, toAPIError(ctx, err), r.URL)
return
}
// Call site replication hook.
//
// We encode the xml bytes as base64 to ensure there are no encoding
// errors.
cfgStr := base64.StdEncoding.EncodeToString(corsBytes)
replLogIf(ctx, globalSiteReplicationSys.BucketMetaHook(ctx, madmin.SRBucketMeta{
Type: madmin.SRBucketMetaTypeCorsConfig,
Bucket: bucket,
Cors: &cfgStr,
UpdatedAt: updatedAt,
}))
writeSuccessResponseHeadersOnly(w)
}
// GetBucketCorsHandler - GET bucket cors.
func (api objectAPIHandlers) GetBucketCorsHandler(w http.ResponseWriter, r *http.Request) {
ctx := newContext(r, w, "GetBucketCors")
defer logger.AuditLog(ctx, w, r, mustGetClaimsFromToken(r))
objAPI := api.ObjectAPI()
if objAPI == nil {
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(ErrServerNotInitialized), r.URL)
return
}
vars := mux.Vars(r)
bucket := vars["bucket"]
if s3Error := checkRequestAuthType(ctx, r, policy.GetBucketCorsAction, bucket, ""); s3Error != ErrNone {
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(s3Error), r.URL)
return
}
if _, err := objAPI.GetBucketInfo(ctx, bucket, BucketOptions{}); err != nil {
writeErrorResponse(ctx, w, toAPIError(ctx, err), r.URL)
return
}
configData, _, err := globalBucketMetadataSys.GetCorsConfigXML(bucket)
if err != nil {
if errors.Is(err, errConfigNotFound) {
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(ErrNoSuchCORSConfiguration), r.URL)
return
}
writeErrorResponse(ctx, w, toAPIError(ctx, err), r.URL)
return
}
writeSuccessResponseXML(w, configData)
}
// DeleteBucketCorsHandler - DELETE bucket cors.
func (api objectAPIHandlers) DeleteBucketCorsHandler(w http.ResponseWriter, r *http.Request) {
ctx := newContext(r, w, "DeleteBucketCors")
defer logger.AuditLog(ctx, w, r, mustGetClaimsFromToken(r))
objAPI := api.ObjectAPI()
if objAPI == nil {
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(ErrServerNotInitialized), r.URL)
return
}
vars := mux.Vars(r)
bucket := vars["bucket"]
if s3Error := checkRequestAuthType(ctx, r, policy.DeleteBucketCorsAction, bucket, ""); s3Error != ErrNone {
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(s3Error), r.URL)
return
}
if _, err := objAPI.GetBucketInfo(ctx, bucket, BucketOptions{}); err != nil {
writeErrorResponse(ctx, w, toAPIError(ctx, err), r.URL)
return
}
updatedAt, err := updateLocalBucketCORSMetadata(ctx, objAPI, bucket, nil)
if err != nil {
writeErrorResponse(ctx, w, toAPIError(ctx, err), r.URL)
return
}
replLogIf(ctx, globalSiteReplicationSys.BucketMetaHook(ctx, madmin.SRBucketMeta{
Type: madmin.SRBucketMetaTypeCorsConfig,
Bucket: bucket,
Cors: nil,
UpdatedAt: updatedAt,
}))
writeSuccessNoContent(w)
}
-164
View File
@@ -1,164 +0,0 @@
// Copyright (c) 2015-2021 MinIO, Inc.
//
// This file is part of MinIO Object Storage stack
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package cmd
import (
"bytes"
"net/http"
"net/http/httptest"
"testing"
"github.com/minio/minio/internal/auth"
)
const testCORSDoc = `<CORSConfiguration><CORSRule><AllowedOrigin>http://example.com</AllowedOrigin><AllowedMethod>GET</AllowedMethod><AllowedMethod>PUT</AllowedMethod><ExposeHeader>ETag</ExposeHeader><MaxAgeSeconds>3000</MaxAgeSeconds></CORSRule></CORSConfiguration>`
func TestBucketCorsHandlers(t *testing.T) {
ExecObjectLayerAPITest(ExecObjectLayerAPITestArgs{t: t, objAPITest: testBucketCorsHandlers, endpoints: []string{"PutBucketCors", "GetBucketCors", "DeleteBucketCors"}})
}
func testBucketCorsHandlers(obj ObjectLayer, instanceType, bucketName string, apiRouter http.Handler,
creds auth.Credentials, t *testing.T,
) {
// PUT
req, err := newTestSignedRequestV4(http.MethodPut, getBucketCorsURL("", bucketName),
int64(len(testCORSDoc)), bytes.NewReader([]byte(testCORSDoc)), creds.AccessKey, creds.SecretKey, nil)
if err != nil {
t.Fatal(err)
}
rec := httptest.NewRecorder()
apiRouter.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("PUT cors: expected 200, got %d: %s", rec.Code, rec.Body.String())
}
// GET returns what we stored
req, err = newTestSignedRequestV4(http.MethodGet, getBucketCorsURL("", bucketName),
0, nil, creds.AccessKey, creds.SecretKey, nil)
if err != nil {
t.Fatal(err)
}
rec = httptest.NewRecorder()
apiRouter.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("GET cors: expected 200, got %d: %s", rec.Code, rec.Body.String())
}
if !bytes.Contains(rec.Body.Bytes(), []byte("http://example.com")) {
t.Fatalf("GET cors: body missing origin: %s", rec.Body.String())
}
// DELETE
req, err = newTestSignedRequestV4(http.MethodDelete, getBucketCorsURL("", bucketName),
0, nil, creds.AccessKey, creds.SecretKey, nil)
if err != nil {
t.Fatal(err)
}
rec = httptest.NewRecorder()
apiRouter.ServeHTTP(rec, req)
if rec.Code != http.StatusNoContent {
t.Fatalf("DELETE cors: expected 204, got %d", rec.Code)
}
// GET after delete → 404 NoSuchCORSConfiguration
req, err = newTestSignedRequestV4(http.MethodGet, getBucketCorsURL("", bucketName),
0, nil, creds.AccessKey, creds.SecretKey, nil)
if err != nil {
t.Fatal(err)
}
rec = httptest.NewRecorder()
apiRouter.ServeHTTP(rec, req)
if rec.Code != http.StatusNotFound {
t.Fatalf("GET cors after delete: expected 404, got %d", rec.Code)
}
// Malformed XML → 400
req, err = newTestSignedRequestV4(http.MethodPut, getBucketCorsURL("", bucketName),
int64(len("<bad>")), bytes.NewReader([]byte("<bad>")), creds.AccessKey, creds.SecretKey, nil)
if err != nil {
t.Fatal(err)
}
rec = httptest.NewRecorder()
apiRouter.ServeHTTP(rec, req)
if rec.Code != http.StatusBadRequest {
t.Fatalf("PUT malformed cors: expected 400, got %d", rec.Code)
}
// Missing Content-MD5 is rejected before the body is parsed.
req, err = newTestRequest(http.MethodPut, getBucketCorsURL("", bucketName),
int64(len(testCORSDoc)), bytes.NewReader([]byte(testCORSDoc)))
if err != nil {
t.Fatal(err)
}
req.Header.Del("Content-Md5")
if err = signRequestV4(req, creds.AccessKey, creds.SecretKey); err != nil {
t.Fatal(err)
}
rec = httptest.NewRecorder()
apiRouter.ServeHTTP(rec, req)
if rec.Code != http.StatusBadRequest || !bytes.Contains(rec.Body.Bytes(), []byte("<Code>MissingContentMD5</Code>")) {
t.Fatalf("PUT cors without Content-MD5: expected MissingContentMD5, got %d: %s", rec.Code, rec.Body.String())
}
// A signed but incorrect Content-MD5 is rejected while reading the body.
req, err = newTestRequest(http.MethodPut, getBucketCorsURL("", bucketName),
int64(len(testCORSDoc)), bytes.NewReader([]byte(testCORSDoc)))
if err != nil {
t.Fatal(err)
}
req.Header.Set("Content-Md5", getMD5HashBase64([]byte("different body")))
if err = signRequestV4(req, creds.AccessKey, creds.SecretKey); err != nil {
t.Fatal(err)
}
rec = httptest.NewRecorder()
apiRouter.ServeHTTP(rec, req)
if rec.Code != http.StatusBadRequest || !bytes.Contains(rec.Body.Bytes(), []byte("<Code>BadDigest</Code>")) {
t.Fatalf("PUT cors with bad Content-MD5: expected BadDigest, got %d: %s", rec.Code, rec.Body.String())
}
// Re-PUT the config so the store→GetCorsConfig→enforce seam below has
// something to enforce (the earlier DELETE removed it).
req, err = newTestSignedRequestV4(http.MethodPut, getBucketCorsURL("", bucketName),
int64(len(testCORSDoc)), bytes.NewReader([]byte(testCORSDoc)), creds.AccessKey, creds.SecretKey, nil)
if err != nil {
t.Fatal(err)
}
rec = httptest.NewRecorder()
apiRouter.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("PUT cors (re-put): expected 200, got %d: %s", rec.Code, rec.Body.String())
}
// End-to-end enforcement: drive an OPTIONS preflight through the real
// corsHandler wrapper (not applyBucketCors in isolation), exercising the
// full store -> globalBucketMetadataSys.GetCorsConfig -> enforce seam.
wrapped := corsHandler(apiRouter)
preflightURL := getBucketCorsURL("", bucketName)
preflightReq := httptest.NewRequest(http.MethodOptions, preflightURL, nil)
preflightReq.Header.Set("Origin", "http://example.com")
preflightReq.Header.Set("Access-Control-Request-Method", http.MethodGet)
rec = httptest.NewRecorder()
wrapped.ServeHTTP(rec, preflightReq)
if rec.Code != http.StatusOK {
t.Fatalf("OPTIONS preflight via corsHandler: expected 200, got %d: %s", rec.Code, rec.Body.String())
}
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "http://example.com" {
t.Fatalf("OPTIONS preflight via corsHandler: expected Access-Control-Allow-Origin echoed, got %q", got)
}
}
-740
View File
@@ -1,740 +0,0 @@
// Copyright (c) 2015-2021 MinIO, Inc.
//
// This file is part of MinIO Object Storage stack
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package cmd
import (
"context"
"fmt"
"net/http"
"net/http/httptest"
"strings"
"sync/atomic"
"testing"
"github.com/minio/minio/internal/auth"
"github.com/minio/minio/internal/bucket/cors"
)
type corsLookupCountingObjectLayer struct {
ObjectLayer
getObjectNInfoCalls atomic.Int64
}
func (o *corsLookupCountingObjectLayer) GetObjectNInfo(ctx context.Context, bucket, object string, rs *HTTPRangeSpec, h http.Header, opts ObjectOptions) (*GetObjectReader, error) {
o.getObjectNInfoCalls.Add(1)
return o.ObjectLayer.GetObjectNInfo(ctx, bucket, object, rs, h, opts)
}
func TestPerBucketCorsPreflight(t *testing.T) {
cfg := &cors.Config{CORSRules: []cors.Rule{{
AllowedOrigins: []string{"http://example.com"},
AllowedMethods: []string{"GET", "PUT"},
AllowedHeaders: []string{"*"},
ExposeHeaders: []string{"ETag"},
MaxAgeSeconds: 3000,
}}}
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodOptions, "/mybucket/obj", nil)
req.Header.Set("Origin", "http://example.com")
req.Header.Set("Access-Control-Request-Method", "GET")
req.Header.Set("Access-Control-Request-Headers", "X-Amz-Date")
handled := applyBucketCors(rec, req, cfg)
if !handled {
t.Fatal("expected preflight to be handled")
}
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "http://example.com" {
t.Fatalf("allow-origin = %q", got)
}
if got := rec.Header().Get("Access-Control-Allow-Credentials"); got != "true" {
t.Fatalf("allow-credentials = %q", got)
}
if got := rec.Header().Get("Access-Control-Allow-Methods"); got != "GET, PUT" {
t.Fatalf("allow-methods = %q", got)
}
if got := rec.Header().Get("Access-Control-Allow-Headers"); got != "X-Amz-Date" {
t.Fatalf("allow-headers = %q", got)
}
if got := rec.Header().Get("Access-Control-Expose-Headers"); got != "ETag" {
t.Fatalf("expose-headers = %q", got)
}
if got := rec.Header().Get("Access-Control-Max-Age"); got != "3000" {
t.Fatalf("max-age = %q", got)
}
requireCorsVary(t, rec.Header())
if rec.Code != http.StatusOK {
t.Fatalf("preflight status = %d", rec.Code)
}
requireCorsOriginVary(t, rec.Header())
}
func TestPerBucketCorsActualRequestNoMatchVariesByOrigin(t *testing.T) {
cfg := &cors.Config{CORSRules: []cors.Rule{{
AllowedOrigins: []string{"https://allowed.example.com"},
AllowedMethods: []string{"GET"},
}}}
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodGet, "/mybucket/obj", nil)
req.Header.Set("Origin", "https://denied.example.com")
if handled := applyBucketCors(rec, req, cfg); handled {
t.Fatal("actual request must continue when CORS does not match")
}
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "" {
t.Fatalf("allow-origin = %q", got)
}
requireCorsOriginVary(t, rec.Header())
}
func TestPerBucketCorsPreflightNoMatch(t *testing.T) {
cfg := &cors.Config{CORSRules: []cors.Rule{{
AllowedOrigins: []string{"http://example.com"},
AllowedMethods: []string{"GET"},
}}}
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodOptions, "/mybucket/obj", nil)
req.Header.Set("Origin", "http://evil.com")
req.Header.Set("Access-Control-Request-Method", "GET")
handled := applyBucketCors(rec, req, cfg)
if !handled {
t.Fatal("expected preflight to be handled (rejected)")
}
if rec.Code != http.StatusForbidden {
t.Fatalf("expected 403 for disallowed origin, got %d", rec.Code)
}
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "" {
t.Fatalf("rejected preflight returned allow-origin %q", got)
}
requireCorsVary(t, rec.Header())
}
func TestPerBucketCorsPreflightWildcardOriginAndZeroMaxAge(t *testing.T) {
doc := `<CORSConfiguration><CORSRule><AllowedOrigin>*</AllowedOrigin><AllowedMethod>GET</AllowedMethod><AllowedMethod>HEAD</AllowedMethod><AllowedHeader>*</AllowedHeader><ExposeHeader>ETag</ExposeHeader><MaxAgeSeconds>0</MaxAgeSeconds></CORSRule></CORSConfiguration>`
cfg, err := cors.ParseBucketCorsConfig(strings.NewReader(doc))
if err != nil {
t.Fatal(err)
}
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodOptions, "/mybucket/obj", nil)
req.Header.Set("Origin", "https://app.example.com")
req.Header.Set("Access-Control-Request-Method", "GET")
req.Header.Set("Access-Control-Request-Headers", "RANGE")
if handled := applyBucketCors(rec, req, cfg); !handled {
t.Fatal("expected preflight to be handled")
}
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "*" {
t.Fatalf("allow-origin = %q", got)
}
if got := rec.Header().Get("Access-Control-Allow-Credentials"); got != "" {
t.Fatalf("allow-credentials = %q", got)
}
if got := rec.Header().Get("Access-Control-Allow-Methods"); got != "GET, HEAD" {
t.Fatalf("allow-methods = %q", got)
}
if got := rec.Header().Get("Access-Control-Allow-Headers"); got != "RANGE" {
t.Fatalf("allow-headers = %q", got)
}
if got := rec.Header().Get("Access-Control-Expose-Headers"); got != "ETag" {
t.Fatalf("expose-headers = %q", got)
}
if got := rec.Header().Get("Access-Control-Max-Age"); got != "0" {
t.Fatalf("max-age = %q", got)
}
requireCorsVary(t, rec.Header())
}
func TestPerBucketCorsPreflightUsesFirstFullyMatchingRule(t *testing.T) {
cfg := &cors.Config{CORSRules: []cors.Rule{
{
AllowedOrigins: []string{"https://app.example.com"},
AllowedMethods: []string{"GET"},
AllowedHeaders: []string{"x-a"},
ExposeHeaders: []string{"x-rule-a"},
MaxAgeSeconds: 1,
},
{
AllowedOrigins: []string{"https://app.example.com"},
AllowedMethods: []string{"GET", "HEAD"},
AllowedHeaders: []string{"*"},
ExposeHeaders: []string{"x-rule-b"},
MaxAgeSeconds: 2,
},
}}
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodOptions, "/mybucket/obj", nil)
req.Header.Set("Origin", "https://app.example.com")
req.Header.Set("Access-Control-Request-Method", "GET")
req.Header.Set("Access-Control-Request-Headers", "X-B")
if handled := applyBucketCors(rec, req, cfg); !handled {
t.Fatal("expected preflight to be handled")
}
if got := rec.Header().Get("Access-Control-Expose-Headers"); got != "x-rule-b" {
t.Fatalf("selected rule expose-headers = %q", got)
}
if got := rec.Header().Get("Access-Control-Allow-Methods"); got != "GET, HEAD" {
t.Fatalf("selected rule allow-methods = %q", got)
}
if got := rec.Header().Get("Access-Control-Max-Age"); got != "2" {
t.Fatalf("selected rule max-age = %q", got)
}
}
func TestPerBucketCorsActualRequest(t *testing.T) {
cfg := &cors.Config{CORSRules: []cors.Rule{{
AllowedOrigins: []string{"*"},
AllowedMethods: []string{"GET"},
ExposeHeaders: []string{"ETag"},
}}}
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodGet, "/mybucket/obj", nil)
req.Header.Set("Origin", "http://any.com")
handled := applyBucketCors(rec, req, cfg)
if handled {
t.Fatal("actual (non-preflight) request must not be terminated by CORS")
}
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "*" {
t.Fatalf("allow-origin = %q", got)
}
if got := rec.Header().Get("Access-Control-Allow-Credentials"); got != "" {
t.Fatalf("allow-credentials = %q", got)
}
if got := rec.Header().Get("Access-Control-Expose-Headers"); got != "ETag" {
t.Fatalf("expose-headers = %q", got)
}
}
func TestPerBucketCorsOriginPatternResponse(t *testing.T) {
cfg := &cors.Config{CORSRules: []cors.Rule{{
AllowedOrigins: []string{"https://app.example.com", "https://*", "*"},
AllowedMethods: []string{"GET"},
}}}
tests := []struct {
origin string
wantOrigin string
wantCredentials string
}{
{"https://app.example.com", "https://app.example.com", "true"},
{"https://other.example.com", "https://other.example.com", "true"},
{"http://other.example.com", "*", ""},
}
for _, tt := range tests {
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodGet, "/mybucket/obj", nil)
req.Header.Set("Origin", tt.origin)
if handled := applyBucketCors(rec, req, cfg); handled {
t.Fatal("actual request must not be terminated by CORS")
}
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != tt.wantOrigin {
t.Fatalf("origin %q: allow-origin = %q, want %q", tt.origin, got, tt.wantOrigin)
}
if got := rec.Header().Get("Access-Control-Allow-Credentials"); got != tt.wantCredentials {
t.Fatalf("origin %q: allow-credentials = %q, want %q", tt.origin, got, tt.wantCredentials)
}
}
}
func TestBucketCorsMetadataErrorFailsClosed(t *testing.T) {
oldObjectAPI := newObjectLayerFn()
oldMetadataSys := globalBucketMetadataSys
setObjectLayer(nil)
globalBucketMetadataSys = NewBucketMetadataSys()
defer func() {
setObjectLayer(oldObjectAPI)
globalBucketMetadataSys = oldMetadataSys
}()
wrapped := corsHandler(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusNoContent)
}))
for _, method := range []string{http.MethodGet, http.MethodOptions} {
rec := httptest.NewRecorder()
req := httptest.NewRequest(method, getGetObjectURL("", "cors-metadata-error", "object"), nil)
req.Header.Set("Origin", "https://app.example.com")
if method == http.MethodOptions {
req.Header.Set("Access-Control-Request-Method", http.MethodGet)
}
wrapped.ServeHTTP(rec, req)
if rec.Code != http.StatusNoContent {
t.Fatalf("%s status = %d, want %d", method, rec.Code, http.StatusNoContent)
}
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "" {
t.Fatalf("%s metadata error fell back to global allow-origin %q", method, got)
}
if got := rec.Header().Get("Access-Control-Allow-Credentials"); got != "" {
t.Fatalf("%s metadata error fell back to global credentials %q", method, got)
}
}
}
func TestBucketCorsSkipsMetadataLookupWithoutOrigin(t *testing.T) {
ExecObjectLayerAPITest(ExecObjectLayerAPITestArgs{
t: t,
objAPITest: testBucketCorsSkipsMetadataLookupWithoutOrigin,
endpoints: []string{"GetObject"},
})
}
func testBucketCorsSkipsMetadataLookupWithoutOrigin(obj ObjectLayer, _ string, _ string, _ http.Handler, _ auth.Credentials, t *testing.T) {
oldObjectAPI := newObjectLayerFn()
counting := &corsLookupCountingObjectLayer{ObjectLayer: obj}
setObjectLayer(counting)
defer setObjectLayer(oldObjectAPI)
wrapped := corsHandler(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusNoContent)
}))
rec := httptest.NewRecorder()
wrapped.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, getGetObjectURL("", "api", "v1/login"), nil))
if rec.Code != http.StatusNoContent {
t.Fatalf("status = %d, want %d", rec.Code, http.StatusNoContent)
}
requireCorsOriginVary(t, rec.Header())
if got := counting.getObjectNInfoCalls.Load(); got != 0 {
t.Fatalf("request without Origin performed %d bucket metadata reads", got)
}
}
func TestBucketCorsOriginlessPreflightShapeUsesGlobalHandler(t *testing.T) {
nextCalled := false
wrapped := corsHandler(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
nextCalled = true
w.WriteHeader(http.StatusTeapot)
}))
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodOptions, getGetObjectURL("", "api", "v1/login"), nil)
req.Header.Set("Access-Control-Request-Method", http.MethodGet)
wrapped.ServeHTTP(rec, req)
if rec.Code != http.StatusNoContent {
t.Fatalf("status = %d, want %d", rec.Code, http.StatusNoContent)
}
if nextCalled {
t.Fatal("originless preflight-shaped OPTIONS reached the application handler")
}
requireCorsOriginVary(t, rec.Header())
}
func TestBucketCorsNoConfigUsesGlobalFallback(t *testing.T) {
ExecObjectLayerAPITest(ExecObjectLayerAPITestArgs{
t: t,
objAPITest: testBucketCorsNoConfigUsesGlobalFallback,
endpoints: []string{"GetBucketCors"},
})
}
func TestBucketCorsMissingBucketUsesGlobalFallback(t *testing.T) {
ExecObjectLayerAPITest(ExecObjectLayerAPITestArgs{
t: t,
objAPITest: testBucketCorsMissingBucketUsesGlobalFallback,
endpoints: []string{"GetBucketCors"},
})
}
func testBucketCorsMissingBucketUsesGlobalFallback(obj ObjectLayer, _ string, bucket string, _ http.Handler, _ auth.Credentials, t *testing.T) {
// Model a fully started server: bucket metadata loading has completed, so
// a name that is not resident is genuinely not a CORS-bearing bucket.
restore := markBucketMetadataInitialized(t)
defer restore()
// A non-resident bucket name must not cause any bucket-metadata disk read.
oldObjectAPI := newObjectLayerFn()
counting := &corsLookupCountingObjectLayer{ObjectLayer: obj}
setObjectLayer(counting)
defer setObjectLayer(oldObjectAPI)
before := bucketMetadataMapLen()
wrapped := corsHandler(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusNotFound)
}))
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodGet, getGetObjectURL("", bucket+"-missing", "object"), nil)
req.Header.Set("Origin", "https://app.example.com")
wrapped.ServeHTTP(rec, req)
if rec.Code != http.StatusNotFound {
t.Fatalf("status = %d, want %d", rec.Code, http.StatusNotFound)
}
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "https://app.example.com" {
t.Fatalf("allow-origin = %q", got)
}
if got := rec.Header().Get("Access-Control-Allow-Credentials"); got != "true" {
t.Fatalf("allow-credentials = %q", got)
}
// Regression guard: the pre-auth CORS lookup for a non-existent bucket must
// neither read bucket metadata from disk nor cache a synthetic entry.
if got := counting.getObjectNInfoCalls.Load(); got != 0 {
t.Fatalf("missing-bucket CORS lookup performed %d bucket metadata reads", got)
}
if after := bucketMetadataMapLen(); after != before {
t.Fatalf("missing-bucket CORS lookup grew metadataMap from %d to %d", before, after)
}
}
func testBucketCorsNoConfigUsesGlobalFallback(_ ObjectLayer, _ string, bucket string, _ http.Handler, _ auth.Credentials, t *testing.T) {
wrapped := corsHandler(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusNoContent)
}))
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodGet, getGetObjectURL("", bucket, "object"), nil)
req.Header.Set("Origin", "https://app.example.com")
wrapped.ServeHTTP(rec, req)
if rec.Code != http.StatusNoContent {
t.Fatalf("status = %d, want %d", rec.Code, http.StatusNoContent)
}
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "https://app.example.com" {
t.Fatalf("allow-origin = %q", got)
}
if got := rec.Header().Get("Access-Control-Allow-Credentials"); got != "true" {
t.Fatalf("allow-credentials = %q", got)
}
}
func TestPerBucketCorsActualPatternOriginSupportsCredentials(t *testing.T) {
cfg := &cors.Config{CORSRules: []cors.Rule{{
AllowedOrigins: []string{"https://*.example.com"},
AllowedMethods: []string{"GET"},
}}}
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodGet, "/mybucket/obj", nil)
req.Header.Set("Origin", "https://app.example.com")
if handled := applyBucketCors(rec, req, cfg); handled {
t.Fatal("actual request must continue")
}
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "https://app.example.com" {
t.Fatalf("allow-origin = %q", got)
}
if got := rec.Header().Get("Access-Control-Allow-Credentials"); got != "true" {
t.Fatalf("allow-credentials = %q", got)
}
}
func TestPerBucketCorsActualNullOriginSurvivesForwardingMiddleware(t *testing.T) {
next := setBucketForwardingMiddleware(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusNoContent)
}))
t.Run("per-bucket null origin", func(t *testing.T) {
cfg := &cors.Config{CORSRules: []cors.Rule{{
AllowedOrigins: []string{"null"},
AllowedMethods: []string{"GET"},
}}}
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodGet, "/mybucket/obj", nil)
req.Header.Set("Origin", "null")
if handled := applyBucketCors(rec, req, cfg); handled {
t.Fatal("actual request must continue")
}
next.ServeHTTP(rec, req)
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "null" {
t.Fatalf("allow-origin = %q", got)
}
if got := rec.Header().Get("Access-Control-Allow-Credentials"); got != "true" {
t.Fatalf("allow-credentials = %q", got)
}
})
t.Run("legacy unmarked null origin", func(t *testing.T) {
rec := httptest.NewRecorder()
rec.Header().Set("Access-Control-Allow-Origin", "null")
req := httptest.NewRequest(http.MethodGet, "/mybucket/obj", nil)
next.ServeHTTP(rec, req)
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "*" {
t.Fatalf("allow-origin = %q", got)
}
})
}
func requireCorsVary(t *testing.T, header http.Header) {
t.Helper()
values := strings.Join(header.Values("Vary"), ",")
for _, want := range []string{"Origin", "Access-Control-Request-Method", "Access-Control-Request-Headers"} {
if !strings.Contains(values, want) {
t.Fatalf("Vary = %q, missing %q", values, want)
}
}
}
func requireCorsOriginVary(t *testing.T, header http.Header) {
t.Helper()
if values := strings.Join(header.Values("Vary"), ","); !strings.Contains(values, "Origin") {
t.Fatalf("Vary = %q, missing Origin", values)
}
}
// markBucketMetadataInitialized marks the global bucket-metadata subsystem as
// fully loaded, modeling a running server (the API test harness sets up the
// subsystem but does not run Init). It returns a function that restores the
// previous state.
func markBucketMetadataInitialized(t *testing.T) func() {
t.Helper()
sys := globalBucketMetadataSys
if sys == nil {
t.Fatal("globalBucketMetadataSys is nil")
}
sys.Lock()
prev := sys.initialized
sys.initialized = true
sys.Unlock()
return func() {
sys.Lock()
sys.initialized = prev
sys.Unlock()
}
}
// bucketMetadataMapLen returns the number of resident bucket-metadata entries.
func bucketMetadataMapLen() int {
sys := globalBucketMetadataSys
if sys == nil {
return 0
}
sys.RLock()
defer sys.RUnlock()
return len(sys.metadataMap)
}
// TestBucketCorsUnknownBucketDoesNotGrowMetadata is the regression guard for
// the pre-auth resource-exhaustion path: an unauthenticated, Origin-bearing
// request whose first path segment is not a real bucket must fall back to the
// global CORS policy without loading bucket metadata from disk and without
// caching a synthetic entry. Before the resident-only lookup, each distinct
// name grew metadataMap by one and issued an erasure metadata probe.
func TestBucketCorsUnknownBucketDoesNotGrowMetadata(t *testing.T) {
ExecObjectLayerAPITest(ExecObjectLayerAPITestArgs{
t: t,
objAPITest: testBucketCorsUnknownBucketDoesNotGrowMetadata,
endpoints: []string{"GetBucketCors"},
})
}
func testBucketCorsUnknownBucketDoesNotGrowMetadata(obj ObjectLayer, _ string, _ string, _ http.Handler, _ auth.Credentials, t *testing.T) {
restore := markBucketMetadataInitialized(t)
defer restore()
oldObjectAPI := newObjectLayerFn()
counting := &corsLookupCountingObjectLayer{ObjectLayer: obj}
setObjectLayer(counting)
defer setObjectLayer(oldObjectAPI)
wrapped := corsHandler(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusNotFound)
}))
before := bucketMetadataMapLen()
// Console/admin routes plus enough distinct valid names to make accidental
// cache growth or one metadata probe per name unambiguous.
names := []string{"minio", "api"}
for i := 0; i < 500; i++ {
names = append(names, fmt.Sprintf("cors-missing-%03d", i))
}
for _, name := range names {
for _, method := range []string{http.MethodGet, http.MethodOptions} {
rec := httptest.NewRecorder()
req := httptest.NewRequest(method, getGetObjectURL("", name, "obj"), nil)
req.Header.Set("Origin", "https://app.example.com")
if method == http.MethodOptions {
req.Header.Set("Access-Control-Request-Method", http.MethodGet)
}
wrapped.ServeHTTP(rec, req)
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "https://app.example.com" {
t.Fatalf("%s/%s: allow-origin = %q, want global fallback", name, method, got)
}
}
}
for _, path := range []string{"/../obj", "/A/obj", "/x/obj", "/minio/admin/v3/info", "/api/v1/login"} {
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodGet, path, nil)
req.Header.Set("Origin", "https://app.example.com")
wrapped.ServeHTTP(rec, req)
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "https://app.example.com" {
t.Fatalf("%s: allow-origin = %q, want global fallback", path, got)
}
}
if got := counting.getObjectNInfoCalls.Load(); got != 0 {
t.Fatalf("unknown-bucket CORS lookups performed %d bucket metadata reads", got)
}
if after := bucketMetadataMapLen(); after != before {
t.Fatalf("unknown-bucket CORS lookups grew metadataMap from %d to %d", before, after)
}
}
func TestBucketCorsStartupMissFailsClosedWithoutIO(t *testing.T) {
ExecObjectLayerAPITest(ExecObjectLayerAPITestArgs{
t: t,
objAPITest: testBucketCorsStartupMissFailsClosedWithoutIO,
endpoints: []string{"GetBucketCors"},
})
}
func testBucketCorsStartupMissFailsClosedWithoutIO(obj ObjectLayer, _ string, _ string, _ http.Handler, _ auth.Credentials, t *testing.T) {
oldObjectAPI := newObjectLayerFn()
oldMetadataSys := globalBucketMetadataSys
counting := &corsLookupCountingObjectLayer{ObjectLayer: obj}
setObjectLayer(counting)
globalBucketMetadataSys = NewBucketMetadataSys()
defer func() {
setObjectLayer(oldObjectAPI)
globalBucketMetadataSys = oldMetadataSys
}()
innerCalled := false
wrapped := corsHandler(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
innerCalled = true
w.WriteHeader(http.StatusNoContent)
}))
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodGet, "/startup-missing/object", nil)
req.Header.Set("Origin", "https://app.example.com")
wrapped.ServeHTTP(rec, req)
if !innerCalled || rec.Code != http.StatusNoContent {
t.Fatalf("startup miss did not reach inner handler: called=%v status=%d", innerCalled, rec.Code)
}
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "" {
t.Fatalf("startup miss used permissive global CORS: %q", got)
}
if got := counting.getObjectNInfoCalls.Load(); got != 0 {
t.Fatalf("startup miss performed %d metadata reads", got)
}
if got := globalBucketMetadataSys.Count(); got != 0 {
t.Fatalf("startup miss grew metadataMap to %d", got)
}
}
// markBucketMetadataLoadFailed records a bucket as one whose metadata failed to
// load at startup while the subsystem is Initialized, modeling the degraded
// state where a real bucket is not resident. Returns a restore function.
func markBucketMetadataLoadFailed(t *testing.T, bucket string) func() {
t.Helper()
sys := globalBucketMetadataSys
if sys == nil {
t.Fatal("globalBucketMetadataSys is nil")
}
sys.Lock()
_, had := sys.loadFailed[bucket]
sys.loadFailed[bucket] = struct{}{}
sys.Unlock()
return func() {
sys.Lock()
if !had {
delete(sys.loadFailed, bucket)
}
sys.Unlock()
}
}
// TestBucketCorsLoadFailedBucketFailsClosed guards P1: a real bucket whose
// metadata could not be loaded at startup (present in loadFailed, subsystem
// Initialized) must NOT be answered with the permissive global CORS policy. We
// cannot rule out a restrictive per-bucket config for it, so it must fail
// closed — without a synchronous disk read.
func TestBucketCorsLoadFailedBucketFailsClosed(t *testing.T) {
ExecObjectLayerAPITest(ExecObjectLayerAPITestArgs{
t: t,
objAPITest: testBucketCorsLoadFailedBucketFailsClosed,
endpoints: []string{"GetBucketCors"},
})
}
func testBucketCorsLoadFailedBucketFailsClosed(obj ObjectLayer, _ string, _ string, _ http.Handler, _ auth.Credentials, t *testing.T) {
restoreInit := markBucketMetadataInitialized(t)
defer restoreInit()
restoreFail := markBucketMetadataLoadFailed(t, "strict-cors-bucket")
defer restoreFail()
oldObjectAPI := newObjectLayerFn()
counting := &corsLookupCountingObjectLayer{ObjectLayer: obj}
setObjectLayer(counting)
defer setObjectLayer(oldObjectAPI)
wrapped := corsHandler(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusNotFound)
}))
for _, method := range []string{http.MethodGet, http.MethodOptions} {
rec := httptest.NewRecorder()
req := httptest.NewRequest(method, getGetObjectURL("", "strict-cors-bucket", "object"), nil)
req.Header.Set("Origin", "https://app.example.com")
if method == http.MethodOptions {
req.Header.Set("Access-Control-Request-Method", http.MethodGet)
}
wrapped.ServeHTTP(rec, req)
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "" {
t.Fatalf("%s: load-failed bucket fell back to global allow-origin %q", method, got)
}
if got := rec.Header().Get("Access-Control-Allow-Credentials"); got != "" {
t.Fatalf("%s: load-failed bucket fell back to global credentials %q", method, got)
}
}
if got := counting.getObjectNInfoCalls.Load(); got != 0 {
t.Fatalf("load-failed CORS lookup performed %d synchronous bucket metadata reads", got)
}
}
// TestBucketCorsInternalBucketFailsClosed guards P2: an Origin-bearing request
// whose first path segment is the reserved .minio.sys namespace must preserve
// GetConfig's errInvalidArgument semantics and fail closed, not fall back to
// the permissive global CORS policy.
func TestBucketCorsInternalBucketFailsClosed(t *testing.T) {
ExecObjectLayerAPITest(ExecObjectLayerAPITestArgs{
t: t,
objAPITest: testBucketCorsInternalBucketFailsClosed,
endpoints: []string{"GetBucketCors"},
})
}
func testBucketCorsInternalBucketFailsClosed(_ ObjectLayer, _ string, _ string, _ http.Handler, _ auth.Credentials, t *testing.T) {
restoreInit := markBucketMetadataInitialized(t)
defer restoreInit()
wrapped := corsHandler(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusNotFound)
}))
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodGet, getGetObjectURL("", minioMetaBucket, "object"), nil)
req.Header.Set("Origin", "https://app.example.com")
wrapped.ServeHTTP(rec, req)
if got := rec.Header().Get("Access-Control-Allow-Origin"); got != "" {
t.Fatalf("internal bucket fell back to global allow-origin %q", got)
}
if got := rec.Header().Get("Access-Control-Allow-Credentials"); got != "" {
t.Fatalf("internal bucket fell back to global credentials %q", got)
}
}
File diff suppressed because it is too large Load Diff
+16 -20
View File
@@ -463,20 +463,13 @@ func (api objectAPIHandlers) DeleteMultipleObjectsHandler(w http.ResponseWriter,
// Make sure to update context to print ObjectNames for multi objects. // Make sure to update context to print ObjectNames for multi objects.
ctx = updateReqContext(ctx, objects...) ctx = updateReqContext(ctx, objects...)
// Call checkRequestAuthType to populate ReqInfo.AccessKey before GetBucketInfo()
// Ignore errors here to preserve the S3 error behavior of GetBucketInfo()
checkRequestAuthType(ctx, r, policy.DeleteObjectAction, bucket, "")
deleteObjectsFn := objectAPI.DeleteObjects deleteObjectsFn := objectAPI.DeleteObjects
reqInfo := logger.GetReqInfo(ctx) // Return Malformed XML as S3 spec if the number of objects is empty
if reqInfo == nil {
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(ErrAccessDenied), r.URL)
return
}
reqInfo.BucketName = bucket
reqInfo.ObjectName = ""
if s3Err := authenticateRequest(ctx, r, policy.DeleteObjectAction); s3Err != ErrNone {
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(s3Err), r.URL)
return
}
// Return Malformed XML as S3 spec if the number of objects is empty.
if len(deleteObjectsReq.Objects) == 0 || len(deleteObjectsReq.Objects) > maxDeleteList { if len(deleteObjectsReq.Objects) == 0 || len(deleteObjectsReq.Objects) > maxDeleteList {
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(ErrMalformedXML), r.URL) writeErrorResponse(ctx, w, errorCodes.ToAPIErr(ErrMalformedXML), r.URL)
return return
@@ -506,9 +499,7 @@ func (api objectAPIHandlers) DeleteMultipleObjectsHandler(w http.ResponseWriter,
vc, _ := globalBucketVersioningSys.Get(bucket) vc, _ := globalBucketVersioningSys.Get(bucket)
oss := make([]*objSweeper, len(deleteObjectsReq.Objects)) oss := make([]*objSweeper, len(deleteObjectsReq.Objects))
for index, object := range deleteObjectsReq.Objects { for index, object := range deleteObjectsReq.Objects {
reqInfo.ObjectName = object.ObjectName if apiErrCode := checkRequestAuthTypeWithVID(ctx, r, policy.DeleteObjectAction, bucket, object.ObjectName, object.VersionID); apiErrCode != ErrNone {
reqInfo.VersionID = object.VersionID
if apiErrCode := authorizeRequest(ctx, r, deleteObjectAction(object.VersionID)); apiErrCode != ErrNone {
if apiErrCode == ErrSignatureDoesNotMatch || apiErrCode == ErrInvalidAccessKeyID { if apiErrCode == ErrSignatureDoesNotMatch || apiErrCode == ErrInvalidAccessKeyID {
writeErrorResponse(ctx, w, errorCodes.ToAPIErr(apiErrCode), r.URL) writeErrorResponse(ctx, w, errorCodes.ToAPIErr(apiErrCode), r.URL)
return return
@@ -1424,7 +1415,7 @@ func (api objectAPIHandlers) PostPolicyBucketHandler(w http.ResponseWriter, r *h
Object: ObjectInfo{Name: objInfo.Name}, Object: ObjectInfo{Name: objInfo.Name},
ReqParams: extractReqParams(r), ReqParams: extractReqParams(r),
RespElements: extractRespElements(w), RespElements: extractRespElements(w),
UserAgent: fmt.Sprintf("%s Silo-Fan-Out (failed: %v)", r.UserAgent(), errs[i]), UserAgent: fmt.Sprintf("%s MinIO-Fan-Out (failed: %v)", r.UserAgent(), errs[i]),
Host: handlers.GetSourceIP(r), Host: handlers.GetSourceIP(r),
}) })
continue continue
@@ -1443,7 +1434,7 @@ func (api objectAPIHandlers) PostPolicyBucketHandler(w http.ResponseWriter, r *h
Object: objInfo, Object: objInfo,
ReqParams: extractReqParams(r), ReqParams: extractReqParams(r),
RespElements: extractRespElements(w), RespElements: extractRespElements(w),
UserAgent: r.UserAgent() + " " + "Silo-Fan-Out", UserAgent: r.UserAgent() + " " + "MinIO-Fan-Out",
Host: handlers.GetSourceIP(r), Host: handlers.GetSourceIP(r),
}) })
} }
@@ -1471,7 +1462,7 @@ func (api objectAPIHandlers) PostPolicyBucketHandler(w http.ResponseWriter, r *h
Object: eventArgsList[i].Object, Object: eventArgsList[i].Object,
ReqParams: extractReqParams(r), ReqParams: extractReqParams(r),
RespElements: extractRespElements(w), RespElements: extractRespElements(w),
UserAgent: r.UserAgent() + " " + "Silo-Fan-Out", UserAgent: r.UserAgent() + " " + "MinIO-Fan-Out",
Host: handlers.GetSourceIP(r), Host: handlers.GetSourceIP(r),
}) })
@@ -1766,7 +1757,7 @@ func (api objectAPIHandlers) DeleteBucketHandler(w http.ResponseWriter, r *http.
if globalDNSConfig != nil { if globalDNSConfig != nil {
if err := globalDNSConfig.Delete(bucket); err != nil { if err := globalDNSConfig.Delete(bucket); err != nil {
dnsLogIf(ctx, fmt.Errorf("Unable to delete bucket DNS entry %w; please delete it manually because the bucket no longer exists on Silo", err)) dnsLogIf(ctx, fmt.Errorf("Unable to delete bucket DNS entry %w, please delete it manually, bucket on MinIO no longer exists", err))
writeErrorResponse(ctx, w, toAPIError(ctx, err), r.URL) writeErrorResponse(ctx, w, toAPIError(ctx, err), r.URL)
return return
} }
@@ -1853,7 +1844,12 @@ func (api objectAPIHandlers) PutBucketObjectLockConfigHandler(w http.ResponseWri
// We encode the xml bytes as base64 to ensure there are no encoding // We encode the xml bytes as base64 to ensure there are no encoding
// errors. // errors.
cfgStr := base64.StdEncoding.EncodeToString(configData) cfgStr := base64.StdEncoding.EncodeToString(configData)
replLogIf(ctx, globalSiteReplicationSys.BucketMetaHook(ctx, newSRBucketObjectLockMeta(bucket, &cfgStr, updatedAt))) replLogIf(ctx, globalSiteReplicationSys.BucketMetaHook(ctx, madmin.SRBucketMeta{
Type: madmin.SRBucketMetaTypeObjectLockConfig,
Bucket: bucket,
ObjectLockConfig: &cfgStr,
UpdatedAt: updatedAt,
}))
// Write success response. // Write success response.
writeSuccessResponseHeadersOnly(w) writeSuccessResponseHeadersOnly(w)
+18 -71
View File
@@ -24,52 +24,12 @@ import (
"io" "io"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"net/url"
"strconv" "strconv"
"testing" "testing"
"github.com/minio/minio/internal/auth" "github.com/minio/minio/internal/auth"
) )
func TestListObjectsNonExistentBucketHandler(t *testing.T) {
ExecObjectLayerAPITest(ExecObjectLayerAPITestArgs{t: t, objAPITest: testListObjectsNonExistentBucketHandler})
}
func testListObjectsNonExistentBucketHandler(_ ObjectLayer, instanceType, _ string, apiRouter http.Handler,
credentials auth.Credentials, t *testing.T,
) {
const bucket = "missing-bucket"
testCases := []struct {
name string
query url.Values
}{
{name: "ListObjects", query: url.Values{"prefix": {"/"}}},
{name: "ListObjectsV2", query: url.Values{"list-type": {"2"}, "prefix": {"/"}}},
{name: "ListObjectVersions", query: url.Values{"versions": {""}, "prefix": {"/"}}},
}
for _, tc := range testCases {
req, err := newTestSignedRequestV4(http.MethodGet, makeTestTargetURL("", bucket, "", tc.query), 0, nil,
credentials.AccessKey, credentials.SecretKey, nil)
if err != nil {
t.Fatalf("%s: %s: failed to create request: %v", instanceType, tc.name, err)
}
rec := httptest.NewRecorder()
apiRouter.ServeHTTP(rec, req)
if rec.Code != http.StatusNotFound {
t.Errorf("%s: %s: expected status %d, got %d", instanceType, tc.name, http.StatusNotFound, rec.Code)
}
var apiErr APIErrorResponse
if err = xml.Unmarshal(rec.Body.Bytes(), &apiErr); err != nil {
t.Fatalf("%s: %s: failed to decode error response: %v", instanceType, tc.name, err)
}
if apiErr.Code != "NoSuchBucket" {
t.Errorf("%s: %s: expected NoSuchBucket, got %q", instanceType, tc.name, apiErr.Code)
}
}
}
// Wrapper for calling RemoveBucket HTTP handler tests for both Erasure multiple disks and single node setup. // Wrapper for calling RemoveBucket HTTP handler tests for both Erasure multiple disks and single node setup.
func TestRemoveBucketHandler(t *testing.T) { func TestRemoveBucketHandler(t *testing.T) {
ExecObjectLayerAPITest(ExecObjectLayerAPITestArgs{t: t, objAPITest: testRemoveBucketHandler, endpoints: []string{"RemoveBucket"}}) ExecObjectLayerAPITest(ExecObjectLayerAPITestArgs{t: t, objAPITest: testRemoveBucketHandler, endpoints: []string{"RemoveBucket"}})
@@ -1018,23 +978,14 @@ func testAPIDeleteMultipleObjectsVersionIDNullCondition(obj ObjectLayer, instanc
policyBytes := fmt.Appendf(nil, `{ policyBytes := fmt.Appendf(nil, `{
"Version":"2012-10-17", "Version":"2012-10-17",
"Statement":[ "Statement":[{
{ "Effect":"Allow",
"Effect":"Allow", "Principal":"*",
"Principal":"*", "Action":"s3:DeleteObject",
"Action":"s3:DeleteObject", "Resource":"arn:aws:s3:::%s/*",
"Resource":"arn:aws:s3:::%s/*", "Condition":{"Null":{"s3:versionid":"true"}}
"Condition":{"Null":{"s3:versionid":"true"}} }]
}, }`, bucketName)
{
"Effect":"Allow",
"Principal":"*",
"Action":"s3:DeleteObjectVersion",
"Resource":"arn:aws:s3:::%s/*",
"Condition":{"StringEquals":{"s3:versionid":"%s"}}
}
]
}`, bucketName, bucketName, versionIDs["with-version-id"])
policyReq, err := newTestSignedRequestV4(http.MethodPut, getPutPolicyURL("", bucketName), int64(len(policyBytes)), policyReq, err := newTestSignedRequestV4(http.MethodPut, getPutPolicyURL("", bucketName), int64(len(policyBytes)),
bytes.NewReader(policyBytes), credentials.AccessKey, credentials.SecretKey, nil) bytes.NewReader(policyBytes), credentials.AccessKey, credentials.SecretKey, nil)
if err != nil { if err != nil {
@@ -1080,30 +1031,29 @@ func testAPIDeleteMultipleObjectsVersionIDNullCondition(obj ObjectLayer, instanc
t.Errorf("%s: %q was not a successful delete-marker creation: %+v", instanceType, objectName, response.DeletedObjects) t.Errorf("%s: %q was not a successful delete-marker creation: %+v", instanceType, objectName, response.DeletedObjects)
} }
} }
if object, ok := deleted["with-version-id"]; !ok || object.VersionID != versionIDs["with-version-id"] { if len(deleted) != 2 {
t.Errorf("%s: matching explicit version was not deleted: %+v", instanceType, response.DeletedObjects)
}
if len(deleted) != 3 {
t.Errorf("%s: unexpected deleted objects: %+v", instanceType, response.DeletedObjects) t.Errorf("%s: unexpected deleted objects: %+v", instanceType, response.DeletedObjects)
} }
errorsByKey := make(map[string]DeleteError, len(response.Errors)) errorsByKey := make(map[string]DeleteError, len(response.Errors))
for _, deleteErr := range response.Errors { for _, deleteErr := range response.Errors {
errorsByKey[deleteErr.Key] = deleteErr errorsByKey[deleteErr.Key] = deleteErr
} }
for objectName, versionID := range map[string]string{"with-null-version-id": nullVersionID} { for objectName, versionID := range map[string]string{
"with-version-id": versionIDs["with-version-id"],
"with-null-version-id": nullVersionID,
} {
deleteErr, ok := errorsByKey[objectName] deleteErr, ok := errorsByKey[objectName]
if !ok || deleteErr.VersionID != versionID || deleteErr.Code != errorCodes[ErrAccessDenied].Code { if !ok || deleteErr.VersionID != versionID || deleteErr.Code != errorCodes[ErrAccessDenied].Code {
t.Errorf("%s: %q did not return AccessDenied for version %q: %+v", instanceType, objectName, versionID, response.Errors) t.Errorf("%s: %q did not return AccessDenied for version %q: %+v", instanceType, objectName, versionID, response.Errors)
} }
} }
if len(errorsByKey) != 1 { if len(errorsByKey) != 2 {
t.Errorf("%s: unexpected delete errors: %+v", instanceType, response.Errors) t.Errorf("%s: unexpected delete errors: %+v", instanceType, response.Errors)
} }
// A simple delete adds a marker and keeps the old version. The null-version // A simple delete adds a marker and keeps the old version. The explicitly
// delete remains denied because its per-entry condition does not match. // named version must also remain because its policy condition did not match.
for _, objectName := range []string{"without-version-id-before", "without-version-id-after", "with-null-version-id"} { for objectName, versionID := range versionIDs {
versionID := versionIDs[objectName]
if _, err = obj.GetObjectInfo(t.Context(), bucketName, objectName, ObjectOptions{VersionID: versionID}); err != nil { if _, err = obj.GetObjectInfo(t.Context(), bucketName, objectName, ObjectOptions{VersionID: versionID}); err != nil {
t.Errorf("%s: version %s of %q was not preserved: %v", instanceType, versionID, objectName, err) t.Errorf("%s: version %s of %q was not preserved: %v", instanceType, versionID, objectName, err)
} }
@@ -1113,10 +1063,7 @@ func testAPIDeleteMultipleObjectsVersionIDNullCondition(obj ObjectLayer, instanc
t.Errorf("%s: simple delete of %q did not hide the latest object behind a delete marker: %v", instanceType, objectName, err) t.Errorf("%s: simple delete of %q did not hide the latest object behind a delete marker: %v", instanceType, objectName, err)
} }
} }
if _, err = obj.GetObjectInfo(t.Context(), bucketName, "with-version-id", ObjectOptions{VersionID: versionIDs["with-version-id"]}); !isErrVersionNotFound(err) && !isErrObjectNotFound(err) { for _, objectName := range []string{"with-version-id", "with-null-version-id"} {
t.Errorf("%s: matching explicit version still exists: %v", instanceType, err)
}
for _, objectName := range []string{"with-null-version-id"} {
if info, err := obj.GetObjectInfo(t.Context(), bucketName, objectName, ObjectOptions{}); err != nil { if info, err := obj.GetObjectInfo(t.Context(), bucketName, objectName, ObjectOptions{}); err != nil {
t.Errorf("%s: denied version delete removed latest %q: %v", instanceType, objectName, err) t.Errorf("%s: denied version delete removed latest %q: %v", instanceType, objectName, err)
} else if info.VersionID != versionIDs[objectName] { } else if info.VersionID != versionIDs[objectName] {
-123
View File
@@ -29,7 +29,6 @@ import (
"github.com/minio/madmin-go/v3" "github.com/minio/madmin-go/v3"
"github.com/minio/minio-go/v7/pkg/set" "github.com/minio/minio-go/v7/pkg/set"
"github.com/minio/minio-go/v7/pkg/tags" "github.com/minio/minio-go/v7/pkg/tags"
"github.com/minio/minio/internal/bucket/cors"
bucketsse "github.com/minio/minio/internal/bucket/encryption" bucketsse "github.com/minio/minio/internal/bucket/encryption"
"github.com/minio/minio/internal/bucket/lifecycle" "github.com/minio/minio/internal/bucket/lifecycle"
objectlock "github.com/minio/minio/internal/bucket/object/lock" objectlock "github.com/minio/minio/internal/bucket/object/lock"
@@ -51,14 +50,6 @@ type BucketMetadataSys struct {
initialized bool initialized bool
group *singleflight.Group group *singleflight.Group
metadataMap map[string]BucketMetadata metadataMap map[string]BucketMetadata
// loadFailed tracks real buckets whose metadata could not be loaded at
// startup (concurrentLoad) or during a refresh. Such buckets are NOT
// resident in metadataMap even though the subsystem is Initialized, so a
// plain map miss cannot distinguish "not a bucket" from "known bucket whose
// config we could not read". Callers that must fail closed for a real but
// unreadable bucket (e.g. per-bucket CORS) consult this set. It is bounded
// by the number of load failures and is empty in normal operation.
loadFailed map[string]struct{}
} }
// Count returns number of bucket metadata map entries. // Count returns number of bucket metadata map entries.
@@ -75,7 +66,6 @@ func (sys *BucketMetadataSys) Remove(buckets ...string) {
for _, bucket := range buckets { for _, bucket := range buckets {
sys.group.Forget(bucket) sys.group.Forget(bucket)
delete(sys.metadataMap, bucket) delete(sys.metadataMap, bucket)
delete(sys.loadFailed, bucket)
globalBucketMonitor.DeleteBucket(bucket) globalBucketMonitor.DeleteBucket(bucket)
} }
sys.Unlock() sys.Unlock()
@@ -93,11 +83,6 @@ func (sys *BucketMetadataSys) RemoveStaleBuckets(diskBuckets set.StringSet) {
delete(sys.metadataMap, bucket) delete(sys.metadataMap, bucket)
globalBucketMonitor.DeleteBucket(bucket) globalBucketMonitor.DeleteBucket(bucket)
} }
for bucket := range sys.loadFailed {
if !diskBuckets.Contains(bucket) {
delete(sys.loadFailed, bucket)
}
}
} }
// Set - sets a new metadata in-memory. // Set - sets a new metadata in-memory.
@@ -109,7 +94,6 @@ func (sys *BucketMetadataSys) Set(bucket string, meta BucketMetadata) {
if !isMinioMetaBucketName(bucket) { if !isMinioMetaBucketName(bucket) {
sys.Lock() sys.Lock()
sys.metadataMap[bucket] = meta sys.metadataMap[bucket] = meta
delete(sys.loadFailed, bucket)
sys.Unlock() sys.Unlock()
} }
} }
@@ -150,9 +134,6 @@ func (sys *BucketMetadataSys) updateAndParse(ctx context.Context, bucket string,
case bucketTaggingConfig: case bucketTaggingConfig:
meta.TaggingConfigXML = configData meta.TaggingConfigXML = configData
meta.TaggingConfigUpdatedAt = updatedAt meta.TaggingConfigUpdatedAt = updatedAt
case bucketCorsConfig:
meta.CorsConfigXML = configData
meta.CorsConfigUpdatedAt = updatedAt
case bucketQuotaConfigFile: case bucketQuotaConfigFile:
meta.QuotaConfigJSON = configData meta.QuotaConfigJSON = configData
meta.QuotaConfigUpdatedAt = updatedAt meta.QuotaConfigUpdatedAt = updatedAt
@@ -378,98 +359,6 @@ func (sys *BucketMetadataSys) GetSSEConfig(bucket string) (*bucketsse.BucketSSEC
return meta.sseConfig, meta.EncryptionConfigUpdatedAt, nil return meta.sseConfig, meta.EncryptionConfigUpdatedAt, nil
} }
// GetCorsConfig returns the CORS configuration for the given bucket.
// The returned object must not be modified.
func (sys *BucketMetadataSys) GetCorsConfig(bucket string) (*cors.Config, time.Time, error) {
meta, _, err := sys.GetConfig(GlobalContext, bucket)
if err != nil {
return nil, time.Time{}, err
}
if meta.corsConfigErr != nil {
return nil, meta.CorsConfigUpdatedAt, meta.corsConfigErr
}
if meta.corsConfig == nil {
return nil, time.Time{}, errConfigNotFound
}
return meta.corsConfig, meta.CorsConfigUpdatedAt, nil
}
// GetResidentCorsConfig returns the CORS configuration for the given bucket
// using only bucket metadata that is already resident in memory. Unlike
// GetCorsConfig it never loads metadata from disk and never caches a new
// entry.
//
// The per-request CORS middleware runs before authentication, for every
// Origin-bearing request, using the validated first path segment as the bucket
// name.
// Routing that through GetCorsConfig (which loads and caches) let an
// unauthenticated client grow metadataMap without bound and trigger an
// erasure metadata probe for every distinct, attacker-controlled,
// non-existent name it sent with an Origin header (e.g. /minio/... , /api/... ,
// or random buckets). Every bucket that can carry a CORS document is made
// resident when the document is written (Set) and when metadata is loaded at
// startup (Init/concurrentLoad), so a resident-only read is complete for real
// buckets while costing only an in-memory map lookup for everything else.
//
// While bucket metadata is still loading (not yet Initialized) a non-resident
// bucket returns errBucketMetadataNotInitialized so the caller fails closed
// rather than answering with the permissive global policy for a bucket whose
// restrictive CORS document may simply not be loaded yet.
func (sys *BucketMetadataSys) GetResidentCorsConfig(bucket string) (*cors.Config, time.Time, error) {
if isMinioMetaBucketName(bucket) {
// Preserve GetConfig's semantics for the internal namespace: this is
// not a real bucket, and returning a non-errConfigNotFound error makes
// the CORS middleware fail closed rather than answer for .minio.sys
// with the permissive global policy.
return nil, time.Time{}, errInvalidArgument
}
if isReservedOrInvalidBucket(bucket, true) {
return nil, time.Time{}, errConfigNotFound
}
sys.RLock()
meta, ok := sys.metadataMap[bucket]
_, failed := sys.loadFailed[bucket]
initialized := sys.initialized
sys.RUnlock()
if ok {
if meta.corsConfigErr != nil {
return nil, meta.CorsConfigUpdatedAt, meta.corsConfigErr
}
if meta.corsConfig == nil {
return nil, time.Time{}, errConfigNotFound
}
return meta.corsConfig, meta.CorsConfigUpdatedAt, nil
}
// Not resident. Two cases must not be conflated:
// - metadata is still loading (!initialized), or this is a real bucket
// whose metadata failed to load: we cannot rule out a restrictive CORS
// config, so fail closed rather than answer with the global policy.
// - a fully initialized subsystem with no record of the name: it is not a
// bucket that can carry CORS, so fall back to the global policy without
// loading or caching metadata for an arbitrary, client-supplied name.
if !initialized || failed {
return nil, time.Time{}, errBucketMetadataNotInitialized
}
return nil, time.Time{}, errConfigNotFound
}
// GetCorsConfigXML returns the raw stored CORS configuration XML for the
// given bucket, preserving the document exactly as it was PUT (including
// the S3 xmlns and any unmodeled elements).
func (sys *BucketMetadataSys) GetCorsConfigXML(bucket string) ([]byte, time.Time, error) {
meta, _, err := sys.GetConfig(GlobalContext, bucket)
if err != nil {
return nil, time.Time{}, err
}
if meta.corsConfigErr != nil {
return nil, meta.CorsConfigUpdatedAt, meta.corsConfigErr
}
if len(meta.CorsConfigXML) == 0 {
return nil, time.Time{}, errConfigNotFound
}
return meta.CorsConfigXML, meta.CorsConfigUpdatedAt, nil
}
// CreatedAt returns the time of creation of bucket // CreatedAt returns the time of creation of bucket
func (sys *BucketMetadataSys) CreatedAt(bucket string) (time.Time, error) { func (sys *BucketMetadataSys) CreatedAt(bucket string) (time.Time, error) {
meta, _, err := sys.GetConfig(GlobalContext, bucket) meta, _, err := sys.GetConfig(GlobalContext, bucket)
@@ -650,13 +539,8 @@ func (sys *BucketMetadataSys) concurrentLoad(ctx context.Context, buckets []stri
sys.Lock() sys.Lock()
for i, meta := range bucketMetas { for i, meta := range bucketMetas {
if errs[i] != nil { if errs[i] != nil {
// Real bucket whose metadata could not be loaded: record it so
// consumers that must fail closed (per-bucket CORS) can tell it
// apart from a name that is not a bucket at all.
sys.loadFailed[buckets[i]] = struct{}{}
continue continue
} }
delete(sys.loadFailed, buckets[i])
sys.metadataMap[buckets[i]] = meta sys.metadataMap[buckets[i]] = meta
} }
sys.Unlock() sys.Unlock()
@@ -706,9 +590,6 @@ func (sys *BucketMetadataSys) refreshBucketsMetadataLoop(ctx context.Context) {
meta, err := loadBucketMetadata(ctx, sys.objAPI, bucket) meta, err := loadBucketMetadata(ctx, sys.objAPI, bucket)
if err != nil { if err != nil {
internalLogIf(ctx, err, logger.WarningKind) internalLogIf(ctx, err, logger.WarningKind)
sys.Lock()
sys.loadFailed[bucket] = struct{}{}
sys.Unlock()
wait() // wait to proceed to next entry. wait() // wait to proceed to next entry.
continue continue
} }
@@ -719,8 +600,6 @@ func (sys *BucketMetadataSys) refreshBucketsMetadataLoop(ctx context.Context) {
updated = true updated = true
sys.metadataMap[bucket] = meta sys.metadataMap[bucket] = meta
} }
// A successful (re)load clears any earlier load failure.
delete(sys.loadFailed, bucket)
sys.Unlock() sys.Unlock()
if updated { if updated {
@@ -768,7 +647,6 @@ func (sys *BucketMetadataSys) init(ctx context.Context, buckets []string) {
func (sys *BucketMetadataSys) Reset() { func (sys *BucketMetadataSys) Reset() {
sys.Lock() sys.Lock()
clear(sys.metadataMap) clear(sys.metadataMap)
clear(sys.loadFailed)
sys.Unlock() sys.Unlock()
} }
@@ -776,7 +654,6 @@ func (sys *BucketMetadataSys) Reset() {
func NewBucketMetadataSys() *BucketMetadataSys { func NewBucketMetadataSys() *BucketMetadataSys {
return &BucketMetadataSys{ return &BucketMetadataSys{
metadataMap: make(map[string]BucketMetadata), metadataMap: make(map[string]BucketMetadata),
loadFailed: make(map[string]struct{}),
group: &singleflight.Group{}, group: &singleflight.Group{},
} }
} }
+1 -40
View File
@@ -31,7 +31,6 @@ import (
"github.com/minio/madmin-go/v3" "github.com/minio/madmin-go/v3"
"github.com/minio/minio-go/v7/pkg/tags" "github.com/minio/minio-go/v7/pkg/tags"
"github.com/minio/minio/internal/bucket/cors"
bucketsse "github.com/minio/minio/internal/bucket/encryption" bucketsse "github.com/minio/minio/internal/bucket/encryption"
"github.com/minio/minio/internal/bucket/lifecycle" "github.com/minio/minio/internal/bucket/lifecycle"
objectlock "github.com/minio/minio/internal/bucket/object/lock" objectlock "github.com/minio/minio/internal/bucket/object/lock"
@@ -59,9 +58,6 @@ var (
enabledBucketVersioningConfig = []byte(`<VersioningConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Status>Enabled</Status></VersioningConfiguration>`) enabledBucketVersioningConfig = []byte(`<VersioningConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Status>Enabled</Status></VersioningConfiguration>`)
) )
// Bucket CORS configuration file.
const bucketCorsConfig = "cors.xml"
//go:generate msgp -file $GOFILE //go:generate msgp -file $GOFILE
// BucketMetadata contains bucket metadata. // BucketMetadata contains bucket metadata.
@@ -84,7 +80,6 @@ type BucketMetadata struct {
ReplicationConfigXML []byte ReplicationConfigXML []byte
BucketTargetsConfigJSON []byte BucketTargetsConfigJSON []byte
BucketTargetsConfigMetaJSON []byte BucketTargetsConfigMetaJSON []byte
CorsConfigXML []byte
PolicyConfigUpdatedAt time.Time PolicyConfigUpdatedAt time.Time
ObjectLockConfigUpdatedAt time.Time ObjectLockConfigUpdatedAt time.Time
@@ -97,7 +92,6 @@ type BucketMetadata struct {
NotificationConfigUpdatedAt time.Time NotificationConfigUpdatedAt time.Time
BucketTargetsConfigUpdatedAt time.Time BucketTargetsConfigUpdatedAt time.Time
BucketTargetsConfigMetaUpdatedAt time.Time BucketTargetsConfigMetaUpdatedAt time.Time
CorsConfigUpdatedAt time.Time
// Add a new UpdatedAt field and update lastUpdate function // Add a new UpdatedAt field and update lastUpdate function
// Unexported fields. Must be updated atomically. // Unexported fields. Must be updated atomically.
@@ -112,8 +106,6 @@ type BucketMetadata struct {
replicationConfig *replication.Config replicationConfig *replication.Config
bucketTargetConfig *madmin.BucketTargets bucketTargetConfig *madmin.BucketTargets
bucketTargetConfigMeta map[string]string bucketTargetConfigMeta map[string]string
corsConfig *cors.Config
corsConfigErr error
} }
// newBucketMetadata creates BucketMetadata with the supplied name and Created to Now. // newBucketMetadata creates BucketMetadata with the supplied name and Created to Now.
@@ -168,9 +160,6 @@ func (b BucketMetadata) lastUpdate() (t time.Time) {
if b.BucketTargetsConfigMetaUpdatedAt.After(t) { if b.BucketTargetsConfigMetaUpdatedAt.After(t) {
t = b.BucketTargetsConfigMetaUpdatedAt t = b.BucketTargetsConfigMetaUpdatedAt
} }
if b.CorsConfigUpdatedAt.After(t) {
t = b.CorsConfigUpdatedAt
}
return t return t
} }
@@ -262,12 +251,6 @@ func loadBucketMetadataParse(ctx context.Context, objectAPI ObjectLayer, bucket
return b, err return b, err
} }
} }
if b.corsConfigErr != nil {
// Keep the rest of the bucket metadata available so an operator can
// replace or delete a CORS document accepted by an older, more lenient
// build. Defer unrelated metadata migration until CORS is repaired.
return b, nil
}
// migrate unencrypted remote targets // migrate unencrypted remote targets
if err = b.migrateTargetConfig(ctx, objectAPI); err != nil { if err = b.migrateTargetConfig(ctx, objectAPI); err != nil {
@@ -327,27 +310,8 @@ func (b *BucketMetadata) parseAllConfigs(ctx context.Context, objectAPI ObjectLa
b.taggingConfig = nil b.taggingConfig = nil
} }
b.corsConfigErr = nil
if len(b.CorsConfigXML) != 0 {
cfg, corsErr := cors.ParseBucketCorsConfig(bytes.NewReader(b.CorsConfigXML))
if corsErr == nil {
corsErr = cfg.Validate()
}
if corsErr != nil {
b.corsConfig = nil
b.corsConfigErr = fmt.Errorf("invalid bucket CORS configuration: %w", corsErr)
} else {
b.corsConfig = cfg
}
} else {
b.corsConfig = nil
}
if bytes.Equal(b.ObjectLockConfigXML, enabledBucketObjectLockConfig) { if bytes.Equal(b.ObjectLockConfigXML, enabledBucketObjectLockConfig) {
config, versioningErr := versioning.ParseConfig(bytes.NewReader(b.VersioningConfigXML)) b.VersioningConfigXML = enabledBucketVersioningConfig
if versioningErr != nil || !config.Enabled() {
b.VersioningConfigXML = enabledBucketVersioningConfig
}
} }
if len(b.ObjectLockConfigXML) != 0 { if len(b.ObjectLockConfigXML) != 0 {
@@ -539,9 +503,6 @@ func (b *BucketMetadata) Save(ctx context.Context, api ObjectLayer) error {
if err := b.parseAllConfigs(ctx, api); err != nil { if err := b.parseAllConfigs(ctx, api); err != nil {
return err return err
} }
if b.corsConfigErr != nil {
return b.corsConfigErr
}
data := make([]byte, 4, b.Msgsize()+4) data := make([]byte, 4, b.Msgsize()+4)
+5 -55
View File
@@ -108,12 +108,6 @@ func (z *BucketMetadata) DecodeMsg(dc *msgp.Reader) (err error) {
err = msgp.WrapError(err, "BucketTargetsConfigMetaJSON") err = msgp.WrapError(err, "BucketTargetsConfigMetaJSON")
return return
} }
case "CorsConfigXML":
z.CorsConfigXML, err = dc.ReadBytes(z.CorsConfigXML)
if err != nil {
err = msgp.WrapError(err, "CorsConfigXML")
return
}
case "PolicyConfigUpdatedAt": case "PolicyConfigUpdatedAt":
z.PolicyConfigUpdatedAt, err = dc.ReadTime() z.PolicyConfigUpdatedAt, err = dc.ReadTime()
if err != nil { if err != nil {
@@ -180,12 +174,6 @@ func (z *BucketMetadata) DecodeMsg(dc *msgp.Reader) (err error) {
err = msgp.WrapError(err, "BucketTargetsConfigMetaUpdatedAt") err = msgp.WrapError(err, "BucketTargetsConfigMetaUpdatedAt")
return return
} }
case "CorsConfigUpdatedAt":
z.CorsConfigUpdatedAt, err = dc.ReadTime()
if err != nil {
err = msgp.WrapError(err, "CorsConfigUpdatedAt")
return
}
default: default:
err = dc.Skip() err = dc.Skip()
if err != nil { if err != nil {
@@ -199,9 +187,9 @@ func (z *BucketMetadata) DecodeMsg(dc *msgp.Reader) (err error) {
// EncodeMsg implements msgp.Encodable // EncodeMsg implements msgp.Encodable
func (z *BucketMetadata) EncodeMsg(en *msgp.Writer) (err error) { func (z *BucketMetadata) EncodeMsg(en *msgp.Writer) (err error) {
// map header, size 27 // map header, size 25
// write "Name" // write "Name"
err = en.Append(0xde, 0x0, 0x1b, 0xa4, 0x4e, 0x61, 0x6d, 0x65) err = en.Append(0xde, 0x0, 0x19, 0xa4, 0x4e, 0x61, 0x6d, 0x65)
if err != nil { if err != nil {
return return
} }
@@ -340,16 +328,6 @@ func (z *BucketMetadata) EncodeMsg(en *msgp.Writer) (err error) {
err = msgp.WrapError(err, "BucketTargetsConfigMetaJSON") err = msgp.WrapError(err, "BucketTargetsConfigMetaJSON")
return return
} }
// write "CorsConfigXML"
err = en.Append(0xad, 0x43, 0x6f, 0x72, 0x73, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x58, 0x4d, 0x4c)
if err != nil {
return
}
err = en.WriteBytes(z.CorsConfigXML)
if err != nil {
err = msgp.WrapError(err, "CorsConfigXML")
return
}
// write "PolicyConfigUpdatedAt" // write "PolicyConfigUpdatedAt"
err = en.Append(0xb5, 0x50, 0x6f, 0x6c, 0x69, 0x63, 0x79, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x64, 0x41, 0x74) err = en.Append(0xb5, 0x50, 0x6f, 0x6c, 0x69, 0x63, 0x79, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x64, 0x41, 0x74)
if err != nil { if err != nil {
@@ -460,25 +438,15 @@ func (z *BucketMetadata) EncodeMsg(en *msgp.Writer) (err error) {
err = msgp.WrapError(err, "BucketTargetsConfigMetaUpdatedAt") err = msgp.WrapError(err, "BucketTargetsConfigMetaUpdatedAt")
return return
} }
// write "CorsConfigUpdatedAt"
err = en.Append(0xb3, 0x43, 0x6f, 0x72, 0x73, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x64, 0x41, 0x74)
if err != nil {
return
}
err = en.WriteTime(z.CorsConfigUpdatedAt)
if err != nil {
err = msgp.WrapError(err, "CorsConfigUpdatedAt")
return
}
return return
} }
// MarshalMsg implements msgp.Marshaler // MarshalMsg implements msgp.Marshaler
func (z *BucketMetadata) MarshalMsg(b []byte) (o []byte, err error) { func (z *BucketMetadata) MarshalMsg(b []byte) (o []byte, err error) {
o = msgp.Require(b, z.Msgsize()) o = msgp.Require(b, z.Msgsize())
// map header, size 27 // map header, size 25
// string "Name" // string "Name"
o = append(o, 0xde, 0x0, 0x1b, 0xa4, 0x4e, 0x61, 0x6d, 0x65) o = append(o, 0xde, 0x0, 0x19, 0xa4, 0x4e, 0x61, 0x6d, 0x65)
o = msgp.AppendString(o, z.Name) o = msgp.AppendString(o, z.Name)
// string "Created" // string "Created"
o = append(o, 0xa7, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x64) o = append(o, 0xa7, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x64)
@@ -519,9 +487,6 @@ func (z *BucketMetadata) MarshalMsg(b []byte) (o []byte, err error) {
// string "BucketTargetsConfigMetaJSON" // string "BucketTargetsConfigMetaJSON"
o = append(o, 0xbb, 0x42, 0x75, 0x63, 0x6b, 0x65, 0x74, 0x54, 0x61, 0x72, 0x67, 0x65, 0x74, 0x73, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x4d, 0x65, 0x74, 0x61, 0x4a, 0x53, 0x4f, 0x4e) o = append(o, 0xbb, 0x42, 0x75, 0x63, 0x6b, 0x65, 0x74, 0x54, 0x61, 0x72, 0x67, 0x65, 0x74, 0x73, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x4d, 0x65, 0x74, 0x61, 0x4a, 0x53, 0x4f, 0x4e)
o = msgp.AppendBytes(o, z.BucketTargetsConfigMetaJSON) o = msgp.AppendBytes(o, z.BucketTargetsConfigMetaJSON)
// string "CorsConfigXML"
o = append(o, 0xad, 0x43, 0x6f, 0x72, 0x73, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x58, 0x4d, 0x4c)
o = msgp.AppendBytes(o, z.CorsConfigXML)
// string "PolicyConfigUpdatedAt" // string "PolicyConfigUpdatedAt"
o = append(o, 0xb5, 0x50, 0x6f, 0x6c, 0x69, 0x63, 0x79, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x64, 0x41, 0x74) o = append(o, 0xb5, 0x50, 0x6f, 0x6c, 0x69, 0x63, 0x79, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x64, 0x41, 0x74)
o = msgp.AppendTime(o, z.PolicyConfigUpdatedAt) o = msgp.AppendTime(o, z.PolicyConfigUpdatedAt)
@@ -555,9 +520,6 @@ func (z *BucketMetadata) MarshalMsg(b []byte) (o []byte, err error) {
// string "BucketTargetsConfigMetaUpdatedAt" // string "BucketTargetsConfigMetaUpdatedAt"
o = append(o, 0xd9, 0x20, 0x42, 0x75, 0x63, 0x6b, 0x65, 0x74, 0x54, 0x61, 0x72, 0x67, 0x65, 0x74, 0x73, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x4d, 0x65, 0x74, 0x61, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x64, 0x41, 0x74) o = append(o, 0xd9, 0x20, 0x42, 0x75, 0x63, 0x6b, 0x65, 0x74, 0x54, 0x61, 0x72, 0x67, 0x65, 0x74, 0x73, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x4d, 0x65, 0x74, 0x61, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x64, 0x41, 0x74)
o = msgp.AppendTime(o, z.BucketTargetsConfigMetaUpdatedAt) o = msgp.AppendTime(o, z.BucketTargetsConfigMetaUpdatedAt)
// string "CorsConfigUpdatedAt"
o = append(o, 0xb3, 0x43, 0x6f, 0x72, 0x73, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x64, 0x41, 0x74)
o = msgp.AppendTime(o, z.CorsConfigUpdatedAt)
return return
} }
@@ -663,12 +625,6 @@ func (z *BucketMetadata) UnmarshalMsg(bts []byte) (o []byte, err error) {
err = msgp.WrapError(err, "BucketTargetsConfigMetaJSON") err = msgp.WrapError(err, "BucketTargetsConfigMetaJSON")
return return
} }
case "CorsConfigXML":
z.CorsConfigXML, bts, err = msgp.ReadBytesBytes(bts, z.CorsConfigXML)
if err != nil {
err = msgp.WrapError(err, "CorsConfigXML")
return
}
case "PolicyConfigUpdatedAt": case "PolicyConfigUpdatedAt":
z.PolicyConfigUpdatedAt, bts, err = msgp.ReadTimeBytes(bts) z.PolicyConfigUpdatedAt, bts, err = msgp.ReadTimeBytes(bts)
if err != nil { if err != nil {
@@ -735,12 +691,6 @@ func (z *BucketMetadata) UnmarshalMsg(bts []byte) (o []byte, err error) {
err = msgp.WrapError(err, "BucketTargetsConfigMetaUpdatedAt") err = msgp.WrapError(err, "BucketTargetsConfigMetaUpdatedAt")
return return
} }
case "CorsConfigUpdatedAt":
z.CorsConfigUpdatedAt, bts, err = msgp.ReadTimeBytes(bts)
if err != nil {
err = msgp.WrapError(err, "CorsConfigUpdatedAt")
return
}
default: default:
bts, err = msgp.Skip(bts) bts, err = msgp.Skip(bts)
if err != nil { if err != nil {
@@ -755,6 +705,6 @@ func (z *BucketMetadata) UnmarshalMsg(bts []byte) (o []byte, err error) {
// Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message // Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message
func (z *BucketMetadata) Msgsize() (s int) { func (z *BucketMetadata) Msgsize() (s int) {
s = 3 + 5 + msgp.StringPrefixSize + len(z.Name) + 8 + msgp.TimeSize + 12 + msgp.BoolSize + 17 + msgp.BytesPrefixSize + len(z.PolicyConfigJSON) + 22 + msgp.BytesPrefixSize + len(z.NotificationConfigXML) + 19 + msgp.BytesPrefixSize + len(z.LifecycleConfigXML) + 20 + msgp.BytesPrefixSize + len(z.ObjectLockConfigXML) + 20 + msgp.BytesPrefixSize + len(z.VersioningConfigXML) + 20 + msgp.BytesPrefixSize + len(z.EncryptionConfigXML) + 17 + msgp.BytesPrefixSize + len(z.TaggingConfigXML) + 16 + msgp.BytesPrefixSize + len(z.QuotaConfigJSON) + 21 + msgp.BytesPrefixSize + len(z.ReplicationConfigXML) + 24 + msgp.BytesPrefixSize + len(z.BucketTargetsConfigJSON) + 28 + msgp.BytesPrefixSize + len(z.BucketTargetsConfigMetaJSON) + 14 + msgp.BytesPrefixSize + len(z.CorsConfigXML) + 22 + msgp.TimeSize + 26 + msgp.TimeSize + 26 + msgp.TimeSize + 23 + msgp.TimeSize + 21 + msgp.TimeSize + 27 + msgp.TimeSize + 26 + msgp.TimeSize + 25 + msgp.TimeSize + 28 + msgp.TimeSize + 29 + msgp.TimeSize + 34 + msgp.TimeSize + 20 + msgp.TimeSize s = 3 + 5 + msgp.StringPrefixSize + len(z.Name) + 8 + msgp.TimeSize + 12 + msgp.BoolSize + 17 + msgp.BytesPrefixSize + len(z.PolicyConfigJSON) + 22 + msgp.BytesPrefixSize + len(z.NotificationConfigXML) + 19 + msgp.BytesPrefixSize + len(z.LifecycleConfigXML) + 20 + msgp.BytesPrefixSize + len(z.ObjectLockConfigXML) + 20 + msgp.BytesPrefixSize + len(z.VersioningConfigXML) + 20 + msgp.BytesPrefixSize + len(z.EncryptionConfigXML) + 17 + msgp.BytesPrefixSize + len(z.TaggingConfigXML) + 16 + msgp.BytesPrefixSize + len(z.QuotaConfigJSON) + 21 + msgp.BytesPrefixSize + len(z.ReplicationConfigXML) + 24 + msgp.BytesPrefixSize + len(z.BucketTargetsConfigJSON) + 28 + msgp.BytesPrefixSize + len(z.BucketTargetsConfigMetaJSON) + 22 + msgp.TimeSize + 26 + msgp.TimeSize + 26 + msgp.TimeSize + 23 + msgp.TimeSize + 21 + msgp.TimeSize + 27 + msgp.TimeSize + 26 + msgp.TimeSize + 25 + msgp.TimeSize + 28 + msgp.TimeSize + 29 + msgp.TimeSize + 34 + msgp.TimeSize
return return
} }
-41
View File
@@ -1,41 +0,0 @@
// Copyright (c) 2015-2026 MinIO, Inc.
//
// This file is part of MinIO Object Storage stack
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package cmd
import "testing"
func TestBucketMetadataCorsRoundTrip(t *testing.T) {
meta := newBucketMetadata("test-cors")
meta.CorsConfigXML = []byte(`<CORSConfiguration><CORSRule><AllowedOrigin>*</AllowedOrigin><AllowedMethod>GET</AllowedMethod></CORSRule></CORSConfiguration>`)
meta.CorsConfigUpdatedAt = UTCNow()
buf, err := meta.MarshalMsg(nil)
if err != nil {
t.Fatal(err)
}
var got BucketMetadata
if _, err := got.UnmarshalMsg(buf); err != nil {
t.Fatal(err)
}
if string(got.CorsConfigXML) != string(meta.CorsConfigXML) {
t.Fatalf("CorsConfigXML not preserved: %q", string(got.CorsConfigXML))
}
if !got.CorsConfigUpdatedAt.Equal(meta.CorsConfigUpdatedAt) {
t.Fatalf("CorsConfigUpdatedAt not preserved")
}
}
+12 -12
View File
@@ -25,6 +25,8 @@ import (
"github.com/minio/minio/internal/auth" "github.com/minio/minio/internal/auth"
objectlock "github.com/minio/minio/internal/bucket/object/lock" objectlock "github.com/minio/minio/internal/bucket/object/lock"
"github.com/minio/minio/internal/bucket/replication"
xhttp "github.com/minio/minio/internal/http"
"github.com/minio/minio/internal/logger" "github.com/minio/minio/internal/logger"
"github.com/minio/pkg/v3/policy" "github.com/minio/pkg/v3/policy"
) )
@@ -148,11 +150,7 @@ func enforceRetentionBypassForDelete(ctx context.Context, r *http.Request, bucke
} }
// https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lock-overview.html#object-lock-retention-modes // https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lock-overview.html#object-lock-retention-modes
// If you try to delete objects protected by governance mode and have s3:BypassGovernanceRetention, the operation will succeed. // If you try to delete objects protected by governance mode and have s3:BypassGovernanceRetention, the operation will succeed.
if reqInfo := logger.GetReqInfo(ctx); reqInfo != nil { if checkRequestAuthType(ctx, r, policy.BypassGovernanceRetentionAction, bucket, object.ObjectName) != ErrNone {
reqInfo.BucketName = bucket
reqInfo.ObjectName = object.ObjectName
}
if authorizeRequest(ctx, r, policy.BypassGovernanceRetentionAction) != ErrNone {
return errAuthentication return errAuthentication
} }
} }
@@ -200,7 +198,7 @@ func enforceRetentionBypassForPut(ctx context.Context, r *http.Request, oi Objec
byPassSet, r, cred, owner) byPassSet, r, cred, owner)
// Governance mode retention period cannot be shortened, if x-amz-bypass-governance is not set. // Governance mode retention period cannot be shortened, if x-amz-bypass-governance is not set.
if !byPassSet { if !byPassSet {
if objRetention.Mode != objectlock.RetGovernance || objRetention.RetainUntilDate.Before(ret.RetainUntilDate.Time) { if objRetention.Mode != objectlock.RetGovernance || objRetention.RetainUntilDate.Before((ret.RetainUntilDate.Time)) {
return ObjectLocked{Bucket: oi.Bucket, Object: oi.Name, VersionID: oi.VersionID} return ObjectLocked{Bucket: oi.Bucket, Object: oi.Name, VersionID: oi.VersionID}
} }
} }
@@ -211,7 +209,7 @@ func enforceRetentionBypassForPut(ctx context.Context, r *http.Request, oi Objec
case objectlock.RetCompliance: case objectlock.RetCompliance:
// Compliance retention mode cannot be changed or shortened. // Compliance retention mode cannot be changed or shortened.
// https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lock-overview.html#object-lock-retention-modes // https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lock-overview.html#object-lock-retention-modes
if objRetention.Mode != objectlock.RetCompliance || objRetention.RetainUntilDate.Before(ret.RetainUntilDate.Time) { if objRetention.Mode != objectlock.RetCompliance || objRetention.RetainUntilDate.Before((ret.RetainUntilDate.Time)) {
return ObjectLocked{Bucket: oi.Bucket, Object: oi.Name, VersionID: oi.VersionID} return ObjectLocked{Bucket: oi.Bucket, Object: oi.Name, VersionID: oi.VersionID}
} }
apiErr := isPutRetentionAllowed(oi.Bucket, oi.Name, apiErr := isPutRetentionAllowed(oi.Bucket, oi.Name,
@@ -244,7 +242,7 @@ func enforceRetentionBypassForPut(ctx context.Context, r *http.Request, oi Objec
// For objects in "Compliance" mode, retention date cannot be shortened, and mode cannot be altered. // For objects in "Compliance" mode, retention date cannot be shortened, and mode cannot be altered.
// For objects with legal hold header set, the s3:PutObjectLegalHold permission is expected to be set // For objects with legal hold header set, the s3:PutObjectLegalHold permission is expected to be set
// Both legal hold and retention can be applied independently on an object // Both legal hold and retention can be applied independently on an object
func checkPutObjectLockAllowed(ctx context.Context, rq *http.Request, bucket, object string, getObjectInfoFn GetObjectInfoFn, retentionPermErr, legalHoldPermErr APIErrorCode, replicaTrusted bool) (objectlock.RetMode, objectlock.RetentionDate, objectlock.ObjectLegalHold, APIErrorCode) { func checkPutObjectLockAllowed(ctx context.Context, rq *http.Request, bucket, object string, getObjectInfoFn GetObjectInfoFn, retentionPermErr, legalHoldPermErr APIErrorCode) (objectlock.RetMode, objectlock.RetentionDate, objectlock.ObjectLegalHold, APIErrorCode) {
var mode objectlock.RetMode var mode objectlock.RetMode
var retainDate objectlock.RetentionDate var retainDate objectlock.RetentionDate
var legalHold objectlock.ObjectLegalHold var legalHold objectlock.ObjectLegalHold
@@ -271,7 +269,9 @@ func checkPutObjectLockAllowed(ctx context.Context, rq *http.Request, bucket, ob
return mode, retainDate, legalHold, toAPIErrorCode(ctx, err) return mode, retainDate, legalHold, toAPIErrorCode(ctx, err)
} }
if opts.VersionID != "" && !replicaTrusted { replica := rq.Header.Get(xhttp.AmzBucketReplicationStatus) == replication.Replica.String()
if opts.VersionID != "" && !replica {
if objInfo, err := getObjectInfoFn(ctx, bucket, object, opts); err == nil { if objInfo, err := getObjectInfoFn(ctx, bucket, object, opts); err == nil {
r := objectlock.GetObjectRetentionMeta(objInfo.UserDefined) r := objectlock.GetObjectRetentionMeta(objInfo.UserDefined)
t, err := objectlock.UTCNowNTP() t, err := objectlock.UTCNowNTP()
@@ -307,8 +307,8 @@ func checkPutObjectLockAllowed(ctx context.Context, rq *http.Request, bucket, ob
if err != nil { if err != nil {
return mode, retainDate, legalHold, toAPIErrorCode(ctx, err) return mode, retainDate, legalHold, toAPIErrorCode(ctx, err)
} }
rMode, rDate, err := objectlock.ParseObjectLockRetentionHeaders(rq.Header, replicaTrusted) rMode, rDate, err := objectlock.ParseObjectLockRetentionHeaders(rq.Header)
if err != nil && (!replicaTrusted || rMode != "" || !rDate.IsZero()) { if err != nil && (!replica || rMode != "" || !rDate.IsZero()) {
return mode, retainDate, legalHold, toAPIErrorCode(ctx, err) return mode, retainDate, legalHold, toAPIErrorCode(ctx, err)
} }
if retentionPermErr != ErrNone { if retentionPermErr != ErrNone {
@@ -316,7 +316,7 @@ func checkPutObjectLockAllowed(ctx context.Context, rq *http.Request, bucket, ob
} }
return rMode, rDate, legalHold, ErrNone return rMode, rDate, legalHold, ErrNone
} }
if replicaTrusted { // replica inherits retention metadata only from source if replica { // replica inherits retention metadata only from source
return "", objectlock.RetentionDate{}, legalHold, ErrNone return "", objectlock.RetentionDate{}, legalHold, ErrNone
} }
if !retentionRequested && retentionCfg.Validity > 0 { if !retentionRequested && retentionCfg.Validity > 0 {
+1 -5
View File
@@ -617,7 +617,7 @@ func (api objectAPIHandlers) ValidateBucketReplicationCredsHandler(w http.Respon
ReplicationValidityCheck: true, // set this to validate the replication config ReplicationValidityCheck: true, // set this to validate the replication config
}, },
} }
obj := replicationValidationObject(rule) obj := path.Join(minioReservedBucket, globalLocalNodeNameHex, "deleteme")
ui, err := c.PutObject(ctx, clnt.Bucket, obj, reader, int64(len(buf)), "", "", putOpts) ui, err := c.PutObject(ctx, clnt.Bucket, obj, reader, int64(len(buf)), "", "", putOpts)
if err != nil && !isReplicationPermissionCheck(ErrorRespToObjectError(err, bucket, obj)) { if err != nil && !isReplicationPermissionCheck(ErrorRespToObjectError(err, bucket, obj)) {
writeErrorResponse(ctx, w, errorCodes.ToAPIErrWithErr(ErrReplicationValidationError, fmt.Errorf("s3:ReplicateObject permissions missing for replication user: %w", err)), r.URL) writeErrorResponse(ctx, w, errorCodes.ToAPIErrWithErr(ErrReplicationValidationError, fmt.Errorf("s3:ReplicateObject permissions missing for replication user: %w", err)), r.URL)
@@ -658,7 +658,3 @@ func (api objectAPIHandlers) ValidateBucketReplicationCredsHandler(w http.Respon
// Write success response. // Write success response.
writeSuccessResponseHeadersOnly(w) writeSuccessResponseHeadersOnly(w)
} }
func replicationValidationObject(rule replication.Rule) string {
return path.Join(rule.Prefix(), minioReservedBucket, globalLocalNodeNameHex, "deleteme")
}
-20
View File
@@ -20,7 +20,6 @@ package cmd
import ( import (
"fmt" "fmt"
"net/http" "net/http"
"path"
"testing" "testing"
"time" "time"
@@ -288,22 +287,3 @@ func TestReplicationResyncwrapper(t *testing.T) {
} }
} }
} }
func TestReplicationValidationObjectUsesRulePrefix(t *testing.T) {
tests := []struct {
name string
rule replication.Rule
want string
}{
{name: "empty prefix", rule: replication.Rule{}, want: path.Join(minioReservedBucket, globalLocalNodeNameHex, "deleteme")},
{name: "filter prefix", rule: replication.Rule{Filter: replication.Filter{Prefix: "data/"}}, want: path.Join("data", minioReservedBucket, globalLocalNodeNameHex, "deleteme")},
{name: "and prefix", rule: replication.Rule{Filter: replication.Filter{And: replication.And{Prefix: "archive/"}}}, want: path.Join("archive", minioReservedBucket, globalLocalNodeNameHex, "deleteme")},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
if got := replicationValidationObject(test.rule); got != test.want {
t.Fatalf("replicationValidationObject() = %q, want %q", got, test.want)
}
})
}
}
+9 -34
View File
@@ -17,30 +17,7 @@
package cmd package cmd
import ( import "runtime"
"runtime"
"strconv"
"time"
)
// upstreamCopyrightEndYear is the last year of upstream MinIO, Inc. development
// this fork derives from. It is fixed: the fork's own maintenance does not
// extend the upstream copyright term.
const upstreamCopyrightEndYear = "2025"
// forkCopyrightStartYear is the year the Silo fork began.
const forkCopyrightStartYear = "2025"
// copyrightEndYear returns the end year shown in copyright notices for the
// fork's own modifications. Release builds inject CopyrightYear through
// ldflags; source builds keep the "0000" placeholder, for which the current
// year is used instead.
func copyrightEndYear() string {
if CopyrightYear == "" || CopyrightYear == "0000" {
return strconv.Itoa(time.Now().Year())
}
return CopyrightYear
}
// DO NOT EDIT THIS FILE DIRECTLY. These are build-time constants // DO NOT EDIT THIS FILE DIRECTLY. These are build-time constants
// set through ‘buildscripts/gen-ldflags.go’. // set through ‘buildscripts/gen-ldflags.go’.
@@ -72,22 +49,20 @@ var (
// MinioOSARCH - OS and ARCH. // MinioOSARCH - OS and ARCH.
minioOSARCH = runtime.GOOS + "-" + runtime.GOARCH minioOSARCH = runtime.GOOS + "-" + runtime.GOARCH
// MinioReleaseBaseURL is retained for source compatibility. Silo does not // MinioReleaseBaseURL - release url without os and arch.
// provide an in-place update endpoint. MinioReleaseBaseURL = "https://dl.min.io/server/minio/release/"
MinioReleaseBaseURL = ""
// MinioReleaseURL - release URL. // MinioReleaseURL - release URL.
MinioReleaseURL = MinioReleaseBaseURL + minioOSARCH + SlashSeparator MinioReleaseURL = MinioReleaseBaseURL + minioOSARCH + SlashSeparator
// MinioStoreName - Silo product name. The identifier is retained to avoid a // MinioStoreName - MinIO store name.
// source-only rename across compatibility-sensitive code. MinioStoreName = "MinIO"
MinioStoreName = "Silo"
// MinioUAName - Silo user agent name. // MinioUAName - MinIO user agent name.
MinioUAName = "Silo" MinioUAName = "MinIO"
// MinioBannerName - Silo banner name for startup message. // MinioBannerName - MinIO banner name for startup message.
MinioBannerName = "Silo Object Storage Server" MinioBannerName = "MinIO Object Storage Server"
// MinioLicense - MinIO server license. // MinioLicense - MinIO server license.
MinioLicense = "GNU AGPLv3 - https://www.gnu.org/licenses/agpl-3.0.html" MinioLicense = "GNU AGPLv3 - https://www.gnu.org/licenses/agpl-3.0.html"
+199
View File
@@ -0,0 +1,199 @@
// Copyright (c) 2015-2022 MinIO, Inc.
//
// This file is part of MinIO Object Storage stack
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package cmd
import (
"bytes"
"compress/gzip"
"context"
"encoding/json"
"errors"
"fmt"
"math/rand"
"net/url"
"time"
"github.com/minio/madmin-go/v3"
)
var callhomeLeaderLockTimeout = newDynamicTimeout(30*time.Second, 10*time.Second)
// initCallhome will start the callhome task in the background.
func initCallhome(ctx context.Context, objAPI ObjectLayer) {
if !globalCallhomeConfig.Enabled() {
return
}
go func() {
r := rand.New(rand.NewSource(time.Now().UnixNano()))
// Leader node (that successfully acquires the lock inside runCallhome)
// will keep performing the callhome. If the leader goes down for some reason,
// the lock will be released and another node will acquire it and take over
// because of this loop.
for {
if !globalCallhomeConfig.Enabled() {
return
}
if !runCallhome(ctx, objAPI) {
// callhome was disabled or context was canceled
return
}
// callhome running on a different node.
// sleep for some time and try again.
duration := max(time.Duration(r.Float64()*float64(globalCallhomeConfig.FrequencyDur())),
// Make sure to sleep at least a second to avoid high CPU ticks.
time.Second)
time.Sleep(duration)
}
}()
}
func runCallhome(ctx context.Context, objAPI ObjectLayer) bool {
// Make sure only 1 callhome is running on the cluster.
locker := objAPI.NewNSLock(minioMetaBucket, "callhome/runCallhome.lock")
lkctx, err := locker.GetLock(ctx, callhomeLeaderLockTimeout)
if err != nil {
// lock timedout means some other node is the leader,
// cycle back return 'true'
return true
}
ctx = lkctx.Context()
defer locker.Unlock(lkctx)
// Perform callhome once and then keep running it at regular intervals.
performCallhome(ctx)
callhomeTimer := time.NewTimer(globalCallhomeConfig.FrequencyDur())
defer callhomeTimer.Stop()
for {
if !globalCallhomeConfig.Enabled() {
// Stop the processing as callhome got disabled
return false
}
select {
case <-ctx.Done():
// indicates that we do not need to run callhome anymore
return false
case <-callhomeTimer.C:
if !globalCallhomeConfig.Enabled() {
// Stop the processing as callhome got disabled
return false
}
performCallhome(ctx)
// Reset the timer for next cycle.
callhomeTimer.Reset(globalCallhomeConfig.FrequencyDur())
}
}
}
func performCallhome(ctx context.Context) {
deadline := 10 * time.Second // Default deadline is 10secs for callhome
objectAPI := newObjectLayerFn()
if objectAPI == nil {
internalLogIf(ctx, errors.New("Callhome: object layer not ready"))
return
}
healthCtx, healthCancel := context.WithTimeout(ctx, deadline)
defer healthCancel()
healthInfoCh := make(chan madmin.HealthInfo)
query := url.Values{}
for _, k := range madmin.HealthDataTypesList {
query.Set(string(k), "true")
}
healthInfo := madmin.HealthInfo{
TimeStamp: time.Now().UTC(),
Version: madmin.HealthInfoVersion,
Minio: madmin.MinioHealthInfo{
Info: madmin.MinioInfo{
DeploymentID: globalDeploymentID(),
},
},
}
go fetchHealthInfo(healthCtx, objectAPI, &query, healthInfoCh, healthInfo)
for {
select {
case hi, hasMore := <-healthInfoCh:
if !hasMore {
auditOptions := AuditLogOptions{Event: "callhome:diagnostics"}
// Received all data. Send to SUBNET and return
err := sendHealthInfo(ctx, healthInfo)
if err != nil {
internalLogIf(ctx, fmt.Errorf("Unable to perform callhome: %w", err))
auditOptions.Error = err.Error()
}
auditLogInternal(ctx, auditOptions)
return
}
healthInfo = hi
case <-healthCtx.Done():
return
}
}
}
const (
subnetHealthPath = "/api/health/upload"
)
func sendHealthInfo(ctx context.Context, healthInfo madmin.HealthInfo) error {
url := globalSubnetConfig.BaseURL + subnetHealthPath
filename := fmt.Sprintf("health_%s.json.gz", UTCNow().Format("20060102150405"))
url += "?filename=" + filename
_, err := globalSubnetConfig.Upload(url, filename, createHealthJSONGzip(ctx, healthInfo))
return err
}
func createHealthJSONGzip(ctx context.Context, healthInfo madmin.HealthInfo) []byte {
var b bytes.Buffer
gzWriter := gzip.NewWriter(&b)
header := struct {
Version string `json:"version"`
}{Version: healthInfo.Version}
enc := json.NewEncoder(gzWriter)
if e := enc.Encode(header); e != nil {
internalLogIf(ctx, fmt.Errorf("Could not encode health info header: %w", e))
return nil
}
if e := enc.Encode(healthInfo); e != nil {
internalLogIf(ctx, fmt.Errorf("Could not encode health info: %w", e))
return nil
}
gzWriter.Flush()
gzWriter.Close()
return b.Bytes()
}

Some files were not shown because too many files have changed in this diff Show More