Compare commits

..

1 Commits

Author SHA1 Message Date
Feng Ruohang 5d57938939 add github ci/cd pipeline 2026-02-14 17:20:37 +08:00
1071 changed files with 22187 additions and 94294 deletions
+51
View File
@@ -0,0 +1,51 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: community, triage
assignees: ''
---
## NOTE
All GitHub issues are addressed on a best-effort basis at MinIO's sole discretion. There are no Service Level Agreements (SLA) or Objectives (SLO). Remember our [Code of Conduct](https://github.com/minio/minio/blob/master/code_of_conduct.md) when engaging with MinIO Engineers and the larger community.
For urgent issues (e.g. production down, etc.), subscribe to [SUBNET](https://min.io/pricing?jmp=github) for direct to engineering support.
<!--- Provide a general summary of the issue in the Title above -->
## Expected Behavior
<!--- If you're describing a bug, tell us what should happen -->
<!--- If you're suggesting a change/improvement, tell us how it should work -->
## Current Behavior
<!--- If describing a bug, tell us what happens instead of the expected behavior -->
<!--- If suggesting a change/improvement, explain the difference from current behavior -->
## Possible Solution
<!--- Not obligatory, but suggest a fix/reason for the bug, -->
<!--- or ideas how to implement the addition or change -->
## Steps to Reproduce (for bugs)
<!--- Provide a link to a live example, or an unambiguous set of steps to -->
<!--- reproduce this bug. Include code to reproduce, if relevant -->
<!--- and make sure you have followed https://github.com/minio/minio/tree/release/docs/debugging to capture relevant logs -->
1.
2.
3.
4.
## Context
<!--- How has this issue affected you? What are you trying to accomplish? -->
<!--- Providing context helps us come up with a solution that is most useful in the real world -->
## Regression
<!-- Is this issue a regression? (Yes / No) -->
<!-- If Yes, optionally please include minio version or commit id or PR# that caused this regression, if you have these details. -->
## Your Environment
<!--- Include as many relevant details about the environment you experienced the bug in -->
* Version used (`minio --version`):
* Server setup and configuration:
* Operating System and version (`uname -a`):
+3 -11
View File
@@ -7,14 +7,6 @@ assignees: ''
---
Report bugs in the PGSTY SILO server (`pgsty/silo`) here. Community maintainers
handle reports on a best-effort basis. There is no SLA, SLO, or emergency
production-support channel. Follow the
[Code of Conduct](https://github.com/pgsty/silo/blob/main/code_of_conduct.md).
Report suspected vulnerabilities privately through
[SECURITY.md](https://github.com/pgsty/silo/blob/main/SECURITY.md).
For patches, see the [contribution guide](https://github.com/pgsty/silo/blob/main/CONTRIBUTING.md).
<!--- Provide a general summary of the issue in the Title above -->
## Expected Behavior
@@ -32,7 +24,7 @@ For patches, see the [contribution guide](https://github.com/pgsty/silo/blob/mai
## Steps to Reproduce (for bugs)
<!--- Provide a link to a live example, or an unambiguous set of steps to -->
<!--- reproduce this bug. Include code to reproduce, if relevant -->
<!--- and include the relevant Silo logs with secrets and credentials removed -->
<!--- and make sure you have followed https://github.com/minio/minio/tree/release/docs/debugging to capture relevant logs -->
1.
2.
@@ -45,10 +37,10 @@ For patches, see the [contribution guide](https://github.com/pgsty/silo/blob/mai
## Regression
<!-- Is this issue a regression? (Yes / No) -->
<!-- If Yes, optionally include the Silo version, commit id, or PR that caused the regression. -->
<!-- If Yes, optionally please include minio version or commit id or PR# that caused this regression, if you have these details. -->
## Your Environment
<!--- Include as many relevant details about the environment you experienced the bug in -->
* Version used (`silo --version`):
* Version used (`minio --version`):
* Server setup and configuration:
* Operating System and version (`uname -a`):
+6 -6
View File
@@ -1,8 +1,8 @@
blank_issues_enabled: false
contact_links:
- name: Silo Documentation
url: https://silo.pgsty.com/docs/
about: Installation, configuration, operations, and compatibility guidance
- name: Private Security Report
url: https://github.com/pgsty/silo/security/advisories/new
about: Privately report a suspected vulnerability in Silo
- name: MinIO Community Support
url: https://slack.min.io
about: Community support via Slack - for questions and discussions
- name: MinIO Enterprise Support (SUBNET)
url: https://min.io/pricing
about: Enterprise support with SLA - for production deployments and feature requests
@@ -7,9 +7,6 @@ assignees: ''
---
Suggest improvements to the PGSTY SILO server (`pgsty/silo`) here.
For patches, see the [contribution guide](https://github.com/pgsty/silo/blob/main/CONTRIBUTING.md).
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+6 -25
View File
@@ -1,17 +1,8 @@
## Contribution Licensing (no CLA, inbound=outbound, DCO required)
This pull request contributes to PGSTY SILO (`pgsty/silo`). Code contributions
are accepted under AGPL-3.0-or-later, the same license as the server.
This project does not use a CLA or require a separate Apache-2.0 license grant.
By submitting this pull request I represent that I have the right to contribute
the code changes under this repository's
[GNU Affero General Public License v3.0 or later](https://www.gnu.org/licenses/agpl-3.0.html)
and retain copyright in my original work. Existing copyright and license
notices remain intact; separately licensed material keeps its applicable terms.
Every commit must carry a DCO `Signed-off-by` trailer
(`git commit -s`) certifying the
[Developer Certificate of Origin](https://developercertificate.org/) — see
[CONTRIBUTING.md](https://github.com/pgsty/silo/blob/main/CONTRIBUTING.md).
## Community Contribution License
All community contributions in this pull request are licensed to the project maintainers
under the terms of the [Apache 2 license](https://www.apache.org/licenses/LICENSE-2.0).
By creating this pull request I represent that I have the right to license the
contributions to the project maintainers under the Apache 2 license.
## Description
@@ -22,12 +13,6 @@ Every commit must carry a DCO `Signed-off-by` trailer
## How to test this PR?
## Compatibility impact
<!-- Note effects on APIs, clients, MINIO_* configuration, metrics, headers,
routes, storage metadata, module/import paths, upgrades, or rollback. -->
## Types of changes
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
@@ -35,11 +20,7 @@ routes, storage metadata, module/import paths, upgrades, or rollback. -->
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Checklist:
- [ ] All commits are signed off (`git commit -s`) per the [DCO](https://developercertificate.org/)
- [ ] Fixes a regression (If yes, please add `commit-id` or `PR #` here)
- [ ] Unit tests added/updated
- [ ] `make verifiers` passes
- [ ] Relevant package tests and `make build` pass
- [ ] Compatibility and rollback impact documented
- [ ] Internal documentation updated
- [ ] Public documentation update opened in `pgsty/silo.pgsty.com`, if needed
- [ ] Create a documentation update request [here](https://github.com/minio/docs/issues/new?label=doc-change,title=Doc+Updated+Needed+For+PR+github.com%2fminio%2fminio%2fpull%2fNNNNN)
+59 -49
View File
@@ -4,9 +4,9 @@ env:
- CGO_ENABLED=0
builds:
- id: silo
- id: minio
main: .
binary: silo
binary: minio
goos:
- linux
- darwin
@@ -23,64 +23,74 @@ builds:
- "{{ .Env.LDFLAGS }}"
archives:
- id: silo
- id: minio
ids:
- silo
name_template: "silo_{{ .Env.PKG_VERSION }}_{{ .Os }}_{{ .Arch }}"
# Explicit so the license materials cannot silently drop out of the
# binary archives: GoReleaser's default file globs would miss NOTICE.
# CREDITS stays out deliberately -- at 1.8MB it would dominate the
# archive; it remains available in the repository and the OCI image.
files:
- README.md
- minio
name_template: "minio_{{ .Env.PKG_VERSION }}_{{ .Os }}_{{ .Arch }}"
dockers:
- id: minio-amd64
ids:
- minio
goos: linux
goarch: amd64
dockerfile: Dockerfile.goreleaser
use: buildx
image_templates:
- "pgsty/minio:{{ .Tag }}-amd64"
- "pgsty/minio:latest-amd64"
build_flag_templates:
- "--platform=linux/amd64"
- "--label=org.opencontainers.image.version={{ .Tag }}"
- "--label=org.opencontainers.image.created={{ .Date }}"
- "--label=org.opencontainers.image.revision={{ .FullCommit }}"
extra_files:
- dockerscripts/docker-entrypoint.sh
- LICENSE
- NOTICE
- CREDITS
- id: minio-arm64
ids:
- minio
goos: linux
goarch: arm64
dockerfile: Dockerfile.goreleaser
use: buildx
image_templates:
- "pgsty/minio:{{ .Tag }}-arm64"
- "pgsty/minio:latest-arm64"
build_flag_templates:
- "--platform=linux/arm64"
- "--label=org.opencontainers.image.version={{ .Tag }}"
- "--label=org.opencontainers.image.created={{ .Date }}"
- "--label=org.opencontainers.image.revision={{ .FullCommit }}"
extra_files:
- dockerscripts/docker-entrypoint.sh
- LICENSE
- CREDITS
docker_manifests:
- name_template: "pgsty/minio:{{ .Tag }}"
image_templates:
- "pgsty/minio:{{ .Tag }}-amd64"
- "pgsty/minio:{{ .Tag }}-arm64"
- name_template: "pgsty/minio:latest"
image_templates:
- "pgsty/minio:latest-amd64"
- "pgsty/minio:latest-arm64"
checksum:
name_template: "silo_{{ .Env.PKG_VERSION }}_checksums.txt"
name_template: "minio_{{ .Env.PKG_VERSION }}_checksums.txt"
algorithm: sha256
# Generate one SPDX JSON document per platform archive. SBOMs are created
# before the checksum stage, so the signed checksum manifest covers both the
# archives and their corresponding software bills of materials.
sboms:
- id: silo-archives
artifacts: archive
# Avoid network-backed package enrichment: the release SBOM must be
# reproducible from the artifact alone and the PR gate must work offline.
args:
- "$artifact"
- "--output"
- "spdx-json=$document"
env:
- SYFT_FILE_METADATA_CATALOGER_ENABLED=true
- SYFT_CHECK_FOR_APP_UPDATE=false
# A keyless Sigstore bundle is the detached signature for the checksum
# manifest. Consumers can verify the whole archive/SBOM set without trusting a
# long-lived project key copied into the repository.
signs:
- id: silo-checksums
cmd: cosign
signature: "${artifact}.sigstore.json"
args:
- sign-blob
- "--bundle=${signature}"
- "${artifact}"
- --yes
artifacts: checksum
output: true
release:
github:
owner: pgsty
name: silo
draft: true
name: minio
draft: false
prerelease: false
replace_existing_draft: true
replace_existing_artifacts: false
mode: replace
# Draft replacement matches the release name; keep it identical to the tag.
replace_existing_artifacts: true
name_template: "{{ .Tag }}"
changelog:
+1
View File
@@ -0,0 +1 @@
<svg data-name="Layer 1" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 162.612 24.465"><path d="M52.751.414h9.108v23.63h-9.108zM41.711.74l-18.488 9.92a.919.919 0 0 1-.856 0L3.879.74A2.808 2.808 0 0 0 2.558.414h-.023A2.4 2.4 0 0 0 0 2.641v21.376h9.1V13.842a.918.918 0 0 1 1.385-.682l10.361 5.568a3.634 3.634 0 0 0 3.336.028l10.933-5.634a.917.917 0 0 1 1.371.69v10.205h9.1V2.641A2.4 2.4 0 0 0 43.055.414h-.023a2.808 2.808 0 0 0-1.321.326zm65.564-.326h-9.237v10.755a.913.913 0 0 1-1.338.706L72.762.675a2.824 2.824 0 0 0-1.191-.261h-.016a2.4 2.4 0 0 0-2.535 2.227v21.377h9.163V13.275a.914.914 0 0 1 1.337-.707l24.032 11.2a2.813 2.813 0 0 0 1.188.26 2.4 2.4 0 0 0 2.535-2.227zm7.161 23.63V.414h4.191v23.63zm28.856.421c-11.274 0-19.272-4.7-19.272-12.232C124.02 4.741 132.066 0 143.292 0s19.32 4.7 19.32 12.233-7.902 12.232-19.32 12.232zm0-21.333c-8.383 0-14.84 3.217-14.84 9.1 0 5.926 6.457 9.1 14.84 9.1s14.887-3.174 14.887-9.1c0-5.883-6.504-9.1-14.887-9.1z" fill="#c72c48"/></svg>

After

Width:  |  Height:  |  Size: 978 B

-111
View File
@@ -1,111 +0,0 @@
name: silo
arch: ${NFPM_ARCH}
platform: linux
version: ${PKG_VERSION}
version_schema: none
release: ${NFPM_RELEASE}
section: utils
priority: optional
maintainer: "Ruohang Feng (@Vonng) <rh@vonng.com>"
description: S3-Interface Libre Object Storage, a community-maintained S3-compatible server.
vendor: PGSTY
homepage: https://silo.pgsty.com
license: AGPL-3.0-or-later
contents:
- src: ${NFPM_SOURCE}
dst: /usr/bin/silo
expand: true
file_info:
mode: 0755
owner: root
group: root
- src: ${NFPM_UNIT}
dst: /usr/lib/systemd/system/silo.service
expand: true
file_info:
mode: 0644
owner: root
group: root
- src: ${NFPM_DEFAULTS}
dst: /etc/default/silo
type: config|noreplace
expand: true
file_info:
mode: 0644
owner: root
group: root
- src: ${NFPM_SYSUSERS}
dst: /usr/lib/sysusers.d/silo.conf
expand: true
file_info:
mode: 0644
owner: root
group: root
# The license materials are declared once per packager: nfpm only honors
# type: license on rpm and silently drops such entries from deb and apk, so
# the rpm keeps its %license flag while deb and apk carry plain files at the
# same path.
- src: ${NFPM_LICENSE}
dst: /usr/share/doc/silo/LICENSE
type: license
packager: rpm
expand: true
file_info:
mode: 0644
owner: root
group: root
- src: ${NFPM_NOTICE}
dst: /usr/share/doc/silo/NOTICE
type: license
packager: rpm
expand: true
file_info:
mode: 0644
owner: root
group: root
- src: ${NFPM_LICENSE}
dst: /usr/share/doc/silo/LICENSE
packager: deb
expand: true
file_info:
mode: 0644
owner: root
group: root
- src: ${NFPM_NOTICE}
dst: /usr/share/doc/silo/NOTICE
packager: deb
expand: true
file_info:
mode: 0644
owner: root
group: root
- src: ${NFPM_LICENSE}
dst: /usr/share/doc/silo/LICENSE
packager: apk
expand: true
file_info:
mode: 0644
owner: root
group: root
- src: ${NFPM_NOTICE}
dst: /usr/share/doc/silo/NOTICE
packager: apk
expand: true
file_info:
mode: 0644
owner: root
group: root
scripts:
postinstall: buildscripts/package/postinstall.sh
preremove: buildscripts/package/preremove.sh
rpm:
group: Applications/File
compression: gzip:9
deb:
compression: gzip
fields:
License: AGPL-3.0-or-later
-105
View File
@@ -1,105 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg"
width="1500" height="570" viewBox="0 0 1500 570"
preserveAspectRatio="xMidYMid meet"
role="img" aria-labelledby="silo-logo-title silo-logo-desc"
shape-rendering="geometricPrecision">
<title id="silo-logo-title">SILO logo</title>
<desc id="silo-logo-desc">The SILO horizontal lockup: the circular silo emblem on the left, the SILO wordmark on the right.</desc>
<defs>
<!-- Emblem gradient, in the emblem's local coordinates; the group transform maps it. -->
<linearGradient id="silo-logo-mark-color" x1="276" y1="720" x2="742" y2="286" gradientUnits="userSpaceOnUse">
<stop offset="0" stop-color="#064A83"/>
<stop offset="0.5" stop-color="#007FA8"/>
<stop offset="1" stop-color="#22C7C9"/>
</linearGradient>
<!-- Wordmark gradient, in the wordmark's local coordinates. -->
<linearGradient id="silo-logo-word-color" x1="44.0" y1="-94.4" x2="1950.0" y2="794.4" gradientUnits="userSpaceOnUse">
<stop class="silo-logo-wm-a" offset="0.06" stop-color="#1d588c"/>
<stop class="silo-logo-wm-b" offset="0.94" stop-color="#b4762e"/>
</linearGradient>
<style>
/* Light theme values of --pg-strong / --copper; dark theme swaps in its own pair. */
@media (prefers-color-scheme: dark) {
.silo-logo-wm-a { stop-color: #7fb8e8; }
.silo-logo-wm-b { stop-color: #e0a35c; }
}
</style>
</defs>
<!-- Circular emblem, shifted from its native viewBox (230 213 570 570) to the 0..570 square. -->
<g id="silo-logo-mark" transform="translate(-230 -213)" fill="url(#silo-logo-mark-color)">
<!-- Outer circular band, intentionally opened at the lower-right plinth. -->
<path d="
M 734 676
A 283.5 278.5 0 1 0 310 692
L 359 692
A 247 248.5 0 1 1 688 676
Z"/>
<!-- Flowing left wall; its upper tangent matches the inner ellipse. -->
<path d="
M 300.57 375
C 292 390 300 430 328 450
C 343 461 357 472 374 481
C 410 501 426 517 426 544
L 426 676
L 336 676
C 308 647 286 608 274 565
C 262 522 263 479 272 439
C 278 413 286 389 300.57 375
Z"/>
<!-- Right column with tangent-continuous upper shoulder. -->
<path d="
M 602 373
Q 602 368 607 370
C 619 374 634 381 634 389
L 634 647
Q 634 649 636 649
L 708 649
L 734 676
L 602 676
Z"/>
<!-- Lower circular cap. -->
<path d="
M 310 692
L 714 692
C 668 743 596 774 512 774
C 428 774 355 743 310 692
Z"/>
<!-- Main silo body, with a tangent-continuous right shoulder. -->
<path d="
M 389 389
C 389 374 447 351 512 351
C 540 351 565 354 580 359
Q 583 360 583 364
L 583 676
L 443 676
L 443 541
C 443 509 426 490 389 470
Z"/>
<!-- Peaked roof with softly tapered, burr-free tips. -->
<path d="
M 512 274
L 647 365
Q 649 370 647 376
C 609 350 563 337 512 337
C 460 337 414 350 377 376
Q 375 370 377 365
Z"/>
</g>
<!-- Wordmark, scaled 3/7 to a 300-unit cap height and centered on the emblem's axis. -->
<g id="silo-logo-word" transform="translate(645.429 135) scale(0.428571)"
fill="url(#silo-logo-word-color)" fill-rule="nonzero">
<path id="silo-logo-s" d="M0 592V492H134V551L167 584H374L408 550V434L375 401H110L2 293V108L110 0H426L534 108V209H400V149L367 116H169L136 149V252L169 285H434L542 393V590L432 700H108Z"/>
<path id="silo-logo-i" d="M637 0H773V700H637Z"/>
<path id="silo-logo-l" d="M888 0H1024V585H1374V700H888Z"/>
<path id="silo-logo-o" d="M1404 585V115L1519 0H1879L1994 115V585L1879 700H1519ZM1807 584 1858 533V167L1807 116H1591L1540 167V533L1591 584Z"/>
</g>
</svg>

Before

Width:  |  Height:  |  Size: 3.8 KiB

-30
View File
@@ -1,30 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg"
width="1624" height="570" viewBox="0 0 1994 700"
preserveAspectRatio="xMidYMid meet"
role="img" aria-labelledby="title desc"
shape-rendering="geometricPrecision">
<title id="title">SILO wordmark</title>
<desc id="desc">The SILO wordmark set in Chakra Petch Bold, outlined, with the blue-to-copper brand gradient.</desc>
<defs>
<linearGradient id="silo-wordmark-color" x1="44.0" y1="-94.4" x2="1950.0" y2="794.4" gradientUnits="userSpaceOnUse">
<stop class="wm-a" offset="0.06" stop-color="#1d588c"/>
<stop class="wm-b" offset="0.94" stop-color="#b4762e"/>
</linearGradient>
<style>
/* Light theme values of --pg-strong / --copper; dark theme swaps in its own pair. */
@media (prefers-color-scheme: dark) {
.wm-a { stop-color: #7fb8e8; }
.wm-b { stop-color: #e0a35c; }
}
</style>
</defs>
<g fill="url(#silo-wordmark-color)" fill-rule="nonzero">
<path id="silo-s" d="M0 592V492H134V551L167 584H374L408 550V434L375 401H110L2 293V108L110 0H426L534 108V209H400V149L367 116H169L136 149V252L169 285H434L542 393V590L432 700H108Z"/>
<path id="silo-i" d="M637 0H773V700H637Z"/>
<path id="silo-l" d="M888 0H1024V585H1374V700H888Z"/>
<path id="silo-o" d="M1404 585V115L1519 0H1879L1994 115V585L1879 700H1519ZM1807 584 1858 533V167L1807 116H1591L1540 167V533L1591 584Z"/>
</g>
</svg>

Before

Width:  |  Height:  |  Size: 1.4 KiB

-82
View File
@@ -1,82 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg"
width="570" height="570" viewBox="230 213 570 570"
preserveAspectRatio="xMidYMid meet"
role="img" aria-labelledby="title desc"
shape-rendering="geometricPrecision">
<title id="title">SILO emblem</title>
<desc id="desc">A smooth circular SILO mark with a peaked roof, flowing left wall, columns, and a curved base.</desc>
<defs>
<linearGradient id="silo-color" x1="276" y1="720" x2="742" y2="286" gradientUnits="userSpaceOnUse">
<stop offset="0" stop-color="#064A83"/>
<stop offset="0.5" stop-color="#007FA8"/>
<stop offset="1" stop-color="#22C7C9"/>
</linearGradient>
</defs>
<g fill="url(#silo-color)">
<!-- Outer circular band, intentionally opened at the lower-right plinth. -->
<path d="
M 734 676
A 283.5 278.5 0 1 0 310 692
L 359 692
A 247 248.5 0 1 1 688 676
Z"/>
<!-- Flowing left wall; its upper tangent matches the inner ellipse. -->
<path d="
M 300.57 375
C 292 390 300 430 328 450
C 343 461 357 472 374 481
C 410 501 426 517 426 544
L 426 676
L 336 676
C 308 647 286 608 274 565
C 262 522 263 479 272 439
C 278 413 286 389 300.57 375
Z"/>
<!-- Right column with tangent-continuous upper shoulder. -->
<path d="
M 602 373
Q 602 368 607 370
C 619 374 634 381 634 389
L 634 647
Q 634 649 636 649
L 708 649
L 734 676
L 602 676
Z"/>
<!-- Lower circular cap. -->
<path d="
M 310 692
L 714 692
C 668 743 596 774 512 774
C 428 774 355 743 310 692
Z"/>
<!-- Main silo body, with a tangent-continuous right shoulder. -->
<path d="
M 389 389
C 389 374 447 351 512 351
C 540 351 565 354 580 359
Q 583 360 583 364
L 583 676
L 443 676
L 443 541
C 443 509 426 490 389 470
Z"/>
<!-- Peaked roof with softly tapered, burr-free tips. -->
<path d="
M 512 274
L 647 365
Q 649 370 647 376
C 609 350 563 337 512 337
C 460 337 414 350 377 376
Q 375 370 377 365
Z"/>
</g>
</svg>

Before

Width:  |  Height:  |  Size: 2.2 KiB

-43
View File
@@ -1,43 +0,0 @@
name: DCO
on:
pull_request:
branches:
- main
permissions:
contents: read
jobs:
check:
name: Verify DCO sign-off
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
# Every non-merge commit in the pull request must carry a Signed-off-by
# trailer matching the commit author's email, certifying the Developer
# Certificate of Origin 1.1 (https://developercertificate.org/).
# Only commits authored from a GitHub-issued bot address are exempt; a
# display name is attacker-controlled and must never grant the exemption.
- name: Check Signed-off-by trailers
env:
BASE_SHA: ${{ github.event.pull_request.base.sha }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
fail=0
while read -r sha; do
author_name="$(git log -1 --format='%an' "${sha}")"
author_email="$(git log -1 --format='%ae' "${sha}")"
case "${author_email}" in
*"[bot]@users.noreply.github.com") continue ;;
esac
if ! git log -1 --format='%(trailers:key=Signed-off-by,valueonly)' "${sha}" |
grep -qiF "<${author_email}>"; then
echo "::error::commit ${sha} by ${author_name} <${author_email}> lacks a matching Signed-off-by trailer; sign with 'git commit -s', repair with 'git rebase --signoff'"
fail=1
fi
done < <(git rev-list --no-merges "${BASE_SHA}..${HEAD_SHA}")
exit "${fail}"
+14
View File
@@ -0,0 +1,14 @@
name: 'Dependency Review'
on: [pull_request]
permissions:
contents: read
jobs:
dependency-review:
runs-on: ubuntu-latest
steps:
- name: 'Checkout Repository'
uses: actions/checkout@v4
- name: 'Dependency Review'
uses: actions/dependency-review-action@v4
-422
View File
@@ -1,422 +0,0 @@
name: Publish Docker Image
on:
workflow_dispatch:
inputs:
tag:
description: "Published RELEASE.* tag to package as pgsty/silo"
required: true
type: string
recovery:
description: "Run the current main workflow against an already-published tag"
required: false
default: false
type: boolean
permissions:
contents: read
id-token: write
attestations: write
artifact-metadata: write
concurrency:
group: docker-release
cancel-in-progress: false
jobs:
publish:
runs-on: ubuntu-latest
steps:
# Images are built from a published release rather than from the build
# that produced it, so an abandoned draft can never leave :latest
# pointing at something nobody shipped.
- name: Validate published release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
INPUT_TAG: ${{ inputs.tag }}
run: |
set -euo pipefail
TAG="${INPUT_TAG}"
VERSION_HYPHEN="${TAG#RELEASE.}"
PKG_VERSION="$(echo "${VERSION_HYPHEN}" | sed -E 's/^([0-9]{4})-([0-9]{2})-([0-9]{2})T([0-9]{2})-([0-9]{2})-([0-9]{2})Z$/\1\2\3\4\5\6.0.0/')"
if [ "${PKG_VERSION}" = "${VERSION_HYPHEN}" ]; then
echo "Invalid release tag: ${TAG}"
exit 1
fi
IS_DRAFT="$(gh release view "${TAG}" --repo "${GITHUB_REPOSITORY}" --json isDraft --jq .isDraft)"
IS_PRERELEASE="$(gh release view "${TAG}" --repo "${GITHUB_REPOSITORY}" --json isPrerelease --jq .isPrerelease)"
PUBLISHED_AT="$(gh release view "${TAG}" --repo "${GITHUB_REPOSITORY}" --json publishedAt --jq .publishedAt)"
LATEST_TAG="$(gh release view --repo "${GITHUB_REPOSITORY}" --json tagName --jq .tagName)"
if [ "${IS_DRAFT}" != false ] || [ "${IS_PRERELEASE}" != false ] || [ -z "${PUBLISHED_AT}" ]; then
echo "${TAG} must be a published, non-prerelease GitHub Release"
exit 1
fi
# This workflow moves :latest, so it must not run for an older tag.
if [ "${TAG}" != "${LATEST_TAG}" ]; then
echo "Refusing to replace Docker latest with non-latest release ${TAG} (latest is ${LATEST_TAG})"
exit 1
fi
{
echo "RELEASE_TAG=${TAG}"
echo "PKG_VERSION=${PKG_VERSION}"
echo "PUBLISHED_AT=${PUBLISHED_AT}"
} >> "${GITHUB_ENV}"
- name: Validate Docker Hub credentials
run: |
set -euo pipefail
if [ -z "${{ secrets.DOCKERHUB_USERNAME }}" ] || [ -z "${{ secrets.DOCKERHUB_TOKEN }}" ]; then
echo "Missing Docker Hub credentials. Set DOCKERHUB_USERNAME and DOCKERHUB_TOKEN repository secrets."
exit 1
fi
- name: Checkout release tag
uses: actions/checkout@v7
with:
ref: ${{ inputs.tag }}
fetch-depth: 0
- name: Verify workflow identity matches release source
env:
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
RECOVERY: ${{ inputs.recovery }}
run: |
set -euo pipefail
CHECKED_OUT_REVISION="$(git rev-parse HEAD)"
if [ "${CHECKED_OUT_REVISION}" != "${GITHUB_SHA}" ]; then
if [ "${RECOVERY}" != "true" ] || [ "${GITHUB_REF}" != "refs/heads/${DEFAULT_BRANCH}" ]; then
echo "Checked out ${CHECKED_OUT_REVISION}, but workflow identity is ${GITHUB_SHA}. Dispatch from ${RELEASE_TAG}, or use recovery from ${DEFAULT_BRANCH}." >&2
exit 1
fi
echo "Recovery workflow ${GITHUB_SHA} is packaging published source ${CHECKED_OUT_REVISION}."
fi
- name: Prepare verified Docker contexts
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
assets_dir="docker-release/assets"
mkdir -p "${assets_dir}"
amd64_archive="silo_${PKG_VERSION}_linux_amd64.tar.gz"
arm64_archive="silo_${PKG_VERSION}_linux_arm64.tar.gz"
checksums="silo_${PKG_VERSION}_checksums.txt"
gh release download "${RELEASE_TAG}" --repo "${GITHUB_REPOSITORY}" \
--dir "${assets_dir}" \
--pattern "${amd64_archive}" \
--pattern "${arm64_archive}" \
--pattern "${checksums}"
# The binaries going into the images are the published ones, checked
# against the published checksums, not a rebuild that merely ought to
# match them. awk matches the manifest filename column exactly: a
# substring grep would also pull in the archive's .sbom.json line,
# whose file is deliberately not downloaded in this lane.
cd "${assets_dir}"
awk -v name="${amd64_archive}" '$2 == name' "${checksums}" | sha256sum --check
awk -v name="${arm64_archive}" '$2 == name' "${checksums}" | sha256sum --check
# The checksum manifest and both archives must have provenance from
# this repository's release workflow at the exact checked-out tag.
for artifact in "${checksums}" "${amd64_archive}" "${arm64_archive}"; do
gh attestation verify "${artifact}" \
--repo "${GITHUB_REPOSITORY}" \
--signer-workflow "${GITHUB_REPOSITORY}/.github/workflows/release.yml" \
--source-digest "$(git -C "${GITHUB_WORKSPACE}" rev-parse HEAD)" \
--source-ref "refs/tags/${RELEASE_TAG}" >/dev/null
done
cd "${GITHUB_WORKSPACE}"
# Dockerfile.goreleaser expects the binary at the context root and
# the entrypoint scripts under dockerscripts/, which is the layout
# GoReleaser used to assemble via extra_files.
for arch in amd64 arm64; do
context="docker-release/${arch}"
archive="${assets_dir}/silo_${PKG_VERSION}_linux_${arch}.tar.gz"
mkdir -p "${context}/dockerscripts"
tar -xzf "${archive}" -C "${context}" silo
cp Dockerfile.goreleaser Dockerfile.distroless LICENSE NOTICE CREDITS "${context}/"
cp dockerscripts/docker-entrypoint.sh dockerscripts/build-static-curl.sh \
"${context}/dockerscripts/"
done
# The classic image bundles mcli. Resolve its two archive digests
# from the immutable published release instead of trusting defaults
# copied into an older Server tag. This also gives a recovery run a
# narrow override when a tag selected the right mcli release but
# accidentally retained stale archive pins.
MC_REPO="$(awk -F= '/^ARG MC_REPO=/{print $2; exit}' Dockerfile.goreleaser)"
MC_VERSION="$(awk -F= '/^ARG MC_VERSION=/{print $2; exit}' Dockerfile.goreleaser)"
test -n "${MC_REPO}"
test -n "${MC_VERSION}"
MC_VERSION_HYPHEN="${MC_VERSION#RELEASE.}"
MC_PKG_VERSION="$(echo "${MC_VERSION_HYPHEN}" | sed -E 's/^([0-9]{4})-([0-9]{2})-([0-9]{2})T([0-9]{2})-([0-9]{2})-([0-9]{2})Z$/\1\2\3\4\5\6.0.0/')"
if [ "${MC_PKG_VERSION}" = "${MC_VERSION_HYPHEN}" ]; then
echo "Invalid bundled mcli tag: ${MC_VERSION}" >&2
exit 1
fi
if [ "$(gh release view "${MC_VERSION}" --repo "${MC_REPO}" --json isDraft --jq .isDraft)" != false ] || \
[ "$(gh release view "${MC_VERSION}" --repo "${MC_REPO}" --json isPrerelease --jq .isPrerelease)" != false ] || \
[ "$(gh release view "${MC_VERSION}" --repo "${MC_REPO}" --json isImmutable --jq .isImmutable)" != true ]; then
echo "Bundled mcli ${MC_REPO}@${MC_VERSION} must be a published immutable release" >&2
exit 1
fi
mc_checksums="mcli_${MC_PKG_VERSION}_checksums.txt"
gh release download "${MC_VERSION}" --repo "${MC_REPO}" \
--dir "${assets_dir}" --pattern "${mc_checksums}"
gh attestation verify "${assets_dir}/${mc_checksums}" \
--repo "${MC_REPO}" \
--signer-workflow "${MC_REPO}/.github/workflows/release.yml" \
--source-ref "refs/tags/${MC_VERSION}" >/dev/null
MC_AMD64_SHA256="$(awk -v name="mcli_${MC_PKG_VERSION}_linux_amd64.tar.gz" '$2 == name {print $1}' "${assets_dir}/${mc_checksums}")"
MC_ARM64_SHA256="$(awk -v name="mcli_${MC_PKG_VERSION}_linux_arm64.tar.gz" '$2 == name {print $1}' "${assets_dir}/${mc_checksums}")"
[[ "${MC_AMD64_SHA256}" =~ ^[0-9a-f]{64}$ ]]
[[ "${MC_ARM64_SHA256}" =~ ^[0-9a-f]{64}$ ]]
{
echo "MC_AMD64_SHA256=${MC_AMD64_SHA256}"
echo "MC_ARM64_SHA256=${MC_ARM64_SHA256}"
} >> "${GITHUB_ENV}"
echo "RELEASE_REVISION=$(git rev-parse HEAD)" >> "${GITHUB_ENV}"
- name: Set up QEMU
uses: docker/setup-qemu-action@v4
with:
platforms: arm64
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Login to Docker Hub
uses: docker/login-action@v4
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Install Syft
uses: anchore/sbom-action/download-syft@e22c389904149dbc22b58101806040fa8d37a610 # v0.24.0
with:
syft-version: v1.50.0
- name: Build and push amd64 image
id: build-amd64
uses: docker/build-push-action@v7
with:
context: docker-release/amd64
file: docker-release/amd64/Dockerfile.goreleaser
platforms: linux/amd64
push: true
build-args: |
MC_AMD64_SHA256=${{ env.MC_AMD64_SHA256 }}
MC_ARM64_SHA256=${{ env.MC_ARM64_SHA256 }}
tags: |
pgsty/silo:${{ env.RELEASE_TAG }}-amd64
pgsty/silo:latest-amd64
labels: |
org.opencontainers.image.version=${{ env.RELEASE_TAG }}
org.opencontainers.image.created=${{ env.PUBLISHED_AT }}
org.opencontainers.image.revision=${{ env.RELEASE_REVISION }}
- name: Build and push arm64 image
id: build-arm64
uses: docker/build-push-action@v7
with:
context: docker-release/arm64
file: docker-release/arm64/Dockerfile.goreleaser
platforms: linux/arm64
push: true
build-args: |
MC_AMD64_SHA256=${{ env.MC_AMD64_SHA256 }}
MC_ARM64_SHA256=${{ env.MC_ARM64_SHA256 }}
tags: |
pgsty/silo:${{ env.RELEASE_TAG }}-arm64
pgsty/silo:latest-arm64
labels: |
org.opencontainers.image.version=${{ env.RELEASE_TAG }}
org.opencontainers.image.created=${{ env.PUBLISHED_AT }}
org.opencontainers.image.revision=${{ env.RELEASE_REVISION }}
# The distroless variant is a pilot published alongside the classic
# image; it ships the silo binary alone and relies on the native
# `silo healthcheck` subcommand for container health.
# Design: https://silo.pgsty.com/compatibility/feature/healthcheck/
- name: Build and push amd64 distroless image
id: build-amd64-distroless
uses: docker/build-push-action@v7
with:
context: docker-release/amd64
file: docker-release/amd64/Dockerfile.distroless
platforms: linux/amd64
push: true
tags: |
pgsty/silo:${{ env.RELEASE_TAG }}-distroless-amd64
pgsty/silo:distroless-amd64
labels: |
org.opencontainers.image.version=${{ env.RELEASE_TAG }}
org.opencontainers.image.created=${{ env.PUBLISHED_AT }}
org.opencontainers.image.revision=${{ env.RELEASE_REVISION }}
- name: Build and push arm64 distroless image
id: build-arm64-distroless
uses: docker/build-push-action@v7
with:
context: docker-release/arm64
file: docker-release/arm64/Dockerfile.distroless
platforms: linux/arm64
push: true
tags: |
pgsty/silo:${{ env.RELEASE_TAG }}-distroless-arm64
pgsty/silo:distroless-arm64
labels: |
org.opencontainers.image.version=${{ env.RELEASE_TAG }}
org.opencontainers.image.created=${{ env.PUBLISHED_AT }}
org.opencontainers.image.revision=${{ env.RELEASE_REVISION }}
- name: Verify HEALTHCHECK survived the distroless push
run: |
set -euo pipefail
# HEALTHCHECK is a Docker extension absent from the OCI image
# spec, and a publish path can drop it silently. Check the pushed
# architecture image now, before the versioned and rolling
# multi-arch manifests are created, so a broken health config
# stops their promotion. (The architecture-suffixed tags above
# are already public by this point - full staging-then-promote
# would be a workflow-wide redesign shared with the classic
# image lanes.)
docker pull "pgsty/silo:${RELEASE_TAG}-distroless-amd64" >/dev/null
test "$(docker inspect -f '{{json .Config.Healthcheck.Test}}' "pgsty/silo:${RELEASE_TAG}-distroless-amd64")" \
= '["CMD","/usr/bin/silo","healthcheck","ready"]'
- name: Publish multi-architecture manifests
run: |
set -euo pipefail
mkdir -p docker-release/metadata
docker buildx imagetools create \
--tag "pgsty/silo:${RELEASE_TAG}" \
--metadata-file docker-release/metadata/release.json \
"pgsty/silo:${RELEASE_TAG}-amd64" \
"pgsty/silo:${RELEASE_TAG}-arm64"
docker buildx imagetools create \
--tag "pgsty/silo:latest" \
--metadata-file docker-release/metadata/latest.json \
"pgsty/silo:latest-amd64" \
"pgsty/silo:latest-arm64"
docker buildx imagetools inspect "pgsty/silo:${RELEASE_TAG}"
docker buildx imagetools inspect "pgsty/silo:latest"
docker buildx imagetools create \
--tag "pgsty/silo:${RELEASE_TAG}-distroless" \
--metadata-file docker-release/metadata/release-distroless.json \
"pgsty/silo:${RELEASE_TAG}-distroless-amd64" \
"pgsty/silo:${RELEASE_TAG}-distroless-arm64"
docker buildx imagetools create \
--tag "pgsty/silo:distroless" \
--metadata-file docker-release/metadata/distroless.json \
"pgsty/silo:distroless-amd64" \
"pgsty/silo:distroless-arm64"
docker buildx imagetools inspect "pgsty/silo:${RELEASE_TAG}-distroless"
docker buildx imagetools inspect "pgsty/silo:distroless"
RELEASE_DIGEST="$(jq -r '."containerimage.descriptor".digest' docker-release/metadata/release.json)"
LATEST_DIGEST="$(jq -r '."containerimage.descriptor".digest' docker-release/metadata/latest.json)"
if ! [[ "${RELEASE_DIGEST}" =~ ^sha256:[0-9a-f]{64}$ ]]; then
echo "Invalid release manifest digest: ${RELEASE_DIGEST}" >&2
exit 1
fi
if [ "${RELEASE_DIGEST}" != "${LATEST_DIGEST}" ]; then
echo "Release and latest tags resolved to different manifests" >&2
exit 1
fi
echo "SILO_IMAGE_DIGEST=${RELEASE_DIGEST}" >> "${GITHUB_ENV}"
DISTROLESS_RELEASE_DIGEST="$(jq -r '."containerimage.descriptor".digest' docker-release/metadata/release-distroless.json)"
DISTROLESS_ROLLING_DIGEST="$(jq -r '."containerimage.descriptor".digest' docker-release/metadata/distroless.json)"
if ! [[ "${DISTROLESS_RELEASE_DIGEST}" =~ ^sha256:[0-9a-f]{64}$ ]]; then
echo "Invalid distroless manifest digest: ${DISTROLESS_RELEASE_DIGEST}" >&2
exit 1
fi
if [ "${DISTROLESS_RELEASE_DIGEST}" != "${DISTROLESS_ROLLING_DIGEST}" ]; then
echo "Distroless release and rolling tags resolved to different manifests" >&2
exit 1
fi
echo "SILO_DISTROLESS_DIGEST=${DISTROLESS_RELEASE_DIGEST}" >> "${GITHUB_ENV}"
- name: Generate architecture image SBOMs
env:
AMD64_DIGEST: ${{ steps.build-amd64.outputs.digest }}
ARM64_DIGEST: ${{ steps.build-arm64.outputs.digest }}
DISTROLESS_AMD64_DIGEST: ${{ steps.build-amd64-distroless.outputs.digest }}
DISTROLESS_ARM64_DIGEST: ${{ steps.build-arm64-distroless.outputs.digest }}
SYFT_CHECK_FOR_APP_UPDATE: "false"
run: |
set -euo pipefail
mkdir -p docker-release/sbom
# Each per-architecture digest names an OCI index (image plus the
# provenance attestation buildx attaches), and Syft's platform
# default on an index follows the amd64 runner - an arm64-only
# index would fail outright. Select the platform explicitly.
syft "registry:index.docker.io/pgsty/silo@${AMD64_DIGEST}" \
--platform linux/amd64 \
--output "spdx-json=docker-release/sbom/linux-amd64.spdx.json"
syft "registry:index.docker.io/pgsty/silo@${ARM64_DIGEST}" \
--platform linux/arm64 \
--output "spdx-json=docker-release/sbom/linux-arm64.spdx.json"
syft "registry:index.docker.io/pgsty/silo@${DISTROLESS_AMD64_DIGEST}" \
--platform linux/amd64 \
--output "spdx-json=docker-release/sbom/linux-amd64-distroless.spdx.json"
syft "registry:index.docker.io/pgsty/silo@${DISTROLESS_ARM64_DIGEST}" \
--platform linux/arm64 \
--output "spdx-json=docker-release/sbom/linux-arm64-distroless.spdx.json"
- name: Attest amd64 image SBOM
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-name: index.docker.io/pgsty/silo
subject-digest: ${{ steps.build-amd64.outputs.digest }}
sbom-path: docker-release/sbom/linux-amd64.spdx.json
push-to-registry: true
- name: Attest arm64 image SBOM
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-name: index.docker.io/pgsty/silo
subject-digest: ${{ steps.build-arm64.outputs.digest }}
sbom-path: docker-release/sbom/linux-arm64.spdx.json
push-to-registry: true
- name: Attest amd64 distroless image SBOM
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-name: index.docker.io/pgsty/silo
subject-digest: ${{ steps.build-amd64-distroless.outputs.digest }}
sbom-path: docker-release/sbom/linux-amd64-distroless.spdx.json
push-to-registry: true
- name: Attest arm64 distroless image SBOM
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-name: index.docker.io/pgsty/silo
subject-digest: ${{ steps.build-arm64-distroless.outputs.digest }}
sbom-path: docker-release/sbom/linux-arm64-distroless.spdx.json
push-to-registry: true
- name: Attest multi-architecture image provenance
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-name: index.docker.io/pgsty/silo
subject-digest: ${{ env.SILO_IMAGE_DIGEST }}
push-to-registry: true
- name: Attest multi-architecture distroless image provenance
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-name: index.docker.io/pgsty/silo
subject-digest: ${{ env.SILO_DISTROLESS_DIGEST }}
push-to-registry: true
-234
View File
@@ -1,234 +0,0 @@
name: Finalize Release Packages
# Manual-only lane that runs AFTER the maintainer has GPG-signed the RPMs in a
# Draft release (buildscripts/sign-release-rpms.sh --upload) and BEFORE the
# release is published. GPG signing rewrites the RPM bytes, which strands the
# SBOMs, the packages checksum manifest, and the attestations that release.yml
# generated from the as-built packages. This lane regenerates those materials
# from the published (signed) bytes under the workflow identity, so the
# sigstore layer describes exactly what the release ships.
on:
workflow_dispatch:
inputs:
tag:
description: "Draft RELEASE.* tag whose signed RPMs need refreshed SBOMs and checksums"
required: true
type: string
permissions:
contents: write
id-token: write
attestations: write
artifact-metadata: write
concurrency:
group: release-${{ inputs.tag }}
cancel-in-progress: false
jobs:
finalize:
runs-on: ubuntu-latest
steps:
- name: Checkout release tag
uses: actions/checkout@v7
with:
ref: ${{ inputs.tag }}
fetch-depth: 0
- name: Verify workflow identity matches release source
run: |
set -euo pipefail
CHECKED_OUT_REVISION="$(git rev-parse HEAD)"
if [ "${CHECKED_OUT_REVISION}" != "${GITHUB_SHA}" ]; then
echo "Checked out ${CHECKED_OUT_REVISION}, but workflow identity is ${GITHUB_SHA}. Dispatch from the release tag." >&2
exit 1
fi
- name: Compute release variables
env:
# Environment passthrough keeps the dispatch input out of the script
# source, mirroring release.yml.
INPUT_TAG: ${{ inputs.tag }}
run: |
set -euo pipefail
TAG="${INPUT_TAG}"
if [[ ! "${TAG}" =~ ^RELEASE\.[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}-[0-9]{2}-[0-9]{2}Z$ ]]; then
echo "Invalid release tag format: ${TAG}" >&2
exit 1
fi
VERSION_HYPHEN="${TAG#RELEASE.}"
PKG_VERSION="$(echo "${VERSION_HYPHEN}" | sed -E 's/^([0-9]{4})-([0-9]{2})-([0-9]{2})T([0-9]{2})-([0-9]{2})-([0-9]{2})Z$/\1\2\3\4\5\6.0.0/')"
{
echo "RELEASE_TAG=${TAG}"
echo "PKG_VERSION=${PKG_VERSION}"
} >> "${GITHUB_ENV}"
- name: Refuse to touch a published release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
if [ "$(gh release view "${RELEASE_TAG}" --repo "${GITHUB_REPOSITORY}" --json isDraft --jq .isDraft)" != "true" ]; then
echo "${RELEASE_TAG} is not a Draft release; finalize runs only before publishing." >&2
exit 1
fi
- name: Install Syft
uses: anchore/sbom-action/download-syft@e22c389904149dbc22b58101806040fa8d37a610 # v0.24.0
with:
syft-version: v1.50.0
- name: Install Cosign
uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2
with:
cosign-release: v3.1.2
- name: Download and verify the package set
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
# sign-release-rpms.sh owns the package identity; import its values
# the same way test-release.yml does so the lanes cannot drift.
eval "$(grep -E '^expected_(release|fingerprint)=' buildscripts/sign-release-rpms.sh)"
test -n "${expected_release}"
test -n "${expected_fingerprint}"
packages_dir="finalize/packages"
sidecar_dir="finalize/sidecars"
mkdir -p "${packages_dir}" "${sidecar_dir}"
manifest="silo_${PKG_VERSION}_packages_checksums.txt"
# Exactly the twelve manifest subjects land in packages_dir; the
# tarball SBOMs in the release root do not match these patterns.
gh release download "${RELEASE_TAG}" --repo "${GITHUB_REPOSITORY}" --dir "${packages_dir}" \
--pattern '*.rpm' --pattern '*.deb' --pattern '*.apk' \
--pattern '*.rpm.sbom.json' --pattern '*.deb.sbom.json' --pattern '*.apk.sbom.json'
gh release download "${RELEASE_TAG}" --repo "${GITHUB_REPOSITORY}" --dir "${sidecar_dir}" \
--pattern '*.rpm.sha256sum' --pattern "${manifest}"
cd "${packages_dir}"
subject_count="$(find . -maxdepth 1 -type f | wc -l | tr -d ' ')"
if [ "${subject_count}" -ne 12 ]; then
echo "Expected twelve package subjects, found ${subject_count}" >&2
find . -maxdepth 1 -type f >&2
exit 1
fi
# The signed RPMs must match the .sha256sum sidecars the signing
# script regenerated and uploaded alongside them.
for rpm_file in "silo-${PKG_VERSION}-${expected_release}.x86_64.rpm" \
"silo-${PKG_VERSION}-${expected_release}.aarch64.rpm"; do
test -s "${rpm_file}"
actual="$(sha256sum "${rpm_file}" | awk '{print $1}')"
recorded="$(awk '{print $1}' "../sidecars/${rpm_file}.sha256sum")"
if [ "${actual}" != "${recorded}" ]; then
echo "Digest mismatch for ${rpm_file}: sidecar ${recorded}, asset ${actual}" >&2
exit 1
fi
done
# Everything the maintainer did not re-sign must still match the
# manifest release.yml generated: this lane refreshes RPM materials,
# it does not accept drift anywhere else.
for package_file in *.deb *.apk *.deb.sbom.json *.apk.sbom.json; do
awk -v name="${package_file}" '$2 == name' "../sidecars/${manifest}" | sha256sum --check
done
- name: Verify RPM GPG signatures
run: |
set -euo pipefail
eval "$(grep -E '^expected_fingerprint=' buildscripts/sign-release-rpms.sh)"
sudo apt-get update
sudo apt-get install --yes rpm
sudo rpmkeys --import buildscripts/pgsty-rpm-signing-key.asc
key_id="$(printf '%s' "${expected_fingerprint}" | tail -c 8 | tr '[:upper:]' '[:lower:]')"
for rpm_file in finalize/packages/*.rpm; do
signature_output="$(sudo rpmkeys --checksig --verbose "${rpm_file}")"
printf '%s\n' "${signature_output}"
if ! printf '%s\n' "${signature_output}" | tr '[:upper:]' '[:lower:]' | grep -q "key id ${key_id}: ok"; then
echo "Signature verification failed for ${rpm_file}" >&2
exit 1
fi
done
- name: Regenerate RPM SBOMs and the packages checksum manifest
env:
SYFT_CHECK_FOR_APP_UPDATE: "false"
run: |
set -euo pipefail
cd finalize/packages
for rpm_file in *.rpm; do
rm -f "${rpm_file}.sbom.json"
syft "${rpm_file}" --output "spdx-json=${rpm_file}.sbom.json"
done
manifest="silo_${PKG_VERSION}_packages_checksums.txt"
mapfile -t subjects < <(find . -maxdepth 1 -type f \
\( -name '*.rpm' -o -name '*.deb' -o -name '*.apk' -o -name '*.sbom.json' \) | sort)
if [ "${#subjects[@]}" -ne 12 ]; then
echo "Expected six packages and six SBOMs, found ${#subjects[@]} subjects" >&2
exit 1
fi
sha256sum "${subjects[@]}" | sed 's# \./# #' > "${manifest}"
cosign sign-blob --bundle="${manifest}.sigstore.json" "${manifest}" --yes
- name: Attest the finalized package artifacts
id: attest-finalize
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-path: |
finalize/packages/*.rpm
finalize/packages/*.rpm.sbom.json
finalize/packages/*_checksums.txt
finalize/packages/*_checksums.txt.sigstore.json
finalize/sidecars/*.rpm.sha256sum
- name: Preserve finalize provenance bundle as a release asset
env:
BUNDLE_PATH: ${{ steps.attest-finalize.outputs.bundle-path }}
run: |
set -euo pipefail
test -s "${BUNDLE_PATH}"
cp "${BUNDLE_PATH}" "finalize/packages/silo_${PKG_VERSION}_packages_provenance.sigstore.json"
- name: Upload finalized assets to the Draft release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
eval "$(grep -E '^expected_release=' buildscripts/sign-release-rpms.sh)"
manifest="silo_${PKG_VERSION}_packages_checksums.txt"
cd finalize/packages
files=(
"silo-${PKG_VERSION}-${expected_release}.x86_64.rpm.sbom.json"
"silo-${PKG_VERSION}-${expected_release}.aarch64.rpm.sbom.json"
"${manifest}"
"${manifest}.sigstore.json"
"silo_${PKG_VERSION}_packages_provenance.sigstore.json"
)
gh release upload "${RELEASE_TAG}" --repo "${GITHUB_REPOSITORY}" --clobber "${files[@]}"
for asset in "${files[@]}"; do
local_digest="sha256:$(sha256sum "${asset}" | awk '{print $1}')"
remote_digest=""
for attempt in 1 2 3 4 5; do
remote_digest="$(gh release view "${RELEASE_TAG}" --repo "${GITHUB_REPOSITORY}" --json assets \
--jq ".assets[] | select(.name == \"${asset}\") | .digest")"
if [ "${local_digest}" = "${remote_digest}" ]; then
break
fi
if [ "${attempt}" -lt 5 ]; then
sleep 2
fi
done
if [ "${local_digest}" != "${remote_digest}" ]; then
echo "GitHub asset digest mismatch for ${asset}" >&2
echo "Local: ${local_digest}" >&2
echo "Remote: ${remote_digest}" >&2
exit 1
fi
echo "Verified GitHub asset: ${asset} ${remote_digest}"
done
+39
View File
@@ -0,0 +1,39 @@
name: Crosscompile
on:
pull_request:
branches:
- master
# This ensures that previous jobs for the PR are canceled when the PR is
# updated.
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
build:
name: Build Tests with Go ${{ matrix.go-version }} on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
go-version: [1.24.x]
os: [ubuntu-latest]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go-version }}
check-latest: true
- name: Build on ${{ matrix.os }}
if: matrix.os == 'ubuntu-latest'
env:
CGO_ENABLED: 0
GO111MODULE: on
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make crosscompile
+44
View File
@@ -0,0 +1,44 @@
name: Healing Functional Tests
on:
pull_request:
branches:
- master
# This ensures that previous jobs for the PR are canceled when the PR is
# updated.
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
build:
name: Go ${{ matrix.go-version }} on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
go-version: [1.24.x]
os: [ubuntu-latest]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go-version }}
check-latest: true
- name: Build on ${{ matrix.os }}
if: matrix.os == 'ubuntu-latest'
env:
CGO_ENABLED: 0
GO111MODULE: on
MINIO_KMS_SECRET_KEY: "my-minio-key:oyArl7zlPECEduNbB1KXgdzDn2Bdpvvw0l8VO51HQnY="
MINIO_KMS_AUTO_ENCRYPTION: on
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make verify-healing
make verify-healing-inconsistent-versions
make verify-healing-with-root-disks
make verify-healing-with-rewrite
+42
View File
@@ -0,0 +1,42 @@
name: Linters and Tests
on:
pull_request:
branches:
- master
# This ensures that previous jobs for the PR are canceled when the PR is
# updated.
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
build:
name: Go ${{ matrix.go-version }} on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
go-version: [1.24.x]
os: [ubuntu-latest]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go-version }}
check-latest: true
- name: Build on ${{ matrix.os }}
if: matrix.os == 'ubuntu-latest'
env:
CGO_ENABLED: 0
GO111MODULE: on
run: |
sudo apt install jq -y
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make
make test
make test-race
+39
View File
@@ -0,0 +1,39 @@
name: Resiliency Functional Tests
on:
pull_request:
branches:
- master
# This ensures that previous jobs for the PR are canceled when the PR is
# updated.
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
build:
name: Go ${{ matrix.go-version }} on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
go-version: [1.24.x]
os: [ubuntu-latest]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go-version }}
check-latest: true
- name: Build on ${{ matrix.os }}
if: matrix.os == 'ubuntu-latest'
env:
CGO_ENABLED: 0
GO111MODULE: on
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make test-resiliency
+23 -140
View File
@@ -1,159 +1,42 @@
name: Go CI
name: Functional Tests
on:
pull_request:
branches:
- main
push:
branches:
- main
workflow_dispatch:
- master
# Cancel superseded runs for the same PR; never cancel main push runs.
# Keyed on PR number (not head_ref) so fork PRs sharing a branch name
# do not collide.
# This ensures that previous jobs for the PR are canceled when the PR is
# updated.
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
group: ${{ github.workflow }}-${{ github.head_ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
verify:
name: Format, Build, Vet
runs-on: ubuntu-latest
timeout-minutes: 20
build:
name: Go ${{ matrix.go-version }} on ${{ matrix.os }} - healing
runs-on: ${{ matrix.os }}
strategy:
matrix:
go-version: [1.24.x]
os: [ubuntu-latest]
steps:
- uses: actions/checkout@v7
- uses: actions/setup-go@v7
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version-file: go.mod
cache: true
- name: Check gofmt
run: |
mapfile -t unformatted < <(gofmt -l main.go cmd internal \
buildscripts/rebrand-guard buildscripts/helm-migration-guard)
if [ "${#unformatted[@]}" -ne 0 ]; then
echo "The following files are not gofmt-formatted:"
printf '%s\n' "${unformatted[@]}"
gofmt -d "${unformatted[@]}"
exit 1
fi
- name: Build
go-version: ${{ matrix.go-version }}
check-latest: true
- name: Build on ${{ matrix.os }}
if: matrix.os == 'ubuntu-latest'
env:
CGO_ENABLED: 0
run: go build ./...
- name: Vet
run: go vet ./...
- name: Verify rebrand compatibility contracts
run: |
go run ./buildscripts/rebrand-guard
buildscripts/verify-rebrand.sh
dockerscripts/docker-entrypoint_test.sh
quality:
name: Lint, Generated Files
runs-on: ubuntu-latest
timeout-minutes: 25
steps:
- uses: actions/checkout@v7
- uses: actions/setup-go@v7
with:
go-version-file: go.mod
cache: true
- name: Lint
run: make lint
- name: Check generated files
run: make check-gen
race-s3select:
name: Race, S3 Select
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v7
- uses: actions/setup-go@v7
with:
go-version-file: go.mod
cache: true
- name: Run S3 Select tests under race detector
run: go test -race ./internal/s3select/... -count=1
- name: Run conditional PUT tests under race detector
run: go test -race ./cmd -run '^Test(PoolsConditionalPut|SinglePoolConditionalPutHTTP)' -count=1 -timeout=5m
crosscompile:
name: Cross Compile
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v7
- uses: actions/setup-go@v7
with:
go-version-file: go.mod
cache: true
- name: Build supported targets
run: make crosscompile
test-internal:
name: Test internal/
runs-on: ubuntu-latest
timeout-minutes: 25
steps:
- uses: actions/checkout@v7
- uses: actions/setup-go@v7
with:
go-version-file: go.mod
cache: true
# internal/http listener tests bind [::1]; runners disable IPv6 by default.
- name: Enable IPv6
GO111MODULE: on
MINIO_KMS_SECRET_KEY: "my-minio-key:OSMM+vkKUTCvQs9YL/CVMIMt43HFhkUpqJxTmGl6rYw="
MINIO_KMS_AUTO_ENCRYPTION: on
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
- name: Run internal tests
env:
CGO_ENABLED: 0
MINIO_API_REQUESTS_MAX: "10000"
run: go test ./internal/... -count=1
test-cmd:
name: Test cmd/
runs-on: ubuntu-latest
timeout-minutes: 35
steps:
- uses: actions/checkout@v7
- uses: actions/setup-go@v7
with:
go-version-file: go.mod
cache: true
# Some server tests bind IPv6 listeners; runners disable IPv6 by default.
- name: Enable IPv6
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
- name: Run cmd tests
env:
CGO_ENABLED: 0
MINIO_API_REQUESTS_MAX: "10000"
# cmd/ is one large package; raise go test's default 10m per-package
# timeout so slower runners fail on the job timeout, not a panic.
run: go test ./cmd/ -count=1 -timeout 30m
make verify
make test-timeout
+30
View File
@@ -0,0 +1,30 @@
name: Helm Chart linting
on:
pull_request:
branches:
- master
# This ensures that previous jobs for the PR are canceled when the PR is
# updated.
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
release:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Install Helm
uses: azure/setup-helm@v4
- name: Run helm lint
run: |
cd helm/minio
helm lint .
+161
View File
@@ -0,0 +1,161 @@
name: IAM integration
on:
pull_request:
branches:
- master
# This ensures that previous jobs for the PR are canceled when the PR is
# updated.
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
iam-matrix-test:
name: "[Go=${{ matrix.go-version }}|ldap=${{ matrix.ldap }}|etcd=${{ matrix.etcd }}|openid=${{ matrix.openid }}]"
runs-on: ubuntu-latest
services:
openldap:
image: quay.io/minio/openldap
ports:
- "389:389"
- "636:636"
env:
LDAP_ORGANIZATION: "MinIO Inc"
LDAP_DOMAIN: "min.io"
LDAP_ADMIN_PASSWORD: "admin"
etcd:
image: "quay.io/coreos/etcd:v3.5.1"
env:
ETCD_LISTEN_CLIENT_URLS: "http://0.0.0.0:2379"
ETCD_ADVERTISE_CLIENT_URLS: "http://0.0.0.0:2379"
ports:
- "2379:2379"
options: >-
--health-cmd "etcdctl endpoint health"
--health-interval 10s
--health-timeout 5s
--health-retries 5
openid:
image: quay.io/minio/dex
ports:
- "5556:5556"
env:
DEX_LDAP_SERVER: "openldap:389"
openid2:
image: quay.io/minio/dex
ports:
- "5557:5557"
env:
DEX_LDAP_SERVER: "openldap:389"
DEX_ISSUER: "http://127.0.0.1:5557/dex"
DEX_WEB_HTTP: "0.0.0.0:5557"
strategy:
# When ldap, etcd or openid vars are empty below, those external servers
# are turned off - i.e. if ldap="", then ldap server is not enabled for
# the tests.
matrix:
go-version: [1.24.x]
ldap: ["", "localhost:389"]
etcd: ["", "http://localhost:2379"]
openid: ["", "http://127.0.0.1:5556/dex"]
exclude:
# exclude combos where all are empty.
- ldap: ""
etcd: ""
openid: ""
# exclude combos where both ldap and openid IDPs are specified.
- ldap: "localhost:389"
openid: "http://127.0.0.1:5556/dex"
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go-version }}
check-latest: true
- name: Test LDAP/OpenID/Etcd combo
env:
_MINIO_LDAP_TEST_SERVER: ${{ matrix.ldap }}
_MINIO_ETCD_TEST_SERVER: ${{ matrix.etcd }}
_MINIO_OPENID_TEST_SERVER: ${{ matrix.openid }}
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make test-iam
- name: Test with multiple OpenID providers
if: matrix.openid == 'http://127.0.0.1:5556/dex'
env:
_MINIO_LDAP_TEST_SERVER: ${{ matrix.ldap }}
_MINIO_ETCD_TEST_SERVER: ${{ matrix.etcd }}
_MINIO_OPENID_TEST_SERVER: ${{ matrix.openid }}
_MINIO_OPENID_TEST_SERVER_2: "http://127.0.0.1:5557/dex"
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make test-iam
- name: Test with Access Management Plugin enabled
env:
_MINIO_LDAP_TEST_SERVER: ${{ matrix.ldap }}
_MINIO_ETCD_TEST_SERVER: ${{ matrix.etcd }}
_MINIO_OPENID_TEST_SERVER: ${{ matrix.openid }}
_MINIO_POLICY_PLUGIN_TEST_ENDPOINT: "http://127.0.0.1:8080"
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
go run docs/iam/access-manager-plugin.go &
make test-iam
- name: Test MinIO Old Version data to IAM import current version
if: matrix.ldap == 'ldaphost:389'
env:
_MINIO_LDAP_TEST_SERVER: ${{ matrix.ldap }}
run: |
make test-iam-ldap-upgrade-import
- name: Test LDAP for automatic site replication
if: matrix.ldap == 'localhost:389'
run: |
make test-site-replication-ldap
- name: Test OIDC for automatic site replication
if: matrix.openid == 'http://127.0.0.1:5556/dex'
run: |
make test-site-replication-oidc
iam-import-with-missing-entities:
name: Test IAM import in new cluster with missing entities
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go-version }}
check-latest: true
- name: Checkout minio-iam-testing
uses: actions/checkout@v4
with:
repository: minio/minio-iam-testing
path: minio-iam-testing
- name: Test import of IAM artifacts when in fresh cluster there are missing groups etc
run: |
make test-iam-import-with-missing-entities
iam-import-with-openid:
name: Test IAM import in new cluster with opendid configurations
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go-version }}
check-latest: true
- name: Checkout minio-iam-testing
uses: actions/checkout@v4
with:
repository: minio/minio-iam-testing
path: minio-iam-testing
- name: Test import of IAM artifacts when in fresh cluster with openid configurations
run: |
make test-iam-import-with-openid
+18
View File
@@ -0,0 +1,18 @@
# @format
name: Issue Workflow
on:
issues:
types:
- opened
jobs:
add-to-project:
name: Add issue to project
runs-on: ubuntu-latest
steps:
- uses: actions/add-to-project@v0.5.0
with:
project-url: https://github.com/orgs/miniohq/projects/2
github-token: ${{ secrets.BOT_PAT }}
+24
View File
@@ -0,0 +1,24 @@
name: 'Lock Threads'
on:
schedule:
- cron: '0 0 * * *'
workflow_dispatch:
permissions:
issues: write
concurrency:
group: lock
jobs:
action:
runs-on: ubuntu-latest
steps:
- uses: dessant/lock-threads@v3
with:
github-token: ${{ github.token }}
issue-inactive-days: '365'
exclude-any-issue-labels: 'do-not-close'
issue-lock-reason: 'resolved'
log-output: true
+81
View File
@@ -0,0 +1,81 @@
name: Mint Tests
on:
pull_request:
branches:
- master
# This ensures that previous jobs for the PR are canceled when the PR is
# updated.
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
mint-test:
runs-on: mint
timeout-minutes: 120
steps:
- name: cleanup #https://github.com/actions/checkout/issues/273
run: |
sudo -S rm -rf ${GITHUB_WORKSPACE}
mkdir ${GITHUB_WORKSPACE}
- name: checkout-step
uses: actions/checkout@v4
- name: setup-go-step
uses: actions/setup-go@v5
with:
go-version: 1.24.x
- name: github sha short
id: vars
run: echo "sha_short=$(git rev-parse --short HEAD)" >> $GITHUB_OUTPUT
- name: build-minio
run: |
TAG="quay.io/minio/minio:${{ steps.vars.outputs.sha_short }}" make docker
- name: multipart uploads test
run: |
${GITHUB_WORKSPACE}/.github/workflows/multipart/migrate.sh "${{ steps.vars.outputs.sha_short }}"
- name: compress and encrypt
run: |
${GITHUB_WORKSPACE}/.github/workflows/run-mint.sh "compress-encrypt" "minio" "minio123" "${{ steps.vars.outputs.sha_short }}"
- name: multiple pools
run: |
${GITHUB_WORKSPACE}/.github/workflows/run-mint.sh "pools" "minio" "minio123" "${{ steps.vars.outputs.sha_short }}"
- name: standalone erasure
run: |
${GITHUB_WORKSPACE}/.github/workflows/run-mint.sh "erasure" "minio" "minio123" "${{ steps.vars.outputs.sha_short }}"
# FIXME: renable this back when we have a valid way to add deadlines for PUT()s (internode CreateFile)
# - name: resiliency
# run: |
# ${GITHUB_WORKSPACE}/.github/workflows/run-mint.sh "resiliency" "minio" "minio123" "${{ steps.vars.outputs.sha_short }}"
- name: The job must cleanup
if: ${{ always() }}
run: |
export JOB_NAME=${{ steps.vars.outputs.sha_short }}
for mode in $(echo compress-encrypt pools erasure); do
docker-compose -f ${GITHUB_WORKSPACE}/.github/workflows/mint/minio-${mode}.yaml down || true
docker-compose -f ${GITHUB_WORKSPACE}/.github/workflows/mint/minio-${mode}.yaml rm || true
done
docker-compose -f ${GITHUB_WORKSPACE}/.github/workflows/multipart/docker-compose-site1.yaml rm -s -f || true
docker-compose -f ${GITHUB_WORKSPACE}/.github/workflows/multipart/docker-compose-site2.yaml rm -s -f || true
for volume in $(docker volume ls -q | grep minio); do
docker volume rm ${volume} || true
done
docker rmi -f quay.io/minio/minio:${{ steps.vars.outputs.sha_short }}
docker system prune -f || true
docker volume prune -f || true
docker volume rm $(docker volume ls -q -f dangling=true) || true
@@ -0,0 +1,80 @@
version: '3.7'
# Settings and configurations that are common for all containers
x-minio-common: &minio-common
image: quay.io/minio/minio:${JOB_NAME}
command: server --console-address ":9001" http://minio{1...4}/cdata{1...2}
expose:
- "9000"
- "9001"
environment:
MINIO_CI_CD: "on"
MINIO_ROOT_USER: "minio"
MINIO_ROOT_PASSWORD: "minio123"
MINIO_COMPRESSION_ENABLE: "on"
MINIO_COMPRESSION_MIME_TYPES: "*"
MINIO_COMPRESSION_ALLOW_ENCRYPTION: "on"
MINIO_KMS_SECRET_KEY: "my-minio-key:OSMM+vkKUTCvQs9YL/CVMIMt43HFhkUpqJxTmGl6rYw="
healthcheck:
test: ["CMD", "mc", "ready", "local"]
interval: 5s
timeout: 5s
retries: 5
# starts 4 docker containers running minio server instances.
# using nginx reverse proxy, load balancing, you can access
# it through port 9000.
services:
minio1:
<<: *minio-common
hostname: minio1
volumes:
- cdata1-1:/cdata1
- cdata1-2:/cdata2
minio2:
<<: *minio-common
hostname: minio2
volumes:
- cdata2-1:/cdata1
- cdata2-2:/cdata2
minio3:
<<: *minio-common
hostname: minio3
volumes:
- cdata3-1:/cdata1
- cdata3-2:/cdata2
minio4:
<<: *minio-common
hostname: minio4
volumes:
- cdata4-1:/cdata1
- cdata4-2:/cdata2
nginx:
image: nginx:1.19.2-alpine
hostname: nginx
volumes:
- ./nginx-4-node.conf:/etc/nginx/nginx.conf:ro
ports:
- "9000:9000"
- "9001:9001"
depends_on:
- minio1
- minio2
- minio3
- minio4
## By default this config uses default local driver,
## For custom volumes replace with volume driver configuration.
volumes:
cdata1-1:
cdata1-2:
cdata2-1:
cdata2-2:
cdata3-1:
cdata3-2:
cdata4-1:
cdata4-2:
+51
View File
@@ -0,0 +1,51 @@
version: '3.7'
# Settings and configurations that are common for all containers
x-minio-common: &minio-common
image: quay.io/minio/minio:${JOB_NAME}
command: server --console-address ":9001" edata{1...4}
expose:
- "9000"
- "9001"
environment:
MINIO_CI_CD: "on"
MINIO_ROOT_USER: "minio"
MINIO_ROOT_PASSWORD: "minio123"
MINIO_KMS_SECRET_KEY: "my-minio-key:OSMM+vkKUTCvQs9YL/CVMIMt43HFhkUpqJxTmGl6rYw="
healthcheck:
test: ["CMD", "mc", "ready", "local"]
interval: 5s
timeout: 5s
retries: 5
# starts 4 docker containers running minio server instances.
# using nginx reverse proxy, load balancing, you can access
# it through port 9000.
services:
minio1:
<<: *minio-common
hostname: minio1
volumes:
- edata1-1:/edata1
- edata1-2:/edata2
- edata1-3:/edata3
- edata1-4:/edata4
nginx:
image: nginx:1.19.2-alpine
hostname: nginx
volumes:
- ./nginx-1-node.conf:/etc/nginx/nginx.conf:ro
ports:
- "9000:9000"
- "9001:9001"
depends_on:
- minio1
## By default this config uses default local driver,
## For custom volumes replace with volume driver configuration.
volumes:
edata1-1:
edata1-2:
edata1-3:
edata1-4:
+117
View File
@@ -0,0 +1,117 @@
version: '3.7'
# Settings and configurations that are common for all containers
x-minio-common: &minio-common
image: quay.io/minio/minio:${JOB_NAME}
command: server --console-address ":9001" http://minio{1...4}/pdata{1...2} http://minio{5...8}/pdata{1...2}
expose:
- "9000"
- "9001"
environment:
MINIO_CI_CD: "on"
MINIO_ROOT_USER: "minio"
MINIO_ROOT_PASSWORD: "minio123"
MINIO_KMS_SECRET_KEY: "my-minio-key:OSMM+vkKUTCvQs9YL/CVMIMt43HFhkUpqJxTmGl6rYw="
healthcheck:
test: ["CMD", "mc", "ready", "local"]
interval: 5s
timeout: 5s
retries: 5
# starts 4 docker containers running minio server instances.
# using nginx reverse proxy, load balancing, you can access
# it through port 9000.
services:
minio1:
<<: *minio-common
hostname: minio1
volumes:
- pdata1-1:/pdata1
- pdata1-2:/pdata2
minio2:
<<: *minio-common
hostname: minio2
volumes:
- pdata2-1:/pdata1
- pdata2-2:/pdata2
minio3:
<<: *minio-common
hostname: minio3
volumes:
- pdata3-1:/pdata1
- pdata3-2:/pdata2
minio4:
<<: *minio-common
hostname: minio4
volumes:
- pdata4-1:/pdata1
- pdata4-2:/pdata2
minio5:
<<: *minio-common
hostname: minio5
volumes:
- pdata5-1:/pdata1
- pdata5-2:/pdata2
minio6:
<<: *minio-common
hostname: minio6
volumes:
- pdata6-1:/pdata1
- pdata6-2:/pdata2
minio7:
<<: *minio-common
hostname: minio7
volumes:
- pdata7-1:/pdata1
- pdata7-2:/pdata2
minio8:
<<: *minio-common
hostname: minio8
volumes:
- pdata8-1:/pdata1
- pdata8-2:/pdata2
nginx:
image: nginx:1.19.2-alpine
hostname: nginx
volumes:
- ./nginx-8-node.conf:/etc/nginx/nginx.conf:ro
ports:
- "9000:9000"
- "9001:9001"
depends_on:
- minio1
- minio2
- minio3
- minio4
- minio5
- minio6
- minio7
- minio8
## By default this config uses default local driver,
## For custom volumes replace with volume driver configuration.
volumes:
pdata1-1:
pdata1-2:
pdata2-1:
pdata2-2:
pdata3-1:
pdata3-2:
pdata4-1:
pdata4-2:
pdata5-1:
pdata5-2:
pdata6-1:
pdata6-2:
pdata7-1:
pdata7-2:
pdata8-1:
pdata8-2:
@@ -0,0 +1,78 @@
version: '3.7'
# Settings and configurations that are common for all containers
x-minio-common: &minio-common
image: quay.io/minio/minio:${JOB_NAME}
command: server --console-address ":9001" http://minio{1...4}/rdata{1...2}
expose:
- "9000"
- "9001"
environment:
MINIO_CI_CD: "on"
MINIO_ROOT_USER: "minio"
MINIO_ROOT_PASSWORD: "minio123"
MINIO_KMS_SECRET_KEY: "my-minio-key:OSMM+vkKUTCvQs9YL/CVMIMt43HFhkUpqJxTmGl6rYw="
MINIO_DRIVE_MAX_TIMEOUT: "5s"
healthcheck:
test: ["CMD", "mc", "ready", "local"]
interval: 5s
timeout: 5s
retries: 5
# starts 4 docker containers running minio server instances.
# using nginx reverse proxy, load balancing, you can access
# it through port 9000.
services:
minio1:
<<: *minio-common
hostname: minio1
volumes:
- rdata1-1:/rdata1
- rdata1-2:/rdata2
minio2:
<<: *minio-common
hostname: minio2
volumes:
- rdata2-1:/rdata1
- rdata2-2:/rdata2
minio3:
<<: *minio-common
hostname: minio3
volumes:
- rdata3-1:/rdata1
- rdata3-2:/rdata2
minio4:
<<: *minio-common
hostname: minio4
volumes:
- rdata4-1:/rdata1
- rdata4-2:/rdata2
nginx:
image: nginx:1.19.2-alpine
hostname: nginx
volumes:
- ./nginx-4-node.conf:/etc/nginx/nginx.conf:ro
ports:
- "9000:9000"
- "9001:9001"
depends_on:
- minio1
- minio2
- minio3
- minio4
## By default this config uses default local driver,
## For custom volumes replace with volume driver configuration.
volumes:
rdata1-1:
rdata1-2:
rdata2-1:
rdata2-2:
rdata3-1:
rdata3-2:
rdata4-1:
rdata4-2:
+100
View File
@@ -0,0 +1,100 @@
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 4096;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
keepalive_timeout 65;
# include /etc/nginx/conf.d/*.conf;
upstream minio {
server minio1:9000;
}
upstream console {
ip_hash;
server minio1:9001;
}
server {
listen 9000;
listen [::]:9000;
server_name localhost;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
proxy_request_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 300;
# Default is HTTP/1, keepalive is only enabled in HTTP/1.1
proxy_http_version 1.1;
proxy_set_header Connection "";
chunked_transfer_encoding off;
proxy_pass http://minio;
}
}
server {
listen 9001;
listen [::]:9001;
server_name localhost;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
proxy_request_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-NginX-Proxy true;
# This is necessary to pass the correct IP to be hashed
real_ip_header X-Real-IP;
proxy_connect_timeout 300;
# To support websocket
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
chunked_transfer_encoding off;
proxy_pass http://console;
}
}
}
+105
View File
@@ -0,0 +1,105 @@
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 4096;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
keepalive_timeout 65;
# include /etc/nginx/conf.d/*.conf;
upstream minio {
server minio1:9000 max_fails=1 fail_timeout=10s;
server minio2:9000 max_fails=1 fail_timeout=10s;
server minio3:9000 max_fails=1 fail_timeout=10s;
}
upstream console {
ip_hash;
server minio1:9001;
server minio2:9001;
server minio3:9001;
server minio4:9001;
}
server {
listen 9000;
listen [::]:9000;
server_name localhost;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
proxy_request_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 300;
# Default is HTTP/1, keepalive is only enabled in HTTP/1.1
proxy_http_version 1.1;
proxy_set_header Connection "";
chunked_transfer_encoding off;
proxy_pass http://minio;
}
}
server {
listen 9001;
listen [::]:9001;
server_name localhost;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
proxy_request_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-NginX-Proxy true;
# This is necessary to pass the correct IP to be hashed
real_ip_header X-Real-IP;
proxy_connect_timeout 300;
# To support websocket
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
chunked_transfer_encoding off;
proxy_pass http://console;
}
}
}
+114
View File
@@ -0,0 +1,114 @@
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 4096;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
keepalive_timeout 65;
# include /etc/nginx/conf.d/*.conf;
upstream minio {
server minio1:9000 max_fails=1 fail_timeout=10s;
server minio2:9000 max_fails=1 fail_timeout=10s;
server minio3:9000 max_fails=1 fail_timeout=10s;
server minio4:9000 max_fails=1 fail_timeout=10s;
server minio5:9000 max_fails=1 fail_timeout=10s;
server minio6:9000 max_fails=1 fail_timeout=10s;
server minio7:9000 max_fails=1 fail_timeout=10s;
server minio8:9000 max_fails=1 fail_timeout=10s;
}
upstream console {
ip_hash;
server minio1:9001;
server minio2:9001;
server minio3:9001;
server minio4:9001;
server minio5:9001;
server minio6:9001;
server minio7:9001;
server minio8:9001;
}
server {
listen 9000;
listen [::]:9000;
server_name localhost;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
proxy_request_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 300;
# Default is HTTP/1, keepalive is only enabled in HTTP/1.1
proxy_http_version 1.1;
proxy_set_header Connection "";
chunked_transfer_encoding off;
proxy_pass http://minio;
}
}
server {
listen 9001;
listen [::]:9001;
server_name localhost;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
proxy_request_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-NginX-Proxy true;
# This is necessary to pass the correct IP to be hashed
real_ip_header X-Real-IP;
proxy_connect_timeout 300;
# To support websocket
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
chunked_transfer_encoding off;
proxy_pass http://console;
}
}
}
+106
View File
@@ -0,0 +1,106 @@
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 4096;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
keepalive_timeout 65;
# include /etc/nginx/conf.d/*.conf;
upstream minio {
server minio1:9000 max_fails=1 fail_timeout=10s;
server minio2:9000 max_fails=1 fail_timeout=10s;
server minio3:9000 max_fails=1 fail_timeout=10s;
server minio4:9000 max_fails=1 fail_timeout=10s;
}
upstream console {
ip_hash;
server minio1:9001;
server minio2:9001;
server minio3:9001;
server minio4:9001;
}
server {
listen 9000;
listen [::]:9000;
server_name localhost;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
proxy_request_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 300;
# Default is HTTP/1, keepalive is only enabled in HTTP/1.1
proxy_http_version 1.1;
proxy_set_header Connection "";
chunked_transfer_encoding off;
proxy_pass http://minio;
}
}
server {
listen 9001;
listen [::]:9001;
server_name localhost;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
proxy_request_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-NginX-Proxy true;
# This is necessary to pass the correct IP to be hashed
real_ip_header X-Real-IP;
proxy_connect_timeout 300;
# To support websocket
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
chunked_transfer_encoding off;
proxy_pass http://console;
}
}
}
@@ -0,0 +1,66 @@
version: '3.7'
# Settings and configurations that are common for all containers
x-minio-common: &minio-common
image: quay.io/minio/minio:${RELEASE}
command: server http://site1-minio{1...4}/data{1...2}
environment:
- MINIO_PROMETHEUS_AUTH_TYPE=public
- CI=true
# starts 4 docker containers running minio server instances.
# using nginx reverse proxy, load balancing, you can access
# it through port 9000.
services:
site1-minio1:
<<: *minio-common
hostname: site1-minio1
volumes:
- site1-data1-1:/data1
- site1-data1-2:/data2
site1-minio2:
<<: *minio-common
hostname: site1-minio2
volumes:
- site1-data2-1:/data1
- site1-data2-2:/data2
site1-minio3:
<<: *minio-common
hostname: site1-minio3
volumes:
- site1-data3-1:/data1
- site1-data3-2:/data2
site1-minio4:
<<: *minio-common
hostname: site1-minio4
volumes:
- site1-data4-1:/data1
- site1-data4-2:/data2
site1-nginx:
image: nginx:1.19.2-alpine
hostname: site1-nginx
volumes:
- ./nginx-site1.conf:/etc/nginx/nginx.conf:ro
ports:
- "9001:9001"
depends_on:
- site1-minio1
- site1-minio2
- site1-minio3
- site1-minio4
## By default this config uses default local driver,
## For custom volumes replace with volume driver configuration.
volumes:
site1-data1-1:
site1-data1-2:
site1-data2-1:
site1-data2-2:
site1-data3-1:
site1-data3-2:
site1-data4-1:
site1-data4-2:
@@ -0,0 +1,66 @@
version: '3.7'
# Settings and configurations that are common for all containers
x-minio-common: &minio-common
image: quay.io/minio/minio:${RELEASE}
command: server http://site2-minio{1...4}/data{1...2}
environment:
- MINIO_PROMETHEUS_AUTH_TYPE=public
- CI=true
# starts 4 docker containers running minio server instances.
# using nginx reverse proxy, load balancing, you can access
# it through port 9000.
services:
site2-minio1:
<<: *minio-common
hostname: site2-minio1
volumes:
- site2-data1-1:/data1
- site2-data1-2:/data2
site2-minio2:
<<: *minio-common
hostname: site2-minio2
volumes:
- site2-data2-1:/data1
- site2-data2-2:/data2
site2-minio3:
<<: *minio-common
hostname: site2-minio3
volumes:
- site2-data3-1:/data1
- site2-data3-2:/data2
site2-minio4:
<<: *minio-common
hostname: site2-minio4
volumes:
- site2-data4-1:/data1
- site2-data4-2:/data2
site2-nginx:
image: nginx:1.19.2-alpine
hostname: site2-nginx
volumes:
- ./nginx-site2.conf:/etc/nginx/nginx.conf:ro
ports:
- "9002:9002"
depends_on:
- site2-minio1
- site2-minio2
- site2-minio3
- site2-minio4
## By default this config uses default local driver,
## For custom volumes replace with volume driver configuration.
volumes:
site2-data1-1:
site2-data1-2:
site2-data2-1:
site2-data2-2:
site2-data3-1:
site2-data3-2:
site2-data4-1:
site2-data4-2:
+147
View File
@@ -0,0 +1,147 @@
#!/bin/bash
set -x
## change working directory
cd .github/workflows/multipart/
function cleanup() {
docker-compose -f docker-compose-site1.yaml rm -s -f || true
docker-compose -f docker-compose-site2.yaml rm -s -f || true
for volume in $(docker volume ls -q | grep minio); do
docker volume rm ${volume} || true
done
docker system prune -f || true
docker volume prune -f || true
docker volume rm $(docker volume ls -q -f dangling=true) || true
}
cleanup
if [ ! -f ./mc ]; then
wget --quiet -O mc https://dl.minio.io/client/mc/release/linux-amd64/mc &&
chmod +x mc
fi
export RELEASE=RELEASE.2023-08-29T23-07-35Z
docker-compose -f docker-compose-site1.yaml up -d
docker-compose -f docker-compose-site2.yaml up -d
sleep 30s
./mc alias set site1 http://site1-nginx:9001 minioadmin minioadmin --api s3v4
./mc alias set site2 http://site2-nginx:9002 minioadmin minioadmin --api s3v4
./mc ready site1/
./mc ready site2/
./mc admin replicate add site1 site2
./mc mb site1/testbucket/
./mc cp -r --quiet /usr/bin site1/testbucket/
sleep 5
./s3-check-md5 -h
failed_count_site1=$(./s3-check-md5 -versions -access-key minioadmin -secret-key minioadmin -endpoint http://site1-nginx:9001 -bucket testbucket 2>&1 | grep FAILED | wc -l)
failed_count_site2=$(./s3-check-md5 -versions -access-key minioadmin -secret-key minioadmin -endpoint http://site2-nginx:9002 -bucket testbucket 2>&1 | grep FAILED | wc -l)
if [ $failed_count_site1 -ne 0 ]; then
echo "failed with multipart on site1 uploads"
exit 1
fi
if [ $failed_count_site2 -ne 0 ]; then
echo "failed with multipart on site2 uploads"
exit 1
fi
./mc cp -r --quiet /usr/bin site1/testbucket/
sleep 5
failed_count_site1=$(./s3-check-md5 -versions -access-key minioadmin -secret-key minioadmin -endpoint http://site1-nginx:9001 -bucket testbucket 2>&1 | grep FAILED | wc -l)
failed_count_site2=$(./s3-check-md5 -versions -access-key minioadmin -secret-key minioadmin -endpoint http://site2-nginx:9002 -bucket testbucket 2>&1 | grep FAILED | wc -l)
## we do not need to fail here, since we are going to test
## upgrading to master, healing and being able to recover
## the last version.
if [ $failed_count_site1 -ne 0 ]; then
echo "failed with multipart on site1 uploads ${failed_count_site1}"
fi
if [ $failed_count_site2 -ne 0 ]; then
echo "failed with multipart on site2 uploads ${failed_count_site2}"
fi
export RELEASE=${1}
docker-compose -f docker-compose-site1.yaml up -d
docker-compose -f docker-compose-site2.yaml up -d
./mc ready site1/
./mc ready site2/
for i in $(seq 1 10); do
# mc admin heal -r --remove when used against a LB endpoint
# behaves flaky, let this run 10 times before giving up
./mc admin heal -r --remove --json site1/ 2>&1 >/dev/null
./mc admin heal -r --remove --json site2/ 2>&1 >/dev/null
done
failed_count_site1=$(./s3-check-md5 -versions -access-key minioadmin -secret-key minioadmin -endpoint http://site1-nginx:9001 -bucket testbucket 2>&1 | grep FAILED | wc -l)
failed_count_site2=$(./s3-check-md5 -versions -access-key minioadmin -secret-key minioadmin -endpoint http://site2-nginx:9002 -bucket testbucket 2>&1 | grep FAILED | wc -l)
if [ $failed_count_site1 -ne 0 ]; then
echo "failed with multipart on site1 uploads"
exit 1
fi
if [ $failed_count_site2 -ne 0 ]; then
echo "failed with multipart on site2 uploads"
exit 1
fi
# Add user group test
./mc admin user add site1 site-replication-issue-user site-replication-issue-password
./mc admin group add site1 site-replication-issue-group site-replication-issue-user
max_wait_attempts=30
wait_interval=5
attempt=1
while true; do
diff <(./mc admin group info site1 site-replication-issue-group) <(./mc admin group info site2 site-replication-issue-group)
if [[ $? -eq 0 ]]; then
echo "Outputs are consistent."
break
fi
remaining_attempts=$((max_wait_attempts - attempt))
if ((attempt >= max_wait_attempts)); then
echo "Outputs remain inconsistent after $max_wait_attempts attempts. Exiting with error."
exit 1
else
echo "Outputs are inconsistent. Waiting for $wait_interval seconds (attempt $attempt/$max_wait_attempts)."
sleep $wait_interval
fi
((attempt++))
done
status=$(./mc admin group info site1 site-replication-issue-group --json | jq .groupStatus | tr -d '"')
if [[ $status == "enabled" ]]; then
echo "Success"
else
echo "Expected status: enabled, actual status: $status"
exit 1
fi
cleanup
## change working directory
cd ../../../
@@ -0,0 +1,61 @@
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 4096;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
keepalive_timeout 65;
# include /etc/nginx/conf.d/*.conf;
upstream minio {
server site1-minio1:9000;
server site1-minio2:9000;
server site1-minio3:9000;
server site1-minio4:9000;
}
server {
listen 9001;
listen [::]:9001;
server_name localhost;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
proxy_request_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 300;
# Default is HTTP/1, keepalive is only enabled in HTTP/1.1
proxy_http_version 1.1;
proxy_set_header Connection "";
chunked_transfer_encoding off;
proxy_pass http://minio;
}
}
}
@@ -0,0 +1,61 @@
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 4096;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
keepalive_timeout 65;
# include /etc/nginx/conf.d/*.conf;
upstream minio {
server site2-minio1:9000;
server site2-minio2:9000;
server site2-minio3:9000;
server site2-minio4:9000;
}
server {
listen 9002;
listen [::]:9002;
server_name localhost;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
proxy_request_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 300;
# Default is HTTP/1, keepalive is only enabled in HTTP/1.1
proxy_http_version 1.1;
proxy_set_header Connection "";
chunked_transfer_encoding off;
proxy_pass http://minio;
}
}
}
+53 -163
View File
@@ -1,103 +1,39 @@
name: Release
# Retry contract: an absent or single unfinalized Draft may be rebuilt from
# scratch; a published release or a Draft carrying finalize's GPG-derived
# provenance marker is terminal for this lane. The per-tag lock serializes
# workflows, but a maintainer must not publish the Draft while this job runs.
on:
push:
tags:
- "RELEASE.*"
workflow_dispatch:
inputs:
tag:
description: "Release tag (e.g. RELEASE.2026-03-24T12-00-00Z)"
required: true
permissions:
contents: write
id-token: write
attestations: write
artifact-metadata: write
concurrency:
group: release-${{ github.event_name == 'workflow_dispatch' && inputs.tag || github.ref_name }}
cancel-in-progress: false
packages: write
jobs:
release:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v7
uses: actions/checkout@v4
with:
fetch-depth: 0
# Build the code at the tag being released, not whatever branch the
# dispatch ran from. On a tag push this is the tag ref already; on
# workflow_dispatch it pins the checkout to the requested tag so the
# artifacts cannot be built from one ref and published under another.
ref: ${{ github.event.inputs.tag || github.ref }}
- name: Set up Go
uses: actions/setup-go@v7
uses: actions/setup-go@v5
with:
go-version-file: go.mod
cache: true
- name: Verify clean checkout
run: |
set -euo pipefail
# GitHub's OIDC certificate records GITHUB_SHA, not the ref passed to
# actions/checkout. A manual dispatch must therefore be launched from
# the release tag itself; otherwise the provenance identity would
# describe different source from the bytes being published.
CHECKED_OUT_REVISION="$(git rev-parse HEAD)"
if [ "${CHECKED_OUT_REVISION}" != "${GITHUB_SHA}" ]; then
echo "Checked out ${CHECKED_OUT_REVISION}, but workflow identity is ${GITHUB_SHA}. Dispatch from the release tag." >&2
exit 1
fi
if [ -n "$(git status --porcelain)" ]; then
echo "Refusing to release from a dirty working tree:" >&2
git status --porcelain >&2
exit 1
fi
- name: Verify rebrand compatibility contracts
run: |
set -euo pipefail
go run ./buildscripts/rebrand-guard
buildscripts/verify-rebrand.sh
dockerscripts/docker-entrypoint_test.sh
- name: Compute release variables
env:
# Passed through the environment, never interpolated into the script
# body: a dispatch input reaches bash as data, so it cannot inject
# commands the way a `${{ ... }}` splice into the source would.
INPUT_TAG: ${{ github.event.inputs.tag }}
run: |
set -euo pipefail
TAG="${INPUT_TAG:-${GITHUB_REF_NAME}}"
# Whitelist the exact tag shape before the value is used anywhere. bash
# =~ anchors ^...$ to the whole string (not per line, as sed would), so
# a tag carrying a newline cannot pass and then smuggle extra lines into
# $GITHUB_ENV below.
if [[ ! "${TAG}" =~ ^RELEASE\.[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}-[0-9]{2}-[0-9]{2}Z$ ]]; then
echo "Invalid release tag format: ${TAG}" >&2
exit 1
fi
if ! TAG_COMMIT="$(git rev-parse "${TAG}^{commit}" 2>/dev/null)"; then
echo "Release tag ${TAG} does not resolve to a commit" >&2
exit 1
fi
HEAD_COMMIT="$(git rev-parse HEAD)"
if [ "${TAG_COMMIT}" != "${HEAD_COMMIT}" ]; then
echo "Release tag ${TAG} resolves to ${TAG_COMMIT}, checkout is ${HEAD_COMMIT}" >&2
exit 1
fi
TAG="${GITHUB_REF_NAME}"
VERSION_HYPHEN="${TAG#RELEASE.}"
PKG_VERSION="$(echo "${VERSION_HYPHEN}" | sed -E 's/^([0-9]{4})-([0-9]{2})-([0-9]{2})T([0-9]{2})-([0-9]{2})-([0-9]{2})Z$/\1\2\3\4\5\6.0.0/')"
if [ "${PKG_VERSION}" = "${VERSION_HYPHEN}" ]; then
echo "Invalid release tag format: ${TAG}"
exit 1
fi
VERSION_COLON="$(echo "${VERSION_HYPHEN}" | sed -E 's/T([0-9]{2})-([0-9]{2})-([0-9]{2})Z$/T\1:\2:\3Z/')"
LDFLAGS="$(MINIO_RELEASE=RELEASE go run buildscripts/gen-ldflags.go "${VERSION_COLON}")"
@@ -111,136 +47,90 @@ jobs:
echo "Package version: ${PKG_VERSION}"
echo "LDFLAGS: ${LDFLAGS}"
- name: Check existing release state
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Validate Docker Hub credentials
run: |
set -euo pipefail
buildscripts/check-release-state.sh "${RELEASE_TAG}"
if [ -z "${{ secrets.DOCKERHUB_USERNAME }}" ] || [ -z "${{ secrets.DOCKERHUB_TOKEN }}" ]; then
echo "Missing Docker Hub credentials. Set DOCKERHUB_USERNAME and DOCKERHUB_TOKEN repository secrets."
exit 1
fi
# Both installer actions are pinned to immutable commits. The explicit
# tool versions keep the release format reproducible across workflow
# reruns while the installers verify the downloaded executables.
- name: Install Syft
uses: anchore/sbom-action/download-syft@e22c389904149dbc22b58101806040fa8d37a610 # v0.24.0
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
with:
syft-version: v1.50.0
platforms: arm64
- name: Install Cosign
uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
cosign-release: v3.1.2
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build Draft release with GoReleaser
uses: goreleaser/goreleaser-action@v7
- name: Build and publish with GoReleaser
uses: goreleaser/goreleaser-action@v6
with:
version: "~> v2"
args: release --clean --skip=validate --config .github/goreleaser.yml
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GORELEASER_CURRENT_TAG: ${{ env.RELEASE_TAG }}
LDFLAGS: ${{ env.LDFLAGS }}
PKG_VERSION: ${{ env.PKG_VERSION }}
- name: Verify binary provenance stamps
- name: Install pkger
run: |
set -euo pipefail
buildscripts/verify-build-provenance.sh
- name: Install nFPM
run: |
set -euo pipefail
go install github.com/goreleaser/nfpm/v2/cmd/nfpm@v2.47.0
go install github.com/minio/pkger/v2@v2.6.18
echo "$(go env GOPATH)/bin" >> "${GITHUB_PATH}"
- name: Build nFPM packages
- name: Prepare package layout
run: |
set -euo pipefail
buildscripts/package-release.sh
- name: Generate package SBOMs and signed checksum manifest
env:
SYFT_CHECK_FOR_APP_UPDATE: "false"
run: |
set -euo pipefail
packages_dir="dist/packages"
mapfile -t packages < <(find "${packages_dir}" -maxdepth 1 -type f \
\( -name '*.rpm' -o -name '*.deb' -o -name '*.apk' \) | sort)
if [ "${#packages[@]}" -ne 6 ]; then
echo "Expected six Linux packages, found ${#packages[@]}" >&2
printf '%s\n' "${packages[@]}" >&2
exit 1
fi
for package in "${packages[@]}"; do
syft "${package}" --output "spdx-json=${package}.sbom.json"
done
manifest="${packages_dir}/silo_${PKG_VERSION}_packages_checksums.txt"
(
cd "${packages_dir}"
mapfile -t subjects < <(find . -maxdepth 1 -type f \
\( -name '*.rpm' -o -name '*.deb' -o -name '*.apk' -o -name '*.sbom.json' \) | sort)
if [ "${#subjects[@]}" -ne 12 ]; then
echo "Expected six packages and six SBOMs, found ${#subjects[@]} subjects" >&2
copy_binary() {
local arch="$1"
local pattern="$2"
local src
src="$(find dist -maxdepth 2 -type f -path "dist/${pattern}/minio" | head -n1 || true)"
if [ -z "${src}" ]; then
echo "Missing GoReleaser binary for ${arch} (${pattern})"
exit 1
fi
sha256sum "${subjects[@]}" | sed 's# \./# #' > "$(basename "${manifest}")"
)
cosign sign-blob --bundle="${manifest}.sigstore.json" "${manifest}" --yes
mkdir -p "dist/linux-${arch}"
cp "${src}" "dist/linux-${arch}/minio.${RELEASE_TAG}"
}
- name: Attest downloadable release artifacts
id: attest-release
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-path: |
dist/*.tar.gz
dist/*.zip
dist/*.sbom.json
dist/*_checksums.txt
dist/*.sigstore.json
dist/packages/*.rpm
dist/packages/*.deb
dist/packages/*.apk
dist/packages/*.sha256sum
dist/packages/*.sbom.json
dist/packages/*_checksums.txt
dist/packages/*.sigstore.json
copy_binary amd64 "minio_linux_amd64*"
copy_binary arm64 "minio_linux_arm64*"
- name: Preserve provenance bundle as a release asset
env:
BUNDLE_PATH: ${{ steps.attest-release.outputs.bundle-path }}
- name: Build standard pkger packages
run: |
set -euo pipefail
test -s "${BUNDLE_PATH}"
cp "${BUNDLE_PATH}" "dist/silo_${PKG_VERSION}_provenance.sigstore.json"
pkger -r "${RELEASE_TAG}" --appName minio --releaseDir dist --ignore
- name: Confirm unfinalized Draft release state
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REQUIRE_DRAFT: "true"
run: |
set -euo pipefail
buildscripts/check-release-state.sh "${RELEASE_TAG}"
# Keep only full package files; drop convenience symlinks (minio.rpm/minio.deb/minio.apk)
find dist/linux-* -maxdepth 1 -type l \
\( -name 'minio.rpm' -o -name 'minio.deb' -o -name 'minio.apk' \) -delete
- name: Upload nFPM packages to Draft release
find dist -maxdepth 2 -type f \
\( -name '*.rpm' -o -name '*.deb' -o -name '*.apk' -o -name '*.sha256sum' -o -name 'downloads-minio.json' \) | sort
- name: Upload pkger artifacts to GitHub release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
mapfile -t files < <(find dist/packages -maxdepth 1 -type f \
\( -name '*.rpm' -o -name '*.deb' -o -name '*.apk' -o -name '*.sha256sum' \
-o -name '*.sbom.json' -o -name '*_checksums.txt' -o -name '*.sigstore.json' \) | sort)
mapfile -t files < <(find dist -maxdepth 2 -type f \
\( -name '*.rpm' -o -name '*.deb' -o -name '*.apk' -o -name '*.sha256sum' -o -name 'downloads-minio.json' \) | sort)
if [ "${#files[@]}" -eq 0 ]; then
echo "No packages were generated."
exit 1
fi
gh release upload "${RELEASE_TAG}" "${files[@]}" \
"dist/silo_${PKG_VERSION}_provenance.sigstore.json"
gh release upload "${RELEASE_TAG}" "${files[@]}" --clobber
- name: Upload dist artifact
if: always()
uses: actions/upload-artifact@v7
uses: actions/upload-artifact@v4
with:
name: dist
path: dist/
+79
View File
@@ -0,0 +1,79 @@
name: MinIO advanced tests
on:
pull_request:
branches:
- master
# This ensures that previous jobs for the PR are canceled when the PR is
# updated.
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
replication-test:
name: Advanced Tests with Go ${{ matrix.go-version }}
runs-on: ubuntu-latest
strategy:
matrix:
go-version: [1.24.x]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go-version }}
check-latest: true
- name: Test Decom
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make test-decom
- name: Test ILM
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make test-ilm
make test-ilm-transition
- name: Test PBAC
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make test-pbac
- name: Test Config File
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make test-configfile
- name: Test Replication
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make test-replication
- name: Test MinIO IDP for automatic site replication
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make test-site-replication-minio
- name: Test Versioning
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make test-versioning
- name: Test Multipart upload with failures
run: |
sudo sysctl net.ipv6.conf.all.disable_ipv6=0
sudo sysctl net.ipv6.conf.default.disable_ipv6=0
make test-multipart
+34
View File
@@ -0,0 +1,34 @@
name: Root lockdown tests
on:
pull_request:
branches:
- master
# This ensures that previous jobs for the PR are canceled when the PR is
# updated.
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
build:
name: Go ${{ matrix.go-version }} on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
go-version: [1.24.x]
os: [ubuntu-latest]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go-version }}
check-latest: true
- name: Start root lockdown tests
run: |
make test-root-disable
+9
View File
@@ -0,0 +1,9 @@
-----BEGIN CERTIFICATE-----
MIIBKDCB26ADAgECAhB6vebGMUfKnmBKyqoApRSOMAUGAytlcDAbMRkwFwYDVQQD
DBByb290QHBsYXkubWluLmlvMB4XDTIwMDQzMDE1MjIyNVoXDTI1MDQyOTE1MjIy
NVowGzEZMBcGA1UEAwwQcm9vdEBwbGF5Lm1pbi5pbzAqMAUGAytlcAMhALzn735W
fmSH/ghKs+4iPWziZMmWdiWr/sqvqeW+WwSxozUwMzAOBgNVHQ8BAf8EBAMCB4Aw
EwYDVR0lBAwwCgYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAFBgMrZXADQQDZOrGK
b2ATkDlu2pTcP3LyhSBDpYh7V4TvjRkBTRgjkacCzwFLm+mh+7US8V4dBpIDsJ4u
uWoF0y6vbLVGIlkG
-----END CERTIFICATE-----
+3
View File
@@ -0,0 +1,3 @@
-----BEGIN PRIVATE KEY-----
MC4CAQAwBQYDK2VwBCIEID9E7FSYWrMD+VjhI6q545cYT9YOyFxZb7UnjEepYDRc
-----END PRIVATE KEY-----
+64
View File
@@ -0,0 +1,64 @@
#!/bin/bash
set -ex
export MODE="$1"
export ACCESS_KEY="$2"
export SECRET_KEY="$3"
export JOB_NAME="$4"
export MINT_MODE="full"
docker system prune -f || true
docker volume prune -f || true
docker volume rm $(docker volume ls -f dangling=true) || true
## change working directory
cd .github/workflows/mint
## always pull latest
docker pull docker.io/minio/mint:edge
docker-compose -f minio-${MODE}.yaml up -d
sleep 1m
docker system prune -f || true
docker volume prune -f || true
docker volume rm $(docker volume ls -q -f dangling=true) || true
# Stop two nodes, one of each pool, to check that all S3 calls work while quorum is still there
[ "${MODE}" == "pools" ] && docker-compose -f minio-${MODE}.yaml stop minio2
[ "${MODE}" == "pools" ] && docker-compose -f minio-${MODE}.yaml stop minio6
# Pause one node, to check that all S3 calls work while one node goes wrong
[ "${MODE}" == "resiliency" ] && docker-compose -f minio-${MODE}.yaml pause minio4
docker run --rm --net=mint_default \
--name="mint-${MODE}-${JOB_NAME}" \
-e SERVER_ENDPOINT="nginx:9000" \
-e ACCESS_KEY="${ACCESS_KEY}" \
-e SECRET_KEY="${SECRET_KEY}" \
-e ENABLE_HTTPS=0 \
-e MINT_MODE="${MINT_MODE}" \
docker.io/minio/mint:edge
# FIXME: enable this after fixing aws-sdk-java-v2 tests
# # unpause the node, to check that all S3 calls work while one node goes wrong
# [ "${MODE}" == "resiliency" ] && docker-compose -f minio-${MODE}.yaml unpause minio4
# [ "${MODE}" == "resiliency" ] && docker run --rm --net=mint_default \
# --name="mint-${MODE}-${JOB_NAME}" \
# -e SERVER_ENDPOINT="nginx:9000" \
# -e ACCESS_KEY="${ACCESS_KEY}" \
# -e SECRET_KEY="${SECRET_KEY}" \
# -e ENABLE_HTTPS=0 \
# -e MINT_MODE="${MINT_MODE}" \
# docker.io/minio/mint:edge
docker-compose -f minio-${MODE}.yaml down || true
sleep 10s
docker system prune -f || true
docker volume prune -f || true
docker volume rm $(docker volume ls -q -f dangling=true) || true
## change working directory
cd ../../../
+22
View File
@@ -0,0 +1,22 @@
name: Shell formatting checks
on:
pull_request:
branches:
- master
permissions:
contents: read
jobs:
build:
name: runner / shfmt
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: luizm/action-sh-checker@master
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SHFMT_OPTS: "-s"
with:
sh_checker_shellcheck_disable: true # disable for now
+31 -472
View File
@@ -5,74 +5,25 @@ on:
pull_request:
paths:
- ".github/goreleaser.yml"
- ".github/nfpm.yml"
- "Dockerfile.goreleaser"
- "Dockerfile.distroless"
- "cmd/healthcheck-main.go"
- "cmd/main.go"
- "dockerscripts/build-static-curl.sh"
- "dockerscripts/docker-entrypoint.sh"
- "dockerscripts/docker-entrypoint_test.sh"
- "silo.service"
- "silo.env"
- "silo.sysusers"
- "buildscripts/package-release.sh"
- "buildscripts/package/postinstall.sh"
- "buildscripts/package/preremove.sh"
- "buildscripts/package/lifecycle_test.sh"
- "buildscripts/minio-upgrade.sh"
- "buildscripts/sign-release-rpms.sh"
- "buildscripts/verify-build-provenance.sh"
- "buildscripts/check-release-state.sh"
- "buildscripts/check-release-state_test.sh"
- "buildscripts/verify-rebrand.sh"
- "buildscripts/verify-helm-migration.sh"
- "buildscripts/helm-migration-guard/**"
- "helm/silo/**"
- "buildscripts/rebrand-guard/**"
- "buildscripts/gen-ldflags.go"
- "minio.service"
- ".github/workflows/release.yml"
- ".github/workflows/docker-release.yml"
- ".github/workflows/finalize-release.yml"
- ".github/workflows/test-release.yml"
- ".gitignore"
permissions:
contents: read
jobs:
curl:
name: Static curl (${{ matrix.arch }})
runs-on: ${{ matrix.runner }}
strategy:
matrix:
include:
- arch: amd64
runner: ubuntu-latest
- arch: arm64
runner: ubuntu-24.04-arm
steps:
- uses: actions/checkout@v7
- name: Build and exercise curl in an empty runtime
run: |
docker build --build-arg TARGETARCH=${{ matrix.arch }} \
--target curl-runtime -f Dockerfile.goreleaser -t silo-curl-test .
docker run --rm silo-curl-test --version | tee curl-version.txt
grep -F 'curl 8.22.0 ' curl-version.txt
grep -F 'HTTP2' curl-version.txt
docker run --rm silo-curl-test --fail --silent --show-error \
--connect-timeout 15 --max-time 60 https://curl.se/robots.txt
validate:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v7
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Go
uses: actions/setup-go@v7
uses: actions/setup-go@v5
with:
go-version-file: go.mod
cache: true
@@ -84,447 +35,55 @@ jobs:
VERSION_COLON="2026-02-14T12:00:00Z"
PKG_VERSION="20260214120000.0.0"
LDFLAGS="$(MINIO_RELEASE=RELEASE go run buildscripts/gen-ldflags.go "${VERSION_COLON}")"
{
echo "RELEASE_TAG=${RELEASE_TAG}"
echo "PKG_VERSION=${PKG_VERSION}"
echo "LDFLAGS=${LDFLAGS}"
} >> "${GITHUB_ENV}"
echo "RELEASE_TAG=${RELEASE_TAG}" >> "${GITHUB_ENV}"
echo "PKG_VERSION=${PKG_VERSION}" >> "${GITHUB_ENV}"
echo "LDFLAGS=${LDFLAGS}" >> "${GITHUB_ENV}"
echo "PKG_VERSION: ${PKG_VERSION}"
echo "LDFLAGS: ${LDFLAGS}"
- name: GoReleaser config check
uses: goreleaser/goreleaser-action@v7
uses: goreleaser/goreleaser-action@v6
with:
version: "~> v2"
args: check --config .github/goreleaser.yml
- name: Validate Helm chart and legacy upgrade identity
run: buildscripts/verify-helm-migration.sh
- name: Install Syft
uses: anchore/sbom-action/download-syft@e22c389904149dbc22b58101806040fa8d37a610 # v0.24.0
with:
syft-version: v1.50.0
- name: Build snapshot artifacts
uses: goreleaser/goreleaser-action@v7
uses: goreleaser/goreleaser-action@v6
with:
version: "~> v2"
# A pull-request snapshot has no trusted release identity. Exercise
# the SBOM/checksum pipeline here, and reserve keyless signing for
# the tag-triggered release workflow with GitHub OIDC.
args: release --snapshot --clean --skip=publish,docker,sign --config .github/goreleaser.yml
args: release --snapshot --clean --skip=publish,docker --config .github/goreleaser.yml
env:
LDFLAGS: ${{ env.LDFLAGS }}
PKG_VERSION: ${{ env.PKG_VERSION }}
- name: Verify archive SBOM and checksum coverage
- name: Install pkger
run: |
set -euo pipefail
mapfile -t archives < <(find dist -maxdepth 1 -type f -name 'silo_*.tar.gz' | sort)
mapfile -t sboms < <(find dist -maxdepth 1 -type f -name 'silo_*.tar.gz.sbom.json' | sort)
test "${#archives[@]}" -eq 6
test "${#sboms[@]}" -eq 6
manifest="dist/silo_${PKG_VERSION}_checksums.txt"
test -s "${manifest}"
(
cd dist
sha256sum --check "$(basename "${manifest}")"
)
test "$(wc -l < "${manifest}" | tr -d ' ')" -eq 12
- name: Verify binary provenance stamps
run: |
set -euo pipefail
buildscripts/verify-build-provenance.sh
- name: Install package validation tools
run: |
set -euo pipefail
go install github.com/goreleaser/nfpm/v2/cmd/nfpm@v2.47.0
go install github.com/minio/pkger/v2@v2.6.18
echo "$(go env GOPATH)/bin" >> "${GITHUB_PATH}"
sudo apt-get update
sudo apt-get install --yes rpm binutils
- name: Package snapshot binaries with nFPM
- name: Package snapshot binaries with pkger
run: |
set -euo pipefail
buildscripts/package-release.sh
- name: Validate package names, checksums, metadata, and payload
run: |
set -euo pipefail
cd dist/packages
# The signing script asserts these same values, but it runs on the
# maintainer's machine after the release workflow has already built
# and uploaded. Take its expectations as the single source of truth
# so nfpm.yml and the signing script cannot drift apart without
# failing here first, while a fix is still cheap.
#
# This grep is deliberately limited to the eight identity variables,
# all of which are single-line. That is what makes the eval safe:
# should one ever become multi-line, the grep captures an
# unterminated quote and the eval aborts on a syntax error under
# set -e rather than quietly binding an empty value and comparing
# against nothing. expected_payload is multi-line by design and must
# stay out of this set for the same reason.
eval "$(grep -E '^expected_(release|vendor|packager|url|summary|description|license|group)=' \
../../buildscripts/sign-release-rpms.sh)"
for value in "${expected_release}" "${expected_vendor}" "${expected_packager}" \
"${expected_url}" "${expected_summary}" "${expected_description}" \
"${expected_license}" "${expected_group}"; do
test -n "${value}"
done
# These are the public download names; a drift here breaks every
# script that fetches packages by URL. RPM and DEB carry the PGSTY
# release segment; APK cannot (Alpine pkgrel admits only -r<integer>),
# so it stays bare. package-release.sh builds the same three shapes.
expected=(
"silo-${PKG_VERSION}-${expected_release}.aarch64.rpm"
"silo-${PKG_VERSION}-${expected_release}.x86_64.rpm"
"silo_${PKG_VERSION}-${expected_release}_amd64.deb"
"silo_${PKG_VERSION}-${expected_release}_arm64.deb"
"silo_${PKG_VERSION}_aarch64.apk"
"silo_${PKG_VERSION}_x86_64.apk"
)
for package in "${expected[@]}"; do
test -s "${package}"
test -s "${package}.sha256sum"
sha256sum --check "${package}.sha256sum"
done
test "$(find . -maxdepth 1 -type f \( -name '*.rpm' -o -name '*.deb' -o -name '*.apk' \) | wc -l)" -eq 6
service_sha="$(sha256sum ../../silo.service | awk '{print $1}')"
defaults_sha="$(sha256sum ../../silo.env | awk '{print $1}')"
sysusers_sha="$(sha256sum ../../silo.sysusers | awk '{print $1}')"
license_sha="$(sha256sum ../../LICENSE | awk '{print $1}')"
notice_sha="$(sha256sum ../../NOTICE | awk '{print $1}')"
rpm_file="silo-${PKG_VERSION}-${expected_release}.x86_64.rpm"
test "$(rpm -qp --queryformat '%{RELEASE}' "${rpm_file}")" = "${expected_release}"
test "$(rpm -qp --queryformat '%{VENDOR}' "${rpm_file}")" = "${expected_vendor}"
test "$(rpm -qp --queryformat '%{PACKAGER}' "${rpm_file}")" = "${expected_packager}"
test "$(rpm -qp --queryformat '%{URL}' "${rpm_file}")" = "${expected_url}"
test "$(rpm -qp --queryformat '%{SUMMARY}' "${rpm_file}")" = "${expected_summary}"
test "$(rpm -qp --queryformat '%{DESCRIPTION}' "${rpm_file}")" = "${expected_description}"
test "$(rpm -qp --queryformat '%{LICENSE}' "${rpm_file}")" = "${expected_license}"
test "$(rpm -qp --queryformat '%{GROUP}' "${rpm_file}")" = "${expected_group}"
rpm -qpl "${rpm_file}" | grep -Fx '/usr/bin/silo'
rpm -qpl "${rpm_file}" | grep -Fx '/usr/lib/systemd/system/silo.service'
rpm -qpl "${rpm_file}" | grep -Fx '/etc/default/silo'
rpm -qpl "${rpm_file}" | grep -Fx '/usr/lib/sysusers.d/silo.conf'
rpm -qpl "${rpm_file}" | grep -Fx '/usr/share/doc/silo/LICENSE'
rpm -qpl "${rpm_file}" | grep -Fx '/usr/share/doc/silo/NOTICE'
test "$(rpm -qpl "${rpm_file}" | wc -l)" -eq 6
# nfpm only honors type: license on rpm, which is why nfpm.yml
# declares the license materials once per packager. Pin the rpm
# %license flag so that split cannot silently regress.
rpm -qp --queryformat '[%{FILEFLAGS:fflags} %{FILENAMES}\n]' "${rpm_file}" \
| grep -Fx 'l /usr/share/doc/silo/LICENSE'
rpm -qp --queryformat '[%{FILEFLAGS:fflags} %{FILENAMES}\n]' "${rpm_file}" \
| grep -Fx 'l /usr/share/doc/silo/NOTICE'
if rpm -qp --conflicts "${rpm_file}" | grep -qi minio; then
echo "RPM must not declare a cross-name conflict with MinIO" >&2
exit 1
fi
if rpm -qp --obsoletes "${rpm_file}" | grep -qi minio; then
echo "RPM must not obsolete a MinIO package" >&2
exit 1
fi
if rpm -qp --provides "${rpm_file}" | grep -qi minio; then
echo "RPM must not provide a MinIO package alias" >&2
exit 1
fi
deb_file="silo_${PKG_VERSION}-${expected_release}_amd64.deb"
test "$(dpkg-deb --field "${deb_file}" Maintainer)" = "${expected_packager}"
test "$(dpkg-deb --field "${deb_file}" Version)" = "${PKG_VERSION}-${expected_release}"
test "$(dpkg-deb --field "${deb_file}" License)" = "${expected_license}"
test "$(dpkg-deb --field "${deb_file}" Section)" = "utils"
test "$(dpkg-deb --field "${deb_file}" Homepage)" = "${expected_url}"
test "$(dpkg-deb --field "${deb_file}" Description)" = "${expected_description}"
dpkg-deb --contents "${deb_file}" | grep -E 'usr/bin/silo$'
dpkg-deb --contents "${deb_file}" | grep -E 'usr/lib/systemd/system/silo\.service$'
dpkg-deb --contents "${deb_file}" | grep -E 'etc/default/silo$'
dpkg-deb --contents "${deb_file}" | grep -E 'usr/lib/sysusers\.d/silo\.conf$'
dpkg-deb --contents "${deb_file}" | grep -E 'usr/share/doc/silo/LICENSE$'
dpkg-deb --contents "${deb_file}" | grep -E 'usr/share/doc/silo/NOTICE$'
test "$(dpkg-deb --contents "${deb_file}" | awk '$1 !~ /^d/ { count++ } END { print count + 0 }')" -eq 6
test -z "$(dpkg-deb --field "${deb_file}" Conflicts)"
test -z "$(dpkg-deb --field "${deb_file}" Replaces)"
test -z "$(dpkg-deb --field "${deb_file}" Provides)"
apk_info="$(tar -xOzf "silo_${PKG_VERSION}_x86_64.apk" .PKGINFO)"
grep -Fx "pkgver = ${PKG_VERSION}" <<< "${apk_info}"
grep -Fx "url = ${expected_url}" <<< "${apk_info}"
grep -Fx "maintainer = ${expected_packager}" <<< "${apk_info}"
grep -Fx "license = ${expected_license}" <<< "${apk_info}"
grep -Fx "pkgdesc = ${expected_description}" <<< "${apk_info}"
tar -tzf "silo_${PKG_VERSION}_x86_64.apk" | grep -Fx 'usr/bin/silo'
tar -tzf "silo_${PKG_VERSION}_x86_64.apk" | grep -Fx 'usr/lib/systemd/system/silo.service'
tar -tzf "silo_${PKG_VERSION}_x86_64.apk" | grep -Fx 'etc/default/silo'
tar -tzf "silo_${PKG_VERSION}_x86_64.apk" | grep -Fx 'usr/lib/sysusers.d/silo.conf'
tar -tzf "silo_${PKG_VERSION}_x86_64.apk" | grep -Fx 'usr/share/doc/silo/LICENSE'
tar -tzf "silo_${PKG_VERSION}_x86_64.apk" | grep -Fx 'usr/share/doc/silo/NOTICE'
test "$(tar -tzf "silo_${PKG_VERSION}_x86_64.apk" | awk '$0 !~ /^\./ && $0 !~ /\/$/ { count++ } END { print count + 0 }')" -eq 6
if grep -Ei '^provides = .*minio' <<< "${apk_info}"; then
echo "APK must not provide a MinIO package alias" >&2
exit 1
fi
for arch in amd64 arm64; do
if [ "${arch}" = amd64 ]; then
rpm_arch=x86_64
deb_arch=amd64
apk_arch=x86_64
else
rpm_arch=aarch64
deb_arch=arm64
apk_arch=aarch64
fi
test "$(rpm -qp --queryformat '%{ARCH}' "silo-${PKG_VERSION}-${expected_release}.${rpm_arch}.rpm")" = "${rpm_arch}"
test "$(dpkg-deb --field "silo_${PKG_VERSION}-${expected_release}_${deb_arch}.deb" Architecture)" = "${deb_arch}"
grep -Fx "arch = ${apk_arch}" <<< "$(tar -xOzf "silo_${PKG_VERSION}_${apk_arch}.apk" .PKGINFO)"
# Accepted weakness: this takes the first match, unsorted, where
# find_binary in package-release.sh demands exactly one. It cannot
# be reached with an ambiguous match today, because packaging runs
# earlier in this same job and hard-fails on one. Revisit if
# goamd64 gains a second level, or if find_binary's exactly-one
# contract is ever relaxed -- at that point this weak copy would be
# the only one left choosing silently.
source_binary="$(find .. -maxdepth 2 -type f -path "../silo_linux_${arch}*/silo" | head -n 1)"
source_sha="$(sha256sum "${source_binary}" | awk '{print $1}')"
# Do not pipe rpm2cpio here: Debian's build exits non-zero even when
# it writes a correct payload, which trips `set -o pipefail`. Use
# rpm's own digests instead -- -K checks the payload against the
# header, and FILEDIGESTS is the sha256 rpm itself verifies on
# install.
rpm -K "silo-${PKG_VERSION}-${expected_release}.${rpm_arch}.rpm"
rpm_sha="$(rpm -qp --queryformat '[%{FILENAMES} %{FILEDIGESTS}\n]' \
"silo-${PKG_VERSION}-${expected_release}.${rpm_arch}.rpm" | awk '$1 == "/usr/bin/silo" { print $2 }')"
rpm_service_sha="$(rpm -qp --queryformat '[%{FILENAMES} %{FILEDIGESTS}\n]' \
"silo-${PKG_VERSION}-${expected_release}.${rpm_arch}.rpm" | awk '$1 == "/usr/lib/systemd/system/silo.service" { print $2 }')"
rpm_defaults_sha="$(rpm -qp --queryformat '[%{FILENAMES} %{FILEDIGESTS}\n]' \
"silo-${PKG_VERSION}-${expected_release}.${rpm_arch}.rpm" | awk '$1 == "/etc/default/silo" { print $2 }')"
rpm_sysusers_sha="$(rpm -qp --queryformat '[%{FILENAMES} %{FILEDIGESTS}\n]' \
"silo-${PKG_VERSION}-${expected_release}.${rpm_arch}.rpm" | awk '$1 == "/usr/lib/sysusers.d/silo.conf" { print $2 }')"
rpm_license_sha="$(rpm -qp --queryformat '[%{FILENAMES} %{FILEDIGESTS}\n]' \
"silo-${PKG_VERSION}-${expected_release}.${rpm_arch}.rpm" | awk '$1 == "/usr/share/doc/silo/LICENSE" { print $2 }')"
rpm_notice_sha="$(rpm -qp --queryformat '[%{FILENAMES} %{FILEDIGESTS}\n]' \
"silo-${PKG_VERSION}-${expected_release}.${rpm_arch}.rpm" | awk '$1 == "/usr/share/doc/silo/NOTICE" { print $2 }')"
deb_sha="$(ar p "silo_${PKG_VERSION}-${expected_release}_${deb_arch}.deb" data.tar.gz | tar -xzOf - ./usr/bin/silo | sha256sum | awk '{print $1}')"
deb_service_sha="$(ar p "silo_${PKG_VERSION}-${expected_release}_${deb_arch}.deb" data.tar.gz | tar -xzOf - ./usr/lib/systemd/system/silo.service | sha256sum | awk '{print $1}')"
deb_defaults_sha="$(ar p "silo_${PKG_VERSION}-${expected_release}_${deb_arch}.deb" data.tar.gz | tar -xzOf - ./etc/default/silo | sha256sum | awk '{print $1}')"
deb_sysusers_sha="$(ar p "silo_${PKG_VERSION}-${expected_release}_${deb_arch}.deb" data.tar.gz | tar -xzOf - ./usr/lib/sysusers.d/silo.conf | sha256sum | awk '{print $1}')"
deb_license_sha="$(ar p "silo_${PKG_VERSION}-${expected_release}_${deb_arch}.deb" data.tar.gz | tar -xzOf - ./usr/share/doc/silo/LICENSE | sha256sum | awk '{print $1}')"
deb_notice_sha="$(ar p "silo_${PKG_VERSION}-${expected_release}_${deb_arch}.deb" data.tar.gz | tar -xzOf - ./usr/share/doc/silo/NOTICE | sha256sum | awk '{print $1}')"
apk_sha="$(tar -xzOf "silo_${PKG_VERSION}_${apk_arch}.apk" usr/bin/silo | sha256sum | awk '{print $1}')"
apk_service_sha="$(tar -xzOf "silo_${PKG_VERSION}_${apk_arch}.apk" usr/lib/systemd/system/silo.service | sha256sum | awk '{print $1}')"
apk_defaults_sha="$(tar -xzOf "silo_${PKG_VERSION}_${apk_arch}.apk" etc/default/silo | sha256sum | awk '{print $1}')"
apk_sysusers_sha="$(tar -xzOf "silo_${PKG_VERSION}_${apk_arch}.apk" usr/lib/sysusers.d/silo.conf | sha256sum | awk '{print $1}')"
apk_license_sha="$(tar -xzOf "silo_${PKG_VERSION}_${apk_arch}.apk" usr/share/doc/silo/LICENSE | sha256sum | awk '{print $1}')"
apk_notice_sha="$(tar -xzOf "silo_${PKG_VERSION}_${apk_arch}.apk" usr/share/doc/silo/NOTICE | sha256sum | awk '{print $1}')"
test "${source_sha}" = "${rpm_sha}"
test "${source_sha}" = "${deb_sha}"
test "${source_sha}" = "${apk_sha}"
test "${service_sha}" = "${rpm_service_sha}"
test "${service_sha}" = "${deb_service_sha}"
test "${service_sha}" = "${apk_service_sha}"
test "${defaults_sha}" = "${rpm_defaults_sha}"
test "${defaults_sha}" = "${deb_defaults_sha}"
test "${defaults_sha}" = "${apk_defaults_sha}"
test "${sysusers_sha}" = "${rpm_sysusers_sha}"
test "${sysusers_sha}" = "${deb_sysusers_sha}"
test "${sysusers_sha}" = "${apk_sysusers_sha}"
test "${license_sha}" = "${rpm_license_sha}"
test "${license_sha}" = "${deb_license_sha}"
test "${license_sha}" = "${apk_license_sha}"
test "${notice_sha}" = "${rpm_notice_sha}"
test "${notice_sha}" = "${deb_notice_sha}"
test "${notice_sha}" = "${apk_notice_sha}"
done
find . -maxdepth 1 -type f | sort
- name: Build release runtime image and verify graceful shutdown
run: |
set -euo pipefail
# docker-release.yml is workflow_dispatch only, so this is the only
# automated build of the release runtime layer and entrypoint before a
# real publish. Assemble a minimal image from the linux/amd64 binary
# goreleaser already produced; the mcli-download build stage is skipped
# on purpose to keep this gate offline and deterministic.
ctx="$(mktemp -d)"
tar -xzf "dist/silo_${PKG_VERSION}_linux_amd64.tar.gz" -C "${ctx}" silo
cp dockerscripts/docker-entrypoint.sh "${ctx}/docker-entrypoint.sh"
{
echo "FROM registry.access.redhat.com/ubi9/ubi-micro:latest"
echo "COPY silo /usr/bin/silo"
echo "COPY docker-entrypoint.sh /usr/bin/docker-entrypoint.sh"
echo "RUN mkdir -p /data && chmod 0777 /data && chmod +x /usr/bin/silo /usr/bin/docker-entrypoint.sh"
echo 'ENV HOME=/tmp'
echo 'ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"]'
echo 'CMD ["silo"]'
} > "${ctx}/Dockerfile"
docker build -t silo-runtime-test:snapshot "${ctx}"
# PID 1 must be silo, not the entry shell, on every privilege path, so
# a SIGTERM from docker stop reaches the server and it exits gracefully
# instead of being killed at the stop timeout. Regression guard for the
# exec-into-chroot entrypoint fix.
assert_graceful() {
name="$1"; shift
docker rm -f "${name}" >/dev/null 2>&1 || true
docker run -d --name "${name}" \
-e MINIO_CI_CD=1 -e MINIO_ROOT_USER=ciadmin -e MINIO_ROOT_PASSWORD=ciadmin-secret-123 \
"$@" silo-runtime-test:snapshot silo server /data --address :9000 >/dev/null
up=""
for _ in $(seq 1 60); do
if docker logs "${name}" 2>&1 | grep -q "API:"; then up=1; break; fi
if [ "$(docker inspect -f '{{.State.Running}}' "${name}")" != "true" ]; then break; fi
sleep 1
done
if [ -z "${up}" ]; then echo "server did not start (${name}):"; docker logs "${name}" | tail -5; exit 1; fi
pid1="$(docker exec "${name}" cat /proc/1/comm 2>/dev/null || echo '?')"
start="$(date +%s)"; docker stop -t 15 "${name}" >/dev/null; end="$(date +%s)"
code="$(docker inspect -f '{{.State.ExitCode}}' "${name}")"
elapsed=$((end - start))
echo "${name}: pid1=${pid1} stop=${elapsed}s exit=${code}"
graceful=0; docker logs "${name}" 2>&1 | grep -q "Exiting on signal" && graceful=1
docker rm -f "${name}" >/dev/null 2>&1 || true
[ "${graceful}" = "1" ] || { echo "no graceful-shutdown log (${name}) - signal not forwarded"; exit 1; }
[ "${code}" = "0" ] || { echo "non-zero exit (${name}): ${code}"; exit 1; }
[ "${elapsed}" -lt 10 ] || { echo "shutdown too slow (${name}): ${elapsed}s - signal not forwarded"; exit 1; }
}
assert_graceful silo-rt-default
assert_graceful silo-rt-dropuser -e MINIO_USERNAME=silo-user -e MINIO_GROUPNAME=silo-group
assert_graceful silo-rt-rootless --user 1001:1001
# The compatibility shim translates only the legacy first argv token;
# the image contains no /usr/bin/minio file.
docker run --rm silo-runtime-test:snapshot sh -c 'test ! -e /usr/bin/minio'
docker run --rm -d --name silo-rt-legacy \
-e MINIO_CI_CD=1 -e MINIO_ROOT_USER=ciadmin -e MINIO_ROOT_PASSWORD=ciadmin-secret-123 \
silo-runtime-test:snapshot minio server /data --address :9000 >/dev/null
sleep 2
test "$(docker exec silo-rt-legacy cat /proc/1/comm)" = silo
docker rm -f silo-rt-legacy >/dev/null
- name: Build distroless runtime image and verify native healthcheck
run: |
set -euo pipefail
# Unlike the classic image, Dockerfile.distroless has no release
# download stages, so the real shipped file can be built and gated
# here. It must keep working with nothing in it but the silo
# binary: no shell, no mc, no entrypoint script.
ctx="$(mktemp -d)"
tar -xzf "dist/silo_${PKG_VERSION}_linux_amd64.tar.gz" -C "${ctx}" silo
cp Dockerfile.distroless LICENSE NOTICE CREDITS "${ctx}/"
docker build -t silo-distroless-test:snapshot -f "${ctx}/Dockerfile.distroless" "${ctx}"
# HEALTHCHECK is a Docker extension absent from the OCI image
# spec; assert the exact probe command survived into the image
# config, not merely a substring of it.
test "$(docker inspect -f '{{json .Config.Healthcheck.Test}}' silo-distroless-test:snapshot)" \
= '["CMD","/usr/bin/silo","healthcheck","ready"]'
# /data ships in the image layer world-writable (issue #55):
# there is no entrypoint left to repair ownership at runtime.
# Export the rootfs once, then assert each required and each
# forbidden entry individually: tar's member-argument mode exits
# non-zero on any missing name, which under pipefail masks a
# found forbidden file, and -tv prints symlinks as 'name ->
# target' which defeats $-anchored greps.
probe="$(docker create silo-distroless-test:snapshot server /data)"
docker export "${probe}" -o "${ctx}/rootfs.tar"
docker rm "${probe}" >/dev/null
tar -tf "${ctx}/rootfs.tar" > "${ctx}/names.txt"
tar -tvf "${ctx}/rootfs.tar" > "${ctx}/verbose.txt"
grep -E '^drwxrwxrwx.* data/$' "${ctx}/verbose.txt" >/dev/null
for want in usr/bin/silo licenses/LICENSE licenses/NOTICE licenses/CREDITS; do
grep -Fxq "${want}" "${ctx}/names.txt" || { echo "missing ${want}"; exit 1; }
done
for forbid in bin/sh usr/bin/sh busybox/sh usr/bin/minio usr/bin/mc usr/bin/mcli; do
if grep -Fxq "${forbid}" "${ctx}/names.txt"; then
echo "distroless image unexpectedly contains ${forbid}"
copy_binary() {
local arch="$1"
local pattern="$2"
local src
src="$(find dist -maxdepth 2 -type f -path "dist/${pattern}/minio" | head -n1 || true)"
if [ -z "${src}" ]; then
echo "Missing GoReleaser binary for ${arch} (${pattern})"
exit 1
fi
done
# The baked-in healthcheck must drive Docker's health state on its
# own, the probe binary must be directly exec-able without any
# shell, and SIGTERM must still reach PID 1 (the server binary is
# the entrypoint) for a graceful stop.
assert_distroless() {
name="$1"; shift
docker rm -f "${name}" >/dev/null 2>&1 || true
docker run -d --name "${name}" \
-e MINIO_CI_CD=1 -e MINIO_ROOT_USER=ciadmin -e MINIO_ROOT_PASSWORD=ciadmin-secret-123 \
"$@" silo-distroless-test:snapshot server /data --address :9000 >/dev/null
status=""
for _ in $(seq 1 90); do
status="$(docker inspect -f '{{.State.Health.Status}}' "${name}" 2>/dev/null || echo '?')"
if [ "${status}" = "healthy" ]; then break; fi
if [ "$(docker inspect -f '{{.State.Running}}' "${name}")" != "true" ]; then break; fi
sleep 1
done
if [ "${status}" != "healthy" ]; then
echo "container never became healthy (${name}): status=${status}"
docker logs "${name}" 2>&1 | tail -5
exit 1
fi
docker exec "${name}" /usr/bin/silo healthcheck ready
docker exec "${name}" /usr/bin/silo healthcheck cluster
start="$(date +%s)"; docker stop -t 15 "${name}" >/dev/null; end="$(date +%s)"
code="$(docker inspect -f '{{.State.ExitCode}}' "${name}")"
elapsed=$((end - start))
graceful=0; docker logs "${name}" 2>&1 | grep -q "Exiting on signal" && graceful=1
echo "${name}: health=${status} stop=${elapsed}s exit=${code}"
docker rm -f "${name}" >/dev/null 2>&1 || true
[ "${graceful}" = "1" ] || { echo "no graceful-shutdown log (${name}) - signal not forwarded"; exit 1; }
[ "${code}" = "0" ] || { echo "non-zero exit (${name}): ${code}"; exit 1; }
[ "${elapsed}" -lt 10 ] || { echo "shutdown too slow (${name}): ${elapsed}s - signal not forwarded"; exit 1; }
mkdir -p "dist/linux-${arch}"
cp "${src}" "dist/linux-${arch}/minio.${RELEASE_TAG}"
}
assert_distroless silo-dl-default
assert_distroless silo-dl-rootless --user 1001:1001
- name: Validate release scripts
run: |
set -euo pipefail
bash -n buildscripts/package-release.sh
bash -n buildscripts/minio-upgrade.sh
bash -n buildscripts/sign-release-rpms.sh
bash -n buildscripts/verify-build-provenance.sh
bash -n buildscripts/check-release-state.sh
bash -n buildscripts/check-release-state_test.sh
bash -n buildscripts/verify-rebrand.sh
bash -n buildscripts/verify-helm-migration.sh
sh -n buildscripts/package/postinstall.sh
sh -n buildscripts/package/preremove.sh
bash -n buildscripts/package/lifecycle_test.sh
buildscripts/package/lifecycle_test.sh
bash -n dockerscripts/docker-entrypoint_test.sh
bash -n dockerscripts/build-static-curl.sh
dockerscripts/docker-entrypoint_test.sh
go run ./buildscripts/rebrand-guard
buildscripts/verify-rebrand.sh
test -x buildscripts/package-release.sh
test -x buildscripts/sign-release-rpms.sh
test -x buildscripts/verify-build-provenance.sh
test -x buildscripts/check-release-state.sh
test -x buildscripts/check-release-state_test.sh
test -x buildscripts/verify-rebrand.sh
test -x buildscripts/verify-helm-migration.sh
test -x buildscripts/package/postinstall.sh
test -x buildscripts/package/preremove.sh
test -x buildscripts/package/lifecycle_test.sh
test -x dockerscripts/docker-entrypoint_test.sh
buildscripts/check-release-state_test.sh
copy_binary amd64 "minio_linux_amd64*"
copy_binary arm64 "minio_linux_arm64*"
pkger -r "${RELEASE_TAG}" --appName minio --releaseDir dist --ignore
# Keep only full package files; drop convenience symlinks (minio.rpm/minio.deb/minio.apk)
find dist/linux-* -maxdepth 1 -type l \
\( -name 'minio.rpm' -o -name 'minio.deb' -o -name 'minio.apk' \) -delete
find dist -maxdepth 2 -type f \
\( -name '*.rpm' -o -name '*.deb' -o -name '*.apk' -o -name '*.sha256sum' -o -name 'downloads-minio.json' \) | sort
+15
View File
@@ -0,0 +1,15 @@
---
name: Spelling
on: [pull_request]
jobs:
run:
name: Spell Check with Typos
runs-on: ubuntu-latest
steps:
- name: Checkout Actions Repository
uses: actions/checkout@v4
- name: Check spelling of repo
uses: crate-ci/typos@master
+34
View File
@@ -0,0 +1,34 @@
name: Upgrade old version tests
on:
pull_request:
branches:
- master
# This ensures that previous jobs for the PR are canceled when the PR is
# updated.
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
build:
name: Go ${{ matrix.go-version }} on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
go-version: [1.24.x]
os: [ubuntu-latest]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go-version }}
check-latest: true
- name: Start upgrade tests
run: |
make test-upgrade
+13 -18
View File
@@ -1,36 +1,31 @@
name: VulnCheck
on:
pull_request:
branches:
- main
- master
push:
branches:
- main
workflow_dispatch:
- master
permissions:
contents: read
contents: read # to fetch code (actions/checkout)
jobs:
vulncheck:
name: Analysis
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Check out code
uses: actions/checkout@v7
- name: Check out code into the Go module directory
uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v7
uses: actions/setup-go@v5
with:
go-version-file: go.mod
cache: true
- name: Install govulncheck
run: |
go install golang.org/x/vuln/cmd/govulncheck@v1.8.0
echo "$(go env GOPATH)/bin" >> "${GITHUB_PATH}"
go-version: 1.24.x
cached: false
- name: Get official govulncheck
run: go install golang.org/x/vuln/cmd/govulncheck@latest
shell: bash
- name: Run govulncheck
run: govulncheck -show verbose ./...
shell: bash
+1 -6
View File
@@ -2,7 +2,6 @@
cover.out
*~
minio
silo
!*/
site/
**/*.test
@@ -56,16 +55,12 @@ xattr
xl-meta
.gitignore
.goreleaser.yml
dist/
.claude/
.codex/
AGENTS.md
CLAUDE.md
_bmad/
_bmad-output/
docs/security/
docs/rebranding.md
.release-sign/
+1 -1
View File
@@ -5,7 +5,7 @@ linters:
- durationcheck
- forcetypeassert
- gocritic
- gomodguard_v2
- gomodguard
- govet
- ineffassign
- misspell
-162
View File
@@ -1,162 +0,0 @@
# Changelog
## Unreleased
The entries below describe source changes on main since the latest published Server.
**The latest published Server remains 20260903.** These changes are not in its
binaries, packages or images. See the [component matrix](https://silo.pgsty.com/compatibility/versions/)
and [complete commit range](https://github.com/pgsty/silo/compare/RELEASE.2026-09-03T13-18-01Z...main).
### Authorization and security
- Persist IAM deletion revisions and parent revocation boundaries so stale site
events cannot restore deleted identities, policies or their older grants
(#191, #192). Peer deletion notifications reload committed storage; deliberate
recreation requires a newer revision, and credentials issued before the
parent's revocation remain invalid.
**Coordinated upgrade required:** upgrade every participating node and site.
Mixed old/new nodes sharing an IAM backend and rolling downgrade are
unsupported. Back up complete IAM storage and encryption material; an admin
export of live records omits deletion history. Reissue credentials for
recreated parents and explicitly reconcile pre-upgrade revocations whose
history is already lost. Restoring an older backup can lose later revocations;
keep affected sites isolated until reconciliation/rekeying is complete. See
[the operator runbook](https://github.com/pgsty/silo.pgsty.com/blob/7bd2d57c2ce5aaa804d0b1a2fe0e5eed69d15235/content/operations/replication/iam-upgrade.md).
- Enforce an absolute HTTP/1 request-header deadline through the connection
wrapper (#196). Repeated small reads no longer extend that deadline, and
`--read-header-timeout` / `MINIO_READ_HEADER_TIMEOUT` now reaches the HTTP
server. HTTP/1 request bodies retain the rolling idle timeout; this does not
impose a total upload/download duration. A shorter setting also constrains
TLS handshake reads. The wrapper's strict header mode is not applied to HTTP/2.
- Reject unsigned `x-amz-*` request headers that could turn a signed PUT into a
copy of another object accessible to the signer (SN-2026-011). The latest
public Server is affected; the fix is on main. See [the advisory ledger](docs/security/advisories.md).
- Align signed request fields with policy conditions and enforce header-only
presigned payload checksums. See [the signed-header review](https://silo.pgsty.com/blog/design/signed-header-coverage/).
- **Breaking policy semantics:** separate self-service `admin:ChangeMyPassword`
from `admin:CreateUser`. Built-in read-only policies follow the split. Preserve
both denies if the previous combined restriction must survive upgrades or
rollback. Saved policies are not rewritten. Deploy with the matching Console
and pkg; see [the migration guide](docs/iam/password-permissions.md).
### Object storage and replication
- Preserve object tags during multi-pool metadata reconciliation by reading the
resolved tag field together with its revision (#189). Previously, reconciliation
could replace existing tags with an empty value.
- Preserve the tag revision on SSE-KMS metadata replication (#193), and advance
tag revisions monotonically on local PUT/DELETE tagging (#196). Empty tags
participate in reconciliation as an ordered deletion, preventing older
events from restoring removed tags. SSE-C key rotation also retains the tag
revision. Malformed historical revisions can fail and retry; their missing
history is not reconstructed by the upgrade.
- Complete delete-marker version purges and preserve their identity and retry
state through MRF recovery (#196). Recovery accepts a 405 marker response only
when its version, bucket, object name and modification time match the task.
Purge audit status is normalized from `COMPLETE` to `COMPLETED`.
Thanks to Julien Laurenceau (@julienlau) for the investigation and proposed
fix in #184 that helped shape this follow-up.
- Restore only the six replication-specific metadata fields after ordinary
request metadata extraction (#194). This prevents transport-only `aws-chunked`
from being stored as Content-Encoding while preserving the signed-header
protections. Trusted Snowball entries no longer inherit the outer archive's
ordinary metadata. Thanks to Mikhail Khadarenka (@chodorenko) for the fix in #187.
**Existing data:** these repairs prevent new errors; they do not scan or rewrite
historical object metadata, recover lost tags or prove that old purge work has
converged. Follow the [read-only audit procedure](https://github.com/pgsty/silo.pgsty.com/blob/7bd2d57c2ce5aaa804d0b1a2fe0e5eed69d15235/content/operations/replication/replica-metadata-audit.md)
before planning any repair of stored state.
- Evaluate conditional multipart completion against the logical current object
across all pools while holding the existing object lock. A stale `If-Match`
can no longer replace newer data in another pool, and the current ETag is no
longer rejected because the upload resides next to an older copy. Conditions
are evaluated once; a current delete marker counts as an absent object.
**Availability change:** if any pool's metadata cannot be read, conditional
completion fails even when another pool can still serve GET/HEAD. This also
applies when the unreadable pool may not hold the object: absence cannot be
verified. Retry after the pool recovers. Unconditional completion and the
single-pool path retain their existing behavior.
- Evaluate ordinary multi-pool conditional PUT against the logical current
object across all pools, including draining pools, under the existing object
lock (#207). A stale destination copy no longer accepts a stale ETag or rejects
the current one; a current delete marker is treated as absence.
**Availability change:** if any pool's object metadata cannot be verified,
the condition fails even when GET can use another pool; read-quorum failures
return 503. Restore readability or heal before retrying. Unconditional PUT,
single-pool conditions and internal replication retain their existing behavior.
A public condition with a destination `versionId` compares the current object
while preserving the requested write version. This change does not retire
stale copies in other pools, undo historical accepted overwrites or provide
a new global clock-ordering guarantee. The multipart-completion repair in #190
neither introduced nor repaired this separate PUT defect.
- Reconcile ordinary single-object version DELETE across all pools, including
null versions, delete markers and unqualified directory-marker DELETE. This
applies the deletion to every resolved pool copy under existing quorum
rules. Pending outbound delete replication retains versions until the
existing replication worker completes their purge; a successful response
does not imply immediate physical removal from every drive. Unreadable
pools now consistently return 503 instead of depending on pool traversal
order; insufficient read quorum returns `SlowDownRead`. This extends the
existing failure surface. Retry after recovery.
Cleanup failures also return an error. Batch deletion already fans out across
pools; replication and scanner cleanup keep their existing contracts. See
[scope and limitations](docs/bucket/lifecycle/access-tiering-removal.md#version-deletion-scope).
- Remove the opt-in GET-frequency pool-tiering feature from PR #60, including
its tracker, mover, scanner hooks, configuration, XML actions and metrics.
Accept and ignore retired configuration/XML and preserve ordinary statistics
when reading v9 caches. See [migration notes](docs/bucket/lifecycle/access-tiering-removal.md).
The [decision record](docs/investigations/access-tiering-revert.md) preserves
the feature's introduction, subsequent fixes, rollback scope and review history.
- Preserve the independent multi-pool write, metadata, healing and conditional
deletion fixes from PR #178, including shared remote-tier reference protection.
- Enforce `If-Match` on DELETE, preserve retention and independently ordered
Object Lock/tag updates, and correctly retransmit encrypted replicas.
- Preserve plaintext part sizes and raw SSE-C replicas; prevent SSE-C
compression, honor key-rotation checksums, and complete attributes pagination.
- Repair federated CopyObject checksums, destination timestamps, reserved
metadata, encrypted-object forwarding, legal hold and KMS context.
- Make resync counters, target selection, cancellation and worker lifetimes
reflect actual work, and report bounded MRF drops.
- Converge bucket metadata with deterministic source state, deletion tombstones,
creation time recovery and diagnostics. The mixed-version export gate requires
coordinated upgrades before tombstones are exported. See [the #77 record](docs/investigations/issue-77-current.md).
- Include per-bucket CORS in metadata export/import, close metadata publication
and logger races, and report effective bucket quotas in metrics.
### Console, dependencies and delivery
- Restore embedded Console login over loopback TLS, trusted-proxy handling and
all four WebSocket connection limits. Preserve Go TLS defaults across transports.
- Directly require `github.com/pgsty/silo-pkg/v3` v3.14.0; select Console
`v0.0.0-20260913015128-417559bb2c97` and MC
`v0.0.0-20260913012246-4f609a4da3bb` with explicit PGSTY replacements.
- Pin upstream minio-go `v7.3.1-0.20260910142817-60bd07042d49`; refresh Go x/*
modules and security fixes including bounded AMQP frame handling. Keep Go
1.27.1 and go-systemd v22.6.0's NetBSD compatibility replacement.
- Refresh container base digests and build static curl 8.22.0 from verified
source for both Linux architectures. Pin the actual mcli 20260913 archives and
hashes. Helm's client image follows that release; its Server image still names
the latest published Server 20260903.
The dependency update passed the final candidate's Go, vulnerability and Test
Release workflows; native curl builds passed on both architectures. A local
ARM64 image passed startup, health, S3 transfer and embedded Console checks.
These checks do not publish a Server tag or production image and do not replace
cluster upgrade/rollback acceptance for the next release. Dated investigations
retain the exact source and runtime boundaries they tested.
## RELEASE.2026-09-03T13-18-01Z
Published source: `9b11dc9469e650815b775cb47b039610644f5da4`.
[Complete release notes](https://silo.pgsty.com/blog/release/silo-20260903/) ·
[GitHub release](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-09-03T13-18-01Z)
This release ships Go 1.27.1, silo-pkg v3.13.2, upstream minio-go `0e78d3f18efe`,
mcli 20260903 and embedded Console source `464a59d73ada` (v2.3.0 version identity).
Installing the newer standalone mcli or Console does not replace components
inside this existing Server binary or image.
Earlier releases: [release archive](https://github.com/pgsty/silo/releases).
+1
View File
@@ -0,0 +1 @@
charts.min.io
+3 -13
View File
@@ -1,17 +1,7 @@
# AGPLv3 Compliance
Silo is distributed under the [GNU Affero General Public License v3.0](LICENSE).
It incorporates source code from the MinIO project and preserves the original
copyright, license, and attribution notices in [`NOTICE`](NOTICE),
[`CREDITS`](CREDITS), and source-file headers.
We have designed MinIO as an Open Source software for the Open Source software community. This requires applications to consider whether their usage of MinIO is in compliance with the GNU AGPLv3 [license](https://github.com/minio/minio/blob/master/LICENSE).
You are responsible for determining how the AGPLv3 applies to your use,
modification, deployment, and distribution of Silo and its dependencies. The
Silo maintainers cannot provide legal advice or determine whether a particular
application or service satisfies the license. Consult qualified counsel when
the obligations are material to your deployment.
MinIO cannot make the determination as to whether your application's usage of MinIO is in compliance with the AGPLv3 license requirements. You should instead rely on your own legal counsel or licensing specialists to audit and ensure your application is in compliance with the licenses of MinIO and all other open-source projects with which your application integrates or interacts. We understand that AGPLv3 licensing is complex and nuanced. It is for that reason we strongly encourage using experts in licensing to make any such determinations around compliance instead of relying on apocryphal or anecdotal advice.
If you convey modified binaries or provide network access to a modified
version, review the complete AGPLv3 text and ensure that the corresponding
source and notices are made available as required. Dependency licenses and
separate notices continue to apply independently.
[MinIO Commercial Licensing](https://min.io/pricing) is the best option for applications that trigger AGPLv3 obligations (e.g. open sourcing your application). Applications using MinIO - or any other OSS-licensed code - without validating their usage do so at their own risk.
+30 -105
View File
@@ -1,65 +1,55 @@
# Contributing to Silo
# MinIO Contribution Guide [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io) [![Docker Pulls](https://img.shields.io/docker/pulls/minio/minio.svg?maxAge=604800)](https://hub.docker.com/r/minio/minio/)
Silo welcomes focused contributions that improve security, reliability,
compatibility, packaging, tests, or maintainability. This repository preserves
MinIO-compatible interfaces and storage formats, so changes must identify and
test any compatibility impact.
``MinIO`` community welcomes your contribution. To make the process as seamless as possible, we recommend you read this contribution guide.
## Development Workflow
Fork the current Silo source repository, create a topic branch, and submit a
pull request. Discuss broad or compatibility-sensitive changes in an issue
before implementation.
Start by forking the MinIO GitHub repository, make changes in a branch and then send a pull request. We encourage pull requests to discuss code changes. Here are the steps in details:
### Set up a checkout
### Setup your MinIO GitHub Repository
Fork [MinIO upstream](https://github.com/minio/minio/fork) source repository to your own personal repository. Copy the URL of your MinIO fork (you will need it for the `git clone` command below).
```sh
git clone https://github.com/pgsty/silo
cd silo
go build -o silo .
./silo --version
git clone https://github.com/minio/minio
cd minio
go install -v
ls $(go env GOPATH)/bin/minio
```
### Keep the lineage remote separate
### Set up git remote as ``upstream``
```sh
git remote add lineage https://github.com/minio/minio
git fetch lineage
$ cd minio
$ git remote add upstream https://github.com/minio/minio
$ git fetch upstream
$ git merge upstream/master
...
```
Do not merge an upstream branch into a pull request unless the maintainers have
agreed on the scope. Silo intentionally carries a small downstream delta.
### Create your feature branch
Create a separate branch before making code changes:
Before making code changes, make sure you create a separate branch for these changes
```
git checkout -b my-new-feature
```
### Test Silo server changes
### Test MinIO server changes
Before opening a pull request:
After your code changes, make sure
- Add or update tests for changed behavior.
- Run `make verifiers`.
- If `make rebrand-guard` reports a changed compatibility set, review the
listed identifiers; when the change is intended, refresh the baseline with
`go run ./buildscripts/rebrand-guard --write` and commit
`buildscripts/rebrand-guard/compat-baseline.json`.
- Run the smallest relevant package tests, then `make test` when practical.
- Run `make build` and confirm the generated executable is `silo`.
- Explain any preserved `MINIO_*`, `minio_*`, `x-minio-*`, `/minio/*`,
`.minio.sys`, ARN, module/import-path, or serialized compatibility name.
- To add test cases for the new code. If you have questions about how to do it, please ask on our [Slack](https://slack.min.io) channel.
- To run `make verifiers`
- To squash your commits into a single commit. `git rebase -i`. It's okay to force update your pull request.
- To run `make test` and `make build` completes.
### Commit changes
After verification, commit your changes with a concise message and a DCO
sign-off (see [Licensing of Contributions](#licensing-of-contributions)):
After verification, commit your changes. This is a [great post](https://chris.beams.io/posts/git-commit/) on how to write useful commit messages
```
git commit -s -am 'Fix object replication retry handling'
git commit -am 'Add some feature'
```
### Push to the branch
@@ -72,77 +62,13 @@ git push origin my-new-feature
### Create a Pull Request
Pull requests should include motivation, reproduction steps where applicable,
test evidence, compatibility notes, and documentation impact. Public product
documentation is owned by the separate
[`pgsty/silo.pgsty.com`](https://github.com/pgsty/silo.pgsty.com) repository.
## Licensing of Contributions
Code contributions to PGSTY SILO (`pgsty/silo`) are accepted under the
[GNU AGPL v3.0 or later](LICENSE), the same license as the server. Submit issues
and pull requests to this repository's maintainers. No separate Apache-2.0
license grant to SILO or upstream MinIO maintainers is required.
* **No CLA.** We do not ask you to sign a Contributor License Agreement and we
do not take your copyright. Contributions are accepted inbound=outbound: you
keep the copyright to your changes and license them under the same
AGPL-3.0-or-later as the project itself. The maintainers receive no rights
beyond the project license.
* **DCO sign-off required.** Every commit must carry a
`Signed-off-by: Your Name <you@example.com>` trailer certifying the
[Developer Certificate of Origin 1.1](https://developercertificate.org/) —
your statement that you have the right to submit the code under the project
license. Sign each commit with:
```
git commit -s
```
Forgot some? Repair your branch with `git rebase --signoff` and force-push.
CI rejects pull requests containing unsigned commits; the sign-off email
must match the commit author email. (Lowercase `-s` is the plain-text DCO
sign-off; cryptographic `-S`/GPG signing is welcome but independent.)
* **Provenance.** Only submit code you are entitled to submit. This matters
more here than in most projects: Silo carries a downstream delta over an
upstream code base, and cherry-picks from the lineage remote or other forks
are routine. When relaying a patch written by someone else, preserve original
authorship (`git cherry-pick -x`, keep the author field and any existing
`Signed-off-by` trailers) and add your own sign-off as the person passing it
along. Never import code from a proprietary distribution.
* **File headers.** Preserve existing copyright and license notices in inherited
and third-party files. New original files name their actual copyright holders
and use AGPL-3.0-or-later. Use a header such as the following, then append the
standard AGPL boilerplate:
```
// Copyright (c) 2026 Your Name
```
* **Separately licensed material.** Documentation contributions in `docs/`
follow its existing [CC BY 4.0 license](docs/LICENSE). Third-party components
and earlier Apache-2.0 contributions retain their original licenses and
attribution; this policy does not relicense earlier work.
* **Squash merges** must keep the `Signed-off-by:` trailers in the resulting
commit message.
* **Authorship and tooling.** The human contributor is the author of the commit
and the sole signatory of its DCO sign-off. Attribution trailers for
assistive tooling (for example `Co-Authored-By:` naming an AI assistant) are
informational only: they record which tools were used, and do not create
authorship, co-authorship, or any copyright claim. Whoever signs off remains
responsible for the content of the commit, whatever produced it.
Pull requests can be created via GitHub. Refer to [this document](https://help.github.com/articles/creating-a-pull-request/) for detailed steps on how to create a pull request. After a Pull Request gets peer reviewed and approved, it will be merged.
## FAQs
### How does Silo manage dependencies?
### How does ``MinIO`` manage dependencies?
Silo uses Go modules. Preserve the compatibility module and import paths in
`go.mod`; downstream forks are selected with explicit `replace` directives.
``MinIO`` uses `go mod` to manage its dependencies.
- Run `go get foo/bar` in the source folder to add the dependency to `go.mod` file.
@@ -151,7 +77,6 @@ To remove a dependency
- Edit your code and remove the import reference.
- Run `go mod tidy` in the source folder to remove dependency from `go.mod` file.
### What are the coding guidelines?
### What are the coding guidelines for MinIO?
Follow the existing Go style, run `gofmt` on changed Go files, and keep changes
compact. See the Go project's [code review comments](https://go.dev/wiki/CodeReviewComments).
``MinIO`` is fully conformant with Golang style. Refer: [Effective Go](https://github.com/golang/go/wiki/CodeReviewComments) article from Golang project. If you observe offending code, please feel free to send a pull request or ping us on [Slack](https://slack.min.io).
-184
View File
File diff suppressed because one or more lines are too long
+6116 -4832
View File
File diff suppressed because it is too large Load Diff
+18
View File
@@ -0,0 +1,18 @@
FROM minio/minio:latest
ARG TARGETARCH
ARG RELEASE
RUN chmod -R 777 /usr/bin
COPY ./minio-${TARGETARCH}.${RELEASE} /usr/bin/minio
COPY ./minio-${TARGETARCH}.${RELEASE}.minisig /usr/bin/minio.minisig
COPY ./minio-${TARGETARCH}.${RELEASE}.sha256sum /usr/bin/minio.sha256sum
COPY dockerscripts/docker-entrypoint.sh /usr/bin/docker-entrypoint.sh
ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"]
VOLUME ["/data"]
CMD ["minio"]
+3
View File
@@ -0,0 +1,3 @@
FROM minio/minio:edge
CMD ["minio", "server", "/data"]
-50
View File
@@ -1,50 +0,0 @@
# The distroless variant ships exactly one program: the silo binary.
# No shell, no mc, no curl, no entrypoint script; health checking is
# provided by the binary itself (`silo healthcheck`).
# Design note: https://silo.pgsty.com/compatibility/feature/healthcheck/
# A distroless final stage cannot RUN anything, so /data is prepared in a
# throwaway stage. It ships world-writable (see pgsty/silo#55): Docker
# seeds fresh volumes from the image-layer mountpoint, no entrypoint
# exists to repair ownership at runtime, and 0777 is what keeps every
# privilege mode working, --user included.
FROM busybox:1.37.0 AS prep
RUN mkdir -p /prep/data && chmod 0777 /prep/data
FROM gcr.io/distroless/static-debian12:latest
LABEL org.opencontainers.image.title="Silo" \
org.opencontainers.image.description="S3-Interface Libre Object Storage (distroless)" \
org.opencontainers.image.url="https://silo.pgsty.com" \
org.opencontainers.image.source="https://github.com/pgsty/silo" \
org.opencontainers.image.licenses="AGPL-3.0-or-later" \
maintainer="PGSTY <https://silo.pgsty.com>"
ENV MINIO_ACCESS_KEY_FILE=access_key \
MINIO_SECRET_KEY_FILE=secret_key \
MINIO_ROOT_USER_FILE=access_key \
MINIO_ROOT_PASSWORD_FILE=secret_key \
MINIO_KMS_SECRET_KEY_FILE=kms_master_key \
MINIO_CONFIG_ENV_FILE=config.env \
HOME=/tmp
COPY --chmod=0755 silo /usr/bin/silo
# COPY of a directory copies its contents, not the directory entry, so an
# empty /prep/data would arrive as a default root:0755 /data and non-root
# runs would fail storage init. Copying the parent makes data/ itself a
# copied entry, which --chmod then actually applies to.
COPY --from=prep --chmod=0777 /prep/ /
COPY LICENSE NOTICE CREDITS /licenses/
EXPOSE 9000
VOLUME ["/data"]
# Exec form is mandatory: there is no /bin/sh in this image. `ready`
# rather than `live` because Docker health feeds start-order gating
# (readiness semantics); the two are identical unless KMS/etcd are used.
# The outer timeout stays above the probe's own 5s deadline so the
# probe can report its diagnostic line instead of being SIGKILLed.
HEALTHCHECK --interval=30s --timeout=10s --start-period=2m --start-interval=2s --retries=3 \
CMD ["/usr/bin/silo", "healthcheck", "ready"]
ENTRYPOINT ["/usr/bin/silo"]
+8 -86
View File
@@ -1,112 +1,34 @@
FROM golang:1.27.1-alpine@sha256:cf6fca6641884b8433441b2b0652976f975e1d0fdd26d177eaaf8596087f3125 AS curl-build
ARG TARGETARCH
COPY dockerscripts/build-static-curl.sh /build/build-static-curl
RUN /bin/sh /build/build-static-curl
# Exercise the exact shipped curl without a dynamic loader or shared libraries.
FROM scratch AS curl-runtime
COPY --from=curl-build /go/bin/curl /curl
COPY --from=curl-build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
ENTRYPOINT ["/curl"]
FROM golang:1.27.1-alpine@sha256:cf6fca6641884b8433441b2b0652976f975e1d0fdd26d177eaaf8596087f3125 AS build
ARG TARGETARCH
ENV GOPATH=/go
ENV CGO_ENABLED=0
ARG MC_REPO=pgsty/mc
ARG MC_VERSION=RELEASE.2026-09-13T00-00-00Z
ARG MC_AMD64_SHA256=9d2a92de9c7b887d9b944fe9ddce68d23f1b6df3415092e737594e56593f5e2b
ARG MC_ARM64_SHA256=3d82e9ea6c601c4cb44fe5dd5f2ad1b7d7d64369378110f9ada9c524688a452a
RUN apk add -U --no-cache \
ca-certificates \
bash \
curl \
jq && \
case "${TARGETARCH}" in \
amd64) MC_ARCH=amd64; MC_PINNED_SHA256="${MC_AMD64_SHA256}" ;; \
arm64) MC_ARCH=arm64; MC_PINNED_SHA256="${MC_ARM64_SHA256}" ;; \
*) echo "Unsupported TARGETARCH=${TARGETARCH}"; exit 1 ;; \
esac && \
if [ "${MC_VERSION}" = "latest" ]; then \
MC_RELEASE_URL="https://api.github.com/repos/${MC_REPO}/releases/latest"; \
else \
MC_RELEASE_URL="https://api.github.com/repos/${MC_REPO}/releases/tags/${MC_VERSION}"; \
fi && \
curl -fsSL "${MC_RELEASE_URL}" -o /tmp/mc-release.json && \
MC_ARCHIVE_URL=$(jq -r --arg arch "${MC_ARCH}" \
'.assets[] | select(.name | endswith("_linux_" + $arch + ".tar.gz")) | .browser_download_url' \
/tmp/mc-release.json | head -n 1) && \
MC_CHECKSUM_URL=$(jq -r \
'.assets[] | select(.name | endswith("_checksums.txt")) | .browser_download_url' \
/tmp/mc-release.json | head -n 1) && \
[ -n "${MC_ARCHIVE_URL}" ] || { echo "Cannot find mcli archive for linux/${MC_ARCH}"; exit 1; } && \
[ -n "${MC_CHECKSUM_URL}" ] || { echo "Cannot find mcli checksums file"; exit 1; } && \
ARCHIVE_NAME=$(basename "${MC_ARCHIVE_URL}") && \
echo "Downloading ${ARCHIVE_NAME} ..." && \
curl -fsSL "${MC_ARCHIVE_URL}" -o /tmp/mcli.tar.gz && \
curl -fsSL "${MC_CHECKSUM_URL}" -o /tmp/mcli_checksums.txt && \
EXPECTED=$(grep " ${ARCHIVE_NAME}$" /tmp/mcli_checksums.txt | awk '{print $1}') && \
ACTUAL=$(sha256sum /tmp/mcli.tar.gz | awk '{print $1}') && \
[ -n "${EXPECTED}" ] || { echo "Checksum entry not found for ${ARCHIVE_NAME}"; exit 1; } && \
[ "${EXPECTED}" = "${MC_PINNED_SHA256}" ] || { echo "Published checksum drift for ${ARCHIVE_NAME}"; exit 1; } && \
[ "${MC_PINNED_SHA256}" = "${ACTUAL}" ] || { echo "Checksum mismatch: expected ${MC_PINNED_SHA256}, got ${ACTUAL}"; exit 1; } && \
echo "Checksum OK: ${ACTUAL}" && \
mkdir -p /tmp/mcli-extract && \
tar -xzf /tmp/mcli.tar.gz -C /tmp/mcli-extract/ && \
if [ -f /tmp/mcli-extract/mcli ]; then \
cp /tmp/mcli-extract/mcli /go/bin/mcli; \
elif [ -f /tmp/mcli-extract/mc ]; then \
cp /tmp/mcli-extract/mc /go/bin/mcli; \
else \
echo "No mc or mcli binary found in archive:"; ls -la /tmp/mcli-extract/; exit 1; \
fi && \
chmod +x /go/bin/mcli && \
ln -sf mcli /go/bin/mc
FROM registry.access.redhat.com/ubi9/ubi:latest@sha256:206b65b8ee0f04b992818c9a51b29081b14974630d4850bc358097d0c44ea156 AS certs
FROM registry.access.redhat.com/ubi9/ubi:latest AS certs
RUN dnf -y install ca-certificates && \
update-ca-trust && \
cp /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem /tmp/ca-certificates.crt && \
dnf clean all && \
rm -rf /var/cache/dnf
FROM registry.access.redhat.com/ubi9/ubi-micro:latest@sha256:f332c99eb8f798a8486821c91937f10ad64ee83d7e739303be2df051040918f6
FROM registry.access.redhat.com/ubi9/ubi-micro:latest
LABEL org.opencontainers.image.title="Silo" \
org.opencontainers.image.description="S3-Interface Libre Object Storage" \
org.opencontainers.image.url="https://silo.pgsty.com" \
org.opencontainers.image.source="https://github.com/pgsty/silo" \
org.opencontainers.image.licenses="AGPL-3.0-or-later" \
maintainer="PGSTY <https://silo.pgsty.com>"
LABEL maintainer="pgsty <https://github.com/pgsty/minio>" \
description="MinIO community fork, build by pgsty"
ENV MINIO_ACCESS_KEY_FILE=access_key \
MINIO_SECRET_KEY_FILE=secret_key \
MINIO_ROOT_USER_FILE=access_key \
MINIO_ROOT_PASSWORD_FILE=secret_key \
MINIO_KMS_SECRET_KEY_FILE=kms_master_key \
MINIO_UPDATE_MINISIGN_PUBKEY="RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav" \
MINIO_CONFIG_ENV_FILE=config.env \
HOME=/tmp \
MC_CONFIG_DIR=/tmp/.mc
COPY --from=certs /tmp/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
COPY silo /usr/bin/silo
COPY --from=build /go/bin/mcli /usr/bin/mcli
COPY --from=curl-build /go/bin/curl /usr/bin/curl
COPY --from=curl-build /go/share/curl /licenses/curl
COPY minio /usr/bin/minio
COPY dockerscripts/docker-entrypoint.sh /usr/bin/docker-entrypoint.sh
COPY LICENSE /licenses/LICENSE
COPY NOTICE /licenses/NOTICE
COPY CREDITS /licenses/CREDITS
RUN chmod +x /usr/bin/silo /usr/bin/mcli /usr/bin/docker-entrypoint.sh && \
ln -sf mcli /usr/bin/mc
RUN chmod +x /usr/bin/minio /usr/bin/docker-entrypoint.sh
EXPOSE 9000
VOLUME ["/data"]
ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"]
CMD ["silo"]
CMD ["minio"]
+71
View File
@@ -0,0 +1,71 @@
FROM golang:1.24-alpine as build
ARG TARGETARCH
ARG RELEASE
ENV GOPATH=/go
ENV CGO_ENABLED=0
# Install curl and minisign
RUN apk add -U --no-cache ca-certificates && \
apk add -U --no-cache curl && \
go install aead.dev/minisign/cmd/minisign@v0.2.1
# Download minio binary and signature files
RUN curl -s -q https://dl.min.io/server/minio/hotfixes/linux-${TARGETARCH}/archive/minio.${RELEASE} -o /go/bin/minio && \
curl -s -q https://dl.min.io/server/minio/hotfixes/linux-${TARGETARCH}/archive/minio.${RELEASE}.minisig -o /go/bin/minio.minisig && \
curl -s -q https://dl.min.io/server/minio/hotfixes/linux-${TARGETARCH}/archive/minio.${RELEASE}.sha256sum -o /go/bin/minio.sha256sum && \
chmod +x /go/bin/minio
# Download mc binary and signature files
RUN curl -s -q https://dl.min.io/client/mc/release/linux-${TARGETARCH}/mc -o /go/bin/mc && \
curl -s -q https://dl.min.io/client/mc/release/linux-${TARGETARCH}/mc.minisig -o /go/bin/mc.minisig && \
curl -s -q https://dl.min.io/client/mc/release/linux-${TARGETARCH}/mc.sha256sum -o /go/bin/mc.sha256sum && \
chmod +x /go/bin/mc
RUN if [ "$TARGETARCH" = "amd64" ]; then \
curl -L -s -q https://github.com/moparisthebest/static-curl/releases/latest/download/curl-${TARGETARCH} -o /go/bin/curl; \
chmod +x /go/bin/curl; \
fi
# Verify binary signature using public key "RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGavRUN"
RUN minisign -Vqm /go/bin/minio -x /go/bin/minio.minisig -P RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav && \
minisign -Vqm /go/bin/mc -x /go/bin/mc.minisig -P RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav
FROM registry.access.redhat.com/ubi9/ubi-micro:latest
ARG RELEASE
LABEL name="MinIO" \
vendor="MinIO Inc <dev@min.io>" \
maintainer="MinIO Inc <dev@min.io>" \
version="${RELEASE}" \
release="${RELEASE}" \
summary="MinIO is a High Performance Object Storage, API compatible with Amazon S3 cloud storage service." \
description="MinIO object storage is fundamentally different. Designed for performance and the S3 API, it is 100% open-source. MinIO is ideal for large, private cloud environments with stringent security requirements and delivers mission-critical availability across a diverse range of workloads."
ENV MINIO_ACCESS_KEY_FILE=access_key \
MINIO_SECRET_KEY_FILE=secret_key \
MINIO_ROOT_USER_FILE=access_key \
MINIO_ROOT_PASSWORD_FILE=secret_key \
MINIO_KMS_SECRET_KEY_FILE=kms_master_key \
MINIO_UPDATE_MINISIGN_PUBKEY="RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav" \
MINIO_CONFIG_ENV_FILE=config.env \
MC_CONFIG_DIR=/tmp/.mc
RUN chmod -R 777 /usr/bin
COPY --from=build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
COPY --from=build /go/bin/minio* /usr/bin/
COPY --from=build /go/bin/mc* /usr/bin/
COPY --from=build /go/bin/cur* /usr/bin/
COPY CREDITS /licenses/CREDITS
COPY LICENSE /licenses/LICENSE
COPY dockerscripts/docker-entrypoint.sh /usr/bin/docker-entrypoint.sh
EXPOSE 9000
VOLUME ["/data"]
ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"]
CMD ["minio"]
+73
View File
@@ -0,0 +1,73 @@
FROM golang:1.24-alpine AS build
ARG TARGETARCH
ARG RELEASE
ENV GOPATH=/go
ENV CGO_ENABLED=0
WORKDIR /build
# Install curl and minisign
RUN apk add -U --no-cache ca-certificates && \
apk add -U --no-cache curl && \
apk add -U --no-cache bash && \
go install aead.dev/minisign/cmd/minisign@v0.2.1
# Download minio binary and signature files
RUN curl -s -q https://dl.min.io/server/minio/release/linux-${TARGETARCH}/archive/minio.${RELEASE} -o /go/bin/minio && \
curl -s -q https://dl.min.io/server/minio/release/linux-${TARGETARCH}/archive/minio.${RELEASE}.minisig -o /go/bin/minio.minisig && \
curl -s -q https://dl.min.io/server/minio/release/linux-${TARGETARCH}/archive/minio.${RELEASE}.sha256sum -o /go/bin/minio.sha256sum && \
chmod +x /go/bin/minio
# Download mc binary and signature files
RUN curl -s -q https://dl.min.io/client/mc/release/linux-${TARGETARCH}/mc -o /go/bin/mc && \
curl -s -q https://dl.min.io/client/mc/release/linux-${TARGETARCH}/mc.minisig -o /go/bin/mc.minisig && \
curl -s -q https://dl.min.io/client/mc/release/linux-${TARGETARCH}/mc.sha256sum -o /go/bin/mc.sha256sum && \
chmod +x /go/bin/mc
# Verify binary signature using public key "RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGavRUN"
RUN minisign -Vqm /go/bin/minio -x /go/bin/minio.minisig -P RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav && \
minisign -Vqm /go/bin/mc -x /go/bin/mc.minisig -P RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav
COPY dockerscripts/download-static-curl.sh /build/download-static-curl
RUN chmod +x /build/download-static-curl && \
/build/download-static-curl
FROM registry.access.redhat.com/ubi9/ubi-micro:latest
ARG RELEASE
LABEL name="MinIO" \
vendor="MinIO Inc <dev@min.io>" \
maintainer="MinIO Inc <dev@min.io>" \
version="${RELEASE}" \
release="${RELEASE}" \
summary="MinIO is a High Performance Object Storage, API compatible with Amazon S3 cloud storage service." \
description="MinIO object storage is fundamentally different. Designed for performance and the S3 API, it is 100% open-source. MinIO is ideal for large, private cloud environments with stringent security requirements and delivers mission-critical availability across a diverse range of workloads."
ENV MINIO_ACCESS_KEY_FILE=access_key \
MINIO_SECRET_KEY_FILE=secret_key \
MINIO_ROOT_USER_FILE=access_key \
MINIO_ROOT_PASSWORD_FILE=secret_key \
MINIO_KMS_SECRET_KEY_FILE=kms_master_key \
MINIO_UPDATE_MINISIGN_PUBKEY="RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav" \
MINIO_CONFIG_ENV_FILE=config.env \
MC_CONFIG_DIR=/tmp/.mc
RUN chmod -R 777 /usr/bin
COPY --from=build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
COPY --from=build /go/bin/minio* /usr/bin/
COPY --from=build /go/bin/mc* /usr/bin/
COPY --from=build /go/bin/curl* /usr/bin/
COPY CREDITS /licenses/CREDITS
COPY LICENSE /licenses/LICENSE
COPY dockerscripts/docker-entrypoint.sh /usr/bin/docker-entrypoint.sh
EXPOSE 9000
VOLUME ["/data"]
ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"]
CMD ["minio"]
+71
View File
@@ -0,0 +1,71 @@
FROM golang:1.24-alpine AS build
ARG TARGETARCH
ARG RELEASE
ENV GOPATH=/go
ENV CGO_ENABLED=0
# Install curl and minisign
RUN apk add -U --no-cache ca-certificates && \
apk add -U --no-cache curl && \
go install aead.dev/minisign/cmd/minisign@v0.2.1
# Download minio binary and signature files
RUN curl -s -q https://dl.min.io/server/minio/release/linux-${TARGETARCH}/archive/minio.${RELEASE} -o /go/bin/minio && \
curl -s -q https://dl.min.io/server/minio/release/linux-${TARGETARCH}/archive/minio.${RELEASE}.minisig -o /go/bin/minio.minisig && \
curl -s -q https://dl.min.io/server/minio/release/linux-${TARGETARCH}/archive/minio.${RELEASE}.sha256sum -o /go/bin/minio.sha256sum && \
chmod +x /go/bin/minio
# Download mc binary and signature files
RUN curl -s -q https://dl.min.io/client/mc/release/linux-${TARGETARCH}/mc -o /go/bin/mc && \
curl -s -q https://dl.min.io/client/mc/release/linux-${TARGETARCH}/mc.minisig -o /go/bin/mc.minisig && \
curl -s -q https://dl.min.io/client/mc/release/linux-${TARGETARCH}/mc.sha256sum -o /go/bin/mc.sha256sum && \
chmod +x /go/bin/mc
RUN if [ "$TARGETARCH" = "amd64" ]; then \
curl -L -s -q https://github.com/moparisthebest/static-curl/releases/latest/download/curl-${TARGETARCH} -o /go/bin/curl; \
chmod +x /go/bin/curl; \
fi
# Verify binary signature using public key "RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGavRUN"
RUN minisign -Vqm /go/bin/minio -x /go/bin/minio.minisig -P RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav && \
minisign -Vqm /go/bin/mc -x /go/bin/mc.minisig -P RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav
FROM registry.access.redhat.com/ubi8/ubi-micro:latest
ARG RELEASE
LABEL name="MinIO" \
vendor="MinIO Inc <dev@min.io>" \
maintainer="MinIO Inc <dev@min.io>" \
version="${RELEASE}" \
release="${RELEASE}" \
summary="MinIO is a High Performance Object Storage, API compatible with Amazon S3 cloud storage service." \
description="MinIO object storage is fundamentally different. Designed for performance and the S3 API, it is 100% open-source. MinIO is ideal for large, private cloud environments with stringent security requirements and delivers mission-critical availability across a diverse range of workloads."
ENV MINIO_ACCESS_KEY_FILE=access_key \
MINIO_SECRET_KEY_FILE=secret_key \
MINIO_ROOT_USER_FILE=access_key \
MINIO_ROOT_PASSWORD_FILE=secret_key \
MINIO_KMS_SECRET_KEY_FILE=kms_master_key \
MINIO_UPDATE_MINISIGN_PUBKEY="RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav" \
MINIO_CONFIG_ENV_FILE=config.env \
MC_CONFIG_DIR=/tmp/.mc
RUN chmod -R 777 /usr/bin
COPY --from=build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
COPY --from=build /go/bin/minio* /usr/bin/
COPY --from=build /go/bin/mc* /usr/bin/
COPY --from=build /go/bin/cur* /usr/bin/
COPY CREDITS /licenses/CREDITS
COPY LICENSE /licenses/LICENSE
COPY dockerscripts/docker-entrypoint.sh /usr/bin/docker-entrypoint.sh
EXPOSE 9000
VOLUME ["/data"]
ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"]
CMD ["minio"]
+5
View File
@@ -0,0 +1,5 @@
FROM scratch
COPY minio /minio
CMD ["/minio"]
+69 -84
View File
@@ -4,11 +4,10 @@ LDFLAGS := $(shell go run buildscripts/gen-ldflags.go)
GOOS ?= $(shell go env GOOS)
GOARCH ?= $(shell go env GOARCH)
GOLANGCI_VERSION ?= v2.13.1
VERSION ?= $(shell git describe --tags)
REPO ?= docker.io/pgsty
TAG ?= $(REPO)/silo:$(VERSION)
REPO ?= quay.io/minio
TAG ?= $(REPO)/minio:$(VERSION)
GOLANGCI_DIR = .bin/golangci/$(GOLANGCI_VERSION)
GOLANGCI = $(GOLANGCI_DIR)/golangci-lint
@@ -24,61 +23,35 @@ help: ## print this help
getdeps: ## fetch necessary dependencies
@mkdir -p ${GOPATH}/bin
@if [ ! -x "$(GOLANGCI)" ]; then \
set -e; \
echo "Installing golangci-lint $(GOLANGCI_VERSION)"; \
script=$$(mktemp); \
trap 'rm -f "$$script"' EXIT; \
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/$(GOLANGCI_VERSION)/install.sh -o "$$script"; \
sh "$$script" -b $(GOLANGCI_DIR) $(GOLANGCI_VERSION); \
fi
@echo "Installing golangci-lint" && curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(GOLANGCI_DIR)
crosscompile: ## cross compile Silo
crosscompile: ## cross compile minio
@(env bash $(PWD)/buildscripts/cross-compile.sh)
verifiers: lint check-gen rebrand-guard
rebrand-guard: ## verify Silo branding and protected compatibility identifiers
@go run ./buildscripts/rebrand-guard
@env bash $(PWD)/buildscripts/verify-rebrand.sh
@env bash $(PWD)/dockerscripts/docker-entrypoint_test.sh
credits: ## regenerate CREDITS from the licenses of Go modules linked into the binary
@env bash $(PWD)/buildscripts/gen-credits.sh
verifiers: lint check-gen
check-gen: ## check for updated autogenerated files
@go generate ./... >/dev/null
@go mod tidy -compat=1.27
@env bash $(PWD)/buildscripts/gen-credits.sh
@changed=$$(git diff --name-only -- '*_gen.go' '*_gen_test.go' '*_msgp_test.go' '*_string.go' go.mod go.sum CREDITS); \
if [ -n "$$changed" ]; then \
echo "Non-committed generated changes detected:"; \
echo "$$changed"; \
exit 1; \
fi
@untracked=$$(git ls-files --others --exclude-standard -- '*_gen.go' '*_gen_test.go' '*_msgp_test.go' '*_string.go'); \
if [ -n "$$untracked" ]; then \
echo "Untracked generated files detected:"; \
echo "$$untracked"; \
exit 1; \
fi
@go mod tidy -compat=1.21
@(! git diff --name-only | grep '_gen.go$$') || (echo "Non-committed changes in auto-generated code is detected, please commit them to proceed." && false)
@(! git diff --name-only | grep 'go.sum') || (echo "Non-committed changes in auto-generated go.sum is detected, please commit them to proceed." && false)
lint: getdeps ## runs golangci-lint suite of linters
@echo "Running $@ check"
@$(GOLANGCI) run --build-tags kqueue --timeout=10m --config ./.golangci.yml
@if command -v typos >/dev/null 2>&1; then typos ./; else echo "typos binary is not found.. skipping.."; fi
@command typos && typos ./ || echo "typos binary is not found.. skipping.."
lint-fix: getdeps ## runs golangci-lint suite of linters with automatic fixes
@echo "Running $@ check"
@$(GOLANGCI) run --build-tags kqueue --timeout=10m --config ./.golangci.yml --fix
check: test
test: verifiers build ## builds Silo, runs linters, tests
test: verifiers build ## builds minio, runs linters, tests
@echo "Running unit tests"
@MINIO_API_REQUESTS_MAX=10000 CGO_ENABLED=0 go test -v -tags kqueue,dev ./...
test-root-disable: install-race
@echo "Running Silo root lockdown tests"
@echo "Running minio root lockdown tests"
@env bash $(PWD)/buildscripts/disable-root.sh
test-ilm: install-race
@@ -94,7 +67,7 @@ test-pbac: install-race
@env bash $(PWD)/docs/iam/policies/pbac-tests.sh
test-decom: install-race
@echo "Running Silo decom tests"
@echo "Running minio decom tests"
@env bash $(PWD)/docs/distributed/decom.sh
@env bash $(PWD)/docs/distributed/decom-encrypted.sh
@env bash $(PWD)/docs/distributed/decom-encrypted-sse-s3.sh
@@ -102,17 +75,17 @@ test-decom: install-race
@env bash $(PWD)/docs/distributed/decom-encrypted-kes.sh
test-versioning: install-race
@echo "Running Silo versioning tests"
@echo "Running minio versioning tests"
@env bash $(PWD)/docs/bucket/versioning/versioning-tests.sh
test-configfile: install-race
@env bash $(PWD)/docs/distributed/distributed-from-config-file.sh
test-upgrade:
@echo "Running MinIO-to-Silo upgrade tests"
test-upgrade: install-race
@echo "Running minio upgrade tests"
@(env bash $(PWD)/buildscripts/minio-upgrade.sh)
test-race: verifiers build ## builds Silo, runs linters, tests (race)
test-race: verifiers build ## builds minio, runs linters, tests (race)
@echo "Running unit tests under -race"
@(env bash $(PWD)/buildscripts/race.sh)
@@ -160,9 +133,9 @@ test-site-replication-oidc: install-race ## verify automatic site replication
@echo "Running tests for automatic site replication of IAM (with OIDC)"
@(env bash $(PWD)/docs/site-replication/run-multi-site-oidc.sh)
test-site-replication-silo: install-race ## verify automatic site replication
@echo "Running tests for automatic site replication of IAM (with Silo IDP)"
@(env bash $(PWD)/docs/site-replication/run-multi-site-silo-idp.sh)
test-site-replication-minio: install-race ## verify automatic site replication
@echo "Running tests for automatic site replication of IAM (with MinIO IDP)"
@(env bash $(PWD)/docs/site-replication/run-multi-site-minio-idp.sh)
@echo "Running tests for automatic site replication of SSE-C objects"
@(env bash $(PWD)/docs/site-replication/run-ssec-object-replication.sh)
@echo "Running tests for automatic site replication of SSE-C objects with SSE-KMS enabled for bucket"
@@ -178,11 +151,11 @@ test-timeout: install-race ## test multipart
@echo "Test server timeout"
@(env bash $(PWD)/buildscripts/test-timeout.sh)
verify: install-race ## verify Silo in various setups
verify: install-race ## verify minio various setups
@echo "Verifying build with race"
@(env bash $(PWD)/buildscripts/verify-build.sh)
verify-healing: install-race ## verify healing and replacing disks with the Silo binary
verify-healing: install-race ## verify healing and replacing disks with minio binary
@echo "Verify healing build with race"
@(env bash $(PWD)/buildscripts/verify-healing.sh)
@(env bash $(PWD)/buildscripts/verify-healing-empty-erasure-set.sh)
@@ -203,49 +176,59 @@ verify-healing-inconsistent-versions: install-race ## verify resolving inconsist
build-debugging:
@(env bash $(PWD)/docs/debugging/build.sh)
build: checks build-debugging ## builds Silo to $(PWD)
@echo "Building Silo binary to './silo'"
@CGO_ENABLED=0 GOOS=$(GOOS) GOARCH=$(GOARCH) go build -tags kqueue -trimpath --ldflags "$(LDFLAGS)" -o $(PWD)/silo 1>/dev/null
build: checks build-debugging ## builds minio to $(PWD)
@echo "Building minio binary to './minio'"
@CGO_ENABLED=0 GOOS=$(GOOS) GOARCH=$(GOARCH) go build -tags kqueue -trimpath --ldflags "$(LDFLAGS)" -o $(PWD)/minio 1>/dev/null
docker: checks build-debugging ## builds the local Linux Silo container image
@echo "Building Silo container image '$(TAG)'"
@set -e; \
context=$$(mktemp -d); \
trap 'rm -rf "$$context"' EXIT; \
CGO_ENABLED=0 GOOS=linux GOARCH=$(GOARCH) go build -tags kqueue -trimpath \
--ldflags "$(LDFLAGS)" -o "$$context/silo"; \
mkdir -p "$$context/dockerscripts"; \
cp Dockerfile.goreleaser LICENSE NOTICE CREDITS "$$context/"; \
cp dockerscripts/docker-entrypoint.sh dockerscripts/build-static-curl.sh \
"$$context/dockerscripts/"; \
docker build -q --no-cache --platform linux/$(GOARCH) -t $(TAG) --build-arg TARGETARCH=$(GOARCH) \
-f "$$context/Dockerfile.goreleaser" "$$context"
hotfix-vars:
$(eval LDFLAGS := $(shell MINIO_RELEASE="RELEASE" MINIO_HOTFIX="hotfix.$(shell git rev-parse --short HEAD)" go run buildscripts/gen-ldflags.go $(shell git describe --tags --abbrev=0 | \
sed 's#RELEASE\.\([0-9]\+\)-\([0-9]\+\)-\([0-9]\+\)T\([0-9]\+\)-\([0-9]\+\)-\([0-9]\+\)Z#\1-\2-\3T\4:\5:\6Z#')))
$(eval VERSION := $(shell git describe --tags --abbrev=0).hotfix.$(shell git rev-parse --short HEAD))
docker-distroless: checks build-debugging ## builds the local Linux Silo distroless container image
@echo "Building Silo distroless container image '$(TAG)-distroless'"
@set -e; \
context=$$(mktemp -d); \
trap 'rm -rf "$$context"' EXIT; \
CGO_ENABLED=0 GOOS=linux GOARCH=$(GOARCH) go build -tags kqueue -trimpath \
--ldflags "$(LDFLAGS)" -o "$$context/silo"; \
cp Dockerfile.distroless LICENSE NOTICE CREDITS "$$context/"; \
docker build -q --no-cache --platform linux/$(GOARCH) -t $(TAG)-distroless \
-f "$$context/Dockerfile.distroless" "$$context"
hotfix: hotfix-vars clean install ## builds minio binary with hotfix tags
@wget -q -c https://github.com/minio/pkger/releases/download/v2.3.11/pkger_2.3.11_linux_amd64.deb
@wget -q -c https://raw.githubusercontent.com/minio/minio-service/v1.1.1/linux-systemd/distributed/minio.service
@sudo apt install ./pkger_2.3.11_linux_amd64.deb --yes
@mkdir -p minio-release/$(GOOS)-$(GOARCH)/archive
@cp -af ./minio minio-release/$(GOOS)-$(GOARCH)/minio
@cp -af ./minio minio-release/$(GOOS)-$(GOARCH)/minio.$(VERSION)
@minisign -qQSm minio-release/$(GOOS)-$(GOARCH)/minio.$(VERSION) -s "${CRED_DIR}/minisign.key" < "${CRED_DIR}/minisign-passphrase"
@sha256sum < minio-release/$(GOOS)-$(GOARCH)/minio.$(VERSION) | sed 's, -,minio.$(VERSION),g' > minio-release/$(GOOS)-$(GOARCH)/minio.$(VERSION).sha256sum
@cp -af minio-release/$(GOOS)-$(GOARCH)/minio.$(VERSION)* minio-release/$(GOOS)-$(GOARCH)/archive/
@pkger -r $(VERSION) --ignore
hotfix-push: hotfix
@scp -q -r minio-release/$(GOOS)-$(GOARCH)/* minio@dl-0.minio.io:~/releases/server/minio/hotfixes/linux-$(GOOS)/
@scp -q -r minio-release/$(GOOS)-$(GOARCH)/* minio@dl-0.minio.io:~/releases/server/minio/hotfixes/linux-$(GOOS)/archive
@scp -q -r minio-release/$(GOOS)-$(GOARCH)/* minio@dl-1.minio.io:~/releases/server/minio/hotfixes/linux-$(GOOS)/
@scp -q -r minio-release/$(GOOS)-$(GOARCH)/* minio@dl-1.minio.io:~/releases/server/minio/hotfixes/linux-$(GOOS)/archive
@echo "Published new hotfix binaries at https://dl.min.io/server/minio/hotfixes/linux-$(GOOS)/archive/minio.$(VERSION)"
docker-hotfix-push: docker-hotfix
@docker push -q $(TAG) && echo "Published new container $(TAG)"
docker-hotfix: hotfix-push checks ## builds minio docker container with hotfix tags
@echo "Building minio docker image '$(TAG)'"
@docker build -q --no-cache -t $(TAG) --build-arg RELEASE=$(VERSION) . -f Dockerfile.hotfix
docker: build ## builds minio docker container
@echo "Building minio docker image '$(TAG)'"
@docker build -q --no-cache -t $(TAG) . -f Dockerfile
test-resiliency: build
@echo "Running resiliency tests"
@(DOCKER_COMPOSE_FILE=$(PWD)/docs/resiliency/docker-compose.yaml env bash $(PWD)/docs/resiliency/resiliency-tests.sh)
install-race: checks build-debugging ## builds Silo to $(PWD)
@echo "Building Silo binary with -race to './silo'"
@GORACE=history_size=7 CGO_ENABLED=1 go build -tags kqueue,dev -race -trimpath --ldflags "$(LDFLAGS)" -o $(PWD)/silo 1>/dev/null
@echo "Installing Silo binary with -race to '$(GOPATH)/bin/silo'"
@mkdir -p $(GOPATH)/bin && cp -af $(PWD)/silo $(GOPATH)/bin/silo
install-race: checks build-debugging ## builds minio to $(PWD)
@echo "Building minio binary with -race to './minio'"
@GORACE=history_size=7 CGO_ENABLED=1 go build -tags kqueue,dev -race -trimpath --ldflags "$(LDFLAGS)" -o $(PWD)/minio 1>/dev/null
@echo "Installing minio binary with -race to '$(GOPATH)/bin/minio'"
@mkdir -p $(GOPATH)/bin && cp -af $(PWD)/minio $(GOPATH)/bin/minio
install: build ## builds Silo and installs it to $GOPATH/bin.
@echo "Installing Silo binary to '$(GOPATH)/bin/silo'"
@mkdir -p $(GOPATH)/bin && cp -af $(PWD)/silo $(GOPATH)/bin/silo
@echo "Installation successful. To learn more, try \"silo --help\"."
install: build ## builds minio and installs it to $GOPATH/bin.
@echo "Installing minio binary to '$(GOPATH)/bin/minio'"
@mkdir -p $(GOPATH)/bin && cp -af $(PWD)/minio $(GOPATH)/bin/minio
@echo "Installation successful. To learn more, try \"minio --help\"."
clean: ## cleanup all generated assets
@echo "Cleaning up all the generated files"
@@ -253,8 +236,10 @@ clean: ## cleanup all generated assets
@find . -name '*~' | xargs rm -fv
@find . -name '.#*#' | xargs rm -fv
@find . -name '#*#' | xargs rm -fv
@rm -rvf silo
@rm -rvf minio
@rm -rvf build
@rm -rvf release
@rm -rvf .verify*
@rm -rvf minio-release
@rm -rvf minio.RELEASE*.hotfix.*
@rm -rvf pkger_*.deb
+1 -7
View File
@@ -1,4 +1,4 @@
MinIO Project, (C) 2015-2025 MinIO, Inc.
MinIO Project, (C) 2015-2023 MinIO, Inc.
This product includes software developed at MinIO, Inc.
(https://min.io/).
@@ -7,9 +7,3 @@ The MinIO project contains unmodified/modified subcomponents too with
separate copyright notices and license terms. Your use of the source
code for these subcomponents is subject to the terms and conditions
of GNU Affero General Public License 3.0.
Silo Project modifications, (C) 2025-2026 PGSTY.
Silo is an independent community-maintained project incorporating MinIO
source code. It is not affiliated with or endorsed by MinIO, Inc. Modified
source and Silo release artifacts are maintained by the Silo project.
+12 -12
View File
@@ -1,12 +1,12 @@
# Silo Pull Request Guidelines
# MinIO Pull Request Guidelines
These guidelines ensure high-quality commits in Silo's GitHub repositories, maintaining
a clear, valuable commit history for our open-source projects. They apply to all contributors,
These guidelines ensure high-quality commits in MinIO’s GitHub repositories, maintaining
a clear, valuable commit history for our open-source projects. They apply to all contributors,
fostering efficient reviews and robust code.
## Why Pull Requests?
Pull Requests (PRs) drive quality in Silo's codebase by:
Pull Requests (PRs) drive quality in MinIO’s codebase by:
- Enabling peer review without pair programming.
- Documenting changes for future reference.
- Ensuring commits tell a clear story of development.
@@ -15,12 +15,12 @@ Pull Requests (PRs) drive quality in Silo's codebase by:
## Crafting a Quality PR
A strong Silo PR:
A strong MinIO PR:
- Delivers a complete, valuable change (feature, bug fix, or improvement).
- Has a concise title (e.g., `[S3] Fix bucket policy parsing #1234`) and a summary with context, referencing issues (e.g., `#1234`).
- Contains well-written, logical commits explaining *why* changes were made (e.g., “Add S3 bucket tagging support so that users can organize resources efficiently”).
- Is small, focused, and easy to review—ideally one commit, unless multiple commits better narrate complex work.
- Adheres to Silo's coding standards (e.g., Go style, error handling, testing).
- Adheres to MinIO’s coding standards (e.g., Go style, error handling, testing).
PRs must flow smoothly through review to reach production. Large PRs should be split into smaller, manageable ones.
@@ -48,14 +48,14 @@ PRs must flow smoothly through review to reach production. Large PRs should be s
## Reviewing PRs
Reviewers ensure Silo's commit history remains a clear, reliable record. Responsibilities include:
Reviewers ensure MinIO’s commit history remains a clear, reliable record. Responsibilities include:
1. **Commit Quality**:
- Verify each commit explains *why* the change was made (e.g., “So that…”).
- Request rebasing if commits are unclear, redundant, or lack context (e.g., “Please squash typo fixes into the parent commit”).
2. **Code Quality**:
- Check adherence to Silo's Go standards (e.g., error handling, documentation).
- Check adherence to MinIO’s Go standards (e.g., error handling, documentation).
- Ensure tests cover new code and pass CI.
- Flag bugs or critical issues for immediate fixes; suggest non-blocking improvements as follow-up issues.
@@ -65,7 +65,7 @@ Reviewers ensure Silo's commit history remains a clear, reliable record. Respons
- If unable to complete the review, tag another reviewer (e.g., `@username please take over`).
4. **Shared Responsibility**:
- All Silo contributors are reviewers. The first commenter on a PR owns the review unless they delegate.
- All MinIO contributors are reviewers. The first commenter on a PR owns the review unless they delegate.
- Multiple reviewers are encouraged for complex PRs.
5. **No Self-Edits**:
@@ -80,14 +80,14 @@ Reviewers ensure Silo's commit history remains a clear, reliable record. Respons
- **Small PRs**: Easier to review, faster to merge. Split large changes logically.
- **Clear Commits**: Use `git rebase -i` to refine history before submitting.
- **Engage Early**: Discuss complex changes in a GitHub issue before coding.
- **Engage Early**: Discuss complex changes in issues or Slack (https://slack.min.io) before coding.
- **Be Responsive**: Address reviewer feedback promptly to keep PRs moving.
- **Learn from Reviews**: Use feedback to improve future contributions.
## Resources
- [Silo Contribution Guide](CONTRIBUTING.md)
- [MinIO Coding Standards](https://github.com/minio/minio/blob/master/CONTRIBUTING.md)
- [Effective Commit Messages](https://mislav.net/2014/02/hidden-documentation/)
- [GitHub PR Tips](https://github.com/blog/1943-how-to-write-the-perfect-pull-request)
By following these guidelines, we ensure Silo's codebase remains high-quality, maintainable, and a joy to contribute to. Happy coding!
By following these guidelines, we ensure MinIO’s codebase remains high-quality, maintainable, and a joy to contribute to. Happy coding!
+115 -176
View File
@@ -1,191 +1,130 @@
<h1 align="center">
<a href="https://silo.pgsty.com/">
<img src=".github/silo-logo.svg" alt="Silo" width="160">
</a>
</h1>
<p align="center">
<strong>S3-compatible object storage — a MinIO fork maintained by PGSTY</strong>
</p>
<p align="center">
<a href="https://silo.pgsty.com/">Website</a> ·
<a href="https://silo.pgsty.com/docs/">Documentation</a> ·
<a href="https://silo.pgsty.com/download/">Download</a> ·
<a href="https://silo.pgsty.com/tags/silo/">Release Notes</a> ·
<a href="https://silo.pgsty.com/compatibility/server/">Compatibility</a> ·
<a href="https://silo.pgsty.com/about/manifesto/">Manifesto</a> ·
<a href="SECURITY.md">Security</a> ·
<a href="README_ZH.md">中文</a>
</p>
<p align="center">
<a href="https://silo.pgsty.com/"><img alt="Website" src="https://img.shields.io/badge/Website-silo.pgsty.com-1d588c"></a>
<a href="https://github.com/pgsty/silo/releases"><img alt="GitHub Release" src="https://img.shields.io/github/v/release/pgsty/silo?include_prereleases&label=release&logo=github"></a>
<a href="https://hub.docker.com/r/pgsty/silo"><img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/pgsty/minio?logo=docker"></a>
<a href="go.mod"><img alt="Go Version" src="https://img.shields.io/github/go-mod/go-version/pgsty/silo?logo=go"></a>
<a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/license-AGPLv3-blue"></a>
</p>
> [!IMPORTANT]
> **PGSTY Silo** (hereinafter “Silo”) is an independent, community-maintained fork of the open-source MinIO server, published by [Pigsty](https://pigsty.io) from [`pgsty/silo`](https://github.com/pgsty/silo). It is not affiliated with, endorsed by, or sponsored by MinIO, Inc. “MinIO” is used only to identify the upstream project and compatibility lineage.
> **This is a community-maintained fork of [minio/minio](https://github.com/minio/minio), maintained by [Pigsty](https://pigsty.io).**
> This project is **not** affiliated with, endorsed by, or sponsored by MinIO, Inc.
> "MinIO" is a trademark of MinIO, Inc., used here solely to identify the upstream project.
>
> Changes from upstream are minimal:
> - Restored the embedded management console version reference
> - Updated documentation links and Go module paths to point to this repository
>
> Distributed under the original [GNU AGPLv3](LICENSE) license.
# MinIO Quickstart Guide
[![Website: minio.pigsty.io](https://img.shields.io/badge/website-minio.pigsty.io-slategray?style=flat&logo=cilium&logoColor=white)](https://pigsty.io) [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io) [![license](https://img.shields.io/badge/license-AGPL%20V3-blue)](https://github.com/pgsty/minio/blob/master/LICENSE)
MinIO is a high-performance, S3-compatible object storage solution released under the GNU AGPL v3.0 license.
Designed for speed and scalability, it powers AI/ML, analytics, and data-intensive workloads with industry-leading performance.
- S3 API Compatible – Seamless integration with existing S3 tools
- Built for AI & Analytics – Optimized for large-scale data pipelines
- High Performance – Ideal for demanding storage workloads.
This README provides instructions for building MinIO from source and deploying onto baremetal hardware.
Use the [MinIO Documentation](https://github.com/minio/docs) project to build and host a local copy of the documentation.
## Install from Source
Use the following commands to compile and run a standalone MinIO server from source.
If you do not have a working Golang environment, please follow [How to install Golang](https://golang.org/doc/install). Minimum version required is [go1.24](https://golang.org/dl/#stable)
```sh
go install github.com/pgsty/minio@latest
```
You can alternatively run `go build` and use the `GOOS` and `GOARCH` environment variables to control the OS and architecture target.
For example:
```
env GOOS=linux GOARCH=arm64 go build
```
Start MinIO by running `minio server PATH` where `PATH` is any empty folder on your local filesystem.
The MinIO deployment starts using default root credentials `minioadmin:minioadmin`.
You can test the deployment using the MinIO Console, an embedded web-based object browser built into MinIO Server.
Point a web browser running on the host machine to <http://127.0.0.1:9000> and log in with the root credentials.
You can use the Browser to create buckets, upload objects, and browse the contents of the MinIO server.
You can also connect using any S3-compatible tool, such as the MinIO Client `mc` commandline tool:
```sh
mc alias set local http://localhost:9000 minioadmin minioadmin
mc admin info local
```
See [Test using MinIO Client `mc`](#test-using-minio-client-mc) for more information on using the `mc` commandline tool.
For application developers, see <https://min.io/docs/minio/linux/developers/minio-drivers.html> to view MinIO SDKs for supported languages.
## Build Docker Image
You can use the `docker build .` command to build a Docker image on your local host machine.
You must first [build MinIO](#install-from-source) and ensure the `minio` binary exists in the project root.
The following command builds the Docker image using the default `Dockerfile` in the root project directory with the repository and image tag `myminio:minio`
```sh
docker build -t myminio:minio .
```
Use `docker image ls` to confirm the image exists in your local repository.
You can run the server using standard Docker invocation:
```sh
docker run -p 9000:9000 -p 9001:9001 myminio:minio server /tmp/minio --console-address :9001
```
Complete documentation for building Docker containers, managing custom images, or loading images into orchestration platforms is out of scope for this documentation.
You can modify the `Dockerfile` and `dockerscripts/docker-entrypoint.sh` as-needed to reflect your specific image requirements.
## Install using Helm Charts
There are two paths for installing MinIO onto Kubernetes infrastructure:
- Use the [MinIO Operator](https://github.com/minio/operator)
- Use the community-maintained [Helm charts](https://github.com/pgsty/minio/tree/master/helm/minio)
The Community Helm chart has instructions in the folder-level README.
## Test MinIO Connectivity
### Test using MinIO Console
MinIO Server comes with an embedded web based object browser.
Point your web browser to <http://127.0.0.1:9000> to ensure your server has started successfully.
> [!NOTE]
> Renamed from `pgsty/minio` to `pgsty/silo`, default branch `master` → `main`, on 2026-08-06. Artifacts under the original MinIO identity stay published on the archived [`minio`](https://github.com/pgsty/silo/tree/minio) branch and in releases up to [`RELEASE.2026-08-04T00-00-00Z`](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-08-04T00-00-00Z).
> MinIO runs console on random port by default, if you wish to choose a specific port use `--console-address` to pick a specific interface and port.
## Current release and main branch
### Test using MinIO Client `mc`
The latest published Server is [20260903](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-09-03T13-18-01Z).
As of 2026-09-13, the main branch has newer security, storage, Console and
shared-package changes that have not shipped in a Server release. See
[CHANGELOG.md](CHANGELOG.md) and the [component version matrix](https://silo.pgsty.com/compatibility/versions/)
for the exact release/source boundary, including SN-2026-011 and password-policy migration.
`mc` provides a modern alternative to UNIX commands like ls, cat, cp, mirror, diff etc. It supports filesystems and Amazon S3 compatible cloud storage services.
## Overview
The following commands set a local alias, validate the server information, create a bucket, copy data to that bucket, and list the contents of the bucket.
PGSTY SILO keeps one maintained release line of the open-source MinIO server alive after upstream ended community distribution: builds, packages, multi-arch images, security fixes, and the full web console. Pigsty runs it in production as its PostgreSQL backup repository.
It follows one rule — **the product and its delivery surfaces are renamed; the protocol and your data are not.** Everything else lives on [silo.pgsty.com](https://silo.pgsty.com/).
**Related:** [`pgsty/mc`](https://github.com/pgsty/mc) client (shipped as `mcli`) · [`pgsty/silo-console`](https://github.com/pgsty/silo-console) · [`pgsty/silo-pkg`](https://github.com/pgsty/silo-pkg) · [`pgsty/pigsty`](https://github.com/pgsty/pigsty)
<p align="center">
<img src="https://silo.pgsty.com/images/silo-console/console-metrics-simple.webp" alt="Silo Console">
</p>
## Quick Start
```bash
docker run -d --name silo -p 9000:9000 -p 9001:9001 \
-e MINIO_ROOT_USER=minioadmin \
-e MINIO_ROOT_PASSWORD=change-me-long-password \
-v "$PWD/data:/data" \
docker.io/pgsty/silo:latest server /data --console-address ":9001"
```sh
mc alias set local http://localhost:9000 minioadmin minioadmin
mc admin info
mc mb data
mc cp ~/Downloads/mydata data/
mc ls data/
```
<p align="center">
<img src="https://silo.pgsty.com/images/silo-console/console-login.webp" alt="Silo Console">
</p>
Follow the MinIO Client [Quickstart Guide](https://min.io/docs/minio/linux/reference/minio-mc.html#quickstart) for further instructions.
Console on <http://localhost:9001>, S3 API on <http://localhost:9000>. The image bundles the client as `mcli`:
## Explore Further
```bash
docker exec silo mcli alias set local http://127.0.0.1:9000 minioadmin change-me-long-password
docker exec silo mcli mb local/demo && docker exec silo mcli ls local
```
- [MinIO Erasure Code Overview](https://min.io/docs/minio/linux/operations/concepts/erasure-coding.html)
- [Use `mc` with MinIO Server](https://min.io/docs/minio/linux/reference/minio-mc.html)
- [Use `minio-go` SDK with MinIO Server](https://min.io/docs/minio/linux/developers/go/minio-go.html)
- [MinIO V3 Metrics Reference](docs/metrics/v3.md)
> [!WARNING]
> For production, pin a release, use unique credentials and TLS, monitor the service, keep independent backups, and test recovery. Start from the [documentation](https://silo.pgsty.com/docs/).
## Contribute
## Install
Please follow MinIO [Contributor's Guide](https://github.com/minio/minio/blob/master/CONTRIBUTING.md) for guidance on making new contributions to the repository.
| Method | Where |
| :-- | :-- |
| Container | [`pgsty/silo`](https://hub.docker.com/r/pgsty/silo), multi-arch for `linux/amd64` and `linux/arm64` |
| Binaries | [GitHub Releases](https://github.com/pgsty/silo/releases) — Linux, macOS, Windows on `amd64` and `arm64` |
| Packages | RPM, DEB, and APK, also via the [Pigsty repository](https://pigsty.io/docs/repo/) |
| Kubernetes | Helm chart, see [Download & Install](https://silo.pgsty.com/download/) |
| Source | `go build -o silo . && ./silo --version` |
## License
Every release ships checksums, SPDX SBOMs, Sigstore-signed manifests, and GitHub build attestations. Installation methods and verification commands are documented at [Download & Install](https://silo.pgsty.com/download/); migrating from upstream MinIO — taking over an existing `minio.service` and its `/etc/default/minio`, and keeping data ownership stable with a `/etc/systemd/system/silo.service.d/10-legacy-user.conf` drop-in — is covered by the [migration guide](https://silo.pgsty.com/compatibility/migration/) and the [binary & service notes](https://silo.pgsty.com/compatibility/binary/).
## Compatibility
The S3 API, `MINIO_*` variables, `minio_*` metrics, `x-minio-*` headers, `/minio/*` routes, the `github.com/minio/*` import paths, and the on-disk format (including `.minio.sys`) are preserved and held in place by a CI compatibility check. Only Silo-owned delivery surfaces change: the `silo` executable, package, service, Helm chart, and container image — no `minio` binary alias is installed.
Every divergence from upstream is listed in the code-verified [compatibility audit](https://silo.pgsty.com/compatibility/server/). Treat each release as a downstream upgrade: pin versions, read the [release notes](https://silo.pgsty.com/tags/silo/), and keep a rollback path.
### TLS and Go upgrades
TLS key exchange follows Go's defaults across the S3 listener, node links,
replication, identity providers, etcd, and external HTTP services. If an endpoint
cannot accept ML-KEM, `GODEBUG=tlsmlkem=0` disables the default hybrid exchanges
for the process; certificate verification remains enabled. This option does not
disable ML-DSA signatures or resolve every TLS reset. Prefer updating the
incompatible endpoint before removing the temporary setting.
If only the new SecP hybrids cause problems, `GODEBUG=tlssecpmlkem=0` disables
those groups while retaining X25519MLKEM768.
For builds targeting Go 1.27, setting either `SSL_CERT_FILE` or `SSL_CERT_DIR`
on macOS replaces Keychain trust with on-disk roots and Go's verifier. Stale or
incomplete CA paths can break previously trusted connections; unset inherited
values to restore Keychain trust. Explicit certificates in the configured `CAs`
directory remain additive to the selected root pool.
Go 1.27 binaries require macOS 13 or later. See the
[Go release notes](https://go.dev/doc/go1.27) and the
[SILO stack investigation](docs/investigations/go127-stack.md).
## Security & Contributing
Report vulnerabilities privately as described in [`SECURITY.md`](SECURITY.md); every fix ships with a public [advisory](https://silo.pgsty.com/blog/security/). Contributions are accepted inbound=outbound under AGPL-3.0-or-later with no CLA — only DCO sign-off (`git commit -s`) is required; see [`CONTRIBUTING.md`](CONTRIBUTING.md).
## Contributors
**41 community contributors** build SILO, Console, mcli, shared packages, and related projects. The list includes maintainers and every human Issue or PR author, ordered by merged PRs, other PRs, then issue reports. Gold rings highlight significant contributions.
<p align="center">
<a href="https://github.com/Vonng"><img src="https://silo.pgsty.com/images/contributors/Vonng.svg" width="60" height="60" alt="@Vonng" title="@Vonng — Maintains SILO, Console, mcli, shared packages, releases, and documentation"></a>
<a href="https://github.com/h5vx"><img src="https://silo.pgsty.com/images/contributors/h5vx.svg" width="60" height="60" alt="@h5vx" title="@h5vx — Implemented per-bucket CORS configuration and enforcement"></a>
<a href="https://github.com/mrjavadseydi"><img src="https://silo.pgsty.com/images/contributors/mrjavadseydi.svg" width="60" height="60" alt="@mrjavadseydi" title="@mrjavadseydi — Fixed effective bucket quota metrics; proposed access-frequency ILM"></a>
<a href="https://github.com/Dansyuqri"><img src="https://silo.pgsty.com/images/contributors/Dansyuqri.svg" width="60" height="60" alt="@Dansyuqri" title="@Dansyuqri — Added ChecksumType to multipart completion responses"></a>
<a href="https://github.com/ycjlin"><img src="https://silo.pgsty.com/images/contributors/ycjlin.svg" width="60" height="60" alt="@ycjlin" title="@ycjlin — Fixed missing-bucket ListObjects semantics"></a>
<a href="https://github.com/pinginfo"><img src="https://silo.pgsty.com/images/contributors/pinginfo.svg" width="60" height="60" alt="@pinginfo" title="@pinginfo — Repaired bucket notification streaming"></a>
<a href="https://github.com/ZouhairCharef"><img src="https://silo.pgsty.com/images/contributors/ZouhairCharef.svg" width="60" height="60" alt="@ZouhairCharef" title="@ZouhairCharef — Patched CVE-2026-34986 in go-jose"></a>
<a href="https://github.com/mfredenhagen"><img src="https://silo.pgsty.com/images/contributors/mfredenhagen.svg" width="60" height="60" alt="@mfredenhagen" title="@mfredenhagen — Patched CVE-2026-39883 in OpenTelemetry"></a>
<a href="https://github.com/waterkip"><img src="https://silo.pgsty.com/images/contributors/waterkip.svg" width="60" height="60" alt="@waterkip" title="@waterkip — Repointed documentation links to the SILO portal"></a>
<a href="https://github.com/mikemikimike"><img src="https://silo.pgsty.com/images/contributors/mikemikimike.svg" width="60" height="60" alt="@mikemikimike" title="@mikemikimike — Contributed the replicated SSE-C plaintext part-size fix"></a>
<a href="https://github.com/metaneutrons"><img src="https://silo.pgsty.com/images/contributors/metaneutrons.svg" width="60" height="60" alt="@metaneutrons" title="@metaneutrons — Reported and proposed explicit-version delete authorization"></a>
<a href="https://github.com/magicxor"><img src="https://silo.pgsty.com/images/contributors/magicxor.svg" width="60" height="60" alt="@magicxor" title="@magicxor — Reported and proposed conditional DELETE support for If-Match"></a>
<a href="https://github.com/davinkevin"><img src="https://silo.pgsty.com/images/contributors/davinkevin.svg" width="60" height="60" alt="@davinkevin" title="@davinkevin — Proposed the distroless container image and dependency automation"></a>
<a href="https://github.com/lem21h"><img src="https://silo.pgsty.com/images/contributors/lem21h.svg" width="48" height="48" alt="@lem21h" title="@lem21h — Proposed robustness and goroutine improvements"></a>
<a href="https://github.com/sulin37392"><img src="https://silo.pgsty.com/images/contributors/sulin37392.svg" width="48" height="48" alt="@sulin37392" title="@sulin37392 — Proposed dependency updates"></a>
<a href="https://github.com/cbornet"><img src="https://silo.pgsty.com/images/contributors/cbornet.svg" width="60" height="60" alt="@cbornet" title="@cbornet — Reported multipart and streaming checksum defects and missing-bucket semantics"></a>
<a href="https://github.com/vampywiz17"><img src="https://silo.pgsty.com/images/contributors/vampywiz17.svg" width="60" height="60" alt="@vampywiz17" title="@vampywiz17 — Reported LDAP TLS and Console login regressions"></a>
<a href="https://github.com/orenyomtov"><img src="https://silo.pgsty.com/images/contributors/orenyomtov.svg" width="60" height="60" alt="@orenyomtov" title="@orenyomtov — Reported the unsigned-header CopyObject cross-object read (SN-2026-011)"></a>
<a href="https://github.com/mumu-lab"><img src="https://silo.pgsty.com/images/contributors/mumu-lab.svg" width="48" height="48" alt="@mumu-lab" title="@mumu-lab — Reported bucket quota metrics reading a deprecated field"></a>
<a href="https://github.com/jvasile"><img src="https://silo.pgsty.com/images/contributors/jvasile.svg" width="48" height="48" alt="@jvasile" title="@jvasile — Reported missing user, group, and defaults in Debian packages"></a>
<a href="https://github.com/pmezhuev"><img src="https://silo.pgsty.com/images/contributors/pmezhuev.svg" width="48" height="48" alt="@pmezhuev" title="@pmezhuev — Reported missing RPM package signatures"></a>
<a href="https://github.com/TLINDEN"><img src="https://silo.pgsty.com/images/contributors/TLINDEN.svg" width="48" height="48" alt="@TLINDEN" title="@TLINDEN — Reported the missing client in release tarballs"></a>
<a href="https://github.com/makinikm"><img src="https://silo.pgsty.com/images/contributors/makinikm.svg" width="48" height="48" alt="@makinikm" title="@makinikm — Reported the missing client in the container image"></a>
<a href="https://github.com/meesudzu"><img src="https://silo.pgsty.com/images/contributors/meesudzu.svg" width="48" height="48" alt="@meesudzu" title="@meesudzu — Requested the migration guide from upstream MinIO"></a>
<a href="https://github.com/kuldeep-link11"><img src="https://silo.pgsty.com/images/contributors/kuldeep-link11.svg" width="48" height="48" alt="@kuldeep-link11" title="@kuldeep-link11 — Reported NATS JWT credentials and target reload issues"></a>
<a href="https://github.com/sargarass"><img src="https://silo.pgsty.com/images/contributors/sargarass.svg" width="48" height="48" alt="@sargarass" title="@sargarass — Reported ListMultipartUploads prefix and pagination semantics"></a>
<a href="https://github.com/liuhaodongliu990-cmyk"><img src="https://silo.pgsty.com/images/contributors/liuhaodongliu990-cmyk.svg" width="48" height="48" alt="@liuhaodongliu990-cmyk" title="@liuhaodongliu990-cmyk — Reported indeterminate progress for prefix downloads"></a>
<a href="https://github.com/Xavier-777"><img src="https://silo.pgsty.com/images/contributors/Xavier-777.svg" width="48" height="48" alt="@Xavier-777" title="@Xavier-777 — Reported Console lifecycle management and file preview gaps"></a>
<a href="https://github.com/spaceg00se-r"><img src="https://silo.pgsty.com/images/contributors/spaceg00se-r.svg" width="48" height="48" alt="@spaceg00se-r" title="@spaceg00se-r — Requested cpuv1 support and reported a workflow token failure"></a>
<a href="https://github.com/kh0mka"><img src="https://silo.pgsty.com/images/contributors/kh0mka.svg" width="48" height="48" alt="@kh0mka" title="@kh0mka — Reported inter-node I/O timeouts in ReadFileStreamHandler"></a>
<a href="https://github.com/bagutzu"><img src="https://silo.pgsty.com/images/contributors/bagutzu.svg" width="48" height="48" alt="@bagutzu" title="@bagutzu — Requested KES-compatible external KMS and OpenBao support"></a>
<a href="https://github.com/DestroyLee"><img src="https://silo.pgsty.com/images/contributors/DestroyLee.svg" width="48" height="48" alt="@DestroyLee" title="@DestroyLee — Reported the missing documentation navigation"></a>
<a href="https://github.com/mosesdd"><img src="https://silo.pgsty.com/images/contributors/mosesdd.svg" width="48" height="48" alt="@mosesdd" title="@mosesdd — Requested a maintained Helm chart"></a>
<a href="https://github.com/zylpsrs"><img src="https://silo.pgsty.com/images/contributors/zylpsrs.svg" width="48" height="48" alt="@zylpsrs" title="@zylpsrs — Reported missing Console tiering and site replication"></a>
<a href="https://github.com/heroes1412"><img src="https://silo.pgsty.com/images/contributors/heroes1412.svg" width="48" height="48" alt="@heroes1412" title="@heroes1412 — Reported the unusable profiling option"></a>
<a href="https://github.com/redfoxfox"><img src="https://silo.pgsty.com/images/contributors/redfoxfox.svg" width="48" height="48" alt="@redfoxfox" title="@redfoxfox — Reported Chinese documentation availability"></a>
<a href="https://github.com/jiadzh"><img src="https://silo.pgsty.com/images/contributors/jiadzh.svg" width="48" height="48" alt="@jiadzh" title="@jiadzh — Requested Windows build guidance"></a>
<a href="https://github.com/AntonOfTheWoods"><img src="https://silo.pgsty.com/images/contributors/AntonOfTheWoods.svg" width="48" height="48" alt="@AntonOfTheWoods" title="@AntonOfTheWoods — Asked for clarity on Helm chart and operator options"></a>
<a href="https://github.com/chalukyaj"><img src="https://silo.pgsty.com/images/contributors/chalukyaj.svg" width="48" height="48" alt="@chalukyaj" title="@chalukyaj — Proposed making the SILO Operator easier to discover"></a>
<a href="https://github.com/nsanitate"><img src="https://silo.pgsty.com/images/contributors/nsanitate.svg" width="48" height="48" alt="@nsanitate" title="@nsanitate — Proposed CNCF Sandbox governance"></a>
<a href="https://github.com/Kesavaambati"><img src="https://silo.pgsty.com/images/contributors/Kesavaambati.svg" width="48" height="48" alt="@Kesavaambati" title="@Kesavaambati — Asked about community support and image maintenance"></a>
</p>
[View the full contribution record](CONTRIBUTORS.md) for each person's proposals, fixes, and reports.
## Background
Upstream wound down its community edition: the web console was cut back to a stub, prebuilt community binaries stopped, and the community repository was archived. Silo exists to keep those deployments running. The fork is a means, not an identity — if upstream restores its community edition, we will narrow our scope and offer the fixes back.
The [**Manifesto**](https://silo.pgsty.com/about/manifesto/) is the project's public commitment in eleven articles, under one discipline: every article is either something already done with public evidence, or something explicitly refused. In short:
- **Compatibility contract** — the protocol and your data do not change, and every release documents its tested rollback target and path.
- **The license cannot change** — AGPLv3, no CLA, no copyright aggregation; nobody here, ourselves included, holds enough copyright to relicense on everyone else's behalf.
- **The never list**, append-only — no paywalling existing features, no registration wall on downloads, no telemetry (upstream's phone-home paths are removed outright), no CLA, no license change, no trademark enforcement against normal use.
- **Security and release discipline** — a public advisory for every fix, and a release every one to two months, at most a quarter apart. Judge both against the public record.
Essays: [MinIO Is Dead](https://silo.pgsty.com/blog/post/minio-is-dead/) · [Who Takes Over?](https://silo.pgsty.com/blog/post/minio-alternative/) · [Long Live MinIO](https://silo.pgsty.com/blog/post/minio-resurrect/) · [Promise Kept](https://silo.pgsty.com/blog/post/minio-promise-kept/)
## License & Trademark
Silo is [AGPL-3.0-or-later](LICENSE), derived from [`minio/minio`](https://github.com/minio/minio) with upstream copyright and third-party notices preserved in [`NOTICE`](NOTICE) and [`CREDITS`](CREDITS). MinIO is a trademark of MinIO, Inc.; the name is used here only to identify the upstream project and compatibility lineage.
Details: [license](https://silo.pgsty.com/about/license/) · [attribution](https://silo.pgsty.com/about/attribution/) · [trademark](https://silo.pgsty.com/about/trademark/)
- MinIO source is licensed under the [GNU AGPLv3](LICENSE).
- MinIO [documentation](docs/) is licensed under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/).
- [License Compliance](COMPLIANCE.md)
-170
View File
@@ -1,170 +0,0 @@
<h1 align="center">
<a href="https://silo.pgsty.com/zh/">
<img src=".github/silo-logo.svg" alt="Silo" width="160">
</a>
</h1>
<p align="center">
<strong>S3 兼容对象存储 —— 由 PGSTY 维护的 MinIO 社区分支</strong>
</p>
<p align="center">
<a href="https://silo.pgsty.com/zh/">官网</a> ·
<a href="https://silo.pgsty.com/zh/docs/">文档</a> ·
<a href="https://silo.pgsty.com/zh/download/">下载</a> ·
<a href="https://silo.pgsty.com/zh/tags/silo/">版本说明</a> ·
<a href="https://silo.pgsty.com/zh/compatibility/server/">兼容性</a> ·
<a href="https://silo.pgsty.com/zh/about/manifesto/">宣言</a> ·
<a href="SECURITY.md">安全策略</a> ·
<a href="README.md">English</a>
</p>
<p align="center">
<a href="https://silo.pgsty.com/zh/"><img alt="官网" src="https://img.shields.io/badge/%E5%AE%98%E7%BD%91-silo.pgsty.com%2Fzh-1d588c"></a>
<a href="https://github.com/pgsty/silo/releases"><img alt="GitHub Release" src="https://img.shields.io/github/v/release/pgsty/silo?include_prereleases&label=release&logo=github"></a>
<a href="https://hub.docker.com/r/pgsty/silo"><img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/pgsty/minio?logo=docker"></a>
<a href="go.mod"><img alt="Go Version" src="https://img.shields.io/github/go-mod/go-version/pgsty/silo?logo=go"></a>
<a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/license-AGPLv3-blue"></a>
</p>
> [!IMPORTANT]
> **PGSTY Silo**(以下简称 Silo)是由 [Pigsty](https://pigsty.cc) 独立维护、从 [`pgsty/silo`](https://github.com/pgsty/silo) 发布的开源 MinIO 社区分支。本项目与 MinIO, Inc. 不存在隶属、背书或赞助关系;文中使用 “MinIO” 仅用于说明上游项目及兼容谱系。
> [!NOTE]
> 2026-08-06,本仓库由 `pgsty/minio` 更名为 `pgsty/silo`,默认分支由 `master` 更名为 `main`。以原 MinIO 形态维持的归档构件仍位于归档的 [`minio`](https://github.com/pgsty/silo/tree/minio) 分支,以及截止 [`RELEASE.2026-08-04T00-00-00Z`](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-08-04T00-00-00Z) 的历次发布中。
## 当前发行版与主分支
最新已发布的 Server 仍为 [20260903](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-09-03T13-18-01Z)。
截至 2026-09-13,主分支已合入更新的安全、存储、Console 与共享包改动,但尚未发布新 Server。
准确的已发布/源码边界见 [CHANGELOG.md](CHANGELOG.md) 与[组件版本矩阵](https://silo.pgsty.com/zh/compatibility/versions/),
其中包括 SN-2026-011 修复状态与密码权限迁移要求。
## 概述
上游停止社区发行后,Silo 为开源 MinIO 服务端维护一条持续可用的版本线:构建、软件包、多架构镜像、安全修复与完整 Web 控制台。Pigsty 在生产环境中用它承载 PostgreSQL 备份存储。
它只遵循一条原则:**改名的是产品与交付物,不是协议与你的数据。** 其余内容都在 [silo.pgsty.com](https://silo.pgsty.com/zh/)。
**相关项目:**[`pgsty/mc`](https://github.com/pgsty/mc) 客户端(以 `mcli` 发行) · [`pgsty/silo-console`](https://github.com/pgsty/silo-console) · [`pgsty/silo-pkg`](https://github.com/pgsty/silo-pkg) · [`pgsty/pigsty`](https://github.com/pgsty/pigsty)
<p align="center">
<img src="https://silo.pgsty.com/images/silo-console/console-metrics-simple.webp" alt="Silo 控制台">
</p>
## 快速上手
```bash
docker run -d --name silo -p 9000:9000 -p 9001:9001 \
-e MINIO_ROOT_USER=minioadmin \
-e MINIO_ROOT_PASSWORD=change-me-long-password \
-v "$PWD/data:/data" \
docker.io/pgsty/silo:latest server /data --console-address ":9001"
```
<p align="center">
<img src="https://silo.pgsty.com/images/silo-console/console-login.webp" alt="Silo 控制台">
</p>
控制台位于 <http://localhost:9001>,S3 API 位于 <http://localhost:9000>。镜像内置客户端 `mcli`:
```bash
docker exec silo mcli alias set local http://127.0.0.1:9000 minioadmin change-me-long-password
docker exec silo mcli mb local/demo && docker exec silo mcli ls local
```
> [!WARNING]
> 生产环境应锁定版本,使用独立凭据与 TLS,配置监控,保留独立备份,并验证恢复流程。请从[文档](https://silo.pgsty.com/zh/docs/)开始。
## 安装
| 方式 | 位置 |
| :-- | :-- |
| 容器镜像 | [`pgsty/silo`](https://hub.docker.com/r/pgsty/silo),支持 `linux/amd64` 与 `linux/arm64` |
| 二进制 | [GitHub Releases](https://github.com/pgsty/silo/releases),覆盖 Linux、macOS、Windows 的 `amd64` 与 `arm64` |
| 软件包 | RPM、DEB、APK,也可通过 [Pigsty 软件仓库](https://pigsty.cc/docs/repo/) 安装 |
| Kubernetes | Helm Chart,参见[下载与安装](https://silo.pgsty.com/zh/download/) |
| 源码构建 | `go build -o silo . && ./silo --version` |
每个版本都附带校验和、SPDX SBOM、Sigstore 签名清单与 GitHub 构建证明。完整安装方式与验证命令见[下载与安装](https://silo.pgsty.com/zh/download/);从上游 MinIO 迁移 —— 接管既有 `minio.service` 与 `/etc/default/minio`,并用 `/etc/systemd/system/silo.service.d/10-legacy-user.conf` drop-in 保持数据属主不变 —— 见[迁移指南](https://silo.pgsty.com/zh/compatibility/migration/)与[二进制与服务说明](https://silo.pgsty.com/zh/compatibility/binary/)。
## 兼容性
S3 API、`MINIO_*` 环境变量、`minio_*` 指标、`x-minio-*` 头、`/minio/*` 路由、`github.com/minio/*` 导入路径与磁盘格式(含 `.minio.sys`)原样保留,并由 CI 兼容性门禁冻结。只有 Silo 自有交付面改名:`silo` 可执行文件、软件包、服务、Helm Chart 与容器镜像 —— 原生交付物不会安装 `minio` 二进制别名。
与上游的全部分歧,以逐项核验代码的[兼容性审计](https://silo.pgsty.com/zh/compatibility/server/)形式维护。每个版本仍应视为下游升级:锁定版本,阅读[版本说明](https://silo.pgsty.com/zh/tags/silo/),并保留回滚路径。
## 安全与贡献
请按照 [`SECURITY.md`](SECURITY.md) 私密报告漏洞;每项修复都会发布公开[安全公告](https://silo.pgsty.com/zh/blog/security/)。本项目不要求签署 CLA:贡献按 AGPL-3.0-or-later(inbound=outbound)接收,只需 DCO 签署(`git commit -s`),详见 [`CONTRIBUTING.md`](CONTRIBUTING.md)。
## 贡献者
**41 位社区贡献者**共同建设 SILO、Console、mcli、公共包与相关项目。名单包含维护者,以及所有提出 Issue 或 PR 的真人作者;按已合并 PR、其他 PR、Issue 报告排序,黄圈标记显著贡献。
<p align="center">
<a href="https://github.com/Vonng"><img src="https://silo.pgsty.com/images/contributors/Vonng.svg" width="60" height="60" alt="@Vonng" title="@Vonng — 维护 SILO、Console、mcli、公共包、发行与文档"></a>
<a href="https://github.com/h5vx"><img src="https://silo.pgsty.com/images/contributors/h5vx.svg" width="60" height="60" alt="@h5vx" title="@h5vx — 实现单桶 CORS 配置与请求执行"></a>
<a href="https://github.com/mrjavadseydi"><img src="https://silo.pgsty.com/images/contributors/mrjavadseydi.svg" width="60" height="60" alt="@mrjavadseydi" title="@mrjavadseydi — 修复有效桶配额指标,并提交按访问频率分层的 ILM 方案"></a>
<a href="https://github.com/Dansyuqri"><img src="https://silo.pgsty.com/images/contributors/Dansyuqri.svg" width="60" height="60" alt="@Dansyuqri" title="@Dansyuqri — 为分片上传完成响应补充 ChecksumType"></a>
<a href="https://github.com/ycjlin"><img src="https://silo.pgsty.com/images/contributors/ycjlin.svg" width="60" height="60" alt="@ycjlin" title="@ycjlin — 修复缺失桶的 ListObjects 语义"></a>
<a href="https://github.com/pinginfo"><img src="https://silo.pgsty.com/images/contributors/pinginfo.svg" width="60" height="60" alt="@pinginfo" title="@pinginfo — 修复桶通知的流式输出"></a>
<a href="https://github.com/ZouhairCharef"><img src="https://silo.pgsty.com/images/contributors/ZouhairCharef.svg" width="60" height="60" alt="@ZouhairCharef" title="@ZouhairCharef — 修复 go-jose 中的 CVE-2026-34986"></a>
<a href="https://github.com/mfredenhagen"><img src="https://silo.pgsty.com/images/contributors/mfredenhagen.svg" width="60" height="60" alt="@mfredenhagen" title="@mfredenhagen — 修复 OpenTelemetry 中的 CVE-2026-39883"></a>
<a href="https://github.com/waterkip"><img src="https://silo.pgsty.com/images/contributors/waterkip.svg" width="60" height="60" alt="@waterkip" title="@waterkip — 将文档链接指向 SILO 门户"></a>
<a href="https://github.com/mikemikimike"><img src="https://silo.pgsty.com/images/contributors/mikemikimike.svg" width="60" height="60" alt="@mikemikimike" title="@mikemikimike — 提交 SSE-C 复制分片明文尺寸修复"></a>
<a href="https://github.com/metaneutrons"><img src="https://silo.pgsty.com/images/contributors/metaneutrons.svg" width="60" height="60" alt="@metaneutrons" title="@metaneutrons — 报告并提交显式版本删除鉴权方案"></a>
<a href="https://github.com/magicxor"><img src="https://silo.pgsty.com/images/contributors/magicxor.svg" width="60" height="60" alt="@magicxor" title="@magicxor — 报告并提交 DELETE If-Match 条件请求支持方案"></a>
<a href="https://github.com/davinkevin"><img src="https://silo.pgsty.com/images/contributors/davinkevin.svg" width="60" height="60" alt="@davinkevin" title="@davinkevin — 提交 distroless 容器镜像与依赖自动更新方案"></a>
<a href="https://github.com/lem21h"><img src="https://silo.pgsty.com/images/contributors/lem21h.svg" width="48" height="48" alt="@lem21h" title="@lem21h — 提交健壮性与 goroutine 改进"></a>
<a href="https://github.com/sulin37392"><img src="https://silo.pgsty.com/images/contributors/sulin37392.svg" width="48" height="48" alt="@sulin37392" title="@sulin37392 — 提交依赖更新"></a>
<a href="https://github.com/cbornet"><img src="https://silo.pgsty.com/images/contributors/cbornet.svg" width="60" height="60" alt="@cbornet" title="@cbornet — 报告分片与流式校验和缺陷及缺失桶语义问题"></a>
<a href="https://github.com/vampywiz17"><img src="https://silo.pgsty.com/images/contributors/vampywiz17.svg" width="60" height="60" alt="@vampywiz17" title="@vampywiz17 — 报告 LDAP TLS 与 Console 登录回归"></a>
<a href="https://github.com/orenyomtov"><img src="https://silo.pgsty.com/images/contributors/orenyomtov.svg" width="60" height="60" alt="@orenyomtov" title="@orenyomtov — 报告未签名头导致的 CopyObject 跨对象读取(SN-2026-011)"></a>
<a href="https://github.com/mumu-lab"><img src="https://silo.pgsty.com/images/contributors/mumu-lab.svg" width="48" height="48" alt="@mumu-lab" title="@mumu-lab — 报告桶配额指标读取已弃用字段的问题"></a>
<a href="https://github.com/jvasile"><img src="https://silo.pgsty.com/images/contributors/jvasile.svg" width="48" height="48" alt="@jvasile" title="@jvasile — 报告 Debian 包缺少用户、用户组与默认配置"></a>
<a href="https://github.com/pmezhuev"><img src="https://silo.pgsty.com/images/contributors/pmezhuev.svg" width="48" height="48" alt="@pmezhuev" title="@pmezhuev — 报告 RPM 包缺少 GPG 签名"></a>
<a href="https://github.com/TLINDEN"><img src="https://silo.pgsty.com/images/contributors/TLINDEN.svg" width="48" height="48" alt="@TLINDEN" title="@TLINDEN — 报告发布压缩包缺少客户端"></a>
<a href="https://github.com/makinikm"><img src="https://silo.pgsty.com/images/contributors/makinikm.svg" width="48" height="48" alt="@makinikm" title="@makinikm — 报告容器镜像缺少客户端"></a>
<a href="https://github.com/meesudzu"><img src="https://silo.pgsty.com/images/contributors/meesudzu.svg" width="48" height="48" alt="@meesudzu" title="@meesudzu — 提出从上游 MinIO 迁移的指南需求"></a>
<a href="https://github.com/kuldeep-link11"><img src="https://silo.pgsty.com/images/contributors/kuldeep-link11.svg" width="48" height="48" alt="@kuldeep-link11" title="@kuldeep-link11 — 报告 NATS JWT 凭据与通知目标重载问题"></a>
<a href="https://github.com/sargarass"><img src="https://silo.pgsty.com/images/contributors/sargarass.svg" width="48" height="48" alt="@sargarass" title="@sargarass — 报告 ListMultipartUploads 前缀与分页语义问题"></a>
<a href="https://github.com/liuhaodongliu990-cmyk"><img src="https://silo.pgsty.com/images/contributors/liuhaodongliu990-cmyk.svg" width="48" height="48" alt="@liuhaodongliu990-cmyk" title="@liuhaodongliu990-cmyk — 报告前缀下载进度显示异常"></a>
<a href="https://github.com/Xavier-777"><img src="https://silo.pgsty.com/images/contributors/Xavier-777.svg" width="48" height="48" alt="@Xavier-777" title="@Xavier-777 — 报告 Console 生命周期管理与文件预览缺失"></a>
<a href="https://github.com/spaceg00se-r"><img src="https://silo.pgsty.com/images/contributors/spaceg00se-r.svg" width="48" height="48" alt="@spaceg00se-r" title="@spaceg00se-r — 提出 cpuv1 支持需求并报告工作流令牌错误"></a>
<a href="https://github.com/kh0mka"><img src="https://silo.pgsty.com/images/contributors/kh0mka.svg" width="48" height="48" alt="@kh0mka" title="@kh0mka — 报告 ReadFileStreamHandler 节点间 I/O 超时"></a>
<a href="https://github.com/bagutzu"><img src="https://silo.pgsty.com/images/contributors/bagutzu.svg" width="48" height="48" alt="@bagutzu" title="@bagutzu — 提出兼容 KES 的外部 KMS 与 OpenBao 支持需求"></a>
<a href="https://github.com/DestroyLee"><img src="https://silo.pgsty.com/images/contributors/DestroyLee.svg" width="48" height="48" alt="@DestroyLee" title="@DestroyLee — 报告文档目录导航缺失"></a>
<a href="https://github.com/mosesdd"><img src="https://silo.pgsty.com/images/contributors/mosesdd.svg" width="48" height="48" alt="@mosesdd" title="@mosesdd — 提出维护 Helm Chart 的需求"></a>
<a href="https://github.com/zylpsrs"><img src="https://silo.pgsty.com/images/contributors/zylpsrs.svg" width="48" height="48" alt="@zylpsrs" title="@zylpsrs — 报告 Console 缺少分层与站点复制"></a>
<a href="https://github.com/heroes1412"><img src="https://silo.pgsty.com/images/contributors/heroes1412.svg" width="48" height="48" alt="@heroes1412" title="@heroes1412 — 报告性能分析选项不可用"></a>
<a href="https://github.com/redfoxfox"><img src="https://silo.pgsty.com/images/contributors/redfoxfox.svg" width="48" height="48" alt="@redfoxfox" title="@redfoxfox — 报告中文文档站点不可用"></a>
<a href="https://github.com/jiadzh"><img src="https://silo.pgsty.com/images/contributors/jiadzh.svg" width="48" height="48" alt="@jiadzh" title="@jiadzh — 提出 Windows 构建指导需求"></a>
<a href="https://github.com/AntonOfTheWoods"><img src="https://silo.pgsty.com/images/contributors/AntonOfTheWoods.svg" width="48" height="48" alt="@AntonOfTheWoods" title="@AntonOfTheWoods — 提出明确 Helm Chart 与 Operator 选项的需求"></a>
<a href="https://github.com/chalukyaj"><img src="https://silo.pgsty.com/images/contributors/chalukyaj.svg" width="48" height="48" alt="@chalukyaj" title="@chalukyaj — 提出改善 SILO Operator 可发现性的建议"></a>
<a href="https://github.com/nsanitate"><img src="https://silo.pgsty.com/images/contributors/nsanitate.svg" width="48" height="48" alt="@nsanitate" title="@nsanitate — 提出加入 CNCF Sandbox 的治理建议"></a>
<a href="https://github.com/Kesavaambati"><img src="https://silo.pgsty.com/images/contributors/Kesavaambati.svg" width="48" height="48" alt="@Kesavaambati" title="@Kesavaambati — 提出社区支持与容器镜像维护问题"></a>
</p>
[查看完整贡献记录](CONTRIBUTORS.md),了解每位贡献者的提案、修复与问题报告。
## 背景
本项目因上游收缩社区版而生:Web 控制台被削减为残桩、社区预编译制品停发、社区仓库被归档。Silo 的存在就是让这些部署继续跑下去。Fork 是手段,不是身份 —— 若上游恢复社区版承诺,我们乐意收缩范围,并把修复回馈上游。
[**宣言**](https://silo.pgsty.com/zh/about/manifesto/)是项目的公开承诺,共十一条,通篇遵循一项纪律:**每一条,要么是已经在做且有公开证据的事实,要么是刻意拒绝的承诺。** 摘要:
- **兼容性合同** —— 协议与数据不改,每个版本都标注经过测试的回滚目标与路径。
- **许可证无法变更** —— AGPLv3、无 CLA、不做版权聚合;包括我们自己在内,没有人握有足够版权代表所有贡献者重新授权。
- **永不清单**(只增不减)—— 永不将既有功能移入付费墙、永不给下载设注册墙、永不加入遥测(上游回连路径已整体移除)、永不引入 CLA、永不变更许可证、永不以商标追究正常使用。
- **安全与发布纪律** —— 每项安全修复配一篇公开公告;通常每一到两个月发布一版,最长不超过一个季度。请拿公开记录检验这两条。
延伸阅读:[MinIO已死](https://silo.pgsty.com/zh/blog/post/minio-is-dead/) · [谁能接盘?](https://silo.pgsty.com/zh/blog/post/minio-alternative/) · [MinIO 复生](https://silo.pgsty.com/zh/blog/post/minio-resurrect/) · [承诺兑现](https://silo.pgsty.com/zh/blog/post/minio-promise-kept/)
## 许可证与商标
Silo 采用 [AGPL-3.0-or-later](LICENSE),衍生自 [`minio/minio`](https://github.com/minio/minio),上游版权与第三方声明完整保留于 [`NOTICE`](NOTICE) 与 [`CREDITS`](CREDITS)。MinIO 是 MinIO, Inc. 的商标,此处使用仅为标识上游项目与兼容谱系。
详见:[许可证](https://silo.pgsty.com/zh/about/license/) · [署名归属](https://silo.pgsty.com/zh/about/attribution/) · [商标声明](https://silo.pgsty.com/zh/about/trademark/)
+32 -35
View File
@@ -1,45 +1,42 @@
# Security Policy
Silo is an independent, community-maintained object-storage server derived from
the open-source MinIO server. Upstream MinIO security contacts do not handle
Silo-specific fixes or release notes.
## Supported Versions
Security fixes are tracked on the active development branch and summarized in
[docs/security/advisories.md](docs/security/advisories.md). Only the current
Silo release line is supported unless an advisory says otherwise.
## Inherited Fix Evidence
The canonical ledger also records security fixes inherited from upstream when
they are part of the Silo release baseline. Source and fork commits are linked
separately even when the fork preserves the original commit object and SHA.
- [CVE-2025-62506](https://github.com/advisories/GHSA-jjjj-jwhf-8rgr):
upstream [PR #21642](https://github.com/minio/minio/pull/21642) merged as
[`minio/minio@c1a49490`](https://github.com/minio/minio/commit/c1a49490c78e9c3ebcad86ba0662319138ace190),
inherited unchanged as
[`pgsty/silo@c1a49490`](https://github.com/pgsty/silo/commit/c1a49490c78e9c3ebcad86ba0662319138ace190),
and is present in every Silo community release beginning with
[`RELEASE.2025-12-03T12-00-00Z`](https://github.com/pgsty/silo/releases/tag/RELEASE.2025-12-03T12-00-00Z).
The inherited [service-account](https://github.com/pgsty/silo/blob/c1a49490c78e9c3ebcad86ba0662319138ace190/cmd/admin-handlers-users_test.go#L211-L212)
and [STS](https://github.com/pgsty/silo/blob/c1a49490c78e9c3ebcad86ba0662319138ace190/cmd/sts-handlers_test.go#L45-L46)
regression groups remain part of `go test ./cmd`; see the
[canonical ledger](docs/security/advisories.md#inherited-upstream-advisory-baseline)
for the operator-facing record.
We always provide security updates for the [latest release](https://github.com/minio/minio/releases/latest).
Whenever there is a security update you just need to upgrade to the latest version.
## Reporting a Vulnerability
For vulnerabilities in this fork:
All security bugs in [minio/minio](https://github,com/minio/minio) (or other minio/* repositories)
should be reported by email to security@min.io. Your email will be acknowledged within 48 hours,
and you'll receive a more detailed response to your email within 72 hours indicating the next steps
in handling your report.
1. Follow the fork-specific expectations in [VULNERABILITY_REPORT.md](VULNERABILITY_REPORT.md).
2. Prefer this repository's [private GitHub security advisory](https://github.com/pgsty/silo/security/advisories/new) workflow.
3. If private reporting is unavailable, contact the maintainers through the
repository without publishing exploit details until a private channel is
established.
4. If you confirm the issue also affects upstream `minio/minio`, report it upstream separately.
Please, provide a detailed explanation of the issue. In particular, outline the type of the security
issue (DoS, authentication bypass, information disclose, ...) and the assumptions you're making (e.g. do
you need access credentials for a successful exploit).
## Disclosure Process
If you have not received a reply to your email within 48 hours or you have not heard from the security team
for the past five days please contact the security team directly:
Fork-specific fixes and user-visible upgrade notes are published in [docs/security/advisories.md](docs/security/advisories.md). The fork-specific triage and remediation process is described in [VULNERABILITY_REPORT.md](VULNERABILITY_REPORT.md).
- Primary security coordinator: aead@min.io
- Secondary coordinator: harsha@min.io
- If you receive no response: dev@min.io
### Disclosure Process
MinIO uses the following disclosure process:
1. Once the security report is received one member of the security team tries to verify and reproduce
the issue and determines the impact it has.
2. A member of the security team will respond and either confirm or reject the security report.
If the report is rejected the response explains why.
3. Code is audited to find any potential similar problems.
4. Fixes are prepared for the latest release.
5. On the date that the fixes are applied a security advisory will be published on <https://blog.min.io>.
Please inform us in your report email whether MinIO should mention your contribution w.r.t. fixing
the security issue. By default MinIO will **not** publish this information to protect your privacy.
This process can take some time, especially when coordination is required with maintainers of other projects.
Every effort will be made to handle the bug in as timely a manner as possible, however it's important that we
follow the process described above to ensure that disclosures are handled consistently.
+20 -19
View File
@@ -1,37 +1,38 @@
# Vulnerability Management Policy
This document describes how the Silo maintainers investigate,
assess, and remediate reported vulnerabilities affecting this fork, any
directly shipped component, or a direct / indirect dependency used by this
repository.
This document formally describes the process of addressing and managing a
reported vulnerability that has been found in the MinIO server code base,
any directly connected ecosystem component or a direct / indirect dependency
of the code base.
## Scope
This policy covers vulnerability reports opened by repository maintainers or
external third parties against Silo itself, its release artifacts, or
dependencies that materially affect this fork.
The vulnerability management policy described in this document covers the
process of investigating, assessing and resolving a vulnerability report
opened by a MinIO employee or an external third party.
It defines the information needed for triage and the expected remediation
workflow for supported fixes.
Therefore, it lists pre-conditions and actions that should be performed to
resolve and fix a reported vulnerability.
## Vulnerability Management Process
A useful vulnerability report should contain the following information:
The vulnerability management process requires that the vulnerability report
contains the following information:
- The project / component that contains the reported vulnerability.
- A description of the vulnerability. In particular, the type of the
reported vulnerability and how it might be exploited. Alternatively,
a well-established vulnerability identifier, such as a CVE or GHSA ID, can
be used instead.
reported vulnerability and how it might be exploited. Alternatively,
a well-established vulnerability identifier, e.g. CVE number, can be
used instead.
Based on the report, the Silo maintainers investigate:
Based on the description mentioned above, a MinIO engineer or security team
member investigates:
- Whether the reported vulnerability exists.
- The conditions that are required such that the vulnerability can be exploited.
- Which releases, branches, or deployment paths are affected.
- The steps required to fix the vulnerability.
If the vulnerability exists in this fork itself, the maintainers will, when
feasible, fix the issue or implement reasonable countermeasures such that the
vulnerability can no longer be exploited. Fork-specific upgrade notes and
security advisories are published in `docs/security/advisories.md`.
In general, if the vulnerability exists in one of the MinIO code bases
itself - not in a code dependency - then MinIO will, if possible, fix
the vulnerability or implement reasonable countermeasures such that the
vulnerability cannot be exploited anymore.
+1
View File
@@ -0,0 +1 @@
theme: jekyll-theme-minimal
-76
View File
@@ -1,76 +0,0 @@
#!/usr/bin/env bash
# Fail closed before a release job can replace published or finalized assets.
# An ordinary Draft is retry state; a finalized Draft contains GPG-derived
# materials and must never be replaced by the build lane.
set -euo pipefail
release_tag="${1:-}"
fixture="${2:-}"
repository="${GITHUB_REPOSITORY:-pgsty/silo}"
require_draft="${REQUIRE_DRAFT:-false}"
if ! command -v jq >/dev/null 2>&1; then
echo "jq is required to inspect GitHub release state" >&2
exit 1
fi
if [[ ! "${release_tag}" =~ ^RELEASE\.[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}-[0-9]{2}-[0-9]{2}Z$ ]]; then
echo "Invalid release tag format: ${release_tag:-<empty>}" >&2
exit 1
fi
if [ -n "${fixture}" ]; then
release_json="$(<"${fixture}")"
else
error_file="$(mktemp)"
trap 'rm -f "${error_file}"' EXIT
if ! release_json="$(
gh api --paginate "repos/${repository}/releases?per_page=100" --jq '.[]' 2>"${error_file}" |
jq --arg tag "${release_tag}" -s '[.[] | select(.tag_name == $tag)]'
)"; then
cat "${error_file}" >&2
exit 1
fi
fi
if ! jq -e 'type == "array" and all(.[]; type == "object" and (.tag_name | type == "string") and (.draft | type == "boolean"))' \
<<<"${release_json}" >/dev/null 2>&1; then
echo "Invalid release state response for ${release_tag}" >&2
exit 1
fi
if ! jq -e --arg tag "${release_tag}" 'all(.[]; .tag_name == $tag)' \
<<<"${release_json}" >/dev/null 2>&1; then
echo "Release state returned a tag other than ${release_tag}" >&2
exit 1
fi
release_count="$(jq 'length' <<<"${release_json}")"
if [ "${release_count}" -eq 0 ]; then
if [ "${require_draft}" = "true" ]; then
echo "Expected one Draft release for ${release_tag}, found none" >&2
exit 1
fi
echo "No existing release for ${release_tag}."
exit 0
fi
if [ "${release_count}" -ne 1 ]; then
echo "Refusing to choose among ${release_count} releases for ${release_tag}; clean duplicate Drafts first" >&2
exit 1
fi
if [ "$(jq -r '.[0].draft' <<<"${release_json}")" != "true" ]; then
echo "Refusing to overwrite published release ${release_tag}" >&2
exit 1
fi
finalize_markers="$(jq '[.[0].assets[]? | select(.name | endswith("_packages_provenance.sigstore.json"))] | length' <<<"${release_json}")"
if [ "${finalize_markers}" -ne 0 ]; then
echo "Refusing to replace finalized Draft ${release_tag}" >&2
exit 1
fi
echo "Existing unfinalized Draft ${release_tag} will be replaced from scratch."
-68
View File
@@ -1,68 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
checker="${script_dir}/check-release-state.sh"
tag="RELEASE.2026-08-29T00-00-00Z"
fixture="$(mktemp)"
stdout_file="$(mktemp)"
stderr_file="$(mktemp)"
trap 'rm -f "${fixture}" "${stdout_file}" "${stderr_file}"' EXIT
expect_success() {
if ! "${checker}" "$@" >"${stdout_file}" 2>"${stderr_file}"; then
cat "${stderr_file}" >&2
return 1
fi
}
expect_failure() {
if "${checker}" "$@" >"${stdout_file}" 2>"${stderr_file}"; then
echo "Expected release-state check to fail: $*" >&2
return 1
fi
}
printf '[]\n' >"${fixture}"
expect_success "${tag}" "${fixture}"
grep -qF "No existing release for ${tag}." "${stdout_file}"
if REQUIRE_DRAFT=true "${checker}" "${tag}" "${fixture}" >"${stdout_file}" 2>"${stderr_file}"; then
echo "Expected required-Draft check to fail when no release exists" >&2
exit 1
fi
grep -qF "Expected one Draft release for ${tag}, found none" "${stderr_file}"
printf '[{"tag_name":"%s","draft":true,"assets":[]}]\n' "${tag}" >"${fixture}"
expect_success "${tag}" "${fixture}"
grep -qF "Existing unfinalized Draft ${tag} will be replaced from scratch." "${stdout_file}"
if ! REQUIRE_DRAFT=true "${checker}" "${tag}" "${fixture}" >"${stdout_file}" 2>"${stderr_file}"; then
cat "${stderr_file}" >&2
exit 1
fi
printf '[{"tag_name":"%s","draft":true,"assets":[{"name":"silo_20260829000000.0.0_packages_provenance.sigstore.json"}]}]\n' "${tag}" >"${fixture}"
expect_failure "${tag}" "${fixture}"
grep -qF "Refusing to replace finalized Draft ${tag}" "${stderr_file}"
printf '[{"tag_name":"%s","draft":false}]\n' "${tag}" >"${fixture}"
expect_failure "${tag}" "${fixture}"
grep -qF "Refusing to overwrite published release ${tag}" "${stderr_file}"
printf '[{"tag_name":"%s","draft":true},{"tag_name":"%s","draft":true}]\n' "${tag}" "${tag}" >"${fixture}"
expect_failure "${tag}" "${fixture}"
grep -qF "Refusing to choose among 2 releases" "${stderr_file}"
printf '[{"tag_name":"RELEASE.2026-08-28T00-00-00Z","draft":true}]\n' >"${fixture}"
expect_failure "${tag}" "${fixture}"
grep -qF "other than ${tag}" "${stderr_file}"
printf '{not-json}\n' >"${fixture}"
expect_failure "${tag}" "${fixture}"
grep -qF "Invalid release state response for ${tag}" "${stderr_file}"
expect_failure "not-a-release-tag" "${fixture}"
grep -qF "Invalid release tag format" "${stderr_file}"
echo "release-state decision tests passed"
+1 -1
View File
@@ -7,7 +7,7 @@ _init() {
## Minimum required versions for build dependencies
GIT_VERSION="1.0"
GO_VERSION="1.27.1"
GO_VERSION="1.16"
OSX_VERSION="10.8"
KNAME=$(uname -s)
ARCH=$(uname -m)
+2 -5
View File
@@ -8,11 +8,8 @@ function _init() {
## All binaries are static make sure to disable CGO.
export CGO_ENABLED=0
## Cross-compile only the OS/arch combinations we actually publish, kept in
## sync with the goos/goarch matrix in .github/goreleaser.yml. Compile-checking
## targets we never ship (ppc64le, s390x, mips*, riscv64, 386, arm, the BSDs)
## spent CI minutes on unshipped code and timed the gate out on a cold cache.
SUPPORTED_OSARCH="linux/amd64 linux/arm64 darwin/amd64 darwin/arm64 windows/amd64 windows/arm64"
## List of architectures and OS to test coss compilation.
SUPPORTED_OSARCH="linux/ppc64le linux/mips64 linux/amd64 linux/arm64 linux/s390x darwin/arm64 darwin/amd64 freebsd/amd64 windows/amd64 linux/arm linux/386 netbsd/amd64 linux/mips openbsd/amd64 linux/riscv64"
}
function _build() {
+23 -22
View File
@@ -3,7 +3,7 @@
set -x
export MINIO_CI_CD=1
killall -9 silo
killall -9 minio
rm -rf ${HOME}/tmp/dist
@@ -19,27 +19,28 @@ done
echo $args
for ((i = 0; i < $((nr_servers)); i++)); do
(silo server --address ":$((9100 + i))" $args 2>&1 >/tmp/log$i.txt) &
(minio server --address ":$((9100 + i))" $args 2>&1 >/tmp/log$i.txt) &
done
sleep 10s
if [ ! -f ./mc ]; then
"$(git rev-parse --show-toplevel)/buildscripts/install-mcli.sh" ./mc
wget --quiet -O ./mc https://dl.minio.io/client/mc/release/linux-amd64/./mc &&
chmod +x mc
fi
set +e
export MC_HOST_siloadm=http://minioadmin:minioadmin@localhost:9100/
./mc ready siloadm
export MC_HOST_minioadm=http://minioadmin:minioadmin@localhost:9100/
./mc ready minioadm
./mc ls siloadm/
./mc ls minioadm/
./mc admin config set siloadm/ api root_access=off
./mc admin config set minioadm/ api root_access=off
sleep 3s # let things settle a little
./mc ls siloadm/
./mc ls minioadm/
if [ $? -eq 0 ]; then
echo "listing succeeded, 'minioadmin' was not disabled"
exit 1
@@ -47,38 +48,38 @@ fi
set -e
killall -9 silo
killall -9 minio
export MINIO_API_ROOT_ACCESS=on
for ((i = 0; i < $((nr_servers)); i++)); do
(silo server --address ":$((9100 + i))" $args 2>&1 >/tmp/log$i.txt) &
(minio server --address ":$((9100 + i))" $args 2>&1 >/tmp/log$i.txt) &
done
set +e
./mc ready siloadm/
./mc ready minioadm/
./mc ls siloadm/
./mc ls minioadm/
if [ $? -ne 0 ]; then
echo "listing failed, 'minioadmin' should be enabled"
exit 1
fi
killall -9 silo
killall -9 minio
rm -rf /tmp/multisitea/
rm -rf /tmp/multisiteb/
echo "Setup site-replication and then disable root credentials"
silo server --address 127.0.0.1:9001 "http://127.0.0.1:9001/tmp/multisitea/data/disterasure/xl{1...4}" \
minio server --address 127.0.0.1:9001 "http://127.0.0.1:9001/tmp/multisitea/data/disterasure/xl{1...4}" \
"http://127.0.0.1:9002/tmp/multisitea/data/disterasure/xl{5...8}" >/tmp/sitea_1.log 2>&1 &
silo server --address 127.0.0.1:9002 "http://127.0.0.1:9001/tmp/multisitea/data/disterasure/xl{1...4}" \
minio server --address 127.0.0.1:9002 "http://127.0.0.1:9001/tmp/multisitea/data/disterasure/xl{1...4}" \
"http://127.0.0.1:9002/tmp/multisitea/data/disterasure/xl{5...8}" >/tmp/sitea_2.log 2>&1 &
silo server --address 127.0.0.1:9003 "http://127.0.0.1:9003/tmp/multisiteb/data/disterasure/xl{1...4}" \
minio server --address 127.0.0.1:9003 "http://127.0.0.1:9003/tmp/multisiteb/data/disterasure/xl{1...4}" \
"http://127.0.0.1:9004/tmp/multisiteb/data/disterasure/xl{5...8}" >/tmp/siteb_1.log 2>&1 &
silo server --address 127.0.0.1:9004 "http://127.0.0.1:9003/tmp/multisiteb/data/disterasure/xl{1...4}" \
minio server --address 127.0.0.1:9004 "http://127.0.0.1:9003/tmp/multisiteb/data/disterasure/xl{1...4}" \
"http://127.0.0.1:9004/tmp/multisiteb/data/disterasure/xl{5...8}" >/tmp/siteb_2.log 2>&1 &
export MC_HOST_sitea=http://minioadmin:minioadmin@127.0.0.1:9001
@@ -95,19 +96,19 @@ export MC_HOST_siteb=http://minioadmin:minioadmin@127.0.0.1:9004
./mc admin user info siteb foobar
killall -9 silo
killall -9 minio
echo "turning off root access, however site replication must continue"
export MINIO_API_ROOT_ACCESS=off
silo server --address 127.0.0.1:9001 "http://127.0.0.1:9001/tmp/multisitea/data/disterasure/xl{1...4}" \
minio server --address 127.0.0.1:9001 "http://127.0.0.1:9001/tmp/multisitea/data/disterasure/xl{1...4}" \
"http://127.0.0.1:9002/tmp/multisitea/data/disterasure/xl{5...8}" >/tmp/sitea_1.log 2>&1 &
silo server --address 127.0.0.1:9002 "http://127.0.0.1:9001/tmp/multisitea/data/disterasure/xl{1...4}" \
minio server --address 127.0.0.1:9002 "http://127.0.0.1:9001/tmp/multisitea/data/disterasure/xl{1...4}" \
"http://127.0.0.1:9002/tmp/multisitea/data/disterasure/xl{5...8}" >/tmp/sitea_2.log 2>&1 &
silo server --address 127.0.0.1:9003 "http://127.0.0.1:9003/tmp/multisiteb/data/disterasure/xl{1...4}" \
minio server --address 127.0.0.1:9003 "http://127.0.0.1:9003/tmp/multisiteb/data/disterasure/xl{1...4}" \
"http://127.0.0.1:9004/tmp/multisiteb/data/disterasure/xl{5...8}" >/tmp/siteb_1.log 2>&1 &
silo server --address 127.0.0.1:9004 "http://127.0.0.1:9003/tmp/multisiteb/data/disterasure/xl{1...4}" \
minio server --address 127.0.0.1:9004 "http://127.0.0.1:9003/tmp/multisiteb/data/disterasure/xl{1...4}" \
"http://127.0.0.1:9004/tmp/multisiteb/data/disterasure/xl{5...8}" >/tmp/siteb_2.log 2>&1 &
export MC_HOST_sitea=http://foobar:foo12345@127.0.0.1:9001
-125
View File
@@ -1,125 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
# Regenerates CREDITS from the license text of every Go module linked into the
# silo binary. The module set is what `go list -deps` reports for the main
# package, so test-only and tool dependencies stay out: CREDITS documents what
# a shipped binary actually contains. check-gen runs this and fails on a diff,
# which keeps CREDITS from drifting when go.mod changes.
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
repo_dir="$(cd "${script_dir}/.." && pwd)"
cd "${repo_dir}"
out_file="${1:-${repo_dir}/CREDITS}"
tmp_file="${out_file}.tmp"
trap 'rm -f "${tmp_file}"' EXIT
rule_dash='----------------------------------------------------------------'
rule_equal='================================================================'
# These modules repackage Go standard library code and publish no license
# file; their source files carry the Go Authors' BSD-style header pointing at
# the Go project license, so that text is reproduced for them.
stdlib_derived='github.com/minio/colorjson github.com/minio/csvparser github.com/minio/filepath'
is_stdlib_derived() {
case " ${stdlib_derived} " in
*" $1 "*) return 0 ;;
*) return 1 ;;
esac
}
# Command substitution strips trailing newlines; printf adds exactly one back,
# so every entry ends the same way regardless of how the license file ends.
emit_text() {
printf '%s\n' "$(cat "$1")"
}
# The module cache must hold every dependency before .Dir can resolve.
go mod download
# The Go project license text is taken from the pinned golang.org/x/sys module
# rather than GOROOT: Homebrew's Go does not ship GOROOT/LICENSE, and the
# module copy is version-locked by go.mod, so the output cannot vary with the
# machine's toolchain packaging.
go_license="$(go list -m -f '{{.Dir}}' golang.org/x/sys)/LICENSE"
if [ ! -f "${go_license}" ]; then
echo "Missing Go license text: ${go_license}" >&2
exit 1
fi
{
printf '%s\n' \
'Silo bundles third-party software under the licenses reproduced below.' \
'Generated by buildscripts/gen-credits.sh (make credits) from the Go' \
'modules linked into the silo binary. Do not edit by hand.' \
'' \
"${rule_equal}" \
''
printf '%s\n%s\n%s\n' 'Go (the standard library)' 'https://golang.org/' "${rule_dash}"
emit_text "${go_license}"
printf '\n%s\n\n' "${rule_equal}"
# The dependency closure is GOOS/GOARCH-specific: platform-only modules
# (darwin's go-m1cpu, windows' wmi, ...) enter and leave it with the host.
# Pin the primary shipped target and the release build tags so regenerating
# CREDITS produces identical output on every machine, including CI.
GOOS=linux GOARCH=amd64 go list -deps -tags kqueue \
-f '{{if and (not .Standard) .Module}}{{.Module.Path}}{{end}}' . \
| LC_ALL=C sort -u \
| grep -vx 'github.com/minio/minio' \
| xargs go list -m -f '{{.Path}}|{{with .Replace}}{{.Path}}{{end}}|{{.Dir}}' \
| while IFS='|' read -r path replacement dir; do
if [ -z "${dir}" ] || [ ! -d "${dir}" ]; then
echo "Module cache directory missing for ${path}" >&2
exit 1
fi
name="${path}"
url="https://${path}"
# A same-path replace only pins a version; the annotation is for
# dependencies actually served from a different repository.
if [ -n "${replacement}" ] && [ "${replacement}" != "${path}" ]; then
name="${path} (replaced by ${replacement})"
url="https://${replacement}"
fi
printf '%s\n%s\n%s\n' "${name}" "${url}" "${rule_dash}"
if is_stdlib_derived "${path}"; then
printf '%s\n%s\n\n' \
'This module repackages Go standard library code and publishes no' \
'license file; the Go project license below applies per its file headers.'
emit_text "${go_license}"
else
license_file=''
for candidate in LICENSE LICENSE.txt LICENSE.md COPYING COPYING.txt LICENCE UNLICENSE; do
if [ -f "${dir}/${candidate}" ]; then
license_file="${dir}/${candidate}"
break
fi
done
if [ -z "${license_file}" ]; then
echo "No license file found for ${path} in ${dir}" >&2
exit 1
fi
emit_text "${license_file}"
fi
# Apache License 2.0 section 4(d) requires redistributing the NOTICE
# file contents alongside the licensed work.
for notice in NOTICE NOTICE.txt; do
if [ -f "${dir}/${notice}" ]; then
printf '\n%s\n\n' 'Bundled NOTICE file:'
emit_text "${dir}/${notice}"
break
fi
done
printf '\n%s\n\n' "${rule_equal}"
done
} > "${tmp_file}"
mv "${tmp_file}" "${out_file}"
+2 -6
View File
@@ -38,12 +38,8 @@ func genLDFlags(version string) string {
ldflagsStr += " -X github.com/minio/minio/cmd.ReleaseTag=" + releaseTag
ldflagsStr += " -X github.com/minio/minio/cmd.CommitID=" + commitID()
ldflagsStr += " -X github.com/minio/minio/cmd.ShortCommitID=" + commitID()[:12]
// GOPATH/GOROOT are deliberately not stamped in. They only seed the logger's
// source-path trim list, which -trimpath already makes moot (paths are
// relative in the binary, so there is no build-machine prefix left to trim),
// and stamping them baked the builder's absolute paths into the released
// binary - defeating -trimpath and reproducible builds. cmd.GOPATH/GOROOT
// keep their empty defaults, exactly as a plain `go build` leaves them.
ldflagsStr += " -X github.com/minio/minio/cmd.GOPATH=" + os.Getenv("GOPATH")
ldflagsStr += " -X github.com/minio/minio/cmd.GOROOT=" + os.Getenv("GOROOT")
return ldflagsStr
}
+30 -19
View File
@@ -5,34 +5,45 @@ set -o pipefail
set -x
WORK_DIR="$PWD/.verify-$RANDOM"
SILO_CONFIG_DIR="$WORK_DIR/.silo"
SILO=("$PWD/silo" --config-dir "$SILO_CONFIG_DIR" server)
MINIO_CONFIG_DIR="$WORK_DIR/.minio"
MINIO=("$PWD/minio" --config-dir "$MINIO_CONFIG_DIR" server)
if [ ! -x "$PWD/silo" ]; then
echo "Silo executable binary not found in current directory"
if [ ! -x "$PWD/minio" ]; then
echo "minio executable binary not found in current directory"
exit 1
fi
if [ ! -x "$PWD/silo" ]; then
echo "Silo executable binary not found in current directory"
if [ ! -x "$PWD/minio" ]; then
echo "minio executable binary not found in current directory"
exit 1
fi
function start_silo_4drive() {
function start_minio_4drive() {
start_port=$1
export MINIO_ROOT_USER=silo
export MINIO_ROOT_PASSWORD=silo1234
export MC_HOST_silo="http://silo:silo1234@127.0.0.1:${start_port}/"
export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=minio123
export MC_HOST_minio="http://minio:minio123@127.0.0.1:${start_port}/"
unset MINIO_KMS_AUTO_ENCRYPTION # do not auto-encrypt objects
export MINIO_CI_CD=1
mkdir ${WORK_DIR}
C_PWD=${PWD}
if [ ! -x "$PWD/mc" ]; then
"$(git rev-parse --show-toplevel)/buildscripts/install-mcli.sh" "$PWD/mc"
MC_BUILD_DIR="mc-$RANDOM"
if ! git clone --quiet https://github.com/minio/mc "$MC_BUILD_DIR"; then
echo "failed to download https://github.com/minio/mc"
purge "${MC_BUILD_DIR}"
exit 1
fi
(cd "${MC_BUILD_DIR}" && go build -o "$C_PWD/mc")
# remove mc source.
purge "${MC_BUILD_DIR}"
fi
"${SILO[@]}" --address ":$start_port" "${WORK_DIR}/disk{1...4}" >"${WORK_DIR}/server1.log" 2>&1 &
"${MINIO[@]}" --address ":$start_port" "${WORK_DIR}/disk{1...4}" >"${WORK_DIR}/server1.log" 2>&1 &
pid=$!
disown $pid
sleep 5
@@ -45,30 +56,30 @@ function start_silo_4drive() {
exit 1
fi
"${PWD}/mc" mb --with-versioning silo/bucket
"${PWD}/mc" mb --with-versioning minio/bucket
for i in $(seq 1 4); do
"${PWD}/mc" cp /etc/hosts silo/bucket/testobj
"${PWD}/mc" cp /etc/hosts minio/bucket/testobj
sudo chown -R root. "${WORK_DIR}/disk${i}"
"${PWD}/mc" cp /etc/hosts silo/bucket/testobj
"${PWD}/mc" cp /etc/hosts minio/bucket/testobj
sudo chown -R ${USER}. "${WORK_DIR}/disk${i}"
done
for vid in $("${PWD}/mc" ls --json --versions silo/bucket/testobj | jq -r .versionId); do
"${PWD}/mc" cat --vid "${vid}" silo/bucket/testobj | md5sum
for vid in $("${PWD}/mc" ls --json --versions minio/bucket/testobj | jq -r .versionId); do
"${PWD}/mc" cat --vid "${vid}" minio/bucket/testobj | md5sum
done
pkill silo
pkill minio
sleep 3
}
function main() {
start_port=$(shuf -i 10000-65000 -n 1)
start_silo_4drive ${start_port}
start_minio_4drive ${start_port}
}
function purge() {
-331
View File
@@ -1,331 +0,0 @@
// Copyright 2026 PGSTY contributors.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// helm-migration-guard compares a rendered legacy MinIO chart with the Silo
// upgrade candidate. Product labels, images, and commands may change; resource
// identity, selectors, PVCs, storage mounts, ports, secrets, and service-account
// references must not.
package main
import (
"encoding/json"
"errors"
"fmt"
"io"
"os"
"sort"
"strings"
"go.yaml.in/yaml/v3"
)
type resource struct {
key string
doc map[string]any
}
func main() {
if len(os.Args) != 3 {
fatal(errors.New("usage: helm-migration-guard OLD_RENDER NEW_RENDER"))
}
oldResources, err := readResources(os.Args[1])
if err != nil {
fatal(err)
}
newResources, err := readResources(os.Args[2])
if err != nil {
fatal(err)
}
if err := compare(oldResources, newResources); err != nil {
fatal(err)
}
fmt.Printf("Silo Helm migration identity is stable across %d rendered resources\n", len(oldResources))
}
func readResources(path string) (map[string]resource, error) {
file, err := os.Open(path)
if err != nil {
return nil, fmt.Errorf("open %s: %w", path, err)
}
defer file.Close()
resources := make(map[string]resource)
decoder := yaml.NewDecoder(file)
for document := 1; ; document++ {
var doc map[string]any
err = decoder.Decode(&doc)
if errors.Is(err, io.EOF) {
break
}
if err != nil {
return nil, fmt.Errorf("decode %s document %d: %w", path, document, err)
}
if len(doc) == 0 || text(doc["kind"]) == "" {
continue
}
metadata := object(doc["metadata"])
key := strings.Join([]string{text(doc["kind"]), text(metadata["namespace"]), text(metadata["name"])}, "/")
if _, exists := resources[key]; exists {
return nil, fmt.Errorf("%s contains duplicate resource %s", path, key)
}
resources[key] = resource{key: key, doc: doc}
}
return resources, nil
}
func compare(oldResources, newResources map[string]resource) error {
for key := range oldResources {
if _, ok := newResources[key]; !ok {
return fmt.Errorf("legacy resource would be removed or renamed: %s", key)
}
}
for key := range newResources {
if _, ok := oldResources[key]; !ok {
return fmt.Errorf("upgrade candidate unexpectedly adds a resource: %s", key)
}
}
keys := make([]string, 0, len(oldResources))
for key := range oldResources {
keys = append(keys, key)
}
sort.Strings(keys)
for _, key := range keys {
oldDoc := oldResources[key].doc
newDoc := newResources[key].doc
kind := text(oldDoc["kind"])
switch kind {
case "Service":
if err := same(key, "Service selector", at(oldDoc, "spec", "selector"), at(newDoc, "spec", "selector")); err != nil {
return err
}
if err := same(key, "Service ports", at(oldDoc, "spec", "ports"), at(newDoc, "spec", "ports")); err != nil {
return err
}
case "Deployment", "StatefulSet":
if err := same(key, "workload selector", at(oldDoc, "spec", "selector"), at(newDoc, "spec", "selector")); err != nil {
return err
}
}
if kind == "StatefulSet" {
if err := same(key, "StatefulSet serviceName", at(oldDoc, "spec", "serviceName"), at(newDoc, "spec", "serviceName")); err != nil {
return err
}
if err := same(key, "volume claim templates", claimTemplates(oldDoc), claimTemplates(newDoc)); err != nil {
return err
}
}
if kind == "PersistentVolumeClaim" {
if err := same(key, "PVC specification", at(oldDoc, "spec"), at(newDoc, "spec")); err != nil {
return err
}
}
if kind == "Secret" {
if err := same(key, "Secret keys", secretKeys(oldDoc), secretKeys(newDoc)); err != nil {
return err
}
}
if kind == "Deployment" || kind == "StatefulSet" || kind == "Job" {
if err := comparePod(key, kind, oldDoc, newDoc); err != nil {
return err
}
}
}
return nil
}
func comparePod(key, kind string, oldDoc, newDoc map[string]any) error {
oldPod := object(at(oldDoc, "spec", "template", "spec"))
newPod := object(at(newDoc, "spec", "template", "spec"))
if err := same(key, "service account", oldPod["serviceAccountName"], newPod["serviceAccountName"]); err != nil {
return err
}
if err := same(key, "referenced volume sources", volumeSources(oldPod), volumeSources(newPod)); err != nil {
return err
}
oldContainers := containers(oldPod)
newContainers := containers(newPod)
if err := same(key, "container identities", sortedKeys(oldContainers), sortedKeys(newContainers)); err != nil {
return err
}
for _, name := range sortedKeys(oldContainers) {
oldContainer := oldContainers[name]
newContainer := newContainers[name]
if err := same(key, name+" ports", oldContainer["ports"], newContainer["ports"]); err != nil {
return err
}
if err := same(key, name+" environment", oldContainer["env"], newContainer["env"]); err != nil {
return err
}
if err := same(key, name+" envFrom", oldContainer["envFrom"], newContainer["envFrom"]); err != nil {
return err
}
if err := same(key, name+" storage mounts", normalizedMounts(oldContainer, oldPod), normalizedMounts(newContainer, newPod)); err != nil {
return err
}
image := text(newContainer["image"])
if strings.HasPrefix(image, "pgsty/minio:") || strings.HasPrefix(image, "docker.io/pgsty/minio:") {
return fmt.Errorf("%s container %s still uses frozen image %s", key, name, image)
}
command := commandText(newContainer)
if strings.Contains(command, "/usr/bin/minio") {
return fmt.Errorf("%s container %s still invokes /usr/bin/minio", key, name)
}
if (kind == "Deployment" || kind == "StatefulSet") && strings.Contains(image, "pgsty/silo:") {
if !strings.Contains(command, "silo") || !strings.Contains(command, "server") {
return fmt.Errorf("%s container %s does not invoke the Silo server: %q", key, name, command)
}
}
}
return nil
}
func claimTemplates(doc map[string]any) []string {
var result []string
for _, raw := range list(at(doc, "spec", "volumeClaimTemplates")) {
claim := object(raw)
metadata := object(claim["metadata"])
result = append(result, text(metadata["name"])+"="+canonical(claim["spec"]))
}
sort.Strings(result)
return result
}
func secretKeys(doc map[string]any) []string {
var result []string
for _, section := range []string{"data", "stringData"} {
for key := range object(doc[section]) {
result = append(result, section+":"+key)
}
}
sort.Strings(result)
return result
}
func volumeSources(pod map[string]any) []string {
var result []string
for _, raw := range list(pod["volumes"]) {
volume := cloneObject(object(raw))
delete(volume, "name")
result = append(result, canonical(volume))
}
sort.Strings(result)
return result
}
func volumeSourceByName(pod map[string]any) map[string]string {
result := make(map[string]string)
for _, raw := range list(pod["volumes"]) {
volume := cloneObject(object(raw))
name := text(volume["name"])
delete(volume, "name")
result[name] = canonical(volume)
}
return result
}
func normalizedMounts(container, pod map[string]any) []string {
sources := volumeSourceByName(pod)
var result []string
for _, raw := range list(container["volumeMounts"]) {
mount := cloneObject(object(raw))
name := text(mount["name"])
delete(mount, "name")
mount["source"] = sources[name]
result = append(result, canonical(mount))
}
sort.Strings(result)
return result
}
func containers(pod map[string]any) map[string]map[string]any {
result := make(map[string]map[string]any)
for _, section := range []string{"initContainers", "containers"} {
for _, raw := range list(pod[section]) {
container := object(raw)
result[section+":"+text(container["name"])] = container
}
}
return result
}
func commandText(container map[string]any) string {
var parts []string
for _, field := range []string{"command", "args"} {
for _, value := range list(container[field]) {
parts = append(parts, text(value))
}
}
return strings.Join(parts, " ")
}
func at(root map[string]any, path ...string) any {
var current any = root
for _, part := range path {
current = object(current)[part]
}
return current
}
func object(value any) map[string]any {
if value == nil {
return map[string]any{}
}
result, _ := value.(map[string]any)
return result
}
func cloneObject(value map[string]any) map[string]any {
result := make(map[string]any, len(value))
for key, item := range value {
result[key] = item
}
return result
}
func list(value any) []any {
result, _ := value.([]any)
return result
}
func text(value any) string {
result, _ := value.(string)
return result
}
func sortedKeys[T any](values map[string]T) []string {
result := make([]string, 0, len(values))
for key := range values {
result = append(result, key)
}
sort.Strings(result)
return result
}
func same(resourceKey, field string, oldValue, newValue any) error {
oldCanonical := canonical(oldValue)
newCanonical := canonical(newValue)
if oldCanonical != newCanonical {
return fmt.Errorf("%s changes %s\nold: %s\nnew: %s", resourceKey, field, oldCanonical, newCanonical)
}
return nil
}
func canonical(value any) string {
data, err := json.Marshal(value)
if err != nil {
return fmt.Sprintf("<unmarshalable %T: %v>", value, err)
}
return string(data)
}
func fatal(err error) {
fmt.Fprintf(os.Stderr, "Silo Helm migration check failed: %v\n", err)
os.Exit(1)
}
-91
View File
@@ -1,91 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
if [ "$#" -ne 1 ]; then
echo "usage: $0 TARGET" >&2
exit 2
fi
target=$1
target_dir=$(dirname "${target}")
if [ ! -d "${target_dir}" ]; then
echo "target directory does not exist: ${target_dir}" >&2
exit 1
fi
sha256_file() {
if command -v sha256sum >/dev/null 2>&1; then
sha256sum "$1" | awk '{print $1}'
else
shasum -a 256 "$1" | awk '{print $1}'
fi
}
if [ -n "${MCLI_BIN:-}" ]; then
if [ ! -f "${MCLI_BIN}" ]; then
echo "MCLI_BIN is not a regular file: ${MCLI_BIN}" >&2
exit 1
fi
if ! printf '%s\n' "${MCLI_SHA256:-}" | grep -Eq '^[0-9a-fA-F]{64}$'; then
echo "MCLI_SHA256 must contain the expected SHA-256 for MCLI_BIN" >&2
exit 1
fi
actual=$(sha256_file "${MCLI_BIN}")
if [ "${actual}" != "${MCLI_SHA256,,}" ]; then
echo "MCLI_BIN checksum mismatch: expected ${MCLI_SHA256,,}, got ${actual}" >&2
exit 1
fi
install -m 0755 "${MCLI_BIN}" "${target}"
exit 0
fi
release=${MCLI_RELEASE:-RELEASE.2026-09-13T00-00-00Z}
version_hyphen=${release#RELEASE.}
package_version=$(printf '%s\n' "${version_hyphen}" | sed -E 's/^([0-9]{4})-([0-9]{2})-([0-9]{2})T([0-9]{2})-([0-9]{2})-([0-9]{2})Z$/\1\2\3\4\5\6.0.0/')
if [ "${package_version}" = "${version_hyphen}" ]; then
echo "invalid MCLI_RELEASE: ${release}" >&2
exit 1
fi
case $(uname -s) in
Linux) os=linux ;;
Darwin) os=darwin ;;
*) echo "unsupported mcli host OS: $(uname -s)" >&2; exit 1 ;;
esac
case $(uname -m) in
x86_64 | amd64) arch=amd64 ;;
aarch64 | arm64) arch=arm64 ;;
*) echo "unsupported mcli host architecture: $(uname -m)" >&2; exit 1 ;;
esac
archive="mcli_${package_version}_${os}_${arch}.tar.gz"
checksums="mcli_${package_version}_checksums.txt"
base_url="https://github.com/pgsty/mc/releases/download/${release}"
tmp_dir=$(mktemp -d "${TMPDIR:-/tmp}/silo-mcli.XXXXXX")
trap 'rm -rf "${tmp_dir}"' EXIT
curl --fail --location --retry 3 --silent --show-error \
"${base_url}/${checksums}" --output "${tmp_dir}/${checksums}"
curl --fail --location --retry 3 --silent --show-error \
"${base_url}/${archive}" --output "${tmp_dir}/${archive}"
expected=$(awk -v asset="${archive}" '
{
name=$2
sub(/^\*/, "", name)
if (name == asset && length($1) == 64 && $1 ~ /^[0-9a-fA-F]+$/) print tolower($1)
}
' "${tmp_dir}/${checksums}")
if ! printf '%s\n' "${expected}" | grep -Eq '^[0-9a-f]{64}$'; then
echo "checksum manifest does not contain exactly one valid entry for ${archive}" >&2
exit 1
fi
actual=$(sha256_file "${tmp_dir}/${archive}")
if [ "${actual}" != "${expected}" ]; then
echo "downloaded ${archive} checksum mismatch" >&2
exit 1
fi
tar -xzf "${tmp_dir}/${archive}" -C "${tmp_dir}" mcli
install -m 0755 "${tmp_dir}/mcli" "${target}"
-48
View File
@@ -1,48 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
if [ "$#" -ne 3 ]; then
echo "usage: $0 SOURCE SHA256 TARGET" >&2
exit 2
fi
source_ref=$1
expected=${2,,}
target=$3
if ! printf '%s\n' "${expected}" | grep -Eq '^[0-9a-f]{64}$'; then
echo "expected checksum must be a lowercase SHA-256 digest" >&2
exit 1
fi
if [ ! -d "$(dirname "${target}")" ]; then
echo "target directory does not exist: $(dirname "${target}")" >&2
exit 1
fi
tmp_file=$(mktemp "${TMPDIR:-/tmp}/silo-fixture.XXXXXX")
trap 'rm -f "${tmp_file}"' EXIT
case ${source_ref} in
https://*)
curl --fail --location --retry 3 --silent --show-error \
"${source_ref}" --output "${tmp_file}"
;;
*)
if [ ! -f "${source_ref}" ]; then
echo "fixture is not a regular file: ${source_ref}" >&2
exit 1
fi
cp "${source_ref}" "${tmp_file}"
;;
esac
if command -v sha256sum >/dev/null 2>&1; then
actual=$(sha256sum "${tmp_file}" | awk '{print $1}')
else
actual=$(shasum -a 256 "${tmp_file}" | awk '{print $1}')
fi
if [ "${actual}" != "${expected}" ]; then
echo "fixture checksum mismatch: expected ${expected}, got ${actual}" >&2
exit 1
fi
install -m 0755 "${tmp_file}" "${target}"
@@ -1,7 +1,7 @@
#!/bin/bash
# This script tests IAM migration from an old MinIO compatibility fixture into
# the current Silo server.
# This script is used to test the migration of IAM content from old minio
# instance to new minio instance.
#
# To run it locally, start the LDAP server in github.com/minio/minio-iam-testing
# repo (e.g. make podman-run), and then run this script.
@@ -15,7 +15,6 @@
OLD_VERSION=RELEASE.2024-03-26T22-10-45Z
OLD_BINARY_LINK=https://dl.min.io/server/minio/release/linux-amd64/archive/minio.${OLD_VERSION}
OLD_BINARY_SHA256=2050199d89e3057571620a1d453118fed5bd2de9d4f3b266b11365fdf984d676
__init__() {
if which curl &>/dev/null; then
@@ -28,16 +27,17 @@ __init__() {
export GOPATH=/tmp/gopath
export PATH="${PATH}":"${GOPATH}"/bin
if [ ! -x "${GOPATH}/bin/mc" ]; then
echo "Installing verified compatible client fixture"
mkdir -p "${GOPATH}/bin"
"$(git rev-parse --show-toplevel)/buildscripts/install-mcli.sh" "${GOPATH}/bin/mc"
if which mc &>/dev/null; then
echo "mc is already installed"
else
echo "Installing mc:"
go install github.com/minio/mc@latest
fi
if [ ! -x ./minio.${OLD_VERSION} ]; then
echo "Installing verified upstream compatibility fixture minio.${OLD_VERSION}"
"$(git rev-parse --show-toplevel)/buildscripts/install-verified-fixture.sh" \
"${OLD_BINARY_LINK}" "${OLD_BINARY_SHA256}" "minio.${OLD_VERSION}"
echo "Downloading minio.${OLD_VERSION} binary"
curl -o minio.${OLD_VERSION} ${OLD_BINARY_LINK}
chmod +x minio.${OLD_VERSION}
fi
if [ -z "$_MINIO_LDAP_TEST_SERVER" ]; then
@@ -49,7 +49,7 @@ __init__() {
}
create_iam_content_in_old_minio() {
echo "Creating IAM content in the old MinIO compatibility fixture."
echo "Creating IAM content in old minio instance."
MINIO_CI_CD=1 ./minio.${OLD_VERSION} server /tmp/data/{1...4} &
sleep 5
@@ -80,9 +80,9 @@ create_iam_content_in_old_minio() {
}
import_iam_content_in_new_minio() {
echo "Importing IAM content into the current Silo instance."
# Assume the current Silo binary exists.
MINIO_CI_CD=1 ./silo server /tmp/data/{1...4} &
echo "Importing IAM content in new minio instance."
# Assume current minio binary exists.
MINIO_CI_CD=1 ./minio server /tmp/data/{1...4} &
sleep 5
set -x
+95 -106
View File
@@ -1,124 +1,113 @@
#!/usr/bin/env bash
#!/bin/bash
set -euo pipefail
# Exercise both directions of the on-disk compatibility contract using an
# immutable pre-rebrand image and a container built from the current checkout.
# Every Docker resource is uniquely named and removed explicitly; this test
# never prunes unrelated images, containers, networks, or volumes.
repo_dir="$(git rev-parse --show-toplevel)"
old_image="${OLD_IMAGE:-docker.io/pgsty/minio@sha256:b6bfe7239bfc83fb90d31612d9704d86039dd714f7904b3f1ad68f211e602372}"
new_image="${NEW_IMAGE:-silo-upgrade-test:dev}"
suffix="$(date +%s)-$$"
network="silo-upgrade-net-${suffix}"
volume="silo-upgrade-data-${suffix}"
old_container="silo-upgrade-old-${suffix}"
new_container="silo-upgrade-new-${suffix}"
rollback_container="silo-upgrade-rollback-${suffix}"
root_user=silo-upgrade-admin
root_password=silo-upgrade-secret-123
trap 'cleanup $LINENO' ERR
# shellcheck disable=SC2120
cleanup() {
status=$?
trap - EXIT
if [ "${status}" -ne 0 ]; then
for name in "${old_container}" "${new_container}" "${rollback_container}"; do
docker logs "${name}" 2>/dev/null | tail -n 80 >&2 || true
done
fi
if [ "${KEEP_UPGRADE_TEST_RESOURCES:-0}" = 1 ]; then
printf 'Retained Docker resources for inspection: %s %s\n' "${network}" "${volume}" >&2
exit "${status}"
fi
docker rm -f "${old_container}" "${new_container}" "${rollback_container}" >/dev/null 2>&1 || true
docker network rm "${network}" >/dev/null 2>&1 || true
docker volume rm "${volume}" >/dev/null 2>&1 || true
exit "${status}"
}
trap cleanup EXIT
trap 'exit 130' INT TERM
MINIO_VERSION=dev /tmp/gopath/bin/docker-compose \
-f "buildscripts/upgrade-tests/compose.yml" \
down || true
wait_ready() {
name="$1"
ready=""
for _ in $(seq 1 90); do
if docker logs "${name}" 2>&1 | grep -q 'API:'; then
ready=1
break
fi
if [ "$(docker inspect -f '{{.State.Running}}' "${name}")" != true ]; then
break
fi
sleep 1
MINIO_VERSION=dev /tmp/gopath/bin/docker-compose \
-f "buildscripts/upgrade-tests/compose.yml" \
rm || true
for volume in $(docker volume ls -q | grep upgrade); do
docker volume rm ${volume} || true
done
if [ -z "${ready}" ]; then
echo "Server did not become ready: ${name}" >&2
return 1
docker volume prune -f
docker system prune -f || true
docker volume prune -f || true
docker volume rm $(docker volume ls -q -f dangling=true) || true
}
verify_checksum_after_heal() {
local sum1
sum1=$(curl -s "$2" | sha256sum)
mc admin heal --json -r "$1" >/dev/null # test after healing
local sum1_heal
sum1_heal=$(curl -s "$2" | sha256sum)
if [ "${sum1_heal}" != "${sum1}" ]; then
echo "mismatch expected ${sum1_heal}, got ${sum1}"
exit 1
fi
}
start_server() {
name="$1"
image="$2"
shift 2
docker run -d --name "${name}" --network "${network}" \
--mount "source=${volume},target=/data" \
-e MINIO_CI_CD=1 -e MINIO_ROOT_USER="${root_user}" -e MINIO_ROOT_PASSWORD="${root_password}" \
"${image}" "$@" >/dev/null
wait_ready "${name}"
docker exec "${name}" mcli alias set local http://127.0.0.1:9000 "${root_user}" "${root_password}" >/dev/null
verify_checksum_mc() {
local expected
expected=$(mc cat "$1" | sha256sum)
local got
got=$(mc cat "$2" | sha256sum)
if [ "${expected}" != "${got}" ]; then
echo "mismatch - expected ${expected}, got ${got}"
exit 1
fi
echo "matches - ${expected}, got ${got}"
}
stop_server() {
name="$1"
docker stop -t 20 "${name}" >/dev/null
test "$(docker inspect -f '{{.State.ExitCode}}' "${name}")" = 0
docker logs "${name}" 2>&1 | grep -q 'Exiting on signal'
docker rm "${name}" >/dev/null
add_alias() {
for i in $(seq 1 4); do
echo "... attempting to add alias $i"
until (mc alias set minio http://127.0.0.1:9000 minioadmin minioadmin); do
echo "...waiting... for 5secs" && sleep 5
done
done
echo "Sleeping for nginx"
sleep 20
}
command -v docker >/dev/null
docker info >/dev/null
if ! docker image inspect "${old_image}" >/dev/null 2>&1; then
docker pull "${old_image}"
fi
__init__() {
sudo apt install curl -y
export GOPATH=/tmp/gopath
export PATH=${PATH}:${GOPATH}/bin
if [ "${SILO_UPGRADE_SKIP_BUILD:-0}" != 1 ]; then
make -C "${repo_dir}" docker TAG="${new_image}"
fi
docker image inspect "${new_image}" >/dev/null
go install github.com/minio/mc@latest
docker network create "${network}" >/dev/null
docker volume create "${volume}" >/dev/null
## this is needed because github actions don't have
## docker-compose on all runners
COMPOSE_VERSION=v2.35.1
mkdir -p /tmp/gopath/bin/
wget -O /tmp/gopath/bin/docker-compose https://github.com/docker/compose/releases/download/${COMPOSE_VERSION}/docker-compose-linux-x86_64
chmod +x /tmp/gopath/bin/docker-compose
start_server "${old_container}" "${old_image}" minio server /data --address :9000
docker exec "${old_container}" mcli mb local/compat >/dev/null
docker exec "${old_container}" mcli version enable local/compat >/dev/null
printf 'old-version-1\n' | docker exec -i "${old_container}" mcli pipe local/compat/versioned.txt >/dev/null
printf 'old-version-2\n' | docker exec -i "${old_container}" mcli pipe local/compat/versioned.txt >/dev/null
test "$(docker exec "${old_container}" mcli ls --versions local/compat/versioned.txt | grep -c 'versioned.txt')" -ge 2
docker exec "${old_container}" mcli mb --with-lock local/locked >/dev/null
printf 'locked-by-old\n' | docker exec -i "${old_container}" mcli pipe local/locked/object.txt >/dev/null
dd if=/dev/zero bs=1048576 count=70 2>/dev/null | docker exec -i "${old_container}" mcli pipe local/compat/multipart.bin >/dev/null
test "$(docker exec "${old_container}" mcli stat --json local/compat/multipart.bin | jq -r '.size')" = 73400320
docker exec "${old_container}" mcli admin user add local migration-user migration-secret-123 >/dev/null
docker exec "${old_container}" mcli admin policy attach local readwrite --user migration-user >/dev/null
stop_server "${old_container}"
cleanup
start_server "${new_container}" "${new_image}" silo server /data --address :9000
test "$(docker exec "${new_container}" mcli cat local/compat/versioned.txt)" = old-version-2
test "$(docker exec "${new_container}" mcli cat local/locked/object.txt)" = locked-by-old
test "$(docker exec "${new_container}" mcli stat --json local/compat/multipart.bin | jq -r '.size')" = 73400320
docker exec "${new_container}" mcli admin user info local migration-user >/dev/null
docker exec "${new_container}" mcli alias set migrated http://127.0.0.1:9000 migration-user migration-secret-123 >/dev/null
printf 'written-by-silo\n' | docker exec -i "${new_container}" mcli pipe migrated/compat/silo.txt >/dev/null
stop_server "${new_container}"
TAG=minio/minio:dev make docker
start_server "${rollback_container}" "${old_image}" minio server /data --address :9000
test "$(docker exec "${rollback_container}" mcli cat local/compat/versioned.txt)" = old-version-2
test "$(docker exec "${rollback_container}" mcli cat local/compat/silo.txt)" = written-by-silo
test "$(docker exec "${rollback_container}" mcli stat --json local/compat/multipart.bin | jq -r '.size')" = 73400320
stop_server "${rollback_container}"
MINIO_VERSION=RELEASE.2019-12-19T22-52-26Z docker-compose \
-f "buildscripts/upgrade-tests/compose.yml" \
up -d --build
echo "MinIO-to-Silo data upgrade and rollback checks passed"
add_alias
mc mb minio/minio-test/
mc cp ./minio minio/minio-test/to-read/
mc cp /etc/hosts minio/minio-test/to-read/hosts
mc anonymous set download minio/minio-test
verify_checksum_mc ./minio minio/minio-test/to-read/minio
curl -s http://127.0.0.1:9000/minio-test/to-read/hosts | sha256sum
MINIO_VERSION=dev /tmp/gopath/bin/docker-compose -f "buildscripts/upgrade-tests/compose.yml" stop
}
main() {
MINIO_VERSION=dev /tmp/gopath/bin/docker-compose -f "buildscripts/upgrade-tests/compose.yml" up -d --build
add_alias
verify_checksum_after_heal minio/minio-test http://127.0.0.1:9000/minio-test/to-read/hosts
verify_checksum_mc ./minio minio/minio-test/to-read/minio
verify_checksum_mc /etc/hosts minio/minio-test/to-read/hosts
cleanup
}
(__init__ "$@" && main "$@")
+30 -19
View File
@@ -5,16 +5,16 @@ if [ -n "$TEST_DEBUG" ]; then
fi
WORK_DIR="$PWD/.verify-$RANDOM"
SILO_CONFIG_DIR="$WORK_DIR/.silo"
SILO=("$PWD/silo" --config-dir "$SILO_CONFIG_DIR" server)
MINIO_CONFIG_DIR="$WORK_DIR/.minio"
MINIO=("$PWD/minio" --config-dir "$MINIO_CONFIG_DIR" server)
if [ ! -x "$PWD/silo" ]; then
echo "Silo executable binary not found in current directory"
if [ ! -x "$PWD/minio" ]; then
echo "minio executable binary not found in current directory"
exit 1
fi
if [ ! -x "$PWD/silo" ]; then
echo "Silo executable binary not found in current directory"
if [ ! -x "$PWD/minio" ]; then
echo "minio executable binary not found in current directory"
exit 1
fi
@@ -30,9 +30,9 @@ catch() {
echo "error on line $1"
fi
echo "Cleaning up instances of Silo"
pkill silo || true
pkill -9 silo || true
echo "Cleaning up instances of MinIO"
pkill minio || true
pkill -9 minio || true
purge "$WORK_DIR"
if [ $# -ne 0 ]; then
exit $#
@@ -41,21 +41,32 @@ catch() {
catch
function start_silo_10drive() {
function start_minio_10drive() {
start_port=$1
export MINIO_ROOT_USER=silo
export MINIO_ROOT_PASSWORD=silo1234
export MC_HOST_silo="http://silo:silo1234@127.0.0.1:${start_port}/"
export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=minio123
export MC_HOST_minio="http://minio:minio123@127.0.0.1:${start_port}/"
unset MINIO_KMS_AUTO_ENCRYPTION # do not auto-encrypt objects
export MINIO_CI_CD=1
mkdir ${WORK_DIR}
C_PWD=${PWD}
if [ ! -x "$PWD/mc" ]; then
"$(git rev-parse --show-toplevel)/buildscripts/install-mcli.sh" "$PWD/mc"
MC_BUILD_DIR="mc-$RANDOM"
if ! git clone --quiet https://github.com/minio/mc "$MC_BUILD_DIR"; then
echo "failed to download https://github.com/minio/mc"
purge "${MC_BUILD_DIR}"
exit 1
fi
(cd "${MC_BUILD_DIR}" && go build -o "$C_PWD/mc")
# remove mc source.
purge "${MC_BUILD_DIR}"
fi
"${SILO[@]}" --address ":$start_port" "${WORK_DIR}/disk{1...10}" >"${WORK_DIR}/server1.log" 2>&1 &
"${MINIO[@]}" --address ":$start_port" "${WORK_DIR}/disk{1...10}" >"${WORK_DIR}/server1.log" 2>&1 &
pid=$!
disown $pid
sleep 5
@@ -68,10 +79,10 @@ function start_silo_10drive() {
exit 1
fi
"${PWD}/mc" mb --with-versioning silo/bucket
"${PWD}/mc" mb --with-versioning minio/bucket
export AWS_ACCESS_KEY_ID=silo
export AWS_SECRET_ACCESS_KEY=silo1234
export AWS_ACCESS_KEY_ID=minio
export AWS_SECRET_ACCESS_KEY=minio123
aws --endpoint-url http://localhost:"$start_port" s3api create-multipart-upload --bucket bucket --key obj-1 >upload-id.json
uploadId=$(jq -r '.UploadId' upload-id.json)
@@ -109,7 +120,7 @@ EOF
function main() {
start_port=$(shuf -i 10000-65000 -n 1)
start_silo_10drive ${start_port}
start_minio_10drive ${start_port}
}
main "$@"
-157
View File
@@ -1,157 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
repo_dir="$(cd "${script_dir}/.." && pwd)"
dist_dir="${DIST_DIR:-${repo_dir}/dist}"
nfpm_config="${NFPM_CONFIG:-${repo_dir}/.github/nfpm.yml}"
if [ -z "${PKG_VERSION:-}" ]; then
echo "PKG_VERSION is required" >&2
exit 1
fi
# Re-validate the shape release.yml derived from the tag. The packages are
# named from this, so a malformed value would ship under a name no repository
# can order against.
if ! [[ "${PKG_VERSION}" =~ ^[0-9]{14}\.0\.0$ ]]; then
echo "Invalid PKG_VERSION: ${PKG_VERSION}" >&2
exit 1
fi
# The PGSTY release segment, PGDG-style. sign-release-rpms.sh declares the
# same value as expected_release, and test-release.yml asserts the two agree.
PKG_RELEASE="1PGSTY"
if ! command -v nfpm >/dev/null 2>&1; then
echo "nfpm is required" >&2
exit 1
fi
if [ ! -f "${nfpm_config}" ]; then
echo "Missing nFPM config: ${nfpm_config}" >&2
exit 1
fi
# nfpm resolves a relative content src against the current directory, not
# against the config file, so the unit path is passed in absolute. Otherwise
# this only works when invoked from the repository root and fails elsewhere on
# a message that names the file rather than the cause.
unit_file="${repo_dir}/silo.service"
defaults_file="${repo_dir}/silo.env"
sysusers_file="${repo_dir}/silo.sysusers"
license_file="${repo_dir}/LICENSE"
notice_file="${repo_dir}/NOTICE"
postinstall_file="${repo_dir}/buildscripts/package/postinstall.sh"
preremove_file="${repo_dir}/buildscripts/package/preremove.sh"
if [ ! -f "${unit_file}" ]; then
echo "Missing systemd unit: ${unit_file}" >&2
exit 1
fi
if [ ! -f "${defaults_file}" ]; then
echo "Missing defaults file: ${defaults_file}" >&2
exit 1
fi
if [ ! -f "${sysusers_file}" ]; then
echo "Missing sysusers file: ${sysusers_file}" >&2
exit 1
fi
for distributed_doc in "${license_file}" "${notice_file}"; do
if [ ! -s "${distributed_doc}" ]; then
echo "Missing license material: ${distributed_doc}" >&2
exit 1
fi
done
for lifecycle_script in "${postinstall_file}" "${preremove_file}"; do
if [ ! -x "${lifecycle_script}" ]; then
echo "Missing executable package lifecycle script: ${lifecycle_script}" >&2
exit 1
fi
done
packages_dir="${dist_dir}/packages"
mkdir -p "${packages_dir}"
# Two spaces, no trailing newline: sign-release-rpms.sh parses these files to
# check download integrity before it signs, and regenerates them afterwards in
# the same shape.
sha256_file() {
local file="$1"
local digest
if command -v sha256sum >/dev/null 2>&1; then
digest="$(sha256sum "${file}" | awk '{print $1}')"
else
digest="$(shasum -a 256 "${file}" | awk '{print $1}')"
fi
printf '%s %s' "${digest}" "$(basename "${file}")" > "${file}.sha256sum"
}
find_binary() {
local goarch="$1"
local matches
local count
# Must resolve to exactly one binary. Picking the first of several build
# variants (an added goamd64 level, a stale dist entry) would silently ship a
# package whose contents do not match its name.
matches="$(find "${dist_dir}" -maxdepth 2 -type f \
-path "${dist_dir}/silo_linux_${goarch}*/silo" | sort)"
count="$(printf '%s' "${matches}" | grep -c . || true)"
if [ "${count}" -eq 0 ]; then
echo "Missing GoReleaser binary for linux/${goarch}" >&2
exit 1
fi
if [ "${count}" -ne 1 ]; then
echo "Expected exactly one GoReleaser binary for linux/${goarch}, found ${count}:" >&2
printf '%s\n' "${matches}" >&2
exit 1
fi
printf '%s\n' "${matches}"
}
build_arch() {
local goarch="$1"
local rpm_arch="$2"
local deb_arch="$3"
local apk_arch="$4"
local source
local rpm_file
local deb_file
local apk_file
source="$(find_binary "${goarch}")"
# These names are the public download names and must not drift; RPM and DEB
# carry the PGSTY release number (nfpm renders it as the RPM Release tag and
# as the Debian revision after a dash). APK stays bare: Alpine pkgrel only
# admits -r<integer>, so a lettered release cannot ride along there.
rpm_file="${packages_dir}/silo-${PKG_VERSION}-${PKG_RELEASE}.${rpm_arch}.rpm"
deb_file="${packages_dir}/silo_${PKG_VERSION}-${PKG_RELEASE}_${deb_arch}.deb"
apk_file="${packages_dir}/silo_${PKG_VERSION}_${apk_arch}.apk"
(
cd "${repo_dir}"
export NFPM_UNIT="${unit_file}" NFPM_DEFAULTS="${defaults_file}" NFPM_SYSUSERS="${sysusers_file}" \
NFPM_LICENSE="${license_file}" NFPM_NOTICE="${notice_file}"
PKG_VERSION="${PKG_VERSION}" NFPM_RELEASE="${PKG_RELEASE}" NFPM_ARCH="${goarch}" NFPM_SOURCE="${source}" \
nfpm package --config "${nfpm_config}" --packager rpm --target "${rpm_file}"
PKG_VERSION="${PKG_VERSION}" NFPM_RELEASE="${PKG_RELEASE}" NFPM_ARCH="${goarch}" NFPM_SOURCE="${source}" \
nfpm package --config "${nfpm_config}" --packager deb --target "${deb_file}"
PKG_VERSION="${PKG_VERSION}" NFPM_RELEASE='' NFPM_ARCH="${goarch}" NFPM_SOURCE="${source}" \
nfpm package --config "${nfpm_config}" --packager apk --target "${apk_file}"
)
sha256_file "${rpm_file}"
sha256_file "${deb_file}"
sha256_file "${apk_file}"
}
build_arch amd64 x86_64 amd64 x86_64
build_arch arm64 aarch64 arm64 aarch64
find "${packages_dir}" -maxdepth 1 -type f | sort
-172
View File
@@ -1,172 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
repo_dir="$(cd "${script_dir}/../.." && pwd)"
postinstall="${script_dir}/postinstall.sh"
preremove="${script_dir}/preremove.sh"
test_dir="$(mktemp -d)"
fakebin="${test_dir}/bin"
log_file="${test_dir}/calls.log"
useradd_shell=/usr/sbin/nologin
[ -x "${useradd_shell}" ] || useradd_shell=/sbin/nologin
busybox_shell=/sbin/nologin
[ -x "${busybox_shell}" ] || busybox_shell=/bin/false
cleanup() {
rm -rf "${test_dir}"
}
trap cleanup EXIT
mkdir -p "${fakebin}"
touch "${log_file}"
# One dispatcher represents every external command used by the lifecycle
# scripts. The tested scripts run with no host utilities in PATH, so a green
# result cannot create a real account or touch the host service manager.
cat > "${fakebin}/fake-command" <<'EOF'
#!/bin/sh
set -eu
command_name=${0##*/}
case "${command_name}" in
id)
[ "${PACKAGE_TEST_USER_EXISTS:-0}" = 1 ]
;;
getent)
[ "${PACKAGE_TEST_GROUP_EXISTS:-0}" = 1 ]
;;
systemd-sysusers|useradd|addgroup|adduser|systemctl)
{
printf '%s' "${command_name}"
for argument in "$@"; do
printf ' %s' "${argument}"
done
printf '\n'
} >> "${PACKAGE_TEST_LOG}"
;;
*)
echo "unexpected fake command: ${command_name}" >&2
exit 1
;;
esac
EOF
chmod +x "${fakebin}/fake-command"
link_command() {
ln -sf fake-command "${fakebin}/$1"
}
unlink_optional_commands() {
rm -f \
"${fakebin}/systemd-sysusers" \
"${fakebin}/useradd" \
"${fakebin}/adduser" \
"${fakebin}/addgroup"
}
reset_log() {
: > "${log_file}"
}
run_postinstall() {
PACKAGE_TEST_LOG="${log_file}" \
PACKAGE_TEST_USER_EXISTS="${1}" \
PACKAGE_TEST_GROUP_EXISTS="${2}" \
PATH="${fakebin}" \
/bin/sh "${postinstall}"
}
run_preremove() {
PACKAGE_TEST_LOG="${log_file}" PATH="${fakebin}" \
/bin/sh "${preremove}" "$@"
}
assert_log_line() {
grep -Fx -- "$1" "${log_file}" >/dev/null
}
reject_log_text() {
if grep -F -- "$1" "${log_file}" >/dev/null; then
echo "unexpected lifecycle call containing '$1':" >&2
cat "${log_file}" >&2
exit 1
fi
}
link_command id
link_command getent
link_command systemctl
# Clean install through systemd-sysusers. Side-by-side safety is represented
# by the fact that the only service-manager operation is daemon-reload: no old
# service is stopped, disabled, enabled, masked, or restarted.
unlink_optional_commands
link_command systemd-sysusers
reset_log
run_postinstall 0 0
assert_log_line "systemd-sysusers /usr/lib/sysusers.d/silo.conf"
assert_log_line "systemctl daemon-reload"
test "$(wc -l < "${log_file}" | tr -d ' ')" -eq 2
# An existing service account is preserved without modification.
reset_log
run_postinstall 1 0
test "$(cat "${log_file}")" = "systemctl daemon-reload"
# useradd creates a private group only when one does not already exist. An
# administrator may pre-create group silo with the legacy GID; that group must
# be reused rather than causing installation to fail.
unlink_optional_commands
link_command useradd
reset_log
run_postinstall 0 0
assert_log_line "useradd --system --user-group --no-create-home --shell ${useradd_shell} --comment Silo object storage service silo"
reject_log_text "--gid silo"
reset_log
run_postinstall 0 1
assert_log_line "useradd --system --gid silo --no-create-home --shell ${useradd_shell} --comment Silo object storage service silo"
reject_log_text "--user-group"
# BusyBox follows the same existing-group contract.
unlink_optional_commands
link_command adduser
link_command addgroup
reset_log
run_postinstall 0 0
assert_log_line "addgroup -S silo"
assert_log_line "adduser -S -D -H -G silo -s ${busybox_shell} silo"
reset_log
run_postinstall 0 1
reject_log_text "addgroup"
assert_log_line "adduser -S -D -H -G silo -s ${busybox_shell} silo"
# Debian remove, RPM erase, and Alpine deinstall stop the Silo unit. Upgrade
# arguments must leave the running service alone.
for removal_argument in remove 0 20260214120000.0.0-r0; do
reset_log
run_preremove "${removal_argument}"
test "$(cat "${log_file}")" = "systemctl disable --now silo.service"
done
for upgrade_argument in upgrade 1; do
reset_log
run_preremove "${upgrade_argument}"
test ! -s "${log_file}"
done
# The package deliberately leaves legacy ownership changes to an explicit
# systemd drop-in. Lifecycle scripts must never rewrite ownership or touch the
# old unit, and the base unit must expose overridable User/Group directives.
grep -Fx 'User=silo' "${repo_dir}/silo.service" >/dev/null
grep -Fx 'Group=silo' "${repo_dir}/silo.service" >/dev/null
if grep -Ein '\b(chown|chgrp|usermod|groupmod)\b|minio\.service' \
"${postinstall}" "${preremove}"; then
echo "package lifecycle scripts must not mutate data ownership or the legacy service" >&2
exit 1
fi
echo "Silo package lifecycle checks passed"
-44
View File
@@ -1,44 +0,0 @@
#!/bin/sh
set -eu
sysusers_file=/usr/lib/sysusers.d/silo.conf
group_exists() {
if command -v getent >/dev/null 2>&1; then
getent group silo >/dev/null 2>&1
return
fi
[ -r /etc/group ] || return 1
while IFS=: read -r group_name _; do
[ "${group_name}" = silo ] && return 0
done < /etc/group
return 1
}
if ! id -u silo >/dev/null 2>&1; then
if command -v systemd-sysusers >/dev/null 2>&1; then
systemd-sysusers "${sysusers_file}"
elif command -v useradd >/dev/null 2>&1; then
nologin_shell=/usr/sbin/nologin
[ -x "${nologin_shell}" ] || nologin_shell=/sbin/nologin
if group_exists; then
useradd --system --gid silo --no-create-home --shell "${nologin_shell}" --comment "Silo object storage service" silo
else
useradd --system --user-group --no-create-home --shell "${nologin_shell}" --comment "Silo object storage service" silo
fi
elif command -v adduser >/dev/null 2>&1 && command -v addgroup >/dev/null 2>&1; then
nologin_shell=/sbin/nologin
[ -x "${nologin_shell}" ] || nologin_shell=/bin/false
group_exists || addgroup -S silo
adduser -S -D -H -G silo -s "${nologin_shell}" silo
else
echo "Unable to create the silo system account: systemd-sysusers, useradd, or BusyBox adduser is required" >&2
exit 1
fi
fi
if command -v systemctl >/dev/null 2>&1; then
systemctl daemon-reload >/dev/null 2>&1 || true
fi
-14
View File
@@ -1,14 +0,0 @@
#!/bin/sh
set -eu
# Debian passes "remove" for an actual removal, RPM passes 0 to %preun, and
# Alpine runs pre-deinstall only for removal and passes the old dotted version.
# Upgrade paths deliberately leave the running service untouched.
case "${1:-}" in
remove|0|*.*)
if command -v systemctl >/dev/null 2>&1; then
systemctl disable --now silo.service >/dev/null 2>&1 || true
fi
;;
esac
-52
View File
@@ -1,52 +0,0 @@
-----BEGIN PGP PUBLIC KEY BLOCK-----
mQINBGaV5PwBEACbErI+7yOrsXTT3mR83O6Fw9WyHJqozhyNPF3dA1gAtWpfWqd4
S9x6vBjVwUbIRn21jYgov0hDiaLABNQhRzifvVr0r1IjBW8lhA8zJGaO42Uz0aBW
YIkajOklsXgYMX+gSmy5WXzM31sDQVMnzptHh9dwW067hMM5pJKDslu2pLMwSb9K
QgIFcYsaR0taBkcDg4dNu1gncriD/GcdXIS0/V4R82DIYeIqj2S0lt0jDTACbUz3
C6esrTw2XerCeHKHb9c/V+KMhqvLJOOpy/aJWLrTGBoaH7xw6v0qg32OYiBxlUj9
VEzoQbDfbRkR+jlxiuYP3scUs/ziKrSh+0mshVbeuLRSNfuHLa7C4xTEnATcgD1J
MZeMaJXIcDt+DN+1aHVQjY5YNvr5wA3ykxW51uReZf7/odgqVW3+1rhW5pd8NQKQ
qoVUHOtIrC9KaiGfrczEtJTNUxcNZV9eBgcKHYDXB2hmR2pIf7WvydgXTs/qIsXg
SIzfKjisi795Dd5GrvdLYXVnu9YzylWlkJ5rjod1wnSxkI/CcCJaoPLnXZA9KV7A
cpMWWaUEXP/XBIwIU+vxDd1taBIaPIOv1KIdzvG7QqAQtf5Lphi5HfaGvBud/CVt
mvWhRPJMr1J0ER2xAgU2iZR7dN0vSF6zDqc0W09RAoC0nDS3tupDX2BrOwARAQAB
tCRSdW9oYW5nIEZlbmcgKFBpZ3N0eSkgPHJoQHZvbm5nLmNvbT6JAlEEEwEIADsW
IQSVkqe8emguczM3bgnnk12Nub2LIAUCZpXk/AIbAwULCQgHAgIiAgYVCgkICwIE
FgIDAQIeBwIXgAAKCRDnk12Nub2LIOMuEACBLVc09O4icFwc45R3KMvOMu14Egpn
UkpmBKhErjup0TIunzI0zZH6HG8LGuf6XEdH4ItCJeLg5349UE00BUHNmxk2coo2
u4Wtu28LPqmxb6sqpuRAaefedU6vqfs7YN6WWp52pVF1KdOHkIOcgAQ9z3ZHdosM
I/Y/UxO2t4pjdCAfJHOmGPrbgLcHSMpoLLxjuf3YIwS5NSfjNDd0Y8sKFUcMGLCF
5P0lv5feLLdZvh2Una34UmHKhZlXC5E3vlY9bf/LgsRzXRFQosD0RsCXbz3Tk+zF
+j/eP3WhUvJshqIDuY6eJYCzMjiA8sM5gety+htVJuD0mewp+qAhjxE0d4bIr4qO
BKQzBt9tT2ackCPdgW42VPS+IZymm1oMET0hgZfKiVpwsKO6qxeWn4RW2jJ0zkUJ
MsrrxOPFdZQAtuFcLwa5PUAHHs6XQT2vzxDpeE9lInQ14lshofU5ZKIeb9sbvb/w
P+xnDqvZ1pcotEIBvDK0S0jHbHHqtioIUdDFvdCBlBlYP1TQRNPlJ7TJDBBvhj8i
fmjQsYSV1u36aHOJVGYNHv+SyJpVd3nHCZn97ADM9qHnDm7xljyHXPzIx4FMmBGJ
UTiLH5yxa1xhWr42Iv3TykaQJVbpydmBuegFR8WbWitAvVqI3HvRG+FalLsjJruc
8YDAf7gHdj/937kCDQRmleT8ARAAmJxscC76NZzqFBiaeq2+aJxOt1HGPqKb4pbz
jLKRX9sFkeXuzhfZaNDljnr2yrnQ75rit9Aah/loEhbSHanNUDCNmvOeSEISr9yA
yfOnqlcVOtcwWQK57n6MvlCSM8Js3jdoSmCFHVtdFFwxejE5ok0dk1VFYDIg6DRk
ZBMuxGO7ZJW7TzCxhK4AL+NNYA2wX6b+IVMn6CA9kwNwCNrrnGHR1sblSxZp7lPo
+GsqzYY0LXGR2eEicgKd4lk38gaO8Q4d1mlpX95vgdhGKxR+CM26y9QU0qrO1hXP
Fw6lX9HfIUkVNrqAa1mzgneYXivnLvcj8gc7bFAdweX4MyBHsmiPm32WqjUJFAmw
kcKYaiyfDJ+1wusa/b+7RCnshWc8B9udYbXfvcpOGgphpUuvomKT8at3ToJfEWmR
BzToYYTsgAAX8diY/X53BHCE/+MhLccglEUYNZyBRkTwDLrS9QgNkhrADaTwxsv1
8PwnVKve/ZxwOU0QGf4ZOhA2YQOE5hkRDR5uY2OHsOS5vHsd9Y6kNNnO8EBy99d1
QiBJOW3AP0nr4Cj1/NhdigAujsYRKiCAuPT7dgqART58VU4bZ3PgonMlziLe7+ht
YYxV+wyP6LVqicDd0MLLvG7r/JOiWuABOUxsFFaRecehoPJjeAEQxnWJjedokXKL
HVOFaEkAEQEAAYkCNgQYAQgAIBYhBJWSp7x6aC5zMzduCeeTXY25vYsgBQJmleT8
AhsMAAoJEOeTXY25vYsgG8sP/3UdsWuiwTsf/x4BTW82K+Uk9YwZDnUNH+4dUMED
bKT1C6CbuSZ7Mnbi2rVsmGzOMs9MehIx6Ko8/iCR2OCeWi8Q+wM+iffAfWuT1GK6
7f/VIfoYBUWEa+kvDcPgEbd5Tu7ZdUO/jROVBSlXRSjzK9LpIj7GozBTJ8Vqy5x7
oqbWPPEYtGDVHime8o6f5/wfhNgL3mFnoq6srK7KhwACwfTXlNqAlGiXGa30Yj+b
Cj6IvmxoII49E67/ovMEmzDCb3RXiaL6OATy25P+HQJvWvAam7Qq5Xn+bZg65Mup
vXq3zoX0a7EKXc5vsJVNtTlXO1ATdYszKP5uNzkHrNAN52VRYaowq1vPy/MVMbSI
rL/hTFKr7ZNhmC7jmS3OuJyCYQsfEerubtBUuc/W6JDc2oTI3xOG1S2Zj8f4PxLl
H7vMG4E+p6eOrUGw6VQXjFsH9GtwhkPh/ZGMKENb2+JztJ02674Cok4s5c/lZFKz
mmRUcNjX2bm2K0GfGG5/hAog/CHCeUZvwIh4hZLkdeJ1QsIYpN8xbvY7QP6yh4VB
XrL18+2sontZ45MsGResrRibB35x7IrCrxZsVtRJZthHqshiORPatgy+AiWcAtEv
UWEnnC1xBSasNebw4fSE8AJg9JMCRw+3GAetlotOeW9q7PN6yrXD9rGuV/QquQNd
/c7w
=4rRi
-----END PGP PUBLIC KEY BLOCK-----
File diff suppressed because it is too large Load Diff
-406
View File
@@ -1,406 +0,0 @@
// Copyright 2026 PGSTY contributors.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// rebrand-guard records the compatibility identifiers that a product rebrand
// must not accidentally rename. It intentionally excludes product branding and
// delivery names, which are validated by buildscripts/verify-rebrand.sh.
package main
import (
"bytes"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"flag"
"fmt"
"go/ast"
"go/parser"
"go/token"
"os"
"os/exec"
"path/filepath"
"regexp"
"sort"
"strconv"
"strings"
)
const manifestVersion = 4
var (
minioImportRE = regexp.MustCompile(`github\.com/minio/[A-Za-z0-9_./-]+`)
envRE = regexp.MustCompile(`\b_?MINIO_[A-Z0-9_]+\b`)
metricRE = regexp.MustCompile(`\bminio_[A-Za-z0-9_]+\b`)
headerRE = regexp.MustCompile(`(?i)\bx-minio-[a-z0-9_-]+\b`)
routeRE = regexp.MustCompile(`^/[A-Za-z0-9._~!$&'()*+,;=:@%/?{}=-]*`)
storageRE = regexp.MustCompile(`\.minio\.sys(?:/[A-Za-z0-9._${}-]+)*`)
policyRE = regexp.MustCompile(`(?:arn:minio|minio:s3)[A-Za-z0-9_:/.*${}-]*`)
brandRE = regexp.MustCompile(`(?i)(^|[^a-z0-9_])minio([^a-z0-9_]|$)`)
)
type manifest struct {
Version int `json:"version"`
ModulePath string `json:"module_path"`
MinioImports []string `json:"minio_imports"`
Environment []string `json:"environment"`
Metrics []string `json:"metrics"`
Headers []string `json:"headers"`
Routes []string `json:"routes"`
RouteRoots []string `json:"route_roots"`
GridRoutes []string `json:"grid_routes"`
StorageMarkers []string `json:"storage_markers"`
PolicyValues []string `json:"policy_values"`
BrandAllowlist []string `json:"brand_allowlist"`
}
func main() {
write := flag.Bool("write", false, "replace the checked-in compatibility baseline")
flag.Parse()
repo, err := gitOutput("rev-parse", "--show-toplevel")
if err != nil {
fatal(err)
}
repo = strings.TrimSpace(repo)
baselinePath := filepath.Join(repo, "buildscripts", "rebrand-guard", "compat-baseline.json")
current, err := collect(repo)
if err != nil {
fatal(err)
}
if *write {
if err := writeManifest(baselinePath, current); err != nil {
fatal(err)
}
fmt.Printf("wrote %s\n", baselinePath)
printSummary(current)
return
}
want, err := readManifest(baselinePath)
if err != nil {
fatal(err)
}
if err := compare(want, current); err != nil {
fatal(err)
}
printSummary(current)
fmt.Println("Silo rebrand compatibility baseline is unchanged")
}
func collect(repo string) (manifest, error) {
files, err := trackedFiles(repo)
if err != nil {
return manifest{}, err
}
sets := map[string]map[string]struct{}{
"imports": {},
"env": {},
"metrics": {},
"headers": {},
"routes": {},
"roots": {},
"grid": {},
"storage": {},
"policy": {},
"brand": {},
}
modulePath := ""
fset := token.NewFileSet()
for _, rel := range files {
// Investigation artifacts contain synthetic routes and archived configurations.
if rel == "SILO_REBRANDING_MIGRATION.md" ||
strings.HasPrefix(rel, "docs/investigations/") ||
strings.HasPrefix(rel, "buildscripts/rebrand-guard/") ||
strings.HasPrefix(rel, "buildscripts/helm-migration-guard/") {
continue
}
path := filepath.Join(repo, filepath.FromSlash(rel))
data, err := os.ReadFile(path)
if errors.Is(err, os.ErrNotExist) {
continue
}
if err != nil {
return manifest{}, fmt.Errorf("read %s: %w", rel, err)
}
if bytes.IndexByte(data, 0) >= 0 {
continue
}
text := string(data)
addMatches(sets["env"], envRE, text, false)
addMatches(sets["headers"], headerRE, text, true)
addMatches(sets["storage"], storageRE, text, false)
addMatches(sets["policy"], policyRE, text, false)
if strings.HasSuffix(rel, ".go") && (strings.HasPrefix(rel, "cmd/") || strings.HasPrefix(rel, "internal/")) {
addMatches(sets["metrics"], metricRE, text, false)
}
if rel == "go.mod" {
addMatches(sets["imports"], minioImportRE, text, false)
for _, line := range strings.Split(text, "\n") {
fields := strings.Fields(line)
if len(fields) == 2 && fields[0] == "module" {
modulePath = fields[1]
break
}
}
}
if strings.HasSuffix(rel, ".go") {
file, err := parser.ParseFile(fset, path, data, parser.SkipObjectResolution)
if err != nil {
return manifest{}, fmt.Errorf("parse %s: %w", rel, err)
}
for _, spec := range file.Imports {
value, err := strconv.Unquote(spec.Path.Value)
if err == nil && strings.HasPrefix(value, "github.com/minio/") {
sets["imports"][value] = struct{}{}
}
}
if !strings.HasSuffix(rel, "_test.go") {
// Test files hold request paths for fixtures, not served routes.
collectStringMatches(sets["routes"], routeRE, file)
if strings.HasPrefix(rel, "cmd/") || strings.HasPrefix(rel, "internal/") {
collectBrandStrings(sets["brand"], rel, file)
}
}
collectNamedStringValues(sets["roots"], rel, file, "minioReservedBucket")
if rel == "internal/grid/manager.go" {
collectStringMatches(sets["grid"], routeRE, file)
}
}
}
// This was a shell-local PID variable in the generated inspect script,
// never a supported environment setting.
delete(sets["env"], "MINIO_SRVR_PID")
if modulePath == "" {
return manifest{}, errors.New("go.mod module path was not found")
}
return manifest{
Version: manifestVersion,
ModulePath: modulePath,
MinioImports: sorted(sets["imports"]),
Environment: sorted(sets["env"]),
Metrics: sorted(sets["metrics"]),
Headers: sorted(sets["headers"]),
Routes: sorted(sets["routes"]),
RouteRoots: sorted(sets["roots"]),
GridRoutes: sorted(sets["grid"]),
StorageMarkers: sorted(sets["storage"]),
PolicyValues: sorted(sets["policy"]),
BrandAllowlist: sorted(sets["brand"]),
}, nil
}
func collectBrandStrings(dst map[string]struct{}, rel string, file *ast.File) {
ast.Inspect(file, func(node ast.Node) bool {
literal, ok := node.(*ast.BasicLit)
if !ok || literal.Kind != token.STRING {
return true
}
value, err := strconv.Unquote(literal.Value)
if err != nil || !brandRE.MatchString(value) || strings.HasPrefix(value, "github.com/minio/") {
return true
}
dst[filepath.ToSlash(rel)+"="+strconv.Quote(value)] = struct{}{}
return true
})
}
func collectNamedStringValues(dst map[string]struct{}, rel string, file *ast.File, names ...string) {
wanted := make(map[string]struct{}, len(names))
for _, name := range names {
wanted[name] = struct{}{}
}
for _, decl := range file.Decls {
gen, ok := decl.(*ast.GenDecl)
if !ok {
continue
}
for _, rawSpec := range gen.Specs {
spec, ok := rawSpec.(*ast.ValueSpec)
if !ok {
continue
}
for i, name := range spec.Names {
if _, ok := wanted[name.Name]; !ok || i >= len(spec.Values) {
continue
}
literal, ok := spec.Values[i].(*ast.BasicLit)
if !ok || literal.Kind != token.STRING {
continue
}
value, err := strconv.Unquote(literal.Value)
if err == nil {
dst[filepath.ToSlash(rel)+":"+name.Name+"="+value] = struct{}{}
}
}
}
}
}
func collectStringMatches(dst map[string]struct{}, re *regexp.Regexp, file *ast.File) {
ast.Inspect(file, func(node ast.Node) bool {
literal, ok := node.(*ast.BasicLit)
if !ok || literal.Kind != token.STRING {
return true
}
value, err := strconv.Unquote(literal.Value)
if err == nil {
addMatches(dst, re, value, false)
}
return true
})
}
func trackedFiles(repo string) ([]string, error) {
cmd := exec.Command("git", "-C", repo, "ls-files", "--cached", "-z")
out, err := cmd.Output()
if err != nil {
return nil, fmt.Errorf("git ls-files: %w", err)
}
parts := bytes.Split(out, []byte{0})
files := make([]string, 0, len(parts))
for _, part := range parts {
if len(part) > 0 {
files = append(files, string(part))
}
}
return files, nil
}
func addMatches(dst map[string]struct{}, re *regexp.Regexp, text string, lower bool) {
for _, match := range re.FindAllString(text, -1) {
if lower {
match = strings.ToLower(match)
}
dst[match] = struct{}{}
}
}
func sorted(set map[string]struct{}) []string {
values := make([]string, 0, len(set))
for value := range set {
values = append(values, value)
}
sort.Strings(values)
return values
}
func writeManifest(path string, value manifest) error {
data, err := json.MarshalIndent(value, "", " ")
if err != nil {
return err
}
data = append(data, '\n')
return os.WriteFile(path, data, 0o644)
}
func readManifest(path string) (manifest, error) {
data, err := os.ReadFile(path)
if err != nil {
return manifest{}, fmt.Errorf("read compatibility baseline (run go run ./buildscripts/rebrand-guard --write once): %w", err)
}
var value manifest
if err := json.Unmarshal(data, &value); err != nil {
return manifest{}, err
}
if value.Version != manifestVersion {
return manifest{}, fmt.Errorf("unsupported compatibility baseline version %d", value.Version)
}
return value, nil
}
func compare(want, got manifest) error {
var failures []string
if want.ModulePath != got.ModulePath {
failures = append(failures, fmt.Sprintf("module_path: want %q, got %q", want.ModulePath, got.ModulePath))
}
checks := []struct {
name string
want, got []string
}{
{"minio_imports", want.MinioImports, got.MinioImports},
{"environment", want.Environment, got.Environment},
{"metrics", want.Metrics, got.Metrics},
{"headers", want.Headers, got.Headers},
{"routes", want.Routes, got.Routes},
{"route_roots", want.RouteRoots, got.RouteRoots},
{"grid_routes", want.GridRoutes, got.GridRoutes},
{"storage_markers", want.StorageMarkers, got.StorageMarkers},
{"policy_values", want.PolicyValues, got.PolicyValues},
{"brand_allowlist", want.BrandAllowlist, got.BrandAllowlist},
}
for _, check := range checks {
if missing, added := setDiff(check.want, check.got); len(missing) > 0 || len(added) > 0 {
var b strings.Builder
fmt.Fprintf(&b, "%s compatibility set changed", check.name)
for _, value := range missing {
fmt.Fprintf(&b, "\n - %s", value)
}
for _, value := range added {
fmt.Fprintf(&b, "\n + %s", value)
}
failures = append(failures, b.String())
}
}
if len(failures) > 0 {
return errors.New(strings.Join(failures, "\n"))
}
return nil
}
func setDiff(want, got []string) (missing, added []string) {
wantSet := make(map[string]struct{}, len(want))
gotSet := make(map[string]struct{}, len(got))
for _, value := range want {
wantSet[value] = struct{}{}
}
for _, value := range got {
gotSet[value] = struct{}{}
}
for _, value := range want {
if _, ok := gotSet[value]; !ok {
missing = append(missing, value)
}
}
for _, value := range got {
if _, ok := wantSet[value]; !ok {
added = append(added, value)
}
}
return missing, added
}
func printSummary(value manifest) {
fmt.Printf("compatibility manifest: imports=%d env=%d metrics=%d headers=%d routes=%d roots=%d grid=%d storage=%d policy=%d brand=%d sha256=%s\n",
len(value.MinioImports), len(value.Environment), len(value.Metrics), len(value.Headers),
len(value.Routes), len(value.RouteRoots), len(value.GridRoutes), len(value.StorageMarkers), len(value.PolicyValues),
len(value.BrandAllowlist), manifestDigest(value))
}
func manifestDigest(value manifest) string {
data, _ := json.Marshal(value)
sum := sha256.Sum256(data)
return hex.EncodeToString(sum[:])
}
func gitOutput(args ...string) (string, error) {
out, err := exec.Command("git", args...).CombinedOutput()
if err != nil {
return "", fmt.Errorf("git %s: %s: %w", strings.Join(args, " "), strings.TrimSpace(string(out)), err)
}
return string(out), nil
}
func fatal(err error) {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
+23 -14
View File
@@ -6,29 +6,38 @@ set -x
set -e
WORK_DIR="$PWD/.verify-$RANDOM"
SILO_CONFIG_DIR="$WORK_DIR/.silo"
SILO=("$PWD/silo" --config-dir "$SILO_CONFIG_DIR" server)
MINIO_CONFIG_DIR="$WORK_DIR/.minio"
MINIO=("$PWD/minio" --config-dir "$MINIO_CONFIG_DIR" server)
if [ ! -x "$PWD/silo" ]; then
echo "Silo executable binary not found in current directory"
if [ ! -x "$PWD/minio" ]; then
echo "minio executable binary not found in current directory"
exit 1
fi
function start_silo_5drive() {
function start_minio_5drive() {
start_port=$1
export MINIO_ROOT_USER=silo
export MINIO_ROOT_PASSWORD=silo1234
export MC_HOST_silo="http://silo:silo1234@127.0.0.1:${start_port}/"
export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=minio123
export MC_HOST_minio="http://minio:minio123@127.0.0.1:${start_port}/"
unset MINIO_KMS_AUTO_ENCRYPTION # do not auto-encrypt objects
export MINIO_CI_CD=1
mkdir -p "${WORK_DIR}"
"$(git rev-parse --show-toplevel)/buildscripts/install-mcli.sh" "${WORK_DIR}/mc"
MC_BUILD_DIR="mc-$RANDOM"
if ! git clone --quiet https://github.com/minio/mc "$MC_BUILD_DIR"; then
echo "failed to download https://github.com/minio/mc"
purge "${MC_BUILD_DIR}"
exit 1
fi
(cd "${MC_BUILD_DIR}" && go build -o "$WORK_DIR/mc")
# remove mc source.
purge "${MC_BUILD_DIR}"
"${WORK_DIR}/mc" cp --quiet -r "buildscripts/cicd-corpus/" "${WORK_DIR}/cicd-corpus/"
"${SILO[@]}" --address ":$start_port" "${WORK_DIR}/cicd-corpus/disk{1...5}" >"${WORK_DIR}/server1.log" 2>&1 &
"${MINIO[@]}" --address ":$start_port" "${WORK_DIR}/cicd-corpus/disk{1...5}" >"${WORK_DIR}/server1.log" 2>&1 &
pid=$!
disown $pid
sleep 5
@@ -41,16 +50,16 @@ function start_silo_5drive() {
exit 1
fi
"${WORK_DIR}/mc" stat silo/bucket/testobj
"${WORK_DIR}/mc" stat minio/bucket/testobj
pkill silo
pkill minio
sleep 3
}
function main() {
start_port=$(shuf -i 10000-65000 -n 1)
start_silo_5drive ${start_port}
start_minio_5drive ${start_port}
}
function purge() {
+36 -30
View File
@@ -5,42 +5,48 @@ set -o pipefail
set -x
WORK_DIR="$PWD/.verify-$RANDOM"
SILO_CONFIG_DIR="$WORK_DIR/.silo"
MINIO_OLD=("$PWD/minio.RELEASE.2020-10-28T08-16-50Z" --config-dir "$SILO_CONFIG_DIR" server)
SILO=("$PWD/silo" --config-dir "$SILO_CONFIG_DIR" server)
MINIO_CONFIG_DIR="$WORK_DIR/.minio"
MINIO_OLD=("$PWD/minio.RELEASE.2020-10-28T08-16-50Z" --config-dir "$MINIO_CONFIG_DIR" server)
MINIO=("$PWD/minio" --config-dir "$MINIO_CONFIG_DIR" server)
if [ ! -x "$PWD/silo" ]; then
echo "Silo executable binary not found in current directory"
if [ ! -x "$PWD/minio" ]; then
echo "minio executable binary not found in current directory"
exit 1
fi
function download_old_release() {
if [ ! -x minio.RELEASE.2020-10-28T08-16-50Z ]; then
: "${SILO_LEGACY_FIXTURE_2020:?set SILO_LEGACY_FIXTURE_2020 to the audited legacy binary}"
: "${SILO_LEGACY_SHA256_2020:?set SILO_LEGACY_SHA256_2020 to its audited SHA-256}"
"$(git rev-parse --show-toplevel)/buildscripts/install-verified-fixture.sh" \
"${SILO_LEGACY_FIXTURE_2020}" "${SILO_LEGACY_SHA256_2020}" \
"minio.RELEASE.2020-10-28T08-16-50Z"
if [ ! -f minio.RELEASE.2020-10-28T08-16-50Z ]; then
curl --silent -O https://dl.minio.io/server/minio/release/linux-amd64/archive/minio.RELEASE.2020-10-28T08-16-50Z
chmod a+x minio.RELEASE.2020-10-28T08-16-50Z
fi
}
function verify_rewrite() {
start_port=$1
export MINIO_ACCESS_KEY=silo
export MINIO_SECRET_KEY=silo1234
export MC_HOST_silo="http://silo:silo1234@127.0.0.1:${start_port}/"
export MINIO_ACCESS_KEY=minio
export MINIO_SECRET_KEY=minio123
export MC_HOST_minio="http://minio:minio123@127.0.0.1:${start_port}/"
unset MINIO_KMS_AUTO_ENCRYPTION # do not auto-encrypt objects
export MINIO_CI_CD=1
mkdir -p "${WORK_DIR}"
"$(git rev-parse --show-toplevel)/buildscripts/install-mcli.sh" "${WORK_DIR}/mc"
MC_BUILD_DIR="mc-$RANDOM"
if ! git clone --quiet https://github.com/minio/mc "$MC_BUILD_DIR"; then
echo "failed to download https://github.com/minio/mc"
purge "${MC_BUILD_DIR}"
exit 1
fi
(cd "${MC_BUILD_DIR}" && go build -o "$WORK_DIR/mc")
# remove mc source.
purge "${MC_BUILD_DIR}"
"${MINIO_OLD[@]}" --address ":$start_port" "${WORK_DIR}/xl{1...16}" >"${WORK_DIR}/server1.log" 2>&1 &
pid=$!
disown $pid
"${WORK_DIR}/mc" ready silo/
"${WORK_DIR}/mc" ready minio/
if ! ps -p ${pid} 1>&2 >/dev/null; then
echo "server1 log:"
@@ -50,30 +56,30 @@ function verify_rewrite() {
exit 1
fi
"${WORK_DIR}/mc" mb silo/healing-rewrite-bucket --quiet --with-lock
"${WORK_DIR}/mc" mb minio/healing-rewrite-bucket --quiet --with-lock
"${WORK_DIR}/mc" cp \
buildscripts/verify-build.sh \
silo/healing-rewrite-bucket/ \
minio/healing-rewrite-bucket/ \
--disable-multipart --quiet
"${WORK_DIR}/mc" cp \
buildscripts/verify-build.sh \
silo/healing-rewrite-bucket/ \
minio/healing-rewrite-bucket/ \
--disable-multipart --quiet
"${WORK_DIR}/mc" cp \
buildscripts/verify-build.sh \
silo/healing-rewrite-bucket/ \
minio/healing-rewrite-bucket/ \
--disable-multipart --quiet
kill ${pid}
sleep 3
"${SILO[@]}" --address ":$start_port" "${WORK_DIR}/xl{1...16}" >"${WORK_DIR}/server1.log" 2>&1 &
"${MINIO[@]}" --address ":$start_port" "${WORK_DIR}/xl{1...16}" >"${WORK_DIR}/server1.log" 2>&1 &
pid=$!
disown $pid
"${WORK_DIR}/mc" ready silo/
"${WORK_DIR}/mc" ready minio/
if ! ps -p ${pid} 1>&2 >/dev/null; then
echo "server1 log:"
@@ -86,8 +92,8 @@ function verify_rewrite() {
if ! ./s3-check-md5 \
-debug \
-versions \
-access-key silo \
-secret-key silo1234 \
-access-key minio \
-secret-key minio123 \
-endpoint "http://127.0.0.1:${start_port}/" 2>&1 | grep INTACT; then
echo "server1 log:"
cat "${WORK_DIR}/server1.log"
@@ -95,7 +101,7 @@ function verify_rewrite() {
mkdir -p inspects
(
cd inspects
"${WORK_DIR}/mc" admin inspect silo/healing-rewrite-bucket/verify-build.sh/**
"${WORK_DIR}/mc" admin inspect minio/healing-rewrite-bucket/verify-build.sh/**
)
"${WORK_DIR}/mc" mb play/inspects
@@ -105,14 +111,14 @@ function verify_rewrite() {
exit 1
fi
go run ./buildscripts/heal-manual.go "127.0.0.1:${start_port}" "silo" "silo1234"
go run ./buildscripts/heal-manual.go "127.0.0.1:${start_port}" "minio" "minio123"
sleep 1
if ! ./s3-check-md5 \
-debug \
-versions \
-access-key silo \
-secret-key silo1234 \
-access-key minio \
-secret-key minio123 \
-endpoint http://127.0.0.1:${start_port}/ 2>&1 | grep INTACT; then
echo "server1 log:"
cat "${WORK_DIR}/server1.log"
@@ -120,7 +126,7 @@ function verify_rewrite() {
mkdir -p inspects
(
cd inspects
"${WORK_DIR}/mc" admin inspect silo/healing-rewrite-bucket/verify-build.sh/**
"${WORK_DIR}/mc" admin inspect minio/healing-rewrite-bucket/verify-build.sh/**
)
"${WORK_DIR}/mc" mb play/inspects
-287
View File
@@ -1,287 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
# These are the single source of truth for the package identity: .github/nfpm.yml
# must agree with them, and test-release.yml asserts that it does. Drift the
# other way round would only surface here, on the maintainer's machine, after
# the build has already run and uploaded.
expected_fingerprint="9592A7BC7A682E7333376E09E7935D8DB9BD8B20"
expected_release="1PGSTY"
expected_vendor="PGSTY"
expected_packager="Ruohang Feng (@Vonng) <rh@vonng.com>"
expected_url="https://silo.pgsty.com"
expected_summary="S3-Interface Libre Object Storage, a community-maintained S3-compatible server."
expected_description="S3-Interface Libre Object Storage, a community-maintained S3-compatible server."
expected_license="AGPL-3.0-or-later"
expected_group="Applications/File"
expected_payload="/etc/default/silo
/usr/bin/silo
/usr/lib/systemd/system/silo.service
/usr/lib/sysusers.d/silo.conf
/usr/share/doc/silo/LICENSE
/usr/share/doc/silo/NOTICE"
repository="${GH_REPO:-pgsty/silo}"
container="${DNFUPDATE_CONTAINER:-dnfupdate}"
upload=false
release_tag=""
usage() {
cat <<'EOF'
Usage: buildscripts/sign-release-rpms.sh RELEASE.TAG [--upload] [--repo OWNER/REPO] [--container NAME]
Downloads the two unsigned RPMs from a Draft GitHub Release, signs them with
the expected Pigsty key in the local dnfupdate container, verifies the result,
and regenerates their .sha256sum files. Nothing is uploaded unless --upload is
provided.
EOF
}
while [ "$#" -gt 0 ]; do
case "$1" in
--upload)
upload=true
;;
--repo)
shift
if [ "$#" -eq 0 ]; then
echo "--repo requires OWNER/REPO" >&2
exit 1
fi
repository="$1"
;;
--container)
shift
if [ "$#" -eq 0 ]; then
echo "--container requires a name" >&2
exit 1
fi
container="$1"
;;
-h|--help)
usage
exit 0
;;
-*)
echo "Unknown option: $1" >&2
usage >&2
exit 1
;;
*)
if [ -n "${release_tag}" ]; then
echo "Only one release tag may be specified" >&2
exit 1
fi
release_tag="$1"
;;
esac
shift
done
if [ -z "${release_tag}" ]; then
usage >&2
exit 1
fi
for command in docker gh; do
if ! command -v "${command}" >/dev/null 2>&1; then
echo "${command} is required" >&2
exit 1
fi
done
version_hyphen="${release_tag#RELEASE.}"
package_version="$(printf '%s\n' "${version_hyphen}" | sed -E \
's/^([0-9]{4})-([0-9]{2})-([0-9]{2})T([0-9]{2})-([0-9]{2})-([0-9]{2})Z$/\1\2\3\4\5\6.0.0/')"
if [ "${package_version}" = "${version_hyphen}" ]; then
echo "Invalid release tag: ${release_tag}" >&2
exit 1
fi
if [ "$(gh release view "${release_tag}" --repo "${repository}" --json isDraft --jq .isDraft)" != "true" ]; then
echo "Refusing to sign: ${release_tag} is not a Draft release" >&2
exit 1
fi
if [ "$(docker inspect --format '{{.State.Running}}' "${container}" 2>/dev/null || true)" != "true" ]; then
echo "Signing container is not running: ${container}" >&2
exit 1
fi
secret_fingerprints="$(docker exec "${container}" \
gpg --batch --with-colons --list-secret-keys 2>/dev/null |
awk -F: '$1 == "fpr" { print toupper($10) }')"
if ! printf '%s\n' "${secret_fingerprints}" | grep -Fxq "${expected_fingerprint}"; then
echo "Expected signing key is not available in ${container}: ${expected_fingerprint}" >&2
exit 1
fi
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
repo_dir="$(cd "${script_dir}/.." && pwd)"
work_root="${SIGN_WORKDIR:-${repo_dir}/.release-sign}"
mkdir -p "${work_root}"
work_dir="$(mktemp -d "${work_root}/${release_tag}.XXXXXX")"
unsigned_dir="${work_dir}/unsigned"
signed_dir="${work_dir}/signed"
mkdir -p "${unsigned_dir}" "${signed_dir}"
chmod 700 "${work_dir}" "${unsigned_dir}" "${signed_dir}"
rpm_files=(
"silo-${package_version}-${expected_release}.x86_64.rpm"
"silo-${package_version}-${expected_release}.aarch64.rpm"
)
download_patterns=()
for rpm_file in "${rpm_files[@]}"; do
download_patterns+=(--pattern "${rpm_file}" --pattern "${rpm_file}.sha256sum")
done
echo "Downloading RPMs from Draft release ${repository}@${release_tag}"
gh release download "${release_tag}" --repo "${repository}" \
--dir "${unsigned_dir}" "${download_patterns[@]}"
sha256_digest() {
local file="$1"
if command -v sha256sum >/dev/null 2>&1; then
sha256sum "${file}" | awk '{print $1}'
else
shasum -a 256 "${file}" | awk '{print $1}'
fi
}
for rpm_file in "${rpm_files[@]}"; do
rpm_path="${unsigned_dir}/${rpm_file}"
checksum_path="${rpm_path}.sha256sum"
test -s "${rpm_path}"
test -s "${checksum_path}"
actual_line="$(sha256_digest "${rpm_path}") ${rpm_file}"
published_line="$(tr -d '\n' < "${checksum_path}")"
if [ "${actual_line}" != "${published_line}" ]; then
echo "Checksum mismatch for ${rpm_file}" >&2
exit 1
fi
done
safe_tag="$(printf '%s' "${release_tag}" | tr -c 'A-Za-z0-9._-' '_')"
container_dir="/tmp/silo-sign-${safe_tag}-$$"
docker exec "${container}" mkdir -p "${container_dir}"
cleanup_container() {
local rpm_file
for rpm_file in "${rpm_files[@]}"; do
docker exec "${container}" rm -f "${container_dir}/${rpm_file}" >/dev/null 2>&1 || true
done
docker exec "${container}" rmdir "${container_dir}" >/dev/null 2>&1 || true
}
trap cleanup_container EXIT
assert_rpm_tag() {
local rpm_path="$1"
local tag="$2"
local expected="$3"
local actual
actual="$(docker exec "${container}" rpm -qp --queryformat "%{${tag}}" "${rpm_path}")"
if [ "${actual}" != "${expected}" ]; then
echo "Unexpected RPM ${tag}: ${actual}" >&2
echo "Expected RPM ${tag}: ${expected}" >&2
exit 1
fi
}
for rpm_file in "${rpm_files[@]}"; do
case "${rpm_file}" in
*.x86_64.rpm)
expected_arch="x86_64"
;;
*.aarch64.rpm)
expected_arch="aarch64"
;;
*)
echo "Unexpected RPM filename: ${rpm_file}" >&2
exit 1
;;
esac
echo "Signing ${rpm_file} with ${expected_fingerprint}"
docker cp "${unsigned_dir}/${rpm_file}" "${container}:${container_dir}/${rpm_file}" >/dev/null
container_rpm="${container_dir}/${rpm_file}"
assert_rpm_tag "${container_rpm}" NAME silo
assert_rpm_tag "${container_rpm}" VERSION "${package_version}"
assert_rpm_tag "${container_rpm}" RELEASE "${expected_release}"
assert_rpm_tag "${container_rpm}" ARCH "${expected_arch}"
assert_rpm_tag "${container_rpm}" VENDOR "${expected_vendor}"
assert_rpm_tag "${container_rpm}" PACKAGER "${expected_packager}"
assert_rpm_tag "${container_rpm}" URL "${expected_url}"
assert_rpm_tag "${container_rpm}" LICENSE "${expected_license}"
assert_rpm_tag "${container_rpm}" GROUP "${expected_group}"
assert_rpm_tag "${container_rpm}" SUMMARY "${expected_summary}"
assert_rpm_tag "${container_rpm}" DESCRIPTION "${expected_description}"
rpm_payload="$(docker exec "${container}" rpm -qpl "${container_rpm}")"
if [ "${rpm_payload}" != "${expected_payload}" ]; then
echo "Unexpected RPM payload for ${rpm_file}:" >&2
printf '%s\n' "${rpm_payload}" >&2
exit 1
fi
docker exec "${container}" rpmsign \
--define "_gpg_name ${expected_fingerprint}" \
--addsign "${container_rpm}"
signature_output="$(docker exec "${container}" rpmkeys --checksig --verbose "${container_rpm}")"
printf '%s\n' "${signature_output}"
if ! printf '%s\n' "${signature_output}" | tr '[:upper:]' '[:lower:]' | grep -q 'key id b9bd8b20: ok'; then
echo "Signature verification failed for ${rpm_file}" >&2
exit 1
fi
docker cp "${container}:${container_dir}/${rpm_file}" "${signed_dir}/${rpm_file}" >/dev/null
signed_digest="$(sha256_digest "${signed_dir}/${rpm_file}")"
printf '%s %s' "${signed_digest}" "${rpm_file}" > "${signed_dir}/${rpm_file}.sha256sum"
docker exec "${container}" rpm -qp --queryformat \
$'Name: %{NAME}\nVersion: %{VERSION}-%{RELEASE}\nArch: %{ARCH}\nVendor: %{VENDOR}\nPackager: %{PACKAGER}\nURL: %{URL}\n' \
"${container_rpm}"
echo "SHA256: ${signed_digest}"
done
if [ "${upload}" = true ]; then
upload_files=()
for rpm_file in "${rpm_files[@]}"; do
upload_files+=("${signed_dir}/${rpm_file}" "${signed_dir}/${rpm_file}.sha256sum")
done
echo "Replacing RPMs in Draft release ${release_tag}"
gh release upload "${release_tag}" --repo "${repository}" --clobber "${upload_files[@]}"
for rpm_file in "${rpm_files[@]}"; do
for asset in "${rpm_file}" "${rpm_file}.sha256sum"; do
local_digest="sha256:$(sha256_digest "${signed_dir}/${asset}")"
remote_digest=""
for attempt in 1 2 3 4 5; do
remote_digest="$(gh release view "${release_tag}" --repo "${repository}" --json assets \
--jq ".assets[] | select(.name == \"${asset}\") | .digest")"
if [ "${local_digest}" = "${remote_digest}" ]; then
break
fi
if [ "${attempt}" -lt 5 ]; then
sleep 2
fi
done
if [ "${local_digest}" != "${remote_digest}" ]; then
echo "GitHub asset digest mismatch for ${asset}" >&2
echo "Local: ${local_digest}" >&2
echo "Remote: ${remote_digest}" >&2
exit 1
fi
echo "Verified GitHub asset: ${asset} ${remote_digest}"
done
done
else
echo
echo "Signed RPMs are ready for review in: ${signed_dir}"
echo "Re-run with --upload to replace the RPM assets in the Draft release."
fi
+32 -21
View File
@@ -5,16 +5,16 @@ if [ -n "$TEST_DEBUG" ]; then
fi
WORK_DIR="$PWD/.verify-$RANDOM"
SILO_CONFIG_DIR="$WORK_DIR/.silo"
SILO=("$PWD/silo" --config-dir "$SILO_CONFIG_DIR" server)
MINIO_CONFIG_DIR="$WORK_DIR/.minio"
MINIO=("$PWD/minio" --config-dir "$MINIO_CONFIG_DIR" server)
if [ ! -x "$PWD/silo" ]; then
echo "Silo executable binary not found in current directory"
if [ ! -x "$PWD/minio" ]; then
echo "minio executable binary not found in current directory"
exit 1
fi
if [ ! -x "$PWD/silo" ]; then
echo "Silo executable binary not found in current directory"
if [ ! -x "$PWD/minio" ]; then
echo "minio executable binary not found in current directory"
exit 1
fi
@@ -30,9 +30,9 @@ catch() {
echo "error on line $1"
fi
echo "Cleaning up instances of Silo"
pkill silo || true
pkill -9 silo || true
echo "Cleaning up instances of MinIO"
pkill minio || true
pkill -9 minio || true
purge "$WORK_DIR"
if [ $# -ne 0 ]; then
exit $#
@@ -70,20 +70,31 @@ function send_put_object_request() {
return 0
}
function test_silo_with_timeout() {
function test_minio_with_timeout() {
start_port=$1
export MINIO_ROOT_USER=silo
export MINIO_ROOT_PASSWORD=silo1234
export MC_HOST_silo="http://silo:silo1234@127.0.0.1:${start_port}/"
export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=minio123
export MC_HOST_minio="http://minio:minio123@127.0.0.1:${start_port}/"
export MINIO_CI_CD=1
mkdir ${WORK_DIR}
C_PWD=${PWD}
if [ ! -x "$PWD/mc" ]; then
"$(git rev-parse --show-toplevel)/buildscripts/install-mcli.sh" "$PWD/mc"
MC_BUILD_DIR="mc-$RANDOM"
if ! git clone --quiet https://github.com/minio/mc "$MC_BUILD_DIR"; then
echo "failed to download https://github.com/minio/mc"
purge "${MC_BUILD_DIR}"
exit 1
fi
(cd "${MC_BUILD_DIR}" && go build -o "$C_PWD/mc")
# remove mc source.
purge "${MC_BUILD_DIR}"
fi
"${SILO[@]}" --address ":$start_port" --read-header-timeout ${srv_hdr_timeout}s --idle-timeout ${srv_idle_timeout}s "${WORK_DIR}/disk/" >"${WORK_DIR}/server1.log" 2>&1 &
"${MINIO[@]}" --address ":$start_port" --read-header-timeout ${srv_hdr_timeout}s --idle-timeout ${srv_idle_timeout}s "${WORK_DIR}/disk/" >"${WORK_DIR}/server1.log" 2>&1 &
pid=$!
disown $pid
sleep 1
@@ -98,20 +109,20 @@ function test_silo_with_timeout() {
set -e
"${PWD}/mc" mb silo/testbucket
"${PWD}/mc" anonymous set public silo/testbucket
"${PWD}/mc" mb minio/testbucket
"${PWD}/mc" anonymous set public minio/testbucket
# slow header writing
send_put_object_request 20 0 && exit -1
"${PWD}/mc" stat silo/testbucket/testobject && exit -1
"${PWD}/mc" stat minio/testbucket/testobject && exit -1
# quick header write and slow bodywrite
send_put_object_request 0 40 && exit -1
"${PWD}/mc" stat silo/testbucket/testobject && exit -1
"${PWD}/mc" stat minio/testbucket/testobject && exit -1
# quick header and body write
send_put_object_request 1 1 || exit -1
"${PWD}/mc" stat silo/testbucket/testobject || exit -1
"${PWD}/mc" stat minio/testbucket/testobject || exit -1
}
function main() {
@@ -120,7 +131,7 @@ function main() {
export srv_idle_timeout=5
export -f gen_put_request
test_silo_with_timeout ${start_port}
test_minio_with_timeout ${start_port}
}
main "$@"
+74
View File
@@ -0,0 +1,74 @@
# Settings and configurations that are common for all containers
x-minio-common: &minio-common
image: minio/minio:${MINIO_VERSION}
command: server http://minio{1...4}/data{1...3}
env_file:
- ./minio.env
expose:
- "9000"
- "9001"
# starts 4 docker containers running minio server instances.
# using nginx reverse proxy, load balancing, you can access
# it through port 9000.
services:
minio1:
<<: *minio-common
hostname: minio1
volumes:
- data1-1:/data1
- data1-2:/data2
- data1-3:/data3
minio2:
<<: *minio-common
hostname: minio2
volumes:
- data2-1:/data1
- data2-2:/data2
- data2-3:/data3
minio3:
<<: *minio-common
hostname: minio3
volumes:
- data3-1:/data1
- data3-2:/data2
- data3-3:/data3
minio4:
<<: *minio-common
hostname: minio4
volumes:
- data4-1:/data1
- data4-2:/data2
- data4-3:/data3
nginx:
image: nginx:1.19.2-alpine
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf:ro
ports:
- "9000:9000"
- "9001:9001"
depends_on:
- minio1
- minio2
- minio3
- minio4
## By default this config uses default local driver,
## For custom volumes replace with volume driver configuration.
volumes:
data1-1:
data1-2:
data1-3:
data2-1:
data2-2:
data2-3:
data3-1:
data3-2:
data3-3:
data4-1:
data4-2:
data4-3:
+3
View File
@@ -0,0 +1,3 @@
MINIO_ACCESS_KEY=minioadmin
MINIO_SECRET_KEY=minioadmin
MINIO_BROWSER=off

Some files were not shown because too many files have changed in this diff Show More