Compare commits

...

382 Commits

Author SHA1 Message Date
Feng Ruohang 69d5e92791 Merge pull request #220 from AleksaMCode/fix/lambda-targetidset-isempty
Fix `TargetIDSet.IsEmpty` and add regression test
2026-09-23 08:07:09 +08:00
AleksaMCode efd5698597 lambda/event: fix TargetIDSet.IsEmpty and add regression test
Signed-off-by: AleksaMCode <aleksamcode@gmail.com>
2026-09-17 22:06:44 +02:00
Feng Ruohang 2fde3cf535 docs: update ec description 2026-09-17 09:42:26 +08:00
Feng Ruohang 2a4d51406b test: build the purge-response fixture ARN at runtime
The rebrand compatibility guard tracks every literal policy value that
names the upstream brand. Construct the replication ARN the way the
other replication fixtures do instead of adding a new literal.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 23:50:09 +08:00
Feng Ruohang 0596685ae7 docs: record the delete-marker purge, listing quorum and migration tag repairs
Describe the three correctness repairs merged after the 20260916 candidate
was prepared, together with their known remaining limits (#217, #218).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 23:44:44 +08:00
Feng Ruohang 358ab38fb0 fix: report a purged version as a delete marker only for stored markers
The exact-version purge path copies the looked-up DeleteMarker flag
into the DELETE response so that removing a delete marker keeps its
x-amz-delete-marker header. That flag is also set for a data version
whose purge is pending, because the lookup exposes such a version as
deleted for visibility. Retrying the purge after the replication
configuration was removed therefore answered with
x-amz-delete-marker: true and raised ObjectRemovedDeleteMarkerCreated
for a data version.

Only a stored marker carries no erasure layout; use that to decide the
response identity. Found by the adversarial review of the purge change.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 22:53:12 +08:00
Feng Ruohang 254b19ac07 fix(replication): recheck queued delete-marker creations under the lock
A delete-marker creation task carries the marker as it looked when it
was queued by the DELETE handler, a GET/HEAD/LIST heal, the scanner or
MRF. Tasks from several frontends serialize on the per-object
replication lock, so a task can run after the user has purged that
marker and the purge has already reached the targets. The target no
longer holds the marker, so the queued creation recreated it there with
the original VersionID and mtime. This is the late-create sequence seen
in the 2026-09-16 three-site runs: purge 204 on both targets, then about
85 ms later a replicated creation for the same VersionID.

Re-read the source version under the replication lock before sending a
creation. A missing version, a non-marker version, or a version under
purge makes the task stale; it is dropped without touching the source.
A read that cannot confirm either way is retried through MRF instead of
being treated as absence.

Creations already on the wire, replays from other sites, and cleanup of
minority residue after a crash are not covered by this check.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 22:35:08 +08:00
Feng Ruohang eb4f5e5b31 Fix exact-version purges and delete-marker metadata healing
Require write-quorum absence for missing purge retries, aggregate removed and already-absent votes only for physical purges, and resolve receiver purges by version across pools. Preserve marker metadata through creation and healing while retaining DELETE response semantics.

Add real-disk quorum, pool, callback, metadata, heal and outbound replication regressions. Synchronize capacity fixture installation and restoration with concurrent IAM readers.

Signed-off-by: Feng Ruohang <rh@vonng.com>
(cherry picked from commit 22ba4d426677aa07470927fc349a43afd87e19b6)
2026-09-16 22:24:27 +08:00
Feng Ruohang 8d06424b12 fix: retain quorate null versions after newer minorities
Recount only when the existing selection is below quorum and each original
nonempty stream has one ordinary null version with identical EC headers.
Reuse the existing header grouping and leave history pruning unchanged.

Add deterministic signed LIST, complete-metadata permutation, excluded-history,
object-read, scanner/heal, Walk and migration-consumer regressions.

Signed-off-by: Feng Ruohang <rh@vonng.com>
(cherry picked from commit 9804e4deb4d5a18cca640be72bff9b47a415f7de)
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 21:50:03 +08:00
Feng Ruohang fced863036 fix(storage): preserve tags during pool migration
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 20:22:37 +08:00
Feng Ruohang 5af0865aab Merge pull request #214 from pgsty/codex/release-prep-20260916
Prepare the SILO 20260916000000 source candidate with released component pins, CPU metrics synchronization, regression coverage and release tooling updates.

All 11 checks passed on e7963381ba. Signed release artifacts, image publication and deployment remain separate deliverables tracked by #203.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 19:30:07 +08:00
Feng Ruohang 83821f0f1f Merge pull request #216 from pgsty/codex/docs-published-design-links-20260916
docs: link retained guides to published design records
2026-09-16 18:50:28 +08:00
Feng Ruohang bf053386a4 docs: link compression and federation to maintained design records
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 18:26:03 +08:00
Feng Ruohang e7963381ba test: synchronize tag replication capacity fixtures
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 17:37:47 +08:00
Feng Ruohang 37c0edc7ca test: use canonical octal permissions in LZ4 fixture
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 17:33:58 +08:00
Feng Ruohang a2fe70424e release: prepare SILO 20260916000000 candidate
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 17:30:29 +08:00
Feng Ruohang 027d43b4eb fix: synchronize CPU resource metrics reads
Hold resourceMetricsMapMu.RLock while the v3 CPU collector reads both the
subsystem map and its nested idle/iowait metrics. The unsynchronized reads
were inherited from upstream commit f7b665347 (minio/minio#19560).

Add value and concurrent Prometheus Gather regression tests, run them in
the targeted race CI job, and record the existing CPU metric names in the
compatibility inventory.

Fixes #210

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 17:09:19 +08:00
Feng Ruohang f99ed829b5 Merge pull request #213 from pgsty/codex/pr198-release-compat
fix: preserve multipart defaults and rollback compatibility
2026-09-16 16:42:56 +08:00
Feng Ruohang 956a1a8e33 Merge pull request #212 from pgsty/codex/docs-migration-cleanup
docs: consolidate repository entry points and retire migrated work records
2026-09-16 16:39:40 +08:00
Feng Ruohang 82f0a9828e fix: preserve multipart defaults and rollback compatibility
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 16:30:43 +08:00
Feng Ruohang 584b5a3a8f docs: link the canonical recovery runbooks
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 16:19:56 +08:00
Feng Ruohang 39f49d548f Merge remote-tracking branch 'origin/main' into codex/docs-migration-cleanup
Signed-off-by: Feng Ruohang <rh@vonng.com>

# Conflicts:
#	CONTRIBUTORS.md
2026-09-16 16:13:31 +08:00
Feng Ruohang a168576adb Merge pull request #208 from pgsty/codex/conditional-put-release-notes
docs: describe conditional PUT behavior and tested recovery guidance
2026-09-16 16:09:22 +08:00
Feng Ruohang e85c4c8dfb docs: preserve reviewed contributor records and daily card references
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 16:09:00 +08:00
Feng Ruohang 0e3c43778e Merge pull request #211 from pgsty/codex/daily-repository-cards
ci: update README repository cards daily at 00:00 UTC
2026-09-16 15:40:09 +08:00
Feng Ruohang 9101fe78db ci: refresh README repository cards daily at 00:00 UTC
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 15:27:48 +08:00
Feng Ruohang 82948d6306 Merge pull request #198 from mrjavadseydi/fix/issue-79-list-multipart-uploads
Retain the original contributor commit and integrate the reviewed durable listing, cancellation confirmation and migration fixes. Capacity acceptance and delayed creation-write fencing remain open in issue #79.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 13:40:10 +08:00
Feng Ruohang dfb4b2a1d2 fix: return HTTP 503 when multipart scan capacity is exhausted
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 13:24:52 +08:00
Feng Ruohang 143f6970d8 fix: make multipart discovery and cancellation explicit and bounded
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 13:16:37 +08:00
Feng Ruohang ea5dac5a99 Merge current main into multipart listing contribution
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 12:51:20 +08:00
Feng Ruohang 3c26a8b0b5 Merge pull request #209 from pgsty/codex/credit-console-share-reporter
fix: integrate the bounded Console sharing proxy and credit its reporter
2026-09-16 12:19:29 +08:00
Feng Ruohang 2fabd436c0 fix: select the bounded Console sharing proxy
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 12:05:00 +08:00
Feng Ruohang 07c68d6054 docs: credit Jiri Pejchal for the Console sharing report
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 11:48:38 +08:00
Feng Ruohang 15090dc4fd docs: retire migrated working material and clarify documentation ownership
Remove migrated investigation and security documents, repair their incoming links, and align the English/Chinese READMEs with current module and release boundaries. Track AGENTS.md as the shared repository guide and make CLAUDE.md import it; keep working artifacts outside the repository.

Publish pgsty/silo.pgsty.com commit c7682185e2832b981629e1c17aef74fc778c6ca1 before publishing this cleanup: the new documentation routes are not live yet.

Validation: make rebrand-guard; 92 Markdown files checked for deletion-induced broken relative paths; 199 source-to-site references and anchors resolve in the paired site build; git diff --check.
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 11:22:48 +08:00
Feng Ruohang e791640dac docs: describe merged conditional PUT behavior and recovery guidance
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 11:21:16 +08:00
Feng Ruohang 9b4ae82a29 Merge pull request #207 from pgsty/codex/conditional-put-pools
fix(pools): evaluate cross-pool PUT conditions against the current object
2026-09-16 11:17:02 +08:00
Feng Ruohang 4620be394b test: synchronize conditional PUT disk fixtures
Replace unsynchronized getDisks swaps with backing disk-list updates under
erasureDisksMu, matching the existing GetDisks reader lock. Apply the same
helper to capacity and read-fault adapters while preserving nested restore
ordering.

Add a regression that overlaps fixture changes with the real IAM Walk
reader, and run the conditional PUT suite under the race detector in CI.
The regression reproduces the old fixture race; ten fixed race iterations
pass without warnings. Production conditional PUT behavior is unchanged.

Refs #199

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 10:53:24 +08:00
Feng Ruohang 5e7d603083 fix(pools): evaluate cross-pool PUT conditions against the current object
Multi-pool PUT selected a destination by capacity and evaluated
If-Match/If-None-Match only against that destination's local object
state. An empty or stale destination could accept a stale ETag or
If-None-Match:* while another pool held the current object, replacing
it; a current ETag could instead be rejected with 412 or 404.

Under PUT's existing pools-layer object lock, resolve the comparison
object with objectPoolInfos (including draining pools), treat a latest
delete marker as absence, fail closed on unreadable pool metadata, and
clear an accepted callback before destination dispatch. Replica and
data-movement callbacks keep their addressed-version semantics and
metadata reconciliation.

Reproduced on 40220bd836 and RELEASE.2026-09-03T13-18-01Z with six
signed HTTP scenarios: four defect cases failed, two controls passed.

Refs #199

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 08:21:21 +08:00
Feng Ruohang fb7c406ddc Merge pull request #206 from pgsty/codex/main-consolidation-20260916
test: restore valid credentials in integration fixtures
2026-09-16 08:18:56 +08:00
Feng Ruohang 416826f61f Merge pull request #205 from pgsty/codex/release-notes-followups
docs: complete September correctness and upgrade notes
2026-09-16 08:16:23 +08:00
Feng Ruohang a2e2f3ee82 test: restore valid credentials in integration fixtures
Port the shell-fixture changes from ebc9937d97b27871dcc4bb91d4b5771d3550b76a. Match the existing minimum secret length consistently across startup, aliases and helper commands.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 08:08:08 +08:00
Feng Ruohang 70c7ec4a9f docs: link reviewed runbook sources before website deployment
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 08:04:53 +08:00
Feng Ruohang 2b722b7e92 docs: complete September correctness and upgrade notes
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 07:51:11 +08:00
mr javad seydi 4cbb074ccd fix: make multipart upload listing S3-compatible
Signed-off-by: mr javad seydi <seydi.birjand@gmail.com>
2026-09-15 23:17:55 +03:30
Feng Ruohang 40220bd836 Merge pull request #196 from pgsty/codex/merge-r4-r8
Complete R5, R6 and R8 on the main baseline containing R4 and R7.

Preserve the individual signed repairs, independent Opus 5 Max review, and integration validation evidence.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 01:14:06 +08:00
Feng Ruohang df0dfa0a34 docs: record R4-R8 integration review and validation
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 00:58:31 +08:00
Feng Ruohang 80684fed59 chore: align integrated repair tests with contribution checks
Use the actual author and AGPL-3.0-or-later notices for new R6/R8 tests, preserving all test bodies and the Linux build tag. Record the existing replication ARN prefix used by the new R6 fixture in the compatibility inventory; no wire behavior changes.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 00:44:34 +08:00
Feng Ruohang 055030ea53 fix(http): honor configured request header deadlines
Signed-off-by: Feng Ruohang <rh@vonng.com>
(cherry picked from commit 0d48d32d7e038ae1ea5966f3d7e0cb86780a6311)
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 00:37:56 +08:00
Feng Ruohang aea3882c95 docs: record R6 integration verification
(cherry picked from commit d38edb2c46182d3a8fa96e040604493d20a4b478)
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 00:37:56 +08:00
Feng Ruohang 0c61128d23 fix(replication): retry marker purges through persisted MRF
(cherry picked from commit cf381a7151ef25fc95ace5fedcd767fa19410de2)
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 00:37:55 +08:00
Feng Ruohang 680eac66e4 fix(replication): preserve ordered tag deletions
Persist, send and reconcile empty tag states together with their revision
across COPY, PUT and multipart replication. Advance local tag mutations
under the existing locks and preserve current tags during replication ACK.

Cover signed HTTP, persistent single/multiple pool state, KMS, SSE-C key
rotation, ordering, retry and duplicate requests. Record real Opus plan
consensus, implementation review and local verification evidence.

Signed-off-by: Feng Ruohang <rh@vonng.com>
(cherry picked from commit 115fe8b12329d147adbaf817faa1737392ecbf9b)
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 00:37:55 +08:00
Feng Ruohang 9f3037e941 Merge pull request #194 from pgsty/codex/r7-replication-content-encoding
fix(replication): preserve normalized replica metadata
2026-09-16 00:31:03 +08:00
Feng Ruohang 5f00f6762f Merge main after R4 validation into R7 candidate
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 00:21:21 +08:00
Feng Ruohang af2b1794d3 Merge pull request #193 from pgsty/codex/r4-kms-tag-timestamp
fix(replication): preserve SSE-KMS tag timestamps
2026-09-16 00:20:10 +08:00
Feng Ruohang d371f77dcc docs: record final Opus 5 Max R7 implementation review
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 00:10:31 +08:00
Feng Ruohang 022722a7a7 chore: align R4 contribution notices and record final review
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 00:08:09 +08:00
Feng Ruohang 03027727d1 fix(replication): preserve SSE-KMS tag timestamps
Carry the already-parsed source tagging timestamp through the KMS options
constructor so replica COPY can apply newer tag updates on explicitly or
automatically encrypted destinations.

Cover all option encryption modes and signed COPY persistence for newer,
stale, duplicate and timestamp-less updates, including bucket defaults.
Preserve the real Opus 5.0/max plan review, consensus and local validation.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 00:03:52 +08:00
Feng Ruohang 4fcdf37ce6 fix(replication): preserve normalized replica metadata
Restore only the six replication-specific metadata fields after trust
validation, so streaming uploads retain their actual content encoding and
Snowball entries do not inherit ordinary metadata from the outer archive.

Include helper, authenticated PUT/COPY/multipart and Snowball regressions,
plus the R7 investigation, actual Opus 5 consensus and local verification.
The production change is based on PR #187 by Mikhail Khadarenka.

Co-authored-by: Mikhail Khadarenka <chodorenko@gmail.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-16 00:00:17 +08:00
Feng Ruohang 9ebe81c1b3 Merge pull request #192 from pgsty/codex/iam-revision-tombstones
fix(iam): retain revocation versions through replay and recovery
2026-09-15 23:14:22 +08:00
Feng Ruohang e5f5c9e7f6 Merge pull request #191 from pgsty/codex/iam-peer-delete-reload
fix(iam): reload committed state on peer deletion notifications
2026-09-15 23:10:38 +08:00
Feng Ruohang 7b4cacc392 chore: align IAM file notices with contribution policy
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-15 22:59:04 +08:00
Feng Ruohang a0dd7dae9b fix(iam): resume site healing after leadership changes
Keep one healing loop per process across replication configuration reloads. Reacquire leadership after a lease is canceled and allow shutdown while waiting, so temporary quorum loss cannot permanently stop revocation propagation.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-15 22:59:04 +08:00
Feng Ruohang 709d50a916 fix(iam): persist revocations across site replay and recovery
Retain source-ordered tombstones and parent grant boundaries across both IAM backends, cache reloads, and deliberate identity recreation. Reconcile deletions through a versioned, bounded replication protocol with restart-aware acknowledgements.

Cover inherited group grants, STS retention, same-key service recreation, absolute expiration, and failures after the durable commit. Document coordinated upgrades and the remaining consistency boundaries.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-15 22:59:04 +08:00
Feng Ruohang dff81f293b chore: align IAM test notice with contribution policy
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-15 22:58:52 +08:00
Feng Ruohang f653a6ea03 fix(iam): reload committed state on peer deletion notifications
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-15 22:58:52 +08:00
Feng Ruohang 47d239f84f Merge pull request #190 from pgsty/codex/fix-pool-multipart-preconditions
fix(storage): evaluate multipart preconditions across pools
2026-09-15 21:57:26 +08:00
Feng Ruohang e069fe9d92 fix(storage): evaluate multipart preconditions across pools
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-15 21:13:37 +08:00
Feng Ruohang d848fb52b5 Merge pull request #189 from pgsty/codex/fix-pool-tag-reconciliation
fix(storage): preserve tags during pool reconciliation
2026-09-15 19:38:19 +08:00
Feng Ruohang 3ce8319251 fix(storage): preserve tags during pool reconciliation
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-15 19:26:58 +08:00
Feng Ruohang 9df0f4abaf Merge pull request #188 from pgsty/codex/remove-access-tiering-final
Remove access-frequency pool tiering and preserve independent multi-pool correctness fixes. Reconcile ordinary addressed-version DELETE across pools, retaining existing quorum and compatibility boundaries.

Verified delivery head: 4d0693cb8c. All 11 final CI checks and three qualified Linux upgrade runs passed. Introduction, retirement decisions and historical unresolved observations are documented.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-15 15:09:48 +08:00
Feng Ruohang 4d0693cb8c docs: record controlled retirement evidence and qualified upgrade runs
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-15 14:56:20 +08:00
Feng Ruohang bf59e3f222 docs: record retirement execution and blocked Linux acceptance
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-15 12:16:45 +08:00
Feng Ruohang 41aa846097 style(storage): satisfy callback selection lint rule
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-15 11:31:32 +08:00
Feng Ruohang 4093fa0d78 docs: record access tiering introduction and retirement decisions
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-15 11:27:02 +08:00
Feng Ruohang 13bf126ebd fix(storage): reuse resolved copies for version DELETE callbacks
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-15 11:27:02 +08:00
Feng Ruohang 1cf529ce8a fix(storage): reconcile ordinary version DELETE across pools
Delete every copy of an explicitly addressed UUID, null version or delete
marker under the pool lock. Preserve retention and replication callbacks,
report unreadable pools and cleanup failures, and keep movement, incoming
replication, expiration and free-version cleanup on their existing paths.

Retain the separately developed general DELETE repair and replace its
access-mover-only coverage with a real interrupted rebalance copy followed
by HTTP deletion. Cover unqualified directory-marker DELETE and document
the existing pool-order-dependent 503 behavior that this makes consistent.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-15 11:27:01 +08:00
Feng Ruohang 9b76a21675 revert: remove access-frequency ILM tiering (#60)
Reverse the first-parent diff of a3df317ae0,
including the feature branch compatibility and mover follow-up fixes.
Retain the independent multi-pool correctness fixes from #178 and migrate
their shared test fixture away from access-tier code.

Tolerate retired ILM keys and XML, read old v9 statistics while writing v8,
and document migration without moving objects or rewriting their metadata.
Include regression coverage using a historical scanner/writer v9 fixture.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-15 11:27:01 +08:00
Feng Ruohang 89637554d6 Merge pull request #182 from pgsty/codex/docs-current-state-20260913
docs: align release notes and current component status
2026-09-13 10:50:22 +08:00
Feng Ruohang 2dd1e00da4 docs: align release notes and current component status
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-13 10:39:57 +08:00
Feng Ruohang 5d955b5b74 Merge pull request #181 from pgsty/codex/deps-release-20260913
Coordinate September dependency releases and replace vulnerable bundled curl
2026-09-13 10:14:57 +08:00
Feng Ruohang f7808a172c deps: pin the coordinated September 13 SILO stack
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-13 09:54:51 +08:00
Feng Ruohang acc9b514f5 Follow the curl builder rename in delivery verification
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-13 09:08:38 +08:00
Feng Ruohang 31ba01c5d5 Refresh Go dependencies and build current static curl for both architectures
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-13 09:01:09 +08:00
Feng Ruohang 48ec10312f Merge pull request #180 from pgsty/codex/issue-77-metadata-convergence
fix(replication): preserve bucket metadata source times and converge deletions
2026-09-12 18:07:36 +08:00
Feng Ruohang 114dc10529 docs: archive issue 77 design, adversarial reviews and acceptance evidence
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-12 17:52:25 +08:00
Feng Ruohang 461e9a7210 test(replication): satisfy diagnostic regression style checks
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-12 17:17:18 +08:00
Feng Ruohang fcbb93e895 fix(replication): diagnose unusable metadata without a heal source
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-12 17:09:34 +08:00
Feng Ruohang 62cf066ff5 fix(replication): recover physical creation time and align policy status
An independent adversarial review of the bucket metadata convergence
work found three defects it had introduced.

GetBucketInfo overwrote the physical creation probe with cached
metadata, which a bucket that never held a configuration legitimately
lacks. The new creation-time requirement then failed every policy, tag,
SSE, quota, versioning and Object Lock write on such a bucket, with no
operator recovery path, and initial synchronization skipped it silently.
Return the physical result unchanged when metadata is not requested, as
ListBuckets already does, recover the time during initial
synchronization, and pass it to MakeBucketHook so peers adopt the same
bucket generation.

Replication status compared parsed policies statement by statement while
heal compares the canonical key. An upgraded peer that stored an
equivalent statement order was therefore reported as mismatched forever,
and heal never had anything to write. Compare the key heal compares;
per-site presence counting is unchanged.

Heal diagnostics shared one log key across four conditions, so a real
peer RPC failure could be deduplicated away by an earlier message, and
they were logged at error level for the normal transient of a peer that
does not have the bucket yet. Give each reason its own key at warning
level, report only a field state that exists and still cannot be
ordered, and diagnose nothing when no site holds a state to propagate.

The recovery test now runs against the real ObjectLayer; the stub it
replaced returned the expected time and hid the defect. The policy
status test uses a statement order the canonical encoder reorders, and
adoption coverage is extended past a real field time.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-12 14:01:35 +08:00
Feng Ruohang 1ee64a8d89 fix(replication): gate metadata tombstone export during rolling upgrades
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-12 13:50:55 +08:00
Feng Ruohang 01aaef2b50 fix(replication): converge bucket metadata using deterministic source states
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-12 13:50:55 +08:00
Feng Ruohang bcc62afe3d fix(replication): apply bucket metadata source times under the metadata lock
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-12 13:50:55 +08:00
Feng Ruohang 5c57658163 Merge pull request #179 from pgsty/codex/federation-copy-fixes
fix(federation): preserve committed copy times and reject raw SSE-C replicas
2026-09-12 02:00:57 +08:00
Feng Ruohang f175e98c34 fix(federation): bind copy timestamps to committed writes
Return the committed object or part time on federation write responses and
capture it for CopyObject and UploadPartCopy without a follow-up read.
Keep successful writes compatible with targets that do not supply a time.

Reject authenticated raw SSE-C replica CopyObject across deployments before
forwarding. Extend the existing SSE fixtures to verify empty objects,
stored checksums, KMS contexts and multipart sources.

Refs: #169, #168, #171
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-12 01:44:22 +08:00
Feng Ruohang 12f631b502 Merge pull request #178 from pgsty/codex/multipool-correctness-20260911
fix(storage): reconcile multi-pool writes and conditional deletes
2026-09-11 20:34:58 +08:00
Feng Ruohang ccb676e60c fix(storage): preserve shared tier references during pool cleanup
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-11 20:23:40 +08:00
Feng Ruohang 51d41345f7 test: record Linux restart and OIDC release acceptance
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-11 20:05:56 +08:00
Feng Ruohang e59a3d938e fix(storage): serialize and reconcile multi-pool object updates
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-11 20:05:56 +08:00
Feng Ruohang b32f2d9dd0 Merge pull request #177 from pgsty/codex/release-consolidation-20260911
Fix signed payloads, Object Lock and federated copies; secure AMQP
2026-09-11 17:08:34 +08:00
Feng Ruohang d63c92e393 build(deps): secure AMQP frames and select merged Console
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-11 16:56:33 +08:00
Feng Ruohang 68127c5a63 build(deps): embed the consolidated Console source
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-11 16:40:42 +08:00
Feng Ruohang c4b5e1cb45 fix(auth): enforce header-only presigned payload checksums
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-11 16:24:42 +08:00
Feng Ruohang 9a303f5096 fix(object): preserve retention and federated copy destination state
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-11 16:24:42 +08:00
Feng Ruohang 87d8b5967f fix(auth): align signed request and policy condition semantics
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-11 16:24:42 +08:00
Feng Ruohang f760046c44 Merge remote-tracking branch 'origin/main' into codex/release-consolidation-20260911
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-11 16:13:42 +08:00
Feng Ruohang 93e7ef4bcc Merge pull request #175 from pgsty/codex/upstream-sdk-password-20260910
fix(iam)!: split self-service password permissions and update SDK stack
2026-09-10 17:51:08 +08:00
Feng Ruohang a4229b366f build(deps): align final coordinated SILO source pins
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-10 17:39:01 +08:00
Feng Ruohang 420340bc14 docs(iam): explain breaking password-policy semantics
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-10 16:58:21 +08:00
Ayush Sharma e7e87402ed fix: forward the legal hold as an Object Lock header on federated CopyObject
A cross-deployment CopyObject that requests
`x-amz-object-lock-legal-hold: ON` answered 200 while the destination
carried no hold. The resolved value reached the remote as ordinary user
metadata, `X-Amz-Meta-X-Amz-Object-Lock-Legal-Hold`, so nothing applied it.
Retention requested on the same copy survived, which is what made the loss
easy to miss.

The federation branch passes the resolved metadata map straight to
`Core.PutObject` as `PutObjectOptions.UserMetadata`. minio-go's `Header()`
writes the typed lock fields first, then prefixes every UserMetadata key it
does not recognise with `x-amz-meta-`; `supportedHeaders` covers
`x-amz-object-lock-mode` and `x-amz-object-lock-retain-until-date` but not
`x-amz-object-lock-legal-hold`, and `isAmzHeader` does not match it either.
Retention therefore arrives as real headers and the hold does not. The
high-level `validate()` that would have rejected the key never runs, because
`Core.PutObject` goes straight to the low-level PUT.

Carry the hold on the typed `LegalHold` option and forward a cloned map with
the raw key removed. The clone matters twice: typed fields are written before
the UserMetadata loop, so a leftover raw key would add a bogus `x-amz-meta-`
entry beside the correct header, and the proxy's own response and event
metadata are rebuilt from the resolved values rather than the forwarding map,
which no longer carries the hold.

Retention stays in the map deliberately. It already passes through as a
standard header, and moving it to the typed `RetainUntilDate` field would
format with `time.RFC3339` and truncate a retain-until date to whole seconds.

The new test asserts the wire: the remote must receive
`X-Amz-Object-Lock-Legal-Hold` and never the `x-amz-meta-` spelling, and the
destination version must actually store the hold. It fails without the change
with "legal hold forwarded as user metadata [ON]".

Fixes #166

Signed-off-by: Ayush Sharma <72848455+Aeirx@users.noreply.github.com>
2026-09-10 13:15:10 +05:30
Feng Ruohang a164e1dda1 fix(iam): separate password changes and refresh coordinated SDK dependencies
Use ChangeMyPassword for the authenticated user and keep CreateUser for other users. Coordinate silo-pkg b3760f56ec23, mcli fa22b40b4eb7, Console 1b95b6cec652 and upstream minio-go 78bfa91607c2. Add legacy-policy and SDK streaming regressions plus upgrade guidance.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-10 15:16:21 +08:00
Feng Ruohang 2f61325d4a Merge pull request #174 from pgsty/codex/contributor-orenyomtov-20260910
docs: credit @orenyomtov for the SN-2026-011 report
2026-09-10 11:34:52 +08:00
Feng Ruohang a6145e1d2e docs: credit @orenyomtov for the SN-2026-011 report
Add Oren Yomtov (github.com/orenyomtov) to the contributor wall in
README.md, README_ZH.md, and CONTRIBUTORS.md, and to the Issue reports
table, for the private disclosure of the unsigned-header CopyObject
cross-object read fixed as SN-2026-011 (#173). Community contributor
count 40 -> 41.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-10 11:24:12 +08:00
Feng Ruohang 5232546690 Merge pull request #173 from pgsty/codex/unsigned-amz-header-copy-20260909
fix(auth): reject unsigned x-amz-* headers to close CopyObject confused-deputy (SN-2026-011)
2026-09-10 10:10:49 +08:00
Feng Ruohang 0c46cb641b docs(security): record SN-2026-011 (unsigned x-amz-* header CopyObject)
Ledger entry for the confused-deputy fix in 123325430: an unsigned
x-amz-copy-source header turned a presigned or signed PUT into a
server-side copy of any object the signing key can read. Reported by
Oren Yomtov; inherited from upstream minio/minio; CVE requested.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-10 09:57:26 +08:00
Feng Ruohang 1233254309 fix(auth): reject unsigned x-amz-* headers to close CopyObject confused-deputy
A presigned or signed PUT authorized for a single object could be turned
into a server-side copy of any object the signing key can read by adding
an unsigned x-amz-copy-source header, executed as the signer. SigV4
verification only walked the signed-headers list, never the headers that
actually arrived; the meta-header check matched only X-Amz-Meta- and ran
only on the presigned path, so an unsigned x-amz-* header outside the
list was never seen while the router still dispatched the PUT to
CopyObjectHandler.

Reject any x-amz-* request header not covered by the signed headers, on
both the presigned (doesPresignedSignatureMatch) and Authorization-header
(doesSignatureMatch) paths, matching AWS S3. The check tests membership
in the signed set rather than value equality, so a header whose first
value is empty (e.g. {"", "/src/secret"}) cannot slip through.
X-Amz-Content-Sha256 is exempt (payload hash: read from the query for
presigned requests and bound into the string-to-sign for signed ones, so
it is self-protected) and X-Amz-Signature-Age is exempt (an internal
scratch header written after verification, so repeated verification of
the same request stays idempotent). The synthesized X-Amz-Tagging header
in PutObjectTagging is now injected after signature verification.

Tests that previously added x-amz-copy-source and friends after signing
(relying on the vulnerable behavior) now re-sign, mirroring real S3
clients. Adds checkUnsignedHeaders unit cases and TestPresignedVerifyIdempotent.

Reported by Oren Yomtov. Inherited unchanged from upstream minio/minio.
Tracked as SN-2026-011.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-10 09:57:01 +08:00
Feng Ruohang 8a2fe9b7a0 Merge pull request #164 from pgsty/codex/main-consolidation-20260909
fix: honor TLS defaults and accept valid bucket metadata reloads
2026-09-09 19:49:08 +08:00
Feng Ruohang f26ee6bf0a test: cover metadata reload with equal maximum timestamps
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-09 19:35:13 +08:00
Feng Ruohang d69c4ccfe4 Merge TLS default key exchange compatibility fixes
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-09 19:31:35 +08:00
Feng Ruohang 086e619505 Merge accepted bucket metadata reload correction
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-09 19:31:10 +08:00
Feng Ruohang 48e1846525 fix(tls): honor Go key exchange defaults across transports
Remove the eight explicit curve overrides so Go 1.27 honors tlsmlkem=0
across Server listeners, node links and outbound transports. Remove the
unused shared curve option and add wire-level regression coverage.

Document CA trust and TLS upgrade behavior, retain the investigation
artifacts, and exclude their synthetic routes from the rebrand guard.
The product compatibility baseline remains unchanged.

Validation: focused race tests, HTTP tests, lint, compatibility guard
positive/negative controls, and a fresh Linux build with three isolated
OIDC integration scenarios all pass.

Adversarial review: Claude Code Fable 5.1, max effort.
Final verdict: APPROVE FOR COMMIT.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-09 18:50:12 +08:00
Feng Ruohang bcc8871b1d Merge pull request #163 from pgsty/fix/issue-158-federated-copy-sse
fix: forward plaintext on federated CopyObject of SSE objects (#158)
2026-09-09 18:02:45 +08:00
Feng Ruohang 25cb3511c9 test: cover multipart SSE sources on federated CopyObject
A multipart SSE-S3 source is encrypted per part, so its logical size is
the sum of the parts' decrypted sizes and the decrypting reader crosses
a part boundary. Copy such a source across the federation to a plain
and to an SSE-S3 destination and check the destination plaintext and
single-encryption size.

The test router registers routes in endpoint order and the plain
PutObject route has no query matcher, so the multipart endpoints are
listed first.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FodsDpa6VkghaeRE6WjmEe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-09 17:51:32 +08:00
Feng Ruohang cfefc049c1 fix: send the SSE-KMS context as a JSON object on federated copies
putOptsFromReq handed the parsed kms.Context straight to
encrypt.NewSSEKMS. kms.Context implements encoding.TextMarshaler, so the
SDK serialized it as a JSON string, and a request without a context
still produced one because the nil Context is a typed nil inside the
interface value and marshals to "{}". The receiving ParseHTTP rejects
both forms, so every federated CopyObject to an SSE-KMS destination
failed with InvalidArgument once the forwarded stream was correct.

Pass a plain map, or nothing when no context was requested, and cover
SSE-KMS destinations with and without an explicit context.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FodsDpa6VkghaeRE6WjmEe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-09 17:51:32 +08:00
Feng Ruohang af56d17630 fix: forward plaintext on federated CopyObject of SSE objects (#158)
The legacy etcd bucket-federation branch of CopyObjectHandler reads its
source through getObjectNInfo, which yields the decrypted and
decompressed bytes, but it also ran the destination encryption locally
and then forwarded that stream to the remote PutObject with the source's
stored size and the destination SSE option. SSE to plain and plain to
SSE therefore failed on a Content-Length mismatch, while SSE to SSE
matched by coincidence: the remote encrypted the ciphertext a second
time and stored an unreadable object, and a destination GET returned
the inner ciphertext with HTTP 200.

The remote write owns the destination's storage transformations, so
hand it the logical bytes at their logical size and let it encrypt
exactly once. Compression was already excluded on this branch; apply
the same rule to encryption, size the forwarded reader by actualSize,
and declare that size on the forwarded PutObject.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FodsDpa6VkghaeRE6WjmEe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-09 17:51:32 +08:00
Feng Ruohang d1105bbb3d Merge pull request #162 from pgsty/codex/replication-reliability-20260909
fix(replication): complete purges, expose MRF drops, and cancel resyncs reliably
2026-09-09 14:55:35 +08:00
Feng Ruohang 66fe61ff65 test: reuse replication compatibility fixtures
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-09 14:44:02 +08:00
Feng Ruohang a1141a43f2 style: format replication regression fixture
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-09 14:37:24 +08:00
Feng Ruohang 702f113f51 fix(replication): scope resync cancellation and drain worker lifecycle
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-09 14:37:02 +08:00
Feng Ruohang 63aace4099 fix(replication): expose bounded MRF queue drops
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-09 14:37:02 +08:00
Feng Ruohang 9d7094b770 fix(replication): complete single-object delete marker purges
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-09 14:37:02 +08:00
Feng Ruohang 450dcb8484 Merge pull request #161 from pgsty/codex/server-dependency-refresh-20260909
build: refresh maintained SILO stack dependencies
2026-09-09 12:04:16 +08:00
Feng Ruohang 4074d00b96 build: refresh maintained SILO stack dependencies
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-09 11:49:13 +08:00
Feng Ruohang 75ba0ce402 fix: drop the broken bucket-metadata reload publication guard (issue #105 T3)
The #105 T3 change (PR #156) tried to keep the resident metadata cache
monotonic by guarding peer-reload publication on lastUpdate(). But
lastUpdate() is the max of per-config timestamps and cannot order whole
records: a node caching {policy@20, CORS@10} that receives a newer CORS@15
still has lastUpdate()==20, so the guard rejects the legitimately-newer
record and the periodic refresh (same comparator) cannot repair it. A
paused reload could also resurrect deleted resident state.

Per the maintainer decision, revert the reload publication to its original
unconditional (acceptable-until-refresh) behavior:
- remove setReloaded and restore the plain Set plus notification/target
  registry updates in LoadBucketMetadataHandler;
- restore refreshBucketsMetadataLoop's own lastUpdate() staleness check and
  globalEventNotifier.set / globalBucketTargetSys.set publication;
- restore the unconditional GetConfig cache-miss publication;
- document the known freshness limitation at the reload site (the periodic
  refresh is best-effort and cannot repair an equal-maximum-timestamp
  divergence).

The T1 lifecycle merge-under-lock (UpdateExpiryLCConfig) and both T2 fixes
(DeleteBucket takes metadata.lock before deleting; saveMetadata and
loadBucketMetadataParseUnderLock recheck physical bucket existence) are
kept fully intact.

Tests:
- drop the T3 reproductions (overlapping-reload resident-cache test and the
  peer-reload-preserves-current-targets publication test);
- add lockBucketMetadataAcquireHook, a nil-in-production atomic test hook in
  the shared metadata.lock path, so tests can deterministically observe a
  caller (notably DeleteBucket, whose lock is taken through its
  erasureServerPools receiver and is invisible to an injected object layer)
  reaching the lock;
- rewrite the T2 delete-race ghost test to hold metadata.lock MID-SAVE (past
  saveMetadata's existence recheck) and synchronize on the delete's actual
  lock attempt via the hook, so it isolates the lock-before-delete fix:
  removing only DeleteBucket's metadata.lock (recheck kept) now fails it;
- rewrite the cancellation test to observe the delete's actual lock attempt,
  then cancel and await its error while still holding the lock, so a
  scheduling-delayed delete stopped by the canceled context can no longer
  pass on a broken tree.

Refs #105. Follows #156.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-08 16:13:54 +08:00
Feng Ruohang da142327f2 Merge pull request #159 from pgsty/fix/issue-99-100-followups
fix: repair federated CopyObject checksum edge cases (empty body, inherited, multipart-suffix)
2026-09-08 15:58:22 +08:00
Feng Ruohang a3df317ae0 Merge pull request #60 from mrjavadseydi/feat/access-based-ilm
[ILM] Relocate hot objects across server pools by GET frequency
2026-09-08 15:42:15 +08:00
Feng Ruohang 2c50d11f72 fix: correct federated CopyObject checksum edge cases (#99 follow-ups)
Three residual checksum defects in the legacy etcd federation branch of
CopyObjectHandler, found by post-merge review of #157.

1. Empty-source 500 regression. A checksum-less object gains the S3 default
   CRC-64NVME (WantServerSideChecksumType is set), but minio-go streams no
   trailing checksum for a 0-byte body (contentLength == 0), so the remote
   computed none, federatedChecksumValue was empty, hash.NewChecksumWithType
   returned nil, and the handler returned 500 -- so every empty-object
   federated copy failed. For a 0-byte source, forward the empty-content digest
   as an ordinary checksum request header instead, so the remote validates,
   persists and returns it, matching the local path (e.g. CRC32 "AAAAAA==").

2. Inherited full-object checksum dropped. When the source already carries a
   full-object checksum, the local path sets dstOpts.WantChecksum, not
   WantServerSideChecksumType (only multipart-composite sources are promoted).
   The federated branch inspected only WantServerSideChecksumType, so a
   checksum-bearing source's checksum was silently discarded on a federated
   copy that requested no algorithm. Forward WantChecksum.Encoded (always a
   plain digest) as a checksum header so the remote validates and persists it,
   and bind the returned value, matching local persistence.

3. Multipart-suffixed remote value accepted. The bind accepted a value like
   "NSRBwg==-0": NewChecksumWithType parses the "-N" as ChecksumMultipart with
   WantParts 0 and the length-only validator passes, so the destination was
   returned as COMPOSITE. A single forwarded PutObject must yield a full-object
   digest, so reject a multipart-marked parsed value in addition to the
   existing nil (missing/malformed) rejection.

A forwarded checksum request header is stripped from objInfo.UserDefined so it
is not mistaken for object metadata.

Out of scope: the SSE federated-copy corruption (srcInfo.Reader/Size mismatch
for encrypted sources) predates this work and is filed separately.

New federated regressions cover empty source with requested and default
checksum (200 + correct value + persisted), an inherited full-object checksum
preserved without a requested algorithm, and a multipart-suffixed remote value
rejected. Red/green verified for each against the merged code.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-08 15:38:11 +08:00
Feng Ruohang 5ac33e1583 test: make access move failure recovery deterministic
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-08 15:28:33 +08:00
Feng Ruohang d57c4e8407 Merge remote-tracking branch 'origin/main' into codex/access-tiering-ci-20260908
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-08 15:08:34 +08:00
Feng Ruohang 374de0fa32 fix: make access tier moves preserve versions and isolate writes
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-08 15:08:34 +08:00
Feng Ruohang 80e23dc9f2 Merge pull request #156 from pgsty/codex/bucket-metadata-merge-20260908
fix: preserve bucket metadata across concurrent updates and reloads
2026-09-08 15:03:52 +08:00
Feng Ruohang 2cc0e3c6ed test: construct notification fixtures with the event ARN type
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-08 14:52:48 +08:00
Feng Ruohang f9da3b919d fix: order bucket deletion and metadata publication safely
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-08 14:43:15 +08:00
Feng Ruohang 1309853f57 Merge remote-tracking branch 'origin/main' into codex/access-tiering-ci-20260908
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-08 14:34:51 +08:00
Feng Ruohang 39b8e6c30a Merge remote-tracking branch 'origin/main' into codex/bucket-metadata-merge-20260908
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-08 14:24:22 +08:00
Feng Ruohang 9a6e1477f4 ci: record access tiering compatibility identifiers
Record the ten documented MINIO_ILM_ACCESS settings, the internal object
metadata stamp, and the tracker storage-path suffix introduced by this PR.
The guard places the /ilm/access string in its routes set, but the value
is a component of the tracker object prefix, not a public HTTP endpoint.

Keep all existing compatibility entries. The guard, delivery rebrand check,
and Docker entrypoint compatibility tests pass with the refreshed manifest.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-08 13:18:50 +08:00
Feng Ruohang 49375ed2d3 Merge pull request #157 from pgsty/codex/federated-copy-merge-20260908
fix: preserve metadata and checksums in federated object copies
2026-09-08 13:16:30 +08:00
Feng Ruohang f817b5261c Merge pull request #151 from nikitapogromsky/fix/idempotent-add-system-target
logger: make AddSystemTarget idempotent
2026-09-08 13:14:30 +08:00
Feng Ruohang 885bd2c20a fix: reject missing remote checksums on federated copies
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-08 13:05:45 +08:00
Feng Ruohang 89b75e7913 Merge branch 'main' into feat/access-based-ilm 2026-09-08 13:04:25 +08:00
Feng Ruohang 079ebb1926 fix: serialize logger initialization and publish the console target safely
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-08 12:58:21 +08:00
Feng Ruohang 04c29aac11 Merge branch 'audit/issue-105-bucketmeta-races' into codex/bucket-metadata-merge-20260908
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-08 12:55:51 +08:00
Feng Ruohang 95e7a190b3 Merge branch 'fix/issue-99-100-federated-copyobject' into codex/federated-copy-merge-20260908
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-08 12:55:51 +08:00
Feng Ruohang 8e2392e48f Merge remote-tracking branch 'origin/main' into codex/logger-idempotency-20260908
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-08 12:55:50 +08:00
Feng Ruohang 3abe0d95a5 Merge pull request #149 from pgsty/codex/embedded-console-compat
fix: restore embedded Console proxy and WebSocket configuration
2026-09-08 10:09:21 +08:00
Feng Ruohang 9c6c9805de fix: select the validated Console mainline for embedding
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-08 10:00:45 +08:00
nikitapogromsky 5cb900bfad logger: make AddSystemTarget idempotent
Subscribe re-registered the console target on every console-log subscription, producing duplicate minio_logger_webhook_* series on each /minio/metrics/v3 scrape. Fixes #150

Signed-off-by: nikitapogromsky <129324283+nikitapogromsky@users.noreply.github.com>
2026-09-07 12:03:39 +03:00
Feng Ruohang 0af5d22286 fix: restore embedded Console proxy and WebSocket configuration
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-07 13:36:24 +08:00
Feng Ruohang f1687f402b fix: scope rebrand checks to the retired repository
Match the exact retired repository while preserving links to distinct repositories and historical issue titles in contributor credits. Continue rejecting live links to the retired repository in those credits.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-07 11:49:23 +08:00
Feng Ruohang ce606df2c4 fix: align contribution tooling with SILO AGPL policy
Clarify SILO contribution ownership and preserve prior copyright notices. Consolidate issue templates and route the legacy credits command through the maintained generator.

Validation: make rebrand-guard; bash -n update-credits.sh; regenerated credits match CREDITS; template and link checks.
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-07 11:33:21 +08:00
Feng Ruohang fd44dc4e9b docs: include the latest merged community contribution
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-07 11:01:04 +08:00
Feng Ruohang 479745e764 docs: credit contributors across the SILO repositories
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-07 10:58:47 +08:00
Feng Ruohang cc1c54475f Merge pull request #146 from pgsty/fix/issue-108-console-loopback-tls
fix: keep embedded Console login working over loopback TLS (#108)
2026-09-07 10:53:45 +08:00
Feng Ruohang e7654d470c fix: close residual bucket-metadata races (issue #105 audit)
Audit of the three deferred #105 follow-ups. Each reproduces with a
deterministic red test in cmd/bucket-metadata-race_test.go, and each fix is
the minimal change that turns its test green while preserving the
<bucket>.lck -> metadata.lock -> .metadata.bin lock order established by #103.

1. Lifecycle expiry merge lost update (persistent). PeerBucketLCConfigHandler
   and healBucketILMExpiry read the current lifecycle with an unlocked
   GetConfigFromDisk, merged the replicated expiry rules with the local
   transition rules, then wrote the pre-computed blob via Update. Any lifecycle
   transition change committed between the merge read and the merge write was
   silently lost. New BucketMetadataSys.UpdateExpiryLCConfig performs the read,
   merge, and save under one metadata.lock; mergeExpiryWithLCConfig now takes
   the locked snapshot and validates object-lock retention from it instead of
   re-reading (avoids a re-entrant metadata load under the lock).

2. DeleteBucket ghost .metadata.bin (persistent). DeleteBucket took only
   <bucket>.lck while config writers take only metadata.lock, so a writer that
   was mid-save could re-create .metadata.bin after the prefix purge. The purge
   now runs under metadata.lock, with a best-effort unlocked fallback so a
   delete is never blocked from completing.

3. Overlapping peer reloads publishing a stale resident cache (freshness only;
   the persisted record stays correct). LoadBucketMetadataHandler and the
   GetConfig cache-miss path published with an unconditional Set, so a reload
   that read an older revision could overwrite a newer resident record until the
   next refresh. New BucketMetadataSys.setReloaded (and a matching GetConfig
   guard) refuses to regress a newer resident record, mirroring
   refreshBucketsMetadataLoop.

Verification: go build -tags kqueue,dev ./...; go vet ./cmd; gofmt clean;
rebrand-guard baseline unchanged; go test -tags kqueue,dev ./cmd (207s) green;
new tests plus the #103 metadata suite green under -race.

Refs #105. Parent #102. Foundation #103.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-07 10:33:11 +08:00
Feng Ruohang 4b25f7e819 fix: return and persist checksum on federated CopyObject (#99)
The legacy etcd federation branch of CopyObjectHandler forwards the copied
bytes with minio-go Core.PutObject but never asked the remote for a checksum
and discarded any it returned, so a cross-deployment whole-object copy that
requested a checksum returned 200 with an empty checksum, and a checksum-less
source did not gain the S3 default CRC-64NVME that the local path assigns. The
request was neither honored nor rejected. This is the whole-object counterpart
of #72, which repaired the same class of defect for federated UploadPartCopy.

When a server-side checksum is wanted -- explicitly requested, inherited from a
multipart source, or the CRC-64NVME default for a checksum-less object, all
already captured in dstOpts.WantServerSideChecksumType -- the forwarded
PutObject now streams a trailing checksum of that type, so the remote computes
and persists it and echoes it in the response. The value the remote reports for
that exact write is bound into objInfo.Checksum, matching how the local
CopyObject path carries checksums into the CopyObjectResult. Reading the value
from the same UploadInfo that produced the ETag keeps the pair bound to one
write.

Only the requested algorithm is returned; a malformed or absent remote value
leaves objInfo.Checksum unset, so an ordinary copy that wanted no checksum
still returns none. Two small mapping helpers convert between the server's
hash.ChecksumType and the minio-go request type and response field.

New end-to-end tests drive the real federation branch through
getRemoteInstanceClient and minio-go into a second in-process deployment and
assert that CRC32/CRC32C/SHA256/CRC64NVME and the no-algorithm default are all
returned in the CopyObjectResult and persisted on the destination, and that a
requested algorithm never leaks other algorithms into the response.

Fixes #99

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-07 10:32:59 +08:00
Feng Ruohang 711b092f86 fix: keep embedded Console login working over loopback TLS (#108)
The embedded Console reaches the S3/STS API at https://127.0.0.1:<port>
(minioConfigToConsoleFeatures), a loopback endpoint whose TLS certificate is
not expected to carry a 127.0.0.1 SAN. silo-console v2.3.x began verifying
every outbound TLS peer, so the Console's STS AssumeRole handshake to that
loopback endpoint now fails certificate validation and BOTH local and LDAP
logins fail with a generic "invalid login". The failure happens in the Console
HTTP client before any request reaches a server auth/STS/LDAP handler, so no
server-side auth error is logged, matching the report.

Restore the documented loopback bypass by opting the embedded Console into its
endpoint-scoped CONSOLE_MINIO_SERVER_TLS_SKIP_VERIFY switch whenever the server
falls back to the 127.0.0.1 endpoint under TLS. The exemption is scoped to that
single loopback origin inside Console; every other HTTPS peer (IdP, Prometheus,
webhooks) stays verified, preserving the v2.3.x hardening. An explicitly
configured endpoint is reached under its own verified name and is never
exempted. initConsoleServer unsets CONSOLE_* before re-deriving them, so the
switch cannot be supplied by the operator on the embedded path; the server must
assert it.

Fixes #108

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-07 10:32:59 +08:00
Feng Ruohang 2bc103b80c fix: strip all reserved metadata on federated CopyObject (#100)
The legacy etcd federation branch of CopyObjectHandler forwards the copied
source metadata to the remote deployment with minio-go Core.PutObject, after
removing only two reserved keys (compression and actual-size). Every small
object is stored inline, so its stored metadata also carries
X-Minio-Internal-inline-data; the remote's setRequestLimitMiddleware rejects
any request bearing a reserved-prefix header (containsReservedMetadata), so
the forwarded write failed with 400 InvalidArgument "Your metadata headers
are not supported." for the default COPY metadata directive.

A plain federated PutObject must not carry any internal storage metadata, so
strip the whole reserved-prefix class before forwarding instead of an
enumerated subset. Enumerating a third key would only defer the next leak:
besides inline-data, replication bookkeeping (replica/replication status and
timestamps) is added to the same map earlier in the handler and would be
rejected just the same. None of these keys is required by the remote for a
correct plain PutObject; they are internal storage details the remote sets
for itself. The stripping uses stringsHasPrefixFold, matching the remote's
own case-insensitive detection. Ordinary user metadata (x-amz-meta-*) is
untouched and still copied.

The pre-existing UUID ETag on the federated write (no Content-MD5 is sent) is
out of scope and left unchanged, as recorded in the issue.

A new end-to-end test drives the real federation branch through
getRemoteInstanceClient and minio-go into a second in-process deployment,
copying an inline source with the default COPY directive. It asserts the copy
now succeeds, that no reserved-prefix header reaches the remote on any
forwarded request, and that copied user metadata survives.

Fixes #100

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-07 10:32:59 +08:00
Feng Ruohang ad873c7357 Merge pull request #132 from mrjavadseydi/fix/issue-106-bucket-quota-metrics
fix: report effective bucket quotas in metrics
2026-09-07 09:54:01 +08:00
Feng Ruohang 0af0907eff Merge pull request #134 from pgsty/fix/issue-120-ssec-replica-retransmit
fix: retransmit and re-order Object Lock for SSE-C replicas (single erasure set)
2026-09-07 00:24:45 +08:00
Feng Ruohang e27ba2bc14 Merge pull request #131 from pgsty/fix/issue-117-lock-resend-compare
fix: stop re-replicating an object whose retention was removed
2026-09-07 00:22:21 +08:00
Feng Ruohang 236e163c0b fix: repair an undecodable SSE-C replica on retransmit
PutObjectHandler's precondition callback ran DecryptObjectInfo on the
stored object before checkPreconditionsPUT, so an authenticated raw
SSE-C replica overwrite was rejected when the stored version could not
decrypt. A replica a pre-fix destination (issue #109) left as
compress(ciphertext) or a re-encrypted body has an invalid decrypted
length, so DecryptObjectInfo returned errObjectTampered and the
retransmission that repairs it never ran -- the version stayed damaged
through resync. #134's raw-replica exemption only covered the
version/ETag duplicate check inside checkPreconditionsPUT, one step too
late.

Skip the stored object's decryption precondition only for a PURE raw
SSE-C replica overwrite (a trusted SSE-C replica write with no public
precondition), keyed on the incoming request's restored SSE-C metadata,
the same predicate checkPreconditionsPUT uses. Such a write fully
replaces the object, so requiring the damaged stored version to decrypt
is both wrong and unnecessary. A conditional request keeps the check:
DecryptObjectInfo also normalizes the stored sealed ETag to the
client-visible one, and If-Match/If-None-Match must compare against
that, not the sealed ETag -- skipping it for every replica inverted both
conditions. Ordinary writes and non-SSE-C replicas are unchanged.

Adds red/green regressions: a raw retransmit over a version staged as an
undecodable body returns 500 XMinioObjectTampered before this change and
200 with full customer-key recovery after; and a conditional replica PUT
(If-Match / If-None-Match) on the client-visible ETag is honoured rather
than inverted. Fixes the single-PUT compression-damage recovery gap in

Signed-off-by: Feng Ruohang <rh@vonng.com>
#120.
2026-09-07 00:11:04 +08:00
Feng Ruohang 7220210e8d test: reconcile #134 fixtures with #119 and refresh the rebrand baseline
Rebased onto current main. #119 made PutObjectPart derive an encrypted
part's plaintext length and reject a part that cannot be a valid sio
stream, so TestReplicaLockReconcileNullVersion's completeNullMPU fixture
(a 4-byte plaintext part under SSE-C metadata) no longer stores; build it
with sio.Encrypt like the other encrypted-part fixtures. Also regenerate
the rebrand-guard baseline for the replication SSE header the retransmit
path reintroduces (headers 84 -> 85). Mechanical integration only.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-07 00:11:04 +08:00
Feng Ruohang 34cbca97ea docs: point the multi-pool lock follow-up at pgsty/silo#133
Fill the tracked-issue number into the scope comments of the single
erasure set Object Lock reconcile added for SSE-C replica retransmit.
No behaviour change.

Refs pgsty/silo#120
Refs pgsty/silo#133

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-07 00:11:04 +08:00
Feng Ruohang 109d824e5f fix: retransmit and re-order Object Lock for SSE-C replicas (single erasure set)
Issue #120 routes an existing SSE-C replica through PutObjectHandler and
NewMultipartUploadHandler. On main those handlers assigned the incoming retention
and legal hold directly, without the source-timestamp ordering #111 added to
CopyObjectHandler and without persisting the ordering timestamps, so in
active-active replication a retransmit carrying an older value could overwrite a
destination version's newer lock state.

Share #111's ordering decision as applyReplicatedObjectLock in
cmd/bucket-object-lock.go and call it from CopyObject, PUT and multipart
initiation. A request that is not an actual trusted replica keeps ordinary write
semantics (a validated value is applied and stamped now); only a real replica
update is ordered against the stored version, so a marker-only peer write no
longer drops a validated hold or default retention. CopyObject keeps its SSE-C
key-rotation encMetadata reconciliation inline. putReplicationOpts now emits a
stored retention ordering timestamp even when the value keys are absent, so a
removal recorded on the retransmit PUT path still replicates onward. replicateAll
marks Failed and carries the error when putReplicationOpts fails.

The handler decision is made against the version as it stands then, which a
concurrent lock update can outrun before the write commits, and for multipart
across the whole initiation-to-completion span. Close that window under the
object write lock the receiving erasure set holds: a trusted SSE-C replica full
write sets ObjectOptions.ReplicaLockReconcile, and erasureObjects.PutObject and
CompleteMultipartUpload re-run the ordering (reconcileStoredObjectLock, which
orders retention and legal hold independently by their reserved timestamps)
against the destination version read on that set before committing. Persisted
upload metadata records the null version as an empty VersionID, so completion
looks that up as the null version rather than the latest. The reconcile runs only
against an existing version; a not-found destination keeps the write's own
accepted lock, including a pre-upgrade upload that persisted values without
ordering timestamps, and a non-not-found read error fails the write. Scoped to
the SSE-C paths this issue enables; CopyObject is left as #111 wrote it.

Scope: this orders Object Lock against the destination version under the write
lock and is correct for a single erasure set. A multi-pool deployment -- where a
version can have duplicate copies across pools, object ModTime ties do not track
per-field lock timestamps, and the object namespace lock is per-pool -- needs a
cross-pool lock-safe reconcile and is deliberately out of scope here, tracked in
pgsty/silo#TBD-multipool-lock.

Tests: TestAPISSECReplicaRetransmitObjectLockOrdering and its multipart sibling;
TestAPIReplicaMultipartNewerHoldSurvivesCompletion and
TestReplicaPutObjectLockReconcileUnderWriteLock (a hold or retention reaching the
version after the handler decision, or after multipart initiation, survives the
commit; a pre-upgrade upload on an absent version keeps its lock);
TestReplicaLockReconcileNullVersion (a null-version completion reconciles the null
version, not a coexisting UUID version, and an absent null version keeps its
accepted lock); TestAPIReplicaMarkerOnlyAppliesObjectLock; TestReplicaStoredLock;
the timestamp-only putReplicationOpts round trip; and the retransmit, exemption
and target-head tests. The #111 CopyObject replica suite and the existing #120
suite stay green, as do the PUT/multipart handler and object-layer regression
suites. Compatibility: the shared helper preserves #111's CopyObject behavior; a
non-replica PUT or multipart initiation that sets Object Lock now also stamps the
reserved ordering timestamp, matching CopyObject since #111; only trusted SSE-C
replica writes take the in-lock reconcile.

Refs pgsty/silo#120

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-07 00:11:04 +08:00
Feng Ruohang 87746913fc fix: retransmit existing SSE-C replicas instead of metadata-copying them
The replication sender's target HEAD carries no SSE-C customer key, so for
an SSE-C object the target answers 400 and replicateAll fell into a
metadata-only CopyObject that fails on any non-empty SSE-C object (the
undecryptable source checksum makes the target recompute one and rewrite
the data with a plaintext-sized reader). Once a non-empty SSE-C replica
existed, tag, retention and legal-hold changes never reached it, a heal
never retransmitted, and a resync neither repaired the replica nor
counted it correctly. Forcing a full retransmit alone was not enough:
checkPreconditionsPUT rejects a write whose PreserveETag and VersionID
match the stored version, only the single-part sealed ETag is truncated
before that comparison, so a multipart SSE-C retransmit answered 412,
which the sender turns into success. Inherited from upstream ad04afe38.

Select replicateAll when the SSE-C HEAD cannot answer (the two previous
assignments were dead: rAction still forced the metadata path), exempt an
authenticated replica write that carries an SSE-C seal from the duplicate
version and ETag rejection (the predicate is the incoming write's
restored SSE-C metadata, not what the destination holds), and send the
internal replication marker on the resync accounting HEAD for SSE-C
objects so a peer answers with the replica metadata instead of 400.

Tests: TestAPISSECReplicaRetransmitOverExistingVersion (multipart replica
initiation over the same version and ETag answered 412 on main, now 200
with parts sent and plaintext readback; single-part and zero-byte writes
unchanged), TestAPISSECReplicaWriteExemptionIsKeyedOnTheIncomingWrite
(plaintext replica over an SSE-C version still 412; SSE-C replica over a
plaintext version exempted and readable), and
TestAPISSECReplicationTargetHead (keyless HEAD 400, missing key 404,
marked HEAD 200 with metadata, metadata CopyObject ExcessData on a
non-empty object) on ErasureSD and Erasure. Compatibility: every update
of an SSE-C object now retransmits its bytes; a peer that rejects the
internal marker fails the accounting HEAD as before; the #109 destination
fix must be deployed first or a retransmitted replica is transformed
again.

Fixes pgsty/silo#120

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-07 00:11:04 +08:00
Feng Ruohang 7935c84f9a fix: recognize timestamp-only retention-removal tombstone in resend compare
retentionRemovedAtSource only recognized representation (1) of a removed
retention: the object lock key present with an empty value. But a removal
that arrived by replication persists representation (2): restoreRetention
(and the receiver's replica update path) writes only the retention ordering
timestamp when the mode is empty, leaving the mode and retain-until-date keys
absent. For that shape the helper returned false, so replicationActionForTarget
skipped the GetObjectRetention confirmation and let getReplicationAction's
replicateNone stand, silently dropping a needed removal when the destination
HEAD hides retention behind a permission-filtered credential.

Recognize representation (2) as well: a present retention ordering timestamp
with the mode value absent or empty is a removal. A present timestamp paired
with a non-empty mode is a retention that was set, not removed, and still
returns false.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-07 00:06:47 +08:00
Feng Ruohang c185635b43 fix: treat empty object lock values as absent when comparing
getReplicationAction builds its source map from oi1.UserDefined, where a
removed retention is a present key with an empty value, and its target map
from the destination's HEAD headers, which can never carry those keys because
setObjectHeaders skips empty lock values and FilterObjectLockMetadata drops
both keys when the mode is invalid. The comparison then always reports a
difference, the replicateNone fast path is dead for such versions, and an
otherwise matching version re-copies its metadata on every evaluation.

Skip an entry whose value is empty and whose key is x-amz-object-lock-mode or
x-amz-object-lock-retain-until-date, case-insensitively, in both comparison
loops, using the joined value on the target side. Normalizing only the source
would regress the case where both sides hold the empty pair.

HEAD also omits a real retention from a credential without
s3:GetObjectRetention, which the documented target policy does not grant, so
that normalization alone would read a destination hiding a retention as in
sync and drop the removal. replicationActionForTarget therefore confirms with
the destination before skipping the resend: only an explicit answer, no
retention on the version, clears it. Everything else keeps today's metadata
resend, including a denied or unreachable destination, a mode the SDK does not
recognize, and InvalidRequest, which names a bucket without Object Lock but is
also what a destination answers when its own read of that configuration fails.
The null version an existing object resync excludes is never reopened.

Tests: TestGetReplicationActionEmptyObjectLockValues (eight cases, red on 2
and 3 before this change), TestRetentionRemovedAtSource,
TestTargetRetentionConfirmedAbsent,
TestReplicationActionForTargetRetentionRemoval,
TestReplicationActionForTargetNullVersionResync and
TestEmptyRetentionValuesAreOmittedFromObjectResponseHeaders.
Compatibility: sender side only, no wire or storage change, so a fixed source
converges against any destination version.

Fixes pgsty/silo#117

Signed-off-by: Feng Ruohang <rh@vonng.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
2026-09-07 00:06:47 +08:00
Feng Ruohang c201148738 Merge pull request #145 from pgsty/fix/issue-10-conditional-delete
fix: support conditional DeleteObject (If-Match) with atomic precondition
2026-09-06 23:58:42 +08:00
Feng Ruohang 53adb21c52 Merge pull request #142 from pgsty/fix/issue-141-resync-dispatch-scope
fix: scope resync worker dispatch to the target being resynced
2026-09-06 23:57:17 +08:00
Feng Ruohang 58b0ee36ca Merge pull request #140 from pgsty/fix/issue-139-resync-classification
fix: count resync success by replication outcome, not target existence
2026-09-06 23:56:57 +08:00
Feng Ruohang 8a1f594add Merge pull request #138 from pgsty/fix/issue-136-resync-counter-flush
fix: persist an honest resync terminal status about object counts
2026-09-06 23:55:59 +08:00
Feng Ruohang 5b9959617e Merge pull request #130 from pgsty/docs/issue-116-startup-readiness
docs: describe the startup readiness window of the health probes
2026-09-06 23:55:37 +08:00
Feng Ruohang da19d91b64 Merge pull request #143 from pgsty/fix/issue-107-chunked-checksum
fix: honor a header-delivered checksum advertised as a chunked trailer
2026-09-06 23:55:33 +08:00
Feng Ruohang 40bee4b7ba fix(delete): honor If-Match precondition on DeleteObject (#10)
DeleteObject ignored the If-Match request header and always deleted the
object (204). AWS S3 conditional deletes require that when If-Match is
provided and does not match the object's current ETag, the delete is
refused with 412 Precondition Failed and the object is left intact.

The precondition is evaluated in erasureServerPools.DeleteObject, while
the server-pool delete lock is held, before the delete-marker short-circuit
and before any version is removed. It runs against the version that will
actually be deleted: pinfo.ObjInfo for a normal delete, or the specifically
addressed version (read under the held lock) for a version-scoped delete,
since getPoolInfoExistingWithOpts strips VersionID. The check is a pure
function (no ResponseWriter writes) and returns PreConditionFailed, which
toAPIError maps to 412; CheckPrecondFn is cleared before lower layers run
so the precondition is evaluated exactly once.

Semantics:
- If-Match mismatch on a live object -> 412, object preserved.
- If-Match "*" requires a live object; a delete-marker-latest -> 412, and
  an explicitly addressed delete-marker version -> 412 (getObjectInfo
  returns the marker with MethodNotAllowed; the marker is the precondition
  target, not a 405).
- SSE-C/SSE-KMS: compared against the public ETag derived without the
  customer key, so a satisfiable condition is never falsely rejected.
- Explicit versionId -> evaluated against that version; a missing addressed
  version -> NoSuchVersion whether or not the key exists; a missing object
  (no versionId) -> NoSuchKey; no If-Match -> unchanged (including the
  unconditional version-scoped delete's error behavior).

Scope: atomicity is guaranteed for a single erasure set (the default
deployment). Multi-pool conditional-delete atomicity (concurrent writers
across pools, cross-pool version selection) is tracked as a follow-up.

Tests: pure-helper unit test (delete marker, "*", SSE-C without key);
object-layer tests (unversioned match/mismatch/missing, versioned
delete-marker-latest and addressed delete-marker version, explicit-version
selection, missing version on present and absent keys, read-quorum loss);
handler tests (412/204/wildcard/404) across both backends, with red/green
demonstrated per guard.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-06 23:05:27 +08:00
Feng Ruohang 6a9b5d6763 fix: honor header checksum when x-amz-trailer is advertised on non-trailer chunked PUT (#107)
The AWS Java SDK v2, with chunked encoding enabled (its default), sends a
PutObject as a non-trailer signed aws-chunked stream
(x-amz-content-sha256: STREAMING-AWS4-HMAC-SHA256-PAYLOAD). When a checksum
algorithm is set it puts the precomputed value in the x-amz-checksum-crc32
header, yet still advertises the checksum in x-amz-trailer even though no
trailer chunk is ever sent.

GetContentChecksum treated any x-amz-trailer-advertised checksum as trailing
with an empty value, deferring it to a trailer. For the non-trailer auth type
the handler sets req.Trailer = nil, so at EOF the hash.Reader looked the value
up in a nil trailer, got "", and returned XAmzContentChecksumMismatch (HTTP
400) even though the correct value sat in the request header. Real S3 accepts
the request, and disabling chunked encoding removed the trailer advertisement,
matching the reported symptom.

Honor the header value directly when a trailer-advertised checksum is already
present in the request headers; fall back to trailing delivery only when the
header is absent. When the header carries the checksum but it does not parse,
reject the request with ErrInvalidChecksum instead of falling through to a
no-validation path, so a malformed client-supplied checksum is never silently
dropped. This also restores the checksum echo on the response and the stored
value, while keeping genuine trailer uploads and wrong-checksum rejection
intact.

Fixes #107.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-06 22:48:29 +08:00
Feng Ruohang 0720ed477e fix(replication): scope resync dispatch to the target being resynced
The resync worker pool runs for a single target (opts.arn), but the dispatch
loop admitted any object whose ExistingObjResync.mustResync() was true for ANY
target. On a bucket with per-target rules (A and B), a resync of A would pull
in objects that only qualify for B - even with a single active resync, since
qualification is any-target. After the outcome-based classification (previous
change) such a cross-target object leaves A absent from its per-object result
and is counted as an A failure - an object A was never responsible for.

Scope admission to the resync's own target: dispatch an object only if it must
resync for opts.arn specifically (mustResyncTarget), via a small pure helper
objectNeedsResyncForARN. Only opts.arn carries this resync's ResetID, and that
reset is already folded into its per-target decision, so the per-target check
both scopes dispatch and honors the reset. Each target has its own resyncBucket,
so no cross-target object is dropped - it is handled by that target's resync.
The classifier's absent-ARN failure path is now unreachable for normally
dispatched objects and remains only as defense-in-depth (e.g. a config/lock
error before replication is attempted).

Delete-marker/version-purge handling, the null-version exclusion, the
finalization ordering and the outcome-based classification are unchanged.

Adds a table-driven regression for the predicate: with A/B rules and a resync
of A, an object qualifying only for B is not admitted; any-target scoping
admits it and fails the test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-06 21:57:52 +08:00
Feng Ruohang 46e82eb54d fix(replication): count resync success by outcome, not target existence
The resync worker classified each object by whether the target version
merely existed (a tgt.StatObject HEAD), ignoring the outcome of the
replicateObject/replicateDelete call it had just made. A quota-rejected
update leaves the old version in place, so StatObject succeeded and the
resync recorded a false success - reported as Completed / N success /
0 failed and persisted across restart (issue #139). #134's SSE-C HEAD
marker made StatObject succeed for SSE-C too, exposing it there. The delete
path had the mirror flaw (a failed delete leaves the object, so the HEAD
succeeded), and FailedSize was never incremented (a failed 196,608-byte
object counted as 1 failed / 0 bytes).

replicateObject and replicateDelete already build the per-target
replicatedInfos (each replicatedTargetInfo carries Arn, ReplicationStatus
and Err) but discarded it. Return it (callers that only trigger replication
ignore the value - a Go call statement discards it, so the queue paths are
unchanged) and classify the resync from the target whose Arn == opts.arn via
a small pure helper:

- Completed without error -> replicated (+ that target's size, falling back
  to the object size).
- Failed or errored -> failed (+ the object size, fixing FailedSize).
- opts.arn absent from the result (not attempted) -> failed; a resync that
  cannot confirm the object reached the target is not a success.

The StatObject-existence block (including the delete-marker/MethodNotAllowed
special case, now subsumed by the delete outcome) is removed. #134's SSE-C
HEAD marker is left intact - it is needed for genuine SSE-C success.

Adds a table-driven regression for the classifier covering a completed
update, a failed update over an existing version, an errored-but-Completed
result, a delete failure, a delete-marker success (zero bytes) and an
un-attempted ARN. Classifying by existence makes the failed cases count
success and fails the test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-06 18:48:15 +08:00
Feng Ruohang f8ca4a8656 fix(replication): keep resync Completed status honest about object counts
resyncBucket could publish and persist a Completed resync status that did
not actually cover every object, in two ways:

1. It joined only the producer workers before the deferred markStatus ran,
   not the goroutine that folds each worker result into the status, so a
   Completed status could omit the last object (or a failed object) until the
   periodic ~1m flush (issue #136). The same finalization also closed the
   result channel on early-return paths while workers were still in flight,
   risking a send-on-closed-channel panic and a lost result.
2. markStatus persists under its own background context, so if the parent
   context was cancelled during the drain - workers then return without
   sending their computed result - or a worker dropped a result on the
   resync-cancel signal, a bare Completed was still recorded with counts that
   no longer matched the objects seen.

Fixes (count integrity only; the inherited cancellation deadlock, walker leak,
and single-token routing are tracked as separate follow-ups):

- Centralize shutdown in a resyncResults helper whose finish() stops the
  workers (closes inputs, waits for them to exit) before closing the result
  channel and waiting for the consumer to drain, then lets the deferred
  markStatus persist the final counts. finish() now runs on every exit path.
- Record a dropped result via sendResyncResult (a worker consuming the
  resync-cancel token returns without sending), and in the finalizer downgrade
  a Completed status to Failed via finalResyncStatus when the parent context
  was cancelled or a worker aborted - so a persisted Completed never
  misrepresents an incomplete resync.

Deterministic tests: an on-disk round-trip of the terminal status (complete
counts stay Completed; parent-cancel-during-drain and worker-abort each
downgrade to Failed), and testing/synctest drain/worker-order assertions that
fail deterministically if a finish() wait is removed. The inherited
cancellation structure (inline Walk, the dispatch send, the worker cancel
branches) is left unchanged for the follow-ups.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-06 15:28:55 +08:00
Feng Ruohang e7d0e62f16 Merge pull request #135 from pgsty/fix/attributes-encrypted-parts-test-reconcile
test: reconcile encrypted-parts attributes test with #119 write validation
2026-09-06 09:59:54 +08:00
Feng Ruohang aee290fc34 test: reconcile encrypted-parts attributes test with #119 write validation
TestAPIGetObjectAttributesEncryptedPartLengths (from #128) built its
fixtures by PutObjectPart-ing plaintext bodies under encrypted-object
metadata with per-part sizes 5245473 and 1. Since #119, PutObjectPart
always derives an encrypted part's plaintext length from the bytes
written and rejects a part that cannot be a valid sio stream, so those
fixtures can no longer be created through a normal write and both
variants failed at write time.

Such an on-disk shape now only exists as pre-#119 data or from an old
peer, which is exactly the state the GetObjectAttributes per-part
tamper check (#128) defends. Inject that ObjectInfo directly through a
stub object layer (the setObjectLayer pattern used by the #110 tamper
test) and exercise the handler, which is what this test pins. The
handler path, the crafted part sizes, and both assertions
(separately-encrypted-parts -> ErrObjectTampered; legacy-single-stream
-> stored fragment sizes) are unchanged. Test-only; reconciles two
already-merged correct changes (#119 and #128).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-06 09:51:09 +08:00
Feng Ruohang 62cce2b152 Merge pull request #121 from pgsty/fix/issue-110-tampered-status
fix: return 500 for unreadable objects instead of 206
2026-09-06 09:26:28 +08:00
Feng Ruohang 65d4806a7b Merge pull request #127 from pgsty/fix/issue-112-bucket-metadata-cors
fix: include per-bucket CORS in bucket metadata export and import
2026-09-06 09:26:21 +08:00
Feng Ruohang 765757473a Merge pull request #126 from pgsty/fix/issue-118-no-compressed-ssec
fix: exclude SSE-C objects from compression
2026-09-06 09:26:12 +08:00
Feng Ruohang 425bd7fff1 Merge pull request #124 from pgsty/fix/issue-119-ssec-part-actual-size
fix: record plaintext part sizes for replicated SSE-C multipart parts
2026-09-06 09:26:04 +08:00
Feng Ruohang e12e739a53 Merge pull request #128 from pgsty/fix/issue-114-115-object-attributes-parts
fix: report logical part sizes and end pagination correctly in GetObjectAttributes
2026-09-06 09:25:28 +08:00
Feng Ruohang 8b736dee34 Merge pull request #123 from pgsty/fix/issue-113-rotation-checksum-algorithm
fix: honor a requested checksum algorithm on SSE-C key rotation
2026-09-06 09:25:02 +08:00
Feng Ruohang 32e75c27bf Merge pull request #122 from pgsty/fix/issue-109-raw-ssec-replica
fix: store raw SSE-C replicas verbatim on the destination
2026-09-06 09:24:26 +08:00
Feng Ruohang 885ca604a1 Merge pull request #129 from pgsty/fix/issue-111-object-lock-replica-ordering
fix: order value-less replicated Object Lock updates by timestamp
2026-09-06 09:23:34 +08:00
Feng Ruohang d10382d0dc build: refresh rebrand compatibility baseline for the SSE-C replica helper
The raw SSE-C seal headers moved from an inline slice in
cmd/object-multipart-handlers.go to the shared isRawSSECReplica helper
in cmd/replication-trust.go (already recorded), so regenerate the
compat-baseline allowlist to drop the three stale object-multipart
entries. No behaviour change; make rebrand-guard is green.

Refs pgsty/silo#109

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-06 09:08:22 +08:00
mr javad seydi 0db4bf3b00 fix: report effective bucket quotas in metrics
Signed-off-by: mr javad seydi <seydi.birjand@gmail.com>
2026-09-05 21:23:46 +03:30
Feng Ruohang 87c621965d docs: describe the startup readiness window of the health probes
After a restart a node's remote erasure drives that could not be connected
during startup stay uninstalled until the next connectDisks pass, about 15
seconds later. During that window the liveness, readiness and both cluster
probes answer 200, admin info shows every drive online and mcli ready agrees,
because the cluster probes aggregate each peer's report of its own local
drives rather than the drives this node has installed. A PUT through that
node can still fail with 503 SlowDownWrite and a cross-node GET can answer
404 NoSuchKey until the window closes. The behaviour is inherited from
upstream and reproduced on the 0806 and 0903 releases alike.

Document the window and the bounded data-path check (PUT through each node,
read each object through every node, fixed deadline, re-read acknowledged
objects) that automation should use instead of the probes, and correct the
readiness probe description, which also fails on request-queue overload and
an unreachable KMS. No product change: the probes keep their documented
purpose, and changing them or the reconnect cadence was judged unproven
tuning in the agreed plan.

Refs pgsty/silo#116

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-05 17:17:05 +08:00
Feng Ruohang b2dca43fda fix: order value-less replicated Object Lock updates by timestamp
CopyObjectHandler rebuilt the destination metadata with the public Object Lock keys stripped (cmd/object-handlers.go:1708) and then restored a value only inside retentionMode.Valid() and legalHold.Status.Valid() (cmd/object-handlers.go:1715 and :1732), so a replica update that carried no retention or legal-hold value never reached the ordering comparison and silently erased whatever the destination held, however new it was; a retention removal that did win recorded no ordering timestamp either, so cmd/bucket-object-lock.go:370 later read an unparseable stored timestamp and let an older retained value back in. Each replica field is now decided on its source timestamp first and its incoming value second, and both restore helpers write the stored timestamp back before returning early on an empty stored value, which is the only way a removal timestamp survives the REPLACE metadata directive. Legal hold stays deliberately asymmetric: S3 has no legal-hold removal, an explicitly empty status is already rejected as invalid, and an absent status conveys no change even when an orphaned timestamp arrives with it, so only a valid ON or OFF can win.

Three inherited defects would have defeated that ordering, so they are fixed here too. The SSE-KMS branch of putOptsFromHeaders built its own ObjectOptions and dropped the parsed lock timestamps, leaving every replicated lock update unordered on a bucket with default KMS encryption; it now carries them. The in-place SSE-C key rotation snapshots the stored reserved metadata into encMetadata before the lock decision exists and merges it back afterwards to preserve the encryption headers, reinstating the ordering timestamp the decision had just replaced; the snapshot is now reconciled with the decision for a trusted replica. Finally, the value-less handling applies only to an actual replica: a trusted peer that sends the replication marker without REPLICA status keeps the previous behaviour, so a REPLACE copy carrying no lock headers still writes a version with no retention and no hold.

Tests: TestAPICopyObjectReplicaAbsentLockFieldsPreserveNewerState, TestAPICopyObjectReplicaRetentionRemovalKeepsOrderingTimestamp, TestAPICopyObjectReplicaObjectLockOrdering, TestAPICopyObjectReplicaRetentionRemovalUnderBucketKMS, TestAPICopyObjectReplicaLockTimestampSurvivesSSECKeyRotation and TestAPICopyObjectMarkerOnlyLeavesObjectLockUnchanged, all on ErasureSD and Erasure. Compatibility: no API, wire or stored-field change, and a field arriving with no source timestamp is unordered and now preserves destination state, so an un-upgraded 0806 peer keeps replicating safely while it still runs the old erasing receiver.

Fixes pgsty/silo#111

Signed-off-by: Feng Ruohang <rh@vonng.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-05 16:58:11 +08:00
Feng Ruohang 33a91d972f fix: carry per-bucket CORS through bucket metadata export/import
The admin bucket-metadata handlers enumerate every bucket config by name, and
per-bucket CORS was never added to that enumeration: export omitted cors.xml
(cmd/admin-bucket-handlers.go:414 cfgFiles) and import ignored the entry
outright, with no case in applyImportedBucketMetadata (:598) or SetStatus
(:629), so a CORS-only archive reported 0/0 buckets imported and a restored
bucket silently lost its configuration. Export now writes the stored document
verbatim and import validates it with the same parser and validator as
PutBucketCorsHandler, merging it under the existing bucket metadata lock and
announcing it through the dedicated SRBucketMetaTypeCorsConfig event; the local
CORS timestamp rule is extracted into localCORSUpdatedAt and reused so an
imported document always lands strictly above bucket creation, which matters
because the import stamps its fields before creating any missing bucket and a
CORS event below Created is dropped as an older bucket incarnation.

The import reads one byte past the declared entry size so archive/zip reaches
EOF and verifies the entry checksum, otherwise a corrupt or over-long entry
carrying well formed XML would overwrite the stored document; and the CORS
event is sent even when the shared bucket metadata hook failed, so an
unreachable peer cannot withhold an already committed CORS document from the
reachable ones.

Tests: TestAdminBucketMetadataCORSRoundTrip and
TestAdminBucketMetadataCORSImportReplicatesPastPeerFailure (new, ErasureSD and
Erasure).
Compatibility: the ZIP gains one entry, older archives stay importable and
leave CORS untouched; no mcli or madmin-go change is needed because
madmin.BucketStatus already carries Cors and mcli copies the export ZIP
verbatim.

Fixes pgsty/silo#112

Signed-off-by: Feng Ruohang <rh@vonng.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
2026-09-05 16:21:12 +08:00
Feng Ruohang 2cbd48a3c3 fix: end GetObjectAttributes part pagination correctly
GetObjectAttributes decided truncation by comparing the last returned
part number with the part count (cmd/object-handlers.go:720), which is
only a coincidence of contiguous numbering. Sparse parts 1/3 reported a
complete page as truncated with a marker that loops, and parts 1/3/5
with max-parts=1 stopped after part 3 and silently dropped part 5. Set
IsTruncated in the break that proves an eligible part was left
unreturned, and zero NextPartNumberMarker when the listing is complete,
as ListObjectParts already does. Also reject negative x-amz-max-parts
and x-amz-part-number-marker in getAndValidateAttributesOpts with the
same API errors ListObjectParts uses, instead of answering an invalid
request with an empty parts listing; an absent or zero max-parts still
means the default page size.

Tests: TestAPIGetObjectAttributesPartsPagination (sparse 1/3/5 and
contiguous 1/2 walks on ErasureSD and Erasure),
TestGetAndValidateAttributesOptsPartsRange, and the sparse variants of
TestAPIGetObjectAttributesMultipartLogicalPartSize. Compatibility: no
field is added or removed; IsTruncated and NextPartNumberMarker change
only where they were wrong, and negative pagination values that no SDK
sends now fail fast.

Fixes pgsty/silo#115

Signed-off-by: Feng Ruohang <rh@vonng.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
2026-09-05 16:17:39 +08:00
Feng Ruohang b5409ca112 fix: report logical part sizes in GetObjectAttributes
GetObjectAttributes filled ObjectPart.Size from the on-disk part length
(cmd/object-handlers.go:715), so every compressed or encrypted multipart
object reported transformed sizes that do not sum to the logical
ObjectSize the same response returns from objInfo.GetActualSize().
Report each part's uploaded plaintext length instead: a compressed part
uses its recorded ActualSize, and a separately encrypted part derives the
plaintext length with sio.DecryptedSize, because ActualSize is the
ciphertext length for a replicated SSE-C part and zero for parts written
before actualSize existed.

Only parts of an encrypted multipart object are streams of their own. A
legacy encrypted object carries no multipart marker and is one continuous
stream that the erasure writer split into storage fragments, so those
fragments keep their stored size. Where a part is a stream, one whose
length cannot be a valid encrypted stream has no logical length, and the
request now fails with XMinioObjectTampered rather than reporting the
ciphertext length; DecryptObjectInfo does not catch that case, because
ObjectInfo.isMultipart gives up on the first bad part and only the object
total is then validated.

Tests: TestAPIGetObjectAttributesMultipartLogicalPartSize (plain,
compressed, SSE-C and compressed+SSE-C, consecutive and sparse part
numbers), TestAPIGetObjectAttributesCompressedEmptyTrailingPart,
TestAPIGetObjectAttributesEncryptedPartLengths, and a part-size assertion
added to TestAPISSECMultipartReplicationTrust. Compatibility: the XML
shape is unchanged and nothing is written to disk, only the value of the
existing Size element is corrected.

Fixes pgsty/silo#114

Signed-off-by: Feng Ruohang <rh@vonng.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
2026-09-05 16:17:39 +08:00
Feng Ruohang 35bd75948a fix: exclude SSE-C objects from compression
With compression allow_encryption=on an SSE-C object is stored as
encrypt(s2(plaintext)), while replication reads it raw (NoDecryption at
cmd/erasure-object.go:257) and putReplicationOpts drops the internal
compression and actual-size headers (cmd/bucket-replication.go:786). The
replica keeps the source seal with no compression marker, so a GET with the
correct customer key returns HTTP 200 and the raw S2 stream instead of the
object, and the source records the transfer as COMPLETED.

Widen the one condition in excludeForCompression (cmd/object-api-utils.go:613)
so SSE-C is never compressed, whatever allow_encryption says. This covers all
four producers at once, PutObject, NewMultipartUpload, CopyObject and
PutObjectExtract, plus any future caller of isCompressible.
crypto.SSEC.IsRequested ignores copy-source headers, so a copy is judged on its
destination key only, and a raw SSE-C replica write is unaffected because it
carries no public SSE-C headers. allow_encryption keeps its meaning for SSE-S3
and SSE-KMS, where the server owns the key and decompresses before replicating.

Tests: TestAPISSECCompressionReplicaStaysReadable (single PUT and multipart),
TestAPISSECCompressionProducerMatrix, TestAPISSECCompressionSkippedOnCopyObject,
TestAPISSECCompressionSkippedOnSnowballExtract and the control
TestSSECBatchReplicationCannotRead in cmd/compression-ssec_test.go. Two existing
expectations pinned the removed shape and are updated:
TestAPICopyObjectSSECKeyRotationNullVersionCompressesRewrite is renamed
TestAPICopyObjectSSECKeyRotationNullVersionSkipsCompression and now expects an
uncompressed rewrite, keeping its body, checksum, version and ETag assertions;
the SSE-C compressed-encrypted variant of
TestAPICopyObjectServerSideChecksumEncryption becomes compressible-extension and
expects an uncompressed destination, its SSE-S3 sibling keeping the compressed
coverage.

Compatibility: a deliberate behaviour change. Deployments with
allow_encryption=on no longer store new or rewritten SSE-C data compressed, so
those writes cost more space; objects already stored compressed keep working on
the source and are the concern of pgsty/silo#109, which rejects them at
replication time. Multipart uploads initiated before this change keep
compressing their parts from the metadata saved at initiation. Upstream
468a9fae8 refused this combination at PUT time and a2cab0255 removed the guard;
upstream master is still unguarded, so this is a deliberate divergence.

Fixes pgsty/silo#118

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-05 15:51:24 +08:00
Feng Ruohang 0c8d74205b fix: record plaintext part sizes for replicated SSE-C multipart parts
Trusted SSE-C replication uploads parts as raw ciphertext with the
ciphertext length as Content-Length, and erasureObjects.PutObjectPart only
derived the plaintext length when the caller passed a negative size, so
each replicated part persisted the ciphertext length as ActualSize (the
field defined as the uploaded size without encryption bytes). On the
replica, partNumberToRangeSpec turned those lengths into a plaintext range,
so GET/HEAD ?partNumber=N returned the wrong bytes and shifted
Content-Range (2560, 2560 and 5120 bytes for 5 MiB, 5 MiB and 1 MiB
parts), and a later decommission or rebalance re-uploaded the parts with
the stale value and recomputed the object-level actual-size from their sum,
after which a whole-object GET advertised a Content-Length larger than the
body it wrote.

Derive the plaintext length of an encrypted, uncompressed part from the
bytes actually written (sio.DecryptedSize) in PutObjectPart, the single
place a part is persisted, rejecting a length that cannot be a valid
stream before the part is committed; and derive part lengths from
part.Size in partNumberToRangeSpec for encrypted, uncompressed objects,
returning an error instead of a nil range, so replicas already on disk
read correctly without a resync. Compressed parts keep ActualSize.

Tests: TestAPISSECReplicaPartNumberReads (three-part SSE-C replica,
?partNumber=N bytes, Content-Length and Content-Range equal the source)
and TestSSECReplicaPartActualSizeDataMovement (replay through the data
movement path leaves part and object sizes at plaintext values) fail on
main and pass with the fix on ErasureSD and Erasure;
TestAPIGetObjectWithPartNumberHandler, TestAPISSECMultipartReplicationTrust
and TestAPIListObjectPartsHandler stay green. Compatibility: no wire or
API change; objects an unfixed server already moved carry a poisoned
object-level actual-size and need a rewrite or resync.

Fixes pgsty/silo#119

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-05 15:33:23 +08:00
Feng Ruohang fcc4d77895 fix: honor a requested checksum algorithm on SSE-C key rotation
An in-place SSE-C key rotation takes the fast path at cmd/object-handlers.go:1523
that only rewraps the object key, while every line that turns
x-amz-checksum-algorithm into a stored checksum lives in the re-encrypting else
branch at 1571-1609, so a requested algorithm was silently dropped and the stale
source checksum was kept and reported. Extend the canRotateKeyInPlace guard so a
client request carrying the header falls through to the copy that recomputes,
stores and reports it. Replica-trusted requests keep the fast path: getOpts
leaves their source reader encrypted, so a rewrite would hash ciphertext, and a
replica has to keep the checksum its source assigned.

Tests: TestAPICopyObjectSSECKeyRotationChecksumAlgorithm (new, red before the
guard), TestAPICopyObjectSSECKeyRotationKeepsChecksumAbsence (new, pins the
accepted limitation that a headerless rotation preserves the stored checksum
state including absence, gaining no default CRC64NVME) and
TestAPICopyObjectSSECKeyRotationReplicaKeepsFastPath (new, pins the replica
carve-out on a non-empty and on a zero byte source).
Compatibility: no API or wire change; a rotation without the header and every
replica-trusted rotation are unchanged, while a client rotation carrying the
header now rewrites the object data, so the ETag changes, a multipart source
collapses to a single part object, the copy replicates as an object rather than
as metadata, and the rewritten bytes are compressed if compression is enabled for
that object, as AWS CopyObject documents. Upstream MinIO carries the same
defect from 2718d9a43 (minio/minio#21399); this is a deliberate divergence.

Fixes pgsty/silo#113

Signed-off-by: Feng Ruohang <rh@vonng.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
2026-09-05 15:25:08 +08:00
Feng Ruohang c52acc1a5d fix: store raw SSE-C replicas verbatim on the destination
A raw SSE-C replica write carries the source ciphertext and the source seal
in X-Minio-Replication-Server-Side-Encryption-* headers but no public SSE-C
request headers, so crypto.Requested() was false and PutObjectHandler and
NewMultipartUploadHandler applied the destination's default encryption and
compression to bytes that were already ciphertext (upstream 468a9fae8,
"Enable replication of SSE-C objects", never exempted the raw path). With
destination default SSE-S3 the replica's IV and seal were overwritten and
GET returned 400; with destination compression the replica stored
compress(ciphertext) and GET failed, while the source reported COMPLETED.

Recognize a validated raw SSE-C replica (replicaTrusted plus a seal header,
shared helper isRawSSECReplica) and skip bucket default encryption,
compression and the encryption branch on the single PUT path, and default
encryption plus compression on the multipart initiation path, which already
skipped key generation. On the sender, reject replication of an object that
is both compressed and SSE-C, since the wire carries no compression state
and the destination would otherwise store an undetectable S2 stream, and
make replicateObject/replicateAll report a putReplicationOpts failure as
Failed instead of Completed.

Tests: TestAPISSECReplicaSkipsDestinationTransforms (single PUT and
multipart under destination default SSE-S3, compression and an explicit SSE
header, plus an untrusted control), TestAPISSECMultipartReplicaRoundTripWith
Compression, and TestPutReplicationOptsRejectsCompressedSSEC fail on main
and pass with the fix on ErasureSD and Erasure; the replication-trust,
multipart and PutObject suites stay green. Compatibility: no wire, API or
metadata change; the destination change applies only to trusted replica
writes carrying a source seal; replicas already transformed must be
rewritten from an intact source (see pgsty/silo#120 for why a resync does
not do that yet).

Fixes pgsty/silo#109

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-05 15:07:22 +08:00
Feng Ruohang 5703426b3c fix: return 500 for unreadable objects instead of 206
ErrObjectTampered was mapped to http.StatusPartialContent since upstream
ca6b4773e (2017), so a GET or HEAD of an object the server cannot decode
(invalid encrypted size, malformed actual-size, bad multipart ETag shape)
answered with a success status and an XML error document that SDKs handed
back as object content; boto3 returned the XML as Body and a zero-length
HEAD as success. Every origin of errObjectTampered is a stored-state
defect, not caller input, so map the entry to 500 Internal Server Error
and keep the XMinioObjectTampered code and message. The comment records
the deliberate divergence from upstream.

Tests: TestObjectTamperedGETHEADStatus (signed GET and HEAD, ErasureSD and
Erasure) fails with 206 on main and passes with 500; TestAPIErrCode,
TestAPIErrCodeDefinition, TestAPIHeadObjectHandler,
TestAPIHeadObjectHandlerWithEncryption and TestAPIGetObjectHandler stay
green. Compatibility: only the status line of one MinIO-specific error
changes; clients now retry damaged-object reads per their 5xx policy.

Fixes pgsty/silo#110

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L7qJqWwy8oFA6aCXWRzXQe
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-05 14:50:00 +08:00
Feng Ruohang f0bd164b92 docs: refresh community contributors
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-04 19:21:26 +08:00
Feng Ruohang 9936a69d89 ci: recover container publication from verified component pins
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-04 15:10:43 +08:00
Feng Ruohang ce2326c946 build: pin the published mcli 20260903 archives
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-04 14:38:54 +08:00
Feng Ruohang 4c164907f5 build: align Helm defaults with the published release tag
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-04 14:36:12 +08:00
Feng Ruohang 9b11dc9469 build: converge the final 20260903 dependency graph
Adopt silo-pkg v3.13.2, the 20260903 mcli release, and the latest validated SILO Console commit. Move maintained source imports to the pgsty/silo-pkg module path, refresh the resulting dependency closure and credits, and keep only documented legacy minio/pkg transitive residue.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-03 20:54:40 +08:00
Feng Ruohang 1eccc6908b Merge branch 'codex/ci-fixes': CI repairs for the 20260903 release
Flush S3 Select records queued before an error message (an inherited select
race that the CVE-2026-39414 regression test caught under the race detector
in CI), update golang.org/x/crypto to v0.56.0 for GO-2026-6354/6355 reachable
through the SFTP server, move the toolchain to Go 1.27.1, and extend the
message writer tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QwirCh4nsrJCQp2WaXoVLK
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-03 07:09:11 +08:00
Feng Ruohang 202371afbd docs: record the x/crypto SSH fixes and Go 1.27.1 in the advisory ledger
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QwirCh4nsrJCQp2WaXoVLK
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-03 06:53:09 +08:00
Feng Ruohang 41ef4411d9 build: move to Go 1.27.1
Go 1.27.1 (2026-09-01) carries fixes to the runtime, compiler, net/http,
encoding/json, os and database/sql. The go directive, the dependency check
script and the image build stage follow it; CI reads the version from
go.mod. The remaining golang.org/x modules the server requires are already
at their latest releases.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QwirCh4nsrJCQp2WaXoVLK
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-03 06:52:33 +08:00
Feng Ruohang 035aa6c201 test: cover the S3 Select message writer ordering
Exercise the writer directly and decode the event stream with the client
parser: a buffered plus a queued record before an error, an error with
nothing queued, a record larger than the staging buffer on both the success
and the error path, and the unchanged success ordering of records, Stats and
End.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QwirCh4nsrJCQp2WaXoVLK
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-03 06:52:33 +08:00
Feng Ruohang edf36bcbfa deps: update golang.org/x/crypto to v0.56.0
GO-2026-6354 and GO-2026-6355 (denial of service on deadlocked SSH channels)
are reachable through the SFTP server, which listens with x/crypto/ssh.
v0.56.0 carries the fixes; govulncheck reports no reachable vulnerability.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QwirCh4nsrJCQp2WaXoVLK
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-03 06:43:49 +08:00
Feng Ruohang 3598c4305d fix: flush records queued before an S3 Select error message
The writer goroutine selects between the record channel and the error
channel at random. When Evaluate had just queued a record and then reported
an error, the error case could win first; it flushed only the staging
buffer, so the queued record was dropped in the exit drain and the client
saw the error without the records that preceded it. Stage whatever is queued
before flushing and writing the error. The CVE-2026-39414 regression test
asserted this contract and failed once under the race detector in CI; a new
unit test exercises the ordering directly.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QwirCh4nsrJCQp2WaXoVLK
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-03 06:43:49 +08:00
Feng Ruohang 94fbb6df6d fix: never record a resident bucket as a CORS load failure
A bucket that is resident keeps its last loaded metadata through a failed
refresh, exactly like every other bucket configuration, so the load-failure
set only ever holds buckets that were never loaded. This makes the CORS
lookup's precedence explicit: resident document first, fail-closed only for
a bucket that has no loaded document.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-03 00:45:02 +08:00
Feng Ruohang f4c1286c9d fix: apply replicated Object Lock updates only when newer than the stored state
A replicated CopyObject uses the REPLACE metadata directive, so the map the
handler compared replication timestamps against had already been rebuilt
from the request and filtered of Object Lock keys: the stored retention and
legal-hold timestamps were never seen, every replica update was applied
regardless of order, and the legal-hold timestamp was written under the
retention key. A stale replica could turn a newer legal hold off or shorten
a newer retention.

Capture the stored Object Lock state before the metadata is rebuilt, apply a
replica update only when its source timestamp is newer, put the stored state
back when the update is stale, and keep each timestamp under its own key.
Inherited from upstream; recorded in the advisory ledger.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-03 00:45:02 +08:00
Feng Ruohang 6e112d1856 build: point Helm defaults at the 20260903 release
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 23:11:32 +08:00
Feng Ruohang e62dc6e023 Merge branch 'codex/prerelease-cleanup': pre-release cleanup and dependency pins
Fifteen reviewed commits: streaming-trailer trust fix and a single
replication-trust evaluator, Object Lock versioning invariant, resident-only
CORS lookup with fail-closed startup and load-failure states, dead-code
removal, compatibility baseline without exported symbols, upstream minio-go
with Console 43f8447fd and mcli 20260901, advisory ledger SN-2026-006..010,
contributor records, verification scripts with a valid root password, and a
deterministic dynamic-timeout test.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 21:34:30 +08:00
Feng Ruohang 84e1580a47 fix: clear the load-failure bit when GetConfig reloads a bucket on demand
A successful on-demand load makes the bucket resident, so the failure
recorded by an earlier startup or refresh no longer applies. Clearing it here
keeps the set's invariant exact instead of waiting for the next refresh.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 21:23:30 +08:00
Feng Ruohang ebac0ca73b test: give the dynamic timeout tests a private random source
TestDynamicTimeoutAdjustExponential and TestDynamicTimeoutAdjustNormal
seeded the global generator and then drew from it while other tests in the
package may use the same generator, so the sample was not the one the seed
promised and the exponential case failed once in a full race run. A private
source makes both tests deterministic.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 20:04:57 +08:00
Feng Ruohang 3f9c79e919 fix: keep load-failed buckets fail-closed in the CORS lookup
Restore the load-failure set that the cleanup removed. A presigned URL is
authenticated by its signature, so for such requests the bucket's CORS
document is the only origin boundary a browser enforces; a real bucket whose
metadata failed to load must therefore not be answered with the global
policy, and without this bit it is indistinguishable from a name that is not
a bucket. Two helpers own the set's lifecycle; the resident-only lookup and
the removal of the internal-namespace special case stay.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 19:51:54 +08:00
Feng Ruohang ec2979ca48 fix: keep the CORS lookup fail-closed until bucket metadata is loaded
Restore the startup guard removed by the previous cleanup: while bucket
metadata is still loading, a non-resident name may be a bucket with a
restrictive CORS document, so the request gets no CORS answer instead of the
global policy. After startup a non-resident name still falls back to the
global policy without any metadata I/O; the separate load-failure set stays
removed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 18:49:17 +08:00
Feng Ruohang 21646eebd2 fix: derive the Object Lock versioning rule from the parsed configuration
The load-time normalization compared the stored lock document with the
canonical enabled document byte for byte, so a lock configuration that also
carries a default retention rule kept a suspended or prefix-excluded
versioning document. Decide from the parsed configuration instead, after it
is parsed, so every writer that goes through Save, including the site
replication versioning and heal paths, ends with plain Enabled versioning on
a locked bucket. Receiving a lock configuration on a bucket created without
lock now enables versioning as well; the test that asserted the opposite is
updated, and a new test covers a rule-bearing lock document with suspended
and prefix-excluded versioning through Update, Get, and reload.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 18:49:16 +08:00
Feng Ruohang 5711996231 build: use an eight-character root password in the verification scripts
The rebrand shortened the scripts' root password from minio123 to silo123.
The server requires at least eight characters, so every script that starts
a server with it failed at startup and verify-build.sh then waited forever
on mc ready. No workflow runs these scripts, which is why it went unnoticed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 15:53:08 +08:00
Feng Ruohang 0079723d35 deps: return to upstream minio-go and embed Console 43f8447fd
The silo-go fork is retired: silo-pkg v3.13.0, Console v2.3.0, and mcli
20260901 all require upstream github.com/minio/minio-go/v7 again. Drop the
replacement and require the same upstream pre-release they use
(v7.3.1-0.20260828014306-0e78d3f18efe, one commit past the fork's base).

Console moves from e07ef01 (v2.2.1 plus pins) to 43f8447fd, the last commit
of the v2.3.0 line before Console adopted the github.com/pgsty/silo-pkg/v3
module path. It carries the six v2.3.0 security fixes (forwarding-header
trust, outbound TLS verification, credential redaction, WebSocket session and
connection caps), the IAM wildcard and session-identity fixes, and their
regenerated assets, while still consuming silo-pkg through the existing
replacement. mc stays on the last commit before its own path migration;
silo-pkg stays on the last commit that declares the github.com/minio/pkg/v3
path, which differs from v3.13.0 only by that path change. CREDITS follows
the module set.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 14:51:10 +08:00
Feng Ruohang bc3b35f975 build: track only served routes in the compatibility baseline
The baseline recorded 9,051 exported symbols of the main and internal
packages, which nothing outside this module can import, and 112 request
paths that exist only in test fixtures. Both changed with almost every
functional commit and protected no compatibility promise. The guard now
records routes from non-test files only, ignores untracked files, and drops
the symbol set; the baseline shrinks from 522 KB to 46 KB. CONTRIBUTING
explains when to refresh it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 14:08:11 +08:00
Feng Ruohang 00d864ed0e docs: record advisories SN-2026-006 to 010 and refresh contributors
Ledger entries for the zero-byte SSE-C key check, GetObjectAttributes
authentication, replication request trust, user and group status
authorization, and DeleteObjectVersion authorization, plus the Go 1.27
toolchain refresh. The ledger names pgsty/silo, CONTRIBUTORS lists the
per-bucket CORS, ChecksumType, and NoSuchBucket contributors, and the CORS
design record states its merged status without the review logs.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 14:08:11 +08:00
Feng Ruohang 8f2a30d9af build: bundle mcli 20260901 and describe the component pins
mcli RELEASE.2026-09-01 carries the credential redaction fixes for --debug
output. The go.mod comments now state what the Console and silo-pkg pins are:
the last commits that consume silo-pkg through the github.com/minio/pkg/v3
replacement, since silo-pkg v3.13.0 and Console v2.3.0 moved to the
pgsty/silo-pkg module path.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 14:06:56 +08:00
Feng Ruohang d9766d7378 chore: drop the wait_pipe lint exclusion and use gomodguard_v2
Assigning the two pipe halves before returning them removes the gofumpt and
gofmt disagreement that needed a permanent formatter exclusion. gomodguard is
deprecated in golangci-lint v2.12; the v2 linter takes the same configuration.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 14:06:56 +08:00
Feng Ruohang 5594d284fc chore: remove dead code left by earlier fixes
checkSSECCopySourceKey duplicated the source-key authentication that both
CopyObject read paths already perform; the DeleteObjects signature-error
branch became unreachable once signatures are verified once per request; the
discrete PostgreSQL and MySQL notification environment constants have had no
reader since DSNs became mandatory.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 14:06:56 +08:00
Feng Ruohang 632eb4729a refactor: serve non-resident buckets with the global CORS policy
The pre-authentication CORS lookup stays resident-only, so client-supplied
path segments still cause no metadata I/O and no cache growth. The
fail-closed states for startup, load failures, and the internal namespace
are gone: CORS is a browser response policy rather than an authorization
boundary, and failing closed only denied browser clients CORS headers while
bucket metadata was still loading. A bucket whose stored CORS document does
not parse still gets no CORS headers. This removes the loadFailed bookkeeping
and the unused GetCorsConfig, HasAllowedOrigin, and generic Update path for
CORS; tests use the CORS-specific writer.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 14:06:56 +08:00
Feng Ruohang 3b5de82f5a fix: keep plain Enabled versioning when Object Lock is enabled on a bucket
Site adoption and ForceCreate preserved a suspended or prefix-excluded
versioning configuration while bootstrapping Object Lock, persisting a state
that PutBucketVersioning itself rejects: objects under an excluded prefix in
a WORM bucket were not versioned and escaped retention. enablePeerBucketVersioning
now takes the lock intent and replaces such configurations with plain Enabled
versioning, and metadata loading ignores prefix exclusions on a locked bucket
as it ignored suspension before. The adoption tests assert the normalized
state and keep the timestamp-preservation checks on valid documents.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 14:06:56 +08:00
Feng Ruohang 76195f1c68 fix: keep streaming trailers visible after replication headers are stripped
A request that does not earn replication trust continues with a clone whose
internal replication headers are removed. r.Clone copies the Trailer map, but
the streaming body reader created from the original request fills the
original map, so a trailing checksum was never seen by the hash reader and
PutObject and UploadPart with STREAMING-UNSIGNED-PAYLOAD-TRAILER failed with
XAmzContentChecksumMismatch whenever an untrusted X-Minio-Source-* header was
present. Share the trailer map with the clone, as the Snowball path already
does for its per-entry requests, and cover both handlers with a test.

The marker evaluation that was copied into six handlers now lives in
evaluateReplicationTrust so the rule (a declared replica without the
replication permission is rejected; trust needs the exact marker plus the
permission) is defined once.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PvgysXDmhPBBimCReYtA8q
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 14:06:56 +08:00
Feng Ruohang 6586fbfd0d Merge pull request #103 from pgsty/codex/issue-102-metadata-lock
fix: serialize whole-record bucket metadata updates
2026-09-02 07:30:40 +08:00
Feng Ruohang 53b09f4e25 style: apply gofumpt to migration globals
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:20:53 +08:00
Feng Ruohang 59812446ad fix: surface metadata migration failures after locking
Degrade to the loaded in-memory configuration only when the short migration lock cannot be acquired. Preserve parse, I/O, KMS, and persistence errors once the lock is held.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:20:53 +08:00
Feng Ruohang 1c2f59604d docs: record metadata-lock rolling-upgrade limits
State that old cors-config.lock and new metadata.lock do not interoperate during a rolling upgrade, while the unchanged on-disk format remains rollback-compatible.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:20:53 +08:00
Feng Ruohang b3a6d5dbf8 fix: bound bucket metadata migration locking
Use a short dedicated migration lock timeout and fall back to the already loaded legacy or target configuration in memory when persistence is contended. Never fall back to an unlocked metadata save, and keep peer fan-out contexts free of stale lock markers.\n\nRefs: #102

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:20:53 +08:00
Feng Ruohang 11de51ec26 fix: limit imported metadata replication to changed fields
Release metadata.lock with defer and dispatch only the configuration types actually present in the import. Avoid stamping unrelated peer metadata with the import timestamp.\n\nRefs: #102

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:20:53 +08:00
Feng Ruohang 10f3a8590b fix: harden bucket metadata creation edge cases
Preserve existing records only for ForceCreate, reject ghost metadata on genuine creation, keep object-lock versioning invariants, and complete metadata saves after caller cancellation. Expand deterministic coverage for peer bulk, lifecycle delete, ghost creation, and cancellation.\n\nRefs: #102

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:20:53 +08:00
Feng Ruohang 55c22abec1 docs: describe the shared bucket metadata lock
Replace the obsolete CORS-only lock and #77 references with metadata.lock and issue #102. Document lock order, fan-out placement, covered writers, and deliberate follow-ups.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:20:53 +08:00
Feng Ruohang 32a1b81e4c fix: rebase imports and migrations under metadata.lock
Apply only validated import fields to a fresh locked record, block ForceCreate after real read errors, and route legacy or target-config migration saves through the shared lock. Compute lifecycle deletion state from the locked record.\n\nRefs: #102

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:20:53 +08:00
Feng Ruohang 62d8c649fe fix: preserve metadata during bucket creation and adoption
After storage bucket creation, merge required versioning and lock defaults into the latest on-disk metadata under metadata.lock. Avoid ForceCreate and site-adoption rewrites that replaced existing bucket configuration.\n\nRefs: #102

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:20:53 +08:00
Feng Ruohang 312397739e test: reproduce ForceCreate bucket metadata clobber
Create policy and CORS state, force-create the existing bucket, and require the original Created time and both metadata fields to survive.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:20:53 +08:00
Feng Ruohang 7ade0c045b fix: serialize bucket metadata updates across config types
Use one per-bucket metadata.lock for ordinary updates, CORS transitions, and legacy bulk replication. Persist and update the local cache while locked, then release before peer metadata reload fan-out.\n\nRefs: #102

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:20:53 +08:00
Feng Ruohang f9f9fa6c90 test: reproduce cross-type bucket metadata lost updates
Pause one whole-record writer at the metadata PutObject boundary and let a different config writer commit from the same stale snapshot. Assert that policy+CORS and tagging+SSE both survive on disk and in the resident cache.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:20:53 +08:00
Feng Ruohang 0a9c777795 Merge pull request #104 from pgsty/codex/issue-58-delete-version-authz
fix: authorize explicit version deletes with DeleteObjectVersion
2026-09-02 07:20:41 +08:00
Feng Ruohang d2d47a41fb chore: refresh delete authorization compatibility snapshot
Accept the additional DeleteObjects route literal introduced by the issue #58 regression matrix.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:08:39 +08:00
Feng Ruohang dee2c3a02b fix: preserve multi-delete authentication and audit context
Authenticate DeleteObjects before validating entry count, remove the obsolete per-version auth helper, and pin the marker-only request to the ordinary authorization path.\n\nRefs: #58

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:08:02 +08:00
Feng Ruohang c4140609b7 test: verify least-privilege delete replication
Document that replication targets retain the DeleteObject plus ReplicateDelete contract and extend the existing two-site test with a target user that lacks DeleteObjectVersion. Use mc for explicit version deletion so the gate no longer depends on AWS CLI.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:08:02 +08:00
Feng Ruohang 75a6734e49 fix: authorize explicit version deletes with DeleteObjectVersion
Select DeleteObject or DeleteObjectVersion from each request's effective version ID for single and multi-delete. Authenticate multi-delete once, bind version conditions to each XML entry, and keep the established DeleteObject plus ReplicateDelete target contract with explicit version denies honored only on trusted replication.\n\nRefs: #58

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:08:02 +08:00
Feng Ruohang f8b598f1d3 test: reproduce explicit-version delete authorization mismatch
Prove that DeleteObjectVersion-only principals cannot delete named UUID or null versions while DeleteObject-only principals can, contrary to the S3 action mapping in issue #58.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 07:08:02 +08:00
Feng Ruohang fe9dd90b22 Merge pull request #101 from pgsty/codex/fix-cors-replication-trust
fix: harden CORS and replication request trust
2026-09-02 07:07:49 +08:00
Feng Ruohang 04b097fd9f chore: refresh compatibility and lint baselines
Accept the new CORS test routes, resident getter, and replication header literals in the rebrand guard. Apply gofumpt, context-first helper ordering, and spelling fixes required by CI.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 02:31:20 +08:00
Feng Ruohang ab3ae99ca3 fix: preserve Snowball request defaults across workers
Snapshot per-entry requests after applying bucket encryption defaults but before streaming trailers are consumed. Keep authorization failures fatal while retaining Snowball ignore-errors behavior for object-lock failures.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 00:20:06 +08:00
Feng Ruohang 5db7be4ee4 fix: validate replication within the rule prefix
Place synthetic permission-check objects under each enabled rule's effective prefix, so least-privilege target policies are validated against the namespace they will actually replicate.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 00:03:33 +08:00
Feng Ruohang ff44527a3c fix: isolate Snowball replication trust per entry
Evaluate PutObject and ReplicateObject permissions with immutable per-entry request snapshots during concurrent Snowball extraction. Preserve the first API error without sharing mutable handler state, and cover prefix-scoped trust under the race detector.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-02 00:02:24 +08:00
Feng Ruohang c9ad746732 fix: verify replication permissions in validity probes
Evaluate ReplicateObject or ReplicateDelete before returning the no-op validation response, so underprivileged target credentials fail during replication setup instead of at runtime.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-01 23:16:09 +08:00
Feng Ruohang f3438b2602 fix: validate CORS state in replication status
Count only valid live CORS states in per-site summaries. Treat baselines and tombstones as absent, and diagnose malformed payloads or missing source timestamps.\n\nRefs: #77

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-01 23:16:04 +08:00
Feng Ruohang 938603458d fix: harden CORS and replication request trust
Keep pre-authentication CORS lookups resident-only so attacker-controlled path segments cannot trigger metadata I/O or grow the metadata cache. Preserve fail-closed behavior for startup, load failures, invalid metadata, and the internal namespace.

Centralize replication request trust after authentication, distinguish general replication from replica-only privileges, and gate SSE-C ciphertext handling, source metadata, object-lock bypasses, event suppression, delete semantics, and replica status on the appropriate permission. Add least-privilege, multipart, PostPolicy, CORS amplification, and compatibility regressions.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-09-01 20:50:08 +08:00
Feng Ruohang d5e763b072 Merge pull request #98 from pgsty/fix/attributes-ssec-replication-authz
fix: authorize SSE-C attribute reads by replication permission
2026-08-30 18:42:10 +08:00
Feng Ruohang 74c97d005d fix: authorize SSE-C attribute reads by replication permission
GetObjectAttributes lets a replication peer read SSE-C attributes without
presenting the customer key. That carve-out was keyed on the
X-Minio-Source-Replication-Request header alone, which any client can set,
so a caller holding only s3:GetObject could read an SSE-C object's ETag,
plaintext size, part list, and checksums without the key -- the response was
byte-identical to one made with the correct key.

Gate the carve-out on s3:ReplicateObject for the target object, mirroring
CopyObjectHandler's existing replication check.

The pre-existing test asserted the carve-out with root credentials, which
hold every action and therefore cannot tell a gated check apart from an
ungated one. Add least-privilege cases that do: a reader without
s3:ReplicateObject is now refused, a caller that holds it keeps the
carve-out, and the ordinary key-bearing path is unchanged.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-30 15:50:24 +08:00
Feng Ruohang 1be684ba29 Merge pull request #97 from pgsty/codex/pin-prerelease-components
deps: pin reviewed pre-release components
2026-08-29 20:49:58 +08:00
Feng Ruohang 1367364c40 Merge remote-tracking branch 'origin/main' into codex/pin-prerelease-components 2026-08-29 20:33:44 +08:00
Feng Ruohang 5f44fe369c Merge pull request #94 from pgsty/codex/release-idempotency
ci: make server release retries tag-idempotent
2026-08-29 20:33:14 +08:00
Feng Ruohang c6cc136833 Merge remote-tracking branch 'origin/main' into codex/release-idempotency 2026-08-29 20:12:38 +08:00
Feng Ruohang 97e31af331 Merge pull request #88 from pgsty/codex/cors-hotpath
fix: skip bucket CORS lookup without an origin
2026-08-29 20:11:59 +08:00
Feng Ruohang 0507c3d56a Merge remote-tracking branch 'origin/main' into codex/cors-hotpath 2026-08-29 20:02:51 +08:00
Feng Ruohang 87b9895bc0 Merge pull request #91 from pgsty/codex/issue-77-status-accounting
fix: report site replication metadata per site
2026-08-29 20:00:53 +08:00
Feng Ruohang ca1f11b708 Merge remote-tracking branch 'origin/main' into codex/issue-77-status-accounting 2026-08-29 19:50:27 +08:00
Feng Ruohang 514a041f4a Merge pull request #90 from pgsty/codex/issue-78-preserve-bucket-config
fix: preserve bucket configs during site adoption
2026-08-29 19:50:04 +08:00
Feng Ruohang f8106cd693 Merge remote-tracking branch 'origin/main' into codex/issue-78-preserve-bucket-config 2026-08-29 19:39:50 +08:00
Feng Ruohang b1ba685ac5 Merge pull request #89 from pgsty/codex/issue-76-object-lock-wire
fix: replicate object lock config in its own field
2026-08-29 19:39:22 +08:00
Feng Ruohang bc7658f035 Merge pull request #93 from pgsty/codex/issue-50-after-checksum-contract
fix: reject composite CRC64NVME checksums
2026-08-29 19:38:48 +08:00
Feng Ruohang f377b5c95b Merge remote-tracking branch 'origin/main' into codex/issue-50-after-checksum-contract
# Conflicts:
#	cmd/erasure-multipart-fullobject_test.go
2026-08-29 19:28:06 +08:00
Feng Ruohang 45381222b7 Merge pull request #92 from pgsty/codex/reject-unsupported-checksums
fix: reject unsupported checksum assertions
2026-08-29 19:25:39 +08:00
Feng Ruohang 4d6e1ea8ea deps: pin reviewed pre-release components
Select merged MCLI, Console, and silo-pkg source commits under GOWORK=off so the Server module graph matches the reviewed client, policy, and environment behavior.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 19:23:43 +08:00
Feng Ruohang 20aaefa89c Merge remote-tracking branch 'origin/main' into codex/reject-unsupported-checksums 2026-08-29 19:14:51 +08:00
Feng Ruohang 82509ddc0a Merge pull request #95 from pgsty/codex/get-object-attributes-ssec
fix: authenticate SSE-C for object attributes
2026-08-29 19:14:34 +08:00
Feng Ruohang f2ba439022 Merge remote-tracking branch 'origin/main' into codex/get-object-attributes-ssec 2026-08-29 19:04:14 +08:00
Feng Ruohang 2ba1f38503 Merge pull request #87 from pgsty/codex/issue-82-after-83
fix: authenticate SSE-C keys on zero-byte reads
2026-08-29 19:03:57 +08:00
Feng Ruohang 8d58343263 Merge pull request #96 from pgsty/codex/crc64-completion-after-foundation
fix: reject composite CRC64NVME completion
2026-08-29 18:51:09 +08:00
Feng Ruohang 2b2e0d2a5d Merge pull request #86 from pgsty/codex/issue-83-copy-null-version-release
fix: keep rewritten CopyObject data and metadata consistent
2026-08-29 18:49:25 +08:00
Feng Ruohang e407dc58eb Merge pull request #85 from pgsty/codex/server-prerelease-foundation
fix: restore pre-release server corrections
2026-08-29 18:40:26 +08:00
Feng Ruohang 32b2aa49f1 fix: reject composite CRC64NVME completion
Remove the remaining type-only canonicalization at CompleteMultipartUpload while preserving legacy uploads stored as FULL_OBJECT.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 18:36:02 +08:00
Feng Ruohang 8448512a1f docs: clarify layered SSE-C key authentication
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 18:21:13 +08:00
Feng Ruohang 150e7b5f9e fix: preserve global CORS response semantics
Guard only the per-bucket metadata lookup, then retain the global handler Vary and originless preflight behavior for non-CORS traffic.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 18:19:57 +08:00
Feng Ruohang 21870fa2e7 fix: preserve replicated object attributes reads
Keep the existing trusted replication carve-out while authenticating ordinary SSE-C GetObjectAttributes requests.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 18:18:01 +08:00
Feng Ruohang d4c8da162b fix: reject composite CRC64NVME trailers
Apply the full-object-only rule to declared streaming checksum trailers and cover the HTTP mutation path.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 18:15:51 +08:00
Feng Ruohang fc7bf7b295 test: keep CORS hot-path coverage route-neutral
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 17:58:30 +08:00
Feng Ruohang d28885d0e5 fix: reject composite CRC64NVME checksums
Return InvalidArgument for CRC64NVME with COMPOSITE at multipart initiation and PutObject instead of silently canonicalizing the request to FULL_OBJECT.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 17:32:36 +08:00
Feng Ruohang c4fd97d0bf test: align zero-byte rotation key errors
After zero-byte reads authenticate SSE-C at the object layer, equal invalid rotation keys match non-empty reads and return AccessDenied before the rotation-specific distinction.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 17:32:25 +08:00
Feng Ruohang b73581b05d fix: authenticate SSE-C keys on zero-byte reads
Unseal supplied SSE-C and copy-source keys after range and request preconditions when a zero-byte read cannot build a decryptor. Preserve internal no-decryption, replication, restore, and absent-header reads.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 17:32:25 +08:00
Feng Ruohang 474cd5801e fix: authenticate SSE-C for object attributes
Unseal the supplied customer key after request preconditions before returning object size, ETag, checksum, or part attributes. Cover zero-byte and non-empty objects with correct, wrong, and missing keys.\n\nRefs #84.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 17:26:37 +08:00
Feng Ruohang ee9252a608 ci: make server release retries tag-idempotent
Serialize release and finalize work by tag, replace only one unfinalized Draft, refuse published or finalized release state, and pin GoReleaser to the checked-out tag. Add fail-closed release-state fixtures to the release pipeline.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 17:25:40 +08:00
Feng Ruohang fb406fdc94 fix: report site replication metadata per site
Count only each site own valid bucket metadata, populate quota totals, and keep malformed fields from suppressing unrelated bucket statistics. Emit bounded diagnostics for invalid payloads.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 17:25:11 +08:00
Feng Ruohang dd3bdb8086 fix: preserve bucket configs during site adoption
Keep existing Object Lock and enabled versioning documents and timestamps when adopting a same-name bucket. Bootstrap missing configs, enable suspended versioning, and retain custom excluded-prefix settings.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 17:24:27 +08:00
Feng Ruohang 7c103389f5 fix: reject unsupported checksum assertions
Reject unimplemented x-amz-checksum value and trailer names instead of accepting uploads without verification. Apply the same contract to PutObject, multipart initiation and parts, CopyObject, and UploadPartCopy while preserving the five supported algorithms.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 17:23:26 +08:00
Feng Ruohang 3861f33cba fix: replicate object lock config in its own field
Populate ObjectLockConfig for live, initial-sync, and heal events, while accepting the legacy Tags field during rolling upgrades. Exercise signed admin dispatch and remote-heal transport on both object-layer backends.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 16:54:00 +08:00
Feng Ruohang 07d92db523 fix: skip bucket CORS lookup without an origin
Bypass per-bucket metadata work for non-CORS traffic, including admin and Console routes. Keep operational metadata errors fail-closed and pin the existing global fallback for missing buckets.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 16:34:51 +08:00
Feng Ruohang 5732930102 test: cover null-version copy rewrite directions
Exercise the silent compression-on-copy path, compressed SSE-C re-encryption, and the equal-invalid-key error contract on both object-layer backends.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 16:28:42 +08:00
Feng Ruohang 2aea7fe9c4 fix: preserve named targets in config environment files
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 16:00:21 +08:00
Feng Ruohang ffb70eb373 fix: re-encrypt a key rotation the object layer has to rewrite
A key rotation rewraps the object key held in metadata; it never re-encrypts
the stored bytes. CopyObjectHandler took that shortcut whenever the request
looked like a same-object SSE-C rotation, on the assumption that the object
layer would then leave the stored bytes alone. That is the same assumption
copyRewritesObjectData() was added to stop making.

When the source is a null version on a bucket that gained versioning after the
object was written, the object layer cannot reference that version and falls
back to PutObject. The reader at that point holds plaintext decrypted with the
old key and no EncryptFn is set, so the destination ends up storing plaintext
under metadata that claims the object is SSE-C encrypted. A subsequent GET
failed with "sio: unsupported version".

Gate the rotation shortcut on the same prediction the compression metadata
already uses. When the object layer stores new object data the rotation falls
through to the regular re-encrypting copy, which decrypts with the old key and
re-encrypts with the new one. The source version selection moves next to the
gate because both decisions need it.

That fallback authenticates the source key through the source decryptor, which
GetObjectNInfo does not build for a zero byte object. Check the key explicitly
before the destination is written, so the gate cannot turn a rotation that the
shortcut rejected with AccessDenied into one that succeeds. The re-encrypting
copy regenerates the encrypted ETag, unlike an in-place rotation; the test
records that difference.

The other three object layer CopyObject callers that set metadataOnly -
PostRestoreObjectHandler, updateRestoreMetadata and batchKeyRotate - address
the same version on both sides and never set Versioned, so they only reach the
two in-place cases already covered by the copyRewritesObjectData table.

Signed-off-by: Feng Ruohang <rh@vonng.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fk3PAD7DHCYzcyegYWqAmt
2026-08-29 16:00:21 +08:00
Feng Ruohang e73436c99d fix: decrypt CopyObject checksums with destination key
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 16:00:21 +08:00
Feng Ruohang 0b0ae2423a fix: keep copy metadata consistent with a rewritten null version
CopyObjectHandler recorded the source compression metadata whenever the copy
was metadata-only, on the assumption that the object layer would then leave
the stored bytes alone. That assumption does not hold. Both
erasureServerPools.CopyObject and erasureSets.CopyObject only skip a data
rewrite in three cases, and otherwise fall back to a full PutObject.

The reachable gap is a copy whose source is a null version on a bucket that
gained versioning after the object was written. Neither version ID is set, so
the self-referential version branch is skipped, the data is rewritten as
plaintext, and the preserved compression metadata then described bytes that
no longer exist. A subsequent GET failed with "s2: corrupt input".

Mirror the object layer's decision in copyRewritesObjectData and record the
compression metadata from it, so the metadata always describes whichever
bytes are finally stored. The source version selection that lets a versioned
metadata-only copy add a self-referential version moves next to the same
decision, since both depend on the effective metadata-only value.

Signed-off-by: Feng Ruohang <rh@vonng.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 16:00:21 +08:00
Feng Ruohang 7e079ff05c fix: validate explicit multipart checksum type
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 16:00:21 +08:00
Feng Ruohang 229fe2b3c3 fix: authorize group status changes by target status
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 16:00:21 +08:00
Feng Ruohang 38ed9d1e1f docs(security): record inherited upstream advisory
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 16:00:21 +08:00
Feng Ruohang 47cd7807d3 test: align federated client version
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 16:00:21 +08:00
Feng Ruohang 04d3d316d2 Merge pull request #81 from pgsty/codex/issue-75-closeout-tests
test: cover asymmetric CORS site counts
2026-08-29 10:13:38 +08:00
Feng Ruohang 4bb8c813ac test: cover asymmetric CORS site counts
Exercise no-site, local-only, remote-only, and both-site CORS summary accounting through the real siteReplicationStatus seam.

Refs #75

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 10:03:01 +08:00
Feng Ruohang b6ef7e430c Merge pull request #80 from pgsty/codex/issue-75-cors-hardening
fix: complete per-bucket CORS release hardening
2026-08-29 09:44:16 +08:00
Feng Ruohang 91d9091758 docs: record final bucket CORS acceptance
Record the frozen B2+B3 commit, combined local gates, raw SigV4 validation, real two-site offline/delete/heal/restart evidence, and the separate public documentation QA boundary.

Refs #75

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 09:29:34 +08:00
Feng Ruohang 0eebc928f7 fix: complete bucket CORS protocol validation
Integrate the strict B3 XML, validation, checksum, wildcard, MaxAge, and Origin-null response contract with the C-prime site-replication register from #75.

Preserve fail-closed metadata behavior and rejected-preflight cache variation while keeping legacy-invalid development metadata readable and repairable through a valid CORS PUT or DELETE.

Add combined parser, handler, browser-response, namespace, replication, restart, and legacy-repair regressions, and update the internal design contract.

Refs #75

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 09:10:58 +08:00
Feng Ruohang 724f8703d8 fix: make bucket CORS replication converge
Define a deterministic CORS replication register with durable tombstones, strict source timestamps, equal-time conflict ordering, full-state status, and heal convergence.

Serialize local and peer CORS transitions with a distributed namespace lock, validate canonical transport payloads, preserve initial-sync deletes, and fail closed on metadata errors.

Add adversarial, concurrent, restart, status, heal, signed admin-dispatch, protocol, and middleware coverage together with the reviewed site-replication design record.

Refs #75

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-29 01:33:24 +08:00
Feng Ruohang e4e3007da6 Merge pull request #71 from h5vx/feature/per-bucket-cors
feat: per-bucket CORS configuration with S3 ?cors enforcement\n\nRelease hardening and site-replication convergence follow-up: #75.
2026-08-28 10:34:52 +08:00
h5vx 13e6458d90 feat: replicate per-bucket CORS across sites and harden the protocol path
Site replication emitted SRBucketMetaTypeCorsConfig on PutBucketCors, but
the peer receive/apply, initial-sync, status, and heal paths did not carry
the CORS metadata. Replicated sites could therefore diverge on CORS config
even though the originating request succeeded.

Complete every site-replication path for CORS, mirroring the SSEConfig
pattern:
  - peer apply: PeerBucketCorsConfigHandler + item.Cors handling in
    PeerBucketMetadataUpdateHandler, with an updatedAt staleness guard
  - initial sync: push existing CorsConfigXML via BucketMetaHook
  - status: parse per-site CorsConfig, count/compare, surface
    CorsCfgMismatch/HasCorsCfgSet/ReplicatedCorsConfig, and include CORS in
    the bucket-stats aggregation filter
  - heal: healCORSMetadata, including nil -> delete propagation

Also harden the request/config path:
  - PutBucketCors validates the supplied Content-MD5/checksum via
    validateLengthAndChecksum
  - CORS validation rejects more than one wildcard per AllowedOrigin/
    AllowedHeader and enforces the 255-char rule ID limit
  - preflight responses Vary on Origin, Access-Control-Request-Method, and
    Access-Control-Request-Headers

Add focused tests for the CORS SR transport round-trip, the metadata
equality helper, and the new validation constraints.

Signed-off-by: h5vx <h5v@protonmail.com>
2026-08-27 12:46:47 +05:00
Feng Ruohang 590aeaa7d1 Merge pull request #74 from pgsty/codex/issue-48-baddigest
fix: align multipart completion checksum errors
2026-08-27 09:53:41 +08:00
Feng Ruohang 5d152416de fix: align multipart completion checksum errors
Return AWS-compatible errors for CompleteMultipartUpload checksum failures without changing the global streaming checksum mapping. Compare explicit multipart checksum types symmetrically, distinguish missing composite part checksums, and preserve the CRC64NVME canonicalization pending a direct AWS probe.

Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-27 09:05:49 +08:00
Feng Ruohang edc8be6ed1 deps: adopt Silo Go v7.3.1 stack
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-27 02:40:19 +08:00
Feng Ruohang 49c8aeac40 Merge pull request #37 from ycjlin/fix/listobjects-nosuchbucket-prefix
fix: ListObjects should return NoSuchBucket for prefix on missing bucket
2026-08-26 19:13:42 +08:00
Feng Ruohang e9c5340be9 fix: return NoSuchBucket from listing shortcuts
ListObjects shortcuts can return EOF before consulting storage, causing missing buckets to appear as empty listings. Verify bucket existence only on those shortcuts so the normal listing path retains the upstream fan-out optimization.

Cover ListObjects, ListObjectsV2, and ListObjectVersions at the object layer and verify HTTP 404 NoSuchBucket responses.

Fixes #32

Co-authored-by: Jason Lin <jason@JasondeMacBook-Air.local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-26 19:02:25 +08:00
Feng Ruohang 2e2377d1c6 Merge pull request #73 from pgsty/codex/issue-21478
fix: authorize user status changes by target status
2026-08-26 14:28:04 +08:00
Feng Ruohang 58735ee382 fix: authorize user status changes by target status
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-26 14:19:39 +08:00
Feng Ruohang a96116b128 Merge pull request #57 from Dansyuqri/feat-add-checksumtype-completemultipartupload-response
feat: add ChecksumType to CompleteMultipartUpload response struct
2026-08-26 10:27:54 +08:00
Feng Ruohang c4b9d38d8a Merge branch 'main' into feat-add-checksumtype-completemultipartupload-response 2026-08-26 10:15:15 +08:00
Feng Ruohang 8d76a255c4 fix: return the remote part checksum to federated UploadPartCopy (#72)
The legacy etcd federation branch of CopyObjectPartHandler forwards copied
bytes with minio-go Core.PutObjectPart, which can only recover a checksum
from response headers. After the server-side part checksum work, the remote
computes and persists the checksum, but an AWS-compatible UploadPart response
correctly omits a checksum the request did not supply, so the proxy had
nothing to put in CopyPartResult.

The destination now returns the non-empty checksum fields of the PartInfo
produced by that exact write, but only when the request carries the
minio-federated application token that getRemoteInstanceClient already
attaches. Ordinary UploadPart responses are unchanged, and the checksum type
is deliberately not returned because UploadPart does not carry it. The
User-Agent is a response-shape hint only: it never gates authorization,
visibility or validation, and it can expose nothing beyond the checksum of
the body the caller just uploaded.

Reading the checksum from the same PartInfo that produced the response ETag
also keeps the pair bound to one write, so a concurrent overwrite of the same
part number cannot publish another writer's checksum.

Tests cover the application token gating matrix including lookalike tokens,
the real minio-go response parser, concurrent overwrites of one part number,
and an in-process two-deployment probe that drives the federation branch
through the real getRemoteInstanceClient into a real PutObjectPartHandler for
both FULL_OBJECT and COMPOSITE uploads.

Fixes #64

Signed-off-by: Feng Ruohang <rh@vonng.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 01:06:50 +08:00
h5vx c5bc57b7a3 fix: gofumpt formatting and record CORS symbols in rebrand baseline
Reformat cors_test.go per gofumpt and regenerate the rebrand-guard
compatibility baseline to record the per-bucket CORS feature's new
exported symbols (internal/bucket/cors types and BucketMetadata/
BucketMetadataSys additions).

Signed-off-by: h5vx <h5v@protonmail.com>
2026-08-25 19:24:44 +05:00
h5vx 3814818537 fix: address CORS final-review findings (multi-rule preflight, raw GET, e2e test)
Signed-off-by: h5vx <h5v@protonmail.com>
2026-08-25 19:24:44 +05:00
h5vx 7a49a7a3da feat: enforce per-bucket CORS with global fallback
Signed-off-by: h5vx <h5v@protonmail.com>
2026-08-25 19:24:44 +05:00
h5vx ff3395d3c6 feat: implement S3 per-bucket CORS handlers
Signed-off-by: h5vx <h5v@protonmail.com>
2026-08-25 19:24:44 +05:00
h5vx ce4525632f feat: persist per-bucket CORS config in bucket metadata
Signed-off-by: h5vx <h5v@protonmail.com>
2026-08-25 19:24:44 +05:00
h5vx 1c9a2431fe feat: add internal/bucket/cors CORS config type and matching
Signed-off-by: h5vx <h5v@protonmail.com>
2026-08-25 19:24:44 +05:00
Feng Ruohang f2520f3346 fix: return checksums from CopyObject
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-24 20:59:27 +08:00
Feng Ruohang 05df6e70d7 fix: preserve transform state on metadata-only copies
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-24 20:46:31 +08:00
Feng Ruohang c0e7159771 fix: checksum CopyObject data before compression
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-24 20:33:54 +08:00
Feng Ruohang 56c67dacf1 Document bare ARN policy hardening
Record the new strict named-policy and service-account write behavior, the additional admin-policy validation it activates, the compatibility boundaries retained for stored/imported/replicated policies, and the required manual rewrite for legacy bare ARN prefixes.
2026-08-24 18:01:22 +08:00
Feng Ruohang eee05a17c3 Reject bare ARN policies on admin writes
Use silo-pkg v3.12 strict validation when creating named policies and when creating or updating service-account session policies. Keep stored policy loads, IAM import, site replication, and STS inline policies on the permissive compatibility path.
2026-08-24 17:59:13 +08:00
Feng Ruohang 45eb2e423d fix(ci): align compatibility fixtures and generated credits 2026-08-24 15:03:45 +08:00
Feng Ruohang 6b0998157c fix: harden config environment file parsing
Trim whitespace around assignments, preserve whitespace inside matching quotes, validate portable variable names, and report redacted file-and-line diagnostics. Check config-file Setenv failures instead of silently ignoring invalid entries.\n\nFixes #65
2026-08-24 13:43:44 +08:00
Feng Ruohang 7fea6d5a5f fix: compute multipart part checksums server-side
Accept checksum-enabled UploadPart and UploadPartCopy requests when clients omit optional per-part checksum headers. Compute over the logical plaintext stream, persist the result, and return it from CopyPartResult while preserving client checksum validation.\n\nRefs #46; follow-ups #63 and #64 remain out of scope.
2026-08-24 12:20:52 +08:00
Feng Ruohang 68eeb002f6 chore: record notification migration compatibility symbols 2026-08-24 12:20:13 +08:00
Feng Ruohang c565987b9c docs: align database notification migration guidance 2026-08-24 02:29:22 +08:00
Feng Ruohang f1ba683582 fix: require DSNs for legacy database notifications
Reject pre-KV PostgreSQL and MySQL targets that lack a canonical connection string, propagate the typed migration error to the fatal startup boundary, and stop emitting unregistered discrete connection keys.\n\nCloses the implementation for #53; release and issue closure remain separate gates.
2026-08-24 02:22:20 +08:00
Feng Ruohang 43f4bb7ed4 chore(deps): align the SILO Go dependency stack
Standardize the related SILO components on Go 1.27 tooling, etcd 3.7.1, current Go-maintained modules, shared runtime versions, and explicit security and portability pins.

Keep the shared package Go 1.26 consumer floor, isolate lint tooling from product dependency selection, and preserve upstream-compatible import paths.
2026-08-24 01:08:46 +08:00
mr javad seydi 7a060cab1e feat(ilm): relocate hot objects across server pools by GET frequency
Keep NVMe/HDD pool pairs useful without remote tiering: promote objects that
are read often, demote previously moved objects once they go idle, and leave
the feature off until operators set a two-pool topology.

Signed-off-by: mr javad seydi <seydi.birjand@gmail.com>
2026-08-15 13:48:22 +03:30
Shooks d014a12cff feat: add ChecksumType to CompleteMultipartUpload response struct
Signed-off-by: Shooks <justanormalme@gmail.com>
2026-08-09 17:52:38 +08:00
Feng Ruohang 100e2e57a7 docs: restore the legacy-user drop-in path in both READMEs
Condensing the READMEs dropped the systemd drop-in that keeps data ownership
stable when silo.service takes over an existing minio.service, which the
rebrand guard pins in both files precisely so it cannot vanish quietly. The
path is back as a clause in the install note rather than a section, so the
guard is satisfied and the migration-critical detail stays discoverable in
the repository, not only on the portal.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-07 19:46:28 +08:00
Feng Ruohang 6e20e74774 docs: record contributors and rework the READMEs around them
GitHub generates no contributor graph for forks, so this fork had no
attribution record at all. CONTRIBUTORS.md becomes that record, in three
tiers: the four contributors with code merged into main, each listed with
the pull request and the commit that carries their authorship; the five who
opened pull requests; and the twenty-two who filed the bug reports and
compatibility findings that shaped the releases.

Both READMEs are condensed and gain a Contributors section that shows the
avatars and points at the full record, with README_ZH realigned to README
section for section.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-07 19:34:16 +08:00
Feng Ruohang 3be10fcc1a feat(ci): add a manual finalize lane for signed Draft packages
GPG-signing the RPMs rewrites their bytes after release.yml has already
generated SBOMs, the packages checksum manifest, and attestations from
the as-built files. The new workflow_dispatch-only finalize-release lane
runs between signing and publishing: it refuses non-Draft releases,
verifies the signed RPMs against their sha256 sidecars and the committed
PGSTY public key, confirms every other package still matches the
original manifest, regenerates the RPM SBOMs and the manifest from the
published bytes, cosign-signs the manifest under the workflow identity,
and attests the finalized set before replacing the assets in place.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 22:21:17 +08:00
Feng Ruohang b14ea22aa8 fix(ci): match checksum manifest entries exactly in docker-release
The substring grep for the archive line also captured the archive's
.sbom.json entry, whose file is deliberately not downloaded in this
lane, so the sha256sum check failed on every run. awk now matches the
manifest filename column exactly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 22:21:17 +08:00
Feng Ruohang 86a7782900 docs(readme): update the docker pulls badge
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 22:19:41 +08:00
Feng Ruohang 4679314556 build(packages): unify the rpm and deb release segment as 1PGSTY
The rpm carried a bare -1 release while the deb carried none. Both now
ship the PGDG-style 1PGSTY segment: silo-VER-1PGSTY.arch.rpm and
silo_VER-1PGSTY_arch.deb, with nfpm rendering it as the RPM Release tag
and the Debian revision respectively. The apk stays bare because Alpine
pkgrel admits only -r<integer>.

sign-release-rpms.sh declares the value once as expected_release;
package-release.sh names artifacts from its own PKG_RELEASE copy, and
test-release.yml evals the signing script's value and asserts both the
download names and the packaged RPM Release/DEB Version against it, so
the copies cannot drift apart silently.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 21:02:15 +08:00
Feng Ruohang 062a91beed fix: pin the CREDITS module closure to the shipped linux target
The go list -deps closure is GOOS-specific, so the CREDITS generated on
darwin (carrying go-m1cpu) could never match a Linux regeneration
(ethtool, numcpus), and the check-gen guard failed on its first CI run.
Generation now pins linux/amd64 with the release build tags, so the
output is identical on every host.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 19:48:33 +08:00
Feng Ruohang 16b78eb4e5 build: point the release surfaces at RELEASE.2026-08-06
Bundle mcli RELEASE.2026-08-06T00-00-00Z in the container image (pin
and both architecture digests; interop with the new server was
verified end to end during release acceptance) and align the CI
fixture default in install-mcli.sh.

Point the Helm chart defaults at the first real silo image tag:
docker.io/pgsty/silo is a fresh repository, so the inherited
RELEASE.2026-08-04 default could never pull. Chart version moves to
7.0.1 with appVersion RELEASE.2026-08-06T00-00-00Z; the packaging
tripwire in verify-helm-migration.sh follows.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 19:10:56 +08:00
Feng Ruohang 9462cce16e fix: rewrite the healthcheck output selection as a switch
gocritic's ifElseChain check fails make lint on the --json/--quiet
branch introduced by b6d47b739; the quality job runs make lint on
every push to main. Behavior is unchanged (verified by re-running the
json/quiet success and failure cases byte-for-byte).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 19:09:26 +08:00
Feng Ruohang b6d47b739c fix: harden healthcheck and distroless lanes per adversarial review
Findings from an adversarial review (Codex, gpt-5.6-sol at max effort)
of 2ff594f4b and 4c34d2309, each independently verified before fixing:

- SBOM generation: buildx attaches a provenance attestation, so every
  per-arch digest names an OCI index; Syft's platform default on an
  amd64 runner cannot resolve an arm64-only index and the step dies.
  Pass --platform explicitly on all four Syft calls (the two classic
  lanes had the same latent defect - the renamed workflow has not run
  yet, which is why it never fired).
- Release ordering: the HEALTHCHECK survival check now runs against
  the pushed architecture image before the versioned and rolling
  multi-arch manifests are created, so a broken health config blocks
  their promotion; the comment now states honestly that the
  arch-suffixed tags are already public at that point.
- Gate assertions: tar's member-argument mode exits non-zero on any
  missing name, which under pipefail masked a found forbidden file
  when exactly one of them existed; -tv prints symlinks as
  'name -> target', defeating $-anchored greps; and the licenses
  check proved only one-of-three. Export the rootfs once and assert
  every required and forbidden entry individually (busybox/sh and
  usr/bin/mc[li] now covered), and match the image healthcheck as an
  exact array instead of a substring.
- Probe target vs CLI-configured servers: a probe process cannot see
  PID 1's argv, so --url gains EnvVar MINIO_HEALTHCHECK_URL as the
  documented way to point the baked-in HEALTHCHECK at a server whose
  address/TLS comes from command-line arguments (verified end to end:
  server on --address :9010, env var alone turns the container
  healthy). Baseline regenerated for the new env token.
- IPv6 zone identifiers: serialize probe URLs via url.URL.String()
  so [fe80::1%eth0]:9000 becomes a valid %25-escaped URL (tests added).
- Boolean flags: read --json/--quiet via Bool() so --json=false is
  false, instead of IsSet() which treats any occurrence as true.
- Docker's HEALTHCHECK timeout raised to 10s: an outer deadline equal
  to the probe's own 5s always SIGKILLed the probe before it could
  print its diagnostic line.
- test-release path filter now also triggers on cmd/healthcheck-main.go
  and cmd/main.go, so subcommand regressions run the image gate.

Not adopted: require_text's comment-insensitivity in verify-rebrand.sh
(snapshot-tripwire by design, consistent with its other assertions -
the semantic check lives in the CI gate now), and full
staging-then-promote tag publishing (a workflow-wide redesign shared
with the classic lanes, tracked as follow-up).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 17:27:22 +08:00
Feng Ruohang 4c34d23099 build(docker): add the distroless image variant as a pilot
Publish pgsty/silo:<RELEASE>-distroless (and a rolling :distroless tag)
alongside the classic image: gcr.io/distroless/static-debian12 plus
exactly one program, the silo binary. No shell, no mc, no curl, no
entrypoint script - the binary is the ENTRYPOINT and the baked-in
exec-form HEALTHCHECK runs 'silo healthcheck ready'. The classic image
and its mc-based health checks are deliberately unchanged.
Design: silo.pgsty.com/compatibility/feature/healthcheck/

/data is created in the image layer, world-writable, because Docker
seeds fresh volumes from the layer mountpoint and no entrypoint exists
to repair ownership at runtime (issue #55); the parent directory is
copied from a throwaway busybox stage since COPY of a directory copies
contents rather than the entry itself, which would silently leave
/data at root:0755 and break every non-root run. The MINIO_USERNAME
drop-user path is not supported in this variant; use --user.

test-release.yml now builds the real Dockerfile.distroless on every
gate run (it has no download stages, so it stays offline) and asserts:
the HEALTHCHECK survives into the image config, /data ships 0777, no
shell and no /usr/bin/minio are present, Docker's health state turns
healthy from the baked probe alone, the probe binary execs without a
shell, and SIGTERM still stops the server gracefully - as root and as
--user 1001:1001. docker-release.yml gains the distroless build lanes,
multi-arch manifests, SBOM and provenance attestations, and a
release-blocking check that the pushed manifest still carries the
HEALTHCHECK (a Docker extension absent from the OCI image spec).

Verified locally on linux/arm64: full gate assertions plus bare
'docker run ... healthcheck' exit-code semantics and --version
passthrough.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 16:40:04 +08:00
Feng Ruohang 2ff594f4bb feat: add the native silo healthcheck subcommand
Add 'silo healthcheck [live|ready|cluster|cluster-read]', a thin
anonymous HTTP client for the server's own /minio/health/* endpoints,
so containers without a shell, curl, or mc can still run health
checks. Design: silo.pgsty.com/compatibility/feature/healthcheck/

The check vocabulary maps 1:1 onto the health API paths; the probe
target is derived from the server's own --address/MINIO_ADDRESS
contract with HTTPS auto-detected from the certs directory, and can
be overridden with --url. Exit codes are 0/1 only (Docker reserves 2);
diagnostics (x-minio-server-status, quorum headers) go into a single
output line for docker inspect. The request is strictly anonymous (a
credentialed request would be rejected by the reserved-path guard),
the transport bypasses HTTP_PROXY, and certificate verification is
skipped to match kubelet HTTPS probe behavior. Cluster checks default
to a 15s deadline so the server's 10s cluster_deadline can elapse.

Compatibility notes: the preserved /minio/health/* path literals and
the MINIO_ADDRESS env var are upstream wire/config surface, reused on
purpose; the rebrand-guard baseline is regenerated for the new route
literals (tests included) with zero new exported symbols. The docker
entrypoint argv translation learns the new command name.

Verified: unit tests, entrypoint tests, go vet, plus an end-to-end
run against a live server covering all four checks, --maintenance
(412), --json, usage errors, unreachable and timeout paths.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 16:32:16 +08:00
Feng Ruohang 219670d317 build: ship LICENSE and NOTICE in every package and image
The RPM, DEB and APK payloads carried no license material at all, and
the OCI image copied LICENSE and CREDITS but not NOTICE. For an AGPL
distribution that is not defensible packaging. Every delivery channel
now carries the same license set:

- nfpm packages install LICENSE and NOTICE under /usr/share/doc/silo/.
  The entries are declared once per packager because nfpm only honors
  type: license on rpm and silently drops such entries from deb and apk
  -- the rpm keeps its %license flag, deb and apk carry plain files at
  the same path. CREDITS stays out of the OS packages deliberately: at
  1.8MB it would dominate an otherwise ~40MB payload.
- The OCI image gains /licenses/NOTICE next to the existing LICENSE and
  CREDITS, in the Dockerfile and in both context-staging paths (the
  make docker target and docker-release.yml).
- The GoReleaser archives list their files explicitly; the default
  globs would keep missing NOTICE.

NOTICE's first line now reads 2015-2025 to match the banner constants
fixed in b57275be3 -- it had kept upstream's stale 2015-2023 claim --
and verify-rebrand.sh pins both NOTICE copyright lines.

The payload locks move in the same commit: sign-release-rpms.sh expects
the six-file listing, and the test-release validation asserts presence,
count, the rpm %license flag, and sha256 equality of the packaged
LICENSE and NOTICE against the repository files across all three
formats and both architectures.

Also dropped the unit's WorkingDirectory=/usr/local, a leftover from
upstream's /usr/local/bin install layout. Debian policy 9.1.2 keeps
packages out of /usr/local entirely; the binary already installs to
/usr/bin and the service does not depend on its working directory.

Verified locally with nfpm v2.47.0 (the CI-pinned version): built all
six packages via package-release.sh, then ran the test-release
validation step verbatim (extracted from the workflow YAML) and the
signing script's payload comparison against them in an Ubuntu 24.04
container -- both pass. Built the arm64 image from the real
Dockerfile.goreleaser and confirmed /licenses holds CREDITS, LICENSE
and NOTICE. make rebrand-guard passes with the new NOTICE pins.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-06 15:03:35 +08:00
Feng Ruohang 6bd9cf77ef build: regenerate CREDITS from the linked module set and guard it in CI
The shipped CREDITS predated the fork's dependency work: 38 of the
modules in the current go.mod, among them lestrrat-go/jwx, go-tpm,
go-spiffe and the charmbracelet family, had no entry at all, while
entries lingered for modules no longer in the build. An attribution
file that silently drifts from go.mod is worse than none, so it is now
generated, not curated.

buildscripts/gen-credits.sh (make credits) rebuilds the file from the
licenses of every module go list -deps reports for the main package --
the set actually linked into the silo binary, so test-only and tool
dependencies stay out. Entries keep the established name / URL /
license-text layout. New over the old file:

- Modules replaced in go.mod are annotated with the repository that
  actually serves them, so the pgsty/silo-console, pgsty/mc and
  pgsty/silo-pkg forks are named next to their upstream import paths.
- Bundled NOTICE files are reproduced after the license text, which
  Apache License 2.0 section 4(d) requires when redistributing; 23
  modules carry one.
- minio/colorjson, minio/csvparser and minio/filepath publish no
  license file at all; they repackage Go standard library code and
  their sources carry the Go Authors' BSD-style header, so the Go
  project license is reproduced for them with a note saying why.
- The Go license text itself comes from the pinned golang.org/x/sys
  module rather than GOROOT, because Homebrew's Go omits
  GOROOT/LICENSE and the module copy is version-locked.

check-gen now runs the generator and fails on a CREDITS diff, the same
treatment go.mod and go.sum already get, so dependency changes cannot
leave stale attributions behind. Output is deterministic: two runs are
byte-identical, and LC_ALL=C sorting plus version-pinned inputs keep it
that way across machines.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-06 15:03:09 +08:00
Feng Ruohang a6d6d9b028 build: update the embedded Console to SILO Console v2.1.1
Pins pgsty/silo-console at 72fc0a5e, the v2.1.1 tag, replacing the
v2.1.0 commit from earlier today. The pseudo-version form stays because
the compatibility module path github.com/minio/console carries no /v2
suffix, so a v2.x semver tag is not directly selectable.

v2.1.1 is 5 commits over v2.1.0. The one substantive change for the
server is a fix that keeps unresolved i18n placeholders out of the
metrics dashboard legends; the rest is the v2.1.1 changelog, dropping
the container-image advertising from the Console docs, and regenerated
embedded web assets.

Verified: go mod tidy -compat=1.26 is a clean no-op afterwards, which is
what make check-gen asserts in CI, and go build ./... and go vet ./...
pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-06 14:16:35 +08:00
Feng Ruohang 05be686b84 docs: update README and svg log 2026-08-06 14:08:01 +08:00
Feng Ruohang b57275be34 docs: adopt the no-CLA plus DCO policy and fix copyright terms
Brings the server in line with the contribution and copyright policy
already adopted in pgsty/mc.

Copyright terms were derived from the clock: startupBanner overwrote the
ldflags-injected CopyrightYear with time.Now().Year() and printed it as
the end of MinIO, Inc.'s term, so every January would have extended the
upstream copyright claim past the 2025 end of upstream development, and
release builds silently discarded the injected year. Both banners now
credit MinIO, Inc. for 2015-2025 and PGSTY from 2025 through the
release-stamped year, falling back to the current year in source builds.
NOTICE names PGSTY as the holder of the fork's modifications, matching
the banners and the packaging vendor field.

Contributions are accepted inbound=outbound under AGPL-3.0-or-later with
no CLA - the core is Copyright MinIO, Inc., so the combined work can
never be relicensed and a CLA would buy nothing. What a fork carrying a
downstream delta does need is provenance, so DCO 1.1 sign-off is now
mandatory and enforced by a workflow that rejects unsigned non-bot
commits. CONTRIBUTING documents sign-off, repair, cherry-pick provenance,
dual copyright headers, trailer preservation across squash merges, and
that assistive-tooling trailers carry no authorship or copyright claim.

verify-rebrand.sh pins the copyright split and the policy files so
neither can regress.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
2026-08-06 12:43:58 +08:00
Feng Ruohang 6740e6978f ci: move the workflow actions onto the Node 24 runtime
actions/checkout@v4 and actions/setup-go@v5 declare runs.using: node20, so
every Go CI job printed the runner's Node 20 deprecation warning. The
release, test-release and docker-release workflows carried six more node20
actions that never appeared in that warning only because those workflows
run on a tag or a dispatch rather than on every push: upload-artifact v4,
goreleaser-action v6, build-push v6 and the docker qemu/buildx/login v3
line. All of them move to the current majors, which declare node24. The
SHA-pinned rest already qualified -- attest and sbom-action are node24,
cosign-installer is composite.

Checked against what those majors removed. build-push v7 dropped
DOCKER_BUILD_NO_SUMMARY and DOCKER_BUILD_EXPORT_RETENTION_DAYS and
setup-buildx v4 dropped its deprecated inputs, none of which are used here.
checkout v7 refuses to check out fork PRs under pull_request_target and
workflow_run, and no workflow here triggers on either.

setup-go v6 exports GOTOOLCHAIN=local, the one change with teeth. go.mod
declares `go 1.26.5` and carries no toolchain directive, every job resolves
that same version through go-version-file, and nothing in the module graph
asks for more, so there is nothing left for the go command to fetch:
`go build ./...` passes under GOTOOLCHAIN=local. The tools installed from
outside the module are in range too -- nfpm v2.47.0 requires 1.26.4 and
govulncheck v1.6.0 requires 1.25.0 -- and golangci-lint arrives as a
prebuilt binary from upstream's install script, not through the toolchain.

The runner floor that comes with node24 (v2.327.1) is met by GitHub-hosted
runners, and every job here is ubuntu-latest.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 10:46:29 +08:00
Feng Ruohang 62717d7bf6 build: update the embedded Console to SILO Console v2.1.0
Pins pgsty/silo-console at 343d9442, the v2.1.0 tag, replacing the v2.0.0 commit
from 2026-08-04. The pseudo-version is still the form used here because the
compatibility module path github.com/minio/console carries no /v2 suffix, so a
v2.x semver tag is not directly selectable; the comment above the directive now
names the release the commit actually corresponds to.

v2.1.0 is 23 commits over v2.0.0. The substantive ones for the server are a
hand-rolled EN/ZH i18n layer with a language toggle covering login, navigation,
dashboard, buckets and object browser, identity and configuration, event
destinations, observability and admin tools; a dashboard migration to MinIO
Metrics V3; and removal of the remaining SUBNET references from health
reporting, which matches this repository's own decision to close the upstream
telemetry surfaces. The rest is UI polish and regenerated embedded web assets.

Verified: go mod tidy -compat=1.26 is a clean no-op afterwards, which is what
make check-gen asserts in CI, and go build ./... and go vet ./... pass.

Not included here: the GitHub Action version bumps and the .gitignore entry
sitting in the working tree are someone else's change and stay uncommitted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 10:04:23 +08:00
Feng Ruohang f1c77d5a2b ci: let the READMEs name the old repository without pointing at it
The previous commit added a rename note to both READMEs and broke the gate I
added one commit earlier. That gate rejected the string pgsty/minio anywhere
outside three guard files, which was the wrong invariant: the point is that the
old name must never be a live target, not that it must never be spoken. A README
explaining where the MinIO-named artifacts went has to name them, and doing so
is the opposite of stranding a reader on a dead repository.

Split into two rules that say what is actually meant:

- No live URL may resolve to the old repository anywhere, READMEs included -
  github.com/pgsty/minio and hub.docker.com/r/pgsty/minio are rejected outright.
  This is the rule that protects users.
- The bare name is allowed only in five files: the pinned pre-rebrand image
  digest in the upgrade test, the two guards that refuse a legacy image, and the
  two READMEs.

Both halves are negative-tested: adding a github.com/pgsty/minio link to
README.md fails rule one, and adding a bare pgsty/minio to docs/docker/README.md
fails rule two.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 09:47:48 +08:00
Feng Ruohang c47733abc4 docs: point readers at the archived minio branch
Someone arriving at main after the rename has one question the README did not
answer: where did the MinIO-named builds go. Both READMEs now say it directly —
the archived minio branch and the releases up to RELEASE.2026-08-04T00-00-00Z
hold the artifacts maintained under the original identity, and those assets and
the docker.io/pgsty/minio image stay published and unmodified.

The note also records the rename itself with its date, and repeats the one fact
that determines whether an operator has any work to do: only product and
artifact names changed. MINIO_* variables, minio_* metrics, x-minio-* headers,
/minio/* routes and the on-disk format are the same.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 09:46:31 +08:00
Feng Ruohang c46b16ec62 chore: cut over to pgsty/silo and main
The transitional references land in one commit, because they are only correct
together: the repository is pgsty/silo, its default branch is main, and nothing
in the tree should point a user at the old names.

Changed:

- Workflow branch filters. go.yml and vulncheck.yml gated on `branches: master`
  for both push and pull_request, so renaming the default branch would have
  taken automatic CI offline with no error and no signal - the workflows would
  simply never trigger again. They now name main.
- Release target. goreleaser's `release.github.name` becomes silo, which is
  what actually decides where a tagged build publishes. sign-release-rpms.sh's
  GH_REPO default follows.
- The OCI `image.source` label, the Helm chart `sources` entry, the security
  advisory link in the issue-template config, and the go.mod comment citing the
  LDAP TLS fix.
- 115 occurrences across README, README_ZH, SECURITY, CONTRIBUTING and 30 docs
  pages, including 72 links that also carried the master branch in their path.
  Those matter most: GitHub redirects clone, fetch, push and web URLs after a
  rename, but raw.githubusercontent.com does not, and neither follows a branch
  rename - every one of those links would 404 twice over.
- Three error strings in cmd/erasure-sets.go, cmd/storage-errors.go and
  internal/config/errors.go that print an issue URL to operators. These are Go
  string literals inside rebrand-guard's brand allowlist, so the baseline is
  regenerated. The regeneration removes exactly those three entries and adds
  none; all twelve other protected sets, including the 9014 exported symbols,
  are byte-identical.
- The transitional-naming disclaimers in README, README_ZH, SECURITY and
  CONTRIBUTING are dropped, since they no longer describe anything.

Deliberately unchanged, all three because they exist to reject or freeze the old
name rather than to point at it:

- buildscripts/minio-upgrade.sh pins pgsty/minio@sha256:b6bfe72... - the frozen
  pre-rebrand image is the control group for the MinIO-to-Silo upgrade test.
- helm-migration-guard rejects any rendered container still pulling pgsty/minio.
- verify-rebrand.sh rejects the same in the delivery surfaces.

Also unchanged: docs/config/README.md links to pgsty/mc/blob/master, and that
repository's default branch really is still master. It moves when mc does.

verify-rebrand.sh gains three assertions so this cannot silently regress: no
source reference may name pgsty/minio outside the three allowlisted guards, no
link may target pgsty/silo's master branch, and go.yml and vulncheck.yml must
filter on main. Both new rejections were negative-tested - reintroducing a
master branch filter and adding a pgsty/minio URL each fail the gate with the
specific message.

This commit assumes the rename actually happens. Until the GitHub branch and
repository renames are executed, the links it introduces do not resolve.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 09:28:06 +08:00
Feng Ruohang fd2ca1c6d2 docs: rebrand the repository documentation, templates and dashboards
README, README_ZH, SECURITY, COMPLIANCE, CONTRIBUTING, NOTICE,
code_of_conduct, the vulnerability and PR-etiquette documents, the GitHub issue
and pull request templates, and the docs/ tree all present Silo as the product.
The Grafana dashboards under docs/metrics/prometheus/grafana/ have their panel
titles and descriptions rebranded while every minio_* query, label and
expression is left alone, so existing alerts and recording rules keep matching.

The distinction the review demanded is applied per hit rather than by
search-and-replace:

- Product and command text becomes Silo and silo: install and run instructions,
  systemd examples, compose services, download links, badges.
- Protocol and interface text keeps MinIO: MINIO_* variables, minio_* metrics,
  x-minio-* headers, /minio/* routes, .minio.sys, arn:minio, and API field and
  error names.
- Attribution keeps MinIO and gains the fork's own: the AGPL obligations,
  original copyright, CREDITS and NOTICE stay, with the modification notice
  added alongside rather than replacing them.
- Historical and third-party references are left as facts, not rewritten for
  brand tidiness.

README and README_ZH each carry an explicit non-affiliation notice, document
the side-by-side package migration including the
/etc/systemd/system/silo.service.d/10-legacy-user.conf drop-in for keeping a
legacy UID/GID, and state that recursive chown is never performed. The trademark
attribution uses the policy's approved "based on MinIO technology" wording, not
the shortened form the policy rejects.

github.com/pgsty/minio links are left in place and labelled transitional. The
repository has not been renamed, and rewriting them now would produce documented
URLs that 404 until the cutover; they change in the cutover commit together with
the goreleaser release target, the OCI source label and the raw-content branch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 08:49:30 +08:00
Feng Ruohang 6613c2a3cb test: pin the external fixtures and run the suites against the silo binary
The test and verification scripts invoked ./minio and pulled their tooling from
upstream infrastructure with no integrity check. Every `curl | tar` of a client
or an old server binary was an unverified execution path in a script that
regularly runs as a privileged user, and several fetched a floating "latest".

Two installers replace all of it:

- install-mcli.sh resolves a pinned pgsty/mc release, downloads the archive and
  its checksum manifest, requires exactly one valid manifest entry for the
  asset, verifies it, and installs. MCLI_BIN with a mandatory MCLI_SHA256 lets
  an offline or air-gapped run supply its own binary, still checksum-checked.
- install-verified-fixture.sh takes source, expected SHA-256 and target, and
  refuses anything that does not match. Sources may be a URL or a local file.

Every script that previously downloaded mc now calls install-mcli.sh. The three
places that genuinely need an upstream artifact - the old MinIO server binary
for the LDAP IAM upgrade-import test, the 2021 mc for the three-site
replication test, and the functional-tests.sh fixture - go through
install-verified-fixture.sh with the digest recorded inline. Those dl.min.io
URLs remain on purpose: they are historical upstream artifacts needed to prove
upgrade compatibility, and they are now pinned and verified rather than
trusted.

The scripts otherwise switch to ./silo, silo.service, the silo container and
compose service names, and SILO_CONFIG_DIR. run-multi-site-minio-idp.sh is
renamed to run-multi-site-silo-idp.sh with the Makefile target following.
buildscripts/minio-upgrade.sh keeps its name and its `minio server` argv - it
exists to test the MinIO-to-Silo upgrade, so the old side must stay old - but
it is now pinned to an image digest rather than a tag, and its `docker system
prune` and `docker volume prune` calls are removed. Those ran unfiltered
against the developer's whole Docker installation; the resiliency tests had the
same problem and lose their prune and `docker ps -q` sweeps too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 08:48:58 +08:00
Feng Ruohang bd8df51665 ci: gate the rebrand on compatibility, packaging and provenance evidence
A rebrand of this size cannot be reviewed by reading the diff. The risk is not
what was renamed but what was renamed by accident: an environment variable, a
metric, a header, a route, a storage marker or an exported symbol that some
deployment depends on. So the invariants get a machine check.

rebrand-guard extracts the identifiers a product rename must not touch - the
go.mod module path, github.com/minio/* imports, MINIO_* variables, minio_*
metric literals, x-minio-* headers, route string literals, the hardcoded grid
routes in internal/grid/manager.go, the reserved bucket root, .minio.sys
markers, arn:minio and minio:s3 policy values, every exported symbol in cmd/
and internal/, and every branded string literal - and compares them to a
checked-in baseline. Any removal or addition fails the build, in both
directions, so a future change cannot quietly drop one either.

The recorded delta against the pre-rebrand tree is: module path, 137 imports,
19 metrics, 15 storage markers, 58 policy values and all 9014 exported symbols
byte-identical; environment 437 -> 436, headers 85 -> 84, routes 331 -> 330.
Each of those three is accounted for. MINIO_CONFIG_DIR and MINIO_VERSION were a
shell local and a compose variable in test scripts, never server settings.
x-minio-common was a docker-compose extension key, never an HTTP header.
/api/health/upload was the SUBNET upload path in the deleted callhome.go.

verify-rebrand.sh covers what the identifier guard cannot see: that the retired
upstream delivery paths stay deleted, that the packaging and container files
name silo, that silo.service keeps its Conflicts and both EnvironmentFile
lines, that silo.env contains no active assignment that would shadow the legacy
file, that no active delivery surface still publishes pgsty/minio, and that no
non-test Go source contains a dl.min.io, subnet.min.io, api.min.io,
slack.min.io or play.min.io URL.

Both, plus the entrypoint argv test, run in go.yml, release.yml and
test-release.yml. test-release.yml also gets the path filters right: it was
watching minio.service, which no longer exists, so the packaging gate would
have gone silently dormant. It now watches the unit, defaults, sysusers,
lifecycle scripts, helm chart and both guards, and asserts the four-file
payload, the absence of any cross-name Provides/Obsoletes/Conflicts, and that a
container started with the legacy `minio server` argv really runs silo as PID 1.

release.yml and docker-release.yml additionally verify that the checked-out
revision matches GITHUB_SHA before building. GitHub's OIDC certificate records
GITHUB_SHA, not the ref passed to actions/checkout, so a manual dispatch from
the wrong ref would have produced provenance describing different source from
the bytes being published. They now emit SBOMs and attestations for archives,
packages and images, and docker-release verifies the release workflow's
attestation on every artifact it consumes. Images publish to pgsty/silo.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 08:48:39 +08:00
Feng Ruohang e071bb77e4 build(helm): replace the minio chart with a silo chart that preserves identity
helm/minio becomes helm/silo: chart name silo, version 6.0.0 -> 7.0.0, the
MinIO wordmark icon replaced with the project's own, image.repository and
mcImage.repository pointing at pgsty/silo, and the container command changed to
silo. User-visible titles, comments and documentation links are rebranded. The
MINIO_* environment variables and every existing values key are kept - the
first Silo chart is a rename, not a values-schema migration.

The hard problem is that a chart rename normally rewrites Kubernetes resource
identity, and a StatefulSet's selector and volumeClaimTemplate are immutable.
An existing release upgraded carelessly would either fail or orphan its PVCs.
Two things address that:

- Templates no longer derive the container name from .Chart.Name. It comes from
  a helper, so nameOverride can pin it, which means an existing release can be
  upgraded with nameOverride=minio, fullnameOverride=<existing-fullname> and
  serviceAccount.name=minio-sa and render byte-stable identity while switching
  chart and image.

- helm-migration-guard and verify-helm-migration.sh make that a gate rather
  than a documented hope. The script lints the chart, renders it in distributed
  and standalone modes plus the optional templates, then renders the legacy
  chart from a pinned commit and the new chart with those three overrides and
  compares resource identity. The guard additionally rejects any rendered
  container still pulling pgsty/minio or invoking /usr/bin/minio. It runs
  through a pinned alpine/helm image when helm is not installed locally, so the
  gate does not depend on the developer's machine. Currently green over 7
  compared resources.

Rollback is asymmetric and the README says so: the old chart with the new image
survives via the entrypoint argv shim, but the new chart with an old MinIO
image does not, because `silo server` is not a command that binary knows. Only
`helm rollback` is supported, never an image-only downgrade.

Not addressed here: the default image tag is pgsty/silo:RELEASE.2026-08-04T00-00-00Z,
which does not exist yet. The chart must not be published until the first Silo
image is pushed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 08:48:17 +08:00
Feng Ruohang 30749911bd build(docker): ship the silo binary and translate the legacy argv command
Dockerfile.goreleaser now copies silo to /usr/bin/silo, defaults to
CMD ["silo"], and labels the image as Silo. MINIO_UPDATE_MINISIGN_PUBKEY is
removed from the image environment: with the updater permanently disabled it
was dead weight, and leaving upstream's verification key in a Silo image
implied a trust relationship that does not exist. The MINIO_* runtime
environment variables, ports, volume and health endpoints are unchanged.

The image keeps shipping mcli with an /usr/bin/mc symlink. That is the client,
not a MinIO-branded alias for the server binary, and the Helm post-install job
and existing container scripts call it by name.

docker-entrypoint.sh translates a legacy first argument: `minio server /data`
becomes `silo server /data`, so an existing `command: minio server ...` in
compose or a Pod spec keeps working across the image swap. The translation is
argv-level only - no file named minio is installed, and an explicitly
overridden `entrypoint: /usr/bin/minio` still fails, which is the honest
outcome since that path genuinely no longer exists.

The entrypoint also fixes an unrelated startup hazard it was already carrying:
when the image runs under an arbitrary UID, HOME points at an unreadable /root
and the server probes its default config directory during initialization. It
now falls back to /tmp when HOME is unset, /root, missing or unwritable.

docker-entrypoint_test.sh pins all of it - empty argv, legacy minio, native
silo, bare flags, and an explicit shell command that must stay explicit. It is
wired into make rebrand-guard and into the go.yml, release.yml and
test-release.yml gates, so removing the shim breaks CI rather than breaking
users. Passing locally.

download-static-curl.sh gains checksum verification for the curl it fetches
into the build stage.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 08:47:59 +08:00
Feng Ruohang 15ab10833b build: rename the delivery artifacts to silo and complete the package payload
Everything a user installs is renamed, and the package finally installs enough
to be startable on a clean host.

Artifact names
- goreleaser.yml: build id, binary, archive and checksum manifest become silo_*.
  release.github.name stays "minio" with a comment - the GitHub repository has
  not been renamed yet, and pointing at pgsty/silo before the rename would 404.
  Also adds per-archive SPDX SBOMs and a keyless cosign signature over the
  checksum manifest, so the signed manifest covers archives and SBOMs together.
- nfpm.yml: package name silo, and the binary moves from /usr/local/bin/minio
  to /usr/bin/silo. /usr/local is not on the default PATH of a systemd unit and
  is not FHS-correct for a distribution package.
- package-release.sh, sign-release-rpms.sh and verify-build-provenance.sh follow
  the new names; the RPM signing script asserts NAME=silo and the new four-file
  payload. nfpm is now invoked from the repository root so relative script paths
  in the config resolve regardless of the caller's directory.

Package relationships are deliberately empty
No Provides, Obsoletes, Replaces or package-level Conflicts. Obsoletes: minio
cannot distinguish a pgsty package from upstream's own identically named one,
so an unattended dnf upgrade could silently swap a different vendor's product
for this one. With no relationships, both packages coexist, their file sets do
not overlap, and migration and rollback are single explicit commands. The
mutual exclusion lives in the unit instead: silo.service carries
Conflicts=minio.service plus After=minio.service.

Payload, from two files to four
- /usr/bin/silo
- /usr/lib/systemd/system/silo.service
- /etc/default/silo, installed config|noreplace
- /usr/lib/sysusers.d/silo.conf

The old package shipped a unit referencing an account nothing created, so a
clean install could not start. postinstall.sh now creates the silo system
account through systemd-sysusers, useradd or BusyBox adduser in that order and
runs daemon-reload. It never stops a service, never chowns data and never
touches /etc/default/minio. preremove.sh disables silo.service only on a real
removal - Debian "remove", RPM 0, Alpine's dotted version - so upgrades leave
the running service alone. lifecycle_test.sh exercises both against a stubbed
PATH, so a green run cannot create an account or touch the host.

silo.service reads /etc/default/minio then /etc/default/silo, in that order, so
an existing node's MINIO_* values keep working and the new file overrides them.
The packaged silo.env therefore ships comments only: any active assignment
would shadow the legacy file with an empty value.

Makefile: build/install/install-race produce ./silo, and the docker target now
assembles a context from a locally built linux binary plus Dockerfile.goreleaser
instead of the deleted Dockerfile. The hotfix, hotfix-push, docker-hotfix and
docker-hotfix-push targets are gone - they downloaded upstream's pkger, signed
with upstream's minisign key and scp'd to dl-N.minio.io. verifiers now depends
on a new rebrand-guard target.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 08:47:44 +08:00
Feng Ruohang 77bdc4c0cd feat(server): present Silo identity and close the inherited upstream services
Two coupled changes that must land together, because the same files carry both:
the server now identifies itself as Silo, and every path that would have called
home to a MinIO-operated service is closed.

Product identity
- build-constants.go: store name, UA name and startup banner become Silo. The
  Go identifiers (MinioStoreName, MinioBannerName, ...) keep their names on
  purpose - renaming exported symbols would churn the compatibility surface for
  a cosmetic gain, and the rebrand guard freezes that surface.
- main.go, server-startup-msg.go, ftp-server.go and the user-visible log, help
  and error strings across cmd/ and internal/ switch to Silo. Original MinIO
  copyright, LICENSE, NOTICE and CREDITS are untouched; --version now prints
  the upstream copyright, the pgsty modification notice, and the trademark
  policy's approved "based on MinIO technology" attribution.
- api-headers.go: the HTTP Server header becomes "Silo". This is the one
  externally observable identity change, so TestCommonHeadersUseSiloProductName
  pins it - probes that sniff for "MinIO" must move to capability detection.
- Prometheus metric HELP strings keep their MinIO wording. They are part of the
  metrics contract the guard protects, not product copy.

Configuration directory
- config-dir.go: new installs use ~/.silo. If only ~/.minio exists it is still
  read, with a one-time notice and no files moved. If both exist ~/.silo wins
  and an ambiguity warning is emitted; an explicit --config-dir always wins.
  Covered by TestSelectDefaultConfigDir. The internal .minio.sys layout is
  never renamed - this rule applies to the user config directory only.

Upstream service lockdown
- globalInplaceUpdateDisabled is now true at initialization rather than being
  set from MINIO_UPDATE. common-main.go still parses MINIO_UPDATE so upgrading
  nodes do not fail on an unknown key, but warns that the value is ignored;
  there is no way to re-enable the updater. TestInplaceUpdateCannotBeEnabled
  guards that. Without this, an admin with mc could have overwritten
  /usr/bin/silo with an upstream MinIO binary.
- verifyBinary and commitBinary refuse early; the ServerUpdate v1/v2 admin
  routes and the peer-rest update endpoints stay registered and keep returning
  the existing programmatic error, so clients see a stable failure rather than
  a 404.
- MinioReleaseBaseURL and defaultMinisignPubkey are emptied: no dl.min.io
  download root, and upstream's minisign key is no longer a trust root for
  anything this fork ships.
- cmd/callhome.go is deleted and internal/config/subnet/ is reduced to parsing
  its old keys and reporting that the integration is disabled. config-current.go
  warns instead of failing when callhome or SUBNET settings are present, so an
  upgraded node with those keys still starts.
- internal/config/errors.go replaces the MinIO Slack and support entry points
  with Silo documentation and issue links. Error codes and programmatic fields
  are unchanged.

Verified: the compatibility baseline is unchanged except for the deliberate
removal of the /api/health/upload SUBNET route; go build, go vet and the full
cmd/ and internal/ unit suites pass; a locally built binary starts, serves
S3/Admin/metrics on the unchanged /minio/* routes, answers with Server: Silo,
and falls back to a pre-existing ~/.minio with the expected notice.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 08:46:58 +08:00
Feng Ruohang 15def34dce chore: drop the unpublished upstream delivery and branding residue
The tree still carried a complete second publishing pipeline inherited from
upstream that pgsty has never operated, plus MinIO-branded artwork the fork has
no right to redistribute. None of it is a pgsty release artifact, so the
history-is-immutable rule does not protect it: nothing here was ever published
under this repository's name, and git history plus the pre-cutover tag keep the
provenance.

Removed, by category:

- charts.min.io machinery: CNAME (charts.min.io), _config.yml, index.yaml,
  helm-reindex.sh and 90 helm-releases/minio-*.tgz. GitHub Pages is not enabled
  on this repository, so this was dormant - but enabling Pages for any reason
  would have stood up a convincing impostor of the official MinIO chart
  repository from our own domain.

- Dockerfiles that consume upstream binaries: Dockerfile, Dockerfile.cicd,
  Dockerfile.hotfix, Dockerfile.release, Dockerfile.release.old_cpu,
  Dockerfile.scratch and docker-buildx.sh. These fetch or expect a MinIO-built
  binary; a Silo image must only ever contain an artifact this repository built
  from this commit. Dockerfile.goreleaser, the one the release workflow
  actually uses, stays.

- buildscripts/upgrade-tests/ (compose.yml, minio.env, nginx.conf), which
  pinned minio/minio images through a MINIO_VERSION compose variable. The
  upgrade test is rebuilt on image digests in a later commit.

- docs/hotfixes.md, describing an upstream hotfix distribution channel that
  does not exist for this fork.

- MinIO trademark artwork: .github/logo.svg (a recoloured MinIO wordmark), the
  docs/screenshots/ set, the Grafana dashboard screenshots and
  docs/federation/lookup/bucket-lookup.png. The dashboards themselves are kept
  and rebranded separately; only the images carrying MinIO chrome go.

Deliberately not touched: published releases, tags, image digests and package
signatures remain byte-identical. This commit only removes paths that were
never part of a pgsty release.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 08:46:31 +08:00
758 changed files with 64560 additions and 19268 deletions
-51
View File
@@ -1,51 +0,0 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: community, triage
assignees: ''
---
## NOTE
All GitHub issues are addressed on a best-effort basis at MinIO's sole discretion. There are no Service Level Agreements (SLA) or Objectives (SLO). Remember our [Code of Conduct](https://github.com/minio/minio/blob/master/code_of_conduct.md) when engaging with MinIO Engineers and the larger community.
For urgent issues (e.g. production down, etc.), subscribe to [SUBNET](https://min.io/pricing?jmp=github) for direct to engineering support.
<!--- Provide a general summary of the issue in the Title above -->
## Expected Behavior
<!--- If you're describing a bug, tell us what should happen -->
<!--- If you're suggesting a change/improvement, tell us how it should work -->
## Current Behavior
<!--- If describing a bug, tell us what happens instead of the expected behavior -->
<!--- If suggesting a change/improvement, explain the difference from current behavior -->
## Possible Solution
<!--- Not obligatory, but suggest a fix/reason for the bug, -->
<!--- or ideas how to implement the addition or change -->
## Steps to Reproduce (for bugs)
<!--- Provide a link to a live example, or an unambiguous set of steps to -->
<!--- reproduce this bug. Include code to reproduce, if relevant -->
<!--- and make sure you have followed https://github.com/minio/minio/tree/release/docs/debugging to capture relevant logs -->
1.
2.
3.
4.
## Context
<!--- How has this issue affected you? What are you trying to accomplish? -->
<!--- Providing context helps us come up with a solution that is most useful in the real world -->
## Regression
<!-- Is this issue a regression? (Yes / No) -->
<!-- If Yes, optionally please include minio version or commit id or PR# that caused this regression, if you have these details. -->
## Your Environment
<!--- Include as many relevant details about the environment you experienced the bug in -->
* Version used (`minio --version`):
* Server setup and configuration:
* Operating System and version (`uname -a`):
+11 -3
View File
@@ -7,6 +7,14 @@ assignees: ''
---
Report bugs in the PGSTY SILO server (`pgsty/silo`) here. Community maintainers
handle reports on a best-effort basis. There is no SLA, SLO, or emergency
production-support channel. Follow the
[Code of Conduct](https://github.com/pgsty/silo/blob/main/code_of_conduct.md).
Report suspected vulnerabilities privately through
[SECURITY.md](https://github.com/pgsty/silo/blob/main/SECURITY.md).
For patches, see the [contribution guide](https://github.com/pgsty/silo/blob/main/CONTRIBUTING.md).
<!--- Provide a general summary of the issue in the Title above -->
## Expected Behavior
@@ -24,7 +32,7 @@ assignees: ''
## Steps to Reproduce (for bugs)
<!--- Provide a link to a live example, or an unambiguous set of steps to -->
<!--- reproduce this bug. Include code to reproduce, if relevant -->
<!--- and make sure you have followed https://github.com/minio/minio/tree/release/docs/debugging to capture relevant logs -->
<!--- and include the relevant Silo logs with secrets and credentials removed -->
1.
2.
@@ -37,10 +45,10 @@ assignees: ''
## Regression
<!-- Is this issue a regression? (Yes / No) -->
<!-- If Yes, optionally please include minio version or commit id or PR# that caused this regression, if you have these details. -->
<!-- If Yes, optionally include the Silo version, commit id, or PR that caused the regression. -->
## Your Environment
<!--- Include as many relevant details about the environment you experienced the bug in -->
* Version used (`minio --version`):
* Version used (`silo --version`):
* Server setup and configuration:
* Operating System and version (`uname -a`):
+6 -6
View File
@@ -1,8 +1,8 @@
blank_issues_enabled: false
contact_links:
- name: MinIO Community Support
url: https://slack.min.io
about: Community support via Slack - for questions and discussions
- name: MinIO Enterprise Support (SUBNET)
url: https://min.io/pricing
about: Enterprise support with SLA - for production deployments and feature requests
- name: Silo Documentation
url: https://silo.pgsty.com/docs/
about: Installation, configuration, operations, and compatibility guidance
- name: Private Security Report
url: https://github.com/pgsty/silo/security/advisories/new
about: Privately report a suspected vulnerability in Silo
@@ -7,6 +7,9 @@ assignees: ''
---
Suggest improvements to the PGSTY SILO server (`pgsty/silo`) here.
For patches, see the [contribution guide](https://github.com/pgsty/silo/blob/main/CONTRIBUTING.md).
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+25 -6
View File
@@ -1,8 +1,17 @@
## Community Contribution License
All community contributions in this pull request are licensed to the project maintainers
under the terms of the [Apache 2 license](https://www.apache.org/licenses/LICENSE-2.0).
By creating this pull request I represent that I have the right to license the
contributions to the project maintainers under the Apache 2 license.
## Contribution Licensing (no CLA, inbound=outbound, DCO required)
This pull request contributes to PGSTY SILO (`pgsty/silo`). Code contributions
are accepted under AGPL-3.0-or-later, the same license as the server.
This project does not use a CLA or require a separate Apache-2.0 license grant.
By submitting this pull request I represent that I have the right to contribute
the code changes under this repository's
[GNU Affero General Public License v3.0 or later](https://www.gnu.org/licenses/agpl-3.0.html)
and retain copyright in my original work. Existing copyright and license
notices remain intact; separately licensed material keeps its applicable terms.
Every commit must carry a DCO `Signed-off-by` trailer
(`git commit -s`) certifying the
[Developer Certificate of Origin](https://developercertificate.org/) — see
[CONTRIBUTING.md](https://github.com/pgsty/silo/blob/main/CONTRIBUTING.md).
## Description
@@ -13,6 +22,12 @@ contributions to the project maintainers under the Apache 2 license.
## How to test this PR?
## Compatibility impact
<!-- Note effects on APIs, clients, MINIO_* configuration, metrics, headers,
routes, storage metadata, module/import paths, upgrades, or rollback. -->
## Types of changes
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
@@ -20,7 +35,11 @@ contributions to the project maintainers under the Apache 2 license.
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Checklist:
- [ ] All commits are signed off (`git commit -s`) per the [DCO](https://developercertificate.org/)
- [ ] Fixes a regression (If yes, please add `commit-id` or `PR #` here)
- [ ] Unit tests added/updated
- [ ] `make verifiers` passes
- [ ] Relevant package tests and `make build` pass
- [ ] Compatibility and rollback impact documented
- [ ] Internal documentation updated
- [ ] Create a documentation update request [here](https://github.com/minio/docs/issues/new?label=doc-change,title=Doc+Updated+Needed+For+PR+github.com%2fminio%2fminio%2fpull%2fNNNNN)
- [ ] Public documentation update opened in `pgsty/silo.pgsty.com`, if needed
+49 -8
View File
@@ -4,9 +4,9 @@ env:
- CGO_ENABLED=0
builds:
- id: minio
- id: silo
main: .
binary: minio
binary: silo
goos:
- linux
- darwin
@@ -23,23 +23,64 @@ builds:
- "{{ .Env.LDFLAGS }}"
archives:
- id: minio
- id: silo
ids:
- minio
name_template: "minio_{{ .Env.PKG_VERSION }}_{{ .Os }}_{{ .Arch }}"
- silo
name_template: "silo_{{ .Env.PKG_VERSION }}_{{ .Os }}_{{ .Arch }}"
# Explicit so the license materials cannot silently drop out of the
# binary archives: GoReleaser's default file globs would miss NOTICE.
# CREDITS stays out deliberately -- at 1.8MB it would dominate the
# archive; it remains available in the repository and the OCI image.
files:
- README.md
- LICENSE
- NOTICE
checksum:
name_template: "minio_{{ .Env.PKG_VERSION }}_checksums.txt"
name_template: "silo_{{ .Env.PKG_VERSION }}_checksums.txt"
algorithm: sha256
# Generate one SPDX JSON document per platform archive. SBOMs are created
# before the checksum stage, so the signed checksum manifest covers both the
# archives and their corresponding software bills of materials.
sboms:
- id: silo-archives
artifacts: archive
# Avoid network-backed package enrichment: the release SBOM must be
# reproducible from the artifact alone and the PR gate must work offline.
args:
- "$artifact"
- "--output"
- "spdx-json=$document"
env:
- SYFT_FILE_METADATA_CATALOGER_ENABLED=true
- SYFT_CHECK_FOR_APP_UPDATE=false
# A keyless Sigstore bundle is the detached signature for the checksum
# manifest. Consumers can verify the whole archive/SBOM set without trusting a
# long-lived project key copied into the repository.
signs:
- id: silo-checksums
cmd: cosign
signature: "${artifact}.sigstore.json"
args:
- sign-blob
- "--bundle=${signature}"
- "${artifact}"
- --yes
artifacts: checksum
output: true
release:
github:
owner: pgsty
name: minio
name: silo
draft: true
prerelease: false
mode: append
replace_existing_draft: true
replace_existing_artifacts: false
mode: replace
# Draft replacement matches the release name; keep it identical to the tag.
name_template: "{{ .Tag }}"
changelog:
-6
View File
@@ -1,6 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="255 293 539 140" fill="none">
<path d="M255.222 385.861 304.199 383.375c1.061 6.446 3.22 11.358 6.478 14.734 5.303 5.465 12.879 8.197 22.728 8.197 7.348 0 13.011-1.397 16.988-4.19 3.977-2.794 5.966-6.032 5.966-9.716 0-3.5-1.894-6.631-5.682-9.394-3.788-2.763-12.576-5.372-26.364-7.828-22.576-4.113-38.674-9.577-48.295-16.392-9.697-6.815-14.546-15.502-14.546-26.062 0-6.938 2.481-13.491 7.443-19.662 4.962-6.17 12.424-11.02 22.386-14.55 9.962-3.53 23.617-5.295 40.966-5.295 21.288 0 37.519 3.208 48.693 9.624 11.174 6.416 17.822 16.622 19.943 30.621l-48.523 2.302c-1.288-6.078-3.996-10.498-8.125-13.261-4.129-2.763-9.83-4.144-17.103-4.144-5.984 0-10.492 1.028-13.522 3.085-3.03 2.056-4.545 4.558-4.545 7.505 0 2.149 1.25 4.083 3.75 5.802 2.424 1.78 8.182 3.438 17.273 4.973 22.5 3.93 38.617 7.905 48.352 11.926 9.735 4.022 16.818 9.01 21.25 14.965 4.432 5.956 6.648 12.617 6.648 19.984 0 8.657-2.954 16.638-8.863 23.944-5.909 7.306-14.167 12.847-24.773 16.623-10.606 3.776-23.977 5.664-40.113 5.664-28.333 0-47.954-4.42-58.863-13.261-10.909-8.841-17.083-20.076-18.522-33.706Z" fill="#287CAB"/>
<path d="M434.578 295.519h43.304v135.007h-43.304z" fill="#287CAB"/>
<path d="M517.881 295.519h43.209v101.762h67.436v33.245H517.881Z" fill="#287CAB"/>
<path d="M639.4 363.115c0-22.041 6.359-39.201 19.077-51.48 12.718-12.28 30.427-18.419 53.129-18.419 23.274 0 41.206 6.032 53.796 18.096 12.591 12.064 18.886 28.963 18.886 50.697 0 15.778-2.75 28.717-8.251 38.817-5.5 10.099-13.449 17.958-23.846 23.575-10.397 5.618-23.353 8.427-38.869 8.427-15.77 0-28.822-2.425-39.155-7.275-10.333-4.85-18.711-12.525-25.133-23.023-6.423-10.499-9.634-23.637-9.634-39.416Zm43.209.184c0 13.63 2.623 23.422 7.869 29.377 5.246 5.956 12.384 8.934 21.413 8.934 9.284 0 16.47-2.916 21.557-8.749 5.087-5.833 7.631-16.301 7.631-31.403 0-12.709-2.655-21.995-7.965-27.858-5.31-5.863-12.511-8.795-21.604-8.795-8.712 0-15.707 2.978-20.985 8.933-5.278 5.956-7.916 15.81-7.916 29.561Z" fill="#287CAB"/>
</svg>

Before

Width:  |  Height:  |  Size: 2.0 KiB

+77 -4
View File
@@ -1,4 +1,4 @@
name: minio
name: silo
arch: ${NFPM_ARCH}
platform: linux
version: ${PKG_VERSION}
@@ -7,26 +7,99 @@ release: ${NFPM_RELEASE}
section: utils
priority: optional
maintainer: "Ruohang Feng (@Vonng) <rh@vonng.com>"
description: S3-Interface Libre Object Storage, Community-maintained MinIO server fork.
description: S3-Interface Libre Object Storage, a community-maintained S3-compatible server.
vendor: PGSTY
homepage: https://silo.pgsty.com
license: AGPL-3.0-or-later
contents:
- src: ${NFPM_SOURCE}
dst: /usr/local/bin/minio
dst: /usr/bin/silo
expand: true
file_info:
mode: 0755
owner: root
group: root
- src: ${NFPM_UNIT}
dst: /usr/lib/systemd/system/minio.service
dst: /usr/lib/systemd/system/silo.service
expand: true
file_info:
mode: 0644
owner: root
group: root
- src: ${NFPM_DEFAULTS}
dst: /etc/default/silo
type: config|noreplace
expand: true
file_info:
mode: 0644
owner: root
group: root
- src: ${NFPM_SYSUSERS}
dst: /usr/lib/sysusers.d/silo.conf
expand: true
file_info:
mode: 0644
owner: root
group: root
# The license materials are declared once per packager: nfpm only honors
# type: license on rpm and silently drops such entries from deb and apk, so
# the rpm keeps its %license flag while deb and apk carry plain files at the
# same path.
- src: ${NFPM_LICENSE}
dst: /usr/share/doc/silo/LICENSE
type: license
packager: rpm
expand: true
file_info:
mode: 0644
owner: root
group: root
- src: ${NFPM_NOTICE}
dst: /usr/share/doc/silo/NOTICE
type: license
packager: rpm
expand: true
file_info:
mode: 0644
owner: root
group: root
- src: ${NFPM_LICENSE}
dst: /usr/share/doc/silo/LICENSE
packager: deb
expand: true
file_info:
mode: 0644
owner: root
group: root
- src: ${NFPM_NOTICE}
dst: /usr/share/doc/silo/NOTICE
packager: deb
expand: true
file_info:
mode: 0644
owner: root
group: root
- src: ${NFPM_LICENSE}
dst: /usr/share/doc/silo/LICENSE
packager: apk
expand: true
file_info:
mode: 0644
owner: root
group: root
- src: ${NFPM_NOTICE}
dst: /usr/share/doc/silo/NOTICE
packager: apk
expand: true
file_info:
mode: 0644
owner: root
group: root
scripts:
postinstall: buildscripts/package/postinstall.sh
preremove: buildscripts/package/preremove.sh
rpm:
group: Applications/File
+105
View File
@@ -0,0 +1,105 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg"
width="1500" height="570" viewBox="0 0 1500 570"
preserveAspectRatio="xMidYMid meet"
role="img" aria-labelledby="silo-logo-title silo-logo-desc"
shape-rendering="geometricPrecision">
<title id="silo-logo-title">SILO logo</title>
<desc id="silo-logo-desc">The SILO horizontal lockup: the circular silo emblem on the left, the SILO wordmark on the right.</desc>
<defs>
<!-- Emblem gradient, in the emblem's local coordinates; the group transform maps it. -->
<linearGradient id="silo-logo-mark-color" x1="276" y1="720" x2="742" y2="286" gradientUnits="userSpaceOnUse">
<stop offset="0" stop-color="#064A83"/>
<stop offset="0.5" stop-color="#007FA8"/>
<stop offset="1" stop-color="#22C7C9"/>
</linearGradient>
<!-- Wordmark gradient, in the wordmark's local coordinates. -->
<linearGradient id="silo-logo-word-color" x1="44.0" y1="-94.4" x2="1950.0" y2="794.4" gradientUnits="userSpaceOnUse">
<stop class="silo-logo-wm-a" offset="0.06" stop-color="#1d588c"/>
<stop class="silo-logo-wm-b" offset="0.94" stop-color="#b4762e"/>
</linearGradient>
<style>
/* Light theme values of --pg-strong / --copper; dark theme swaps in its own pair. */
@media (prefers-color-scheme: dark) {
.silo-logo-wm-a { stop-color: #7fb8e8; }
.silo-logo-wm-b { stop-color: #e0a35c; }
}
</style>
</defs>
<!-- Circular emblem, shifted from its native viewBox (230 213 570 570) to the 0..570 square. -->
<g id="silo-logo-mark" transform="translate(-230 -213)" fill="url(#silo-logo-mark-color)">
<!-- Outer circular band, intentionally opened at the lower-right plinth. -->
<path d="
M 734 676
A 283.5 278.5 0 1 0 310 692
L 359 692
A 247 248.5 0 1 1 688 676
Z"/>
<!-- Flowing left wall; its upper tangent matches the inner ellipse. -->
<path d="
M 300.57 375
C 292 390 300 430 328 450
C 343 461 357 472 374 481
C 410 501 426 517 426 544
L 426 676
L 336 676
C 308 647 286 608 274 565
C 262 522 263 479 272 439
C 278 413 286 389 300.57 375
Z"/>
<!-- Right column with tangent-continuous upper shoulder. -->
<path d="
M 602 373
Q 602 368 607 370
C 619 374 634 381 634 389
L 634 647
Q 634 649 636 649
L 708 649
L 734 676
L 602 676
Z"/>
<!-- Lower circular cap. -->
<path d="
M 310 692
L 714 692
C 668 743 596 774 512 774
C 428 774 355 743 310 692
Z"/>
<!-- Main silo body, with a tangent-continuous right shoulder. -->
<path d="
M 389 389
C 389 374 447 351 512 351
C 540 351 565 354 580 359
Q 583 360 583 364
L 583 676
L 443 676
L 443 541
C 443 509 426 490 389 470
Z"/>
<!-- Peaked roof with softly tapered, burr-free tips. -->
<path d="
M 512 274
L 647 365
Q 649 370 647 376
C 609 350 563 337 512 337
C 460 337 414 350 377 376
Q 375 370 377 365
Z"/>
</g>
<!-- Wordmark, scaled 3/7 to a 300-unit cap height and centered on the emblem's axis. -->
<g id="silo-logo-word" transform="translate(645.429 135) scale(0.428571)"
fill="url(#silo-logo-word-color)" fill-rule="nonzero">
<path id="silo-logo-s" d="M0 592V492H134V551L167 584H374L408 550V434L375 401H110L2 293V108L110 0H426L534 108V209H400V149L367 116H169L136 149V252L169 285H434L542 393V590L432 700H108Z"/>
<path id="silo-logo-i" d="M637 0H773V700H637Z"/>
<path id="silo-logo-l" d="M888 0H1024V585H1374V700H888Z"/>
<path id="silo-logo-o" d="M1404 585V115L1519 0H1879L1994 115V585L1879 700H1519ZM1807 584 1858 533V167L1807 116H1591L1540 167V533L1591 584Z"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 3.8 KiB

+43
View File
@@ -0,0 +1,43 @@
name: DCO
on:
pull_request:
branches:
- main
permissions:
contents: read
jobs:
check:
name: Verify DCO sign-off
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
# Every non-merge commit in the pull request must carry a Signed-off-by
# trailer matching the commit author's email, certifying the Developer
# Certificate of Origin 1.1 (https://developercertificate.org/).
# Only commits authored from a GitHub-issued bot address are exempt; a
# display name is attacker-controlled and must never grant the exemption.
- name: Check Signed-off-by trailers
env:
BASE_SHA: ${{ github.event.pull_request.base.sha }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
fail=0
while read -r sha; do
author_name="$(git log -1 --format='%an' "${sha}")"
author_email="$(git log -1 --format='%ae' "${sha}")"
case "${author_email}" in
*"[bot]@users.noreply.github.com") continue ;;
esac
if ! git log -1 --format='%(trailers:key=Signed-off-by,valueonly)' "${sha}" |
grep -qiF "<${author_email}>"; then
echo "::error::commit ${sha} by ${author_name} <${author_email}> lacks a matching Signed-off-by trailer; sign with 'git commit -s', repair with 'git rebase --signoff'"
fail=1
fi
done < <(git rev-list --no-merges "${BASE_SHA}..${HEAD_SHA}")
exit "${fail}"
+281 -29
View File
@@ -4,12 +4,20 @@ on:
workflow_dispatch:
inputs:
tag:
description: "Published RELEASE.* tag to package as pgsty/minio"
description: "Published RELEASE.* tag to package as pgsty/silo"
required: true
type: string
recovery:
description: "Run the current main workflow against an already-published tag"
required: false
default: false
type: boolean
permissions:
contents: read
id-token: write
attestations: write
artifact-metadata: write
concurrency:
group: docker-release
@@ -66,11 +74,26 @@ jobs:
fi
- name: Checkout release tag
uses: actions/checkout@v4
uses: actions/checkout@v7
with:
ref: ${{ inputs.tag }}
fetch-depth: 0
- name: Verify workflow identity matches release source
env:
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
RECOVERY: ${{ inputs.recovery }}
run: |
set -euo pipefail
CHECKED_OUT_REVISION="$(git rev-parse HEAD)"
if [ "${CHECKED_OUT_REVISION}" != "${GITHUB_SHA}" ]; then
if [ "${RECOVERY}" != "true" ] || [ "${GITHUB_REF}" != "refs/heads/${DEFAULT_BRANCH}" ]; then
echo "Checked out ${CHECKED_OUT_REVISION}, but workflow identity is ${GITHUB_SHA}. Dispatch from ${RELEASE_TAG}, or use recovery from ${DEFAULT_BRANCH}." >&2
exit 1
fi
echo "Recovery workflow ${GITHUB_SHA} is packaging published source ${CHECKED_OUT_REVISION}."
fi
- name: Prepare verified Docker contexts
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@@ -79,9 +102,9 @@ jobs:
assets_dir="docker-release/assets"
mkdir -p "${assets_dir}"
amd64_archive="minio_${PKG_VERSION}_linux_amd64.tar.gz"
arm64_archive="minio_${PKG_VERSION}_linux_arm64.tar.gz"
checksums="minio_${PKG_VERSION}_checksums.txt"
amd64_archive="silo_${PKG_VERSION}_linux_amd64.tar.gz"
arm64_archive="silo_${PKG_VERSION}_linux_arm64.tar.gz"
checksums="silo_${PKG_VERSION}_checksums.txt"
gh release download "${RELEASE_TAG}" --repo "${GITHUB_REPOSITORY}" \
--dir "${assets_dir}" \
--pattern "${amd64_archive}" \
@@ -90,10 +113,22 @@ jobs:
# The binaries going into the images are the published ones, checked
# against the published checksums, not a rebuild that merely ought to
# match them.
# match them. awk matches the manifest filename column exactly: a
# substring grep would also pull in the archive's .sbom.json line,
# whose file is deliberately not downloaded in this lane.
cd "${assets_dir}"
grep -F " ${amd64_archive}" "${checksums}" | sha256sum --check
grep -F " ${arm64_archive}" "${checksums}" | sha256sum --check
awk -v name="${amd64_archive}" '$2 == name' "${checksums}" | sha256sum --check
awk -v name="${arm64_archive}" '$2 == name' "${checksums}" | sha256sum --check
# The checksum manifest and both archives must have provenance from
# this repository's release workflow at the exact checked-out tag.
for artifact in "${checksums}" "${amd64_archive}" "${arm64_archive}"; do
gh attestation verify "${artifact}" \
--repo "${GITHUB_REPOSITORY}" \
--signer-workflow "${GITHUB_REPOSITORY}/.github/workflows/release.yml" \
--source-digest "$(git -C "${GITHUB_WORKSPACE}" rev-parse HEAD)" \
--source-ref "refs/tags/${RELEASE_TAG}" >/dev/null
done
cd "${GITHUB_WORKSPACE}"
# Dockerfile.goreleaser expects the binary at the context root and
@@ -101,70 +136,287 @@ jobs:
# GoReleaser used to assemble via extra_files.
for arch in amd64 arm64; do
context="docker-release/${arch}"
archive="${assets_dir}/minio_${PKG_VERSION}_linux_${arch}.tar.gz"
archive="${assets_dir}/silo_${PKG_VERSION}_linux_${arch}.tar.gz"
mkdir -p "${context}/dockerscripts"
tar -xzf "${archive}" -C "${context}" minio
cp Dockerfile.goreleaser LICENSE CREDITS "${context}/"
cp dockerscripts/docker-entrypoint.sh dockerscripts/download-static-curl.sh \
tar -xzf "${archive}" -C "${context}" silo
cp Dockerfile.goreleaser Dockerfile.distroless LICENSE NOTICE CREDITS "${context}/"
cp dockerscripts/docker-entrypoint.sh dockerscripts/build-static-curl.sh \
"${context}/dockerscripts/"
done
# The classic image bundles mcli. Resolve its two archive digests
# from the immutable published release instead of trusting defaults
# copied into an older Server tag. This also gives a recovery run a
# narrow override when a tag selected the right mcli release but
# accidentally retained stale archive pins.
MC_REPO="$(awk -F= '/^ARG MC_REPO=/{print $2; exit}' Dockerfile.goreleaser)"
MC_VERSION="$(awk -F= '/^ARG MC_VERSION=/{print $2; exit}' Dockerfile.goreleaser)"
test -n "${MC_REPO}"
test -n "${MC_VERSION}"
MC_VERSION_HYPHEN="${MC_VERSION#RELEASE.}"
MC_PKG_VERSION="$(echo "${MC_VERSION_HYPHEN}" | sed -E 's/^([0-9]{4})-([0-9]{2})-([0-9]{2})T([0-9]{2})-([0-9]{2})-([0-9]{2})Z$/\1\2\3\4\5\6.0.0/')"
if [ "${MC_PKG_VERSION}" = "${MC_VERSION_HYPHEN}" ]; then
echo "Invalid bundled mcli tag: ${MC_VERSION}" >&2
exit 1
fi
if [ "$(gh release view "${MC_VERSION}" --repo "${MC_REPO}" --json isDraft --jq .isDraft)" != false ] || \
[ "$(gh release view "${MC_VERSION}" --repo "${MC_REPO}" --json isPrerelease --jq .isPrerelease)" != false ] || \
[ "$(gh release view "${MC_VERSION}" --repo "${MC_REPO}" --json isImmutable --jq .isImmutable)" != true ]; then
echo "Bundled mcli ${MC_REPO}@${MC_VERSION} must be a published immutable release" >&2
exit 1
fi
mc_checksums="mcli_${MC_PKG_VERSION}_checksums.txt"
gh release download "${MC_VERSION}" --repo "${MC_REPO}" \
--dir "${assets_dir}" --pattern "${mc_checksums}"
gh attestation verify "${assets_dir}/${mc_checksums}" \
--repo "${MC_REPO}" \
--signer-workflow "${MC_REPO}/.github/workflows/release.yml" \
--source-ref "refs/tags/${MC_VERSION}" >/dev/null
MC_AMD64_SHA256="$(awk -v name="mcli_${MC_PKG_VERSION}_linux_amd64.tar.gz" '$2 == name {print $1}' "${assets_dir}/${mc_checksums}")"
MC_ARM64_SHA256="$(awk -v name="mcli_${MC_PKG_VERSION}_linux_arm64.tar.gz" '$2 == name {print $1}' "${assets_dir}/${mc_checksums}")"
[[ "${MC_AMD64_SHA256}" =~ ^[0-9a-f]{64}$ ]]
[[ "${MC_ARM64_SHA256}" =~ ^[0-9a-f]{64}$ ]]
{
echo "MC_AMD64_SHA256=${MC_AMD64_SHA256}"
echo "MC_ARM64_SHA256=${MC_ARM64_SHA256}"
} >> "${GITHUB_ENV}"
echo "RELEASE_REVISION=$(git rev-parse HEAD)" >> "${GITHUB_ENV}"
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
uses: docker/setup-qemu-action@v4
with:
platforms: arm64
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
uses: docker/setup-buildx-action@v4
- name: Login to Docker Hub
uses: docker/login-action@v3
uses: docker/login-action@v4
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Install Syft
uses: anchore/sbom-action/download-syft@e22c389904149dbc22b58101806040fa8d37a610 # v0.24.0
with:
syft-version: v1.50.0
- name: Build and push amd64 image
uses: docker/build-push-action@v6
id: build-amd64
uses: docker/build-push-action@v7
with:
context: docker-release/amd64
file: docker-release/amd64/Dockerfile.goreleaser
platforms: linux/amd64
push: true
build-args: |
MC_AMD64_SHA256=${{ env.MC_AMD64_SHA256 }}
MC_ARM64_SHA256=${{ env.MC_ARM64_SHA256 }}
tags: |
pgsty/minio:${{ env.RELEASE_TAG }}-amd64
pgsty/minio:latest-amd64
pgsty/silo:${{ env.RELEASE_TAG }}-amd64
pgsty/silo:latest-amd64
labels: |
org.opencontainers.image.version=${{ env.RELEASE_TAG }}
org.opencontainers.image.created=${{ env.PUBLISHED_AT }}
org.opencontainers.image.revision=${{ env.RELEASE_REVISION }}
- name: Build and push arm64 image
uses: docker/build-push-action@v6
id: build-arm64
uses: docker/build-push-action@v7
with:
context: docker-release/arm64
file: docker-release/arm64/Dockerfile.goreleaser
platforms: linux/arm64
push: true
build-args: |
MC_AMD64_SHA256=${{ env.MC_AMD64_SHA256 }}
MC_ARM64_SHA256=${{ env.MC_ARM64_SHA256 }}
tags: |
pgsty/minio:${{ env.RELEASE_TAG }}-arm64
pgsty/minio:latest-arm64
pgsty/silo:${{ env.RELEASE_TAG }}-arm64
pgsty/silo:latest-arm64
labels: |
org.opencontainers.image.version=${{ env.RELEASE_TAG }}
org.opencontainers.image.created=${{ env.PUBLISHED_AT }}
org.opencontainers.image.revision=${{ env.RELEASE_REVISION }}
# The distroless variant is a pilot published alongside the classic
# image; it ships the silo binary alone and relies on the native
# `silo healthcheck` subcommand for container health.
# Design: https://silo.pgsty.com/compatibility/feature/healthcheck/
- name: Build and push amd64 distroless image
id: build-amd64-distroless
uses: docker/build-push-action@v7
with:
context: docker-release/amd64
file: docker-release/amd64/Dockerfile.distroless
platforms: linux/amd64
push: true
tags: |
pgsty/silo:${{ env.RELEASE_TAG }}-distroless-amd64
pgsty/silo:distroless-amd64
labels: |
org.opencontainers.image.version=${{ env.RELEASE_TAG }}
org.opencontainers.image.created=${{ env.PUBLISHED_AT }}
org.opencontainers.image.revision=${{ env.RELEASE_REVISION }}
- name: Build and push arm64 distroless image
id: build-arm64-distroless
uses: docker/build-push-action@v7
with:
context: docker-release/arm64
file: docker-release/arm64/Dockerfile.distroless
platforms: linux/arm64
push: true
tags: |
pgsty/silo:${{ env.RELEASE_TAG }}-distroless-arm64
pgsty/silo:distroless-arm64
labels: |
org.opencontainers.image.version=${{ env.RELEASE_TAG }}
org.opencontainers.image.created=${{ env.PUBLISHED_AT }}
org.opencontainers.image.revision=${{ env.RELEASE_REVISION }}
- name: Verify HEALTHCHECK survived the distroless push
run: |
set -euo pipefail
# HEALTHCHECK is a Docker extension absent from the OCI image
# spec, and a publish path can drop it silently. Check the pushed
# architecture image now, before the versioned and rolling
# multi-arch manifests are created, so a broken health config
# stops their promotion. (The architecture-suffixed tags above
# are already public by this point - full staging-then-promote
# would be a workflow-wide redesign shared with the classic
# image lanes.)
docker pull "pgsty/silo:${RELEASE_TAG}-distroless-amd64" >/dev/null
test "$(docker inspect -f '{{json .Config.Healthcheck.Test}}' "pgsty/silo:${RELEASE_TAG}-distroless-amd64")" \
= '["CMD","/usr/bin/silo","healthcheck","ready"]'
- name: Publish multi-architecture manifests
run: |
set -euo pipefail
mkdir -p docker-release/metadata
docker buildx imagetools create \
--tag "pgsty/minio:${RELEASE_TAG}" \
"pgsty/minio:${RELEASE_TAG}-amd64" \
"pgsty/minio:${RELEASE_TAG}-arm64"
--tag "pgsty/silo:${RELEASE_TAG}" \
--metadata-file docker-release/metadata/release.json \
"pgsty/silo:${RELEASE_TAG}-amd64" \
"pgsty/silo:${RELEASE_TAG}-arm64"
docker buildx imagetools create \
--tag "pgsty/minio:latest" \
"pgsty/minio:latest-amd64" \
"pgsty/minio:latest-arm64"
docker buildx imagetools inspect "pgsty/minio:${RELEASE_TAG}"
docker buildx imagetools inspect "pgsty/minio:latest"
--tag "pgsty/silo:latest" \
--metadata-file docker-release/metadata/latest.json \
"pgsty/silo:latest-amd64" \
"pgsty/silo:latest-arm64"
docker buildx imagetools inspect "pgsty/silo:${RELEASE_TAG}"
docker buildx imagetools inspect "pgsty/silo:latest"
docker buildx imagetools create \
--tag "pgsty/silo:${RELEASE_TAG}-distroless" \
--metadata-file docker-release/metadata/release-distroless.json \
"pgsty/silo:${RELEASE_TAG}-distroless-amd64" \
"pgsty/silo:${RELEASE_TAG}-distroless-arm64"
docker buildx imagetools create \
--tag "pgsty/silo:distroless" \
--metadata-file docker-release/metadata/distroless.json \
"pgsty/silo:distroless-amd64" \
"pgsty/silo:distroless-arm64"
docker buildx imagetools inspect "pgsty/silo:${RELEASE_TAG}-distroless"
docker buildx imagetools inspect "pgsty/silo:distroless"
RELEASE_DIGEST="$(jq -r '."containerimage.descriptor".digest' docker-release/metadata/release.json)"
LATEST_DIGEST="$(jq -r '."containerimage.descriptor".digest' docker-release/metadata/latest.json)"
if ! [[ "${RELEASE_DIGEST}" =~ ^sha256:[0-9a-f]{64}$ ]]; then
echo "Invalid release manifest digest: ${RELEASE_DIGEST}" >&2
exit 1
fi
if [ "${RELEASE_DIGEST}" != "${LATEST_DIGEST}" ]; then
echo "Release and latest tags resolved to different manifests" >&2
exit 1
fi
echo "SILO_IMAGE_DIGEST=${RELEASE_DIGEST}" >> "${GITHUB_ENV}"
DISTROLESS_RELEASE_DIGEST="$(jq -r '."containerimage.descriptor".digest' docker-release/metadata/release-distroless.json)"
DISTROLESS_ROLLING_DIGEST="$(jq -r '."containerimage.descriptor".digest' docker-release/metadata/distroless.json)"
if ! [[ "${DISTROLESS_RELEASE_DIGEST}" =~ ^sha256:[0-9a-f]{64}$ ]]; then
echo "Invalid distroless manifest digest: ${DISTROLESS_RELEASE_DIGEST}" >&2
exit 1
fi
if [ "${DISTROLESS_RELEASE_DIGEST}" != "${DISTROLESS_ROLLING_DIGEST}" ]; then
echo "Distroless release and rolling tags resolved to different manifests" >&2
exit 1
fi
echo "SILO_DISTROLESS_DIGEST=${DISTROLESS_RELEASE_DIGEST}" >> "${GITHUB_ENV}"
- name: Generate architecture image SBOMs
env:
AMD64_DIGEST: ${{ steps.build-amd64.outputs.digest }}
ARM64_DIGEST: ${{ steps.build-arm64.outputs.digest }}
DISTROLESS_AMD64_DIGEST: ${{ steps.build-amd64-distroless.outputs.digest }}
DISTROLESS_ARM64_DIGEST: ${{ steps.build-arm64-distroless.outputs.digest }}
SYFT_CHECK_FOR_APP_UPDATE: "false"
run: |
set -euo pipefail
mkdir -p docker-release/sbom
# Each per-architecture digest names an OCI index (image plus the
# provenance attestation buildx attaches), and Syft's platform
# default on an index follows the amd64 runner - an arm64-only
# index would fail outright. Select the platform explicitly.
syft "registry:index.docker.io/pgsty/silo@${AMD64_DIGEST}" \
--platform linux/amd64 \
--output "spdx-json=docker-release/sbom/linux-amd64.spdx.json"
syft "registry:index.docker.io/pgsty/silo@${ARM64_DIGEST}" \
--platform linux/arm64 \
--output "spdx-json=docker-release/sbom/linux-arm64.spdx.json"
syft "registry:index.docker.io/pgsty/silo@${DISTROLESS_AMD64_DIGEST}" \
--platform linux/amd64 \
--output "spdx-json=docker-release/sbom/linux-amd64-distroless.spdx.json"
syft "registry:index.docker.io/pgsty/silo@${DISTROLESS_ARM64_DIGEST}" \
--platform linux/arm64 \
--output "spdx-json=docker-release/sbom/linux-arm64-distroless.spdx.json"
- name: Attest amd64 image SBOM
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-name: index.docker.io/pgsty/silo
subject-digest: ${{ steps.build-amd64.outputs.digest }}
sbom-path: docker-release/sbom/linux-amd64.spdx.json
push-to-registry: true
- name: Attest arm64 image SBOM
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-name: index.docker.io/pgsty/silo
subject-digest: ${{ steps.build-arm64.outputs.digest }}
sbom-path: docker-release/sbom/linux-arm64.spdx.json
push-to-registry: true
- name: Attest amd64 distroless image SBOM
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-name: index.docker.io/pgsty/silo
subject-digest: ${{ steps.build-amd64-distroless.outputs.digest }}
sbom-path: docker-release/sbom/linux-amd64-distroless.spdx.json
push-to-registry: true
- name: Attest arm64 distroless image SBOM
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-name: index.docker.io/pgsty/silo
subject-digest: ${{ steps.build-arm64-distroless.outputs.digest }}
sbom-path: docker-release/sbom/linux-arm64-distroless.spdx.json
push-to-registry: true
- name: Attest multi-architecture image provenance
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-name: index.docker.io/pgsty/silo
subject-digest: ${{ env.SILO_IMAGE_DIGEST }}
push-to-registry: true
- name: Attest multi-architecture distroless image provenance
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-name: index.docker.io/pgsty/silo
subject-digest: ${{ env.SILO_DISTROLESS_DIGEST }}
push-to-registry: true
+234
View File
@@ -0,0 +1,234 @@
name: Finalize Release Packages
# Manual-only lane that runs AFTER the maintainer has GPG-signed the RPMs in a
# Draft release (buildscripts/sign-release-rpms.sh --upload) and BEFORE the
# release is published. GPG signing rewrites the RPM bytes, which strands the
# SBOMs, the packages checksum manifest, and the attestations that release.yml
# generated from the as-built packages. This lane regenerates those materials
# from the published (signed) bytes under the workflow identity, so the
# sigstore layer describes exactly what the release ships.
on:
workflow_dispatch:
inputs:
tag:
description: "Draft RELEASE.* tag whose signed RPMs need refreshed SBOMs and checksums"
required: true
type: string
permissions:
contents: write
id-token: write
attestations: write
artifact-metadata: write
concurrency:
group: release-${{ inputs.tag }}
cancel-in-progress: false
jobs:
finalize:
runs-on: ubuntu-latest
steps:
- name: Checkout release tag
uses: actions/checkout@v7
with:
ref: ${{ inputs.tag }}
fetch-depth: 0
- name: Verify workflow identity matches release source
run: |
set -euo pipefail
CHECKED_OUT_REVISION="$(git rev-parse HEAD)"
if [ "${CHECKED_OUT_REVISION}" != "${GITHUB_SHA}" ]; then
echo "Checked out ${CHECKED_OUT_REVISION}, but workflow identity is ${GITHUB_SHA}. Dispatch from the release tag." >&2
exit 1
fi
- name: Compute release variables
env:
# Environment passthrough keeps the dispatch input out of the script
# source, mirroring release.yml.
INPUT_TAG: ${{ inputs.tag }}
run: |
set -euo pipefail
TAG="${INPUT_TAG}"
if [[ ! "${TAG}" =~ ^RELEASE\.[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}-[0-9]{2}-[0-9]{2}Z$ ]]; then
echo "Invalid release tag format: ${TAG}" >&2
exit 1
fi
VERSION_HYPHEN="${TAG#RELEASE.}"
PKG_VERSION="$(echo "${VERSION_HYPHEN}" | sed -E 's/^([0-9]{4})-([0-9]{2})-([0-9]{2})T([0-9]{2})-([0-9]{2})-([0-9]{2})Z$/\1\2\3\4\5\6.0.0/')"
{
echo "RELEASE_TAG=${TAG}"
echo "PKG_VERSION=${PKG_VERSION}"
} >> "${GITHUB_ENV}"
- name: Refuse to touch a published release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
if [ "$(gh release view "${RELEASE_TAG}" --repo "${GITHUB_REPOSITORY}" --json isDraft --jq .isDraft)" != "true" ]; then
echo "${RELEASE_TAG} is not a Draft release; finalize runs only before publishing." >&2
exit 1
fi
- name: Install Syft
uses: anchore/sbom-action/download-syft@e22c389904149dbc22b58101806040fa8d37a610 # v0.24.0
with:
syft-version: v1.50.0
- name: Install Cosign
uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2
with:
cosign-release: v3.1.2
- name: Download and verify the package set
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
# sign-release-rpms.sh owns the package identity; import its values
# the same way test-release.yml does so the lanes cannot drift.
eval "$(grep -E '^expected_(release|fingerprint)=' buildscripts/sign-release-rpms.sh)"
test -n "${expected_release}"
test -n "${expected_fingerprint}"
packages_dir="finalize/packages"
sidecar_dir="finalize/sidecars"
mkdir -p "${packages_dir}" "${sidecar_dir}"
manifest="silo_${PKG_VERSION}_packages_checksums.txt"
# Exactly the twelve manifest subjects land in packages_dir; the
# tarball SBOMs in the release root do not match these patterns.
gh release download "${RELEASE_TAG}" --repo "${GITHUB_REPOSITORY}" --dir "${packages_dir}" \
--pattern '*.rpm' --pattern '*.deb' --pattern '*.apk' \
--pattern '*.rpm.sbom.json' --pattern '*.deb.sbom.json' --pattern '*.apk.sbom.json'
gh release download "${RELEASE_TAG}" --repo "${GITHUB_REPOSITORY}" --dir "${sidecar_dir}" \
--pattern '*.rpm.sha256sum' --pattern "${manifest}"
cd "${packages_dir}"
subject_count="$(find . -maxdepth 1 -type f | wc -l | tr -d ' ')"
if [ "${subject_count}" -ne 12 ]; then
echo "Expected twelve package subjects, found ${subject_count}" >&2
find . -maxdepth 1 -type f >&2
exit 1
fi
# The signed RPMs must match the .sha256sum sidecars the signing
# script regenerated and uploaded alongside them.
for rpm_file in "silo-${PKG_VERSION}-${expected_release}.x86_64.rpm" \
"silo-${PKG_VERSION}-${expected_release}.aarch64.rpm"; do
test -s "${rpm_file}"
actual="$(sha256sum "${rpm_file}" | awk '{print $1}')"
recorded="$(awk '{print $1}' "../sidecars/${rpm_file}.sha256sum")"
if [ "${actual}" != "${recorded}" ]; then
echo "Digest mismatch for ${rpm_file}: sidecar ${recorded}, asset ${actual}" >&2
exit 1
fi
done
# Everything the maintainer did not re-sign must still match the
# manifest release.yml generated: this lane refreshes RPM materials,
# it does not accept drift anywhere else.
for package_file in *.deb *.apk *.deb.sbom.json *.apk.sbom.json; do
awk -v name="${package_file}" '$2 == name' "../sidecars/${manifest}" | sha256sum --check
done
- name: Verify RPM GPG signatures
run: |
set -euo pipefail
eval "$(grep -E '^expected_fingerprint=' buildscripts/sign-release-rpms.sh)"
sudo apt-get update
sudo apt-get install --yes rpm
sudo rpmkeys --import buildscripts/pgsty-rpm-signing-key.asc
key_id="$(printf '%s' "${expected_fingerprint}" | tail -c 8 | tr '[:upper:]' '[:lower:]')"
for rpm_file in finalize/packages/*.rpm; do
signature_output="$(sudo rpmkeys --checksig --verbose "${rpm_file}")"
printf '%s\n' "${signature_output}"
if ! printf '%s\n' "${signature_output}" | tr '[:upper:]' '[:lower:]' | grep -q "key id ${key_id}: ok"; then
echo "Signature verification failed for ${rpm_file}" >&2
exit 1
fi
done
- name: Regenerate RPM SBOMs and the packages checksum manifest
env:
SYFT_CHECK_FOR_APP_UPDATE: "false"
run: |
set -euo pipefail
cd finalize/packages
for rpm_file in *.rpm; do
rm -f "${rpm_file}.sbom.json"
syft "${rpm_file}" --output "spdx-json=${rpm_file}.sbom.json"
done
manifest="silo_${PKG_VERSION}_packages_checksums.txt"
mapfile -t subjects < <(find . -maxdepth 1 -type f \
\( -name '*.rpm' -o -name '*.deb' -o -name '*.apk' -o -name '*.sbom.json' \) | sort)
if [ "${#subjects[@]}" -ne 12 ]; then
echo "Expected six packages and six SBOMs, found ${#subjects[@]} subjects" >&2
exit 1
fi
sha256sum "${subjects[@]}" | sed 's# \./# #' > "${manifest}"
cosign sign-blob --bundle="${manifest}.sigstore.json" "${manifest}" --yes
- name: Attest the finalized package artifacts
id: attest-finalize
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-path: |
finalize/packages/*.rpm
finalize/packages/*.rpm.sbom.json
finalize/packages/*_checksums.txt
finalize/packages/*_checksums.txt.sigstore.json
finalize/sidecars/*.rpm.sha256sum
- name: Preserve finalize provenance bundle as a release asset
env:
BUNDLE_PATH: ${{ steps.attest-finalize.outputs.bundle-path }}
run: |
set -euo pipefail
test -s "${BUNDLE_PATH}"
cp "${BUNDLE_PATH}" "finalize/packages/silo_${PKG_VERSION}_packages_provenance.sigstore.json"
- name: Upload finalized assets to the Draft release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
eval "$(grep -E '^expected_release=' buildscripts/sign-release-rpms.sh)"
manifest="silo_${PKG_VERSION}_packages_checksums.txt"
cd finalize/packages
files=(
"silo-${PKG_VERSION}-${expected_release}.x86_64.rpm.sbom.json"
"silo-${PKG_VERSION}-${expected_release}.aarch64.rpm.sbom.json"
"${manifest}"
"${manifest}.sigstore.json"
"silo_${PKG_VERSION}_packages_provenance.sigstore.json"
)
gh release upload "${RELEASE_TAG}" --repo "${GITHUB_REPOSITORY}" --clobber "${files[@]}"
for asset in "${files[@]}"; do
local_digest="sha256:$(sha256sum "${asset}" | awk '{print $1}')"
remote_digest=""
for attempt in 1 2 3 4 5; do
remote_digest="$(gh release view "${RELEASE_TAG}" --repo "${GITHUB_REPOSITORY}" --json assets \
--jq ".assets[] | select(.name == \"${asset}\") | .digest")"
if [ "${local_digest}" = "${remote_digest}" ]; then
break
fi
if [ "${attempt}" -lt 5 ]; then
sleep 2
fi
done
if [ "${local_digest}" != "${remote_digest}" ]; then
echo "GitHub asset digest mismatch for ${asset}" >&2
echo "Local: ${local_digest}" >&2
echo "Remote: ${remote_digest}" >&2
exit 1
fi
echo "Verified GitHub asset: ${asset} ${remote_digest}"
done
+38 -19
View File
@@ -3,13 +3,13 @@ name: Go CI
on:
pull_request:
branches:
- master
- main
push:
branches:
- master
- main
workflow_dispatch:
# Cancel superseded runs for the same PR; never cancel master push runs.
# Cancel superseded runs for the same PR; never cancel main push runs.
# Keyed on PR number (not head_ref) so fork PRs sharing a branch name
# do not collide.
concurrency:
@@ -25,20 +25,21 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- uses: actions/setup-go@v5
- uses: actions/setup-go@v7
with:
go-version-file: go.mod
cache: true
- name: Check gofmt
run: |
unformatted=$(gofmt -l main.go cmd internal)
if [ -n "${unformatted}" ]; then
mapfile -t unformatted < <(gofmt -l main.go cmd internal \
buildscripts/rebrand-guard buildscripts/helm-migration-guard)
if [ "${#unformatted[@]}" -ne 0 ]; then
echo "The following files are not gofmt-formatted:"
echo "${unformatted}"
gofmt -d ${unformatted}
printf '%s\n' "${unformatted[@]}"
gofmt -d "${unformatted[@]}"
exit 1
fi
@@ -50,14 +51,20 @@ jobs:
- name: Vet
run: go vet ./...
- name: Verify rebrand compatibility contracts
run: |
go run ./buildscripts/rebrand-guard
buildscripts/verify-rebrand.sh
dockerscripts/docker-entrypoint_test.sh
quality:
name: Lint, Generated Files
runs-on: ubuntu-latest
timeout-minutes: 25
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- uses: actions/setup-go@v5
- uses: actions/setup-go@v7
with:
go-version-file: go.mod
cache: true
@@ -73,9 +80,9 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- uses: actions/setup-go@v5
- uses: actions/setup-go@v7
with:
go-version-file: go.mod
cache: true
@@ -83,14 +90,26 @@ jobs:
- name: Run S3 Select tests under race detector
run: go test -race ./internal/s3select/... -count=1
- name: Run conditional PUT tests under race detector
run: go test -race ./cmd -run '^Test(PoolsConditionalPut|SinglePoolConditionalPutHTTP)' -count=1 -timeout=5m
- name: Run multipart listing and cancellation tests under race detector
run: go test -race ./cmd -run '^Test(MultipartListing|MultipartAbort|PaginateMultipartUploads|ListMultipartUploads)' -count=1 -timeout=5m
- name: Run CPU metrics tests under race detector
run: go test -race ./cmd -run '^TestLoadCPUMetrics' -count=1 -timeout=5m
- name: Run tag replication tests under race detector
run: go test -race ./cmd -run '^TestAPITagging' -count=1 -timeout=5m
crosscompile:
name: Cross Compile
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- uses: actions/setup-go@v5
- uses: actions/setup-go@v7
with:
go-version-file: go.mod
cache: true
@@ -103,9 +122,9 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 25
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- uses: actions/setup-go@v5
- uses: actions/setup-go@v7
with:
go-version-file: go.mod
cache: true
@@ -127,9 +146,9 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 35
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- uses: actions/setup-go@v5
- uses: actions/setup-go@v7
with:
go-version-file: go.mod
cache: true
+133 -8
View File
@@ -1,5 +1,10 @@
name: Release
# Retry contract: an absent or single unfinalized Draft may be rebuilt from
# scratch; a published release or a Draft carrying finalize's GPG-derived
# provenance marker is terminal for this lane. The per-tag lock serializes
# workflows, but a maintainer must not publish the Draft while this job runs.
on:
push:
tags:
@@ -12,13 +17,20 @@ on:
permissions:
contents: write
id-token: write
attestations: write
artifact-metadata: write
concurrency:
group: release-${{ github.event_name == 'workflow_dispatch' && inputs.tag || github.ref_name }}
cancel-in-progress: false
jobs:
release:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
uses: actions/checkout@v7
with:
fetch-depth: 0
# Build the code at the tag being released, not whatever branch the
@@ -28,7 +40,7 @@ jobs:
ref: ${{ github.event.inputs.tag || github.ref }}
- name: Set up Go
uses: actions/setup-go@v5
uses: actions/setup-go@v7
with:
go-version-file: go.mod
cache: true
@@ -36,12 +48,28 @@ jobs:
- name: Verify clean checkout
run: |
set -euo pipefail
# GitHub's OIDC certificate records GITHUB_SHA, not the ref passed to
# actions/checkout. A manual dispatch must therefore be launched from
# the release tag itself; otherwise the provenance identity would
# describe different source from the bytes being published.
CHECKED_OUT_REVISION="$(git rev-parse HEAD)"
if [ "${CHECKED_OUT_REVISION}" != "${GITHUB_SHA}" ]; then
echo "Checked out ${CHECKED_OUT_REVISION}, but workflow identity is ${GITHUB_SHA}. Dispatch from the release tag." >&2
exit 1
fi
if [ -n "$(git status --porcelain)" ]; then
echo "Refusing to release from a dirty working tree:" >&2
git status --porcelain >&2
exit 1
fi
- name: Verify rebrand compatibility contracts
run: |
set -euo pipefail
go run ./buildscripts/rebrand-guard
buildscripts/verify-rebrand.sh
dockerscripts/docker-entrypoint_test.sh
- name: Compute release variables
env:
# Passed through the environment, never interpolated into the script
@@ -59,6 +87,15 @@ jobs:
echo "Invalid release tag format: ${TAG}" >&2
exit 1
fi
if ! TAG_COMMIT="$(git rev-parse "${TAG}^{commit}" 2>/dev/null)"; then
echo "Release tag ${TAG} does not resolve to a commit" >&2
exit 1
fi
HEAD_COMMIT="$(git rev-parse HEAD)"
if [ "${TAG_COMMIT}" != "${HEAD_COMMIT}" ]; then
echo "Release tag ${TAG} resolves to ${TAG_COMMIT}, checkout is ${HEAD_COMMIT}" >&2
exit 1
fi
VERSION_HYPHEN="${TAG#RELEASE.}"
PKG_VERSION="$(echo "${VERSION_HYPHEN}" | sed -E 's/^([0-9]{4})-([0-9]{2})-([0-9]{2})T([0-9]{2})-([0-9]{2})-([0-9]{2})Z$/\1\2\3\4\5\6.0.0/')"
VERSION_COLON="$(echo "${VERSION_HYPHEN}" | sed -E 's/T([0-9]{2})-([0-9]{2})-([0-9]{2})Z$/T\1:\2:\3Z/')"
@@ -74,13 +111,34 @@ jobs:
echo "Package version: ${PKG_VERSION}"
echo "LDFLAGS: ${LDFLAGS}"
- name: Build Draft release with GoReleaser
uses: goreleaser/goreleaser-action@v6
- name: Check existing release state
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
buildscripts/check-release-state.sh "${RELEASE_TAG}"
# Both installer actions are pinned to immutable commits. The explicit
# tool versions keep the release format reproducible across workflow
# reruns while the installers verify the downloaded executables.
- name: Install Syft
uses: anchore/sbom-action/download-syft@e22c389904149dbc22b58101806040fa8d37a610 # v0.24.0
with:
version: "~> v2"
syft-version: v1.50.0
- name: Install Cosign
uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2
with:
cosign-release: v3.1.2
- name: Build Draft release with GoReleaser
uses: goreleaser/goreleaser-action@f06c13b6b1a9625abc9e6e439d9c05a8f2190e94 # v7.2.3
with:
version: v2.18.1
args: release --clean --skip=validate --config .github/goreleaser.yml
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GORELEASER_CURRENT_TAG: ${{ env.RELEASE_TAG }}
LDFLAGS: ${{ env.LDFLAGS }}
PKG_VERSION: ${{ env.PKG_VERSION }}
@@ -100,22 +158,89 @@ jobs:
set -euo pipefail
buildscripts/package-release.sh
- name: Generate package SBOMs and signed checksum manifest
env:
SYFT_CHECK_FOR_APP_UPDATE: "false"
run: |
set -euo pipefail
packages_dir="dist/packages"
mapfile -t packages < <(find "${packages_dir}" -maxdepth 1 -type f \
\( -name '*.rpm' -o -name '*.deb' -o -name '*.apk' \) | sort)
if [ "${#packages[@]}" -ne 6 ]; then
echo "Expected six Linux packages, found ${#packages[@]}" >&2
printf '%s\n' "${packages[@]}" >&2
exit 1
fi
for package in "${packages[@]}"; do
syft "${package}" --output "spdx-json=${package}.sbom.json"
done
manifest="${packages_dir}/silo_${PKG_VERSION}_packages_checksums.txt"
(
cd "${packages_dir}"
mapfile -t subjects < <(find . -maxdepth 1 -type f \
\( -name '*.rpm' -o -name '*.deb' -o -name '*.apk' -o -name '*.sbom.json' \) | sort)
if [ "${#subjects[@]}" -ne 12 ]; then
echo "Expected six packages and six SBOMs, found ${#subjects[@]} subjects" >&2
exit 1
fi
sha256sum "${subjects[@]}" | sed 's# \./# #' > "$(basename "${manifest}")"
)
cosign sign-blob --bundle="${manifest}.sigstore.json" "${manifest}" --yes
- name: Attest downloadable release artifacts
id: attest-release
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-path: |
dist/*.tar.gz
dist/*.zip
dist/*.sbom.json
dist/*_checksums.txt
dist/*.sigstore.json
dist/packages/*.rpm
dist/packages/*.deb
dist/packages/*.apk
dist/packages/*.sha256sum
dist/packages/*.sbom.json
dist/packages/*_checksums.txt
dist/packages/*.sigstore.json
- name: Preserve provenance bundle as a release asset
env:
BUNDLE_PATH: ${{ steps.attest-release.outputs.bundle-path }}
run: |
set -euo pipefail
test -s "${BUNDLE_PATH}"
cp "${BUNDLE_PATH}" "dist/silo_${PKG_VERSION}_provenance.sigstore.json"
- name: Confirm unfinalized Draft release state
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REQUIRE_DRAFT: "true"
run: |
set -euo pipefail
buildscripts/check-release-state.sh "${RELEASE_TAG}"
- name: Upload nFPM packages to Draft release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
mapfile -t files < <(find dist/packages -maxdepth 1 -type f \
\( -name '*.rpm' -o -name '*.deb' -o -name '*.apk' -o -name '*.sha256sum' \) | sort)
\( -name '*.rpm' -o -name '*.deb' -o -name '*.apk' -o -name '*.sha256sum' \
-o -name '*.sbom.json' -o -name '*_checksums.txt' -o -name '*.sigstore.json' \) | sort)
if [ "${#files[@]}" -eq 0 ]; then
echo "No packages were generated."
exit 1
fi
gh release upload "${RELEASE_TAG}" "${files[@]}"
gh release upload "${RELEASE_TAG}" "${files[@]}" \
"dist/silo_${PKG_VERSION}_provenance.sigstore.json"
- name: Upload dist artifact
if: always()
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: dist
path: dist/
+100
View File
@@ -0,0 +1,100 @@
name: Repository Cards
on:
schedule:
# 00:00 UTC = 08:00 Asia/Shanghai. GitHub may queue scheduled runs.
- cron: "0 0 * * *"
workflow_dispatch:
push:
branches: [main]
paths:
- .github/workflows/repository-cards.yml
- .github/silo.svg
- buildscripts/repository-cards/**
pull_request:
branches: [main]
paths:
- .github/workflows/repository-cards.yml
- .github/silo.svg
- buildscripts/repository-cards/**
permissions:
contents: read
concurrency:
group: repository-cards-${{ github.event.pull_request.number || 'publish' }}
cancel-in-progress: false
jobs:
check:
name: Validate repository cards
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
persist-credentials: false
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
with:
python-version: "3.13"
- run: python -m pip install -r buildscripts/repository-cards/requirements.txt
- run: python -m unittest discover -s buildscripts/repository-cards -p 'test_*.py' -v
publish:
name: Update README images
needs: check
if: github.repository == 'pgsty/silo' && github.ref == 'refs/heads/main' && github.event_name != 'pull_request'
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: write
issues: read
pull-requests: read
env:
OUTPUT_BRANCH: codex/repository-cards
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
with:
python-version: "3.13"
- run: python -m pip install -r buildscripts/repository-cards/requirements.txt
- name: Load the generated-assets branch
shell: bash
run: |
set -euo pipefail
artifacts="$RUNNER_TEMP/repository-cards"
if git ls-remote --exit-code --heads origin "$OUTPUT_BRANCH"; then
git fetch --depth=1 origin "$OUTPUT_BRANCH"
git worktree add --detach "$artifacts" FETCH_HEAD
else
status=$?
# Exit 2 means no matching ref; transport/auth failures must stop.
if [ "$status" -ne 2 ]; then exit "$status"; fi
git worktree add --detach "$artifacts" HEAD
git -C "$artifacts" checkout --orphan "$OUTPUT_BRANCH"
git -C "$artifacts" rm -rf .
fi
- name: Refresh contributor and star cards
env:
GH_TOKEN: ${{ github.token }}
run: python buildscripts/repository-cards/update.py --output "$RUNNER_TEMP/repository-cards"
- name: Publish changed assets
shell: bash
run: |
set -euo pipefail
cd "$RUNNER_TEMP/repository-cards"
git add README.md history.json curated.json contributors.json \
contributors-light.svg contributors-dark.svg \
star-history-light.svg star-history-dark.svg
if git diff --cached --quiet; then
echo "Repository cards are already current."
exit 0
fi
git config user.name 'github-actions[bot]'
git config user.email '41898282+github-actions[bot]@users.noreply.github.com'
git commit -s -m "chore: update repository cards $(date -u +%F)"
# A normal push preserves history and refuses concurrent overwrites.
git push origin "HEAD:refs/heads/$OUTPUT_BRANCH"
+327 -72
View File
@@ -4,16 +4,38 @@ on:
workflow_dispatch:
pull_request:
paths:
- "go.mod"
- "go.sum"
- ".github/goreleaser.yml"
- ".github/nfpm.yml"
- "Dockerfile.goreleaser"
- "Dockerfile.distroless"
- "cmd/healthcheck-main.go"
- "cmd/main.go"
- "dockerscripts/build-static-curl.sh"
- "dockerscripts/docker-entrypoint.sh"
- "minio.service"
- "dockerscripts/docker-entrypoint_test.sh"
- "silo.service"
- "silo.env"
- "silo.sysusers"
- "buildscripts/package-release.sh"
- "buildscripts/package/postinstall.sh"
- "buildscripts/package/preremove.sh"
- "buildscripts/package/lifecycle_test.sh"
- "buildscripts/minio-upgrade.sh"
- "buildscripts/sign-release-rpms.sh"
- "buildscripts/verify-build-provenance.sh"
- "buildscripts/check-release-state.sh"
- "buildscripts/check-release-state_test.sh"
- "buildscripts/verify-rebrand.sh"
- "buildscripts/verify-helm-migration.sh"
- "buildscripts/helm-migration-guard/**"
- "helm/silo/**"
- "buildscripts/rebrand-guard/**"
- "buildscripts/gen-ldflags.go"
- ".github/workflows/release.yml"
- ".github/workflows/docker-release.yml"
- ".github/workflows/finalize-release.yml"
- ".github/workflows/test-release.yml"
- ".gitignore"
@@ -21,16 +43,38 @@ permissions:
contents: read
jobs:
curl:
name: Static curl (${{ matrix.arch }})
runs-on: ${{ matrix.runner }}
strategy:
matrix:
include:
- arch: amd64
runner: ubuntu-latest
- arch: arm64
runner: ubuntu-24.04-arm
steps:
- uses: actions/checkout@v7
- name: Build and exercise curl in an empty runtime
run: |
docker build --build-arg TARGETARCH=${{ matrix.arch }} \
--target curl-runtime -f Dockerfile.goreleaser -t silo-curl-test .
docker run --rm silo-curl-test --version | tee curl-version.txt
grep -F 'curl 8.22.0 ' curl-version.txt
grep -F 'HTTP2' curl-version.txt
docker run --rm silo-curl-test --fail --silent --show-error \
--connect-timeout 15 --max-time 60 https://curl.se/robots.txt
validate:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Set up Go
uses: actions/setup-go@v5
uses: actions/setup-go@v7
with:
go-version-file: go.mod
cache: true
@@ -51,20 +95,46 @@ jobs:
echo "LDFLAGS: ${LDFLAGS}"
- name: GoReleaser config check
uses: goreleaser/goreleaser-action@v6
uses: goreleaser/goreleaser-action@f06c13b6b1a9625abc9e6e439d9c05a8f2190e94 # v7.2.3
with:
version: "~> v2"
version: v2.18.1
args: check --config .github/goreleaser.yml
- name: Build snapshot artifacts
uses: goreleaser/goreleaser-action@v6
- name: Validate Helm chart and legacy upgrade identity
run: buildscripts/verify-helm-migration.sh
- name: Install Syft
uses: anchore/sbom-action/download-syft@e22c389904149dbc22b58101806040fa8d37a610 # v0.24.0
with:
version: "~> v2"
args: release --snapshot --clean --skip=publish,docker --config .github/goreleaser.yml
syft-version: v1.50.0
- name: Build snapshot artifacts
uses: goreleaser/goreleaser-action@f06c13b6b1a9625abc9e6e439d9c05a8f2190e94 # v7.2.3
with:
version: v2.18.1
# A pull-request snapshot has no trusted release identity. Exercise
# the SBOM/checksum pipeline here, and reserve keyless signing for
# the tag-triggered release workflow with GitHub OIDC.
args: release --snapshot --clean --skip=publish,docker,sign --config .github/goreleaser.yml
env:
LDFLAGS: ${{ env.LDFLAGS }}
PKG_VERSION: ${{ env.PKG_VERSION }}
- name: Verify archive SBOM and checksum coverage
run: |
set -euo pipefail
mapfile -t archives < <(find dist -maxdepth 1 -type f -name 'silo_*.tar.gz' | sort)
mapfile -t sboms < <(find dist -maxdepth 1 -type f -name 'silo_*.tar.gz.sbom.json' | sort)
test "${#archives[@]}" -eq 6
test "${#sboms[@]}" -eq 6
manifest="dist/silo_${PKG_VERSION}_checksums.txt"
test -s "${manifest}"
(
cd dist
sha256sum --check "$(basename "${manifest}")"
)
test "$(wc -l < "${manifest}" | tr -d ' ')" -eq 12
- name: Verify binary provenance stamps
run: |
set -euo pipefail
@@ -88,15 +158,38 @@ jobs:
set -euo pipefail
cd dist/packages
# The signing script asserts these same values, but it runs on the
# maintainer's machine after the release workflow has already built
# and uploaded. Take its expectations as the single source of truth
# so nfpm.yml and the signing script cannot drift apart without
# failing here first, while a fix is still cheap.
#
# This grep is deliberately limited to the eight identity variables,
# all of which are single-line. That is what makes the eval safe:
# should one ever become multi-line, the grep captures an
# unterminated quote and the eval aborts on a syntax error under
# set -e rather than quietly binding an empty value and comparing
# against nothing. expected_payload is multi-line by design and must
# stay out of this set for the same reason.
eval "$(grep -E '^expected_(release|vendor|packager|url|summary|description|license|group)=' \
../../buildscripts/sign-release-rpms.sh)"
for value in "${expected_release}" "${expected_vendor}" "${expected_packager}" \
"${expected_url}" "${expected_summary}" "${expected_description}" \
"${expected_license}" "${expected_group}"; do
test -n "${value}"
done
# These are the public download names; a drift here breaks every
# script that fetches packages by URL.
# script that fetches packages by URL. RPM and DEB carry the PGSTY
# release segment; APK cannot (Alpine pkgrel admits only -r<integer>),
# so it stays bare. package-release.sh builds the same three shapes.
expected=(
"minio-${PKG_VERSION}-1.aarch64.rpm"
"minio-${PKG_VERSION}-1.x86_64.rpm"
"minio_${PKG_VERSION}_aarch64.apk"
"minio_${PKG_VERSION}_amd64.deb"
"minio_${PKG_VERSION}_arm64.deb"
"minio_${PKG_VERSION}_x86_64.apk"
"silo-${PKG_VERSION}-${expected_release}.aarch64.rpm"
"silo-${PKG_VERSION}-${expected_release}.x86_64.rpm"
"silo_${PKG_VERSION}-${expected_release}_amd64.deb"
"silo_${PKG_VERSION}-${expected_release}_arm64.deb"
"silo_${PKG_VERSION}_aarch64.apk"
"silo_${PKG_VERSION}_x86_64.apk"
)
for package in "${expected[@]}"; do
@@ -107,30 +200,14 @@ jobs:
test "$(find . -maxdepth 1 -type f \( -name '*.rpm' -o -name '*.deb' -o -name '*.apk' \) | wc -l)" -eq 6
# The signing script asserts these same values, but it runs on the
# maintainer's machine after the release workflow has already built
# and uploaded. Take its expectations as the single source of truth
# so nfpm.yml and the signing script cannot drift apart without
# failing here first, while a fix is still cheap.
#
# This grep is deliberately limited to the seven identity variables,
# all of which are single-line. That is what makes the eval safe:
# should one ever become multi-line, the grep captures an
# unterminated quote and the eval aborts on a syntax error under
# set -e rather than quietly binding an empty value and comparing
# against nothing. expected_payload is multi-line by design and must
# stay out of this set for the same reason.
eval "$(grep -E '^expected_(vendor|packager|url|summary|description|license|group)=' \
../../buildscripts/sign-release-rpms.sh)"
for value in "${expected_vendor}" "${expected_packager}" "${expected_url}" \
"${expected_summary}" "${expected_description}" \
"${expected_license}" "${expected_group}"; do
test -n "${value}"
done
service_sha="$(sha256sum ../../silo.service | awk '{print $1}')"
defaults_sha="$(sha256sum ../../silo.env | awk '{print $1}')"
sysusers_sha="$(sha256sum ../../silo.sysusers | awk '{print $1}')"
license_sha="$(sha256sum ../../LICENSE | awk '{print $1}')"
notice_sha="$(sha256sum ../../NOTICE | awk '{print $1}')"
service_sha="$(sha256sum ../../minio.service | awk '{print $1}')"
rpm_file="minio-${PKG_VERSION}-1.x86_64.rpm"
rpm_file="silo-${PKG_VERSION}-${expected_release}.x86_64.rpm"
test "$(rpm -qp --queryformat '%{RELEASE}' "${rpm_file}")" = "${expected_release}"
test "$(rpm -qp --queryformat '%{VENDOR}' "${rpm_file}")" = "${expected_vendor}"
test "$(rpm -qp --queryformat '%{PACKAGER}' "${rpm_file}")" = "${expected_packager}"
test "$(rpm -qp --queryformat '%{URL}' "${rpm_file}")" = "${expected_url}"
@@ -138,31 +215,68 @@ jobs:
test "$(rpm -qp --queryformat '%{DESCRIPTION}' "${rpm_file}")" = "${expected_description}"
test "$(rpm -qp --queryformat '%{LICENSE}' "${rpm_file}")" = "${expected_license}"
test "$(rpm -qp --queryformat '%{GROUP}' "${rpm_file}")" = "${expected_group}"
# Both payload entries: the unit file is as much a part of the
# package as the binary, and losing it would install a server with
# nothing to start it.
rpm -qpl "${rpm_file}" | grep -Fx '/usr/local/bin/minio'
rpm -qpl "${rpm_file}" | grep -Fx '/usr/lib/systemd/system/minio.service'
test "$(rpm -qpl "${rpm_file}" | wc -l)" -eq 2
rpm -qpl "${rpm_file}" | grep -Fx '/usr/bin/silo'
rpm -qpl "${rpm_file}" | grep -Fx '/usr/lib/systemd/system/silo.service'
rpm -qpl "${rpm_file}" | grep -Fx '/etc/default/silo'
rpm -qpl "${rpm_file}" | grep -Fx '/usr/lib/sysusers.d/silo.conf'
rpm -qpl "${rpm_file}" | grep -Fx '/usr/share/doc/silo/LICENSE'
rpm -qpl "${rpm_file}" | grep -Fx '/usr/share/doc/silo/NOTICE'
test "$(rpm -qpl "${rpm_file}" | wc -l)" -eq 6
# nfpm only honors type: license on rpm, which is why nfpm.yml
# declares the license materials once per packager. Pin the rpm
# %license flag so that split cannot silently regress.
rpm -qp --queryformat '[%{FILEFLAGS:fflags} %{FILENAMES}\n]' "${rpm_file}" \
| grep -Fx 'l /usr/share/doc/silo/LICENSE'
rpm -qp --queryformat '[%{FILEFLAGS:fflags} %{FILENAMES}\n]' "${rpm_file}" \
| grep -Fx 'l /usr/share/doc/silo/NOTICE'
if rpm -qp --conflicts "${rpm_file}" | grep -qi minio; then
echo "RPM must not declare a cross-name conflict with MinIO" >&2
exit 1
fi
if rpm -qp --obsoletes "${rpm_file}" | grep -qi minio; then
echo "RPM must not obsolete a MinIO package" >&2
exit 1
fi
if rpm -qp --provides "${rpm_file}" | grep -qi minio; then
echo "RPM must not provide a MinIO package alias" >&2
exit 1
fi
deb_file="minio_${PKG_VERSION}_amd64.deb"
deb_file="silo_${PKG_VERSION}-${expected_release}_amd64.deb"
test "$(dpkg-deb --field "${deb_file}" Maintainer)" = "${expected_packager}"
test "$(dpkg-deb --field "${deb_file}" Version)" = "${PKG_VERSION}"
test "$(dpkg-deb --field "${deb_file}" Version)" = "${PKG_VERSION}-${expected_release}"
test "$(dpkg-deb --field "${deb_file}" License)" = "${expected_license}"
test "$(dpkg-deb --field "${deb_file}" Section)" = "utils"
test "$(dpkg-deb --field "${deb_file}" Homepage)" = "${expected_url}"
test "$(dpkg-deb --field "${deb_file}" Description)" = "${expected_description}"
dpkg-deb --contents "${deb_file}" | grep -E 'usr/local/bin/minio$'
dpkg-deb --contents "${deb_file}" | grep -E 'usr/lib/systemd/system/minio\.service$'
dpkg-deb --contents "${deb_file}" | grep -E 'usr/bin/silo$'
dpkg-deb --contents "${deb_file}" | grep -E 'usr/lib/systemd/system/silo\.service$'
dpkg-deb --contents "${deb_file}" | grep -E 'etc/default/silo$'
dpkg-deb --contents "${deb_file}" | grep -E 'usr/lib/sysusers\.d/silo\.conf$'
dpkg-deb --contents "${deb_file}" | grep -E 'usr/share/doc/silo/LICENSE$'
dpkg-deb --contents "${deb_file}" | grep -E 'usr/share/doc/silo/NOTICE$'
test "$(dpkg-deb --contents "${deb_file}" | awk '$1 !~ /^d/ { count++ } END { print count + 0 }')" -eq 6
test -z "$(dpkg-deb --field "${deb_file}" Conflicts)"
test -z "$(dpkg-deb --field "${deb_file}" Replaces)"
test -z "$(dpkg-deb --field "${deb_file}" Provides)"
apk_info="$(tar -xOzf "minio_${PKG_VERSION}_x86_64.apk" .PKGINFO)"
apk_info="$(tar -xOzf "silo_${PKG_VERSION}_x86_64.apk" .PKGINFO)"
grep -Fx "pkgver = ${PKG_VERSION}" <<< "${apk_info}"
grep -Fx "url = ${expected_url}" <<< "${apk_info}"
grep -Fx "maintainer = ${expected_packager}" <<< "${apk_info}"
grep -Fx "license = ${expected_license}" <<< "${apk_info}"
grep -Fx "pkgdesc = ${expected_description}" <<< "${apk_info}"
tar -tzf "minio_${PKG_VERSION}_x86_64.apk" | grep -Fx 'usr/local/bin/minio'
tar -tzf "minio_${PKG_VERSION}_x86_64.apk" | grep -Fx 'usr/lib/systemd/system/minio.service'
tar -tzf "silo_${PKG_VERSION}_x86_64.apk" | grep -Fx 'usr/bin/silo'
tar -tzf "silo_${PKG_VERSION}_x86_64.apk" | grep -Fx 'usr/lib/systemd/system/silo.service'
tar -tzf "silo_${PKG_VERSION}_x86_64.apk" | grep -Fx 'etc/default/silo'
tar -tzf "silo_${PKG_VERSION}_x86_64.apk" | grep -Fx 'usr/lib/sysusers.d/silo.conf'
tar -tzf "silo_${PKG_VERSION}_x86_64.apk" | grep -Fx 'usr/share/doc/silo/LICENSE'
tar -tzf "silo_${PKG_VERSION}_x86_64.apk" | grep -Fx 'usr/share/doc/silo/NOTICE'
test "$(tar -tzf "silo_${PKG_VERSION}_x86_64.apk" | awk '$0 !~ /^\./ && $0 !~ /\/$/ { count++ } END { print count + 0 }')" -eq 6
if grep -Ei '^provides = .*minio' <<< "${apk_info}"; then
echo "APK must not provide a MinIO package alias" >&2
exit 1
fi
for arch in amd64 arm64; do
if [ "${arch}" = amd64 ]; then
@@ -175,9 +289,9 @@ jobs:
apk_arch=aarch64
fi
test "$(rpm -qp --queryformat '%{ARCH}' "minio-${PKG_VERSION}-1.${rpm_arch}.rpm")" = "${rpm_arch}"
test "$(dpkg-deb --field "minio_${PKG_VERSION}_${deb_arch}.deb" Architecture)" = "${deb_arch}"
grep -Fx "arch = ${apk_arch}" <<< "$(tar -xOzf "minio_${PKG_VERSION}_${apk_arch}.apk" .PKGINFO)"
test "$(rpm -qp --queryformat '%{ARCH}' "silo-${PKG_VERSION}-${expected_release}.${rpm_arch}.rpm")" = "${rpm_arch}"
test "$(dpkg-deb --field "silo_${PKG_VERSION}-${expected_release}_${deb_arch}.deb" Architecture)" = "${deb_arch}"
grep -Fx "arch = ${apk_arch}" <<< "$(tar -xOzf "silo_${PKG_VERSION}_${apk_arch}.apk" .PKGINFO)"
# Accepted weakness: this takes the first match, unsorted, where
# find_binary in package-release.sh demands exactly one. It cannot
@@ -186,7 +300,7 @@ jobs:
# goamd64 gains a second level, or if find_binary's exactly-one
# contract is ever relaxed -- at that point this weak copy would be
# the only one left choosing silently.
source_binary="$(find .. -maxdepth 2 -type f -path "../minio_linux_${arch}*/minio" | head -n 1)"
source_binary="$(find .. -maxdepth 2 -type f -path "../silo_linux_${arch}*/silo" | head -n 1)"
source_sha="$(sha256sum "${source_binary}" | awk '{print $1}')"
# Do not pipe rpm2cpio here: Debian's build exits non-zero even when
@@ -194,15 +308,31 @@ jobs:
# rpm's own digests instead -- -K checks the payload against the
# header, and FILEDIGESTS is the sha256 rpm itself verifies on
# install.
rpm -K "minio-${PKG_VERSION}-1.${rpm_arch}.rpm"
rpm -K "silo-${PKG_VERSION}-${expected_release}.${rpm_arch}.rpm"
rpm_sha="$(rpm -qp --queryformat '[%{FILENAMES} %{FILEDIGESTS}\n]' \
"minio-${PKG_VERSION}-1.${rpm_arch}.rpm" | awk '$1 == "/usr/local/bin/minio" { print $2 }')"
"silo-${PKG_VERSION}-${expected_release}.${rpm_arch}.rpm" | awk '$1 == "/usr/bin/silo" { print $2 }')"
rpm_service_sha="$(rpm -qp --queryformat '[%{FILENAMES} %{FILEDIGESTS}\n]' \
"minio-${PKG_VERSION}-1.${rpm_arch}.rpm" | awk '$1 == "/usr/lib/systemd/system/minio.service" { print $2 }')"
deb_sha="$(ar p "minio_${PKG_VERSION}_${deb_arch}.deb" data.tar.gz | tar -xzOf - ./usr/local/bin/minio | sha256sum | awk '{print $1}')"
deb_service_sha="$(ar p "minio_${PKG_VERSION}_${deb_arch}.deb" data.tar.gz | tar -xzOf - ./usr/lib/systemd/system/minio.service | sha256sum | awk '{print $1}')"
apk_sha="$(tar -xzOf "minio_${PKG_VERSION}_${apk_arch}.apk" usr/local/bin/minio | sha256sum | awk '{print $1}')"
apk_service_sha="$(tar -xzOf "minio_${PKG_VERSION}_${apk_arch}.apk" usr/lib/systemd/system/minio.service | sha256sum | awk '{print $1}')"
"silo-${PKG_VERSION}-${expected_release}.${rpm_arch}.rpm" | awk '$1 == "/usr/lib/systemd/system/silo.service" { print $2 }')"
rpm_defaults_sha="$(rpm -qp --queryformat '[%{FILENAMES} %{FILEDIGESTS}\n]' \
"silo-${PKG_VERSION}-${expected_release}.${rpm_arch}.rpm" | awk '$1 == "/etc/default/silo" { print $2 }')"
rpm_sysusers_sha="$(rpm -qp --queryformat '[%{FILENAMES} %{FILEDIGESTS}\n]' \
"silo-${PKG_VERSION}-${expected_release}.${rpm_arch}.rpm" | awk '$1 == "/usr/lib/sysusers.d/silo.conf" { print $2 }')"
rpm_license_sha="$(rpm -qp --queryformat '[%{FILENAMES} %{FILEDIGESTS}\n]' \
"silo-${PKG_VERSION}-${expected_release}.${rpm_arch}.rpm" | awk '$1 == "/usr/share/doc/silo/LICENSE" { print $2 }')"
rpm_notice_sha="$(rpm -qp --queryformat '[%{FILENAMES} %{FILEDIGESTS}\n]' \
"silo-${PKG_VERSION}-${expected_release}.${rpm_arch}.rpm" | awk '$1 == "/usr/share/doc/silo/NOTICE" { print $2 }')"
deb_sha="$(ar p "silo_${PKG_VERSION}-${expected_release}_${deb_arch}.deb" data.tar.gz | tar -xzOf - ./usr/bin/silo | sha256sum | awk '{print $1}')"
deb_service_sha="$(ar p "silo_${PKG_VERSION}-${expected_release}_${deb_arch}.deb" data.tar.gz | tar -xzOf - ./usr/lib/systemd/system/silo.service | sha256sum | awk '{print $1}')"
deb_defaults_sha="$(ar p "silo_${PKG_VERSION}-${expected_release}_${deb_arch}.deb" data.tar.gz | tar -xzOf - ./etc/default/silo | sha256sum | awk '{print $1}')"
deb_sysusers_sha="$(ar p "silo_${PKG_VERSION}-${expected_release}_${deb_arch}.deb" data.tar.gz | tar -xzOf - ./usr/lib/sysusers.d/silo.conf | sha256sum | awk '{print $1}')"
deb_license_sha="$(ar p "silo_${PKG_VERSION}-${expected_release}_${deb_arch}.deb" data.tar.gz | tar -xzOf - ./usr/share/doc/silo/LICENSE | sha256sum | awk '{print $1}')"
deb_notice_sha="$(ar p "silo_${PKG_VERSION}-${expected_release}_${deb_arch}.deb" data.tar.gz | tar -xzOf - ./usr/share/doc/silo/NOTICE | sha256sum | awk '{print $1}')"
apk_sha="$(tar -xzOf "silo_${PKG_VERSION}_${apk_arch}.apk" usr/bin/silo | sha256sum | awk '{print $1}')"
apk_service_sha="$(tar -xzOf "silo_${PKG_VERSION}_${apk_arch}.apk" usr/lib/systemd/system/silo.service | sha256sum | awk '{print $1}')"
apk_defaults_sha="$(tar -xzOf "silo_${PKG_VERSION}_${apk_arch}.apk" etc/default/silo | sha256sum | awk '{print $1}')"
apk_sysusers_sha="$(tar -xzOf "silo_${PKG_VERSION}_${apk_arch}.apk" usr/lib/sysusers.d/silo.conf | sha256sum | awk '{print $1}')"
apk_license_sha="$(tar -xzOf "silo_${PKG_VERSION}_${apk_arch}.apk" usr/share/doc/silo/LICENSE | sha256sum | awk '{print $1}')"
apk_notice_sha="$(tar -xzOf "silo_${PKG_VERSION}_${apk_arch}.apk" usr/share/doc/silo/NOTICE | sha256sum | awk '{print $1}')"
test "${source_sha}" = "${rpm_sha}"
test "${source_sha}" = "${deb_sha}"
@@ -210,6 +340,18 @@ jobs:
test "${service_sha}" = "${rpm_service_sha}"
test "${service_sha}" = "${deb_service_sha}"
test "${service_sha}" = "${apk_service_sha}"
test "${defaults_sha}" = "${rpm_defaults_sha}"
test "${defaults_sha}" = "${deb_defaults_sha}"
test "${defaults_sha}" = "${apk_defaults_sha}"
test "${sysusers_sha}" = "${rpm_sysusers_sha}"
test "${sysusers_sha}" = "${deb_sysusers_sha}"
test "${sysusers_sha}" = "${apk_sysusers_sha}"
test "${license_sha}" = "${rpm_license_sha}"
test "${license_sha}" = "${deb_license_sha}"
test "${license_sha}" = "${apk_license_sha}"
test "${notice_sha}" = "${rpm_notice_sha}"
test "${notice_sha}" = "${deb_notice_sha}"
test "${notice_sha}" = "${apk_notice_sha}"
done
find . -maxdepth 1 -type f | sort
@@ -223,19 +365,20 @@ jobs:
# goreleaser already produced; the mcli-download build stage is skipped
# on purpose to keep this gate offline and deterministic.
ctx="$(mktemp -d)"
tar -xzf "dist/minio_${PKG_VERSION}_linux_amd64.tar.gz" -C "${ctx}" minio
tar -xzf "dist/silo_${PKG_VERSION}_linux_amd64.tar.gz" -C "${ctx}" silo
cp dockerscripts/docker-entrypoint.sh "${ctx}/docker-entrypoint.sh"
{
echo "FROM registry.access.redhat.com/ubi9/ubi-micro:latest"
echo "COPY minio /usr/bin/minio"
echo "COPY silo /usr/bin/silo"
echo "COPY docker-entrypoint.sh /usr/bin/docker-entrypoint.sh"
echo "RUN mkdir -p /data && chmod 0777 /data && chmod +x /usr/bin/minio /usr/bin/docker-entrypoint.sh"
echo "RUN mkdir -p /data && chmod 0777 /data && chmod +x /usr/bin/silo /usr/bin/docker-entrypoint.sh"
echo 'ENV HOME=/tmp'
echo 'ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"]'
echo 'CMD ["minio"]'
echo 'CMD ["silo"]'
} > "${ctx}/Dockerfile"
docker build -t minio-runtime-test:snapshot "${ctx}"
docker build -t silo-runtime-test:snapshot "${ctx}"
# PID 1 must be minio, not the entry shell, on every privilege path, so
# PID 1 must be silo, not the entry shell, on every privilege path, so
# a SIGTERM from docker stop reaches the server and it exits gracefully
# instead of being killed at the stop timeout. Regression guard for the
# exec-into-chroot entrypoint fix.
@@ -244,7 +387,7 @@ jobs:
docker rm -f "${name}" >/dev/null 2>&1 || true
docker run -d --name "${name}" \
-e MINIO_CI_CD=1 -e MINIO_ROOT_USER=ciadmin -e MINIO_ROOT_PASSWORD=ciadmin-secret-123 \
"$@" minio-runtime-test:snapshot minio server /data --address :9000 >/dev/null
"$@" silo-runtime-test:snapshot silo server /data --address :9000 >/dev/null
up=""
for _ in $(seq 1 60); do
if docker logs "${name}" 2>&1 | grep -q "API:"; then up=1; break; fi
@@ -263,15 +406,127 @@ jobs:
[ "${code}" = "0" ] || { echo "non-zero exit (${name}): ${code}"; exit 1; }
[ "${elapsed}" -lt 10 ] || { echo "shutdown too slow (${name}): ${elapsed}s - signal not forwarded"; exit 1; }
}
assert_graceful minio-rt-default
assert_graceful minio-rt-dropuser -e MINIO_USERNAME=minio-user -e MINIO_GROUPNAME=minio-group
assert_graceful silo-rt-default
assert_graceful silo-rt-dropuser -e MINIO_USERNAME=silo-user -e MINIO_GROUPNAME=silo-group
assert_graceful silo-rt-rootless --user 1001:1001
# The compatibility shim translates only the legacy first argv token;
# the image contains no /usr/bin/minio file.
docker run --rm silo-runtime-test:snapshot sh -c 'test ! -e /usr/bin/minio'
docker run --rm -d --name silo-rt-legacy \
-e MINIO_CI_CD=1 -e MINIO_ROOT_USER=ciadmin -e MINIO_ROOT_PASSWORD=ciadmin-secret-123 \
silo-runtime-test:snapshot minio server /data --address :9000 >/dev/null
sleep 2
test "$(docker exec silo-rt-legacy cat /proc/1/comm)" = silo
docker rm -f silo-rt-legacy >/dev/null
- name: Build distroless runtime image and verify native healthcheck
run: |
set -euo pipefail
# Unlike the classic image, Dockerfile.distroless has no release
# download stages, so the real shipped file can be built and gated
# here. It must keep working with nothing in it but the silo
# binary: no shell, no mc, no entrypoint script.
ctx="$(mktemp -d)"
tar -xzf "dist/silo_${PKG_VERSION}_linux_amd64.tar.gz" -C "${ctx}" silo
cp Dockerfile.distroless LICENSE NOTICE CREDITS "${ctx}/"
docker build -t silo-distroless-test:snapshot -f "${ctx}/Dockerfile.distroless" "${ctx}"
# HEALTHCHECK is a Docker extension absent from the OCI image
# spec; assert the exact probe command survived into the image
# config, not merely a substring of it.
test "$(docker inspect -f '{{json .Config.Healthcheck.Test}}' silo-distroless-test:snapshot)" \
= '["CMD","/usr/bin/silo","healthcheck","ready"]'
# /data ships in the image layer world-writable (issue #55):
# there is no entrypoint left to repair ownership at runtime.
# Export the rootfs once, then assert each required and each
# forbidden entry individually: tar's member-argument mode exits
# non-zero on any missing name, which under pipefail masks a
# found forbidden file, and -tv prints symlinks as 'name ->
# target' which defeats $-anchored greps.
probe="$(docker create silo-distroless-test:snapshot server /data)"
docker export "${probe}" -o "${ctx}/rootfs.tar"
docker rm "${probe}" >/dev/null
tar -tf "${ctx}/rootfs.tar" > "${ctx}/names.txt"
tar -tvf "${ctx}/rootfs.tar" > "${ctx}/verbose.txt"
grep -E '^drwxrwxrwx.* data/$' "${ctx}/verbose.txt" >/dev/null
for want in usr/bin/silo licenses/LICENSE licenses/NOTICE licenses/CREDITS; do
grep -Fxq "${want}" "${ctx}/names.txt" || { echo "missing ${want}"; exit 1; }
done
for forbid in bin/sh usr/bin/sh busybox/sh usr/bin/minio usr/bin/mc usr/bin/mcli; do
if grep -Fxq "${forbid}" "${ctx}/names.txt"; then
echo "distroless image unexpectedly contains ${forbid}"
exit 1
fi
done
# The baked-in healthcheck must drive Docker's health state on its
# own, the probe binary must be directly exec-able without any
# shell, and SIGTERM must still reach PID 1 (the server binary is
# the entrypoint) for a graceful stop.
assert_distroless() {
name="$1"; shift
docker rm -f "${name}" >/dev/null 2>&1 || true
docker run -d --name "${name}" \
-e MINIO_CI_CD=1 -e MINIO_ROOT_USER=ciadmin -e MINIO_ROOT_PASSWORD=ciadmin-secret-123 \
"$@" silo-distroless-test:snapshot server /data --address :9000 >/dev/null
status=""
for _ in $(seq 1 90); do
status="$(docker inspect -f '{{.State.Health.Status}}' "${name}" 2>/dev/null || echo '?')"
if [ "${status}" = "healthy" ]; then break; fi
if [ "$(docker inspect -f '{{.State.Running}}' "${name}")" != "true" ]; then break; fi
sleep 1
done
if [ "${status}" != "healthy" ]; then
echo "container never became healthy (${name}): status=${status}"
docker logs "${name}" 2>&1 | tail -5
exit 1
fi
docker exec "${name}" /usr/bin/silo healthcheck ready
docker exec "${name}" /usr/bin/silo healthcheck cluster
start="$(date +%s)"; docker stop -t 15 "${name}" >/dev/null; end="$(date +%s)"
code="$(docker inspect -f '{{.State.ExitCode}}' "${name}")"
elapsed=$((end - start))
graceful=0; docker logs "${name}" 2>&1 | grep -q "Exiting on signal" && graceful=1
echo "${name}: health=${status} stop=${elapsed}s exit=${code}"
docker rm -f "${name}" >/dev/null 2>&1 || true
[ "${graceful}" = "1" ] || { echo "no graceful-shutdown log (${name}) - signal not forwarded"; exit 1; }
[ "${code}" = "0" ] || { echo "non-zero exit (${name}): ${code}"; exit 1; }
[ "${elapsed}" -lt 10 ] || { echo "shutdown too slow (${name}): ${elapsed}s - signal not forwarded"; exit 1; }
}
assert_distroless silo-dl-default
assert_distroless silo-dl-rootless --user 1001:1001
- name: Validate release scripts
run: |
set -euo pipefail
bash -n buildscripts/package-release.sh
bash -n buildscripts/minio-upgrade.sh
bash -n buildscripts/sign-release-rpms.sh
bash -n buildscripts/verify-build-provenance.sh
bash -n buildscripts/check-release-state.sh
bash -n buildscripts/check-release-state_test.sh
bash -n buildscripts/verify-rebrand.sh
bash -n buildscripts/verify-helm-migration.sh
sh -n buildscripts/package/postinstall.sh
sh -n buildscripts/package/preremove.sh
bash -n buildscripts/package/lifecycle_test.sh
buildscripts/package/lifecycle_test.sh
bash -n dockerscripts/docker-entrypoint_test.sh
bash -n dockerscripts/build-static-curl.sh
dockerscripts/docker-entrypoint_test.sh
go run ./buildscripts/rebrand-guard
buildscripts/verify-rebrand.sh
test -x buildscripts/package-release.sh
test -x buildscripts/sign-release-rpms.sh
test -x buildscripts/verify-build-provenance.sh
test -x buildscripts/check-release-state.sh
test -x buildscripts/check-release-state_test.sh
test -x buildscripts/verify-rebrand.sh
test -x buildscripts/verify-helm-migration.sh
test -x buildscripts/package/postinstall.sh
test -x buildscripts/package/preremove.sh
test -x buildscripts/package/lifecycle_test.sh
test -x dockerscripts/docker-entrypoint_test.sh
buildscripts/check-release-state_test.sh
+5 -5
View File
@@ -3,10 +3,10 @@ name: VulnCheck
on:
pull_request:
branches:
- master
- main
push:
branches:
- master
- main
workflow_dispatch:
permissions:
@@ -19,17 +19,17 @@ jobs:
timeout-minutes: 20
steps:
- name: Check out code
uses: actions/checkout@v4
uses: actions/checkout@v7
- name: Set up Go
uses: actions/setup-go@v5
uses: actions/setup-go@v7
with:
go-version-file: go.mod
cache: true
- name: Install govulncheck
run: |
go install golang.org/x/vuln/cmd/govulncheck@v1.6.0
go install golang.org/x/vuln/cmd/govulncheck@v1.8.0
echo "$(go env GOPATH)/bin" >> "${GITHUB_PATH}"
- name: Run govulncheck
+5 -2
View File
@@ -2,6 +2,7 @@
cover.out
*~
minio
silo
!*/
site/
**/*.test
@@ -61,8 +62,10 @@ dist/
.claude/
.codex/
AGENTS.md
CLAUDE.md
_bmad/
_bmad-output/
# Investigation and editorial working files belong outside this repository.
docs/investigations/
docs/security/
docs/rebranding.md
.release-sign/
+1 -1
View File
@@ -5,7 +5,7 @@ linters:
- durationcheck
- forcetypeassert
- gocritic
- gomodguard
- gomodguard_v2
- govet
- ineffassign
- misspell
+84
View File
@@ -0,0 +1,84 @@
# SILO Repository Guide
## Project map
- Server: `pgsty/silo` (this checkout, default branch `main`)
- Console: `pgsty/silo-console` (usual sibling checkout `../silo-console`)
- Client: `pgsty/mc`, shipped as `mcli` (usual sibling checkout `../mc`)
- Shared packages: `pgsty/silo-pkg` (usual sibling checkout `../silo-pkg`)
- Documentation: `pgsty/silo.pgsty.com` (usual sibling checkout `../silo.pgsty.com`),
published at <https://silo.pgsty.com/>
The server executable, package, systemd service, and container image use `silo`;
the public Docker image is `docker.io/pgsty/silo`. The server module remains
`github.com/minio/minio`. Consult the current `go.mod` and release configuration
for selected component versions and compatibility replacements.
## Supported stack and compatibility policy
The maintained, release-gating product graph is the coordinated PGSTY stack:
`silo` + `silo-console` + `mc` + `silo-pkg`.
Keep compatibility with upstream MinIO/MC on a best-effort basis. Preserve
inexpensive wire, configuration, CLI, migration, and import compatibility when
it helps users, and document known differences. Do not infer from source
lineage, MinIO-compatible protocols, retained `MINIO_*`/`MC_*` names, or an
inherited upstream test that unmodified upstream MinIO is a supported release
target.
Upstream-only compatibility checks may remain as advisory evidence, but they
must not force a downgrade of a maintained SILO component, a fork-only API
workaround, or a release block. Make upstream compatibility a hard gate only
when the user explicitly requests that scope.
## Dependency selection
- When PGSTY maintains a component, import and require it directly under its own
module path. In particular, prefer `github.com/pgsty/silo-pkg/v3` over
`github.com/minio/pkg/v3` in maintained SILO source.
- Do not use an upstream package merely to make unmodified upstream MinIO or MC
compile. Unavoidable transitive upstream modules should be documented and
kept separate from the maintained product dependency.
- `github.com/minio/minio-go/v7` is the explicit exception: use the verified
upstream module/commit while it contains the required fixes; do not recreate
a SILO fork without a concrete functional divergence.
- Check the current `go.mod` before changing versions. Coordinate breaking
import-path changes across package, client, Console, and server releases.
## Documentation and delivery
The companion site owns product, operations, migration, security-advisory,
design, and release documentation. Maintain English and Chinese content in
that repository and link to its canonical URLs from this one. The site's
`content/docs/` is the documentation entry point; detailed pages live under
`content/operations/`, `administration/`, `reference/`, `compatibility/`,
`about/`, and `blog/`, following the existing structure.
Keep repository entry points and contributor instructions here. Retained
upstream documents, examples, and test fixtures under `docs/` may be updated
when a code or tooling change requires it; do not build a second documentation
site in this tree.
Investigation plans, prompts, AI session transcripts, execution logs, temporary
reports, and private security material belong outside the checkout, for example
in a task-specific directory under `~/tmp/`. Do not recreate
`docs/investigations/`, `docs/security/`, or `docs/rebranding.md`, or move their
working material to another repository directory. Extract reusable, verified
knowledge into the companion site; keep private evidence outside public Git.
Compatibility pages may continue to describe similarities with upstream, but
must label that compatibility as best effort. The supported and tested server
for Console and mcli administration is `pgsty/silo`.
Before removing or moving documentation, check repository links and scripts,
the companion site's links and anchors, and references from this guide. Use
fixed commit URLs for historical evidence that should survive deletion from
the current tree. Run the site's `make check` for site changes and this
repository's `make rebrand-guard` for documentation cleanup that changes the
identifier inventory. When new public URLs are involved, publish the site
before publishing source changes that depend on those URLs.
Keep changes in the repository that owns the affected surface. Treat every
repository commit, tag, release, image, documentation update, and deployment as
a separate deliverable. Follow `CONTRIBUTING.md`, including DCO sign-off
(`git commit -s`). `CLAUDE.md` imports this guide so both agents use one policy.
+238
View File
@@ -0,0 +1,238 @@
# Changelog
## Unreleased
Preparation target: `RELEASE.2026-09-16T00-00-00Z` (package version
`20260916000000.0.0`). The entries below describe the candidate changes since
the latest published Server.
**The latest published Server remains 20260903.** These changes are not in its
binaries, packages or images. See the [component matrix](https://silo.pgsty.com/compatibility/versions/)
and [complete commit range](https://github.com/pgsty/silo/compare/RELEASE.2026-09-03T13-18-01Z...main).
### Authorization and security
- Synchronize CPU metrics reads with resource-metrics updates (#210), preventing
concurrent map access from terminating the server during Prometheus scraping.
Metric names, values and authentication requirements are unchanged.
- Restrict embedded Console's anonymous sharing proxy to object-content GETs
at the configured S3 origin, and reject every redirect. Internal metrics,
system paths and non-download S3 operations cannot be reached through it.
Normal public, presigned and versioned downloads remain available without a
new setting; a full sharing-disable switch is not introduced. See
[Console #56](https://github.com/pgsty/silo-console/pull/56) and the
[design record](https://github.com/pgsty/silo-console/issues/52).
Thanks to Jiri Pejchal (@jiri-pejchal) for the report.
- Persist IAM deletion revisions and parent revocation boundaries so stale site
events cannot restore deleted identities, policies or their older grants
(#191, #192). Peer deletion notifications reload committed storage; deliberate
recreation requires a newer revision, and credentials issued before the
parent's revocation remain invalid.
**Coordinated upgrade required:** upgrade every participating node and site.
Mixed old/new nodes sharing an IAM backend and rolling downgrade are
unsupported. Back up complete IAM storage and encryption material; an admin
export of live records omits deletion history. Reissue credentials for
recreated parents and explicitly reconcile pre-upgrade revocations whose
history is already lost. Restoring an older backup can lose later revocations;
keep affected sites isolated until reconciliation/rekeying is complete. See
[the operator runbook](https://silo.pgsty.com/operations/replication/iam-upgrade/).
- Enforce an absolute HTTP/1 request-header deadline through the connection
wrapper (#196). Repeated small reads no longer extend that deadline, and
`--read-header-timeout` / `MINIO_READ_HEADER_TIMEOUT` now reaches the HTTP
server. HTTP/1 request bodies retain the rolling idle timeout; this does not
impose a total upload/download duration. A shorter setting also constrains
TLS handshake reads. The wrapper's strict header mode is not applied to HTTP/2.
- Reject unsigned `x-amz-*` request headers that could turn a signed PUT into a
copy of another object accessible to the signer (SN-2026-011). The latest
public Server is affected; the fix is on main. See [the advisory ledger](https://silo.pgsty.com/about/security-advisories/).
- Align signed request fields with policy conditions and enforce header-only
presigned payload checksums. See [the signed-header review](https://silo.pgsty.com/blog/design/signed-header-coverage/).
- **Breaking policy semantics:** separate self-service `admin:ChangeMyPassword`
from `admin:CreateUser`. Built-in read-only policies follow the split. Preserve
both denies if the previous combined restriction must survive upgrades or
rollback. Saved policies are not rewritten. Deploy with the matching Console
and pkg; see [the migration guide](docs/iam/password-permissions.md).
### Object storage and replication
- Make `ListMultipartUploads` discover quorum-valid uploads from durable state
across pools, erasure sets and drives, then apply S3 prefix, delimiter,
marker, ordering and 1,000-entry pagination semantics globally (#198). New uploads
store their canonical bucket and key as reserved fields in the existing
quorum-written `xl.meta`; completion removes those upload-only fields. Native
markers remain usable after their upload is completed or canceled. Strict
listing returns a diagnostic 503 for legacy uploads or uncertain coverage;
the default remains the released exact-key/cache-based `legacy` behavior.
Opt into strict mode only through `MINIO_API_MULTIPART_LISTING=strict`, after
upgrading every writer, draining old uploads, checking the read-only admin
`multipart-preflight` report and validating scan capacity. The process-only
setting is not persisted into shared API configuration. Historical
`multipart_listing` keys are ignored and can be removed with a targeted
`mcli admin config reset ALIAS api multipart_listing` before rollback. Per-process
admission, directory-entry, worker and time budgets bound scan scheduling;
each page still scans durable state. See [issue #79](https://github.com/pgsty/silo/issues/79)
and its [design record](https://silo.pgsty.com/blog/design/list-multipart-uploads/).
Thanks to mr javad seydi (@mrjavadseydi) for the original implementation.
- Retain released read-quorum and best-effort multipart cancellation in default
legacy mode. Strict mode requires majority deletion acknowledgements and
permits retries below read quorum. When most drives were already empty,
failed deletion of an observed remnant now returns 503 instead of being
masked by empty-drive successes. Wrong-key or wrong-bucket cancellation
preserves the valid upload's cache entry; successful cancellation notifies
peers with the request context after the distributed lock is released.
The HTTP response for an absent
upload remains 204; this is not proof of physical cleanup. **Known boundary:**
delayed creation writes can still restore an upload after cancellation;
this change does not add a durable creation fence.
- Preserve object tags during multi-pool metadata reconciliation by reading the
resolved tag field together with its revision (#189). Previously, reconciliation
could replace existing tags with an empty value.
- Preserve the tag revision on SSE-KMS metadata replication (#193), and advance
tag revisions monotonically on local PUT/DELETE tagging (#196). Empty tags
participate in reconciliation as an ordered deletion, preventing older
events from restoring removed tags. SSE-C key rotation also retains the tag
revision. Malformed historical revisions can fail and retry; their missing
history is not reconstructed by the upgrade.
- Complete delete-marker version purges and preserve their identity and retry
state through MRF recovery (#196). Recovery accepts a 405 marker response only
when its version, bucket, object name and modification time match the task.
Purge audit status is normalized from `COMPLETE` to `COMPLETED`.
Thanks to Julien Laurenceau (@julienlau) for the investigation and proposed
fix in #184 that helped shape this follow-up.
- Restore only the six replication-specific metadata fields after ordinary
request metadata extraction (#194). This prevents transport-only `aws-chunked`
from being stored as Content-Encoding while preserving the signed-header
protections. Trusted Snowball entries no longer inherit the outer archive's
ordinary metadata. Thanks to Mikhail Khadarenka (@chodorenko) for the fix in #187.
**Existing data:** these repairs prevent new errors; they do not scan or rewrite
historical object metadata, recover lost tags or prove that old purge work has
converged. Follow the [read-only audit procedure](https://silo.pgsty.com/operations/replication/replica-metadata-audit/)
before planning any repair of stored state.
- Make exact-version delete-marker purges converge in replicated buckets
(`eb4f5e5b3`, `254b19ac0`, `358ab38fb`). A purge no longer creates the
marker on drives that lacked it; a retried purge of a missing version is
acknowledged only when a write-quorum majority of drives report it absent;
purge results merge removed and reliably absent replies; healing a marker
preserves its stored replication and purge metadata; and a queued marker
creation is re-checked against the source under the replication lock before
it is sent, so a purge that already reached the targets is not undone by a
stale task from another frontend, a GET/LIST heal, the scanner or MRF.
Purging a data version whose earlier purge is still pending reports it as a
data version. **Known limitations:** creations already in flight or replayed
from another site, and minority marker copies left by a crash after a
majority-acknowledged purge, are tracked in #217. See
[the replication reliability record](https://silo.pgsty.com/blog/design/replication-reliability/).
- Keep a null object version that has listing quorum when a newer minority of
drives sorts first (`8d06424b1`). The resolver recounts per header only when
the original selection lacks quorum, every non-empty drive stream holds
exactly one ordinary null version and all share the same erasure layout;
mixed histories keep their previous behavior. **Known limitation:** a
successful ListObjects can still omit readable keys during rolling restarts
with concurrent overwrites (#218). Do not run destination-deleting sync tools
against a listing taken during a rolling restart; list again once the
cluster is stable.
- Carry object tags through rebalance and decommission for ordinary and
multipart writes (`fced86303`). Both migration entry points restore the tags
and their revision fields when rewriting the object in the destination pool.
Tags dropped by earlier migrations are not recovered; audit tag-dependent
lifecycle and policy rules for pools migrated with an older build.
- Evaluate conditional multipart completion against the logical current object
across all pools while holding the existing object lock. A stale `If-Match`
can no longer replace newer data in another pool, and the current ETag is no
longer rejected because the upload resides next to an older copy. Conditions
are evaluated once; a current delete marker counts as an absent object.
**Availability change:** if any pool's metadata cannot be read, conditional
completion fails even when another pool can still serve GET/HEAD. This also
applies when the unreadable pool may not hold the object: absence cannot be
verified. Retry after the pool recovers. Unconditional completion and the
single-pool path retain their existing behavior.
- Evaluate ordinary multi-pool conditional PUT against the logical current
object across all pools, including draining pools, under the existing object
lock (#207). A stale destination copy no longer accepts a stale ETag or rejects
the current one; a current delete marker is treated as absence.
**Availability change:** if any pool's object metadata cannot be verified,
the condition fails even when GET can use another pool; read-quorum failures
return 503. Restore readability or heal before retrying. Unconditional PUT,
single-pool conditions and internal replication retain their existing behavior.
A public condition with a destination `versionId` compares the current object
while preserving the requested write version. This change does not retire
stale copies in other pools, undo historical accepted overwrites or provide
a new global clock-ordering guarantee. The multipart-completion repair in #190
neither introduced nor repaired this separate PUT defect.
- Reconcile ordinary single-object version DELETE across all pools, including
null versions, delete markers and unqualified directory-marker DELETE. This
applies the deletion to every resolved pool copy under existing quorum
rules. Pending outbound delete replication retains versions until the
existing replication worker completes their purge; a successful response
does not imply immediate physical removal from every drive. Unreadable
pools now consistently return 503 instead of depending on pool traversal
order; insufficient read quorum returns `SlowDownRead`. This extends the
existing failure surface. Retry after recovery.
Cleanup failures also return an error. Batch deletion already fans out across
pools; replication and scanner cleanup keep their existing contracts. See
[scope and limitations](docs/bucket/lifecycle/access-tiering-removal.md#version-deletion-scope).
- Remove the opt-in GET-frequency pool-tiering feature from PR #60, including
its tracker, mover, scanner hooks, configuration, XML actions and metrics.
Accept and ignore retired configuration/XML and preserve ordinary statistics
when reading v9 caches. See [migration notes](docs/bucket/lifecycle/access-tiering-removal.md).
The [decision record](https://silo.pgsty.com/compatibility/access-tiering-removal/) preserves
the feature's introduction, subsequent fixes, rollback scope and review history.
- Preserve the independent multi-pool write, metadata, healing and conditional
deletion fixes from PR #178, including shared remote-tier reference protection.
- Enforce `If-Match` on DELETE, preserve retention and independently ordered
Object Lock/tag updates, and correctly retransmit encrypted replicas.
- Preserve plaintext part sizes and raw SSE-C replicas; prevent SSE-C
compression, honor key-rotation checksums, and complete attributes pagination.
- Repair federated CopyObject checksums, destination timestamps, reserved
metadata, encrypted-object forwarding, legal hold and KMS context.
- Make resync counters, target selection, cancellation and worker lifetimes
reflect actual work, and report bounded MRF drops.
- Converge bucket metadata with deterministic source state, deletion tombstones,
creation time recovery and diagnostics. The mixed-version export gate requires
coordinated upgrades before tombstones are exported. See [the #77 record](https://silo.pgsty.com/blog/design/bucket-metadata-convergence/).
- Include per-bucket CORS in metadata export/import, close metadata publication
and logger races, and report effective bucket quotas in metrics.
### Console, dependencies and delivery
- Restore embedded Console login over loopback TLS, trusted-proxy handling and
all four WebSocket connection limits. Preserve Go TLS defaults across transports.
- Directly require `github.com/pgsty/silo-pkg/v3` v3.14.1; select released Console
v2.4.1 (`v0.0.0-20260916075814-1360e26d976d`) and mcli 20260916
(`v0.0.0-20260916070421-e952aa78f10a`) with explicit PGSTY replacements.
The embedded frontend identifies itself as Console v2.4.1.
- Pin upstream minio-go `v7.3.1-0.20260915093545-32e1f32cb176` to handle
CopyObject errors embedded in HTTP 200 responses. Update JWX to v3.3.0 for
JSON field-name escaping, strfmt to v0.27.2 for Go 1.27 hostname validation,
and LZ4 to v4.1.30 for frame-reader, partial-read and concurrency fixes.
Retain the earlier Go x/* and bounded AMQP frame updates, Go 1.27.1, and
go-systemd v22.6.0's NetBSD compatibility replacement.
- Refresh container base digests and build static curl 8.22.0 from verified
source for both Linux architectures. Pin the published mcli 20260916 archives
and hashes in the container and update the client installer default.
- Prepare Helm chart 7.0.3 with Server and client defaults for the September 16
batch. Publish the chart only after the corresponding Server image exists.
- Pin GoReleaser v2.18.1 and its action commit identically in snapshot and release
workflows. Dependency-only PRs now run the Test Release Pipeline too.
Validation of earlier source revisions does not establish acceptance of this
candidate. Final source, package, image and multi-process checks are tracked
separately in [#203](https://github.com/pgsty/silo/issues/203). No Server release
or production rollout is implied by this preparation target.
## RELEASE.2026-09-03T13-18-01Z
Published source: `9b11dc9469e650815b775cb47b039610644f5da4`.
[Complete release notes](https://silo.pgsty.com/blog/release/silo-20260903/) ·
[GitHub release](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-09-03T13-18-01Z)
This release ships Go 1.27.1, silo-pkg v3.13.2, upstream minio-go `0e78d3f18efe`,
mcli 20260903 and embedded Console source `464a59d73ada` (v2.3.0 version identity).
Installing the newer standalone mcli or Console does not replace components
inside this existing Server binary or image.
Earlier releases: [release archive](https://github.com/pgsty/silo/releases).
+1
View File
@@ -0,0 +1 @@
@AGENTS.md
-1
View File
@@ -1 +0,0 @@
charts.min.io
+13 -3
View File
@@ -1,7 +1,17 @@
# AGPLv3 Compliance
We have designed MinIO as an Open Source software for the Open Source software community. This requires applications to consider whether their usage of MinIO is in compliance with the GNU AGPLv3 [license](https://github.com/minio/minio/blob/master/LICENSE).
Silo is distributed under the [GNU Affero General Public License v3.0](LICENSE).
It incorporates source code from the MinIO project and preserves the original
copyright, license, and attribution notices in [`NOTICE`](NOTICE),
[`CREDITS`](CREDITS), and source-file headers.
MinIO cannot make the determination as to whether your application's usage of MinIO is in compliance with the AGPLv3 license requirements. You should instead rely on your own legal counsel or licensing specialists to audit and ensure your application is in compliance with the licenses of MinIO and all other open-source projects with which your application integrates or interacts. We understand that AGPLv3 licensing is complex and nuanced. It is for that reason we strongly encourage using experts in licensing to make any such determinations around compliance instead of relying on apocryphal or anecdotal advice.
You are responsible for determining how the AGPLv3 applies to your use,
modification, deployment, and distribution of Silo and its dependencies. The
Silo maintainers cannot provide legal advice or determine whether a particular
application or service satisfies the license. Consult qualified counsel when
the obligations are material to your deployment.
[MinIO Commercial Licensing](https://min.io/pricing) is the best option for applications that trigger AGPLv3 obligations (e.g. open sourcing your application). Applications using MinIO - or any other OSS-licensed code - without validating their usage do so at their own risk.
If you convey modified binaries or provide network access to a modified
version, review the complete AGPLv3 text and ensure that the corresponding
source and notices are made available as required. Dependency licenses and
separate notices continue to apply independently.
+115 -30
View File
@@ -1,55 +1,65 @@
# MinIO Contribution Guide [![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io) [![Docker Pulls](https://img.shields.io/docker/pulls/minio/minio.svg?maxAge=604800)](https://hub.docker.com/r/minio/minio/)
# Contributing to Silo
``MinIO`` community welcomes your contribution. To make the process as seamless as possible, we recommend you read this contribution guide.
Silo welcomes focused contributions that improve security, reliability,
compatibility, packaging, tests, or maintainability. This repository preserves
MinIO-compatible interfaces and storage formats, so changes must identify and
test any compatibility impact.
## Development Workflow
Start by forking the MinIO GitHub repository, make changes in a branch and then send a pull request. We encourage pull requests to discuss code changes. Here are the steps in details:
Fork the current Silo source repository, create a topic branch, and submit a
pull request. Discuss broad or compatibility-sensitive changes in an issue
before implementation.
### Setup your MinIO GitHub Repository
Fork [MinIO upstream](https://github.com/minio/minio/fork) source repository to your own personal repository. Copy the URL of your MinIO fork (you will need it for the `git clone` command below).
### Set up a checkout
```sh
git clone https://github.com/minio/minio
cd minio
go install -v
ls $(go env GOPATH)/bin/minio
git clone https://github.com/pgsty/silo
cd silo
go build -o silo .
./silo --version
```
### Set up git remote as ``upstream``
### Keep the lineage remote separate
```sh
$ cd minio
$ git remote add upstream https://github.com/minio/minio
$ git fetch upstream
$ git merge upstream/master
...
git remote add lineage https://github.com/minio/minio
git fetch lineage
```
Do not merge an upstream branch into a pull request unless the maintainers have
agreed on the scope. Silo intentionally carries a small downstream delta.
### Create your feature branch
Before making code changes, make sure you create a separate branch for these changes
Create a separate branch before making code changes:
```
git checkout -b my-new-feature
```
### Test MinIO server changes
### Test Silo server changes
After your code changes, make sure
Before opening a pull request:
- To add test cases for the new code. If you have questions about how to do it, please ask on our [Slack](https://slack.min.io) channel.
- To run `make verifiers`
- To squash your commits into a single commit. `git rebase -i`. It's okay to force update your pull request.
- To run `make test` and `make build` completes.
- Add or update tests for changed behavior.
- Run `make verifiers`.
- If `make rebrand-guard` reports a changed compatibility set, review the
listed identifiers; when the change is intended, refresh the baseline with
`go run ./buildscripts/rebrand-guard --write` and commit
`buildscripts/rebrand-guard/compat-baseline.json`.
- Run the smallest relevant package tests, then `make test` when practical.
- Run `make build` and confirm the generated executable is `silo`.
- Explain any preserved `MINIO_*`, `minio_*`, `x-minio-*`, `/minio/*`,
`.minio.sys`, ARN, module/import-path, or serialized compatibility name.
### Commit changes
After verification, commit your changes. This is a [great post](https://chris.beams.io/posts/git-commit/) on how to write useful commit messages
After verification, commit your changes with a concise message and a DCO
sign-off (see [Licensing of Contributions](#licensing-of-contributions)):
```
git commit -am 'Add some feature'
git commit -s -am 'Fix object replication retry handling'
```
### Push to the branch
@@ -62,13 +72,87 @@ git push origin my-new-feature
### Create a Pull Request
Pull requests can be created via GitHub. Refer to [this document](https://help.github.com/articles/creating-a-pull-request/) for detailed steps on how to create a pull request. After a Pull Request gets peer reviewed and approved, it will be merged.
Pull requests should include motivation, reproduction steps where applicable,
test evidence, compatibility notes, and documentation impact. Public product
documentation is owned by the separate
[`pgsty/silo.pgsty.com`](https://github.com/pgsty/silo.pgsty.com) repository.
## Contributor recognition
Every human issue or pull-request author is recognized in [CONTRIBUTORS.md](CONTRIBUTORS.md),
including open issues, draft PRs, and PRs closed without merging. Merged fixes,
adopted proposals, and reports that lead to fixes receive greater prominence;
first participation guides the remaining order. Work incorporated through a
later PR retains credit without changing the original PR's recorded status.
Security disclosures are credited with the reporter's agreement. DCO sign-off
applies to code commits, not to opening an issue.
## Licensing of Contributions
Code contributions to PGSTY SILO (`pgsty/silo`) are accepted under the
[GNU AGPL v3.0 or later](LICENSE), the same license as the server. Submit issues
and pull requests to this repository's maintainers. No separate Apache-2.0
license grant to SILO or upstream MinIO maintainers is required.
* **No CLA.** We do not ask you to sign a Contributor License Agreement and we
do not take your copyright. Contributions are accepted inbound=outbound: you
keep the copyright to your changes and license them under the same
AGPL-3.0-or-later as the project itself. The maintainers receive no rights
beyond the project license.
* **DCO sign-off required.** Every commit must carry a
`Signed-off-by: Your Name <you@example.com>` trailer certifying the
[Developer Certificate of Origin 1.1](https://developercertificate.org/) —
your statement that you have the right to submit the code under the project
license. Sign each commit with:
```
git commit -s
```
Forgot some? Repair your branch with `git rebase --signoff` and force-push.
CI rejects pull requests containing unsigned commits; the sign-off email
must match the commit author email. (Lowercase `-s` is the plain-text DCO
sign-off; cryptographic `-S`/GPG signing is welcome but independent.)
* **Provenance.** Only submit code you are entitled to submit. This matters
more here than in most projects: Silo carries a downstream delta over an
upstream code base, and cherry-picks from the lineage remote or other forks
are routine. When relaying a patch written by someone else, preserve original
authorship (`git cherry-pick -x`, keep the author field and any existing
`Signed-off-by` trailers) and add your own sign-off as the person passing it
along. Never import code from a proprietary distribution.
* **File headers.** Preserve existing copyright and license notices in inherited
and third-party files. New original files name their actual copyright holders
and use AGPL-3.0-or-later. Use a header such as the following, then append the
standard AGPL boilerplate:
```
// Copyright (c) 2026 Your Name
```
* **Separately licensed material.** Documentation contributions in `docs/`
follow its existing [CC BY 4.0 license](docs/LICENSE). Third-party components
and earlier Apache-2.0 contributions retain their original licenses and
attribution; this policy does not relicense earlier work.
* **Squash merges** must keep the `Signed-off-by:` trailers in the resulting
commit message.
* **Authorship and tooling.** The human contributor is the author of the commit
and the sole signatory of its DCO sign-off. Attribution trailers for
assistive tooling (for example `Co-Authored-By:` naming an AI assistant) are
informational only: they record which tools were used, and do not create
authorship, co-authorship, or any copyright claim. Whoever signs off remains
responsible for the content of the commit, whatever produced it.
## FAQs
### How does ``MinIO`` manage dependencies?
### How does Silo manage dependencies?
``MinIO`` uses `go mod` to manage its dependencies.
Silo uses Go modules. Preserve the compatibility module and import paths in
`go.mod`; downstream forks are selected with explicit `replace` directives.
- Run `go get foo/bar` in the source folder to add the dependency to `go.mod` file.
@@ -77,6 +161,7 @@ To remove a dependency
- Edit your code and remove the import reference.
- Run `go mod tidy` in the source folder to remove dependency from `go.mod` file.
### What are the coding guidelines for MinIO?
### What are the coding guidelines?
``MinIO`` is fully conformant with Golang style. Refer: [Effective Go](https://github.com/golang/go/wiki/CodeReviewComments) article from Golang project. If you observe offending code, please feel free to send a pull request or ping us on [Slack](https://slack.min.io).
Follow the existing Go style, run `gofmt` on changed Go files, and keep changes
compact. See the Go project's [code review comments](https://go.dev/wiki/CodeReviewComments).
+180
View File
@@ -0,0 +1,180 @@
# Contributors
<!-- Generated by silo.pgsty.com/bin/contributors.py. -->
As of **2026-09-16**, **50 community contributors** are credited in SILO and related projects. The roll includes maintainers and every human issue / PR author, plus the previously acknowledged security disclosure.
Opening an issue or PR counts, whether open, closed, draft, or unmerged. Each account appears once. Merged PR authors come first, followed by other PR authors and reporters. Within each group, significant fixes, adopted proposals, and reports that led to fixes take priority, followed by first participation. Gold rings highlight significant contributions.
When a proposal was incorporated through a later repair, its adoption is credited separately from the original PR status. A closed issue does not necessarily mean a fix, and merged source does not imply a published release. Automated accounts are excluded. Upstream authorship remains in Git history and copyright notices.
<p align="center">
<a href="https://github.com/Vonng"><img src="https://silo.pgsty.com/images/contributors/Vonng.svg" width="60" height="60" alt="@Vonng" title="@Vonng — Maintains SILO, Console, mcli, shared packages, releases, and documentation"></a>
<a href="https://github.com/ZouhairCharef"><img src="https://silo.pgsty.com/images/contributors/ZouhairCharef.svg" width="60" height="60" alt="@ZouhairCharef" title="@ZouhairCharef — Patched CVE-2026-34986 in go-jose"></a>
<a href="https://github.com/mfredenhagen"><img src="https://silo.pgsty.com/images/contributors/mfredenhagen.svg" width="60" height="60" alt="@mfredenhagen" title="@mfredenhagen — Patched CVE-2026-39883 in OpenTelemetry"></a>
<a href="https://github.com/pinginfo"><img src="https://silo.pgsty.com/images/contributors/pinginfo.svg" width="60" height="60" alt="@pinginfo" title="@pinginfo — Repaired bucket notification streaming"></a>
<a href="https://github.com/ycjlin"><img src="https://silo.pgsty.com/images/contributors/ycjlin.svg" width="60" height="60" alt="@ycjlin" title="@ycjlin — Fixed missing-bucket ListObjects semantics"></a>
<a href="https://github.com/waterkip"><img src="https://silo.pgsty.com/images/contributors/waterkip.svg" width="60" height="60" alt="@waterkip" title="@waterkip — Repointed documentation links to the SILO portal"></a>
<a href="https://github.com/Dansyuqri"><img src="https://silo.pgsty.com/images/contributors/Dansyuqri.svg" width="60" height="60" alt="@Dansyuqri" title="@Dansyuqri — Added ChecksumType to multipart completion responses"></a>
<a href="https://github.com/mrjavadseydi"><img src="https://silo.pgsty.com/images/contributors/mrjavadseydi.svg" width="60" height="60" alt="@mrjavadseydi" title="@mrjavadseydi — Fixed bucket quota metrics; contributed access-frequency ILM and S3-compatible multipart listing proposals"></a>
<a href="https://github.com/h5vx"><img src="https://silo.pgsty.com/images/contributors/h5vx.svg" width="60" height="60" alt="@h5vx" title="@h5vx — Implemented per-bucket CORS configuration and enforcement"></a>
<a href="https://github.com/nikitapogromsky"><img src="https://silo.pgsty.com/images/contributors/nikitapogromsky.svg" width="60" height="60" alt="@nikitapogromsky" title="@nikitapogromsky — Fixed duplicate logger targets that broke Metrics V3 collection"></a>
<a href="https://github.com/Aeirx"><img src="https://silo.pgsty.com/images/contributors/Aeirx.svg" width="60" height="60" alt="@Aeirx" title="@Aeirx — Preserved Object Lock legal-hold headers in federated CopyObject"></a>
<a href="https://github.com/magicxor"><img src="https://silo.pgsty.com/images/contributors/magicxor.svg" width="60" height="60" alt="@magicxor" title="@magicxor — Reported and proposed conditional DELETE support for If-Match"></a>
<a href="https://github.com/davinkevin"><img src="https://silo.pgsty.com/images/contributors/davinkevin.svg" width="60" height="60" alt="@davinkevin" title="@davinkevin — Proposed the distroless container image and dependency automation"></a>
<a href="https://github.com/metaneutrons"><img src="https://silo.pgsty.com/images/contributors/metaneutrons.svg" width="60" height="60" alt="@metaneutrons" title="@metaneutrons — Reported and proposed explicit-version delete authorization"></a>
<a href="https://github.com/mikemikimike"><img src="https://silo.pgsty.com/images/contributors/mikemikimike.svg" width="60" height="60" alt="@mikemikimike" title="@mikemikimike — Contributed the replicated SSE-C plaintext part-size fix"></a>
<a href="https://github.com/julienlau"><img src="https://silo.pgsty.com/images/contributors/julienlau.svg" width="60" height="60" alt="@julienlau" title="@julienlau — Reported MRF queue and delete-marker retry defects; contributed repair proposals, tests, and documentation"></a>
<a href="https://github.com/chodorenko"><img src="https://silo.pgsty.com/images/contributors/chodorenko.svg" width="60" height="60" alt="@chodorenko" title="@chodorenko — Reported and contributed the fix for aws-chunked leaking into replica metadata"></a>
<a href="https://github.com/sulin37392"><img src="https://silo.pgsty.com/images/contributors/sulin37392.svg" width="48" height="48" alt="@sulin37392" title="@sulin37392 — Proposed dependency updates"></a>
<a href="https://github.com/lem21h"><img src="https://silo.pgsty.com/images/contributors/lem21h.svg" width="48" height="48" alt="@lem21h" title="@lem21h — Proposed robustness and goroutine improvements"></a>
<a href="https://github.com/vampywiz17"><img src="https://silo.pgsty.com/images/contributors/vampywiz17.svg" width="60" height="60" alt="@vampywiz17" title="@vampywiz17 — Reported LDAP TLS and Console login regressions"></a>
<a href="https://github.com/cbornet"><img src="https://silo.pgsty.com/images/contributors/cbornet.svg" width="60" height="60" alt="@cbornet" title="@cbornet — Reported multipart and streaming checksum defects and missing-bucket semantics"></a>
<a href="https://github.com/orenyomtov"><img src="https://silo.pgsty.com/images/contributors/orenyomtov.svg" width="60" height="60" alt="@orenyomtov" title="@orenyomtov — Reported the unsigned-header CopyObject cross-object read (SN-2026-011)"></a>
<a href="https://github.com/jiri-pejchal"><img src="https://silo.pgsty.com/images/contributors/jiri-pejchal.svg" width="60" height="60" alt="@jiri-pejchal" title="@jiri-pejchal — Reported the RabbitMQ client vulnerability and the Console share proxy&#x27;s exposure of internal metrics"></a>
<a href="https://github.com/AEGEGE"><img src="https://silo.pgsty.com/images/contributors/AEGEGE.svg" width="60" height="60" alt="@AEGEGE" title="@AEGEGE — Reported the slow-HTTP denial-of-service vulnerability"></a>
<a href="https://github.com/mosesdd"><img src="https://silo.pgsty.com/images/contributors/mosesdd.svg" width="48" height="48" alt="@mosesdd" title="@mosesdd — Requested a maintained Helm chart"></a>
<a href="https://github.com/Xavier-777"><img src="https://silo.pgsty.com/images/contributors/Xavier-777.svg" width="48" height="48" alt="@Xavier-777" title="@Xavier-777 — Reported Console lifecycle management and file preview gaps"></a>
<a href="https://github.com/jiadzh"><img src="https://silo.pgsty.com/images/contributors/jiadzh.svg" width="48" height="48" alt="@jiadzh" title="@jiadzh — Requested Windows build guidance"></a>
<a href="https://github.com/AntonOfTheWoods"><img src="https://silo.pgsty.com/images/contributors/AntonOfTheWoods.svg" width="48" height="48" alt="@AntonOfTheWoods" title="@AntonOfTheWoods — Asked for clarity on Helm chart and operator options"></a>
<a href="https://github.com/TLINDEN"><img src="https://silo.pgsty.com/images/contributors/TLINDEN.svg" width="48" height="48" alt="@TLINDEN" title="@TLINDEN — Reported the missing client in release tarballs"></a>
<a href="https://github.com/zylpsrs"><img src="https://silo.pgsty.com/images/contributors/zylpsrs.svg" width="48" height="48" alt="@zylpsrs" title="@zylpsrs — Reported missing Console tiering and site replication"></a>
<a href="https://github.com/nsanitate"><img src="https://silo.pgsty.com/images/contributors/nsanitate.svg" width="48" height="48" alt="@nsanitate" title="@nsanitate — Proposed CNCF Sandbox governance"></a>
<a href="https://github.com/makinikm"><img src="https://silo.pgsty.com/images/contributors/makinikm.svg" width="48" height="48" alt="@makinikm" title="@makinikm — Reported the missing client in the container image"></a>
<a href="https://github.com/spaceg00se-r"><img src="https://silo.pgsty.com/images/contributors/spaceg00se-r.svg" width="48" height="48" alt="@spaceg00se-r" title="@spaceg00se-r — Requested cpuv1 support and reported a workflow token failure"></a>
<a href="https://github.com/heroes1412"><img src="https://silo.pgsty.com/images/contributors/heroes1412.svg" width="48" height="48" alt="@heroes1412" title="@heroes1412 — Reported the unusable profiling option"></a>
<a href="https://github.com/chalukyaj"><img src="https://silo.pgsty.com/images/contributors/chalukyaj.svg" width="48" height="48" alt="@chalukyaj" title="@chalukyaj — Proposed making the SILO Operator easier to discover"></a>
<a href="https://github.com/jvasile"><img src="https://silo.pgsty.com/images/contributors/jvasile.svg" width="48" height="48" alt="@jvasile" title="@jvasile — Reported missing user, group, and defaults in Debian packages"></a>
<a href="https://github.com/Kesavaambati"><img src="https://silo.pgsty.com/images/contributors/Kesavaambati.svg" width="48" height="48" alt="@Kesavaambati" title="@Kesavaambati — Asked about community support and image maintenance"></a>
<a href="https://github.com/redfoxfox"><img src="https://silo.pgsty.com/images/contributors/redfoxfox.svg" width="48" height="48" alt="@redfoxfox" title="@redfoxfox — Reported Chinese documentation availability"></a>
<a href="https://github.com/kuldeep-link11"><img src="https://silo.pgsty.com/images/contributors/kuldeep-link11.svg" width="48" height="48" alt="@kuldeep-link11" title="@kuldeep-link11 — Reported NATS JWT credentials and target reload issues"></a>
<a href="https://github.com/meesudzu"><img src="https://silo.pgsty.com/images/contributors/meesudzu.svg" width="48" height="48" alt="@meesudzu" title="@meesudzu — Requested the migration guide from upstream MinIO"></a>
<a href="https://github.com/pmezhuev"><img src="https://silo.pgsty.com/images/contributors/pmezhuev.svg" width="48" height="48" alt="@pmezhuev" title="@pmezhuev — Reported missing RPM package signatures"></a>
<a href="https://github.com/kh0mka"><img src="https://silo.pgsty.com/images/contributors/kh0mka.svg" width="48" height="48" alt="@kh0mka" title="@kh0mka — Reported inter-node I/O timeouts and OIDC configuration-fetch failures"></a>
<a href="https://github.com/bagutzu"><img src="https://silo.pgsty.com/images/contributors/bagutzu.svg" width="48" height="48" alt="@bagutzu" title="@bagutzu — Requested KES-compatible external KMS and OpenBao support"></a>
<a href="https://github.com/liuhaodongliu990-cmyk"><img src="https://silo.pgsty.com/images/contributors/liuhaodongliu990-cmyk.svg" width="48" height="48" alt="@liuhaodongliu990-cmyk" title="@liuhaodongliu990-cmyk — Reported indeterminate progress for prefix downloads"></a>
<a href="https://github.com/DestroyLee"><img src="https://silo.pgsty.com/images/contributors/DestroyLee.svg" width="48" height="48" alt="@DestroyLee" title="@DestroyLee — Reported the missing documentation navigation"></a>
<a href="https://github.com/sargarass"><img src="https://silo.pgsty.com/images/contributors/sargarass.svg" width="48" height="48" alt="@sargarass" title="@sargarass — Reported ListMultipartUploads prefix and pagination semantics"></a>
<a href="https://github.com/mumu-lab"><img src="https://silo.pgsty.com/images/contributors/mumu-lab.svg" width="48" height="48" alt="@mumu-lab" title="@mumu-lab — Reported bucket quota metrics reading a deprecated field"></a>
<a href="https://github.com/haiming236"><img src="https://silo.pgsty.com/images/contributors/haiming236.svg" width="48" height="48" alt="@haiming236" title="@haiming236 — Proposed a built-in image processing pipeline"></a>
<a href="https://github.com/tiredenzo"><img src="https://silo.pgsty.com/images/contributors/tiredenzo.svg" width="48" height="48" alt="@tiredenzo" title="@tiredenzo — Reported inconsistent documentation for two-drive EC:1 support"></a>
<a href="https://github.com/aschyolkin"><img src="https://silo.pgsty.com/images/contributors/aschyolkin.svg" width="48" height="48" alt="@aschyolkin" title="@aschyolkin — Reported a data race in CPU metrics collection"></a>
</p>
## Merged pull requests
| Contributor | Contribution | Record |
| :-- | :-- | :-- |
| [@Vonng](https://github.com/Vonng) | Maintains SILO, Console, mcli, shared packages, releases, and documentation | [pgsty/silo: Merged PRs (77)](https://github.com/pgsty/silo/issues?q=author%3AVonng+is%3Apr+is%3Amerged)<br>PRs closed without merging: [pgsty/silo#155](https://github.com/pgsty/silo/pull/155), [pgsty/silo#195](https://github.com/pgsty/silo/pull/195)<br>[pgsty/silo: Closed issues (60)](https://github.com/pgsty/silo/issues?q=author%3AVonng+is%3Aissue+is%3Aclosed)<br>Open issues: [pgsty/silo#199](https://github.com/pgsty/silo/issues/199), [pgsty/silo#200](https://github.com/pgsty/silo/issues/200), [pgsty/silo#201](https://github.com/pgsty/silo/issues/201), [pgsty/silo#202](https://github.com/pgsty/silo/issues/202), [pgsty/silo#203](https://github.com/pgsty/silo/issues/203)<br>[pgsty/silo-console: Merged PRs (22)](https://github.com/pgsty/silo-console/issues?q=author%3AVonng+is%3Apr+is%3Amerged)<br>[pgsty/silo-console: Closed issues (32)](https://github.com/pgsty/silo-console/issues?q=author%3AVonng+is%3Aissue+is%3Aclosed)<br>Open issues: [pgsty/silo-console#32](https://github.com/pgsty/silo-console/issues/32), [pgsty/silo-console#34](https://github.com/pgsty/silo-console/issues/34)<br>[pgsty/mc: Merged PRs (24)](https://github.com/pgsty/mc/issues?q=author%3AVonng+is%3Apr+is%3Amerged)<br>[pgsty/mc: Closed issues (18)](https://github.com/pgsty/mc/issues?q=author%3AVonng+is%3Aissue+is%3Aclosed)<br>Merged PRs: [pgsty/silo-pkg#1](https://github.com/pgsty/silo-pkg/pull/1), [pgsty/silo-pkg#2](https://github.com/pgsty/silo-pkg/pull/2), [pgsty/silo-pkg#3](https://github.com/pgsty/silo-pkg/pull/3), [pgsty/silo-pkg#4](https://github.com/pgsty/silo-pkg/pull/4), [pgsty/silo-pkg#5](https://github.com/pgsty/silo-pkg/pull/5), [pgsty/silo-pkg#6](https://github.com/pgsty/silo-pkg/pull/6), [pgsty/silo-pkg#7](https://github.com/pgsty/silo-pkg/pull/7), [pgsty/silo-pkg#8](https://github.com/pgsty/silo-pkg/pull/8), [pgsty/silo-pkg#9](https://github.com/pgsty/silo-pkg/pull/9)<br>[pgsty/silo.pgsty.com: Merged PRs (24)](https://github.com/pgsty/silo.pgsty.com/issues?q=author%3AVonng+is%3Apr+is%3Amerged)<br>Open PRs: [pgsty/silo.pgsty.com#25](https://github.com/pgsty/silo.pgsty.com/pull/25) |
| [@ZouhairCharef](https://github.com/ZouhairCharef) | Patched CVE-2026-34986 in go-jose | Merged PRs: [pgsty/silo#18](https://github.com/pgsty/silo/pull/18) |
| [@mfredenhagen](https://github.com/mfredenhagen) | Patched CVE-2026-39883 in OpenTelemetry | Merged PRs: [pgsty/silo#19](https://github.com/pgsty/silo/pull/19) |
| [@pinginfo](https://github.com/pinginfo) | Repaired bucket notification streaming | Merged PRs: [pgsty/silo#34](https://github.com/pgsty/silo/pull/34) |
| [@ycjlin](https://github.com/ycjlin) | Fixed missing-bucket ListObjects semantics | Merged PRs: [pgsty/silo#37](https://github.com/pgsty/silo/pull/37) |
| [@waterkip](https://github.com/waterkip) | Repointed documentation links to the SILO portal | Merged PRs: [pgsty/silo#41](https://github.com/pgsty/silo/pull/41) |
| [@Dansyuqri](https://github.com/Dansyuqri) | Added ChecksumType to multipart completion responses | Merged PRs: [pgsty/silo#57](https://github.com/pgsty/silo/pull/57) |
| [@mrjavadseydi](https://github.com/mrjavadseydi) | Fixed bucket quota metrics; contributed access-frequency ILM and S3-compatible multipart listing proposals | Merged PRs: [pgsty/silo#60](https://github.com/pgsty/silo/pull/60), [pgsty/silo#132](https://github.com/pgsty/silo/pull/132), [pgsty/silo#198](https://github.com/pgsty/silo/pull/198)<br>The access-frequency feature in [#60](https://github.com/pgsty/silo/pull/60) was merged and later removed. The multipart-listing contribution in [#198](https://github.com/pgsty/silo/pull/198) was merged after follow-up durability and cancellation repairs; it remains unreleased. |
| [@h5vx](https://github.com/h5vx) | Implemented per-bucket CORS configuration and enforcement | Merged PRs: [pgsty/silo#71](https://github.com/pgsty/silo/pull/71) |
| [@nikitapogromsky](https://github.com/nikitapogromsky) | Fixed duplicate logger targets that broke Metrics V3 collection | Merged PRs: [pgsty/silo#151](https://github.com/pgsty/silo/pull/151)<br>Closed issues: [pgsty/silo#150](https://github.com/pgsty/silo/issues/150) |
| [@Aeirx](https://github.com/Aeirx) | Preserved Object Lock legal-hold headers in federated CopyObject | Merged PRs: [pgsty/silo#172](https://github.com/pgsty/silo/pull/172) |
## Other pull-request authors
| Contributor | Contribution | Record |
| :-- | :-- | :-- |
| [@magicxor](https://github.com/magicxor) | Reported and proposed conditional DELETE support for If-Match | PRs closed without merging: [pgsty/silo#12](https://github.com/pgsty/silo/pull/12)<br>Closed issues: [pgsty/silo#10](https://github.com/pgsty/silo/issues/10)<br>The original PR closed after conditional DELETE support landed in [#145](https://github.com/pgsty/silo/pull/145). |
| [@davinkevin](https://github.com/davinkevin) | Proposed the distroless container image and dependency automation | PRs closed without merging: [pgsty/silo#21](https://github.com/pgsty/silo/pull/21)<br>Open issues: [pgsty/silo#20](https://github.com/pgsty/silo/issues/20)<br>The distroless proposal was superseded by the image shipped in [RELEASE.2026-08-06](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-08-06T00-00-00Z). |
| [@metaneutrons](https://github.com/metaneutrons) | Reported and proposed explicit-version delete authorization | PRs closed without merging: [pgsty/silo#59](https://github.com/pgsty/silo/pull/59)<br>Closed issues: [pgsty/silo#58](https://github.com/pgsty/silo/issues/58)<br>The original PR was superseded; its authorization action split informed [#104](https://github.com/pgsty/silo/pull/104). |
| [@mikemikimike](https://github.com/mikemikimike) | Contributed the replicated SSE-C plaintext part-size fix | PRs closed without merging: [pgsty/silo#125](https://github.com/pgsty/silo/pull/125)<br>The original PR was superseded; its plaintext-size fix was incorporated through [#124](https://github.com/pgsty/silo/pull/124), with attribute handling in [#128](https://github.com/pgsty/silo/pull/128). |
| [@julienlau](https://github.com/julienlau) | Reported MRF queue and delete-marker retry defects; contributed repair proposals, tests, and documentation | PRs closed without merging: [pgsty/silo#184](https://github.com/pgsty/silo/pull/184)<br>Closed issues: [pgsty/silo#152](https://github.com/pgsty/silo/issues/152), [pgsty/silo#153](https://github.com/pgsty/silo/issues/153)<br>The original PR was not merged directly. Its purge-state and recovery analysis, proposed fix, tests, and documentation helped shape [#196](https://github.com/pgsty/silo/pull/196); see the [maintainer acknowledgement](https://github.com/pgsty/silo/pull/184#issuecomment-5689295222). |
| [@chodorenko](https://github.com/chodorenko) | Reported and contributed the fix for aws-chunked leaking into replica metadata | PRs closed without merging: [pgsty/silo#187](https://github.com/pgsty/silo/pull/187)<br>Closed issues: [pgsty/silo#185](https://github.com/pgsty/silo/issues/185)<br>The implementation was incorporated in [#194](https://github.com/pgsty/silo/pull/194), with [co-author credit](https://github.com/pgsty/silo/commit/4fcdf37ce656152b32ad0f615d47f5e3f9748c3a); the original PR was closed as superseded. |
| [@sulin37392](https://github.com/sulin37392) | Proposed dependency updates | PRs closed without merging: [pgsty/silo#8](https://github.com/pgsty/silo/pull/8) |
| [@lem21h](https://github.com/lem21h) | Proposed robustness and goroutine improvements | PRs closed without merging: [pgsty/silo#36](https://github.com/pgsty/silo/pull/36) |
## Issue reports
| Contributor | Contribution | Record |
| :-- | :-- | :-- |
| [@vampywiz17](https://github.com/vampywiz17) | Reported LDAP TLS and Console login regressions | Closed issues: [pgsty/silo#15](https://github.com/pgsty/silo/issues/15), [pgsty/silo#108](https://github.com/pgsty/silo/issues/108) |
| [@cbornet](https://github.com/cbornet) | Reported multipart and streaming checksum defects and missing-bucket semantics | Closed issues: [pgsty/silo#31](https://github.com/pgsty/silo/issues/31), [pgsty/silo#32](https://github.com/pgsty/silo/issues/32), [pgsty/silo#107](https://github.com/pgsty/silo/issues/107) |
| [@orenyomtov](https://github.com/orenyomtov) | Reported the unsigned-header CopyObject cross-object read (SN-2026-011) | Security disclosure credited in [SN-2026-011](https://silo.pgsty.com/about/security-advisories/#sn-2026-011), with the source fix in [#173](https://github.com/pgsty/silo/pull/173). This credit is retained separately from public issue/PR authorship. |
| [@jiri-pejchal](https://github.com/jiri-pejchal) | Reported the RabbitMQ client vulnerability and the Console share proxy's exposure of internal metrics | Closed issues: [pgsty/silo#176](https://github.com/pgsty/silo/issues/176)<br>Closed issues: [pgsty/silo-console#52](https://github.com/pgsty/silo-console/issues/52) |
| [@AEGEGE](https://github.com/AEGEGE) | Reported the slow-HTTP denial-of-service vulnerability | Closed issues: [pgsty/silo#183](https://github.com/pgsty/silo/issues/183)<br>The reproduced slow-header defect was fixed on main through [#196](https://github.com/pgsty/silo/pull/196). |
| [@mosesdd](https://github.com/mosesdd) | Requested a maintained Helm chart | Closed issues: [pgsty/silo#1](https://github.com/pgsty/silo/issues/1) |
| [@Xavier-777](https://github.com/Xavier-777) | Reported Console lifecycle management and file preview gaps | Closed issues: [pgsty/silo#2](https://github.com/pgsty/silo/issues/2), [pgsty/silo#17](https://github.com/pgsty/silo/issues/17) |
| [@jiadzh](https://github.com/jiadzh) | Requested Windows build guidance | Closed issues: [pgsty/silo#3](https://github.com/pgsty/silo/issues/3) |
| [@AntonOfTheWoods](https://github.com/AntonOfTheWoods) | Asked for clarity on Helm chart and operator options | Closed issues: [pgsty/silo#5](https://github.com/pgsty/silo/issues/5) |
| [@TLINDEN](https://github.com/TLINDEN) | Reported the missing client in release tarballs | Closed issues: [pgsty/silo#4](https://github.com/pgsty/silo/issues/4) |
| [@zylpsrs](https://github.com/zylpsrs) | Reported missing Console tiering and site replication | Closed issues: [pgsty/silo#6](https://github.com/pgsty/silo/issues/6) |
| [@nsanitate](https://github.com/nsanitate) | Proposed CNCF Sandbox governance | Closed issues: [pgsty/silo#7](https://github.com/pgsty/silo/issues/7) |
| [@makinikm](https://github.com/makinikm) | Reported the missing client in the container image | Closed issues: [pgsty/silo#9](https://github.com/pgsty/silo/issues/9) |
| [@spaceg00se-r](https://github.com/spaceg00se-r) | Requested cpuv1 support and reported a workflow token failure | Closed issues: [pgsty/silo#11](https://github.com/pgsty/silo/issues/11), [pgsty/silo#14](https://github.com/pgsty/silo/issues/14) |
| [@heroes1412](https://github.com/heroes1412) | Reported the unusable profiling option | Closed issues: [pgsty/silo#13](https://github.com/pgsty/silo/issues/13) |
| [@chalukyaj](https://github.com/chalukyaj) | Proposed making the SILO Operator easier to discover | Open issues: [pgsty/silo#30](https://github.com/pgsty/silo/issues/30) |
| [@jvasile](https://github.com/jvasile) | Reported missing user, group, and defaults in Debian packages | Closed issues: [pgsty/silo#33](https://github.com/pgsty/silo/issues/33) |
| [@Kesavaambati](https://github.com/Kesavaambati) | Asked about community support and image maintenance | Closed issues: [pgsty/silo#35](https://github.com/pgsty/silo/issues/35) |
| [@redfoxfox](https://github.com/redfoxfox) | Reported Chinese documentation availability | Closed issues: [pgsty/silo#38](https://github.com/pgsty/silo/issues/38) |
| [@kuldeep-link11](https://github.com/kuldeep-link11) | Reported NATS JWT credentials and target reload issues | Closed issues: [pgsty/silo#39](https://github.com/pgsty/silo/issues/39)<br>Open issues: [pgsty/silo#40](https://github.com/pgsty/silo/issues/40) |
| [@meesudzu](https://github.com/meesudzu) | Requested the migration guide from upstream MinIO | Closed issues: [pgsty/silo#42](https://github.com/pgsty/silo/issues/42) |
| [@pmezhuev](https://github.com/pmezhuev) | Reported missing RPM package signatures | Closed issues: [pgsty/silo#43](https://github.com/pgsty/silo/issues/43) |
| [@kh0mka](https://github.com/kh0mka) | Reported inter-node I/O timeouts and OIDC configuration-fetch failures | Closed issues: [pgsty/silo#51](https://github.com/pgsty/silo/issues/51), [pgsty/silo#154](https://github.com/pgsty/silo/issues/154) |
| [@bagutzu](https://github.com/bagutzu) | Requested KES-compatible external KMS and OpenBao support | Open issues: [pgsty/silo#61](https://github.com/pgsty/silo/issues/61) |
| [@liuhaodongliu990-cmyk](https://github.com/liuhaodongliu990-cmyk) | Reported indeterminate progress for prefix downloads | Closed issues: [pgsty/silo#62](https://github.com/pgsty/silo/issues/62) |
| [@DestroyLee](https://github.com/DestroyLee) | Reported the missing documentation navigation | Closed issues: [pgsty/silo.pgsty.com#4](https://github.com/pgsty/silo.pgsty.com/issues/4) |
| [@sargarass](https://github.com/sargarass) | Reported ListMultipartUploads prefix and pagination semantics | Open issues: [pgsty/silo#79](https://github.com/pgsty/silo/issues/79) |
| [@mumu-lab](https://github.com/mumu-lab) | Reported bucket quota metrics reading a deprecated field | Closed issues: [pgsty/silo#106](https://github.com/pgsty/silo/issues/106) |
| [@haiming236](https://github.com/haiming236) | Proposed a built-in image processing pipeline | Open issues: [pgsty/silo#160](https://github.com/pgsty/silo/issues/160) |
| [@tiredenzo](https://github.com/tiredenzo) | Reported inconsistent documentation for two-drive EC:1 support | Open issues: [pgsty/silo#197](https://github.com/pgsty/silo/issues/197) |
| [@aschyolkin](https://github.com/aschyolkin) | Reported a data race in CPU metrics collection | Open issues: [pgsty/silo#210](https://github.com/pgsty/silo/issues/210) |
## Audit scope
Counts below come from every page of the GitHub issue / PR records, across all states, including automated accounts. There are 49 distinct human public issue / PR authors, plus 1 credited security reporter. Excluded automated accounts: `Copilot`, `dependabot[bot]`.
| Repository | Issues | Pull requests |
| :-- | --: | --: |
| [pgsty/silo](https://github.com/pgsty/silo) | 108 | 99 |
| [pgsty/silo-console](https://github.com/pgsty/silo-console) | 35 | 22 |
| [pgsty/mc](https://github.com/pgsty/mc) | 18 | 25 |
| [pgsty/silo-pkg](https://github.com/pgsty/silo-pkg) | 0 | 9 |
| [pgsty/silo-go](https://github.com/pgsty/silo-go) | 0 | 0 |
| [pgsty/kes](https://github.com/pgsty/kes) | 0 | 0 |
| [pgsty/silo.pgsty.com](https://github.com/pgsty/silo.pgsty.com) | 1 | 26 |
| [pgsty/minio-docs](https://github.com/pgsty/minio-docs) | 0 | 0 |
| **Total** | **162** | **181** |
## Maintainer record
<details>
<summary>@Vonng — complete issue / PR record</summary>
Merged PRs: [pgsty/silo#44](https://github.com/pgsty/silo/pull/44), [pgsty/silo#45](https://github.com/pgsty/silo/pull/45), [pgsty/silo#56](https://github.com/pgsty/silo/pull/56), [pgsty/silo#66](https://github.com/pgsty/silo/pull/66), [pgsty/silo#69](https://github.com/pgsty/silo/pull/69), [pgsty/silo#70](https://github.com/pgsty/silo/pull/70), [pgsty/silo#72](https://github.com/pgsty/silo/pull/72), [pgsty/silo#73](https://github.com/pgsty/silo/pull/73), [pgsty/silo#74](https://github.com/pgsty/silo/pull/74), [pgsty/silo#80](https://github.com/pgsty/silo/pull/80), [pgsty/silo#81](https://github.com/pgsty/silo/pull/81), [pgsty/silo#85](https://github.com/pgsty/silo/pull/85), [pgsty/silo#86](https://github.com/pgsty/silo/pull/86), [pgsty/silo#87](https://github.com/pgsty/silo/pull/87), [pgsty/silo#88](https://github.com/pgsty/silo/pull/88), [pgsty/silo#89](https://github.com/pgsty/silo/pull/89), [pgsty/silo#90](https://github.com/pgsty/silo/pull/90), [pgsty/silo#91](https://github.com/pgsty/silo/pull/91), [pgsty/silo#92](https://github.com/pgsty/silo/pull/92), [pgsty/silo#93](https://github.com/pgsty/silo/pull/93), [pgsty/silo#94](https://github.com/pgsty/silo/pull/94), [pgsty/silo#95](https://github.com/pgsty/silo/pull/95), [pgsty/silo#96](https://github.com/pgsty/silo/pull/96), [pgsty/silo#97](https://github.com/pgsty/silo/pull/97), [pgsty/silo#98](https://github.com/pgsty/silo/pull/98), [pgsty/silo#101](https://github.com/pgsty/silo/pull/101), [pgsty/silo#103](https://github.com/pgsty/silo/pull/103), [pgsty/silo#104](https://github.com/pgsty/silo/pull/104), [pgsty/silo#121](https://github.com/pgsty/silo/pull/121), [pgsty/silo#122](https://github.com/pgsty/silo/pull/122), [pgsty/silo#123](https://github.com/pgsty/silo/pull/123), [pgsty/silo#124](https://github.com/pgsty/silo/pull/124), [pgsty/silo#126](https://github.com/pgsty/silo/pull/126), [pgsty/silo#127](https://github.com/pgsty/silo/pull/127), [pgsty/silo#128](https://github.com/pgsty/silo/pull/128), [pgsty/silo#129](https://github.com/pgsty/silo/pull/129), [pgsty/silo#130](https://github.com/pgsty/silo/pull/130), [pgsty/silo#131](https://github.com/pgsty/silo/pull/131), [pgsty/silo#134](https://github.com/pgsty/silo/pull/134), [pgsty/silo#135](https://github.com/pgsty/silo/pull/135), [pgsty/silo#138](https://github.com/pgsty/silo/pull/138), [pgsty/silo#140](https://github.com/pgsty/silo/pull/140), [pgsty/silo#142](https://github.com/pgsty/silo/pull/142), [pgsty/silo#143](https://github.com/pgsty/silo/pull/143), [pgsty/silo#145](https://github.com/pgsty/silo/pull/145), [pgsty/silo#146](https://github.com/pgsty/silo/pull/146), [pgsty/silo#149](https://github.com/pgsty/silo/pull/149), [pgsty/silo#156](https://github.com/pgsty/silo/pull/156), [pgsty/silo#157](https://github.com/pgsty/silo/pull/157), [pgsty/silo#159](https://github.com/pgsty/silo/pull/159), [pgsty/silo#161](https://github.com/pgsty/silo/pull/161), [pgsty/silo#162](https://github.com/pgsty/silo/pull/162), [pgsty/silo#163](https://github.com/pgsty/silo/pull/163), [pgsty/silo#164](https://github.com/pgsty/silo/pull/164), [pgsty/silo#173](https://github.com/pgsty/silo/pull/173), [pgsty/silo#174](https://github.com/pgsty/silo/pull/174), [pgsty/silo#175](https://github.com/pgsty/silo/pull/175), [pgsty/silo#177](https://github.com/pgsty/silo/pull/177), [pgsty/silo#178](https://github.com/pgsty/silo/pull/178), [pgsty/silo#179](https://github.com/pgsty/silo/pull/179), [pgsty/silo#180](https://github.com/pgsty/silo/pull/180), [pgsty/silo#181](https://github.com/pgsty/silo/pull/181), [pgsty/silo#182](https://github.com/pgsty/silo/pull/182), [pgsty/silo#188](https://github.com/pgsty/silo/pull/188), [pgsty/silo#189](https://github.com/pgsty/silo/pull/189), [pgsty/silo#190](https://github.com/pgsty/silo/pull/190), [pgsty/silo#191](https://github.com/pgsty/silo/pull/191), [pgsty/silo#192](https://github.com/pgsty/silo/pull/192), [pgsty/silo#193](https://github.com/pgsty/silo/pull/193), [pgsty/silo#194](https://github.com/pgsty/silo/pull/194), [pgsty/silo#196](https://github.com/pgsty/silo/pull/196), [pgsty/silo#205](https://github.com/pgsty/silo/pull/205), [pgsty/silo#206](https://github.com/pgsty/silo/pull/206), [pgsty/silo#207](https://github.com/pgsty/silo/pull/207), [pgsty/silo#208](https://github.com/pgsty/silo/pull/208), [pgsty/silo#209](https://github.com/pgsty/silo/pull/209), [pgsty/silo#211](https://github.com/pgsty/silo/pull/211)
PRs closed without merging: [pgsty/silo#155](https://github.com/pgsty/silo/pull/155), [pgsty/silo#195](https://github.com/pgsty/silo/pull/195)
Closed issues: [pgsty/silo#22](https://github.com/pgsty/silo/issues/22), [pgsty/silo#23](https://github.com/pgsty/silo/issues/23), [pgsty/silo#24](https://github.com/pgsty/silo/issues/24), [pgsty/silo#25](https://github.com/pgsty/silo/issues/25), [pgsty/silo#26](https://github.com/pgsty/silo/issues/26), [pgsty/silo#27](https://github.com/pgsty/silo/issues/27), [pgsty/silo#28](https://github.com/pgsty/silo/issues/28), [pgsty/silo#46](https://github.com/pgsty/silo/issues/46), [pgsty/silo#47](https://github.com/pgsty/silo/issues/47), [pgsty/silo#48](https://github.com/pgsty/silo/issues/48), [pgsty/silo#49](https://github.com/pgsty/silo/issues/49), [pgsty/silo#50](https://github.com/pgsty/silo/issues/50), [pgsty/silo#52](https://github.com/pgsty/silo/issues/52), [pgsty/silo#53](https://github.com/pgsty/silo/issues/53), [pgsty/silo#55](https://github.com/pgsty/silo/issues/55), [pgsty/silo#63](https://github.com/pgsty/silo/issues/63), [pgsty/silo#64](https://github.com/pgsty/silo/issues/64), [pgsty/silo#65](https://github.com/pgsty/silo/issues/65), [pgsty/silo#67](https://github.com/pgsty/silo/issues/67), [pgsty/silo#68](https://github.com/pgsty/silo/issues/68), [pgsty/silo#75](https://github.com/pgsty/silo/issues/75), [pgsty/silo#76](https://github.com/pgsty/silo/issues/76), [pgsty/silo#77](https://github.com/pgsty/silo/issues/77), [pgsty/silo#78](https://github.com/pgsty/silo/issues/78), [pgsty/silo#82](https://github.com/pgsty/silo/issues/82), [pgsty/silo#83](https://github.com/pgsty/silo/issues/83), [pgsty/silo#84](https://github.com/pgsty/silo/issues/84), [pgsty/silo#99](https://github.com/pgsty/silo/issues/99), [pgsty/silo#100](https://github.com/pgsty/silo/issues/100), [pgsty/silo#102](https://github.com/pgsty/silo/issues/102), [pgsty/silo#105](https://github.com/pgsty/silo/issues/105), [pgsty/silo#109](https://github.com/pgsty/silo/issues/109), [pgsty/silo#110](https://github.com/pgsty/silo/issues/110), [pgsty/silo#111](https://github.com/pgsty/silo/issues/111), [pgsty/silo#112](https://github.com/pgsty/silo/issues/112), [pgsty/silo#113](https://github.com/pgsty/silo/issues/113), [pgsty/silo#114](https://github.com/pgsty/silo/issues/114), [pgsty/silo#115](https://github.com/pgsty/silo/issues/115), [pgsty/silo#116](https://github.com/pgsty/silo/issues/116), [pgsty/silo#117](https://github.com/pgsty/silo/issues/117), [pgsty/silo#118](https://github.com/pgsty/silo/issues/118), [pgsty/silo#119](https://github.com/pgsty/silo/issues/119), [pgsty/silo#120](https://github.com/pgsty/silo/issues/120), [pgsty/silo#133](https://github.com/pgsty/silo/issues/133), [pgsty/silo#136](https://github.com/pgsty/silo/issues/136), [pgsty/silo#137](https://github.com/pgsty/silo/issues/137), [pgsty/silo#139](https://github.com/pgsty/silo/issues/139), [pgsty/silo#141](https://github.com/pgsty/silo/issues/141), [pgsty/silo#144](https://github.com/pgsty/silo/issues/144), [pgsty/silo#147](https://github.com/pgsty/silo/issues/147), [pgsty/silo#148](https://github.com/pgsty/silo/issues/148), [pgsty/silo#158](https://github.com/pgsty/silo/issues/158), [pgsty/silo#165](https://github.com/pgsty/silo/issues/165), [pgsty/silo#166](https://github.com/pgsty/silo/issues/166), [pgsty/silo#167](https://github.com/pgsty/silo/issues/167), [pgsty/silo#168](https://github.com/pgsty/silo/issues/168), [pgsty/silo#169](https://github.com/pgsty/silo/issues/169), [pgsty/silo#170](https://github.com/pgsty/silo/issues/170), [pgsty/silo#171](https://github.com/pgsty/silo/issues/171), [pgsty/silo#204](https://github.com/pgsty/silo/issues/204)
Open issues: [pgsty/silo#199](https://github.com/pgsty/silo/issues/199), [pgsty/silo#200](https://github.com/pgsty/silo/issues/200), [pgsty/silo#201](https://github.com/pgsty/silo/issues/201), [pgsty/silo#202](https://github.com/pgsty/silo/issues/202), [pgsty/silo#203](https://github.com/pgsty/silo/issues/203)
Merged PRs: [pgsty/silo-console#9](https://github.com/pgsty/silo-console/pull/9), [pgsty/silo-console#10](https://github.com/pgsty/silo-console/pull/10), [pgsty/silo-console#11](https://github.com/pgsty/silo-console/pull/11), [pgsty/silo-console#38](https://github.com/pgsty/silo-console/pull/38), [pgsty/silo-console#39](https://github.com/pgsty/silo-console/pull/39), [pgsty/silo-console#40](https://github.com/pgsty/silo-console/pull/40), [pgsty/silo-console#41](https://github.com/pgsty/silo-console/pull/41), [pgsty/silo-console#42](https://github.com/pgsty/silo-console/pull/42), [pgsty/silo-console#43](https://github.com/pgsty/silo-console/pull/43), [pgsty/silo-console#44](https://github.com/pgsty/silo-console/pull/44), [pgsty/silo-console#45](https://github.com/pgsty/silo-console/pull/45), [pgsty/silo-console#46](https://github.com/pgsty/silo-console/pull/46), [pgsty/silo-console#47](https://github.com/pgsty/silo-console/pull/47), [pgsty/silo-console#48](https://github.com/pgsty/silo-console/pull/48), [pgsty/silo-console#49](https://github.com/pgsty/silo-console/pull/49), [pgsty/silo-console#50](https://github.com/pgsty/silo-console/pull/50), [pgsty/silo-console#51](https://github.com/pgsty/silo-console/pull/51), [pgsty/silo-console#53](https://github.com/pgsty/silo-console/pull/53), [pgsty/silo-console#54](https://github.com/pgsty/silo-console/pull/54), [pgsty/silo-console#55](https://github.com/pgsty/silo-console/pull/55), [pgsty/silo-console#56](https://github.com/pgsty/silo-console/pull/56), [pgsty/silo-console#57](https://github.com/pgsty/silo-console/pull/57)
Closed issues: [pgsty/silo-console#1](https://github.com/pgsty/silo-console/issues/1), [pgsty/silo-console#2](https://github.com/pgsty/silo-console/issues/2), [pgsty/silo-console#3](https://github.com/pgsty/silo-console/issues/3), [pgsty/silo-console#4](https://github.com/pgsty/silo-console/issues/4), [pgsty/silo-console#5](https://github.com/pgsty/silo-console/issues/5), [pgsty/silo-console#6](https://github.com/pgsty/silo-console/issues/6), [pgsty/silo-console#7](https://github.com/pgsty/silo-console/issues/7), [pgsty/silo-console#8](https://github.com/pgsty/silo-console/issues/8), [pgsty/silo-console#12](https://github.com/pgsty/silo-console/issues/12), [pgsty/silo-console#13](https://github.com/pgsty/silo-console/issues/13), [pgsty/silo-console#14](https://github.com/pgsty/silo-console/issues/14), [pgsty/silo-console#15](https://github.com/pgsty/silo-console/issues/15), [pgsty/silo-console#16](https://github.com/pgsty/silo-console/issues/16), [pgsty/silo-console#17](https://github.com/pgsty/silo-console/issues/17), [pgsty/silo-console#18](https://github.com/pgsty/silo-console/issues/18), [pgsty/silo-console#19](https://github.com/pgsty/silo-console/issues/19), [pgsty/silo-console#20](https://github.com/pgsty/silo-console/issues/20), [pgsty/silo-console#21](https://github.com/pgsty/silo-console/issues/21), [pgsty/silo-console#22](https://github.com/pgsty/silo-console/issues/22), [pgsty/silo-console#23](https://github.com/pgsty/silo-console/issues/23), [pgsty/silo-console#24](https://github.com/pgsty/silo-console/issues/24), [pgsty/silo-console#25](https://github.com/pgsty/silo-console/issues/25), [pgsty/silo-console#26](https://github.com/pgsty/silo-console/issues/26), [pgsty/silo-console#27](https://github.com/pgsty/silo-console/issues/27), [pgsty/silo-console#28](https://github.com/pgsty/silo-console/issues/28), [pgsty/silo-console#29](https://github.com/pgsty/silo-console/issues/29), [pgsty/silo-console#30](https://github.com/pgsty/silo-console/issues/30), [pgsty/silo-console#31](https://github.com/pgsty/silo-console/issues/31), [pgsty/silo-console#33](https://github.com/pgsty/silo-console/issues/33), [pgsty/silo-console#35](https://github.com/pgsty/silo-console/issues/35), [pgsty/silo-console#36](https://github.com/pgsty/silo-console/issues/36), [pgsty/silo-console#37](https://github.com/pgsty/silo-console/issues/37)
Open issues: [pgsty/silo-console#32](https://github.com/pgsty/silo-console/issues/32), [pgsty/silo-console#34](https://github.com/pgsty/silo-console/issues/34)
Merged PRs: [pgsty/mc#1](https://github.com/pgsty/mc/pull/1), [pgsty/mc#2](https://github.com/pgsty/mc/pull/2), [pgsty/mc#3](https://github.com/pgsty/mc/pull/3), [pgsty/mc#4](https://github.com/pgsty/mc/pull/4), [pgsty/mc#8](https://github.com/pgsty/mc/pull/8), [pgsty/mc#9](https://github.com/pgsty/mc/pull/9), [pgsty/mc#10](https://github.com/pgsty/mc/pull/10), [pgsty/mc#11](https://github.com/pgsty/mc/pull/11), [pgsty/mc#13](https://github.com/pgsty/mc/pull/13), [pgsty/mc#22](https://github.com/pgsty/mc/pull/22), [pgsty/mc#24](https://github.com/pgsty/mc/pull/24), [pgsty/mc#27](https://github.com/pgsty/mc/pull/27), [pgsty/mc#32](https://github.com/pgsty/mc/pull/32), [pgsty/mc#33](https://github.com/pgsty/mc/pull/33), [pgsty/mc#34](https://github.com/pgsty/mc/pull/34), [pgsty/mc#35](https://github.com/pgsty/mc/pull/35), [pgsty/mc#36](https://github.com/pgsty/mc/pull/36), [pgsty/mc#37](https://github.com/pgsty/mc/pull/37), [pgsty/mc#38](https://github.com/pgsty/mc/pull/38), [pgsty/mc#39](https://github.com/pgsty/mc/pull/39), [pgsty/mc#40](https://github.com/pgsty/mc/pull/40), [pgsty/mc#41](https://github.com/pgsty/mc/pull/41), [pgsty/mc#42](https://github.com/pgsty/mc/pull/42), [pgsty/mc#43](https://github.com/pgsty/mc/pull/43)
Closed issues: [pgsty/mc#5](https://github.com/pgsty/mc/issues/5), [pgsty/mc#6](https://github.com/pgsty/mc/issues/6), [pgsty/mc#7](https://github.com/pgsty/mc/issues/7), [pgsty/mc#12](https://github.com/pgsty/mc/issues/12), [pgsty/mc#14](https://github.com/pgsty/mc/issues/14), [pgsty/mc#15](https://github.com/pgsty/mc/issues/15), [pgsty/mc#16](https://github.com/pgsty/mc/issues/16), [pgsty/mc#17](https://github.com/pgsty/mc/issues/17), [pgsty/mc#18](https://github.com/pgsty/mc/issues/18), [pgsty/mc#19](https://github.com/pgsty/mc/issues/19), [pgsty/mc#20](https://github.com/pgsty/mc/issues/20), [pgsty/mc#21](https://github.com/pgsty/mc/issues/21), [pgsty/mc#23](https://github.com/pgsty/mc/issues/23), [pgsty/mc#25](https://github.com/pgsty/mc/issues/25), [pgsty/mc#28](https://github.com/pgsty/mc/issues/28), [pgsty/mc#29](https://github.com/pgsty/mc/issues/29), [pgsty/mc#30](https://github.com/pgsty/mc/issues/30), [pgsty/mc#31](https://github.com/pgsty/mc/issues/31)
Merged PRs: [pgsty/silo-pkg#1](https://github.com/pgsty/silo-pkg/pull/1), [pgsty/silo-pkg#2](https://github.com/pgsty/silo-pkg/pull/2), [pgsty/silo-pkg#3](https://github.com/pgsty/silo-pkg/pull/3), [pgsty/silo-pkg#4](https://github.com/pgsty/silo-pkg/pull/4), [pgsty/silo-pkg#5](https://github.com/pgsty/silo-pkg/pull/5), [pgsty/silo-pkg#6](https://github.com/pgsty/silo-pkg/pull/6), [pgsty/silo-pkg#7](https://github.com/pgsty/silo-pkg/pull/7), [pgsty/silo-pkg#8](https://github.com/pgsty/silo-pkg/pull/8), [pgsty/silo-pkg#9](https://github.com/pgsty/silo-pkg/pull/9)
Merged PRs: [pgsty/silo.pgsty.com#2](https://github.com/pgsty/silo.pgsty.com/pull/2), [pgsty/silo.pgsty.com#3](https://github.com/pgsty/silo.pgsty.com/pull/3), [pgsty/silo.pgsty.com#5](https://github.com/pgsty/silo.pgsty.com/pull/5), [pgsty/silo.pgsty.com#6](https://github.com/pgsty/silo.pgsty.com/pull/6), [pgsty/silo.pgsty.com#7](https://github.com/pgsty/silo.pgsty.com/pull/7), [pgsty/silo.pgsty.com#8](https://github.com/pgsty/silo.pgsty.com/pull/8), [pgsty/silo.pgsty.com#9](https://github.com/pgsty/silo.pgsty.com/pull/9), [pgsty/silo.pgsty.com#10](https://github.com/pgsty/silo.pgsty.com/pull/10), [pgsty/silo.pgsty.com#11](https://github.com/pgsty/silo.pgsty.com/pull/11), [pgsty/silo.pgsty.com#13](https://github.com/pgsty/silo.pgsty.com/pull/13), [pgsty/silo.pgsty.com#14](https://github.com/pgsty/silo.pgsty.com/pull/14), [pgsty/silo.pgsty.com#15](https://github.com/pgsty/silo.pgsty.com/pull/15), [pgsty/silo.pgsty.com#16](https://github.com/pgsty/silo.pgsty.com/pull/16), [pgsty/silo.pgsty.com#17](https://github.com/pgsty/silo.pgsty.com/pull/17), [pgsty/silo.pgsty.com#19](https://github.com/pgsty/silo.pgsty.com/pull/19), [pgsty/silo.pgsty.com#20](https://github.com/pgsty/silo.pgsty.com/pull/20), [pgsty/silo.pgsty.com#21](https://github.com/pgsty/silo.pgsty.com/pull/21), [pgsty/silo.pgsty.com#22](https://github.com/pgsty/silo.pgsty.com/pull/22), [pgsty/silo.pgsty.com#23](https://github.com/pgsty/silo.pgsty.com/pull/23), [pgsty/silo.pgsty.com#24](https://github.com/pgsty/silo.pgsty.com/pull/24), [pgsty/silo.pgsty.com#26](https://github.com/pgsty/silo.pgsty.com/pull/26), [pgsty/silo.pgsty.com#27](https://github.com/pgsty/silo.pgsty.com/pull/27), [pgsty/silo.pgsty.com#28](https://github.com/pgsty/silo.pgsty.com/pull/28), [pgsty/silo.pgsty.com#29](https://github.com/pgsty/silo.pgsty.com/pull/29)
Open PRs: [pgsty/silo.pgsty.com#25](https://github.com/pgsty/silo.pgsty.com/pull/25)
</details>
## Keeping this record current
The reviewed [website contributor data](https://github.com/pgsty/silo.pgsty.com/blob/main/data/home/contributors.yaml) is shared by this record, the component records, README avatar walls, and both website languages. After auditing all issue / PR pages and reviewing adoption evidence, update that data and run `python3 bin/contributors.py` and `python3 bin/contributor_avatars.py` in the site checkout. Validate the generated records with `python3 bin/contributors.py --check`.
Code contributions follow the no-CLA, DCO policy in [CONTRIBUTING.md](CONTRIBUTING.md).
+4864 -6148
View File
File diff suppressed because it is too large Load Diff
-18
View File
@@ -1,18 +0,0 @@
FROM minio/minio:latest
ARG TARGETARCH
ARG RELEASE
RUN chmod -R 777 /usr/bin
COPY ./minio-${TARGETARCH}.${RELEASE} /usr/bin/minio
COPY ./minio-${TARGETARCH}.${RELEASE}.minisig /usr/bin/minio.minisig
COPY ./minio-${TARGETARCH}.${RELEASE}.sha256sum /usr/bin/minio.sha256sum
COPY dockerscripts/docker-entrypoint.sh /usr/bin/docker-entrypoint.sh
ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"]
VOLUME ["/data"]
CMD ["minio"]
-3
View File
@@ -1,3 +0,0 @@
FROM minio/minio:edge
CMD ["minio", "server", "/data"]
+50
View File
@@ -0,0 +1,50 @@
# The distroless variant ships exactly one program: the silo binary.
# No shell, no mc, no curl, no entrypoint script; health checking is
# provided by the binary itself (`silo healthcheck`).
# Design note: https://silo.pgsty.com/compatibility/feature/healthcheck/
# A distroless final stage cannot RUN anything, so /data is prepared in a
# throwaway stage. It ships world-writable (see pgsty/silo#55): Docker
# seeds fresh volumes from the image-layer mountpoint, no entrypoint
# exists to repair ownership at runtime, and 0777 is what keeps every
# privilege mode working, --user included.
FROM busybox:1.37.0 AS prep
RUN mkdir -p /prep/data && chmod 0777 /prep/data
FROM gcr.io/distroless/static-debian12:latest
LABEL org.opencontainers.image.title="Silo" \
org.opencontainers.image.description="S3-Interface Libre Object Storage (distroless)" \
org.opencontainers.image.url="https://silo.pgsty.com" \
org.opencontainers.image.source="https://github.com/pgsty/silo" \
org.opencontainers.image.licenses="AGPL-3.0-or-later" \
maintainer="PGSTY <https://silo.pgsty.com>"
ENV MINIO_ACCESS_KEY_FILE=access_key \
MINIO_SECRET_KEY_FILE=secret_key \
MINIO_ROOT_USER_FILE=access_key \
MINIO_ROOT_PASSWORD_FILE=secret_key \
MINIO_KMS_SECRET_KEY_FILE=kms_master_key \
MINIO_CONFIG_ENV_FILE=config.env \
HOME=/tmp
COPY --chmod=0755 silo /usr/bin/silo
# COPY of a directory copies its contents, not the directory entry, so an
# empty /prep/data would arrive as a default root:0755 /data and non-root
# runs would fail storage init. Copying the parent makes data/ itself a
# copied entry, which --chmod then actually applies to.
COPY --from=prep --chmod=0777 /prep/ /
COPY LICENSE NOTICE CREDITS /licenses/
EXPOSE 9000
VOLUME ["/data"]
# Exec form is mandatory: there is no /bin/sh in this image. `ready`
# rather than `live` because Docker health feeds start-order gating
# (readiness semantics); the two are identical unless KMS/etcd are used.
# The outer timeout stays above the probe's own 5s deadline so the
# probe can report its diagnostic line instead of being SIGKILLed.
HEALTHCHECK --interval=30s --timeout=10s --start-period=2m --start-interval=2s --retries=3 \
CMD ["/usr/bin/silo", "healthcheck", "ready"]
ENTRYPOINT ["/usr/bin/silo"]
+34 -18
View File
@@ -1,4 +1,15 @@
FROM golang:1.26.5-alpine AS build
FROM golang:1.27.1-alpine@sha256:cf6fca6641884b8433441b2b0652976f975e1d0fdd26d177eaaf8596087f3125 AS curl-build
ARG TARGETARCH
COPY dockerscripts/build-static-curl.sh /build/build-static-curl
RUN /bin/sh /build/build-static-curl
# Exercise the exact shipped curl without a dynamic loader or shared libraries.
FROM scratch AS curl-runtime
COPY --from=curl-build /go/bin/curl /curl
COPY --from=curl-build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
ENTRYPOINT ["/curl"]
FROM golang:1.27.1-alpine@sha256:cf6fca6641884b8433441b2b0652976f975e1d0fdd26d177eaaf8596087f3125 AS build
ARG TARGETARCH
@@ -6,7 +17,9 @@ ENV GOPATH=/go
ENV CGO_ENABLED=0
ARG MC_REPO=pgsty/mc
ARG MC_VERSION=latest
ARG MC_VERSION=RELEASE.2026-09-16T00-00-00Z
ARG MC_AMD64_SHA256=4ba2814fd5507fbe6b4d237c359750b9119d28d7217495fa5b48002fcbd397ef
ARG MC_ARM64_SHA256=b7008ca2a1bc5735b6585981c59a3640a0daa152dc789df3d1a0d0438787de82
RUN apk add -U --no-cache \
ca-certificates \
@@ -14,8 +27,8 @@ RUN apk add -U --no-cache \
curl \
jq && \
case "${TARGETARCH}" in \
amd64) MC_ARCH=amd64 ;; \
arm64) MC_ARCH=arm64 ;; \
amd64) MC_ARCH=amd64; MC_PINNED_SHA256="${MC_AMD64_SHA256}" ;; \
arm64) MC_ARCH=arm64; MC_PINNED_SHA256="${MC_ARM64_SHA256}" ;; \
*) echo "Unsupported TARGETARCH=${TARGETARCH}"; exit 1 ;; \
esac && \
if [ "${MC_VERSION}" = "latest" ]; then \
@@ -39,7 +52,8 @@ RUN apk add -U --no-cache \
EXPECTED=$(grep " ${ARCHIVE_NAME}$" /tmp/mcli_checksums.txt | awk '{print $1}') && \
ACTUAL=$(sha256sum /tmp/mcli.tar.gz | awk '{print $1}') && \
[ -n "${EXPECTED}" ] || { echo "Checksum entry not found for ${ARCHIVE_NAME}"; exit 1; } && \
[ "${EXPECTED}" = "${ACTUAL}" ] || { echo "Checksum mismatch: expected ${EXPECTED}, got ${ACTUAL}"; exit 1; } && \
[ "${EXPECTED}" = "${MC_PINNED_SHA256}" ] || { echo "Published checksum drift for ${ARCHIVE_NAME}"; exit 1; } && \
[ "${MC_PINNED_SHA256}" = "${ACTUAL}" ] || { echo "Checksum mismatch: expected ${MC_PINNED_SHA256}, got ${ACTUAL}"; exit 1; } && \
echo "Checksum OK: ${ACTUAL}" && \
mkdir -p /tmp/mcli-extract && \
tar -xzf /tmp/mcli.tar.gz -C /tmp/mcli-extract/ && \
@@ -53,44 +67,46 @@ RUN apk add -U --no-cache \
chmod +x /go/bin/mcli && \
ln -sf mcli /go/bin/mc
COPY dockerscripts/download-static-curl.sh /build/download-static-curl
RUN chmod +x /build/download-static-curl && \
/build/download-static-curl
FROM registry.access.redhat.com/ubi9/ubi:latest AS certs
FROM registry.access.redhat.com/ubi9/ubi:latest@sha256:206b65b8ee0f04b992818c9a51b29081b14974630d4850bc358097d0c44ea156 AS certs
RUN dnf -y install ca-certificates && \
update-ca-trust && \
cp /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem /tmp/ca-certificates.crt && \
dnf clean all && \
rm -rf /var/cache/dnf
FROM registry.access.redhat.com/ubi9/ubi-micro:latest
FROM registry.access.redhat.com/ubi9/ubi-micro:latest@sha256:f332c99eb8f798a8486821c91937f10ad64ee83d7e739303be2df051040918f6
LABEL maintainer="pgsty <https://github.com/pgsty/minio>" \
description="MinIO community fork, build by pgsty"
LABEL org.opencontainers.image.title="Silo" \
org.opencontainers.image.description="S3-Interface Libre Object Storage" \
org.opencontainers.image.url="https://silo.pgsty.com" \
org.opencontainers.image.source="https://github.com/pgsty/silo" \
org.opencontainers.image.licenses="AGPL-3.0-or-later" \
maintainer="PGSTY <https://silo.pgsty.com>"
ENV MINIO_ACCESS_KEY_FILE=access_key \
MINIO_SECRET_KEY_FILE=secret_key \
MINIO_ROOT_USER_FILE=access_key \
MINIO_ROOT_PASSWORD_FILE=secret_key \
MINIO_KMS_SECRET_KEY_FILE=kms_master_key \
MINIO_UPDATE_MINISIGN_PUBKEY="RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav" \
MINIO_CONFIG_ENV_FILE=config.env \
HOME=/tmp \
MC_CONFIG_DIR=/tmp/.mc
COPY --from=certs /tmp/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
COPY minio /usr/bin/minio
COPY silo /usr/bin/silo
COPY --from=build /go/bin/mcli /usr/bin/mcli
COPY --from=build /go/bin/curl* /usr/bin/
COPY --from=curl-build /go/bin/curl /usr/bin/curl
COPY --from=curl-build /go/share/curl /licenses/curl
COPY dockerscripts/docker-entrypoint.sh /usr/bin/docker-entrypoint.sh
COPY LICENSE /licenses/LICENSE
COPY NOTICE /licenses/NOTICE
COPY CREDITS /licenses/CREDITS
RUN chmod +x /usr/bin/minio /usr/bin/mcli /usr/bin/docker-entrypoint.sh && \
RUN chmod +x /usr/bin/silo /usr/bin/mcli /usr/bin/docker-entrypoint.sh && \
ln -sf mcli /usr/bin/mc
EXPOSE 9000
VOLUME ["/data"]
ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"]
CMD ["minio"]
CMD ["silo"]
-71
View File
@@ -1,71 +0,0 @@
FROM golang:1.26.5-alpine as build
ARG TARGETARCH
ARG RELEASE
ENV GOPATH=/go
ENV CGO_ENABLED=0
# Install curl and minisign
RUN apk add -U --no-cache ca-certificates && \
apk add -U --no-cache curl && \
go install aead.dev/minisign/cmd/minisign@v0.2.1
# Download minio binary and signature files
RUN curl -s -q https://dl.min.io/server/minio/hotfixes/linux-${TARGETARCH}/archive/minio.${RELEASE} -o /go/bin/minio && \
curl -s -q https://dl.min.io/server/minio/hotfixes/linux-${TARGETARCH}/archive/minio.${RELEASE}.minisig -o /go/bin/minio.minisig && \
curl -s -q https://dl.min.io/server/minio/hotfixes/linux-${TARGETARCH}/archive/minio.${RELEASE}.sha256sum -o /go/bin/minio.sha256sum && \
chmod +x /go/bin/minio
# Download mc binary and signature files
RUN curl -s -q https://dl.min.io/client/mc/release/linux-${TARGETARCH}/mc -o /go/bin/mc && \
curl -s -q https://dl.min.io/client/mc/release/linux-${TARGETARCH}/mc.minisig -o /go/bin/mc.minisig && \
curl -s -q https://dl.min.io/client/mc/release/linux-${TARGETARCH}/mc.sha256sum -o /go/bin/mc.sha256sum && \
chmod +x /go/bin/mc
RUN if [ "$TARGETARCH" = "amd64" ]; then \
curl -L -s -q https://github.com/moparisthebest/static-curl/releases/latest/download/curl-${TARGETARCH} -o /go/bin/curl; \
chmod +x /go/bin/curl; \
fi
# Verify binary signature using public key "RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGavRUN"
RUN minisign -Vqm /go/bin/minio -x /go/bin/minio.minisig -P RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav && \
minisign -Vqm /go/bin/mc -x /go/bin/mc.minisig -P RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav
FROM registry.access.redhat.com/ubi9/ubi-micro:latest
ARG RELEASE
LABEL name="MinIO" \
vendor="MinIO Inc <dev@min.io>" \
maintainer="MinIO Inc <dev@min.io>" \
version="${RELEASE}" \
release="${RELEASE}" \
summary="MinIO is a High Performance Object Storage, API compatible with Amazon S3 cloud storage service." \
description="MinIO object storage is fundamentally different. Designed for performance and the S3 API, it is 100% open-source. MinIO is ideal for large, private cloud environments with stringent security requirements and delivers mission-critical availability across a diverse range of workloads."
ENV MINIO_ACCESS_KEY_FILE=access_key \
MINIO_SECRET_KEY_FILE=secret_key \
MINIO_ROOT_USER_FILE=access_key \
MINIO_ROOT_PASSWORD_FILE=secret_key \
MINIO_KMS_SECRET_KEY_FILE=kms_master_key \
MINIO_UPDATE_MINISIGN_PUBKEY="RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav" \
MINIO_CONFIG_ENV_FILE=config.env \
MC_CONFIG_DIR=/tmp/.mc
RUN chmod -R 777 /usr/bin
COPY --from=build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
COPY --from=build /go/bin/minio* /usr/bin/
COPY --from=build /go/bin/mc* /usr/bin/
COPY --from=build /go/bin/cur* /usr/bin/
COPY CREDITS /licenses/CREDITS
COPY LICENSE /licenses/LICENSE
COPY dockerscripts/docker-entrypoint.sh /usr/bin/docker-entrypoint.sh
EXPOSE 9000
VOLUME ["/data"]
ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"]
CMD ["minio"]
-73
View File
@@ -1,73 +0,0 @@
FROM golang:1.26.5-alpine AS build
ARG TARGETARCH
ARG RELEASE
ENV GOPATH=/go
ENV CGO_ENABLED=0
WORKDIR /build
# Install curl and minisign
RUN apk add -U --no-cache ca-certificates && \
apk add -U --no-cache curl && \
apk add -U --no-cache bash && \
go install aead.dev/minisign/cmd/minisign@v0.2.1
# Download minio binary and signature files
RUN curl -s -q https://dl.min.io/server/minio/release/linux-${TARGETARCH}/archive/minio.${RELEASE} -o /go/bin/minio && \
curl -s -q https://dl.min.io/server/minio/release/linux-${TARGETARCH}/archive/minio.${RELEASE}.minisig -o /go/bin/minio.minisig && \
curl -s -q https://dl.min.io/server/minio/release/linux-${TARGETARCH}/archive/minio.${RELEASE}.sha256sum -o /go/bin/minio.sha256sum && \
chmod +x /go/bin/minio
# Download mc binary and signature files
RUN curl -s -q https://dl.min.io/client/mc/release/linux-${TARGETARCH}/mc -o /go/bin/mc && \
curl -s -q https://dl.min.io/client/mc/release/linux-${TARGETARCH}/mc.minisig -o /go/bin/mc.minisig && \
curl -s -q https://dl.min.io/client/mc/release/linux-${TARGETARCH}/mc.sha256sum -o /go/bin/mc.sha256sum && \
chmod +x /go/bin/mc
# Verify binary signature using public key "RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGavRUN"
RUN minisign -Vqm /go/bin/minio -x /go/bin/minio.minisig -P RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav && \
minisign -Vqm /go/bin/mc -x /go/bin/mc.minisig -P RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav
COPY dockerscripts/download-static-curl.sh /build/download-static-curl
RUN chmod +x /build/download-static-curl && \
/build/download-static-curl
FROM registry.access.redhat.com/ubi9/ubi-micro:latest
ARG RELEASE
LABEL name="MinIO" \
vendor="MinIO Inc <dev@min.io>" \
maintainer="MinIO Inc <dev@min.io>" \
version="${RELEASE}" \
release="${RELEASE}" \
summary="MinIO is a High Performance Object Storage, API compatible with Amazon S3 cloud storage service." \
description="MinIO object storage is fundamentally different. Designed for performance and the S3 API, it is 100% open-source. MinIO is ideal for large, private cloud environments with stringent security requirements and delivers mission-critical availability across a diverse range of workloads."
ENV MINIO_ACCESS_KEY_FILE=access_key \
MINIO_SECRET_KEY_FILE=secret_key \
MINIO_ROOT_USER_FILE=access_key \
MINIO_ROOT_PASSWORD_FILE=secret_key \
MINIO_KMS_SECRET_KEY_FILE=kms_master_key \
MINIO_UPDATE_MINISIGN_PUBKEY="RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav" \
MINIO_CONFIG_ENV_FILE=config.env \
MC_CONFIG_DIR=/tmp/.mc
RUN chmod -R 777 /usr/bin
COPY --from=build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
COPY --from=build /go/bin/minio* /usr/bin/
COPY --from=build /go/bin/mc* /usr/bin/
COPY --from=build /go/bin/curl* /usr/bin/
COPY CREDITS /licenses/CREDITS
COPY LICENSE /licenses/LICENSE
COPY dockerscripts/docker-entrypoint.sh /usr/bin/docker-entrypoint.sh
EXPOSE 9000
VOLUME ["/data"]
ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"]
CMD ["minio"]
-71
View File
@@ -1,71 +0,0 @@
FROM golang:1.26.5-alpine AS build
ARG TARGETARCH
ARG RELEASE
ENV GOPATH=/go
ENV CGO_ENABLED=0
# Install curl and minisign
RUN apk add -U --no-cache ca-certificates && \
apk add -U --no-cache curl && \
go install aead.dev/minisign/cmd/minisign@v0.2.1
# Download minio binary and signature files
RUN curl -s -q https://dl.min.io/server/minio/release/linux-${TARGETARCH}/archive/minio.${RELEASE} -o /go/bin/minio && \
curl -s -q https://dl.min.io/server/minio/release/linux-${TARGETARCH}/archive/minio.${RELEASE}.minisig -o /go/bin/minio.minisig && \
curl -s -q https://dl.min.io/server/minio/release/linux-${TARGETARCH}/archive/minio.${RELEASE}.sha256sum -o /go/bin/minio.sha256sum && \
chmod +x /go/bin/minio
# Download mc binary and signature files
RUN curl -s -q https://dl.min.io/client/mc/release/linux-${TARGETARCH}/mc -o /go/bin/mc && \
curl -s -q https://dl.min.io/client/mc/release/linux-${TARGETARCH}/mc.minisig -o /go/bin/mc.minisig && \
curl -s -q https://dl.min.io/client/mc/release/linux-${TARGETARCH}/mc.sha256sum -o /go/bin/mc.sha256sum && \
chmod +x /go/bin/mc
RUN if [ "$TARGETARCH" = "amd64" ]; then \
curl -L -s -q https://github.com/moparisthebest/static-curl/releases/latest/download/curl-${TARGETARCH} -o /go/bin/curl; \
chmod +x /go/bin/curl; \
fi
# Verify binary signature using public key "RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGavRUN"
RUN minisign -Vqm /go/bin/minio -x /go/bin/minio.minisig -P RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav && \
minisign -Vqm /go/bin/mc -x /go/bin/mc.minisig -P RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav
FROM registry.access.redhat.com/ubi8/ubi-micro:latest
ARG RELEASE
LABEL name="MinIO" \
vendor="MinIO Inc <dev@min.io>" \
maintainer="MinIO Inc <dev@min.io>" \
version="${RELEASE}" \
release="${RELEASE}" \
summary="MinIO is a High Performance Object Storage, API compatible with Amazon S3 cloud storage service." \
description="MinIO object storage is fundamentally different. Designed for performance and the S3 API, it is 100% open-source. MinIO is ideal for large, private cloud environments with stringent security requirements and delivers mission-critical availability across a diverse range of workloads."
ENV MINIO_ACCESS_KEY_FILE=access_key \
MINIO_SECRET_KEY_FILE=secret_key \
MINIO_ROOT_USER_FILE=access_key \
MINIO_ROOT_PASSWORD_FILE=secret_key \
MINIO_KMS_SECRET_KEY_FILE=kms_master_key \
MINIO_UPDATE_MINISIGN_PUBKEY="RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav" \
MINIO_CONFIG_ENV_FILE=config.env \
MC_CONFIG_DIR=/tmp/.mc
RUN chmod -R 777 /usr/bin
COPY --from=build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
COPY --from=build /go/bin/minio* /usr/bin/
COPY --from=build /go/bin/mc* /usr/bin/
COPY --from=build /go/bin/cur* /usr/bin/
COPY CREDITS /licenses/CREDITS
COPY LICENSE /licenses/LICENSE
COPY dockerscripts/docker-entrypoint.sh /usr/bin/docker-entrypoint.sh
EXPOSE 9000
VOLUME ["/data"]
ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"]
CMD ["minio"]
-5
View File
@@ -1,5 +0,0 @@
FROM scratch
COPY minio /minio
CMD ["/minio"]
+64 -67
View File
@@ -4,11 +4,11 @@ LDFLAGS := $(shell go run buildscripts/gen-ldflags.go)
GOOS ?= $(shell go env GOOS)
GOARCH ?= $(shell go env GOARCH)
GOLANGCI_VERSION ?= v2.11.3
GOLANGCI_VERSION ?= v2.13.1
VERSION ?= $(shell git describe --tags)
REPO ?= quay.io/minio
TAG ?= $(REPO)/minio:$(VERSION)
REPO ?= docker.io/pgsty
TAG ?= $(REPO)/silo:$(VERSION)
GOLANGCI_DIR = .bin/golangci/$(GOLANGCI_VERSION)
GOLANGCI = $(GOLANGCI_DIR)/golangci-lint
@@ -33,15 +33,24 @@ getdeps: ## fetch necessary dependencies
sh "$$script" -b $(GOLANGCI_DIR) $(GOLANGCI_VERSION); \
fi
crosscompile: ## cross compile minio
crosscompile: ## cross compile Silo
@(env bash $(PWD)/buildscripts/cross-compile.sh)
verifiers: lint check-gen
verifiers: lint check-gen rebrand-guard
rebrand-guard: ## verify Silo branding and protected compatibility identifiers
@go run ./buildscripts/rebrand-guard
@env bash $(PWD)/buildscripts/verify-rebrand.sh
@env bash $(PWD)/dockerscripts/docker-entrypoint_test.sh
credits: ## regenerate CREDITS from the licenses of Go modules linked into the binary
@env bash $(PWD)/buildscripts/gen-credits.sh
check-gen: ## check for updated autogenerated files
@go generate ./... >/dev/null
@go mod tidy -compat=1.26
@changed=$$(git diff --name-only -- '*_gen.go' '*_gen_test.go' '*_msgp_test.go' '*_string.go' go.mod go.sum); \
@go mod tidy -compat=1.27
@env bash $(PWD)/buildscripts/gen-credits.sh
@changed=$$(git diff --name-only -- '*_gen.go' '*_gen_test.go' '*_msgp_test.go' '*_string.go' go.mod go.sum CREDITS); \
if [ -n "$$changed" ]; then \
echo "Non-committed generated changes detected:"; \
echo "$$changed"; \
@@ -64,12 +73,12 @@ lint-fix: getdeps ## runs golangci-lint suite of linters with automatic fixes
@$(GOLANGCI) run --build-tags kqueue --timeout=10m --config ./.golangci.yml --fix
check: test
test: verifiers build ## builds minio, runs linters, tests
test: verifiers build ## builds Silo, runs linters, tests
@echo "Running unit tests"
@MINIO_API_REQUESTS_MAX=10000 CGO_ENABLED=0 go test -v -tags kqueue,dev ./...
test-root-disable: install-race
@echo "Running minio root lockdown tests"
@echo "Running Silo root lockdown tests"
@env bash $(PWD)/buildscripts/disable-root.sh
test-ilm: install-race
@@ -85,7 +94,7 @@ test-pbac: install-race
@env bash $(PWD)/docs/iam/policies/pbac-tests.sh
test-decom: install-race
@echo "Running minio decom tests"
@echo "Running Silo decom tests"
@env bash $(PWD)/docs/distributed/decom.sh
@env bash $(PWD)/docs/distributed/decom-encrypted.sh
@env bash $(PWD)/docs/distributed/decom-encrypted-sse-s3.sh
@@ -93,17 +102,17 @@ test-decom: install-race
@env bash $(PWD)/docs/distributed/decom-encrypted-kes.sh
test-versioning: install-race
@echo "Running minio versioning tests"
@echo "Running Silo versioning tests"
@env bash $(PWD)/docs/bucket/versioning/versioning-tests.sh
test-configfile: install-race
@env bash $(PWD)/docs/distributed/distributed-from-config-file.sh
test-upgrade: install-race
@echo "Running minio upgrade tests"
test-upgrade:
@echo "Running MinIO-to-Silo upgrade tests"
@(env bash $(PWD)/buildscripts/minio-upgrade.sh)
test-race: verifiers build ## builds minio, runs linters, tests (race)
test-race: verifiers build ## builds Silo, runs linters, tests (race)
@echo "Running unit tests under -race"
@(env bash $(PWD)/buildscripts/race.sh)
@@ -151,9 +160,9 @@ test-site-replication-oidc: install-race ## verify automatic site replication
@echo "Running tests for automatic site replication of IAM (with OIDC)"
@(env bash $(PWD)/docs/site-replication/run-multi-site-oidc.sh)
test-site-replication-minio: install-race ## verify automatic site replication
@echo "Running tests for automatic site replication of IAM (with MinIO IDP)"
@(env bash $(PWD)/docs/site-replication/run-multi-site-minio-idp.sh)
test-site-replication-silo: install-race ## verify automatic site replication
@echo "Running tests for automatic site replication of IAM (with Silo IDP)"
@(env bash $(PWD)/docs/site-replication/run-multi-site-silo-idp.sh)
@echo "Running tests for automatic site replication of SSE-C objects"
@(env bash $(PWD)/docs/site-replication/run-ssec-object-replication.sh)
@echo "Running tests for automatic site replication of SSE-C objects with SSE-KMS enabled for bucket"
@@ -169,11 +178,11 @@ test-timeout: install-race ## test multipart
@echo "Test server timeout"
@(env bash $(PWD)/buildscripts/test-timeout.sh)
verify: install-race ## verify minio various setups
verify: install-race ## verify Silo in various setups
@echo "Verifying build with race"
@(env bash $(PWD)/buildscripts/verify-build.sh)
verify-healing: install-race ## verify healing and replacing disks with minio binary
verify-healing: install-race ## verify healing and replacing disks with the Silo binary
@echo "Verify healing build with race"
@(env bash $(PWD)/buildscripts/verify-healing.sh)
@(env bash $(PWD)/buildscripts/verify-healing-empty-erasure-set.sh)
@@ -194,59 +203,49 @@ verify-healing-inconsistent-versions: install-race ## verify resolving inconsist
build-debugging:
@(env bash $(PWD)/docs/debugging/build.sh)
build: checks build-debugging ## builds minio to $(PWD)
@echo "Building minio binary to './minio'"
@CGO_ENABLED=0 GOOS=$(GOOS) GOARCH=$(GOARCH) go build -tags kqueue -trimpath --ldflags "$(LDFLAGS)" -o $(PWD)/minio 1>/dev/null
build: checks build-debugging ## builds Silo to $(PWD)
@echo "Building Silo binary to './silo'"
@CGO_ENABLED=0 GOOS=$(GOOS) GOARCH=$(GOARCH) go build -tags kqueue -trimpath --ldflags "$(LDFLAGS)" -o $(PWD)/silo 1>/dev/null
hotfix-vars:
$(eval LDFLAGS := $(shell MINIO_RELEASE="RELEASE" MINIO_HOTFIX="hotfix.$(shell git rev-parse --short HEAD)" go run buildscripts/gen-ldflags.go $(shell git describe --tags --abbrev=0 | \
sed 's#RELEASE\.\([0-9]\+\)-\([0-9]\+\)-\([0-9]\+\)T\([0-9]\+\)-\([0-9]\+\)-\([0-9]\+\)Z#\1-\2-\3T\4:\5:\6Z#')))
$(eval VERSION := $(shell git describe --tags --abbrev=0).hotfix.$(shell git rev-parse --short HEAD))
docker: checks build-debugging ## builds the local Linux Silo container image
@echo "Building Silo container image '$(TAG)'"
@set -e; \
context=$$(mktemp -d); \
trap 'rm -rf "$$context"' EXIT; \
CGO_ENABLED=0 GOOS=linux GOARCH=$(GOARCH) go build -tags kqueue -trimpath \
--ldflags "$(LDFLAGS)" -o "$$context/silo"; \
mkdir -p "$$context/dockerscripts"; \
cp Dockerfile.goreleaser LICENSE NOTICE CREDITS "$$context/"; \
cp dockerscripts/docker-entrypoint.sh dockerscripts/build-static-curl.sh \
"$$context/dockerscripts/"; \
docker build -q --no-cache --platform linux/$(GOARCH) -t $(TAG) --build-arg TARGETARCH=$(GOARCH) \
-f "$$context/Dockerfile.goreleaser" "$$context"
hotfix: hotfix-vars clean install ## builds minio binary with hotfix tags
@wget -q -c https://github.com/minio/pkger/releases/download/v2.3.11/pkger_2.3.11_linux_amd64.deb
@wget -q -c https://raw.githubusercontent.com/minio/minio-service/v1.1.1/linux-systemd/distributed/minio.service
@sudo apt install ./pkger_2.3.11_linux_amd64.deb --yes
@mkdir -p minio-release/$(GOOS)-$(GOARCH)/archive
@cp -af ./minio minio-release/$(GOOS)-$(GOARCH)/minio
@cp -af ./minio minio-release/$(GOOS)-$(GOARCH)/minio.$(VERSION)
@minisign -qQSm minio-release/$(GOOS)-$(GOARCH)/minio.$(VERSION) -s "${CRED_DIR}/minisign.key" < "${CRED_DIR}/minisign-passphrase"
@sha256sum < minio-release/$(GOOS)-$(GOARCH)/minio.$(VERSION) | sed 's, -,minio.$(VERSION),g' > minio-release/$(GOOS)-$(GOARCH)/minio.$(VERSION).sha256sum
@cp -af minio-release/$(GOOS)-$(GOARCH)/minio.$(VERSION)* minio-release/$(GOOS)-$(GOARCH)/archive/
@pkger -r $(VERSION) --ignore
hotfix-push: hotfix
@scp -q -r minio-release/$(GOOS)-$(GOARCH)/* minio@dl-0.minio.io:~/releases/server/minio/hotfixes/linux-$(GOOS)/
@scp -q -r minio-release/$(GOOS)-$(GOARCH)/* minio@dl-0.minio.io:~/releases/server/minio/hotfixes/linux-$(GOOS)/archive
@scp -q -r minio-release/$(GOOS)-$(GOARCH)/* minio@dl-1.minio.io:~/releases/server/minio/hotfixes/linux-$(GOOS)/
@scp -q -r minio-release/$(GOOS)-$(GOARCH)/* minio@dl-1.minio.io:~/releases/server/minio/hotfixes/linux-$(GOOS)/archive
@echo "Published new hotfix binaries at https://dl.min.io/server/minio/hotfixes/linux-$(GOOS)/archive/minio.$(VERSION)"
docker-hotfix-push: docker-hotfix
@docker push -q $(TAG) && echo "Published new container $(TAG)"
docker-hotfix: hotfix-push checks ## builds minio docker container with hotfix tags
@echo "Building minio docker image '$(TAG)'"
@docker build -q --no-cache -t $(TAG) --build-arg RELEASE=$(VERSION) . -f Dockerfile.hotfix
docker: build ## builds minio docker container
@echo "Building minio docker image '$(TAG)'"
@docker build -q --no-cache -t $(TAG) . -f Dockerfile
docker-distroless: checks build-debugging ## builds the local Linux Silo distroless container image
@echo "Building Silo distroless container image '$(TAG)-distroless'"
@set -e; \
context=$$(mktemp -d); \
trap 'rm -rf "$$context"' EXIT; \
CGO_ENABLED=0 GOOS=linux GOARCH=$(GOARCH) go build -tags kqueue -trimpath \
--ldflags "$(LDFLAGS)" -o "$$context/silo"; \
cp Dockerfile.distroless LICENSE NOTICE CREDITS "$$context/"; \
docker build -q --no-cache --platform linux/$(GOARCH) -t $(TAG)-distroless \
-f "$$context/Dockerfile.distroless" "$$context"
test-resiliency: build
@echo "Running resiliency tests"
@(DOCKER_COMPOSE_FILE=$(PWD)/docs/resiliency/docker-compose.yaml env bash $(PWD)/docs/resiliency/resiliency-tests.sh)
install-race: checks build-debugging ## builds minio to $(PWD)
@echo "Building minio binary with -race to './minio'"
@GORACE=history_size=7 CGO_ENABLED=1 go build -tags kqueue,dev -race -trimpath --ldflags "$(LDFLAGS)" -o $(PWD)/minio 1>/dev/null
@echo "Installing minio binary with -race to '$(GOPATH)/bin/minio'"
@mkdir -p $(GOPATH)/bin && cp -af $(PWD)/minio $(GOPATH)/bin/minio
install-race: checks build-debugging ## builds Silo to $(PWD)
@echo "Building Silo binary with -race to './silo'"
@GORACE=history_size=7 CGO_ENABLED=1 go build -tags kqueue,dev -race -trimpath --ldflags "$(LDFLAGS)" -o $(PWD)/silo 1>/dev/null
@echo "Installing Silo binary with -race to '$(GOPATH)/bin/silo'"
@mkdir -p $(GOPATH)/bin && cp -af $(PWD)/silo $(GOPATH)/bin/silo
install: build ## builds minio and installs it to $GOPATH/bin.
@echo "Installing minio binary to '$(GOPATH)/bin/minio'"
@mkdir -p $(GOPATH)/bin && cp -af $(PWD)/minio $(GOPATH)/bin/minio
@echo "Installation successful. To learn more, try \"minio --help\"."
install: build ## builds Silo and installs it to $GOPATH/bin.
@echo "Installing Silo binary to '$(GOPATH)/bin/silo'"
@mkdir -p $(GOPATH)/bin && cp -af $(PWD)/silo $(GOPATH)/bin/silo
@echo "Installation successful. To learn more, try \"silo --help\"."
clean: ## cleanup all generated assets
@echo "Cleaning up all the generated files"
@@ -254,10 +253,8 @@ clean: ## cleanup all generated assets
@find . -name '*~' | xargs rm -fv
@find . -name '.#*#' | xargs rm -fv
@find . -name '#*#' | xargs rm -fv
@rm -rvf minio
@rm -rvf silo
@rm -rvf build
@rm -rvf release
@rm -rvf .verify*
@rm -rvf minio-release
@rm -rvf minio.RELEASE*.hotfix.*
@rm -rvf pkger_*.deb
+7 -1
View File
@@ -1,4 +1,4 @@
MinIO Project, (C) 2015-2023 MinIO, Inc.
MinIO Project, (C) 2015-2025 MinIO, Inc.
This product includes software developed at MinIO, Inc.
(https://min.io/).
@@ -7,3 +7,9 @@ The MinIO project contains unmodified/modified subcomponents too with
separate copyright notices and license terms. Your use of the source
code for these subcomponents is subject to the terms and conditions
of GNU Affero General Public License 3.0.
Silo Project modifications, (C) 2025-2026 PGSTY.
Silo is an independent community-maintained project incorporating MinIO
source code. It is not affiliated with or endorsed by MinIO, Inc. Modified
source and Silo release artifacts are maintained by the Silo project.
+12 -12
View File
@@ -1,12 +1,12 @@
# MinIO Pull Request Guidelines
# Silo Pull Request Guidelines
These guidelines ensure high-quality commits in MinIO’s GitHub repositories, maintaining
a clear, valuable commit history for our open-source projects. They apply to all contributors,
These guidelines ensure high-quality commits in Silo's GitHub repositories, maintaining
a clear, valuable commit history for our open-source projects. They apply to all contributors,
fostering efficient reviews and robust code.
## Why Pull Requests?
Pull Requests (PRs) drive quality in MinIO’s codebase by:
Pull Requests (PRs) drive quality in Silo's codebase by:
- Enabling peer review without pair programming.
- Documenting changes for future reference.
- Ensuring commits tell a clear story of development.
@@ -15,12 +15,12 @@ Pull Requests (PRs) drive quality in MinIO’s codebase by:
## Crafting a Quality PR
A strong MinIO PR:
A strong Silo PR:
- Delivers a complete, valuable change (feature, bug fix, or improvement).
- Has a concise title (e.g., `[S3] Fix bucket policy parsing #1234`) and a summary with context, referencing issues (e.g., `#1234`).
- Contains well-written, logical commits explaining *why* changes were made (e.g., “Add S3 bucket tagging support so that users can organize resources efficiently”).
- Is small, focused, and easy to review—ideally one commit, unless multiple commits better narrate complex work.
- Adheres to MinIO’s coding standards (e.g., Go style, error handling, testing).
- Adheres to Silo's coding standards (e.g., Go style, error handling, testing).
PRs must flow smoothly through review to reach production. Large PRs should be split into smaller, manageable ones.
@@ -48,14 +48,14 @@ PRs must flow smoothly through review to reach production. Large PRs should be s
## Reviewing PRs
Reviewers ensure MinIO’s commit history remains a clear, reliable record. Responsibilities include:
Reviewers ensure Silo's commit history remains a clear, reliable record. Responsibilities include:
1. **Commit Quality**:
- Verify each commit explains *why* the change was made (e.g., “So that…”).
- Request rebasing if commits are unclear, redundant, or lack context (e.g., “Please squash typo fixes into the parent commit”).
2. **Code Quality**:
- Check adherence to MinIO’s Go standards (e.g., error handling, documentation).
- Check adherence to Silo's Go standards (e.g., error handling, documentation).
- Ensure tests cover new code and pass CI.
- Flag bugs or critical issues for immediate fixes; suggest non-blocking improvements as follow-up issues.
@@ -65,7 +65,7 @@ Reviewers ensure MinIO’s commit history remains a clear, reliable record. Resp
- If unable to complete the review, tag another reviewer (e.g., `@username please take over`).
4. **Shared Responsibility**:
- All MinIO contributors are reviewers. The first commenter on a PR owns the review unless they delegate.
- All Silo contributors are reviewers. The first commenter on a PR owns the review unless they delegate.
- Multiple reviewers are encouraged for complex PRs.
5. **No Self-Edits**:
@@ -80,14 +80,14 @@ Reviewers ensure MinIO’s commit history remains a clear, reliable record. Resp
- **Small PRs**: Easier to review, faster to merge. Split large changes logically.
- **Clear Commits**: Use `git rebase -i` to refine history before submitting.
- **Engage Early**: Discuss complex changes in issues or Slack (https://slack.min.io) before coding.
- **Engage Early**: Discuss complex changes in a GitHub issue before coding.
- **Be Responsive**: Address reviewer feedback promptly to keep PRs moving.
- **Learn from Reviews**: Use feedback to improve future contributions.
## Resources
- [MinIO Coding Standards](https://github.com/minio/minio/blob/master/CONTRIBUTING.md)
- [Silo Contribution Guide](CONTRIBUTING.md)
- [Effective Commit Messages](https://mislav.net/2014/02/hidden-documentation/)
- [GitHub PR Tips](https://github.com/blog/1943-how-to-write-the-perfect-pull-request)
By following these guidelines, we ensure MinIO’s codebase remains high-quality, maintainable, and a joy to contribute to. Happy coding!
By following these guidelines, we ensure Silo's codebase remains high-quality, maintainable, and a joy to contribute to. Happy coding!
+122 -103
View File
@@ -1,156 +1,175 @@
<h1 align="center">
<img src=".github/silo-word.svg" alt="SILO" height="80">
<a href="https://silo.pgsty.com/">
<img src=".github/silo-logo.svg" alt="Silo" width="160">
</a>
</h1>
<p align="center">
<strong>A conservatively maintained MinIO fork</strong><br>
Security maintenance, versioned release artifacts, and operational continuity for existing deployments.
<strong>S3-compatible object storage — a MinIO fork maintained by PGSTY</strong>
</p>
<p align="center">
<a href="https://silo.pgsty.com/">Website</a> ·
<a href="https://silo.pgsty.com/docs/">Documentation</a> ·
<a href="https://silo.pgsty.com/download/">Download</a> ·
<a href="https://silo.pgsty.com/blog/">Blog</a> ·
<a href="https://github.com/pgsty/minio/releases">Releases</a> ·
<a href="https://silo.pgsty.com/tags/silo/">Release Notes</a> ·
<a href="https://silo.pgsty.com/compatibility/server/">Compatibility</a> ·
<a href="https://silo.pgsty.com/about/manifesto/">Manifesto</a> ·
<a href="SECURITY.md">Security</a> ·
<a href="README_ZH.md">中文</a>
</p>
<p align="center">
<a href="https://github.com/pgsty/minio/releases"><img alt="GitHub Release" src="https://img.shields.io/github/v/release/pgsty/minio?include_prereleases&label=release&logo=github"></a>
<a href="https://hub.docker.com/r/pgsty/minio"><img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/pgsty/minio?logo=docker"></a>
<a href="go.mod"><img alt="Go Version" src="https://img.shields.io/github/go-mod/go-version/pgsty/minio?logo=go"></a>
<a href="https://silo.pgsty.com/"><img alt="Website" src="https://img.shields.io/badge/Website-silo.pgsty.com-1d588c"></a>
<a href="https://github.com/pgsty/silo/releases"><img alt="GitHub Release" src="https://img.shields.io/github/v/release/pgsty/silo?include_prereleases&label=release&logo=github"></a>
<a href="https://hub.docker.com/r/pgsty/silo"><img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/pgsty/minio?logo=docker"></a>
<a href="go.mod"><img alt="Go Version" src="https://img.shields.io/github/go-mod/go-version/pgsty/silo?logo=go"></a>
<a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/license-AGPLv3-blue"></a>
</p>
> [!IMPORTANT]
> Silo is an independent, community-maintained fork of the open-source MinIO server, published by [Pigsty](https://pigsty.io) from [`pgsty/minio`](https://github.com/pgsty/minio). It is not affiliated with, endorsed by, or sponsored by MinIO, Inc. “MinIO” is used only to identify the upstream project and compatibility lineage.
> **PGSTY Silo** (hereinafter “Silo”) is an independent, community-maintained fork of the open-source MinIO server, published by [Pigsty](https://pigsty.io) from [`pgsty/silo`](https://github.com/pgsty/silo). It is not affiliated with, endorsed by, or sponsored by MinIO, Inc. “MinIO” is used only to identify the upstream project and compatibility lineage.
> [!NOTE]
> Renamed from `pgsty/minio` to `pgsty/silo`, default branch `master` → `main`, on 2026-08-06. Artifacts under the original MinIO identity stay published on the archived [`minio`](https://github.com/pgsty/silo/tree/minio) branch and in releases up to [`RELEASE.2026-08-04T00-00-00Z`](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-08-04T00-00-00Z).
## Current release and main branch
The latest published Server is [20260903](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-09-03T13-18-01Z).
As of 2026-09-16, the main branch has newer security, storage, Console and
shared-package changes that have not shipped in a Server release. See
[CHANGELOG.md](CHANGELOG.md) and the [component version matrix](https://silo.pgsty.com/compatibility/versions/)
for the exact release/source boundary, including SN-2026-011 and password-policy migration.
## Overview
Silo maintains one downstream release line based on MinIO [`RELEASE.2025-12-03T12-00-00Z`](https://github.com/minio/minio/releases/tag/RELEASE.2025-12-03T12-00-00Z). It provides maintained builds and release artifacts for existing MinIO-compatible deployments after upstream community distribution ended. Pigsty uses this fork for object storage as an optional PG backup repo.
PGSTY SILO keeps one maintained release line of the open-source MinIO server alive after upstream ended community distribution: builds, packages, multi-arch images, security fixes, and the full web console. Pigsty runs it in production as its PostgreSQL backup repository.
The official project portal is [silo.pgsty.com](https://silo.pgsty.com/). It brings documentation, downloads, release and security notes, and project background together. English is served at the site root; Chinese is available under [/zh/](https://silo.pgsty.com/zh/).
It follows one rule — **the product and its delivery surfaces are renamed; the protocol and your data are not.** Everything else lives on [silo.pgsty.com](https://silo.pgsty.com/).
## Find the Right Resource
**Related:** [`pgsty/mc`](https://github.com/pgsty/mc) client (shipped as `mcli`) · [`pgsty/silo-console`](https://github.com/pgsty/silo-console) · [`pgsty/silo-pkg`](https://github.com/pgsty/silo-pkg) · [`pgsty/pigsty`](https://github.com/pgsty/pigsty)
| Looking for | Canonical location |
| :-- | :-- |
| Project overview and navigation | [Silo Website](https://silo.pgsty.com/) ([中文](https://silo.pgsty.com/zh/)) |
| Installation methods and downloads | [Download & Install](https://silo.pgsty.com/download/) ([中文](https://silo.pgsty.com/zh/download/)) |
| Operations, administration, development, and reference | [Documentation](https://silo.pgsty.com/docs/) ([中文](https://silo.pgsty.com/zh/docs/)) |
| Project news, release notes, and security notes | [Blog](https://silo.pgsty.com/blog/), including [releases](https://silo.pgsty.com/blog/release/) and [security](https://silo.pgsty.com/blog/security/) |
| Versioned binaries, checksums, and source archives | [GitHub Releases](https://github.com/pgsty/minio/releases) |
| Bug reports and feature discussions | [GitHub Issues](https://github.com/pgsty/minio/issues) |
| License, attribution, and trademark information | [License](https://silo.pgsty.com/about/license/), [Attribution](https://silo.pgsty.com/about/attribution/), and [Trademark](https://silo.pgsty.com/about/trademark/) |
## Maintenance Policy
The active release line covers:
- build and dependency maintenance;
- applicable security fixes and advisories;
- focused fixes for reproducible defects;
- versioned binaries, packages, checksums, and multi-architecture images;
- the web console, client, documentation, and Pigsty integration.
Changes are kept narrow and tested where practical. Maintenance is best effort; no response, remediation, or release schedule is guaranteed.
### Out of scope
- a separate product roadmap, new storage engine, or speculative S3 features;
- broad rewrites or changes that materially expand the downstream delta;
- historical releases or multiple support branches;
- commercial support, SLAs, 24×7 coverage, or SUBNET access;
- deployment design, access control, monitoring, backup, or recovery.
## Compatibility
Silo aims to preserve:
- MinIO-compatible S3 APIs, configuration, environment variables, and CLI conventions;
- `RELEASE.YYYY-MM-DDTHH-MM-SSZ` tags, container entrypoints, and common deployment workflows.
Compatibility is the default constraint. Silo preserves existing wire, client, configuration, and operational behavior whenever doing so remains safe. Compatibility is broken only when necessary to close a major security issue, and the release notes must identify the affected behavior and migration path. Treat each release as a downstream upgrade: pin versions, review [release notes](https://silo.pgsty.com/blog/release/) and [security advisories](docs/security/advisories.md), keep a rollback path, and test before production use.
## Downloads and Release Artifacts
Use [Download & Install](https://silo.pgsty.com/download/) to choose an installation method. GitHub Releases remains the source for versioned server binaries, checksums, and source archives.
| Artifact | Location |
| :-- | :-- |
| Source | [`github.com/pgsty/minio`](https://github.com/pgsty/minio) |
| Container image | [`pgsty/minio`](https://hub.docker.com/r/pgsty/minio), multi-arch for `linux/amd64` and `linux/arm64` |
| Server binaries and checksums | [GitHub Releases](https://github.com/pgsty/minio/releases) for Linux, macOS, and Windows on `amd64` and `arm64` |
| Linux packages | RPM, DEB, and APK artifacts, also distributed through the [Pigsty repository](https://pigsty.io/docs/repo/) |
| Client | [`pgsty/mc`](https://github.com/pgsty/mc), bundled in the container as `mcli` with an `mc` compatibility alias |
| Console | Maintained [`georgmangold/console`](https://github.com/georgmangold/console) fork, embedded in the server build |
| Shared library | [`pgsty/silo-pkg`](https://github.com/pgsty/silo-pkg) v3.7.0, consumed through a `replace` directive while preserving `github.com/minio/pkg/v3` import paths ([release notes](https://silo.pgsty.com/blog/release/pkg-3.7.0/)) |
<p align="center">
<img src="https://silo.pgsty.com/images/silo-console/console-metrics-simple.webp" alt="Silo Console">
</p>
## Quick Start
For local evaluation:
```bash
mkdir -p data
export MINIO_ROOT_USER=minioadmin
export MINIO_ROOT_PASSWORD=change-me-long-password
docker run -d --name silo \
-p 9000:9000 \
-p 9001:9001 \
-e MINIO_ROOT_USER \
-e MINIO_ROOT_PASSWORD \
docker run -d --name silo -p 9000:9000 -p 9001:9001 \
-e MINIO_ROOT_USER=minioadmin \
-e MINIO_ROOT_PASSWORD=change-me-long-password \
-v "$PWD/data:/data" \
pgsty/minio:latest server /data --console-address ":9001"
docker.io/pgsty/silo:latest server /data --console-address ":9001"
```
Open the console at <http://localhost:9001>; the S3 API listens on <http://localhost:9000>.
<p align="center">
<img src="https://silo.pgsty.com/images/silo-console/console-login.webp" alt="Silo Console">
</p>
The image includes the compatible client as `mcli`:
Console on <http://localhost:9001>, S3 API on <http://localhost:9000>. The image bundles the client as `mcli`:
```bash
docker exec silo mcli alias set local http://127.0.0.1:9000 \
"$MINIO_ROOT_USER" "$MINIO_ROOT_PASSWORD"
docker exec silo mcli mb local/demo
docker exec silo mcli ls local
docker exec silo mcli alias set local http://127.0.0.1:9000 minioadmin change-me-long-password
docker exec silo mcli mb local/demo && docker exec silo mcli ls local
```
> [!WARNING]
> For production, pin a release, use unique credentials and TLS, monitor the service, keep independent backups, and test recovery.
> For production, pin a release, use unique credentials and TLS, monitor the service, keep independent backups, and test recovery. Start from the [documentation](https://silo.pgsty.com/docs/).
Build the server from source:
## Install
```bash
go build -o minio .
./minio --version
```
| Method | Where |
| :-- | :-- |
| Container | [`pgsty/silo`](https://hub.docker.com/r/pgsty/silo), multi-arch for `linux/amd64` and `linux/arm64` |
| Binaries | [GitHub Releases](https://github.com/pgsty/silo/releases) — Linux, macOS, Windows on `amd64` and `arm64` |
| Packages | RPM, DEB, and APK, also via the [Pigsty repository](https://pigsty.io/docs/repo/) |
| Kubernetes | Helm chart, see [Download & Install](https://silo.pgsty.com/download/) |
| Source | `go build -o silo . && ./silo --version` |
For other installation paths—including native packages, binaries, Podman, Kubernetes, source, and Pigsty Ansible—use [Download & Install](https://silo.pgsty.com/download/). For production deployment and administration, start with the [Silo documentation](https://silo.pgsty.com/docs/). Pigsty users can also use the [Pigsty MinIO module](https://pigsty.io/docs/minio/).
Every release ships checksums, SPDX SBOMs, Sigstore-signed manifests, and GitHub build attestations. Installation methods and verification commands are documented at [Download & Install](https://silo.pgsty.com/download/); migrating from upstream MinIO — taking over an existing `minio.service` and its `/etc/default/minio`, and keeping data ownership stable with a `/etc/systemd/system/silo.service.d/10-legacy-user.conf` drop-in — is covered by the [migration guide](https://silo.pgsty.com/compatibility/migration/) and the [binary & service notes](https://silo.pgsty.com/compatibility/binary/).
## Security
## Compatibility
Security fixes target the active `master` branch and are recorded in the [advisory log](docs/security/advisories.md) and the portal's [security notes](https://silo.pgsty.com/blog/security/). Report vulnerabilities privately as described in [`SECURITY.md`](SECURITY.md) and [`VULNERABILITY_REPORT.md`](VULNERABILITY_REPORT.md). Report issues that also affect upstream MinIO there as well.
Silo preserves S3 and storage-format compatibility, including existing `MINIO_*` variables, `minio_*` metrics, `x-minio-*` headers, `/minio/*` routes, and `.minio.sys` data. CI guards selected compatibility identifiers; release notes document intentional security and behavior changes. Silo-owned delivery surfaces use the `silo` executable, package, service, Helm chart, and container image; no `minio` server binary alias is installed.
## Contributing
The supported release stack is `pgsty/silo` + `pgsty/silo-console` + `pgsty/mc` + `pgsty/silo-pkg`; compatibility with unmodified upstream MinIO/MC is best effort. The Server, Console, and client retain their historical module paths where needed, while maintained code imports `github.com/pgsty/silo-pkg/v3` directly. The SDK `github.com/minio/minio-go/v7` is an explicit upstream dependency. See the current [go.mod](go.mod) for versions and replacements.
Useful contributions include security and dependency updates, reproducible bug fixes, tests, release automation, packaging, and documentation.
Every divergence from upstream is listed in the code-verified [compatibility audit](https://silo.pgsty.com/compatibility/server/). Treat each release as a downstream upgrade: pin versions, read the [release notes](https://silo.pgsty.com/tags/silo/), and keep a rollback path.
Issues and pull requests should include the affected version, reproduction steps, impact, expected behavior, tests, and compatibility notes. Discuss large changes in an issue first.
### TLS and Go upgrades
The following TLS repair is on main and is not included in Server 20260903.
With that repair, TLS key exchange follows Go's defaults across the S3 listener, node links,
replication, identity providers, etcd, and external HTTP services. If an endpoint
cannot accept ML-KEM, `GODEBUG=tlsmlkem=0` disables the default hybrid exchanges
for the process; certificate verification remains enabled. This option does not
disable ML-DSA signatures or resolve every TLS reset. Prefer updating the
incompatible endpoint before removing the temporary setting.
If only the new SecP hybrids cause problems, `GODEBUG=tlssecpmlkem=0` disables
those groups while retaining X25519MLKEM768.
For builds targeting Go 1.27, setting either `SSL_CERT_FILE` or `SSL_CERT_DIR`
on macOS replaces Keychain trust with on-disk roots and Go's verifier. Stale or
incomplete CA paths can break previously trusted connections; unset inherited
values to restore Keychain trust. Explicit certificates in the configured `CAs`
directory remain additive to the selected root pool.
Go 1.27 binaries require macOS 13 or later. See the
[Go release notes](https://go.dev/doc/go1.27) and the
[Go 1.27 TLS and OIDC discovery guide](https://silo.pgsty.com/blog/design/go127-tls-oidc-discovery/).
## Documentation ownership
User documentation is maintained at [silo.pgsty.com](https://silo.pgsty.com/docs/),
with source in [pgsty/silo.pgsty.com](https://github.com/pgsty/silo.pgsty.com).
The remaining `docs/` tree contains inherited references, examples, and tooling
fixtures. Investigation logs, AI work records, and temporary reports are kept
outside this repository; reusable findings belong in the companion site.
See [AGENTS.md](AGENTS.md) for repository ownership and maintenance rules.
## Security & Contributing
Report vulnerabilities privately as described in [`SECURITY.md`](SECURITY.md); every fix ships with a public [advisory](https://silo.pgsty.com/blog/security/). Contributions are accepted inbound=outbound under AGPL-3.0-or-later with no CLA — only DCO sign-off (`git commit -s`) is required; see [`CONTRIBUTING.md`](CONTRIBUTING.md).
## Contributors
<!-- Generated by silo.pgsty.com/bin/contributors.py. -->
Every human issue or pull-request author is part of the SILO community, including open and unmerged work. Merged fixes, adopted proposals, and actionable reports receive priority, with first participation guiding the remaining order. Gold rings highlight reviewed significant contributions.
<a href="CONTRIBUTORS.md">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/pgsty/silo/codex/repository-cards/contributors-dark.svg">
<img src="https://raw.githubusercontent.com/pgsty/silo/codex/repository-cards/contributors-light.svg" alt="SILO community contributors">
</picture>
</a>
[View contribution notes and actual PR status](CONTRIBUTORS.md).
## Star History
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/pgsty/silo/codex/repository-cards/star-history-dark.svg">
<img src="https://raw.githubusercontent.com/pgsty/silo/codex/repository-cards/star-history-light.svg" alt="SILO GitHub star history">
</picture>
## Background
This project was created in response to changes in the upstream community distribution and maintenance model. The maintainer’s analysis, alternatives considered, and early maintenance record are documented below:
Upstream wound down its community edition: the web console was cut back to a stub, prebuilt community binaries stopped, and the community repository was archived. Silo exists to keep those deployments running. The fork is a means, not an identity — if upstream restores its community edition, we will narrow our scope and offer the fixes back.
| Essay | Subject |
| :-- | :-- |
| [MinIO Is Dead](https://silo.pgsty.com/blog/post/minio-is-dead/) | Changes to the upstream project and distribution model |
| [MinIO Is Dead, Long Live MinIO](https://silo.pgsty.com/blog/post/minio-resurrect/) | Establishing the fork and its release pipeline |
| [Two months into maintaining a MinIO fork](https://silo.pgsty.com/blog/post/minio-promise-kept/) | Initial security and maintenance work |
The [**Manifesto**](https://silo.pgsty.com/about/manifesto/) is the project's public commitment in eleven articles, under one discipline: every article is either something already done with public evidence, or something explicitly refused. In short:
## License and Trademark
- **Compatibility contract** — the protocol and your data do not change, and every release documents its tested rollback target and path.
- **The license cannot change** — AGPLv3, no CLA, no copyright aggregation; nobody here, ourselves included, holds enough copyright to relicense on everyone else's behalf.
- **The never list**, append-only — no paywalling existing features, no registration wall on downloads, no telemetry (upstream's phone-home paths are removed outright), no CLA, no license change, no trademark enforcement against normal use.
- **Security and release discipline** — a public advisory for every fix, and a release every one to two months, at most a quarter apart. Judge both against the public record.
The server remains licensed under the [GNU Affero General Public License v3.0](LICENSE). See [`CREDITS`](CREDITS) for upstream authorship and attribution. MinIO is a trademark of MinIO, Inc. Silo and `pgsty/minio` are independent community efforts and are not affiliated with or endorsed by MinIO, Inc.
Essays: [MinIO Is Dead](https://silo.pgsty.com/blog/post/minio-is-dead/) · [Who Takes Over?](https://silo.pgsty.com/blog/post/minio-alternative/) · [Long Live MinIO](https://silo.pgsty.com/blog/post/minio-resurrect/) · [Promise Kept](https://silo.pgsty.com/blog/post/minio-promise-kept/)
## License & Trademark
Silo is [AGPL-3.0-or-later](LICENSE), derived from [`minio/minio`](https://github.com/minio/minio) with upstream copyright and third-party notices preserved in [`NOTICE`](NOTICE) and [`CREDITS`](CREDITS). MinIO is a trademark of MinIO, Inc.; the name is used here only to identify the upstream project and compatibility lineage.
Details: [license](https://silo.pgsty.com/about/license/) · [attribution](https://silo.pgsty.com/about/attribution/) · [trademark](https://silo.pgsty.com/about/trademark/)
+104 -110
View File
@@ -1,163 +1,157 @@
<h1 align="center">
<img src=".github/silo.svg" alt="" height="80">
<img src=".github/silo-word.svg" alt="SILO" height="80">
<a href="https://silo.pgsty.com/zh/">
<img src=".github/silo-logo.svg" alt="Silo" width="160">
</a>
</h1>
<p align="center">
<strong>审慎维护的 MinIO 社区分支</strong><br>
为现有部署提供安全维护、带版本的发行产物与持续运维支持。
<strong>S3 兼容对象存储 —— 由 PGSTY 维护的 MinIO 社区分支</strong>
</p>
<p align="center">
<a href="https://silo.pgsty.com/zh/">官网</a> ·
<a href="https://silo.pgsty.com/zh/docs/">文档</a> ·
<a href="https://silo.pgsty.com/zh/download/">下载</a> ·
<a href="https://silo.pgsty.com/zh/blog/">博客</a> ·
<a href="https://github.com/pgsty/minio/releases">版本发布</a> ·
<a href="https://silo.pgsty.com/zh/tags/silo/">版本说明</a> ·
<a href="https://silo.pgsty.com/zh/compatibility/server/">兼容性</a> ·
<a href="https://silo.pgsty.com/zh/about/manifesto/">宣言</a> ·
<a href="SECURITY.md">安全策略</a> ·
<a href="README.md">English</a>
</p>
<p align="center">
<a href="https://github.com/pgsty/minio/releases"><img alt="GitHub Release" src="https://img.shields.io/github/v/release/pgsty/minio?include_prereleases&label=release&logo=github"></a>
<a href="https://hub.docker.com/r/pgsty/minio"><img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/pgsty/minio?logo=docker"></a>
<a href="go.mod"><img alt="Go Version" src="https://img.shields.io/github/go-mod/go-version/pgsty/minio?logo=go"></a>
<a href="https://silo.pgsty.com/zh/"><img alt="官网" src="https://img.shields.io/badge/%E5%AE%98%E7%BD%91-silo.pgsty.com%2Fzh-1d588c"></a>
<a href="https://github.com/pgsty/silo/releases"><img alt="GitHub Release" src="https://img.shields.io/github/v/release/pgsty/silo?include_prereleases&label=release&logo=github"></a>
<a href="https://hub.docker.com/r/pgsty/silo"><img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/pgsty/minio?logo=docker"></a>
<a href="go.mod"><img alt="Go Version" src="https://img.shields.io/github/go-mod/go-version/pgsty/silo?logo=go"></a>
<a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/license-AGPLv3-blue"></a>
</p>
> [!IMPORTANT]
> Silo 是由 [Pigsty](https://pigsty.cc) 独立维护、从 [`pgsty/minio`](https://github.com/pgsty/minio) 发布的开源 MinIO 社区分支。本项目与 MinIO, Inc. 不存在隶属、背书或赞助关系;文中使用 “MinIO” 仅用于说明上游项目及兼容谱系。
> **PGSTY Silo**(以下简称 Silo)是由 [Pigsty](https://pigsty.cc) 独立维护、从 [`pgsty/silo`](https://github.com/pgsty/silo) 发布的开源 MinIO 社区分支。本项目与 MinIO, Inc. 不存在隶属、背书或赞助关系;文中使用 “MinIO” 仅用于说明上游项目及兼容谱系。
> [!NOTE]
> 2026-08-06,本仓库由 `pgsty/minio` 更名为 `pgsty/silo`,默认分支由 `master` 更名为 `main`。以原 MinIO 形态维持的归档构件仍位于归档的 [`minio`](https://github.com/pgsty/silo/tree/minio) 分支,以及截止 [`RELEASE.2026-08-04T00-00-00Z`](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-08-04T00-00-00Z) 的历次发布中。
## 当前发行版与主分支
最新已发布的 Server 仍为 [20260903](https://github.com/pgsty/silo/releases/tag/RELEASE.2026-09-03T13-18-01Z)。
截至 2026-09-16,主分支已合入更新的安全、存储、Console 与共享包改动,但尚未发布新 Server。
准确的已发布/源码边界见 [CHANGELOG.md](CHANGELOG.md) 与[组件版本矩阵](https://silo.pgsty.com/zh/compatibility/versions/),
其中包括 SN-2026-011 修复状态与密码权限迁移要求。
## 概述
Silo 维护一条基于 MinIO [`RELEASE.2025-12-03T12-00-00Z`](https://github.com/minio/minio/releases/tag/RELEASE.2025-12-03T12-00-00Z) 的下游版本线,为上游停止社区发行后仍在运行 MinIO 兼容部署的用户提供持续构建与发行产物。
上游停止社区发行后,Silo 为开源 MinIO 服务端维护一条持续可用的版本线:构建、软件包、多架构镜像、安全修复与完整 Web 控制台。Pigsty 在生产环境中用它承载 PostgreSQL 备份存储。
Pigsty 使用本分支提供对象存储,包括 PostgreSQL 备份存储。
它只遵循一条原则:**改名的是产品与交付物,不是协议与你的数据。** 其余内容都在 [silo.pgsty.com](https://silo.pgsty.com/zh/)。
项目统一门户为 [silo.pgsty.com](https://silo.pgsty.com/zh/),集中提供文档、下载安装、版本与安全动态及项目背景。中文内容位于 `/zh/`,英文内容位于站点根路径。
**相关项目:**[`pgsty/mc`](https://github.com/pgsty/mc) 客户端(以 `mcli` 发行) · [`pgsty/silo-console`](https://github.com/pgsty/silo-console) · [`pgsty/silo-pkg`](https://github.com/pgsty/silo-pkg) · [`pgsty/pigsty`](https://github.com/pgsty/pigsty)
## 按需求选择入口
<p align="center">
<img src="https://silo.pgsty.com/images/silo-console/console-metrics-simple.webp" alt="Silo 控制台">
</p>
| 需求 | 权威入口 |
| :-- | :-- |
| 项目概览与全站导航 | [Silo 中文门户](https://silo.pgsty.com/zh/)([English](https://silo.pgsty.com/)) |
| 安装方式与软件下载 | [下载与安装](https://silo.pgsty.com/zh/download/)([English](https://silo.pgsty.com/download/)) |
| 运维、管理、开发与参考指南 | [中文文档](https://silo.pgsty.com/zh/docs/)([English](https://silo.pgsty.com/docs/)) |
| 项目动态、版本说明与安全通告 | [博客](https://silo.pgsty.com/zh/blog/),包括[版本发布](https://silo.pgsty.com/zh/blog/release/)与[安全通告](https://silo.pgsty.com/zh/blog/security/) |
| 带版本的二进制、校验和与源码归档 | [GitHub Releases](https://github.com/pgsty/minio/releases) |
| 缺陷报告与功能讨论 | [GitHub Issues](https://github.com/pgsty/minio/issues) |
| 私密漏洞报告 | [`SECURITY.md`](SECURITY.md) 与 [`VULNERABILITY_REPORT.md`](VULNERABILITY_REPORT.md) |
| 许可证、署名与商标信息 | [许可证](https://silo.pgsty.com/zh/about/license/)、[署名归属](https://silo.pgsty.com/zh/about/attribution/)与[商标政策](https://silo.pgsty.com/zh/about/trademark/) |
## 维护政策
活跃版本线的维护范围包括:
- 构建与依赖项维护;
- 适用的安全修复与公告;
- 针对可复现缺陷的范围明确的修复;
- 带版本的二进制、软件包、校验和与多架构镜像;
- Web Console、客户端、文档与 Pigsty 集成。
改动保持克制,并在可行时提供测试。所有维护均为尽力而为,不承诺固定的响应、修复或发布时间。
### 范围之外
- 独立产品路线图、新存储引擎或假设性的 S3 新特性;
- 大规模重写或显著扩大下游差异的改动;
- 历史版本或多条支持分支;
- 商业支持、SLA、7×24 服务或 SUBNET 服务;
- 部署设计、访问控制、监控、备份与恢复。
## 兼容策略
Silo 尽量保留:
- `minio` 可执行文件与 `github.com/minio/minio` module path;
- MinIO 兼容的 S3 API、配置、环境变量与命令行约定;
- `RELEASE.YYYY-MM-DDTHH-MM-SSZ` 标签、容器入口与常见部署方式。
兼容性是默认约束。只要不会留下安全问题,Silo 就保留既有的协议、客户端、配置与运维行为;只有在修复重大安全问题确有必要时才会打破兼容,并在版本说明中明确受影响行为与迁移方式。每个版本仍应视为下游升级:锁定版本,阅读[版本说明](https://silo.pgsty.com/zh/blog/release/)与[安全公告](docs/security/advisories.md),保留回滚路径,并在生产使用前完成测试。
## 下载与发行产物
请先在[下载与安装](https://silo.pgsty.com/zh/download/)页面选择合适的安装方式;GitHub Releases 仍是带版本服务端二进制、校验和与源码归档的获取位置。
| 产物 | 位置 |
| :-- | :-- |
| 源码 | [`github.com/pgsty/minio`](https://github.com/pgsty/minio) |
| 容器镜像 | [`pgsty/minio`](https://hub.docker.com/r/pgsty/minio),支持 `linux/amd64` 与 `linux/arm64` 多架构清单 |
| 服务端二进制与校验和 | [GitHub Releases](https://github.com/pgsty/minio/releases),覆盖 Linux、macOS、Windows 的 `amd64` 与 `arm64` |
| Linux 软件包 | RPM、DEB、APK,并通过 [Pigsty 软件仓库](https://pigsty.cc/docs/repo/) 分发 |
| 客户端 | [`pgsty/mc`](https://github.com/pgsty/mc),容器内以 `mcli` 提供,并保留 `mc` 兼容别名 |
| 管理控制台 | 社区维护的 [`georgmangold/console`](https://github.com/georgmangold/console),嵌入服务端构建 |
| 共享库 | [`pgsty/silo-pkg`](https://github.com/pgsty/silo-pkg) v3.7.0,通过 `replace` 指令使用,同时保留 `github.com/minio/pkg/v3` 导入路径([版本说明](https://silo.pgsty.com/zh/blog/release/pkg-3.7.0/)) |
## 快速开始
本地体验:
## 快速上手
```bash
mkdir -p data
export MINIO_ROOT_USER=minioadmin
export MINIO_ROOT_PASSWORD=change-me-long-password
docker run -d --name silo \
-p 9000:9000 \
-p 9001:9001 \
-e MINIO_ROOT_USER \
-e MINIO_ROOT_PASSWORD \
docker run -d --name silo -p 9000:9000 -p 9001:9001 \
-e MINIO_ROOT_USER=minioadmin \
-e MINIO_ROOT_PASSWORD=change-me-long-password \
-v "$PWD/data:/data" \
pgsty/minio:latest server /data --console-address ":9001"
docker.io/pgsty/silo:latest server /data --console-address ":9001"
```
管理控制台位于 <http://localhost:9001>,S3 API 位于 <http://localhost:9000>。
<p align="center">
<img src="https://silo.pgsty.com/images/silo-console/console-login.webp" alt="Silo 控制台">
</p>
镜像内置兼容客户端 `mcli`:
控制台位于 <http://localhost:9001>,S3 API 位于 <http://localhost:9000>。镜像内置客户端 `mcli`:
```bash
docker exec silo mcli alias set local http://127.0.0.1:9000 \
"$MINIO_ROOT_USER" "$MINIO_ROOT_PASSWORD"
docker exec silo mcli mb local/demo
docker exec silo mcli ls local
docker exec silo mcli alias set local http://127.0.0.1:9000 minioadmin change-me-long-password
docker exec silo mcli mb local/demo && docker exec silo mcli ls local
```
> [!WARNING]
> 生产环境应锁定版本,使用独立凭据与 TLS,配置监控,保留独立备份,并验证恢复流程。
> 生产环境应锁定版本,使用独立凭据与 TLS,配置监控,保留独立备份,并验证恢复流程。请从[文档](https://silo.pgsty.com/zh/docs/)开始。
从源码构建服务端:
## 安装
```bash
go build -o minio .
./minio --version
```
| 方式 | 位置 |
| :-- | :-- |
| 容器镜像 | [`pgsty/silo`](https://hub.docker.com/r/pgsty/silo),支持 `linux/amd64` 与 `linux/arm64` |
| 二进制 | [GitHub Releases](https://github.com/pgsty/silo/releases),覆盖 Linux、macOS、Windows 的 `amd64` 与 `arm64` |
| 软件包 | RPM、DEB、APK,也可通过 [Pigsty 软件仓库](https://pigsty.cc/docs/repo/) 安装 |
| Kubernetes | Helm Chart,参见[下载与安装](https://silo.pgsty.com/zh/download/) |
| 源码构建 | `go build -o silo . && ./silo --version` |
其他安装方式(包括原生软件包、二进制、Podman、Kubernetes、源码构建与 Pigsty Ansible)请前往[下载与安装](https://silo.pgsty.com/zh/download/);生产部署与管理请从 [Silo 中文文档](https://silo.pgsty.com/zh/docs/)开始。Pigsty 用户也可以直接使用 [Pigsty MinIO 模块](https://pigsty.cc/docs/minio/)。
每个版本都附带校验和、SPDX SBOM、Sigstore 签名清单与 GitHub 构建证明。完整安装方式与验证命令见[下载与安装](https://silo.pgsty.com/zh/download/);从上游 MinIO 迁移 —— 接管既有 `minio.service` 与 `/etc/default/minio`,并用 `/etc/systemd/system/silo.service.d/10-legacy-user.conf` drop-in 保持数据属主不变 —— 见[迁移指南](https://silo.pgsty.com/zh/compatibility/migration/)与[二进制与服务说明](https://silo.pgsty.com/zh/compatibility/binary/)。
## 安全
## 兼容性
安全修复面向活跃的 `master` 分支,并记录在仓库[安全公告](docs/security/advisories.md)与门户[安全通告](https://silo.pgsty.com/zh/blog/security/)中。请按照 [`SECURITY.md`](SECURITY.md) 与 [`VULNERABILITY_REPORT.md`](VULNERABILITY_REPORT.md) 私密报告漏洞;同时影响上游 MinIO 的问题也应向上游报告。
Silo 保留 S3 与存储格式兼容性,包括既有 `MINIO_*` 环境变量、`minio_*` 指标、`x-minio-*` 头、`/minio/*` 路由与 `.minio.sys` 数据。CI 守卫检查选定的兼容性标识,有意的安全与行为变化在发布说明中记录。Silo 自有交付面使用 `silo` 可执行文件、软件包、服务、Helm Chart 与容器镜像;原生交付物不会安装 `minio` 服务端二进制别名。
## 参与贡献
正式支持和发布验收的组合为 `pgsty/silo` + `pgsty/silo-console` + `pgsty/mc` + `pgsty/silo-pkg`,对未修改的上游 MinIO/MC 尽最大努力保持兼容。Server、Console 与客户端按需保留历史模块路径,维护源码直接导入 `github.com/pgsty/silo-pkg/v3`;SDK `github.com/minio/minio-go/v7` 是明确保留的上游依赖。具体版本与 replace 以当前 [go.mod](go.mod) 为准。
欢迎安全与依赖项更新、可复现缺陷修复、测试、发布自动化、打包与文档改进。
与上游的全部分歧,以逐项核验代码的[兼容性审计](https://silo.pgsty.com/zh/compatibility/server/)形式维护。每个版本仍应视为下游升级:锁定版本,阅读[版本说明](https://silo.pgsty.com/zh/tags/silo/),并保留回滚路径。
Issue 与 Pull Request 应说明受影响版本、复现步骤、影响、预期行为、测试与兼容性说明。大型改动请先提交 Issue 讨论。
### TLS 与 Go 升级
Server 恢复 Go 默认密钥交换策略的修复已在 main,尚未包含在 Server 20260903。
`GODEBUG=tlsmlkem=0`、`tlssecpmlkem=0` 的适用范围、macOS 根证书来源变化,
以及 OIDC discovery 的诊断方法见 [Go 1.27 TLS 与 OIDC 指南](https://silo.pgsty.com/zh/blog/design/go127-tls-oidc-discovery/)。
## 文档归属
用户文档统一维护在 [silo.pgsty.com](https://silo.pgsty.com/zh/docs/),源码位于
[pgsty/silo.pgsty.com](https://github.com/pgsty/silo.pgsty.com)。本仓库保留的 `docs/`
主要是继承的参考材料、示例和工具测试夹具。调查日志、AI 工作记录与临时报告放在仓库外;
可复用的结论应整理进伴生文档站。仓库职责与维护规则见 [AGENTS.md](AGENTS.md)。
## 安全与贡献
请按照 [`SECURITY.md`](SECURITY.md) 私密报告漏洞;每项修复都会发布公开[安全公告](https://silo.pgsty.com/zh/blog/security/)。本项目不要求签署 CLA:贡献按 AGPL-3.0-or-later(inbound=outbound)接收,只需 DCO 签署(`git commit -s`),详见 [`CONTRIBUTING.md`](CONTRIBUTING.md)。
## 贡献者
<!-- Generated by silo.pgsty.com/bin/contributors.py. -->
每位 issue 或 PR 的作者都是 SILO 社区的一员,包括尚未合并的工作。已合并的修复、被采纳的方案和有效报告优先展示,其余参考首次参与时间;金色圆环突出经过审核的显著贡献。
<a href="CONTRIBUTORS.md">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/pgsty/silo/codex/repository-cards/contributors-dark.svg">
<img src="https://raw.githubusercontent.com/pgsty/silo/codex/repository-cards/contributors-light.svg" alt="SILO 社区贡献者">
</picture>
</a>
[查看贡献记录与实际 PR 状态](CONTRIBUTORS.md)。
## Star History
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/pgsty/silo/codex/repository-cards/star-history-dark.svg">
<img src="https://raw.githubusercontent.com/pgsty/silo/codex/repository-cards/star-history-light.svg" alt="SILO GitHub 星标历史">
</picture>
## 背景
本项目源于上游社区发行与维护模式的变化。维护者对相关变化的分析、替代方案评估与早期维护记录见以下文章:
本项目因上游收缩社区版而生:Web 控制台被削减为残桩、社区预编译制品停发、社区仓库被归档。Silo 的存在就是让这些部署继续跑下去。Fork 是手段,不是身份 —— 若上游恢复社区版承诺,我们乐意收缩范围,并把修复回馈上游。
| 文章 | 主题 |
| :-- | :-- |
| [MinIO已死](https://silo.pgsty.com/zh/blog/post/minio-is-dead/) | 上游项目与发行模式的变化 |
| [MinIO已死,谁能接盘?](https://silo.pgsty.com/zh/blog/post/minio-alternative/) | 可选替代方案评估 |
| [MinIO 已死,MinIO 复生](https://silo.pgsty.com/zh/blog/post/minio-resurrect/) | 建立分支及其发行流水线 |
| [续命 MinIO:承诺兑现](https://silo.pgsty.com/zh/blog/post/minio-promise-kept/) | 初期安全与维护工作 |
[**宣言**](https://silo.pgsty.com/zh/about/manifesto/)是项目的公开承诺,共十一条,通篇遵循一项纪律:**每一条,要么是已经在做且有公开证据的事实,要么是刻意拒绝的承诺。** 摘要:
- **兼容性合同** —— 协议与数据不改,每个版本都标注经过测试的回滚目标与路径。
- **许可证无法变更** —— AGPLv3、无 CLA、不做版权聚合;包括我们自己在内,没有人握有足够版权代表所有贡献者重新授权。
- **永不清单**(只增不减)—— 永不将既有功能移入付费墙、永不给下载设注册墙、永不加入遥测(上游回连路径已整体移除)、永不引入 CLA、永不变更许可证、永不以商标追究正常使用。
- **安全与发布纪律** —— 每项安全修复配一篇公开公告;通常每一到两个月发布一版,最长不超过一个季度。请拿公开记录检验这两条。
延伸阅读:[MinIO已死](https://silo.pgsty.com/zh/blog/post/minio-is-dead/) · [谁能接盘?](https://silo.pgsty.com/zh/blog/post/minio-alternative/) · [MinIO 复生](https://silo.pgsty.com/zh/blog/post/minio-resurrect/) · [承诺兑现](https://silo.pgsty.com/zh/blog/post/minio-promise-kept/)
## 许可证与商标
服务端继续采用 [GNU Affero General Public License v3.0](LICENSE) 发布。上游作者与署名信息见 [`CREDITS`](CREDITS)。
Silo 采用 [AGPL-3.0-or-later](LICENSE),衍生自 [`minio/minio`](https://github.com/minio/minio),上游版权与第三方声明完整保留于 [`NOTICE`](NOTICE) 与 [`CREDITS`](CREDITS)。MinIO 是 MinIO, Inc. 的商标,此处使用仅为标识上游项目与兼容谱系。
MinIO 是 MinIO, Inc. 的商标。Silo、Pigsty 与 `pgsty/minio` 均为独立社区项目,与 MinIO, Inc. 不存在隶属或背书关系。
详见:[许可证](https://silo.pgsty.com/zh/about/license/) · [署名归属](https://silo.pgsty.com/zh/about/attribution/) · [商标声明](https://silo.pgsty.com/zh/about/trademark/)
+30 -5
View File
@@ -1,20 +1,45 @@
# Security Policy
This repository is the `pgsty/minio` community fork of `minio/minio`. Upstream MinIO security contacts do not handle fork-specific fixes or release notes for this repository.
Silo is an independent, community-maintained object-storage server derived from
the open-source MinIO server. Upstream MinIO security contacts do not handle
Silo-specific fixes or release notes.
## Supported Versions
Security fixes are tracked on the active `master` branch and summarized in [docs/security/advisories.md](docs/security/advisories.md).
Security fixes are tracked on the active development branch and summarized in
[the security advisory ledger](https://silo.pgsty.com/about/security-advisories/). Only the current
Silo release line is supported unless an advisory says otherwise.
## Inherited Fix Evidence
The canonical ledger also records security fixes inherited from upstream when
they are part of the Silo release baseline. Source and fork commits are linked
separately even when the fork preserves the original commit object and SHA.
- [CVE-2025-62506](https://github.com/advisories/GHSA-jjjj-jwhf-8rgr):
upstream [PR #21642](https://github.com/minio/minio/pull/21642) merged as
[`minio/minio@c1a49490`](https://github.com/minio/minio/commit/c1a49490c78e9c3ebcad86ba0662319138ace190),
inherited unchanged as
[`pgsty/silo@c1a49490`](https://github.com/pgsty/silo/commit/c1a49490c78e9c3ebcad86ba0662319138ace190),
and is present in every Silo community release beginning with
[`RELEASE.2025-12-03T12-00-00Z`](https://github.com/pgsty/silo/releases/tag/RELEASE.2025-12-03T12-00-00Z).
The inherited [service-account](https://github.com/pgsty/silo/blob/c1a49490c78e9c3ebcad86ba0662319138ace190/cmd/admin-handlers-users_test.go#L211-L212)
and [STS](https://github.com/pgsty/silo/blob/c1a49490c78e9c3ebcad86ba0662319138ace190/cmd/sts-handlers_test.go#L45-L46)
regression groups remain part of `go test ./cmd`; see the
[canonical ledger](https://silo.pgsty.com/about/security-advisories/#inherited)
for the operator-facing record.
## Reporting a Vulnerability
For vulnerabilities in this fork:
1. Follow the fork-specific expectations in [VULNERABILITY_REPORT.md](VULNERABILITY_REPORT.md).
2. Prefer the `pgsty/minio` repository's GitHub security reporting workflow when it is available.
3. If private reporting is not available, contact the maintainers through the `pgsty/minio` repository before publishing detailed exploit information.
2. Prefer this repository's [private GitHub security advisory](https://github.com/pgsty/silo/security/advisories/new) workflow.
3. If private reporting is unavailable, contact the maintainers through the
repository without publishing exploit details until a private channel is
established.
4. If you confirm the issue also affects upstream `minio/minio`, report it upstream separately.
## Disclosure Process
Fork-specific fixes and user-visible upgrade notes are published in [docs/security/advisories.md](docs/security/advisories.md). The fork-specific triage and remediation process is described in [VULNERABILITY_REPORT.md](VULNERABILITY_REPORT.md).
Fork-specific fixes and user-visible upgrade notes are published in [the security advisory ledger](https://silo.pgsty.com/about/security-advisories/). The fork-specific triage and remediation process is described in [VULNERABILITY_REPORT.md](VULNERABILITY_REPORT.md).
+4 -4
View File
@@ -1,6 +1,6 @@
# Vulnerability Management Policy
This document describes how the `pgsty/minio` maintainers investigate,
This document describes how the Silo maintainers investigate,
assess, and remediate reported vulnerabilities affecting this fork, any
directly shipped component, or a direct / indirect dependency used by this
repository.
@@ -8,7 +8,7 @@ repository.
## Scope
This policy covers vulnerability reports opened by repository maintainers or
external third parties against `pgsty/minio` itself, its release artifacts, or
external third parties against Silo itself, its release artifacts, or
dependencies that materially affect this fork.
It defines the information needed for triage and the expected remediation
@@ -24,7 +24,7 @@ A useful vulnerability report should contain the following information:
a well-established vulnerability identifier, such as a CVE or GHSA ID, can
be used instead.
Based on the report, the `pgsty/minio` maintainers investigate:
Based on the report, the Silo maintainers investigate:
- Whether the reported vulnerability exists.
- The conditions that are required such that the vulnerability can be exploited.
@@ -34,4 +34,4 @@ Based on the report, the `pgsty/minio` maintainers investigate:
If the vulnerability exists in this fork itself, the maintainers will, when
feasible, fix the issue or implement reasonable countermeasures such that the
vulnerability can no longer be exploited. Fork-specific upgrade notes and
security advisories are published in `docs/security/advisories.md`.
security advisories are published in [the security advisory ledger](https://silo.pgsty.com/about/security-advisories/).
-1
View File
@@ -1 +0,0 @@
theme: jekyll-theme-minimal
+76
View File
@@ -0,0 +1,76 @@
#!/usr/bin/env bash
# Fail closed before a release job can replace published or finalized assets.
# An ordinary Draft is retry state; a finalized Draft contains GPG-derived
# materials and must never be replaced by the build lane.
set -euo pipefail
release_tag="${1:-}"
fixture="${2:-}"
repository="${GITHUB_REPOSITORY:-pgsty/silo}"
require_draft="${REQUIRE_DRAFT:-false}"
if ! command -v jq >/dev/null 2>&1; then
echo "jq is required to inspect GitHub release state" >&2
exit 1
fi
if [[ ! "${release_tag}" =~ ^RELEASE\.[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}-[0-9]{2}-[0-9]{2}Z$ ]]; then
echo "Invalid release tag format: ${release_tag:-<empty>}" >&2
exit 1
fi
if [ -n "${fixture}" ]; then
release_json="$(<"${fixture}")"
else
error_file="$(mktemp)"
trap 'rm -f "${error_file}"' EXIT
if ! release_json="$(
gh api --paginate "repos/${repository}/releases?per_page=100" --jq '.[]' 2>"${error_file}" |
jq --arg tag "${release_tag}" -s '[.[] | select(.tag_name == $tag)]'
)"; then
cat "${error_file}" >&2
exit 1
fi
fi
if ! jq -e 'type == "array" and all(.[]; type == "object" and (.tag_name | type == "string") and (.draft | type == "boolean"))' \
<<<"${release_json}" >/dev/null 2>&1; then
echo "Invalid release state response for ${release_tag}" >&2
exit 1
fi
if ! jq -e --arg tag "${release_tag}" 'all(.[]; .tag_name == $tag)' \
<<<"${release_json}" >/dev/null 2>&1; then
echo "Release state returned a tag other than ${release_tag}" >&2
exit 1
fi
release_count="$(jq 'length' <<<"${release_json}")"
if [ "${release_count}" -eq 0 ]; then
if [ "${require_draft}" = "true" ]; then
echo "Expected one Draft release for ${release_tag}, found none" >&2
exit 1
fi
echo "No existing release for ${release_tag}."
exit 0
fi
if [ "${release_count}" -ne 1 ]; then
echo "Refusing to choose among ${release_count} releases for ${release_tag}; clean duplicate Drafts first" >&2
exit 1
fi
if [ "$(jq -r '.[0].draft' <<<"${release_json}")" != "true" ]; then
echo "Refusing to overwrite published release ${release_tag}" >&2
exit 1
fi
finalize_markers="$(jq '[.[0].assets[]? | select(.name | endswith("_packages_provenance.sigstore.json"))] | length' <<<"${release_json}")"
if [ "${finalize_markers}" -ne 0 ]; then
echo "Refusing to replace finalized Draft ${release_tag}" >&2
exit 1
fi
echo "Existing unfinalized Draft ${release_tag} will be replaced from scratch."
+68
View File
@@ -0,0 +1,68 @@
#!/usr/bin/env bash
set -euo pipefail
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
checker="${script_dir}/check-release-state.sh"
tag="RELEASE.2026-08-29T00-00-00Z"
fixture="$(mktemp)"
stdout_file="$(mktemp)"
stderr_file="$(mktemp)"
trap 'rm -f "${fixture}" "${stdout_file}" "${stderr_file}"' EXIT
expect_success() {
if ! "${checker}" "$@" >"${stdout_file}" 2>"${stderr_file}"; then
cat "${stderr_file}" >&2
return 1
fi
}
expect_failure() {
if "${checker}" "$@" >"${stdout_file}" 2>"${stderr_file}"; then
echo "Expected release-state check to fail: $*" >&2
return 1
fi
}
printf '[]\n' >"${fixture}"
expect_success "${tag}" "${fixture}"
grep -qF "No existing release for ${tag}." "${stdout_file}"
if REQUIRE_DRAFT=true "${checker}" "${tag}" "${fixture}" >"${stdout_file}" 2>"${stderr_file}"; then
echo "Expected required-Draft check to fail when no release exists" >&2
exit 1
fi
grep -qF "Expected one Draft release for ${tag}, found none" "${stderr_file}"
printf '[{"tag_name":"%s","draft":true,"assets":[]}]\n' "${tag}" >"${fixture}"
expect_success "${tag}" "${fixture}"
grep -qF "Existing unfinalized Draft ${tag} will be replaced from scratch." "${stdout_file}"
if ! REQUIRE_DRAFT=true "${checker}" "${tag}" "${fixture}" >"${stdout_file}" 2>"${stderr_file}"; then
cat "${stderr_file}" >&2
exit 1
fi
printf '[{"tag_name":"%s","draft":true,"assets":[{"name":"silo_20260829000000.0.0_packages_provenance.sigstore.json"}]}]\n' "${tag}" >"${fixture}"
expect_failure "${tag}" "${fixture}"
grep -qF "Refusing to replace finalized Draft ${tag}" "${stderr_file}"
printf '[{"tag_name":"%s","draft":false}]\n' "${tag}" >"${fixture}"
expect_failure "${tag}" "${fixture}"
grep -qF "Refusing to overwrite published release ${tag}" "${stderr_file}"
printf '[{"tag_name":"%s","draft":true},{"tag_name":"%s","draft":true}]\n' "${tag}" "${tag}" >"${fixture}"
expect_failure "${tag}" "${fixture}"
grep -qF "Refusing to choose among 2 releases" "${stderr_file}"
printf '[{"tag_name":"RELEASE.2026-08-28T00-00-00Z","draft":true}]\n' >"${fixture}"
expect_failure "${tag}" "${fixture}"
grep -qF "other than ${tag}" "${stderr_file}"
printf '{not-json}\n' >"${fixture}"
expect_failure "${tag}" "${fixture}"
grep -qF "Invalid release state response for ${tag}" "${stderr_file}"
expect_failure "not-a-release-tag" "${fixture}"
grep -qF "Invalid release tag format" "${stderr_file}"
echo "release-state decision tests passed"
+1 -1
View File
@@ -7,7 +7,7 @@ _init() {
## Minimum required versions for build dependencies
GIT_VERSION="1.0"
GO_VERSION="1.16"
GO_VERSION="1.27.1"
OSX_VERSION="10.8"
KNAME=$(uname -s)
ARCH=$(uname -m)
+22 -23
View File
@@ -3,7 +3,7 @@
set -x
export MINIO_CI_CD=1
killall -9 minio
killall -9 silo
rm -rf ${HOME}/tmp/dist
@@ -19,28 +19,27 @@ done
echo $args
for ((i = 0; i < $((nr_servers)); i++)); do
(minio server --address ":$((9100 + i))" $args 2>&1 >/tmp/log$i.txt) &
(silo server --address ":$((9100 + i))" $args 2>&1 >/tmp/log$i.txt) &
done
sleep 10s
if [ ! -f ./mc ]; then
wget --quiet -O ./mc https://dl.minio.io/client/mc/release/linux-amd64/./mc &&
chmod +x mc
"$(git rev-parse --show-toplevel)/buildscripts/install-mcli.sh" ./mc
fi
set +e
export MC_HOST_minioadm=http://minioadmin:minioadmin@localhost:9100/
./mc ready minioadm
export MC_HOST_siloadm=http://minioadmin:minioadmin@localhost:9100/
./mc ready siloadm
./mc ls minioadm/
./mc ls siloadm/
./mc admin config set minioadm/ api root_access=off
./mc admin config set siloadm/ api root_access=off
sleep 3s # let things settle a little
./mc ls minioadm/
./mc ls siloadm/
if [ $? -eq 0 ]; then
echo "listing succeeded, 'minioadmin' was not disabled"
exit 1
@@ -48,38 +47,38 @@ fi
set -e
killall -9 minio
killall -9 silo
export MINIO_API_ROOT_ACCESS=on
for ((i = 0; i < $((nr_servers)); i++)); do
(minio server --address ":$((9100 + i))" $args 2>&1 >/tmp/log$i.txt) &
(silo server --address ":$((9100 + i))" $args 2>&1 >/tmp/log$i.txt) &
done
set +e
./mc ready minioadm/
./mc ready siloadm/
./mc ls minioadm/
./mc ls siloadm/
if [ $? -ne 0 ]; then
echo "listing failed, 'minioadmin' should be enabled"
exit 1
fi
killall -9 minio
killall -9 silo
rm -rf /tmp/multisitea/
rm -rf /tmp/multisiteb/
echo "Setup site-replication and then disable root credentials"
minio server --address 127.0.0.1:9001 "http://127.0.0.1:9001/tmp/multisitea/data/disterasure/xl{1...4}" \
silo server --address 127.0.0.1:9001 "http://127.0.0.1:9001/tmp/multisitea/data/disterasure/xl{1...4}" \
"http://127.0.0.1:9002/tmp/multisitea/data/disterasure/xl{5...8}" >/tmp/sitea_1.log 2>&1 &
minio server --address 127.0.0.1:9002 "http://127.0.0.1:9001/tmp/multisitea/data/disterasure/xl{1...4}" \
silo server --address 127.0.0.1:9002 "http://127.0.0.1:9001/tmp/multisitea/data/disterasure/xl{1...4}" \
"http://127.0.0.1:9002/tmp/multisitea/data/disterasure/xl{5...8}" >/tmp/sitea_2.log 2>&1 &
minio server --address 127.0.0.1:9003 "http://127.0.0.1:9003/tmp/multisiteb/data/disterasure/xl{1...4}" \
silo server --address 127.0.0.1:9003 "http://127.0.0.1:9003/tmp/multisiteb/data/disterasure/xl{1...4}" \
"http://127.0.0.1:9004/tmp/multisiteb/data/disterasure/xl{5...8}" >/tmp/siteb_1.log 2>&1 &
minio server --address 127.0.0.1:9004 "http://127.0.0.1:9003/tmp/multisiteb/data/disterasure/xl{1...4}" \
silo server --address 127.0.0.1:9004 "http://127.0.0.1:9003/tmp/multisiteb/data/disterasure/xl{1...4}" \
"http://127.0.0.1:9004/tmp/multisiteb/data/disterasure/xl{5...8}" >/tmp/siteb_2.log 2>&1 &
export MC_HOST_sitea=http://minioadmin:minioadmin@127.0.0.1:9001
@@ -96,19 +95,19 @@ export MC_HOST_siteb=http://minioadmin:minioadmin@127.0.0.1:9004
./mc admin user info siteb foobar
killall -9 minio
killall -9 silo
echo "turning off root access, however site replication must continue"
export MINIO_API_ROOT_ACCESS=off
minio server --address 127.0.0.1:9001 "http://127.0.0.1:9001/tmp/multisitea/data/disterasure/xl{1...4}" \
silo server --address 127.0.0.1:9001 "http://127.0.0.1:9001/tmp/multisitea/data/disterasure/xl{1...4}" \
"http://127.0.0.1:9002/tmp/multisitea/data/disterasure/xl{5...8}" >/tmp/sitea_1.log 2>&1 &
minio server --address 127.0.0.1:9002 "http://127.0.0.1:9001/tmp/multisitea/data/disterasure/xl{1...4}" \
silo server --address 127.0.0.1:9002 "http://127.0.0.1:9001/tmp/multisitea/data/disterasure/xl{1...4}" \
"http://127.0.0.1:9002/tmp/multisitea/data/disterasure/xl{5...8}" >/tmp/sitea_2.log 2>&1 &
minio server --address 127.0.0.1:9003 "http://127.0.0.1:9003/tmp/multisiteb/data/disterasure/xl{1...4}" \
silo server --address 127.0.0.1:9003 "http://127.0.0.1:9003/tmp/multisiteb/data/disterasure/xl{1...4}" \
"http://127.0.0.1:9004/tmp/multisiteb/data/disterasure/xl{5...8}" >/tmp/siteb_1.log 2>&1 &
minio server --address 127.0.0.1:9004 "http://127.0.0.1:9003/tmp/multisiteb/data/disterasure/xl{1...4}" \
silo server --address 127.0.0.1:9004 "http://127.0.0.1:9003/tmp/multisiteb/data/disterasure/xl{1...4}" \
"http://127.0.0.1:9004/tmp/multisiteb/data/disterasure/xl{5...8}" >/tmp/siteb_2.log 2>&1 &
export MC_HOST_sitea=http://foobar:foo12345@127.0.0.1:9001
+125
View File
@@ -0,0 +1,125 @@
#!/usr/bin/env bash
set -euo pipefail
# Regenerates CREDITS from the license text of every Go module linked into the
# silo binary. The module set is what `go list -deps` reports for the main
# package, so test-only and tool dependencies stay out: CREDITS documents what
# a shipped binary actually contains. check-gen runs this and fails on a diff,
# which keeps CREDITS from drifting when go.mod changes.
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
repo_dir="$(cd "${script_dir}/.." && pwd)"
cd "${repo_dir}"
out_file="${1:-${repo_dir}/CREDITS}"
tmp_file="${out_file}.tmp"
trap 'rm -f "${tmp_file}"' EXIT
rule_dash='----------------------------------------------------------------'
rule_equal='================================================================'
# These modules repackage Go standard library code and publish no license
# file; their source files carry the Go Authors' BSD-style header pointing at
# the Go project license, so that text is reproduced for them.
stdlib_derived='github.com/minio/colorjson github.com/minio/csvparser github.com/minio/filepath'
is_stdlib_derived() {
case " ${stdlib_derived} " in
*" $1 "*) return 0 ;;
*) return 1 ;;
esac
}
# Command substitution strips trailing newlines; printf adds exactly one back,
# so every entry ends the same way regardless of how the license file ends.
emit_text() {
printf '%s\n' "$(cat "$1")"
}
# The module cache must hold every dependency before .Dir can resolve.
go mod download
# The Go project license text is taken from the pinned golang.org/x/sys module
# rather than GOROOT: Homebrew's Go does not ship GOROOT/LICENSE, and the
# module copy is version-locked by go.mod, so the output cannot vary with the
# machine's toolchain packaging.
go_license="$(go list -m -f '{{.Dir}}' golang.org/x/sys)/LICENSE"
if [ ! -f "${go_license}" ]; then
echo "Missing Go license text: ${go_license}" >&2
exit 1
fi
{
printf '%s\n' \
'Silo bundles third-party software under the licenses reproduced below.' \
'Generated by buildscripts/gen-credits.sh (make credits) from the Go' \
'modules linked into the silo binary. Do not edit by hand.' \
'' \
"${rule_equal}" \
''
printf '%s\n%s\n%s\n' 'Go (the standard library)' 'https://golang.org/' "${rule_dash}"
emit_text "${go_license}"
printf '\n%s\n\n' "${rule_equal}"
# The dependency closure is GOOS/GOARCH-specific: platform-only modules
# (darwin's go-m1cpu, windows' wmi, ...) enter and leave it with the host.
# Pin the primary shipped target and the release build tags so regenerating
# CREDITS produces identical output on every machine, including CI.
GOOS=linux GOARCH=amd64 go list -deps -tags kqueue \
-f '{{if and (not .Standard) .Module}}{{.Module.Path}}{{end}}' . \
| LC_ALL=C sort -u \
| grep -vx 'github.com/minio/minio' \
| xargs go list -m -f '{{.Path}}|{{with .Replace}}{{.Path}}{{end}}|{{.Dir}}' \
| while IFS='|' read -r path replacement dir; do
if [ -z "${dir}" ] || [ ! -d "${dir}" ]; then
echo "Module cache directory missing for ${path}" >&2
exit 1
fi
name="${path}"
url="https://${path}"
# A same-path replace only pins a version; the annotation is for
# dependencies actually served from a different repository.
if [ -n "${replacement}" ] && [ "${replacement}" != "${path}" ]; then
name="${path} (replaced by ${replacement})"
url="https://${replacement}"
fi
printf '%s\n%s\n%s\n' "${name}" "${url}" "${rule_dash}"
if is_stdlib_derived "${path}"; then
printf '%s\n%s\n\n' \
'This module repackages Go standard library code and publishes no' \
'license file; the Go project license below applies per its file headers.'
emit_text "${go_license}"
else
license_file=''
for candidate in LICENSE LICENSE.txt LICENSE.md COPYING COPYING.txt LICENCE UNLICENSE; do
if [ -f "${dir}/${candidate}" ]; then
license_file="${dir}/${candidate}"
break
fi
done
if [ -z "${license_file}" ]; then
echo "No license file found for ${path} in ${dir}" >&2
exit 1
fi
emit_text "${license_file}"
fi
# Apache License 2.0 section 4(d) requires redistributing the NOTICE
# file contents alongside the licensed work.
for notice in NOTICE NOTICE.txt; do
if [ -f "${dir}/${notice}" ]; then
printf '\n%s\n\n' 'Bundled NOTICE file:'
emit_text "${dir}/${notice}"
break
fi
done
printf '\n%s\n\n' "${rule_equal}"
done
} > "${tmp_file}"
mv "${tmp_file}" "${out_file}"
+19 -30
View File
@@ -5,45 +5,34 @@ set -o pipefail
set -x
WORK_DIR="$PWD/.verify-$RANDOM"
MINIO_CONFIG_DIR="$WORK_DIR/.minio"
MINIO=("$PWD/minio" --config-dir "$MINIO_CONFIG_DIR" server)
SILO_CONFIG_DIR="$WORK_DIR/.silo"
SILO=("$PWD/silo" --config-dir "$SILO_CONFIG_DIR" server)
if [ ! -x "$PWD/minio" ]; then
echo "minio executable binary not found in current directory"
if [ ! -x "$PWD/silo" ]; then
echo "Silo executable binary not found in current directory"
exit 1
fi
if [ ! -x "$PWD/minio" ]; then
echo "minio executable binary not found in current directory"
if [ ! -x "$PWD/silo" ]; then
echo "Silo executable binary not found in current directory"
exit 1
fi
function start_minio_4drive() {
function start_silo_4drive() {
start_port=$1
export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=minio123
export MC_HOST_minio="http://minio:minio123@127.0.0.1:${start_port}/"
export MINIO_ROOT_USER=silo
export MINIO_ROOT_PASSWORD=silo1234
export MC_HOST_silo="http://silo:silo1234@127.0.0.1:${start_port}/"
unset MINIO_KMS_AUTO_ENCRYPTION # do not auto-encrypt objects
export MINIO_CI_CD=1
mkdir ${WORK_DIR}
C_PWD=${PWD}
if [ ! -x "$PWD/mc" ]; then
MC_BUILD_DIR="mc-$RANDOM"
if ! git clone --quiet https://github.com/minio/mc "$MC_BUILD_DIR"; then
echo "failed to download https://github.com/minio/mc"
purge "${MC_BUILD_DIR}"
exit 1
fi
(cd "${MC_BUILD_DIR}" && go build -o "$C_PWD/mc")
# remove mc source.
purge "${MC_BUILD_DIR}"
"$(git rev-parse --show-toplevel)/buildscripts/install-mcli.sh" "$PWD/mc"
fi
"${MINIO[@]}" --address ":$start_port" "${WORK_DIR}/disk{1...4}" >"${WORK_DIR}/server1.log" 2>&1 &
"${SILO[@]}" --address ":$start_port" "${WORK_DIR}/disk{1...4}" >"${WORK_DIR}/server1.log" 2>&1 &
pid=$!
disown $pid
sleep 5
@@ -56,30 +45,30 @@ function start_minio_4drive() {
exit 1
fi
"${PWD}/mc" mb --with-versioning minio/bucket
"${PWD}/mc" mb --with-versioning silo/bucket
for i in $(seq 1 4); do
"${PWD}/mc" cp /etc/hosts minio/bucket/testobj
"${PWD}/mc" cp /etc/hosts silo/bucket/testobj
sudo chown -R root. "${WORK_DIR}/disk${i}"
"${PWD}/mc" cp /etc/hosts minio/bucket/testobj
"${PWD}/mc" cp /etc/hosts silo/bucket/testobj
sudo chown -R ${USER}. "${WORK_DIR}/disk${i}"
done
for vid in $("${PWD}/mc" ls --json --versions minio/bucket/testobj | jq -r .versionId); do
"${PWD}/mc" cat --vid "${vid}" minio/bucket/testobj | md5sum
for vid in $("${PWD}/mc" ls --json --versions silo/bucket/testobj | jq -r .versionId); do
"${PWD}/mc" cat --vid "${vid}" silo/bucket/testobj | md5sum
done
pkill minio
pkill silo
sleep 3
}
function main() {
start_port=$(shuf -i 10000-65000 -n 1)
start_minio_4drive ${start_port}
start_silo_4drive ${start_port}
}
function purge() {
+331
View File
@@ -0,0 +1,331 @@
// Copyright 2026 PGSTY contributors.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// helm-migration-guard compares a rendered legacy MinIO chart with the Silo
// upgrade candidate. Product labels, images, and commands may change; resource
// identity, selectors, PVCs, storage mounts, ports, secrets, and service-account
// references must not.
package main
import (
"encoding/json"
"errors"
"fmt"
"io"
"os"
"sort"
"strings"
"go.yaml.in/yaml/v3"
)
type resource struct {
key string
doc map[string]any
}
func main() {
if len(os.Args) != 3 {
fatal(errors.New("usage: helm-migration-guard OLD_RENDER NEW_RENDER"))
}
oldResources, err := readResources(os.Args[1])
if err != nil {
fatal(err)
}
newResources, err := readResources(os.Args[2])
if err != nil {
fatal(err)
}
if err := compare(oldResources, newResources); err != nil {
fatal(err)
}
fmt.Printf("Silo Helm migration identity is stable across %d rendered resources\n", len(oldResources))
}
func readResources(path string) (map[string]resource, error) {
file, err := os.Open(path)
if err != nil {
return nil, fmt.Errorf("open %s: %w", path, err)
}
defer file.Close()
resources := make(map[string]resource)
decoder := yaml.NewDecoder(file)
for document := 1; ; document++ {
var doc map[string]any
err = decoder.Decode(&doc)
if errors.Is(err, io.EOF) {
break
}
if err != nil {
return nil, fmt.Errorf("decode %s document %d: %w", path, document, err)
}
if len(doc) == 0 || text(doc["kind"]) == "" {
continue
}
metadata := object(doc["metadata"])
key := strings.Join([]string{text(doc["kind"]), text(metadata["namespace"]), text(metadata["name"])}, "/")
if _, exists := resources[key]; exists {
return nil, fmt.Errorf("%s contains duplicate resource %s", path, key)
}
resources[key] = resource{key: key, doc: doc}
}
return resources, nil
}
func compare(oldResources, newResources map[string]resource) error {
for key := range oldResources {
if _, ok := newResources[key]; !ok {
return fmt.Errorf("legacy resource would be removed or renamed: %s", key)
}
}
for key := range newResources {
if _, ok := oldResources[key]; !ok {
return fmt.Errorf("upgrade candidate unexpectedly adds a resource: %s", key)
}
}
keys := make([]string, 0, len(oldResources))
for key := range oldResources {
keys = append(keys, key)
}
sort.Strings(keys)
for _, key := range keys {
oldDoc := oldResources[key].doc
newDoc := newResources[key].doc
kind := text(oldDoc["kind"])
switch kind {
case "Service":
if err := same(key, "Service selector", at(oldDoc, "spec", "selector"), at(newDoc, "spec", "selector")); err != nil {
return err
}
if err := same(key, "Service ports", at(oldDoc, "spec", "ports"), at(newDoc, "spec", "ports")); err != nil {
return err
}
case "Deployment", "StatefulSet":
if err := same(key, "workload selector", at(oldDoc, "spec", "selector"), at(newDoc, "spec", "selector")); err != nil {
return err
}
}
if kind == "StatefulSet" {
if err := same(key, "StatefulSet serviceName", at(oldDoc, "spec", "serviceName"), at(newDoc, "spec", "serviceName")); err != nil {
return err
}
if err := same(key, "volume claim templates", claimTemplates(oldDoc), claimTemplates(newDoc)); err != nil {
return err
}
}
if kind == "PersistentVolumeClaim" {
if err := same(key, "PVC specification", at(oldDoc, "spec"), at(newDoc, "spec")); err != nil {
return err
}
}
if kind == "Secret" {
if err := same(key, "Secret keys", secretKeys(oldDoc), secretKeys(newDoc)); err != nil {
return err
}
}
if kind == "Deployment" || kind == "StatefulSet" || kind == "Job" {
if err := comparePod(key, kind, oldDoc, newDoc); err != nil {
return err
}
}
}
return nil
}
func comparePod(key, kind string, oldDoc, newDoc map[string]any) error {
oldPod := object(at(oldDoc, "spec", "template", "spec"))
newPod := object(at(newDoc, "spec", "template", "spec"))
if err := same(key, "service account", oldPod["serviceAccountName"], newPod["serviceAccountName"]); err != nil {
return err
}
if err := same(key, "referenced volume sources", volumeSources(oldPod), volumeSources(newPod)); err != nil {
return err
}
oldContainers := containers(oldPod)
newContainers := containers(newPod)
if err := same(key, "container identities", sortedKeys(oldContainers), sortedKeys(newContainers)); err != nil {
return err
}
for _, name := range sortedKeys(oldContainers) {
oldContainer := oldContainers[name]
newContainer := newContainers[name]
if err := same(key, name+" ports", oldContainer["ports"], newContainer["ports"]); err != nil {
return err
}
if err := same(key, name+" environment", oldContainer["env"], newContainer["env"]); err != nil {
return err
}
if err := same(key, name+" envFrom", oldContainer["envFrom"], newContainer["envFrom"]); err != nil {
return err
}
if err := same(key, name+" storage mounts", normalizedMounts(oldContainer, oldPod), normalizedMounts(newContainer, newPod)); err != nil {
return err
}
image := text(newContainer["image"])
if strings.HasPrefix(image, "pgsty/minio:") || strings.HasPrefix(image, "docker.io/pgsty/minio:") {
return fmt.Errorf("%s container %s still uses frozen image %s", key, name, image)
}
command := commandText(newContainer)
if strings.Contains(command, "/usr/bin/minio") {
return fmt.Errorf("%s container %s still invokes /usr/bin/minio", key, name)
}
if (kind == "Deployment" || kind == "StatefulSet") && strings.Contains(image, "pgsty/silo:") {
if !strings.Contains(command, "silo") || !strings.Contains(command, "server") {
return fmt.Errorf("%s container %s does not invoke the Silo server: %q", key, name, command)
}
}
}
return nil
}
func claimTemplates(doc map[string]any) []string {
var result []string
for _, raw := range list(at(doc, "spec", "volumeClaimTemplates")) {
claim := object(raw)
metadata := object(claim["metadata"])
result = append(result, text(metadata["name"])+"="+canonical(claim["spec"]))
}
sort.Strings(result)
return result
}
func secretKeys(doc map[string]any) []string {
var result []string
for _, section := range []string{"data", "stringData"} {
for key := range object(doc[section]) {
result = append(result, section+":"+key)
}
}
sort.Strings(result)
return result
}
func volumeSources(pod map[string]any) []string {
var result []string
for _, raw := range list(pod["volumes"]) {
volume := cloneObject(object(raw))
delete(volume, "name")
result = append(result, canonical(volume))
}
sort.Strings(result)
return result
}
func volumeSourceByName(pod map[string]any) map[string]string {
result := make(map[string]string)
for _, raw := range list(pod["volumes"]) {
volume := cloneObject(object(raw))
name := text(volume["name"])
delete(volume, "name")
result[name] = canonical(volume)
}
return result
}
func normalizedMounts(container, pod map[string]any) []string {
sources := volumeSourceByName(pod)
var result []string
for _, raw := range list(container["volumeMounts"]) {
mount := cloneObject(object(raw))
name := text(mount["name"])
delete(mount, "name")
mount["source"] = sources[name]
result = append(result, canonical(mount))
}
sort.Strings(result)
return result
}
func containers(pod map[string]any) map[string]map[string]any {
result := make(map[string]map[string]any)
for _, section := range []string{"initContainers", "containers"} {
for _, raw := range list(pod[section]) {
container := object(raw)
result[section+":"+text(container["name"])] = container
}
}
return result
}
func commandText(container map[string]any) string {
var parts []string
for _, field := range []string{"command", "args"} {
for _, value := range list(container[field]) {
parts = append(parts, text(value))
}
}
return strings.Join(parts, " ")
}
func at(root map[string]any, path ...string) any {
var current any = root
for _, part := range path {
current = object(current)[part]
}
return current
}
func object(value any) map[string]any {
if value == nil {
return map[string]any{}
}
result, _ := value.(map[string]any)
return result
}
func cloneObject(value map[string]any) map[string]any {
result := make(map[string]any, len(value))
for key, item := range value {
result[key] = item
}
return result
}
func list(value any) []any {
result, _ := value.([]any)
return result
}
func text(value any) string {
result, _ := value.(string)
return result
}
func sortedKeys[T any](values map[string]T) []string {
result := make([]string, 0, len(values))
for key := range values {
result = append(result, key)
}
sort.Strings(result)
return result
}
func same(resourceKey, field string, oldValue, newValue any) error {
oldCanonical := canonical(oldValue)
newCanonical := canonical(newValue)
if oldCanonical != newCanonical {
return fmt.Errorf("%s changes %s\nold: %s\nnew: %s", resourceKey, field, oldCanonical, newCanonical)
}
return nil
}
func canonical(value any) string {
data, err := json.Marshal(value)
if err != nil {
return fmt.Sprintf("<unmarshalable %T: %v>", value, err)
}
return string(data)
}
func fatal(err error) {
fmt.Fprintf(os.Stderr, "Silo Helm migration check failed: %v\n", err)
os.Exit(1)
}
+91
View File
@@ -0,0 +1,91 @@
#!/usr/bin/env bash
set -euo pipefail
if [ "$#" -ne 1 ]; then
echo "usage: $0 TARGET" >&2
exit 2
fi
target=$1
target_dir=$(dirname "${target}")
if [ ! -d "${target_dir}" ]; then
echo "target directory does not exist: ${target_dir}" >&2
exit 1
fi
sha256_file() {
if command -v sha256sum >/dev/null 2>&1; then
sha256sum "$1" | awk '{print $1}'
else
shasum -a 256 "$1" | awk '{print $1}'
fi
}
if [ -n "${MCLI_BIN:-}" ]; then
if [ ! -f "${MCLI_BIN}" ]; then
echo "MCLI_BIN is not a regular file: ${MCLI_BIN}" >&2
exit 1
fi
if ! printf '%s\n' "${MCLI_SHA256:-}" | grep -Eq '^[0-9a-fA-F]{64}$'; then
echo "MCLI_SHA256 must contain the expected SHA-256 for MCLI_BIN" >&2
exit 1
fi
actual=$(sha256_file "${MCLI_BIN}")
if [ "${actual}" != "${MCLI_SHA256,,}" ]; then
echo "MCLI_BIN checksum mismatch: expected ${MCLI_SHA256,,}, got ${actual}" >&2
exit 1
fi
install -m 0755 "${MCLI_BIN}" "${target}"
exit 0
fi
release=${MCLI_RELEASE:-RELEASE.2026-09-16T00-00-00Z}
version_hyphen=${release#RELEASE.}
package_version=$(printf '%s\n' "${version_hyphen}" | sed -E 's/^([0-9]{4})-([0-9]{2})-([0-9]{2})T([0-9]{2})-([0-9]{2})-([0-9]{2})Z$/\1\2\3\4\5\6.0.0/')
if [ "${package_version}" = "${version_hyphen}" ]; then
echo "invalid MCLI_RELEASE: ${release}" >&2
exit 1
fi
case $(uname -s) in
Linux) os=linux ;;
Darwin) os=darwin ;;
*) echo "unsupported mcli host OS: $(uname -s)" >&2; exit 1 ;;
esac
case $(uname -m) in
x86_64 | amd64) arch=amd64 ;;
aarch64 | arm64) arch=arm64 ;;
*) echo "unsupported mcli host architecture: $(uname -m)" >&2; exit 1 ;;
esac
archive="mcli_${package_version}_${os}_${arch}.tar.gz"
checksums="mcli_${package_version}_checksums.txt"
base_url="https://github.com/pgsty/mc/releases/download/${release}"
tmp_dir=$(mktemp -d "${TMPDIR:-/tmp}/silo-mcli.XXXXXX")
trap 'rm -rf "${tmp_dir}"' EXIT
curl --fail --location --retry 3 --silent --show-error \
"${base_url}/${checksums}" --output "${tmp_dir}/${checksums}"
curl --fail --location --retry 3 --silent --show-error \
"${base_url}/${archive}" --output "${tmp_dir}/${archive}"
expected=$(awk -v asset="${archive}" '
{
name=$2
sub(/^\*/, "", name)
if (name == asset && length($1) == 64 && $1 ~ /^[0-9a-fA-F]+$/) print tolower($1)
}
' "${tmp_dir}/${checksums}")
if ! printf '%s\n' "${expected}" | grep -Eq '^[0-9a-f]{64}$'; then
echo "checksum manifest does not contain exactly one valid entry for ${archive}" >&2
exit 1
fi
actual=$(sha256_file "${tmp_dir}/${archive}")
if [ "${actual}" != "${expected}" ]; then
echo "downloaded ${archive} checksum mismatch" >&2
exit 1
fi
tar -xzf "${tmp_dir}/${archive}" -C "${tmp_dir}" mcli
install -m 0755 "${tmp_dir}/mcli" "${target}"
+48
View File
@@ -0,0 +1,48 @@
#!/usr/bin/env bash
set -euo pipefail
if [ "$#" -ne 3 ]; then
echo "usage: $0 SOURCE SHA256 TARGET" >&2
exit 2
fi
source_ref=$1
expected=${2,,}
target=$3
if ! printf '%s\n' "${expected}" | grep -Eq '^[0-9a-f]{64}$'; then
echo "expected checksum must be a lowercase SHA-256 digest" >&2
exit 1
fi
if [ ! -d "$(dirname "${target}")" ]; then
echo "target directory does not exist: $(dirname "${target}")" >&2
exit 1
fi
tmp_file=$(mktemp "${TMPDIR:-/tmp}/silo-fixture.XXXXXX")
trap 'rm -f "${tmp_file}"' EXIT
case ${source_ref} in
https://*)
curl --fail --location --retry 3 --silent --show-error \
"${source_ref}" --output "${tmp_file}"
;;
*)
if [ ! -f "${source_ref}" ]; then
echo "fixture is not a regular file: ${source_ref}" >&2
exit 1
fi
cp "${source_ref}" "${tmp_file}"
;;
esac
if command -v sha256sum >/dev/null 2>&1; then
actual=$(sha256sum "${tmp_file}" | awk '{print $1}')
else
actual=$(shasum -a 256 "${tmp_file}" | awk '{print $1}')
fi
if [ "${actual}" != "${expected}" ]; then
echo "fixture checksum mismatch: expected ${expected}, got ${actual}" >&2
exit 1
fi
install -m 0755 "${tmp_file}" "${target}"
@@ -1,7 +1,7 @@
#!/bin/bash
# This script is used to test the migration of IAM content from old minio
# instance to new minio instance.
# This script tests IAM migration from an old MinIO compatibility fixture into
# the current Silo server.
#
# To run it locally, start the LDAP server in github.com/minio/minio-iam-testing
# repo (e.g. make podman-run), and then run this script.
@@ -15,6 +15,7 @@
OLD_VERSION=RELEASE.2024-03-26T22-10-45Z
OLD_BINARY_LINK=https://dl.min.io/server/minio/release/linux-amd64/archive/minio.${OLD_VERSION}
OLD_BINARY_SHA256=2050199d89e3057571620a1d453118fed5bd2de9d4f3b266b11365fdf984d676
__init__() {
if which curl &>/dev/null; then
@@ -27,17 +28,16 @@ __init__() {
export GOPATH=/tmp/gopath
export PATH="${PATH}":"${GOPATH}"/bin
if which mc &>/dev/null; then
echo "mc is already installed"
else
echo "Installing mc:"
go install github.com/minio/mc@latest
if [ ! -x "${GOPATH}/bin/mc" ]; then
echo "Installing verified compatible client fixture"
mkdir -p "${GOPATH}/bin"
"$(git rev-parse --show-toplevel)/buildscripts/install-mcli.sh" "${GOPATH}/bin/mc"
fi
if [ ! -x ./minio.${OLD_VERSION} ]; then
echo "Downloading minio.${OLD_VERSION} binary"
curl -o minio.${OLD_VERSION} ${OLD_BINARY_LINK}
chmod +x minio.${OLD_VERSION}
echo "Installing verified upstream compatibility fixture minio.${OLD_VERSION}"
"$(git rev-parse --show-toplevel)/buildscripts/install-verified-fixture.sh" \
"${OLD_BINARY_LINK}" "${OLD_BINARY_SHA256}" "minio.${OLD_VERSION}"
fi
if [ -z "$_MINIO_LDAP_TEST_SERVER" ]; then
@@ -49,7 +49,7 @@ __init__() {
}
create_iam_content_in_old_minio() {
echo "Creating IAM content in old minio instance."
echo "Creating IAM content in the old MinIO compatibility fixture."
MINIO_CI_CD=1 ./minio.${OLD_VERSION} server /tmp/data/{1...4} &
sleep 5
@@ -80,9 +80,9 @@ create_iam_content_in_old_minio() {
}
import_iam_content_in_new_minio() {
echo "Importing IAM content in new minio instance."
# Assume current minio binary exists.
MINIO_CI_CD=1 ./minio server /tmp/data/{1...4} &
echo "Importing IAM content into the current Silo instance."
# Assume the current Silo binary exists.
MINIO_CI_CD=1 ./silo server /tmp/data/{1...4} &
sleep 5
set -x
+113 -102
View File
@@ -1,113 +1,124 @@
#!/bin/bash
#!/usr/bin/env bash
trap 'cleanup $LINENO' ERR
set -euo pipefail
# Exercise both directions of the on-disk compatibility contract using an
# immutable pre-rebrand image and a container built from the current checkout.
# Every Docker resource is uniquely named and removed explicitly; this test
# never prunes unrelated images, containers, networks, or volumes.
repo_dir="$(git rev-parse --show-toplevel)"
old_image="${OLD_IMAGE:-docker.io/pgsty/minio@sha256:b6bfe7239bfc83fb90d31612d9704d86039dd714f7904b3f1ad68f211e602372}"
new_image="${NEW_IMAGE:-silo-upgrade-test:dev}"
suffix="$(date +%s)-$$"
network="silo-upgrade-net-${suffix}"
volume="silo-upgrade-data-${suffix}"
old_container="silo-upgrade-old-${suffix}"
new_container="silo-upgrade-new-${suffix}"
rollback_container="silo-upgrade-rollback-${suffix}"
root_user=silo-upgrade-admin
root_password=silo-upgrade-secret-123
# shellcheck disable=SC2120
cleanup() {
MINIO_VERSION=dev /tmp/gopath/bin/docker-compose \
-f "buildscripts/upgrade-tests/compose.yml" \
down || true
MINIO_VERSION=dev /tmp/gopath/bin/docker-compose \
-f "buildscripts/upgrade-tests/compose.yml" \
rm || true
for volume in $(docker volume ls -q | grep upgrade); do
docker volume rm ${volume} || true
done
docker volume prune -f
docker system prune -f || true
docker volume prune -f || true
docker volume rm $(docker volume ls -q -f dangling=true) || true
}
verify_checksum_after_heal() {
local sum1
sum1=$(curl -s "$2" | sha256sum)
mc admin heal --json -r "$1" >/dev/null # test after healing
local sum1_heal
sum1_heal=$(curl -s "$2" | sha256sum)
if [ "${sum1_heal}" != "${sum1}" ]; then
echo "mismatch expected ${sum1_heal}, got ${sum1}"
exit 1
fi
}
verify_checksum_mc() {
local expected
expected=$(mc cat "$1" | sha256sum)
local got
got=$(mc cat "$2" | sha256sum)
if [ "${expected}" != "${got}" ]; then
echo "mismatch - expected ${expected}, got ${got}"
exit 1
fi
echo "matches - ${expected}, got ${got}"
}
add_alias() {
for i in $(seq 1 4); do
echo "... attempting to add alias $i"
until (mc alias set minio http://127.0.0.1:9000 minioadmin minioadmin); do
echo "...waiting... for 5secs" && sleep 5
status=$?
trap - EXIT
if [ "${status}" -ne 0 ]; then
for name in "${old_container}" "${new_container}" "${rollback_container}"; do
docker logs "${name}" 2>/dev/null | tail -n 80 >&2 || true
done
fi
if [ "${KEEP_UPGRADE_TEST_RESOURCES:-0}" = 1 ]; then
printf 'Retained Docker resources for inspection: %s %s\n' "${network}" "${volume}" >&2
exit "${status}"
fi
docker rm -f "${old_container}" "${new_container}" "${rollback_container}" >/dev/null 2>&1 || true
docker network rm "${network}" >/dev/null 2>&1 || true
docker volume rm "${volume}" >/dev/null 2>&1 || true
exit "${status}"
}
trap cleanup EXIT
trap 'exit 130' INT TERM
wait_ready() {
name="$1"
ready=""
for _ in $(seq 1 90); do
if docker logs "${name}" 2>&1 | grep -q 'API:'; then
ready=1
break
fi
if [ "$(docker inspect -f '{{.State.Running}}' "${name}")" != true ]; then
break
fi
sleep 1
done
echo "Sleeping for nginx"
sleep 20
if [ -z "${ready}" ]; then
echo "Server did not become ready: ${name}" >&2
return 1
fi
}
__init__() {
sudo apt install curl -y
export GOPATH=/tmp/gopath
export PATH=${PATH}:${GOPATH}/bin
go install github.com/minio/mc@latest
## this is needed because github actions don't have
## docker-compose on all runners
COMPOSE_VERSION=v2.35.1
mkdir -p /tmp/gopath/bin/
wget -O /tmp/gopath/bin/docker-compose https://github.com/docker/compose/releases/download/${COMPOSE_VERSION}/docker-compose-linux-x86_64
chmod +x /tmp/gopath/bin/docker-compose
cleanup
TAG=minio/minio:dev make docker
MINIO_VERSION=RELEASE.2019-12-19T22-52-26Z docker-compose \
-f "buildscripts/upgrade-tests/compose.yml" \
up -d --build
add_alias
mc mb minio/minio-test/
mc cp ./minio minio/minio-test/to-read/
mc cp /etc/hosts minio/minio-test/to-read/hosts
mc anonymous set download minio/minio-test
verify_checksum_mc ./minio minio/minio-test/to-read/minio
curl -s http://127.0.0.1:9000/minio-test/to-read/hosts | sha256sum
MINIO_VERSION=dev /tmp/gopath/bin/docker-compose -f "buildscripts/upgrade-tests/compose.yml" stop
start_server() {
name="$1"
image="$2"
shift 2
docker run -d --name "${name}" --network "${network}" \
--mount "source=${volume},target=/data" \
-e MINIO_CI_CD=1 -e MINIO_ROOT_USER="${root_user}" -e MINIO_ROOT_PASSWORD="${root_password}" \
"${image}" "$@" >/dev/null
wait_ready "${name}"
docker exec "${name}" mcli alias set local http://127.0.0.1:9000 "${root_user}" "${root_password}" >/dev/null
}
main() {
MINIO_VERSION=dev /tmp/gopath/bin/docker-compose -f "buildscripts/upgrade-tests/compose.yml" up -d --build
add_alias
verify_checksum_after_heal minio/minio-test http://127.0.0.1:9000/minio-test/to-read/hosts
verify_checksum_mc ./minio minio/minio-test/to-read/minio
verify_checksum_mc /etc/hosts minio/minio-test/to-read/hosts
cleanup
stop_server() {
name="$1"
docker stop -t 20 "${name}" >/dev/null
test "$(docker inspect -f '{{.State.ExitCode}}' "${name}")" = 0
docker logs "${name}" 2>&1 | grep -q 'Exiting on signal'
docker rm "${name}" >/dev/null
}
(__init__ "$@" && main "$@")
command -v docker >/dev/null
docker info >/dev/null
if ! docker image inspect "${old_image}" >/dev/null 2>&1; then
docker pull "${old_image}"
fi
if [ "${SILO_UPGRADE_SKIP_BUILD:-0}" != 1 ]; then
make -C "${repo_dir}" docker TAG="${new_image}"
fi
docker image inspect "${new_image}" >/dev/null
docker network create "${network}" >/dev/null
docker volume create "${volume}" >/dev/null
start_server "${old_container}" "${old_image}" minio server /data --address :9000
docker exec "${old_container}" mcli mb local/compat >/dev/null
docker exec "${old_container}" mcli version enable local/compat >/dev/null
printf 'old-version-1\n' | docker exec -i "${old_container}" mcli pipe local/compat/versioned.txt >/dev/null
printf 'old-version-2\n' | docker exec -i "${old_container}" mcli pipe local/compat/versioned.txt >/dev/null
test "$(docker exec "${old_container}" mcli ls --versions local/compat/versioned.txt | grep -c 'versioned.txt')" -ge 2
docker exec "${old_container}" mcli mb --with-lock local/locked >/dev/null
printf 'locked-by-old\n' | docker exec -i "${old_container}" mcli pipe local/locked/object.txt >/dev/null
dd if=/dev/zero bs=1048576 count=70 2>/dev/null | docker exec -i "${old_container}" mcli pipe local/compat/multipart.bin >/dev/null
test "$(docker exec "${old_container}" mcli stat --json local/compat/multipart.bin | jq -r '.size')" = 73400320
docker exec "${old_container}" mcli admin user add local migration-user migration-secret-123 >/dev/null
docker exec "${old_container}" mcli admin policy attach local readwrite --user migration-user >/dev/null
stop_server "${old_container}"
start_server "${new_container}" "${new_image}" silo server /data --address :9000
test "$(docker exec "${new_container}" mcli cat local/compat/versioned.txt)" = old-version-2
test "$(docker exec "${new_container}" mcli cat local/locked/object.txt)" = locked-by-old
test "$(docker exec "${new_container}" mcli stat --json local/compat/multipart.bin | jq -r '.size')" = 73400320
docker exec "${new_container}" mcli admin user info local migration-user >/dev/null
docker exec "${new_container}" mcli alias set migrated http://127.0.0.1:9000 migration-user migration-secret-123 >/dev/null
printf 'written-by-silo\n' | docker exec -i "${new_container}" mcli pipe migrated/compat/silo.txt >/dev/null
stop_server "${new_container}"
start_server "${rollback_container}" "${old_image}" minio server /data --address :9000
test "$(docker exec "${rollback_container}" mcli cat local/compat/versioned.txt)" = old-version-2
test "$(docker exec "${rollback_container}" mcli cat local/compat/silo.txt)" = written-by-silo
test "$(docker exec "${rollback_container}" mcli stat --json local/compat/multipart.bin | jq -r '.size')" = 73400320
stop_server "${rollback_container}"
echo "MinIO-to-Silo data upgrade and rollback checks passed"
+19 -30
View File
@@ -5,16 +5,16 @@ if [ -n "$TEST_DEBUG" ]; then
fi
WORK_DIR="$PWD/.verify-$RANDOM"
MINIO_CONFIG_DIR="$WORK_DIR/.minio"
MINIO=("$PWD/minio" --config-dir "$MINIO_CONFIG_DIR" server)
SILO_CONFIG_DIR="$WORK_DIR/.silo"
SILO=("$PWD/silo" --config-dir "$SILO_CONFIG_DIR" server)
if [ ! -x "$PWD/minio" ]; then
echo "minio executable binary not found in current directory"
if [ ! -x "$PWD/silo" ]; then
echo "Silo executable binary not found in current directory"
exit 1
fi
if [ ! -x "$PWD/minio" ]; then
echo "minio executable binary not found in current directory"
if [ ! -x "$PWD/silo" ]; then
echo "Silo executable binary not found in current directory"
exit 1
fi
@@ -30,9 +30,9 @@ catch() {
echo "error on line $1"
fi
echo "Cleaning up instances of MinIO"
pkill minio || true
pkill -9 minio || true
echo "Cleaning up instances of Silo"
pkill silo || true
pkill -9 silo || true
purge "$WORK_DIR"
if [ $# -ne 0 ]; then
exit $#
@@ -41,32 +41,21 @@ catch() {
catch
function start_minio_10drive() {
function start_silo_10drive() {
start_port=$1
export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=minio123
export MC_HOST_minio="http://minio:minio123@127.0.0.1:${start_port}/"
export MINIO_ROOT_USER=silo
export MINIO_ROOT_PASSWORD=silo1234
export MC_HOST_silo="http://silo:silo1234@127.0.0.1:${start_port}/"
unset MINIO_KMS_AUTO_ENCRYPTION # do not auto-encrypt objects
export MINIO_CI_CD=1
mkdir ${WORK_DIR}
C_PWD=${PWD}
if [ ! -x "$PWD/mc" ]; then
MC_BUILD_DIR="mc-$RANDOM"
if ! git clone --quiet https://github.com/minio/mc "$MC_BUILD_DIR"; then
echo "failed to download https://github.com/minio/mc"
purge "${MC_BUILD_DIR}"
exit 1
fi
(cd "${MC_BUILD_DIR}" && go build -o "$C_PWD/mc")
# remove mc source.
purge "${MC_BUILD_DIR}"
"$(git rev-parse --show-toplevel)/buildscripts/install-mcli.sh" "$PWD/mc"
fi
"${MINIO[@]}" --address ":$start_port" "${WORK_DIR}/disk{1...10}" >"${WORK_DIR}/server1.log" 2>&1 &
"${SILO[@]}" --address ":$start_port" "${WORK_DIR}/disk{1...10}" >"${WORK_DIR}/server1.log" 2>&1 &
pid=$!
disown $pid
sleep 5
@@ -79,10 +68,10 @@ function start_minio_10drive() {
exit 1
fi
"${PWD}/mc" mb --with-versioning minio/bucket
"${PWD}/mc" mb --with-versioning silo/bucket
export AWS_ACCESS_KEY_ID=minio
export AWS_SECRET_ACCESS_KEY=minio123
export AWS_ACCESS_KEY_ID=silo
export AWS_SECRET_ACCESS_KEY=silo1234
aws --endpoint-url http://localhost:"$start_port" s3api create-multipart-upload --bucket bucket --key obj-1 >upload-id.json
uploadId=$(jq -r '.UploadId' upload-id.json)
@@ -120,7 +109,7 @@ EOF
function main() {
start_port=$(shuf -i 10000-65000 -n 1)
start_minio_10drive ${start_port}
start_silo_10drive ${start_port}
}
main "$@"
+50 -13
View File
@@ -20,6 +20,10 @@ if ! [[ "${PKG_VERSION}" =~ ^[0-9]{14}\.0\.0$ ]]; then
exit 1
fi
# The PGSTY release segment, PGDG-style. sign-release-rpms.sh declares the
# same value as expected_release, and test-release.yml asserts the two agree.
PKG_RELEASE="1PGSTY"
if ! command -v nfpm >/dev/null 2>&1; then
echo "nfpm is required" >&2
exit 1
@@ -34,11 +38,37 @@ fi
# against the config file, so the unit path is passed in absolute. Otherwise
# this only works when invoked from the repository root and fails elsewhere on
# a message that names the file rather than the cause.
unit_file="${repo_dir}/minio.service"
unit_file="${repo_dir}/silo.service"
defaults_file="${repo_dir}/silo.env"
sysusers_file="${repo_dir}/silo.sysusers"
license_file="${repo_dir}/LICENSE"
notice_file="${repo_dir}/NOTICE"
postinstall_file="${repo_dir}/buildscripts/package/postinstall.sh"
preremove_file="${repo_dir}/buildscripts/package/preremove.sh"
if [ ! -f "${unit_file}" ]; then
echo "Missing systemd unit: ${unit_file}" >&2
exit 1
fi
if [ ! -f "${defaults_file}" ]; then
echo "Missing defaults file: ${defaults_file}" >&2
exit 1
fi
if [ ! -f "${sysusers_file}" ]; then
echo "Missing sysusers file: ${sysusers_file}" >&2
exit 1
fi
for distributed_doc in "${license_file}" "${notice_file}"; do
if [ ! -s "${distributed_doc}" ]; then
echo "Missing license material: ${distributed_doc}" >&2
exit 1
fi
done
for lifecycle_script in "${postinstall_file}" "${preremove_file}"; do
if [ ! -x "${lifecycle_script}" ]; then
echo "Missing executable package lifecycle script: ${lifecycle_script}" >&2
exit 1
fi
done
packages_dir="${dist_dir}/packages"
mkdir -p "${packages_dir}"
@@ -68,7 +98,7 @@ find_binary() {
# variants (an added goamd64 level, a stale dist entry) would silently ship a
# package whose contents do not match its name.
matches="$(find "${dist_dir}" -maxdepth 2 -type f \
-path "${dist_dir}/minio_linux_${goarch}*/minio" | sort)"
-path "${dist_dir}/silo_linux_${goarch}*/silo" | sort)"
count="$(printf '%s' "${matches}" | grep -c . || true)"
if [ "${count}" -eq 0 ]; then
@@ -96,18 +126,25 @@ build_arch() {
source="$(find_binary "${goarch}")"
# These names are the public download names and must not drift; RPM carries a
# release number, DEB and APK do not, matching what pkger produced.
rpm_file="${packages_dir}/minio-${PKG_VERSION}-1.${rpm_arch}.rpm"
deb_file="${packages_dir}/minio_${PKG_VERSION}_${deb_arch}.deb"
apk_file="${packages_dir}/minio_${PKG_VERSION}_${apk_arch}.apk"
# These names are the public download names and must not drift; RPM and DEB
# carry the PGSTY release number (nfpm renders it as the RPM Release tag and
# as the Debian revision after a dash). APK stays bare: Alpine pkgrel only
# admits -r<integer>, so a lettered release cannot ride along there.
rpm_file="${packages_dir}/silo-${PKG_VERSION}-${PKG_RELEASE}.${rpm_arch}.rpm"
deb_file="${packages_dir}/silo_${PKG_VERSION}-${PKG_RELEASE}_${deb_arch}.deb"
apk_file="${packages_dir}/silo_${PKG_VERSION}_${apk_arch}.apk"
PKG_VERSION="${PKG_VERSION}" NFPM_RELEASE=1 NFPM_ARCH="${goarch}" NFPM_SOURCE="${source}" NFPM_UNIT="${unit_file}" \
nfpm package --config "${nfpm_config}" --packager rpm --target "${rpm_file}"
PKG_VERSION="${PKG_VERSION}" NFPM_RELEASE='' NFPM_ARCH="${goarch}" NFPM_SOURCE="${source}" NFPM_UNIT="${unit_file}" \
nfpm package --config "${nfpm_config}" --packager deb --target "${deb_file}"
PKG_VERSION="${PKG_VERSION}" NFPM_RELEASE='' NFPM_ARCH="${goarch}" NFPM_SOURCE="${source}" NFPM_UNIT="${unit_file}" \
nfpm package --config "${nfpm_config}" --packager apk --target "${apk_file}"
(
cd "${repo_dir}"
export NFPM_UNIT="${unit_file}" NFPM_DEFAULTS="${defaults_file}" NFPM_SYSUSERS="${sysusers_file}" \
NFPM_LICENSE="${license_file}" NFPM_NOTICE="${notice_file}"
PKG_VERSION="${PKG_VERSION}" NFPM_RELEASE="${PKG_RELEASE}" NFPM_ARCH="${goarch}" NFPM_SOURCE="${source}" \
nfpm package --config "${nfpm_config}" --packager rpm --target "${rpm_file}"
PKG_VERSION="${PKG_VERSION}" NFPM_RELEASE="${PKG_RELEASE}" NFPM_ARCH="${goarch}" NFPM_SOURCE="${source}" \
nfpm package --config "${nfpm_config}" --packager deb --target "${deb_file}"
PKG_VERSION="${PKG_VERSION}" NFPM_RELEASE='' NFPM_ARCH="${goarch}" NFPM_SOURCE="${source}" \
nfpm package --config "${nfpm_config}" --packager apk --target "${apk_file}"
)
sha256_file "${rpm_file}"
sha256_file "${deb_file}"
+172
View File
@@ -0,0 +1,172 @@
#!/usr/bin/env bash
set -euo pipefail
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
repo_dir="$(cd "${script_dir}/../.." && pwd)"
postinstall="${script_dir}/postinstall.sh"
preremove="${script_dir}/preremove.sh"
test_dir="$(mktemp -d)"
fakebin="${test_dir}/bin"
log_file="${test_dir}/calls.log"
useradd_shell=/usr/sbin/nologin
[ -x "${useradd_shell}" ] || useradd_shell=/sbin/nologin
busybox_shell=/sbin/nologin
[ -x "${busybox_shell}" ] || busybox_shell=/bin/false
cleanup() {
rm -rf "${test_dir}"
}
trap cleanup EXIT
mkdir -p "${fakebin}"
touch "${log_file}"
# One dispatcher represents every external command used by the lifecycle
# scripts. The tested scripts run with no host utilities in PATH, so a green
# result cannot create a real account or touch the host service manager.
cat > "${fakebin}/fake-command" <<'EOF'
#!/bin/sh
set -eu
command_name=${0##*/}
case "${command_name}" in
id)
[ "${PACKAGE_TEST_USER_EXISTS:-0}" = 1 ]
;;
getent)
[ "${PACKAGE_TEST_GROUP_EXISTS:-0}" = 1 ]
;;
systemd-sysusers|useradd|addgroup|adduser|systemctl)
{
printf '%s' "${command_name}"
for argument in "$@"; do
printf ' %s' "${argument}"
done
printf '\n'
} >> "${PACKAGE_TEST_LOG}"
;;
*)
echo "unexpected fake command: ${command_name}" >&2
exit 1
;;
esac
EOF
chmod +x "${fakebin}/fake-command"
link_command() {
ln -sf fake-command "${fakebin}/$1"
}
unlink_optional_commands() {
rm -f \
"${fakebin}/systemd-sysusers" \
"${fakebin}/useradd" \
"${fakebin}/adduser" \
"${fakebin}/addgroup"
}
reset_log() {
: > "${log_file}"
}
run_postinstall() {
PACKAGE_TEST_LOG="${log_file}" \
PACKAGE_TEST_USER_EXISTS="${1}" \
PACKAGE_TEST_GROUP_EXISTS="${2}" \
PATH="${fakebin}" \
/bin/sh "${postinstall}"
}
run_preremove() {
PACKAGE_TEST_LOG="${log_file}" PATH="${fakebin}" \
/bin/sh "${preremove}" "$@"
}
assert_log_line() {
grep -Fx -- "$1" "${log_file}" >/dev/null
}
reject_log_text() {
if grep -F -- "$1" "${log_file}" >/dev/null; then
echo "unexpected lifecycle call containing '$1':" >&2
cat "${log_file}" >&2
exit 1
fi
}
link_command id
link_command getent
link_command systemctl
# Clean install through systemd-sysusers. Side-by-side safety is represented
# by the fact that the only service-manager operation is daemon-reload: no old
# service is stopped, disabled, enabled, masked, or restarted.
unlink_optional_commands
link_command systemd-sysusers
reset_log
run_postinstall 0 0
assert_log_line "systemd-sysusers /usr/lib/sysusers.d/silo.conf"
assert_log_line "systemctl daemon-reload"
test "$(wc -l < "${log_file}" | tr -d ' ')" -eq 2
# An existing service account is preserved without modification.
reset_log
run_postinstall 1 0
test "$(cat "${log_file}")" = "systemctl daemon-reload"
# useradd creates a private group only when one does not already exist. An
# administrator may pre-create group silo with the legacy GID; that group must
# be reused rather than causing installation to fail.
unlink_optional_commands
link_command useradd
reset_log
run_postinstall 0 0
assert_log_line "useradd --system --user-group --no-create-home --shell ${useradd_shell} --comment Silo object storage service silo"
reject_log_text "--gid silo"
reset_log
run_postinstall 0 1
assert_log_line "useradd --system --gid silo --no-create-home --shell ${useradd_shell} --comment Silo object storage service silo"
reject_log_text "--user-group"
# BusyBox follows the same existing-group contract.
unlink_optional_commands
link_command adduser
link_command addgroup
reset_log
run_postinstall 0 0
assert_log_line "addgroup -S silo"
assert_log_line "adduser -S -D -H -G silo -s ${busybox_shell} silo"
reset_log
run_postinstall 0 1
reject_log_text "addgroup"
assert_log_line "adduser -S -D -H -G silo -s ${busybox_shell} silo"
# Debian remove, RPM erase, and Alpine deinstall stop the Silo unit. Upgrade
# arguments must leave the running service alone.
for removal_argument in remove 0 20260214120000.0.0-r0; do
reset_log
run_preremove "${removal_argument}"
test "$(cat "${log_file}")" = "systemctl disable --now silo.service"
done
for upgrade_argument in upgrade 1; do
reset_log
run_preremove "${upgrade_argument}"
test ! -s "${log_file}"
done
# The package deliberately leaves legacy ownership changes to an explicit
# systemd drop-in. Lifecycle scripts must never rewrite ownership or touch the
# old unit, and the base unit must expose overridable User/Group directives.
grep -Fx 'User=silo' "${repo_dir}/silo.service" >/dev/null
grep -Fx 'Group=silo' "${repo_dir}/silo.service" >/dev/null
if grep -Ein '\b(chown|chgrp|usermod|groupmod)\b|minio\.service' \
"${postinstall}" "${preremove}"; then
echo "package lifecycle scripts must not mutate data ownership or the legacy service" >&2
exit 1
fi
echo "Silo package lifecycle checks passed"
+44
View File
@@ -0,0 +1,44 @@
#!/bin/sh
set -eu
sysusers_file=/usr/lib/sysusers.d/silo.conf
group_exists() {
if command -v getent >/dev/null 2>&1; then
getent group silo >/dev/null 2>&1
return
fi
[ -r /etc/group ] || return 1
while IFS=: read -r group_name _; do
[ "${group_name}" = silo ] && return 0
done < /etc/group
return 1
}
if ! id -u silo >/dev/null 2>&1; then
if command -v systemd-sysusers >/dev/null 2>&1; then
systemd-sysusers "${sysusers_file}"
elif command -v useradd >/dev/null 2>&1; then
nologin_shell=/usr/sbin/nologin
[ -x "${nologin_shell}" ] || nologin_shell=/sbin/nologin
if group_exists; then
useradd --system --gid silo --no-create-home --shell "${nologin_shell}" --comment "Silo object storage service" silo
else
useradd --system --user-group --no-create-home --shell "${nologin_shell}" --comment "Silo object storage service" silo
fi
elif command -v adduser >/dev/null 2>&1 && command -v addgroup >/dev/null 2>&1; then
nologin_shell=/sbin/nologin
[ -x "${nologin_shell}" ] || nologin_shell=/bin/false
group_exists || addgroup -S silo
adduser -S -D -H -G silo -s "${nologin_shell}" silo
else
echo "Unable to create the silo system account: systemd-sysusers, useradd, or BusyBox adduser is required" >&2
exit 1
fi
fi
if command -v systemctl >/dev/null 2>&1; then
systemctl daemon-reload >/dev/null 2>&1 || true
fi
+14
View File
@@ -0,0 +1,14 @@
#!/bin/sh
set -eu
# Debian passes "remove" for an actual removal, RPM passes 0 to %preun, and
# Alpine runs pre-deinstall only for removal and passes the old dotted version.
# Upgrade paths deliberately leave the running service untouched.
case "${1:-}" in
remove|0|*.*)
if command -v systemctl >/dev/null 2>&1; then
systemctl disable --now silo.service >/dev/null 2>&1 || true
fi
;;
esac
+52
View File
@@ -0,0 +1,52 @@
-----BEGIN PGP PUBLIC KEY BLOCK-----
mQINBGaV5PwBEACbErI+7yOrsXTT3mR83O6Fw9WyHJqozhyNPF3dA1gAtWpfWqd4
S9x6vBjVwUbIRn21jYgov0hDiaLABNQhRzifvVr0r1IjBW8lhA8zJGaO42Uz0aBW
YIkajOklsXgYMX+gSmy5WXzM31sDQVMnzptHh9dwW067hMM5pJKDslu2pLMwSb9K
QgIFcYsaR0taBkcDg4dNu1gncriD/GcdXIS0/V4R82DIYeIqj2S0lt0jDTACbUz3
C6esrTw2XerCeHKHb9c/V+KMhqvLJOOpy/aJWLrTGBoaH7xw6v0qg32OYiBxlUj9
VEzoQbDfbRkR+jlxiuYP3scUs/ziKrSh+0mshVbeuLRSNfuHLa7C4xTEnATcgD1J
MZeMaJXIcDt+DN+1aHVQjY5YNvr5wA3ykxW51uReZf7/odgqVW3+1rhW5pd8NQKQ
qoVUHOtIrC9KaiGfrczEtJTNUxcNZV9eBgcKHYDXB2hmR2pIf7WvydgXTs/qIsXg
SIzfKjisi795Dd5GrvdLYXVnu9YzylWlkJ5rjod1wnSxkI/CcCJaoPLnXZA9KV7A
cpMWWaUEXP/XBIwIU+vxDd1taBIaPIOv1KIdzvG7QqAQtf5Lphi5HfaGvBud/CVt
mvWhRPJMr1J0ER2xAgU2iZR7dN0vSF6zDqc0W09RAoC0nDS3tupDX2BrOwARAQAB
tCRSdW9oYW5nIEZlbmcgKFBpZ3N0eSkgPHJoQHZvbm5nLmNvbT6JAlEEEwEIADsW
IQSVkqe8emguczM3bgnnk12Nub2LIAUCZpXk/AIbAwULCQgHAgIiAgYVCgkICwIE
FgIDAQIeBwIXgAAKCRDnk12Nub2LIOMuEACBLVc09O4icFwc45R3KMvOMu14Egpn
UkpmBKhErjup0TIunzI0zZH6HG8LGuf6XEdH4ItCJeLg5349UE00BUHNmxk2coo2
u4Wtu28LPqmxb6sqpuRAaefedU6vqfs7YN6WWp52pVF1KdOHkIOcgAQ9z3ZHdosM
I/Y/UxO2t4pjdCAfJHOmGPrbgLcHSMpoLLxjuf3YIwS5NSfjNDd0Y8sKFUcMGLCF
5P0lv5feLLdZvh2Una34UmHKhZlXC5E3vlY9bf/LgsRzXRFQosD0RsCXbz3Tk+zF
+j/eP3WhUvJshqIDuY6eJYCzMjiA8sM5gety+htVJuD0mewp+qAhjxE0d4bIr4qO
BKQzBt9tT2ackCPdgW42VPS+IZymm1oMET0hgZfKiVpwsKO6qxeWn4RW2jJ0zkUJ
MsrrxOPFdZQAtuFcLwa5PUAHHs6XQT2vzxDpeE9lInQ14lshofU5ZKIeb9sbvb/w
P+xnDqvZ1pcotEIBvDK0S0jHbHHqtioIUdDFvdCBlBlYP1TQRNPlJ7TJDBBvhj8i
fmjQsYSV1u36aHOJVGYNHv+SyJpVd3nHCZn97ADM9qHnDm7xljyHXPzIx4FMmBGJ
UTiLH5yxa1xhWr42Iv3TykaQJVbpydmBuegFR8WbWitAvVqI3HvRG+FalLsjJruc
8YDAf7gHdj/937kCDQRmleT8ARAAmJxscC76NZzqFBiaeq2+aJxOt1HGPqKb4pbz
jLKRX9sFkeXuzhfZaNDljnr2yrnQ75rit9Aah/loEhbSHanNUDCNmvOeSEISr9yA
yfOnqlcVOtcwWQK57n6MvlCSM8Js3jdoSmCFHVtdFFwxejE5ok0dk1VFYDIg6DRk
ZBMuxGO7ZJW7TzCxhK4AL+NNYA2wX6b+IVMn6CA9kwNwCNrrnGHR1sblSxZp7lPo
+GsqzYY0LXGR2eEicgKd4lk38gaO8Q4d1mlpX95vgdhGKxR+CM26y9QU0qrO1hXP
Fw6lX9HfIUkVNrqAa1mzgneYXivnLvcj8gc7bFAdweX4MyBHsmiPm32WqjUJFAmw
kcKYaiyfDJ+1wusa/b+7RCnshWc8B9udYbXfvcpOGgphpUuvomKT8at3ToJfEWmR
BzToYYTsgAAX8diY/X53BHCE/+MhLccglEUYNZyBRkTwDLrS9QgNkhrADaTwxsv1
8PwnVKve/ZxwOU0QGf4ZOhA2YQOE5hkRDR5uY2OHsOS5vHsd9Y6kNNnO8EBy99d1
QiBJOW3AP0nr4Cj1/NhdigAujsYRKiCAuPT7dgqART58VU4bZ3PgonMlziLe7+ht
YYxV+wyP6LVqicDd0MLLvG7r/JOiWuABOUxsFFaRecehoPJjeAEQxnWJjedokXKL
HVOFaEkAEQEAAYkCNgQYAQgAIBYhBJWSp7x6aC5zMzduCeeTXY25vYsgBQJmleT8
AhsMAAoJEOeTXY25vYsgG8sP/3UdsWuiwTsf/x4BTW82K+Uk9YwZDnUNH+4dUMED
bKT1C6CbuSZ7Mnbi2rVsmGzOMs9MehIx6Ko8/iCR2OCeWi8Q+wM+iffAfWuT1GK6
7f/VIfoYBUWEa+kvDcPgEbd5Tu7ZdUO/jROVBSlXRSjzK9LpIj7GozBTJ8Vqy5x7
oqbWPPEYtGDVHime8o6f5/wfhNgL3mFnoq6srK7KhwACwfTXlNqAlGiXGa30Yj+b
Cj6IvmxoII49E67/ovMEmzDCb3RXiaL6OATy25P+HQJvWvAam7Qq5Xn+bZg65Mup
vXq3zoX0a7EKXc5vsJVNtTlXO1ATdYszKP5uNzkHrNAN52VRYaowq1vPy/MVMbSI
rL/hTFKr7ZNhmC7jmS3OuJyCYQsfEerubtBUuc/W6JDc2oTI3xOG1S2Zj8f4PxLl
H7vMG4E+p6eOrUGw6VQXjFsH9GtwhkPh/ZGMKENb2+JztJ02674Cok4s5c/lZFKz
mmRUcNjX2bm2K0GfGG5/hAog/CHCeUZvwIh4hZLkdeJ1QsIYpN8xbvY7QP6yh4VB
XrL18+2sontZ45MsGResrRibB35x7IrCrxZsVtRJZthHqshiORPatgy+AiWcAtEv
UWEnnC1xBSasNebw4fSE8AJg9JMCRw+3GAetlotOeW9q7PN6yrXD9rGuV/QquQNd
/c7w
=4rRi
-----END PGP PUBLIC KEY BLOCK-----
File diff suppressed because it is too large Load Diff
+405
View File
@@ -0,0 +1,405 @@
// Copyright 2026 PGSTY contributors.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// rebrand-guard records the compatibility identifiers that a product rebrand
// must not accidentally rename. It intentionally excludes product branding and
// delivery names, which are validated by buildscripts/verify-rebrand.sh.
package main
import (
"bytes"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"flag"
"fmt"
"go/ast"
"go/parser"
"go/token"
"os"
"os/exec"
"path/filepath"
"regexp"
"sort"
"strconv"
"strings"
)
const manifestVersion = 4
var (
minioImportRE = regexp.MustCompile(`github\.com/minio/[A-Za-z0-9_./-]+`)
envRE = regexp.MustCompile(`\b_?MINIO_[A-Z0-9_]+\b`)
metricRE = regexp.MustCompile(`\bminio_[A-Za-z0-9_]+\b`)
headerRE = regexp.MustCompile(`(?i)\bx-minio-[a-z0-9_-]+\b`)
routeRE = regexp.MustCompile(`^/[A-Za-z0-9._~!$&'()*+,;=:@%/?{}=-]*`)
storageRE = regexp.MustCompile(`\.minio\.sys(?:/[A-Za-z0-9._${}-]+)*`)
policyRE = regexp.MustCompile(`(?:arn:minio|minio:s3)[A-Za-z0-9_:/.*${}-]*`)
brandRE = regexp.MustCompile(`(?i)(^|[^a-z0-9_])minio([^a-z0-9_]|$)`)
)
type manifest struct {
Version int `json:"version"`
ModulePath string `json:"module_path"`
MinioImports []string `json:"minio_imports"`
Environment []string `json:"environment"`
Metrics []string `json:"metrics"`
Headers []string `json:"headers"`
Routes []string `json:"routes"`
RouteRoots []string `json:"route_roots"`
GridRoutes []string `json:"grid_routes"`
StorageMarkers []string `json:"storage_markers"`
PolicyValues []string `json:"policy_values"`
BrandAllowlist []string `json:"brand_allowlist"`
}
func main() {
write := flag.Bool("write", false, "replace the checked-in compatibility baseline")
flag.Parse()
repo, err := gitOutput("rev-parse", "--show-toplevel")
if err != nil {
fatal(err)
}
repo = strings.TrimSpace(repo)
baselinePath := filepath.Join(repo, "buildscripts", "rebrand-guard", "compat-baseline.json")
current, err := collect(repo)
if err != nil {
fatal(err)
}
if *write {
if err := writeManifest(baselinePath, current); err != nil {
fatal(err)
}
fmt.Printf("wrote %s\n", baselinePath)
printSummary(current)
return
}
want, err := readManifest(baselinePath)
if err != nil {
fatal(err)
}
if err := compare(want, current); err != nil {
fatal(err)
}
printSummary(current)
fmt.Println("Silo rebrand compatibility baseline is unchanged")
}
func collect(repo string) (manifest, error) {
files, err := trackedFiles(repo)
if err != nil {
return manifest{}, err
}
sets := map[string]map[string]struct{}{
"imports": {},
"env": {},
"metrics": {},
"headers": {},
"routes": {},
"roots": {},
"grid": {},
"storage": {},
"policy": {},
"brand": {},
}
modulePath := ""
fset := token.NewFileSet()
for _, rel := range files {
// Migration notes and guard fixtures contain archived identifiers.
if rel == "SILO_REBRANDING_MIGRATION.md" ||
strings.HasPrefix(rel, "buildscripts/rebrand-guard/") ||
strings.HasPrefix(rel, "buildscripts/helm-migration-guard/") {
continue
}
path := filepath.Join(repo, filepath.FromSlash(rel))
data, err := os.ReadFile(path)
if errors.Is(err, os.ErrNotExist) {
continue
}
if err != nil {
return manifest{}, fmt.Errorf("read %s: %w", rel, err)
}
if bytes.IndexByte(data, 0) >= 0 {
continue
}
text := string(data)
addMatches(sets["env"], envRE, text, false)
addMatches(sets["headers"], headerRE, text, true)
addMatches(sets["storage"], storageRE, text, false)
addMatches(sets["policy"], policyRE, text, false)
if strings.HasSuffix(rel, ".go") && (strings.HasPrefix(rel, "cmd/") || strings.HasPrefix(rel, "internal/")) {
addMatches(sets["metrics"], metricRE, text, false)
}
if rel == "go.mod" {
addMatches(sets["imports"], minioImportRE, text, false)
for _, line := range strings.Split(text, "\n") {
fields := strings.Fields(line)
if len(fields) == 2 && fields[0] == "module" {
modulePath = fields[1]
break
}
}
}
if strings.HasSuffix(rel, ".go") {
file, err := parser.ParseFile(fset, path, data, parser.SkipObjectResolution)
if err != nil {
return manifest{}, fmt.Errorf("parse %s: %w", rel, err)
}
for _, spec := range file.Imports {
value, err := strconv.Unquote(spec.Path.Value)
if err == nil && strings.HasPrefix(value, "github.com/minio/") {
sets["imports"][value] = struct{}{}
}
}
if !strings.HasSuffix(rel, "_test.go") {
// Test files hold request paths for fixtures, not served routes.
collectStringMatches(sets["routes"], routeRE, file)
if strings.HasPrefix(rel, "cmd/") || strings.HasPrefix(rel, "internal/") {
collectBrandStrings(sets["brand"], rel, file)
}
}
collectNamedStringValues(sets["roots"], rel, file, "minioReservedBucket")
if rel == "internal/grid/manager.go" {
collectStringMatches(sets["grid"], routeRE, file)
}
}
}
// This was a shell-local PID variable in the generated inspect script,
// never a supported environment setting.
delete(sets["env"], "MINIO_SRVR_PID")
if modulePath == "" {
return manifest{}, errors.New("go.mod module path was not found")
}
return manifest{
Version: manifestVersion,
ModulePath: modulePath,
MinioImports: sorted(sets["imports"]),
Environment: sorted(sets["env"]),
Metrics: sorted(sets["metrics"]),
Headers: sorted(sets["headers"]),
Routes: sorted(sets["routes"]),
RouteRoots: sorted(sets["roots"]),
GridRoutes: sorted(sets["grid"]),
StorageMarkers: sorted(sets["storage"]),
PolicyValues: sorted(sets["policy"]),
BrandAllowlist: sorted(sets["brand"]),
}, nil
}
func collectBrandStrings(dst map[string]struct{}, rel string, file *ast.File) {
ast.Inspect(file, func(node ast.Node) bool {
literal, ok := node.(*ast.BasicLit)
if !ok || literal.Kind != token.STRING {
return true
}
value, err := strconv.Unquote(literal.Value)
if err != nil || !brandRE.MatchString(value) || strings.HasPrefix(value, "github.com/minio/") {
return true
}
dst[filepath.ToSlash(rel)+"="+strconv.Quote(value)] = struct{}{}
return true
})
}
func collectNamedStringValues(dst map[string]struct{}, rel string, file *ast.File, names ...string) {
wanted := make(map[string]struct{}, len(names))
for _, name := range names {
wanted[name] = struct{}{}
}
for _, decl := range file.Decls {
gen, ok := decl.(*ast.GenDecl)
if !ok {
continue
}
for _, rawSpec := range gen.Specs {
spec, ok := rawSpec.(*ast.ValueSpec)
if !ok {
continue
}
for i, name := range spec.Names {
if _, ok := wanted[name.Name]; !ok || i >= len(spec.Values) {
continue
}
literal, ok := spec.Values[i].(*ast.BasicLit)
if !ok || literal.Kind != token.STRING {
continue
}
value, err := strconv.Unquote(literal.Value)
if err == nil {
dst[filepath.ToSlash(rel)+":"+name.Name+"="+value] = struct{}{}
}
}
}
}
}
func collectStringMatches(dst map[string]struct{}, re *regexp.Regexp, file *ast.File) {
ast.Inspect(file, func(node ast.Node) bool {
literal, ok := node.(*ast.BasicLit)
if !ok || literal.Kind != token.STRING {
return true
}
value, err := strconv.Unquote(literal.Value)
if err == nil {
addMatches(dst, re, value, false)
}
return true
})
}
func trackedFiles(repo string) ([]string, error) {
cmd := exec.Command("git", "-C", repo, "ls-files", "--cached", "-z")
out, err := cmd.Output()
if err != nil {
return nil, fmt.Errorf("git ls-files: %w", err)
}
parts := bytes.Split(out, []byte{0})
files := make([]string, 0, len(parts))
for _, part := range parts {
if len(part) > 0 {
files = append(files, string(part))
}
}
return files, nil
}
func addMatches(dst map[string]struct{}, re *regexp.Regexp, text string, lower bool) {
for _, match := range re.FindAllString(text, -1) {
if lower {
match = strings.ToLower(match)
}
dst[match] = struct{}{}
}
}
func sorted(set map[string]struct{}) []string {
values := make([]string, 0, len(set))
for value := range set {
values = append(values, value)
}
sort.Strings(values)
return values
}
func writeManifest(path string, value manifest) error {
data, err := json.MarshalIndent(value, "", " ")
if err != nil {
return err
}
data = append(data, '\n')
return os.WriteFile(path, data, 0o644)
}
func readManifest(path string) (manifest, error) {
data, err := os.ReadFile(path)
if err != nil {
return manifest{}, fmt.Errorf("read compatibility baseline (run go run ./buildscripts/rebrand-guard --write once): %w", err)
}
var value manifest
if err := json.Unmarshal(data, &value); err != nil {
return manifest{}, err
}
if value.Version != manifestVersion {
return manifest{}, fmt.Errorf("unsupported compatibility baseline version %d", value.Version)
}
return value, nil
}
func compare(want, got manifest) error {
var failures []string
if want.ModulePath != got.ModulePath {
failures = append(failures, fmt.Sprintf("module_path: want %q, got %q", want.ModulePath, got.ModulePath))
}
checks := []struct {
name string
want, got []string
}{
{"minio_imports", want.MinioImports, got.MinioImports},
{"environment", want.Environment, got.Environment},
{"metrics", want.Metrics, got.Metrics},
{"headers", want.Headers, got.Headers},
{"routes", want.Routes, got.Routes},
{"route_roots", want.RouteRoots, got.RouteRoots},
{"grid_routes", want.GridRoutes, got.GridRoutes},
{"storage_markers", want.StorageMarkers, got.StorageMarkers},
{"policy_values", want.PolicyValues, got.PolicyValues},
{"brand_allowlist", want.BrandAllowlist, got.BrandAllowlist},
}
for _, check := range checks {
if missing, added := setDiff(check.want, check.got); len(missing) > 0 || len(added) > 0 {
var b strings.Builder
fmt.Fprintf(&b, "%s compatibility set changed", check.name)
for _, value := range missing {
fmt.Fprintf(&b, "\n - %s", value)
}
for _, value := range added {
fmt.Fprintf(&b, "\n + %s", value)
}
failures = append(failures, b.String())
}
}
if len(failures) > 0 {
return errors.New(strings.Join(failures, "\n"))
}
return nil
}
func setDiff(want, got []string) (missing, added []string) {
wantSet := make(map[string]struct{}, len(want))
gotSet := make(map[string]struct{}, len(got))
for _, value := range want {
wantSet[value] = struct{}{}
}
for _, value := range got {
gotSet[value] = struct{}{}
}
for _, value := range want {
if _, ok := gotSet[value]; !ok {
missing = append(missing, value)
}
}
for _, value := range got {
if _, ok := wantSet[value]; !ok {
added = append(added, value)
}
}
return missing, added
}
func printSummary(value manifest) {
fmt.Printf("compatibility manifest: imports=%d env=%d metrics=%d headers=%d routes=%d roots=%d grid=%d storage=%d policy=%d brand=%d sha256=%s\n",
len(value.MinioImports), len(value.Environment), len(value.Metrics), len(value.Headers),
len(value.Routes), len(value.RouteRoots), len(value.GridRoutes), len(value.StorageMarkers), len(value.PolicyValues),
len(value.BrandAllowlist), manifestDigest(value))
}
func manifestDigest(value manifest) string {
data, _ := json.Marshal(value)
sum := sha256.Sum256(data)
return hex.EncodeToString(sum[:])
}
func gitOutput(args ...string) (string, error) {
out, err := exec.Command("git", args...).CombinedOutput()
if err != nil {
return "", fmt.Errorf("git %s: %s: %w", strings.Join(args, " "), strings.TrimSpace(string(out)), err)
}
return string(out), nil
}
func fatal(err error) {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
+1
View File
@@ -0,0 +1 @@
__pycache__/
+163
View File
@@ -0,0 +1,163 @@
# Copyright (c) 2026 Feng Ruohang
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
"""Pure, self-contained SVG rendering for SILO's README cards."""
from datetime import date, timedelta
from html import escape
import math
import xml.etree.ElementTree as ET
themes = {
'light': dict(bg='#ffffff', wash='#f2f7fc', edge='#d9e3ee', ink='#16222e',
muted='#62758a', blue='#1d588c', copper='#b4762e', grid='#e5edf5',
line='#2b6ca3', ring='#dce5ef', field='#f7f9fc', label='#3d4e61'),
'dark': dict(bg='#101923', wash='#152738', edge='#2b3c50', ink='#e8eef6',
muted='#93a3b8', blue='#7fb8e8', copper='#e0a35c', grid='#263749',
line='#5da2dd', ring='#3a4e63', field='#0b1119', label='#b6c2d2'),
}
def read_emblem(path):
emblem = ET.parse(path).getroot()
body = ''.join(ET.tostring(child, encoding='unicode') for child in emblem
if child.tag.rsplit('}', 1)[-1] in ('defs', 'g'))
return '\n'.join(line.rstrip() for line in body.splitlines()).strip()
def txt(x, y, value, size=14, color=None, weight=400, anchor='start', mono=False, spacing=None):
family = 'Menlo,Consolas,monospace' if mono else 'Arial,Helvetica,sans-serif'
extra = f' letter-spacing="{spacing}"' if spacing is not None else ''
return (f'<text x="{x}" y="{y}" font-family="{family}" font-size="{size}" '
f'font-weight="{weight}" fill="{color}" text-anchor="{anchor}"{extra}>'
f'{escape(str(value))}</text>')
def start(height, theme, title, description, emblem_body):
t = themes[theme]
return [f'<svg xmlns="http://www.w3.org/2000/svg" width="1000" height="{height}" '
f'viewBox="0 0 1000 {height}" role="img" aria-labelledby="title desc">',
f'<title id="title">{escape(title)}</title><desc id="desc">{escape(description)}</desc>',
'<defs><linearGradient id="surface" x1="0" y1="1" x2="1" y2="0">'
f'<stop offset="0" stop-color="{t["bg"]}"/>'
f'<stop offset="1" stop-color="{t["wash"]}"/></linearGradient>'
'<linearGradient id="accent" x1="0" y1="0" x2="1" y2="0">'
f'<stop offset="0" stop-color="{t["blue"]}"/>'
f'<stop offset="1" stop-color="{t["copper"]}"/></linearGradient>'
'<linearGradient id="area" x1="0" y1="0" x2="0" y2="1">'
f'<stop offset="0" stop-color="{t["line"]}" stop-opacity=".22"/>'
f'<stop offset="1" stop-color="{t["line"]}" stop-opacity=".015"/>'
'</linearGradient></defs>',
f'<rect x=".75" y=".75" width="998.5" height="{height-1.5}" rx="22" '
f'fill="url(#surface)" stroke="{t["edge"]}" stroke-width="1.5"/>',
f'<svg x="40" y="25" width="25" height="25" viewBox="230 213 570 570">{emblem_body}</svg>']
def heading(parts, t, eyebrow, title, subtitle, value, value_label):
parts.extend([
txt(76, 43, eyebrow, 11, t['muted'], 600, mono=True, spacing=1.7),
txt(40, 94, title, 32, t['ink'], 700),
txt(41, 123, subtitle, 14, t['muted']),
txt(958, 89, f'{value:,}', 45, t['ink'], 700, anchor='end'),
txt(957, 114, value_label, 10, t['muted'], 600, anchor='end', mono=True, spacing=1.5),
f'<path d="M40 146 H960" stroke="{t["edge"]}"/>',
])
def contributors(theme, people, snapshot, emblem_body):
height = 206 + 76 * math.ceil(len(people) / 10)
t = themes[theme]
parts = start(height, theme, f'SILO community — {len(people)} contributors',
f'The existing SILO community roll, including code, proposals and reports across related projects. '
f'Gold rings retain the existing significant-contribution designation. Snapshot {snapshot}.', emblem_body)
heading(parts, t, 'SILO / COMMUNITY', 'Contributors',
'Code, proposals & reports across SILO and related projects', len(people), 'COMMUNITY CONTRIBUTORS')
for row in range(math.ceil(len(people) / 10)):
group = people[row * 10:(row + 1) * 10]
row_width = len(group) * 91
for col, person in enumerate(group):
x = (1000 - row_width) / 2 + col * 91 + 45.5
y = 199 + row * 76
identifier = f'avatar-{row}-{col}'
featured = bool(person.get('featured'))
parts.append(f'<g><title>@{escape(person["handle"])} — {escape(person["what"])}</title>')
parts.append(f'<defs><clipPath id="{identifier}"><circle cx="{x}" cy="{y}" r="29"/></clipPath></defs>')
if featured:
parts.append(f'<circle cx="{x}" cy="{y}" r="34" fill="{t["copper"]}" opacity=".09"/>')
if person.get('avatarDataUrl'):
parts.append(f'<image x="{x-29}" y="{y-29}" width="58" height="58" '
f'clip-path="url(#{identifier})" href="{escape(person["avatarDataUrl"])}"/>')
else:
parts.append(f'<circle cx="{x}" cy="{y}" r="29" fill="{t["ring"]}"/>')
parts.append(txt(x, y + 9, person['handle'][0].upper(), 26, t['ink'], 700, 'middle'))
parts.append(f'<circle cx="{x}" cy="{y}" r="30.5" fill="none" '
f'stroke="{t["copper"] if featured else t["ring"]}" stroke-width="{2 if featured else 1.25}"/></g>')
parts.extend([
f'<path d="M40 {height-42} H960" stroke="{t["edge"]}"/>',
f'<circle cx="47" cy="{height-21}" r="4" fill="none" stroke="{t["copper"]}" stroke-width="1.5"/>',
txt(61, height-17, 'Gold rings mark significant contributions', 12, t['muted']),
txt(959, height-17, f'AS OF {snapshot}', 10, t['muted'], 500, 'end', mono=True, spacing=.6),
'</svg>',
])
return ''.join(parts)
def stars(theme, history, snapshot, emblem_body):
points = history['points']
star_count = points[-1]['stars']
t = themes[theme]
provenance = ('Initial history reconstructed · Daily totals since ' + history['bootstrap']['through'] + ' · UTC'
if history['bootstrap']['reconstructed'] else 'Observed daily star totals · UTC')
parts = start(558, theme, f'SILO star history — {star_count:,} stars',
f'GitHub repository pgsty/silo. {star_count:,} stars as of {snapshot}. ' +
provenance, emblem_body)
heading(parts, t, 'SILO / GITHUB', 'Star History', 'pgsty/silo', star_count, 'GITHUB STARS')
left, right, top, bottom = 76, 958, 177, 440
begin = date.fromisoformat(points[0]['date'])
end = date.fromisoformat(points[-1]['date'])
days = max(1, (end - begin).days)
maximum = max(500, math.ceil(max(p['stars'] for p in points) / 500) * 500)
tick_step = 10 ** max(0, int(math.log10(maximum)))
xy = lambda day, n: (left + (right-left)*(date.fromisoformat(day)-begin).days / days,
bottom-(bottom-top)*n/maximum)
for value in range(0, maximum+1, tick_step):
y = xy(points[0]['date'], value)[1]
parts.append(f'<path d="M{left} {y:.2f} H{right}" stroke="{t["grid"]}" stroke-dasharray="4 6"/>')
parts.append(txt(left-16, round(y+4, 2), f'{value / 1000:g}k' if value >= 1000 else str(value), 12, t['muted'], anchor='end'))
dates = sorted({begin + timedelta(days=round((end-begin).days*i/5)) for i in range(6)})
ticks = [(d.isoformat(), d.strftime('%b %Y') if days > 90 else d.strftime('%b %d')) for d in dates]
for day, label in ticks:
x = xy(day, 0)[0]
parts.append(f'<path d="M{x:.2f} {top} V{bottom}" stroke="{t["grid"]}" stroke-opacity=".65"/>')
anchor = 'start' if day == points[0]['date'] else 'end' if day == snapshot else 'middle'
parts.append(txt(round(x, 2), 466, label, 12, t['muted'], anchor=anchor))
coords = [xy(p['date'], p['stars']) for p in points]
line = 'M' + ' L'.join(f'{x:.2f} {y:.2f}' for x, y in coords)
area = line + f' L{coords[-1][0]:.2f} {bottom} L{left} {bottom} Z'
parts.extend([
f'<path d="{area}" fill="url(#area)"/>',
f'<path d="{line}" fill="none" stroke="url(#accent)" stroke-width="3" '
'stroke-linecap="round" stroke-linejoin="round"/>',
f'<path d="M{left} {bottom} H{right}" stroke="{t["edge"]}"/>',
])
x, y = coords[-1]
parts.extend([
f'<circle cx="{x}" cy="{y}" r="10" fill="{t["copper"]}" opacity=".12"/>',
f'<circle cx="{x}" cy="{y}" r="5" fill="{t["copper"]}" stroke="{t["bg"]}" stroke-width="2"/>',
f'<path d="M40 491 H960" stroke="{t["edge"]}"/>',
txt(40, 516, f'{begin:%b %Y} — {end:%b %Y}'.upper(), 10, t['muted'], 500, mono=True, spacing=.7),
txt(959, 516, f'SNAPSHOT {snapshot}', 10, t['muted'], 500, 'end', mono=True, spacing=.6),
txt(40, 539, provenance, 11, t['muted']),
'</svg>',
])
return ''.join(parts)
@@ -0,0 +1 @@
PyYAML==6.0.3
+178
View File
@@ -0,0 +1,178 @@
# Copyright (c) 2026 Feng Ruohang
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
"""Regression checks for historical accuracy, contributor scope and SVG safety."""
import base64
import json
from pathlib import Path
import tempfile
import unittest
from unittest.mock import patch
from urllib.error import URLError
import xml.etree.ElementTree as ET
import render
import update
NS = {'s': 'http://www.w3.org/2000/svg'}
PNG = base64.b64decode('iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+a7mgAAAAASUVORK5CYII=')
def person(handle='Alice', group='reports', featured=False):
return {'handle': handle, 'group': group, 'featured': featured,
'what': 'A reviewed contribution', 'firstContribution': '2026-09-01'}
def history():
return {'repository': 'pgsty/silo',
'bootstrap': {'through': '2026-09-15', 'reconstructed': True},
'points': [{'date': '2026-09-14', 'stars': 100}, {'date': '2026-09-15', 'stars': 105}]}
class HistoryTests(unittest.TestCase):
def test_new_day_preserves_old_counts_and_unstars(self):
before = history()
after = update.update_history(before, '2026-09-16', 103)
self.assertEqual(after['points'][:-1], before['points'])
self.assertEqual(after['points'][-1], {'date': '2026-09-16', 'stars': 103})
self.assertEqual(before, history())
def test_same_day_rerun_replaces_instead_of_appending(self):
first = update.update_history(history(), '2026-09-15', 107)
self.assertEqual(len(first['points']), 2)
self.assertEqual(first, update.update_history(first, '2026-09-15', 107))
def test_missing_days_are_not_invented(self):
result = update.update_history(history(), '2026-09-18', 106)
self.assertEqual([p['date'] for p in result['points']], ['2026-09-14', '2026-09-15', '2026-09-18'])
def test_rejects_wrong_repository_and_corrupt_history(self):
cases = []
wrong = history(); wrong['repository'] = 'someone/else'; cases.append(wrong)
duplicate = history(); duplicate['points'].append(duplicate['points'][-1]); cases.append(duplicate)
unordered = history(); unordered['points'].reverse(); cases.append(unordered)
negative = history(); negative['points'][0]['stars'] = -1; cases.append(negative)
future = history(); future['points'][-1]['date'] = '2026-09-20'; cases.append(future)
for case in cases:
with self.subTest(case=case), self.assertRaises(ValueError):
update.update_history(case, '2026-09-16', 100)
def test_first_run_has_no_fabricated_history(self):
result = update.update_history(None, '2026-09-16', 10)
self.assertFalse(result['bootstrap']['reconstructed'])
self.assertEqual(result['points'], [{'date': '2026-09-16', 'stars': 10}])
class ContributorTests(unittest.TestCase):
def test_retries_truncated_json_before_using_it(self):
with patch('update.request', side_effect=[b'{"partial":', b'{"ok":true}']), patch('update.time.sleep'):
self.assertEqual(update.GitHub('').get('repos/pgsty/silo'), {'ok': True})
def test_paginates_past_one_full_page(self):
class API(update.GitHub):
def __init__(self): self.calls = []
def get(self, path):
self.calls.append(path)
return list(range(100)) if 'page=1&' in path else [100]
api = API()
self.assertEqual(len(list(api.issues('pgsty/silo'))), 101)
self.assertIn('state=all', api.calls[0])
self.assertIn('page=2&', api.calls[1])
def test_bots_deduplication_unmerged_work_and_reviewed_credit(self):
def issue(login, kind='issue', user_type='User'):
item = {'user': {'login': login, 'type': user_type, 'avatar_url': ''}, 'created_at': '2026-09-02T00:00:00Z'}
if kind != 'issue': item['pull_request'] = {'merged_at': None if kind == 'open' else '2026-09-03T00:00:00Z'}
return item
class API:
def issues(self, _repo):
return [issue('alice'), issue('Bob', 'open'), issue('Bob', 'merged'),
issue('Carol', 'open'), issue('Copilot'), issue('robot', user_type='Bot')]
curated = {'repositories': ['pgsty/silo', 'pgsty/mc'], 'bots': ['Copilot'],
'people': [person('Alice', featured=True), person('Reporter'), person('Copilot')]}
result = update.collect_people(API(), curated)
self.assertEqual({p['handle'] for p in result}, {'Alice', 'Bob', 'Carol', 'Reporter'})
self.assertEqual(result[0]['handle'], 'Bob')
self.assertEqual(result[1]['handle'], 'Carol')
self.assertTrue(next(p for p in result if p['handle'] == 'Alice')['featured'])
self.assertEqual(next(p for p in result if p['handle'] == 'Bob')['group'], 'code')
self.assertFalse(next(p for p in result if p['handle'] == 'Carol')['featured'])
def test_newer_reviewed_preview_survives_until_site_catches_up(self):
remote = {'updated': '2026-09-16T03:00:00+00:00'}
cached = {'updated': '2026-09-16T04:00:00+00:00'}
self.assertIs(update.select_curated(remote, cached), cached)
newer = {'updated': '2026-09-17T03:00:00+00:00'}
self.assertIs(update.select_curated(newer, cached), newer)
def test_avatar_failure_reuses_raster_cache(self):
previous = {**person(), 'avatarDataUrl': update.raster_data_url(PNG)}
with patch('update.request', side_effect=URLError('unavailable')):
result = update.add_avatars(None, [{**person(), 'avatarUrl': 'https://avatars.githubusercontent.com/u/1'}], [previous])
self.assertEqual(result[0]['avatarDataUrl'], previous['avatarDataUrl'])
with self.assertRaises(ValueError): update.raster_data_url(b'<svg onload="bad()"/>')
with self.assertRaises(ValueError): update.cached_avatar({'avatarDataUrl': 'data:image/svg+xml;base64,PHN2Zy8+'})
def test_fetch_failure_leaves_published_assets_untouched(self):
class API:
def get(self, path):
if path == 'repos/pgsty/silo': return {'full_name': 'pgsty/silo', 'stargazers_count': 106}
raise URLError('roster unavailable')
with tempfile.TemporaryDirectory() as directory:
out = Path(directory)
original = json.dumps(history())
(out / 'history.json').write_text(original)
(out / 'contributors-light.svg').write_text('previous image')
with self.assertRaises(URLError): update.refresh(out, API(), Path('.'))
self.assertEqual((out / 'history.json').read_text(), original)
self.assertEqual((out / 'contributors-light.svg').read_text(), 'previous image')
class RenderTests(unittest.TestCase):
def test_real_emblem_generates_clean_xml(self):
emblem = render.read_emblem(Path(__file__).resolve().parents[2] / '.github/silo.svg')
svg = render.contributors('light', [person()], '2026-09-16', emblem)
ET.fromstring(svg)
self.assertTrue(all(line == line.rstrip() for line in svg.splitlines()))
def test_all_avatars_fit_when_the_roster_grows(self):
people = [{**person(f'person-{i}'), 'avatarDataUrl': update.raster_data_url(PNG)} for i in range(151)]
for theme in ('light', 'dark'):
root = ET.fromstring(render.contributors(theme, people, '2026-09-16', ''))
images = root.findall('.//s:image', NS)
self.assertEqual(len(images), 151)
footer = float(root.attrib['height']) - 42
self.assertTrue(all(float(i.attrib['y']) + float(i.attrib['height']) < footer for i in images))
self.assertTrue(all(i.attrib['href'].startswith('data:image/png;base64,') for i in images))
def test_untrusted_text_is_escaped(self):
data = [{**person(), 'what': '<script>alert("x")</script> & contributions'}]
svg = render.contributors('light', data, '2026-09-16', '')
root = ET.fromstring(svg)
self.assertEqual(root.findall('.//s:script', NS), [])
self.assertIn('&lt;script&gt;', svg)
def test_single_point_and_decreasing_star_history_render(self):
for data in (update.update_history(None, '2026-09-16', 0), update.update_history(history(), '2026-09-16', 90)):
for theme in ('light', 'dark'):
svg = render.stars(theme, data, '2026-09-16', '')
ET.fromstring(svg)
self.assertNotIn('nan', svg.lower())
self.assertNotIn('inf', svg.lower())
if __name__ == '__main__':
unittest.main()
+295
View File
@@ -0,0 +1,295 @@
#!/usr/bin/env python3
# Copyright (c) 2026 Feng Ruohang
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
"""Refresh the generated-asset checkout; publishing is handled by the workflow."""
import argparse
import base64
from concurrent.futures import ThreadPoolExecutor
from datetime import date, datetime, timezone
import json
from http.client import IncompleteRead
import os
from pathlib import Path
import re
import sys
import time
from urllib.error import HTTPError, URLError
from urllib.parse import urlparse
from urllib.request import Request, urlopen
import xml.etree.ElementTree as ET
import yaml
import render
REPOSITORY = 'pgsty/silo'
SOURCE = 'repos/pgsty/silo.pgsty.com/contents/data/home/contributors.yaml?ref=main'
GROUPS = ('code', 'proposed', 'reports')
HANDLE = re.compile(r'[A-Za-z0-9][A-Za-z0-9-]{0,38}\Z')
def request(url, token='', limit=8 * 1024 * 1024):
headers = {'User-Agent': 'silo-repository-cards', 'Accept': 'application/vnd.github+json'}
if urlparse(url).netloc == 'api.github.com':
headers['X-GitHub-Api-Version'] = '2022-11-28'
if token:
headers['Authorization'] = f'Bearer {token}'
for attempt in range(3):
try:
with urlopen(Request(url, headers=headers), timeout=25) as response:
data = response.read(limit + 1)
if len(data) > limit:
raise ValueError('Response exceeds the size limit')
expected = response.headers.get('Content-Length')
if expected is not None and len(data) != int(expected):
raise URLError('Incomplete response body')
return data
except HTTPError as exc:
if exc.code < 500 or attempt == 2:
raise
except (URLError, TimeoutError, IncompleteRead):
if attempt == 2:
raise
time.sleep(attempt + 1)
class GitHub:
def __init__(self, token):
self.token = token
def get(self, path):
for attempt in range(3):
try:
return json.loads(request('https://api.github.com/' + path, self.token))
except (json.JSONDecodeError, UnicodeDecodeError) as exc:
if attempt == 2:
raise ValueError(f'Incomplete or invalid GitHub JSON: {path}') from exc
time.sleep(attempt + 1)
def issues(self, repository):
page = 1
while True:
batch = self.get(f'repos/{repository}/issues?state=all&per_page=100&page={page}&sort=created&direction=asc')
if not isinstance(batch, list):
raise ValueError(f'Invalid issues response for {repository}')
yield from batch
if len(batch) < 100:
return
page += 1
def curated_snapshot(data, revision):
updated = str(data['updated'])
datetime.fromisoformat(updated)
repositories = [item['repo'] for item in data['repositories']]
if not repositories or any(not re.fullmatch(r'pgsty/[A-Za-z0-9_.-]+', repo) for repo in repositories):
raise ValueError('Invalid contributor repository scope')
people = []
for group in GROUPS:
for entry in data[group]:
if not HANDLE.fullmatch(entry['handle']):
raise ValueError('Invalid GitHub contributor handle')
people.append({
'handle': entry['handle'], 'group': group,
'featured': bool(entry.get('featured')), 'what': entry['what'],
'firstContribution': str(entry.get('firstContribution', '9999-12-31')),
})
if not people or len({p['handle'].lower() for p in people}) != len(people):
raise ValueError('Empty or duplicate contributor roster')
return {'updated': updated, 'revision': revision, 'repositories': repositories,
'bots': data.get('bots', ['Copilot', 'dependabot[bot]']), 'people': people}
def select_curated(remote, cached):
# The initial, approved preview can contain reviewed credit not published by
# the companion site yet. Keep that newer snapshot until the site catches up.
if cached and datetime.fromisoformat(cached['updated']) > datetime.fromisoformat(remote['updated']):
return cached
return remote
def collect_people(api, curated):
bots = {name.lower() for name in curated['bots']}
people = {p['handle'].lower(): dict(p) for p in curated['people']
if p['handle'].lower() not in bots and not p['handle'].lower().endswith('[bot]')}
order = {p['handle'].lower(): index for index, p in enumerate(curated['people'])}
for repository in curated['repositories']:
print(f'Reading issue and PR authors: {repository}', flush=True)
for issue in api.issues(repository):
user = issue.get('user') or {}
handle = user.get('login', '')
key = handle.lower()
if user.get('type') != 'User' or key in bots or key.endswith('[bot]'):
continue
if not HANDLE.fullmatch(handle):
raise ValueError('Invalid issue author')
pr = issue.get('pull_request')
group = 'code' if pr and pr.get('merged_at') else 'proposed' if pr else 'reports'
first = issue['created_at'][:10]
date.fromisoformat(first)
person = people.setdefault(key, {
'handle': handle, 'group': group, 'featured': False,
'what': 'Contributed an issue or pull request to SILO and related projects',
'firstContribution': first,
})
person['avatarUrl'] = user.get('avatar_url', '')
person['firstContribution'] = min(person['firstContribution'], first)
if GROUPS.index(group) < GROUPS.index(person['group']):
person['group'] = group
if not people:
raise ValueError('No human contributors were collected')
return sorted(people.values(), key=lambda p: (
GROUPS.index(p['group']), not p['featured'],
order.get(p['handle'].lower(), len(order)), p['firstContribution'], p['handle'].lower()))
def raster_data_url(data):
if data.startswith(b'\x89PNG\r\n\x1a\n'):
mime = 'image/png'
elif data.startswith(b'\xff\xd8\xff'):
mime = 'image/jpeg'
elif data.startswith((b'GIF87a', b'GIF89a')):
mime = 'image/gif'
elif data[:4] == b'RIFF' and data[8:12] == b'WEBP':
mime = 'image/webp'
else:
raise ValueError('Avatar is not a raster image')
return f'data:{mime};base64,' + base64.b64encode(data).decode('ascii')
def cached_avatar(person):
value = person.get('avatarDataUrl', '')
if not value:
return ''
prefix, encoded = value.split(',', 1)
if prefix not in ('data:image/png;base64', 'data:image/jpeg;base64', 'data:image/gif;base64', 'data:image/webp;base64'):
raise ValueError('Invalid cached avatar format')
raw = base64.b64decode(encoded, validate=True)
if len(raw) > 512 * 1024 or raster_data_url(raw) != value:
raise ValueError('Invalid cached avatar')
return value
def add_avatars(api, people, previous):
cached = {p['handle'].lower(): cached_avatar(p) for p in previous}
def update(person):
person = dict(person)
try:
url = person.pop('avatarUrl', '') or api.get('users/' + person['handle'])['avatar_url']
parsed = urlparse(url)
if parsed.scheme != 'https' or parsed.netloc != 'avatars.githubusercontent.com':
raise ValueError('Unexpected avatar host')
data = request(url + ('&' if '?' in url else '?') + 's=96', limit=512 * 1024)
person['avatarDataUrl'] = raster_data_url(data)
except (HTTPError, URLError, TimeoutError, IncompleteRead, ValueError, KeyError) as exc:
person.pop('avatarUrl', None)
person['avatarDataUrl'] = cached.get(person['handle'].lower(), '')
print(f'Avatar fallback for @{person["handle"]}: {type(exc).__name__}', file=sys.stderr)
return person
with ThreadPoolExecutor(max_workers=6) as pool:
return list(pool.map(update, people))
def update_history(history, day, stars):
date.fromisoformat(day)
if type(stars) is not int or stars < 0:
raise ValueError('Invalid repository star count')
if history is None:
history = {'repository': REPOSITORY, 'bootstrap': {'through': day, 'reconstructed': False}, 'points': []}
if history['repository'] != REPOSITORY:
raise ValueError('Star history belongs to a different repository')
date.fromisoformat(history['bootstrap']['through'])
dates = []
for point in history['points']:
date.fromisoformat(point['date'])
if type(point['stars']) is not int or point['stars'] < 0:
raise ValueError('Invalid historical star count')
dates.append(point['date'])
if dates != sorted(set(dates)) or any(d > day for d in dates):
raise ValueError('History contains duplicate, unordered, or future dates')
# Replace today's observation, preserve previous days, and allow unstars.
points = [dict(p) for p in history['points'] if p['date'] != day]
points.append({'date': day, 'stars': stars})
return {**history, 'points': points}
def read_json(path, default=None):
return json.loads(path.read_text()) if path.exists() else default
def refresh(output, api, source_root):
day = datetime.now(timezone.utc).date().isoformat()
metadata = api.get('repos/' + REPOSITORY)
if metadata['full_name'].lower() != REPOSITORY:
raise ValueError('Unexpected repository metadata')
history = update_history(read_json(output / 'history.json'), day, metadata['stargazers_count'])
source = api.get(SOURCE)
reviewed = yaml.safe_load(base64.b64decode(source['content'], validate=False))
curated = select_curated(curated_snapshot(reviewed, source['sha']), read_json(output / 'curated.json'))
people = collect_people(api, curated)
previous = read_json(output / 'contributors.json', {}).get('people', [])
people = add_avatars(api, people, previous)
emblem = render.read_emblem(source_root / '.github/silo.svg')
payloads = {}
for theme in ('light', 'dark'):
payloads[f'contributors-{theme}.svg'] = render.contributors(theme, people, day, emblem) + '\n'
payloads[f'star-history-{theme}.svg'] = render.stars(theme, history, day, emblem) + '\n'
for svg in payloads.values():
ET.fromstring(svg)
for name, data in {
'history.json': history,
'curated.json': curated,
'contributors.json': {'repository': REPOSITORY, 'updated': day, 'people': people},
}.items():
payloads[name] = json.dumps(data, indent=2, ensure_ascii=False) + '\n'
payloads['README.md'] = f'''# SILO repository cards
Generated by [Repository Cards](https://github.com/pgsty/silo/actions/workflows/repository-cards.yml)
at 00:00 UTC daily (08:00 Asia/Shanghai). GitHub may queue scheduled runs.
Snapshot: {day}. {metadata['stargazers_count']:,} stars; {len(people)} community contributors.
- `contributors-light.svg` / `contributors-dark.svg`: human issue and PR authors across the SILO project scope, plus reviewed acknowledgements. Bots are excluded. Gold rings follow the reviewed companion-site roster; new authors are collected automatically.
- `star-history-light.svg` / `star-history-dark.svg`: initial history reconstructed from the then-current stargazers; later points are daily observed totals, including decreases. Missing days are not fabricated.
- `curated.json`: a cache of reviewed contributor credit from `pgsty/silo.pgsty.com/data/home/contributors.yaml`. The approved initial preview may be newer than the published site; a newer reviewed snapshot is retained until the site catches up.
- `contributors.json`: generated contributor data and embedded raster avatars. Failed avatar refreshes use the previous image, or an initial when no image is available.
- `history.json`: persistent daily totals. Keep this file when regenerating images.
The SVGs are self-contained. Source and instructions live on the default branch;
this branch contains generated assets only. Do not merge it into `main`.
'''
# Collect and validate everything before touching the publication checkout.
output.mkdir(parents=True, exist_ok=True)
for filename, text in payloads.items():
(output / filename).write_text(text)
print(f'{day}: {len(people)} contributors; {metadata["stargazers_count"]:,} stars; {len(history["points"])} history points')
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--output', type=Path, required=True)
args = parser.parse_args()
configured = os.environ.get('GITHUB_REPOSITORY', REPOSITORY)
if configured.lower() != REPOSITORY:
raise SystemExit('This workflow is scoped to pgsty/silo')
refresh(args.output, GitHub(os.environ.get('GH_TOKEN', '')), Path(__file__).resolve().parents[2])
if __name__ == '__main__':
main()
+14 -23
View File
@@ -6,38 +6,29 @@ set -x
set -e
WORK_DIR="$PWD/.verify-$RANDOM"
MINIO_CONFIG_DIR="$WORK_DIR/.minio"
MINIO=("$PWD/minio" --config-dir "$MINIO_CONFIG_DIR" server)
SILO_CONFIG_DIR="$WORK_DIR/.silo"
SILO=("$PWD/silo" --config-dir "$SILO_CONFIG_DIR" server)
if [ ! -x "$PWD/minio" ]; then
echo "minio executable binary not found in current directory"
if [ ! -x "$PWD/silo" ]; then
echo "Silo executable binary not found in current directory"
exit 1
fi
function start_minio_5drive() {
function start_silo_5drive() {
start_port=$1
export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=minio123
export MC_HOST_minio="http://minio:minio123@127.0.0.1:${start_port}/"
export MINIO_ROOT_USER=silo
export MINIO_ROOT_PASSWORD=silo1234
export MC_HOST_silo="http://silo:silo1234@127.0.0.1:${start_port}/"
unset MINIO_KMS_AUTO_ENCRYPTION # do not auto-encrypt objects
export MINIO_CI_CD=1
MC_BUILD_DIR="mc-$RANDOM"
if ! git clone --quiet https://github.com/minio/mc "$MC_BUILD_DIR"; then
echo "failed to download https://github.com/minio/mc"
purge "${MC_BUILD_DIR}"
exit 1
fi
(cd "${MC_BUILD_DIR}" && go build -o "$WORK_DIR/mc")
# remove mc source.
purge "${MC_BUILD_DIR}"
mkdir -p "${WORK_DIR}"
"$(git rev-parse --show-toplevel)/buildscripts/install-mcli.sh" "${WORK_DIR}/mc"
"${WORK_DIR}/mc" cp --quiet -r "buildscripts/cicd-corpus/" "${WORK_DIR}/cicd-corpus/"
"${MINIO[@]}" --address ":$start_port" "${WORK_DIR}/cicd-corpus/disk{1...5}" >"${WORK_DIR}/server1.log" 2>&1 &
"${SILO[@]}" --address ":$start_port" "${WORK_DIR}/cicd-corpus/disk{1...5}" >"${WORK_DIR}/server1.log" 2>&1 &
pid=$!
disown $pid
sleep 5
@@ -50,16 +41,16 @@ function start_minio_5drive() {
exit 1
fi
"${WORK_DIR}/mc" stat minio/bucket/testobj
"${WORK_DIR}/mc" stat silo/bucket/testobj
pkill minio
pkill silo
sleep 3
}
function main() {
start_port=$(shuf -i 10000-65000 -n 1)
start_minio_5drive ${start_port}
start_silo_5drive ${start_port}
}
function purge() {
+30 -36
View File
@@ -5,48 +5,42 @@ set -o pipefail
set -x
WORK_DIR="$PWD/.verify-$RANDOM"
MINIO_CONFIG_DIR="$WORK_DIR/.minio"
MINIO_OLD=("$PWD/minio.RELEASE.2020-10-28T08-16-50Z" --config-dir "$MINIO_CONFIG_DIR" server)
MINIO=("$PWD/minio" --config-dir "$MINIO_CONFIG_DIR" server)
SILO_CONFIG_DIR="$WORK_DIR/.silo"
MINIO_OLD=("$PWD/minio.RELEASE.2020-10-28T08-16-50Z" --config-dir "$SILO_CONFIG_DIR" server)
SILO=("$PWD/silo" --config-dir "$SILO_CONFIG_DIR" server)
if [ ! -x "$PWD/minio" ]; then
echo "minio executable binary not found in current directory"
if [ ! -x "$PWD/silo" ]; then
echo "Silo executable binary not found in current directory"
exit 1
fi
function download_old_release() {
if [ ! -f minio.RELEASE.2020-10-28T08-16-50Z ]; then
curl --silent -O https://dl.minio.io/server/minio/release/linux-amd64/archive/minio.RELEASE.2020-10-28T08-16-50Z
chmod a+x minio.RELEASE.2020-10-28T08-16-50Z
if [ ! -x minio.RELEASE.2020-10-28T08-16-50Z ]; then
: "${SILO_LEGACY_FIXTURE_2020:?set SILO_LEGACY_FIXTURE_2020 to the audited legacy binary}"
: "${SILO_LEGACY_SHA256_2020:?set SILO_LEGACY_SHA256_2020 to its audited SHA-256}"
"$(git rev-parse --show-toplevel)/buildscripts/install-verified-fixture.sh" \
"${SILO_LEGACY_FIXTURE_2020}" "${SILO_LEGACY_SHA256_2020}" \
"minio.RELEASE.2020-10-28T08-16-50Z"
fi
}
function verify_rewrite() {
start_port=$1
export MINIO_ACCESS_KEY=minio
export MINIO_SECRET_KEY=minio123
export MC_HOST_minio="http://minio:minio123@127.0.0.1:${start_port}/"
export MINIO_ACCESS_KEY=silo
export MINIO_SECRET_KEY=silo1234
export MC_HOST_silo="http://silo:silo1234@127.0.0.1:${start_port}/"
unset MINIO_KMS_AUTO_ENCRYPTION # do not auto-encrypt objects
export MINIO_CI_CD=1
MC_BUILD_DIR="mc-$RANDOM"
if ! git clone --quiet https://github.com/minio/mc "$MC_BUILD_DIR"; then
echo "failed to download https://github.com/minio/mc"
purge "${MC_BUILD_DIR}"
exit 1
fi
(cd "${MC_BUILD_DIR}" && go build -o "$WORK_DIR/mc")
# remove mc source.
purge "${MC_BUILD_DIR}"
mkdir -p "${WORK_DIR}"
"$(git rev-parse --show-toplevel)/buildscripts/install-mcli.sh" "${WORK_DIR}/mc"
"${MINIO_OLD[@]}" --address ":$start_port" "${WORK_DIR}/xl{1...16}" >"${WORK_DIR}/server1.log" 2>&1 &
pid=$!
disown $pid
"${WORK_DIR}/mc" ready minio/
"${WORK_DIR}/mc" ready silo/
if ! ps -p ${pid} 1>&2 >/dev/null; then
echo "server1 log:"
@@ -56,30 +50,30 @@ function verify_rewrite() {
exit 1
fi
"${WORK_DIR}/mc" mb minio/healing-rewrite-bucket --quiet --with-lock
"${WORK_DIR}/mc" mb silo/healing-rewrite-bucket --quiet --with-lock
"${WORK_DIR}/mc" cp \
buildscripts/verify-build.sh \
minio/healing-rewrite-bucket/ \
silo/healing-rewrite-bucket/ \
--disable-multipart --quiet
"${WORK_DIR}/mc" cp \
buildscripts/verify-build.sh \
minio/healing-rewrite-bucket/ \
silo/healing-rewrite-bucket/ \
--disable-multipart --quiet
"${WORK_DIR}/mc" cp \
buildscripts/verify-build.sh \
minio/healing-rewrite-bucket/ \
silo/healing-rewrite-bucket/ \
--disable-multipart --quiet
kill ${pid}
sleep 3
"${MINIO[@]}" --address ":$start_port" "${WORK_DIR}/xl{1...16}" >"${WORK_DIR}/server1.log" 2>&1 &
"${SILO[@]}" --address ":$start_port" "${WORK_DIR}/xl{1...16}" >"${WORK_DIR}/server1.log" 2>&1 &
pid=$!
disown $pid
"${WORK_DIR}/mc" ready minio/
"${WORK_DIR}/mc" ready silo/
if ! ps -p ${pid} 1>&2 >/dev/null; then
echo "server1 log:"
@@ -92,8 +86,8 @@ function verify_rewrite() {
if ! ./s3-check-md5 \
-debug \
-versions \
-access-key minio \
-secret-key minio123 \
-access-key silo \
-secret-key silo1234 \
-endpoint "http://127.0.0.1:${start_port}/" 2>&1 | grep INTACT; then
echo "server1 log:"
cat "${WORK_DIR}/server1.log"
@@ -101,7 +95,7 @@ function verify_rewrite() {
mkdir -p inspects
(
cd inspects
"${WORK_DIR}/mc" admin inspect minio/healing-rewrite-bucket/verify-build.sh/**
"${WORK_DIR}/mc" admin inspect silo/healing-rewrite-bucket/verify-build.sh/**
)
"${WORK_DIR}/mc" mb play/inspects
@@ -111,14 +105,14 @@ function verify_rewrite() {
exit 1
fi
go run ./buildscripts/heal-manual.go "127.0.0.1:${start_port}" "minio" "minio123"
go run ./buildscripts/heal-manual.go "127.0.0.1:${start_port}" "silo" "silo1234"
sleep 1
if ! ./s3-check-md5 \
-debug \
-versions \
-access-key minio \
-secret-key minio123 \
-access-key silo \
-secret-key silo1234 \
-endpoint http://127.0.0.1:${start_port}/ 2>&1 | grep INTACT; then
echo "server1 log:"
cat "${WORK_DIR}/server1.log"
@@ -126,7 +120,7 @@ function verify_rewrite() {
mkdir -p inspects
(
cd inspects
"${WORK_DIR}/mc" admin inspect minio/healing-rewrite-bucket/verify-build.sh/**
"${WORK_DIR}/mc" admin inspect silo/healing-rewrite-bucket/verify-build.sh/**
)
"${WORK_DIR}/mc" mb play/inspects
+15 -10
View File
@@ -7,16 +7,21 @@ set -euo pipefail
# other way round would only surface here, on the maintainer's machine, after
# the build has already run and uploaded.
expected_fingerprint="9592A7BC7A682E7333376E09E7935D8DB9BD8B20"
expected_release="1PGSTY"
expected_vendor="PGSTY"
expected_packager="Ruohang Feng (@Vonng) <rh@vonng.com>"
expected_url="https://silo.pgsty.com"
expected_summary="S3-Interface Libre Object Storage, Community-maintained MinIO server fork."
expected_description="S3-Interface Libre Object Storage, Community-maintained MinIO server fork."
expected_summary="S3-Interface Libre Object Storage, a community-maintained S3-compatible server."
expected_description="S3-Interface Libre Object Storage, a community-maintained S3-compatible server."
expected_license="AGPL-3.0-or-later"
expected_group="Applications/File"
expected_payload="/usr/lib/systemd/system/minio.service
/usr/local/bin/minio"
repository="${GH_REPO:-pgsty/minio}"
expected_payload="/etc/default/silo
/usr/bin/silo
/usr/lib/systemd/system/silo.service
/usr/lib/sysusers.d/silo.conf
/usr/share/doc/silo/LICENSE
/usr/share/doc/silo/NOTICE"
repository="${GH_REPO:-pgsty/silo}"
container="${DNFUPDATE_CONTAINER:-dnfupdate}"
upload=false
release_tag=""
@@ -122,8 +127,8 @@ mkdir -p "${unsigned_dir}" "${signed_dir}"
chmod 700 "${work_dir}" "${unsigned_dir}" "${signed_dir}"
rpm_files=(
"minio-${package_version}-1.x86_64.rpm"
"minio-${package_version}-1.aarch64.rpm"
"silo-${package_version}-${expected_release}.x86_64.rpm"
"silo-${package_version}-${expected_release}.aarch64.rpm"
)
download_patterns=()
@@ -159,7 +164,7 @@ for rpm_file in "${rpm_files[@]}"; do
done
safe_tag="$(printf '%s' "${release_tag}" | tr -c 'A-Za-z0-9._-' '_')"
container_dir="/tmp/minio-sign-${safe_tag}-$$"
container_dir="/tmp/silo-sign-${safe_tag}-$$"
docker exec "${container}" mkdir -p "${container_dir}"
cleanup_container() {
@@ -203,9 +208,9 @@ for rpm_file in "${rpm_files[@]}"; do
docker cp "${unsigned_dir}/${rpm_file}" "${container}:${container_dir}/${rpm_file}" >/dev/null
container_rpm="${container_dir}/${rpm_file}"
assert_rpm_tag "${container_rpm}" NAME minio
assert_rpm_tag "${container_rpm}" NAME silo
assert_rpm_tag "${container_rpm}" VERSION "${package_version}"
assert_rpm_tag "${container_rpm}" RELEASE 1
assert_rpm_tag "${container_rpm}" RELEASE "${expected_release}"
assert_rpm_tag "${container_rpm}" ARCH "${expected_arch}"
assert_rpm_tag "${container_rpm}" VENDOR "${expected_vendor}"
assert_rpm_tag "${container_rpm}" PACKAGER "${expected_packager}"
+21 -32
View File
@@ -5,16 +5,16 @@ if [ -n "$TEST_DEBUG" ]; then
fi
WORK_DIR="$PWD/.verify-$RANDOM"
MINIO_CONFIG_DIR="$WORK_DIR/.minio"
MINIO=("$PWD/minio" --config-dir "$MINIO_CONFIG_DIR" server)
SILO_CONFIG_DIR="$WORK_DIR/.silo"
SILO=("$PWD/silo" --config-dir "$SILO_CONFIG_DIR" server)
if [ ! -x "$PWD/minio" ]; then
echo "minio executable binary not found in current directory"
if [ ! -x "$PWD/silo" ]; then
echo "Silo executable binary not found in current directory"
exit 1
fi
if [ ! -x "$PWD/minio" ]; then
echo "minio executable binary not found in current directory"
if [ ! -x "$PWD/silo" ]; then
echo "Silo executable binary not found in current directory"
exit 1
fi
@@ -30,9 +30,9 @@ catch() {
echo "error on line $1"
fi
echo "Cleaning up instances of MinIO"
pkill minio || true
pkill -9 minio || true
echo "Cleaning up instances of Silo"
pkill silo || true
pkill -9 silo || true
purge "$WORK_DIR"
if [ $# -ne 0 ]; then
exit $#
@@ -70,31 +70,20 @@ function send_put_object_request() {
return 0
}
function test_minio_with_timeout() {
function test_silo_with_timeout() {
start_port=$1
export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=minio123
export MC_HOST_minio="http://minio:minio123@127.0.0.1:${start_port}/"
export MINIO_ROOT_USER=silo
export MINIO_ROOT_PASSWORD=silo1234
export MC_HOST_silo="http://silo:silo1234@127.0.0.1:${start_port}/"
export MINIO_CI_CD=1
mkdir ${WORK_DIR}
C_PWD=${PWD}
if [ ! -x "$PWD/mc" ]; then
MC_BUILD_DIR="mc-$RANDOM"
if ! git clone --quiet https://github.com/minio/mc "$MC_BUILD_DIR"; then
echo "failed to download https://github.com/minio/mc"
purge "${MC_BUILD_DIR}"
exit 1
fi
(cd "${MC_BUILD_DIR}" && go build -o "$C_PWD/mc")
# remove mc source.
purge "${MC_BUILD_DIR}"
"$(git rev-parse --show-toplevel)/buildscripts/install-mcli.sh" "$PWD/mc"
fi
"${MINIO[@]}" --address ":$start_port" --read-header-timeout ${srv_hdr_timeout}s --idle-timeout ${srv_idle_timeout}s "${WORK_DIR}/disk/" >"${WORK_DIR}/server1.log" 2>&1 &
"${SILO[@]}" --address ":$start_port" --read-header-timeout ${srv_hdr_timeout}s --idle-timeout ${srv_idle_timeout}s "${WORK_DIR}/disk/" >"${WORK_DIR}/server1.log" 2>&1 &
pid=$!
disown $pid
sleep 1
@@ -109,20 +98,20 @@ function test_minio_with_timeout() {
set -e
"${PWD}/mc" mb minio/testbucket
"${PWD}/mc" anonymous set public minio/testbucket
"${PWD}/mc" mb silo/testbucket
"${PWD}/mc" anonymous set public silo/testbucket
# slow header writing
send_put_object_request 20 0 && exit -1
"${PWD}/mc" stat minio/testbucket/testobject && exit -1
"${PWD}/mc" stat silo/testbucket/testobject && exit -1
# quick header write and slow bodywrite
send_put_object_request 0 40 && exit -1
"${PWD}/mc" stat minio/testbucket/testobject && exit -1
"${PWD}/mc" stat silo/testbucket/testobject && exit -1
# quick header and body write
send_put_object_request 1 1 || exit -1
"${PWD}/mc" stat minio/testbucket/testobject || exit -1
"${PWD}/mc" stat silo/testbucket/testobject || exit -1
}
function main() {
@@ -131,7 +120,7 @@ function main() {
export srv_idle_timeout=5
export -f gen_put_request
test_minio_with_timeout ${start_port}
test_silo_with_timeout ${start_port}
}
main "$@"
-74
View File
@@ -1,74 +0,0 @@
# Settings and configurations that are common for all containers
x-minio-common: &minio-common
image: minio/minio:${MINIO_VERSION}
command: server http://minio{1...4}/data{1...3}
env_file:
- ./minio.env
expose:
- "9000"
- "9001"
# starts 4 docker containers running minio server instances.
# using nginx reverse proxy, load balancing, you can access
# it through port 9000.
services:
minio1:
<<: *minio-common
hostname: minio1
volumes:
- data1-1:/data1
- data1-2:/data2
- data1-3:/data3
minio2:
<<: *minio-common
hostname: minio2
volumes:
- data2-1:/data1
- data2-2:/data2
- data2-3:/data3
minio3:
<<: *minio-common
hostname: minio3
volumes:
- data3-1:/data1
- data3-2:/data2
- data3-3:/data3
minio4:
<<: *minio-common
hostname: minio4
volumes:
- data4-1:/data1
- data4-2:/data2
- data4-3:/data3
nginx:
image: nginx:1.19.2-alpine
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf:ro
ports:
- "9000:9000"
- "9001:9001"
depends_on:
- minio1
- minio2
- minio3
- minio4
## By default this config uses default local driver,
## For custom volumes replace with volume driver configuration.
volumes:
data1-1:
data1-2:
data1-3:
data2-1:
data2-2:
data2-3:
data3-1:
data3-2:
data3-3:
data4-1:
data4-2:
data4-3:
-3
View File
@@ -1,3 +0,0 @@
MINIO_ACCESS_KEY=minioadmin
MINIO_SECRET_KEY=minioadmin
MINIO_BROWSER=off
-68
View File
@@ -1,68 +0,0 @@
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
# include /etc/nginx/conf.d/*.conf;
upstream minio {
server minio1:9000;
server minio2:9000;
server minio3:9000;
server minio4:9000;
}
# main minio
server {
listen 9000;
listen [::]:9000;
server_name localhost;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 300;
# Default is HTTP/1, keepalive is only enabled in HTTP/1.1
proxy_http_version 1.1;
proxy_set_header Connection "";
chunked_transfer_encoding off;
proxy_pass http://minio;
}
}
}
+1 -1
View File
@@ -41,7 +41,7 @@ while IFS= read -r binary; do
grep -F 'vcs.' <<< "${info}" >&2 || true
exit 1
fi
done < <(find "${dist_dir}" -maxdepth 2 -type f \( -name 'minio' -o -name 'minio.exe' \) | sort)
done < <(find "${dist_dir}" -maxdepth 2 -type f \( -name 'silo' -o -name 'silo.exe' \) | sort)
if [ "${count}" -ne "${expected_count}" ]; then
echo "Expected ${expected_count} release binaries, found ${count}" >&2
+59 -70
View File
@@ -5,8 +5,8 @@ set -e
set -E
set -o pipefail
if [ ! -x "$PWD/minio" ]; then
echo "minio executable binary not found in current directory"
if [ ! -x "$PWD/silo" ]; then
echo "Silo executable binary not found in current directory"
exit 1
fi
@@ -15,196 +15,196 @@ WORK_DIR="$PWD/.verify-$RANDOM"
export MINT_MODE=core
export MINT_DATA_DIR="$WORK_DIR/data"
export SERVER_ENDPOINT="127.0.0.1:9000"
export MC_HOST_verify="http://minio:minio123@${SERVER_ENDPOINT}/"
export MC_HOST_verify_ipv6="http://minio:minio123@[::1]:9000/"
export ACCESS_KEY="minio"
export SECRET_KEY="minio123"
export MC_HOST_verify="http://silo:silo1234@${SERVER_ENDPOINT}/"
export MC_HOST_verify_ipv6="http://silo:silo1234@[::1]:9000/"
export ACCESS_KEY="silo"
export SECRET_KEY="silo1234"
export ENABLE_HTTPS=0
export GO111MODULE=on
export GOGC=25
export ENABLE_ADMIN=1
export MINIO_CI_CD=1
MINIO_CONFIG_DIR="$WORK_DIR/.minio"
MINIO=("$PWD/minio" --config-dir "$MINIO_CONFIG_DIR")
SILO_CONFIG_DIR="$WORK_DIR/.silo"
SILO=("$PWD/silo" --config-dir "$SILO_CONFIG_DIR")
FILE_1_MB="$MINT_DATA_DIR/datafile-1-MB"
FILE_65_MB="$MINT_DATA_DIR/datafile-65-MB"
FUNCTIONAL_TESTS="$WORK_DIR/functional-tests.sh"
function start_minio_fs() {
function start_silo_fs() {
export MINIO_ROOT_USER=$ACCESS_KEY
export MINIO_ROOT_PASSWORD=$SECRET_KEY
"${MINIO[@]}" server "${WORK_DIR}/fs-disk" >"$WORK_DIR/fs-minio.log" 2>&1 &
"${SILO[@]}" server "${WORK_DIR}/fs-disk" >"$WORK_DIR/fs-silo.log" 2>&1 &
"${WORK_DIR}/mc" ready verify
}
function start_minio_erasure() {
"${MINIO[@]}" server "${WORK_DIR}/erasure-disk1" "${WORK_DIR}/erasure-disk2" "${WORK_DIR}/erasure-disk3" "${WORK_DIR}/erasure-disk4" >"$WORK_DIR/erasure-minio.log" 2>&1 &
function start_silo_erasure() {
"${SILO[@]}" server "${WORK_DIR}/erasure-disk1" "${WORK_DIR}/erasure-disk2" "${WORK_DIR}/erasure-disk3" "${WORK_DIR}/erasure-disk4" >"$WORK_DIR/erasure-silo.log" 2>&1 &
"${WORK_DIR}/mc" ready verify
}
function start_minio_erasure_sets() {
function start_silo_erasure_sets() {
export MINIO_ENDPOINTS="${WORK_DIR}/erasure-disk-sets{1...32}"
"${MINIO[@]}" server >"$WORK_DIR/erasure-minio-sets.log" 2>&1 &
"${SILO[@]}" server >"$WORK_DIR/erasure-silo-sets.log" 2>&1 &
"${WORK_DIR}/mc" ready verify
}
function start_minio_pool_erasure_sets() {
function start_silo_pool_erasure_sets() {
export MINIO_ROOT_USER=$ACCESS_KEY
export MINIO_ROOT_PASSWORD=$SECRET_KEY
export MINIO_ENDPOINTS="http://127.0.0.1:9000${WORK_DIR}/pool-disk-sets{1...4} http://127.0.0.1:9001${WORK_DIR}/pool-disk-sets{5...8}"
"${MINIO[@]}" server --address ":9000" >"$WORK_DIR/pool-minio-9000.log" 2>&1 &
"${MINIO[@]}" server --address ":9001" >"$WORK_DIR/pool-minio-9001.log" 2>&1 &
"${SILO[@]}" server --address ":9000" >"$WORK_DIR/pool-silo-9000.log" 2>&1 &
"${SILO[@]}" server --address ":9001" >"$WORK_DIR/pool-silo-9001.log" 2>&1 &
"${WORK_DIR}/mc" ready verify
}
function start_minio_pool_erasure_sets_ipv6() {
function start_silo_pool_erasure_sets_ipv6() {
export MINIO_ROOT_USER=$ACCESS_KEY
export MINIO_ROOT_PASSWORD=$SECRET_KEY
export MINIO_ENDPOINTS="http://[::1]:9000${WORK_DIR}/pool-disk-sets-ipv6{1...4} http://[::1]:9001${WORK_DIR}/pool-disk-sets-ipv6{5...8}"
"${MINIO[@]}" server --address="[::1]:9000" >"$WORK_DIR/pool-minio-ipv6-9000.log" 2>&1 &
"${MINIO[@]}" server --address="[::1]:9001" >"$WORK_DIR/pool-minio-ipv6-9001.log" 2>&1 &
"${SILO[@]}" server --address="[::1]:9000" >"$WORK_DIR/pool-silo-ipv6-9000.log" 2>&1 &
"${SILO[@]}" server --address="[::1]:9001" >"$WORK_DIR/pool-silo-ipv6-9001.log" 2>&1 &
"${WORK_DIR}/mc" ready verify_ipv6
}
function start_minio_dist_erasure() {
function start_silo_dist_erasure() {
export MINIO_ROOT_USER=$ACCESS_KEY
export MINIO_ROOT_PASSWORD=$SECRET_KEY
export MINIO_ENDPOINTS="http://127.0.0.1:9000${WORK_DIR}/dist-disk1 http://127.0.0.1:9001${WORK_DIR}/dist-disk2 http://127.0.0.1:9002${WORK_DIR}/dist-disk3 http://127.0.0.1:9003${WORK_DIR}/dist-disk4"
for i in $(seq 0 3); do
"${MINIO[@]}" server --address ":900${i}" >"$WORK_DIR/dist-minio-900${i}.log" 2>&1 &
"${SILO[@]}" server --address ":900${i}" >"$WORK_DIR/dist-silo-900${i}.log" 2>&1 &
done
"${WORK_DIR}/mc" ready verify
}
function run_test_fs() {
start_minio_fs
start_silo_fs
(cd "$WORK_DIR" && "$FUNCTIONAL_TESTS")
rv=$?
pkill minio
pkill silo
sleep 3
if [ "$rv" -ne 0 ]; then
cat "$WORK_DIR/fs-minio.log"
cat "$WORK_DIR/fs-silo.log"
fi
rm -f "$WORK_DIR/fs-minio.log"
rm -f "$WORK_DIR/fs-silo.log"
return "$rv"
}
function run_test_erasure_sets() {
start_minio_erasure_sets
start_silo_erasure_sets
(cd "$WORK_DIR" && "$FUNCTIONAL_TESTS")
rv=$?
pkill minio
pkill silo
sleep 3
if [ "$rv" -ne 0 ]; then
cat "$WORK_DIR/erasure-minio-sets.log"
cat "$WORK_DIR/erasure-silo-sets.log"
fi
rm -f "$WORK_DIR/erasure-minio-sets.log"
rm -f "$WORK_DIR/erasure-silo-sets.log"
return "$rv"
}
function run_test_pool_erasure_sets() {
start_minio_pool_erasure_sets
start_silo_pool_erasure_sets
(cd "$WORK_DIR" && "$FUNCTIONAL_TESTS")
rv=$?
pkill minio
pkill silo
sleep 3
if [ "$rv" -ne 0 ]; then
for i in $(seq 0 1); do
echo "server$i log:"
cat "$WORK_DIR/pool-minio-900$i.log"
cat "$WORK_DIR/pool-silo-900$i.log"
done
fi
for i in $(seq 0 1); do
rm -f "$WORK_DIR/pool-minio-900$i.log"
rm -f "$WORK_DIR/pool-silo-900$i.log"
done
return "$rv"
}
function run_test_pool_erasure_sets_ipv6() {
start_minio_pool_erasure_sets_ipv6
start_silo_pool_erasure_sets_ipv6
export SERVER_ENDPOINT="[::1]:9000"
(cd "$WORK_DIR" && "$FUNCTIONAL_TESTS")
rv=$?
pkill minio
pkill silo
sleep 3
if [ "$rv" -ne 0 ]; then
for i in $(seq 0 1); do
echo "server$i log:"
cat "$WORK_DIR/pool-minio-ipv6-900$i.log"
cat "$WORK_DIR/pool-silo-ipv6-900$i.log"
done
fi
for i in $(seq 0 1); do
rm -f "$WORK_DIR/pool-minio-ipv6-900$i.log"
rm -f "$WORK_DIR/pool-silo-ipv6-900$i.log"
done
return "$rv"
}
function run_test_erasure() {
start_minio_erasure
start_silo_erasure
(cd "$WORK_DIR" && "$FUNCTIONAL_TESTS")
rv=$?
pkill minio
pkill silo
sleep 3
if [ "$rv" -ne 0 ]; then
cat "$WORK_DIR/erasure-minio.log"
cat "$WORK_DIR/erasure-silo.log"
fi
rm -f "$WORK_DIR/erasure-minio.log"
rm -f "$WORK_DIR/erasure-silo.log"
return "$rv"
}
function run_test_dist_erasure() {
start_minio_dist_erasure
start_silo_dist_erasure
(cd "$WORK_DIR" && "$FUNCTIONAL_TESTS")
rv=$?
pkill minio
pkill silo
sleep 3
if [ "$rv" -ne 0 ]; then
echo "server1 log:"
cat "$WORK_DIR/dist-minio-9000.log"
cat "$WORK_DIR/dist-silo-9000.log"
echo "server2 log:"
cat "$WORK_DIR/dist-minio-9001.log"
cat "$WORK_DIR/dist-silo-9001.log"
echo "server3 log:"
cat "$WORK_DIR/dist-minio-9002.log"
cat "$WORK_DIR/dist-silo-9002.log"
echo "server4 log:"
cat "$WORK_DIR/dist-minio-9003.log"
cat "$WORK_DIR/dist-silo-9003.log"
fi
rm -f "$WORK_DIR/dist-minio-9000.log" "$WORK_DIR/dist-minio-9001.log" "$WORK_DIR/dist-minio-9002.log" "$WORK_DIR/dist-minio-9003.log"
rm -f "$WORK_DIR/dist-silo-9000.log" "$WORK_DIR/dist-silo-9001.log" "$WORK_DIR/dist-silo-9002.log" "$WORK_DIR/dist-silo-9003.log"
return "$rv"
}
@@ -216,31 +216,21 @@ function purge() {
function __init__() {
echo "Initializing environment"
mkdir -p "$WORK_DIR"
mkdir -p "$MINIO_CONFIG_DIR"
mkdir -p "$SILO_CONFIG_DIR"
mkdir -p "$MINT_DATA_DIR"
MC_BUILD_DIR="mc-$RANDOM"
if ! git clone --quiet https://github.com/minio/mc "$MC_BUILD_DIR"; then
echo "failed to download https://github.com/minio/mc"
purge "${MC_BUILD_DIR}"
exit 1
fi
(cd "${MC_BUILD_DIR}" && go build -o "${WORK_DIR}/mc")
# remove mc source.
purge "${MC_BUILD_DIR}"
"$(git rev-parse --show-toplevel)/buildscripts/install-mcli.sh" "${WORK_DIR}/mc"
shred -n 1 -s 1M - 1>"$FILE_1_MB" 2>/dev/null
shred -n 1 -s 65M - 1>"$FILE_65_MB" 2>/dev/null
## version is purposefully set to '3' for minio to migrate configuration file
echo '{"version": "3", "credential": {"accessKey": "minio", "secretKey": "minio123"}, "region": "us-east-1"}' >"$MINIO_CONFIG_DIR/config.json"
echo '{"version": "3", "credential": {"accessKey": "silo", "secretKey": "silo1234"}, "region": "us-east-1"}' >"$SILO_CONFIG_DIR/config.json"
if ! wget -q -O "$FUNCTIONAL_TESTS" https://raw.githubusercontent.com/minio/mc/master/functional-tests.sh; then
echo "failed to download https://raw.githubusercontent.com/minio/mc/master/functional-tests.sh"
exit 1
fi
"$(git rev-parse --show-toplevel)/buildscripts/install-verified-fixture.sh" \
https://raw.githubusercontent.com/pgsty/mc/4c4dcc4b55baf238cd0c81030d77945b3828f157/functional-tests.sh \
9b98c8152b294d567b9bc732869226dd4f65d0f4d84092dc066a900a66a9e22c \
"$FUNCTIONAL_TESTS"
sed -i 's|-sS|-sSg|g' "$FUNCTIONAL_TESTS"
chmod a+x "$FUNCTIONAL_TESTS"
@@ -292,7 +282,6 @@ function main() {
purge "$WORK_DIR"
}
(__init__ "$@" && main "$@")
rv=$?
purge "$WORK_DIR"
exit "$rv"
trap 'purge "$WORK_DIR"' EXIT
__init__ "$@"
main "$@"
@@ -4,18 +4,18 @@
set -E
set -o pipefail
if [ ! -x "$PWD/minio" ]; then
echo "minio executable binary not found in current directory"
if [ ! -x "$PWD/silo" ]; then
echo "Silo executable binary not found in current directory"
exit 1
fi
WORK_DIR="$PWD/.verify-$RANDOM"
MINIO_CONFIG_DIR="$WORK_DIR/.minio"
MINIO=("$PWD/minio" --config-dir "$MINIO_CONFIG_DIR" server)
SILO_CONFIG_DIR="$WORK_DIR/.silo"
SILO=("$PWD/silo" --config-dir "$SILO_CONFIG_DIR" server)
function start_minio_3_node() {
export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=minio123
function start_silo_3_node() {
export MINIO_ROOT_USER=silo
export MINIO_ROOT_PASSWORD=silo1234
export MINIO_ERASURE_SET_DRIVE_COUNT=6
export MINIO_CI_CD=1
@@ -25,26 +25,26 @@ function start_minio_3_node() {
args="$args http://127.0.0.1:$((start_port + i))${WORK_DIR}/$i/1/ http://127.0.0.1:$((start_port + i))${WORK_DIR}/$i/2/ http://127.0.0.1:$((start_port + i))${WORK_DIR}/$i/3/ http://127.0.0.1:$((start_port + i))${WORK_DIR}/$i/4/ http://127.0.0.1:$((start_port + i))${WORK_DIR}/$i/5/ http://127.0.0.1:$((start_port + i))${WORK_DIR}/$i/6/"
done
"${MINIO[@]}" --address ":$((start_port + 1))" $args >"${WORK_DIR}/dist-minio-server1.log" 2>&1 &
"${SILO[@]}" --address ":$((start_port + 1))" $args >"${WORK_DIR}/dist-silo-server1.log" 2>&1 &
pid1=$!
disown ${pid1}
"${MINIO[@]}" --address ":$((start_port + 2))" $args >"${WORK_DIR}/dist-minio-server2.log" 2>&1 &
"${SILO[@]}" --address ":$((start_port + 2))" $args >"${WORK_DIR}/dist-silo-server2.log" 2>&1 &
pid2=$!
disown $pid2
"${MINIO[@]}" --address ":$((start_port + 3))" $args >"${WORK_DIR}/dist-minio-server3.log" 2>&1 &
"${SILO[@]}" --address ":$((start_port + 3))" $args >"${WORK_DIR}/dist-silo-server3.log" 2>&1 &
pid3=$!
disown $pid3
export MC_HOST_myminio="http://minio:minio123@127.0.0.1:$((start_port + 1))"
export MC_HOST_mysilo="http://silo:silo1234@127.0.0.1:$((start_port + 1))"
timeout 15m /tmp/mc ready myminio || fail
timeout 15m /tmp/mc ready mysilo || fail
# Wait for all drives to be online and formatted
while [ $(/tmp/mc admin info --json myminio | jq '.info.servers[].drives[].state | select(. != "ok")' | wc -l) -gt 0 ]; do sleep 1; done
while [ $(/tmp/mc admin info --json mysilo | jq '.info.servers[].drives[].state | select(. != "ok")' | wc -l) -gt 0 ]; do sleep 1; done
# Wait for all drives to be healed
while [ $(/tmp/mc admin info --json myminio | jq '.info.servers[].drives[].healing | select(. != null) | select(. == true)' | wc -l) -gt 0 ]; do sleep 1; done
while [ $(/tmp/mc admin info --json mysilo | jq '.info.servers[].drives[].healing | select(. != null) | select(. == true)' | wc -l) -gt 0 ]; do sleep 1; done
# Wait for Status: in MinIO output
while true; do
@@ -66,26 +66,26 @@ function start_minio_3_node() {
done
if ! ps -p $pid1 1>&2 >/dev/null; then
echo "minio-server-1 is not running." && fail
echo "silo-server-1 is not running." && fail
fi
if ! ps -p $pid2 1>&2 >/dev/null; then
echo "minio-server-2 is not running." && fail
echo "silo-server-2 is not running." && fail
fi
if ! ps -p $pid3 1>&2 >/dev/null; then
echo "minio-server-3 is not running." && fail
echo "silo-server-3 is not running." && fail
fi
if ! pkill minio; then
if ! pkill silo; then
fail
fi
sleep 1
if pgrep minio; then
if pgrep silo; then
# forcibly killing, to proceed further properly.
if ! pkill -9 minio; then
echo "no minio process running anymore, proceed."
if ! pkill -9 silo; then
echo "no Silo process running anymore, proceed."
fi
fi
}
@@ -93,7 +93,7 @@ function start_minio_3_node() {
function fail() {
for i in $(seq 1 3); do
echo "server$i log:"
cat "${WORK_DIR}/dist-minio-server$i.log"
cat "${WORK_DIR}/dist-silo-server$i.log"
done
echo "FAILED"
purge "$WORK_DIR"
@@ -101,7 +101,7 @@ function fail() {
}
function check_online() {
if ! grep -q 'API:' ${WORK_DIR}/dist-minio-*.log; then
if ! grep -q 'API:' ${WORK_DIR}/dist-silo-*.log; then
echo "1"
fi
}
@@ -113,19 +113,18 @@ function purge() {
function __init__() {
echo "Initializing environment"
mkdir -p "$WORK_DIR"
mkdir -p "$MINIO_CONFIG_DIR"
mkdir -p "$SILO_CONFIG_DIR"
## version is purposefully set to '3' for minio to migrate configuration file
echo '{"version": "3", "credential": {"accessKey": "minio", "secretKey": "minio123"}, "region": "us-east-1"}' >"$MINIO_CONFIG_DIR/config.json"
echo '{"version": "3", "credential": {"accessKey": "silo", "secretKey": "silo1234"}, "region": "us-east-1"}' >"$SILO_CONFIG_DIR/config.json"
if [ ! -f /tmp/mc ]; then
wget --quiet -O /tmp/mc https://dl.minio.io/client/mc/release/linux-amd64/mc &&
chmod +x /tmp/mc
"$(git rev-parse --show-toplevel)/buildscripts/install-mcli.sh" /tmp/mc
fi
}
function perform_test() {
start_minio_3_node $2
start_silo_3_node $2
echo "Testing Distributed Erasure setup healing of drives"
echo "Remove the contents of the disks belonging to '${1}' erasure set"
@@ -133,7 +132,7 @@ function perform_test() {
rm -rf ${WORK_DIR}/${1}/*/
set -x
start_minio_3_node $2
start_silo_3_node $2
}
function main() {
+10 -10
View File
@@ -4,20 +4,20 @@ set -E
set -o pipefail
set -x
if [ ! -x "$PWD/minio" ]; then
echo "minio executable binary not found in current directory"
if [ ! -x "$PWD/silo" ]; then
echo "Silo executable binary not found in current directory"
exit 1
fi
WORK_DIR="$(mktemp -d)"
MINIO_CONFIG_DIR="$WORK_DIR/.minio"
MINIO=("$PWD/minio" --config-dir "$MINIO_CONFIG_DIR" server)
SILO_CONFIG_DIR="$WORK_DIR/.silo"
SILO=("$PWD/silo" --config-dir "$SILO_CONFIG_DIR" server)
function start_minio() {
function start_silo() {
start_port=$1
export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=minio123
export MINIO_ROOT_USER=silo
export MINIO_ROOT_PASSWORD=silo1234
unset MINIO_KMS_AUTO_ENCRYPTION # do not auto-encrypt objects
unset MINIO_CI_CD
unset CI
@@ -28,7 +28,7 @@ function start_minio() {
done
for i in $(seq 1 4); do
"${MINIO[@]}" --address ":$((start_port + i))" ${args[@]} 2>&1 >"${WORK_DIR}/server$i.log" &
"${SILO[@]}" --address ":$((start_port + i))" ${args[@]} 2>&1 >"${WORK_DIR}/server$i.log" &
done
# Wait until all nodes return 403
@@ -60,7 +60,7 @@ function prepare_block_devices() {
# Start a distributed MinIO setup, unmount one disk and check if it is formatted
function main() {
start_port=$(shuf -i 10000-65000 -n 1)
start_minio ${start_port}
start_silo ${start_port}
# Unmount the disk, after the unmount the device id
# /tmp/xxx/mnt/disk4 will be the same as '/' and it
@@ -82,7 +82,7 @@ function main() {
}
function cleanup() {
pkill minio
pkill silo
sudo umount ${WORK_DIR}/mnt/disk{1..3}/
sudo rm /dev/minio-loopdisk*
rm -rf "$WORK_DIR"
+30 -31
View File
@@ -4,23 +4,23 @@
set -E
set -o pipefail
if [ ! -x "$PWD/minio" ]; then
echo "minio executable binary not found in current directory"
if [ ! -x "$PWD/silo" ]; then
echo "Silo executable binary not found in current directory"
exit 1
fi
WORK_DIR="$PWD/.verify-$RANDOM"
MINIO_CONFIG_DIR="$WORK_DIR/.minio"
MINIO=("$PWD/minio" --config-dir "$MINIO_CONFIG_DIR" server)
SILO_CONFIG_DIR="$WORK_DIR/.silo"
SILO=("$PWD/silo" --config-dir "$SILO_CONFIG_DIR" server)
GOPATH=/tmp/gopath
function start_minio_3_node() {
function start_silo_3_node() {
for i in $(seq 1 3); do
rm "${WORK_DIR}/dist-minio-server$i.log"
rm "${WORK_DIR}/dist-silo-server$i.log"
done
export MINIO_ROOT_USER=minio
export MINIO_ROOT_PASSWORD=minio123
export MINIO_ROOT_USER=silo
export MINIO_ROOT_PASSWORD=silo1234
export MINIO_ERASURE_SET_DRIVE_COUNT=6
export MINIO_CI_CD=1
@@ -34,51 +34,51 @@ function start_minio_3_node() {
args="$args http://127.0.0.1:$((start_port + 1))${WORK_DIR}/1/${d}/ http://127.0.0.1:$((start_port + 2))${WORK_DIR}/2/${d}/ http://127.0.0.1:$((start_port + 3))${WORK_DIR}/3/${d}/ "
done
"${MINIO[@]}" --address ":$((start_port + 1))" $args >"${WORK_DIR}/dist-minio-server1.log" 2>&1 &
"${SILO[@]}" --address ":$((start_port + 1))" $args >"${WORK_DIR}/dist-silo-server1.log" 2>&1 &
pid1=$!
disown ${pid1}
"${MINIO[@]}" --address ":$((start_port + 2))" $args >"${WORK_DIR}/dist-minio-server2.log" 2>&1 &
"${SILO[@]}" --address ":$((start_port + 2))" $args >"${WORK_DIR}/dist-silo-server2.log" 2>&1 &
pid2=$!
disown $pid2
"${MINIO[@]}" --address ":$((start_port + 3))" $args >"${WORK_DIR}/dist-minio-server3.log" 2>&1 &
"${SILO[@]}" --address ":$((start_port + 3))" $args >"${WORK_DIR}/dist-silo-server3.log" 2>&1 &
pid3=$!
disown $pid3
export MC_HOST_myminio="http://minio:minio123@127.0.0.1:$((start_port + 1))"
timeout 15m /tmp/mc ready myminio || fail
export MC_HOST_mysilo="http://silo:silo1234@127.0.0.1:$((start_port + 1))"
timeout 15m /tmp/mc ready mysilo || fail
[ ${first_time} -eq 0 ] && upload_objects
[ ${first_time} -ne 0 ] && sleep 120
if ! ps -p $pid1 1>&2 >/dev/null; then
echo "minio server 1 is not running" && fail
echo "silo server 1 is not running" && fail
fi
if ! ps -p $pid2 1>&2 >/dev/null; then
echo "minio server 2 is not running" && fail
echo "silo server 2 is not running" && fail
fi
if ! ps -p $pid3 1>&2 >/dev/null; then
echo "minio server 3 is not running" && fail
echo "silo server 3 is not running" && fail
fi
if ! pkill minio; then
if ! pkill silo; then
fail
fi
sleep 1
if pgrep minio; then
if pgrep silo; then
# forcibly killing, to proceed further properly.
if ! pkill -9 minio; then
echo "no minio process running anymore, proceed."
if ! pkill -9 silo; then
echo "no Silo process running anymore, proceed."
fi
fi
}
function check_heal() {
if ! grep -q 'API:' ${WORK_DIR}/dist-minio-*.log; then
if ! grep -q 'API:' ${WORK_DIR}/dist-silo-*.log; then
return 1
fi
@@ -103,9 +103,9 @@ function purge() {
function fail() {
for i in $(seq 1 3); do
echo "server$i log:"
cat "${WORK_DIR}/dist-minio-server$i.log"
cat "${WORK_DIR}/dist-silo-server$i.log"
done
pkill -9 minio
pkill -9 silo
echo "FAILED"
purge "$WORK_DIR"
exit 1
@@ -114,28 +114,27 @@ function fail() {
function __init__() {
echo "Initializing environment"
mkdir -p "$WORK_DIR"
mkdir -p "$MINIO_CONFIG_DIR"
mkdir -p "$SILO_CONFIG_DIR"
## version is purposefully set to '3' for minio to migrate configuration file
echo '{"version": "3", "credential": {"accessKey": "minio", "secretKey": "minio123"}, "region": "us-east-1"}' >"$MINIO_CONFIG_DIR/config.json"
echo '{"version": "3", "credential": {"accessKey": "silo", "secretKey": "silo1234"}, "region": "us-east-1"}' >"$SILO_CONFIG_DIR/config.json"
if [ ! -f /tmp/mc ]; then
wget --quiet -O /tmp/mc https://dl.minio.io/client/mc/release/linux-amd64/mc &&
chmod +x /tmp/mc
"$(git rev-parse --show-toplevel)/buildscripts/install-mcli.sh" /tmp/mc
fi
}
function upload_objects() {
/tmp/mc mb myminio/testbucket/
/tmp/mc mb mysilo/testbucket/
for ((i = 0; i < 20; i++)); do
echo "my content" | /tmp/mc pipe myminio/testbucket/file-$i
echo "my content" | /tmp/mc pipe mysilo/testbucket/file-$i
done
}
function perform_test() {
start_port=$2
start_minio_3_node $start_port
start_silo_3_node $start_port
echo "Testing Distributed Erasure setup healing of drives"
echo "Remove the contents of the disks belonging to '${1}' node"
@@ -143,7 +142,7 @@ function perform_test() {
rm -rf ${WORK_DIR}/${1}/*/
set -x
start_minio_3_node $start_port
start_silo_3_node $start_port
check_heal ${1}
rv=$?
+122
View File
@@ -0,0 +1,122 @@
#!/usr/bin/env bash
set -euo pipefail
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
repo_dir="$(cd "${script_dir}/.." && pwd)"
baseline_commit="${HELM_LEGACY_COMMIT:-d88f46cce}"
helm_image="${HELM_IMAGE:-alpine/helm:3.18.6@sha256:c6d8088ddb279625a2e1ca3b08b22c18c946d1f65c8b810f28f1597435a1134c}"
work_dir="$(mktemp -d "${TMPDIR:-/tmp}/silo-helm.XXXXXX")"
cleanup() {
rm -rf "${work_dir}"
}
trap cleanup EXIT
cd "${repo_dir}"
git cat-file -e "${baseline_commit}^{commit}"
git archive "${baseline_commit}" helm/minio | tar -x -C "${work_dir}"
if command -v helm >/dev/null 2>&1; then
new_chart="${repo_dir}/helm/silo"
old_chart="${work_dir}/helm/minio"
output_dir="${work_dir}"
helm_run() {
helm "$@"
}
else
command -v docker >/dev/null 2>&1 || {
echo "helm or docker is required" >&2
exit 1
}
new_chart=/repo/helm/silo
old_chart=/check/helm/minio
output_dir=/check
helm_run() {
docker run --rm \
-v "${repo_dir}:/repo:ro" \
-v "${work_dir}:/check" \
"${helm_image}" "$@"
}
fi
helm_run lint "${new_chart}"
helm_run template silo "${new_chart}" \
--namespace silo \
--set rootUser=silo-admin \
--set rootPassword=test-password-123456 >/dev/null
helm_run template silo "${new_chart}" \
--namespace silo \
--set mode=standalone \
--set replicas=1 \
--set persistence.enabled=false \
--set rootUser=silo-admin \
--set rootPassword=test-password-123456 >/dev/null
# Exercise optional templates that the default render leaves dormant.
helm_run template silo-all "${new_chart}" \
--namespace silo \
--set rootUser=silo-admin \
--set rootPassword=test-password-123456 \
--set tls.enabled=true \
--set tls.certSecret=silo-tls \
--set trustedCertsSecret=silo-trusted-ca \
--set ingress.enabled=true \
--set consoleIngress.enabled=true \
--set networkPolicy.enabled=true \
--set podDisruptionBudget.enabled=true \
--set metrics.serviceMonitor.enabled=true \
--set metrics.serviceMonitor.includeNode=true \
--set 'buckets[0].name=chart-test' \
--set 'buckets[0].policy=none' \
--set 'buckets[0].purge=false' >/dev/null
# Existing values commonly address the historical myminio target. Render the
# custom-command path explicitly so both the new and compatibility aliases are
# protected by the release gate rather than only by a source-text assertion.
custom_render="${work_dir}/custom-command.yaml"
helm_run template silo-custom "${new_chart}" \
--namespace silo \
--set rootUser=silo-admin \
--set rootPassword=test-password-123456 \
--set-string 'customCommands[0].command=admin info myminio' \
--show-only templates/configmap.yaml >"${custom_render}"
for expected in \
'alias set mysilo' \
'alias set myminio' \
'runCommand admin info myminio'; do
grep -F -- "${expected}" "${custom_render}" >/dev/null || {
echo "rendered custom command is missing: ${expected}" >&2
exit 1
}
done
old_render="${work_dir}/legacy.yaml"
new_render="${work_dir}/candidate.yaml"
helm_run template my-release "${old_chart}" \
--namespace my-namespace \
--set rootUser=legacy-admin \
--set rootPassword=legacy-password-123456 >"${old_render}"
helm_run template my-release "${new_chart}" \
--namespace my-namespace \
-f "${old_chart}/values.yaml" \
--set rootUser=legacy-admin \
--set rootPassword=legacy-password-123456 \
--set nameOverride=minio \
--set fullnameOverride=my-release-minio \
--set serviceAccount.name=minio-sa \
--set image.repository=pgsty/silo \
--set mcImage.repository=pgsty/silo \
--set-string image.tag=RELEASE.2026-08-04T00-00-00Z \
--set-string mcImage.tag=RELEASE.2026-08-04T00-00-00Z >"${new_render}"
go run ./buildscripts/helm-migration-guard "${old_render}" "${new_render}"
helm_run package "${new_chart}" --destination "${output_dir}" >/dev/null
test -s "${work_dir}/silo-7.0.3.tgz"
if find "${work_dir}" -maxdepth 1 -type f -name 'minio-*.tgz' | grep -q .; then
echo "Helm packaging emitted a legacy MinIO chart name" >&2
exit 1
fi
echo "Silo Helm lint, render, legacy-upgrade, and package checks passed"
+220
View File
@@ -0,0 +1,220 @@
#!/usr/bin/env bash
set -euo pipefail
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
repo_dir="$(cd "${script_dir}/.." && pwd)"
cd "${repo_dir}"
fail() {
echo "Silo rebrand verification failed: $*" >&2
exit 1
}
require_file() {
[ -f "$1" ] || fail "missing required file: $1"
}
require_text() {
local file="$1"
local text="$2"
grep -Fq -- "${text}" "${file}" || fail "${file} does not contain: ${text}"
}
reject_text() {
local file="$1"
local text="$2"
if grep -Fq -- "${text}" "${file}"; then
fail "${file} still contains forbidden delivery text: ${text}"
fi
}
for file in \
.github/goreleaser.yml \
.github/nfpm.yml \
buildscripts/package/lifecycle_test.sh \
buildscripts/verify-helm-migration.sh \
Dockerfile.goreleaser \
Dockerfile.distroless \
dockerscripts/build-static-curl.sh \
dockerscripts/docker-entrypoint.sh \
helm/silo/Chart.yaml \
helm/silo/values.yaml \
silo.service \
silo.env \
silo.sysusers; do
require_file "${file}"
done
for retired_path in \
CNAME _config.yml index.yaml helm-reindex.sh helm/minio helm-releases \
Dockerfile Dockerfile.cicd Dockerfile.hotfix Dockerfile.release \
Dockerfile.release.old_cpu Dockerfile.scratch docker-buildx.sh \
minio.service cmd/callhome.go buildscripts/upgrade-tests .github/logo.svg \
docs/federation/lookup/bucket-lookup.png \
docs/screenshots/Minio_Cloud_Native_Arch.jpg \
docs/screenshots/Minio_Cloud_Native_Arch.png \
docs/screenshots/Minio_Cloud_Native_Arch.svg \
docs/screenshots/Architecture-diagram_distributed_8.jpg \
docs/screenshots/Architecture-diagram_distributed_8.png \
docs/screenshots/Architecture-diagram_distributed_8.svg \
docs/screenshots/Architecture-diagram_distributed_16.jpg \
docs/screenshots/Architecture-diagram_distributed_16.png \
docs/screenshots/Architecture-diagram_distributed_16.svg \
docs/screenshots/Architecture-diagram_distributed_nm.png \
docs/screenshots/Example-1.jpg docs/screenshots/Example-1.png \
docs/screenshots/Example-2.jpg docs/screenshots/Example-2.png \
docs/screenshots/Example-3.jpg docs/screenshots/Example-3.png \
docs/screenshots/pic1.png docs/screenshots/pic2.png \
docs/metrics/prometheus/grafana/grafana-minio.png \
docs/metrics/prometheus/grafana/bucket/grafana-bucket.png \
docs/metrics/prometheus/grafana/node/grafana-node.png \
docs/metrics/prometheus/grafana/replication/grafana-replication-cluster.png \
docs/metrics/prometheus/grafana/replication/grafana-replication-node.png; do
[ ! -e "${retired_path}" ] || fail "retired upstream delivery path remains: ${retired_path}"
done
require_text .github/goreleaser.yml "binary: silo"
require_text .github/goreleaser.yml 'name_template: "silo_{{ .Env.PKG_VERSION }}_{{ .Os }}_{{ .Arch }}"'
require_text .github/goreleaser.yml "sboms:"
require_text .github/goreleaser.yml "artifacts: archive"
require_text .github/goreleaser.yml "cmd: cosign"
# shellcheck disable=SC2016 # Match the literal GoReleaser template variable.
require_text .github/goreleaser.yml 'signature: "${artifact}.sigstore.json"'
require_text .github/nfpm.yml "name: silo"
require_text .github/nfpm.yml "dst: /usr/bin/silo"
require_text .github/nfpm.yml "dst: /etc/default/silo"
require_text .github/nfpm.yml "dst: /usr/lib/sysusers.d/silo.conf"
require_text buildscripts/package/lifecycle_test.sh "Silo package lifecycle checks passed"
require_text buildscripts/verify-helm-migration.sh "Silo Helm lint, render, legacy-upgrade, and package checks passed"
require_text silo.service "Conflicts=minio.service"
require_text silo.service "EnvironmentFile=-/etc/default/minio"
require_text silo.service "EnvironmentFile=-/etc/default/silo"
# shellcheck disable=SC2016 # Match the literal service environment variables.
require_text silo.service 'ExecStart=/usr/bin/silo server $MINIO_OPTS $MINIO_VOLUMES'
require_text README.md "/etc/systemd/system/silo.service.d/10-legacy-user.conf"
require_text README_ZH.md "/etc/systemd/system/silo.service.d/10-legacy-user.conf"
require_text Dockerfile.goreleaser "COPY silo /usr/bin/silo"
require_text Dockerfile.goreleaser 'CMD ["silo"]'
require_text Dockerfile.goreleaser "MC_AMD64_SHA256="
require_text Dockerfile.goreleaser "Published checksum drift"
require_text Dockerfile.distroless 'COPY --chmod=0755 silo /usr/bin/silo'
require_text Dockerfile.distroless 'ENTRYPOINT ["/usr/bin/silo"]'
require_text Dockerfile.distroless '"/usr/bin/silo", "healthcheck", "ready"'
require_text dockerscripts/build-static-curl.sh "sha256sum -c"
require_text helm/silo/Chart.yaml "name: silo"
require_text helm/silo/values.yaml "repository: pgsty/silo"
require_text helm/silo/templates/deployment.yaml "/usr/bin/docker-entrypoint.sh silo server"
require_text helm/silo/templates/statefulset.yaml "/usr/bin/docker-entrypoint.sh silo server"
require_text docs/orchestration/docker-compose/docker-compose.yaml 'http://silo{1...4}/data{1...2}'
require_text docs/resiliency/docker-compose.yaml 'http://silo{1...4}/data{1...8}'
require_text docs/distributed/DECOMMISSION.md 'systemctl restart silo'
# shellcheck disable=SC2016 # Match the literal shell variable.
require_text docs/resiliency/resiliency-tests.sh 'docker exec resiliency-silo$NODE-1'
require_text .github/workflows/release.yml "Attest downloadable release artifacts"
require_text .github/workflows/release.yml "packages_checksums.txt"
require_text .github/workflows/docker-release.yml "Attest multi-architecture image provenance"
require_text .github/workflows/docker-release.yml "index.docker.io/pgsty/silo"
# Copyright notices credit both parties with fixed terms: upstream MinIO
# development ends at its own last year, and the fork's own term starts when
# the fork did. Deriving the upstream end year from the clock would extend
# MinIO's copyright term every January.
require_text cmd/build-constants.go 'upstreamCopyrightEndYear = "2025"'
require_text cmd/build-constants.go 'forkCopyrightStartYear = "2025"'
require_text cmd/main.go 'upstreamCopyrightEndYear'
reject_text cmd/main.go 'CopyrightYear = strconv.Itoa(time.Now().Year())'
require_text NOTICE 'MinIO Project, (C) 2015-2025 MinIO, Inc.'
require_text NOTICE 'Silo Project modifications, (C) 2025-2026 PGSTY.'
# Contribution policy: no CLA, inbound=outbound, DCO sign-off enforced in CI.
require_file .github/workflows/dco.yml
require_text .github/workflows/dco.yml "Signed-off-by"
require_text CONTRIBUTING.md "developercertificate.org"
require_text CONTRIBUTING.md "No CLA"
for file in .github/nfpm.yml Dockerfile.goreleaser silo.service; do
reject_text "${file}" "/usr/bin/minio"
reject_text "${file}" "/usr/local/bin/minio"
done
reject_text Dockerfile.goreleaser "MINIO_UPDATE_MINISIGN_PUBKEY"
reject_text buildscripts/minio-upgrade.sh "docker system prune"
reject_text buildscripts/minio-upgrade.sh "docker volume prune"
reject_text docs/orchestration/docker-compose/docker-compose.yaml 'http://minio{1...4}'
reject_text docs/resiliency/docker-compose.yaml 'http://minio{1...4}'
reject_text docs/distributed/DECOMMISSION.md 'systemctl restart minio'
reject_text docs/resiliency/resiliency-tests.sh 'resiliency-minio'
reject_text docs/resiliency/resiliency-tests.sh 'docker system prune'
reject_text docs/resiliency/resiliency-tests.sh 'docker image prune'
reject_text docs/resiliency/resiliency-tests.sh 'docker ps -q'
if grep -Ev '^[[:space:]]*(#|$)' silo.env | grep -q '='; then
fail "silo.env must not contain active assignments that shadow /etc/default/minio"
fi
if rg -n 'pgsty/minio:' .github/workflows Dockerfile.goreleaser helm/silo; then
fail "an active delivery surface still publishes the frozen pgsty/minio image"
fi
# The repository and its default branch are pgsty/silo and main. The invariant
# is that the old name is never a live target, not that it is never spoken: the
# READMEs have to name it to explain the rename and to point at the archived
# artifacts, which is the opposite of stranding a reader on it. CONTRIBUTORS.md
# also quotes historical issue titles.
#
# So two rules. First, no live URL may resolve to the old repository anywhere,
# READMEs and CONTRIBUTORS.md included.
old_repo_pattern='pgsty/minio(\.git)?([^[:alnum:]_.-]|$)'
stale_repo_url="$(rg -n -e "github\.com/${old_repo_pattern}" -e "hub\.docker\.com/r/${old_repo_pattern}" \
--glob '!.git/**' --glob '!dist/**' \
--glob '!SILO_REBRANDING_MIGRATION.md' \
--glob '!buildscripts/rebrand-guard/compat-baseline.json' . |
sed 's#^\./##' | grep -v '^buildscripts/verify-rebrand\.sh:' || true)"
if [ -n "${stale_repo_url}" ]; then
printf '%s\n' "${stale_repo_url}" >&2
fail "a link still resolves to the pre-rename pgsty/minio repository"
fi
# Second, the bare name may only appear where it is deliberate: the pinned
# pre-rebrand image digest in the upgrade test, the two guards that refuse a
# legacy image, the two READMEs that document the rename and the archived
# minio branch, and historical issue titles in CONTRIBUTORS.md.
repo_guard_allowlist='^(buildscripts/minio-upgrade\.sh|buildscripts/verify-rebrand\.sh|buildscripts/helm-migration-guard/main\.go|README\.md|README_ZH\.md|CONTRIBUTORS\.md):'
stale_repo="$(rg -n "${old_repo_pattern}" --glob '!.git/**' --glob '!dist/**' \
--glob '!SILO_REBRANDING_MIGRATION.md' \
--glob '!buildscripts/rebrand-guard/compat-baseline.json' . |
sed 's#^\./##' | grep -Ev "${repo_guard_allowlist}" || true)"
if [ -n "${stale_repo}" ]; then
printf '%s\n' "${stale_repo}" >&2
fail "a source reference still names the pre-rename pgsty/minio repository"
fi
stale_branch="$(rg -n 'pgsty/silo/(blob/|tree/|raw/)?master' \
--glob '!.git/**' --glob '!dist/**' . || true)"
if [ -n "${stale_branch}" ]; then
printf '%s\n' "${stale_branch}" >&2
fail "a link still targets the retired master branch; raw and Actions URLs do not follow a branch rename"
fi
for workflow in .github/workflows/go.yml .github/workflows/vulncheck.yml; do
if rg -q '^\s+- master$' "${workflow}"; then
fail "${workflow} still filters on master and would go silently dormant on main"
fi
require_text "${workflow}" " - main"
done
network_hits="$(rg -n --glob '*.go' --glob '!**/*_test.go' \
'https?://[^"`[:space:]]*(dl\.min\.io|subnet\.min\.io|api\.min\.io|slack\.min\.io|play\.min\.io)' \
cmd internal || true)"
network_hits="$(printf '%s\n' "${network_hits}" | grep -Ev '^[^:]+:[0-9]+:[[:space:]]*//' || true)"
if [ -n "${network_hits}" ]; then
printf '%s\n' "${network_hits}" >&2
fail "runtime code still contains an upstream MinIO service endpoint"
fi
require_text cmd/build-constants.go 'MinioReleaseBaseURL = ""'
require_text cmd/globals.go "globalInplaceUpdateDisabled = true"
reject_text cmd/globals.go "subnetAdminPublicKey"
reject_text cmd/admin-handlers.go "getSubnetAdminPublicKey"
echo "Silo delivery and runtime rebrand checks passed"
+1 -1
View File
@@ -25,7 +25,7 @@ import (
xhttp "github.com/minio/minio/internal/http"
"github.com/minio/minio/internal/logger"
"github.com/minio/mux"
"github.com/minio/pkg/v3/policy"
"github.com/pgsty/silo-pkg/v3/policy"
)
// Data types used for returning dummy access control
+348
View File
@@ -0,0 +1,348 @@
package cmd
import (
"archive/zip"
"bytes"
"encoding/base64"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"github.com/minio/madmin-go/v3"
"github.com/minio/minio/internal/auth"
"github.com/minio/mux"
)
func corsAdminRequest(t *testing.T, cred auth.Credentials, method, path string, body []byte) *httptest.ResponseRecorder {
t.Helper()
router := mux.NewRouter()
registerAdminRouter(router, true)
req, err := newTestSignedRequestV4(method, adminPathPrefix+adminAPIVersionPrefix+path,
int64(len(body)), bytes.NewReader(body), cred.AccessKey, cred.SecretKey, nil)
if err != nil {
t.Fatal(err)
}
rec := httptest.NewRecorder()
router.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("admin %s: %d: %s", path, rec.Code, rec.Body.String())
}
return rec
}
func corsImportReport(t *testing.T, rec *httptest.ResponseRecorder) madmin.BucketMetaImportErrs {
t.Helper()
var rpt madmin.BucketMetaImportErrs
if err := json.Unmarshal(rec.Body.Bytes(), &rpt); err != nil {
t.Fatalf("import report %q: %v", rec.Body.String(), err)
}
return rpt
}
func corsZip(t *testing.T, entries map[string][]byte) []byte {
t.Helper()
var buf bytes.Buffer
zw := zip.NewWriter(&buf)
for name, data := range entries {
w, err := zw.Create(name)
if err != nil {
t.Fatal(err)
}
if _, err = w.Write(data); err != nil {
t.Fatal(err)
}
}
if err := zw.Close(); err != nil {
t.Fatal(err)
}
return buf.Bytes()
}
// corsCorruptedZip builds an archive holding a stored (uncompressed) cors.xml
// whose payload is altered after the checksum is computed, plus the given
// companion entries. The altered document stays well formed, so only the zip
// checksum tells the two apart.
func corsCorruptedZip(t *testing.T, name string, doc []byte, others map[string][]byte) []byte {
t.Helper()
var buf bytes.Buffer
zw := zip.NewWriter(&buf)
w, err := zw.CreateHeader(&zip.FileHeader{Name: name, Method: zip.Store})
if err != nil {
t.Fatal(err)
}
if _, err = w.Write(doc); err != nil {
t.Fatal(err)
}
for other, data := range others {
ow, err := zw.Create(other)
if err != nil {
t.Fatal(err)
}
if _, err = ow.Write(data); err != nil {
t.Fatal(err)
}
}
if err = zw.Close(); err != nil {
t.Fatal(err)
}
raw := buf.Bytes()
at := bytes.Index(raw, []byte("app.example.com"))
if at < 0 {
t.Fatalf("stored CORS payload not found in archive")
}
raw[at] = 'A'
return raw
}
// TestAdminBucketMetadataCORSRoundTrip covers the export/import round trip for
// per-bucket CORS, per-file error reporting for an invalid document, and that
// an archive without cors.xml leaves an existing configuration alone.
func TestAdminBucketMetadataCORSRoundTrip(t *testing.T) {
ExecObjectLayerAPITest(ExecObjectLayerAPITestArgs{t: t, objAPITest: func(obj ObjectLayer, instanceType, bucket string, _ http.Handler, cred auth.Credentials, t *testing.T) {
corsXML := []byte(testSiteReplicationCORSDoc)
if _, err := updateLocalBucketCORSMetadata(t.Context(), obj, bucket, corsXML); err != nil {
t.Fatal(err)
}
// Export must carry the stored document verbatim.
rec := corsAdminRequest(t, cred, http.MethodGet, "/export-bucket-metadata?bucket="+bucket, nil)
archive := rec.Body.Bytes()
zr, err := zip.NewReader(bytes.NewReader(archive), int64(len(archive)))
if err != nil {
t.Fatal(err)
}
var exported []byte
for _, f := range zr.File {
if f.Name != bucket+"/"+bucketCorsConfig {
continue
}
r, err := f.Open()
if err != nil {
t.Fatal(err)
}
exported, err = io.ReadAll(r)
r.Close()
if err != nil {
t.Fatal(err)
}
}
if !bytes.Equal(exported, corsXML) {
t.Fatalf("%s: exported CORS = %q, want %q", instanceType, exported, corsXML)
}
// Drop the configuration: the archive must then omit the entry.
if _, err = updateLocalBucketCORSMetadata(t.Context(), obj, bucket, nil); err != nil {
t.Fatal(err)
}
if _, _, err = globalBucketMetadataSys.GetCorsConfigXML(bucket); err == nil {
t.Fatalf("%s: CORS still present before restore", instanceType)
}
rec = corsAdminRequest(t, cred, http.MethodGet, "/export-bucket-metadata?bucket="+bucket, nil)
empty := rec.Body.Bytes()
zr, err = zip.NewReader(bytes.NewReader(empty), int64(len(empty)))
if err != nil {
t.Fatal(err)
}
for _, f := range zr.File {
if f.Name == bucket+"/"+bucketCorsConfig {
t.Fatalf("%s: export emitted %s for a bucket without CORS", instanceType, f.Name)
}
}
rec = corsAdminRequest(t, cred, http.MethodPut, "/import-bucket-metadata", archive)
if st := corsImportReport(t, rec).Buckets[bucket]; !st.Cors.IsSet || st.Cors.Err != "" {
t.Fatalf("%s: import report cors = %+v", instanceType, st.Cors)
}
stored, storedAt, err := globalBucketMetadataSys.GetCorsConfigXML(bucket)
if err != nil || !bytes.Equal(stored, corsXML) {
t.Fatalf("%s: restored CORS = %q, err = %v", instanceType, stored, err)
}
created, err := globalBucketMetadataSys.CreatedAt(bucket)
if err != nil {
t.Fatal(err)
}
if !storedAt.After(created) {
t.Fatalf("%s: restored CORS timestamp %v is not after bucket creation %v", instanceType, storedAt, created)
}
// An archive without cors.xml must not remove the configuration.
corsAdminRequest(t, cred, http.MethodPut, "/import-bucket-metadata",
corsZip(t, map[string][]byte{bucket + "/quota.json": []byte(`{"quota":0}`)}))
if stored, _, err = globalBucketMetadataSys.GetCorsConfigXML(bucket); err != nil || !bytes.Equal(stored, corsXML) {
t.Fatalf("%s: import without cors.xml changed CORS: %q, err = %v", instanceType, stored, err)
}
// A bucket the import itself creates must still land above its own
// creation time, otherwise CORS replication would drop the restore.
fresh := "cors-import-created-bucket"
rec = corsAdminRequest(t, cred, http.MethodPut, "/import-bucket-metadata",
corsZip(t, map[string][]byte{fresh + "/" + bucketCorsConfig: corsXML}))
if st := corsImportReport(t, rec).Buckets[fresh]; !st.Cors.IsSet || st.Cors.Err != "" {
t.Fatalf("%s: fresh bucket import report cors = %+v", instanceType, st.Cors)
}
freshStored, freshAt, err := globalBucketMetadataSys.GetCorsConfigXML(fresh)
if err != nil || !bytes.Equal(freshStored, corsXML) {
t.Fatalf("%s: fresh bucket CORS = %q, err = %v", instanceType, freshStored, err)
}
freshCreated, err := globalBucketMetadataSys.CreatedAt(fresh)
if err != nil {
t.Fatal(err)
}
if !freshAt.After(freshCreated) {
t.Fatalf("%s: fresh bucket CORS timestamp %v is not after creation %v", instanceType, freshAt, freshCreated)
}
// An invalid document must fail loudly for that bucket and change nothing.
rec = corsAdminRequest(t, cred, http.MethodPut, "/import-bucket-metadata",
corsZip(t, map[string][]byte{bucket + "/" + bucketCorsConfig: []byte("<CORSConfiguration><CORSRule>")}))
if st := corsImportReport(t, rec).Buckets[bucket]; st.Cors.Err == "" {
t.Fatalf("%s: invalid CORS import reported no error: %+v", instanceType, st)
}
if stored, _, err = globalBucketMetadataSys.GetCorsConfigXML(bucket); err != nil || !bytes.Equal(stored, corsXML) {
t.Fatalf("%s: invalid CORS import changed stored config: %q, err = %v", instanceType, stored, err)
}
// A well formed document carried by a corrupt zip entry must be
// rejected too, leaving the stored document and its timestamp alone
// while the other configs in the same archive still apply.
_, corsAt, err := globalBucketMetadataSys.GetCorsConfigXML(bucket)
if err != nil {
t.Fatal(err)
}
rec = corsAdminRequest(t, cred, http.MethodPut, "/import-bucket-metadata",
corsCorruptedZip(t, bucket+"/"+bucketCorsConfig, corsXML,
map[string][]byte{bucket + "/quota.json": []byte(`{"quota":4096,"quotatype":"hard"}`)}))
st := corsImportReport(t, rec).Buckets[bucket]
if st.Cors.Err == "" {
t.Fatalf("%s: corrupt CORS entry reported no error: %+v", instanceType, st)
}
if !st.Quota.IsSet || st.Quota.Err != "" {
t.Fatalf("%s: corrupt CORS entry blocked the neighboring quota: %+v", instanceType, st.Quota)
}
stored, storedAt, err = globalBucketMetadataSys.GetCorsConfigXML(bucket)
if err != nil || !bytes.Equal(stored, corsXML) || !storedAt.Equal(corsAt) {
t.Fatalf("%s: corrupt CORS entry changed stored config: %q at %v (was %v), err = %v", instanceType, stored, storedAt, corsAt, err)
}
quota, _, err := globalBucketMetadataSys.GetQuotaConfig(t.Context(), bucket)
if err != nil || quota == nil || quota.Quota != 4096 {
t.Fatalf("%s: neighboring quota not applied: %+v, err = %v", instanceType, quota, err)
}
}})
}
// corsPeerStub is a stand-in site-replication peer. It records every
// SRBucketMeta it is asked to apply and answers with status.
func corsPeerStub(t *testing.T, applied chan<- madmin.SRBucketMeta, status int) *httptest.Server {
t.Helper()
return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodPut && applied != nil {
var item madmin.SRBucketMeta
if err := json.NewDecoder(r.Body).Decode(&item); err != nil {
t.Errorf("decode peer apply: %v", err)
w.WriteHeader(http.StatusBadRequest)
return
}
applied <- item
}
w.WriteHeader(status)
}))
}
// TestAdminBucketMetadataCORSImportReplicatesPastPeerFailure pins that an
// imported CORS document reaches the reachable peers even when the shared
// bucket metadata hook failed against an unreachable one, and that both
// failures are still reported for the bucket.
func TestAdminBucketMetadataCORSImportReplicatesPastPeerFailure(t *testing.T) {
ExecObjectLayerAPITest(ExecObjectLayerAPITestArgs{t: t, objAPITest: func(obj ObjectLayer, instanceType, bucket string, _ http.Handler, cred auth.Credentials, t *testing.T) {
ctx := t.Context()
corsXML := []byte(testSiteReplicationCORSDoc)
healthyApplies := make(chan madmin.SRBucketMeta, 4)
healthy := corsPeerStub(t, healthyApplies, http.StatusOK)
defer healthy.Close()
broken := corsPeerStub(t, nil, http.StatusBadRequest)
defer broken.Close()
// With site replication on, admin requests resolve their token signing
// key through the site replicator account, so it has to exist.
serviceCred, err := auth.CreateCredentials(siteReplicatorSvcAcc, "cors-import-service-secret")
if err != nil {
t.Fatal(err)
}
serviceCred.ParentUser = cred.AccessKey
if _, err = globalIAMSys.store.AddServiceAccount(ctx, serviceCred); err != nil {
t.Fatal(err)
}
defer globalIAMSys.DeleteServiceAccount(ctx, serviceCred.AccessKey, false)
globalSiteReplicatorCred.Set(serviceCred.SecretKey)
defer globalSiteReplicatorCred.Set("")
globalSiteReplicationSys.Lock()
oldEnabled, oldState := globalSiteReplicationSys.enabled, globalSiteReplicationSys.state
globalSiteReplicationSys.enabled = true
globalSiteReplicationSys.state = srState{
Name: "cors-import-test",
ServiceAccountAccessKey: serviceCred.AccessKey,
Peers: map[string]madmin.PeerInfo{
globalDeploymentID(): {Name: "local", DeploymentID: globalDeploymentID()},
"peer-healthy": {Name: "healthy", DeploymentID: "peer-healthy", Endpoint: healthy.URL},
"peer-broken": {Name: "broken", DeploymentID: "peer-broken", Endpoint: broken.URL},
},
}
globalSiteReplicationSys.Unlock()
defer func() {
globalSiteReplicationSys.Lock()
globalSiteReplicationSys.enabled, globalSiteReplicationSys.state = oldEnabled, oldState
globalSiteReplicationSys.Unlock()
}()
rec := corsAdminRequest(t, cred, http.MethodPut, "/import-bucket-metadata",
corsZip(t, map[string][]byte{
bucket + "/" + bucketCorsConfig: corsXML,
bucket + "/quota.json": []byte(`{"quota":8192,"quotatype":"hard"}`),
}))
st := corsImportReport(t, rec).Buckets[bucket]
if !st.Cors.IsSet || st.Cors.Err != "" {
t.Fatalf("%s: import report cors = %+v", instanceType, st.Cors)
}
stored, storedAt, err := globalBucketMetadataSys.GetCorsConfigXML(bucket)
if err != nil || !bytes.Equal(stored, corsXML) {
t.Fatalf("%s: stored CORS = %q, err = %v", instanceType, stored, err)
}
// The reachable peer must have been told about the CORS document,
// carrying exactly the timestamp that was saved locally.
var corsSeen, sharedSeen bool
for range 2 {
select {
case item := <-healthyApplies:
if item.Type != madmin.SRBucketMetaTypeCorsConfig {
sharedSeen = item.Bucket == bucket && item.Quota != nil
continue
}
if item.Bucket != bucket || item.Cors == nil || !item.UpdatedAt.Equal(storedAt) {
t.Fatalf("%s: peer CORS event = %#v, want %s at %v", instanceType, item, bucket, storedAt)
}
payload, decErr := base64.StdEncoding.Strict().DecodeString(*item.Cors)
if decErr != nil || !bytes.Equal(payload, corsXML) {
t.Fatalf("%s: peer CORS payload = %q, err = %v", instanceType, payload, decErr)
}
corsSeen = true
case <-time.After(10 * time.Second):
t.Fatalf("%s: healthy peer received no further events (shared=%v cors=%v)", instanceType, sharedSeen, corsSeen)
}
}
if !sharedSeen || !corsSeen {
t.Fatalf("%s: healthy peer events shared=%v cors=%v, want both", instanceType, sharedSeen, corsSeen)
}
// Both hook failures against the unreachable peer stay reported.
if got := strings.Count(st.Err, "->broken:"); got != 2 {
t.Fatalf("%s: bucket error mentions the broken peer %d times, want 2: %q", instanceType, got, st.Err)
}
}})
}
+205 -19
View File
@@ -41,7 +41,7 @@ import (
xhttp "github.com/minio/minio/internal/http"
"github.com/minio/minio/internal/kms"
"github.com/minio/mux"
"github.com/minio/pkg/v3/policy"
"github.com/pgsty/silo-pkg/v3/policy"
)
const (
@@ -76,7 +76,7 @@ func (a adminAPIHandlers) PutBucketQuotaConfigHandler(w http.ResponseWriter, r *
return
}
quotaConfig, err := parseBucketQuota(bucket, data)
_, err = parseBucketQuota(bucket, data)
if err != nil {
writeErrorResponse(ctx, w, toAPIError(ctx, err), r.URL)
return
@@ -94,9 +94,6 @@ func (a adminAPIHandlers) PutBucketQuotaConfigHandler(w http.ResponseWriter, r *
Quota: data,
UpdatedAt: updatedAt,
}
if quotaConfig.Size == 0 && quotaConfig.Quota == 0 {
bucketMeta.Quota = nil
}
// Call site replication hook.
replLogIf(ctx, globalSiteReplicationSys.BucketMetaHook(ctx, bucketMeta))
@@ -417,6 +414,7 @@ func (a adminAPIHandlers) ExportBucketMetadataHandler(w http.ResponseWriter, r *
bucketLifecycleConfig,
bucketSSEConfig,
bucketTaggingConfig,
bucketCorsConfig,
bucketQuotaConfigFile,
objectLockConfig,
bucketVersioningConfig,
@@ -437,7 +435,7 @@ func (a adminAPIHandlers) ExportBucketMetadataHandler(w http.ResponseWriter, r *
writeErrorResponse(ctx, w, exportError(ctx, err, cfgFile, bucket), r.URL)
return
}
configData, err := json.Marshal(config)
configData, err := canonicalBucketPolicy(config)
if err != nil {
writeErrorResponse(ctx, w, exportError(ctx, err, cfgFile, bucket), r.URL)
return
@@ -517,6 +515,19 @@ func (a adminAPIHandlers) ExportBucketMetadataHandler(w http.ResponseWriter, r *
return
}
rawDataFn(bytes.NewReader(configData), cfgPath, len(configData))
case bucketCorsConfig:
// Export the stored document verbatim: GetBucketCors returns
// the bytes exactly as they were PUT, so the archive must
// round-trip them unchanged.
configData, _, err := globalBucketMetadataSys.GetCorsConfigXML(bucket)
if err != nil {
if errors.Is(err, errConfigNotFound) {
continue
}
writeErrorResponse(ctx, w, exportError(ctx, err, cfgFile, bucket), r.URL)
return
}
rawDataFn(bytes.NewReader(configData), cfgPath, len(configData))
case objectLockConfig:
config, _, err := globalBucketMetadataSys.GetObjectLockConfig(bucket)
if err != nil {
@@ -589,6 +600,50 @@ type importMetaReport struct {
madmin.BucketMetaImportErrs
}
type importMetadataFields map[string]struct{}
func (f importMetadataFields) add(configFile string) {
f[configFile] = struct{}{}
}
func applyImportedBucketMetadata(dst *BucketMetadata, src BucketMetadata, fields importMetadataFields) {
for configFile := range fields {
switch configFile {
case bucketPolicyConfig:
dst.PolicyConfigJSON = bytes.Clone(src.PolicyConfigJSON)
dst.PolicyConfigUpdatedAt = src.PolicyConfigUpdatedAt
case bucketNotificationConfig:
dst.NotificationConfigXML = bytes.Clone(src.NotificationConfigXML)
dst.NotificationConfigUpdatedAt = src.NotificationConfigUpdatedAt
case bucketLifecycleConfig:
dst.LifecycleConfigXML = bytes.Clone(src.LifecycleConfigXML)
dst.LifecycleConfigUpdatedAt = src.LifecycleConfigUpdatedAt
case bucketSSEConfig:
dst.EncryptionConfigXML = bytes.Clone(src.EncryptionConfigXML)
dst.EncryptionConfigUpdatedAt = src.EncryptionConfigUpdatedAt
case bucketTaggingConfig:
dst.TaggingConfigXML = bytes.Clone(src.TaggingConfigXML)
dst.TaggingConfigUpdatedAt = src.TaggingConfigUpdatedAt
case bucketQuotaConfigFile:
dst.QuotaConfigJSON = bytes.Clone(src.QuotaConfigJSON)
dst.QuotaConfigUpdatedAt = src.QuotaConfigUpdatedAt
case bucketCorsConfig:
// The import stamps its fields before creating any missing bucket,
// and a CORS event stamped before bucket creation is discarded as
// belonging to an older incarnation, so the imported document takes
// the same monotonic timestamp a local PutBucketCors would assign.
dst.CorsConfigUpdatedAt = localCORSUpdatedAt(*dst, src.CorsConfigUpdatedAt)
dst.CorsConfigXML = bytes.Clone(src.CorsConfigXML)
case objectLockConfig:
dst.ObjectLockConfigXML = bytes.Clone(src.ObjectLockConfigXML)
dst.ObjectLockConfigUpdatedAt = src.ObjectLockConfigUpdatedAt
case bucketVersioningConfig:
dst.VersioningConfigXML = bytes.Clone(src.VersioningConfigXML)
dst.VersioningConfigUpdatedAt = src.VersioningConfigUpdatedAt
}
}
}
func (i *importMetaReport) SetStatus(bucket, fname string, err error) {
st := i.Buckets[bucket]
var errMsg string
@@ -608,6 +663,8 @@ func (i *importMetaReport) SetStatus(bucket, fname string, err error) {
st.Tagging = madmin.MetaStatus{IsSet: true, Err: errMsg}
case bucketQuotaConfigFile:
st.Quota = madmin.MetaStatus{IsSet: true, Err: errMsg}
case bucketCorsConfig:
st.Cors = madmin.MetaStatus{IsSet: true, Err: errMsg}
case objectLockConfig:
st.ObjectLock = madmin.MetaStatus{IsSet: true, Err: errMsg}
case bucketVersioningConfig:
@@ -649,6 +706,16 @@ func (a adminAPIHandlers) ImportBucketMetadataHandler(w http.ResponseWriter, r *
}
bucketMap := make(map[string]*BucketMetadata, len(zr.File))
importedFields := make(map[string]importMetadataFields, len(zr.File))
blockedBuckets := make(map[string]struct{})
markImported := func(bucket, configFile string) {
fields := importedFields[bucket]
if fields == nil {
fields = make(importMetadataFields)
importedFields[bucket] = fields
}
fields.add(configFile)
}
updatedAt := UTCNow()
@@ -664,6 +731,7 @@ func (a adminAPIHandlers) ImportBucketMetadataHandler(w http.ResponseWriter, r *
bucketMap[bucket] = &meta
} else if err != errConfigNotFound {
rpt.SetStatus(bucket, "", err)
blockedBuckets[bucket] = struct{}{}
}
}
@@ -675,6 +743,9 @@ func (a adminAPIHandlers) ImportBucketMetadataHandler(w http.ResponseWriter, r *
continue
}
bucket, fileName := slc[0], slc[1]
if _, blocked := blockedBuckets[bucket]; blocked {
continue
}
if fileName == objectLockConfig {
reader, err := file.Open()
if err != nil {
@@ -708,6 +779,7 @@ func (a adminAPIHandlers) ImportBucketMetadataHandler(w http.ResponseWriter, r *
bucketMap[bucket].ObjectLockConfigXML = configData
bucketMap[bucket].ObjectLockConfigUpdatedAt = updatedAt
markImported(bucket, fileName)
rpt.SetStatus(bucket, fileName, nil)
}
}
@@ -720,6 +792,9 @@ func (a adminAPIHandlers) ImportBucketMetadataHandler(w http.ResponseWriter, r *
continue
}
bucket, fileName := slc[0], slc[1]
if _, blocked := blockedBuckets[bucket]; blocked {
continue
}
if fileName == bucketVersioningConfig {
reader, err := file.Open()
if err != nil {
@@ -764,6 +839,7 @@ func (a adminAPIHandlers) ImportBucketMetadataHandler(w http.ResponseWriter, r *
bucketMap[bucket].VersioningConfigXML = configData
bucketMap[bucket].VersioningConfigUpdatedAt = updatedAt
markImported(bucket, fileName)
rpt.SetStatus(bucket, fileName, nil)
}
}
@@ -781,6 +857,9 @@ func (a adminAPIHandlers) ImportBucketMetadataHandler(w http.ResponseWriter, r *
continue
}
bucket, fileName := slc[0], slc[1]
if _, blocked := blockedBuckets[bucket]; blocked {
continue
}
// create bucket if it does not exist yet.
if _, ok := bucketMap[bucket]; !ok {
@@ -813,6 +892,7 @@ func (a adminAPIHandlers) ImportBucketMetadataHandler(w http.ResponseWriter, r *
bucketMap[bucket].NotificationConfigXML = configData
bucketMap[bucket].NotificationConfigUpdatedAt = updatedAt
markImported(bucket, fileName)
rpt.SetStatus(bucket, fileName, nil)
case bucketPolicyConfig:
// Error out if Content-Length is beyond allowed size.
@@ -839,7 +919,7 @@ func (a adminAPIHandlers) ImportBucketMetadataHandler(w http.ResponseWriter, r *
continue
}
configData, err := json.Marshal(bucketPolicy)
configData, err := canonicalBucketPolicy(bucketPolicy)
if err != nil {
rpt.SetStatus(bucket, fileName, err)
continue
@@ -847,6 +927,7 @@ func (a adminAPIHandlers) ImportBucketMetadataHandler(w http.ResponseWriter, r *
bucketMap[bucket].PolicyConfigJSON = configData
bucketMap[bucket].PolicyConfigUpdatedAt = updatedAt
markImported(bucket, fileName)
rpt.SetStatus(bucket, fileName, nil)
case bucketLifecycleConfig:
bucketLifecycle, err := lifecycle.ParseLifecycleConfig(io.LimitReader(reader, sz))
@@ -879,6 +960,7 @@ func (a adminAPIHandlers) ImportBucketMetadataHandler(w http.ResponseWriter, r *
bucketMap[bucket].LifecycleConfigXML = configData
bucketMap[bucket].LifecycleConfigUpdatedAt = updatedAt
markImported(bucket, fileName)
rpt.SetStatus(bucket, fileName, nil)
case bucketSSEConfig:
// Parse bucket encryption xml
@@ -917,6 +999,7 @@ func (a adminAPIHandlers) ImportBucketMetadataHandler(w http.ResponseWriter, r *
bucketMap[bucket].EncryptionConfigXML = configData
bucketMap[bucket].EncryptionConfigUpdatedAt = updatedAt
markImported(bucket, fileName)
rpt.SetStatus(bucket, fileName, nil)
case bucketTaggingConfig:
tags, err := tags.ParseBucketXML(io.LimitReader(reader, sz))
@@ -933,6 +1016,7 @@ func (a adminAPIHandlers) ImportBucketMetadataHandler(w http.ResponseWriter, r *
bucketMap[bucket].TaggingConfigXML = configData
bucketMap[bucket].TaggingConfigUpdatedAt = updatedAt
markImported(bucket, fileName)
rpt.SetStatus(bucket, fileName, nil)
case bucketQuotaConfigFile:
data, err := io.ReadAll(reader)
@@ -949,6 +1033,33 @@ func (a adminAPIHandlers) ImportBucketMetadataHandler(w http.ResponseWriter, r *
bucketMap[bucket].QuotaConfigJSON = data
bucketMap[bucket].QuotaConfigUpdatedAt = updatedAt
markImported(bucket, fileName)
rpt.SetStatus(bucket, fileName, nil)
case bucketCorsConfig:
if sz > maxBucketCorsSize {
rpt.SetStatus(bucket, fileName, errors.New(ErrEntityTooLarge.String()))
continue
}
// Read one byte past the declared size: stopping exactly at sz
// leaves archive/zip short of EOF, so it never verifies the entry
// checksum and a corrupt entry carrying well formed XML would be
// stored as a valid document. The extra byte also lets the reader
// reject an entry longer than it declares.
corsData, err := io.ReadAll(io.LimitReader(reader, sz+1))
if err != nil {
rpt.SetStatus(bucket, fileName, err)
continue
}
if err = validateCORSReplicationPayload(corsData); err != nil {
rpt.SetStatus(bucket, fileName, fmt.Errorf("%s (%s)", errorCodes[ErrMalformedXML].Description, err))
continue
}
bucketMap[bucket].CorsConfigXML = corsData
bucketMap[bucket].CorsConfigUpdatedAt = updatedAt
markImported(bucket, fileName)
rpt.SetStatus(bucket, fileName, nil)
}
}
@@ -962,22 +1073,97 @@ func (a adminAPIHandlers) ImportBucketMetadataHandler(w http.ResponseWriter, r *
}
for bucket, meta := range bucketMap {
err := globalBucketMetadataSys.save(ctx, *meta)
fields := importedFields[bucket]
if len(fields) == 0 {
continue
}
var merged BucketMetadata
var commitAt time.Time
err := func() error {
lockCtx, unlock, err := lockBucketMetadata(ctx, objectAPI, bucket)
if err != nil {
return err
}
defer unlock()
merged, err = loadBucketMetadataParse(lockCtx, objectAPI, bucket, false)
if err != nil {
return err
}
if err := ensureBucketMetadataCreated(lockCtx, objectAPI, &merged); err != nil {
return err
}
commitAt = UTCNow()
for _, file := range replicatedBucketConfigs {
if _, ok := fields[file]; ok {
commitAt = localBucketConfigUpdatedAt(merged, file, commitAt)
}
}
applyImportedBucketMetadata(&merged, *meta, fields)
for _, file := range replicatedBucketConfigs {
if _, ok := fields[file]; !ok {
continue
}
data, at := replicatedBucketConfig(&merged, file)
payload, _, err := bucketConfigPayload(bucket, file, *data, len(merged.ObjectLockConfigXML) != 0)
if err != nil {
return err
}
*data, *at = payload, commitAt
}
return globalBucketMetadataSys.saveMetadata(lockCtx, objectAPI, &merged)
}()
if err != nil {
rpt.SetStatus(bucket, "", err)
continue
}
// Call site replication hook.
if err = globalSiteReplicationSys.BucketMetaHook(ctx, madmin.SRBucketMeta{
Bucket: bucket,
Quota: meta.QuotaConfigJSON,
Policy: meta.PolicyConfigJSON,
Versioning: enc(meta.VersioningConfigXML),
Tags: enc(meta.TaggingConfigXML),
ObjectLockConfig: enc(meta.ObjectLockConfigXML),
SSEConfig: enc(meta.EncryptionConfigXML),
UpdatedAt: updatedAt,
}); err != nil {
*meta = merged
globalNotificationSys.LoadBucketMetadata(bgContext(ctx), bucket)
hook := madmin.SRBucketMeta{Bucket: bucket, UpdatedAt: commitAt}
var hookNeeded bool
if _, ok := fields[bucketQuotaConfigFile]; ok {
hook.Quota = meta.QuotaConfigJSON
hookNeeded = true
}
if _, ok := fields[bucketPolicyConfig]; ok {
hook.Policy = meta.PolicyConfigJSON
hookNeeded = hookNeeded || len(hook.Policy) != 0
}
if _, ok := fields[bucketVersioningConfig]; ok {
hook.Versioning = enc(meta.VersioningConfigXML)
hookNeeded = true
}
if _, ok := fields[bucketTaggingConfig]; ok {
hook.Tags = enc(meta.TaggingConfigXML)
hookNeeded = true
}
if _, ok := fields[objectLockConfig]; ok {
hook.ObjectLockConfig = enc(meta.ObjectLockConfigXML)
hookNeeded = true
}
if _, ok := fields[bucketSSEConfig]; ok {
hook.SSEConfig = enc(meta.EncryptionConfigXML)
hookNeeded = true
}
if hookNeeded {
err = globalSiteReplicationSys.BucketMetaHook(ctx, hook)
}
if _, ok := fields[bucketPolicyConfig]; ok && len(meta.PolicyConfigJSON) == 0 {
// An omitted bulk Policy cannot express deletion.
err = errors.Join(err, globalSiteReplicationSys.BucketMetaHook(ctx, madmin.SRBucketMeta{
Type: madmin.SRBucketMetaTypePolicy, Bucket: bucket, UpdatedAt: commitAt,
}))
}
if _, ok := fields[bucketCorsConfig]; ok {
// CORS carries its own timestamp, so it replicates through the
// dedicated event rather than the shared bucket metadata hook. It
// is announced even when the shared hook failed: the document is
// already committed locally, and a peer that is unreachable for
// one config must not withhold CORS from the reachable ones.
if corsEvent, live := newBucketCORSReplicationEvent(bucket, *meta); live {
err = errors.Join(err, globalSiteReplicationSys.BucketMetaHook(ctx, corsEvent))
}
}
if err != nil {
rpt.SetStatus(bucket, "", err)
continue
}
+1 -1
View File
@@ -27,7 +27,7 @@ import (
"github.com/minio/madmin-go/v3"
"github.com/minio/minio/internal/auth"
"github.com/minio/minio/internal/config"
"github.com/minio/pkg/v3/policy"
"github.com/pgsty/silo-pkg/v3/policy"
)
// validateAdminReq will validate request against and return whether it is allowed.
+1 -1
View File
@@ -37,7 +37,7 @@ import (
"github.com/minio/minio/internal/config/subnet"
"github.com/minio/minio/internal/logger"
"github.com/minio/mux"
"github.com/minio/pkg/v3/policy"
"github.com/pgsty/silo-pkg/v3/policy"
)
// DelConfigKVHandler - DELETE /minio/admin/v3/del-config-kv
+2 -2
View File
@@ -32,8 +32,8 @@ import (
cfgldap "github.com/minio/minio/internal/config/identity/ldap"
"github.com/minio/minio/internal/config/identity/openid"
"github.com/minio/mux"
"github.com/minio/pkg/v3/ldap"
"github.com/minio/pkg/v3/policy"
"github.com/pgsty/silo-pkg/v3/ldap"
"github.com/pgsty/silo-pkg/v3/policy"
)
func addOrUpdateIDPHandler(ctx context.Context, w http.ResponseWriter, r *http.Request, isUpdate bool) {
+2 -2
View File
@@ -28,8 +28,8 @@ import (
"github.com/minio/madmin-go/v3"
"github.com/minio/minio/internal/auth"
"github.com/minio/mux"
xldap "github.com/minio/pkg/v3/ldap"
"github.com/minio/pkg/v3/policy"
xldap "github.com/pgsty/silo-pkg/v3/ldap"
"github.com/pgsty/silo-pkg/v3/policy"
)
// ListLDAPPolicyMappingEntities lists users/groups mapped to given/all policies.
+1 -1
View File
@@ -25,7 +25,7 @@ import (
"github.com/minio/madmin-go/v3"
"github.com/minio/minio-go/v7/pkg/set"
"github.com/minio/pkg/v3/policy"
"github.com/pgsty/silo-pkg/v3/policy"
)
const dummyRoleARN = "dummy-internal"
+2 -2
View File
@@ -27,8 +27,8 @@ import (
"strings"
"github.com/minio/mux"
"github.com/minio/pkg/v3/env"
"github.com/minio/pkg/v3/policy"
"github.com/pgsty/silo-pkg/v3/env"
"github.com/pgsty/silo-pkg/v3/policy"
)
var (
+4 -2
View File
@@ -33,7 +33,7 @@ import (
"github.com/minio/madmin-go/v3"
xioutil "github.com/minio/minio/internal/ioutil"
"github.com/minio/mux"
"github.com/minio/pkg/v3/policy"
"github.com/pgsty/silo-pkg/v3/policy"
)
// SiteReplicationAdd - PUT /minio/admin/v3/site-replication/add
@@ -255,9 +255,11 @@ func (a adminAPIHandlers) SRPeerReplicateBucketItem(w http.ResponseWriter, r *ht
case madmin.SRBucketMetaTypeTags:
err = globalSiteReplicationSys.PeerBucketTaggingHandler(ctx, item.Bucket, item.Tags, item.UpdatedAt)
case madmin.SRBucketMetaTypeObjectLockConfig:
err = globalSiteReplicationSys.PeerBucketObjectLockConfigHandler(ctx, item.Bucket, item.ObjectLockConfig, item.UpdatedAt)
err = globalSiteReplicationSys.peerBucketObjectLockConfigItem(ctx, item)
case madmin.SRBucketMetaTypeSSEConfig:
err = globalSiteReplicationSys.PeerBucketSSEConfigHandler(ctx, item.Bucket, item.SSEConfig, item.UpdatedAt)
case madmin.SRBucketMetaTypeCorsConfig:
err = globalSiteReplicationSys.PeerBucketCorsConfigHandler(ctx, item.Bucket, item.Cors, item.UpdatedAt)
case madmin.SRBucketMetaLCConfig:
err = globalSiteReplicationSys.PeerBucketLCConfigHandler(ctx, item.Bucket, item.ExpiryLCConfig, item.UpdatedAt)
}
+1 -1
View File
@@ -32,7 +32,7 @@ import (
"github.com/minio/madmin-go/v3"
minio "github.com/minio/minio-go/v7"
"github.com/minio/pkg/v3/sync/errgroup"
"github.com/pgsty/silo-pkg/v3/sync/errgroup"
)
func runAllIAMConcurrencyTests(suite *TestSuiteIAM, c *check) {
+34 -16
View File
@@ -40,8 +40,8 @@ import (
"github.com/minio/minio/internal/config/dns"
"github.com/minio/minio/internal/logger"
"github.com/minio/mux"
xldap "github.com/minio/pkg/v3/ldap"
"github.com/minio/pkg/v3/policy"
xldap "github.com/pgsty/silo-pkg/v3/ldap"
"github.com/pgsty/silo-pkg/v3/policy"
"github.com/puzpuzpuz/xsync/v3"
)
@@ -355,18 +355,25 @@ func (a adminAPIHandlers) ListGroups(w http.ResponseWriter, r *http.Request) {
}
// SetGroupStatus - PUT /minio/admin/v3/set-group-status?group=mygroup1&status=enabled
func setGroupStatusAdminAction(status string) policy.AdminAction {
if madmin.GroupStatus(status) == madmin.GroupDisabled {
return policy.DisableGroupAdminAction
}
return policy.EnableGroupAdminAction
}
func (a adminAPIHandlers) SetGroupStatus(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
objectAPI, _ := validateAdminReq(ctx, w, r, policy.EnableGroupAdminAction)
if objectAPI == nil {
return
}
vars := mux.Vars(r)
group := vars["group"]
status := vars["status"]
objectAPI, _ := validateAdminReq(ctx, w, r, setGroupStatusAdminAction(status))
if objectAPI == nil {
return
}
var (
err error
updatedAt time.Time
@@ -398,18 +405,25 @@ func (a adminAPIHandlers) SetGroupStatus(w http.ResponseWriter, r *http.Request)
}
// SetUserStatus - PUT /minio/admin/v3/set-user-status?accessKey=<access_key>&status=[enabled|disabled]
func setUserStatusAdminAction(status string) policy.AdminAction {
if madmin.AccountStatus(status) == madmin.AccountDisabled {
return policy.DisableUserAdminAction
}
return policy.EnableUserAdminAction
}
func (a adminAPIHandlers) SetUserStatus(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
objectAPI, creds := validateAdminReq(ctx, w, r, policy.EnableUserAdminAction)
if objectAPI == nil {
return
}
vars := mux.Vars(r)
accessKey := vars["accessKey"]
status := vars["status"]
objectAPI, creds := validateAdminReq(ctx, w, r, setUserStatusAdminAction(status))
if objectAPI == nil {
return
}
// you cannot enable or disable yourself.
if accessKey == creds.AccessKey {
writeErrorResponseJSON(ctx, w, toAdminAPIErr(ctx, errInvalidArgument), r.URL)
@@ -488,11 +502,15 @@ func (a adminAPIHandlers) AddUser(w http.ResponseWriter, r *http.Request) {
}
checkDenyOnly := accessKey == cred.AccessKey
action := policy.Action(policy.CreateUserAdminAction)
if checkDenyOnly {
action = policy.ChangeMyPasswordAdminAction
}
if !globalIAMSys.IsAllowed(policy.Args{
AccountName: cred.AccessKey,
Groups: cred.Groups,
Action: policy.CreateUserAdminAction,
Action: action,
ConditionValues: getConditionValues(r, "", cred),
IsOwner: owner,
Claims: cred.Claims,
@@ -859,7 +877,7 @@ func (a adminAPIHandlers) UpdateServiceAccount(w http.ResponseWriter, r *http.Re
var sp *policy.Policy
if len(updateReq.NewPolicy) > 0 {
sp, err = policy.ParseConfig(bytes.NewReader(updateReq.NewPolicy))
sp, err = policy.ParseConfigStrict(bytes.NewReader(updateReq.NewPolicy))
if err != nil {
writeErrorResponseJSON(ctx, w, toAdminAPIErr(ctx, err), r.URL)
return
@@ -1729,7 +1747,7 @@ func (a adminAPIHandlers) AddCannedPolicy(w http.ResponseWriter, r *http.Request
return
}
iamPolicy, err := policy.ParseConfig(bytes.NewReader(iamPolicyBytes))
iamPolicy, err := policy.ParseConfigStrict(bytes.NewReader(iamPolicyBytes))
if err != nil {
writeErrorResponseJSON(ctx, w, toAdminAPIErr(ctx, err), r.URL)
return
@@ -2981,7 +2999,7 @@ func commonAddServiceAccount(r *http.Request, ldap bool) (context.Context, auth.
var sp *policy.Policy
if len(createReq.Policy) > 0 {
sp, err = policy.ParseConfig(bytes.NewReader(createReq.Policy))
sp, err = policy.ParseConfigStrict(bytes.NewReader(createReq.Policy))
if err != nil {
return ctx, auth.Credentials{}, newServiceAccountOpts{}, madmin.AddServiceAccountReq{}, "", toAdminAPIErr(ctx, err)
}
+384 -1
View File
@@ -40,13 +40,54 @@ import (
"github.com/minio/minio-go/v7/pkg/set"
"github.com/minio/minio-go/v7/pkg/signer"
"github.com/minio/minio/internal/auth"
"github.com/minio/pkg/v3/env"
"github.com/pgsty/silo-pkg/v3/env"
"github.com/pgsty/silo-pkg/v3/policy"
)
const (
testDefaultTimeout = 30 * time.Second
)
func TestSetUserStatusAdminAction(t *testing.T) {
tests := []struct {
name string
status string
want policy.AdminAction
}{
{name: "enable", status: string(madmin.AccountEnabled), want: policy.EnableUserAdminAction},
{name: "disable", status: string(madmin.AccountDisabled), want: policy.DisableUserAdminAction},
{name: "invalid preserves authenticated default", status: "invalid", want: policy.EnableUserAdminAction},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := setUserStatusAdminAction(tt.status); got != tt.want {
t.Fatalf("setUserStatusAdminAction(%q) = %q, want %q", tt.status, got, tt.want)
}
})
}
}
func TestSetGroupStatusAdminAction(t *testing.T) {
tests := []struct {
name string
status string
want policy.AdminAction
}{
{name: "enable", status: string(madmin.GroupEnabled), want: policy.EnableGroupAdminAction},
{name: "disable", status: string(madmin.GroupDisabled), want: policy.DisableGroupAdminAction},
{name: "invalid preserves authenticated default", status: "invalid", want: policy.EnableGroupAdminAction},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := setGroupStatusAdminAction(tt.status); got != tt.want {
t.Fatalf("setGroupStatusAdminAction(%q) = %q, want %q", tt.status, got, tt.want)
}
})
}
}
// API suite container for IAM
type TestSuiteIAM struct {
TestSuiteCommon
@@ -202,8 +243,12 @@ func TestIAMInternalIDPServerSuite(t *testing.T) {
suite.SetUpSuite(c)
suite.TestUserCreate(c)
suite.TestUserPasswordActionAuthorization(c)
suite.TestUserStatusActionAuthorization(c)
suite.TestGroupStatusActionAuthorization(c)
suite.TestUserPolicyEscalationBug(c)
suite.TestPolicyCreate(c)
suite.TestServiceAccountBareARNPolicyRejected(c)
suite.TestCannedPolicies(c)
suite.TestGroupAddRemove(c)
suite.TestServiceAccountOpsByAdmin(c)
@@ -312,6 +357,284 @@ func (s *TestSuiteIAM) TestUserCreate(c *check) {
}
}
func (s *TestSuiteIAM) TestUserPasswordActionAuthorization(c *check) {
for _, tt := range []struct {
name string
statements string
self bool
other bool
}{
{"readonly", "", true, false},
{"consolereadonly", "", true, false},
{"password grant", `{"Effect":"Allow","Action":"admin:ChangeMyPassword"}`, true, false},
{"legacy CreateUser deny", `{"Effect":"Deny","Action":"admin:CreateUser","Resource":"arn:aws:s3:::*"}`, true, false},
{"password deny", `{"Effect":"Deny","Action":"admin:ChangeMyPassword"}`, false, false},
{"user admin", `{"Effect":"Allow","Action":"admin:CreateUser"}`, true, true},
{"user admin with password deny", `{"Effect":"Allow","Action":"admin:CreateUser"},{"Effect":"Deny","Action":"admin:ChangeMyPassword"}`, false, true},
{"password deny overrides grant", `{"Effect":"Allow","Action":"admin:ChangeMyPassword"},{"Effect":"Deny","Action":"admin:ChangeMyPassword"}`, false, false},
{"wildcard deny", `{"Effect":"Deny","Action":"admin:*"}`, false, false},
} {
c.Run(tt.name, func(t *testing.T) {
c := &check{t, s.serverType}
ctx, cancel := context.WithTimeout(context.Background(), testDefaultTimeout)
defer cancel()
var users []string
policyName := tt.name
defer func() {
for _, user := range users {
if err := s.adm.RemoveUser(ctx, user); err != nil {
c.Errorf("remove test user: %v", err)
}
}
if tt.statements != "" {
if err := s.adm.RemoveCannedPolicy(ctx, policyName); err != nil {
c.Errorf("remove test policy: %v", err)
}
}
}()
createUser := func() (string, string) {
accessKey, secretKey := mustGenerateCredentials(c)
if err := s.adm.SetUser(ctx, accessKey, secretKey, madmin.AccountEnabled); err != nil {
c.Fatalf("create test user: %v", err)
}
users = append(users, accessKey)
return accessKey, secretKey
}
client := func(accessKey, secretKey string) *madmin.AdminClient {
adm, err := madmin.New(s.endpoint, accessKey, secretKey, s.secure)
if err != nil {
c.Fatal(err)
}
adm.SetCustomTransport(s.TestSuiteCommon.client.Transport)
return adm
}
if tt.statements != "" {
policyName = getRandomBucketName()
doc := []byte(`{"Version":"2012-10-17","Statement":[` + tt.statements + `]}`)
if err := s.adm.AddCannedPolicy(ctx, policyName, doc); err != nil {
c.Fatalf("save test policy: %v", err)
}
}
accessKey, secretKey := createUser()
if _, err := s.adm.AttachPolicy(ctx, madmin.PolicyAssociationReq{
User: accessKey, Policies: []string{policyName},
}); err != nil {
c.Fatalf("attach test policy: %v", err)
}
adm := client(accessKey, secretKey)
_, newSecretKey := mustGenerateCredentials(c)
err := adm.SetUser(ctx, accessKey, newSecretKey, madmin.AccountEnabled)
if tt.self {
if err != nil {
c.Fatalf("change own password: %v", err)
}
if _, err = adm.AccountInfo(ctx, madmin.AccountOpts{}); err == nil {
c.Fatal("old password still authenticates")
}
adm = client(accessKey, newSecretKey)
} else if err == nil || madmin.ToErrorResponse(err).Code != "AccessDenied" {
c.Fatalf("self password change: expected AccessDenied, got %v", err)
}
if _, err := adm.AccountInfo(ctx, madmin.AccountOpts{}); err != nil {
c.Fatalf("current password no longer authenticates: %v", err)
}
target, _ := createUser()
newUser, newUserSecret := mustGenerateCredentials(c)
for _, key := range []string{target, newUser} {
err := adm.SetUser(ctx, key, newUserSecret, madmin.AccountEnabled)
if tt.other {
if err != nil {
c.Fatalf("create or update another user: %v", err)
}
if key == newUser {
users = append(users, newUser)
}
} else if err == nil || madmin.ToErrorResponse(err).Code != "AccessDenied" {
c.Fatalf("create or update another user: expected AccessDenied, got %v", err)
}
}
})
}
}
func (s *TestSuiteIAM) TestUserStatusActionAuthorization(c *check) {
ctx, cancel := context.WithTimeout(context.Background(), testDefaultTimeout)
defer cancel()
var createdUsers []string
var createdPolicies []string
defer func() {
for _, user := range createdUsers {
if err := s.adm.RemoveUser(ctx, user); err != nil {
c.Errorf("unable to remove test user %s: %v", user, err)
}
}
for _, policyName := range createdPolicies {
if err := s.adm.RemoveCannedPolicy(ctx, policyName); err != nil {
c.Errorf("unable to remove test policy %s: %v", policyName, err)
}
}
}()
createUser := func() (string, string) {
accessKey, secretKey := mustGenerateCredentials(c)
if err := s.adm.SetUser(ctx, accessKey, secretKey, madmin.AccountEnabled); err != nil {
c.Fatalf("unable to create test user: %v", err)
}
createdUsers = append(createdUsers, accessKey)
return accessKey, secretKey
}
createStatusClient := func(action policy.AdminAction) *madmin.AdminClient {
accessKey, secretKey := createUser()
policyName := getRandomBucketName()
policyBytes := fmt.Appendf(nil, `{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": ["%s"]
}]
}`, action)
if err := s.adm.AddCannedPolicy(ctx, policyName, policyBytes); err != nil {
c.Fatalf("unable to add status policy: %v", err)
}
createdPolicies = append(createdPolicies, policyName)
if _, err := s.adm.AttachPolicy(ctx, madmin.PolicyAssociationReq{
Policies: []string{policyName},
User: accessKey,
}); err != nil {
c.Fatalf("unable to attach status policy: %v", err)
}
client, err := madmin.NewWithOptions(s.endpoint, &madmin.Options{
Creds: credentials.NewStaticV4(accessKey, secretKey, ""),
Secure: s.secure,
})
if err != nil {
c.Fatalf("unable to create status admin client: %v", err)
}
client.SetCustomTransport(s.TestSuiteCommon.client.Transport)
return client
}
targetAccessKey, _ := createUser()
disableClient := createStatusClient(policy.DisableUserAdminAction)
if err := disableClient.SetUserStatus(ctx, targetAccessKey, madmin.AccountDisabled); err != nil {
c.Fatalf("DisableUser-only client could not disable a user: %v", err)
}
if err := disableClient.SetUserStatus(ctx, targetAccessKey, madmin.AccountEnabled); err == nil || madmin.ToErrorResponse(err).Code != "AccessDenied" {
c.Fatalf("DisableUser-only client unexpectedly enabled a user: %v", err)
}
enableClient := createStatusClient(policy.EnableUserAdminAction)
if err := enableClient.SetUserStatus(ctx, targetAccessKey, madmin.AccountEnabled); err != nil {
c.Fatalf("EnableUser-only client could not enable a user: %v", err)
}
if err := enableClient.SetUserStatus(ctx, targetAccessKey, madmin.AccountDisabled); err == nil || madmin.ToErrorResponse(err).Code != "AccessDenied" {
c.Fatalf("EnableUser-only client unexpectedly disabled a user: %v", err)
}
}
func (s *TestSuiteIAM) TestGroupStatusActionAuthorization(c *check) {
ctx, cancel := context.WithTimeout(context.Background(), testDefaultTimeout)
defer cancel()
var createdUsers []string
var createdPolicies []string
group := getRandomBucketName()
var groupCreated bool
defer func() {
if groupCreated {
if err := s.adm.UpdateGroupMembers(ctx, madmin.GroupAddRemove{
Group: group,
Members: createdUsers[:1],
IsRemove: true,
}); err != nil {
c.Errorf("unable to remove group member: %v", err)
}
if err := s.adm.UpdateGroupMembers(ctx, madmin.GroupAddRemove{Group: group, IsRemove: true}); err != nil {
c.Errorf("unable to remove test group: %v", err)
}
}
for _, user := range createdUsers {
if err := s.adm.RemoveUser(ctx, user); err != nil {
c.Errorf("unable to remove test user %s: %v", user, err)
}
}
for _, policyName := range createdPolicies {
if err := s.adm.RemoveCannedPolicy(ctx, policyName); err != nil {
c.Errorf("unable to remove test policy %s: %v", policyName, err)
}
}
}()
createUser := func() (string, string) {
accessKey, secretKey := mustGenerateCredentials(c)
if err := s.adm.SetUser(ctx, accessKey, secretKey, madmin.AccountEnabled); err != nil {
c.Fatalf("unable to create test user: %v", err)
}
createdUsers = append(createdUsers, accessKey)
return accessKey, secretKey
}
targetAccessKey, _ := createUser()
if err := s.adm.UpdateGroupMembers(ctx, madmin.GroupAddRemove{
Group: group,
Members: []string{targetAccessKey},
}); err != nil {
c.Fatalf("unable to create test group: %v", err)
}
groupCreated = true
createStatusClient := func(action policy.AdminAction) *madmin.AdminClient {
accessKey, secretKey := createUser()
policyName := getRandomBucketName()
policyBytes := fmt.Appendf(nil, `{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": ["%s"]
}]
}`, action)
if err := s.adm.AddCannedPolicy(ctx, policyName, policyBytes); err != nil {
c.Fatalf("unable to add group status policy: %v", err)
}
createdPolicies = append(createdPolicies, policyName)
if _, err := s.adm.AttachPolicy(ctx, madmin.PolicyAssociationReq{
Policies: []string{policyName},
User: accessKey,
}); err != nil {
c.Fatalf("unable to attach group status policy: %v", err)
}
client, err := madmin.NewWithOptions(s.endpoint, &madmin.Options{
Creds: credentials.NewStaticV4(accessKey, secretKey, ""),
Secure: s.secure,
})
if err != nil {
c.Fatalf("unable to create group status admin client: %v", err)
}
client.SetCustomTransport(s.TestSuiteCommon.client.Transport)
return client
}
disableClient := createStatusClient(policy.DisableGroupAdminAction)
if err := disableClient.SetGroupStatus(ctx, group, madmin.GroupDisabled); err != nil {
c.Fatalf("DisableGroup-only client could not disable a group: %v", err)
}
if err := disableClient.SetGroupStatus(ctx, group, madmin.GroupEnabled); err == nil || madmin.ToErrorResponse(err).Code != "AccessDenied" {
c.Fatalf("DisableGroup-only client unexpectedly enabled a group: %v", err)
}
enableClient := createStatusClient(policy.EnableGroupAdminAction)
if err := enableClient.SetGroupStatus(ctx, group, madmin.GroupEnabled); err != nil {
c.Fatalf("EnableGroup-only client could not enable a group: %v", err)
}
if err := enableClient.SetGroupStatus(ctx, group, madmin.GroupDisabled); err == nil || madmin.ToErrorResponse(err).Code != "AccessDenied" {
c.Fatalf("EnableGroup-only client unexpectedly disabled a group: %v", err)
}
}
func (s *TestSuiteIAM) TestUserPolicyEscalationBug(c *check) {
ctx, cancel := context.WithTimeout(context.Background(), testDefaultTimeout)
defer cancel()
@@ -600,6 +923,20 @@ func (s *TestSuiteIAM) TestPolicyCreate(c *check) {
c.Fatalf("invalid policy creation success")
}
for i, resource := range []string{"arn:aws:s3:::", "*arn:aws:s3:::"} {
barePolicyBytes := fmt.Appendf(nil, `{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Deny",
"Action": ["s3:GetObject"],
"Resource": ["%s"]
}]
}`, resource)
if err = s.adm.AddCannedPolicy(ctx, fmt.Sprintf("%s-bare-%d", policy, i), barePolicyBytes); err == nil {
c.Fatalf("bare ARN policy creation succeeded for %q", resource)
}
}
// 3. Create a user, associate policy and verify access
accessKey, secretKey := mustGenerateCredentials(c)
err = s.adm.SetUser(ctx, accessKey, secretKey, madmin.AccountEnabled)
@@ -653,6 +990,51 @@ func (s *TestSuiteIAM) TestPolicyCreate(c *check) {
}
}
func (s *TestSuiteIAM) TestServiceAccountBareARNPolicyRejected(c *check) {
ctx, cancel := context.WithTimeout(context.Background(), testDefaultTimeout)
defer cancel()
barePolicy := []byte(`{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": ["s3:GetObject"],
"NotResource": ["arn:aws:s3:::"]
}]
}`)
if _, err := s.adm.AddServiceAccount(ctx, madmin.AddServiceAccountReq{
TargetUser: globalActiveCred.AccessKey,
Policy: barePolicy,
}); err == nil {
c.Fatal("service account creation accepted a bare ARN policy")
}
validPolicy := []byte(`{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": ["s3:GetObject"],
"Resource": ["arn:aws:s3:::*"]
}]
}`)
credentials, err := s.adm.AddServiceAccount(ctx, madmin.AddServiceAccountReq{
TargetUser: globalActiveCred.AccessKey,
Policy: validPolicy,
})
if err != nil {
c.Fatalf("service account creation rejected an explicit resource: %v", err)
}
defer func() {
_ = s.adm.DeleteServiceAccount(ctx, credentials.AccessKey)
}()
if err = s.adm.UpdateServiceAccount(ctx, credentials.AccessKey, madmin.UpdateServiceAccountReq{
NewPolicy: barePolicy,
}); err == nil {
c.Fatal("service account update accepted a bare ARN policy")
}
}
func (s *TestSuiteIAM) TestCannedPolicies(c *check) {
ctx, cancel := context.WithTimeout(context.Background(), testDefaultTimeout)
defer cancel()
@@ -665,6 +1047,7 @@ func (s *TestSuiteIAM) TestCannedPolicies(c *check) {
defaultPolicies := []string{
"readwrite",
"readonly",
"consolereadonly",
"writeonly",
"diagnostics",
"consoleAdmin",
+35 -43
View File
@@ -60,8 +60,8 @@ import (
"github.com/minio/minio/internal/kms"
"github.com/minio/minio/internal/logger"
"github.com/minio/mux"
xnet "github.com/minio/pkg/v3/net"
"github.com/minio/pkg/v3/policy"
xnet "github.com/pgsty/silo-pkg/v3/net"
"github.com/pgsty/silo-pkg/v3/policy"
"github.com/secure-io/sio-go"
"github.com/zeebo/xxh3"
)
@@ -84,7 +84,7 @@ const (
// ServerUpdateV2Handler - POST /minio/admin/v3/update?updateURL={updateURL}&type=2
// ----------
// updates all minio servers and restarts them gracefully.
// Retained for Admin API compatibility. Silo always returns MethodNotAllowed.
func (a adminAPIHandlers) ServerUpdateV2Handler(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
@@ -320,7 +320,7 @@ func (a adminAPIHandlers) ServerUpdateV2Handler(w http.ResponseWriter, r *http.R
// ServerUpdateHandler - POST /minio/admin/v3/update?updateURL={updateURL}
// ----------
// updates all minio servers and restarts them gracefully.
// Retained for Admin API compatibility. Silo always returns MethodNotAllowed.
func (a adminAPIHandlers) ServerUpdateHandler(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
@@ -330,7 +330,7 @@ func (a adminAPIHandlers) ServerUpdateHandler(w http.ResponseWriter, r *http.Req
}
if globalInplaceUpdateDisabled || currentReleaseTime.IsZero() {
// if MINIO_UPDATE=off - inplace update is disabled, mostly in containers.
// MINIO_UPDATE is retained, but Silo permanently disables in-place updates.
writeErrorResponseJSON(ctx, w, errorCodes.ToAPIErr(ErrMethodNotAllowed), r.URL)
return
}
@@ -2708,10 +2708,10 @@ func fetchHealthInfo(healthCtx context.Context, objectAPI ObjectLayer, query *ur
}
// Server start command regex groups:
// 1 - minio server
// 2 - flags e.g. `--address :9000 --certs-dir /etc/minio/certs`
// 1 - silo server (or the legacy minio command)
// 2 - flags e.g. `--address :9000 --certs-dir /etc/silo/certs`
// 3 - pool args e.g. `https://node{01...16}.domain/data/disk{001...204} https://node{17...32}.domain/data/disk{001...204}`
re := regexp.MustCompile(`^(.*minio\s+server\s+)(--[^\s]+\s+[^\s]+\s+)*(.*)`)
re := regexp.MustCompile(`^(.*silo\s+server\s+|.*minio\s+server\s+)(--[^\s]+\s+[^\s]+\s+)*(.*)`)
// stays unchanged in the anonymized version
cmdLineWithoutPools := re.ReplaceAllString(cmdLine, `$1$2`)
@@ -3282,28 +3282,7 @@ func (a adminAPIHandlers) InspectDataHandler(w http.ResponseWriter, r *http.Requ
stream := estream.NewWriter(w)
defer stream.Close()
clusterKey, err := bytesToPublicKey(getSubnetAdminPublicKey())
if err != nil {
bugLogIf(ctx, stream.AddError(err.Error()))
return
}
err = stream.AddKeyEncrypted(clusterKey)
if err != nil {
bugLogIf(ctx, stream.AddError(err.Error()))
return
}
if b := getClusterMetaInfo(ctx); len(b) > 0 {
w, err := stream.AddEncryptedStream("cluster.info", nil)
if err != nil {
bugLogIf(ctx, err)
return
}
w.Write(b)
w.Close()
}
// Add new key for inspect data.
if err := stream.AddKeyEncrypted(publicKey); err != nil {
if err := addInspectDataKey(stream, publicKey, getClusterMetaInfo(ctx)); err != nil {
bugLogIf(ctx, stream.AddError(err.Error()))
return
}
@@ -3432,7 +3411,7 @@ func (a adminAPIHandlers) InspectDataHandler(w http.ResponseWriter, r *http.Requ
scheme = "http"
}
// save MinIO start script to inspect command
// Save a Silo start script to inspect command.
var scrb bytes.Buffer
fmt.Fprintf(&scrb, `#!/usr/bin/env bash
@@ -3443,30 +3422,43 @@ function main() {
done
# Read content of inspect-input.txt
MINIO_OPTS=$(grep "Server command line args" <./inspect-input.txt | sed "s/Server command line args: //g" | sed -r "s#%s:\/\/#\.\/#g")
SILO_OPTS=$(grep "Server command line args" <./inspect-input.txt | sed "s/Server command line args: //g" | sed -r "s#%s:\/\/#\.\/#g")
# Start MinIO instance using the options
START_CMD="CI=on _MINIO_AUTO_DRIVE_HEALING=off minio server ${MINIO_OPTS} &"
# Start Silo using the options
START_CMD="CI=on _MINIO_AUTO_DRIVE_HEALING=off silo server ${SILO_OPTS} &"
echo
echo "Starting MinIO instance: ${START_CMD}"
echo "Starting Silo: ${START_CMD}"
echo
eval "$START_CMD"
MINIO_SRVR_PID="$!"
echo "MinIO Server PID: ${MINIO_SRVR_PID}"
SILO_SRVR_PID="$!"
echo "Silo Server PID: ${SILO_SRVR_PID}"
echo
echo "Waiting for MinIO instance to get ready!"
echo "Waiting for Silo to get ready!"
sleep 10
}
main "$@"`, scheme)
adminLogIf(ctx, embedFileInZip(inspectZipW, "start-minio.sh", scrb.Bytes(), 0o755))
adminLogIf(ctx, embedFileInZip(inspectZipW, "start-silo.sh", scrb.Bytes(), 0o755))
}
func getSubnetAdminPublicKey() []byte {
if globalIsCICD {
return subnetAdminPublicKeyDev
// addInspectDataKey makes the requester the only recipient of encrypted
// diagnostic data. Silo has no built-in vendor or support-service recipient.
func addInspectDataKey(stream *estream.Writer, publicKey *rsa.PublicKey, clusterInfo []byte) error {
if err := stream.AddKeyEncrypted(publicKey); err != nil {
return err
}
return subnetAdminPublicKey
if len(clusterInfo) == 0 {
return nil
}
w, err := stream.AddEncryptedStream("cluster.info", nil)
if err != nil {
return err
}
if _, err = w.Write(clusterInfo); err != nil {
_ = w.Close()
return err
}
return w.Close()
}
func createHostAnonymizerForFSMode() map[string]string {
+1 -1
View File
@@ -64,7 +64,7 @@ func prepareAdminErasureTestBed(ctx context.Context) (*adminErasureTestBed, erro
return nil, xlErr
}
// Initialize minio server config.
// Initialize Silo server config.
if err := newTestConfig(globalMinioDefaultRegion, objLayer); err != nil {
cancel()
return nil, err
+2
View File
@@ -163,6 +163,7 @@ func registerAdminRouter(router *mux.Router, enableConfigOps bool) {
// StorageInfo operations
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/storageinfo").HandlerFunc(adminMiddleware(adminAPI.StorageInfoHandler, traceAllFlag))
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/multipart-preflight").HandlerFunc(adminMiddleware(adminAPI.MultipartPreflightHandler, traceAllFlag))
// DataUsageInfo operations
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/datausageinfo").HandlerFunc(adminMiddleware(adminAPI.DataUsageInfoHandler, traceAllFlag))
// Metrics operation
@@ -388,6 +389,7 @@ func registerAdminRouter(router *mux.Router, enableConfigOps bool) {
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/site-replication/peer/join").HandlerFunc(adminMiddleware(adminAPI.SRPeerJoin))
adminRouter.Methods(http.MethodPut).Path(adminVersion+"/site-replication/peer/bucket-ops").HandlerFunc(adminMiddleware(adminAPI.SRPeerBucketOps)).Queries("bucket", "{bucket:.*}").Queries("operation", "{operation:.*}")
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/site-replication/peer/iam-item").HandlerFunc(adminMiddleware(adminAPI.SRPeerReplicateIAMItem))
adminRouter.Methods(http.MethodGet, http.MethodPut).Path(adminVersion + "/site-replication/peer/iam-revisions").HandlerFunc(adminMiddleware(adminAPI.SRPeerIAMRevisions))
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/site-replication/peer/bucket-meta").HandlerFunc(adminMiddleware(adminAPI.SRPeerReplicateBucketItem))
adminRouter.Methods(http.MethodGet).Path(adminVersion + "/site-replication/peer/idp-settings").HandlerFunc(adminMiddleware(adminAPI.SRPeerGetIDPSettings))
adminRouter.Methods(http.MethodPut).Path(adminVersion + "/site-replication/edit").HandlerFunc(adminMiddleware(adminAPI.SiteReplicationEdit))
+1 -1
View File
@@ -30,7 +30,7 @@ import (
"github.com/minio/madmin-go/v3"
"github.com/minio/minio/internal/config"
"github.com/minio/minio/internal/kms"
xnet "github.com/minio/pkg/v3/net"
xnet "github.com/pgsty/silo-pkg/v3/net"
)
// getLocalServerProperty - returns madmin.ServerProperties for only the
+43 -3
View File
@@ -48,7 +48,7 @@ import (
levent "github.com/minio/minio/internal/config/lambda/event"
"github.com/minio/minio/internal/event"
"github.com/minio/minio/internal/hash"
"github.com/minio/pkg/v3/policy"
"github.com/pgsty/silo-pkg/v3/policy"
)
// APIError structure
@@ -450,6 +450,9 @@ const (
ErrAdminNoSecretKey
ErrIAMNotInitialized
ErrMultipartListingLegacy
ErrMultipartListingIdentity
ErrSlowDown
apiErrCodeEnd // This is used only for the testing code
)
@@ -1154,7 +1157,7 @@ var errorCodes = errorCodeMap{
},
ErrUnsupportedNotification: {
Code: "UnsupportedNotification",
Description: "MinIO server does not support Topic or Cloud Function based notifications.",
Description: "Silo does not support Topic or Cloud Function based notifications.",
HTTPStatusCode: http.StatusBadRequest,
},
ErrInvalidCopyPartRange: {
@@ -1336,6 +1339,21 @@ var errorCodes = errorCodeMap{
Description: "IAM sub-system not initialized yet, please try again.",
HTTPStatusCode: http.StatusServiceUnavailable,
},
ErrMultipartListingLegacy: {
Code: "MultipartListingNotReady",
Description: "Legacy multipart uploads prevent a complete listing. Upgrade all writers, drain old uploads and run the multipart preflight check.",
HTTPStatusCode: http.StatusServiceUnavailable,
},
ErrMultipartListingIdentity: {
Code: "MultipartListingMetadataInvalid",
Description: "Multipart upload metadata is inconsistent. Run the multipart preflight check to locate the affected storage set.",
HTTPStatusCode: http.StatusServiceUnavailable,
},
ErrSlowDown: {
Code: "SlowDown",
Description: "Please reduce your request rate",
HTTPStatusCode: http.StatusServiceUnavailable,
},
ErrBucketMetadataNotInitialized: {
Code: "XMinioBucketMetadataNotInitialized",
Description: "Bucket metadata not initialized yet, please try again.",
@@ -1523,10 +1541,14 @@ var errorCodes = errorCodeMap{
Description: "Your Host header is malformed.",
HTTPStatusCode: http.StatusBadRequest,
},
// The stored object cannot be served: a server-side data condition, not a
// successful partial read. Upstream maps it to http.StatusPartialContent
// (since ca6b4773e, 2017), which lets SDKs accept the XML error document
// as object content; SILO deliberately diverges and returns 500.
ErrObjectTampered: {
Code: "XMinioObjectTampered",
Description: errObjectTampered.Error(),
HTTPStatusCode: http.StatusPartialContent,
HTTPStatusCode: http.StatusInternalServerError,
},
ErrSiteReplicationInvalidRequest: {
@@ -2169,6 +2191,14 @@ func toAPIErrorCode(ctx context.Context, err error) (apiErr APIErrorCode) {
err = unwrapAll(err)
switch err {
case errMultipartListingLegacy:
apiErr = ErrMultipartListingLegacy
case errMultipartListingIdentity:
apiErr = ErrMultipartListingIdentity
case errCompleteMultipartChecksumMismatch, errCompleteMultipartChecksumTypeMismatch:
apiErr = ErrBadDigest
case errMissingPartChecksum:
apiErr = ErrInvalidRequest
case errInvalidArgument:
apiErr = ErrAdminInvalidArgument
case errNoSuchPolicy:
@@ -2295,6 +2325,8 @@ func toAPIErrorCode(ctx context.Context, err error) (apiErr APIErrorCode) {
}
switch err.(type) {
case SlowDown:
apiErr = ErrSlowDown
case StorageFull:
apiErr = ErrStorageFull
case hash.BadDigest:
@@ -2465,6 +2497,14 @@ func toAPIError(ctx context.Context, err error) APIError {
}
apiErr := errorCodes.ToAPIErr(toAPIErrorCode(ctx, err))
switch {
case errors.Is(err, errCompleteMultipartChecksumMismatch):
apiErr.Description = strings.TrimPrefix(err.Error(), errCompleteMultipartChecksumMismatch.Error()+": ")
case errors.Is(err, errCompleteMultipartChecksumTypeMismatch):
apiErr.Description = strings.TrimPrefix(err.Error(), errCompleteMultipartChecksumTypeMismatch.Error()+": ")
case errors.Is(err, errMissingPartChecksum):
apiErr.Description = strings.TrimPrefix(err.Error(), errMissingPartChecksum.Error()+": ")
}
switch apiErr.Code {
case "NotImplemented":
apiErr = APIError{
+4
View File
@@ -39,6 +39,10 @@ var toAPIErrorTests = []struct {
{err: ObjectNameInvalid{}, errCode: ErrInvalidObjectName},
{err: InvalidUploadID{}, errCode: ErrNoSuchUpload},
{err: InvalidPart{}, errCode: ErrInvalidPart},
{err: errCompleteMultipartChecksumMismatch, errCode: ErrBadDigest},
{err: errCompleteMultipartChecksumTypeMismatch, errCode: ErrBadDigest},
{err: errMissingPartChecksum, errCode: ErrInvalidRequest},
{err: hash.ChecksumMismatch{}, errCode: ErrContentChecksumMismatch},
{err: InsufficientReadQuorum{}, errCode: ErrSlowDownRead},
{err: InsufficientWriteQuorum{}, errCode: ErrSlowDownWrite},
{err: InvalidUploadIDKeyCombination{}, errCode: ErrNotImplemented},

Some files were not shown because too many files have changed in this diff Show More