mirror of
https://github.com/pgsty/minio.git
synced 2026-08-10 00:03:29 +03:00
Feng Ruohang
c46b16ec62
The transitional references land in one commit, because they are only correct together: the repository is pgsty/silo, its default branch is main, and nothing in the tree should point a user at the old names. Changed: - Workflow branch filters. go.yml and vulncheck.yml gated on `branches: master` for both push and pull_request, so renaming the default branch would have taken automatic CI offline with no error and no signal - the workflows would simply never trigger again. They now name main. - Release target. goreleaser's `release.github.name` becomes silo, which is what actually decides where a tagged build publishes. sign-release-rpms.sh's GH_REPO default follows. - The OCI `image.source` label, the Helm chart `sources` entry, the security advisory link in the issue-template config, and the go.mod comment citing the LDAP TLS fix. - 115 occurrences across README, README_ZH, SECURITY, CONTRIBUTING and 30 docs pages, including 72 links that also carried the master branch in their path. Those matter most: GitHub redirects clone, fetch, push and web URLs after a rename, but raw.githubusercontent.com does not, and neither follows a branch rename - every one of those links would 404 twice over. - Three error strings in cmd/erasure-sets.go, cmd/storage-errors.go and internal/config/errors.go that print an issue URL to operators. These are Go string literals inside rebrand-guard's brand allowlist, so the baseline is regenerated. The regeneration removes exactly those three entries and adds none; all twelve other protected sets, including the 9014 exported symbols, are byte-identical. - The transitional-naming disclaimers in README, README_ZH, SECURITY and CONTRIBUTING are dropped, since they no longer describe anything. Deliberately unchanged, all three because they exist to reject or freeze the old name rather than to point at it: - buildscripts/minio-upgrade.sh pins pgsty/minio@sha256:b6bfe72... - the frozen pre-rebrand image is the control group for the MinIO-to-Silo upgrade test. - helm-migration-guard rejects any rendered container still pulling pgsty/minio. - verify-rebrand.sh rejects the same in the delivery surfaces. Also unchanged: docs/config/README.md links to pgsty/mc/blob/master, and that repository's default branch really is still master. It moves when mc does. verify-rebrand.sh gains three assertions so this cannot silently regress: no source reference may name pgsty/minio outside the three allowlisted guards, no link may target pgsty/silo's master branch, and go.yml and vulncheck.yml must filter on main. Both new rejections were negative-tested - reintroducing a master branch filter and adding a pgsty/minio URL each fail the gate with the specific message. This commit assumes the rename actually happens. Until the GitHub branch and repository renames are executed, the links it introduces do not resolve. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
56 lines
4.1 KiB
Markdown
56 lines
4.1 KiB
Markdown
# ILM Tiering Design [](https://hub.docker.com/r/pgsty/silo/)
|
|
|
|
Lifecycle transition functionality provided in [bucket lifecycle guide](https://github.com/pgsty/silo/blob/main/docs/bucket/lifecycle/README.md) allows tiering of content from Silo object store to public clouds or other Silo clusters.
|
|
|
|
Transition tiers can be added to Silo using `mc admin tier add` command to associate a `gcs`, `s3` or `azure` bucket or prefix path on a bucket to the tier name.
|
|
Lifecycle transition rules can be applied to buckets (both versioned and un-versioned) by specifying the tier name defined above as the transition storage class for the lifecycle rule.
|
|
|
|
## Implementation
|
|
|
|
ILM tiering takes place when a object placed in the bucket meets lifecycle transition rules and becomes eligible for tiering. Silo scanner (which runs at one minute intervals, each time scanning one sixteenth of the namespace), picks up the object for tiering. The data is moved to the remote tier in entirety, leaving only the object metadata on Silo.
|
|
|
|
The data on the backend is stored under the `bucket/prefix` specified in the tier configuration with a custom name derived from a randomly generated uuid - e.g. `0b/c4/0bc4fab7-2daf-4d2f-8e39-5c6c6fb7e2d3`. The first two prefixes are characters 1-2,3-4 from the uuid. This format allows tiering to any cloud irrespective of whether the cloud in question supports versioning. The reference to the transitioned object name and transitioned tier is stored as part of the internal metadata for the object (or its version) on Silo.
|
|
|
|
Extra metadata maintained internally in `xl.meta` for a transitioned object
|
|
|
|
```
|
|
...
|
|
"MetaSys": {
|
|
"x-minio-internal-transition-status": "Y29tcGxldGU=",
|
|
"x-minio-internal-transition-tier": "R0NTVElFUjE=",
|
|
"x-minio-internal-transitioned-object": "ZDIvN2MvZDI3Y2MwYWMtZGIzNC00ZGM1LWIxNDUtYjI5MGNjZjU1MjY5"
|
|
},
|
|
```
|
|
|
|
When a transitioned object is restored temporarily to local Silo instance via PostRestoreObject API, the object data is copied back from the remote tier, and additional metadata for the restored object is maintained as referenced below. Once the restore period expires, the local copy of the object is removed by the scanner during its periodic runs.
|
|
|
|
```
|
|
...
|
|
"MetaUsr": {
|
|
"X-Amz-Restore-Expiry-Days": "4",
|
|
"X-Amz-Restore-Request-Date": "Mon, 22 Feb 2021 21:10:09 GMT",
|
|
"x-amz-restore": "ongoing-request=false, expiry-date=Sat, 27 Feb 2021 00:00:00 GMT",
|
|
...
|
|
```
|
|
|
|
### Encrypted/Object locked objects
|
|
|
|
For objects under SSE-S3 or SSE-C encryption, the encrypted content from Silo cluster is copied as is to the remote tier without any decryption. The content is decrypted as it is streamed from remote tier on `GET/HEAD`. Objects under retention are protected because the metadata present on Silo server ensures that the object (version) is not deleted until retention period is over. Administrators need to ensure that the remote tier bucket is under proper access control.
|
|
|
|
### Transition Status
|
|
|
|
Silo specific extension header `X-Minio-Transition` is displayed on `HEAD/GET` to predict expected transition date on the object. Once object is transitioned to the remote tier,`x-amz-storage-class` shows the tier name to which object transitioned. Additional headers such as "X-Amz-Restore-Expiry-Days", "x-amz-restore", and "X-Amz-Restore-Request-Date" are displayed when a object is under restore/has been restored to local Silo cluster.
|
|
|
|
### Expiry or removal events
|
|
|
|
An object that is in transition tier will be deleted once the object hits expiry date or if removed via `mc rm` (`mc rm --vid` in the case of delete of a specific object version). Other rules specific to legal hold and object locking precede any lifecycle rules.
|
|
|
|
### Additional notes
|
|
|
|
Tiering and lifecycle transition are applicable only to erasure/distributed Silo.
|
|
|
|
## Explore Further
|
|
|
|
- [MinIO Go client API reference (S3-compatible SDK)](https://pkg.go.dev/github.com/minio/minio-go/v7#Client.SetBucketLifecycle)
|
|
- [Object Lifecycle Management](https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lifecycle-mgmt.html)
|