mirror of
https://github.com/pgsty/minio.git
synced 2026-08-09 15:53:28 +03:00
docs: rebrand the repository documentation, templates and dashboards
README, README_ZH, SECURITY, COMPLIANCE, CONTRIBUTING, NOTICE, code_of_conduct, the vulnerability and PR-etiquette documents, the GitHub issue and pull request templates, and the docs/ tree all present Silo as the product. The Grafana dashboards under docs/metrics/prometheus/grafana/ have their panel titles and descriptions rebranded while every minio_* query, label and expression is left alone, so existing alerts and recording rules keep matching. The distinction the review demanded is applied per hit rather than by search-and-replace: - Product and command text becomes Silo and silo: install and run instructions, systemd examples, compose services, download links, badges. - Protocol and interface text keeps MinIO: MINIO_* variables, minio_* metrics, x-minio-* headers, /minio/* routes, .minio.sys, arn:minio, and API field and error names. - Attribution keeps MinIO and gains the fork's own: the AGPL obligations, original copyright, CREDITS and NOTICE stay, with the modification notice added alongside rather than replacing them. - Historical and third-party references are left as facts, not rewritten for brand tidiness. README and README_ZH each carry an explicit non-affiliation notice, document the side-by-side package migration including the /etc/systemd/system/silo.service.d/10-legacy-user.conf drop-in for keeping a legacy UID/GID, and state that recursive chown is never performed. The trademark attribution uses the policy's approved "based on MinIO technology" wording, not the shortened form the policy rejects. github.com/pgsty/minio links are left in place and labelled transitional. The repository has not been renamed, and rewriting them now would produce documented URLs that 404 until the cutover; they change in the cutover commit together with the goreleaser release target, the OCI source label and the raw-content branch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Feng Ruohang
@@ -1,15 +1,15 @@
|
||||
# ILM Tiering Design [](https://slack.min.io) [](https://hub.docker.com/r/minio/minio/)
|
||||
# ILM Tiering Design [](https://hub.docker.com/r/pgsty/silo/)
|
||||
|
||||
Lifecycle transition functionality provided in [bucket lifecycle guide](https://github.com/pgsty/minio/blob/master/docs/bucket/lifecycle/README.md) allows tiering of content from MinIO object store to public clouds or other MinIO clusters.
|
||||
Lifecycle transition functionality provided in [bucket lifecycle guide](https://github.com/pgsty/minio/blob/master/docs/bucket/lifecycle/README.md) allows tiering of content from Silo object store to public clouds or other Silo clusters.
|
||||
|
||||
Transition tiers can be added to MinIO using `mc admin tier add` command to associate a `gcs`, `s3` or `azure` bucket or prefix path on a bucket to the tier name.
|
||||
Transition tiers can be added to Silo using `mc admin tier add` command to associate a `gcs`, `s3` or `azure` bucket or prefix path on a bucket to the tier name.
|
||||
Lifecycle transition rules can be applied to buckets (both versioned and un-versioned) by specifying the tier name defined above as the transition storage class for the lifecycle rule.
|
||||
|
||||
## Implementation
|
||||
|
||||
ILM tiering takes place when a object placed in the bucket meets lifecycle transition rules and becomes eligible for tiering. MinIO scanner (which runs at one minute intervals, each time scanning one sixteenth of the namespace), picks up the object for tiering. The data is moved to the remote tier in entirety, leaving only the object metadata on MinIO.
|
||||
ILM tiering takes place when a object placed in the bucket meets lifecycle transition rules and becomes eligible for tiering. Silo scanner (which runs at one minute intervals, each time scanning one sixteenth of the namespace), picks up the object for tiering. The data is moved to the remote tier in entirety, leaving only the object metadata on Silo.
|
||||
|
||||
The data on the backend is stored under the `bucket/prefix` specified in the tier configuration with a custom name derived from a randomly generated uuid - e.g. `0b/c4/0bc4fab7-2daf-4d2f-8e39-5c6c6fb7e2d3`. The first two prefixes are characters 1-2,3-4 from the uuid. This format allows tiering to any cloud irrespective of whether the cloud in question supports versioning. The reference to the transitioned object name and transitioned tier is stored as part of the internal metadata for the object (or its version) on MinIO.
|
||||
The data on the backend is stored under the `bucket/prefix` specified in the tier configuration with a custom name derived from a randomly generated uuid - e.g. `0b/c4/0bc4fab7-2daf-4d2f-8e39-5c6c6fb7e2d3`. The first two prefixes are characters 1-2,3-4 from the uuid. This format allows tiering to any cloud irrespective of whether the cloud in question supports versioning. The reference to the transitioned object name and transitioned tier is stored as part of the internal metadata for the object (or its version) on Silo.
|
||||
|
||||
Extra metadata maintained internally in `xl.meta` for a transitioned object
|
||||
|
||||
@@ -22,7 +22,7 @@ Extra metadata maintained internally in `xl.meta` for a transitioned object
|
||||
},
|
||||
```
|
||||
|
||||
When a transitioned object is restored temporarily to local MinIO instance via PostRestoreObject API, the object data is copied back from the remote tier, and additional metadata for the restored object is maintained as referenced below. Once the restore period expires, the local copy of the object is removed by the scanner during its periodic runs.
|
||||
When a transitioned object is restored temporarily to local Silo instance via PostRestoreObject API, the object data is copied back from the remote tier, and additional metadata for the restored object is maintained as referenced below. Once the restore period expires, the local copy of the object is removed by the scanner during its periodic runs.
|
||||
|
||||
```
|
||||
...
|
||||
@@ -35,11 +35,11 @@ When a transitioned object is restored temporarily to local MinIO instance via P
|
||||
|
||||
### Encrypted/Object locked objects
|
||||
|
||||
For objects under SSE-S3 or SSE-C encryption, the encrypted content from MinIO cluster is copied as is to the remote tier without any decryption. The content is decrypted as it is streamed from remote tier on `GET/HEAD`. Objects under retention are protected because the metadata present on MinIO server ensures that the object (version) is not deleted until retention period is over. Administrators need to ensure that the remote tier bucket is under proper access control.
|
||||
For objects under SSE-S3 or SSE-C encryption, the encrypted content from Silo cluster is copied as is to the remote tier without any decryption. The content is decrypted as it is streamed from remote tier on `GET/HEAD`. Objects under retention are protected because the metadata present on Silo server ensures that the object (version) is not deleted until retention period is over. Administrators need to ensure that the remote tier bucket is under proper access control.
|
||||
|
||||
### Transition Status
|
||||
|
||||
MinIO specific extension header `X-Minio-Transition` is displayed on `HEAD/GET` to predict expected transition date on the object. Once object is transitioned to the remote tier,`x-amz-storage-class` shows the tier name to which object transitioned. Additional headers such as "X-Amz-Restore-Expiry-Days", "x-amz-restore", and "X-Amz-Restore-Request-Date" are displayed when a object is under restore/has been restored to local MinIO cluster.
|
||||
Silo specific extension header `X-Minio-Transition` is displayed on `HEAD/GET` to predict expected transition date on the object. Once object is transitioned to the remote tier,`x-amz-storage-class` shows the tier name to which object transitioned. Additional headers such as "X-Amz-Restore-Expiry-Days", "x-amz-restore", and "X-Amz-Restore-Request-Date" are displayed when a object is under restore/has been restored to local Silo cluster.
|
||||
|
||||
### Expiry or removal events
|
||||
|
||||
@@ -47,9 +47,9 @@ An object that is in transition tier will be deleted once the object hits expiry
|
||||
|
||||
### Additional notes
|
||||
|
||||
Tiering and lifecycle transition are applicable only to erasure/distributed MinIO.
|
||||
Tiering and lifecycle transition are applicable only to erasure/distributed Silo.
|
||||
|
||||
## Explore Further
|
||||
|
||||
- [MinIO | Golang Client API Reference](https://pkg.go.dev/github.com/minio/minio-go/v7#Client.SetBucketLifecycle)
|
||||
- [MinIO Go client API reference (S3-compatible SDK)](https://pkg.go.dev/github.com/minio/minio-go/v7#Client.SetBucketLifecycle)
|
||||
- [Object Lifecycle Management](https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lifecycle-mgmt.html)
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
# Bucket Lifecycle Configuration Quickstart Guide [](https://slack.min.io) [](https://hub.docker.com/r/minio/minio/)
|
||||
# Bucket Lifecycle Configuration Quickstart Guide [](https://hub.docker.com/r/pgsty/silo/)
|
||||
|
||||
Enable object lifecycle configuration on buckets to setup automatic deletion of objects after a specified number of days or a specified date.
|
||||
|
||||
## 1. Prerequisites
|
||||
|
||||
- Install MinIO - [MinIO Quickstart Guide](https://silo.pgsty.com/operations/deployments/baremetal-deploy-minio-on-redhat-linux/).
|
||||
- Install Silo - [Silo Quickstart Guide](https://silo.pgsty.com/operations/deployments/baremetal-deploy-minio-on-redhat-linux/).
|
||||
- Install `mc` - [mc Quickstart Guide](https://silo.pgsty.com/reference/minio-mc/#quickstart)
|
||||
|
||||
## 2. Enable bucket lifecycle configuration
|
||||
@@ -84,9 +84,9 @@ e.g., To scan objects stored under `user-uploads/` prefix and remove versions ol
|
||||
}
|
||||
```
|
||||
|
||||
This JSON rule is equivalent to the following MinIO Client command:
|
||||
This JSON rule is equivalent to the following compatible `mc` command:
|
||||
```
|
||||
mc ilm rule add --noncurrent-expire-days 365 --prefix "user-uploads/" myminio/mydata
|
||||
mc ilm rule add --noncurrent-expire-days 365 --prefix "user-uploads/" mysilo/mydata
|
||||
```
|
||||
|
||||
### 3.2 Automatic removal of noncurrent versions keeping only most recent ones after noncurrent days
|
||||
@@ -113,14 +113,14 @@ e.g, To remove noncurrent versions of all objects keeping the most recent 5 nonc
|
||||
}
|
||||
```
|
||||
|
||||
This JSON rule is equivalent to the following MinIO Client command:
|
||||
This JSON rule is equivalent to the following compatible `mc` command:
|
||||
```
|
||||
mc ilm rule add --noncurrent-expire-days 30 --noncurrent-expire-newer 5 --prefix "user-uploads/" myminio/mydata
|
||||
mc ilm rule add --noncurrent-expire-days 30 --noncurrent-expire-newer 5 --prefix "user-uploads/" mysilo/mydata
|
||||
```
|
||||
|
||||
#### 3.2.a Automatic removal of noncurrent versions keeping only most recent ones immediately (MinIO only extension)
|
||||
#### 3.2.a Automatic removal of noncurrent versions keeping only most recent ones immediately (Silo only extension)
|
||||
|
||||
This is available only on MinIO as an extension to the NewerNoncurrentVersions feature. The following rule makes it possible to remove older noncurrent versions
|
||||
This is available only on Silo as an extension to the NewerNoncurrentVersions feature. The following rule makes it possible to remove older noncurrent versions
|
||||
of objects under the prefix `user-uploads/` as soon as there are more than `N` noncurrent versions of an object.
|
||||
|
||||
```
|
||||
@@ -141,9 +141,9 @@ of objects under the prefix `user-uploads/` as soon as there are more than `N` n
|
||||
```
|
||||
Note: This rule has an implicit zero NoncurrentDays, which makes the expiry of those 'extra' noncurrent versions immediate.
|
||||
|
||||
#### 3.2.b Automatic removal of all versions (MinIO only extension)
|
||||
#### 3.2.b Automatic removal of all versions (Silo only extension)
|
||||
|
||||
This is available only on MinIO as an extension to the Expiration feature. The following rule makes it possible to remove all versions of an object under
|
||||
This is available only on Silo as an extension to the Expiration feature. The following rule makes it possible to remove all versions of an object under
|
||||
the prefix `user-uploads/` as soon as the latest object satisfies the expiration criteria.
|
||||
|
||||
> NOTE: If the latest object is a delete marker then filtering based on `Filter.Tags` is ignored and
|
||||
@@ -188,9 +188,9 @@ When an object has only one version as a delete marker, the latter can be automa
|
||||
|
||||
## 4. Enable ILM transition feature
|
||||
|
||||
In Erasure mode, MinIO supports tiering to public cloud providers such as GCS, AWS and Azure as well as to other MinIO clusters via the ILM transition feature. This will allow transitioning of older objects to a different cluster or the public cloud by setting up transition rules in the bucket lifecycle configuration. This feature enables applications to optimize storage costs by moving less frequently accessed data to a cheaper storage without compromising accessibility of data.
|
||||
In Erasure mode, Silo supports tiering to public cloud providers such as GCS, AWS and Azure as well as to other Silo clusters via the ILM transition feature. This will allow transitioning of older objects to a different cluster or the public cloud by setting up transition rules in the bucket lifecycle configuration. This feature enables applications to optimize storage costs by moving less frequently accessed data to a cheaper storage without compromising accessibility of data.
|
||||
|
||||
To transition objects in a bucket to a destination bucket on a different cluster, applications need to specify a transition tier defined on MinIO instead of storage class while setting up the ILM lifecycle rule.
|
||||
To transition objects in a bucket to a destination bucket on a different cluster, applications need to specify a transition tier defined on Silo instead of storage class while setting up the ILM lifecycle rule.
|
||||
|
||||
> To create a transition tier for transitioning objects to a prefix `testprefix` in `azurebucket` on Azure blob using `mc`:
|
||||
|
||||
@@ -203,10 +203,10 @@ To transition objects in a bucket to a destination bucket on a different cluster
|
||||
Using above tier, set up a lifecycle rule with transition:
|
||||
|
||||
```
|
||||
mc ilm add --expiry-days 365 --transition-days 45 --storage-class "AZURETIER" myminio/srcbucket
|
||||
mc ilm add --expiry-days 365 --transition-days 45 --storage-class "AZURETIER" mysilo/srcbucket
|
||||
```
|
||||
|
||||
Note: In the case of S3, it is possible to create a tier from MinIO running in EC2 to S3 using AWS role attached to EC2 as credentials instead of accesskey/secretkey:
|
||||
Note: In the case of S3, it is possible to create a tier from Silo running in EC2 to S3 using AWS role attached to EC2 as credentials instead of accesskey/secretkey:
|
||||
|
||||
```
|
||||
mc admin tier add s3 source S3TIER --bucket s3bucket --prefix testprefix/ --use-aws-role
|
||||
@@ -224,9 +224,9 @@ aws s3api restore-object --bucket srcbucket \
|
||||
|
||||
`s3:ObjectTransition:Complete` and `s3:ObjectTransition:Failed` events can be used to monitor transition events between the source cluster and transition tier. To watch lifecycle events, you can enable bucket notification on the source bucket with `mc event add` and specify `--event ilm` flag.
|
||||
|
||||
Note that transition event notification is a MinIO extension.
|
||||
Note that transition event notification is a Silo extension.
|
||||
|
||||
## Explore Further
|
||||
|
||||
- [MinIO | Golang Client API Reference](https://pkg.go.dev/github.com/minio/minio-go/v7)
|
||||
- [MinIO Go client API reference (S3-compatible SDK)](https://pkg.go.dev/github.com/minio/minio-go/v7)
|
||||
- [Object Lifecycle Management](https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lifecycle-mgmt.html)
|
||||
|
||||
Reference in New Issue
Block a user