astelm/minio
1
0
Fork 0
mirror of https://github.com/pgsty/minio.git synced 2026-09-15 15:04:04 +03:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #188 from pgsty/codex/remove-access-tiering-final

Browse Source 
Remove access-frequency pool tiering and preserve independent multi-pool correctness fixes. Reconcile ordinary addressed-version DELETE across pools, retaining existing quorum and compatibility boundaries.

Verified delivery head: 4d0693cb8c. All 11 final CI checks and three qualified Linux upgrade runs passed. Introduction, retirement decisions and historical unresolved observations are documented.

Signed-off-by: Feng Ruohang <rh@vonng.com>
This commit is contained in:
Feng Ruohang
2026-09-15 15:09:48 +08:00
committed by GitHub
parent 89637554d6 4d0693cb8c
commit 9df0f4abaf
46 changed files with 1447 additions and 5684 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+21 -4
View File
@@ -22,10 +22,27 @@ and [complete commit range](https://github.com/pgsty/silo/compare/RELEASE.2026-0
### Object storage and replication
- Add GET-frequency-based movement across pools, then preserve versions and
isolate writes during movement. Serialize and reconcile multi-pool object
writes, metadata updates, healing and conditional deletion; preserve shared
remote-tier references until their last local owner is removed.
- Reconcile ordinary single-object version DELETE across all pools, including
null versions, delete markers and unqualified directory-marker DELETE. This
applies the deletion to every resolved pool copy under existing quorum
rules. Pending outbound delete replication retains versions until the
existing replication worker completes their purge; a successful response
does not imply immediate physical removal from every drive. Unreadable
pools now consistently return 503 instead of depending on pool traversal
order; insufficient read quorum returns `SlowDownRead`. This extends the
existing failure surface. Retry after recovery.
Cleanup failures also return an error. Batch deletion already fans out across
pools; replication and scanner cleanup keep their existing contracts. See
[scope and limitations](docs/bucket/lifecycle/access-tiering-removal.md#version-deletion-scope).
- Remove the opt-in GET-frequency pool-tiering feature from PR #60, including
its tracker, mover, scanner hooks, configuration, XML actions and metrics.
Accept and ignore retired configuration/XML and preserve ordinary statistics
when reading v9 caches. See [migration notes](docs/bucket/lifecycle/access-tiering-removal.md).
The [decision record](docs/investigations/access-tiering-revert.md) preserves
the feature's introduction, subsequent fixes, rollback scope and review history.
- Preserve the independent multi-pool write, metadata, healing and conditional
deletion fixes from PR #178, including shared remote-tier reference protection.
- Enforce `If-Match` on DELETE, preserve retention and independently ordered
Object Lock/tag updates, and correctly retransmit encrypted replicas.
- Preserve plaintext part sizes and raw SSE-C replicas; prevent SSE-C
CONTRIBUTORS.md
+1 -1
View File
@@ -62,7 +62,7 @@ the Git history and [NOTICE](NOTICE); this record covers activity in the PGSTY r
| :-- | :-- | :-- |
| [@Vonng](https://github.com/Vonng) | Maintains SILO, Console, mcli, shared packages, releases, and documentation | 85 merged PRs across five repositories; [complete maintainer record](#maintainer-record) |
| [@h5vx](https://github.com/h5vx) | Implemented per-bucket CORS configuration and enforcement | [pgsty/silo#71](https://github.com/pgsty/silo/pull/71) — Merged ([e4e3007da](https://github.com/pgsty/silo/commit/e4e3007da6d7d1198a6a050e34f84566d40a9654)) |
| [@mrjavadseydi](https://github.com/mrjavadseydi) | Fixed effective bucket quota metrics; proposed access-frequency ILM | [pgsty/silo#132](https://github.com/pgsty/silo/pull/132) — Merged ([ad873c735](https://github.com/pgsty/silo/commit/ad873c73571b121293791f13f8dc46ddf5264d60))<br>[pgsty/silo#60](https://github.com/pgsty/silo/pull/60) — Open |
| [@mrjavadseydi](https://github.com/mrjavadseydi) | Fixed effective bucket quota metrics; proposed access-frequency ILM | [pgsty/silo#132](https://github.com/pgsty/silo/pull/132) — Merged ([ad873c735](https://github.com/pgsty/silo/commit/ad873c73571b121293791f13f8dc46ddf5264d60))<br>[pgsty/silo#60](https://github.com/pgsty/silo/pull/60) — Merged; access-frequency feature subsequently removed |
| [@Dansyuqri](https://github.com/Dansyuqri) | Added ChecksumType to multipart completion responses | [pgsty/silo#57](https://github.com/pgsty/silo/pull/57) — Merged ([a96116b12](https://github.com/pgsty/silo/commit/a96116b128bbf2aa42f85eafbf75eb6636cd36ee)) |
| [@ycjlin](https://github.com/ycjlin) | Fixed missing-bucket ListObjects semantics | [pgsty/silo#37](https://github.com/pgsty/silo/pull/37) — Merged ([49c8aeac4](https://github.com/pgsty/silo/commit/49c8aeac403916f52f8588bbe8ee42753d86eeef)) |
| [@pinginfo](https://github.com/pinginfo) | Repaired bucket notification streaming | [pgsty/silo#34](https://github.com/pgsty/silo/pull/34) — Merged ([b7f52ca43](https://github.com/pgsty/silo/commit/b7f52ca4336bc45a48b81b39c8983a5e6882a6fa)) |
buildscripts/rebrand-guard/compat-baseline.json
+1 -11
View File
@@ -289,16 +289,6 @@
"MINIO_IDENTITY_TLS_SKIP_VERIFY",
"MINIO_IDENTITY_TLS_STS_EXPIRY",
"MINIO_IDLE_TIMEOUT",
"MINIO_ILM_ACCESS_BINS",
"MINIO_ILM_ACCESS_BIN_WIDTH",
"MINIO_ILM_ACCESS_FLUSH",
"MINIO_ILM_ACCESS_MAX_SIZE",
"MINIO_ILM_ACCESS_MAX_TRACKED",
"MINIO_ILM_ACCESS_MIN_RESIDENCY",
"MINIO_ILM_ACCESS_POOLS",
"MINIO_ILM_ACCESS_PROMOTE_WATERMARK",
"MINIO_ILM_ACCESS_TIERING",
"MINIO_ILM_ACCESS_WORKERS",
"MINIO_ILM_EXPIRATION_WORKERS",
"MINIO_ILM_TRANSITION_WORKERS",
"MINIO_INTERFACE",
@@ -745,7 +735,6 @@
"/idp/ldap/policy/{operation}",
"/idp/openid/list-access-keys-bulk",
"/ilm",
"/ilm/access",
"/import-bucket-metadata",
"/import-iam",
"/import-iam-v2",
@@ -915,6 +904,7 @@
".minio.sys/config/config.json",
".minio.sys/config/hello.txt",
".minio.sys/config/iam/${username}/identity.json",
".minio.sys/config/ilm/access",
".minio.sys/format.json",
".minio.sys/multipart",
".minio.sys/multipart/bucket/object/uploads.json",
cmd/bucket-lifecycle-audit.go
-1
View File
@@ -39,7 +39,6 @@ const (
lcEventSrc_s3PutObject
lcEventSrc_s3CopyObject
lcEventSrc_s3CompleteMultipartUpload
lcEventSrc_AccessTier
)
//revive:enable:var-naming
cmd/bucket-metadata_test.go
+25
View File
@@ -39,3 +39,28 @@ func TestBucketMetadataCorsRoundTrip(t *testing.T) {
t.Fatalf("CorsConfigUpdatedAt not preserved")
}
}
// A persisted retired extension must not prevent the whole bucket's metadata
// from loading, including unrelated versioning and ordinary lifecycle rules.
func TestBucketMetadataRetiredAccessTiering(t *testing.T) {
meta := newBucketMetadata("retired-access")
meta.LifecycleConfigXML = []byte(`<LifecycleConfiguration><AccessTierQuota>500GiB</AccessTierQuota><Rule><ID>access</ID><Status>Enabled</Status><Filter><Prefix>logs/</Prefix></Filter><AccessTransition><Window>10m</Window><PromoteAfterAccesses>10</PromoteAfterAccesses></AccessTransition></Rule><Rule><ID>ordinary</ID><Status>Enabled</Status><Filter><Prefix>expired/</Prefix></Filter><Expiration><Days>30</Days></Expiration></Rule></LifecycleConfiguration>`)
meta.VersioningConfigXML = []byte(`<VersioningConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Status>Enabled</Status></VersioningConfiguration>`)
data, err := meta.MarshalMsg(nil)
if err != nil {
t.Fatal(err)
}
got := newBucketMetadata(meta.Name)
if _, err := got.UnmarshalMsg(data); err != nil {
t.Fatal(err)
}
if err := got.parseAllConfigs(t.Context(), nil); err != nil {
t.Fatalf("bucket metadata failed to load: %v", err)
}
if got.lifecycleConfig == nil || got.lifecycleConfig.HasActiveRules("logs/") || !got.lifecycleConfig.HasActiveRules("expired/") {
t.Fatal("unexpected lifecycle behavior")
}
if got.versioningConfig == nil || !got.versioningConfig.Enabled() {
t.Fatal("unrelated versioning lost")
}
}
cmd/config-current.go
+1 -4
View File
@@ -227,7 +227,7 @@ func initHelp() {
},
config.HelpKV{
Key: config.ILMSubSys,
Description: "manage ILM settings for expiration, transition, and access-tier workers",
Description: "manage ILM settings for expiration and transition workers",
Optional: true,
},
}
@@ -704,9 +704,6 @@ func applyDynamicConfigForSubSys(ctx context.Context, objAPI ObjectLayer, s conf
if globalExpiryState != nil {
globalExpiryState.ResizeWorkers(ilmCfg.ExpirationWorkers)
}
if globalAccessTierState != nil {
globalAccessTierState.UpdateWorkers(ilmCfg.AccessWorkers)
}
globalILMConfig.update(ilmCfg)
}
}
cmd/data-scanner.go
-14
View File
@@ -899,7 +899,6 @@ type scannerItem struct {
objectName string // Only the object name without prefixes.
replication replicationConfig
lifeCycle *lifecycle.Lifecycle
poolIdx int
Typ fs.FileMode
heal struct {
enabled bool
@@ -910,7 +909,6 @@ type scannerItem struct {
type sizeSummary struct {
totalSize int64
hotTierSize int64
versions uint64
deleteMarkers uint64
replicatedSize int64
@@ -1161,14 +1159,6 @@ eventLoop:
globalExpiryState.enqueueNoncurrentVersions(i.bucket, toDel, noncurrentEvents)
}
i.alertExcessiveVersions(remainingVersions, cumulativeSize)
if globalILMConfig.accessTieringEnabled() {
for idx, oi := range objInfos {
if oi.IsLatest && events[idx].Action == lifecycle.NoneAction {
applyAccessTransition(ctx, i, oi)
break
}
}
}
}
func evalActionFromLifecycle(ctx context.Context, lc lifecycle.Lifecycle, lr lock.Retention, rcfg *replication.Config, obj ObjectInfo) lifecycle.Event {
@@ -1484,8 +1474,6 @@ const (
ILMFreeVersionDelete = "ilm:free-version-delete"
// ILMTransition - audit trail for ILM transitioning.
ILMTransition = " ilm:transition"
// ILMAccessTier - audit trail for moving objects between server pools.
ILMAccessTier = "ilm:access-tier"
)
func auditLogLifecycle(ctx context.Context, oi ObjectInfo, event string, tags map[string]string, traceFn func(event string, metadata map[string]string, err error)) {
@@ -1497,8 +1485,6 @@ func auditLogLifecycle(ctx context.Context, oi ObjectInfo, event string, tags ma
apiName = "ILMFreeVersionDelete"
case ILMTransition:
apiName = "ILMTransition"
case ILMAccessTier:
apiName = "ILMAccessTier"
}
auditLogInternal(ctx, AuditLogOptions{
Event: event,
cmd/data-usage-cache-v9_test.go
+89
View File
@@ -0,0 +1,89 @@
// Copyright (c) 2026 Feng Ruohang
//
// This file is part of Silo Object Storage stack
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package cmd
import (
"bytes"
"encoding/json"
"os"
"reflect"
"testing"
)
// Read a real pre-removal scanner cache with nonzero hot-tier bytes. A v8
// payload with a relabeled header would not exercise the ignored hts field.
func TestDataUsageCacheReadV9(t *testing.T) {
raw, err := os.ReadFile("testdata/data-usage-v9/data-usage-v9.bin")
if err != nil {
t.Fatal(err)
}
if len(raw) == 0 || raw[0] != 9 {
t.Fatal("fixture is not v9")
}
expected, err := os.ReadFile("testdata/data-usage-v9/data-usage-v9.json")
if err != nil {
t.Fatal(err)
}
var want, got dataUsageCache
if err := json.Unmarshal(expected, &want); err != nil {
t.Fatal(err)
}
if err := got.deserialize(bytes.NewReader(raw)); err != nil {
t.Fatal(err)
}
if !got.Info.LastUpdate.Equal(want.Info.LastUpdate) {
t.Fatal("cache timestamp changed")
}
// msgp restores local time while JSON preserves the UTC representation.
want.Info.LastUpdate = got.Info.LastUpdate
if !reflect.DeepEqual(got, want) {
t.Fatalf("v9 ordinary cache fields changed:\ngot: %#v\nwant: %#v", got, want)
}
flat := got.flatten(*got.root())
if flat.Size != 74962 || flat.Objects != 3 || flat.Versions != 5 || flat.DeleteMarkers != 2 {
t.Fatalf("statistics changed: %+v", flat)
}
if flat.AllTierStats == nil || flat.AllTierStats.Tiers["COLD"] != (tierStats{TotalSize: 65536, NumVersions: 1, NumObjects: 1}) {
t.Fatalf("remote tier statistics changed: %+v", flat.AllTierStats)
}
buckets := []BucketInfo{{Name: "v9-bucket"}}
if gotInfo, wantInfo := got.dui(dataUsageRoot, buckets), want.dui(dataUsageRoot, buckets); !reflect.DeepEqual(gotInfo, wantInfo) {
t.Fatalf("bucket usage aggregation changed: %+v != %+v", gotInfo, wantInfo)
}
var buf bytes.Buffer
if err := got.serializeTo(&buf); err != nil {
t.Fatal(err)
}
if buf.Bytes()[0] != 8 {
t.Fatalf("wrote cache version %d, want 8", buf.Bytes()[0])
}
var roundtrip dataUsageCache
if err := roundtrip.deserialize(&buf); err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(got, roundtrip) {
t.Fatal("v8 round trip lost ordinary fields")
}
if err := roundtrip.deserialize(bytes.NewReader(raw[:len(raw)/2])); err == nil {
t.Fatal("truncated v9 cache accepted")
}
raw[0] = 10
if err := roundtrip.deserialize(bytes.NewReader(raw)); err == nil {
t.Fatal("unknown cache version accepted")
}
}
cmd/data-usage-cache.go
+7 -61
View File
@@ -61,7 +61,6 @@ type dataUsageEntry struct {
Children dataUsageHashMap `msg:"ch"`
// These fields do no include any children.
Size int64 `msg:"sz"`
HotTierSize int64 `msg:"hts"`
Objects uint64 `msg:"os"`
Versions uint64 `msg:"vs"` // Versions that are not delete markers.
DeleteMarkers uint64 `msg:"dms"`
@@ -135,8 +134,8 @@ func (ts tierStats) add(u tierStats) tierStats {
}
}
//msgp:encode ignore dataUsageEntryV2 dataUsageEntryV3 dataUsageEntryV4 dataUsageEntryV5 dataUsageEntryV6 dataUsageEntryV7 dataUsageEntryV8
//msgp:marshal ignore dataUsageEntryV2 dataUsageEntryV3 dataUsageEntryV4 dataUsageEntryV5 dataUsageEntryV6 dataUsageEntryV7 dataUsageEntryV8
//msgp:encode ignore dataUsageEntryV2 dataUsageEntryV3 dataUsageEntryV4 dataUsageEntryV5 dataUsageEntryV6 dataUsageEntryV7
//msgp:marshal ignore dataUsageEntryV2 dataUsageEntryV3 dataUsageEntryV4 dataUsageEntryV5 dataUsageEntryV6 dataUsageEntryV7
//msgp:tuple dataUsageEntryV2
type dataUsageEntryV2 struct {
@@ -200,30 +199,14 @@ type dataUsageEntryV7 struct {
Compacted bool `msg:"c"`
}
// dataUsageEntryV8 is the on-disk shape before access-tier accounting was
// introduced. Keep it so caches written by the previous release decode
// without being discarded.
type dataUsageEntryV8 struct {
Children dataUsageHashMap `msg:"ch"`
// These fields do no include any children.
Size int64 `msg:"sz"`
Objects uint64 `msg:"os"`
Versions uint64 `msg:"vs"`
DeleteMarkers uint64 `msg:"dms"`
ObjSizes sizeHistogram `msg:"szs"`
ObjVersions versionsHistogram `msg:"vh"`
AllTierStats *allTierStats `msg:"ats,omitempty"`
Compacted bool `msg:"c"`
}
// dataUsageCache contains a cache of data usage entries latest version.
type dataUsageCache struct {
Info dataUsageCacheInfo
Cache map[string]dataUsageEntry
}
//msgp:encode ignore dataUsageCacheV2 dataUsageCacheV3 dataUsageCacheV4 dataUsageCacheV5 dataUsageCacheV6 dataUsageCacheV7 dataUsageCacheV8
//msgp:marshal ignore dataUsageCacheV2 dataUsageCacheV3 dataUsageCacheV4 dataUsageCacheV5 dataUsageCacheV6 dataUsageCacheV7 dataUsageCacheV8
//msgp:encode ignore dataUsageCacheV2 dataUsageCacheV3 dataUsageCacheV4 dataUsageCacheV5 dataUsageCacheV6 dataUsageCacheV7
//msgp:marshal ignore dataUsageCacheV2 dataUsageCacheV3 dataUsageCacheV4 dataUsageCacheV5 dataUsageCacheV6 dataUsageCacheV7
// dataUsageCacheV2 contains a cache of data usage entries version 2.
type dataUsageCacheV2 struct {
@@ -261,12 +244,6 @@ type dataUsageCacheV7 struct {
Cache map[string]dataUsageEntryV7
}
// dataUsageCacheV8 contains a cache of data usage entries version 8.
type dataUsageCacheV8 struct {
Info dataUsageCacheInfo
Cache map[string]dataUsageEntryV8
}
//msgp:ignore dataUsageEntryInfo
type dataUsageEntryInfo struct {
Name string
@@ -295,7 +272,6 @@ type dataUsageCacheInfo struct {
func (e *dataUsageEntry) addSizes(summary sizeSummary) {
e.Size += summary.totalSize
e.HotTierSize += summary.hotTierSize
e.Versions += summary.versions
e.DeleteMarkers += summary.deleteMarkers
e.ObjSizes.add(summary.totalSize)
@@ -315,7 +291,6 @@ func (e *dataUsageEntry) merge(other dataUsageEntry) {
e.Versions += other.Versions
e.DeleteMarkers += other.DeleteMarkers
e.Size += other.Size
e.HotTierSize += other.HotTierSize
for i, v := range other.ObjSizes[:] {
e.ObjSizes[i] += v
@@ -456,7 +431,6 @@ func (d *dataUsageCache) dui(path string, buckets []BucketInfo) DataUsageInfo {
flat := d.flatten(*e)
dui := DataUsageInfo{
LastUpdate: d.Info.LastUpdate,
ScannerCycle: d.Info.NextCycle,
ObjectsTotalCount: flat.Objects,
VersionsTotalCount: flat.Versions,
DeleteMarkersTotalCount: flat.DeleteMarkers,
@@ -807,7 +781,6 @@ func (d *dataUsageCache) bucketsUsageInfo(buckets []BucketInfo) map[string]Bucke
flat := d.flatten(*e)
bui := BucketUsageInfo{
Size: uint64(flat.Size),
HotTierSize: uint64(max(flat.HotTierSize, 0)),
VersionsCount: flat.Versions,
ObjectsCount: flat.Objects,
DeleteMarkersCount: flat.DeleteMarkers,
@@ -1007,8 +980,8 @@ func (d *dataUsageCache) save(ctx context.Context, store objectIO, name string)
// Bumping the cache version will drop data from previous versions
// and write new data with the new version.
const (
dataUsageCacheVerCurrent = 9
dataUsageCacheVerV8 = 8
dataUsageCacheVerCurrent = 8
dataUsageCacheVerV9 = 9 // Retired access-tier cache; only adds the ignored "hts" entry key.
dataUsageCacheVerV7 = 7
dataUsageCacheVerV6 = 6
dataUsageCacheVerV5 = 5
@@ -1210,34 +1183,7 @@ func (d *dataUsageCache) deserialize(r io.Reader) error {
}
return nil
case dataUsageCacheVerV8:
// Zstd compressed.
dec, err := zstd.NewReader(r, zstd.WithDecoderConcurrency(2))
if err != nil {
return err
}
defer dec.Close()
dold := &dataUsageCacheV8{}
if err = dold.DecodeMsg(msgp.NewReader(dec)); err != nil {
return err
}
d.Info = dold.Info
d.Cache = make(map[string]dataUsageEntry, len(dold.Cache))
for k, v := range dold.Cache {
d.Cache[k] = dataUsageEntry{
Children: v.Children,
Size: v.Size,
Objects: v.Objects,
Versions: v.Versions,
DeleteMarkers: v.DeleteMarkers,
ObjSizes: v.ObjSizes,
ObjVersions: v.ObjVersions,
AllTierStats: v.AllTierStats,
Compacted: v.Compacted,
}
}
return nil
case dataUsageCacheVerCurrent:
case dataUsageCacheVerCurrent, dataUsageCacheVerV9:
// Zstd compressed.
dec, err := zstd.NewReader(r, zstd.WithDecoderConcurrency(2))
if err != nil {
cmd/data-usage-cache_gen.go
+9 -605
View File
@@ -1591,145 +1591,6 @@ func (z *dataUsageCacheV7) Msgsize() (s int) {
return
}
// DecodeMsg implements msgp.Decodable
func (z *dataUsageCacheV8) DecodeMsg(dc *msgp.Reader) (err error) {
var field []byte
_ = field
var zb0001 uint32
zb0001, err = dc.ReadMapHeader()
if err != nil {
err = msgp.WrapError(err)
return
}
for zb0001 > 0 {
zb0001--
field, err = dc.ReadMapKeyPtr()
if err != nil {
err = msgp.WrapError(err)
return
}
switch msgp.UnsafeString(field) {
case "Info":
err = z.Info.DecodeMsg(dc)
if err != nil {
err = msgp.WrapError(err, "Info")
return
}
case "Cache":
var zb0002 uint32
zb0002, err = dc.ReadMapHeader()
if err != nil {
err = msgp.WrapError(err, "Cache")
return
}
if z.Cache == nil {
z.Cache = make(map[string]dataUsageEntryV8, zb0002)
} else if len(z.Cache) > 0 {
clear(z.Cache)
}
for zb0002 > 0 {
zb0002--
var za0001 string
za0001, err = dc.ReadString()
if err != nil {
err = msgp.WrapError(err, "Cache")
return
}
var za0002 dataUsageEntryV8
err = za0002.DecodeMsg(dc)
if err != nil {
err = msgp.WrapError(err, "Cache", za0001)
return
}
z.Cache[za0001] = za0002
}
default:
err = dc.Skip()
if err != nil {
err = msgp.WrapError(err)
return
}
}
}
return
}
// UnmarshalMsg implements msgp.Unmarshaler
func (z *dataUsageCacheV8) UnmarshalMsg(bts []byte) (o []byte, err error) {
var field []byte
_ = field
var zb0001 uint32
zb0001, bts, err = msgp.ReadMapHeaderBytes(bts)
if err != nil {
err = msgp.WrapError(err)
return
}
for zb0001 > 0 {
zb0001--
field, bts, err = msgp.ReadMapKeyZC(bts)
if err != nil {
err = msgp.WrapError(err)
return
}
switch msgp.UnsafeString(field) {
case "Info":
bts, err = z.Info.UnmarshalMsg(bts)
if err != nil {
err = msgp.WrapError(err, "Info")
return
}
case "Cache":
var zb0002 uint32
zb0002, bts, err = msgp.ReadMapHeaderBytes(bts)
if err != nil {
err = msgp.WrapError(err, "Cache")
return
}
if z.Cache == nil {
z.Cache = make(map[string]dataUsageEntryV8, zb0002)
} else if len(z.Cache) > 0 {
clear(z.Cache)
}
for zb0002 > 0 {
var za0002 dataUsageEntryV8
zb0002--
var za0001 string
za0001, bts, err = msgp.ReadStringBytes(bts)
if err != nil {
err = msgp.WrapError(err, "Cache")
return
}
bts, err = za0002.UnmarshalMsg(bts)
if err != nil {
err = msgp.WrapError(err, "Cache", za0001)
return
}
z.Cache[za0001] = za0002
}
default:
bts, err = msgp.Skip(bts)
if err != nil {
err = msgp.WrapError(err)
return
}
}
}
o = bts
return
}
// Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message
func (z *dataUsageCacheV8) Msgsize() (s int) {
s = 1 + 5 + z.Info.Msgsize() + 6 + msgp.MapHeaderSize
if z.Cache != nil {
for za0001, za0002 := range z.Cache {
_ = za0002
s += msgp.StringPrefixSize + len(za0001) + za0002.Msgsize()
}
}
return
}
// DecodeMsg implements msgp.Decodable
func (z *dataUsageEntry) DecodeMsg(dc *msgp.Reader) (err error) {
var field []byte
@@ -1762,12 +1623,6 @@ func (z *dataUsageEntry) DecodeMsg(dc *msgp.Reader) (err error) {
err = msgp.WrapError(err, "Size")
return
}
case "hts":
z.HotTierSize, err = dc.ReadInt64()
if err != nil {
err = msgp.WrapError(err, "HotTierSize")
return
}
case "os":
z.Objects, err = dc.ReadUint64()
if err != nil {
@@ -1946,12 +1801,12 @@ func (z *dataUsageEntry) DecodeMsg(dc *msgp.Reader) (err error) {
// EncodeMsg implements msgp.Encodable
func (z *dataUsageEntry) EncodeMsg(en *msgp.Writer) (err error) {
// check for omitted fields
zb0001Len := uint32(10)
var zb0001Mask uint16 /* 10 bits */
zb0001Len := uint32(9)
var zb0001Mask uint16 /* 9 bits */
_ = zb0001Mask
if z.AllTierStats == nil {
zb0001Len--
zb0001Mask |= 0x100
zb0001Mask |= 0x80
}
// variable map header, size zb0001Len
err = en.Append(0x80 | uint8(zb0001Len))
@@ -1981,16 +1836,6 @@ func (z *dataUsageEntry) EncodeMsg(en *msgp.Writer) (err error) {
err = msgp.WrapError(err, "Size")
return
}
// write "hts"
err = en.Append(0xa3, 0x68, 0x74, 0x73)
if err != nil {
return
}
err = en.WriteInt64(z.HotTierSize)
if err != nil {
err = msgp.WrapError(err, "HotTierSize")
return
}
// write "os"
err = en.Append(0xa2, 0x6f, 0x73)
if err != nil {
@@ -2055,7 +1900,7 @@ func (z *dataUsageEntry) EncodeMsg(en *msgp.Writer) (err error) {
return
}
}
if (zb0001Mask & 0x100) == 0 { // if not omitted
if (zb0001Mask & 0x80) == 0 { // if not omitted
// write "ats"
err = en.Append(0xa3, 0x61, 0x74, 0x73)
if err != nil {
@@ -2136,12 +1981,12 @@ func (z *dataUsageEntry) EncodeMsg(en *msgp.Writer) (err error) {
func (z *dataUsageEntry) MarshalMsg(b []byte) (o []byte, err error) {
o = msgp.Require(b, z.Msgsize())
// check for omitted fields
zb0001Len := uint32(10)
var zb0001Mask uint16 /* 10 bits */
zb0001Len := uint32(9)
var zb0001Mask uint16 /* 9 bits */
_ = zb0001Mask
if z.AllTierStats == nil {
zb0001Len--
zb0001Mask |= 0x100
zb0001Mask |= 0x80
}
// variable map header, size zb0001Len
o = append(o, 0x80|uint8(zb0001Len))
@@ -2158,9 +2003,6 @@ func (z *dataUsageEntry) MarshalMsg(b []byte) (o []byte, err error) {
// string "sz"
o = append(o, 0xa2, 0x73, 0x7a)
o = msgp.AppendInt64(o, z.Size)
// string "hts"
o = append(o, 0xa3, 0x68, 0x74, 0x73)
o = msgp.AppendInt64(o, z.HotTierSize)
// string "os"
o = append(o, 0xa2, 0x6f, 0x73)
o = msgp.AppendUint64(o, z.Objects)
@@ -2182,7 +2024,7 @@ func (z *dataUsageEntry) MarshalMsg(b []byte) (o []byte, err error) {
for za0002 := range z.ObjVersions {
o = msgp.AppendUint64(o, z.ObjVersions[za0002])
}
if (zb0001Mask & 0x100) == 0 { // if not omitted
if (zb0001Mask & 0x80) == 0 { // if not omitted
// string "ats"
o = append(o, 0xa3, 0x61, 0x74, 0x73)
if z.AllTierStats == nil {
@@ -2246,12 +2088,6 @@ func (z *dataUsageEntry) UnmarshalMsg(bts []byte) (o []byte, err error) {
err = msgp.WrapError(err, "Size")
return
}
case "hts":
z.HotTierSize, bts, err = msgp.ReadInt64Bytes(bts)
if err != nil {
err = msgp.WrapError(err, "HotTierSize")
return
}
case "os":
z.Objects, bts, err = msgp.ReadUint64Bytes(bts)
if err != nil {
@@ -2429,7 +2265,7 @@ func (z *dataUsageEntry) UnmarshalMsg(bts []byte) (o []byte, err error) {
// Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message
func (z *dataUsageEntry) Msgsize() (s int) {
s = 1 + 3 + z.Children.Msgsize() + 3 + msgp.Int64Size + 4 + msgp.Int64Size + 3 + msgp.Uint64Size + 3 + msgp.Uint64Size + 4 + msgp.Uint64Size + 4 + msgp.ArrayHeaderSize + (dataUsageBucketLen * (msgp.Uint64Size)) + 3 + msgp.ArrayHeaderSize + (dataUsageVersionLen * (msgp.Uint64Size)) + 4
s = 1 + 3 + z.Children.Msgsize() + 3 + msgp.Int64Size + 3 + msgp.Uint64Size + 3 + msgp.Uint64Size + 4 + msgp.Uint64Size + 4 + msgp.ArrayHeaderSize + (dataUsageBucketLen * (msgp.Uint64Size)) + 3 + msgp.ArrayHeaderSize + (dataUsageVersionLen * (msgp.Uint64Size)) + 4
if z.AllTierStats == nil {
s += msgp.NilSize
} else {
@@ -3422,438 +3258,6 @@ func (z *dataUsageEntryV7) Msgsize() (s int) {
return
}
// DecodeMsg implements msgp.Decodable
func (z *dataUsageEntryV8) DecodeMsg(dc *msgp.Reader) (err error) {
var field []byte
_ = field
var zb0001 uint32
zb0001, err = dc.ReadMapHeader()
if err != nil {
err = msgp.WrapError(err)
return
}
var zb0001Mask uint8 /* 1 bits */
_ = zb0001Mask
for zb0001 > 0 {
zb0001--
field, err = dc.ReadMapKeyPtr()
if err != nil {
err = msgp.WrapError(err)
return
}
switch msgp.UnsafeString(field) {
case "ch":
err = z.Children.DecodeMsg(dc)
if err != nil {
err = msgp.WrapError(err, "Children")
return
}
case "sz":
z.Size, err = dc.ReadInt64()
if err != nil {
err = msgp.WrapError(err, "Size")
return
}
case "os":
z.Objects, err = dc.ReadUint64()
if err != nil {
err = msgp.WrapError(err, "Objects")
return
}
case "vs":
z.Versions, err = dc.ReadUint64()
if err != nil {
err = msgp.WrapError(err, "Versions")
return
}
case "dms":
z.DeleteMarkers, err = dc.ReadUint64()
if err != nil {
err = msgp.WrapError(err, "DeleteMarkers")
return
}
case "szs":
var zb0002 uint32
zb0002, err = dc.ReadArrayHeader()
if err != nil {
err = msgp.WrapError(err, "ObjSizes")
return
}
if zb0002 != uint32(dataUsageBucketLen) {
err = msgp.ArrayError{Wanted: uint32(dataUsageBucketLen), Got: zb0002}
return
}
for za0001 := range z.ObjSizes {
z.ObjSizes[za0001], err = dc.ReadUint64()
if err != nil {
err = msgp.WrapError(err, "ObjSizes", za0001)
return
}
}
case "vh":
var zb0003 uint32
zb0003, err = dc.ReadArrayHeader()
if err != nil {
err = msgp.WrapError(err, "ObjVersions")
return
}
if zb0003 != uint32(dataUsageVersionLen) {
err = msgp.ArrayError{Wanted: uint32(dataUsageVersionLen), Got: zb0003}
return
}
for za0002 := range z.ObjVersions {
z.ObjVersions[za0002], err = dc.ReadUint64()
if err != nil {
err = msgp.WrapError(err, "ObjVersions", za0002)
return
}
}
case "ats":
if dc.IsNil() {
err = dc.ReadNil()
if err != nil {
err = msgp.WrapError(err, "AllTierStats")
return
}
z.AllTierStats = nil
} else {
if z.AllTierStats == nil {
z.AllTierStats = new(allTierStats)
}
var zb0004 uint32
zb0004, err = dc.ReadMapHeader()
if err != nil {
err = msgp.WrapError(err, "AllTierStats")
return
}
for zb0004 > 0 {
zb0004--
field, err = dc.ReadMapKeyPtr()
if err != nil {
err = msgp.WrapError(err, "AllTierStats")
return
}
switch msgp.UnsafeString(field) {
case "ts":
var zb0005 uint32
zb0005, err = dc.ReadMapHeader()
if err != nil {
err = msgp.WrapError(err, "AllTierStats", "Tiers")
return
}
if z.AllTierStats.Tiers == nil {
z.AllTierStats.Tiers = make(map[string]tierStats, zb0005)
} else if len(z.AllTierStats.Tiers) > 0 {
clear(z.AllTierStats.Tiers)
}
for zb0005 > 0 {
zb0005--
var za0003 string
za0003, err = dc.ReadString()
if err != nil {
err = msgp.WrapError(err, "AllTierStats", "Tiers")
return
}
var za0004 tierStats
var zb0006 uint32
zb0006, err = dc.ReadMapHeader()
if err != nil {
err = msgp.WrapError(err, "AllTierStats", "Tiers", za0003)
return
}
for zb0006 > 0 {
zb0006--
field, err = dc.ReadMapKeyPtr()
if err != nil {
err = msgp.WrapError(err, "AllTierStats", "Tiers", za0003)
return
}
switch msgp.UnsafeString(field) {
case "ts":
za0004.TotalSize, err = dc.ReadUint64()
if err != nil {
err = msgp.WrapError(err, "AllTierStats", "Tiers", za0003, "TotalSize")
return
}
case "nv":
za0004.NumVersions, err = dc.ReadInt()
if err != nil {
err = msgp.WrapError(err, "AllTierStats", "Tiers", za0003, "NumVersions")
return
}
case "no":
za0004.NumObjects, err = dc.ReadInt()
if err != nil {
err = msgp.WrapError(err, "AllTierStats", "Tiers", za0003, "NumObjects")
return
}
default:
err = dc.Skip()
if err != nil {
err = msgp.WrapError(err, "AllTierStats", "Tiers", za0003)
return
}
}
}
z.AllTierStats.Tiers[za0003] = za0004
}
default:
err = dc.Skip()
if err != nil {
err = msgp.WrapError(err, "AllTierStats")
return
}
}
}
}
zb0001Mask |= 0x1
case "c":
z.Compacted, err = dc.ReadBool()
if err != nil {
err = msgp.WrapError(err, "Compacted")
return
}
default:
err = dc.Skip()
if err != nil {
err = msgp.WrapError(err)
return
}
}
}
// Clear omitted fields.
if (zb0001Mask & 0x1) == 0 {
z.AllTierStats = nil
}
return
}
// UnmarshalMsg implements msgp.Unmarshaler
func (z *dataUsageEntryV8) UnmarshalMsg(bts []byte) (o []byte, err error) {
var field []byte
_ = field
var zb0001 uint32
zb0001, bts, err = msgp.ReadMapHeaderBytes(bts)
if err != nil {
err = msgp.WrapError(err)
return
}
var zb0001Mask uint8 /* 1 bits */
_ = zb0001Mask
for zb0001 > 0 {
zb0001--
field, bts, err = msgp.ReadMapKeyZC(bts)
if err != nil {
err = msgp.WrapError(err)
return
}
switch msgp.UnsafeString(field) {
case "ch":
bts, err = z.Children.UnmarshalMsg(bts)
if err != nil {
err = msgp.WrapError(err, "Children")
return
}
case "sz":
z.Size, bts, err = msgp.ReadInt64Bytes(bts)
if err != nil {
err = msgp.WrapError(err, "Size")
return
}
case "os":
z.Objects, bts, err = msgp.ReadUint64Bytes(bts)
if err != nil {
err = msgp.WrapError(err, "Objects")
return
}
case "vs":
z.Versions, bts, err = msgp.ReadUint64Bytes(bts)
if err != nil {
err = msgp.WrapError(err, "Versions")
return
}
case "dms":
z.DeleteMarkers, bts, err = msgp.ReadUint64Bytes(bts)
if err != nil {
err = msgp.WrapError(err, "DeleteMarkers")
return
}
case "szs":
var zb0002 uint32
zb0002, bts, err = msgp.ReadArrayHeaderBytes(bts)
if err != nil {
err = msgp.WrapError(err, "ObjSizes")
return
}
if zb0002 != uint32(dataUsageBucketLen) {
err = msgp.ArrayError{Wanted: uint32(dataUsageBucketLen), Got: zb0002}
return
}
for za0001 := range z.ObjSizes {
z.ObjSizes[za0001], bts, err = msgp.ReadUint64Bytes(bts)
if err != nil {
err = msgp.WrapError(err, "ObjSizes", za0001)
return
}
}
case "vh":
var zb0003 uint32
zb0003, bts, err = msgp.ReadArrayHeaderBytes(bts)
if err != nil {
err = msgp.WrapError(err, "ObjVersions")
return
}
if zb0003 != uint32(dataUsageVersionLen) {
err = msgp.ArrayError{Wanted: uint32(dataUsageVersionLen), Got: zb0003}
return
}
for za0002 := range z.ObjVersions {
z.ObjVersions[za0002], bts, err = msgp.ReadUint64Bytes(bts)
if err != nil {
err = msgp.WrapError(err, "ObjVersions", za0002)
return
}
}
case "ats":
if msgp.IsNil(bts) {
bts, err = msgp.ReadNilBytes(bts)
if err != nil {
return
}
z.AllTierStats = nil
} else {
if z.AllTierStats == nil {
z.AllTierStats = new(allTierStats)
}
var zb0004 uint32
zb0004, bts, err = msgp.ReadMapHeaderBytes(bts)
if err != nil {
err = msgp.WrapError(err, "AllTierStats")
return
}
for zb0004 > 0 {
zb0004--
field, bts, err = msgp.ReadMapKeyZC(bts)
if err != nil {
err = msgp.WrapError(err, "AllTierStats")
return
}
switch msgp.UnsafeString(field) {
case "ts":
var zb0005 uint32
zb0005, bts, err = msgp.ReadMapHeaderBytes(bts)
if err != nil {
err = msgp.WrapError(err, "AllTierStats", "Tiers")
return
}
if z.AllTierStats.Tiers == nil {
z.AllTierStats.Tiers = make(map[string]tierStats, zb0005)
} else if len(z.AllTierStats.Tiers) > 0 {
clear(z.AllTierStats.Tiers)
}
for zb0005 > 0 {
var za0004 tierStats
zb0005--
var za0003 string
za0003, bts, err = msgp.ReadStringBytes(bts)
if err != nil {
err = msgp.WrapError(err, "AllTierStats", "Tiers")
return
}
var zb0006 uint32
zb0006, bts, err = msgp.ReadMapHeaderBytes(bts)
if err != nil {
err = msgp.WrapError(err, "AllTierStats", "Tiers", za0003)
return
}
for zb0006 > 0 {
zb0006--
field, bts, err = msgp.ReadMapKeyZC(bts)
if err != nil {
err = msgp.WrapError(err, "AllTierStats", "Tiers", za0003)
return
}
switch msgp.UnsafeString(field) {
case "ts":
za0004.TotalSize, bts, err = msgp.ReadUint64Bytes(bts)
if err != nil {
err = msgp.WrapError(err, "AllTierStats", "Tiers", za0003, "TotalSize")
return
}
case "nv":
za0004.NumVersions, bts, err = msgp.ReadIntBytes(bts)
if err != nil {
err = msgp.WrapError(err, "AllTierStats", "Tiers", za0003, "NumVersions")
return
}
case "no":
za0004.NumObjects, bts, err = msgp.ReadIntBytes(bts)
if err != nil {
err = msgp.WrapError(err, "AllTierStats", "Tiers", za0003, "NumObjects")
return
}
default:
bts, err = msgp.Skip(bts)
if err != nil {
err = msgp.WrapError(err, "AllTierStats", "Tiers", za0003)
return
}
}
}
z.AllTierStats.Tiers[za0003] = za0004
}
default:
bts, err = msgp.Skip(bts)
if err != nil {
err = msgp.WrapError(err, "AllTierStats")
return
}
}
}
}
zb0001Mask |= 0x1
case "c":
z.Compacted, bts, err = msgp.ReadBoolBytes(bts)
if err != nil {
err = msgp.WrapError(err, "Compacted")
return
}
default:
bts, err = msgp.Skip(bts)
if err != nil {
err = msgp.WrapError(err)
return
}
}
}
// Clear omitted fields.
if (zb0001Mask & 0x1) == 0 {
z.AllTierStats = nil
}
o = bts
return
}
// Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message
func (z *dataUsageEntryV8) Msgsize() (s int) {
s = 1 + 3 + z.Children.Msgsize() + 3 + msgp.Int64Size + 3 + msgp.Uint64Size + 3 + msgp.Uint64Size + 4 + msgp.Uint64Size + 4 + msgp.ArrayHeaderSize + (dataUsageBucketLen * (msgp.Uint64Size)) + 3 + msgp.ArrayHeaderSize + (dataUsageVersionLen * (msgp.Uint64Size)) + 4
if z.AllTierStats == nil {
s += msgp.NilSize
} else {
s += 1 + 3 + msgp.MapHeaderSize
if z.AllTierStats.Tiers != nil {
for za0003, za0004 := range z.AllTierStats.Tiers {
_ = za0004
s += msgp.StringPrefixSize + len(za0003) + 1 + 3 + msgp.Uint64Size + 3 + msgp.IntSize + 3 + msgp.IntSize
}
}
}
s += 2 + msgp.BoolSize
return
}
// DecodeMsg implements msgp.Decodable
func (z *dataUsageHash) DecodeMsg(dc *msgp.Reader) (err error) {
{
cmd/data-usage-utils.go
+1 -6
View File
@@ -46,8 +46,7 @@ type BucketTargetUsageInfo struct {
// - total objects in a bucket
// - object size histogram per bucket
type BucketUsageInfo struct {
Size uint64 `json:"size"`
HotTierSize uint64 `json:"hotTierSize,omitempty"`
Size uint64 `json:"size"`
// Following five fields suffixed with V1 are here for backward compatibility
// Total Size for objects that have not yet been replicated
ReplicationPendingSizeV1 uint64 `json:"objectsPendingReplicationTotalSize"`
@@ -79,10 +78,6 @@ type DataUsageInfo struct {
// LastUpdate is the timestamp of when the data usage info was last updated.
// This does not indicate a full scan.
LastUpdate time.Time `json:"lastUpdate"`
// ScannerCycle changes only after a complete scanner pass. Background
// consumers use it to distinguish a partial cache update from a baseline
// that has visited every bucket and server pool.
ScannerCycle uint32 `json:"scannerCycle,omitempty"`
// Objects total count across all buckets
ObjectsTotalCount uint64 `json:"objectsCount"`
cmd/erasure-server-pool-consistency.go
+27 -9
View File
@@ -287,33 +287,51 @@ func (z *erasureServerPools) retireReplicaCopies(ctx context.Context, bucket, ob
return nil
}
// deleteObjectConditional evaluates the condition once against the logical
// deleteObjectReconciled evaluates any condition once against the logical
// version, then removes all its copies under the same lock as pooled writers.
func (z *erasureServerPools) deleteObjectConditional(ctx context.Context, bucket, object string, opts ObjectOptions) (ObjectInfo, error) {
func (z *erasureServerPools) deleteObjectReconciled(ctx context.Context, bucket, object string, opts ObjectOptions) (ObjectInfo, error) {
copies, err := z.objectPoolInfos(ctx, bucket, object, opts)
if err != nil {
return ObjectInfo{}, err
}
primary := copies[0]
if opts.CheckPrecondFn(primary.ObjInfo) {
if opts.CheckPrecondFn != nil && opts.CheckPrecondFn(primary.ObjInfo) {
return ObjectInfo{}, PreConditionFailed{}
}
opts.CheckPrecondFn = nil
opts.NoLock = true
if opts.EvalRetentionBypassFn != nil || opts.EvalMetadataFn != nil {
versions, err := z.metadataPoolInfos(ctx, bucket, object, opts)
if err != nil {
return ObjectInfo{}, err
logical := primary.ObjInfo
var gerr error
switch {
case logical.DeleteMarker:
// Markers can be deleted by version ID. Match the set layer's
// callback inputs instead of rejecting them as metadata updates.
gerr = toObjectErr(errMethodNotAllowed, bucket, object)
if opts.VersionID == "" || opts.DeleteMarker {
gerr = toObjectErr(errFileNotFound, bucket, object)
}
case opts.VersionID != "":
// An addressed version already resolved every copy above.
logical = mergedPoolObjectInfo(copies)
default:
versions, err := z.metadataPoolInfos(ctx, bucket, object, opts)
if err != nil {
return ObjectInfo{}, err
}
logical = mergedPoolObjectInfo(versions)
}
logical := mergedPoolObjectInfo(versions)
// Keep the retention gate first. These callbacks independently evaluate
// the logical version and run once before any deletion; the handler only
// sweeps metadata's transition state after a successful delete.
if opts.EvalRetentionBypassFn != nil {
if err := opts.EvalRetentionBypassFn(logical, nil); err != nil {
if err := opts.EvalRetentionBypassFn(logical, gerr); err != nil {
return ObjectInfo{}, err
}
opts.EvalRetentionBypassFn = nil
}
if opts.EvalMetadataFn != nil {
decision, err := opts.EvalMetadataFn(&logical, nil)
decision, err := opts.EvalMetadataFn(&logical, gerr)
if err != nil {
return ObjectInfo{}, err
}
cmd/erasure-server-pool-consistency_test.go
+634 -8
View File
@@ -24,12 +24,16 @@ import (
"fmt"
"io"
"maps"
"net/http"
"net/http/httptest"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
madmin "github.com/minio/madmin-go/v3"
"github.com/minio/minio/internal/bucket/replication"
xhttp "github.com/minio/minio/internal/http"
)
@@ -62,6 +66,490 @@ func putConsistencyObject(t *testing.T, z *erasureServerPools, bucket, object st
return oi
}
func consistencyRequest(t *testing.T, router http.Handler, method, bucket, object, version string) *httptest.ResponseRecorder {
t.Helper()
url := getGetObjectURL("", bucket, object)
if version != "" {
url += "?versionId=" + version
}
req, err := newTestSignedRequestV4(method, url, 0, nil, globalActiveCred.AccessKey, globalActiveCred.SecretKey, nil)
if err != nil {
t.Fatal(err)
}
rec := httptest.NewRecorder()
router.ServeHTTP(rec, req)
return rec
}
// Exercise the handler's real replication/retention callbacks, including the
// MethodNotAllowed metadata returned for an explicitly addressed delete marker.
func TestPoolsDeleteVersionAPI(t *testing.T) {
z, _ := consistencyPools(t)
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
bucket, router, err := initAPIHandlerTest(ctx, z, nil, MakeBucketOptions{VersioningEnabled: true})
if err != nil {
t.Fatal(err)
}
for _, kind := range []string{"uuid", "null", "marker"} {
for primary := range 2 {
t.Run(fmt.Sprintf("%s/primary=%d", kind, primary), func(t *testing.T) {
object := fmt.Sprintf("%s-%d", kind, primary)
opts := ObjectOptions{Versioned: kind != "null", MTime: UTCNow().Add(-time.Hour)}
var addressed ObjectInfo
if kind == "marker" {
opts.VersionID, opts.DeleteMarker = mustGetUUID(), true
addressed, err = z.serverPools[primary].DeleteObject(ctx, bucket, object, opts)
if err != nil {
t.Fatal(err)
}
} else {
addressed = putConsistencyObject(t, z, bucket, object, primary, "payload", opts)
}
version := addressed.VersionID
if version == "" {
version = nullVersionID
}
opts.VersionID = version
opts.MTime = addressed.ModTime.Add(-time.Minute)
if kind == "marker" {
opts.DeleteMarker = true
if _, err := z.serverPools[1-primary].DeleteObject(ctx, bucket, object, opts); err != nil {
t.Fatal(err)
}
} else {
putConsistencyObject(t, z, bucket, object, 1-primary, "payload", opts)
}
latest := putConsistencyObject(t, z, bucket, object, 1-primary, "keep-latest", ObjectOptions{Versioned: true})
rec := consistencyRequest(t, router, http.MethodDelete, bucket, object, version)
if rec.Code != http.StatusNoContent {
t.Fatalf("DELETE: %d %s", rec.Code, rec.Body.String())
}
if kind == "marker" && (strings.Join(rec.Header()[xhttp.AmzDeleteMarker], "") != "true" || strings.Join(rec.Header()[xhttp.AmzVersionID], "") != version) {
t.Errorf("lost deleted marker response headers: %v", rec.Header())
}
for i, pool := range z.serverPools {
if _, err := pool.GetObjectInfo(ctx, bucket, object, ObjectOptions{VersionID: version}); !isErrVersionNotFound(err) {
t.Errorf("pool %d retained addressed version: %v", i, err)
}
}
for _, method := range []string{http.MethodGet, http.MethodHead} {
if rec := consistencyRequest(t, router, method, bucket, object, version); rec.Code != http.StatusNotFound {
t.Errorf("%s after DELETE: %d %s", method, rec.Code, rec.Body.String())
}
}
if got, err := z.GetObjectInfo(ctx, bucket, object, ObjectOptions{}); err != nil || got.VersionID != latest.VersionID {
t.Errorf("DELETE changed another version: %+v, %v", got, err)
}
if rec := consistencyRequest(t, router, http.MethodDelete, bucket, object, version); rec.Code != http.StatusNoContent {
t.Errorf("idempotent retry: %d %s", rec.Code, rec.Body.String())
}
})
}
}
if rec := consistencyRequest(t, router, http.MethodDelete, bucket, "absent-key", mustGetUUID()); rec.Code != http.StatusNoContent {
t.Errorf("absent key: %d %s", rec.Code, rec.Body.String())
}
}
type consistencyReadFaultDisk struct {
StorageAPI
bucket, object string
}
func (d consistencyReadFaultDisk) ReadVersion(ctx context.Context, origvolume, volume, path, version string, opts ReadOptions) (FileInfo, error) {
if volume == d.bucket && path == d.object {
return FileInfo{}, errDiskNotFound
}
return d.StorageAPI.ReadVersion(ctx, origvolume, volume, path, version, opts)
}
func (d consistencyReadFaultDisk) ReadXL(ctx context.Context, volume, path string, readData bool) (RawFileInfo, error) {
if volume == d.bucket && path == d.object {
return RawFileInfo{}, errDiskNotFound
}
return d.StorageAPI.ReadXL(ctx, volume, path, readData)
}
func TestPoolsDeleteVersionUnreadablePool(t *testing.T) {
z, _ := consistencyPools(t)
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
bucket, router, err := initAPIHandlerTest(ctx, z, nil, MakeBucketOptions{VersioningEnabled: true})
if err != nil {
t.Fatal(err)
}
for _, holding := range []bool{false, true} {
for failed := range 2 {
t.Run(fmt.Sprintf("holding=%t/pool=%d", holding, failed), func(t *testing.T) {
object := fmt.Sprintf("unreadable-%t-%d", holding, failed)
oi := putConsistencyObject(t, z, bucket, object, 1-failed, "payload", ObjectOptions{Versioned: true})
if holding {
putConsistencyObject(t, z, bucket, object, failed, "payload", ObjectOptions{Versioned: true, VersionID: oi.VersionID, MTime: oi.ModTime})
}
set := z.serverPools[failed].getHashedSet(object)
getDisks := set.getDisks
faulty := append([]StorageAPI(nil), getDisks()...)
for i := range faulty {
faulty[i] = consistencyReadFaultDisk{StorageAPI: faulty[i], bucket: bucket, object: object}
}
set.getDisks = func() []StorageAPI { return faulty }
defer func() { set.getDisks = getDisks }()
if rec := consistencyRequest(t, router, http.MethodDelete, bucket, object, oi.VersionID); rec.Code != http.StatusServiceUnavailable {
t.Errorf("unreadable pool DELETE: %d %s", rec.Code, rec.Body.String())
}
if _, err := z.serverPools[1-failed].GetObjectInfo(ctx, bucket, object, ObjectOptions{VersionID: oi.VersionID}); err != nil {
t.Errorf("DELETE lost the readable copy: %v", err)
}
set.getDisks = getDisks
if rec := consistencyRequest(t, router, http.MethodDelete, bucket, object, oi.VersionID); rec.Code != http.StatusNoContent {
t.Errorf("recovered pool retry: %d %s", rec.Code, rec.Body.String())
}
for i, pool := range z.serverPools {
if _, err := pool.GetObjectInfo(ctx, bucket, object, ObjectOptions{VersionID: oi.VersionID}); !isErrVersionNotFound(err) {
t.Errorf("pool %d retained version after retry: %v", i, err)
}
}
})
}
}
}
type consistencyDeleteCountDisk struct {
StorageAPI
bucket, object string
metadataReads, deletes *atomic.Int32
}
func (d consistencyDeleteCountDisk) ReadVersion(ctx context.Context, origvolume, volume, path, version string, opts ReadOptions) (FileInfo, error) {
if volume == d.bucket && path == d.object {
d.metadataReads.Add(1)
}
return d.StorageAPI.ReadVersion(ctx, origvolume, volume, path, version, opts)
}
func (d consistencyDeleteCountDisk) DeleteVersion(ctx context.Context, volume, path string, fi FileInfo, force bool, opts DeleteOptions) error {
if volume == d.bucket && path == d.object {
d.deletes.Add(1)
}
return d.StorageAPI.DeleteVersion(ctx, volume, path, fi, force, opts)
}
func TestPoolsDeleteVersionSingleCopy(t *testing.T) {
z, bucket := consistencyPools(t)
for primary := range 2 {
t.Run(fmt.Sprintf("pool=%d", primary), func(t *testing.T) {
object := fmt.Sprintf("single-copy-%d", primary)
oi := putConsistencyObject(t, z, bucket, object, primary, "payload", ObjectOptions{Versioned: true})
var metadataReads, deletes [2]atomic.Int32
for i, pool := range z.serverPools {
set := pool.getHashedSet(object)
getDisks := set.getDisks
disks := append([]StorageAPI(nil), getDisks()...)
for j := range disks {
disks[j] = consistencyDeleteCountDisk{StorageAPI: disks[j], bucket: bucket, object: object, metadataReads: &metadataReads[i], deletes: &deletes[i]}
}
set.getDisks = func() []StorageAPI { return disks }
defer func() { set.getDisks = getDisks }()
}
metadata, retention := 0, 0
_, err := z.DeleteObject(t.Context(), bucket, object, ObjectOptions{
Versioned: true, VersionID: oi.VersionID,
EvalMetadataFn: func(current *ObjectInfo, err error) (ReplicateDecision, error) {
metadata++
if err != nil || current.VersionID != oi.VersionID {
t.Errorf("metadata callback: %+v, %v", current, err)
}
// Count preflight reads before the physical delete reads its own metadata.
for i := range metadataReads {
if got := metadataReads[i].Load(); got != 16 {
t.Errorf("pool %d metadata reads before callbacks = %d; want once per disk (16)", i, got)
}
}
return ReplicateDecision{}, nil
},
EvalRetentionBypassFn: func(current ObjectInfo, err error) error {
retention++
return err
},
})
if err != nil || metadata != 1 || retention != 1 {
t.Fatalf("DELETE: %v, metadata=%d retention=%d", err, metadata, retention)
}
if deletes[primary].Load() != 16 || deletes[1-primary].Load() != 0 {
t.Errorf("physical deletes per pool = %d, %d; want once per holding disk only", deletes[0].Load(), deletes[1].Load())
}
})
}
}
func TestPoolsDeleteVersionReplicationPurge(t *testing.T) {
z, bucket := consistencyPools(t)
for _, marker := range []bool{false, true} {
t.Run(fmt.Sprintf("marker=%t", marker), func(t *testing.T) {
object := fmt.Sprintf("local-replication-purge-%t", marker)
version := mustGetUUID()
for _, pool := range z.serverPools {
opts := ObjectOptions{Versioned: true, VersionID: version, DeleteMarker: marker}
var err error
if marker {
_, err = pool.DeleteObject(t.Context(), bucket, object, opts)
} else {
_, err = pool.PutObject(t.Context(), bucket, object, mustGetPutObjReader(t, strings.NewReader("payload"), 7, "", ""), opts)
}
if err != nil {
t.Fatal(err)
}
}
dsc := ReplicateDecision{}
dsc.Set(newReplicateTargetDecision("arn1", true, false))
opts := ObjectOptions{Versioned: true, VersionID: version}
for attempt := range 2 {
calls := 0
opts.EvalMetadataFn = func(current *ObjectInfo, gerr error) (ReplicateDecision, error) {
calls++
wantMethodNotAllowed := marker || attempt > 0
if (wantMethodNotAllowed && !isErrMethodNotAllowed(gerr)) || (!wantMethodNotAllowed && gerr != nil) {
t.Errorf("pending purge callback lost set-layer read error: %v", gerr)
}
return dsc, nil
}
got, err := z.DeleteObject(t.Context(), bucket, object, opts)
if err != nil || calls != 1 || got.VersionPurgeStatus != replication.VersionPurgePending || got.replicationDecision != dsc.String() {
t.Fatalf("replication scheduling result: %+v, %v, calls=%d", got, err, calls)
}
for i, pool := range z.serverPools {
got, err := pool.GetObjectInfo(t.Context(), bucket, object, ObjectOptions{VersionID: version})
if (err != nil && (!got.DeleteMarker || !isErrMethodNotAllowed(err))) || got.VersionPurgeStatus != replication.VersionPurgePending {
t.Errorf("pool %d lost pending purge: %+v, %v", i, got, err)
}
}
}
// The local worker completes the purge without ReplicationRequest.
// Both pending copies must be removed, including a pending marker purge.
opts.EvalMetadataFn = nil
opts.DeleteReplication = ReplicationState{
VersionPurgeStatusInternal: "arn1=COMPLETED;",
PurgeTargets: map[string]VersionPurgeStatusType{"arn1": replication.VersionPurgeComplete},
}
if _, err := z.DeleteObject(t.Context(), bucket, object, opts); err != nil {
t.Fatal(err)
}
for i, pool := range z.serverPools {
if _, err := pool.GetObjectInfo(t.Context(), bucket, object, ObjectOptions{VersionID: version}); !isErrVersionNotFound(err) {
t.Errorf("pool %d retained completed replication purge: %v", i, err)
}
}
})
}
}
func TestPoolsDeleteVersionCleanupFailure(t *testing.T) {
z, bucket := consistencyPools(t)
for primary := range 2 {
t.Run(fmt.Sprintf("primary=%d", primary), func(t *testing.T) {
object := fmt.Sprintf("cleanup-failure-%d", primary)
oi := putConsistencyObject(t, z, bucket, object, primary, "payload", ObjectOptions{Versioned: true})
putConsistencyObject(t, z, bucket, object, 1-primary, "payload", ObjectOptions{
Versioned: true, VersionID: oi.VersionID, MTime: oi.ModTime.Add(-time.Minute),
})
set := z.serverPools[1-primary].getHashedSet(object)
getDisks := set.getDisks
faulty := append([]StorageAPI(nil), getDisks()...)
for i := range faulty {
faulty[i] = consistencyDeleteFaultDisk{StorageAPI: faulty[i], bucket: bucket, object: object, version: oi.VersionID}
}
set.getDisks = func() []StorageAPI { return faulty }
defer func() { set.getDisks = getDisks }()
opts := ObjectOptions{Versioned: true, VersionID: oi.VersionID}
if _, err := z.DeleteObject(t.Context(), bucket, object, opts); err == nil {
t.Error("acknowledged DELETE despite failed secondary cleanup")
}
if got, err := z.serverPools[primary].GetObjectInfo(t.Context(), bucket, object, opts); err != nil || got.ETag != oi.ETag {
t.Errorf("lost authoritative copy after secondary failure: %+v, %v", got, err)
}
set.getDisks = getDisks
if _, err := z.DeleteObject(t.Context(), bucket, object, opts); err != nil {
t.Fatal(err)
}
for i, pool := range z.serverPools {
if _, err := pool.GetObjectInfo(t.Context(), bucket, object, opts); !isErrVersionNotFound(err) {
t.Errorf("pool %d retained version after retry: %v", i, err)
}
}
})
}
}
func TestPoolsDeleteVersionCallbacks(t *testing.T) {
z, bucket := consistencyPools(t)
for _, marker := range []bool{false, true} {
t.Run(fmt.Sprintf("marker=%t", marker), func(t *testing.T) {
object := fmt.Sprintf("callbacks-%t", marker)
old, recent := "2026-09-09T09:00:00Z", "2026-09-09T10:00:00Z"
oi := putConsistencyObject(t, z, bucket, object, 0, "payload", ObjectOptions{
Versioned: true, UserDefined: poolLockMetadata("GOVERNANCE", "OFF", recent, old),
})
putConsistencyObject(t, z, bucket, object, 1, "payload", ObjectOptions{
Versioned: true, VersionID: oi.VersionID, MTime: oi.ModTime,
UserDefined: poolLockMetadata("", "ON", old, recent),
})
if marker {
markerOpts := ObjectOptions{Versioned: true, VersionID: mustGetUUID(), DeleteMarker: true, MTime: UTCNow()}
for _, pool := range z.serverPools {
var err error
oi, err = pool.DeleteObject(t.Context(), bucket, object, markerOpts)
if err != nil {
t.Fatal(err)
}
}
}
check := func(current ObjectInfo, err error) {
t.Helper()
if current.VersionID != oi.VersionID || current.DeleteMarker != marker || (marker && !isErrMethodNotAllowed(err)) || (!marker && err != nil) {
t.Errorf("wrong callback version or read error: %+v, %v", current, err)
}
if !marker {
state := storedObjectLockState(current.UserDefined)
if state.mode != "GOVERNANCE" || state.legalHold != "ON" {
t.Errorf("callback did not reconcile independently ordered lock state: %+v", state)
}
}
}
for _, reject := range []string{"retention", "metadata", "none"} {
metadata, retention := 0, 0
denied := errors.New("callback denied deletion")
opts := ObjectOptions{
Versioned: true, VersionID: oi.VersionID,
EvalRetentionBypassFn: func(current ObjectInfo, err error) error {
retention++
check(current, err)
if reject == "retention" {
return denied
}
return nil
},
EvalMetadataFn: func(current *ObjectInfo, err error) (ReplicateDecision, error) {
metadata++
check(*current, err)
if reject == "metadata" {
return ReplicateDecision{}, denied
}
return ReplicateDecision{}, nil
},
}
_, err := z.DeleteObject(t.Context(), bucket, object, opts)
if reject == "none" {
if err != nil || metadata != 1 || retention != 1 {
t.Fatalf("DELETE: %v, metadata=%d retention=%d", err, metadata, retention)
}
} else if !errors.Is(err, denied) {
t.Fatalf("%s rejection was lost: %v", reject, err)
}
for i, pool := range z.serverPools {
current, err := pool.GetObjectInfo(t.Context(), bucket, object, ObjectOptions{VersionID: oi.VersionID})
if reject == "none" {
if !isErrVersionNotFound(err) {
t.Errorf("pool %d retained deleted version: %v", i, err)
}
} else if current.VersionID != oi.VersionID || (err != nil && (!marker || !isErrMethodNotAllowed(err))) {
t.Errorf("pool %d changed despite %s rejection: %+v, %v", i, reject, current, err)
}
}
}
})
}
}
func TestPoolsDeleteVersionSpecialCalls(t *testing.T) {
z, bucket := consistencyPools(t)
t.Run("incoming-marker-replication", func(t *testing.T) {
const object = "incoming-marker"
opts := ObjectOptions{Versioned: true, VersionID: mustGetUUID(), DeleteMarker: true, ReplicationRequest: true}
opts.SetReplicaStatus(replication.Replica)
if _, err := z.DeleteObject(t.Context(), bucket, object, opts); err != nil {
t.Fatalf("replication could not create an absent marker: %v", err)
}
if got, err := z.GetObjectInfo(t.Context(), bucket, object, ObjectOptions{VersionID: opts.VersionID}); !isErrMethodNotAllowed(err) || !got.DeleteMarker {
t.Fatalf("replicated marker missing: %+v, %v", got, err)
}
})
for _, rebalance := range []bool{false, true} {
t.Run(fmt.Sprintf("movement/rebalance=%t", rebalance), func(t *testing.T) {
object := fmt.Sprintf("movement-%t", rebalance)
putConsistencyObject(t, z, bucket, object, 1, "destination", ObjectOptions{Versioned: true})
opts := ObjectOptions{Versioned: true, VersionID: mustGetUUID(), DeleteMarker: true, MTime: UTCNow()}
if _, err := z.serverPools[0].DeleteObject(t.Context(), bucket, object, opts); err != nil {
t.Fatal(err)
}
if rebalance {
z.rebalMu.Lock()
z.rebalMeta = &rebalanceMeta{PoolStats: []*rebalanceStats{{Participating: true, Info: rebalanceInfo{Status: rebalStarted}}, {}}}
z.rebalMu.Unlock()
defer func() { z.rebalMu.Lock(); z.rebalMeta = nil; z.rebalMu.Unlock() }()
} else {
z.poolMetaMutex.Lock()
z.poolMeta.Pools[0].Decommission = &PoolDecommissionInfo{}
z.poolMetaMutex.Unlock()
defer func() { z.poolMetaMutex.Lock(); z.poolMeta.Pools[0].Decommission = nil; z.poolMetaMutex.Unlock() }()
}
// These are the marker-copy options used by rebalance/decommission;
// Source cleanup belongs to the mover.
opts.DataMovement, opts.SrcPoolIdx = true, 0
opts.SkipRebalancing, opts.SkipDecommissioned = rebalance, !rebalance
if _, err := z.DeleteObject(t.Context(), bucket, object, opts); err != nil {
t.Fatal(err)
}
for i, pool := range z.serverPools {
if got, err := pool.GetObjectInfo(t.Context(), bucket, object, ObjectOptions{VersionID: opts.VersionID}); !isErrMethodNotAllowed(err) || !got.DeleteMarker {
t.Errorf("movement lost marker in pool %d: %+v, %v", i, got, err)
}
}
})
}
for _, expiration := range []bool{false, true} {
t.Run(fmt.Sprintf("scanner/expiration=%t", expiration), func(t *testing.T) {
object := fmt.Sprintf("scanner-%t", expiration)
oi := putConsistencyObject(t, z, bucket, object, 0, "payload", ObjectOptions{Versioned: true})
set := z.serverPools[1].getHashedSet(object)
getDisks := set.getDisks
faulty := append([]StorageAPI(nil), getDisks()...)
for i := range faulty {
faulty[i] = consistencyReadFaultDisk{StorageAPI: faulty[i], bucket: bucket, object: object}
}
set.getDisks = func() []StorageAPI { return faulty }
defer func() { set.getDisks = getDisks }()
opts := ObjectOptions{Versioned: true, VersionID: oi.VersionID, InclFreeVersions: !expiration, Expiration: ExpirationOptions{Expire: expiration}}
_, err := z.DeleteObject(t.Context(), bucket, object, opts)
if expiration {
// No lifecycle rule authorizes expiration. Preserve its existing
// version-not-found result, even with an unrelated unreadable pool.
if !isErrVersionNotFound(err) {
t.Errorf("expiration changed its scanner contract: %v", err)
}
} else if err != nil {
t.Errorf("free-version cleanup was forced through all-pool resolution: %v", err)
}
})
}
}
func TestPoolsDeleteUnversionedFanout(t *testing.T) {
z, bucket := consistencyPools(t)
const object = "unversioned-fanout"
for i := range z.serverPools {
putConsistencyObject(t, z, bucket, object, i, "payload", ObjectOptions{})
}
if _, err := z.DeleteObject(t.Context(), bucket, object, ObjectOptions{}); err != nil {
t.Fatal(err)
}
for i, pool := range z.serverPools {
if _, err := pool.GetObjectInfo(t.Context(), bucket, object, ObjectOptions{}); !isErrObjectNotFound(err) {
t.Errorf("unversioned fanout left pool %d readable: %v", i, err)
}
}
}
func TestPoolsConditionalDeleteVersionSelection(t *testing.T) {
z, bucket := consistencyPools(t)
const object = "split-versions"
@@ -108,7 +596,7 @@ func TestPoolsConditionalDeleteReportsOtherPoolFailure(t *testing.T) {
getDisks := set.getDisks
faulty := append([]StorageAPI(nil), getDisks()...)
for i := range faulty {
faulty[i] = accessMoveDeleteFaultDisk{StorageAPI: faulty[i], bucket: bucket, object: object}
faulty[i] = consistencyDeleteFaultDisk{StorageAPI: faulty[i], bucket: bucket, object: object}
}
set.getDisks = func() []StorageAPI { return faulty }
defer func() { set.getDisks = getDisks }()
@@ -135,9 +623,8 @@ func testPoolsConditionalDeleteWriter(t *testing.T, multipart bool) {
ctx, cancel := context.WithTimeout(t.Context(), 10*time.Second)
defer cancel()
reader := mustGetPutObjReader(t, bytes.NewBufferString("after"), 5, "", "")
destination := 1
write := func() error {
_, err := z.PutObject(ctx, bucket, object, reader, ObjectOptions{DataMovement: true, SrcPoolIdx: 0, DstPoolIdx: &destination})
_, err := z.PutObject(ctx, bucket, object, reader, ObjectOptions{})
return err
}
if multipart {
@@ -548,7 +1035,7 @@ func TestPoolsReplicaCleanupFailureCanRetry(t *testing.T) {
getDisks := set.getDisks
faulty := append([]StorageAPI(nil), getDisks()...)
for i := range faulty {
faulty[i] = accessMoveDeleteFaultDisk{StorageAPI: faulty[i], bucket: bucket, object: object, version: oi.VersionID}
faulty[i] = consistencyDeleteFaultDisk{StorageAPI: faulty[i], bucket: bucket, object: object, version: oi.VersionID}
}
set.getDisks = func() []StorageAPI { return faulty }
defer func() { set.getDisks = getDisks }()
@@ -582,6 +1069,8 @@ func TestPoolsRetiringCopyPreservesSharedTierObject(t *testing.T) {
failPrimaryDelete bool
differentRemote bool
restored bool
unconditional bool
skipFreeVersion bool
}{
{name: "metadata-copy"},
{name: "restored-metadata-copy", restored: true},
@@ -589,6 +1078,9 @@ func TestPoolsRetiringCopyPreservesSharedTierObject(t *testing.T) {
{name: "failed-primary-delete", deleting: true, failPrimaryDelete: true},
{name: "failed-primary-delete-distinct-reference", deleting: true, failPrimaryDelete: true, differentRemote: true},
{name: "successful-delete", deleting: true},
{name: "ordinary-failed-primary-delete", deleting: true, unconditional: true, failPrimaryDelete: true},
{name: "ordinary-successful-delete", deleting: true, unconditional: true},
{name: "ordinary-skip-free-version", deleting: true, unconditional: true, skipFreeVersion: true},
} {
t.Run(test.name, func(t *testing.T) {
z, bucket := consistencyPools(t)
@@ -622,16 +1114,20 @@ func TestPoolsRetiringCopyPreservesSharedTierObject(t *testing.T) {
getDisks := set.getDisks
faulty := append([]StorageAPI(nil), getDisks()...)
for i := range faulty {
faulty[i] = accessMoveDeleteFaultDisk{StorageAPI: faulty[i], bucket: bucket, object: object, version: oi.VersionID}
faulty[i] = consistencyDeleteFaultDisk{StorageAPI: faulty[i], bucket: bucket, object: object, version: oi.VersionID}
}
set.getDisks = func() []StorageAPI { return faulty }
defer func() { set.getDisks = getDisks }()
}
if test.deleting {
_, err := z.DeleteObject(t.Context(), bucket, object, ObjectOptions{
Versioned: true, VersionID: oi.VersionID,
opts := ObjectOptions{
Versioned: true, VersionID: oi.VersionID, SkipFreeVersion: test.skipFreeVersion,
CheckPrecondFn: func(info ObjectInfo) bool { return info.ETag != current.ETag },
})
}
if test.unconditional {
opts.CheckPrecondFn = nil
}
_, err := z.DeleteObject(t.Context(), bucket, object, opts)
if (err != nil) != test.failPrimaryDelete {
t.Fatalf("unexpected authoritative delete result: %v", err)
}
@@ -656,6 +1152,7 @@ func TestPoolsRetiringCopyPreservesSharedTierObject(t *testing.T) {
t.Fatalf("lost retained authoritative copy: %v", err)
}
for pool, wantFree := range []bool{primaryDeleted, test.differentRemote} {
wantFree = wantFree && !test.skipFreeVersion
for _, disk := range z.serverPools[pool].getHashedSet(object).getDisks() {
data, err := disk.ReadAll(t.Context(), bucket, pathJoin(object, xlStorageFormatFile))
if errors.Is(err, errFileNotFound) && !wantFree {
@@ -680,3 +1177,132 @@ func TestPoolsRetiringCopyPreservesSharedTierObject(t *testing.T) {
})
}
}
// Fault injection shared by general multi-pool regressions.
type consistencyDeleteFaultDisk struct {
StorageAPI
bucket, object, version string
}
func (d consistencyDeleteFaultDisk) DeleteVersion(ctx context.Context, volume, path string, fi FileInfo, forceDelMarker bool, opts DeleteOptions) error {
if volume == d.bucket && path == d.object && fi.VersionID == d.version {
return errDiskFull
}
return d.StorageAPI.DeleteVersion(ctx, volume, path, fi, forceDelMarker, opts)
}
// Exercise the surviving production rebalance copy, then interrupt the workflow
// before its later source cleanup. Repeated versions are reachable without the
// retired access-tier mover, and an API delete must remove both copies.
func TestPoolsDeleteVersionAfterInterruptedRebalance(t *testing.T) {
z, _ := consistencyPools(t)
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
bucket, router, err := initAPIHandlerTest(ctx, z, nil, MakeBucketOptions{VersioningEnabled: true})
if err != nil {
t.Fatal(err)
}
for source := range 2 {
t.Run(fmt.Sprintf("source=%d", source), func(t *testing.T) {
object := fmt.Sprintf("interrupted-rebalance-%d", source)
original := putConsistencyObject(t, z, bucket, object, source, "original", ObjectOptions{Versioned: true, MTime: UTCNow().Add(-time.Hour)})
stats := []*rebalanceStats{{}, {}}
stats[source] = &rebalanceStats{Participating: true, Info: rebalanceInfo{Status: rebalStarted}}
z.rebalMu.Lock()
z.rebalMeta = &rebalanceMeta{PoolStats: stats}
z.rebalMu.Unlock()
defer func() { z.rebalMu.Lock(); z.rebalMeta = nil; z.rebalMu.Unlock() }()
gr, err := z.serverPools[source].GetObjectNInfo(ctx, bucket, object, nil, nil, ObjectOptions{VersionID: original.VersionID, NoLock: true})
if err != nil {
t.Fatal(err)
}
if err := z.rebalanceObject(ctx, source, bucket, gr); err != nil {
t.Fatalf("rebalance copy: %v", err)
}
// Simulate interruption after rebalanceObject, before the enclosing loop
// removes the source version stack. Stop rebalance before the API request.
z.rebalMu.Lock()
z.rebalMeta = nil
z.rebalMu.Unlock()
for i, pool := range z.serverPools {
got, err := pool.GetObjectInfo(ctx, bucket, object, ObjectOptions{VersionID: original.VersionID})
if err != nil || got.VersionID != original.VersionID || !got.ModTime.Equal(original.ModTime) {
t.Fatalf("copy missing/changed in pool %d: %+v, %v", i, got, err)
}
}
later, err := z.PutObject(ctx, bucket, object, mustGetPutObjReader(t, bytes.NewBufferString("later"), 5, "", ""), ObjectOptions{Versioned: true})
if err != nil {
t.Fatal(err)
}
rec := consistencyRequest(t, router, http.MethodDelete, bucket, object, original.VersionID)
if rec.Code != http.StatusNoContent {
t.Fatalf("DELETE: %d %s", rec.Code, rec.Body.String())
}
for i, pool := range z.serverPools {
if _, err := pool.GetObjectInfo(ctx, bucket, object, ObjectOptions{VersionID: original.VersionID}); !isErrVersionNotFound(err) {
t.Errorf("pool %d retained old version: %v", i, err)
}
}
if got, err := z.GetObjectInfo(ctx, bucket, object, ObjectOptions{}); err != nil || got.VersionID != later.VersionID {
t.Fatalf("other version changed: %+v, %v", got, err)
}
for _, method := range []string{http.MethodGet, http.MethodHead} {
if rec := consistencyRequest(t, router, method, bucket, object, original.VersionID); rec.Code != http.StatusNotFound {
t.Errorf("%s returned %d", method, rec.Code)
}
}
})
}
}
func TestPoolsDeleteDirectoryMarker(t *testing.T) {
z, _ := consistencyPools(t)
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
bucket, router, err := initAPIHandlerTest(ctx, z, nil, MakeBucketOptions{VersioningEnabled: true})
if err != nil {
t.Fatal(err)
}
for holding := range 2 {
for _, unreadable := range []bool{false, true} {
t.Run(fmt.Sprintf("holding=%d/unreadable=%t", holding, unreadable), func(t *testing.T) {
object := fmt.Sprintf("directory-%d-%t/", holding, unreadable)
encoded := encodeDirObject(object)
putConsistencyObject(t, z, bucket, encoded, holding, "", ObjectOptions{})
set := z.serverPools[1-holding].getHashedSet(encoded)
getDisks := set.getDisks
defer func() { set.getDisks = getDisks }()
if unreadable {
disks := append([]StorageAPI(nil), getDisks()...)
for i := range disks {
disks[i] = consistencyReadFaultDisk{StorageAPI: disks[i], bucket: bucket, object: encoded}
}
set.getDisks = func() []StorageAPI { return disks }
}
// No explicit versionId: delOpts permanently deletes the null version.
rec := consistencyRequest(t, router, http.MethodDelete, bucket, object, "")
if unreadable {
if rec.Code != http.StatusServiceUnavailable {
t.Fatalf("unreadable pool: %d %s", rec.Code, rec.Body.String())
}
if _, err := z.serverPools[holding].GetObjectInfo(ctx, bucket, encoded, ObjectOptions{VersionID: nullVersionID}); err != nil {
t.Fatalf("readable directory marker lost: %v", err)
}
set.getDisks = getDisks
rec = consistencyRequest(t, router, http.MethodDelete, bucket, object, "")
}
if rec.Code != http.StatusNoContent {
t.Fatalf("DELETE: %d %s", rec.Code, rec.Body.String())
}
for i, pool := range z.serverPools {
if _, err := pool.GetObjectInfo(ctx, bucket, encoded, ObjectOptions{VersionID: nullVersionID}); !isErrVersionNotFound(err) {
t.Errorf("pool %d retained null version: %v", i, err)
}
}
if rec := consistencyRequest(t, router, http.MethodHead, bucket, object, ""); rec.Code != http.StatusNotFound {
t.Errorf("directory still visible: %d", rec.Code)
}
})
}
}
}
cmd/erasure-server-pool.go
+28 -98
View File
@@ -615,17 +615,7 @@ func (z *erasureServerPools) getPoolIdxExistingNoLock(ctx context.Context, bucke
})
}
func (z *erasureServerPools) getPoolIdxNoLock(ctx context.Context, bucket, object string, size int64, dstPoolIdx *int) (idx int, err error) {
if dstPoolIdx != nil {
if *dstPoolIdx < 0 || *dstPoolIdx >= len(z.serverPools) {
return -1, errInvalidArgument
}
if z.IsSuspended(*dstPoolIdx) || z.IsPoolRebalancing(*dstPoolIdx) {
return -1, toObjectErr(errDiskFull)
}
return *dstPoolIdx, nil
}
func (z *erasureServerPools) getPoolIdxNoLock(ctx context.Context, bucket, object string, size int64) (idx int, err error) {
idx, err = z.getPoolIdxExistingNoLock(ctx, bucket, object)
if err != nil && !isErrObjectNotFound(err) {
return idx, err
@@ -644,23 +634,13 @@ func (z *erasureServerPools) getPoolIdxNoLock(ctx context.Context, bucket, objec
// getPoolIdx returns the found previous object and its corresponding pool idx,
// if none are found falls back to most available space pool, this function is
// designed to be only used by PutObject, CopyObject (newObject creation) and NewMultipartUpload.
func (z *erasureServerPools) getPoolIdx(ctx context.Context, bucket, object string, size int64, dstPoolIdx *int) (idx int, err error) {
return z.getWritePoolIdx(ctx, bucket, object, size, dstPoolIdx, false)
func (z *erasureServerPools) getPoolIdx(ctx context.Context, bucket, object string, size int64) (idx int, err error) {
return z.getWritePoolIdx(ctx, bucket, object, size, false)
}
// getWritePoolIdx keeps the write-allocation policy when the caller already
// holds the pools-layer object lock and must not reacquire a set read lock.
func (z *erasureServerPools) getWritePoolIdx(ctx context.Context, bucket, object string, size int64, dstPoolIdx *int, noLock bool) (idx int, err error) {
if dstPoolIdx != nil {
if *dstPoolIdx < 0 || *dstPoolIdx >= len(z.serverPools) {
return -1, errInvalidArgument
}
if z.IsSuspended(*dstPoolIdx) || z.IsPoolRebalancing(*dstPoolIdx) {
return -1, toObjectErr(errDiskFull)
}
return *dstPoolIdx, nil
}
func (z *erasureServerPools) getWritePoolIdx(ctx context.Context, bucket, object string, size int64, noLock bool) (idx int, err error) {
pinfo, _, err := z.getPoolInfoExistingWithOpts(ctx, bucket, object, ObjectOptions{
NoLock: noLock,
SkipDecommissioned: true,
@@ -683,13 +663,6 @@ func (z *erasureServerPools) getWritePoolIdx(ctx context.Context, bucket, object
return idx, nil
}
func dataMovementDstPool(opts ObjectOptions) *int {
if !opts.DataMovement {
return nil
}
return opts.DstPoolIdx
}
func (z *erasureServerPools) Shutdown(ctx context.Context) error {
g := errgroup.WithNErrs(len(z.serverPools))
@@ -1158,16 +1131,10 @@ func (z *erasureServerPools) PutObject(ctx context.Context, bucket string, objec
object = encodeDirObject(object)
if z.SinglePool() {
idx, err := z.getPoolIdx(ctx, bucket, object, data.Size(), dataMovementDstPool(opts))
_, err := z.getPoolIdx(ctx, bucket, object, data.Size())
if err != nil {
return ObjectInfo{}, err
}
if dataMovementDstPool(opts) != nil && idx == opts.SrcPoolIdx {
return ObjectInfo{}, DataMovementOverwriteErr{
Bucket: bucket, Object: object, VersionID: opts.VersionID,
Err: errDataMovementSrcDstPoolSame,
}
}
return z.serverPools[0].PutObject(ctx, bucket, object, data, opts)
}
@@ -1182,7 +1149,7 @@ func (z *erasureServerPools) PutObject(ctx context.Context, bucket string, objec
}
opts.NoLock = true
idx, err := z.getWritePoolIdx(ctx, bucket, object, data.Size(), dataMovementDstPool(opts), true)
idx, err := z.getWritePoolIdx(ctx, bucket, object, data.Size(), true)
if err != nil {
return ObjectInfo{}, err
}
@@ -1238,30 +1205,11 @@ func (z *erasureServerPools) DeleteObject(ctx context.Context, bucket string, ob
return ObjectInfo{}, z.deletePrefix(ctx, bucket, object)
}
// Access-tier moves must recreate delete markers on the explicitly
// selected destination. The regular data-movement path discovers a pool
// from existing object state, which is ambiguous while both source and
// destination temporarily contain the version stack.
if dstPoolIdx := dataMovementDstPool(opts); dstPoolIdx != nil {
if *dstPoolIdx < 0 || *dstPoolIdx >= len(z.serverPools) {
return ObjectInfo{}, errInvalidArgument
}
if *dstPoolIdx == opts.SrcPoolIdx {
return ObjectInfo{}, DataMovementOverwriteErr{
Bucket: bucket, Object: decodeDirObject(object), VersionID: opts.VersionID,
Err: errDataMovementSrcDstPoolSame,
}
}
if z.IsSuspended(*dstPoolIdx) || z.IsPoolRebalancing(*dstPoolIdx) {
return ObjectInfo{}, toObjectErr(errDiskFull)
}
objInfo, err = z.serverPools[*dstPoolIdx].DeleteObject(ctx, bucket, object, opts)
objInfo.Name = decodeDirObject(object)
return objInfo, err
}
if !z.SinglePool() && opts.CheckPrecondFn != nil {
return z.deleteObjectConditional(ctx, bucket, object, opts)
// Reconcile ordinary addressed-version deletes independently of pool movement.
reconcileVersion := opts.VersionID != "" && !opts.DataMovement &&
!opts.ReplicationRequest && !opts.Expiration.Expire && !opts.InclFreeVersions
if !z.SinglePool() && (opts.CheckPrecondFn != nil || reconcileVersion) {
return z.deleteObjectReconciled(ctx, bucket, object, opts)
}
gopts := opts
@@ -1508,16 +1456,10 @@ func (z *erasureServerPools) CopyObject(ctx context.Context, srcBucket, srcObjec
return oi, err
}
poolIdx, err := z.getPoolIdxNoLock(ctx, dstBucket, dstObject, srcInfo.Size, dataMovementDstPool(dstOpts))
poolIdx, err := z.getPoolIdxNoLock(ctx, dstBucket, dstObject, srcInfo.Size)
if err != nil {
return objInfo, err
}
if dataMovementDstPool(dstOpts) != nil && poolIdx == dstOpts.SrcPoolIdx {
return ObjectInfo{}, DataMovementOverwriteErr{
Bucket: dstBucket, Object: dstObject, VersionID: dstOpts.VersionID,
Err: errDataMovementSrcDstPoolSame,
}
}
if !z.SinglePool() && dstOpts.ReplicaLockReconcile {
dstOpts.replicaObjectInfo = z.replicaObjectInfo
@@ -1977,41 +1919,29 @@ func (z *erasureServerPools) NewMultipartUpload(ctx context.Context, bucket, obj
}()
if z.SinglePool() {
idx, err := z.getPoolIdx(ctx, bucket, object, -1, dataMovementDstPool(opts))
if err != nil {
return nil, err
}
if dataMovementDstPool(opts) != nil && idx == opts.SrcPoolIdx {
return nil, DataMovementOverwriteErr{
Bucket: bucket, Object: object, VersionID: opts.VersionID,
Err: errDataMovementSrcDstPoolSame,
}
}
return z.serverPools[0].NewMultipartUpload(ctx, bucket, object, opts)
}
if dataMovementDstPool(opts) == nil {
for idx, pool := range z.serverPools {
if z.IsSuspended(idx) || z.IsPoolRebalancing(idx) {
continue
}
for idx, pool := range z.serverPools {
if z.IsSuspended(idx) || z.IsPoolRebalancing(idx) {
continue
}
result, err := pool.ListMultipartUploads(ctx, bucket, object, "", "", "", maxUploadsList)
if err != nil {
return nil, err
}
// If there is a multipart upload with the same bucket/object name,
// create the new multipart in the same pool, this will avoid
// creating two multiparts uploads in two different pools.
if len(result.Uploads) != 0 {
return z.serverPools[idx].NewMultipartUpload(ctx, bucket, object, opts)
}
result, err := pool.ListMultipartUploads(ctx, bucket, object, "", "", "", maxUploadsList)
if err != nil {
return nil, err
}
// If there is a multipart upload with the same bucket/object name,
// create the new multipart in the same pool, this will avoid
// creating two multiparts uploads in two different pools
if len(result.Uploads) != 0 {
return z.serverPools[idx].NewMultipartUpload(ctx, bucket, object, opts)
}
}
// any parallel writes on the object will block for this poolIdx
// to return since this holds a read lock on the namespace.
idx, err := z.getPoolIdx(ctx, bucket, object, -1, dataMovementDstPool(opts))
idx, err := z.getPoolIdx(ctx, bucket, object, -1)
if err != nil {
return nil, err
}
@@ -2045,7 +1975,7 @@ func (z *erasureServerPools) PutObjectPart(ctx context.Context, bucket, object,
}
if z.SinglePool() {
_, err := z.getPoolIdx(ctx, bucket, object, data.Size(), dataMovementDstPool(opts))
_, err := z.getPoolIdx(ctx, bucket, object, data.Size())
if err != nil {
return PartInfo{}, err
}
@@ -3219,7 +3149,7 @@ func (z *erasureServerPools) DecomTieredObject(ctx context.Context, bucket, obje
defer ns.Unlock(lkctx)
opts.NoLock = true
}
idx, err := z.getPoolIdxNoLock(ctx, bucket, object, fi.Size, dataMovementDstPool(opts))
idx, err := z.getPoolIdxNoLock(ctx, bucket, object, fi.Size)
if err != nil {
return err
}
cmd/ilm-access-move_test.go
-529
View File
@@ -1,529 +0,0 @@
// Copyright 2026 PGSTY contributors.
// SPDX-License-Identifier: AGPL-3.0-or-later
package cmd
import (
"bytes"
"context"
"crypto/md5"
"encoding/base64"
"encoding/xml"
"errors"
"fmt"
"io"
"maps"
"net/http"
"net/http/httptest"
"sync"
"testing"
"time"
"github.com/minio/minio/internal/dsync"
"github.com/minio/minio/internal/hash"
xhttp "github.com/minio/minio/internal/http"
)
func accessMovePools(t *testing.T) (*erasureServerPools, string) {
t.Helper()
ctx, cancel := context.WithCancel(t.Context())
dirs, err := getRandomDisks(32)
if err != nil {
t.Fatal(err)
}
endpoints := mustGetPoolEndpoints(0, dirs[:16]...)
endpoints = append(endpoints, mustGetPoolEndpoints(1, dirs[16:]...)...)
obj, _, err := initObjectLayer(ctx, endpoints)
if err != nil {
cancel()
removeRoots(dirs)
t.Fatal(err)
}
z := obj.(*erasureServerPools)
previous := newObjectLayerFn()
setObjectLayer(z)
t.Cleanup(func() { cancel(); z.Shutdown(context.Background()); removeRoots(dirs); setObjectLayer(previous) })
bucket := "access-move-test"
if err := z.MakeBucket(ctx, bucket, MakeBucketOptions{VersioningEnabled: true}); err != nil {
t.Fatal(err)
}
return z, bucket
}
func putAccessMoveVersion(t *testing.T, z *erasureServerPools, bucket, object string, pool int, body string, moved bool) ObjectInfo {
t.Helper()
metadata := map[string]string{"test-value": body}
if moved {
metadata[accessTierMetadataKey] = accessTierStamp(pool, time.Now().UnixNano())
}
cs := hash.NewChecksumFromData(hash.ChecksumCRC32C, []byte(body))
oi, err := z.serverPools[pool].PutObject(t.Context(), bucket, object,
mustGetPutObjReader(t, bytes.NewBufferString(body), int64(len(body)), "", ""),
ObjectOptions{Versioned: true, UserDefined: metadata, WantChecksum: cs})
if err != nil {
t.Fatal(err)
}
return oi
}
func assertAccessMoveVersion(t *testing.T, z *erasureServerPools, bucket, object string, pool int, want ObjectInfo, body string) {
t.Helper()
gr, err := z.serverPools[pool].GetObjectNInfo(t.Context(), bucket, object, nil, http.Header{}, ObjectOptions{VersionID: want.VersionID})
if err != nil {
t.Errorf("pool %d version %s: %v", pool, want.VersionID, err)
return
}
got, err := io.ReadAll(gr)
gr.Close()
if err != nil || string(got) != body {
t.Errorf("pool %d payload = %q, err = %v, want %q", pool, got, err, body)
}
if gr.ObjInfo.ETag != want.ETag || !gr.ObjInfo.ModTime.Equal(want.ModTime) {
t.Error("move changed ETag or modification time")
}
if len(want.Checksum) == 0 {
t.Fatal("fixture has no checksum")
}
if !bytes.Equal(gr.ObjInfo.Checksum, want.Checksum) {
t.Error("move changed or dropped the stored checksum")
}
if gr.ObjInfo.UserDefined["test-value"] != body {
t.Error("move changed user metadata")
}
}
func TestAccessMoveVersionStack(t *testing.T) {
z, bucket := accessMovePools(t)
const object = "versions"
old := putAccessMoveVersion(t, z, bucket, object, 1, "old", false)
dm, err := z.serverPools[1].DeleteObject(t.Context(), bucket, object, ObjectOptions{Versioned: true})
if err != nil {
t.Fatal(err)
}
latest := putAccessMoveVersion(t, z, bucket, object, 1, "new", false)
for _, pair := range [][2]int{{1, 0}, {0, 1}} {
if _, err := moveObjectPool(t.Context(), z, bucket, object, pair[0], pair[1], nil); err != nil {
t.Fatal(err)
}
assertAccessMoveVersion(t, z, bucket, object, pair[1], old, "old")
assertAccessMoveVersion(t, z, bucket, object, pair[1], latest, "new")
versions, err := accessObjectVersions(t.Context(), z, pair[1], bucket, object)
if err != nil {
t.Fatal(err)
}
found := false
for _, version := range versions {
if version.VersionID == dm.VersionID && version.Deleted {
found = true
}
}
if !found || len(versions) != 3 {
t.Errorf("move lost a version or delete marker: %+v", versions)
}
if _, err := z.serverPools[pair[0]].GetObjectInfo(t.Context(), bucket, object, ObjectOptions{}); !isErrObjectNotFound(err) {
t.Errorf("source remains after completed move: %v", err)
}
}
}
func TestAccessMovePreservesNestedObject(t *testing.T) {
z, bucket := accessMovePools(t)
parent := putAccessMoveVersion(t, z, bucket, "parent", 1, "parent", false)
child := putAccessMoveVersion(t, z, bucket, "parent/child", 1, "child", false)
if _, err := moveObjectPool(t.Context(), z, bucket, "parent", 1, 0, nil); err != nil {
t.Fatal(err)
}
assertAccessMoveVersion(t, z, bucket, "parent", 0, parent, "parent")
assertAccessMoveVersion(t, z, bucket, "parent/child", 1, child, "child")
}
func TestAccessMoveNullVersion(t *testing.T) {
z, bucket := accessMovePools(t)
const object, body = "null-version", "unversioned"
// A null version can remain in a bucket after versioning is enabled.
oi, err := z.serverPools[1].PutObject(t.Context(), bucket, object,
mustGetPutObjReader(t, bytes.NewBufferString(body), int64(len(body)), "", ""), ObjectOptions{
UserDefined: map[string]string{"test-value": body},
WantChecksum: hash.NewChecksumFromData(hash.ChecksumCRC32C, []byte(body)),
})
if err != nil || oi.VersionID != "" {
t.Fatalf("null version fixture failed: %v %q", err, oi.VersionID)
}
if _, err := moveObjectPool(t.Context(), z, bucket, object, 1, 0, nil); err != nil {
t.Fatal(err)
}
assertAccessMoveVersion(t, z, bucket, object, 0, oi, body)
if _, err := z.serverPools[1].GetObjectInfo(t.Context(), bucket, object, ObjectOptions{}); !isErrObjectNotFound(err) {
t.Fatalf("null version source was not removed: %v", err)
}
}
// Source removal can partially commit before a process or disk fails. The
// destination can therefore hold the only remaining copy of an older version.
func TestAccessMoveRetryPreservesDestinationVersions(t *testing.T) {
z, bucket := accessMovePools(t)
const object = "retry"
onlyAtDestination := putAccessMoveVersion(t, z, bucket, object, 0, "unique-old", true)
source := putAccessMoveVersion(t, z, bucket, object, 1, "source-new", false)
if _, err := moveObjectPool(t.Context(), z, bucket, object, 1, 0, nil); err != nil {
t.Fatal(err)
}
assertAccessMoveVersion(t, z, bucket, object, 0, onlyAtDestination, "unique-old")
assertAccessMoveVersion(t, z, bucket, object, 0, source, "source-new")
}
type accessMoveFaultDisk struct {
StorageAPI
failVersion string
}
func (d accessMoveFaultDisk) RenameData(ctx context.Context, srcVolume, srcPath string, fi FileInfo, dstVolume, dstPath string, opts RenameOptions) (RenameDataResp, error) {
if fi.VersionID == d.failVersion {
return RenameDataResp{}, errDiskFull
}
return d.StorageAPI.RenameData(ctx, srcVolume, srcPath, fi, dstVolume, dstPath, opts)
}
func TestAccessMoveWriteFailureCanResume(t *testing.T) {
z, bucket := accessMovePools(t)
const object = "write-failure"
old := putAccessMoveVersion(t, z, bucket, object, 1, "old", false)
latest := putAccessMoveVersion(t, z, bucket, object, 1, "new", false)
existing := putAccessMoveVersion(t, z, bucket, object, 0, "unique-destination", true)
set := z.serverPools[0].getHashedSet(object)
getDisks := set.getDisks
disks := getDisks()
faulty := make([]StorageAPI, len(disks))
for i, disk := range disks {
faulty[i] = accessMoveFaultDisk{StorageAPI: disk, failVersion: latest.VersionID}
}
set.getDisks = func() []StorageAPI { return faulty }
_, err := moveObjectPool(t.Context(), z, bucket, object, 1, 0, nil)
set.getDisks = getDisks
if err == nil {
t.Fatal("injected destination write failure was ignored")
}
assertAccessMoveVersion(t, z, bucket, object, 1, old, "old")
assertAccessMoveVersion(t, z, bucket, object, 1, latest, "new")
assertAccessMoveVersion(t, z, bucket, object, 0, existing, "unique-destination")
if _, err := moveObjectPool(t.Context(), z, bucket, object, 1, 0, nil); err != nil {
t.Fatal(err)
}
assertAccessMoveVersion(t, z, bucket, object, 0, old, "old")
assertAccessMoveVersion(t, z, bucket, object, 0, latest, "new")
assertAccessMoveVersion(t, z, bucket, object, 0, existing, "unique-destination")
}
func TestAccessMoveExcludesSourceWriter(t *testing.T) {
z, bucket := accessMovePools(t)
const object = "concurrent"
old := putAccessMoveVersion(t, z, bucket, object, 1, "old", false)
_, err := moveObjectPool(t.Context(), z, bucket, object, 1, 0, func(ObjectInfo, uint64) error {
ctx, cancel := context.WithTimeout(t.Context(), 200*time.Millisecond)
defer cancel()
_, err := z.serverPools[1].PutObject(ctx, bucket, object,
mustGetPutObjReader(t, bytes.NewBufferString("racing-write"), 12, "", ""), ObjectOptions{Versioned: true})
if err == nil {
t.Error("source writer committed while move was in progress")
}
if ctx.Err() == nil {
return errors.New("writer did not wait for the move lock")
}
return nil
})
if err != nil {
t.Fatal(err)
}
assertAccessMoveVersion(t, z, bucket, object, 0, old, "old")
}
func TestAccessMoveResumesPartialCopy(t *testing.T) {
z, bucket := accessMovePools(t)
const object = "partial-copy"
old := putAccessMoveVersion(t, z, bucket, object, 1, "old", false)
latest := putAccessMoveVersion(t, z, bucket, object, 1, "new", false)
gr, err := z.serverPools[1].GetObjectNInfo(t.Context(), bucket, object, nil, http.Header{}, ObjectOptions{VersionID: old.VersionID, NoDecryption: true})
if err != nil {
t.Fatal(err)
}
// Model a process stopping after the first copy commits, before cleanup.
if err := moveAccessTierVersion(t.Context(), z, 1, 0, bucket, gr, time.Now().UnixNano()); err != nil {
t.Fatal(err)
}
if _, err := moveObjectPool(t.Context(), z, bucket, object, 1, 0, nil); err != nil {
t.Fatal(err)
}
assertAccessMoveVersion(t, z, bucket, object, 0, old, "old")
assertAccessMoveVersion(t, z, bucket, object, 0, latest, "new")
}
func TestAccessMoveRefusesConflictingVersion(t *testing.T) {
z, bucket := accessMovePools(t)
const object = "conflicting-version"
source := putAccessMoveVersion(t, z, bucket, object, 1, "source", false)
destination, err := z.serverPools[0].PutObject(t.Context(), bucket, object,
mustGetPutObjReader(t, bytes.NewBufferString("target"), 6, "", ""), ObjectOptions{
VersionID: source.VersionID, MTime: source.ModTime,
UserDefined: map[string]string{"test-value": "target"},
WantChecksum: hash.NewChecksumFromData(hash.ChecksumCRC32C, []byte("target")),
})
if err != nil {
t.Fatal(err)
}
if _, err := moveObjectPool(t.Context(), z, bucket, object, 1, 0, nil); !errors.Is(err, errAccessTierNotEligible) {
t.Fatalf("conflicting version should be left intact: %v", err)
}
assertAccessMoveVersion(t, z, bucket, object, 1, source, "source")
assertAccessMoveVersion(t, z, bucket, object, 0, destination, "target")
}
type accessMoveDeleteFaultDisk struct {
StorageAPI
bucket, object, version string
}
func (d accessMoveDeleteFaultDisk) DeleteVersion(ctx context.Context, volume, path string, fi FileInfo, forceDelMarker bool, opts DeleteOptions) error {
if volume == d.bucket && path == d.object && fi.VersionID == d.version {
return errDiskFull
}
return d.StorageAPI.DeleteVersion(ctx, volume, path, fi, forceDelMarker, opts)
}
func TestAccessMoveSourceDeleteFailureCanResume(t *testing.T) {
z, bucket := accessMovePools(t)
const object = "source-delete-failure"
old := putAccessMoveVersion(t, z, bucket, object, 1, "old", false)
latest := putAccessMoveVersion(t, z, bucket, object, 1, "new", false)
set := z.serverPools[1].getHashedSet(object)
getDisks := set.getDisks
faulty := append([]StorageAPI(nil), getDisks()...)
// Commit removal of the old version, then reject the latest version on
// every disk. This fixes the retry boundary without relying on how a
// partially deleted erasure quorum is subsequently resolved or healed.
for i := range faulty {
faulty[i] = accessMoveDeleteFaultDisk{StorageAPI: faulty[i], bucket: bucket, object: object, version: latest.VersionID}
}
set.getDisks = func() []StorageAPI { return faulty }
_, err := moveObjectPool(t.Context(), z, bucket, object, 1, 0, nil)
set.getDisks = getDisks
if err == nil {
t.Fatal("injected partial source purge was ignored")
}
assertAccessMoveVersion(t, z, bucket, object, 0, old, "old")
assertAccessMoveVersion(t, z, bucket, object, 0, latest, "new")
if _, err := z.serverPools[1].GetObjectInfo(t.Context(), bucket, object, ObjectOptions{VersionID: old.VersionID}); !isErrVersionNotFound(err) {
t.Fatalf("old source version should already be removed: %v", err)
}
assertAccessMoveVersion(t, z, bucket, object, 1, latest, "new")
if _, err := moveObjectPool(t.Context(), z, bucket, object, 1, 0, nil); err != nil {
t.Fatal(err)
}
assertAccessMoveVersion(t, z, bucket, object, 0, old, "old")
assertAccessMoveVersion(t, z, bucket, object, 0, latest, "new")
}
// Exercise the public multipart API and compare whole-object and part reads
// across both directions of a real pool move, including raw SSE-C ciphertext.
func TestAccessMoveMultipartChecksums(t *testing.T) {
z, _ := accessMovePools(t)
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
bucket, router, err := initAPIHandlerTest(ctx, z, nil, MakeBucketOptions{VersioningEnabled: true})
if err != nil {
t.Fatal(err)
}
previousTLS := globalIsTLS
globalIsTLS = true
defer func() { globalIsTLS = previousTLS }()
partData, full := multipartChecksumTestData()
for _, tc := range []struct {
typ hash.ChecksumType
ssec bool
}{{hash.ChecksumCRC32, false}, {hash.ChecksumCRC32C, false}, {hash.ChecksumCRC64NVME, false}, {hash.ChecksumCRC32C, true}} {
t.Run(fmt.Sprintf("%s/ssec=%t", tc.typ, tc.ssec), func(t *testing.T) {
object := getRandomObjectName()
headers := map[string]string{}
if tc.ssec {
key := bytes.Repeat([]byte{0x42}, 32)
keyMD5 := md5.Sum(key)
headers[xhttp.AmzServerSideEncryptionCustomerAlgorithm] = xhttp.AmzEncryptionAES
headers[xhttp.AmzServerSideEncryptionCustomerKey] = base64.StdEncoding.EncodeToString(key)
headers[xhttp.AmzServerSideEncryptionCustomerKeyMD5] = base64.StdEncoding.EncodeToString(keyMD5[:])
}
do := func(method, url string, body []byte, extra map[string]string) *httptest.ResponseRecorder {
t.Helper()
h := maps.Clone(headers)
maps.Copy(h, extra)
req, err := newTestSignedRequestV4(method, url, int64(len(body)), bytes.NewReader(body), globalActiveCred.AccessKey, globalActiveCred.SecretKey, h)
if err != nil {
t.Fatal(err)
}
rec := httptest.NewRecorder()
router.ServeHTTP(rec, req)
if rec.Code != http.StatusOK && rec.Code != http.StatusPartialContent {
t.Fatalf("%s %s: %d %s", method, url, rec.Code, rec.Body.String())
}
return rec
}
init := do(http.MethodPost, getNewMultipartURL("", bucket, object), nil, map[string]string{
xhttp.AmzChecksumAlgo: tc.typ.String(), xhttp.AmzChecksumType: xhttp.AmzChecksumTypeFullObject,
})
var upload InitiateMultipartUploadResponse
if err := xml.Unmarshal(init.Body.Bytes(), &upload); err != nil {
t.Fatal(err)
}
parts := make([]CompletePart, len(partData))
for i, data := range partData {
rec := do(http.MethodPut, getPutObjectPartURL("", bucket, object, upload.UploadID, fmt.Sprint(i+1)), data,
map[string]string{tc.typ.Key(): mustChecksum(t, tc.typ, data)})
parts[i] = CompletePart{PartNumber: i + 1, ETag: canonicalizeETag(rec.Header()[xhttp.ETag][0])}
}
body, err := xml.Marshal(CompleteMultipartUpload{Parts: parts})
if err != nil {
t.Fatal(err)
}
do(http.MethodPost, getCompleteMultipartUploadURL("", bucket, object, upload.UploadID), body,
map[string]string{tc.typ.Key(): mustChecksum(t, tc.typ, full), xhttp.AmzChecksumType: xhttp.AmzChecksumTypeFullObject})
source, err := z.GetObjectInfo(t.Context(), bucket, object, ObjectOptions{})
if err != nil || len(source.Checksum) == 0 {
t.Fatalf("source checksum missing: %v", err)
}
src := 0
if _, err := z.serverPools[src].GetObjectInfo(t.Context(), bucket, object, ObjectOptions{}); isErrObjectNotFound(err) {
src = 1
}
for i := 0; i < 2; i++ {
dst := 1 - src
if _, err := moveObjectPool(t.Context(), z, bucket, object, src, dst, nil); err != nil {
t.Fatal(err)
}
got, err := z.serverPools[dst].GetObjectInfo(t.Context(), bucket, object, ObjectOptions{})
if err != nil || !bytes.Equal(got.Checksum, source.Checksum) || got.ETag != source.ETag || got.VersionID != source.VersionID || !got.ModTime.Equal(source.ModTime) {
t.Fatalf("move changed version metadata: %v checksumEqual=%t ETag=%q/%q version=%q/%q modTimeEqual=%t", err, bytes.Equal(got.Checksum, source.Checksum), got.ETag, source.ETag, got.VersionID, source.VersionID, got.ModTime.Equal(source.ModTime))
}
rec := do(http.MethodGet, getGetObjectURL("", bucket, object), nil, map[string]string{xhttp.AmzChecksumMode: "ENABLED"})
if !bytes.Equal(rec.Body.Bytes(), full) || rec.Header().Get(tc.typ.Key()) != mustChecksum(t, tc.typ, full) {
t.Fatal("GET payload or checksum changed after move")
}
for j, data := range partData {
rec := do(http.MethodGet, getGetObjectURL("", bucket, object)+fmt.Sprintf("?partNumber=%d", j+1), nil, nil)
if !bytes.Equal(rec.Body.Bytes(), data) {
t.Fatalf("part %d changed after move", j+1)
}
}
src = dst
}
})
}
}
type accessMoveNamedLocker struct {
*localLocker
address string
}
func (l accessMoveNamedLocker) String() string { return l.address }
func TestAccessMoveSharedDistributedLockers(t *testing.T) {
// Three overlapping sets of real dsync lock servers. Taking independent
// quorum locks for the same object would contend with our own earlier lock.
peers := make([]dsync.NetLocker, 5)
for i := range peers {
peers[i] = accessMoveNamedLocker{localLocker: newLocker(), address: fmt.Sprint(i)}
}
z := &erasureServerPools{}
for i := 0; i < 3; i++ {
setPeers := peers[i : i+3]
set := &erasureObjects{nsMutex: &nsLockMap{isDistErasure: true}, getLockers: func() ([]dsync.NetLocker, string) {
return setPeers, "access-move-fixture"
}}
z.serverPools = append(z.serverPools, &erasureSets{sets: []*erasureObjects{set}, distributionAlgo: formatErasureVersionV3DistributionAlgoV3})
}
ctx, cancel := context.WithTimeout(t.Context(), 5*time.Second)
defer cancel()
_, unlock, err := lockAccessTierObject(ctx, z, "bucket", "object", 1, 2)
if err != nil {
t.Fatal(err)
}
release := sync.OnceFunc(unlock)
defer release()
for i, pool := range z.serverPools {
ctx, cancel := context.WithTimeout(t.Context(), 100*time.Millisecond)
lock := pool.NewNSLock("bucket", "object")
lc, err := lock.GetLock(ctx, globalOperationTimeout)
cancel()
if err == nil {
lock.Unlock(lc)
t.Fatalf("pool %d writer acquired its quorum during the move", i)
}
}
release()
// Every acquired peer lock is released, so ordinary writes can resume.
for _, peer := range peers {
// Distributed Unlock sends releases asynchronously.
lock := (&nsLockMap{isDistErasure: true}).NewNSLock(func() ([]dsync.NetLocker, string) {
return []dsync.NetLocker{peer}, "access-move-fixture"
}, "bucket", "object")
ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second)
lc, err := lock.GetLock(ctx, globalOperationTimeout)
cancel()
if err != nil {
t.Fatalf("peer %s leaked a lock: %v", peer.String(), err)
}
lock.Unlock(lc)
}
}
func TestAccessMoveConfiguredPromotionAndDemotion(t *testing.T) {
z, bucket := accessMovePools(t)
const object = "configured-move"
version := putAccessMoveVersion(t, z, bucket, object, 1, "configured", false)
oldCfg, oldTracker := globalILMConfig.accessCfg(), globalAccessTracker
defer func() { globalILMConfig.update(oldCfg); globalAccessTracker = oldTracker }()
cfg := oldCfg
cfg.AccessTiering, cfg.AccessPools = true, []int{0, 1}
cfg.AccessMinResidency, cfg.AccessPromoteWatermark = 0, 99
globalILMConfig.update(cfg)
globalAccessTracker = newAccessTracker()
lc := accessLifecycleForTest(t)
data, err := xml.Marshal(lc)
if err != nil {
t.Fatal(err)
}
if _, err := globalBucketMetadataSys.Update(t.Context(), bucket, bucketLifecycleConfig, data); err != nil {
t.Fatal(err)
}
state := newAccessTierState(t.Context())
state.z, state.objAPI = z, z
task := accessTierTask{ctx: t.Context(), bucket: bucket, object: object, src: 1, dst: 0, direction: accessTierPromote, bytes: uint64(version.Size)}
// A queued move is rechecked when configuration is disabled dynamically.
cfg.AccessTiering = false
globalILMConfig.update(cfg)
if err := state.processTask(task); !errors.Is(err, errAccessTierNotEligible) {
t.Fatalf("disabled mover = %v", err)
}
cfg.AccessTiering = true
globalILMConfig.update(cfg)
globalAccessTracker.merged.Store(&mergedAccess{binWidth: 60, entries: map[string]accessEntry{
accessKey(bucket, object): {Bins: []uint32{100}, LastAt: time.Now().Unix()},
}})
if reason, ok := state.reservePromotion(task, cfg, 0); !ok {
t.Fatalf("promotion reservation failed: %s", reason)
}
if err := state.processTask(task); err != nil {
t.Fatal(err)
}
assertAccessMoveVersion(t, z, bucket, object, 0, version, "configured")
// Advance the counter view beyond the rule's idle period.
globalAccessTracker.merged.Store(&mergedAccess{binWidth: 60, entries: map[string]accessEntry{
accessKey(bucket, object): {Bins: []uint32{0}, LastAt: time.Now().Add(-2 * time.Hour).Unix()},
}})
task.src, task.dst, task.direction, task.bytes = 0, 1, accessTierDemote, 0
if err := state.processTask(task); err != nil {
t.Fatal(err)
}
assertAccessMoveVersion(t, z, bucket, object, 1, version, "configured")
if state.promotions.Load() != 1 || state.demotions.Load() != 1 || state.bytesMoved.Load() != 2*uint64(version.Size) || len(state.pending) != 0 || len(state.reserved) != 0 {
t.Fatal("promotion/demotion metrics or reservation cleanup are incorrect")
}
}
cmd/ilm-access-tier.go
-1201
View File
File diff suppressed because it is too large Load Diff
cmd/ilm-access-tier_test.go
-263
View File
@@ -1,263 +0,0 @@
// Copyright (c) 2015-2026 MinIO, Inc.
package cmd
import (
"bytes"
"context"
"errors"
"strconv"
"strings"
"testing"
"time"
"github.com/klauspost/compress/zstd"
"github.com/minio/minio/internal/bucket/lifecycle"
"github.com/minio/minio/internal/config/ilm"
"github.com/tinylib/msgp/msgp"
)
func accessLifecycleForTest(t *testing.T) *lifecycle.Lifecycle {
t.Helper()
xml := "<LifecycleConfiguration>" +
"<Rule><ID>access</ID><Status>Enabled</Status>" +
"<AccessTransition><Window>10m</Window><PromoteAfterAccesses>100</PromoteAfterAccesses>" +
"<DemoteAfterAccesses>5</DemoteAfterAccesses><DemoteAfterIdle>1h</DemoteAfterIdle></AccessTransition>" +
"</Rule></LifecycleConfiguration>"
lc, err := lifecycle.ParseLifecycleConfig(strings.NewReader(xml))
if err != nil {
t.Fatal(err)
}
return lc
}
func TestAccessTierStampAndDemoteEligibility(t *testing.T) {
oldTracker := globalAccessTracker
globalAccessTracker = newAccessTracker()
t.Cleanup(func() { globalAccessTracker = oldTracker })
now := time.Now()
cfg := ilm.Config{
AccessTiering: true, AccessPools: []int{0, 1},
AccessBinWidth: time.Minute, AccessBins: 12,
AccessMinResidency: 30 * time.Minute,
}
oi := ObjectInfo{
Bucket: "bucket", Name: "object", Size: 10, IsLatest: true,
UserDefined: map[string]string{
accessTierMetadataKey: "0:" + strconv.FormatInt(now.Add(-2*time.Hour).UnixNano(), 10),
},
}
if !accessDemoteEligible(accessLifecycleForTest(t), oi, 0, cfg, now) {
t.Fatal("cold, resident object should be demotion eligible")
}
oi.UserDefined[accessTierMetadataKey] = "0:" + strconv.FormatInt(now.Add(-time.Minute).UnixNano(), 10)
if accessDemoteEligible(accessLifecycleForTest(t), oi, 0, cfg, now) {
t.Fatal("object inside minimum residency was eligible")
}
oi.UserDefined[accessTierMetadataKey] = "broken"
if accessDemoteEligible(accessLifecycleForTest(t), oi, 0, cfg, now) {
t.Fatal("object with malformed marker was eligible")
}
}
func TestAccessTierReservationsEnforceCaps(t *testing.T) {
state := newAccessTierState(t.Context())
state.usageReady = true
state.baseUsage["a"] = 80
state.baseTotal = 180
cfg := ilm.Config{AccessMaxSize: 200}
task := accessTierTask{ctx: t.Context(), bucket: "a", object: "one", bytes: 30}
if reason, ok := state.reservePromotion(task, cfg, 0); ok || reason != "max-size" {
t.Fatalf("max-size reserve = %q/%v", reason, ok)
}
cfg.AccessMaxSize = 0
if reason, ok := state.reservePromotion(task, cfg, 100); ok || reason != "quota" {
t.Fatalf("quota reserve = %q/%v", reason, ok)
}
task.bytes = 20
if reason, ok := state.reservePromotion(task, cfg, 100); !ok || reason != "" {
t.Fatalf("valid reserve = %q/%v", reason, ok)
}
if got := state.bucketUsageLocked("a"); got != 100 {
t.Fatalf("reserved bucket usage = %d, want 100", got)
}
state.releasePending(task, true)
}
func TestDataMovementDestinationIsGated(t *testing.T) {
dst := 0
if got := dataMovementDstPool(ObjectOptions{DstPoolIdx: &dst}); got != nil {
t.Fatal("destination honored without DataMovement")
}
if got := dataMovementDstPool(ObjectOptions{DataMovement: true, DstPoolIdx: &dst}); got == nil || *got != 0 {
t.Fatalf("destination = %v", got)
}
}
func TestForcedDestinationPoolSelection(t *testing.T) {
z := &erasureServerPools{serverPools: make([]*erasureSets, 2)}
dst := 1
if got, err := z.getPoolIdx(context.Background(), "bucket", "object", 1, &dst); err != nil || got != dst {
t.Fatalf("destination = %d, err = %v", got, err)
}
bad := 2
if _, err := z.getPoolIdx(context.Background(), "bucket", "object", 1, &bad); !errors.Is(err, errInvalidArgument) {
t.Fatalf("out-of-range error = %v", err)
}
z.poolMeta.Pools = make([]PoolStatus, 2)
z.poolMeta.Pools[1].Decommission = &PoolDecommissionInfo{}
if _, err := z.getPoolIdx(context.Background(), "bucket", "object", 1, &dst); err == nil {
t.Fatal("suspended destination was accepted")
}
}
func TestHotTierAccounting(t *testing.T) {
entry := dataUsageEntry{}
entry.addSizes(sizeSummary{totalSize: 100, hotTierSize: 60, versions: 1})
entry.merge(dataUsageEntry{Size: 50, HotTierSize: 25})
if entry.Size != 150 || entry.HotTierSize != 85 {
t.Fatalf("usage = size:%d hot:%d", entry.Size, entry.HotTierSize)
}
}
func TestScannerCyclesApart(t *testing.T) {
tests := []struct {
current, previous uint32
want uint32
}{
{current: 12, previous: 10, want: 2},
{current: 0, previous: ^uint32(0), want: 1},
// A cycle counter reset is not evidence that a complete pass covered
// recent moves, so it must not release conservative deltas.
{current: 1, previous: 100, want: 0},
}
for _, tt := range tests {
if got := scannerCyclesApart(tt.current, tt.previous); got != tt.want {
t.Fatalf("scannerCyclesApart(%d, %d) = %d, want %d", tt.current, tt.previous, got, tt.want)
}
}
}
func TestDataUsageCacheV8Migration(t *testing.T) {
var encoded bytes.Buffer
if err := encoded.WriteByte(dataUsageCacheVerV8); err != nil {
t.Fatal(err)
}
zw, err := zstd.NewWriter(&encoded)
if err != nil {
t.Fatal(err)
}
mw := msgp.NewWriter(zw)
if err = mw.WriteMapHeader(2); err != nil {
t.Fatal(err)
}
if err = mw.WriteString("Info"); err != nil {
t.Fatal(err)
}
info := dataUsageCacheInfo{Name: dataUsageRoot, NextCycle: 17, LastUpdate: time.Now().UTC()}
if err = info.EncodeMsg(mw); err != nil {
t.Fatal(err)
}
if err = mw.WriteString("Cache"); err != nil {
t.Fatal(err)
}
if err = mw.WriteMapHeader(1); err != nil {
t.Fatal(err)
}
if err = mw.WriteString("entry"); err != nil {
t.Fatal(err)
}
// A v8 entry has no hts field. Encoding only populated fields also
// verifies that its map decoder retains normal msgpack compatibility.
if err = mw.WriteMapHeader(2); err != nil {
t.Fatal(err)
}
if err = mw.WriteString("sz"); err != nil {
t.Fatal(err)
}
if err = mw.WriteInt64(123); err != nil {
t.Fatal(err)
}
if err = mw.WriteString("os"); err != nil {
t.Fatal(err)
}
if err = mw.WriteUint64(2); err != nil {
t.Fatal(err)
}
if err = mw.Flush(); err != nil {
t.Fatal(err)
}
if err = zw.Close(); err != nil {
t.Fatal(err)
}
var migrated dataUsageCache
if err = migrated.deserialize(bytes.NewReader(encoded.Bytes())); err != nil {
t.Fatal(err)
}
entry := migrated.Cache["entry"]
if entry.Size != 123 || entry.Objects != 2 || entry.HotTierSize != 0 {
t.Fatalf("migrated entry = size:%d objects:%d hot:%d", entry.Size, entry.Objects, entry.HotTierSize)
}
if migrated.Info.NextCycle != 17 || !migrated.Info.LastUpdate.Equal(info.LastUpdate) {
t.Fatalf("migrated cache info = %+v", migrated.Info)
}
}
// The stamp written on every moved version and the stamp the rollback path
// matches against must agree, otherwise rollback silently skips its own work.
func TestAccessTierStampRoundTrip(t *testing.T) {
movedAt := time.Now().UnixNano()
stamp := accessTierStamp(3, movedAt)
pool, at, ok := parseAccessTierStamp(map[string]string{accessTierMetadataKey: stamp})
if !ok {
t.Fatalf("stamp %q did not parse", stamp)
}
if pool != 3 {
t.Fatalf("pool = %d, want 3", pool)
}
if at.UnixNano() != movedAt {
t.Fatalf("movedAt = %d, want %d", at.UnixNano(), movedAt)
}
// A different move of the same object must not match, so a concurrent
// client overwrite is never mistaken for our own copy.
if stamp == accessTierStamp(3, movedAt+1) {
t.Fatal("stamps from distinct moves collided")
}
if stamp == accessTierStamp(4, movedAt) {
t.Fatal("stamps from distinct pools collided")
}
}
func TestAccessHitsMightPromote(t *testing.T) {
lc := accessLifecycleForTest(t)
hot := accessEntry{Bins: []uint32{100}, HeadAt: 0}
cold := accessEntry{Bins: []uint32{1}, HeadAt: 0}
if !accessHitsMightPromote(lc, "object", hot, 60) {
t.Fatal("object above promote threshold was rejected")
}
if accessHitsMightPromote(lc, "object", cold, 60) {
t.Fatal("object below promote threshold was accepted")
}
xml := "<LifecycleConfiguration><Rule><ID>logs</ID><Status>Enabled</Status>" +
"<Filter><Prefix>logs/</Prefix></Filter>" +
"<AccessTransition><Window>10m</Window><PromoteAfterAccesses>100</PromoteAfterAccesses>" +
"<DemoteAfterAccesses>5</DemoteAfterAccesses><DemoteAfterIdle>1h</DemoteAfterIdle></AccessTransition>" +
"</Rule></LifecycleConfiguration>"
prefixed, err := lifecycle.ParseLifecycleConfig(strings.NewReader(xml))
if err != nil {
t.Fatal(err)
}
if accessHitsMightPromote(prefixed, "data/object", hot, 60) {
t.Fatal("object outside the rule prefix was accepted")
}
if !accessHitsMightPromote(prefixed, "logs/object", hot, 60) {
t.Fatal("object inside the rule prefix was rejected")
}
}
cmd/ilm-access-tracker.go
-598
View File
@@ -1,598 +0,0 @@
// Copyright (c) 2015-2026 MinIO, Inc.
//
// This file is part of MinIO Object Storage stack
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package cmd
import (
"cmp"
"context"
"fmt"
"slices"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/minio/minio/internal/config/ilm"
"github.com/zeebo/xxh3"
)
//go:generate msgp -file=$GOFILE -unexported
//msgp:ignore accessTracker mergedAccess
const (
// accessTrackerPrefix is where each node publishes its own counters.
// One object per node, merged by every node on a timer.
accessTrackerPrefix = minioConfigPrefix + "/ilm/access"
// accessQueueSize bounds the GET -> tracker handoff. Overflow drops
// samples rather than slowing down reads.
accessQueueSize = 100000
// accessMaxDemoteCandidates bounds what the scanner may hand over in a
// single flush interval.
accessMaxDemoteCandidates = 100000
// accessShardStaleFactor multiplies the flush interval to decide when a
// peer's counters are too old to trust, e.g. after a node is removed.
accessShardStaleFactor = 5
)
func accessShardFresh(now int64, shard accessShard, stale int64) bool {
if shard.UpdatedAt <= 0 {
return false
}
if stale <= 0 {
return true
}
age := now - shard.UpdatedAt
return age >= -stale && age <= stale
}
// accessEntry is a rolling hit counter for one object. Bins[0] is the current
// bin and each subsequent bin is one bin-width older, so a rule asking for
// "100 hits in 10 minutes" sums the newest ceil(10m/binWidth) bins.
//
// A fixed window is used rather than an exponentially decayed score because
// the rule is stated to operators in exactly those terms.
type accessEntry struct {
Bins []uint32 `msg:"b"`
HeadAt int64 `msg:"h"` // unix seconds at the start of Bins[0]
LastAt int64 `msg:"l"` // unix seconds of the most recent hit
}
// demoteCandidate is an object the scanner found sitting on a fast pool with
// no recent reads. Candidates ride the node's own counter shard so they reach
// the leader without a new peer RPC.
type demoteCandidate struct {
Bucket string `msg:"b"`
Object string `msg:"o"`
Pool int `msg:"p"`
}
// accessShard is what one node publishes. BinWidth is carried so a peer that
// has not yet picked up a configuration change is ignored rather than merged
// with mismatched bins.
type accessShard struct {
UpdatedAt int64 `msg:"u"`
BinWidth int64 `msg:"bw"`
Entries map[string]accessEntry `msg:"e"`
Demote []demoteCandidate `msg:"d"`
}
// binStart truncates a unix timestamp to the start of its bin.
func binStart(now, binWidth int64) int64 {
if binWidth <= 0 {
return now
}
return now - now%binWidth
}
// rollTo advances the counter to now, zeroing the bins that elapsed since the
// last update and resizing if the configured bin count changed.
func (e *accessEntry) rollTo(now, binWidth int64, nbins int) {
if nbins <= 0 || binWidth <= 0 {
return
}
if len(e.Bins) != nbins {
resized := make([]uint32, nbins)
copy(resized, e.Bins)
e.Bins = resized
}
head := binStart(now, binWidth)
if e.HeadAt == 0 {
e.HeadAt = head
return
}
steps := (head - e.HeadAt) / binWidth
if steps <= 0 {
return
}
if steps >= int64(nbins) {
clear(e.Bins)
} else {
copy(e.Bins[steps:], e.Bins[:nbins-int(steps)])
clear(e.Bins[:steps])
}
e.HeadAt = head
}
// hits returns the number of accesses recorded over the newest bins covering
// window. A window longer than the configured history is clamped to it.
//
// The newest bin is partial, so the covered span is between window-binWidth
// and window. Operators tune resolution with ilm access_bin_width.
func (e accessEntry) hits(window time.Duration, binWidth int64) uint64 {
if binWidth <= 0 || len(e.Bins) == 0 {
return 0
}
n := int((int64(window/time.Second) + binWidth - 1) / binWidth)
if n < 1 {
n = 1
}
if n > len(e.Bins) {
n = len(e.Bins)
}
var total uint64
for _, v := range e.Bins[:n] {
total += uint64(v)
}
return total
}
// total is the whole retained history, used to decide what to evict.
func (e accessEntry) total() uint64 {
var t uint64
for _, v := range e.Bins {
t += uint64(v)
}
return t
}
// mergeFrom adds another node's counters for the same object. Both sides must
// already be rolled to the same head.
func (e *accessEntry) mergeFrom(o accessEntry) {
for i := range e.Bins {
if i < len(o.Bins) {
total := uint64(e.Bins[i]) + uint64(o.Bins[i])
if total > uint64(^uint32(0)) {
total = uint64(^uint32(0))
}
e.Bins[i] = uint32(total)
}
}
if o.LastAt > e.LastAt {
e.LastAt = o.LastAt
}
}
// mergedAccess is an immutable cluster-wide snapshot. Readers take it from an
// atomic pointer, so the hot scanner and sweep paths never take a lock.
type mergedAccess struct {
entries map[string]accessEntry
binWidth int64
at int64
}
func (m *mergedAccess) hits(key string, window time.Duration) uint64 {
if m == nil {
return 0
}
e, ok := m.entries[key]
if !ok {
return 0
}
return e.hits(window, m.binWidth)
}
func (m *mergedAccess) lastAccess(key string) int64 {
if m == nil {
return 0
}
return m.entries[key].LastAt
}
// accessTracker records how often each object is read.
//
// Ownership is deliberately narrow: the live counter map is touched only by
// run()'s goroutine, so it needs no lock. Everything read from elsewhere goes
// through the immutable merged snapshot.
type accessTracker struct {
ch chan string
enabled atomic.Bool
merged atomic.Pointer[mergedAccess]
// Demote candidates arrive from scanner goroutines, so this one does
// need a lock. It is small: only objects we previously promoted.
demoteMu sync.Mutex
demote map[string]demoteCandidate
dropped atomic.Uint64
}
var globalAccessTracker = newAccessTracker()
func newAccessTracker() *accessTracker {
return &accessTracker{
ch: make(chan string, accessQueueSize),
demote: make(map[string]demoteCandidate),
}
}
// accessKey is the tracker's map key. Bucket names cannot contain '/', so the
// join is unambiguous.
func accessKey(bucket, object string) string {
return bucket + "/" + object
}
func splitAccessKey(key string) (bucket, object string, ok bool) {
bucket, object, ok = strings.Cut(key, "/")
if !ok || bucket == "" || object == "" {
return "", "", false
}
return bucket, object, true
}
// note records one read. It is called from the GET path and must never block
// or allocate meaningfully: on a full queue the sample is dropped.
func (t *accessTracker) note(bucket, object string) {
if t == nil || !t.enabled.Load() {
return
}
select {
case t.ch <- accessKey(bucket, object):
default:
t.dropped.Add(1)
}
}
// noteDemoteCandidate is called by the scanner for an object it found on a
// fast pool that has gone quiet. The leader picks these up on the next merge.
func (t *accessTracker) noteDemoteCandidate(bucket, object string, pool int) {
if t == nil || !t.enabled.Load() {
return
}
t.demoteMu.Lock()
defer t.demoteMu.Unlock()
if len(t.demote) >= accessMaxDemoteCandidates {
return
}
t.demote[accessKey(bucket, object)] = demoteCandidate{Bucket: bucket, Object: object, Pool: pool}
}
// hits reports cluster-wide accesses to an object over window.
func (t *accessTracker) hits(bucket, object string, window time.Duration) uint64 {
if t == nil {
return 0
}
return t.merged.Load().hits(accessKey(bucket, object), window)
}
// lastAccess reports the cluster-wide time an object was last read. A zero
// time means "no read on record", which for demotion purposes is idle.
func (t *accessTracker) lastAccess(bucket, object string) time.Time {
if t == nil {
return time.Time{}
}
sec := t.merged.Load().lastAccess(accessKey(bucket, object))
if sec == 0 {
return time.Time{}
}
return time.Unix(sec, 0)
}
// snapshot returns the current merged view, or nil if none has been published.
func (t *accessTracker) snapshot() *mergedAccess {
if t == nil {
return nil
}
return t.merged.Load()
}
// takeDemoteCandidates drains and returns the pending candidates.
func (t *accessTracker) takeDemoteCandidates() []demoteCandidate {
t.demoteMu.Lock()
defer t.demoteMu.Unlock()
if len(t.demote) == 0 {
return nil
}
out := make([]demoteCandidate, 0, len(t.demote))
for _, c := range t.demote {
out = append(out, c)
}
t.demote = make(map[string]demoteCandidate)
return out
}
// restoreDemoteCandidates puts candidates back when the publisher is still
// busy. Dropping access samples is acceptable; dropping the only scanner
// discovery of an idle promoted object would delay demotion by a full scan.
func (t *accessTracker) restoreDemoteCandidates(candidates []demoteCandidate) {
if len(candidates) == 0 {
return
}
t.demoteMu.Lock()
defer t.demoteMu.Unlock()
for _, c := range candidates {
if len(t.demote) >= accessMaxDemoteCandidates {
return
}
t.demote[accessKey(c.Bucket, c.Object)] = c
}
}
// shardName is this node's counter object. The node name is hashed so that
// host:port never has to be escaped into an object key.
func (t *accessTracker) shardName() string {
return fmt.Sprintf("%s/%016x.bin", accessTrackerPrefix, xxh3.HashString(globalLocalNodeName))
}
// run owns the live counter map. It drains reads, and on every flush interval
// hands a marshaled shard to a background publisher.
//
// Everything here is best effort: this is accounting for a background data
// movement decision, not a durability path.
func (t *accessTracker) run(ctx context.Context, objAPI ObjectLayer) {
cfg := globalILMConfig.accessCfg()
t.enabled.Store(cfg.AccessTiering)
live := make(map[string]accessEntry)
if cfg.AccessTiering {
if buf, err := readConfig(ctx, objAPI, t.shardName()); err == nil {
var previous accessShard
if _, err = previous.UnmarshalMsg(buf); err == nil && previous.BinWidth == int64(cfg.AccessBinWidth/time.Second) {
now := time.Now().Unix()
for key, entry := range previous.Entries {
entry.rollTo(now, previous.BinWidth, cfg.AccessBins)
if entry.total() != 0 {
live[key] = entry
}
}
}
}
}
ticker := time.NewTicker(cfg.AccessFlush)
defer ticker.Stop()
// One publisher goroutine keeps object-layer I/O off the drain loop.
type pub struct {
shard accessShard
cfg ilm.Config
}
pubCh := make(chan pub, 1)
go func() {
for p := range pubCh {
t.publish(ctx, objAPI, p.shard, p.cfg)
}
}()
defer close(pubCh)
for {
select {
case <-ctx.Done():
return
case key := <-t.ch:
now := time.Now().Unix()
e := live[key]
e.rollTo(now, int64(cfg.AccessBinWidth/time.Second), cfg.AccessBins)
if len(e.Bins) > 0 {
if e.Bins[0] != ^uint32(0) {
e.Bins[0]++
}
}
e.LastAt = now
live[key] = e
case <-ticker.C:
newCfg := globalILMConfig.accessCfg()
t.enabled.Store(newCfg.AccessTiering)
if newCfg.AccessFlush != cfg.AccessFlush && newCfg.AccessFlush > 0 {
ticker.Reset(newCfg.AccessFlush)
}
cfg = newCfg
if !cfg.AccessTiering {
// Feature turned off: release the counters rather than
// holding a stale working set for the process lifetime.
clear(live)
t.merged.Store(nil)
continue
}
now := time.Now().Unix()
binWidth := int64(cfg.AccessBinWidth / time.Second)
t.evict(live, now, binWidth, cfg)
shard := accessShard{
UpdatedAt: now,
BinWidth: binWidth,
Entries: make(map[string]accessEntry, len(live)),
Demote: t.takeDemoteCandidates(),
}
for k, e := range live {
e.Bins = slices.Clone(e.Bins)
shard.Entries[k] = e
}
select {
case pubCh <- pub{shard: shard, cfg: cfg}:
default:
// Previous publish still running; skip this round
// rather than queueing work we cannot keep up with.
t.restoreDemoteCandidates(shard.Demote)
}
}
}
}
// evict rolls every counter forward and drops the ones with no hits left in
// the retained history, then enforces the tracked-object cap.
//
// ponytail: amortized O(n) sweep on the flush tick; a heap would only pay off
// past ~10M tracked keys.
func (t *accessTracker) evict(live map[string]accessEntry, now, binWidth int64, cfg ilm.Config) {
for k, e := range live {
e.rollTo(now, binWidth, cfg.AccessBins)
if e.total() == 0 {
delete(live, k)
continue
}
live[k] = e
}
if len(live) <= cfg.AccessMaxTracked {
return
}
// Over the cap: keep the hottest and drop the rest. Objects that fall
// out are cold by construction, which is exactly what the demotion path
// already assumes about anything missing from the map.
type kt struct {
key string
total uint64
}
all := make([]kt, 0, len(live))
for k, e := range live {
all = append(all, kt{k, e.total()})
}
slices.SortFunc(all, func(a, b kt) int { return cmp.Compare(b.total, a.total) })
for _, x := range all[cfg.AccessMaxTracked:] {
delete(live, x.key)
}
}
// publish writes this node's shard and rebuilds the merged snapshot from every
// node's shard. Both halves are best effort.
func (t *accessTracker) publish(ctx context.Context, objAPI ObjectLayer, shard accessShard, cfg ilm.Config) {
buf, err := shard.MarshalMsg(nil)
if err != nil {
ilmLogIf(ctx, err)
return
}
if err := saveConfig(ctx, objAPI, t.shardName(), buf); err != nil {
ilmLogIf(ctx, err)
// Still rebuild the snapshot below: our own counters are already
// in hand and a stale peer view beats no view.
}
t.merged.Store(t.mergeShards(ctx, objAPI, shard, cfg))
}
// mergeShards sums every live node's counters, including our own in-memory
// shard so this node's most recent reads are never a flush behind.
func (t *accessTracker) mergeShards(ctx context.Context, objAPI ObjectLayer, own accessShard, cfg ilm.Config) *mergedAccess {
now := time.Now().Unix()
binWidth := int64(cfg.AccessBinWidth / time.Second)
out := &mergedAccess{
entries: make(map[string]accessEntry, len(own.Entries)),
binWidth: binWidth,
at: now,
}
add := func(s accessShard) {
if s.BinWidth != binWidth {
// A peer has not yet picked up a bin-width change; merging
// its bins would silently mis-scale the counts.
return
}
for k, e := range s.Entries {
e.rollTo(now, binWidth, cfg.AccessBins)
cur, ok := out.entries[k]
if !ok {
cur = accessEntry{Bins: make([]uint32, cfg.AccessBins)}
}
cur.mergeFrom(e)
out.entries[k] = cur
}
}
add(own)
ownName := t.shardName()
stale := int64(cfg.AccessFlush/time.Second) * accessShardStaleFactor
for _, name := range t.listShards(ctx, objAPI) {
if name == ownName {
continue
}
buf, err := readConfig(ctx, objAPI, name)
if err != nil {
continue
}
var s accessShard
if _, err := s.UnmarshalMsg(buf); err != nil {
continue
}
if !accessShardFresh(now, s, stale) {
continue // node is gone or wedged
}
add(s)
}
return out
}
func (t *accessTracker) listShards(ctx context.Context, objAPI ObjectLayer) []string {
res, err := objAPI.ListObjects(ctx, minioMetaBucket, accessTrackerPrefix+"/", "", "", maxObjectList)
if err != nil {
return nil
}
names := make([]string, 0, len(res.Objects))
for _, o := range res.Objects {
if strings.HasSuffix(o.Name, ".bin") {
names = append(names, o.Name)
}
}
return names
}
// collectDemoteCandidates returns every node's pending demote candidates. Only
// the leader calls this, right before running a demotion pass.
//
// Our own shard is read back rather than skipped: run() drains the local map
// into the published shard, so anything found since the last leader pass lives
// there, not in memory. The local map is still drained here to pick up
// candidates recorded since that publish.
func (t *accessTracker) collectDemoteCandidates(ctx context.Context, objAPI ObjectLayer, cfg ilm.Config) []demoteCandidate {
seen := make(map[string]struct{})
var out []demoteCandidate
for _, c := range t.takeDemoteCandidates() {
key := accessKey(c.Bucket, c.Object)
seen[key] = struct{}{}
out = append(out, c)
}
now := time.Now().Unix()
stale := int64(cfg.AccessFlush/time.Second) * accessShardStaleFactor
for _, name := range t.listShards(ctx, objAPI) {
buf, err := readConfig(ctx, objAPI, name)
if err != nil {
continue
}
var s accessShard
if _, err := s.UnmarshalMsg(buf); err != nil {
continue
}
if !accessShardFresh(now, s, stale) {
continue
}
for _, c := range s.Demote {
key := accessKey(c.Bucket, c.Object)
if _, dup := seen[key]; dup {
continue
}
seen[key] = struct{}{}
out = append(out, c)
}
}
return out
}
cmd/ilm-access-tracker_gen.go
-742
View File
@@ -1,742 +0,0 @@
// Code generated by github.com/tinylib/msgp DO NOT EDIT.
package cmd
import (
"github.com/tinylib/msgp/msgp"
)
// DecodeMsg implements msgp.Decodable
func (z *accessEntry) DecodeMsg(dc *msgp.Reader) (err error) {
var field []byte
_ = field
var zb0001 uint32
zb0001, err = dc.ReadMapHeader()
if err != nil {
err = msgp.WrapError(err)
return
}
for zb0001 > 0 {
zb0001--
field, err = dc.ReadMapKeyPtr()
if err != nil {
err = msgp.WrapError(err)
return
}
switch msgp.UnsafeString(field) {
case "b":
var zb0002 uint32
zb0002, err = dc.ReadArrayHeader()
if err != nil {
err = msgp.WrapError(err, "Bins")
return
}
if cap(z.Bins) >= int(zb0002) {
z.Bins = (z.Bins)[:zb0002]
} else {
z.Bins = make([]uint32, zb0002)
}
for za0001 := range z.Bins {
z.Bins[za0001], err = dc.ReadUint32()
if err != nil {
err = msgp.WrapError(err, "Bins", za0001)
return
}
}
case "h":
z.HeadAt, err = dc.ReadInt64()
if err != nil {
err = msgp.WrapError(err, "HeadAt")
return
}
case "l":
z.LastAt, err = dc.ReadInt64()
if err != nil {
err = msgp.WrapError(err, "LastAt")
return
}
default:
err = dc.Skip()
if err != nil {
err = msgp.WrapError(err)
return
}
}
}
return
}
// EncodeMsg implements msgp.Encodable
func (z *accessEntry) EncodeMsg(en *msgp.Writer) (err error) {
// map header, size 3
// write "b"
err = en.Append(0x83, 0xa1, 0x62)
if err != nil {
return
}
err = en.WriteArrayHeader(uint32(len(z.Bins)))
if err != nil {
err = msgp.WrapError(err, "Bins")
return
}
for za0001 := range z.Bins {
err = en.WriteUint32(z.Bins[za0001])
if err != nil {
err = msgp.WrapError(err, "Bins", za0001)
return
}
}
// write "h"
err = en.Append(0xa1, 0x68)
if err != nil {
return
}
err = en.WriteInt64(z.HeadAt)
if err != nil {
err = msgp.WrapError(err, "HeadAt")
return
}
// write "l"
err = en.Append(0xa1, 0x6c)
if err != nil {
return
}
err = en.WriteInt64(z.LastAt)
if err != nil {
err = msgp.WrapError(err, "LastAt")
return
}
return
}
// MarshalMsg implements msgp.Marshaler
func (z *accessEntry) MarshalMsg(b []byte) (o []byte, err error) {
o = msgp.Require(b, z.Msgsize())
// map header, size 3
// string "b"
o = append(o, 0x83, 0xa1, 0x62)
o = msgp.AppendArrayHeader(o, uint32(len(z.Bins)))
for za0001 := range z.Bins {
o = msgp.AppendUint32(o, z.Bins[za0001])
}
// string "h"
o = append(o, 0xa1, 0x68)
o = msgp.AppendInt64(o, z.HeadAt)
// string "l"
o = append(o, 0xa1, 0x6c)
o = msgp.AppendInt64(o, z.LastAt)
return
}
// UnmarshalMsg implements msgp.Unmarshaler
func (z *accessEntry) UnmarshalMsg(bts []byte) (o []byte, err error) {
var field []byte
_ = field
var zb0001 uint32
zb0001, bts, err = msgp.ReadMapHeaderBytes(bts)
if err != nil {
err = msgp.WrapError(err)
return
}
for zb0001 > 0 {
zb0001--
field, bts, err = msgp.ReadMapKeyZC(bts)
if err != nil {
err = msgp.WrapError(err)
return
}
switch msgp.UnsafeString(field) {
case "b":
var zb0002 uint32
zb0002, bts, err = msgp.ReadArrayHeaderBytes(bts)
if err != nil {
err = msgp.WrapError(err, "Bins")
return
}
if cap(z.Bins) >= int(zb0002) {
z.Bins = (z.Bins)[:zb0002]
} else {
z.Bins = make([]uint32, zb0002)
}
for za0001 := range z.Bins {
z.Bins[za0001], bts, err = msgp.ReadUint32Bytes(bts)
if err != nil {
err = msgp.WrapError(err, "Bins", za0001)
return
}
}
case "h":
z.HeadAt, bts, err = msgp.ReadInt64Bytes(bts)
if err != nil {
err = msgp.WrapError(err, "HeadAt")
return
}
case "l":
z.LastAt, bts, err = msgp.ReadInt64Bytes(bts)
if err != nil {
err = msgp.WrapError(err, "LastAt")
return
}
default:
bts, err = msgp.Skip(bts)
if err != nil {
err = msgp.WrapError(err)
return
}
}
}
o = bts
return
}
// Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message
func (z *accessEntry) Msgsize() (s int) {
s = 1 + 2 + msgp.ArrayHeaderSize + (len(z.Bins) * (msgp.Uint32Size)) + 2 + msgp.Int64Size + 2 + msgp.Int64Size
return
}
// DecodeMsg implements msgp.Decodable
func (z *accessShard) DecodeMsg(dc *msgp.Reader) (err error) {
var field []byte
_ = field
var zb0001 uint32
zb0001, err = dc.ReadMapHeader()
if err != nil {
err = msgp.WrapError(err)
return
}
for zb0001 > 0 {
zb0001--
field, err = dc.ReadMapKeyPtr()
if err != nil {
err = msgp.WrapError(err)
return
}
switch msgp.UnsafeString(field) {
case "u":
z.UpdatedAt, err = dc.ReadInt64()
if err != nil {
err = msgp.WrapError(err, "UpdatedAt")
return
}
case "bw":
z.BinWidth, err = dc.ReadInt64()
if err != nil {
err = msgp.WrapError(err, "BinWidth")
return
}
case "e":
var zb0002 uint32
zb0002, err = dc.ReadMapHeader()
if err != nil {
err = msgp.WrapError(err, "Entries")
return
}
if z.Entries == nil {
z.Entries = make(map[string]accessEntry, zb0002)
} else if len(z.Entries) > 0 {
clear(z.Entries)
}
for zb0002 > 0 {
zb0002--
var za0001 string
za0001, err = dc.ReadString()
if err != nil {
err = msgp.WrapError(err, "Entries")
return
}
var za0002 accessEntry
err = za0002.DecodeMsg(dc)
if err != nil {
err = msgp.WrapError(err, "Entries", za0001)
return
}
z.Entries[za0001] = za0002
}
case "d":
var zb0003 uint32
zb0003, err = dc.ReadArrayHeader()
if err != nil {
err = msgp.WrapError(err, "Demote")
return
}
if cap(z.Demote) >= int(zb0003) {
z.Demote = (z.Demote)[:zb0003]
} else {
z.Demote = make([]demoteCandidate, zb0003)
}
for za0003 := range z.Demote {
var zb0004 uint32
zb0004, err = dc.ReadMapHeader()
if err != nil {
err = msgp.WrapError(err, "Demote", za0003)
return
}
for zb0004 > 0 {
zb0004--
field, err = dc.ReadMapKeyPtr()
if err != nil {
err = msgp.WrapError(err, "Demote", za0003)
return
}
switch msgp.UnsafeString(field) {
case "b":
z.Demote[za0003].Bucket, err = dc.ReadString()
if err != nil {
err = msgp.WrapError(err, "Demote", za0003, "Bucket")
return
}
case "o":
z.Demote[za0003].Object, err = dc.ReadString()
if err != nil {
err = msgp.WrapError(err, "Demote", za0003, "Object")
return
}
case "p":
z.Demote[za0003].Pool, err = dc.ReadInt()
if err != nil {
err = msgp.WrapError(err, "Demote", za0003, "Pool")
return
}
default:
err = dc.Skip()
if err != nil {
err = msgp.WrapError(err, "Demote", za0003)
return
}
}
}
}
default:
err = dc.Skip()
if err != nil {
err = msgp.WrapError(err)
return
}
}
}
return
}
// EncodeMsg implements msgp.Encodable
func (z *accessShard) EncodeMsg(en *msgp.Writer) (err error) {
// map header, size 4
// write "u"
err = en.Append(0x84, 0xa1, 0x75)
if err != nil {
return
}
err = en.WriteInt64(z.UpdatedAt)
if err != nil {
err = msgp.WrapError(err, "UpdatedAt")
return
}
// write "bw"
err = en.Append(0xa2, 0x62, 0x77)
if err != nil {
return
}
err = en.WriteInt64(z.BinWidth)
if err != nil {
err = msgp.WrapError(err, "BinWidth")
return
}
// write "e"
err = en.Append(0xa1, 0x65)
if err != nil {
return
}
err = en.WriteMapHeader(uint32(len(z.Entries)))
if err != nil {
err = msgp.WrapError(err, "Entries")
return
}
for za0001, za0002 := range z.Entries {
err = en.WriteString(za0001)
if err != nil {
err = msgp.WrapError(err, "Entries")
return
}
err = za0002.EncodeMsg(en)
if err != nil {
err = msgp.WrapError(err, "Entries", za0001)
return
}
}
// write "d"
err = en.Append(0xa1, 0x64)
if err != nil {
return
}
err = en.WriteArrayHeader(uint32(len(z.Demote)))
if err != nil {
err = msgp.WrapError(err, "Demote")
return
}
for za0003 := range z.Demote {
// map header, size 3
// write "b"
err = en.Append(0x83, 0xa1, 0x62)
if err != nil {
return
}
err = en.WriteString(z.Demote[za0003].Bucket)
if err != nil {
err = msgp.WrapError(err, "Demote", za0003, "Bucket")
return
}
// write "o"
err = en.Append(0xa1, 0x6f)
if err != nil {
return
}
err = en.WriteString(z.Demote[za0003].Object)
if err != nil {
err = msgp.WrapError(err, "Demote", za0003, "Object")
return
}
// write "p"
err = en.Append(0xa1, 0x70)
if err != nil {
return
}
err = en.WriteInt(z.Demote[za0003].Pool)
if err != nil {
err = msgp.WrapError(err, "Demote", za0003, "Pool")
return
}
}
return
}
// MarshalMsg implements msgp.Marshaler
func (z *accessShard) MarshalMsg(b []byte) (o []byte, err error) {
o = msgp.Require(b, z.Msgsize())
// map header, size 4
// string "u"
o = append(o, 0x84, 0xa1, 0x75)
o = msgp.AppendInt64(o, z.UpdatedAt)
// string "bw"
o = append(o, 0xa2, 0x62, 0x77)
o = msgp.AppendInt64(o, z.BinWidth)
// string "e"
o = append(o, 0xa1, 0x65)
o = msgp.AppendMapHeader(o, uint32(len(z.Entries)))
for za0001, za0002 := range z.Entries {
o = msgp.AppendString(o, za0001)
o, err = za0002.MarshalMsg(o)
if err != nil {
err = msgp.WrapError(err, "Entries", za0001)
return
}
}
// string "d"
o = append(o, 0xa1, 0x64)
o = msgp.AppendArrayHeader(o, uint32(len(z.Demote)))
for za0003 := range z.Demote {
// map header, size 3
// string "b"
o = append(o, 0x83, 0xa1, 0x62)
o = msgp.AppendString(o, z.Demote[za0003].Bucket)
// string "o"
o = append(o, 0xa1, 0x6f)
o = msgp.AppendString(o, z.Demote[za0003].Object)
// string "p"
o = append(o, 0xa1, 0x70)
o = msgp.AppendInt(o, z.Demote[za0003].Pool)
}
return
}
// UnmarshalMsg implements msgp.Unmarshaler
func (z *accessShard) UnmarshalMsg(bts []byte) (o []byte, err error) {
var field []byte
_ = field
var zb0001 uint32
zb0001, bts, err = msgp.ReadMapHeaderBytes(bts)
if err != nil {
err = msgp.WrapError(err)
return
}
for zb0001 > 0 {
zb0001--
field, bts, err = msgp.ReadMapKeyZC(bts)
if err != nil {
err = msgp.WrapError(err)
return
}
switch msgp.UnsafeString(field) {
case "u":
z.UpdatedAt, bts, err = msgp.ReadInt64Bytes(bts)
if err != nil {
err = msgp.WrapError(err, "UpdatedAt")
return
}
case "bw":
z.BinWidth, bts, err = msgp.ReadInt64Bytes(bts)
if err != nil {
err = msgp.WrapError(err, "BinWidth")
return
}
case "e":
var zb0002 uint32
zb0002, bts, err = msgp.ReadMapHeaderBytes(bts)
if err != nil {
err = msgp.WrapError(err, "Entries")
return
}
if z.Entries == nil {
z.Entries = make(map[string]accessEntry, zb0002)
} else if len(z.Entries) > 0 {
clear(z.Entries)
}
for zb0002 > 0 {
var za0002 accessEntry
zb0002--
var za0001 string
za0001, bts, err = msgp.ReadStringBytes(bts)
if err != nil {
err = msgp.WrapError(err, "Entries")
return
}
bts, err = za0002.UnmarshalMsg(bts)
if err != nil {
err = msgp.WrapError(err, "Entries", za0001)
return
}
z.Entries[za0001] = za0002
}
case "d":
var zb0003 uint32
zb0003, bts, err = msgp.ReadArrayHeaderBytes(bts)
if err != nil {
err = msgp.WrapError(err, "Demote")
return
}
if cap(z.Demote) >= int(zb0003) {
z.Demote = (z.Demote)[:zb0003]
} else {
z.Demote = make([]demoteCandidate, zb0003)
}
for za0003 := range z.Demote {
var zb0004 uint32
zb0004, bts, err = msgp.ReadMapHeaderBytes(bts)
if err != nil {
err = msgp.WrapError(err, "Demote", za0003)
return
}
for zb0004 > 0 {
zb0004--
field, bts, err = msgp.ReadMapKeyZC(bts)
if err != nil {
err = msgp.WrapError(err, "Demote", za0003)
return
}
switch msgp.UnsafeString(field) {
case "b":
z.Demote[za0003].Bucket, bts, err = msgp.ReadStringBytes(bts)
if err != nil {
err = msgp.WrapError(err, "Demote", za0003, "Bucket")
return
}
case "o":
z.Demote[za0003].Object, bts, err = msgp.ReadStringBytes(bts)
if err != nil {
err = msgp.WrapError(err, "Demote", za0003, "Object")
return
}
case "p":
z.Demote[za0003].Pool, bts, err = msgp.ReadIntBytes(bts)
if err != nil {
err = msgp.WrapError(err, "Demote", za0003, "Pool")
return
}
default:
bts, err = msgp.Skip(bts)
if err != nil {
err = msgp.WrapError(err, "Demote", za0003)
return
}
}
}
}
default:
bts, err = msgp.Skip(bts)
if err != nil {
err = msgp.WrapError(err)
return
}
}
}
o = bts
return
}
// Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message
func (z *accessShard) Msgsize() (s int) {
s = 1 + 2 + msgp.Int64Size + 3 + msgp.Int64Size + 2 + msgp.MapHeaderSize
if z.Entries != nil {
for za0001, za0002 := range z.Entries {
_ = za0002
s += msgp.StringPrefixSize + len(za0001) + za0002.Msgsize()
}
}
s += 2 + msgp.ArrayHeaderSize
for za0003 := range z.Demote {
s += 1 + 2 + msgp.StringPrefixSize + len(z.Demote[za0003].Bucket) + 2 + msgp.StringPrefixSize + len(z.Demote[za0003].Object) + 2 + msgp.IntSize
}
return
}
// DecodeMsg implements msgp.Decodable
func (z *demoteCandidate) DecodeMsg(dc *msgp.Reader) (err error) {
var field []byte
_ = field
var zb0001 uint32
zb0001, err = dc.ReadMapHeader()
if err != nil {
err = msgp.WrapError(err)
return
}
for zb0001 > 0 {
zb0001--
field, err = dc.ReadMapKeyPtr()
if err != nil {
err = msgp.WrapError(err)
return
}
switch msgp.UnsafeString(field) {
case "b":
z.Bucket, err = dc.ReadString()
if err != nil {
err = msgp.WrapError(err, "Bucket")
return
}
case "o":
z.Object, err = dc.ReadString()
if err != nil {
err = msgp.WrapError(err, "Object")
return
}
case "p":
z.Pool, err = dc.ReadInt()
if err != nil {
err = msgp.WrapError(err, "Pool")
return
}
default:
err = dc.Skip()
if err != nil {
err = msgp.WrapError(err)
return
}
}
}
return
}
// EncodeMsg implements msgp.Encodable
func (z demoteCandidate) EncodeMsg(en *msgp.Writer) (err error) {
// map header, size 3
// write "b"
err = en.Append(0x83, 0xa1, 0x62)
if err != nil {
return
}
err = en.WriteString(z.Bucket)
if err != nil {
err = msgp.WrapError(err, "Bucket")
return
}
// write "o"
err = en.Append(0xa1, 0x6f)
if err != nil {
return
}
err = en.WriteString(z.Object)
if err != nil {
err = msgp.WrapError(err, "Object")
return
}
// write "p"
err = en.Append(0xa1, 0x70)
if err != nil {
return
}
err = en.WriteInt(z.Pool)
if err != nil {
err = msgp.WrapError(err, "Pool")
return
}
return
}
// MarshalMsg implements msgp.Marshaler
func (z demoteCandidate) MarshalMsg(b []byte) (o []byte, err error) {
o = msgp.Require(b, z.Msgsize())
// map header, size 3
// string "b"
o = append(o, 0x83, 0xa1, 0x62)
o = msgp.AppendString(o, z.Bucket)
// string "o"
o = append(o, 0xa1, 0x6f)
o = msgp.AppendString(o, z.Object)
// string "p"
o = append(o, 0xa1, 0x70)
o = msgp.AppendInt(o, z.Pool)
return
}
// UnmarshalMsg implements msgp.Unmarshaler
func (z *demoteCandidate) UnmarshalMsg(bts []byte) (o []byte, err error) {
var field []byte
_ = field
var zb0001 uint32
zb0001, bts, err = msgp.ReadMapHeaderBytes(bts)
if err != nil {
err = msgp.WrapError(err)
return
}
for zb0001 > 0 {
zb0001--
field, bts, err = msgp.ReadMapKeyZC(bts)
if err != nil {
err = msgp.WrapError(err)
return
}
switch msgp.UnsafeString(field) {
case "b":
z.Bucket, bts, err = msgp.ReadStringBytes(bts)
if err != nil {
err = msgp.WrapError(err, "Bucket")
return
}
case "o":
z.Object, bts, err = msgp.ReadStringBytes(bts)
if err != nil {
err = msgp.WrapError(err, "Object")
return
}
case "p":
z.Pool, bts, err = msgp.ReadIntBytes(bts)
if err != nil {
err = msgp.WrapError(err, "Pool")
return
}
default:
bts, err = msgp.Skip(bts)
if err != nil {
err = msgp.WrapError(err)
return
}
}
}
o = bts
return
}
// Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message
func (z demoteCandidate) Msgsize() (s int) {
s = 1 + 2 + msgp.StringPrefixSize + len(z.Bucket) + 2 + msgp.StringPrefixSize + len(z.Object) + 2 + msgp.IntSize
return
}
cmd/ilm-access-tracker_gen_test.go
-349
View File
@@ -1,349 +0,0 @@
// Code generated by github.com/tinylib/msgp DO NOT EDIT.
package cmd
import (
"bytes"
"testing"
"github.com/tinylib/msgp/msgp"
)
func TestMarshalUnmarshalaccessEntry(t *testing.T) {
v := accessEntry{}
bts, err := v.MarshalMsg(nil)
if err != nil {
t.Fatal(err)
}
left, err := v.UnmarshalMsg(bts)
if err != nil {
t.Fatal(err)
}
if len(left) > 0 {
t.Errorf("%d bytes left over after UnmarshalMsg(): %q", len(left), left)
}
left, err = msgp.Skip(bts)
if err != nil {
t.Fatal(err)
}
if len(left) > 0 {
t.Errorf("%d bytes left over after Skip(): %q", len(left), left)
}
}
func BenchmarkMarshalMsgaccessEntry(b *testing.B) {
v := accessEntry{}
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
v.MarshalMsg(nil)
}
}
func BenchmarkAppendMsgaccessEntry(b *testing.B) {
v := accessEntry{}
bts := make([]byte, 0, v.Msgsize())
bts, _ = v.MarshalMsg(bts[0:0])
b.SetBytes(int64(len(bts)))
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
bts, _ = v.MarshalMsg(bts[0:0])
}
}
func BenchmarkUnmarshalaccessEntry(b *testing.B) {
v := accessEntry{}
bts, _ := v.MarshalMsg(nil)
b.ReportAllocs()
b.SetBytes(int64(len(bts)))
b.ResetTimer()
for i := 0; i < b.N; i++ {
_, err := v.UnmarshalMsg(bts)
if err != nil {
b.Fatal(err)
}
}
}
func TestEncodeDecodeaccessEntry(t *testing.T) {
v := accessEntry{}
var buf bytes.Buffer
msgp.Encode(&buf, &v)
m := v.Msgsize()
if buf.Len() > m {
t.Log("WARNING: TestEncodeDecodeaccessEntry Msgsize() is inaccurate")
}
vn := accessEntry{}
err := msgp.Decode(&buf, &vn)
if err != nil {
t.Error(err)
}
buf.Reset()
msgp.Encode(&buf, &v)
err = msgp.NewReader(&buf).Skip()
if err != nil {
t.Error(err)
}
}
func BenchmarkEncodeaccessEntry(b *testing.B) {
v := accessEntry{}
var buf bytes.Buffer
msgp.Encode(&buf, &v)
b.SetBytes(int64(buf.Len()))
en := msgp.NewWriter(msgp.Nowhere)
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
v.EncodeMsg(en)
}
en.Flush()
}
func BenchmarkDecodeaccessEntry(b *testing.B) {
v := accessEntry{}
var buf bytes.Buffer
msgp.Encode(&buf, &v)
b.SetBytes(int64(buf.Len()))
rd := msgp.NewEndlessReader(buf.Bytes(), b)
dc := msgp.NewReader(rd)
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
err := v.DecodeMsg(dc)
if err != nil {
b.Fatal(err)
}
}
}
func TestMarshalUnmarshalaccessShard(t *testing.T) {
v := accessShard{}
bts, err := v.MarshalMsg(nil)
if err != nil {
t.Fatal(err)
}
left, err := v.UnmarshalMsg(bts)
if err != nil {
t.Fatal(err)
}
if len(left) > 0 {
t.Errorf("%d bytes left over after UnmarshalMsg(): %q", len(left), left)
}
left, err = msgp.Skip(bts)
if err != nil {
t.Fatal(err)
}
if len(left) > 0 {
t.Errorf("%d bytes left over after Skip(): %q", len(left), left)
}
}
func BenchmarkMarshalMsgaccessShard(b *testing.B) {
v := accessShard{}
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
v.MarshalMsg(nil)
}
}
func BenchmarkAppendMsgaccessShard(b *testing.B) {
v := accessShard{}
bts := make([]byte, 0, v.Msgsize())
bts, _ = v.MarshalMsg(bts[0:0])
b.SetBytes(int64(len(bts)))
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
bts, _ = v.MarshalMsg(bts[0:0])
}
}
func BenchmarkUnmarshalaccessShard(b *testing.B) {
v := accessShard{}
bts, _ := v.MarshalMsg(nil)
b.ReportAllocs()
b.SetBytes(int64(len(bts)))
b.ResetTimer()
for i := 0; i < b.N; i++ {
_, err := v.UnmarshalMsg(bts)
if err != nil {
b.Fatal(err)
}
}
}
func TestEncodeDecodeaccessShard(t *testing.T) {
v := accessShard{}
var buf bytes.Buffer
msgp.Encode(&buf, &v)
m := v.Msgsize()
if buf.Len() > m {
t.Log("WARNING: TestEncodeDecodeaccessShard Msgsize() is inaccurate")
}
vn := accessShard{}
err := msgp.Decode(&buf, &vn)
if err != nil {
t.Error(err)
}
buf.Reset()
msgp.Encode(&buf, &v)
err = msgp.NewReader(&buf).Skip()
if err != nil {
t.Error(err)
}
}
func BenchmarkEncodeaccessShard(b *testing.B) {
v := accessShard{}
var buf bytes.Buffer
msgp.Encode(&buf, &v)
b.SetBytes(int64(buf.Len()))
en := msgp.NewWriter(msgp.Nowhere)
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
v.EncodeMsg(en)
}
en.Flush()
}
func BenchmarkDecodeaccessShard(b *testing.B) {
v := accessShard{}
var buf bytes.Buffer
msgp.Encode(&buf, &v)
b.SetBytes(int64(buf.Len()))
rd := msgp.NewEndlessReader(buf.Bytes(), b)
dc := msgp.NewReader(rd)
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
err := v.DecodeMsg(dc)
if err != nil {
b.Fatal(err)
}
}
}
func TestMarshalUnmarshaldemoteCandidate(t *testing.T) {
v := demoteCandidate{}
bts, err := v.MarshalMsg(nil)
if err != nil {
t.Fatal(err)
}
left, err := v.UnmarshalMsg(bts)
if err != nil {
t.Fatal(err)
}
if len(left) > 0 {
t.Errorf("%d bytes left over after UnmarshalMsg(): %q", len(left), left)
}
left, err = msgp.Skip(bts)
if err != nil {
t.Fatal(err)
}
if len(left) > 0 {
t.Errorf("%d bytes left over after Skip(): %q", len(left), left)
}
}
func BenchmarkMarshalMsgdemoteCandidate(b *testing.B) {
v := demoteCandidate{}
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
v.MarshalMsg(nil)
}
}
func BenchmarkAppendMsgdemoteCandidate(b *testing.B) {
v := demoteCandidate{}
bts := make([]byte, 0, v.Msgsize())
bts, _ = v.MarshalMsg(bts[0:0])
b.SetBytes(int64(len(bts)))
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
bts, _ = v.MarshalMsg(bts[0:0])
}
}
func BenchmarkUnmarshaldemoteCandidate(b *testing.B) {
v := demoteCandidate{}
bts, _ := v.MarshalMsg(nil)
b.ReportAllocs()
b.SetBytes(int64(len(bts)))
b.ResetTimer()
for i := 0; i < b.N; i++ {
_, err := v.UnmarshalMsg(bts)
if err != nil {
b.Fatal(err)
}
}
}
func TestEncodeDecodedemoteCandidate(t *testing.T) {
v := demoteCandidate{}
var buf bytes.Buffer
msgp.Encode(&buf, &v)
m := v.Msgsize()
if buf.Len() > m {
t.Log("WARNING: TestEncodeDecodedemoteCandidate Msgsize() is inaccurate")
}
vn := demoteCandidate{}
err := msgp.Decode(&buf, &vn)
if err != nil {
t.Error(err)
}
buf.Reset()
msgp.Encode(&buf, &v)
err = msgp.NewReader(&buf).Skip()
if err != nil {
t.Error(err)
}
}
func BenchmarkEncodedemoteCandidate(b *testing.B) {
v := demoteCandidate{}
var buf bytes.Buffer
msgp.Encode(&buf, &v)
b.SetBytes(int64(buf.Len()))
en := msgp.NewWriter(msgp.Nowhere)
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
v.EncodeMsg(en)
}
en.Flush()
}
func BenchmarkDecodedemoteCandidate(b *testing.B) {
v := demoteCandidate{}
var buf bytes.Buffer
msgp.Encode(&buf, &v)
b.SetBytes(int64(buf.Len()))
rd := msgp.NewEndlessReader(buf.Bytes(), b)
dc := msgp.NewReader(rd)
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
err := v.DecodeMsg(dc)
if err != nil {
b.Fatal(err)
}
}
}
cmd/ilm-access-tracker_test.go
-95
View File
@@ -1,95 +0,0 @@
// Copyright (c) 2015-2026 MinIO, Inc.
package cmd
import (
"math"
"testing"
"time"
"github.com/minio/minio/internal/config/ilm"
)
func TestAccessEntryRollAndHits(t *testing.T) {
entry := accessEntry{Bins: []uint32{3, 2, 1}, HeadAt: 100, LastAt: 107}
entry.rollTo(120, 10, 3)
want := []uint32{0, 0, 3}
for i := range want {
if entry.Bins[i] != want[i] {
t.Fatalf("bins = %v, want %v", entry.Bins, want)
}
}
if entry.HeadAt != 120 || entry.LastAt != 107 {
t.Fatalf("head/last = %d/%d", entry.HeadAt, entry.LastAt)
}
entry = accessEntry{Bins: []uint32{10, 20, 30}, HeadAt: 120}
if got := entry.hits(20*time.Second, 10); got != 30 {
t.Fatalf("20s hits = %d, want 30", got)
}
if got := entry.hits(21*time.Second, 10); got != 60 {
t.Fatalf("21s hits = %d, want 60", got)
}
entry.rollTo(200, 10, 3)
if got := entry.total(); got != 0 {
t.Fatalf("expired total = %d, want 0", got)
}
}
func TestAccessEntryMergeSaturates(t *testing.T) {
entry := accessEntry{Bins: []uint32{math.MaxUint32 - 1}}
entry.mergeFrom(accessEntry{Bins: []uint32{10}, LastAt: 50})
if entry.Bins[0] != math.MaxUint32 || entry.LastAt != 50 {
t.Fatalf("merged entry = %+v", entry)
}
}
func TestAccessTrackerEvictsColdest(t *testing.T) {
tracker := newAccessTracker()
live := map[string]accessEntry{
"a": {Bins: []uint32{1}, HeadAt: 100},
"b": {Bins: []uint32{5}, HeadAt: 100},
"c": {Bins: []uint32{3}, HeadAt: 100},
}
tracker.evict(live, 100, 10, ilm.Config{AccessBins: 1, AccessMaxTracked: 2})
if len(live) != 2 {
t.Fatalf("len = %d, want 2", len(live))
}
if _, ok := live["a"]; ok {
t.Fatal("coldest entry was retained")
}
}
func TestAccessKeyRoundTrip(t *testing.T) {
key := accessKey("bucket", "a/b/c")
bucket, object, ok := splitAccessKey(key)
if !ok || bucket != "bucket" || object != "a/b/c" {
t.Fatalf("split %q = %q/%q/%v", key, bucket, object, ok)
}
}
func TestAccessShardFresh(t *testing.T) {
const now = int64(1000)
if !accessShardFresh(now, accessShard{UpdatedAt: 950}, 100) {
t.Fatal("fresh shard rejected")
}
if accessShardFresh(now, accessShard{UpdatedAt: 899}, 100) {
t.Fatal("stale shard accepted")
}
if accessShardFresh(now, accessShard{UpdatedAt: 1101}, 100) {
t.Fatal("far-future shard accepted")
}
if accessShardFresh(now, accessShard{}, 100) {
t.Fatal("zero timestamp accepted")
}
}
func TestAccessTrackerRestoresDemoteCandidates(t *testing.T) {
tracker := newAccessTracker()
candidate := demoteCandidate{Bucket: "bucket", Object: "object", Pool: 1}
tracker.restoreDemoteCandidates([]demoteCandidate{candidate})
got := tracker.takeDemoteCandidates()
if len(got) != 1 || got[0] != candidate {
t.Fatalf("restored candidates = %+v, want %+v", got, candidate)
}
}
cmd/ilm-config.go
-30
View File
@@ -19,7 +19,6 @@ package cmd
import (
"sync"
"time"
"github.com/minio/minio/internal/config/ilm"
)
@@ -28,16 +27,6 @@ var globalILMConfig = ilmConfig{
cfg: ilm.Config{
ExpirationWorkers: 100,
TransitionWorkers: 100,
// Access tiering stays off until configured, but the counter
// geometry must be sane from the start: the tracker divides by
// AccessBinWidth before any config is loaded.
AccessPromoteWatermark: 85,
AccessBinWidth: time.Minute,
AccessBins: 12,
AccessFlush: time.Minute,
AccessMinResidency: 24 * time.Hour,
AccessWorkers: 10,
AccessMaxTracked: 1000000,
},
}
@@ -60,25 +49,6 @@ func (c *ilmConfig) getTransitionWorkers() int {
return c.cfg.TransitionWorkers
}
// accessCfg returns a copy of the access tiering settings. Callers take the
// whole struct rather than one getter per field because the promotion and
// demotion paths need a consistent view of several knobs at once.
func (c *ilmConfig) accessCfg() ilm.Config {
c.mu.RLock()
defer c.mu.RUnlock()
return c.cfg
}
// accessTieringEnabled is the cheap gate used on the scanner path.
func (c *ilmConfig) accessTieringEnabled() bool {
c.mu.RLock()
defer c.mu.RUnlock()
_, ok := c.cfg.HotPool()
return ok
}
func (c *ilmConfig) update(cfg ilm.Config) {
c.mu.Lock()
defer c.mu.Unlock()
cmd/lceventsrc_string.go
+2 -3
View File
@@ -19,12 +19,11 @@ func _() {
_ = x[lcEventSrc_s3PutObject-8]
_ = x[lcEventSrc_s3CopyObject-9]
_ = x[lcEventSrc_s3CompleteMultipartUpload-10]
_ = x[lcEventSrc_AccessTier-11]
}
const _lcEventSrc_name = "NoneHealScannerDecomRebals3HeadObjects3GetObjects3ListObjectss3PutObjects3CopyObjects3CompleteMultipartUploadAccessTier"
const _lcEventSrc_name = "NoneHealScannerDecomRebals3HeadObjects3GetObjects3ListObjectss3PutObjects3CopyObjects3CompleteMultipartUpload"
var _lcEventSrc_index = [...]uint8{0, 4, 8, 15, 20, 25, 37, 48, 61, 72, 84, 109, 119}
var _lcEventSrc_index = [...]uint8{0, 4, 8, 15, 20, 25, 37, 48, 61, 72, 84, 109}
func (i lcEventSrc) String() string {
idx := int(i) - 0
cmd/metrics-v3-ilm.go
-37
View File
@@ -26,17 +26,6 @@ const (
transitionActiveTasks = "transition_active_tasks"
transitionPendingTasks = "transition_pending_tasks"
transitionMissedImmediateTasks = "transition_missed_immediate_tasks"
accessTierActiveTasks = "access_tier_active_tasks"
accessTierPendingTasks = "access_tier_pending_tasks"
accessTierPromotionsTotal = "access_tier_promotions_total"
accessTierDemotionsTotal = "access_tier_demotions_total"
accessTierBytesMovedTotal = "access_tier_bytes_moved_total"
accessTierFailuresTotal = "access_tier_failures_total"
accessTierSkippedWatermark = "access_tier_skipped_watermark_total"
accessTierSkippedMaxSize = "access_tier_skipped_max_size_total"
accessTierSkippedQuota = "access_tier_skipped_bucket_quota_total"
accessTierHotBytes = "access_tier_hot_bytes"
accessTierSamplesDropped = "access_tier_samples_dropped_total"
versionsScanned = "versions_scanned"
)
@@ -45,17 +34,6 @@ var (
ilmTransitionActiveTasksMD = NewGaugeMD(transitionActiveTasks, "Number of active ILM transition tasks")
ilmTransitionPendingTasksMD = NewGaugeMD(transitionPendingTasks, "Number of pending ILM transition tasks in the queue")
ilmTransitionMissedImmediateTasksMD = NewCounterMD(transitionMissedImmediateTasks, "Number of missed immediate ILM transition tasks")
ilmAccessTierActiveTasksMD = NewGaugeMD(accessTierActiveTasks, "Number of active access-tier pool moves")
ilmAccessTierPendingTasksMD = NewGaugeMD(accessTierPendingTasks, "Number of pending access-tier pool moves")
ilmAccessTierPromotionsTotalMD = NewCounterMD(accessTierPromotionsTotal, "Total objects promoted by access-tier ILM")
ilmAccessTierDemotionsTotalMD = NewCounterMD(accessTierDemotionsTotal, "Total objects demoted by access-tier ILM")
ilmAccessTierBytesMovedTotalMD = NewCounterMD(accessTierBytesMovedTotal, "Total logical bytes moved by access-tier ILM")
ilmAccessTierFailuresTotalMD = NewCounterMD(accessTierFailuresTotal, "Total failed access-tier ILM moves")
ilmAccessTierSkippedWatermarkMD = NewCounterMD(accessTierSkippedWatermark, "Promotions skipped because the hot pool reached its watermark")
ilmAccessTierSkippedMaxSizeMD = NewCounterMD(accessTierSkippedMaxSize, "Promotions skipped because the cluster hot-tier size cap was reached")
ilmAccessTierSkippedQuotaMD = NewCounterMD(accessTierSkippedQuota, "Promotions skipped because the bucket hot-tier quota was reached")
ilmAccessTierHotBytesMD = NewGaugeMD(accessTierHotBytes, "Logical bytes currently accounted to the hot tier", "bucket")
ilmAccessTierSamplesDroppedMD = NewCounterMD(accessTierSamplesDropped, "GET samples dropped because the access tracker queue was full")
ilmVersionsScannedMD = NewCounterMD(versionsScanned, "Total number of object versions checked for ILM actions since server start")
)
@@ -69,21 +47,6 @@ func loadILMMetrics(_ context.Context, m MetricValues, _ *metricsCache) error {
m.Set(transitionPendingTasks, float64(globalTransitionState.PendingTasks()))
m.Set(transitionMissedImmediateTasks, float64(globalTransitionState.MissedImmediateTasks()))
}
if globalAccessTierState != nil {
m.Set(accessTierActiveTasks, float64(globalAccessTierState.ActiveTasks()))
m.Set(accessTierPendingTasks, float64(globalAccessTierState.PendingTasks()))
m.Set(accessTierPromotionsTotal, float64(globalAccessTierState.promotions.Load()))
m.Set(accessTierDemotionsTotal, float64(globalAccessTierState.demotions.Load()))
m.Set(accessTierBytesMovedTotal, float64(globalAccessTierState.bytesMoved.Load()))
m.Set(accessTierFailuresTotal, float64(globalAccessTierState.failures.Load()))
m.Set(accessTierSkippedWatermark, float64(globalAccessTierState.skippedWatermark.Load()))
m.Set(accessTierSkippedMaxSize, float64(globalAccessTierState.skippedMaxSize.Load()))
m.Set(accessTierSkippedQuota, float64(globalAccessTierState.skippedQuota.Load()))
for bucket, bytes := range globalAccessTierState.hotUsageSnapshot() {
m.Set(accessTierHotBytes, float64(bytes), "bucket", bucket)
}
}
m.Set(accessTierSamplesDropped, float64(globalAccessTracker.dropped.Load()))
m.Set(versionsScanned, float64(globalScannerMetrics.lifetime(scannerMetricILM)))
return nil
cmd/metrics-v3.go
-11
View File
@@ -392,17 +392,6 @@ func newMetricGroups(r *prometheus.Registry) *metricsV3Collection {
ilmTransitionActiveTasksMD,
ilmTransitionPendingTasksMD,
ilmTransitionMissedImmediateTasksMD,
ilmAccessTierActiveTasksMD,
ilmAccessTierPendingTasksMD,
ilmAccessTierPromotionsTotalMD,
ilmAccessTierDemotionsTotalMD,
ilmAccessTierBytesMovedTotalMD,
ilmAccessTierFailuresTotalMD,
ilmAccessTierSkippedWatermarkMD,
ilmAccessTierSkippedMaxSizeMD,
ilmAccessTierSkippedQuotaMD,
ilmAccessTierHotBytesMD,
ilmAccessTierSamplesDroppedMD,
ilmVersionsScannedMD,
},
loadILMMetrics,
cmd/object-api-interface.go
-4
View File
@@ -122,10 +122,6 @@ type ObjectOptions struct {
SkipRebalancing bool
SrcPoolIdx int // set by PutObject/CompleteMultipart operations due to rebalance; used to prevent rebalance src, dst pools to be the same
// DstPoolIdx forces a data-movement write onto a specific server pool.
// It is ignored unless DataMovement is true; a pointer keeps pool zero
// distinguishable from the unset value.
DstPoolIdx *int
DataMovement bool // indicates an going decommisionning or rebalacing
cmd/object-handlers.go
-2
View File
@@ -576,8 +576,6 @@ func (api objectAPIHandlers) getObjectHandler(ctx context.Context, objectAPI Obj
return
}
globalAccessTracker.note(bucket, object)
// Notify object accessed via a GET request.
sendEvent(eventArgs{
EventName: event.ObjectAccessedGet,
cmd/server-main.go
-5
View File
@@ -501,7 +501,6 @@ func initAllSubsystems(ctx context.Context) {
globalTierConfigMgr = NewTierConfigMgr()
globalTransitionState = newTransitionState(GlobalContext)
globalAccessTierState = newAccessTierState(GlobalContext)
globalSiteResyncMetrics = newSiteResyncMetrics(GlobalContext)
}
@@ -1070,10 +1069,6 @@ func serverMain(ctx *cli.Context) {
bootstrapTrace("globalTransitionState.Init", func() {
globalTransitionState.Init(newObject)
})
bootstrapTrace("globalAccessTierState.Init", func() {
globalAccessTierState.Init(newObject)
go globalAccessTracker.run(GlobalContext, newObject)
})
go func() {
// Initialize transition tier configuration manager
cmd/testdata/data-usage-v9/README.md
Vendored
+27
View File
@@ -0,0 +1,27 @@
# Data-usage v9 retirement fixture
Generated using unmodified `scanDataFolder` and `dataUsageCache.serializeTo` from
Server commit `89637554d60c27cfc51d2281d0a4fe15e415f06d` (Go 1.27.1, darwin/arm64).
The scanner visits three real local files; its size callback supplies synthetic
version/delete-marker/remote-tier summaries, following `TestDataUsageCacheSerialize`.
It is a scanner/cache compatibility fixture, not a distributed object-store test.
The old scanner counts 74,962 bytes, 3 objects, 5 versions and 2 delete markers.
Its nonzero retired `hts` totals 9,426 bytes. Remote tier `COLD` contains 65,536
bytes, one version and one object. The cache includes nested children, both
histograms, a fixed timestamp and scanner cycle 42. The JSON is the old scanner's
complete expected cache (including `HotTierSize`, which the new reader ignores).
Binary SHA-256: `4c9c7e94cea7758fe9b498b19f639188764b2787582783e6cc950f2c44d52a8b`.
To regenerate, create a detached worktree at that exact source commit, copy
`generate.go.txt` to `cmd/retirement-fixture_test.go`, and run:
```sh
SILO_RETIRE_FIXTURE_DIR=/absolute/output/directory go test ./cmd -run '^TestGenerateAccessRetirementV9Fixture$' -count=1 -v
```
The historical production writer adds the version byte, compresses with zstd
and encodes msgp, including the nonzero `hts` field. Do not regenerate using the
retired implementation or by changing the header of a v8 payload. Map order may
change serialized bytes across regeneration; compare the decoded full cache.
cmd/testdata/data-usage-v9/data-usage-v9.bin
Vendored
BIN
View File
Binary file not shown.
cmd/testdata/data-usage-v9/data-usage-v9.json
Vendored
+118
View File
@@ -0,0 +1,118 @@
{
"Info": {
"Name": "/",
"NextCycle": 42,
"LastUpdate": "2026-09-15T00:00:00Z",
"SkipHealing": true
},
"Cache": {
"/": {
"Children": {
"v9-bucket": {}
},
"Size": 0,
"HotTierSize": 0,
"Objects": 0,
"Versions": 0,
"DeleteMarkers": 0,
"ObjSizes": [
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0
],
"ObjVersions": [
0,
0,
0,
0,
0,
0,
0
],
"AllTierStats": null,
"Compacted": false
},
"v9-bucket": {
"Children": {
"v9-bucket/nested": {}
},
"Size": 1234,
"HotTierSize": 1234,
"Objects": 1,
"Versions": 1,
"DeleteMarkers": 0,
"ObjSizes": [
0,
1,
0,
0,
0,
0,
0,
0,
0,
0,
0
],
"ObjVersions": [
0,
1,
0,
0,
0,
0,
0
],
"AllTierStats": null,
"Compacted": false
},
"v9-bucket/nested": {
"Children": null,
"Size": 73728,
"HotTierSize": 8192,
"Objects": 2,
"Versions": 4,
"DeleteMarkers": 2,
"ObjSizes": [
0,
1,
1,
0,
0,
0,
0,
0,
0,
0,
0
],
"ObjVersions": [
0,
1,
1,
0,
0,
0,
0
],
"AllTierStats": {
"Tiers": {
"COLD": {
"TotalSize": 65536,
"NumVersions": 1,
"NumObjects": 1
}
}
},
"Compacted": true
}
}
}
cmd/testdata/data-usage-v9/generate.go.txt
Vendored
+82
View File
@@ -0,0 +1,82 @@
// Copyright (c) 2026 Feng Ruohang
//
// This file is part of Silo Object Storage stack
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package cmd
import (
"bytes"
"context"
"encoding/json"
"os"
"path/filepath"
"testing"
"time"
"github.com/minio/minio/internal/cachevalue"
)
// Run only on 89637554d, before access-tiering is removed. This uses the real
// folder scanner and v9 serializer with synthetic file-size/tier summaries,
// following TestDataUsageCacheSerialize; it does not relabel a v8 payload.
func TestGenerateAccessRetirementV9Fixture(t *testing.T) {
if dataUsageCacheVerCurrent != 9 { t.Fatal("requires the historical v9 writer") }
base := t.TempDir()
const bucket = "v9-bucket"
createUsageTestFiles(t, base, bucket, []usageTestFile{
{name: "root", size: 1234},
{name: "nested/versions", size: 8192},
{name: "nested/remote", size: 65536},
})
getSize := func(item scannerItem) (s sizeSummary, err error) {
if item.Typ&os.ModeDir != 0 { return s, nil }
info, err := os.Stat(item.Path)
if err != nil { return s, err }
s.totalSize, s.hotTierSize, s.versions = info.Size(), info.Size(), 1
if filepath.Base(item.Path) == "versions" { s.versions, s.deleteMarkers = 3, 2 }
if filepath.Base(item.Path) == "remote" {
s.hotTierSize = 0
s.tiers = map[string]tierStats{"COLD": {TotalSize: uint64(info.Size()), NumVersions: 1, NumObjects: 1}}
}
return s, nil
}
xls := xlStorage{drivePath: base, diskInfoCache: cachevalue.New[DiskInfo]()}
xls.diskInfoCache.InitOnce(time.Second, cachevalue.Opts{}, func(context.Context) (DiskInfo,error) {
return DiskInfo{Total: 1<<40, Free: 1<<40}, nil
})
cache, err := scanDataFolder(t.Context(), nil, &xls, dataUsageCache{Info:dataUsageCacheInfo{Name:bucket, SkipHealing:true}}, getSize, 0, func()bool{return false})
if err != nil { t.Fatal(err) }
root := *cache.find(bucket)
cache.replace(dataUsageRoot, "", dataUsageEntry{})
cache.replace(bucket, dataUsageRoot, root)
cache.Info.Name = dataUsageRoot
cache.Info.LastUpdate = time.Date(2026, 9, 15, 0, 0, 0, 0, time.UTC)
cache.Info.NextCycle = 42
flat := cache.flatten(*cache.root())
if flat.Size != 74962 || flat.Objects != 3 || flat.Versions != 5 || flat.DeleteMarkers != 2 || flat.HotTierSize != 9426 {
t.Fatalf("unexpected scanner fixture: %+v", flat)
}
var buf bytes.Buffer
if err := cache.serializeTo(&buf); err != nil { t.Fatal(err) }
dir := os.Getenv("SILO_RETIRE_FIXTURE_DIR")
if dir == "" { t.Fatal("SILO_RETIRE_FIXTURE_DIR required") }
if err := os.MkdirAll(dir, 0755); err != nil { t.Fatal(err) }
if err := os.WriteFile(filepath.Join(dir,"data-usage-v9.bin"),buf.Bytes(),0644);err != nil{t.Fatal(err)}
expected, err := json.MarshalIndent(cache,""," ")
if err != nil { t.Fatal(err) }
if err := os.WriteFile(filepath.Join(dir,"data-usage-v9.json"),append(expected,'\n'),0644);err != nil{t.Fatal(err)}
t.Logf("old scanner/v9 writer: bytes=%d size=%d objects=%d versions=%d markers=%d hts=%d",buf.Len(),flat.Size,flat.Objects,flat.Versions,flat.DeleteMarkers,flat.HotTierSize)
}
cmd/xl-storage.go
-5
View File
@@ -583,7 +583,6 @@ func (s *xlStorage) NSScanner(ctx context.Context, cache dataUsageCache, updates
}
poolIdx, setIdx, _ := s.GetDiskLoc()
hotPool, hotPoolOK := globalILMConfig.accessCfg().HotPool()
disks, err := objAPI.GetDisks(poolIdx, setIdx)
if err != nil {
@@ -593,7 +592,6 @@ func (s *xlStorage) NSScanner(ctx context.Context, cache dataUsageCache, updates
cache.Info.updates = updates
dataUsageInfo, err := scanDataFolder(ctx, disks, s, cache, func(item scannerItem) (sizeSummary, error) {
item.poolIdx = poolIdx
// Look for `xl.meta/xl.json' at the leaf.
if !strings.HasSuffix(item.Path, SlashSeparator+xlStorageFormatFile) &&
!strings.HasSuffix(item.Path, SlashSeparator+xlStorageFormatFileV1) {
@@ -657,9 +655,6 @@ func (s *xlStorage) NSScanner(ctx context.Context, cache dataUsageCache, updates
sizeS.versions++
}
sizeS.totalSize += sz
if hotPoolOK && poolIdx == hotPool {
sizeS.hotTierSize += sz
}
// Skip tier accounting if object version is a delete-marker or a free-version
// tracking deleted transitioned objects
docs/bucket/lifecycle/README.md
+4 -142
View File
@@ -226,149 +226,11 @@ aws s3api restore-object --bucket srcbucket \
Note that transition event notification is a Silo extension.
## 5. Access-based tiering between server pools
Silo can move frequently read objects to a faster server pool and return them
to a slower pool after they become idle. This is different from remote ILM
transition: the object remains a native local object and all versions move
together.
Access tiering requires at least two server pools. Pool indices follow the
order on the server command line:
~~~sh
silo server /srv/nvme{1...4} /srv/hdd{1...8}
# pool 0 (fast) pool 1 (slow)
~~~
Server pools are erasure-coding expansion units, not individual drives. Each
pool should consist of internally homogeneous media.
The feature is disabled by default. Configure the topology and safety limits
with `mc admin config set`; ILM is a dynamic subsystem, so this applies without
a restart. Environment variables (`MINIO_ILM_ACCESS_TIERING`,
`MINIO_ILM_ACCESS_POOLS`, `MINIO_ILM_ACCESS_MAX_SIZE`,
`MINIO_ILM_ACCESS_PROMOTE_WATERMARK`, `MINIO_ILM_ACCESS_BIN_WIDTH`,
`MINIO_ILM_ACCESS_BINS`, `MINIO_ILM_ACCESS_FLUSH`,
`MINIO_ILM_ACCESS_MIN_RESIDENCY`, `MINIO_ILM_ACCESS_WORKERS`,
`MINIO_ILM_ACCESS_MAX_TRACKED`) are read at process start and override the
stored config.
~~~sh
mc admin config set local ilm \
access_tiering=on \
access_pools="0,1" \
access_max_size="2TiB" \
access_promote_watermark=85 \
access_bin_width=1m \
access_bins=12 \
access_flush=1m \
access_min_residency=24h \
access_workers=10 \
access_max_tracked=1000000
~~~
The pool list is ordered hottest to coldest. With three or more pools,
promotion always targets the first index and demotion always targets the last;
intermediate pools are not hop targets. An access_max_size value of zero means
no cluster-wide logical-byte cap. Promotion also stops when the hottest pool
reaches access_promote_watermark.
New PUTs still land via the usual free-space pool picker; they are not steered
onto the cold pool. Size the capacity pool larger than the hot pool so new
objects tend to land there.
Add an AccessTransition to the bucket lifecycle XML:
~~~xml
<LifecycleConfiguration>
<AccessTierQuota>500GiB</AccessTierQuota>
<Rule>
<ID>hot-logs</ID>
<Status>Enabled</Status>
<Filter>
<And>
<Prefix>logs/</Prefix>
<ObjectSizeGreaterThan>65536</ObjectSizeGreaterThan>
</And>
</Filter>
<AccessTransition>
<Window>10m</Window>
<PromoteAfterAccesses>100</PromoteAfterAccesses>
<DemoteAfterAccesses>5</DemoteAfterAccesses>
<DemoteAfterIdle>24h</DemoteAfterIdle>
</AccessTransition>
</Rule>
</LifecycleConfiguration>
~~~
This promotes a matching object after 100 successful GETs in 10 minutes. An
object becomes eligible to return to the coldest configured pool only after
access tiering has already moved it (the `x-minio-internal-ilm-atier` stamp),
it has stayed put for the server-wide minimum residency, it has been idle at
least 24 hours, and it has no more than 5 GETs in the window. Objects that
landed on the hot pool via a normal PUT never demote. Prefix, tag, and
object-size lifecycle filters are honored.
Access-based moves are a parallel path: they are not lifecycle `Eval` actions
and do not appear in S3 prediction headers. If the same object is also due
for age-based remote `Transition` or expiry, that scanner action wins and
demotion discovery is skipped for that pass; promotions still run from the
GET tracker. Site replication copies expiry rules only, same as remote
Transition, so AccessTransition stays local to the cluster.
AccessTierQuota is an optional bucket-wide cap. Promotion checks, in order:
1. hot-pool used percentage;
2. cluster-wide access_max_size;
3. bucket AccessTierQuota.
Demotion is not blocked by these caps and is processed before promotion.
Access moves pause during rebalance or decommission, never target a suspended
pool, skip remotely transitioned objects and objects with excessive version
counts, and recheck eligibility while holding the object namespace lock.
Moves also lock the source and destination write locations. All lock servers
for those locations must be reachable; otherwise the background move is
deferred. Ordinary S3 requests keep their existing quorum requirements.
Each move preserves version IDs, delete markers, ETags, checksums, encryption
and user metadata. A retry copies only missing versions and retains versions
already at the destination. If the same version ID has conflicting metadata
in the two pools, the move is skipped and both copies are left intact. Source
data is removed only after the complete version stack exists at the destination.
The hit counter is intentionally best effort. Only successfully served GET
requests count; HEAD requests do not. Counters are merged across nodes and
bounded by access_max_tracked. A rule window longer than
access_bin_width multiplied by access_bins is clamped to retained history.
AccessTransition and AccessTierQuota are Silo lifecycle extensions. A stock
AWS SDK that reads and rewrites the lifecycle configuration may discard
unknown fields. Use a raw signed S3 PUT lifecycle request, such as
[setup_ilm_access_tiering.sh](setup_ilm_access_tiering.sh), when installing
the rule. Save the XML above as `rule.xml`, then run:
~~~sh
AWS_ACCESS_KEY_ID=minioadmin AWS_SECRET_ACCESS_KEY=minioadmin \
./setup_ilm_access_tiering.sh http://127.0.0.1:9000 testbucket us-east-1 rule.xml
~~~
Access-tier activity is exposed under /minio/metrics/v3/ilm, including move
counts, moved bytes, queue depth, hot bytes per bucket, failed moves, dropped
GET samples, and separate skip counters for each capacity limit.
To stop scheduling moves, set `access_tiering=off` (and remove any environment
override). Objects already moved remain in their current pools and stay
accessible; disabling the feature does not move them back. Before downgrading
to a release without access tiering, disable it and let active moves finish.
The object storage format is unchanged. The data-usage cache advances from
v8 to v9: this release reads both, but an older binary discards v9 caches and
rebuilds usage statistics through the scanner. Usage and quota statistics can
therefore take time to repopulate after a downgrade. Keep a copy of lifecycle
XML containing Silo extensions, since an older binary may omit those fields
when rewriting a lifecycle rule.
## Explore Further
- [MinIO Go client API reference (S3-compatible SDK)](https://pkg.go.dev/github.com/minio/minio-go/v7)
- [Object Lifecycle Management](https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lifecycle-mgmt.html)
## Access-frequency tiering removal
The opt-in cross-pool access-frequency extension has been removed. For upgrades from a main/snapshot build that included it, see [the migration notes](access-tiering-removal.md). Ordinary lifecycle expiration and remote-tier transitions remain supported.
docs/bucket/lifecycle/access-tiering-removal.md
+36
View File
@@ -0,0 +1,36 @@
# Removing access-frequency pool tiering
PR #60 introduced an opt-in scheduler that moved objects between local server pools according to GET frequency. It and its feature-specific fixes have been removed. This does not remove ordinary lifecycle expiration, transitions to remote tiers, rebalance, decommission, or the general multi-pool correctness fixes from PR #178.
The [introduction and rollback record](../../investigations/access-tiering-revert.md) documents the commit history, scope decision, review corrections and unresolved validation findings.
The published Server 20260903 predates this feature. These instructions concern main/snapshot deployments that included #60; upgrading from the published version does not require access-tier configuration cleanup.
## Before upgrading a build with access tiering
1. Save a copy of the server ILM configuration and each affected bucket's lifecycle XML. Use an API client that preserves the nonstandard XML; do not rely on a client model that silently omits unknown elements.
2. On the old server, set `ilm access_tiering=off` and remove or disable any access-tier environment overrides. Allow in-progress moves to finish before replacing nodes. This reduces movement intermediate states; the removal itself changes no storage RPC protocol.
3. Remove top-level `AccessTierQuota` and rule-level `AccessTransition` elements. **Delete rules whose only action was `AccessTransition`**. For a mixed rule, retain its filter, status, ID and ordinary expiration/transition actions. An access-only rule loads harmlessly after upgrade, but becomes an actionless rule and fails validation on the next lifecycle edit. If no rules remain, delete the lifecycle configuration through the S3 API.
4. Use a coordinated maintenance window: stop the deployment, install the same new binary on every node, then restart all nodes. The existing bootstrap check compares binary checksums; in a four-node test the first new node could not finish starting among three old nodes. Do not assume that an unchanged RPC protocol permits replacing one node at a time and waiting for it to become ready. This removal does not relax that check. Apply the same environment changes on every node: bootstrap also compares server environment settings, so removing an old override on only some nodes can block startup even with matching binaries. The check runs only during startup and is not a safety guarantee for nodes already running different binaries.
5. After restarting, verify object reads, bucket listing, ILM worker settings and a lifecycle edit. Check storage access from every request-serving node to each pool's drives: successful reads or bucket listing establish less than complete drive reachability, and admin disk summaries aggregate server-local state. Retirement acceptance used unique probes and storage trace to confirm every node-to-drive path before and after version deletion. Startup connection times varied, so a fixed sleep is insufficient. Complete distributed upgrade acceptance for the exact binaries before production rollout.
## What happens to stored state
| State | Behavior after removal |
| --- | --- |
| Ten old ILM keys | `access_tiering`, `access_pools`, `access_max_size`, `access_promote_watermark`, `access_bin_width`, `access_bins`, `access_flush`, `access_min_residency`, `access_workers`, `access_max_tracked` are accepted but ignored. Existing transition/expiration worker settings are preserved. |
| Admin configuration | Deprecated keys may still appear in `mcli admin config get ilm`; setting them may succeed but has no effect, even with `access_tiering=on`. Remove obsolete environment settings from deployment manifests. |
| Lifecycle XML | `AccessTierQuota` and `AccessTransition` are ignored when read and omitted when re-encoded. The same parser handles new PUT requests, so these extensions are also silently discarded there; access-only rules still fail action validation. |
| Data-usage cache | Both v8 and v9 caches are read, preserving ordinary counts, sizes, histograms and remote-tier statistics. The retired hot-tier byte count is discarded; subsequent writes use v8. No feature-driven full statistics rebuild is required. |
| Objects already moved | Remain in their current pools with the same versions and timestamps. There is no bulk move-back or object metadata rewrite. |
| Internal leftovers | `x-minio-internal-ilm-atier` and `.minio.sys/config/ilm/access/` counter objects may remain unused. They do not require a cleanup service or an object scan. |
Interrupted rebalance/decommission can leave the same version in more than one pool independently of access tiering. Removing the scheduler does not remove such existing copies. General Object Lock, conditional-delete, metadata reconciliation and shared remote-tier reference protections remain in place.
## Version deletion scope
Ordinary single-object `DELETE ?versionId=...` reconciles the addressed UUID, null version or delete marker across pools. Unqualified DELETE of a directory marker (a key ending in `/`) also addresses its null version and uses this path. A successful request applies the deletion to every resolved pool copy under the existing per-pool quorum rules; other version IDs remain. If outbound delete replication is pending, copies retain `VersionPurgePending` until the existing replication worker completes the purge. Success does not guarantee immediate physical removal from every drive.
If a pool is unreadable, these requests can return 503 even when another pool has a readable copy. Insufficient read quorum returns `503 SlowDownRead`; other failures retain their corresponding error codes. This extends an existing failure surface: previously the result could depend on whether the unreadable pool preceded the successful pool in traversal order; it now fails consistently. Retry after recovery. Cleanup failures also return an error. Ordinary unqualified DELETE retains its existing semantics. Batch `DeleteObjects` already fans out across pools.
Incoming replicated deletes, lifecycle expiration, free-version cleanup and movement-internal calls retain their existing contracts. In particular, an incoming replicated version delete can leave movement duplicates in other pools; this change does not solve that separate case. Expiration scanners process their own pools and may remove duplicate expired copies in later cycles; free-version cleanup remains local to a pool. Do not treat the ordinary DELETE repair as a guarantee for every source of deletion.
docs/bucket/lifecycle/setup_ilm_access_tiering.sh
-37
View File
@@ -1,37 +0,0 @@
#!/bin/sh
# Install a lifecycle XML document without an SDK normalizing away Silo's
# AccessTransition and AccessTierQuota extension elements.
set -eu
if [ "$#" -ne 4 ]; then
echo "usage: AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=... $0 ENDPOINT BUCKET REGION LIFECYCLE_XML" >&2
exit 2
fi
endpoint=$1
bucket=$2
region=$3
lifecycle_file=$4
: "$AWS_ACCESS_KEY_ID"
: "$AWS_SECRET_ACCESS_KEY"
if [ ! -r "$lifecycle_file" ]; then
echo "cannot read lifecycle document: $lifecycle_file" >&2
exit 2
fi
content_md5=$(openssl dgst -md5 -binary "$lifecycle_file" | openssl base64)
endpoint=$(printf '%s' "$endpoint" | sed 's:/*$::')
curl --fail-with-body --silent --show-error \
--request PUT \
--aws-sigv4 "aws:amz:$region:s3" \
--user "$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY" \
--header "Content-MD5: $content_md5" \
--header "Content-Type: application/xml" \
--data-binary "@$lifecycle_file" \
"$endpoint/$bucket?lifecycle"
echo "installed lifecycle configuration on $bucket"
docs/investigations/access-tiering-revert.md
+193
View File
@@ -0,0 +1,193 @@
# 访问频率分层：引入、修复与回退记录
记录日期：2026-09-15。本记录说明 PR #60 的设计、合入后的取舍，以及此次回退为什么同时保留并补齐通用多池正确性修复。操作步骤见[退役迁移说明](../bucket/lifecycle/access-tiering-removal.md)。合并、正式发布和生产部署是不同状态；本记录随回退变更交付，不代表已经发布。
初稿审查时（2026-09-15 03:20 UTC），最终候选尚未移植到远端基线、尚未冻结 PR head，合并前全量检查与三次有效 Linux 运行尚未执行，本变更尚未合并。后续执行状态以承载本记录的 PR 及其绑定提交的验收记录为准；下文的历史实验不替代这些检查。
**当前验收状态：** 回退、普通版本 DELETE 调和、扫描复用及文档由 [PR #188](https://github.com/pgsty/silo/pull/188) 交付。本地与 CI 检查通过；首次 Linux 验收因 DELETE204 后的 HEAD/GET503 停止。随后完成可控机制实验、匹配写入负载对照，并修正准备检查；独立的新轮次 R-Upgrade-2 三次完整升级验收均通过。旧失败没有改判，原单次请求的逐盘状态仍不可追溯。最终合并状态以 PR 为准，正式发布与部署另行验收。详情见第 9 至 11 节。
## 1. 引入的目标和实际范围
[@mrjavadseydi](https://github.com/mrjavadseydi) 在 [PR #60](https://github.com/pgsty/silo/pull/60) 提出了基于 GET 频率的本地池间分层。成功 GET 更新有界滚动计数，后台调度器把热对象提升到配置中的首个池，把已经迁移且变冷的对象降到末个池。功能默认关闭，至少需要两个池；它与普通生命周期过期、远端对象存储 transition、rebalance 和 decommission 是不同机制。
实现不只是一个后台任务：它增加了 GET 计数入口、leader 调度、跨池版本栈复制、来源清理及失败恢复、热池配额、生命周期 XML 扩展、配置项、指标和扫描统计。访问热度统计使 data-usage cache 从 v8 升到 v9。对象本体的存储格式没有因此改变。
搬移要保持完整版本历史、删除标记、null version、时间戳、ETag、校验和和加密元数据；同时需要处理并发写入、目的端已有版本、来源部分删除和远端 tier 引用。合入前的修补和测试针对并覆盖了这些边界，不应把回退解释成贡献无效。贡献者署名继续保留，独立的其他贡献也不回退。
## 2. 可追溯时间线
下表的 PR/Issue 时间使用 UTC；提交链接对应具体代码，不把“报告时间”当作“缺陷首次出现时间”。
| 时间 | 事件 | 本次处理 |
| --- | --- | --- |
| 2026-08-15 10:15 | #60 创建；原始实现 [`7a060cab1`](https://github.com/pgsty/silo/commit/7a060cab1edd5bbc17da7f703bbd1ab7415b6f7c) | 随特性撤销 |
| 2026-09-06 00:09 | [#133](https://github.com/pgsty/silo/issues/133) 报告多池副本写不能权威调和 Object Lock 状态 | 保留解决它的通用修复 |
| 2026-09-06 15:12 | [#144](https://github.com/pgsty/silo/issues/144) 报告条件 DELETE 原子性仅限单个纠删码集合 | 保留解决它的通用修复 |
| 2026-09-08 05:18 | [`9a6e1477f`](https://github.com/pgsty/silo/commit/9a6e1477f45067559def8423d431ee177795134f) 补访问分层兼容标识清单 | 删除功能专属标识，保留有依据的退役兼容 |
| 2026-09-08 07:08 | [`374de0fa3`](https://github.com/pgsty/silo/commit/374de0fa32aa1d6eda57dbba6ac522ebf793b6be) 修复搬移的版本保全、写隔离和删除范围 | 随专属搬移器撤销 |
| 2026-09-08 07:28 | [`5ac33e158`](https://github.com/pgsty/silo/commit/5ac33e1583e838ade3f56c7d60e80e7a854f9a88) 确定性覆盖搬移失败恢复 | 随已删除搬移器的专属测试撤销 |
| 2026-09-08 07:42 | #60 以 [`a3df317ae`](https://github.com/pgsty/silo/commit/a3df317ae0725eb650e4d3e21551154f69be6229) 合入 | 以该 merge 的第一父差异确定功能边界 |
| 2026-09-11 12:34 | [#178](https://github.com/pgsty/silo/pull/178) 合入通用多池写入、元数据与条件删除调和 | 保留，解除测试对访问搬移器的依赖 |
| 2026-09-13 | [`2dd1e00da`](https://github.com/pgsty/silo/commit/2dd1e00da49faf995f2db29807fc6211b8376d7d) 的 CHANGELOG 同时汇总访问分层和通用多池修复 | 拆开表述，不整条删除独立修复历史 |
| 2026-09-15 | 维护者决定收缩访问分层；完成来源分析、三种候选反证、退役兼容、普通版本 DELETE 修补与外部评审 | 形成此次选择性回退 |
#178 中的 [`e59a3d938`](https://github.com/pgsty/silo/commit/e59a3d938ed25c1bcd51efbb4ad6955073d195f7) 提供池级串行化、字段调和和条件删除；[`ccb676e60`](https://github.com/pgsty/silo/commit/ccb676e60cb7441ee65ff7c35f3b7828979101fd) 保护仍被其他副本使用的远端 tier 引用；[`51d41345f`](https://github.com/pgsty/silo/commit/51d41345f7ac532f9c6fea2b7dba5f29da930b8f) 保存 Linux 重启及 OIDC 验收记录。这三项不属于仅为访问频率调度而存在的代码。
截至回退评估时，公开 Server `RELEASE.2026-09-03T13-18-01Z` 早于 #60 合入。退役迁移主要针对运行过后续 main、自行构建或快照版本的实例；不能据此声称正式 release 用户普遍启用过此特性。
## 3. 为什么回退，为什么不能整批撤销后续修复
维护者的取舍是：默认关闭的可选调度能力，对配置、生命周期、缓存、统计和核心多池写入路径带来了过大的维护面。此次移除的是该能力及专属实现，没有测量并宣称吞吐量提升、延迟降低或固定减少一次分布式锁往返。
“后来改过同一文件”不等于“由 #60 引发”。#133/#144 的报告早于 #60 合入；更关键的是，原有 rebalance、decommission 和复制写入也会使同一版本暂时存在于多个池。访问分层消失后，这些状态仍然合法存在。移除调和与锁纪律会重新允许旧副本遮蔽新元数据、条件删除选错版本、清理错误被吞掉等问题。
评估在隔离工作树中实际比较了三条路线：
| 候选 | 通用多池回归 | 普通指定版本 DELETE |
| --- | --- | --- |
| A：撤销 #60，保留 #178 | 原有 13 组通过 | 仍能成功返回后留下可读副本 |
| B：同时撤销 #60 和 #178 的存储修补 | 相同 13 组中 10 组失败 | 问题仍在 |
| C：A 加普通版本 DELETE 调和 | 13 组原有及当时新增的 8 组通过 | 同一复现通过 |
这些是 2026-09-15 的历史对照结果，不是最终 PR head 的发布验收。后续补上目录标记、真实 rebalance 中断等覆盖后，通用多池测试达到 23 组。测试通过不能替代来源分析，来源分析也不能替代最终候选的运行验证。
## 4. 最终保留与删除的边界
- 删除访问 tracker、调度/搬移器、GET 和 scanner 钩子、热池配额、专属配置帮助、生命周期动作、指标及专属测试。
- 保留普通过期、远端 transition、rebalance/decommission、复制写入，以及 #178 的 Object Lock、标签、条件删除、元数据调和与远端引用保护。
- 原有 data-usage 生成解码器恢复到 #60 前的实现；允许读取 v8/v9，利用字段编码跳过已退役热度字段，继续写 v8。普通字段保真由历史真实 v9 样本测试覆盖。
- 仅容忍准确的十个退役 ILM 键；读取生命周期时丢弃退役扩展。纯访问动作的规则需先清理才能再次编辑；混合规则保留普通动作。
- 已经搬移的对象留在当前池；没有全量搬回、自动删除所有重复版本、后台清理服务或对象元数据重写。
曾对本地审查提交 `d06f1c614` 做过声明来源复核：消失的 357 个声明均不在 #60 之前，删除的十个文件均由 #60 引入；#60 原先删换的 41 行旧文本按忽略空白比较有 40 行恢复，剩下一行保留 #178 在已持有池锁时调用 `getWritePoolIdx(..., true)` 的修正，避免对同一对象再次取锁。生成缓存解码器与功能前逐字节一致，原有 13 组通用测试没有删除。这是该审查版本的保全证据，不能把数字脱离 SHA 当作未来所有版本的保证。
## 5. 普通版本 DELETE 是独立补洞
功能移除不会自动消除历史重复副本。普通单对象指定版本 DELETE 因而复用既有调和路径：在池级对象锁内读取每个池的目标版本，计算一次条件及回调，先删除非权威副本，再处理权威副本；任何不可读池或清理错误都不能当作成功。
范围包括 UUID、null version、delete marker，以及原先就被解析为 null version 的未指定版本目录标记 DELETE。入站复制、搬移内部调用、生命周期过期和 free-version 清理保留各自语义；批量 `DeleteObjects` 原本就会向池并发扇出，不是此次遗漏。
删除标记需要向 retention/metadata 回调传入与 set 层相同的 `MethodNotAllowed` 或 `ObjectNotFound` 语义。直接复用拒绝 marker 的元数据更新入口会错误地拒绝合法版本删除。回调从所有副本合并独立更新的 Object Lock 和标签，不能随意只采用一个池的状态。
存在两项明确的成功/失败边界：
1. 读法定多数不足时返回 `503 SlowDownRead`，即使另一个池有可读副本。旧路径的结果会受池遍历顺序影响；新路径把失败语义统一。它是正确性与可用性的取舍，需要恢复后重试。
2. 出站删除复制尚未完成时，成功响应可以表示各副本进入 `VersionPurgePending`，由既有 worker 完成清理。原有每池 quorum 规则也继续适用；不能把成功响应等同于每一块盘立即物理删除。
## 6. 审查如何改变了方案
本地先形成三个线性审查提交：`8fdfdabd9` 移除特性，`d06f1c614` 补普通 DELETE，`6e3fdca97` 去除重复扫描。它们记录审查演进，最终 PR 在独立远端基线上重放，提交 ID 会改变；不应把线性演进误认为三份同时维护的实现。
Claude Code Opus 5 / max 的五轮实现评审要求补齐退役兼容、说明协调停机及环境一致性、验证真实池故障，并纠正运行证据措辞。随后 Claude 与 ZCode 的独立复核再次确认了回退边界和普通 DELETE 语义；最终两轮计划商榷收束了交付流程。
| 意见 | 裁定与处理 |
| --- | --- |
| 指定版本 DELETE 重复扫描所有池 | 接受。第一次扫描已在同一锁内得到目标副本，直接合并其结果。16 盘池在回调前的读取计数从 32 降为 16；这不等于总 I/O 或延迟减半。 |
| N 个副本产生 N 条 DELETE 审计 | 反驳。底层调用只追加上下文标签，HTTP 层向每个配置审计目标发送一次请求事件。成功完成调和时池标签最终指向 primary；不增加额外 NoAuditLog 修改。 |
| retention/metadata 顺序与 set 层不同 | 差异存在，但已有明确注释。保留 retention 优先，拒绝后不删除、不调度删除复制、不 Sweep；不宣称任意自定义回调都能交换。 |
| 把优化 amend 进旧提交并直接丢弃 main 脏修改 | 不 amend 已审历史。先保存完整文件、补丁、哈希及可达恢复引用，核对覆盖后受控恢复。八组新增通用测试承接，访问搬移专用测试由真实 rebalance 中断覆盖替代。 |
| 从当前本地分支直接开 PR | 调整。其祖先含另一个任务的 IAM/超时提交 `ebc9937d9`；从远端 `89637554d` 仅移植本次变更，不夹带或删除独立工作。 |
| 合并之后再跑全量与 Linux 验收 | 不接受。先完成文档和移植，冻结 PR head，再验收；不能把旧 SHA 的测试直接提升为新基线的通过记录。 |
| 不同基线 diff 必须逐字节一致 | 改为每提交 stable patch-id、路径与完整树等价性核验。blob hash 和行号随基线变化，不应成为错误的拒绝依据。 |
`objectPoolInfos` 的并行查询作为独立性能跟进，本次不增加并发实现。Contributor 署名、#132 配额指标、#77 桶元数据、federated COPY、IAM、超时及其他独立修复不因本次取舍被整体撤销。
## 7. 历史运行证据与未定案事项
以下是移植前 `d06f1c614` 的历史实验，不是最终 PR head 的验收替身。
| 运行 | 实际观察 | 不应推导的结论 |
| --- | --- | --- |
| `f590538f`，四节点双池 | 旧版真实 rebalance 中断留下 9 个重复 UUID；协调停机换新后对象四节点可读，旧配置与普通规则可编辑，真实缓存头 v9→v8；删除一个 addressed UUID 后四节点 HEAD/GET 404，另一个版本仍可读 | 没有验证全部 9 个不同 UUID 收敛；物理元数据仅每池抽查一盘；没有整份运行时统计守恒测量 |
| `fd93cd37`，三节点双池 | 停掉一个池所在节点，保留 namespace 锁的 2/3 法定多数；DELETE 返回 503 SlowDownRead，源全部四盘保留目标版本；恢复后 DELETE204，各节点 HEAD/GET404 | 不能推广为跨主机网络、持久盘及压力验收 |
| 被弃用的四节点停池拓扑 | 同时丢失 namespace 锁法定多数，发生客户端超时，记录脚本还遇到 NoneType 错误 | 不是“池读取返回503”的证明 |
| `83676ca2` | 升级后配置/生命周期编辑之后一次 HeadObject 返回503；artifact 没有记录 DELETE 自身响应 | “DELETE之后”仅来自脚本顺序，不能写成已证实 DELETE204 后异常，也不能归类为已修复、既有问题或暂态 |
**开放项：`83676ca2` 的 HEAD503 仍未定案。** 可直接比较的目标阶段历史运行是一失败、一成功；一次未复现不足以关闭问题。仅凭 `SlowDownWrite` 等错误名字也不能给其他失败确定容量或环境根因。
最终候选的合并前复核采用有界规则：要求三次有效升级后 DELETE/HEAD 运行，最多五次总尝试；每次记录 DELETE 码/耗时、失败 HEAD 的节点与版本、GET 错误码、两池全盘元数据、固定间隔重试时序和旧版同拓扑对照。旧版可能保留副本返回200，不要求它满足新增跨池删除契约。
有证据证明在目标操作前失败的 harness 尝试才能不计入有效运行，但仍计入总尝试。任何目标阶段的新503或数据不变量失败都不能通过补跑抹掉，必须暂停合并并定位。三次通过也只满足这项工程检查，不证明历史异常已消失；开放项在合并和正式发布评估时仍须可见。
## 8. 交付与恢复纪律
最终候选使用独立分支；main 的五个旧修改先保存完整内容、二进制补丁、SHA-256 和具名 Git 恢复引用，再核对原 HEAD/哈希及测试覆盖，只恢复这五个文件。禁止用整树 reset 或 clean 代替受控归一；若用户已有新增编辑，应保留并重新核对。
全量 cmd/internal、相关 race、构建、vet、lint、生成文件和兼容检查，以及上述 Linux 验收，绑定最终候选的实际 SHA。若纳入新的远端提交，重新记录基线与 head，复核变更并重跑受影响验收。文档与原始测试记录各自保留其对应版本，不篡改旧失败、不用新通过覆盖旧记录。
逐节点滚动升级未通过既有二进制校验检查，采用[协调停机方案](../bucket/lifecycle/access-tiering-removal.md#before-upgrading-a-build-with-access-tiering)。实验使用单个 Docker Linux VM 和 tmpfs；正式 tag、包、镜像、跨主机及生产部署仍是独立交付。未验证全部重复 UUID 或运行时统计守恒应如实披露，不能反向引入自动搬回/清理需求或无关重构。
## 9. 执行后记：最终基线、恢复窗口与验收
本次实际交付由 [PR #188](https://github.com/pgsty/silo/pull/188) 承载。选择的远端基线为 `89637554d60c27cfc51d2281d0a4fe15e415f06d`，移植没有包含本地独立 IAM/超时提交 `ebc9937d9`。前三项实现和历史文档逐提交通过 stable patch-id 对照；虚拟补回独立 IAM 差异后，完整树与原审查分支一致。
首次本地 lint 发现新增回调选择分支触发 `gocritic/ifElseChain`，因此追加等价的无表达式 `switch` 改写，保持 marker、指定版本和普通元数据查找的条件顺序及分支体。重新固定的代码候选为 [`41aa84609`](https://github.com/pgsty/silo/commit/41aa84609754769cfb1861d7fd060c2e84182b98)。这一提交上，全量 cmd/internal 得到 6,428 个测试及子测试通过、166 个跳过，50 个有测试的包通过；相关 race 得到 283 个测试及子测试通过。`make build`、全包构建、vet、lint、生成文件及 rebrand/compat 检查均通过。[Go CI](https://github.com/pgsty/silo/actions/runs/34925534139)、[DCO](https://github.com/pgsty/silo/actions/runs/34925534110)、[VulnCheck](https://github.com/pgsty/silo/actions/runs/34925534142) 和[发布流水线的测试运行](https://github.com/pgsty/silo/actions/runs/34925534198)共 11 项检查通过；后者没有发布正式制品。
第一次最终候选停池实验 `dcd5c2e5` 在源版本保全断言后失败：停掉另一池返回 `503 SlowDownRead`，源四盘版本保留；恢复后 DELETE 成功，节点 0/2 的 HEAD/GET 返回 404，节点 1 返回 503。DELETE 成功由脚本已通过的 204 断言确定，原输出没有单独保存该次 DELETE 响应。此次失败如实保留，不能把后来的成功写回原记录。
复核发现，`ListBuckets` 以及位于源池的现存版本 GET，不能证明每个协调节点对另一池的读取连接已经恢复。随后进行了旧基线与候选的同拓扑对照，探测的是从未写入过的随机 UUID，并且在这些探测之前没有执行任何 DELETE：
| 高时间分辨率对照 | 桶列表和现存版本 | 从未写入版本的 HEAD/GET | 观察到的恢复窗口 |
| --- | --- | --- | --- |
| `20502a2f`，旧基线 `89637554d` | 三节点均为 200 | 节点 0/2 为 404，节点 1 为 503 SlowDownRead，连续 16 组 | 从重启后的观察循环起算约 1.60–2.50 秒 |
| `246aaf77`，候选 `41aa84609` | 三节点均为 200 | 同样是节点 1 的 503，连续 11 组 | 约 1.67–2.30 秒 |
两边在缺失版本全节点连续三轮返回 404 后执行 DELETE，均得到 204、全节点 HEAD/GET 404、目标 UUID 在八盘均不存在且其他版本可读。另有两次较低时间分辨率诊断未捕捉到窗口，同样保留；这四次诊断不计入三次升级验收。
这给出了基线在零 DELETE 下的正向复现，证明原恢复条件不足。`getLatestObjectInfoWithIdx` 的读选择函数与基线文本相同：现存副本可以遮蔽另一池的读错误；缺失版本则必须确认所有池，不可读时返回 503。Claude 复核后同意修正实验准备条件并继续验收，明确反对把这一路径的 503 改成 404。最初失败没有瞬时 RPC 全貌，不能逐请求追溯每条连接；这些对照也不能给 `83676ca2` 归因。
修订后的验收在升级/恢复后逐节点探测从未写入的 UUID，记录首次全 404 时刻，要求连续三轮全 404；并列保存各节点 `admin info` 的全盘状态。最多等待 30 秒，超时仍失败，DELETE 之后仍严格要求 204/404，不把 503 纳入通过条件。后续失败保留实验资源供即时取证，再受控清理。
修订准备条件后的独立停池验收 `40a59b3b` 通过：离线 DELETE `503 SlowDownRead`，源四盘版本保留；恢复门约 1.48 秒完成，DELETE `204`，三节点 HEAD/GET `404`，目标在八盘均不存在，另一版本仍可读。恢复早期 `admin info` 中也记录到了各节点不同的离线盘视图，最后恢复为全盘正常。
完整升级验收随后实际进行了三次尝试：
| 尝试 | 运行 | 结果与证据边界 |
| --- | --- | --- |
| 1 | `83f88c59` | 准备失败，未启动候选。旧版 rebalance 报 Completed、搬移版本数为 0；源池占用约 6.5%，到平均空闲目标的差值约 3.13%，落入代码既有 5% 容差。增加造数从 64 到 192 个 2 MiB 版本后再试；本次仍计入五次总尝试上限。 |
| 2，第一轮有效运行 | `ea58d0c5` | 通过。实际 rebalance 中断产生跨八盘的重复版本；停机复制同一数据供旧版对照。旧版 DELETE204 后仍可读；候选 DELETE204 后四节点立即及后续固定间隔 HEAD/GET 均404，八盘目标清除、另一版本四节点可读；旧 ILM/生命周期编辑和真实缓存 v9→v8 通过。 |
| 3，第二轮有效运行 | `2059bc6b` | **候选失败，阻断合并。** 两阶段逐节点缺失版本连续三轮404、admin info全盘ok之后，DELETE明确204（约12.98ms）；节点2随后的HEAD503/GET503 SlowDownRead，另外三节点404；八盘快照均无目标，首次重试及后续采样全404，其他版本四节点可读。首次失败保留，不因重试恢复改判。 |
第三次尝试发生后停止剩余验收，保留原容器、卷、元数据与响应时序进行诊断。不能把四节点顺序探测中的“节点2异常”直接解释为永久节点故障：它也可能与采样时间有关。随后在保留环境中，对三个额外重复版本做并发、不同顺序的“刚删 UUID / 从未写入 UUID”对照，候选稳定期均为204/404，未复现；旧版克隆数据的双次 DELETE 对照则遇到 `SlowDownWrite`，没有完成其全部断言。这些是诊断结果，不补入有效升级通过计数，也不证明第三次尝试已解释。
为观察逐盘返回，另在独立临时工作树编译仅增加日志的诊断二进制，**没有进入 PR**。它证明了第二层准备检查盲点：`getObjectFileInfo` 的四个响应信号中，可以只有两个实际 `file version not found`，其余两个是被跳过的盘所保留的 `errDiskOngoingReq`；`objectQuorumFromMeta` 的预期读 quorum 为2，因而仍可返回404。同期 `admin info` 汇集各服务器本地盘态为ok，不能证明请求节点到各盘的路径都可用。一次诊断 DELETE 的逐盘返回为 `[nil, nil, drive not found, drive not found]`，达不到写 quorum 3，故返回 `SlowDownWrite`。
Claude 撤回了此前“逐节点三轮404已是最强全池准备条件”的表述，同意这只能证明读 quorum，不能证明全盘可达或写 quorum。诊断给出了候选机制，但**第三次尝试失败瞬间没有逐请求逐盘日志，仍不足以确定其具体原因**；不能把后来稳定期的成功、诊断中的配额不足，或对错误名的解释当成该次故障的直接证据。
## 10. 首次执行的停止位置与恢复资料
首次执行停止时，代码为 `41aa84609`；后续提交只记录执行，不改变已测试的生产代码。当时有效升级运行是一通过、一失败，未满足三次有效运行全部通过的约定；总尝试3次，没有通过继续补跑消耗剩余次数来冲淡失败。`2059bc6b` 和先前的 `83676ca2` 均保持 **OPEN**。Claude 与 Codex 当时的裁定是 **NO-GO for merge**，没有把证据不足升级为“已修复”“既有问题”或“暂态”。后续收尾见第 11 节；实际合并状态以 [#188](https://github.com/pgsty/silo/pull/188) 为准。
主工作区原五文件的完整内容、二进制补丁和 SHA-256 已归档；另有可达 Git 引用 `refs/archive/access-tiering-main-five-files-20260915`，指向快照 `c7fbfc6ada0f0f2abcbe8c0a9681f07e12dafe52`。逐文件核验快照与原已审内容一致。首次停止时因验收未通过，没有执行五文件恢复，也没有移动或改写独立 IAM 提交 `ebc9937d9`。当时唯一待交付候选在 PR 分支，旧变体冻结等待验收裁定。
本机执行资料归档在 `~/.codex/outputs/silo-access-revert-assessment-20260915/final-execution/`：保存了每次尝试、旧/新基线对照、二进制 SHA-256、准备条件、源/目的全盘元数据、诊断补丁、独立评审意见，以及受控清理记录。原始失败记录不覆写；保存的诊断卷内容用于继续调查，不是生产数据或发布制品。
继续推进需要一次能区分机制的取证：在条件可控的升级实验中，于失败请求当时记录每池每盘的真实应答和错误类型，并同时读刚删版本与从未写入版本；明确区分盘面不同步、请求节点的不可用路径与其他原因。若四盘均真实应答仍返回503，应沿错误归约/元数据路径定位；若盘不可用，应查明连接或初始化状态，并验证准备条件。后续成功本身不能关闭本次失败，更不能通过把不可判定的503改成404来满足验收。正式发布、制品和生产部署继续作为独立交付。
## 11. 收尾复核：准备检查、可控机制与独立新轮次
后续收尾没有继续修改生产代码。旧基线 `89637554d` 与候选 `41aa84609` 使用各自的独立诊断构建，仅对测试桶记录逐盘应答；这些日志补丁没有进入 PR。第一轮完整诊断 `53ebe161` 通过但未复现503，仅作为一个样本保留。第二轮 `c951f762` 得到了不同的、可直接解释的失败：两个池的删除调用均记录 `quorum=3 errs=[nil,nil,nil,drive not found]`，DELETE204、所有读样本404，但八盘快照的盘3、7仍保留目标版本。它符合既有写 quorum 契约，并正向证明旧准备门会放行尚未完成挂盘的协调节点；它没有复现或解释 `2059bc6b` 那一次请求。
审查还纠正了两项推断：后续诊断进程在04:19的重连日志不能用于解释03:56的原失败；错误归约按具体错误值计数，不能把“最高同值计数不足”简单等同于“实际应答盘数不足”。原失败之后的全盘快照也不是失败瞬间的原子快照。这些界限继续保留。
准备检查改用服务器已有的 storage trace：从每个 S3 节点，对各自唯一、从未写入的对象执行 `GetObjectTagging`，该路径等待所有盘；按唯一对象名和后端节点、盘路径匹配真实 `storage.ReadVersion` 应答。四节点双池需要每轮32条实际缺失应答，连续三轮完成才满足准备条件；单纯404或 `admin info` 的本地盘状态不够。探针不创建对象，不改变服务器读写语义。trace 输出是多行 JSON 对象流，按流解析；缺失 trace 证据会使检查失败，不能据此假定对应盘健康。
为了比较相同的跨池写入负载，停止真实 rebalance 夹具后复制两份相同数据。旧基线使用 #178 已有的 `If-Match` 条件 DELETE，候选使用普通指定版本 DELETE，二者都处理同一目标的两个池副本。两臂分别完成96次“已删版本／从未写入版本”的 HEAD/GET 对照，均404，目标八盘均清除。准备检查分别耗时约0.60秒和17.08秒。这是一对匹配样本，没有观察到候选专属差异，不是吞吐比较，也不能排除所有可能的故障机制。
`f55967b7` 另用明确制造的重复副本夹具完成因果实验，而非冒充真实 rebalance：pool0 的四盘由一个节点承载并保留 namespace 锁；pool1 的四盘分布到另外四个节点。先停止 pool1 一盘，DELETE204 后直接确认只有该盘残留目标版本。重新接入这份副本，再停止 pool1 两块已删除该版本的盘，保留“一盘返回版本、一盘返回缺失”的读视图。三个被测协调节点的已删版本均返回 `503 SlowDownRead`，从未写入版本均返回404；同一失败请求的逐盘记录明确为 `[drive not found, drive not found, file version not found, nil]`，没有足够的同值应答达到读 quorum。整个实验没有丢失 pool0 的 namespace 锁，也没有将不确定状态改为404。
该因果实验的部分节点重启未在35秒内恢复全部40条访问路径，原实验因此仍记 FAIL；正向机制观察与这个恢复失败分开记录。随后协调重启全部五节点，独立恢复检查通过，五节点 HEAD/GET 均404。这不是滚动恢复已通过的声明，也不追溯原 `2059bc6b` 的逐盘状态。可控复现确定的是故障机制类别，原单次实例继续 **OPEN**。
Claude 复核上述证据后同意显式开启 **R-Upgrade-2**：原轮次的一通过、一失败和总尝试3次保持原样，不合并计数，不静默重置。新轮使用未经诊断修改的 `41aa84609` 二进制，要求三次有效运行全部通过、最多五次总尝试；任何新503或数据不变量失败仍须停止。每次 DELETE 前后均检查32条路径，响应后的即时 HEAD/GET 先于后置准备检查执行，避免等待掩盖短暂错误。
| 新轮次运行 | 完整运行耗时 | 升级后32路径准备耗时 | 验收结果 |
| --- | --- | --- | --- |
| `8eb016db` | 90.20秒 | 17.04秒 | PASS |
| `2a2b7b64` | 80.83秒 | 0.62秒 | PASS |
| `371e7b9f` | 96.28秒 | 0.60秒 | PASS |
三次均实际走到候选阶段：DELETE204，所有即时及后续 HEAD/GET 样本404，目标在八盘均不存在，其他版本从四节点读回；旧配置、普通生命周期规则编辑及真实缓存 v9→v8 均通过。删除前后各三轮32路径检查也全部通过。准备时间有明显波动，应验证访问路径，不能用固定等待秒数代替检查。这些结果满足修正准备条件后的有界验收；不把有限样本写成“历史503已消失”，不把不同阶段的诊断通过计入新轮次。
新轮仍绑定生产代码 `41aa84609`（Linux 二进制 SHA-256 `28e1339d630a22fa5a0e4659b6182224e81f6cd7cd4079856534390e40a697b1`），后续仅更新文档。合并前须核对源码等价性和最终 CI，按第8节的恢复纪律处理旧五文件，保留独立 IAM 提交。原 `2059bc6b`／`83676ca2`、部分重启恢复边界、单 VM/tmpfs、未验证全部不同重复 UUID 和整份运行时统计守恒继续可见；不为此新增自动搬回、清理服务或读错误降级。
本轮详细资料位于原归档的 `final-execution/closure-20260915/`，包括匹配对照、同请求逐盘日志、`qualification-summary.json`、独立 R-Upgrade-2 账本、诊断补丁和完整卷归档。原失败现场、后续对照及恢复后的卷分别标注时点。临时实验资源在归档验证后清理；这些资料与正式发布制品区分管理。
docs/metrics/prometheus/list.md
-11
View File
@@ -227,17 +227,6 @@ For deployments with [bucket](https://silo.pgsty.com/administration/bucket-repli
| `minio_node_ilm_transition_active_tasks` | Number of active ILM transition tasks. |
| `minio_node_ilm_transition_pending_tasks` | Number of pending ILM transition tasks in the queue. |
| `minio_node_ilm_transition_missed_immediate_tasks` | Number of missed immediate ILM transition tasks. |
| `minio_node_ilm_access_tier_active_tasks` | Number of active access-tier pool moves. |
| `minio_node_ilm_access_tier_pending_tasks` | Number of pending access-tier pool moves. |
| `minio_node_ilm_access_tier_promotions_total` | Total objects promoted by access-tier ILM. |
| `minio_node_ilm_access_tier_demotions_total` | Total objects demoted by access-tier ILM. |
| `minio_node_ilm_access_tier_bytes_moved_total` | Total logical bytes moved by access-tier ILM. |
| `minio_node_ilm_access_tier_failures_total` | Total failed access-tier ILM moves. |
| `minio_node_ilm_access_tier_skipped_watermark_total` | Promotions skipped because the hot pool reached its watermark. |
| `minio_node_ilm_access_tier_skipped_max_size_total` | Promotions skipped because the cluster hot-tier size cap was reached. |
| `minio_node_ilm_access_tier_skipped_bucket_quota_total` | Promotions skipped because the bucket hot-tier quota was reached. |
| `minio_node_ilm_access_tier_hot_bytes` | Logical bytes currently accounted to the hot tier, labeled by bucket. |
| `minio_node_ilm_access_tier_samples_dropped_total` | GET samples dropped because the access tracker queue was full. |
| `minio_node_ilm_versions_scanned` | Total number of object versions checked for ilm actions since server start. |
| `minio_node_ilm_action_count_delete_action` | Total action outcome of lifecycle checks since server start for deleting object |
| `minio_node_ilm_action_count_delete_version_action` | Total action outcome of lifecycle checks since server start for deleting a version |
internal/bucket/lifecycle/access-transition.go
-129
View File
@@ -1,129 +0,0 @@
// Copyright (c) 2015-2026 MinIO, Inc.
//
// This file is part of MinIO Object Storage stack
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package lifecycle
import (
"encoding/xml"
"time"
)
var (
errAccessInvalidDuration = Errorf("Window and DemoteAfterIdle must be valid Go durations, e.g. 10m or 24h")
errAccessInvalidWindow = Errorf("Window must be a positive duration with AccessTransition")
errAccessInvalidPromote = Errorf("PromoteAfterAccesses must be a positive integer with AccessTransition")
errAccessInvalidDemote = Errorf("DemoteAfterAccesses must be smaller than PromoteAfterAccesses and 0 or greater")
errAccessInvalidIdle = Errorf("DemoteAfterIdle must be a positive duration no shorter than Window")
errAccessInvalidQuotaSize = Errorf("AccessTierQuota must be a valid size, e.g. 500GiB")
)
// Duration is a time.Duration that marshals to and from an XML element
// holding a Go duration string, e.g. <Window>10m</Window>.
type Duration time.Duration
// UnmarshalXML parses a duration string such as "10m" or "24h".
func (d *Duration) UnmarshalXML(dec *xml.Decoder, start xml.StartElement) error {
var s string
if err := dec.DecodeElement(&s, &start); err != nil {
return err
}
dur, err := time.ParseDuration(s)
if err != nil {
return errAccessInvalidDuration
}
*d = Duration(dur)
return nil
}
// MarshalXML encodes a non-zero duration, and nothing otherwise.
func (d Duration) MarshalXML(enc *xml.Encoder, start xml.StartElement) error {
if d == 0 {
return nil
}
return enc.EncodeElement(time.Duration(d).String(), start)
}
// D returns the value as a time.Duration.
func (d Duration) D() time.Duration {
return time.Duration(d)
}
// AccessTransition is a Silo extension to the S3 lifecycle rule. It relocates
// an object between server pools based on how often it is read, rather than on
// its age: an object read at least PromoteAfterAccesses times within Window
// moves to the fastest configured pool, and moves back once it has been idle
// for DemoteAfterIdle and its windowed hit count has fallen to
// DemoteAfterAccesses or below.
//
// The gap between the two thresholds, together with DemoteAfterIdle and the
// server-side access_min_residency, is what keeps an object from oscillating
// between pools.
type AccessTransition struct {
XMLName xml.Name `xml:"AccessTransition"`
Window Duration `xml:"Window,omitempty"`
PromoteAfterAccesses int `xml:"PromoteAfterAccesses,omitempty"`
DemoteAfterAccesses int `xml:"DemoteAfterAccesses,omitempty"`
DemoteAfterIdle Duration `xml:"DemoteAfterIdle,omitempty"`
set bool
}
// IsNull returns true if no usable access transition is configured.
func (a AccessTransition) IsNull() bool {
return !a.set || a.PromoteAfterAccesses <= 0
}
// MarshalXML encodes an AccessTransition element, and nothing if unset.
func (a AccessTransition) MarshalXML(enc *xml.Encoder, start xml.StartElement) error {
if !a.set {
return nil
}
type accessTransitionWrapper AccessTransition
return enc.EncodeElement(accessTransitionWrapper(a), start)
}
// UnmarshalXML decodes an AccessTransition element.
func (a *AccessTransition) UnmarshalXML(dec *xml.Decoder, start xml.StartElement) error {
type accessTransitionWrapper AccessTransition
var atw accessTransitionWrapper
if err := dec.DecodeElement(&atw, &start); err != nil {
return err
}
*a = AccessTransition(atw)
a.set = true
return nil
}
// Validate checks the AccessTransition element.
func (a AccessTransition) Validate() error {
if !a.set {
return nil
}
if a.Window <= 0 {
return errAccessInvalidWindow
}
if a.PromoteAfterAccesses <= 0 {
return errAccessInvalidPromote
}
if a.DemoteAfterAccesses < 0 || a.DemoteAfterAccesses >= a.PromoteAfterAccesses {
return errAccessInvalidDemote
}
if a.DemoteAfterIdle <= 0 || a.DemoteAfterIdle < a.Window {
return errAccessInvalidIdle
}
return nil
}
internal/bucket/lifecycle/access-transition_test.go
-219
View File
@@ -1,219 +0,0 @@
// Copyright (c) 2015-2026 MinIO, Inc.
//
// This file is part of MinIO Object Storage stack
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package lifecycle
import (
"bytes"
"encoding/xml"
"strings"
"testing"
"time"
"github.com/minio/minio/internal/bucket/object/lock"
)
const accessTieringXML = `<LifecycleConfiguration>
<AccessTierQuota>500GiB</AccessTierQuota>
<Rule>
<ID>hot-logs</ID>
<Status>Enabled</Status>
<Filter><And><Prefix>logs/</Prefix><ObjectSizeGreaterThan>65536</ObjectSizeGreaterThan></And></Filter>
<AccessTransition>
<Window>10m</Window>
<PromoteAfterAccesses>100</PromoteAfterAccesses>
<DemoteAfterAccesses>5</DemoteAfterAccesses>
<DemoteAfterIdle>24h</DemoteAfterIdle>
</AccessTransition>
</Rule>
</LifecycleConfiguration>`
func TestAccessTransitionParse(t *testing.T) {
lc, err := ParseLifecycleConfig(strings.NewReader(accessTieringXML))
if err != nil {
t.Fatalf("parse: %v", err)
}
if err := lc.Validate(lock.Retention{}); err != nil {
t.Fatalf("validate: %v", err)
}
if got := lc.AccessQuotaBytes(); got != 500*1024*1024*1024 {
t.Fatalf("quota = %d, want %d", got, 500*1024*1024*1024)
}
if !lc.HasAccessTransition() {
t.Fatal("HasAccessTransition = false, want true")
}
at := lc.Rules[0].AccessTransition
if at.Window.D() != 10*time.Minute {
t.Fatalf("window = %v, want 10m", at.Window.D())
}
if at.DemoteAfterIdle.D() != 24*time.Hour {
t.Fatalf("idle = %v, want 24h", at.DemoteAfterIdle.D())
}
if at.PromoteAfterAccesses != 100 || at.DemoteAfterAccesses != 5 {
t.Fatalf("thresholds = %d/%d, want 100/5", at.PromoteAfterAccesses, at.DemoteAfterAccesses)
}
}
// A round trip through Marshal must preserve both the rule element and the
// bucket-wide quota, since PutBucketLifecycle stores whatever we re-encode.
func TestAccessTransitionRoundTrip(t *testing.T) {
lc, err := ParseLifecycleConfig(strings.NewReader(accessTieringXML))
if err != nil {
t.Fatalf("parse: %v", err)
}
buf, err := xml.Marshal(lc)
if err != nil {
t.Fatalf("marshal: %v", err)
}
if !bytes.Contains(buf, []byte("<AccessTierQuota>500GiB</AccessTierQuota>")) {
t.Fatalf("quota lost in round trip: %s", buf)
}
got, err := ParseLifecycleConfig(bytes.NewReader(buf))
if err != nil {
t.Fatalf("reparse: %v", err)
}
if got.AccessQuotaBytes() != lc.AccessQuotaBytes() {
t.Fatalf("quota %d != %d", got.AccessQuotaBytes(), lc.AccessQuotaBytes())
}
if got.Rules[0].AccessTransition != lc.Rules[0].AccessTransition {
t.Fatalf("rule %+v != %+v", got.Rules[0].AccessTransition, lc.Rules[0].AccessTransition)
}
}
// A rule with no AccessTransition must not emit an empty element - otherwise
// every existing lifecycle config would change shape on rewrite.
func TestAccessTransitionUnsetNotMarshalled(t *testing.T) {
lc, err := ParseLifecycleConfig(strings.NewReader(`<LifecycleConfiguration><Rule>
<ID>old</ID><Status>Enabled</Status><Filter><Prefix>a/</Prefix></Filter>
<Expiration><Days>3</Days></Expiration></Rule></LifecycleConfiguration>`))
if err != nil {
t.Fatalf("parse: %v", err)
}
buf, err := xml.Marshal(lc)
if err != nil {
t.Fatalf("marshal: %v", err)
}
if bytes.Contains(buf, []byte("AccessTransition")) || bytes.Contains(buf, []byte("AccessTierQuota")) {
t.Fatalf("unset elements emitted: %s", buf)
}
if lc.HasAccessTransition() {
t.Fatal("HasAccessTransition = true for a plain expiry rule")
}
}
func TestAccessTransitionValidate(t *testing.T) {
tests := []struct {
name string
at AccessTransition
err error
}{
{"ok", AccessTransition{Window: Duration(10 * time.Minute), PromoteAfterAccesses: 100, DemoteAfterAccesses: 5, DemoteAfterIdle: Duration(time.Hour), set: true}, nil},
{"unset", AccessTransition{}, nil},
{"zero window", AccessTransition{PromoteAfterAccesses: 100, DemoteAfterIdle: Duration(time.Hour), set: true}, errAccessInvalidWindow},
{"zero promote", AccessTransition{Window: Duration(time.Minute), DemoteAfterIdle: Duration(time.Hour), set: true}, errAccessInvalidPromote},
{"demote >= promote", AccessTransition{Window: Duration(time.Minute), PromoteAfterAccesses: 5, DemoteAfterAccesses: 5, DemoteAfterIdle: Duration(time.Hour), set: true}, errAccessInvalidDemote},
{"negative demote", AccessTransition{Window: Duration(time.Minute), PromoteAfterAccesses: 5, DemoteAfterAccesses: -1, DemoteAfterIdle: Duration(time.Hour), set: true}, errAccessInvalidDemote},
{"idle shorter than window", AccessTransition{Window: Duration(time.Hour), PromoteAfterAccesses: 5, DemoteAfterIdle: Duration(time.Minute), set: true}, errAccessInvalidIdle},
{"zero idle", AccessTransition{Window: Duration(time.Minute), PromoteAfterAccesses: 5, set: true}, errAccessInvalidIdle},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
if err := tc.at.Validate(); err != tc.err {
t.Fatalf("err = %v, want %v", err, tc.err)
}
})
}
}
func TestAccessTierQuotaInvalid(t *testing.T) {
lc, err := ParseLifecycleConfig(strings.NewReader(`<LifecycleConfiguration>
<AccessTierQuota>not-a-size</AccessTierQuota>
<Rule><ID>r</ID><Status>Enabled</Status><Expiration><Days>3</Days></Expiration></Rule>
</LifecycleConfiguration>`))
if err != nil {
t.Fatalf("parse: %v", err)
}
if err := lc.Validate(lock.Retention{}); err != errAccessInvalidQuotaSize {
t.Fatalf("err = %v, want %v", err, errAccessInvalidQuotaSize)
}
// An invalid quota that somehow reached the evaluator means "unlimited",
// never "zero bytes allowed".
if got := lc.AccessQuotaBytes(); got != 0 {
t.Fatalf("quota = %d, want 0 (unlimited)", got)
}
}
func TestAccessTransitionBadDuration(t *testing.T) {
_, err := ParseLifecycleConfig(strings.NewReader(`<LifecycleConfiguration><Rule>
<ID>r</ID><Status>Enabled</Status>
<AccessTransition><Window>ten minutes</Window><PromoteAfterAccesses>1</PromoteAfterAccesses></AccessTransition>
</Rule></LifecycleConfiguration>`))
if err == nil {
t.Fatal("expected a parse error for a malformed duration")
}
}
// AccessRule must reuse the standard rule filtering: prefix, tags, size and
// Status all have to be honored.
func TestAccessRuleFiltering(t *testing.T) {
lc, err := ParseLifecycleConfig(strings.NewReader(accessTieringXML))
if err != nil {
t.Fatalf("parse: %v", err)
}
tests := []struct {
name string
obj ObjectOpts
want bool
}{
{"match", ObjectOpts{Name: "logs/a.log", Size: 1 << 20, IsLatest: true}, true},
{"wrong prefix", ObjectOpts{Name: "data/a.log", Size: 1 << 20, IsLatest: true}, false},
{"too small", ObjectOpts{Name: "logs/a.log", Size: 1024, IsLatest: true}, false},
{"no name", ObjectOpts{Size: 1 << 20}, false},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
_, id, ok := lc.AccessRule(tc.obj)
if ok != tc.want {
t.Fatalf("ok = %v, want %v", ok, tc.want)
}
if ok && id != "hot-logs" {
t.Fatalf("ruleID = %q, want hot-logs", id)
}
})
}
lc.Rules[0].Status = Disabled
if _, _, ok := lc.AccessRule(ObjectOpts{Name: "logs/a.log", Size: 1 << 20, IsLatest: true}); ok {
t.Fatal("disabled rule still matched")
}
if lc.HasAccessTransition() {
t.Fatal("HasAccessTransition = true with only a disabled rule")
}
}
func TestAccessTransitionCountsAsActiveRule(t *testing.T) {
lc, err := ParseLifecycleConfig(strings.NewReader(accessTieringXML))
if err != nil {
t.Fatal(err)
}
if !lc.HasActiveRules("logs/2026") {
t.Fatal("access-only lifecycle rule was not active for its prefix")
}
if lc.HasActiveRules("data/") {
t.Fatal("access rule was active outside its prefix")
}
}
internal/bucket/lifecycle/lifecycle.go
+4 -59
View File
@@ -26,7 +26,6 @@ import (
"strings"
"time"
"github.com/dustin/go-humanize"
"github.com/google/uuid"
"github.com/minio/minio/internal/bucket/object/lock"
"github.com/minio/minio/internal/bucket/replication"
@@ -102,56 +101,11 @@ func (a Action) Delete() bool {
// Lifecycle - Configuration for bucket lifecycle.
type Lifecycle struct {
XMLName xml.Name `xml:"LifecycleConfiguration"`
// AccessTierQuota caps how many bytes of this bucket access-based ILM
// may keep on the fastest pool, e.g. "500GiB". Empty means unlimited.
// It is bucket-wide rather than per-rule so it cannot be declared
// inconsistently by two rules matching the same object.
AccessTierQuota string `xml:"AccessTierQuota,omitempty"`
XMLName xml.Name `xml:"LifecycleConfiguration"`
Rules []Rule `xml:"Rule"`
ExpiryUpdatedAt *time.Time `xml:"ExpiryUpdatedAt,omitempty"`
}
// HasAccessTransition returns 'true' if any enabled rule carries a usable
// AccessTransition. Used as a cheap per-bucket gate before consulting the
// access tracker.
func (lc Lifecycle) HasAccessTransition() bool {
for _, rule := range lc.Rules {
if rule.Status == Disabled {
continue
}
if !rule.AccessTransition.IsNull() {
return true
}
}
return false
}
// AccessRule returns the first enabled rule matching obj that carries a usable
// AccessTransition, along with its rule ID.
func (lc Lifecycle) AccessRule(obj ObjectOpts) (AccessTransition, string, bool) {
for _, rule := range lc.FilterRules(obj) {
if !rule.AccessTransition.IsNull() {
return rule.AccessTransition, rule.ID, true
}
}
return AccessTransition{}, "", false
}
// AccessQuotaBytes returns the bucket-wide fast-pool byte cap, 0 meaning
// unlimited. The value is validated at PUT time, so a parse failure here is
// treated as unlimited rather than as an error.
func (lc Lifecycle) AccessQuotaBytes() uint64 {
if lc.AccessTierQuota == "" {
return 0
}
sz, err := humanize.ParseBytes(lc.AccessTierQuota)
if err != nil {
return 0
}
return sz
}
// HasTransition returns 'true' if lifecycle document has Transition enabled.
func (lc Lifecycle) HasTransition() bool {
for _, rule := range lc.Rules {
@@ -205,11 +159,11 @@ func (lc *Lifecycle) UnmarshalXML(d *xml.Decoder, start xml.StartElement) (err e
}
lc.ExpiryUpdatedAt = &t
case "AccessTierQuota":
var q string
if err = d.DecodeElement(&q, &se); err != nil {
// Ignore the retired access-tier extension in stored configs.
// PUT uses this parser too, so it also accepts and drops it.
if err = d.Skip(); err != nil {
return err
}
lc.AccessTierQuota = q
default:
return xml.UnmarshalError(fmt.Sprintf("expected element type <Rule> but have <%s>", se.Name.Local))
}
@@ -260,9 +214,6 @@ func (lc Lifecycle) HasActiveRules(prefix string) bool {
if !rule.Transition.IsNull() { // this allows for Transition.Days to be zero.
return true
}
if !rule.AccessTransition.IsNull() {
return true
}
}
return false
}
@@ -303,12 +254,6 @@ func (lc Lifecycle) Validate(lr lock.Retention) error {
return errLifecycleNoRule
}
if lc.AccessTierQuota != "" {
if _, err := humanize.ParseBytes(lc.AccessTierQuota); err != nil {
return errAccessInvalidQuotaSize
}
}
// Validate all the rules in the lifecycle config
for _, r := range lc.Rules {
if err := r.Validate(); err != nil {
internal/bucket/lifecycle/retired-access_test.go
+81
View File
@@ -0,0 +1,81 @@
// Copyright (c) 2026 Feng Ruohang
//
// This file is part of Silo Object Storage stack
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package lifecycle
import (
"bytes"
"encoding/xml"
"strings"
"testing"
"github.com/minio/minio/internal/bucket/object/lock"
)
func TestLifecycleRetiredAccessExtensions(t *testing.T) {
const accessRule = `<Rule><ID>old-access</ID><Status>Enabled</Status><Filter><Prefix>logs/</Prefix></Filter><AccessTransition><Window>10m</Window><PromoteAfterAccesses>100</PromoteAfterAccesses><DemoteAfterAccesses>5</DemoteAfterAccesses><DemoteAfterIdle>24h</DemoteAfterIdle></AccessTransition></Rule>`
const ordinaryRule = `<Rule><ID>expiry</ID><Status>Enabled</Status><Filter><Prefix>expired/</Prefix></Filter><Expiration><Days>30</Days></Expiration></Rule>`
for _, mixed := range []bool{false, true} {
input := `<LifecycleConfiguration><AccessTierQuota>500GiB</AccessTierQuota>` + accessRule
if mixed {
input += ordinaryRule
}
input += `</LifecycleConfiguration>`
lc, err := ParseLifecycleConfig(strings.NewReader(input))
if err != nil {
t.Fatal(err)
}
if lc.HasActiveRules("logs/") {
t.Fatal("retired rule is active")
}
if lc.HasActiveRules("expired/") != mixed {
t.Fatalf("ordinary expiration lost (mixed=%v)", mixed)
}
encoded, err := xml.Marshal(lc)
if err != nil {
t.Fatal(err)
}
if bytes.Contains(encoded, []byte("AccessTierQuota")) || bytes.Contains(encoded, []byte("AccessTransition")) {
t.Fatalf("retired extension re-emitted: %s", encoded)
}
if err := lc.Validate(lock.Retention{}); err == nil {
t.Fatal("actionless rule unexpectedly validates")
}
// Operators remove access-only rules before editing the remaining config.
if mixed {
lc.Rules = lc.Rules[1:]
if err := lc.Validate(lock.Retention{}); err != nil {
t.Fatalf("ordinary ILM edit after removing retired rule: %v", err)
}
}
}
// A quota on a normal rule is accepted and dropped by the PUT parser too.
input := `<LifecycleConfiguration><AccessTierQuota>500GiB</AccessTierQuota>` + ordinaryRule + `</LifecycleConfiguration>`
lc, err := ParseLifecycleConfigWithID(strings.NewReader(input))
if err != nil {
t.Fatal(err)
}
if err := lc.Validate(lock.Retention{}); err != nil {
t.Fatal(err)
}
if _, err := ParseLifecycleConfig(strings.NewReader(`<LifecycleConfiguration><Unknown>1</Unknown></LifecycleConfiguration>`)); err == nil {
t.Fatal("unknown top-level element accepted")
}
if _, err := ParseLifecycleConfig(strings.NewReader(`<LifecycleConfiguration><AccessTierQuota><broken></AccessTierQuota></LifecycleConfiguration>`)); err == nil {
t.Fatal("malformed XML accepted")
}
}
internal/bucket/lifecycle/rule.go
+1 -5
View File
@@ -41,7 +41,6 @@ type Rule struct {
Expiration Expiration `xml:"Expiration,omitempty"`
Transition Transition `xml:"Transition,omitempty"`
DelMarkerExpiration DelMarkerExpiration `xml:"DelMarkerExpiration,omitempty"`
AccessTransition AccessTransition `xml:"AccessTransition,omitempty"`
// FIXME: add a type to catch unsupported AbortIncompleteMultipartUpload AbortIncompleteMultipartUpload `xml:"AbortIncompleteMultipartUpload,omitempty"`
NoncurrentVersionExpiration NoncurrentVersionExpiration `xml:"NoncurrentVersionExpiration,omitempty"`
NoncurrentVersionTransition NoncurrentVersionTransition `xml:"NoncurrentVersionTransition,omitempty"`
@@ -172,13 +171,10 @@ func (r Rule) Validate() error {
if err := r.validateNoncurrentTransition(); err != nil {
return err
}
if err := r.AccessTransition.Validate(); err != nil {
return err
}
if (!r.Filter.Tag.IsEmpty() || len(r.Filter.And.Tags) != 0) && !r.DelMarkerExpiration.Empty() {
return errInvalidRuleDelMarkerExpiration
}
if !r.Expiration.set && !r.Transition.set && !r.NoncurrentVersionExpiration.set && !r.NoncurrentVersionTransition.set && r.DelMarkerExpiration.Empty() && !r.AccessTransition.set {
if !r.Expiration.set && !r.Transition.set && !r.NoncurrentVersionExpiration.set && !r.NoncurrentVersionTransition.set && r.DelMarkerExpiration.Empty() {
return errXMLNotWellFormed
}
return nil
internal/config/ilm/help.go
-93
View File
@@ -22,43 +22,10 @@ import "github.com/minio/minio/internal/config"
const (
transitionWorkers = "transition_workers"
expirationWorkers = "expiration_workers"
accessTiering = "access_tiering"
accessPools = "access_pools"
accessMaxSize = "access_max_size"
accessPromoteWatermark = "access_promote_watermark"
accessBinWidth = "access_bin_width"
accessBins = "access_bins"
accessFlush = "access_flush"
accessMinResidency = "access_min_residency"
accessWorkers = "access_workers"
accessMaxTracked = "access_max_tracked"
// EnvILMTransitionWorkers env variable to configure number of transition workers
EnvILMTransitionWorkers = "MINIO_ILM_TRANSITION_WORKERS"
// EnvILMExpirationWorkers env variable to configure number of expiration workers
EnvILMExpirationWorkers = "MINIO_ILM_EXPIRATION_WORKERS"
// EnvILMAccessTiering env variable to enable access based tiering
EnvILMAccessTiering = "MINIO_ILM_ACCESS_TIERING"
// EnvILMAccessPools env variable listing pool indices hottest first
EnvILMAccessPools = "MINIO_ILM_ACCESS_POOLS"
// EnvILMAccessMaxSize env variable capping bytes held on the hottest pool
EnvILMAccessMaxSize = "MINIO_ILM_ACCESS_MAX_SIZE"
// EnvILMAccessPromoteWatermark env variable for the hottest pool fill limit
EnvILMAccessPromoteWatermark = "MINIO_ILM_ACCESS_PROMOTE_WATERMARK"
// EnvILMAccessBinWidth env variable for the access counter resolution
EnvILMAccessBinWidth = "MINIO_ILM_ACCESS_BIN_WIDTH"
// EnvILMAccessBins env variable for the number of access counter bins
EnvILMAccessBins = "MINIO_ILM_ACCESS_BINS"
// EnvILMAccessFlush env variable for the counter publish and sweep interval
EnvILMAccessFlush = "MINIO_ILM_ACCESS_FLUSH"
// EnvILMAccessMinResidency env variable for the anti-thrash floor
EnvILMAccessMinResidency = "MINIO_ILM_ACCESS_MIN_RESIDENCY"
// EnvILMAccessWorkers env variable to configure number of access tiering workers
EnvILMAccessWorkers = "MINIO_ILM_ACCESS_WORKERS"
// EnvILMAccessMaxTracked env variable capping tracked objects per node
EnvILMAccessMaxTracked = "MINIO_ILM_ACCESS_MAX_TRACKED"
)
var (
@@ -81,65 +48,5 @@ var (
Description: `set the number of expiration workers` + defaultHelpPostfix(expirationWorkers),
Optional: true,
},
config.HelpKV{
Key: accessTiering,
Type: "on|off",
Description: `move objects between fast and slow server pools based on how often they are read` + defaultHelpPostfix(accessTiering),
Optional: true,
},
config.HelpKV{
Key: accessPools,
Type: "string",
Description: `server pool indices for access tiering, hottest first, e.g. "0,1"` + defaultHelpPostfix(accessPools),
Optional: true,
},
config.HelpKV{
Key: accessMaxSize,
Type: "string",
Description: `cap total bytes access tiering keeps on the fastest pool, e.g. "2TiB", 0 for unlimited` + defaultHelpPostfix(accessMaxSize),
Optional: true,
},
config.HelpKV{
Key: accessPromoteWatermark,
Type: "number",
Description: `stop promoting once the fastest pool is this percent full` + defaultHelpPostfix(accessPromoteWatermark),
Optional: true,
},
config.HelpKV{
Key: accessBinWidth,
Type: "duration",
Description: `resolution of the access counter` + defaultHelpPostfix(accessBinWidth),
Optional: true,
},
config.HelpKV{
Key: accessBins,
Type: "number",
Description: `number of access counter bins; bins times bin width caps the rule Window` + defaultHelpPostfix(accessBins),
Optional: true,
},
config.HelpKV{
Key: accessFlush,
Type: "duration",
Description: `how often access counters are published and a promotion sweep runs` + defaultHelpPostfix(accessFlush),
Optional: true,
},
config.HelpKV{
Key: accessMinResidency,
Type: "duration",
Description: `minimum time an object stays on a pool after access tiering moved it` + defaultHelpPostfix(accessMinResidency),
Optional: true,
},
config.HelpKV{
Key: accessWorkers,
Type: "number",
Description: `set the number of access tiering workers` + defaultHelpPostfix(accessWorkers),
Optional: true,
},
config.HelpKV{
Key: accessMaxTracked,
Type: "number",
Description: `maximum number of objects each node keeps access counters for` + defaultHelpPostfix(accessMaxTracked),
Optional: true,
},
}
)
internal/config/ilm/ilm.go
+13 -202
View File
@@ -18,29 +18,12 @@
package ilm
import (
"errors"
"strconv"
"strings"
"time"
"github.com/dustin/go-humanize"
"github.com/minio/minio/internal/config"
"github.com/pgsty/silo-pkg/v3/env"
)
// Errors returned when the access tiering configuration is unusable.
var (
ErrAccessPoolsInvalid = errors.New("ilm access_pools must be a comma separated list of distinct pool indices, hottest first, e.g. \"0,1\"")
ErrAccessPoolsTooFew = errors.New("ilm access_pools needs at least two pools to move objects between")
ErrAccessWatermarkInvalid = errors.New("ilm access_promote_watermark must be between 1 and 100")
ErrAccessBinsInvalid = errors.New("ilm access_bins must be between 2 and 64")
ErrAccessBinWidthInvalid = errors.New("ilm access_bin_width must be at least 1s")
ErrAccessFlushInvalid = errors.New("ilm access_flush must be at least 1s")
ErrAccessResidencyInvalid = errors.New("ilm access_min_residency cannot be negative")
ErrAccessWorkersInvalid = errors.New("ilm access_workers must be a positive integer")
ErrAccessTrackedInvalid = errors.New("ilm access_max_tracked must be a positive integer")
)
// DefaultKVS default configuration values for ILM subsystem
var DefaultKVS = config.KVS{
config.KV{
@@ -51,124 +34,26 @@ var DefaultKVS = config.KVS{
Key: expirationWorkers,
Value: "100",
},
config.KV{
Key: accessTiering,
Value: config.EnableOff,
},
config.KV{
Key: accessPools,
Value: "",
},
config.KV{
Key: accessMaxSize,
Value: "0",
},
config.KV{
Key: accessPromoteWatermark,
Value: "85",
},
config.KV{
Key: accessBinWidth,
Value: "1m",
},
config.KV{
Key: accessBins,
Value: "12",
},
config.KV{
Key: accessFlush,
Value: "1m",
},
config.KV{
Key: accessMinResidency,
Value: "24h",
},
config.KV{
Key: accessWorkers,
Value: "10",
},
config.KV{
Key: accessMaxTracked,
Value: "1000000",
},
}
// Config represents the different configuration values for ILM subsystem
type Config struct {
TransitionWorkers int
ExpirationWorkers int
// AccessTiering enables access-frequency driven relocation of objects
// between server pools. Off by default: it moves data.
AccessTiering bool
// AccessPools lists pool indices hottest first, e.g. []int{0, 1}.
AccessPools []int
// AccessMaxSize caps total bytes held on the hottest pool, 0 == unlimited.
AccessMaxSize uint64
// AccessPromoteWatermark stops promotion once the hottest pool is this
// percentage full.
AccessPromoteWatermark int
// AccessBinWidth and AccessBins size the rolling hit counter. Their
// product is the longest rule Window that can be evaluated.
AccessBinWidth time.Duration
AccessBins int
// AccessFlush is how often each node publishes its counters and the
// leader merges them and runs a promotion sweep.
AccessFlush time.Duration
// AccessMinResidency is the minimum time an object stays put after a
// move, regardless of what the counters say.
AccessMinResidency time.Duration
AccessWorkers int
// AccessMaxTracked caps how many objects each node keeps counters for.
AccessMaxTracked int
}
// HotPool returns the index of the hottest configured pool and whether access
// tiering is usable at all.
func (c Config) HotPool() (int, bool) {
if !c.AccessTiering || len(c.AccessPools) < 2 {
return -1, false
}
return c.AccessPools[0], true
}
// ColdPool returns the index of the coldest configured pool, i.e. where
// demoted objects go.
func (c Config) ColdPool() (int, bool) {
if !c.AccessTiering || len(c.AccessPools) < 2 {
return -1, false
}
return c.AccessPools[len(c.AccessPools)-1], true
}
// HistoryWindow is the longest window the rolling counter can answer for.
func (c Config) HistoryWindow() time.Duration {
return time.Duration(c.AccessBins) * c.AccessBinWidth
}
// parseAccessPools parses "0,1" into []int{0, 1}, rejecting duplicates and
// negative indices. An empty string yields no pools, which disables the
// feature rather than erroring - operators enable the switch before they
// configure the topology.
func parseAccessPools(s string) ([]int, error) {
s = strings.TrimSpace(s)
if s == "" {
return nil, nil
}
var pools []int
seen := make(map[int]struct{})
for _, f := range strings.Split(s, ",") {
idx, err := strconv.Atoi(strings.TrimSpace(f))
if err != nil || idx < 0 {
return nil, ErrAccessPoolsInvalid
}
if _, dup := seen[idx]; dup {
return nil, ErrAccessPoolsInvalid
}
seen[idx] = struct{}{}
pools = append(pools, idx)
}
return pools, nil
// Accept stored access-tier settings without re-enabling the retired feature.
var deprecatedKeys = []string{
"access_tiering",
"access_pools",
"access_max_size",
"access_promote_watermark",
"access_bin_width",
"access_bins",
"access_flush",
"access_min_residency",
"access_workers",
"access_max_tracked",
}
// LookupConfig - lookup ilm config and override with valid environment settings if any.
@@ -178,7 +63,7 @@ func LookupConfig(kvs config.KVS) (cfg Config, err error) {
ExpirationWorkers: 100,
}
if err = config.CheckValidKeys(config.ILMSubSys, kvs, DefaultKVS); err != nil {
if err = config.CheckValidKeys(config.ILMSubSys, kvs, DefaultKVS, deprecatedKeys...); err != nil {
return cfg, err
}
@@ -194,79 +79,5 @@ func LookupConfig(kvs config.KVS) (cfg Config, err error) {
cfg.TransitionWorkers = tw
cfg.ExpirationWorkers = ew
if err := cfg.lookupAccess(kvs); err != nil {
return cfg, err
}
return cfg, nil
}
func (c *Config) lookupAccess(kvs config.KVS) (err error) {
c.AccessTiering, err = config.ParseBool(env.Get(EnvILMAccessTiering, kvs.GetWithDefault(accessTiering, DefaultKVS)))
if err != nil {
return err
}
if c.AccessPools, err = parseAccessPools(env.Get(EnvILMAccessPools, kvs.GetWithDefault(accessPools, DefaultKVS))); err != nil {
return err
}
// Enabling the feature without a usable topology is a configuration
// error worth surfacing at set time rather than silently doing nothing.
if c.AccessTiering && len(c.AccessPools) < 2 {
return ErrAccessPoolsTooFew
}
maxSize := env.Get(EnvILMAccessMaxSize, kvs.GetWithDefault(accessMaxSize, DefaultKVS))
if maxSize == "" || maxSize == "0" {
c.AccessMaxSize = 0
} else if c.AccessMaxSize, err = humanize.ParseBytes(maxSize); err != nil {
return err
}
if c.AccessPromoteWatermark, err = strconv.Atoi(env.Get(EnvILMAccessPromoteWatermark, kvs.GetWithDefault(accessPromoteWatermark, DefaultKVS))); err != nil {
return err
}
if c.AccessPromoteWatermark < 1 || c.AccessPromoteWatermark > 100 {
return ErrAccessWatermarkInvalid
}
if c.AccessBinWidth, err = time.ParseDuration(env.Get(EnvILMAccessBinWidth, kvs.GetWithDefault(accessBinWidth, DefaultKVS))); err != nil {
return err
}
if c.AccessBinWidth < time.Second {
return ErrAccessBinWidthInvalid
}
if c.AccessBins, err = strconv.Atoi(env.Get(EnvILMAccessBins, kvs.GetWithDefault(accessBins, DefaultKVS))); err != nil {
return err
}
if c.AccessBins < 2 || c.AccessBins > 64 {
return ErrAccessBinsInvalid
}
if c.AccessFlush, err = time.ParseDuration(env.Get(EnvILMAccessFlush, kvs.GetWithDefault(accessFlush, DefaultKVS))); err != nil {
return err
}
if c.AccessFlush < time.Second {
return ErrAccessFlushInvalid
}
if c.AccessMinResidency, err = time.ParseDuration(env.Get(EnvILMAccessMinResidency, kvs.GetWithDefault(accessMinResidency, DefaultKVS))); err != nil {
return err
}
if c.AccessMinResidency < 0 {
return ErrAccessResidencyInvalid
}
if c.AccessWorkers, err = strconv.Atoi(env.Get(EnvILMAccessWorkers, kvs.GetWithDefault(accessWorkers, DefaultKVS))); err != nil {
return err
}
if c.AccessWorkers < 1 {
return ErrAccessWorkersInvalid
}
if c.AccessMaxTracked, err = strconv.Atoi(env.Get(EnvILMAccessMaxTracked, kvs.GetWithDefault(accessMaxTracked, DefaultKVS))); err != nil {
return err
}
if c.AccessMaxTracked < 1 {
return ErrAccessTrackedInvalid
}
return nil
}
internal/config/ilm/ilm_test.go
+42 -91
View File
@@ -1,109 +1,60 @@
// Copyright (c) 2015-2026 MinIO, Inc.
// Copyright (c) 2026 Feng Ruohang
//
// This file is part of Silo Object Storage stack
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package ilm
import (
"errors"
"reflect"
"testing"
"time"
"github.com/minio/minio/internal/config"
)
func clearAccessEnv(t *testing.T) {
t.Helper()
for _, name := range []string{
EnvILMAccessTiering, EnvILMAccessPools, EnvILMAccessMaxSize,
EnvILMAccessPromoteWatermark, EnvILMAccessBinWidth, EnvILMAccessBins,
EnvILMAccessFlush, EnvILMAccessMinResidency, EnvILMAccessWorkers,
EnvILMAccessMaxTracked,
} {
// The env helper treats an empty value as unset. t.Setenv restores the
// caller's exact value automatically when the test finishes.
t.Setenv(name, "")
func TestLookupConfigRetiredAccessKeys(t *testing.T) {
t.Setenv(EnvILMTransitionWorkers, "")
t.Setenv(EnvILMExpirationWorkers, "")
kvs := config.KVS{
{Key: "transition_workers", Value: "37"},
{Key: "expiration_workers", Value: "23"},
{Key: "access_tiering", Value: "on"},
{Key: "access_pools", Value: "0,1"},
{Key: "access_max_size", Value: "500GiB"},
{Key: "access_promote_watermark", Value: "85"},
{Key: "access_bin_width", Value: "1m"},
{Key: "access_bins", Value: "12"},
{Key: "access_flush", Value: "1m"},
{Key: "access_min_residency", Value: "24h"},
{Key: "access_workers", Value: "10"},
{Key: "access_max_tracked", Value: "1000000"},
}
}
func TestLookupAccessDefaults(t *testing.T) {
clearAccessEnv(t)
cfg, err := LookupConfig(DefaultKVS.Clone())
if err != nil {
t.Fatal(err)
}
if cfg.AccessTiering {
t.Fatal("access tiering must default off")
}
if cfg.AccessBinWidth != time.Minute || cfg.AccessBins != 12 || cfg.AccessFlush != time.Minute {
t.Fatalf("unexpected counter defaults: width=%s bins=%d flush=%s", cfg.AccessBinWidth, cfg.AccessBins, cfg.AccessFlush)
}
if cfg.AccessWorkers != 10 || cfg.AccessMaxTracked != 1000000 {
t.Fatalf("unexpected worker/map defaults: %d/%d", cfg.AccessWorkers, cfg.AccessMaxTracked)
}
}
func TestLookupAccessEnabled(t *testing.T) {
clearAccessEnv(t)
kvs := DefaultKVS.Clone()
kvs.Set(accessTiering, config.EnableOn)
kvs.Set(accessPools, "2, 0, 1")
kvs.Set(accessMaxSize, "2GiB")
kvs.Set(accessPromoteWatermark, "90")
kvs.Set(accessBinWidth, "30s")
kvs.Set(accessBins, "20")
kvs.Set(accessFlush, "15s")
kvs.Set(accessMinResidency, "2h")
kvs.Set(accessWorkers, "7")
kvs.Set(accessMaxTracked, "1234")
cfg, err := LookupConfig(kvs)
if err != nil {
t.Fatal(err)
}
if !cfg.AccessTiering || !reflect.DeepEqual(cfg.AccessPools, []int{2, 0, 1}) {
t.Fatalf("unexpected topology: enabled=%v pools=%v", cfg.AccessTiering, cfg.AccessPools)
if cfg.TransitionWorkers != 37 || cfg.ExpirationWorkers != 23 {
t.Fatalf("worker settings lost: %+v", cfg)
}
if cfg.AccessMaxSize != 2<<30 || cfg.HistoryWindow() != 10*time.Minute {
t.Fatalf("unexpected size/history: %d/%s", cfg.AccessMaxSize, cfg.HistoryWindow())
// An ordinary ILM edit must keep working with stored retired keys.
kvs.Set("transition_workers", "41")
cfg, err = LookupConfig(kvs)
if err != nil || cfg.TransitionWorkers != 41 || cfg.ExpirationWorkers != 23 {
t.Fatalf("ILM edit: %+v, %v", cfg, err)
}
if hot, ok := cfg.HotPool(); !ok || hot != 2 {
t.Fatalf("HotPool = %d/%v", hot, ok)
}
if cold, ok := cfg.ColdPool(); !ok || cold != 1 {
t.Fatalf("ColdPool = %d/%v", cold, ok)
}
}
func TestLookupAccessRejectsInvalidValues(t *testing.T) {
clearAccessEnv(t)
tests := []struct {
key, value string
want error
}{
{accessPools, "0,0", ErrAccessPoolsInvalid},
{accessPromoteWatermark, "0", ErrAccessWatermarkInvalid},
{accessBinWidth, "500ms", ErrAccessBinWidthInvalid},
{accessBins, "1", ErrAccessBinsInvalid},
{accessFlush, "0s", ErrAccessFlushInvalid},
{accessMinResidency, "-1s", ErrAccessResidencyInvalid},
{accessWorkers, "0", ErrAccessWorkersInvalid},
{accessMaxTracked, "0", ErrAccessTrackedInvalid},
}
for _, tc := range tests {
t.Run(tc.key, func(t *testing.T) {
kvs := DefaultKVS.Clone()
kvs.Set(tc.key, tc.value)
_, err := LookupConfig(kvs)
if !errors.Is(err, tc.want) {
t.Fatalf("error = %v, want %v", err, tc.want)
}
})
}
kvs := DefaultKVS.Clone()
kvs.Set(accessTiering, config.EnableOn)
kvs.Set(accessPools, "0")
if _, err := LookupConfig(kvs); !errors.Is(err, ErrAccessPoolsTooFew) {
t.Fatalf("error = %v, want %v", err, ErrAccessPoolsTooFew)
kvs.Set("access_unknown", "on")
if _, err := LookupConfig(kvs); err == nil {
t.Fatal("unrecognized key accepted")
}
}