metacache: Add option for life extension (#10837)

Add `MINIO_API_EXTEND_LIST_CACHE_LIFE` that will extend 
the life of generated caches for a while.

This changes caches to remain valid until no updates have been 
received for the specified time plus a fixed margin.

This also changes the caches from being invalidated when the *first* 
set finishes until the *last* set has finished plus the specified time 
has passed.
This commit is contained in:
Klaus Post
2020-11-05 11:49:56 -08:00
committed by GitHub
parent b72cac4cf3
commit 0724205f35
5 changed files with 66 additions and 15 deletions
+3
View File
@@ -79,6 +79,9 @@ func (m *metacache) worthKeeping(currentCycle uint64) bool {
case cache.finished() && cache.startedCycle > currentCycle:
// Cycle is somehow bigger.
return false
case cache.finished() && time.Since(cache.lastHandout) > 48*time.Hour:
// Keep only for 2 days. Fallback if crawler is clogged.
return false
case cache.finished() && currentCycle >= dataUsageUpdateDirCycles && cache.startedCycle < currentCycle-dataUsageUpdateDirCycles:
// Cycle is too old to be valuable.
return false