A batch expiration that keeps no versions purges each key as a whole,
and the purge refuses a key when one of its versions is transitioned to
a remote tier, as it refuses a key with a version Object Lock protects.
The job counted the Object Lock refusal as a refused key and completed,
but counted the tier refusal as failed versions and ended Failed, on
every run, with objectsFailed set to the versions the walk saw. ILM
expiry reports the same tier refusal as refused, not failed.
expireAllVersions now counts a key refused for a transitioned version as
refused, as it does a locked key: nothing was removed, the attempt is
not retried, and the job completes with keysRefused, which mcli reports.
A later run of the job checks the key again. A key whose history cannot
be read still fails the purge as before.
TestBatchExpirePurgeRefusesTieredKey runs a purge of a key with a
transitioned older version and requires a complete job with one refused
key and every version kept; TestPurgeAllVersionsRefusesTieredKeys now
requires refused keys rather than failed objects. Before this change
the job ended Failed with two failed objects.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
BatchJobExpire.Start checked the job's submitter before it looked at
the job's report, and refuse() clears Complete. A job that had completed
was therefore turned into a failed one on the next start: one that
completed before the upgrade records no submitter, and one completed
with a temporary credential that has expired since, a deleted user or a
changed password fails the binding. Such a job asked to be submitted
again, inviting an operator to run a destructive expiration twice.
Return before the submitter check when the report says the job is
complete, as replication and key rotation jobs already do.
Tests: a completed job without a submitter, and a completed job whose
temporary credential expired, keep their completed state and delete
nothing when started again; a job that did not complete is still
refused once its credential expired. The first two fail without the
change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
A Walk that returned an error before any result cancelled the run like
an ordinary cancellation: the job was saved as neither complete nor
failed. The listing failure is now recorded first, and the run is saved
as failed with its cursor where it was. Start still returns the error,
so the job is left to a later run, which lists again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
A batch expiration that keeps no versions authorized the versions its
walk saw, and the purge then deleted the versions it read under its
lock. A version written after the walk, or tags changed on one since,
was deleted without being authorized for the submitter.
The purge takes an authorization that a batch purge must set and no
other delete may. Under the object lock, once it has approved the
history and checked every drive, and before its first delete, it
hands the writes it is about to delete to the job: every copy of every
approved version and every revision of one that a drive holds, one per
distinct version ID and tags, as they are then. The job evaluates the
expire actions on each for the submitter. A denial deletes nothing and
ends the job as refused; the time it takes counts against the purge's
budget. A locked version still refuses the whole key, counted as a
refused key. The checks on the walk's snapshot are removed.
Reviewed adversarially; the review's regressions are included.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
A batch job ran with the server's object access, whoever submitted it,
and acted on versions that Object Lock protects.
Submitter: StartBatchJob binds the authenticated credential to the job
(type, access key, parent, identity revision and a keyed fingerprint).
Submission, every start and every object resolve the binding against
IAM and evaluate the action with the submitter's policies and session
policy. A credential that is gone, disabled, expired or changed fails
the job. A policy statement that would need request context the job
does not have refuses the action. With an authorization plugin, the
plugin and the session policy must both allow. A refusal is persisted
as the job's failure reason and ends the job. A job stored without a
binding fails and has to be submitted again; a definition holds one
job type.
Expire deletes each older version on its own, checking retention and
legal hold and authorizing it under the delete's lock; a locked version
is left and counted. Until the exact-key purge is integrated, a
retainVersions: 0 key is refused when a version the walk saw is locked.
Pull writes refuse to replace a locked destination version, use the
destination's versioning, validate copied retention and apply default
retention. Rotation leaves locked versions as they are.
The status reports versionsSkippedLocked, keysRefused and the failure
reason. Expire and rotation persist their cursor together with these
counts, so a resumed job counts each version once. Describe returns a
redacted copy of the job.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
An expiry of all versions (ExpiredObjectAllVersions, DelMarkerExpiration,
and batch expiration with the default purge) asked the pools layer for a
prefix delete. The pools layer rebuilt the options, so the request ran as
a recursive delete of the key's directory in every erasure set of every
pool. Object a/b is stored inside the directory of a, so every key under
a/ was removed with it, however new, with one audit entry and one
notification for a alone. The rebuilt options also skipped the lifecycle
recheck, so on a single pool a version written after the decision was
deleted too, and tiered versions lost their metadata without free
versions.
The expiry worker, the migration expiry and batch expiration now ask for
a purge with the in-memory ObjectOptions.DeleteAllVersions:
- DeleteObject hands it to the purge under the object lock, before the
all-pools check. ILM decides with the unchanged lifecycleEventsAllPools,
on a single pool too; batch needs the version its walk matched to still
be the latest. A second read of each pool's authoritative history
records every copy, older writes of the same ID included, and must
equal the approval. Anything else deletes nothing.
- Before deleting, the purge reads every drive of each set that holds a
copy. It refuses the key when a drive cannot be read, when any revision
of an approved version there is transitioned, or, with Object Lock,
when retention or a legal hold protects one. It then deletes on exactly
the drive handles it checked.
- Versions are deleted by ID with the drives' version delete, which only
removes the object directory once it is empty: oldest first, for each
version older writes before the copy reads choose, and the latest last,
only once every older version is gone from every pool. No marker,
replication state or free version is written. The purge stops at the
first error, repeats a quorum-successful delete on the drives that
missed it while the lock is held, and stops before the latest version
at a budget of 1000 versions or one minute, reads included.
- The ILM audit entry records how many versions were removed. A purge
that failed after it began to delete is audited as partial and sends
no removal event. Batch counts come from the purge; a key whose latest
version changed is not expired, and a failed delete marker fails the
job.
The option is rejected with conflicting options and by every layer below
the pools. Force delete and internal prefix deletes keep DeletePrefix.
Delete-all of a transitioned latest version keeps its current path, and
a key with tiered older versions is refused. On a single pool the purge
relies on the lock-domain fix for writer exclusion.
Reviewed adversarially in two design rounds and four patch rounds; the
last approved with nits, which are applied.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Feng Ruohang <rh@vonng.com>
Adopt silo-pkg v3.13.2, the 20260903 mcli release, and the latest validated SILO Console commit. Move maintained source imports to the pgsty/silo-pkg module path, refresh the resulting dependency closure and credits, and keep only documented legacy minio/pkg transitive residue.
Signed-off-by: Feng Ruohang <rh@vonng.com>
Switch to community-maintained console fork (georgmangold/console v1.9.1)
and update dependencies accordingly. Fix go vet format directive in
grid_test.go and adapt test status code for Go 1.26 HTTP semantics.
* fix: proxy requests to honor global transport
Load the globalProxyEndpoint properly
also, currently, the proxy requests will fail silently for batch cancel
even if the proxy fails; instead,d properly send the corresponding error back
for such proxy failures if opted
* pass the transport to the GetProxyEnpoints function
---------
Co-authored-by: Praveen raj Mani <praveen@minio.io>
A batch job will fail if the retry attempt is not provided. The reason
is that the code mistakenly gets the retry attempts from the job status
rather than the job yaml file.
This will also set a default empty prefix for batch expiration.
Also this will avoid trimming the prefix since the yaml decoder already
does that if no quotes were provided, and we should not trim if quotes
were provided and the user provided a leading or a trailing space.
- Also, fix failure reporting at the end.
- Also, avoid parsing report objects when listing or resuming jobs, this
does not cause any bugs, it is only printing, not useful errors.
Currently the status of a completed or failed batch is held in the
memory, a simple restart will lose the status and the user will not
have any visibility of the job that was long running.
In addition to the metrics, add a new API that reads the batch status
from the drives. A batch job will be cleaned up three days after
completion.
Also add the batch type in the batch id, the reason is that the batch
job request is removed immediately when the job is finished, then we
do not know the type of batch job anymore, hence a difficulty to locate
the job report
This change uses the updated ldap library in minio/pkg (bumped
up to v3). A new config parameter is added for LDAP configuration to
specify extra user attributes to load from the LDAP server and to store
them as additional claims for the user.
A test is added in sts_handlers.go that shows how to access the LDAP
attributes as a claim.
This is in preparation for adding SSH pubkey authentication to MinIO's SFTP
integration.
If used, 'opts.Marker` will cause many missed entries since results are returned
unsorted, and pools are serialized.
Switch to fully concurrent listing and merging across pools to return sorted entries.
Create new code paths for multiple subsystems in the code. This will
make maintaing this easier later.
Also introduce bugLogIf() for errors that should not happen in the first
place.
Make sure to pass a nil pointer as a Transport to minio-go when the API config
is not initialized, this will make sure that we do not pass an interface
with a known type but a nil value.
This will also fix the update of the API remote_transport_deadline
configuration without requiring the cluster restart.
Object names of directory objects qualified for ExpiredObjectAllVersions
must be encoded appropriately before calling on deletePrefix on their
erasure set.
e.g., a directory object and regular objects with overlapping prefixes
could lead to the expiration of regular objects, which is not the
intention of ILM.
```
bucket/dir/ ---> directory object
bucket/dir/obj-1
```
When `bucket/dir/` qualifies for expiration, the current implementation would
remove regular objects under the prefix `bucket/dir/`, in this case,
`bucket/dir/obj-1`.
Feng Ruohang