Two coupled changes that must land together, because the same files carry both:
the server now identifies itself as Silo, and every path that would have called
home to a MinIO-operated service is closed.
Product identity
- build-constants.go: store name, UA name and startup banner become Silo. The
Go identifiers (MinioStoreName, MinioBannerName, ...) keep their names on
purpose - renaming exported symbols would churn the compatibility surface for
a cosmetic gain, and the rebrand guard freezes that surface.
- main.go, server-startup-msg.go, ftp-server.go and the user-visible log, help
and error strings across cmd/ and internal/ switch to Silo. Original MinIO
copyright, LICENSE, NOTICE and CREDITS are untouched; --version now prints
the upstream copyright, the pgsty modification notice, and the trademark
policy's approved "based on MinIO technology" attribution.
- api-headers.go: the HTTP Server header becomes "Silo". This is the one
externally observable identity change, so TestCommonHeadersUseSiloProductName
pins it - probes that sniff for "MinIO" must move to capability detection.
- Prometheus metric HELP strings keep their MinIO wording. They are part of the
metrics contract the guard protects, not product copy.
Configuration directory
- config-dir.go: new installs use ~/.silo. If only ~/.minio exists it is still
read, with a one-time notice and no files moved. If both exist ~/.silo wins
and an ambiguity warning is emitted; an explicit --config-dir always wins.
Covered by TestSelectDefaultConfigDir. The internal .minio.sys layout is
never renamed - this rule applies to the user config directory only.
Upstream service lockdown
- globalInplaceUpdateDisabled is now true at initialization rather than being
set from MINIO_UPDATE. common-main.go still parses MINIO_UPDATE so upgrading
nodes do not fail on an unknown key, but warns that the value is ignored;
there is no way to re-enable the updater. TestInplaceUpdateCannotBeEnabled
guards that. Without this, an admin with mc could have overwritten
/usr/bin/silo with an upstream MinIO binary.
- verifyBinary and commitBinary refuse early; the ServerUpdate v1/v2 admin
routes and the peer-rest update endpoints stay registered and keep returning
the existing programmatic error, so clients see a stable failure rather than
a 404.
- MinioReleaseBaseURL and defaultMinisignPubkey are emptied: no dl.min.io
download root, and upstream's minisign key is no longer a trust root for
anything this fork ships.
- cmd/callhome.go is deleted and internal/config/subnet/ is reduced to parsing
its old keys and reporting that the integration is disabled. config-current.go
warns instead of failing when callhome or SUBNET settings are present, so an
upgraded node with those keys still starts.
- internal/config/errors.go replaces the MinIO Slack and support entry points
with Silo documentation and issue links. Error codes and programmatic fields
are unchanged.
Verified: the compatibility baseline is unchanged except for the deliberate
removal of the /api/health/upload SUBNET route; go build, go vet and the full
cmd/ and internal/ unit suites pass; a locally built binary starts, serves
S3/Admin/metrics on the unchanged /minio/* routes, answers with Server: Silo,
and falls back to a pre-existing ~/.minio with the expected notice.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`go run golang.org/x/tools/gopls/internal/analysis/modernize/cmd/modernize@latest -fix -test ./...` executed.
`go generate ./...` ran afterwards to keep generated.
multi-object deletion may or may not compete with locks
granted for other callers, causing concurrent operations
to succeed on each other.
A continuation of the PR https://github.com/minio/minio/pull/20356
Sometimes, we need historical information in .healing.bin, such as the
number of expired objects that the healing avoids to heal and that can
create drive usage disparency in the same erasure set. For that reason,
this commit will not remove .healing.bin anymore and it will have a new
field called Finished so we know healing is finished in that drive.
move away from map[string]interface{} to map[string]string
to simplify the audit, and also provide concise information.
avoids large allocations under load(), reduces the amount
of audit information generated, as the current implementation
was a bit free-form. instead all datastructures must be
flattened.
allow multipart uploads expiration to be dyamic
It would seem like the new values will take effect
only after a restart for changes in multipart_expiration.
This PR fixes this by making it dynamic as it should have
been.
When a drive is in a failed state when a single node multiple drives
deployment is started, a replacement of a fresh disk will not be
properly healed unless the user restarts the node.
Fix this by always adding the new fresh disk to globalLocalDrivesMap. Also
remove globalLocalDrives for simplification, a map to store local node
drives can still be used since the order of local drives of a node is
not defined.
This change uses the updated ldap library in minio/pkg (bumped
up to v3). A new config parameter is added for LDAP configuration to
specify extra user attributes to load from the LDAP server and to store
them as additional claims for the user.
A test is added in sts_handlers.go that shows how to access the LDAP
attributes as a claim.
This is in preparation for adding SSH pubkey authentication to MinIO's SFTP
integration.
since mid 2018 we do not have any deployments
without deployment-id, it is time to put this
code to rest, this PR removes this old code as
its no longer valuable.
on setups with 1000's of drives these are all
quite expensive operations.
the disk location never changes in the lifetime of a
MinIO cluster, even if it did validate this close to the
disk instead at the higher layer.
Return appropriate errors indicating an invalid drive, so
that the drive is not recognized as part of a valid
drive.
Create new code paths for multiple subsystems in the code. This will
make maintaing this easier later.
Also introduce bugLogIf() for errors that should not happen in the first
place.
the PR in #16541 was incorrect and hand wrong assumptions
about the overall setup, revert this since this expectation
to have offline servers is wrong and we can end up with a
bigger chicken and egg problem.
This reverts commit 5996c8c4d5.
Bonus:
- preserve disk in globalLocalDrives properly upon connectDisks()
- do not return 'nil' from newXLStorage(), getting it ready for
the next set of changes for 'format.json' loading.
globalLocalDrives seem to be not updated during the
HealFormat() leads to a requirement where the server
needs to be restarted for the healing to continue.
Each Put, List, Multipart operations heavily rely on making
GetBucketInfo() call to verify if bucket exists or not on
a regular basis. This has a large performance cost when there
are tons of servers involved.
We did optimize this part by vectorizing the bucket calls,
however its not enough, beyond 100 nodes and this becomes
fairly visible in terms of performance.
- healing must not set the write xattr
because that is the job of active healing
to update. what we need to preserve is
permanent deletes.
- remove older env for drive monitoring and
enable it accordingly, as a global value.
- Move RenameFile to websockets
- Move ReadAll that is primarily is used
for reading 'format.json' to to websockets
- Optimize DiskInfo calls, and provide a way
to make a NoOp DiskInfo call.
reference format is constant for any lifetime of
a minio cluster, we do not have to ever replace
it during HealFormat() as it will never change.
additionally we should simply reject reference
formats that we do not understand early on.
- HealFormat() was leaking healthcheck goroutines for
disks, we are only interested in enabling healthcheck
for the newly formatted disk, not for existing disks.
- When disk is a root-disk a random disk monitor was
leaking while we ignored the drive.
- When loading the disk for each erasure set, we were
leaking goroutines for the prepare-storage.go disks
which were replaced via the globalLocalDrives slice
- avoid disk monitoring utilizing health tokens that
would cause exhaustion in the tokens, prematurely
which were meant for incoming I/O. This is ensured
by avoiding writing O_DIRECT aligned buffer instead
write 2048 worth of content only as O_DSYNC, which is
sufficient.
NOTE: This feature is not retro-active; it will not cater to previous transactions
on existing setups.
To enable this feature, please set ` _MINIO_DRIVE_QUORUM=on` environment
variable as part of systemd service or k8s configmap.
Once this has been enabled, you need to also set `list_quorum`.
```
~ mc admin config set alias/ api list_quorum=auto`
```
A new debugging tool is available to check for any missing counters.
This PR also increases per node bpool memory from 1024 entries
to 2048 entries; along with that, it also moves the byte pool
centrally instead of being per pool.
historically, we have always kept storage-rest-server
and a local storage API separate without much trouble,
since they both can independently operate due to no
special state() between them.
however, over some time, we have added state()
such as
- drive monitoring threads now there will be "2" of
them per drive instead of just 1.
- concurrent tokens available per drive are now twice
instead of just single shared, allowing unexpectedly
high amount of I/O to go through.
- applying serialization by using walkMutexes can now
be adequately honored for both remote callers and local
callers.
`monitorAndConnectEndpoints` will continue to attempt to reconnect offline disks.
Since disks were never closed, a `MarkOffline` would continue to try to check these disks forever.
Close previous disks.
* optimize deletePrefix, use direct set location via object name
instead of fanning out the calls for an object force delete
we can assume the set location and not do fan-out calls
* Apply suggestions from code review
Co-authored-by: Krishnan Parthasarathi <krisis@users.noreply.github.com>
---------
Co-authored-by: Krishnan Parthasarathi <krisis@users.noreply.github.com>
health checks were missing for drives replaced since
- HealFormat() would replace the drives without a health check
- disconnected drives when they reconnect via connectEndpoint()
the loop also loses health checks for local disks and merges
these into a single code.
- other than this separate cleanUp, health check variables to avoid
overloading them with similar requirements.
- also ensure that we compete via context selector for disk monitoring
such that the canceled disks don't linger around longer waiting for
the ticker to trigger.
- allow disabling active monitoring.
Feng Ruohang