Commit Graph

69 Commits

Author SHA1 Message Date
Feng Ruohang 77bdc4c0cd feat(server): present Silo identity and close the inherited upstream services
Two coupled changes that must land together, because the same files carry both:
the server now identifies itself as Silo, and every path that would have called
home to a MinIO-operated service is closed.

Product identity
- build-constants.go: store name, UA name and startup banner become Silo. The
  Go identifiers (MinioStoreName, MinioBannerName, ...) keep their names on
  purpose - renaming exported symbols would churn the compatibility surface for
  a cosmetic gain, and the rebrand guard freezes that surface.
- main.go, server-startup-msg.go, ftp-server.go and the user-visible log, help
  and error strings across cmd/ and internal/ switch to Silo. Original MinIO
  copyright, LICENSE, NOTICE and CREDITS are untouched; --version now prints
  the upstream copyright, the pgsty modification notice, and the trademark
  policy's approved "based on MinIO technology" attribution.
- api-headers.go: the HTTP Server header becomes "Silo". This is the one
  externally observable identity change, so TestCommonHeadersUseSiloProductName
  pins it - probes that sniff for "MinIO" must move to capability detection.
- Prometheus metric HELP strings keep their MinIO wording. They are part of the
  metrics contract the guard protects, not product copy.

Configuration directory
- config-dir.go: new installs use ~/.silo. If only ~/.minio exists it is still
  read, with a one-time notice and no files moved. If both exist ~/.silo wins
  and an ambiguity warning is emitted; an explicit --config-dir always wins.
  Covered by TestSelectDefaultConfigDir. The internal .minio.sys layout is
  never renamed - this rule applies to the user config directory only.

Upstream service lockdown
- globalInplaceUpdateDisabled is now true at initialization rather than being
  set from MINIO_UPDATE. common-main.go still parses MINIO_UPDATE so upgrading
  nodes do not fail on an unknown key, but warns that the value is ignored;
  there is no way to re-enable the updater. TestInplaceUpdateCannotBeEnabled
  guards that. Without this, an admin with mc could have overwritten
  /usr/bin/silo with an upstream MinIO binary.
- verifyBinary and commitBinary refuse early; the ServerUpdate v1/v2 admin
  routes and the peer-rest update endpoints stay registered and keep returning
  the existing programmatic error, so clients see a stable failure rather than
  a 404.
- MinioReleaseBaseURL and defaultMinisignPubkey are emptied: no dl.min.io
  download root, and upstream's minisign key is no longer a trust root for
  anything this fork ships.
- cmd/callhome.go is deleted and internal/config/subnet/ is reduced to parsing
  its old keys and reporting that the integration is disabled. config-current.go
  warns instead of failing when callhome or SUBNET settings are present, so an
  upgraded node with those keys still starts.
- internal/config/errors.go replaces the MinIO Slack and support entry points
  with Silo documentation and issue links. Error codes and programmatic fields
  are unchanged.

Verified: the compatibility baseline is unchanged except for the deliberate
removal of the /api/health/upload SUBNET route; go build, go vet and the full
cmd/ and internal/ unit suites pass; a locally built binary starts, serves
S3/Admin/metrics on the unchanged /minio/* routes, answers with Server: Silo,
and falls back to a pre-existing ~/.minio with the expected notice.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 08:46:58 +08:00
Feng Ruohang 0c14d81510 fix(notify): quote libpq connection parameters
The discrete PostgreSQL connection path concatenated raw values into a libpq keyword/value string. Whitespace, quotes, or backslashes could split a value into additional parameters or make an otherwise valid configuration fail to parse; the path also used the unsupported keyword username instead of user.

Render every generated value as a single-quoted libpq parameter, escape quotes and backslashes, and use the correct user key. Keep the existing connection_string form untouched. The earlier attempt to register migrated PostgreSQL and MySQL fields is deliberately absent because those key names collide with the legacy connection-string tokenizer.

Focused tests cover ordinary values, whitespace, quotes, backslashes, and parameter-shaped input.

Co-authored-by: ChatGPT <noreply@openai.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-08-04 23:00:31 +08:00
Feng Ruohang 162ded3438 fix: register NATS/AMQP notify config keys read by parsers
GetNotifyNATS reads user_credentials, nkey_seed and tls_handshake_first
and GetNotifyAMQP reads immediate, but none of them were registered in
DefaultNATSKVS/DefaultAMQPKVS or the help schema, so CheckValidKeys
rejected any enable=on target carrying them. Worse, the legacy config
migration wrote exactly these keys - including the env var name
MINIO_NOTIFY_NATS_USER_CREDENTIALS used as a config key, because the
NATSUserCredentials constant doubled as both - so a migrated NATS config
failed validation on every load, and the FetchEnabledTargets fail-fast
then silently disabled all bucket notification targets.

- Register user_credentials/nkey_seed/tls_handshake_first (NATS) and
  immediate (AMQP) in the default KVS and help schema; split
  NATSUserCredentials into a real config key plus EnvNATSUserCredentials
  (all env var names byte-stable)
- Fix legacy migration: SetNotifyNATS writes the proper key;
  SetNotifyAMQP no longer writes cfg.Immediate under the internal key
  and now carries both immediate and internal
- Tolerate the legacy MINIO_NOTIFY_NATS_USER_CREDENTIALS key written by
  pre-fix migrations (NATS-scoped, load path only) with fallback read;
  env > user_credentials > legacy key
- Print key names only, never values, in the invalid-keys error of both
  CheckValidKeys forms; rejected values can carry credentials
- Add an AST-based audit test asserting parser reads, migration writes
  and help entries stay within the registered key set for all ten notify
  subsystems, with floor assertions so collector drift fails loudly
- Document (unchanged) FetchEnabledTargets fail-fast and pin it with a
  characterization test

Known same-class gap left in place and pinned by the audit's allowlist:
SetNotifyPostgres/SetNotifyMySQL write five unregistered DSN-era keys;
tracked for a follow-up issue.

Closes #39

Co-authored-by: ChatGPT <noreply@openai.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-08-04 23:00:31 +08:00
Klaus Post b8631cf531 Use new gofumpt (#21613)
Update tinylib. Should fix CI.

`gofumpt -w .&&go generate ./...`
2025-09-28 13:59:21 -07:00
Klaus Post f0b91e5504 Run modernize (#21546)
`go run golang.org/x/tools/gopls/internal/analysis/modernize/cmd/modernize@latest -fix -test ./...` executed.

`go generate ./...` ran afterwards to keep generated.
2025-08-28 19:39:48 -07:00
jiuker 86d9d9b55e fix: use amqp.ParseURL to parse amqp url (#21528) 2025-08-20 21:25:07 -07:00
Matt Lloyd 0e017ab071 feat: support nats nkey seed auth (#21231) 2025-04-26 21:30:57 -07:00
Matt Lloyd 0d7408fc99 feat: support nats tls handshake first (#21008) 2025-04-22 15:12:26 -07:00
Taran Pelkey e88d494775 Migrate golanglint-ci config to V2 (#21081) 2025-03-29 17:56:02 -07:00
Klaus Post 90f5e1e5f6 tests: Do not allow forced type asserts (#20905) 2025-02-18 08:25:55 -08:00
Praveen raj Mani 261111e728 Kafka notify: support batched commits for queue store (#20377)
The items will be saved per target batch and will
be committed to the queue store when the batch is full

Also, periodically commit the batched items to the queue store
based on configured commit_timeout; default is 30s;

Bonus: compress queue store multi writes
2024-09-06 16:06:30 -07:00
Harshavardhana 0f1e8db4c5 all 2xx status codes to be success for audit (#20394) 2024-09-06 15:53:34 -07:00
Anis Eleuch 2a75225569 kafka: _MINIO_KAFKA_DEBUG to enable sarama debug messages (#19849) 2024-06-01 08:02:59 -07:00
Aditya Manthramurthy 5f78691fcf ldap: Add user DN attributes list config param (#19758)
This change uses the updated ldap library in minio/pkg (bumped
up to v3). A new config parameter is added for LDAP configuration to
specify extra user attributes to load from the LDAP server and to store
them as additional claims for the user.

A test is added in sts_handlers.go that shows how to access the LDAP
attributes as a claim.

This is in preparation for adding SSH pubkey authentication to MinIO's SFTP
integration.
2024-05-24 16:05:23 -07:00
Ramon de Klein 701da1282a Validates PostgreSQL table name (#19602) 2024-04-24 10:51:07 -07:00
jiuker d7fb6fddf6 feat: add user specific redis auth (#19285) 2024-03-18 21:37:54 -07:00
jiuker 0aae0180fb feat: add userCredentials for nats (#19139) 2024-02-27 10:11:55 -08:00
Harshavardhana cd419a35fe simplify broker healthcheck by following kafka guidelines (#19082)
fixes #19081
2024-02-20 00:16:35 -08:00
Praveen raj Mani c905d3fe21 fix: Re-use TCP connections for Kafka dials (#18860)
Fixes #18857
2024-01-24 13:10:52 -08:00
Harshavardhana dd2542e96c add codespell action (#18818)
Original work here, #18474,  refixed and updated.
2024-01-17 23:03:17 -08:00
jiuker 34187e047d feat: support elasticsearch notification endpoint compression codec (#18562) 2023-11-30 00:25:03 -08:00
Praveen raj Mani 3369eeb920 Relax batch size limit for kafka events (#18513)
Fixes #18490
2023-11-24 09:07:38 -08:00
jiuker 41091d9472 fix: close http body for es action (#18491) 2023-11-20 22:22:10 -08:00
jiuker f56a182b71 fix: close http body when webhook send (#18487) 2023-11-20 14:40:07 -08:00
Anis Eleuch 6ef8e87492 Support case insensitive kafka SASL mechanism config values (#18398) 2023-11-08 20:04:01 -08:00
Harshavardhana 754f7a8a39 replace io.Discard usage to fix some NUMA copy() latencies (#18394)
replace io.Discard usage to fix NUMA copy() latencies

On NUMA systems copying from 8K buffer allocated via
io.Discard leads to large latency build-up for every

```
copy(new8kbuf, largebuf)
```

can in-cur upto 1ms worth of latencies on NUMA systems
due to memory sharding across NUMA nodes.
2023-11-06 14:26:08 -08:00
Praveen raj Mani c27d0583d4 Send kafka notification messages in batches when queue_dir is enabled (#18164)
Fixes #18124
2023-10-07 08:07:38 -07:00
Anis Eleuch 419e5baf16 fix: webhook notify endpoint with standard ports (#18016) 2023-09-14 20:10:44 -07:00
Aditya Manthramurthy 1c99fb106c Update to minio/pkg/v2 (#17967) 2023-09-04 12:57:37 -07:00
Harshavardhana 0d1fbef751 fix: a possible crash in event target Close() (#17948)
these are possible crashes when the configured
target is still in init() state and never finished
- however a delete config was initiated.
2023-08-30 07:27:45 -07:00
Harshavardhana adb8be069e tune-kafka targets to ensure timeout triggers on hung brokers (#17898)
hung brokers can cause slowness to the entire system
when many callers are hung, leading to large goroutine
build-up.
2023-08-22 20:26:35 -07:00
jiuker fa2a8d7209 fix: drain the req.body into io.Discard correctly (#17881) 2023-08-21 01:09:07 -07:00
Harshavardhana dbd4c2425e fix: kafka broker pings must not be greater than 1sec (#17376) 2023-06-07 11:47:00 -07:00
jiuker 443250d135 fix: for Target isActive use net.Dial instead (#17251) 2023-05-25 09:24:11 -07:00
Praveen raj Mani 57acacd5a7 Support persistent queue store for loggers (#17121) 2023-05-08 21:20:31 -07:00
Praveen raj Mani 1704abaf6b fix: store notification events immediately for persistent queues (#17112) 2023-05-02 07:53:13 -07:00
Harshavardhana 8a9b9832fd add Dial timeout for Kafka broker pings (#17044) 2023-04-17 15:45:01 -07:00
Praveen raj Mani 51f7f9aaa3 Generalize the event store using go generics (#16910) 2023-04-04 10:52:24 -07:00
Harshavardhana b21d3f9b82 event target registration failures must be returned (#16700) 2023-02-23 21:59:14 +05:30
Klaus Post 0b7989aa4b Fix Kafka initialization crash (#16523) 2023-02-02 19:21:19 +05:30
Harshavardhana be92cf5959 change dependency from amqp -> amqp091 (RabbitMQ) official (#16142) 2022-11-28 16:05:06 -08:00
Ray 66239f30ce configuring the nats target to reconnect forever (#16050) 2022-11-11 12:42:41 -08:00
Klaus Post bbc312fce6 Add notification queue metrics (#16026) 2022-11-08 16:36:47 -08:00
Harshavardhana 21251d8c22 initialize streaming events without lazy initialization (#16016) 2022-11-07 08:01:24 -08:00
Harshavardhana 4892a766a8 do not panic if webhook returns an error (#15970) 2022-10-30 16:45:53 -07:00
Anis Elleuch 86bb48792c non-blocking initialization of bucket target notifications (#15571) 2022-09-27 17:23:28 -07:00
Klaus Post ff12080ff5 Remove deprecated io/ioutil (#15707) 2022-09-19 11:05:16 -07:00
Harshavardhana 5e763b71dc use logger.LogOnce to reduce printing disconnection logs (#15408)
fixes #15334

- re-use net/url parsed value for http.Request{}
- remove gosimple, structcheck and unusued due to https://github.com/golangci/golangci-lint/issues/2649
- unwrapErrs upto leafErr to ensure that we store exactly the correct errors
2022-07-27 09:44:59 -07:00
Ray 3bdb92fcad Adding error check for jetstream connection (#15252) 2022-07-07 23:14:47 -07:00
Harshavardhana dd839bf295 add NATS JetStream support (#15201) 2022-07-06 13:29:08 -07:00