Commit Graph

227 Commits

Author SHA1 Message Date
Feng Ruohang 2ca4971d91 build: stop stamping the build machine's GOPATH/GOROOT into the binary
gen-ldflags injected -X cmd.GOPATH / cmd.GOROOT from the builder's environment,
baking absolute paths like /Users/<user>/go into every released binary. That
defeats -trimpath and makes the build unreproducible: a third party rebuilding
the same tag gets different bytes and cannot verify checksums.txt.

The values only seed logger.Init's source-path trim list, and under -trimpath
the binary's paths are already relative, so there is no build-machine prefix
left to trim - the trim list also still gets runtime.GOROOT() and
build.Default.GOPATH at run time. Dropping the two stamps changes no observable
logging behaviour; cmd.GOPATH/GOROOT keep the empty defaults a plain go build
leaves.

Verified: gen-ldflags output no longer contains cmd.GOPATH/GOROOT; a
-trimpath release build has zero occurrences of the builder path (was 1);
Version, ReleaseTag and CommitID stamps are intact.

Co-authored-by: Claude <noreply@anthropic.com>
2026-08-05 00:59:06 +08:00
Feng Ruohang ca674a6967 build: cross-compile only the targets we publish
The crosscompile gate built 15 OS/arch combinations, ten of which we never
ship (ppc64le, mips64, s390x, mips, riscv64, 386, arm, freebsd, netbsd,
openbsd). On a cold CI cache each target costs ~80s, so the full set ran ~21
min and tripped the job's 20-min timeout, cancelling the only CI run for the
release HEAD. Trim the list to the exact goos/goarch matrix the release
actually produces (see .github/goreleaser.yml): linux, darwin, windows on
amd64 and arm64.

This also closes a coverage gap: windows/arm64 is published but was not being
compile-checked. The trimmed set builds all six in well under the timeout.

Note: netbsd is no longer compile-checked here, so the go-systemd v22.6.0 pin
(kept because v22.7.0 does not build on netbsd) loses its CI guard. The pin is
retained deliberately - it is harmless on the platforms we ship and upgrading
go-systemd is a separate decision - but a netbsd regression in a dependency
would now surface only if that build is exercised out of band.

Co-authored-by: Claude <noreply@anthropic.com>
2026-08-05 00:55:31 +08:00
Feng Ruohang cf7df097b2 ci: verify release provenance, names, checksums, and payloads
Refuse dirty release checkouts and verify that every GoReleaser binary records the tagged revision with vcs.modified=false before packaging or publication.

Exercise the complete nFPM output in the release test pipeline: assert the six public names, validate every checksum and package identity field, and prove that RPM, DEB, and APK payloads contain the exact source binary and systemd unit. Validate release scripts and make their identity expectations the single source of truth.

Co-authored-by: ChatGPT <noreply@openai.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-08-04 23:02:16 +08:00
Feng Ruohang 10c7670b80 build: package releases with nFPM under the PGSTY identity
Replace minio/pkger's hard-coded upstream identity with an in-tree nFPM configuration. Packages now name PGSTY as vendor and maintainer, use the SILO homepage and SPDX license, and preserve the established package names, versions, payload paths, modes, and checksum format.

Resolve both the release binary and systemd unit independently of the caller's working directory. The release and test workflows share the same package script, while the signing script consumes the same final metadata contract.

Co-authored-by: ChatGPT <noreply@openai.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-08-04 23:02:16 +08:00
Feng Ruohang 9c799f42d5 build: stage draft releases for local RPM signing
GitHub Actions cannot hold the Pigsty RPM signing key, so the release workflow must stop before publication. Create releases as append-only drafts, refuse to overwrite existing assets, and make publication an explicit action after local signing and review.

Add a maintainer-side signing command that downloads the two RPMs, verifies checksums and package identity, signs them with the Pigsty key, regenerates checksums, and only replaces draft assets when --upload is explicitly requested. Published releases are never modified.

Co-authored-by: ChatGPT <noreply@openai.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-08-04 23:02:16 +08:00
ffgan e2245a0b12 allow cross-compiling support for RISC-V 64 (#21348)
this is minor PR that supports building on RISC-V 64,
this PR is for compilation only. There is no guarantee 
that code is tested and will work in production.
2025-06-08 09:12:05 -07:00
Harshavardhana 0379d6a37f fix: permissions for docker-compose 2025-04-21 09:24:31 -07:00
Harshavardhana e2ed696619 fix: docker-compose link since latest release 2025-04-20 10:05:30 -07:00
Anis Eleuch b8dab7b1a9 Set http server read/write timeout from --idle-timeout (#228) (#20715)
Golang http.Server will call SetReadDeadline overwriting the previous
deadline configuration set after a new connection Accept in the custom
listener code. Therefore, --idle-timeout was not correctly respected.

Make http.Server read/write timeout similar to --idle-timeout.
2024-12-02 18:51:17 +05:30
Harshavardhana 8ce101c174 fix: LDAP service port number in tests 2024-11-11 07:01:29 -08:00
Krishnan Parthasarathi a0f9e9f661 readParts: Return error when quorum unavailable (#20389)
readParts requires that both part.N and part.N.meta files be present.
This change addresses an issue with how an error to return to the upper
layers was picked from most drives where a UploadPart operation 
had failed.
2024-09-06 03:51:23 -07:00
Harshavardhana 7fcb428622 do not print unexpected logs (#20083) 2024-07-12 13:51:54 -07:00
Harshavardhana be84a4fd68 do not proxy invalid object names (#20031) 2024-07-02 14:28:55 -07:00
Klaus Post 2040559f71 Fix SkipReader performance with small initial read (#20030)
If `SkipReader` is called with a small initial buffer it may be doing a huge number if Reads to skip the requested number of bytes. If a small buffer is provided grab a 32K buffer and use that.

Fixes slow execution of `testAPIGetObjectWithMPHandler`.

Bonuses:

* Use `-short` with `-race` test.
* Do all suite test types with `-short`.
* Enable compressed+encrypted in `testAPIGetObjectWithMPHandler`.
* Disable big file tests in `testAPIGetObjectWithMPHandler` when using `-short`.
2024-07-02 08:13:05 -07:00
Anis Eleuch e5335450a4 test: Healing test to avoid infinite waiting for servers to be up (#19954)
tests: Healing test to avoid infinite waiting for servers to be up

Quit after 15 minutes and print server logs instead
2024-06-19 09:00:38 -07:00
Harshavardhana ba9f0f2480 fix: attempt to fix CI/CD upgrade tests with docker-compose (#19926) 2024-06-12 22:08:11 -07:00
Anis Eleuch 1277ad69a6 heal: Remove .healing.bin when all ES drives are healing (#19846)
In the very rare case when all drives in a erasure set need to be healed,
remove .healing.bin from all drives, otherwise it will be stuck in a
loop

Also, fix a unit test that fails sometimes due to wrong test.
2024-05-31 07:48:50 -07:00
Harshavardhana e0fe7cc391 fix: information disclosure bug in preconditions GET (#19810)
precondition check was being honored before, validating
if anonymous access is allowed on the metadata of an
object, leading to metadata disclosure of the following
headers.

```
Last-Modified
Etag
x-amz-version-id
Expires:
Cache-Control:
```

although the information presented is minimal in nature,
and of opaque nature. It still simply discloses that an
object by a specific name exists or not without even having
enough permissions.
2024-05-27 12:17:46 -07:00
Harshavardhana 443c93c634 compute time spent in ILM properly (#19806) 2024-05-24 12:28:51 -07:00
Shubhendu 7c7650b7c3 Add sufficient deadlines and countermeasures to handle hung node scenario (#19688)
Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
Signed-off-by: Harshavardhana <harsha@minio.io>
2024-05-22 16:07:14 -07:00
Poorna e947a844c9 Fix test scripts to use mc ready (#19768) 2024-05-18 11:19:01 -07:00
Poorna 4e2d39293a Fix build script to wait for server to come up (#19767) 2024-05-17 14:43:59 -07:00
Anis Eleuch 9a3c992d7a heal: Fix regression in healing a new fresh drive (#19615) 2024-04-25 14:55:41 -07:00
Aditya Manthramurthy 3212d0c8cd fix: IAM import for LDAP should replace mappings (#19607)
Existing IAM import logic for LDAP creates new mappings when the
normalized form of the mapping key differs from the existing mapping key
in storage. This change effectively replaces the existing mapping key by
first deleting it and then recreating with the normalized form of the
mapping key.

For e.g. if an older deployment had a policy mapped to a user DN -

`UID=alice1,OU=people,OU=hwengg,DC=min,DC=io`

instead of adding a mapping for the normalized form -

`uid=alice1,ou=people,ou=hwengg,dc=min,dc=io`

we should replace the existing mapping.

This ensures that duplicates mappings won't remain after the import.

Some additional cleanup cases are also covered. If there are multiple
mappings for the name normalized key such as:

`UID=alice1,OU=people,OU=hwengg,DC=min,DC=io`
`uid=alice1,ou=people,ou=hwengg,DC=min,DC=io`
`uid=alice1,ou=people,ou=hwengg,dc=min,dc=io`

we check if the list of policies mapped to all these keys are exactly
the same, and if so remove all of them and create a single mapping with
the normalized key. However, if the policies mapped to such keys differ,
the import operation returns an error as the server cannot automatically
pick the "right" list of policies to map.
2024-04-25 08:49:53 -07:00
Harshavardhana 9693c382a8 make renameData() more defensive during overwrites (#19548)
instead upon any error in renameData(), we still
preserve the existing dataDir in some form for
recoverability in strange situations such as out
of disk space type errors.

Bonus: avoid running list and heal() instead allow
versions disparity to return the actual versions,
uuid to heal. Currently limit this to 100 versions
and lesser disparate objects.

an undo now reverts back the xl.meta from xl.meta.bkp
during overwrites on such flaky setups.

Bonus: Save N depth syscalls via skipping the parents
upon overwrites and versioned updates.

Flaky setup examples are stretch clusters with regular
packet drops etc, we need to add some defensive code
around to avoid dangling objects.
2024-04-23 10:15:52 -07:00
Harshavardhana 96d226c0b1 remove frivolous log about abort-multipart failure in replication (#19413) 2024-04-05 04:39:55 -07:00
Harshavardhana 9a012a53ef initialize the disk healer early on (#19143)
This PR fixes a bug that perhaps has been long introduced,
with no visible workarounds. In any deployment, if an entire
erasure set is deleted, there is no way the cluster recovers.
2024-02-27 23:02:14 -08:00
Harshavardhana 1dd8ef09a6 remove unnecessary 'recreate' code (#19136) 2024-02-27 01:47:58 -08:00
Harshavardhana b6e98aed01 fix: found races in accessing globalLocalDrives (#19069)
make a copy before accessing globalLocalDrives

Bonus: update console v0.46.0

Signed-off-by: Harshavardhana <harsha@minio.io>
2024-02-16 17:15:57 -08:00
Harshavardhana 630963fa6b protect tracker copy properly to avoid race (#18984)
```
WARNING: DATA RACE
Write at 0x00c000aac1e0 by goroutine 1133:
  github.com/minio/minio/cmd.(*healingTracker).updateProgress()
      github.com/minio/minio/cmd/background-newdisks-heal-ops.go:183 +0x117
  github.com/minio/minio/cmd.(*erasureObjects).healErasureSet.func5()
      github.com/minio/minio/cmd/global-heal.go:292 +0x1d3

Previous read at 0x00c000aac1e0 by goroutine 1003:
  github.com/minio/minio/cmd.(*allHealState).updateHealStatus()
      github.com/minio/minio/cmd/admin-heal-ops.go:136 +0xcb
  github.com/minio/minio/cmd.(*healingTracker).save()
      github.com/minio/minio/cmd/background-newdisks-heal-ops.go:223 +0x424
```
2024-02-06 08:56:59 -08:00
Harshavardhana 708cebe7f0 add necessary protection err, fileInfo slice reads and writes (#18854)
protection was in place. However, it covered only some
areas, so we re-arranged the code to ensure we could hold
locks properly.

Along with this, remove the DataShardFix code altogether,
in deployments with many drive replacements, this can affect
and lead to quorum loss.
2024-01-24 01:08:23 -08:00
Daniel Jakots 331208bec1 Enable cross compile for openbsd/amd64 (#18666) 2023-12-18 17:08:22 -08:00
Harshavardhana 3c2e1a87e2 fix: support dropping privileges with arbitrary users (#18386)
fixes #18380
2023-11-03 14:18:18 -07:00
Harshavardhana e1e33077e8 fix: tests and resync replication status (#18244) 2023-10-13 17:03:34 -07:00
Harshavardhana fa6d082bfd reduce all major allocations in replication path (#18032)
- remove targetClient for passing around via replicationObjectInfo{}
- remove cloing to object info unnecessarily
- remove objectInfo from replicationObjectInfo{} (only require necessary fields)
2023-09-16 02:28:06 -07:00
Harshavardhana 9ebd10d3f4 Revert "Include SuccessorModTime for FileInfo quorum (#17732)" (#17860)
This reverts commit bf3901342c.

This is to fix a regression caused when there are inconsistent
versions, but one version is in quorum. SuccessorModTime issue
must be fixed differently.
2023-08-16 07:51:33 -07:00
Aditya Manthramurthy 5a1612fe32 Bump up madmin-go and pkg deps (#17469) 2023-06-19 17:53:08 -07:00
Harshavardhana 22b7c8cd8a upgrade pkg and dperf to latest packages (#17448)
- dperf improvements in benchmarking read and write tests
- upgrade mimedb to use latest content-types
2023-06-17 07:31:36 -07:00
Harshavardhana 4a425cbac1 cleanup scripts and apply shfmt (#17284) 2023-05-25 22:07:25 -07:00
Harshavardhana 7ae69accc0 allow root user to be disabled via config settings (#17089) 2023-04-28 12:24:14 -07:00
吴小白 62151a751d build: support loong64 (#17027) 2023-04-13 08:10:23 -07:00
Harshavardhana 8fd6be0827 avoid mknod in GitHub actions (#16991) 2023-04-06 12:02:41 -07:00
Harshavardhana 0a17acdb34 return error if policy changes on disabled groups (#16766) 2023-03-06 10:46:24 -08:00
Aditya Manthramurthy a30cfdd88f Bump up madmin-go to v2 (#16162) 2022-12-06 13:46:50 -08:00
杨刚 8afa6fefd8 add cross-build linux/amd64 (#15949) 2022-10-26 16:05:26 -07:00
Harshavardhana ed5b67720c rename deprecated 'mc policy' -> 'mc anonymous' (#15779) 2022-10-01 11:47:48 -07:00
Harshavardhana cf49da387b enable cross compile for openbsd/amd64 (#15701) 2022-09-19 07:01:22 -07:00
Harshavardhana 2c68a19dfd upgrade all deps and update CREDITS (#15650) 2022-09-16 01:59:45 -07:00
Harshavardhana 8e997eba4a fix: trigger Heal when xl.meta needs healing during PUT (#15661)
This PR is a continuation of the previous change instead
of returning an error, instead trigger a spot heal on the
'xl.meta' and return only after the healing is complete.

This allows for future GETs on the same resource to be
consistent for any version of the object.
2022-09-07 07:25:39 -07:00
Harshavardhana 8902561f3c use new xxml for XML responses to support rare control characters (#15511)
use new xxml/XML responses to support rare control characters

fixes #15023
2022-08-23 17:04:11 -07:00