test: pin the external fixtures and run the suites against the silo binary

The test and verification scripts invoked ./minio and pulled their tooling from
upstream infrastructure with no integrity check. Every `curl | tar` of a client
or an old server binary was an unverified execution path in a script that
regularly runs as a privileged user, and several fetched a floating "latest".

Two installers replace all of it:

- install-mcli.sh resolves a pinned pgsty/mc release, downloads the archive and
  its checksum manifest, requires exactly one valid manifest entry for the
  asset, verifies it, and installs. MCLI_BIN with a mandatory MCLI_SHA256 lets
  an offline or air-gapped run supply its own binary, still checksum-checked.
- install-verified-fixture.sh takes source, expected SHA-256 and target, and
  refuses anything that does not match. Sources may be a URL or a local file.

Every script that previously downloaded mc now calls install-mcli.sh. The three
places that genuinely need an upstream artifact - the old MinIO server binary
for the LDAP IAM upgrade-import test, the 2021 mc for the three-site
replication test, and the functional-tests.sh fixture - go through
install-verified-fixture.sh with the digest recorded inline. Those dl.min.io
URLs remain on purpose: they are historical upstream artifacts needed to prove
upgrade compatibility, and they are now pinned and verified rather than
trusted.

The scripts otherwise switch to ./silo, silo.service, the silo container and
compose service names, and SILO_CONFIG_DIR. run-multi-site-minio-idp.sh is
renamed to run-multi-site-silo-idp.sh with the Makefile target following.
buildscripts/minio-upgrade.sh keeps its name and its `minio server` argv - it
exists to test the MinIO-to-Silo upgrade, so the old side must stay old - but
it is now pinned to an image digest rather than a tag, and its `docker system
prune` and `docker volume prune` calls are removed. Those ran unfiltered
against the developer's whole Docker installation; the resiliency tests had the
same problem and lose their prune and `docker ps -q` sweeps too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Feng Ruohang
2026-08-06 08:48:58 +08:00
parent bd8df51665
commit 6613c2a3cb
52 changed files with 2046 additions and 1989 deletions
+91
View File
@@ -0,0 +1,91 @@
#!/usr/bin/env bash
set -euo pipefail
if [ "$#" -ne 1 ]; then
echo "usage: $0 TARGET" >&2
exit 2
fi
target=$1
target_dir=$(dirname "${target}")
if [ ! -d "${target_dir}" ]; then
echo "target directory does not exist: ${target_dir}" >&2
exit 1
fi
sha256_file() {
if command -v sha256sum >/dev/null 2>&1; then
sha256sum "$1" | awk '{print $1}'
else
shasum -a 256 "$1" | awk '{print $1}'
fi
}
if [ -n "${MCLI_BIN:-}" ]; then
if [ ! -f "${MCLI_BIN}" ]; then
echo "MCLI_BIN is not a regular file: ${MCLI_BIN}" >&2
exit 1
fi
if ! printf '%s\n' "${MCLI_SHA256:-}" | grep -Eq '^[0-9a-fA-F]{64}$'; then
echo "MCLI_SHA256 must contain the expected SHA-256 for MCLI_BIN" >&2
exit 1
fi
actual=$(sha256_file "${MCLI_BIN}")
if [ "${actual}" != "${MCLI_SHA256,,}" ]; then
echo "MCLI_BIN checksum mismatch: expected ${MCLI_SHA256,,}, got ${actual}" >&2
exit 1
fi
install -m 0755 "${MCLI_BIN}" "${target}"
exit 0
fi
release=${MCLI_RELEASE:-RELEASE.2026-08-04T00-00-00Z}
version_hyphen=${release#RELEASE.}
package_version=$(printf '%s\n' "${version_hyphen}" | sed -E 's/^([0-9]{4})-([0-9]{2})-([0-9]{2})T([0-9]{2})-([0-9]{2})-([0-9]{2})Z$/\1\2\3\4\5\6.0.0/')
if [ "${package_version}" = "${version_hyphen}" ]; then
echo "invalid MCLI_RELEASE: ${release}" >&2
exit 1
fi
case $(uname -s) in
Linux) os=linux ;;
Darwin) os=darwin ;;
*) echo "unsupported mcli host OS: $(uname -s)" >&2; exit 1 ;;
esac
case $(uname -m) in
x86_64 | amd64) arch=amd64 ;;
aarch64 | arm64) arch=arm64 ;;
*) echo "unsupported mcli host architecture: $(uname -m)" >&2; exit 1 ;;
esac
archive="mcli_${package_version}_${os}_${arch}.tar.gz"
checksums="mcli_${package_version}_checksums.txt"
base_url="https://github.com/pgsty/mc/releases/download/${release}"
tmp_dir=$(mktemp -d "${TMPDIR:-/tmp}/silo-mcli.XXXXXX")
trap 'rm -rf "${tmp_dir}"' EXIT
curl --fail --location --retry 3 --silent --show-error \
"${base_url}/${checksums}" --output "${tmp_dir}/${checksums}"
curl --fail --location --retry 3 --silent --show-error \
"${base_url}/${archive}" --output "${tmp_dir}/${archive}"
expected=$(awk -v asset="${archive}" '
{
name=$2
sub(/^\*/, "", name)
if (name == asset && $1 ~ /^[0-9a-fA-F]{64}$/) print tolower($1)
}
' "${tmp_dir}/${checksums}")
if ! printf '%s\n' "${expected}" | grep -Eq '^[0-9a-f]{64}$'; then
echo "checksum manifest does not contain exactly one valid entry for ${archive}" >&2
exit 1
fi
actual=$(sha256_file "${tmp_dir}/${archive}")
if [ "${actual}" != "${expected}" ]; then
echo "downloaded ${archive} checksum mismatch" >&2
exit 1
fi
tar -xzf "${tmp_dir}/${archive}" -C "${tmp_dir}" mcli
install -m 0755 "${tmp_dir}/mcli" "${target}"