ci: retry module downloads and re-run jobs no runner picked up

Two causes kept turning main red without any code change:

- docker-sandbox-offline failed when proxy.golang.org dropped a module
  download mid-stream during the image build. The sandbox Dockerfile now
  retries `go mod download` up to three times.
- Hosted macOS arm64 runners are often short, and the required
  integration-test (macos-latest) job then fails with "The job was not
  acquired by Runner of type hosted", which blocks merges. A workflow_run
  follow-up re-runs such a job once, on the first attempt of a run from this
  repository. It uses the single-job re-run endpoint because the job ends as
  cancelled, and the docs do not say whether re-running failed jobs covers
  that. Jobs that failed for other reasons are left alone.
This commit is contained in:
sim
2026-11-05 23:28:46 +08:00
parent 1a35f92605
commit 45acab5fd5
2 changed files with 44 additions and 1 deletions
+42
View File
@@ -0,0 +1,42 @@
name: Rerun unacquired jobs
# Hosted macOS arm64 runners are often short, and a job that waits too long
# fails with "The job was not acquired by Runner of type hosted". That says
# nothing about the code, so re-run that job once (its conclusion can be
# "failure" or "cancelled"). Jobs that failed for any other reason are left alone.
on:
workflow_run:
workflows: [Test, Nix darwin]
types: [completed]
permissions:
actions: write
checks: read
jobs:
rerun:
# Only the first attempt, and only runs from this repository.
if: >-
github.event.workflow_run.conclusion == 'failure' &&
github.event.workflow_run.run_attempt == 1 &&
github.event.workflow_run.head_repository.full_name == github.repository
runs-on: ubuntu-latest
steps:
- name: Re-run a job no runner picked up
env:
GH_TOKEN: ${{ github.token }}
REPO: ${{ github.repository }}
RUN_ID: ${{ github.event.workflow_run.id }}
run: |
jobs=$(gh api "repos/$REPO/actions/runs/$RUN_ID/jobs" --paginate \
-q '.jobs[] | select(.conclusion == "failure" or .conclusion == "cancelled") | .id')
for job in $jobs; do
if gh api "repos/$REPO/check-runs/$job/annotations" -q '.[].message' |
grep -q 'was not acquired by Runner'; then
# One re-run per attempt: a second request fails while the first is running.
echo "Job $job in run $RUN_ID was never picked up by a runner; re-running it."
gh api -X POST "repos/$REPO/actions/jobs/$job/rerun"
exit 0
fi
done
echo "No unacquired job in run $RUN_ID; leaving it as is."
+2 -1
View File
@@ -48,7 +48,8 @@ RUN chmod +x /entrypoint.sh
WORKDIR /workspace
COPY go.mod go.sum ./
RUN go mod download
# The module proxy sometimes drops a download mid-stream; retry before failing the build.
RUN for i in 1 2 3; do go mod download && exit 0; echo "go mod download failed (attempt $i), retrying" >&2; sleep 10; done; exit 1
RUN mkdir -p /go/build-cache /go/pkg/mod && chmod -R 0777 /go/build-cache /go/pkg/mod
ENTRYPOINT ["/entrypoint.sh"]