tenseleyflow/shithub / 2a132b6

Browse files

docs/actions: add checkout canary

Authored by mfwolffe <wolffemf@dukes.jmu.edu>
SHA
2a132b667c604916a519b5d677b3fb940bf2ed00
Parents
33b66c6
Tree
3170643

8 changed files

StatusFile+-
A .shithub/workflows/checkout-canary.yml 17 0
M bench/fixtures/README.md 6 3
A bench/fixtures/actions/checkout-canary.yml 13 0
M docs/internal/actions-runner-api.md 30 21
M docs/internal/actions-schema.md 13 6
M docs/internal/runbooks/actions-runner.md 5 4
M docs/internal/runbooks/actions.md 40 8
M docs/public/user/actions.md 22 9
.shithub/workflows/checkout-canary.ymladded
@@ -0,0 +1,17 @@
1
+name: checkout canary
2
+on:
3
+  push:
4
+    branches: [trunk]
5
+  workflow_dispatch:
6
+jobs:
7
+  checkout:
8
+    runs-on: ubuntu-latest
9
+    steps:
10
+      - uses: actions/checkout@v4
11
+      - name: Verify checkout
12
+        env:
13
+          EXPECTED_SHA: ${{ shithub.sha }}
14
+        run: |
15
+          test -f go.mod
16
+          test "$(git rev-parse HEAD)" = "$EXPECTED_SHA"
17
+          bash scripts/lint-migration-versions.sh
bench/fixtures/README.mdmodified
@@ -43,6 +43,9 @@ done" land with the generators above.
4343
 
4444
 `actions/smoke-run-only.yml` is the canonical first end-to-end Actions
4545
 workflow fixture. Copy it into `.shithub/workflows/smoke.yml` in a
46
-repository with a registered `ubuntu-latest` runner. It intentionally uses only
47
-`run:` steps because the current executor rejects reserved `uses:` aliases
48
-until checkout and artifact execution are implemented.
46
+repository with a registered `ubuntu-latest` runner.
47
+
48
+`actions/checkout-canary.yml` is the next fixture once the run-only smoke is
49
+green. It verifies `actions/checkout@v4` and pins that the runner workspace is
50
+on the workflow run's exact `shithub.sha`. Artifact aliases remain reserved
51
+until artifact execution is implemented.
bench/fixtures/actions/checkout-canary.ymladded
@@ -0,0 +1,13 @@
1
+name: checkout canary
2
+on: [push, workflow_dispatch]
3
+jobs:
4
+  checkout:
5
+    runs-on: ubuntu-latest
6
+    steps:
7
+      - uses: actions/checkout@v4
8
+      - name: Verify checkout
9
+        env:
10
+          EXPECTED_SHA: ${{ shithub.sha }}
11
+        run: |
12
+          test -f go.mod
13
+          test "$(git rev-parse HEAD)" = "$EXPECTED_SHA"
docs/internal/actions-runner-api.mdmodified
@@ -28,18 +28,18 @@ the response includes a job payload and a 15-minute job JWT. That JWT
2828
 has claims:
2929
 
3030
 ```json
31
-{"sub":"runner:<id>","job_id":1,"run_id":1,"repo_id":1,"exp":0,"jti":"..."}
31
+{"sub":"runner:<id>","purpose":"api","job_id":1,"run_id":1,"repo_id":1,"exp":0,"jti":"..."}
3232
 ```
3333
 
3434
 The signing key is derived from `auth.totp_key_b64` with HKDF label
3535
 `actions-runner-jwt-v1`; the raw TOTP/secretbox key is not used
3636
 directly for JWT signing.
3737
 
38
-Job JWTs are single-use. Every job endpoint verifies the signature and
39
-expiry, checks that the path job belongs to the claimed runner/run, and
40
-then inserts `jti` into `runner_jwt_used`. A replay returns 401. To
41
-support multi-step runner flows, successful in-flight job endpoints
42
-return `next_token` and `next_token_expires_at`.
38
+API-purpose job JWTs are single-use. Every job endpoint verifies the
39
+signature and expiry, checks that the path job belongs to the claimed
40
+runner/run, and then inserts `jti` into `runner_jwt_used`. A replay
41
+returns 401. To support multi-step runner flows, successful in-flight job
42
+endpoints return `next_token` and `next_token_expires_at`.
4343
 
4444
 Consumed JWT rows are retained for 30 days after token expiry, then
4545
 pruned by the daily `workflow:cleanup` worker. This keeps the replay
@@ -47,11 +47,21 @@ gate audit trail available for recent jobs without letting the table
4747
 grow unbounded.
4848
 
4949
 `shithubd-runner` consumes the same token chain: it claims with the
50
-registration token, marks the job `running` with the first job JWT, then
51
-uses each returned `next_token` serially for log chunks, step-status
52
-updates, cancel checks, artifact upload requests, and finally the
53
-terminal job-status update. Reusing any consumed job JWT is a replay and
54
-must fail with 401.
50
+registration token, marks the job `running` with the first API-purpose job
51
+JWT, then uses each returned `next_token` serially for log chunks,
52
+step-status updates, cancel checks, artifact upload requests, and finally
53
+the terminal job-status update. Reusing any consumed API-purpose job JWT
54
+is a replay and must fail with 401.
55
+
56
+The heartbeat claim also returns `job.checkout_url` and
57
+`job.checkout_token` for `actions/checkout@v4`. The checkout token is a
58
+separate JWT with `purpose:"checkout"` and the same runner/job/run/repo
59
+scope. It is intentionally reusable while the job is `running`, because
60
+Git smart HTTP performs multiple Basic-authenticated requests during one
61
+checkout. The git HTTP handler accepts it only for `git-upload-pack`, only
62
+for the claimed repository, and only while the database still shows that
63
+the claimed runner is running the job. It is never accepted for pushes or
64
+runner API endpoints.
5565
 
5666
 ## Endpoints
5767
 
@@ -67,11 +77,11 @@ Returns 204 when no matching job is claimable. Returns 200 with
6777
 `token`, `expires_at`, and `job` when a job is claimed. Capacity is
6878
 enforced server-side by counting current `workflow_jobs.status =
6979
 'running'` rows for the runner while holding a row lock on the runner.
70
-The job payload includes resolved `secrets` and `mask_values`; repo
71
-secrets shadow org secrets with the same name. The server also stores
72
-an encrypted claim-time copy of the mask values on
73
-`workflow_job_secret_masks` so later log uploads are scrubbed against
74
-the secrets that were actually handed to the runner, even if an
80
+The job payload includes `checkout_url`, `checkout_token`, resolved
81
+`secrets`, and `mask_values`; repo secrets shadow org secrets with the
82
+same name. The server also stores an encrypted claim-time copy of the mask
83
+values on `workflow_job_secret_masks` so later log uploads are scrubbed
84
+against the secrets that were actually handed to the runner, even if an
7585
 operator rotates or deletes a secret mid-job.
7686
 
7787
 `POST /api/v1/jobs/{id}/logs`
@@ -141,11 +151,10 @@ When a runner reports `status:"cancelled"`, any still-open steps in the
141151
 job are marked cancelled too. This keeps a killed job from leaving queued
142152
 step rows that the UI would otherwise treat as live.
143153
 
144
-S41d PR2 runner execution supports containerized `run:` steps with
145
-per-step log streaming and server-side log finalization. `uses:` aliases
146
-such as `actions/checkout@v4` and artifact upload/download remain
147
-reserved for the later S41d slices that add checkout metadata and
148
-artifact transfer.
154
+Runner execution supports host-side `actions/checkout@v4` followed by
155
+containerized `run:` steps with per-step log streaming and server-side log
156
+finalization. Artifact upload/download aliases remain reserved until the
157
+artifact transfer path lands.
149158
 
150159
 `POST /api/v1/jobs/{id}/artifacts/upload`
151160
 
docs/internal/actions-schema.mdmodified
@@ -134,7 +134,7 @@ Exactly three aliases are reserved at parse time, no exceptions:
134134
 
135135
 | Alias                            | Parser status | Runner status                              |
136136
 | -------------------------------- | ------------- | ------------------------------------------ |
137
-| `actions/checkout@v4`            | accepted      | rejected until checkout support lands      |
137
+| `actions/checkout@v4`            | accepted      | executable with scoped checkout token      |
138138
 | `shithub/upload-artifact@v1`     | accepted      | rejected until artifact upload lands       |
139139
 | `shithub/download-artifact@v1`   | accepted      | rejected until artifact download lands     |
140140
 
@@ -143,11 +143,18 @@ actions) is an Error-severity diagnostic. The marketplace problem is
143143
 explicitly out of scope for v1; revisit only if a real demand exists
144144
 and we have an answer for supply-chain trust.
145145
 
146
-The current Docker executor runs `run:` steps only. It fails a reserved
147
-`uses:` alias deliberately instead of pretending checkout/artifact
148
-semantics exist. This keeps the first end-to-end smoke path honest:
149
-`run:`-only workflows are executable now, while repository checkout and
150
-artifact transfer remain explicit follow-up work.
146
+The current Docker executor runs `actions/checkout@v4` and `run:` steps.
147
+Checkout happens on the runner host before a containerized step mounts the
148
+workspace. The server issues a short-lived checkout-purpose JWT scoped to
149
+the claimed repository and running job; the smart-HTTP handler accepts it
150
+only for read-only `git-upload-pack`. Artifact transfer remains explicit
151
+follow-up work, and the artifact aliases fail deliberately until that path
152
+exists.
153
+
154
+Checkout v1 accepts only `with.fetch-depth`. The default is a depth-1 fetch
155
+of the workflow run's `head_sha`; `fetch-depth: 0` requests full history.
156
+Submodules, LFS, `path`, persisted credentials, and marketplace actions are
157
+rejected because they are not part of this dialect yet.
151158
 
152159
 ### File-size + parser caps
153160
 
docs/internal/runbooks/actions-runner.mdmodified
@@ -1,8 +1,9 @@
11
 # Actions runner smoke runbook
22
 
33
 This runbook validates the runner-facing Actions path. `shithubd-runner`
4
-now claims jobs and executes containerized `run:` steps through Docker or
5
-Podman. The curl flow below remains useful for token/replay debugging.
4
+now claims jobs, performs scoped `actions/checkout@v4`, and executes
5
+containerized `run:` steps through Docker or Podman. The curl flow below
6
+remains useful for token/replay debugging.
67
 
78
 For host provisioning and the systemd/Ansible path, see
89
 [runner-deploy.md](./runner-deploy.md).
@@ -15,8 +16,8 @@ Prereqs:
1516
 - Docker or Podman is installed on the runner host.
1617
 - A repo has a workflow under `.shithub/workflows/*.yml` with
1718
   `runs-on: ubuntu-latest`, and a push/dispatch has enqueued a run.
18
-  S41d PR1 supports `run:` steps; checkout and artifact aliases land in
19
-  the following S41d slices.
19
+  Checkout and `run:` steps are executable; artifact aliases remain
20
+  reserved until artifact transfer lands.
2021
 
2122
 `runs-on` is a runner-label selector, not a hard-coded image name.
2223
 A workflow that says `runs-on: ubuntu-latest` can be claimed by any
docs/internal/runbooks/actions.mdmodified
@@ -19,14 +19,17 @@ workflow_runs + workflow_jobs + workflow_steps + check_runs
1919
 registered runner heartbeat claims a matching queued job
2020
         |
2121
         v
22
-containerized run: steps -> log chunks -> step/job status -> run rollup
22
+actions/checkout@v4 -> containerized run: steps
23
+        |
24
+        v
25
+log chunks -> step/job status -> run rollup
2326
 ```
2427
 
25
-The v1 executor supports containerized `run:` steps. The parser reserves
26
-`actions/checkout@v4`, `shithub/upload-artifact@v1`, and
27
-`shithub/download-artifact@v1`, but the Docker runner rejects `uses:` steps
28
-until checkout metadata and artifact transfer are wired end to end. Do not use
29
-`actions/checkout@v4` in production smoke workflows yet.
28
+The v1 executor supports host-side `actions/checkout@v4` plus containerized
29
+`run:` steps. The checkout token is short-lived, repository-scoped, tied to a
30
+running job, and accepted only for read-only smart-HTTP fetches.
31
+`shithub/upload-artifact@v1` and `shithub/download-artifact@v1` are still
32
+reserved aliases and fail until artifact transfer is wired end to end.
3033
 
3134
 ## First smoke
3235
 
@@ -67,6 +70,31 @@ jobs:
6770
 
6871
 Repeat with `exit 1`; the check should complete with `failure`.
6972
 
73
+## Checkout smoke
74
+
75
+After the run-only smoke passes, verify repository checkout with:
76
+
77
+```yaml
78
+name: checkout smoke
79
+on: [push, workflow_dispatch]
80
+jobs:
81
+  checkout:
82
+    runs-on: ubuntu-latest
83
+    steps:
84
+      - uses: actions/checkout@v4
85
+      - run: test -f README.md
86
+      - run: test "$(git rev-parse HEAD)" = "${{ shithub.sha }}"
87
+```
88
+
89
+Expected result:
90
+
91
+- The claim response contains `checkout_url` and `checkout_token`.
92
+- Git smart HTTP sees the checkout token as Basic auth and permits
93
+  `git-upload-pack` for the claimed repo.
94
+- `git-receive-pack` rejects the same credential with 403.
95
+- The job workspace contains the exact `shithub.sha` commit before the first
96
+  `run:` step starts.
97
+
7098
 ## Live log tail
7199
 
72100
 Step log pages open an SSE stream at:
@@ -168,8 +196,12 @@ active container, and reports terminal `cancelled`.
168196
   capacity, then inspect runner journal output and heartbeat metrics.
169197
 - **Step logs buffer:** verify the Caddy route above and confirm the SSE route
170198
   is still mounted outside compression and short timeouts.
171
-- **`uses:` step fails:** expected for now. Replace with a `run:` step until
172
-  checkout/artifact support lands.
199
+- **`actions/checkout@v4` fails:** confirm the job is still running, the repo
200
+  URL in the runner claim points at this shithub instance, and the runner host
201
+  can reach smart HTTP. The checkout token is not valid after the job leaves
202
+  `running`.
203
+- **Artifact `uses:` step fails:** expected for now. Replace with a `run:`
204
+  step until artifact support lands.
173205
 - **Secrets appear masked inconsistently:** check
174206
   `shithub_actions_log_scrub_replacements_total{location="server"}` and confirm
175207
   the job was claimed after the secret was created or rotated. Mask snapshots
docs/public/user/actions.mdmodified
@@ -27,6 +27,7 @@ a check run on matching pull requests.
2727
 ## What works today
2828
 
2929
 - `push`, `pull_request`, `schedule`, and `workflow_dispatch` triggers
30
+- `actions/checkout@v4` for repository checkout
3031
 - `run:` steps executed in the operator-configured runner image
3132
 - `runs-on:` label matching against registered runners
3233
 - workflow, job, and step `env:`
@@ -41,16 +42,27 @@ runner uses. On shithub.sh, use the labels published by the instance operator.
4142
 
4243
 ## Current limit
4344
 
44
-Use `run:` steps for now. The parser accepts these reserved aliases:
45
+The runner executes `actions/checkout@v4` and `run:` steps. Checkout accepts
46
+the default shallow fetch and `with.fetch-depth`; use `fetch-depth: 0` when a
47
+workflow needs full history:
48
+
49
+```yaml
50
+steps:
51
+  - uses: actions/checkout@v4
52
+    with:
53
+      fetch-depth: "0"
54
+  - run: git describe --tags --always
55
+```
56
+
57
+The parser also accepts these artifact aliases:
4558
 
46
-- `actions/checkout@v4`
4759
 - `shithub/upload-artifact@v1`
4860
 - `shithub/download-artifact@v1`
4961
 
50
-The runner does not execute them yet. A workflow containing those `uses:` steps
51
-will fail until checkout and artifact execution land. If you need repository
52
-files in a smoke workflow today, keep the command self-contained or fetch what
53
-you need explicitly inside a `run:` step.
62
+The runner does not execute artifact aliases yet. A workflow containing those
63
+artifact `uses:` steps will fail until artifact execution lands. Checkout
64
+inputs such as `path`, submodules, LFS, and persisted credentials are not
65
+implemented yet.
5466
 
5567
 ## Expressions
5668
 
@@ -84,9 +96,10 @@ Repo-scoped values shadow organization-scoped values with the same name.
8496
 Most simple CI files need three edits:
8597
 
8698
 1. Move the workflow file from `.github/workflows/` to `.shithub/workflows/`.
87
-2. Replace `uses:` actions with equivalent `run:` commands.
99
+2. Keep `actions/checkout@v4`, but replace marketplace and artifact `uses:`
100
+   actions with equivalent `run:` commands for now.
88101
 3. Confirm `runs-on:` matches a label registered by your shithub operator.
89102
 
90103
 Marketplace actions, Docker actions, composite actions, hosted runner images,
91
-matrix expansion, service containers, and built-in checkout are not part of the
92
-current v1 runner.
104
+matrix expansion, service containers, submodules, LFS, and artifact transfer
105
+are not part of the current v1 runner.