tenseleyflow/shithub / 2a132b6

Browse files

docs/actions: add checkout canary

Authored by mfwolffe <wolffemf@dukes.jmu.edu>
SHA
2a132b667c604916a519b5d677b3fb940bf2ed00
Parents
33b66c6
Tree
3170643

8 changed files

StatusFile+-
A .shithub/workflows/checkout-canary.yml 17 0
M bench/fixtures/README.md 6 3
A bench/fixtures/actions/checkout-canary.yml 13 0
M docs/internal/actions-runner-api.md 30 21
M docs/internal/actions-schema.md 13 6
M docs/internal/runbooks/actions-runner.md 5 4
M docs/internal/runbooks/actions.md 40 8
M docs/public/user/actions.md 22 9
.shithub/workflows/checkout-canary.ymladded
@@ -0,0 +1,17 @@
1
+name: checkout canary
2
+on:
3
+  push:
4
+    branches: [trunk]
5
+  workflow_dispatch:
6
+jobs:
7
+  checkout:
8
+    runs-on: ubuntu-latest
9
+    steps:
10
+      - uses: actions/checkout@v4
11
+      - name: Verify checkout
12
+        env:
13
+          EXPECTED_SHA: ${{ shithub.sha }}
14
+        run: |
15
+          test -f go.mod
16
+          test "$(git rev-parse HEAD)" = "$EXPECTED_SHA"
17
+          bash scripts/lint-migration-versions.sh
bench/fixtures/README.mdmodified
@@ -43,6 +43,9 @@ done" land with the generators above.
43
 
43
 
44
 `actions/smoke-run-only.yml` is the canonical first end-to-end Actions
44
 `actions/smoke-run-only.yml` is the canonical first end-to-end Actions
45
 workflow fixture. Copy it into `.shithub/workflows/smoke.yml` in a
45
 workflow fixture. Copy it into `.shithub/workflows/smoke.yml` in a
46
-repository with a registered `ubuntu-latest` runner. It intentionally uses only
46
+repository with a registered `ubuntu-latest` runner.
47
-`run:` steps because the current executor rejects reserved `uses:` aliases
47
+
48
-until checkout and artifact execution are implemented.
48
+`actions/checkout-canary.yml` is the next fixture once the run-only smoke is
49
+green. It verifies `actions/checkout@v4` and pins that the runner workspace is
50
+on the workflow run's exact `shithub.sha`. Artifact aliases remain reserved
51
+until artifact execution is implemented.
bench/fixtures/actions/checkout-canary.ymladded
@@ -0,0 +1,13 @@
1
+name: checkout canary
2
+on: [push, workflow_dispatch]
3
+jobs:
4
+  checkout:
5
+    runs-on: ubuntu-latest
6
+    steps:
7
+      - uses: actions/checkout@v4
8
+      - name: Verify checkout
9
+        env:
10
+          EXPECTED_SHA: ${{ shithub.sha }}
11
+        run: |
12
+          test -f go.mod
13
+          test "$(git rev-parse HEAD)" = "$EXPECTED_SHA"
docs/internal/actions-runner-api.mdmodified
@@ -28,18 +28,18 @@ the response includes a job payload and a 15-minute job JWT. That JWT
28
 has claims:
28
 has claims:
29
 
29
 
30
 ```json
30
 ```json
31
-{"sub":"runner:<id>","job_id":1,"run_id":1,"repo_id":1,"exp":0,"jti":"..."}
31
+{"sub":"runner:<id>","purpose":"api","job_id":1,"run_id":1,"repo_id":1,"exp":0,"jti":"..."}
32
 ```
32
 ```
33
 
33
 
34
 The signing key is derived from `auth.totp_key_b64` with HKDF label
34
 The signing key is derived from `auth.totp_key_b64` with HKDF label
35
 `actions-runner-jwt-v1`; the raw TOTP/secretbox key is not used
35
 `actions-runner-jwt-v1`; the raw TOTP/secretbox key is not used
36
 directly for JWT signing.
36
 directly for JWT signing.
37
 
37
 
38
-Job JWTs are single-use. Every job endpoint verifies the signature and
38
+API-purpose job JWTs are single-use. Every job endpoint verifies the
39
-expiry, checks that the path job belongs to the claimed runner/run, and
39
+signature and expiry, checks that the path job belongs to the claimed
40
-then inserts `jti` into `runner_jwt_used`. A replay returns 401. To
40
+runner/run, and then inserts `jti` into `runner_jwt_used`. A replay
41
-support multi-step runner flows, successful in-flight job endpoints
41
+returns 401. To support multi-step runner flows, successful in-flight job
42
-return `next_token` and `next_token_expires_at`.
42
+endpoints return `next_token` and `next_token_expires_at`.
43
 
43
 
44
 Consumed JWT rows are retained for 30 days after token expiry, then
44
 Consumed JWT rows are retained for 30 days after token expiry, then
45
 pruned by the daily `workflow:cleanup` worker. This keeps the replay
45
 pruned by the daily `workflow:cleanup` worker. This keeps the replay
@@ -47,11 +47,21 @@ gate audit trail available for recent jobs without letting the table
47
 grow unbounded.
47
 grow unbounded.
48
 
48
 
49
 `shithubd-runner` consumes the same token chain: it claims with the
49
 `shithubd-runner` consumes the same token chain: it claims with the
50
-registration token, marks the job `running` with the first job JWT, then
50
+registration token, marks the job `running` with the first API-purpose job
51
-uses each returned `next_token` serially for log chunks, step-status
51
+JWT, then uses each returned `next_token` serially for log chunks,
52
-updates, cancel checks, artifact upload requests, and finally the
52
+step-status updates, cancel checks, artifact upload requests, and finally
53
-terminal job-status update. Reusing any consumed job JWT is a replay and
53
+the terminal job-status update. Reusing any consumed API-purpose job JWT
54
-must fail with 401.
54
+is a replay and must fail with 401.
55
+
56
+The heartbeat claim also returns `job.checkout_url` and
57
+`job.checkout_token` for `actions/checkout@v4`. The checkout token is a
58
+separate JWT with `purpose:"checkout"` and the same runner/job/run/repo
59
+scope. It is intentionally reusable while the job is `running`, because
60
+Git smart HTTP performs multiple Basic-authenticated requests during one
61
+checkout. The git HTTP handler accepts it only for `git-upload-pack`, only
62
+for the claimed repository, and only while the database still shows that
63
+the claimed runner is running the job. It is never accepted for pushes or
64
+runner API endpoints.
55
 
65
 
56
 ## Endpoints
66
 ## Endpoints
57
 
67
 
@@ -67,11 +77,11 @@ Returns 204 when no matching job is claimable. Returns 200 with
67
 `token`, `expires_at`, and `job` when a job is claimed. Capacity is
77
 `token`, `expires_at`, and `job` when a job is claimed. Capacity is
68
 enforced server-side by counting current `workflow_jobs.status =
78
 enforced server-side by counting current `workflow_jobs.status =
69
 'running'` rows for the runner while holding a row lock on the runner.
79
 'running'` rows for the runner while holding a row lock on the runner.
70
-The job payload includes resolved `secrets` and `mask_values`; repo
80
+The job payload includes `checkout_url`, `checkout_token`, resolved
71
-secrets shadow org secrets with the same name. The server also stores
81
+`secrets`, and `mask_values`; repo secrets shadow org secrets with the
72
-an encrypted claim-time copy of the mask values on
82
+same name. The server also stores an encrypted claim-time copy of the mask
73
-`workflow_job_secret_masks` so later log uploads are scrubbed against
83
+values on `workflow_job_secret_masks` so later log uploads are scrubbed
74
-the secrets that were actually handed to the runner, even if an
84
+against the secrets that were actually handed to the runner, even if an
75
 operator rotates or deletes a secret mid-job.
85
 operator rotates or deletes a secret mid-job.
76
 
86
 
77
 `POST /api/v1/jobs/{id}/logs`
87
 `POST /api/v1/jobs/{id}/logs`
@@ -141,11 +151,10 @@ When a runner reports `status:"cancelled"`, any still-open steps in the
141
 job are marked cancelled too. This keeps a killed job from leaving queued
151
 job are marked cancelled too. This keeps a killed job from leaving queued
142
 step rows that the UI would otherwise treat as live.
152
 step rows that the UI would otherwise treat as live.
143
 
153
 
144
-S41d PR2 runner execution supports containerized `run:` steps with
154
+Runner execution supports host-side `actions/checkout@v4` followed by
145
-per-step log streaming and server-side log finalization. `uses:` aliases
155
+containerized `run:` steps with per-step log streaming and server-side log
146
-such as `actions/checkout@v4` and artifact upload/download remain
156
+finalization. Artifact upload/download aliases remain reserved until the
147
-reserved for the later S41d slices that add checkout metadata and
157
+artifact transfer path lands.
148
-artifact transfer.
149
 
158
 
150
 `POST /api/v1/jobs/{id}/artifacts/upload`
159
 `POST /api/v1/jobs/{id}/artifacts/upload`
151
 
160
 
docs/internal/actions-schema.mdmodified
@@ -134,7 +134,7 @@ Exactly three aliases are reserved at parse time, no exceptions:
134
 
134
 
135
 | Alias                            | Parser status | Runner status                              |
135
 | Alias                            | Parser status | Runner status                              |
136
 | -------------------------------- | ------------- | ------------------------------------------ |
136
 | -------------------------------- | ------------- | ------------------------------------------ |
137
-| `actions/checkout@v4`            | accepted      | rejected until checkout support lands      |
137
+| `actions/checkout@v4`            | accepted      | executable with scoped checkout token      |
138
 | `shithub/upload-artifact@v1`     | accepted      | rejected until artifact upload lands       |
138
 | `shithub/upload-artifact@v1`     | accepted      | rejected until artifact upload lands       |
139
 | `shithub/download-artifact@v1`   | accepted      | rejected until artifact download lands     |
139
 | `shithub/download-artifact@v1`   | accepted      | rejected until artifact download lands     |
140
 
140
 
@@ -143,11 +143,18 @@ actions) is an Error-severity diagnostic. The marketplace problem is
143
 explicitly out of scope for v1; revisit only if a real demand exists
143
 explicitly out of scope for v1; revisit only if a real demand exists
144
 and we have an answer for supply-chain trust.
144
 and we have an answer for supply-chain trust.
145
 
145
 
146
-The current Docker executor runs `run:` steps only. It fails a reserved
146
+The current Docker executor runs `actions/checkout@v4` and `run:` steps.
147
-`uses:` alias deliberately instead of pretending checkout/artifact
147
+Checkout happens on the runner host before a containerized step mounts the
148
-semantics exist. This keeps the first end-to-end smoke path honest:
148
+workspace. The server issues a short-lived checkout-purpose JWT scoped to
149
-`run:`-only workflows are executable now, while repository checkout and
149
+the claimed repository and running job; the smart-HTTP handler accepts it
150
-artifact transfer remain explicit follow-up work.
150
+only for read-only `git-upload-pack`. Artifact transfer remains explicit
151
+follow-up work, and the artifact aliases fail deliberately until that path
152
+exists.
153
+
154
+Checkout v1 accepts only `with.fetch-depth`. The default is a depth-1 fetch
155
+of the workflow run's `head_sha`; `fetch-depth: 0` requests full history.
156
+Submodules, LFS, `path`, persisted credentials, and marketplace actions are
157
+rejected because they are not part of this dialect yet.
151
 
158
 
152
 ### File-size + parser caps
159
 ### File-size + parser caps
153
 
160
 
docs/internal/runbooks/actions-runner.mdmodified
@@ -1,8 +1,9 @@
1
 # Actions runner smoke runbook
1
 # Actions runner smoke runbook
2
 
2
 
3
 This runbook validates the runner-facing Actions path. `shithubd-runner`
3
 This runbook validates the runner-facing Actions path. `shithubd-runner`
4
-now claims jobs and executes containerized `run:` steps through Docker or
4
+now claims jobs, performs scoped `actions/checkout@v4`, and executes
5
-Podman. The curl flow below remains useful for token/replay debugging.
5
+containerized `run:` steps through Docker or Podman. The curl flow below
6
+remains useful for token/replay debugging.
6
 
7
 
7
 For host provisioning and the systemd/Ansible path, see
8
 For host provisioning and the systemd/Ansible path, see
8
 [runner-deploy.md](./runner-deploy.md).
9
 [runner-deploy.md](./runner-deploy.md).
@@ -15,8 +16,8 @@ Prereqs:
15
 - Docker or Podman is installed on the runner host.
16
 - Docker or Podman is installed on the runner host.
16
 - A repo has a workflow under `.shithub/workflows/*.yml` with
17
 - A repo has a workflow under `.shithub/workflows/*.yml` with
17
   `runs-on: ubuntu-latest`, and a push/dispatch has enqueued a run.
18
   `runs-on: ubuntu-latest`, and a push/dispatch has enqueued a run.
18
-  S41d PR1 supports `run:` steps; checkout and artifact aliases land in
19
+  Checkout and `run:` steps are executable; artifact aliases remain
19
-  the following S41d slices.
20
+  reserved until artifact transfer lands.
20
 
21
 
21
 `runs-on` is a runner-label selector, not a hard-coded image name.
22
 `runs-on` is a runner-label selector, not a hard-coded image name.
22
 A workflow that says `runs-on: ubuntu-latest` can be claimed by any
23
 A workflow that says `runs-on: ubuntu-latest` can be claimed by any
docs/internal/runbooks/actions.mdmodified
@@ -19,14 +19,17 @@ workflow_runs + workflow_jobs + workflow_steps + check_runs
19
 registered runner heartbeat claims a matching queued job
19
 registered runner heartbeat claims a matching queued job
20
         |
20
         |
21
         v
21
         v
22
-containerized run: steps -> log chunks -> step/job status -> run rollup
22
+actions/checkout@v4 -> containerized run: steps
23
+        |
24
+        v
25
+log chunks -> step/job status -> run rollup
23
 ```
26
 ```
24
 
27
 
25
-The v1 executor supports containerized `run:` steps. The parser reserves
28
+The v1 executor supports host-side `actions/checkout@v4` plus containerized
26
-`actions/checkout@v4`, `shithub/upload-artifact@v1`, and
29
+`run:` steps. The checkout token is short-lived, repository-scoped, tied to a
27
-`shithub/download-artifact@v1`, but the Docker runner rejects `uses:` steps
30
+running job, and accepted only for read-only smart-HTTP fetches.
28
-until checkout metadata and artifact transfer are wired end to end. Do not use
31
+`shithub/upload-artifact@v1` and `shithub/download-artifact@v1` are still
29
-`actions/checkout@v4` in production smoke workflows yet.
32
+reserved aliases and fail until artifact transfer is wired end to end.
30
 
33
 
31
 ## First smoke
34
 ## First smoke
32
 
35
 
@@ -67,6 +70,31 @@ jobs:
67
 
70
 
68
 Repeat with `exit 1`; the check should complete with `failure`.
71
 Repeat with `exit 1`; the check should complete with `failure`.
69
 
72
 
73
+## Checkout smoke
74
+
75
+After the run-only smoke passes, verify repository checkout with:
76
+
77
+```yaml
78
+name: checkout smoke
79
+on: [push, workflow_dispatch]
80
+jobs:
81
+  checkout:
82
+    runs-on: ubuntu-latest
83
+    steps:
84
+      - uses: actions/checkout@v4
85
+      - run: test -f README.md
86
+      - run: test "$(git rev-parse HEAD)" = "${{ shithub.sha }}"
87
+```
88
+
89
+Expected result:
90
+
91
+- The claim response contains `checkout_url` and `checkout_token`.
92
+- Git smart HTTP sees the checkout token as Basic auth and permits
93
+  `git-upload-pack` for the claimed repo.
94
+- `git-receive-pack` rejects the same credential with 403.
95
+- The job workspace contains the exact `shithub.sha` commit before the first
96
+  `run:` step starts.
97
+
70
 ## Live log tail
98
 ## Live log tail
71
 
99
 
72
 Step log pages open an SSE stream at:
100
 Step log pages open an SSE stream at:
@@ -168,8 +196,12 @@ active container, and reports terminal `cancelled`.
168
   capacity, then inspect runner journal output and heartbeat metrics.
196
   capacity, then inspect runner journal output and heartbeat metrics.
169
 - **Step logs buffer:** verify the Caddy route above and confirm the SSE route
197
 - **Step logs buffer:** verify the Caddy route above and confirm the SSE route
170
   is still mounted outside compression and short timeouts.
198
   is still mounted outside compression and short timeouts.
171
-- **`uses:` step fails:** expected for now. Replace with a `run:` step until
199
+- **`actions/checkout@v4` fails:** confirm the job is still running, the repo
172
-  checkout/artifact support lands.
200
+  URL in the runner claim points at this shithub instance, and the runner host
201
+  can reach smart HTTP. The checkout token is not valid after the job leaves
202
+  `running`.
203
+- **Artifact `uses:` step fails:** expected for now. Replace with a `run:`
204
+  step until artifact support lands.
173
 - **Secrets appear masked inconsistently:** check
205
 - **Secrets appear masked inconsistently:** check
174
   `shithub_actions_log_scrub_replacements_total{location="server"}` and confirm
206
   `shithub_actions_log_scrub_replacements_total{location="server"}` and confirm
175
   the job was claimed after the secret was created or rotated. Mask snapshots
207
   the job was claimed after the secret was created or rotated. Mask snapshots
docs/public/user/actions.mdmodified
@@ -27,6 +27,7 @@ a check run on matching pull requests.
27
 ## What works today
27
 ## What works today
28
 
28
 
29
 - `push`, `pull_request`, `schedule`, and `workflow_dispatch` triggers
29
 - `push`, `pull_request`, `schedule`, and `workflow_dispatch` triggers
30
+- `actions/checkout@v4` for repository checkout
30
 - `run:` steps executed in the operator-configured runner image
31
 - `run:` steps executed in the operator-configured runner image
31
 - `runs-on:` label matching against registered runners
32
 - `runs-on:` label matching against registered runners
32
 - workflow, job, and step `env:`
33
 - workflow, job, and step `env:`
@@ -41,16 +42,27 @@ runner uses. On shithub.sh, use the labels published by the instance operator.
41
 
42
 
42
 ## Current limit
43
 ## Current limit
43
 
44
 
44
-Use `run:` steps for now. The parser accepts these reserved aliases:
45
+The runner executes `actions/checkout@v4` and `run:` steps. Checkout accepts
46
+the default shallow fetch and `with.fetch-depth`; use `fetch-depth: 0` when a
47
+workflow needs full history:
48
+
49
+```yaml
50
+steps:
51
+  - uses: actions/checkout@v4
52
+    with:
53
+      fetch-depth: "0"
54
+  - run: git describe --tags --always
55
+```
56
+
57
+The parser also accepts these artifact aliases:
45
 
58
 
46
-- `actions/checkout@v4`
47
 - `shithub/upload-artifact@v1`
59
 - `shithub/upload-artifact@v1`
48
 - `shithub/download-artifact@v1`
60
 - `shithub/download-artifact@v1`
49
 
61
 
50
-The runner does not execute them yet. A workflow containing those `uses:` steps
62
+The runner does not execute artifact aliases yet. A workflow containing those
51
-will fail until checkout and artifact execution land. If you need repository
63
+artifact `uses:` steps will fail until artifact execution lands. Checkout
52
-files in a smoke workflow today, keep the command self-contained or fetch what
64
+inputs such as `path`, submodules, LFS, and persisted credentials are not
53
-you need explicitly inside a `run:` step.
65
+implemented yet.
54
 
66
 
55
 ## Expressions
67
 ## Expressions
56
 
68
 
@@ -84,9 +96,10 @@ Repo-scoped values shadow organization-scoped values with the same name.
84
 Most simple CI files need three edits:
96
 Most simple CI files need three edits:
85
 
97
 
86
 1. Move the workflow file from `.github/workflows/` to `.shithub/workflows/`.
98
 1. Move the workflow file from `.github/workflows/` to `.shithub/workflows/`.
87
-2. Replace `uses:` actions with equivalent `run:` commands.
99
+2. Keep `actions/checkout@v4`, but replace marketplace and artifact `uses:`
100
+   actions with equivalent `run:` commands for now.
88
 3. Confirm `runs-on:` matches a label registered by your shithub operator.
101
 3. Confirm `runs-on:` matches a label registered by your shithub operator.
89
 
102
 
90
 Marketplace actions, Docker actions, composite actions, hosted runner images,
103
 Marketplace actions, Docker actions, composite actions, hosted runner images,
91
-matrix expansion, service containers, and built-in checkout are not part of the
104
+matrix expansion, service containers, submodules, LFS, and artifact transfer
92
-current v1 runner.
105
+are not part of the current v1 runner.