`49ecaa3`

docs/runner: document DNS allowlist and secret scrub posture

Authored by mfwolffe <wolffemf@dukes.jmu.edu> 2 days ago

SHA: 49ecaa3d0c90a887d1dd838b75f54200cfb71554
Parents: 07bdfb8
Tree: 53f00c4

11 changed files

Status	File	+	-
M	`SECURITY.md`	19	0
M	`deploy/ansible/inventory/production.example`	1	0
M	`deploy/ansible/roles/shithubd-runner/defaults/main.yml`	12	0
M	`deploy/ansible/roles/shithubd-runner/tasks/main.yml`	8	0
M	`deploy/ansible/roles/shithubd-runner/templates/config.toml.j2`	10	0
M	`deploy/runner-config/README.md`	12	0
A	`deploy/runner-config/dnsmasq.conf.j2`	15	0
M	`docs/internal/actions-runner-api.md`	7	0
M	`docs/internal/actions-schema.md`	6	3
M	`docs/internal/runbooks/actions-runner.md`	11	0
M	`docs/internal/runbooks/runner-deploy.md`	31	0

SECURITY.mdmodified

  - pinned seccomp profile at `/etc/shithubd-runner/seccomp.json`
  - `--user 65534:65534`
  - PID, file-descriptor, process, CPU, memory, and log-size caps
++- optional per-container DNS servers for operator-managed egress
++  allowlisting
  The writable `/workspace` mount is deliberate. The v1 engine starts one
  container per step, so checkout/build outputs need a host-backed job
  present, no-new-privileges is set, and the default seccomp profile still
  filters dangerous syscalls.
++Runner network allowlisting is an operator-managed control. The runner
++config records `runner.network_allowlist` and passes
++`engine.dns_servers` to each step container; the deployment role renders
++a dnsmasq allowlist template. DNS filtering must be paired with host
++firewall rules on the runner bridge to block direct-IP egress. Do not
++treat DNS-only filtering as a complete network sandbox.
++
++Actions secrets are decrypted only for the runner job claim that needs
++them. Repo secrets shadow org secrets with the same name. The runner
++receives the resolved secret map plus an exact-value mask set; runner
++logs are scrubbed before upload, and the web API scrubs again before
++persisting chunks. The server-side scrubber carries possible secret
++prefix tails across adjacent chunks so a bypassing runner cannot leak a
++secret by splitting it over multiple log POSTs. Base64-encoded or
++transformed secrets are not masked; workflows must not print secrets in
++derived forms.
++
  Root containers are opt-in per job through an explicit shithub-only
  permissions key:

deploy/ansible/inventory/production.examplemodified

`@@ -52,3 +52,4 @@` grafana_cloud_prom_token=REPLACE_ME # access-policy token
52	# shithub_runner_labels=self-hosted,linux,ubuntu-latest	52	# shithub_runner_labels=self-hosted,linux,ubuntu-latest
53	# shithub_runner_capacity=1	53	# shithub_runner_capacity=1
54	# shithub_runner_default_image=ghcr.io/shithub/runner-nix:1.0	54	# shithub_runner_default_image=ghcr.io/shithub/runner-nix:1.0
		55	+# shithub_runner_dns_servers=172.30.0.1

deploy/ansible/roles/shithubd-runner/defaults/main.ymlmodified

  shithub_runner_poll_interval: 5s
  shithub_runner_workspace_root: /var/lib/shithubd-runner/workspaces
  shithub_runner_workspace_ttl: 24h
++shithub_runner_network_allowlist:
++  - api.github.com
++  - auth.docker.io
++  - codeload.github.com
++  - github.com
++  - objects.githubusercontent.com
++  - production.cloudflare.docker.com
++  - registry-1.docker.io
++  - "*.githubusercontent.com"
  shithub_runner_engine: docker
  shithub_runner_default_image: ghcr.io/shithub/runner-nix:1.0
  shithub_runner_network: bridge
++shithub_runner_dns_servers: []
++shithub_runner_dnsmasq_config: /etc/shithubd-runner/dnsmasq.conf
++shithub_runner_dnsmasq_upstream: 1.1.1.1
  shithub_runner_memory: 2g
  shithub_runner_cpus: "2"
  shithub_runner_seccomp_profile: /etc/shithubd-runner/seccomp.json

deploy/ansible/roles/shithubd-runner/tasks/main.ymlmodified

      mode: "0640"
    notify: restart shithubd-runner
++- name: Runner DNS allowlist template
++  template:
++    src: "{{ playbook_dir }}/../runner-config/dnsmasq.conf.j2"
++    dest: "{{ shithub_runner_dnsmasq_config }}"
++    owner: root
++    group: shithub-runner
++    mode: "0640"
++
  - name: Runner systemd unit
    copy:
      src: "{{ playbook_dir }}/../systemd/shithubd-runner.service"

deploy/ansible/roles/shithubd-runner/templates/config.toml.j2modified

  poll_interval = "{{ shithub_runner_poll_interval }}"
  workspace_root = "{{ shithub_runner_workspace_root }}"
  workspace_ttl = "{{ shithub_runner_workspace_ttl }}"
++{% if shithub_runner_network_allowlist is string %}
++network_allowlist = {{ shithub_runner_network_allowlist.split(",") | map("trim") | list | to_json }}
++{% else %}
++network_allowlist = {{ shithub_runner_network_allowlist | to_json }}
++{% endif %}
  [engine]
  kind = "{{ shithub_runner_engine }}"
  seccomp_profile = "{{ shithub_runner_seccomp_profile }}"
  user = "{{ shithub_runner_container_user }}"
  pids_limit = {{ shithub_runner_pids_limit }}
++{% if shithub_runner_dns_servers is string %}
++dns_servers = {{ shithub_runner_dns_servers.split(",") | map("trim") | list | to_json }}
++{% else %}
++dns_servers = {{ shithub_runner_dns_servers | to_json }}
++{% endif %}
  [log]
  level = "{{ shithub_runner_log_level }}"

deploy/runner-config/README.mdmodified

  Update this file deliberately when changing Docker daemon versions or
  runner syscall posture.
++
++`dnsmasq.conf.j2` is the optional runner DNS allowlist template. The
++Ansible role renders it to `/etc/shithubd-runner/dnsmasq.conf` from
++`shithub_runner_network_allowlist`; operators can run dnsmasq bound to
++their Actions Docker bridge and point step containers at it with
++`engine.dns_servers`.
++
++The dnsmasq template intentionally has no default upstream resolver, so
++names outside the allowlist fail resolution. DNS allowlisting alone does
++not block direct-IP egress or a workflow that brings its own resolver;
++pair it with host firewall rules on the runner bridge for a deny-by-
++default network boundary.

deploy/runner-config/dnsmasq.conf.j2added

++# Managed by Ansible. Optional DNS allowlist resolver for Actions runners.
++#
++# Pair this with a Docker bridge/network that uses this resolver as its only
++# DNS server. This controls name resolution, not direct-IP egress; enforce
++# direct-IP denial with host firewall rules on the runner bridge.
++
++domain-needed
++bogus-priv
++no-resolv
++no-hosts
++
++{% for pattern in shithub_runner_network_allowlist %}
++{% set host = (pattern[2:] if pattern.startswith("*.") else pattern) %}
++server=/{{ host }}/{{ shithub_runner_dnsmasq_upstream }}
++{% endfor %}

docs/internal/actions-runner-api.mdmodified

  `token`, `expires_at`, and `job` when a job is claimed. Capacity is
  enforced server-side by counting current `workflow_jobs.status =
  'running'` rows for the runner while holding a row lock on the runner.
++The job payload includes resolved `secrets` and `mask_values`; repo
++secrets shadow org secrets with the same name.
  `POST /api/v1/jobs/{id}/logs`
  first step in the job receives the chunk. Chunks are base64-decoded,
  capped at 512 KiB raw, and appended to `workflow_step_log_chunks`.
  Duplicate `(step_id, seq)` inserts are accepted as idempotent retries.
++Before append, the API re-scrubs exact secret values from the runner
++claim's visible secret set. It also reprocesses any possible secret
++prefix carried at the end of the prior chunk, so a runner cannot leak a
++secret by splitting it across two log calls.
  `POST /api/v1/jobs/{id}/steps/{step_id}/status`
  - `shithub_actions_runner_registrations_total`
  - `shithub_actions_runner_heartbeats_total{result="claimed|no_job"}`
  - `shithub_actions_runner_jwt_total{result="issued|rejected|replay"}`
++- `shithub_actions_log_scrub_replacements_total{location="server"}`

docs/internal/actions-schema.mdmodified

  Runner log chunks pass through `internal/runner/scrub` before they are
  posted to the API. It masks exact secret values and preserves enough
  tail bytes between chunks to catch a secret split across chunk
--boundaries. S41e follow-up work wires resolved workflow secrets into
++boundaries. S41e wires resolved workflow secrets into the runner claim
--the runner/API mask set and adds server-side defense in depth before
++payload and mask set, then applies the same exact-value scrub again in
--persisting chunks.
++the runner API before persisting chunks. The server path also carries a
++possible secret-prefix tail from the prior persisted chunk, so a runner
++that bypasses client-side scrubbing cannot leak a secret by splitting
++it across adjacent log POSTs.
  ## `shithub.event` payload schema (v1)

docs/internal/runbooks/actions-runner.mdmodified

  poll_interval = "5s"
  workspace_root = "/var/lib/shithubd-runner/workspaces"
  workspace_ttl = "24h"
++network_allowlist = [
++  "api.github.com",
++  "auth.docker.io",
++  "codeload.github.com",
++  "github.com",
++  "objects.githubusercontent.com",
++  "production.cloudflare.docker.com",
++  "registry-1.docker.io",
++  "*.githubusercontent.com",
++]
  [engine]
  kind = "docker"
  seccomp_profile = "/etc/shithubd-runner/seccomp.json"
  user = "65534:65534"
  pids_limit = 512
++dns_servers = []
  ```
  The config path defaults to `/etc/shithubd-runner/config.toml`.

docs/internal/runbooks/runner-deploy.mdmodified

  shithub_runner_seccomp_profile=/etc/shithubd-runner/seccomp.json
  shithub_runner_container_user=65534:65534
  shithub_runner_pids_limit=512
++shithub_runner_dns_servers=172.30.0.1
  ```
  The role writes non-secret config to
  Keep `shithub_runner_workspace_root` under `/var/lib/shithubd-runner`;
  the systemd unit grants runner writes only to that subtree.
++`shithub_runner_network_allowlist` defaults to GitHub source/archive
++hosts plus Docker Hub registry hosts. Override it when a runner must
++fetch from an internal package registry. `shithub_runner_dns_servers`
++is empty by default; set it only after a DNS allowlist resolver exists
++on the runner network.
++
  ## Deploy
  For the runner role only:
  - creates the `shithub-runner` system user and joins it to `docker`
  - uploads `/usr/local/bin/shithubd-runner`
  - renders `/etc/shithubd-runner/config.toml` and `runner.env`
++- renders `/etc/shithubd-runner/dnsmasq.conf` from the network
++  allowlist for operators who run a local DNS allowlist resolver
  - installs the pinned seccomp profile at
    `/etc/shithubd-runner/seccomp.json`
  - installs `deploy/systemd/shithubd-runner.service`
  - step logs and systemd journal include the configured image, network,
    CPU/memory limits, PID limit, container user, and seccomp profile
++## Network Allowlist
++
++The runner config carries two separate network controls:
++
++- `runner.network_allowlist`: the host patterns allowed by the
++  operator's DNS allowlist resolver.
++- `engine.dns_servers`: DNS servers passed to each step container with
++  Docker `--dns`.
++
++For a single-host deployment, create a dedicated Docker bridge for
++Actions jobs, run dnsmasq bound to that bridge, render
++`/etc/shithubd-runner/dnsmasq.conf`, and set
++`shithub_runner_dns_servers` to the bridge address of that resolver.
++The rendered dnsmasq config has no default upstream resolver; names not
++matching the allowlist fail DNS resolution.
++
++DNS filtering is not a complete egress boundary by itself. Block
++direct-IP egress from the Actions bridge with host firewall rules, and
++allow only DNS to the resolver plus established outbound connections
++opened by that resolver. Keep the runner on a separate host from web
++and database services.
++
  ## Rollback
  Stop the runner first so it does not claim new jobs: