shithub Public

Watch 1 Fork 0 Star 0
markdown · 2990 bytes Raw Blame History

Rotate signing keys

Distinct from rotate-secrets.md because signing keys have external verifiers (subscribers, browsers, recipients) that need to keep validating old artifacts during the rollover window.

Webhook HMAC secrets

Per-webhook, not global. Each webhook has its own secret. The owner rotates by editing the webhook in Settings → Webhooks → the hook → "Rotate secret" → set new value.

  • Subscriber MUST be updated to the new secret first (or accept both during the cutover).
  • The previous secret is not kept; immediate rotation cuts off old-signature validation.
  • For ops-driven mass rotation (suspected key store compromise): shithubd webhook rotate-all regenerates every secret. All subscribers will start getting signature mismatch until the owners update their endpoints.

Notification unsubscribe HMAC

The one-click unsubscribe links in emails are HMAC-signed. The key lives in notif.unsubscribe_key. Rotation invalidates every old email's unsubscribe link (a recipient with an unread email containing a link from before rotation will get "invalid link" if they click it after rotation).

Procedure:

  1. Generate a new key: openssl rand -base64 32.
  2. Replace notif.unsubscribe_key in worker.env.
  3. Redeploy worker (ANSIBLE_TAGS=app).
  4. Optional: add a banner to the in-app inbox letting users know unsubscribe links from old emails will not work.

Cursor signing key

internal/pagination/keyset HMACs every cursor we hand out. Rotation invalidates every cursor in flight. The user-visible effect is: any open browser tab with a "Next page" link from before the rotation will return "invalid cursor" → take them back to page 1.

Procedure mirrors the unsubscribe key. Acceptable to do without warning; the worst case is a user clicks "Next" and gets sent back to the top of the list.

TLS certificates (Caddy-managed)

Caddy obtains and renews certs from Let's Encrypt automatically. Manual rotation is rarely needed; if it is:

ssh edge
sudo caddy reload --config /etc/caddy/Caddyfile

If Let's Encrypt is rate-limiting you, switch the caddy_use_acme_staging inventory flag to true, redeploy, verify, then flip back. Production cert reissue is constrained by LE's per-week limits; mass rotation in a single hour will fail.

SSH host keys

The host keys (/etc/ssh/ssh_host_*) are sshd's identity to clients. Rotating them prompts every git client with "WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!" — disruptive.

Only rotate if the host has been compromised. Procedure:

  1. ssh-keygen -A on the host (regenerates all host keys).
  2. systemctl restart sshd.
  3. Publish the new host-key fingerprints somewhere users can verify (status page, security advisory).
  4. Users prune the old line from ~/.ssh/known_hosts and re-accept on first reconnect.

User-facing communication is critical here. Without it, you'll look like a MITM attacker to every developer's SSH client.

View source
1 # Rotate signing keys
2
3 Distinct from `rotate-secrets.md` because signing keys have
4 external verifiers (subscribers, browsers, recipients) that need
5 to keep validating old artifacts during the rollover window.
6
7 ## Webhook HMAC secrets
8
9 Per-webhook, not global. Each webhook has its own secret. The
10 owner rotates by editing the webhook in Settings → Webhooks →
11 the hook → "Rotate secret" → set new value.
12
13 - Subscriber MUST be updated to the new secret first (or accept
14 both during the cutover).
15 - The previous secret is **not** kept; immediate rotation cuts
16 off old-signature validation.
17 - For ops-driven mass rotation (suspected key store compromise):
18 `shithubd webhook rotate-all` regenerates every secret. All
19 subscribers will start getting `signature mismatch` until the
20 owners update their endpoints.
21
22 ## Notification unsubscribe HMAC
23
24 The one-click unsubscribe links in emails are HMAC-signed. The
25 key lives in `notif.unsubscribe_key`. Rotation invalidates every
26 **old email's** unsubscribe link (a recipient with an unread
27 email containing a link from before rotation will get "invalid
28 link" if they click it after rotation).
29
30 Procedure:
31
32 1. Generate a new key: `openssl rand -base64 32`.
33 2. Replace `notif.unsubscribe_key` in `worker.env`.
34 3. Redeploy worker (`ANSIBLE_TAGS=app`).
35 4. Optional: add a banner to the in-app inbox letting users
36 know unsubscribe links from old emails will not work.
37
38 ## Cursor signing key
39
40 `internal/pagination/keyset` HMACs every cursor we hand out.
41 Rotation invalidates every cursor in flight. The user-visible
42 effect is: any open browser tab with a "Next page" link from
43 before the rotation will return "invalid cursor" → take them
44 back to page 1.
45
46 Procedure mirrors the unsubscribe key. Acceptable to do without
47 warning; the worst case is a user clicks "Next" and gets sent
48 back to the top of the list.
49
50 ## TLS certificates (Caddy-managed)
51
52 Caddy obtains and renews certs from Let's Encrypt automatically.
53 Manual rotation is rarely needed; if it is:
54
55 ```sh
56 ssh edge
57 sudo caddy reload --config /etc/caddy/Caddyfile
58 ```
59
60 If Let's Encrypt is rate-limiting you, switch the
61 `caddy_use_acme_staging` inventory flag to `true`, redeploy,
62 verify, then flip back. Production cert reissue is constrained
63 by LE's per-week limits; mass rotation in a single hour will
64 fail.
65
66 ## SSH host keys
67
68 The host keys (`/etc/ssh/ssh_host_*`) are sshd's identity to
69 clients. Rotating them prompts every git client with "WARNING:
70 REMOTE HOST IDENTIFICATION HAS CHANGED!" — disruptive.
71
72 Only rotate if the host has been compromised. Procedure:
73
74 1. `ssh-keygen -A` on the host (regenerates all host keys).
75 2. `systemctl restart sshd`.
76 3. Publish the new host-key fingerprints somewhere users can
77 verify (status page, security advisory).
78 4. Users prune the old line from `~/.ssh/known_hosts` and
79 re-accept on first reconnect.
80
81 User-facing communication is critical here. Without it, you'll
82 look like a MITM attacker to every developer's SSH client.