tenseleyflow/shithub / acfe4d1

Browse files

Clarify Spaces object storage docs

Authored by espadonne
SHA
acfe4d1b669d0335d3e3700c913b65a88ae9c7fc
Parents
be5b659
Tree
97e2f6a

4 changed files

StatusFile+-
M cmd/shithubd/storage.go 5 4
M deploy/cutover/SETUP-GUIDE.md 2 1
M docs/internal/storage.md 6 5
M docs/public/self-host/deploy.md 3 1
cmd/shithubd/storage.gomodified
@@ -26,13 +26,14 @@ var storageCmd = &cobra.Command{
26
26
27
 var storageCheckCmd = &cobra.Command{
27
 var storageCheckCmd = &cobra.Command{
28
 	Use:   "check",
28
 	Use:   "check",
29
-	Short: "Verify S3 round-trip and repos-root writability",
29
+	Short: "Verify object-store round-trip and repos-root writability",
30
 	Long: `Exits 0 when both:
30
 	Long: `Exits 0 when both:
31
-  (a) PUT and GET succeed against the configured S3 bucket, and
31
+  (a) PUT and GET succeed against the configured S3-compatible object bucket, and
32
   (b) the configured repos_root is writable.
32
   (b) the configured repos_root is writable.
33
33
34
-When the S3 block is unconfigured, only (b) is checked. Used in deploy
34
+Production uses DigitalOcean Spaces through that S3-compatible API. When the
35
-smoke tests and as a sanity check from the operator's terminal.`,
35
+S3 block is unconfigured, only (b) is checked. Used in deploy smoke tests and
36
+as a sanity check from the operator's terminal.`,
36
 	RunE: func(cmd *cobra.Command, _ []string) error {
37
 	RunE: func(cmd *cobra.Command, _ []string) error {
37
 		cfg, err := config.Load(nil)
38
 		cfg, err := config.Load(nil)
38
 		if err != nil {
39
 		if err != nil {
deploy/cutover/SETUP-GUIDE.mdmodified
@@ -492,7 +492,8 @@ Fill in:
492
 - `caddy_email: ops@shithub.sh` (Let's Encrypt notifications)
492
 - `caddy_email: ops@shithub.sh` (Let's Encrypt notifications)
493
 - `db_password`, `hook_password` — paste from `/tmp/`.
493
 - `db_password`, `hook_password` — paste from `/tmp/`.
494
 - `session_key`, `totp_key` — paste from `/tmp/`.
494
 - `session_key`, `totp_key` — paste from `/tmp/`.
495
-- `s3_endpoint: nyc3.digitaloceanspaces.com`
495
+- `s3_endpoint: nyc3.digitaloceanspaces.com` (DigitalOcean Spaces via
496
+  the S3-compatible API)
496
 - `s3_region: us-east-1` (Spaces uses this for SigV4)
497
 - `s3_region: us-east-1` (Spaces uses this for SigV4)
497
 - `s3_bucket: shithub-prod`
498
 - `s3_bucket: shithub-prod`
498
 - `s3_access_key_id`, `s3_secret_access_key` — from B2 step 5.
499
 - `s3_access_key_id`, `s3_secret_access_key` — from B2 step 5.
docs/internal/storage.mdmodified
@@ -2,7 +2,7 @@ 
2
 
2
 
3
 shithub has two storage layers:
3
 shithub has two storage layers:
4
 
4
 
5
-1. **Object storage** — S3-compatible (MinIO in dev/test, DigitalOcean Spaces in prod). Used for avatars, attachments, and (post-MVP) LFS objects.
5
+1. **Object storage** — S3-compatible (MinIO in dev/test, DigitalOcean Spaces in prod). Used for avatars, attachments, and (post-MVP) LFS objects. The production bucket is DigitalOcean Spaces; the `s3` naming reflects the compatible API, not AWS.
6
 2. **Repo filesystem storage** — bare git repositories on a local block-storage volume, in a sharded layout owned by the `RepoFS` helper.
6
 2. **Repo filesystem storage** — bare git repositories on a local block-storage volume, in a sharded layout owned by the `RepoFS` helper.
7
 
7
 
8
 Both layers live behind the package `internal/infra/storage`. Path validation is the **security boundary** — every entry that takes user-supplied owner/repo names goes through `RepoPath`, which rejects unsafe inputs against a strict whitelist. If repo paths can be tricked, every later sprint inherits the bug; the test suite *over*-tests this.
8
 Both layers live behind the package `internal/infra/storage`. Path validation is the **security boundary** — every entry that takes user-supplied owner/repo names goes through `RepoPath`, which rejects unsafe inputs against a strict whitelist. If repo paths can be tricked, every later sprint inherits the bug; the test suite *over*-tests this.
@@ -29,16 +29,17 @@ Two implementations:
29
 
29
 
30
 ### Bucket / key scheme
30
 ### Bucket / key scheme
31
 
31
 
32
-Single bucket per environment: `shithub-dev`, `shithub-staging`, `shithub-prod`. Per-scope key prefixes ease policy and tenant isolation:
32
+Single bucket per environment: `shithub-dev`, `shithub-staging`, `shithub-prod`. In production this is a DigitalOcean Spaces bucket configured through the S3-compatible client. Per-scope key prefixes ease policy and tenant isolation:
33
 
33
 
34
 ```
34
 ```
35
 lfs/<owner>/<repo>/<sha256>           # LFS objects (post-MVP, key shape reserved)
35
 lfs/<owner>/<repo>/<sha256>           # LFS objects (post-MVP, key shape reserved)
36
 attachments/<scope>/<id>/<filename>   # issue/PR/comment attachments
36
 attachments/<scope>/<id>/<filename>   # issue/PR/comment attachments
37
-avatars/<owner>/<size>.<ext>          # rendered avatar variants
37
+avatars/<user_id>/<hash>.png          # largest rendered avatar variant
38
+avatars/<user_id>/<hash>-<size>.png   # smaller rendered avatar variants
38
 backups/...                           # S37
39
 backups/...                           # S37
39
 ```
40
 ```
40
 
41
 
41
-Keys are always lowercase.
42
+Avatar uploads are decoded from PNG, JPEG, or GIF and re-encoded to PNG before storage. Keys are always lowercase.
42
 
43
 
43
 ### Semantics worth knowing
44
 ### Semantics worth knowing
44
 
45
 
@@ -107,7 +108,7 @@ All storage settings flow through `internal/infra/config` (see `docs/internal/co
107
 | Key | Type | Default | Notes |
108
 | Key | Type | Default | Notes |
108
 |---|---|---|---|
109
 |---|---|---|---|
109
 | `storage.repos_root` | string | `/data/repos` | Filesystem root for bare repos. Required. |
110
 | `storage.repos_root` | string | `/data/repos` | Filesystem root for bare repos. Required. |
110
-| `storage.s3.endpoint` | string | `""` | Host[:port], no scheme. Empty disables S3. |
111
+| `storage.s3.endpoint` | string | `""` | Host[:port], no scheme. Empty disables object storage. Production uses the DigitalOcean Spaces endpoint. |
111
 | `storage.s3.region` | string | `us-east-1` | Region for SigV4 signing. |
112
 | `storage.s3.region` | string | `us-east-1` | Region for SigV4 signing. |
112
 | `storage.s3.access_key_id` | string | `""` | |
113
 | `storage.s3.access_key_id` | string | `""` | |
113
 | `storage.s3.secret_access_key` | string | `""` | Redacted by `config print`. |
114
 | `storage.s3.secret_access_key` | string | `""` | Redacted by `config print`. |
docs/public/self-host/deploy.mdmodified
@@ -26,7 +26,9 @@ Critical variables:
26
 - `session_key` — base64 32-byte AEAD key. Generate with
26
 - `session_key` — base64 32-byte AEAD key. Generate with
27
   `openssl rand -base64 32`.
27
   `openssl rand -base64 32`.
28
 - `totp_key` — base64 32-byte AEAD key for at-rest TOTP secrets.
28
 - `totp_key` — base64 32-byte AEAD key for at-rest TOTP secrets.
29
-- `s3_*` — bucket name, region, credentials.
29
+- `s3_*` — runtime object storage bucket name, region, and
30
+  credentials. In the reference deployment these point at the
31
+  DigitalOcean Spaces bucket through its S3-compatible API.
30
 - `email_*` — Postmark token or SMTP creds.
32
 - `email_*` — Postmark token or SMTP creds.
31
 - `wireguard_*` — peer keys for the monitoring mesh.
33
 - `wireguard_*` — peer keys for the monitoring mesh.
32
 
34