shithub Public

Watch 1 Fork 0 Star 0

markdown · 3358 bytes Raw Blame History

Caching

The S36 perf-pass standardises on an in-process LRU (internal/cache/lru) with optional TTL and a single-flight wrapper for hot-key dogpile prevention. This doc tracks every cross-request cache and its invalidation contract.

The invariant is: every cached value has a documented invalidation trigger. If you can't name the trigger, the cache is a bug factory.

Active caches

Cache	Key	Value	Invalidator	Bound
`repos/git.AheadBehindCached`	(repo_id, base_oid, head_oid)	(ahead, behind)	OID change ⇒ different key; LRU eviction	4096 entries

Concrete uses:

branchesList (S20 deferral H4) — replaces N git rev-list invocations per page load with one cached lookup per branch. Single-flight collapses concurrent misses on hot branches.

Planned caches (next iterations)

These are listed in the S36 spec; they land as the surfaces they back grow large enough to bench-justify the cache.

Cache	Key	Value	Invalidator
Tree at root	(repo_id, ref_oid)	rendered ls-tree result	push:process bumps default-OID
Ref list	(repo_id)	branches + tags	push:process
File list (finder)	(repo_id, ref_oid)	flat path slice	push:process
Default-branch OID	(repo_id)	OID string	push:process + default-branch swap
Markdown render	(markdown_pipeline_version, body_hash)	rendered HTML	bump pipeline version on goldmark/policy change
Effective-team-set	(actor, org)	team-id slice	team-membership change

Single-flight: when to wrap

Wrap with lru.Group whenever:

The upstream is non-trivial (subprocess, FS walk, multi-row DB read), AND
The key is hot (one popular repo, one busy user), AND
Concurrent misses are realistic (HTTP burst, worker fanout).

Without single-flight, a cache miss under load triggers a stampede that defeats the cache's purpose. The lru.Group wrapper collapses N concurrent misses into one upstream call.

Errors are NOT cached

lru.Group.Do deliberately returns errors without caching them. A transient upstream failure (DB blip, git fork EAGAIN) shouldn't poison subsequent reads. Negative caching (caching the absence of a key) is a separate concern; callers add their own sentinel value when they want it.

TTL: when to use one

Default to no-TTL with explicit invalidation. Use TTL only when:

The data is fully public + anonymous-cacheable (rendered HTML for a public repo's README).
Staleness is measured-low-impact (≤ 60s for hot reads).
An explicit invalidator is wired in addition (TTL is the safety net, not the primary correctness mechanism).

Avoid TTL on personalized content. The "you see your friend's old comment for 30s" UX surprise is not worth the cache hit-rate.

Reading hit-rates

Every cache exposes Stats() lru.Stats{Hits, Misses, Evictions}. The /metrics surface (S37 deploy) will scrape these. CI baseline asserts hit-rate above a per-cache target on the bench run.

Invalidation patterns

The push:process worker is the canonical invalidation source for git-shaped caches. After updating refs:

git.InvalidateAheadBehind(git.AheadBehindKey{...})
// + future: tree, refs, default-branch caches

The (repo_id, ...) key shape lets us scope invalidation to one repo's slice without scanning the whole cache.

View source

  
        1
        # Caching
      
        2
        
        3
        The S36 perf-pass standardises on an in-process LRU
      
        4
        (`internal/cache/lru`) with optional TTL and a single-flight
      
        5
        wrapper for hot-key dogpile prevention. This doc tracks every
      
        6
        cross-request cache and its invalidation contract.
      
        7
        
        8
        The invariant is: **every cached value has a documented
      
        9
        invalidation trigger**. If you can't name the trigger, the cache
      
        10
        is a bug factory.
      
        11
        
        12
        ## Active caches
      
        13
        
        14
        | Cache | Key | Value | Invalidator | Bound |
      
        15
        |---|---|---|---|---|
      
        16
        | `repos/git.AheadBehindCached` | (repo_id, base_oid, head_oid) | (ahead, behind) | OID change ⇒ different key; LRU eviction | 4096 entries |
      
        17
        
        18
        Concrete uses:
      
        19
        - `branchesList` (S20 deferral H4) — replaces N `git rev-list`
      
        20
          invocations per page load with one cached lookup per branch.
      
        21
          Single-flight collapses concurrent misses on hot branches.
      
        22
        
        23
        ## Planned caches (next iterations)
      
        24
        
        25
        These are listed in the S36 spec; they land as the surfaces they
      
        26
        back grow large enough to bench-justify the cache.
      
        27
        
        28
        | Cache | Key | Value | Invalidator |
      
        29
        |---|---|---|---|
      
        30
        | Tree at root | (repo_id, ref_oid) | rendered ls-tree result | push:process bumps default-OID |
      
        31
        | Ref list | (repo_id) | branches + tags | push:process |
      
        32
        | File list (finder) | (repo_id, ref_oid) | flat path slice | push:process |
      
        33
        | Default-branch OID | (repo_id) | OID string | push:process + default-branch swap |
      
        34
        | Markdown render | (markdown_pipeline_version, body_hash) | rendered HTML | bump pipeline version on goldmark/policy change |
      
        35
        | Effective-team-set | (actor, org) | team-id slice | team-membership change |
      
        36
        
        37
        ## Single-flight: when to wrap
      
        38
        
        39
        Wrap with `lru.Group` whenever:
      
        40
        
        41
        1. The upstream is non-trivial (subprocess, FS walk, multi-row DB read), AND
      
        42
        2. The key is hot (one popular repo, one busy user), AND
      
        43
        3. Concurrent misses are realistic (HTTP burst, worker fanout).
      
        44
        
        45
        Without single-flight, a cache miss under load triggers a stampede
      
        46
        that defeats the cache's purpose. The `lru.Group` wrapper collapses
      
        47
        N concurrent misses into one upstream call.
      
        48
        
        49
        ## Errors are NOT cached
      
        50
        
        51
        `lru.Group.Do` deliberately returns errors without caching them. A
      
        52
        transient upstream failure (DB blip, git fork EAGAIN) shouldn't
      
        53
        poison subsequent reads. Negative caching (caching the absence of
      
        54
        a key) is a separate concern; callers add their own sentinel value
      
        55
        when they want it.
      
        56
        
        57
        ## TTL: when to use one
      
        58
        
        59
        Default to no-TTL with explicit invalidation. Use TTL only when:
      
        60
        
        61
        - The data is fully public + anonymous-cacheable (rendered HTML for
      
        62
          a public repo's README).
      
        63
        - Staleness is measured-low-impact (≤ 60s for hot reads).
      
        64
        - An explicit invalidator is wired in addition (TTL is the safety
      
        65
          net, not the primary correctness mechanism).
      
        66
        
        67
        Avoid TTL on personalized content. The "you see your friend's old
      
        68
        comment for 30s" UX surprise is not worth the cache hit-rate.
      
        69
        
        70
        ## Reading hit-rates
      
        71
        
        72
        Every cache exposes `Stats() lru.Stats{Hits, Misses, Evictions}`.
      
        73
        The `/metrics` surface (S37 deploy) will scrape these. CI baseline
      
        74
        asserts hit-rate above a per-cache target on the bench run.
      
        75
        
        76
        ## Invalidation patterns
      
        77
        
        78
        The push:process worker is the canonical invalidation source for
      
        79
        git-shaped caches. After updating refs:
      
        80
        
        81
        ```go
      
        82
        git.InvalidateAheadBehind(git.AheadBehindKey{...})
      
        83
        // + future: tree, refs, default-branch caches
      
        84
        ```
      
        85
        
        86
        The (repo_id, ...) key shape lets us scope invalidation to one
      
        87
        repo's slice without scanning the whole cache.

1	# Caching
2
3	The S36 perf-pass standardises on an in-process LRU
4	(`internal/cache/lru`) with optional TTL and a single-flight
5	wrapper for hot-key dogpile prevention. This doc tracks every
6	cross-request cache and its invalidation contract.
7
8	The invariant is: **every cached value has a documented
9	invalidation trigger**. If you can't name the trigger, the cache
10	is a bug factory.
11
12	## Active caches
13
14	\| Cache \| Key \| Value \| Invalidator \| Bound \|
15	\|---\|---\|---\|---\|---\|
16	\| `repos/git.AheadBehindCached` \| (repo_id, base_oid, head_oid) \| (ahead, behind) \| OID change ⇒ different key; LRU eviction \| 4096 entries \|
17
18	Concrete uses:
19	- `branchesList` (S20 deferral H4) — replaces N `git rev-list`
20	invocations per page load with one cached lookup per branch.
21	Single-flight collapses concurrent misses on hot branches.
22
23	## Planned caches (next iterations)
24
25	These are listed in the S36 spec; they land as the surfaces they
26	back grow large enough to bench-justify the cache.
27
28	\| Cache \| Key \| Value \| Invalidator \|
29	\|---\|---\|---\|---\|
30	\| Tree at root \| (repo_id, ref_oid) \| rendered ls-tree result \| push:process bumps default-OID \|
31	\| Ref list \| (repo_id) \| branches + tags \| push:process \|
32	\| File list (finder) \| (repo_id, ref_oid) \| flat path slice \| push:process \|
33	\| Default-branch OID \| (repo_id) \| OID string \| push:process + default-branch swap \|
34	\| Markdown render \| (markdown_pipeline_version, body_hash) \| rendered HTML \| bump pipeline version on goldmark/policy change \|
35	\| Effective-team-set \| (actor, org) \| team-id slice \| team-membership change \|
36
37	## Single-flight: when to wrap
38
39	Wrap with `lru.Group` whenever:
40
41	1. The upstream is non-trivial (subprocess, FS walk, multi-row DB read), AND
42	2. The key is hot (one popular repo, one busy user), AND
43	3. Concurrent misses are realistic (HTTP burst, worker fanout).
44
45	Without single-flight, a cache miss under load triggers a stampede
46	that defeats the cache's purpose. The `lru.Group` wrapper collapses
47	N concurrent misses into one upstream call.
48
49	## Errors are NOT cached
50
51	`lru.Group.Do` deliberately returns errors without caching them. A
52	transient upstream failure (DB blip, git fork EAGAIN) shouldn't
53	poison subsequent reads. Negative caching (caching the absence of
54	a key) is a separate concern; callers add their own sentinel value
55	when they want it.
56
57	## TTL: when to use one
58
59	Default to no-TTL with explicit invalidation. Use TTL only when:
60
61	- The data is fully public + anonymous-cacheable (rendered HTML for
62	a public repo's README).
63	- Staleness is measured-low-impact (≤ 60s for hot reads).
64	- An explicit invalidator is wired in addition (TTL is the safety
65	net, not the primary correctness mechanism).
66
67	Avoid TTL on personalized content. The "you see your friend's old
68	comment for 30s" UX surprise is not worth the cache hit-rate.
69
70	## Reading hit-rates
71
72	Every cache exposes `Stats() lru.Stats{Hits, Misses, Evictions}`.
73	The `/metrics` surface (S37 deploy) will scrape these. CI baseline
74	asserts hit-rate above a per-cache target on the bench run.
75
76	## Invalidation patterns
77
78	The push:process worker is the canonical invalidation source for
79	git-shaped caches. After updating refs:
80
81	```go
82	git.InvalidateAheadBehind(git.AheadBehindKey{...})
83	// + future: tree, refs, default-branch caches
84	```
85
86	The (repo_id, ...) key shape lets us scope invalidation to one
87	repo's slice without scanning the whole cache.