markdown · 18360 bytes Raw Blame History

CLAUDE.md

Guidance to Claude Code when working in this repository. The parent armfortas/CLAUDE.md governs the compiler and applies here too; this file adds linker-specific discipline on top.

Repository Context

afs-ld is a git submodule of ARMFORTAS, the bespoke ARM64 Fortran compiler. It is the standalone ARM64 Mach-O linker: reads MH_OBJECT from afs-as, static archives (.a), binary dylibs, and TAPI TBD text stubs; emits MH_EXECUTE and MH_DYLIB.

It knows nothing about Fortran. The boundary with the compiler is the CLI — an ld-compatible flag surface.

Rust standard library only. No clap, serde, byteorder, object, goblin, memmap2, yaml-rust. Hand-roll parsers, serializers, and the tiny YAML subset TBD files need. Adding a dependency requires a discussion, a CLAUDE.md update, and a justification.

Build, Test, Lint

cargo build -p afs-ld                          # build linker crate
cargo test  -p afs-ld                          # full suite
cargo clippy -p afs-ld --all-targets -- -D warnings

cargo test --lib -p afs-ld                     # unit tests only
cargo test --test <name> -p afs-ld             # one integration test file
cargo test --test parity_matrix                # vs Apple `ld` across the corpus (Sprint 27)
cargo test --test hello_world                  # executable end-to-end (Sprint 18)
cargo test --test hello_library                # dylib end-to-end (Sprint 18.5)
cargo test --test reader_corpus_round_trip     # afs-as corpus → byte-identity
cargo test --test archive_runtime              # libarmfortas_rt.a reality check
cargo test --test dylib_integration            # clang-built dylib → DylibFile
cargo test -p afs-ld -- <substring>            # filter by test name

Integration tests shell out to xcrun as, xcrun clang, ld, otool, nm, codesign. They require macOS on Apple Silicon and a working Xcode command-line toolchain. Do not stub or skip them — the differential against the system tools is the entire point of the suite. If xcrun is missing, tests emit skipping: ... and return cleanly. Never rewrite a test to always pass.

The --dump* CLI modes exist for manual inspection:

afs-ld --dump         input.o        # Mach-O header, load commands, sections, symbols, relocs
afs-ld --dump-archive libfoo.a       # flavor, members, symbol index
afs-ld --dump-dylib   libfoo.dylib   # install_name, dependencies, rpaths, export trie

Every time a new decoder lands, extend the relevant --dump* output.

Target

  • Architecture: arm64 only. Not arm64e, not arm64_32. Not x86_64.
  • OS: macOS. Mach-O file format, Apple AAPCS64 calling convention.
  • Goal: parity with Apple ld for the binaries armfortas produces and the fortsh milestone. Not a toy. Not a subset. The full Mach-O/dyld contract for our use cases.

Design Philosophy

  • Bespoke. We write every decoder, encoder, and layout pass. No object, no goblin, no mach-object. When something breaks, we read our code.
  • Byte-level round-trip is the invariant. For every wire structure (header, load commands, sections, symbols, strings, relocations, archive members, export-trie nodes), parse(write(x)) == x and write(parse(bytes)) == bytes on every corpus fixture. If a structure can't round-trip, it's not done.
  • Total control. ld has bugs we can't fix, behaviors we can't observe, and refactors we can't predict. Owning the linker closes the loop on every binary armfortas produces.
  • No hidden state. Every transformation between raw wire form and linker-side form is explicit. Raw bits are preserved until the writer phase reshapes them; Raw { cmd, cmdsize, data } is a legitimate forever-variant for load commands we don't decode yet.

Architecture

Pipeline, end-to-end:

args → inputs → resolve → atomize → layout → apply relocs → synth sections → write → sign
 │        │        │         │        │            │              │            │       │
args.rs  input.rs resolve.rs atom.rs layout.rs  reloc/arm64.rs  synth/*.rs  macho/   synth/
                 symbol.rs                                                   writer    code_sig

Module responsibilities

  • src/args.rs — CLI parser. Hand-rolled, streaming argv scan. No clap.
  • src/macho/ — Mach-O 64 read/write.
    • constants.rs: LC_*, MH_*, S_*, N_*, ARM64_RELOC_*, EXPORT_SYMBOL_FLAGS_*, PLATFORM_*. Numeric literals mirroring Apple's <mach-o/loader.h> / <nlist.h> / <reloc.h> / <arm64/reloc.h>. Duplicated from afs-as rather than cross-crate coupled — each submodule owns its copy so they stay independent.
    • reader.rs: MachHeader64, LoadCommand enum, per-command structs (Segment64, Section64Header, SymtabCmd, DysymtabCmd, BuildVersionCmd, DylibCmd, RpathCmd, DyldInfoCmd, LinkEditDataCmd). Every variant has parse(cmdsize, payload) and write(&mut Vec<u8>) paired as round-trip.
    • dylib.rs: DylibFile::parse — pulls LC_ID_DYLIB, dependency chain, rpaths, and routes export-trie bytes through to exports.rs.
    • exports.rs: ExportTrie, ExportKind, cycle-safe walker with MAX_DEPTH=128.
    • writer.rs: emits MH_EXECUTE / MH_DYLIB (lands at Sprint 10+).
    • tbd.rs: TAPI TBD v4 YAML-subset parser (Sprint 6).
  • src/archive.rs — BSD + SysV + GNU-thin static archives. Lazy member fetch via fetch_object_defining(name).
  • src/input.rsObjectFile aggregate: header + load commands + sections + symbols + strings + dysymtab. Sprint 4 adds archive fetching; Sprint 7 introduces InputFile enum.
  • src/symbol.rsRawNlist (wire form, round-trip) + InputSymbol accessors (kind, ext, weak_ref/def, common size/alignment, library ordinal, indirect strx).
  • src/string_table.rs — owned StringTable with suffix-dedup-aware strx → &str lookup.
  • src/section.rsSectionKind taxonomy (code / data / zerofill / TLS / literals / stubs / GOT / compact-unwind / eh_frame) derived from (segname, sectname, flags); InputSection with data + raw-reloc slices.
  • src/reloc/ — ARM64 relocs.
    • mod.rs: RawRelocation (bit-packed), Reloc (fused; ADDEND / SUBTRACTOR prefixes folded into primaries), parse_relocs / write_relocs (reversible), validate_relocs (bounds, referent range, kind-vs-length-vs-pcrel).
    • arm64.rs: reloc application against final addresses (Sprint 11).
    • loh.rs: LOH preservation / relaxation (Sprint 25).
  • src/leb.rs — ULEB128/SLEB128 codec reused by export trie, function-starts deltas, dyld opcode streams, chained fixups.
  • src/diag.rs — diagnostics. Path + byte offset + caret, matching afs-as/src/diag*.rs style. Deterministic output: no wall clock, no pid, no thread-id in error text.
  • src/dump.rs--dump* inspection modes. Every time a reader decodes something new, extend the dump.
  • src/driver.rs — orchestrator (Sprint 20).

Coding Conventions

  • Rust, idiomatic. Use enums for wire structures and linker models. Exhaustive pattern matching everywhere — no catch-all _ arms outside tests. When a new LoadCommand variant lands, every match that inspects LoadCommand has to grow a new arm; the compiler enforces this and that's the point.
  • unsafe only where genuinely required. The one known case is libc::mmap for large input files (Sprint 28). Keep blocks small, comment the invariant, and never let unsafe leak across module boundaries.
  • Byte-level round-trip for every wire structure. Don't add a parser without its writer. Don't add a writer without a test that proves write(parse(x)) == x. See src/macho/reader.rs for the template.
  • Raw wire form stays accessible. A Reloc kind that can't losslessly re-emit its ADDEND prefix is broken. A Name16 stored as String instead of [u8; 16] loses null padding and breaks byte-identity. Preserve the wire.
  • Constants duplicated, not imported. afs-ld/src/macho/constants.rs mirrors Apple's headers numerically. Do not depend on afs-as at a type level. Submodule independence matters more than deduplication.
  • Diagnostics cite input + offset + caret. Every parser error goes through ReadError / ArchiveError variants with explicit at_offset / context / reason. No String::from("something broke") noise.
  • Commit discipline: terse imperative messages, no co-authors, per-file / per-chunk commits, never monoliths. No sprint-number references in commit subjects. Commit often — "if I had to bisect this sprint, which revision would I want to land on?" is the granularity.
  • Tests alongside code. Every new decoder lands with unit tests in the same commit. Every bug fix gets a regression test. Integration tests that assemble/link real fixtures at test time always skip cleanly when xcrun is unavailable — never hard-fail on missing toolchain.
  • No "stubs pass silently." Placeholder code that returns wrong answers is worse than code that panics. If a kind is "not yet implemented", say so with a hard error and a pointer to the sprint that'll finish it. Tests must catch the stub, not paper over it.
  • Don't cut corners without stopping to discuss. If you find yourself about to skip a round-trip test, hardcode a "good enough" offset, or silently accept a malformed input, stop and talk it through. We are not building a toy linker.
  • Avoid rushing through sprints to get to an audit. The hard work of each sprint is the point of the sprint. The audit is the downstream check, not the goal.
  • Always opt for the robust solution. If you find yourself saying "the simple solution is X", stop and ask whether a production linker uses the simple solution or digs deeper. The simple one might be right — but we have to be sure this is not a toy.
  • When unsure, consult .refs/. .refs/llvm/lld/MachO/ is the primary architectural reference; .refs/ld64/src/ is Apple's authoritative implementation for parity edge cases; .refs/mold/src/ informs performance choices. Read before inventing.
  • Run long test jobs judiciously. Think about the grep/filter before you launch — cargo test -p afs-ld -- reader_ beats cargo test --workspace when you only touched the reader. The corpus round-trip is fast (~1s); the parity matrix (Sprint 27) won't be.

Test Architecture

Layered, not a single golden path. Each layer catches a different class of regression:

Test file What it proves
src/**/#[cfg(test)] Unit tests per decoder / encoder / accessor
tests/common/harness.rs Differential harness — afs-ld vs system ld, tolerated-diff classifier
tests/reader_corpus_round_trip.rs Full afs-as corpus → byte-identity for header + LC region, symtab, strtab, relocs
tests/archive_runtime.rs Real libarmfortas_rt.a parses; symbol index resolves
tests/dylib_integration.rs xcrun clang dylib → DylibFile, install_name + exports + libSystem dep
tests/reader_empty.rs CLI contract: empty argv → afs-ld: error: no input files, exit 2
tests/diff_harness_sanity.rs Harness zero-diffs on identical inputs
tests/diff_harness_finds_critical.rs Harness catches intentional byte differences
tests/hello_world.rs Executable end-to-end (Sprint 18)
tests/hello_library.rs Dylib end-to-end (Sprint 18.5)
tests/parity_matrix.rs Corpus byte-level differential vs Apple ld (Sprint 27)
tests/armfortas_integration.rs Parent's integration suite under AFS_LD=1 (Sprint 21)

Corpus fixtures live in tests/corpus/. Every new relocation kind, section kind, or CLI flag lands a corpus entry in the same sprint.

Audit Discipline

After each sprint, a brutally honest audit:

  • Assume nothing works until proven otherwise. Test every claim. Re-run the whole test suite, not just the freshly-written tests.
  • "Placeholder" and "stub" are synonyms for "broken." Wrong answers silently produced are worse than crashes.
  • Check against the Mach-O ABI spec, not just "does it link." Wrong output corrupts dyld's bind/rebase state at runtime; the kernel or loader will tell you, hours later, in ways that are hard to bisect.
  • Don't soften findings. Major means "produces wrong binaries." Critical means "silent corruption that dyld will accept."
  • No deferred items unless they genuinely require a later sprint. Fix it now if it can be fixed now. A deferred item accumulates debt with compound interest.
  • The audit is not a formality. It is the last line of defense before bad linker output gets merged and ships bad binaries downstream.

Completeness Philosophy

Every Mach-O feature modern dyld consumes is in scope. When implementing a decoder or encoder:

  • Implement it fully, not just the subset armfortas or fortsh happens to exercise. Every ARM64_RELOC_* kind, every EXPORT_SYMBOL_FLAGS_* terminal form, every LC_*_DYLIB variant.
  • Don't defer features with "fortsh doesn't use this." Other Mach-O binaries do, and the parity gate (Sprint 27) will surface what we missed.
  • The Mach-O spec + dyld source are the spec. ld behavior is a useful reference, not gospel. Where ld and the documented format disagree, match ld's actual output and document the deviation in the diff-tolerance allowlist.
  • Don't modify tests that reveal real bugs to suit incorrect afs-ld behavior. Fix afs-ld.

Key Technical Decisions

  • No LLVM, no ld64 fork. We own the stack. .refs/llvm/lld/MachO/ is architectural inspiration; we do not link against it.
  • Both LC_DYLD_INFO_ONLY (classic) and LC_DYLD_CHAINED_FIXUPS (modern). Classic first (Sprint 15) so hello-world works on macOS 11+; chained immediately after (Sprint 15.5) so we match the Apple default on macOS 12+. Gate via -fixup_chains / -no_fixup_chains; default depends on -platform_version.
  • Dylib output from day one. The writer is dylib-aware from Sprint 10; the hello-library milestone is Sprint 18.5, not Sprint 25 as originally scoped.
  • Ad-hoc code signing is mandatory. macOS 11+ kills unsigned arm64 binaries at exec time. Sprint 22 ships our own SHA-256 code-signature emitter; we do not shell out to codesign.
  • Owned bytes over borrowed slices, for now. InputSection::data, StringTable::raw, and their peers are Vec<u8>. Input buffers can drop after ObjectFile::parse returns. mmap + borrowed slices arrive in Sprint 28 if profiling justifies the complexity.
  • Ordinals from load-command order. Two-level-namespace ordinals are 1-based positions in LC_*_DYLIB appearance order. Do not renumber on re-export or umbrella expansion.
  • Per-test unique scratch dirs. Cargo runs integration tests in parallel within one process. Two tests writing to /tmp/afs-ld-corpus-{pid}/ race. Use an AtomicUsize counter per test or process-id + thread-id composite. See tests/reader_corpus_round_trip.rs::tempdir for the pattern.

Practical Gotchas

Lessons carved out of actually building the first few sprints:

  • Never rm -rf a directory containing .docs/ without triple-verification. During Sprint 0 we had a near-miss extracting afs-ld into its own repo. Before any destructive operation that touches .docs/, keep a safety copy outside the operation scope, verify the snapshot has the expected file count, and only then proceed. The user has been burned; don't burn them again.
  • Submodule work is three-step. Commit inside afs-ld → push origin/trunk → back in armfortas, git add afs-ld && git commit to bump the pin. Never stop after step one unless explicitly asked.
  • Every new LoadCommand variant needs exhaustive-match updates in at least three places: cmd(), cmdsize(), and src/dump.rs::write_command. The compiler will tell you — listen.
  • #[allow(dead_code)] is a short-lived bridge, not a marker. When a helper is unused because the caller lands in the next commit, #[allow(dead_code)] is acceptable with a comment naming the caller. Remove the allow as soon as the caller lands.
  • Clippy's manual_is_multiple_of / identity_op / unusual_byte_groupings lints fire often on bit-field code. Use .is_multiple_of(8), avoid x | (0 << n), and use constants for unusual bit masks.
  • Integration tests must skip cleanly on missing toolchain. If xcrun as / xcrun clang isn't available, eprintln!("skipping: …") and return. CI on a non-Mac runner must not fail.
  • ASCII helpers for ar_hdr must accept empty → 0. Apple's ar writes all-space date/uid/gid/mode fields on anonymized archives; ascii_decimal returns Ok(0) for an all-whitespace slice.

Key References

  • .refs/llvm/lld/MachO/ — primary architectural reference.
    • Driver.cpp — pipeline shape.
    • InputFiles.cpp — object/archive/dylib parsing.
    • SymbolTable.cpp — resolution and coalescing rules.
    • SyntheticSections.cpp — GOT/stubs/binding opcodes.
    • Arch/ARM64.cpp — reloc arithmetic.
    • Writer.cpp — layout and emission.
  • .refs/ld64/src/ — Apple authoritative. src/ld/ + src/mach_o/.
  • .refs/mold/src/ — performance techniques (mostly ELF; patterns apply).
  • Apple <mach-o/loader.h>, <mach-o/nlist.h>, <mach-o/reloc.h>, <mach-o/arm64/reloc.h> — mirrored numerically in src/macho/constants.rs.
  • dyld open source — bind/rebase/lazy-bind opcode semantics and chained-fixups format.
  • ARM Architecture Reference Manual (ARMv8-A), section C4 — encoding of relocated instructions (ADRP, ADD, LDR, B/BL).

Sprint Roadmap

.docs/sprints/index.md — 32 sprints across 10 phases. Each has an individual markdown file with prerequisites, deliverables, testing strategy, and definition of done. Current completion is in each commit's parent-repo pin bump message.

View source
1 # CLAUDE.md
2
3 Guidance to Claude Code when working in this repository. The parent
4 `armfortas/CLAUDE.md` governs the compiler and applies here too; this file
5 adds linker-specific discipline on top.
6
7 ## Repository Context
8
9 `afs-ld` is a **git submodule** of [ARMFORTAS](https://github.com/FortranGoingOnForty/armfortas),
10 the bespoke ARM64 Fortran compiler. It is the standalone ARM64 Mach-O
11 linker: reads `MH_OBJECT` from `afs-as`, static archives (`.a`), binary
12 dylibs, and TAPI TBD text stubs; emits `MH_EXECUTE` and `MH_DYLIB`.
13
14 It knows nothing about Fortran. The boundary with the compiler is the
15 CLI — an `ld`-compatible flag surface.
16
17 **Rust standard library only.** No `clap`, `serde`, `byteorder`, `object`,
18 `goblin`, `memmap2`, `yaml-rust`. Hand-roll parsers, serializers, and
19 the tiny YAML subset TBD files need. Adding a dependency requires a
20 discussion, a CLAUDE.md update, and a justification.
21
22 ## Build, Test, Lint
23
24 ```bash
25 cargo build -p afs-ld # build linker crate
26 cargo test -p afs-ld # full suite
27 cargo clippy -p afs-ld --all-targets -- -D warnings
28
29 cargo test --lib -p afs-ld # unit tests only
30 cargo test --test <name> -p afs-ld # one integration test file
31 cargo test --test parity_matrix # vs Apple `ld` across the corpus (Sprint 27)
32 cargo test --test hello_world # executable end-to-end (Sprint 18)
33 cargo test --test hello_library # dylib end-to-end (Sprint 18.5)
34 cargo test --test reader_corpus_round_trip # afs-as corpus → byte-identity
35 cargo test --test archive_runtime # libarmfortas_rt.a reality check
36 cargo test --test dylib_integration # clang-built dylib → DylibFile
37 cargo test -p afs-ld -- <substring> # filter by test name
38 ```
39
40 Integration tests shell out to `xcrun as`, `xcrun clang`, `ld`, `otool`,
41 `nm`, `codesign`. They require **macOS on Apple Silicon** and a working
42 Xcode command-line toolchain. Do not stub or skip them — the
43 differential against the system tools is the entire point of the suite.
44 If `xcrun` is missing, tests emit `skipping: ...` and return cleanly.
45 Never rewrite a test to always pass.
46
47 The `--dump*` CLI modes exist for manual inspection:
48
49 ```bash
50 afs-ld --dump input.o # Mach-O header, load commands, sections, symbols, relocs
51 afs-ld --dump-archive libfoo.a # flavor, members, symbol index
52 afs-ld --dump-dylib libfoo.dylib # install_name, dependencies, rpaths, export trie
53 ```
54
55 Every time a new decoder lands, extend the relevant `--dump*` output.
56
57 ## Target
58
59 - **Architecture**: arm64 only. Not arm64e, not arm64_32. Not x86_64.
60 - **OS**: macOS. Mach-O file format, Apple AAPCS64 calling convention.
61 - **Goal**: parity with Apple `ld` for the binaries armfortas produces and
62 the fortsh milestone. Not a toy. Not a subset. The full Mach-O/dyld
63 contract for our use cases.
64
65 ## Design Philosophy
66
67 - **Bespoke.** We write every decoder, encoder, and layout pass. No
68 `object`, no `goblin`, no `mach-object`. When something breaks, we
69 read our code.
70 - **Byte-level round-trip is the invariant.** For every wire structure
71 (header, load commands, sections, symbols, strings, relocations,
72 archive members, export-trie nodes), `parse(write(x)) == x` and
73 `write(parse(bytes)) == bytes` on every corpus fixture. If a
74 structure can't round-trip, it's not done.
75 - **Total control.** `ld` has bugs we can't fix, behaviors we can't
76 observe, and refactors we can't predict. Owning the linker closes the
77 loop on every binary armfortas produces.
78 - **No hidden state.** Every transformation between raw wire form and
79 linker-side form is explicit. Raw bits are preserved until the
80 writer phase reshapes them; `Raw { cmd, cmdsize, data }` is a
81 legitimate forever-variant for load commands we don't decode yet.
82
83 ## Architecture
84
85 Pipeline, end-to-end:
86
87 ```
88 args → inputs → resolve → atomize → layout → apply relocs → synth sections → write → sign
89 │ │ │ │ │ │ │ │ │
90 args.rs input.rs resolve.rs atom.rs layout.rs reloc/arm64.rs synth/*.rs macho/ synth/
91 symbol.rs writer code_sig
92 ```
93
94 ### Module responsibilities
95
96 - **`src/args.rs`** — CLI parser. Hand-rolled, streaming argv scan. No `clap`.
97 - **`src/macho/`** — Mach-O 64 read/write.
98 - `constants.rs`: `LC_*`, `MH_*`, `S_*`, `N_*`, `ARM64_RELOC_*`, `EXPORT_SYMBOL_FLAGS_*`, `PLATFORM_*`. Numeric literals mirroring Apple's `<mach-o/loader.h>` / `<nlist.h>` / `<reloc.h>` / `<arm64/reloc.h>`. **Duplicated from afs-as** rather than cross-crate coupled — each submodule owns its copy so they stay independent.
99 - `reader.rs`: `MachHeader64`, `LoadCommand` enum, per-command structs (`Segment64`, `Section64Header`, `SymtabCmd`, `DysymtabCmd`, `BuildVersionCmd`, `DylibCmd`, `RpathCmd`, `DyldInfoCmd`, `LinkEditDataCmd`). Every variant has `parse(cmdsize, payload)` and `write(&mut Vec<u8>)` paired as round-trip.
100 - `dylib.rs`: `DylibFile::parse` — pulls `LC_ID_DYLIB`, dependency chain, rpaths, and routes export-trie bytes through to `exports.rs`.
101 - `exports.rs`: `ExportTrie`, `ExportKind`, cycle-safe walker with `MAX_DEPTH=128`.
102 - `writer.rs`: emits `MH_EXECUTE` / `MH_DYLIB` (lands at Sprint 10+).
103 - `tbd.rs`: TAPI TBD v4 YAML-subset parser (Sprint 6).
104 - **`src/archive.rs`** — BSD + SysV + GNU-thin static archives. Lazy member fetch via `fetch_object_defining(name)`.
105 - **`src/input.rs`** — `ObjectFile` aggregate: header + load commands + sections + symbols + strings + dysymtab. Sprint 4 adds archive fetching; Sprint 7 introduces `InputFile` enum.
106 - **`src/symbol.rs`** — `RawNlist` (wire form, round-trip) + `InputSymbol` accessors (kind, ext, weak_ref/def, common size/alignment, library ordinal, indirect strx).
107 - **`src/string_table.rs`** — owned `StringTable` with suffix-dedup-aware `strx → &str` lookup.
108 - **`src/section.rs`** — `SectionKind` taxonomy (code / data / zerofill / TLS / literals / stubs / GOT / compact-unwind / eh_frame) derived from `(segname, sectname, flags)`; `InputSection` with data + raw-reloc slices.
109 - **`src/reloc/`** — ARM64 relocs.
110 - `mod.rs`: `RawRelocation` (bit-packed), `Reloc` (fused; ADDEND / SUBTRACTOR prefixes folded into primaries), `parse_relocs` / `write_relocs` (reversible), `validate_relocs` (bounds, referent range, kind-vs-length-vs-pcrel).
111 - `arm64.rs`: reloc application against final addresses (Sprint 11).
112 - `loh.rs`: LOH preservation / relaxation (Sprint 25).
113 - **`src/leb.rs`** — ULEB128/SLEB128 codec reused by export trie, function-starts deltas, dyld opcode streams, chained fixups.
114 - **`src/diag.rs`** — diagnostics. Path + byte offset + caret, matching `afs-as/src/diag*.rs` style. Deterministic output: no wall clock, no pid, no thread-id in error text.
115 - **`src/dump.rs`** — `--dump*` inspection modes. Every time a reader decodes something new, extend the dump.
116 - **`src/driver.rs`** — orchestrator (Sprint 20).
117
118 ## Coding Conventions
119
120 - **Rust, idiomatic.** Use enums for wire structures and linker models.
121 Exhaustive pattern matching everywhere — no catch-all `_` arms
122 outside tests. When a new `LoadCommand` variant lands, every
123 `match` that inspects `LoadCommand` has to grow a new arm; the
124 compiler enforces this and that's the point.
125 - **`unsafe` only where genuinely required.** The one known case is
126 `libc::mmap` for large input files (Sprint 28). Keep blocks small,
127 comment the invariant, and never let unsafe leak across module
128 boundaries.
129 - **Byte-level round-trip for every wire structure.** Don't add a
130 parser without its writer. Don't add a writer without a test that
131 proves `write(parse(x)) == x`. See `src/macho/reader.rs` for the
132 template.
133 - **Raw wire form stays accessible.** A Reloc kind that can't losslessly
134 re-emit its ADDEND prefix is broken. A `Name16` stored as `String`
135 instead of `[u8; 16]` loses null padding and breaks byte-identity.
136 Preserve the wire.
137 - **Constants duplicated, not imported.** `afs-ld/src/macho/constants.rs`
138 mirrors Apple's headers numerically. Do not depend on `afs-as` at a
139 type level. Submodule independence matters more than deduplication.
140 - **Diagnostics cite input + offset + caret.** Every parser error goes
141 through `ReadError` / `ArchiveError` variants with explicit
142 `at_offset` / `context` / `reason`. No `String::from("something
143 broke")` noise.
144 - **Commit discipline**: terse imperative messages, no co-authors,
145 per-file / per-chunk commits, never monoliths. No sprint-number
146 references in commit subjects. Commit often — "if I had to
147 bisect this sprint, which revision would I want to land on?" is
148 the granularity.
149 - **Tests alongside code.** Every new decoder lands with unit tests in
150 the same commit. Every bug fix gets a regression test. Integration
151 tests that assemble/link real fixtures at test time always skip
152 cleanly when `xcrun` is unavailable — never hard-fail on missing
153 toolchain.
154 - **No "stubs pass silently."** Placeholder code that returns wrong
155 answers is worse than code that panics. If a kind is "not yet
156 implemented", say so with a hard error and a pointer to the sprint
157 that'll finish it. Tests must catch the stub, not paper over it.
158 - **Don't cut corners without stopping to discuss.** If you find
159 yourself about to skip a round-trip test, hardcode a "good enough"
160 offset, or silently accept a malformed input, stop and talk it
161 through. We are not building a toy linker.
162 - **Avoid rushing through sprints to get to an audit.** The hard work
163 of each sprint is the point of the sprint. The audit is the
164 downstream check, not the goal.
165 - **Always opt for the robust solution.** If you find yourself saying
166 "the simple solution is X", stop and ask whether a production linker
167 uses the simple solution or digs deeper. The simple one might be
168 right — but we have to be sure this is not a toy.
169 - **When unsure, consult `.refs/`.** `.refs/llvm/lld/MachO/` is the
170 primary architectural reference; `.refs/ld64/src/` is Apple's
171 authoritative implementation for parity edge cases; `.refs/mold/src/`
172 informs performance choices. Read before inventing.
173 - **Run long test jobs judiciously.** Think about the grep/filter before
174 you launch — `cargo test -p afs-ld -- reader_` beats
175 `cargo test --workspace` when you only touched the reader. The
176 corpus round-trip is fast (~1s); the parity matrix (Sprint 27) won't
177 be.
178
179 ## Test Architecture
180
181 Layered, not a single golden path. Each layer catches a different class
182 of regression:
183
184 | Test file | What it proves |
185 |--------------------------------------|----------------|
186 | `src/**/#[cfg(test)]` | Unit tests per decoder / encoder / accessor |
187 | `tests/common/harness.rs` | Differential harness — afs-ld vs system `ld`, tolerated-diff classifier |
188 | `tests/reader_corpus_round_trip.rs` | Full afs-as corpus → byte-identity for header + LC region, symtab, strtab, relocs |
189 | `tests/archive_runtime.rs` | Real `libarmfortas_rt.a` parses; symbol index resolves |
190 | `tests/dylib_integration.rs` | `xcrun clang` dylib → `DylibFile`, install_name + exports + libSystem dep |
191 | `tests/reader_empty.rs` | CLI contract: empty argv → `afs-ld: error: no input files`, exit 2 |
192 | `tests/diff_harness_sanity.rs` | Harness zero-diffs on identical inputs |
193 | `tests/diff_harness_finds_critical.rs` | Harness catches intentional byte differences |
194 | `tests/hello_world.rs` | Executable end-to-end (Sprint 18) |
195 | `tests/hello_library.rs` | Dylib end-to-end (Sprint 18.5) |
196 | `tests/parity_matrix.rs` | Corpus byte-level differential vs Apple `ld` (Sprint 27) |
197 | `tests/armfortas_integration.rs` | Parent's integration suite under `AFS_LD=1` (Sprint 21) |
198
199 Corpus fixtures live in `tests/corpus/`. Every new relocation kind,
200 section kind, or CLI flag lands a corpus entry in the same sprint.
201
202 ## Audit Discipline
203
204 After each sprint, a brutally honest audit:
205
206 - Assume nothing works until proven otherwise. Test every claim.
207 Re-run the whole test suite, not just the freshly-written tests.
208 - "Placeholder" and "stub" are synonyms for "broken." Wrong answers
209 silently produced are worse than crashes.
210 - Check against the Mach-O ABI spec, not just "does it link." Wrong
211 output corrupts dyld's bind/rebase state at runtime; the kernel or
212 loader will tell you, hours later, in ways that are hard to
213 bisect.
214 - Don't soften findings. **Major** means "produces wrong binaries."
215 **Critical** means "silent corruption that dyld will accept."
216 - No deferred items unless they genuinely require a later sprint.
217 Fix it now if it can be fixed now. A deferred item accumulates
218 debt with compound interest.
219 - The audit is not a formality. It is the last line of defense
220 before bad linker output gets merged and ships bad binaries
221 downstream.
222
223 ## Completeness Philosophy
224
225 Every Mach-O feature modern dyld consumes is in scope. When implementing
226 a decoder or encoder:
227
228 - Implement it fully, not just the subset armfortas or fortsh happens to
229 exercise. Every `ARM64_RELOC_*` kind, every `EXPORT_SYMBOL_FLAGS_*`
230 terminal form, every `LC_*_DYLIB` variant.
231 - Don't defer features with "fortsh doesn't use this." Other Mach-O
232 binaries do, and the parity gate (Sprint 27) will surface what we
233 missed.
234 - The Mach-O spec + `dyld` source are the spec. `ld` behavior is a
235 useful reference, not gospel. Where `ld` and the documented format
236 disagree, match `ld`'s actual output and document the deviation in
237 the diff-tolerance allowlist.
238 - Don't modify tests that reveal real bugs to suit incorrect afs-ld
239 behavior. Fix afs-ld.
240
241 ## Key Technical Decisions
242
243 - **No LLVM, no `ld64` fork.** We own the stack. `.refs/llvm/lld/MachO/`
244 is architectural inspiration; we do not link against it.
245 - **Both `LC_DYLD_INFO_ONLY` (classic) and `LC_DYLD_CHAINED_FIXUPS`
246 (modern).** Classic first (Sprint 15) so hello-world works on
247 macOS 11+; chained immediately after (Sprint 15.5) so we match the
248 Apple default on macOS 12+. Gate via `-fixup_chains` /
249 `-no_fixup_chains`; default depends on `-platform_version`.
250 - **Dylib output from day one.** The writer is dylib-aware from
251 Sprint 10; the hello-library milestone is Sprint 18.5, not
252 Sprint 25 as originally scoped.
253 - **Ad-hoc code signing is mandatory.** macOS 11+ kills unsigned arm64
254 binaries at exec time. Sprint 22 ships our own SHA-256 code-signature
255 emitter; we do not shell out to `codesign`.
256 - **Owned bytes over borrowed slices, for now.** `InputSection::data`,
257 `StringTable::raw`, and their peers are `Vec<u8>`. Input buffers
258 can drop after `ObjectFile::parse` returns. `mmap` + borrowed slices
259 arrive in Sprint 28 if profiling justifies the complexity.
260 - **Ordinals from load-command order.** Two-level-namespace ordinals
261 are 1-based positions in `LC_*_DYLIB` appearance order. Do not
262 renumber on re-export or umbrella expansion.
263 - **Per-test unique scratch dirs.** Cargo runs integration tests in
264 parallel within one process. Two tests writing to
265 `/tmp/afs-ld-corpus-{pid}/` race. Use an `AtomicUsize` counter
266 per test or process-id + thread-id composite. See
267 `tests/reader_corpus_round_trip.rs::tempdir` for the pattern.
268
269 ## Practical Gotchas
270
271 Lessons carved out of actually building the first few sprints:
272
273 - **Never `rm -rf` a directory containing `.docs/` without triple-verification.**
274 During Sprint 0 we had a near-miss extracting afs-ld into its own
275 repo. Before any destructive operation that touches `.docs/`, keep
276 a safety copy outside the operation scope, verify the snapshot has
277 the expected file count, and only then proceed. The user has been
278 burned; don't burn them again.
279 - **Submodule work is three-step.** Commit inside afs-ld → push
280 origin/trunk → back in armfortas, `git add afs-ld && git commit`
281 to bump the pin. Never stop after step one unless explicitly asked.
282 - **Every new `LoadCommand` variant needs exhaustive-match updates in
283 at least three places**: `cmd()`, `cmdsize()`, and
284 `src/dump.rs::write_command`. The compiler will tell you — listen.
285 - **`#[allow(dead_code)]` is a short-lived bridge, not a marker.**
286 When a helper is unused because the caller lands in the next commit,
287 `#[allow(dead_code)]` is acceptable with a comment naming the caller.
288 Remove the allow as soon as the caller lands.
289 - **Clippy's `manual_is_multiple_of` / `identity_op` /
290 `unusual_byte_groupings` lints fire often on bit-field code.** Use
291 `.is_multiple_of(8)`, avoid `x | (0 << n)`, and use constants for
292 unusual bit masks.
293 - **Integration tests must skip cleanly on missing toolchain.** If
294 `xcrun as` / `xcrun clang` isn't available, `eprintln!("skipping:
295 …")` and return. CI on a non-Mac runner must not fail.
296 - **ASCII helpers for `ar_hdr` must accept empty → 0.** Apple's `ar`
297 writes all-space `date/uid/gid/mode` fields on anonymized archives;
298 `ascii_decimal` returns `Ok(0)` for an all-whitespace slice.
299
300 ## Key References
301
302 - `.refs/llvm/lld/MachO/` — primary architectural reference.
303 - `Driver.cpp` — pipeline shape.
304 - `InputFiles.cpp` — object/archive/dylib parsing.
305 - `SymbolTable.cpp` — resolution and coalescing rules.
306 - `SyntheticSections.cpp` — GOT/stubs/binding opcodes.
307 - `Arch/ARM64.cpp` — reloc arithmetic.
308 - `Writer.cpp` — layout and emission.
309 - `.refs/ld64/src/` — Apple authoritative. `src/ld/` + `src/mach_o/`.
310 - `.refs/mold/src/` — performance techniques (mostly ELF; patterns apply).
311 - Apple `<mach-o/loader.h>`, `<mach-o/nlist.h>`, `<mach-o/reloc.h>`,
312 `<mach-o/arm64/reloc.h>` — mirrored numerically in
313 `src/macho/constants.rs`.
314 - `dyld` open source — bind/rebase/lazy-bind opcode semantics and
315 chained-fixups format.
316 - ARM Architecture Reference Manual (ARMv8-A), section C4 — encoding
317 of relocated instructions (ADRP, ADD, LDR, B/BL).
318
319 ## Sprint Roadmap
320
321 `.docs/sprints/index.md` — 32 sprints across 10 phases. Each has an
322 individual markdown file with prerequisites, deliverables, testing
323 strategy, and definition of done. Current completion is in each
324 commit's parent-repo pin bump message.