tenseleyflow/bencch / a45c6fb

Browse files

Expand standalone external coverage

Authored by espadonne
SHA
a45c6fbe74410a8115986e5c86e40c9afe2dc97c
Parents
3088db5
Tree
c98150b

4 changed files

StatusFile+-
M .github/workflows/standalone-bootstrap.yml 21 1
M README.md 16 0
M scripts/bootstrap-linked-armfortas.sh 5 0
M scripts/bootstrap-standalone-external.sh 5 0
.github/workflows/standalone-bootstrap.ymlmodified
@@ -53,7 +53,27 @@ jobs:
5353
           grep "armfortas_capture_mode: unavailable" doctor.txt
5454
           grep "linked capture is unavailable in this build" doctor.txt
5555
56
+      - name: Verify external-only introspect
57
+        run: |
58
+          cargo run --manifest-path .bencch-external/Cargo.toml -p afs-tests --bin bencch -- introspect fixtures/fake_compilers/match_42_a.sh fixtures/runtime/if_else.f90 --artifact asm,runtime > introspect.txt
59
+          grep "backend_mode: external-driver" introspect.txt
60
+          grep "requested_artifacts: asm, runtime" introspect.txt
61
+
5662
       - name: Verify external-only compare
5763
         run: |
5864
           cargo run --manifest-path .bencch-external/Cargo.toml -p afs-tests --bin bencch -- compare fixtures/fake_compilers/match_42_a.sh fixtures/fake_compilers/match_42_b.sh --program fixtures/runtime/if_else.f90 > compare.txt
59
-          grep "status: MATCH" compare.txt
65
+          grep "status: match" compare.txt
66
+
67
+      - name: Verify external-only generic suite run
68
+        run: |
69
+          cargo run --manifest-path .bencch-external/Cargo.toml -p afs-tests --bin bencch -- run --suite v2/generic-introspect --case fake_compiler_runtime --all > run.txt
70
+          grep "PASS   v2/generic-introspect::fake_compiler_runtime\\[O0\\]" run.txt
71
+
72
+      - name: Verify linked-only suite guidance
73
+        run: |
74
+          if cargo run --manifest-path .bencch-external/Cargo.toml -p afs-tests --bin bencch -- run --suite frontend --case stage_walk --all > legacy.txt 2>&1; then
75
+            echo "expected linked-only suite to fail in external-only build" >&2
76
+            exit 1
77
+          fi
78
+          grep "case requires linked armfortas capture" legacy.txt
79
+          grep "scripts/bootstrap-linked-armfortas.sh" legacy.txt
README.mdmodified
@@ -45,6 +45,22 @@ cargo run --manifest-path .bencch-external/Cargo.toml -p afs-tests --bin bencch
4545
 That mode drops linked capture entirely and keeps the generic external-driver
4646
 surface available for `compare`, `introspect`, and external-facing `run` work.
4747
48
+Example external-only introspection:
49
+
50
+```bash
51
+cargo run --manifest-path .bencch-external/Cargo.toml -p afs-tests --bin bencch -- introspect fixtures/fake_compilers/match_42_a.sh fixtures/runtime/if_else.f90 --artifact asm,runtime
52
+```
53
+
54
+Example external-only authored suite run:
55
+
56
+```bash
57
+cargo run --manifest-path .bencch-external/Cargo.toml -p afs-tests --bin bencch -- run --suite v2/generic-introspect --case fake_compiler_runtime --all
58
+```
59
+
60
+Legacy rich-stage suites still need linked capture. In an external-only build,
61
+they now fail early with a direct message telling you to use
62
+`scripts/bootstrap-linked-armfortas.sh`.
63
+
4864
 ## Usage
4965
5066
 List suites:
scripts/bootstrap-linked-armfortas.shmodified
@@ -49,6 +49,9 @@ if [[ ! -d "${armfortas_root}/src" ]]; then
4949
 fi
5050
5151
 mkdir -p "${output_dir}/bench"
52
+ln -sfn ../suites "${output_dir}/suites"
53
+ln -sfn ../fixtures "${output_dir}/fixtures"
54
+mkdir -p "${output_dir}/reports"
5255
5356
 cat > "${output_dir}/Cargo.toml" <<EOF
5457
 # Generated by scripts/bootstrap-linked-armfortas.sh
@@ -93,6 +96,8 @@ cat > "${output_dir}/README.md" <<EOF
9396
 This directory is generated by scripts/bootstrap-linked-armfortas.sh.
9497
9598
 armfortas checkout: ${armfortas_root}
99
+suite root: ${repo_root}/suites
100
+fixture root: ${repo_root}/fixtures
96101
97102
 Use it with:
98103
scripts/bootstrap-standalone-external.shmodified
@@ -32,6 +32,9 @@ output_dir="${1:-${repo_root}/.bencch-external}"
3232
 output_dir="$(mkdir -p "${output_dir}" && cd "${output_dir}" && pwd)"
3333
3434
 mkdir -p "${output_dir}/bench"
35
+ln -sfn ../suites "${output_dir}/suites"
36
+ln -sfn ../fixtures "${output_dir}/fixtures"
37
+mkdir -p "${output_dir}/reports"
3538
3639
 cat > "${output_dir}/Cargo.toml" <<EOF
3740
 # Generated by scripts/bootstrap-standalone-external.sh
@@ -75,6 +78,8 @@ cat > "${output_dir}/README.md" <<EOF
7578
 This directory is generated by scripts/bootstrap-standalone-external.sh.
7679
7780
 It builds bencch without linked armfortas capture.
81
+suite root: ${repo_root}/suites
82
+fixture root: ${repo_root}/fixtures
7883
7984
 Use it with:
8085