tenseleyflow/bencch / 59ad34a

Browse files

Broaden standalone workflow coverage

Authored by espadonne
SHA
59ad34ac806c99d62b5f713a83045e83b44267f1
Parents
e080c62
Tree
b8706cb

2 changed files

StatusFile+-
M .github/workflows/standalone-bootstrap.yml 16 0
M README.md 12 0
.github/workflows/standalone-bootstrap.ymlmodified
@@ -69,6 +69,22 @@ jobs:
6969
           cargo run --manifest-path .bencch-external/Cargo.toml -p afs-tests --bin bencch -- run --suite v2/generic-introspect --case fake_compiler_runtime --all > run.txt
7070
           grep "PASS   v2/generic-introspect::fake_compiler_runtime\\[O0\\]" run.txt
7171
72
+      - name: Verify external-only generic compare suite run
73
+        run: |
74
+          cargo run --manifest-path .bencch-external/Cargo.toml -p afs-tests --bin bencch -- run --suite v2/generic-compare --case fake_compilers_match_matrix --all > compare-suite.txt
75
+          grep "PASS   v2/generic-compare::fake_compilers_match_matrix\\[O0\\]" compare-suite.txt
76
+          grep "PASS   v2/generic-compare::fake_compilers_match_matrix\\[O1\\]" compare-suite.txt
77
+          grep "PASS   v2/generic-compare::fake_compilers_match_matrix\\[O2\\]" compare-suite.txt
78
+
79
+      - name: Verify external-only generic failure suites
80
+        run: |
81
+          cargo run --manifest-path .bencch-external/Cargo.toml -p afs-tests --bin bencch -- run --suite v2/generic-failures --case fake_compiler_expected_diagnostic --all > failure-suite.txt
82
+          grep "PASS   v2/generic-failures::fake_compiler_expected_diagnostic\\[O0\\]" failure-suite.txt
83
+          cargo run --manifest-path .bencch-external/Cargo.toml -p afs-tests --bin bencch -- run --suite v2/generic-failure-matrix --case fake_compiler_expected_diagnostic_matrix --all > failure-matrix.txt
84
+          grep "PASS   v2/generic-failure-matrix::fake_compiler_expected_diagnostic_matrix\\[O0\\]" failure-matrix.txt
85
+          grep "PASS   v2/generic-failure-matrix::fake_compiler_expected_diagnostic_matrix\\[O1\\]" failure-matrix.txt
86
+          grep "PASS   v2/generic-failure-matrix::fake_compiler_expected_diagnostic_matrix\\[O2\\]" failure-matrix.txt
87
+
7288
       - name: Verify linked-only suite guidance
7389
         run: |
7490
           if cargo run --manifest-path .bencch-external/Cargo.toml -p afs-tests --bin bencch -- run --suite frontend --case stage_walk --all > legacy.txt 2>&1; then
README.mdmodified
@@ -57,6 +57,18 @@ Example external-only authored suite run:
5757
 cargo run --manifest-path .bencch-external/Cargo.toml -p afs-tests --bin bencch -- run --suite v2/generic-introspect --case fake_compiler_runtime --all
5858
 ```
5959
60
+Example external-only authored compare matrix:
61
+
62
+```bash
63
+cargo run --manifest-path .bencch-external/Cargo.toml -p afs-tests --bin bencch -- run --suite v2/generic-compare --case fake_compilers_match_matrix --all
64
+```
65
+
66
+Example external-only authored failure matrix:
67
+
68
+```bash
69
+cargo run --manifest-path .bencch-external/Cargo.toml -p afs-tests --bin bencch -- run --suite v2/generic-failure-matrix --case fake_compiler_expected_diagnostic_matrix --all
70
+```
71
+
6072
 Legacy rich-stage suites still need linked capture. In an external-only build,
6173
 they now fail early with a direct message telling you to use
6274
 `scripts/bootstrap-linked-armfortas.sh`.