bensch Public

Watch 0 Fork 0 Star 0
markdown · 2191 bytes Raw Blame History

YAML Test Specification Format

bensch interactive tests are defined in YAML files. Each file contains a list of tests with steps and expected output.

Structure

metadata:
  category: "History"
  description: "Tests for command history navigation"

tests:
  - name: "Up arrow recalls previous command"
    steps:
      - send_line: "echo hello"
      - send_key: "Up"
      - send_key: "Enter"
    expect_output: "hello"
    match_type: "contains"

Step Types

Step Description Example
send_line Send text + Enter send_line: "echo hello"
send Send text without Enter send: "partial"
send_key Send a named key send_key: "Up"
send_keys Send multiple keys send_keys: ["C-a", "C-k"]
wait Sleep for seconds wait: 0.3
wait_for_prompt Wait for shell prompt wait_for_prompt: true
expect Wait for pattern in output expect: "pattern"
resize Change terminal size resize: {rows: 24, cols: 80}

Key Names

Standard terminal keys are available via name:

  • Arrow keys: Up, Down, Left, Right
  • Control keys: C-a through C-z, C-c, C-d, C-r, C-l
  • Alt keys: M-f, M-b, M-d, M-.
  • Special: Enter, Tab, Escape, Backspace, Delete
  • Function keys: F1 through F12
  • Navigation: Home, End, PageUp, PageDown

Match Types

Type Description
contains Output contains the expected string (default)
exact Output exactly matches
regex Output matches regex pattern

Test Options

Option Description Default
fresh_session Force a new PTY session false
env Extra environment variables {}
rc_file RC file path /dev/null
expect_not Output must NOT contain this —

Example: Tab Completion Test

- name: "Tab completes filename"
  steps:
    - send_line: "cd /tmp"
    - send_line: "touch testfile.txt"
    - send: "ls testf"
    - send_key: "Tab"
    - send_key: "Enter"
  expect_output: "testfile.txt"
  match_type: "contains"
View source
1 # YAML Test Specification Format
2
3 bensch interactive tests are defined in YAML files. Each file contains a list of tests with steps and expected output.
4
5 ## Structure
6
7 ```yaml
8 metadata:
9 category: "History"
10 description: "Tests for command history navigation"
11
12 tests:
13 - name: "Up arrow recalls previous command"
14 steps:
15 - send_line: "echo hello"
16 - send_key: "Up"
17 - send_key: "Enter"
18 expect_output: "hello"
19 match_type: "contains"
20 ```
21
22 ## Step Types
23
24 | Step | Description | Example |
25 |------|-------------|---------|
26 | `send_line` | Send text + Enter | `send_line: "echo hello"` |
27 | `send` | Send text without Enter | `send: "partial"` |
28 | `send_key` | Send a named key | `send_key: "Up"` |
29 | `send_keys` | Send multiple keys | `send_keys: ["C-a", "C-k"]` |
30 | `wait` | Sleep for seconds | `wait: 0.3` |
31 | `wait_for_prompt` | Wait for shell prompt | `wait_for_prompt: true` |
32 | `expect` | Wait for pattern in output | `expect: "pattern"` |
33 | `resize` | Change terminal size | `resize: {rows: 24, cols: 80}` |
34
35 ## Key Names
36
37 Standard terminal keys are available via name:
38
39 - **Arrow keys**: `Up`, `Down`, `Left`, `Right`
40 - **Control keys**: `C-a` through `C-z`, `C-c`, `C-d`, `C-r`, `C-l`
41 - **Alt keys**: `M-f`, `M-b`, `M-d`, `M-.`
42 - **Special**: `Enter`, `Tab`, `Escape`, `Backspace`, `Delete`
43 - **Function keys**: `F1` through `F12`
44 - **Navigation**: `Home`, `End`, `PageUp`, `PageDown`
45
46 ## Match Types
47
48 | Type | Description |
49 |------|-------------|
50 | `contains` | Output contains the expected string (default) |
51 | `exact` | Output exactly matches |
52 | `regex` | Output matches regex pattern |
53
54 ## Test Options
55
56 | Option | Description | Default |
57 |--------|-------------|---------|
58 | `fresh_session` | Force a new PTY session | `false` |
59 | `env` | Extra environment variables | `{}` |
60 | `rc_file` | RC file path | `/dev/null` |
61 | `expect_not` | Output must NOT contain this | — |
62
63 ## Example: Tab Completion Test
64
65 ```yaml
66 - name: "Tab completes filename"
67 steps:
68 - send_line: "cd /tmp"
69 - send_line: "touch testfile.txt"
70 - send: "ls testf"
71 - send_key: "Tab"
72 - send_key: "Enter"
73 expect_output: "testfile.txt"
74 match_type: "contains"
75 ```