tenseleyflow/bensch / 04a2ed2

-# bensch — Project Overview

-

-## What

-

-bensch is a POSIX shell testing framework extracted from the [fortsh](https://github.com/FortranGoingOnForty/fortsh) project. It tests any shell binary for POSIX compliance, interactive feature correctness, and builtin command compatibility.

-

-## Why

-

-The fortsh project accumulated ~1014 interactive PTY tests and ~3800 POSIX compliance assertions as part of its CI. 93% of these tests are shell-agnostic — they test POSIX behavior and readline-compatible features, not fortsh-specific functionality. Extracting them into a standalone framework lets other shell projects benefit from the same rigor.

-

-## Origin

-

-The test infrastructure was developed over 2025-2026 during fortsh's push to achieve full cross-platform interactive parity (x86 Linux, ARM64 Linux, macOS ARM64). The extraction happened in April 2026 after fortsh v1.6.0 achieved 1014/1014 interactive tests passing on all three platforms.

-

-## Architecture Decisions

-

-**Fresh repo, not filter-branch.** fortsh's git history is about a Fortran compiler project. Carrying 1700+ commits of compiler work into a test framework repo would be noise. Each file was brought over as its own commit, preserving provenance in commit messages.

-

-**Shell profiles over runtime detection.** Rather than auto-detecting shell capabilities at runtime (fragile, slow), bensch uses YAML profiles that declaratively describe what each shell supports. This is explicit, testable, and easy to extend.

-

-**Comparison against reference shell.** POSIX tests don't check against a POSIX specification document — they compare against bash (or any reference shell). This is pragmatic: bash IS the de facto standard, and discrepancies between your shell and bash are the actionable findings.

-

-**Separate framework from test suites.** The Python PTY engine (`framework/`) is fully generic. The test specifications (`suites/`) are organized by portability level (POSIX-generic, bash-extended, fortsh-specific). Adding tests for a new shell means adding a profile and optionally a new suite directory.

-

-## Maintenance

-

-bensch and fortsh will co-evolve. Bugs found through testing fortsh will be fixed in bensch's framework code. New test cases written for fortsh's interactive features will be added to bensch if they're shell-agnostic.

-

-The sync is manual but low-friction: both repos share the same test specification format (YAML) and the same Python framework API. A diff between `fortsh/tests/interactive/` and `bensch/framework/` + `bensch/suites/interactive/` will show divergence.