Visual terminal bench
This is the real Ink app inside a POSIX PTY, displayed by xterm.js in a browser. It is not a reimplementation of the UI. Browser keyboard and mouse reports travel back through the PTY. The scripted provider exercises todos, delegation, questions, approvals, review and independent agents; no external model or real project credentials are used.
Requires Node >=22.19.0, Bun, Python 3, and a POSIX host (Linux/macOS).
npm install
npx playwright install chromium
npm run visual:check
The check starts a loopback-only server, opens headless Chromium, exercises
the UI, captures screenshots, and stops its server/processes. Browser
installation may require npx playwright install --with-deps chromium on
minimal Linux hosts. An existing compatible Chromium can be selected with
PLAYWRIGHT_CHROMIUM_EXECUTABLE=/absolute/path/to/chrome.
The emulator explicitly uses the stable Unicode 11 scalar-width provider before receiving PTY output. This matches the width-two emoji/CJK and combining-mark profile exercised by the native checks; xterm's default Unicode 6 profile treats modern emoji as one cell and can leave stale glyphs after valid cursor updates. The raw-ANSI regression checks both profiles against the same writes. This does not establish support for every extended emoji or grapheme-cluster sequence.
node tools/visual/opentui-question-queue-check.mjs checks the actual OpenTUI
question flow with a loopback provider. It queues a follow-up while a selected
answer is pending, verifies no model request is sent early, and checks the answer
and subsequent queued turn. It does not exercise a paid model or native terminal.
node tools/visual/opentui-recovery-check.mjs runs the actual OpenTUI CLI
against an isolated loopback provider, verifies that unapplied steering survives
restart and retries only after a user command. VISUAL_EXECUTABLE selects a
relocated installed executable and the report records its SHA-256. The SIGKILL
and interrupted-retry variants also run against each native release archive. Set VISUAL_OPENTUI_QUEUE=1 to
check the Enter-to-queue path instead.
Set VISUAL_OPENTUI_QUEUE=1 together with VISUAL_OPENTUI_RECOVERY_CURRENT=1
to interrupt a queued message, review it without restarting, and verify that
only an explicit retry invokes the model. This scenario also runs on each
installed native release archive and in Node 26 CI.
Set VISUAL_OPENTUI_QUEUE=1 with VISUAL_OPENTUI_RECOVERY_ORDER=1 to verify
the displayed order of a steering message and a queued message before and after
restart without issuing a second model request. This also runs in CI and on
each native release archive.
With that mixed-order scenario, set VISUAL_EXECUTABLE to an older native
archive executable and VISUAL_RESTART_EXECUTABLE to the newer executable to
exercise a legacy journal through an upgrade. The check requires old intents
without ordering metadata, verifies the replacement executable, then checks old
and new messages across a second restart. Both executables must be complete,
relocated archives. The report records their hashes; it does not certify other
versions or platforms.
Set VISUAL_OPENTUI_PROCESS_KILL=1
to kill the CLI with SIGKILL after its input is persisted and verify explicit
recovery after a fresh start; the report retains the observed exit code. The
mixed-order scenario can combine this option with VISUAL_OPENTUI_QUEUE=1 and
VISUAL_OPENTUI_RECOVERY_ORDER=1 to kill the active process after both intents
are journaled and verify their order without an implicit model request. The
same check retries the first message explicitly and verifies that the other
remains available. It runs in Node 26 CI and against each native release archive.
The queued-input variant runs in Node 26 CI. Set
VISUAL_OPENTUI_RECOVERY_INTERRUPT=1
to hold the retry journal write and verify that Ctrl+C reports the unaccepted retry
without sending a model request. This interrupted-retry variant also runs in
Node 26 CI and retains its report and screenshots. The scenarios use a controlled
provider and local test plugin, not a native terminal or paid model.
node tools/visual/queue-failure-check.mjs runs the Ink app with an isolated
journal denial. It verifies that an unaccepted queued message returns to the
editor. Set VISUAL_INPUT_WRITE_FAIL=1 to check the first user turn instead.
This is a controlled storage failure, not a native crash test.
node tools/visual/recovery-check.mjs runs the actual Ink application with an
isolated scripted provider. It queues a user message during a held turn,
restarts the PTY session, checks the recovery warning and list, and explicitly
retries the message. Set VISUAL_RECOVERY_CURRENT=1 to interrupt and review an
accepted queued message without restarting; this variant runs in Node 26 CI.
Set VISUAL_STEER_RECOVERY=1 to exercise unapplied steering. The report and screenshots do not prove process-kill or
native-terminal behavior; older journal entries without IDs remain ambiguous.
The standard visual:check smoke uses its source ScriptedProvider and rejects
VISUAL_EXECUTABLE. For native Ink window verification use visual:windows,
which configures its isolated loopback approval/question fixtures. Actual-app
OpenTUI scenarios below also accept a complete relocated native archive.
npm run visual:ink-background runs the source Ink frontend in six isolated
PTY sessions. It checks automatic light/dark terminal reports and verifies that
explicit background preferences remain authoritative after a conflicting reply.
It also checks emphasis in automatic and forced 16-color light and dark modes.
The browser emulates the terminal reports; the report and screenshots do not
establish native support for OSC 11 in any particular terminal.
The standard scenario also checks the semantic agent overview: expand by mouse and keyboard, click state tabs, switch with Left/Right, resize to stacked narrow fields and enter the selected agent's conversation. Screenshots include the expanded and narrow views.
npm run visual:opentui-main-draft verifies a built-in Main draft across a real
PTY restart with an isolated loopback provider, then checks that explicit Enter
sends it once. Set VISUAL_DRAFT_SCENARIO=review, review_mouse, conflict, conflict_many, conflict_mouse,
rapid, quit, or kill to exercise ambiguous input and conflict preview/selection. These
checks do not establish power-loss durability or native terminal behavior.
VISUAL_EXECUTABLE selects a relocated native archive executable; the report
verifies and records its SHA-256.
npm run visual:opentui-app runs the actual application entry point with
--renderer opentui, isolated saved profiles and a bounded loopback HTTP model.
It does not inject a buildApp provider. It verifies pending Main question/draft
state while native provider/model/key/proxy forms complete, actual model and
authorization selection on the next HTTP request, independent form lifetime
during foreground interruption, and exact layout plus session restoration with
--resume. Synthetic keys must be absent from terminal output, provider bodies,
session journals and non-secret profile settings; diagnostic raw keyboard logs
are deliberately excluded. No user profiles or credentials are read.
With VISUAL_APP_DELAY_ANSWER=1, the loopback fixture streams the answer before
delaying its finish; the scenario checks Main is running, then waits for ready
before submitting the next task. This exercises the turn-settlement boundary.
Set VISUAL_EXECUTABLE=/absolute/path/to/extracted/compositor for the same proof
against a native archive. The report records the executable identity. The older
visual:opentui, visual:opentui-windows and visual:opentui-inputs commands keep
their isolated preview fixtures and test different integration boundaries.
npm run visual:opentui-managers checks the actual OpenTUI application with an
offline local catalog and the normal configured loopback provider. While Main
waits for an answer, terminal actions browse the canonical manager panels,
register the catalog, install a package inactive, explicitly enable it and queue
reload. A controlled local catalog update is discovered and queued while Main
is busy; installed bytes stay at the old version until the operation finishes,
then the new version and enabled state survive restart. The plugin's own storage receipt and pane establish when it actually
loads. The scenario also keeps a deliberately closed plugin window closed across
reload, exercises private placement Apply/Save rule/Cancel and restores the saved
rule after a real CLI restart. It checks that a failed deferred removal is reported
without preventing valid changes. It also checks that Ctrl+C cancels a prompt waiting
for plugin changes without closing the app or executing the prompt after reload.
After plugin management, the scenario closes Main, reopens the same window with
Finder, and checks that its saved draft and conversation survive a CLI restart
without an implicit model request.
/windows must remain private host navigation without a
model request. No fixture permission rules or live model services are used.
Artifacts default to dist/visual-opentui-managers/; inspect the PNGs and report.
Set VISUAL_EXECUTABLE to an executable inside a complete extracted archive for
the same native check. The bench preserves CI and records native executable
and renderer hashes.
Before reopening Main's pending answer after manager navigation, the check waits
for its focused native footer, draft and Edit answer button inside the same pane.
The footer paints after the focus-dependent dock moves, so an early title update
cannot admit coordinates from the previous unfocused layout. Mouse targets also
wait outside synchronized output and retain the same completed-write count and
pane text across the coordinate stability check; the real click and saved-answer
assertion remain unchanged.
npm run visual:opentui-navigation checks the actual configured OpenTUI shell's
stable Arrange mode, private native finder, protected Main questions/approvals,
resident form/editor state and route lifetime. The scenario verifies current
workspace window strips and cross-workspace restoration through F2, all nine
workspace mouse targets at 38 columns, and exact saved split-tree/ratio restoration
after restart. Arrange checks identify the actual nested keyboard split and its
painted divider, distinguish saved proportions from cell allocation, exercise both
focused-child resize directions, and explain hidden or absent targets. Pointer
resizing still uses the real terminal path. An additional narrow-tiling flow uses
ordinary settings and Unicode windows on a temporary workspace: two columns fit
at 60×24, two rows fit at 38×24, and insufficient height at 38×16 projects only
the focused window. It checks actual minimum 24×6 tile bounds, visible divider
geometry, pointer and keyboard resize, cancellation, canonical tree preservation,
and the still-pending Main question and private form. Widening and the final CLI
restart retain the exact saved tree and fractions. A focus-publication flow uses
abort-aware file gates to finish ordinary public SDK producers after the user
selects Main, edits a private form, enters Arrange/Finder or changes workspace.
Passive first views and late commands must preserve that intent and native cursor;
explicit focus hints and configured rules keep their opt-in/opt-out behavior.
The same controlled gate proves a still-selected retained Finder route can open,
while an abandoned route cannot create a late window. Gate files control fixture
completion only; all user interaction still crosses the actual terminal.
The inventory-refresh fixture also waits at a gate until Finder has displayed the
still-available pane; releasing it then verifies that the pane becomes unavailable.
The ninth flow publishes three ordinary windows on workspace7 through the public
plugin API. Shift+direction must retain center swapping; Ctrl+Arrow in Arrange
must edge-drop the focused window, retain the unaffected outer ratio, resize only
the new nearest split and do nothing when no directional neighbor exists. Main's
pending Production answer, parked draft/cursor and independent form stay intact.
The resulting keyboard-created topology is retained through the same reload and
restart checks and inspected again from the actual restored window.
The separate deterministic preview
retains its development keymap.
After the batched Finder paste is cancelled, the scenario waits for a fresh
frame with Finder closed and Main focused before typing the next prompt. A
placeholder visible behind Finder does not acknowledge focus.
Artifacts default to dist/visual-opentui-navigation/; VISUAL_EXECUTABLE selects
a complete extracted native candidate rather than a source override.
npm run visual:opentui-controllers selects external approval, question and form
controllers in the actual configured CLI; the editor, history and placement keep
their host implementations. Its six flows exercise the existing F3/F4/F5 decision
contracts, Main answer-versus-draft ownership, independent operation windows,
retained local input state, terminal paste and local mouse coordinates, validated
colored/bold/CJK cells, masked supplied defaults, asynchronous responses and
cancel/unload fail-closed behavior. The requester and controller provider are
separate test-only plugins. A retained ordinary startup form also exercises Main's
FIFO while a real model turn completes, preserving the private form and native
draft/cursor until explicit submission. A normal exact permission rule permits disabling only
that provider through the public extension tool before real /reload; external
gate files control fixture completion, never application state. Private form
window selection waits for the exact Finder query, matching row count and native
highlight. A late result can legitimately have no selection: the bench uses one
ArrowDown only when needed, verifies the selected row and then sends Enter.
It never relies on query text alone to establish a current opening target.
Private form
receipts retain hashes rather than tokens, and terminal/provider/journal scans
exclude the diagnostic raw synthetic keystroke log. Artifacts default to
dist/visual-opentui-controllers/; VISUAL_EXECUTABLE checks a complete extracted
native archive. This scenario does not establish selected editor, placement or
full render/wire parity. The older visual:controllers keeps its complete Ink
composition and its documented custom-editor F2 key unchanged.
npm run visual:opentui-editor checks a selected editor in two fresh actual-CLI
compositions: native private requests, then selected private request controllers.
Its concurrent JSONL observations parse only newline-terminated records; a
partially written final record is observed on the next read. Malformed complete
records still fail the scenario.
Its six bounded flows retain the original F2-submit/Enter-newline contract and
exercise UTF-16 cursor checkpoints, terminal paste, command/file capabilities,
visible busy queue receipts, accepted hook/provider failures, native and selected
private request ownership, and a real four-operation admission limit. A rejected
fifth immediate command must restore the visible editor lease before applying
following input. Canonical extension reload and chat-history resume carry the
unsent checkpoint before destructive disposal; a gated old-generation handler
must lose its capabilities and cannot submit afterward. Disabling the selected
owner fails closed. This is in-process transfer, not a promise to persist unsent
drafts across process restart. During a gated resumed-session startup, Main and
its status bar must both show
Restoring; a submit is rejected with its exact draft and cursor retained, and
no new model request is sent. After startup, the same flow submits successfully.
Finder opens through Alt+W then w while the custom Draft owns F2; known-label
selection retains the shared exact query/count/native
highlight acknowledgement. External gate files control fixture completion only.
Private form receipts retain a token hash, and privacy scans exclude the raw
synthetic keystroke log. Artifacts default to dist/visual-opentui-editor/ with
separate native/ and selected/ reports; VISUAL_EXECUTABLE runs the same checks
against a complete relocated native archive.
npm run visual:opentui-workspace-controllers selects history and placement in
the actual configured CLI, leaving editor, question and private form controls at
@host. Its eight bounded flows exercise private Unicode history search and
cursor retention, native private forms, canonical list/rename/fork/archive and
unarchive controls, explicit resume with model context, and real placement
windows/anchors/Apply/Save across workspaces. Main's selected answer and message
draft stay pending while read-only history and placement work continues. Invalid
or removed targets fail; a gated earlier Apply cannot steal newer navigation.
Hide preserves a lease, while focused Ctrl+C, Core replacement and provider
unload revoke it. Restart restores committed context, rules and the complete
saved window tree, including a deliberately closed pane. Unsent drafts are only
checked across interactive resume, not process exit. The fixture records the
actual selected history resume intent before the bench restarts with that ID.
Dependent history mutations wait for the refreshed native selection/list, in
addition to the service receipt. Inserting or removing a chat shifts the action
rows; pointer targets must remain stable in a completed terminal frame.
After transient placement, the next click also waits for the rendered Apply
confirmation as well as the persisted layout, so it uses the updated geometry.
The separate observer proves private frames, raw inputs and local form results do
not use public ui.render, ui.press or ui.change. Deliberate data.control
calls retain the canonical session service interception payload, including search
and rename arguments; these expected service calls are not described as private
broadcast omissions. Synthetic tokens stay out of all observer events, terminal
output, HTTP requests and journals. External files gate producer completion only.
Artifacts default to dist/visual-opentui-workspace-controllers/;
VISUAL_EXECUTABLE runs the same checks against a complete relocated archive.
The shared Finder helper may pair an explicit command query with its expected
visible label: /plugins selects Plugins without also matching
Plugins · placement. Exact query, result count and selected native cells remain
required; existing callers keep their label-as-query default.
When leaving an existing Finder, the helper observes a completed normal-mode
frame before sending its single F2. This separates a standalone Escape from the
following legacy function-key sequence even if the parser's timer is delayed;
it does not resend the query or retry a failed selection. Navigation's fixture
declares the same explicit lifecycle budget for startup and reload, and records
each awaited startup publication and final ready write for failure diagnosis.
After restart, the checker observes the exact completed startup phase as well
as the ready file: file visibility can precede the awaited SDK call's return and
its completion receipt. The same bounded wait applies; incomplete final JSONL
records cannot acknowledge completion.
Navigation body clicks also require the full normal two-row footer and the
intended focused pane before measuring physical cells. A completed terminal
write can precede final reopened-pane geometry. The helper resets its three
observations when frame, ownership or geometry changes, then sends one click;
it does not retry a missed action or relax the selected-answer assertion.
Appearance's final exit check waits for the exact restored prompt's journal turn
to complete and for the Ready footer before sending its single Ctrl+C. Streamed
answer text alone can precede Core finalization; Ctrl+C during that interval
correctly interrupts the active turn rather than exiting an idle application.
npm run visual:opentui-child runs ten actual configured-CLI checks in isolated
native and selected-controller compositions (six plus four). Ordinary model
task calls and canonical service.agent.spawn create real delegated Cores;
managed create/send/archive uses public agent services with worktrees disabled.
The normal saved loopback provider routes bounded requests. No private Core,
manager or window methods simulate terminal interaction.
Concurrent JSONL reads parse only newline-terminated records; an incomplete
tail waits for the writer, while malformed completed records still fail.
The scenario checks exact allow/deny effects, independent questions and questionnaires, private masked forms, nested parent titles, five same-name siblings while Main waits, settings capacity and unchanged Main Unicode input. Hidden or foreign input cannot answer another request. Native and selected windows hide/reopen with the same pending state; selected factories and leases belong to the actual producer. Main-only provider disable/reload preserves an already-running child's original selected lease. Actual source unregister is covered separately by real Core/host tests; the CLI uses actual source Core replacement or managed archive to verify revocation. Main resume revokes Main descendants, while an independent managed request survives and later managed input uses the newly committed host channel. Archive/application close cancels retained input. The shutdown case waits for a detached child to reach the provider and show its question before closing the application.
The reports retain private synthetic factory/answer receipts, public-hook and
model/journal privacy checks, terminal input/output, current screenshots and
process exits. Raw input necessarily contains synthetic typed values and is
excluded from output privacy claims. Artifacts default to
dist/visual-opentui-child/; VISUAL_EXECUTABLE selects a complete relocated
archive and records its executable/renderer identity. Source success does not
substitute for archive or remote-platform verification.
npm run visual:opentui-conversation-copies runs eight checks in two isolated
compositions through the actual CLI: Ink creates the saved window-N views,
then the same project/profile/journal restarts in configured OpenTUI. Real Ink
keys create session, Tools and Help copies, move them across workspaces, resize
and close a copy. No layout or journal metadata is manufactured by the fixture.
The check compares exact migrated IDs, kinds, topology and ratios, reopens closed
views in Finder, exercises Arrange n and the New Tools/Help actions, and checks
that /chat remains idempotent. Copies share one Main draft/cursor and one model
input destination while retaining independent transcript scroll positions.
The second composition selects the documented F2/Enter editor and selected
question, approval and private form controllers. Factory/input receipts prove
that copies do not create duplicate leases, submit twice, or answer a different
request. Private form fields survive hiding/reopening, and synthetic tokens stay
out of terminal output, provider bodies, journals and public UI hooks. Canonical
reload snapshots before destructive dispose; actual fork/resume revokes a gated
old-generation submit. A final restart checks committed context and exact saved
window state. Unsent draft recovery across process restart is not promised.
Artifacts default to dist/visual-opentui-conversation-copies/; each composition
records the ink → opentui → opentui launch sequence and native executable/
renderer hashes when VISUAL_EXECUTABLE selects a complete relocated archive.
Raw input.jsonl necessarily contains the synthetic typed values and is excluded
from output privacy claims.
npm run visual:opentui-appearance runs eight configured-CLI checks in isolated
native-control and selected-control compositions (five plus three). It executes
the bundled /theme amber|mono|slate commands and inspects actual xterm RGB cells,
selection contrast and single/round/double borders. Two ordinary fixture plugins
publish every portable color, chrome and presentation field through
$.ui.configure. The check verifies contribution replacement/order/null removal,
invalid-call isolation and unchanged user window geometry. Native title controls
use their actual focus/hover/always policy and mouse zoom/close hit targets.
The scenario checks welcome/role labels, default hints, compact versus opt-in ordinary tool arguments and canonical Tools publication intent after a user move. An unrelated Tools copy survives that intent. A pending Main question, shared Unicode draft/cursor/undo, narrow zoom and hidden private password form remain owned across theme changes. Selected controller/editor leases stay singular; explicit rich-cell colors remain literal. Real extension disable/re-enable and the manager's Apply action exercise unload/reload, a gated expired appearance API and the persisted theme. Process restart compares exact window/view metadata and committed model context; it does not promise an unsent draft across exit.
Artifacts default to dist/visual-opentui-appearance/, with separate composition
reports, native executable/renderer hashes, same-buffer captures and privacy
receipts. The fixture validates manifests, catalog hook names and appearance
schemas before launch. Its Finder oracle checks the current selection colors
and each supported border style. Models use only the normal saved loopback
profile; these checks do not establish live-model or physical-terminal quality.
The complete wire protocol remains a separate gate.
npm run visual:opentui-render exercises the configured CLI's public
ui.render/ui.invalidate contract in five native-control and three
selected-control checks. The actual bundled /theme amber|mono|slate hook must
replace the second status row; saving a theme or changing colors alone does not
pass. Configured normal mode reserves exactly two status rows; Arrange/Finder
retain their private footer. The fixture covers conversation, tools, help,
status and plugin:<owner>/<surface>, including independently measured Main
copies, original producer data and producer-owned native input.
Trusted fixture hooks exercise default-first rendering behind an external gate, the real tier/continuation chain, invalid opaque final trees, throw/timeout fallback, asynchronous read-only effect guards and invalidation. Actual resize input must leave exactly two unresolved actual painter calls while their fixture gates remain closed; aborted consumer promises do not establish that limit. Releasing one gate admits only the latest queued public input. Concurrent JSONL observation waits for complete newline-terminated records; malformed complete records still fail the scenario. Actual mouse and keyboard input activates render-added rows/tabs/expanded table details; graphics-disabled output must retain the raster's supplied alt text. This proves the portable fallback, not physical terminal media support. The tmux composer component tests additionally exercise plugin-raster negotiation, updates and cleanup; physical Kitty/tmux observations and their limits are recorded in readiness. The raster component tests also exercise SIXEL selection and geometry loss in the SDK test renderer; they do not establish physical SIXEL terminal behavior. The surface-raster scroll test exercises action-triggered reveal and preservation of manual scrolling while a later image is pending. Main public actions preserve its native or selected editor's Unicode draft/cursor/undo. When the original selected editor truncates its single text row in a narrow copy, the check retains that exact view and width: it verifies the visible edit/caret and Unicode suffix, plus a private SHA256/UTF16/cursor receipt from the original controller snapshot after its real input handler. Raw drafts are never added to that receipt or to public render props. Private request/form/controller values and UI-tool arguments/results must not enter public hook props or model traffic. An ordinary UI-tool operation's documented journal audit is not reclassified as a privacy failure.
Real extension disable/re-enable, manager Apply, fork/resume and process restart
exercise stale hook results, checkpoint ownership, saved theme/context and exact
window/view metadata. Unsent drafts are retained in-process; restart does not
promise unsaved input. Manifests, generated module imports, catalog hook names
and valid override trees are preflighted before a PTY run. Artifacts default to
dist/visual-opentui-render/ with per-composition reports, immutable runtime
hashes, same-buffer captures, public render receipts, original input/ANSI and
private-value scan results. The input log intentionally contains synthetic typed
values and is excluded from privacy claims. CI runs the source scenario; every
native release target runs the same scenario from a fresh relocated archive.
npm run visual:opentui-wire checks the public grid source/send API in
three real Bun PTY processes, without constructing an AppUI or injecting Core
state. The normal process paints a schema-validated JSONL log, exercises
full/diff/flush, styled cells and window chrome, and routes native keyboard,
Unicode paste and global mouse input to the current wire focus. Pointer changes
must appear in actual ANSI output even when no text frame changes. The other
processes verify rejection cleanup and explicit stop with a blocked source.
Five checks retain their PNGs, original ANSI/input, codec log, exact identities
and final process exits under dist/visual-opentui-wire/.
The underlying fixture accepts VISUAL_WIRE_GRAPHICS=kitty|sixel|auto|off for
targeted terminal experiments; the standard scenario defaults to off and
its text-fallback assertions are not a graphics-passthrough qualification.
Set VISUAL_RENDERER_MODULE to an absolute relocated
compositor/node_modules/@agent-compositor/renderer-opentui/index.js to exercise
the exact archived API module; the report records its SHA256. Without this
override the fixture imports the source renderer. The package-client gate is
separate from the configured CLI checks above, which now use protocol 0.7's
native presentation/input profile through the production asynchronous stream.
Pure grid/state and actual SDK tests compare the Ink/OpenTUI painted checkpoints
from the same encoded log, including unflushed diffs and literal control text.
Native draft/undo replay also requires the real input stream; host snapshots
alone intentionally omit the built-in textarea's local state.
The appearance scenario waits for the exact persisted Tools show/hide result, including focus and retained copies. An awaited plugin publication is not an acknowledgment that the asynchronous layout file has finished saving.
Known-label Finder opens in the controller, request-window, manager and cancellation checks share opentui-finder.mjs. The helper waits for one native panel's exact query, expected rows and highlighted choice; it selects a late unselected result before one Enter. Query-only observations do not select or open anything, preserving negative and Closed-row mouse checks. The advanced navigation scenario keeps its deliberate partial-query, stale-route and cancellation interactions separate.
npm run visual:opentui-cancellation checks immediate commands through the actual
configured OpenTUI CLI. Two commands from one plugin start real /bin/sh workers
under exact normal permission rules. Ctrl+C in one window must settle its
operation and kill its PID while the neighboring worker, Main's pending question,
selected answer and draft remain intact. Main interruption must preserve the
other command; its own Ctrl+C must then stop it. A denied third invocation must
create no process. Plugin storage receipts, distinct journal operation IDs and a
completed /reload establish settlement without injecting app or window state.
The provider is bounded loopback HTTP; no paid model or user credentials are used.
Artifacts default to dist/visual-opentui-cancellation/. Set VISUAL_EXECUTABLE
to the executable inside a complete extracted archive for the same native check.
pointer.jsonl records the terminal frame and cell used for each single click;
the scenario waits for a quiet, stable frame before choosing that cell.
npm run visual:opentui-requests checks the configured CLI's independent manual
input windows with a bounded loopback model. Main retains its pending selected
answer and draft while real builtin exec receives exact normal Allow once/Deny
decisions, same-owner direct/builtin questions coexist, a questionnaire returns
its structured answers, and a private form preserves its native cursor and
masked token through validation and hide/reopen. Public commands/actions gate
retained requests before and after actual launcher settlement; Main interruption
and cancellation of one request preserve the others. SDK receipts, exact tool
IDs, file effects and a real reload establish settlement. The private token must
remain absent from terminal output, HTTP bodies and saved journals; raw test
keyboard logs are deliberately excluded. No user credentials or paid provider
are used. Artifacts default to dist/visual-opentui-requests/; VISUAL_EXECUTABLE
selects the executable inside a complete extracted archive for the same proof.
Initial resize waits for the app's complete native pane geometry before starting
the question. Pointer targets exclude synchronized partial output and must keep
their coordinates and painted row stable across three browser animation frames.
npm run visual:windows checks the shipping Ink window controls through the
actual PTY: finder search and cancellation, draft/cursor retention, nearest-split
orientation and resize, directional movement, workspace transfer, fullscreen,
closed-conversation recovery and layout restoration after restart. Separate
fixtures keep an approval and a question pending while local window input is
handled. The fixture inspects only its isolated saved layout and session journal;
it does not mutate AppUI or WindowManager to simulate user actions. F2 exercises
the real terminal shortcut; Alt+Tab can be intercepted by the browser or desktop
and is not used as evidence. All model replies are local scripted fixtures.
For the same checks against an extracted native archive, set
VISUAL_EXECUTABLE=/absolute/path/to/compositor and run the command again. The
report records the exact executable SHA-256 and verifies isolated storage paths;
native model requests use the loopback fixture with an empty API key. Source
plugin-directory overrides are rejected in this mode.
VISUAL_OUTPUT=dist/workflow-proof node tools/visual/workflow-check.mjs checks
dynamic workflow tables with actual bounded workers: run/stage drill-down,
pause/release/resume, closing during worker completion and explicit reopening,
expanded result, clickable state tabs, Left/Right switching, keyboard collapse, narrow fields,
parent completion and restoration. VISUAL_PLUGIN_DIR selects an exact candidate
package; the bench records and verifies its loaded identity. Workflow plugin
certification runs this scenario as well as the legacy Python-journal UI scenario.
For a compiled host, set VISUAL_EXECUTABLE to an executable in a complete,
extracted archive and run node tools/visual/workflow-check.mjs. The native bench
uses a bounded loopback provider and a separately compiled trigger fixture in the
temporary project. It verifies the runtime workflow identity against the bundled
directory/version and records executable/plugin hashes and the six protocol
requests. Native mode rejects VISUAL_PLUGIN_DIR overrides so the shipping
workflow package is exercised.
With the same VISUAL_EXECUTABLE, node tools/visual/agent-table-check.mjs checks
the compiled agent overview while a main stream is held: immediate management,
manual create approval, side reply, filters, keyboard expansion, narrow fields,
close/reopen draft and restart transcript. It expects exactly two loopback
requests. For conversation tabs, run
VISUAL_CONVERSATION_TABS=1 VISUAL_EXECUTABLE=/path/to/extracted/compositor VISUAL_OUTPUT=dist/conversation-tabs-proof node tools/visual/agent-table-check.mjs.
The restart check waits for the private draft files and verifies both independent
Reply fields and committed history after process replacement.
It creates two agents while the main stream is held and checks tab switching,
isolated transcripts, independent drafts, button submission, closed-pane reopening,
narrow layout, persisted selected-window identity and restart history without a
reopen command. File reads only verify saved state; all interactions use actual
terminal input. Nine checks require exactly three loopback requests and zero paid
requests. Independent Reply drafts survive restart after their debounced saves.
For saved agent history, select the same extracted host and run
VISUAL_AGENT_HISTORY=1 VISUAL_EXECUTABLE=/path/to/extracted/compositor VISUAL_OUTPUT=dist/agent-history-proof node tools/visual/agent-table-check.mjs.
It holds the main model while a second agent invokes the actual read tool,
opens Saved activity, filters with mouse and arrow keys, expands saved arguments
and output, checks narrow stacked fields, restarts and reopens the saved result.
Eight checks require exactly one held-main and two tool-cycle loopback requests,
with zero paid requests. The harness only seeds a temporary input file; the tool
call/result must be produced by the actual application and survive its journal.
Both overview and conversation scenarios use normal terminal input and isolated state, record binary mode/hash and fail unexpected model requests. Stream comments keep held fixture connections alive; a separate sixty-second deadline still bounds them.
Run CI=1 npm run visual:check to exercise the same real PTY with inherited CI
flags. The fixture preserves them: interactive output must appear before exit,
just as it does outside CI. Renderer regressions separately check that piped
output remains static.
The browser terminal sets macOptionIsMeta: true, so Option/Alt shortcuts emit
the same escape-prefixed terminal input on Linux and macOS. This setting belongs
to the browser bench; native macOS terminals need their own Option-as-Meta setting.
After cancelling a guarded UI operation, wait for its final panel state before
measuring coordinates for the next action. The tool journal can report a result
before the plugin finishes publishing that panel.
For manual inspection:
VISUAL_OUTPUT=dist/setup-proof node tools/visual/setup-check.mjs checks the real
first-run guide with no initial provider: automatic offer, persisted skip, private
short connection editor, chosen-plugin installation/enable/reload, live refresh
without stealing focus when Extensions disables a selected plugin, draft retention,
narrow layout, completion persistence and a first task through a saved local
endpoint. The guide must make zero completion requests; only the explicit first
task uses the bounded loopback response. Credentials in this fixture are synthetic.
Cancelling either recommendation form also preserves the extension state and
does not contact a model before the normal installation flow continues.
Set VISUAL_EXECUTABLE to the executable from an extracted release archive to
run the same flow against a compiled app. The bench records actual storage paths
for both modes; the report identifies source/binary mode and hashes the selected
binary. Keep the complete archive contents beside that executable.
Set VISUAL_ONBOARDING_OPENTUI=1 to run the same flow with OpenTUI; without it
the scenario uses Ink. Both source and extracted-binary modes support the flag.
Set VISUAL_ONBOARDING_INVALID_MODEL=1 to start with an empty saved model
selection and check that setup opens without overwriting it.
The recommendations step selects the resource monitor, side conversation and
automatic approval review. It checks the dependency and opt-in hints, installs
and activates the selections, and verifies /auto status, /auto on and /auto off without a model request before the first task.
Set VISUAL_ONBOARDING_NO_HUB=1 to start the isolated project with agent-hub
disabled and verify that the recommendation exposes the unavailable capability.
VISUAL_OUTPUT=dist/setup-restrictions node tools/visual/setup-restrictions-check.mjs
uses an explicitly blocked project-checks plugin. It checks the visible unchecked
recommendation, selecting it, and the absence of installation or activation;
set VISUAL_ONBOARDING_OPENTUI=1 for OpenTUI.
VISUAL_OUTPUT=dist/setup-cancellation node tools/visual/setup-cancellation-check.mjs
uses a held local archive response and selects two plugins through the real setup
window. Ink closes the panel; OpenTUI uses Ctrl+C, then closes the panel. The
scenario verifies cancellation reaches the request, the second plugin is not
installed, and setup can reopen. Set VISUAL_ONBOARDING_OPENTUI=1 for OpenTUI.
Set VISUAL_SETUP_CANCEL_KEY=1 to exercise Ctrl+C in Ink instead of closing it.
On OpenTUI the next action waits until the profile form has closed and the guide
has regained focus, because a visible status behind the form is not sufficient.
VISUAL_OUTPUT=dist/setup-load-failure node tools/visual/setup-load-failure-check.mjs
corrupts a selected plugin manifest in the isolated fixture, checks that the
guide reports the unresolved selection after reload, then restores the file and
checks recovery without model requests. Set VISUAL_ONBOARDING_OPENTUI=1 for
OpenTUI. This is a controlled loader failure, not a marketplace availability test.
VISUAL_OUTPUT=dist/workflow-recovery node tools/visual/workflow-recovery-check.mjs
uses a scripted provider to fail a workflow stage, verifies the recovery guidance
and executes the selected-stage retry through the real Ink view. The retry fails
again by design; the persisted attempt count verifies it actually ran.
VISUAL_OUTPUT=dist/workflow-owner node tools/visual/workflow-owner-check.mjs
seeds an isolated running journal with a lock owned by the bench process. It
checks the owner warning, hidden controls and recovery after the synthetic lock
is removed; it does not run a real external workflow process.
VISUAL_OUTPUT=dist/btw-proof node tools/visual/plugin-check.mjs btw checks the
actual side-conversation flow: opening while the main model is held, inline
follow-ups, independent drafts, pane closure/reopening, side-only cancellation,
narrow input and committed history after restart. All requests use a local
protocol fixture. VISUAL_PLUGIN_DIR can select an extracted candidate package;
the fixture checks and records that exact loaded package identity.
The post-restart response uses a file gate: the scenario observes the side
agent running, releases the response, waits for it to become idle, then scrolls
to the new reply. The gate controls only the fixture provider and is reset for
each run.
VISUAL_OUTPUT=dist/opentui-btw npm run visual:opentui-btw uses
the actual OpenTUI CLI and a loopback provider to hold Main while /btw opens,
replies and retains an independent draft. A gated side response lets the test
type during finalization and verify the complete submitted follow-up. It also interrupts a running side
agent with Ctrl+C while verifying that Main keeps running. It closes and reopens the side window,
then restarts the CLI and checks the same history and draft. Set
VISUAL_EXECUTABLE to an executable inside an extracted release archive to
exercise the packaged app; the report records SHA-256 hashes for the executable
and its separately bundled OpenTUI renderer.
npm run visual:serve
Open the printed VISUAL_URL (including its token) in a browser. One client
is active at a time. The app runs in a temporary project with Todos and slash
commands; restarting preserves that project's task data and placement rules.
Ctrl+C stops the server and removes the temporary project. The app's own
Ctrl+C closes only the PTY session; use Restart session to reopen it.
OpenTUI prototype
npm run visual:opentui launches the OpenTUI preview in the same POSIX PTY
and xterm.js browser used by the Ink checks. It uses Bun and the Playwright
Chromium installation or PLAYWRIGHT_CHROMIUM_EXECUTABLE override described
above. The fixture holds one agent in a turn while a second agent receives
input. The scenario exercises native text editing, keyboard and mouse pane
focus, split orientation, swapping, resizing, zoom, independent drafts,
workflow row expansion, terminal resizing and
Ctrl+C cleanup. Set VISUAL_OUTPUT to choose a capture directory.
npm run visual:opentui-windows exercises the host-owned window manager through
the same actual PTY. It checks nested panes, directional focus, title drag/drop,
separator resize and cancellation, workspace moves, and closing/reopening a
running conversation without losing its in-process draft. Actual PTY restarts
verify saved tree geometry, focus, workspace and intentional pane closure, plus
restoration of an empty workspace. The scenario independently reads the app's
layout file named in opentui-fixture.json. It verifies layout persistence;
preview conversation history and unsent drafts do not survive process restart.
The visual server owns the temporary project. For OpenTUI its private
.opentui-home remains there across PTY restarts and is removed with the project
when the server stops. No real home or model credentials are used. Native panes
stay mounted with stable semantic IDs while the compositor changes their frames.
npm run visual:opentui-inputs exercises native questions, questionnaires and
private plugin forms through the actual core callbacks. It checks per-chat
ownership, explicit submission, choices/custom answers, independent message
drafts, focus switching, cancellation and form lifetime across foreground
interruption. Model-received answer receipts and the plugin's own form receipt
are inspected independently of UI labels. The synthetic password must stay out
of terminal output, provider requests and session journals. The bench necessarily
records typed bytes in input.jsonl, so use synthetic test values only. Private
form receipts store a password length/hash, never its submitted text.
The same browser scenario issues real write tool calls from both agents.
It checks the workspace on disk before and after answering, scrolls through the
arguments in a 48×18 terminal, visits Workflows with both requests pending and
exits while a later request is unanswered. The report and screenshots include
the state of each conversation, the observed file contents and PTY exit code.
The plugin scenario searches the real isolated local catalog, installs a project
package inactive, enables it while Main waits for approval, and checks its loaded
version once Main is idle. A controlled local upstream version change exercises
deferred update. Installed files, lock state and the plugin's own command receipt
are checked independently of rendered labels. Disable, cancelled removal and
confirmed removal use the actual management services. No remote catalog or model
endpoint is needed. opentui-fixture.json inside the temporary workspace supplies
the isolated catalog, package and storage paths for those independent checks.
npm run visual:opentui-ghost checks transfers on a PTY that
answers the Kitty capability probe. It compares decoded RGBA frame bytes and
ordering with the bundled idle and thinking animations. This verifies the
emitted image protocol. Physical terminals and multiplexer passthrough need
separate checks.
That two-state command uses the standalone preview, not the configured application's plugin activity mapping. Configured media additionally carries the selected producer's compact presentation and explicit expanded/compact visibility over a negotiated optional native capability. Renderer tests exercise the actual SDK and delayed native-media delivery, including legacy peers and pixel disposal when a composer hides. These tests do not establish GPU display, all activity states or physical SIXEL support. Native acceptance must use a pinned complete archive, original source-frame hashes and actual terminal captures, input and resize receipts. Companion commands can run immediately during held Main work, so native pause/resume checks need no model request. Keep binary frame data behind opaque accessors at React component boundaries: development tracing can recursively expand typed-array props and dominate animation time. Preserve development mode and the original-frame timing oracle when diagnosing this cost; disabling tracing or retiming the assets does not verify the normal renderer.
Render performance
npm run visual:performance -- --motion runs the actual application in the
existing PTY/xterm.js bench with a deterministic 1200-fragment Markdown/Unicode
provider, a live Ghost animation, real keyboard input and pointer movement. It
measures idle input, input during streaming, a draft over long history, hover
round trips and eight seeded viewport sizes. --motion enables the diagnostic
Kitty emulator and omits inherited NO_COLOR in this isolated fixture. Without
it, the test exercises the text-only path. No model credentials are needed.
Capture a baseline before changing the renderer, then compare the candidate with the same flags, fixture and environment:
VISUAL_OUTPUT=dist/perf-before npm run visual:performance -- --motion --baseline
VISUAL_OUTPUT=dist/perf-after npm run visual:performance -- --motion --compare dist/perf-before/report.json
Run performance measurements sequentially on an otherwise quiet machine. Avoid
running the package suite or other visual scenarios at the same time. Results
are observations of that workload/environment, not universal latency promises.
--baseline records existing latency without applying the candidate's 1000 ms
input/hover ceiling. Candidate runs also reject unexpected empty terminal paints
and steady-state scrollback clears. Parser counters track display, scrollback
and line erases without consuming the underlying ANSI commands.
resizeEmptyPaints separately counts an empty emulator paint immediately after
a viewport change when no new application output has arrived. Empty paints after
application output still fail the candidate check, including during resize.
report.json records output chunks/UTF-8 bytes, xterm render events, browser long
tasks, output queue/consume durations, input-to-paint and hover-highlight latency, pointer round trips
and observed native image changes. visual.frames still counts completed output
chunks; it is not FPS. Input timing starts when xterm sends real keyboard bytes
and ends at an xterm render event containing the expected composer text. Pointer
timing ends on the matching OSC 22 response.
Hover-highlight timing separately waits for the expected title-cell bold state
in an xterm render event, so an early cursor response cannot hide a delayed UI.
These timings include the browser,
WebSocket and PTY path; they do not measure physical monitor presentation.
app-performance.json records the fixture process's CPU time, memory and event
loop delay. It is diagnostic fixture instrumentation, not production telemetry.
Reports and PNGs remain under the selected ignored output directory.
The resize comparison checks terminal cell content, width, colors and attributes
after the complete frame is visible and stable across three animation frames.
It canonicalizes unwritten blank cells as spaces; it does not ignore styling or
text differences. Eight reproducible cases are a regression probe, not exhaustive
property-based coverage. Native terminal/GPU compatibility still needs separate
verification.
For a VTE-like capability profile, run
VISUAL_OUTPUT=dist/perf-vte npm run visual:performance -- --vte. This uses the
same real PTY and browser xterm emulator, with Ghost enabled, unsupported native
images falling back to compact status, and OSC 22 disabled. Hover latency is
measured from the actual title highlight; native pointer round trips are absent.
It models a VTE build without SIXEL, not every GNOME Terminal configuration.
Performance runs also retain terminal.events.jsonl: ordered PTY output and
viewport changes. If system Python has PyGObject, GTK3, a modern VTE 2.91 binding
with get_text_format (tested on 0.84.1), and an available X11 display, replay
that trace through the actual VTE engine:
GDK_BACKEND=x11 /usr/bin/python3 tools/visual/vte-replay.py dist/perf-vte
The GTK offscreen widget saves vte-screen.png, text and vte-report.json,
checking final geometry, the conversation header and final result/draft. It
waits for real widget allocation before feeding a resized frame. This supplements
the browser check with native VTE parsing/rendering; it does not measure live
GNOME input latency, GPU presentation or SIXEL when the local build lacks it.
The replay is deliberately optional and fails clearly when prerequisites are
missing. No application state or simulated UI methods are used as evidence.
Evidence
Artifacts default to dist/visual-proof/ (override with VISUAL_OUTPUT):
- PNGs: Ghost composer attachment, pause and manual approval; startup, drag/resize, tasks, saved placement, restored workspace, theme, compact/narrow terminals, slash palette, multiline input, subagent question, delegation, auto mode, review, checkpoints and independent-agent restoration.
report.json: scenario assertions and errors. A failed check also capturesfailure.pngwhen a page exists.terminal.ansi: raw application output.input.jsonl: actual PTY input and resize events, timestamped.
The scenario moves Todos by its title, resizes a separator, sends a prompt, checks the tool's output, saves plugin placement, restarts the app, and opens the saved workspace. Assertions read terminal cells, never AgentCore or WindowManager state. Mouse actions use browser coordinates and xterm's mouse reporting, not synthesized calls to compositor methods.
OSC 22 is deliberately translated into CSS cursors in this emulator, so hover behavior can be asserted. This demonstrates the application's emitted pointer requests, not support in an arbitrary native terminal. Browser screenshots generally omit the OS pointer itself. Native terminal behavior, external provider reliability and exact font rendering are separate checks.
VISUAL_URL can point visual:check at an already-running bench; in that
case the runner closes its browser but leaves the server running. Each
connection starts the app; use a fresh server for a clean initial fixture.
The browser and PTY retain the last viewport dimensions across restart. The scenario also opens marketplace/proxy panes through real terminal input after restoring an independent agent.
Native graphics and compiled distributions
node tools/visual/plugin-check.mjs git-review exercises status, staged/working
diffs, scoped configuration, narrow terminals and restart. It also holds the
scripted agent active while refreshing real Git status, starts a controlled
long-running SDK subprocess through isolated fixture middleware, cancels it from
the panel and verifies that the PID is gone while the agent remains active.
The scenario also closes the panel before starting the held model, opens it
through /git-review while the model is active, and checks that the model stays
working. It then interrupts and continues the agent through terminal input.
Fixture processes and files belong only to the temporary bench project.
npm run visual:background exercises the background-jobs plugin in the real PTY.
It checks Ctrl+C during a busy turn, launches a real background shell, sends a
second user message while it runs, and leaves an unsent draft in the composer.
One completion reads the result and preserves that draft. The scripted provider
records open/closed request lifetimes to reject waiting streams or polling calls.
Evidence lives under dist/visual-background/; this does not certify external
provider reliability.
The default scenario checks portable text rendering and compact image status. Run
VISUAL_GRAPHICS=kitty npm run visual:check to let the browser bench answer the
real Kitty query and emulate emitted RGBA upload/placement/deletion commands.
The canvas overlay consumes PTY bytes only; it does not read AppUI or plugin
state. visual.graphics.stats records dimensions, uploads, placements and
protocol errors; visual.graphics.active reports live placements. This verifies
our protocol transport, not GPU rendering in a native Kitty-compatible client.
For inspecting an extracted release without loading the source fixture:
VISUAL_GRAPHICS=kitty VISUAL_EXECUTABLE=/absolute/path/to/compositor npm run visual:serve
The executable runs in the bench's temporary project and home, with no real
project settings. The compiled executable uses its configured provider rather
than the fixture's scripted provider; startup, mouse placement, pause and
workspace checks need no model requests. The full scripted scenario still uses
the source fixture. VISUAL_EXECUTABLE must point to the executable inside its
complete extracted distribution, beside plugins/ and node_modules/.
npm run visual:mascot checks the Ghost image attachment through the real
PTY and the diagnostic Kitty emulator: source pixels, frame changes, pause,
pet, compact status, draft preservation, disable, and restart. The motion
scenario deliberately omits inherited NO_COLOR in its isolated child; production
still respects it. Combine with VISUAL_EXECUTABLE for a compiled distribution.
Evidence defaults to dist/visual-ghost-lifecycle/.
VISUAL_MASCOT_GIF selects a test GIF copied into the isolated temporary project. An optional VISUAL_EXPECT_MIN_FPS sets an explicit observation threshold; the report records observed image changes/s separately from the file timing. It is not a hardware/terminal-wide performance promise. The 50 fps test asset must contain enough distinct frames to make observed changes meaningful.
node tools/visual/sixel-reference-check.mjs optionally uses ImageMagick to
decode a 64×64 generated SIXEL checkerboard and checks all 4096 pixels (two RGB
levels of tolerance because SIXEL uses percentage colour definitions). This is
an independent encoder check, not native VTE screen verification. Tiny SIXEL
streams are avoided because the local ImageMagick decoder rejects files below
2048 bytes, including its own tiny outputs.
Native settings scenario
Run npm run visual:settings to exercise the Settings toolbar, provider preset,
editable connection form, reasoning effort selection/persistence/reset, masked API key, authenticated proxy form, independent
model/tool switches, cancellation and restoration after restart. The real TUI
runs in the same isolated PTY fixture. It uses fake keys and local dummy
addresses, and does not contact an external model. Artifacts default to
dist/visual-settings/; set VISUAL_OUTPUT for another run. Inspect the PNGs and
report.json. input.jsonl intentionally captures raw input: never use real
credentials in this diagnostic bench. Provider discovery/authentication and
transport behavior are also covered by package tests and npm run eval:proxy.
Dynamic workflow scenario
npm run visual:workflows starts an isolated real PTY with a test-only provider
and fixture plugin. Terminal commands launch the real workflow tool, discover
two workers and hold them at a fixture barrier. Mouse actions pause scheduling;
a terminal command releases the workers, then resume joins their results. The
closed view also stays hidden during an actual workflow inspection tool call. The
scenario inspects stage detail and automatic parent-agent completion, restarts the application, verifies persisted
completion and submits foreground input with the context indicator visible.
It does not use live credentials. Artifacts default to dist/visual-workflows/.
The fixture plugin is outside shipped plugin directories.
Markdown and native pointer fallback
npm run visual:presentation exercises a real PTY with a Markdown response,
checks heading/emphasis/list/table/quote/code cells, resizes the viewport, and
captures title hover with OSC 22 explicitly disabled (the GNOME/VTE case).
VISUAL_EXECUTABLE selects the compiled distribution. This scenario supplies a
loopback, token-scoped model fixture; it uses no external credentials or model.
Default artifacts are dist/visual-presentation/; set VISUAL_OUTPUT separately
for source and binary checks. Inspect the PNGs as well as report.json.
The ordinary bench explicitly requests AGENT_POINTER=on because it implements
OSC 22 in JavaScript. Set VISUAL_POINTER=off for manual native-fallback checks;
the standard pointer scenario expects the emulator feature and uses on.
Browser cursor emulation is not native GNOME Terminal support. Actual VTE/GTK
font rendering is not tested by xterm.js; equal cell dimensions and the same
built application are necessary before comparing source and installed versions.
Human questionnaire scenario
npm run visual:questions runs a three-question form through the actual agent
loop and PTY. It selects rich radio/checkbox choices with the mouse, enters a
custom multiple-choice answer, returns with Back, resizes to 60 columns, fills
free text, edits an earlier answer from review and explicitly submits. The
scripted provider reads the structured tool result and displays the returned
values. Captures default to dist/visual-questions/; inspect the PNGs and
report.json. VISUAL_EXECUTABLE selects the compiled distribution via the same
loopback, token-scoped model fixture. No external models or credentials are used.
Chat history scenario
npm run visual:history seeds named, archived and multi-turn sessions in the
isolated project and opens the actual history pane. Real keyboard/mouse input
pages the list, searches a phrase found only in an earlier response, resizes to
60 columns, renames the selected chat, shows archives, resumes its transcript and
sends a follow-up through the provider. Bare /resume and Escape are also checked.
The provider echoes earlier conversation context; no live credentials are used.
Artifacts default to dist/visual-history/; inspect PNGs and report.json.
Set VISUAL_EXECUTABLE to the built distribution and VISUAL_START_HISTORY=1
to test the binary's bare --resume startup entry. The same loopback model fixture
and isolated home storage are used. Set a fresh VISUAL_OUTPUT for each run.
UX interaction scenario
npm run visual:ux checks readable window projection at 100x30, 80x24 and 60x24,
keyboard access to a hidden pane, fullscreen while composing, and an offscreen
provider action. It then opens a long real exec approval and checks its beginning,
end, tool identity and all permission scopes while scrolling. The command is a
safe print-only fixture and is always denied. The final mono history check reads
actual xterm RGB cells, avoiding a false contrast pass from ANSI color names.
VISUAL_EXECUTABLE selects the compiled distribution with a token-scoped loopback
model fixture. Default output: dist/visual-ux/; inspect PNGs and report.json.
The maintained TUI design guide explains the shared layout, navigation, literal-input, contrast and plugin-contract decisions.
Replacement controller scenario
npm run visual:controllers loads an external test-only plugin with custom editor,
approval, question, private form, placement and history controllers plus its own
structural session repository. Real PTY function keys exercise behavior replacement;
restart/search/resume checks persistence and a 60-column viewport. Artifacts default
to dist/visual-controllers/. VISUAL_EXECUTABLE checks a compiled distribution
using the token-scoped loopback model. Inspect the PNGs and report.json; raw input
captures contain only fake credentials. The fixture backend is not production code.
Internet and attached-project scenario
npm run visual:workspaces uses a separate temporary home and real PTY to check
@~/proj completion, directory submission, quoted external files, cross-project
read, /workspace, a loopback search/fetch chain, and scope restoration after
restart/resume at 60 columns. VISUAL_EXECUTABLE checks the compiled distribution.
Inspect report.json and PNGs in dist/visual-web-workspaces/. The loopback fixture
uses no live credentials or search account; live-provider probes are separate.
Plugin authoring lifecycle
npm run visual:author exercises the real PTY's /plugin flow with an isolated
scripted model: skill/composition guidance, a six-component scaffold, editing
without removing other hooks, a real write approval, structure checking, user
/reload, the authored command with an SDK import, an actual panel, plugin skill
loading and a real local stdio MCP call. It repeats execution after process
restart. It waits for the observed Ready state before
reload. It also disables the owner, observes its panel/MCP contribution disappear,
and re-enables it to exercise restoration. VISUAL_EXECUTABLE selects the compiled distribution; it must contain the
plugin-author skill/resources. VISUAL_OUTPUT selects a fresh capture directory
(default dist/visual-plugin-author/). The scenario supplies no live provider
credentials. Inspect report.json and the PNGs; scripted success checks the
integration path, not arbitrary model-generated plugin quality.
Default marketplace and scoped activation
npm run visual:marketplace starts the actual TUI with no active filesystem
plugins. It browses the offline default catalog, installs packages inactive,
explicitly enables Todos in project scope, verifies restart persistence and
disables it. Real terminal mouse/keyboard actions then install and enable a
package through forms selecting user scope. VISUAL_EXECUTABLE selects an
extracted compiled distribution. Its held model, question and continuation use
an abort-aware token-scoped loopback SSE endpoint, so compiled runs do not select
an external model. All projects/homes are isolated. Existing
positive scenarios now explicitly select their fixture plugins; discovery alone
never activates production code.
Plugin certification scenarios
node tools/visual/plugin-check.mjs git-review exercises status and distinct
staged/working diffs against the real Git fixture. It also checks keyboard
actions, scoped configuration save/cancel and restoration after restart,
48×18 controls, panel close/reopen and an unsent draft. VISUAL_PLUGIN_DIR
selects the exact source or extracted archive candidate.
node tools/visual/plugin-check.mjs <plugin> runs the package-specific scenario
for auto-approve, checkpoint-history, cost, mascot, mcp, network-proxy, todos,
ui-theme or workflows. VISUAL_OUTPUT selects a fresh evidence directory.
VISUAL_PLUGIN_DIR selects the exact extracted archive candidate; the bench
asserts the loaded directory and writes fixture-identity.json. All activation,
configuration, model responses and project data remain isolated fixtures.
The certification wrapper is tools/plugins/visual-scenario.ts; it verifies
required observations and hashes PNG, ANSI, input and report evidence into the
scenario protocol. Interactive actions travel through terminal input. The proxy
password fixture is private and must never appear in visible cells, model context
or ordinary plugin events/logs. Screenshots verify masking; deterministic privacy
cases additionally inspect the receiver and excluded channels.
For a Git-reviewing candidate, VISUAL_GIT_FIXTURE=1 seeds the isolated PTY
project with a real local Git repository, one commit, and distinct staged and
working changes in review.txt. It requires Git on PATH, ignores global/system
Git configuration, and uses no remote. With VISUAL_CERT_PLUGIN, this fixture
explicitly allows the fixed read-only git_review tool; it is a test grant, not
a production permission default. VISUAL_PLUGIN_DIR still selects and verifies
the exact candidate directory.
The marketplace scenario also holds a scripted model turn while real terminal mouse/keyboard input opens Plugins, searches a catalog, installs and enables a package and queues reload. It verifies application after interruption, then keeps an agent question pending while catalog search and provider settings are used. No live model or credentials are involved. At 48×18 it also opens management with Alt+o / Alt+,, checks the compact pending question, expands it with a mouse click before answering, and verifies that later keyboard navigation preserves an unsent message draft.
Project checks panel
node tools/visual/plugin-check.mjs project-checks runs the real Ink panel in an
isolated npm project. It discovers scripts, runs passing/failing checks, saves
output/timeout limits globally and verifies them after restart. A held scripted
model stays active while checks run; cancellation and panel Close must reap the
real npm child without completing the main turn. The scenario also checks draft
preservation during terminal resize and reopening the panel. Input goes through
xterm keyboard/mouse events. VISUAL_PLUGIN_DIR selects the exact source or
unpacked archive; fixture-identity.json records which copy was loaded.
The network-proxy plugin scenario also edits a joined emoji in username and
password fields, checks that Backspace removes the complete grapheme, saves the
form and verifies the isolated stored values without printing the secret. The
wire text and screenshots must keep the password masked. This tests the shared
built-in form editor; it does not change the user's proxy configuration.
CI review panel
node tools/visual/plugin-check.mjs ci-review opens the real /ci panel in an
isolated PTY. The scenario configures an explicit repository and output limits,
selects runs/jobs/failed and full logs, cancels the settings form, saves a new
run limit while the main agent is busy, and checks
Cancel/Close cleanup of actual SDK-owned processes while the scripted main
agent remains held. It also verifies a preserved draft after resize and saved
settings after session restart. During a held CI subprocess, it requires the
visible busy status and verifies its removal after cancellation settles, while
previous results remain visible during pending work. ci-fixture.mjs installs a local executable
gh fixture only on that PTY child's PATH and records its argv; it never uses
user GitHub credentials or a live model. Refresh responses include a request
sequence marker: the scenario waits for the specific response to render before
starting cancellation checks, rather than accepting a previous result or a
subprocess-start record as completion. Reports retain screenshots, input and
terminal output. Screenshot capture waits for an xterm render event and browser
paint frames, and retains a companion text snapshot. Pending/cancelled captures
also assert their expected terminal state before and after the screenshot so a
previous painted frame cannot silently stand in for the requested state.
Cancellation waits for both the error and removal of the panel's busy state;
the conversation's earlier tool-error event alone does not acknowledge cleanup.
These checks do not establish live GitHub network reliability.
Structured test reports
node tools/visual/plugin-check.mjs test-reports uses the real PTY with local
Vitest/Jest-shaped JSON, the actual report tool/panel and a controlled pre-tool
wait. It checks explicit reads, keyboard/pointer failure selection, a missing-file
error retaining labelled previous results, scoped configuration, narrow resize,
restart, the unsent draft and concurrent main-agent work. Cancel, titlebar [x]
and Alt+Q settle the controlled report operation without stopping that model.
At 60×24, PageDown must reach the final line of a long diagnostic; PageUp must
return to the unchanged path field. These are mandatory source and installed
archive observations for the report package. This wait is a
fixture dependency, not a measurement of disk latency. No live model or user
credentials are used. See the package guide.
A repeated successful read must enter the controlled dependency before the bench
releases it and waits for completion. The retained Report loaded. label alone
cannot acknowledge a new operation; configuring against that stale label can
race an active read. The concurrent-read capture also rejects Cancel read.
Immediate controls and conversation recovery
node tools/visual/plugin-check.mjs auto-approve opens the optional audit history
explicitly, then closes it and runs /auto on|off|status while a controlled model
stream stays active. Those short commands must not reopen a pane. It closes the
main conversation with an unsent draft, restores it through Alt+Enter, then uses
/chat to restore/focus one existing conversation and return from workspace 2.
At 60×24 it pages through retained history to find the original held response.
The main model must continue independently, and a later prompt must still get
the second scripted response. The normal keyboard/mouse/restart checks follow.
This is actual source-app PTY evidence, not an external-model quality assessment
or proof of a compiled host archive. The fixture uses no saved profile or key.
Independent resource monitor
VISUAL_OUTPUT=dist/resources-proof node tools/visual/plugin-check.mjs resources
loads the exact selected resources package through the real app. The main model
is held while the user opens the charts, waits for actual sample growth, pauses
across a measured observation window, switches tabs by mouse/keyboard, samples
manually, explicitly queues a selected memory sample while the main turn remains
held, clears history, scrolls narrow controls, closes/reopens and restarts.
It also checks actual system CPU and non-free memory charts and explicitly queues
a selected system observation. Pausing invalidates the current process CPU interval and,
on Linux, the current child CPU interval. Neither old interval can be shared;
chart history is preserved.
The fixture uses actual process counters, not seeded graph values.
VISUAL_OUTPUT=dist/opentui-resources npm run visual:opentui-resources exercises
the same system charts in the actual OpenTUI app while Main is held by a loopback
provider. It verifies explicit sharing without a second provider request and
captures both graphs. VISUAL_EXECUTABLE selects a relocated release binary;
the report records hashes of the executable, renderer and bundled plugin.
For a complete extracted native host, use
VISUAL_RESOURCES=1 VISUAL_EXECUTABLE=/path/to/extracted/compositor VISUAL_OUTPUT=dist/native-resources node tools/visual/agent-table-check.mjs.
This uses the bundled plugin and one held-main loopback request; resource controls
do not produce additional model requests while the main turn is held. The queued
sample is interrupted and the isolated session restarted before it can run.
Both paths record fifteen checks,
including the visual certification requirements, and capture wide/narrow/restart
screenshots. Source-host artifact certification can select an exact installed
plugin with VISUAL_PLUGIN_DIR; native proof uses bundled code only.
OpenTUI agent reply recovery
VISUAL_OUTPUT=dist/visual-opentui-agent-drafts npm run visual:opentui-agent-drafts
uses the actual CLI in an isolated PTY with two managed agents and one
synthetic loopback provider request; no paid model is called. It types independent Reply drafts, switches windows immediately,
waits for the private state files, restarts and verifies both values. It also
types rapidly into a delayed plugin field that republishes each edit, switches
windows immediately and verifies the exact value received by the producer. It
also submits a Reply with Enter and checks the exact request and response. A
delayed submit verifies that later typing is delivered separately, including a
nonempty replacement and a temporarily absent field that require explicit input recovery. The
scenario exercises native surface input admission and restart, not external
terminal compatibility. Set VISUAL_EXECUTABLE to an executable inside an
extracted archive to exercise the packaged CLI; the report records its SHA-256.
Ink conflicting input recovery
VISUAL_OUTPUT=dist/visual-ink-input-recovery npm run visual:ink-input-recovery
uses a real Ink PTY and a private temporary plugin to verify that a conflicting
producer value remains visible until Ctrl+R explicitly restores the retained
later input. Set VISUAL_RECOVERY_MISSING=1 to exercise the temporarily absent
field and its waiting state before the plugin restores a field. This synthetic
flow makes no paid model requests.