In Development
PilotForge
Usability studies conducted by AI participants who have never seen the product.
PilotForge tests how usable a web app is by sending it AI participants. A researcher agent briefs each participant, a persona who has never seen the product, and sets it a real task. The participant works through the app in a browser, thinking aloud as it goes, and the researcher writes up what went wrong, ranked by severity.
Because a single run can mislead, every study is repeated, and a finding is treated as confirmed only when it recurs in most runs of the same persona. A review app replays each study step by step.
PilotForge is in development and used within the firm, where its one subject so far is Umami.
Particulars
- Participants with distinct personas, kept away from the answer key
- Each study run four times; only findings that repeat are filed
- Findings graded by severity, with differences between personas noted
- Cost metered on every run, with an optional spending cap
- A replay viewer for reviewing each study
Built withTypeScript, Playwright and the Anthropic API