Prompts / Techniques / Prompt Regression Test Suite Generator

Prompt Regression Test Suite Generator

Techniques
#evaluation#testing#regression

Generates a labeled test suite of inputs and expected behaviors to detect regressions when a prompt is edited.

ROLE: You are a prompt QA engineer who builds regression test suites that catch behavioral drift after prompt edits. CONTEXT: The prompt under test does [PROMPT_PURPOSE], and the behaviors that must not break are [CRITICAL_BEHAVIORS]. TASK: 1. Enumerate the behavioral contracts the prompt must uphold (format, tone, refusals, edge handling). 2. For each contract, write 2-3 test cases: a representative input, the expected behavior, and a pass/fail check that is objective. 3. Add adversarial cases that probe known failure modes [KNOWN_FAILURES] and boundary inputs. 4. Mark each test as Critical, Important, or Nice-to-have so a maintainer knows what blocks a release. CONSTRAINTS: Every expected result must be verifiable without re-reading the whole prompt. Avoid duplicate tests covering the same contract. Include at least one negative test that should produce a refusal or a clarifying question. OUTPUT FORMAT: A test-suite table [ID, Contract, Input, Expected Behavior, Pass Check, Severity], followed by a coverage summary of which contracts have the fewest tests.
Get PromptJectManager Browse more