Decide what a pass means before you start
We will classify fictional requests for paper signs. READY means both the quantity and pickup day are confirmed. Anything else is NEEDS_DETAIL. For each reply, award one point for the correct label and one for returning only that label. Write the answer key below in your own notes; do not include it in the model's input.
Anthropic's evaluation guidance recommends specific, measurable criteria and cases that resemble the task, including edge cases. Our three cases are a learning exercise, not evidence of production reliability.
Keep these three test cases fixed
Case 1: Please make 3 signs for pickup on Thursday. Case 2: We might need 3 signs for pickup on Thursday; I will confirm later. Case 3: Please make some signs for pickup on Thursday.
Editorial answer key: Case 1 is READY; Cases 2 and 3 are NEEDS_DETAIL. The second request is tentative, and the third lacks a quantity. These labels are our predefined checks, not recorded model results.
Prepare the two prompt versions
Classify this fictional request for paper signs. Return READY only if both the number of signs and the pickup day are confirmed. Otherwise return NEEDS_DETAIL. Reply with exactly one of those labels and nothing else. Request: [replace this bracketed text with exactly one test case].
For Prompt B, copy Prompt A and insert this single example before Request: Example: Four signs, pickup day undecided. Label: NEEDS_DETAIL. Keep everything else identical. You are testing the addition of one example, rather than changing several instructions at once.
Run and record the six replies
Use an app you already have. Keep the app, visible model, mode, and any settings you control the same. Start a fresh chat for each prompt-and-case pair so earlier test replies are not in that chat. Note the date, unknown settings, and any personalization or app instructions you cannot hold fixed.
Run A then B for Case 1, B then A for Case 2, and A then B for Case 3. Save the exact input and first complete reply each time. Use a score sheet with columns Version, Case, Correct label, Label only, and Notes. Each version can earn six points across its three replies: three for correct labels and three for following the label-only format.
Read the failures before choosing a favorite
Score against your written key. Report the two checks separately: correct labels out of three and format passes out of three. If both prompts pass, record a tie; extra prompt text has not shown a benefit on these cases.
Outputs can vary. If you repeat the exercise, repeat the same six pairs and retain all runs rather than choosing the best answer. After editing a prompt, add fresh cases you did not use to design it. Three familiar cases cannot establish a general accuracy rate or rank models. This page supplies a method and answer key; it does not claim that either prompt has won a test.