Generating test cases with AI
The drafting assistant turns a requirement, a specification excerpt or a change note into a list of candidate cases. Each candidate carries the layers of the test case model and a pointer to the text it came from, so you start from a list you can argue with instead of a blank page.
What it works from
- The source text. The requirement, the specification chapter or the change note that describes the behaviour. The assistant reads the text you give it; it does not go looking for documents.
- The scope. Which product, vehicle or capability the text applies to, so a draft does not propose cases for a subsystem you are not testing.
- The constraints. What the bench can actually do. A procedure is only useful if every step
names something reachable: a
shellcommand, a serial write, a file on the machine under test.
What comes back
A draft list. Each draft states the capability, the variant, the procedure, the expectation, and the excerpt of the source text it was derived from. The assistant proposes; it does not decide which proposals count.
Two properties of a draft matter more than how well it is written.
- Traceability. Every draft points back at the part of the source it came from. A reviewer can say "this is not in the requirement" without re-reading the whole document.
- Executability. Every step names something the bench can do. A step that cannot be performed is a case that cannot fail, which is worse than no case at all.
Review before you accept
| Check | What to look for |
|---|---|
| Can it be executed? | Every step resolves to a command the bench can run. A step that needs a person watching a screen is not an automated case. |
| Is the expectation observable? | Pass and fail are distinguished by something the platform records — an exit code, a captured byte, a value in the command result. "The interface looks right" is not an expectation. |
| Is the data complete? | The case carries the parameters and preconditions it needs to run, not just the intent. |
| Does it point at the source? | The excerpt the draft cites actually contains the behaviour the case tests. |
| Is the coverage complete? | The enumeration is plausible, not exhaustive. Check it against the specification, not against the draft. |
Accepting a draft
Drafts are held apart from the case list. Accepting a draft creates the case; discarding one removes it. Nothing reaches a run without that explicit accept step, because a case that nobody approved is not a case.
A model that is confidently wrong is the normal failure mode of drafting, not a rare one. Read the draft against its source excerpt before you accept it.
Limits
- A draft is a hypothesis. It carries no observation of a real device. It is a proposal about what to check, and the acceptance criterion is that the check turns out to be worth making.
- The model does not know your bench. It has no view of the devices, vehicles and environments you run on, and no catalogue of what this product can do. It works from the text you give it and from what you tell it the bench can reach.
- The enumeration is plausible, not exhaustive. A list of twenty candidates is a starting point for coverage work, not a coverage argument.
- A generated expectation inherits the source's ambiguity. If the specification says "responds promptly" and the case expects a duration, the ambiguity is now in the test suite rather than in the document.
See also
- The test case model — the layers a draft has to fit.
- Writing a test case — how a case is written once it exists.
- Remote Shell and Serial — the commands a procedure names.
- Results and reports — where a run's outcome ends up.