The Test Case Model

One capability, one definition, many benches. This page introduces how DK Studio models a test so a single case covers every project, protocol family and device type it has to run on — and how every result can be traced back to the intent that produced it.

The problem the model solves

Take one capability: provision and enrol a digital key. Getting it right means more than one thing:

• The protocol family differs — CCC, ICCE, ICCOA and COT have different frames and different handshakes (Supported Protocols).
• The device type differs — an Apple handset needs an activation handshake before enrolment, a Google handset does not.
• The OEM differs — one implementation returns the status in the first byte of the reply, the other in the fifth.
• The project differs only in data — a different IMSI, a different SIM, a different port, a different platform URL.

Copy the case per project and per protocol and you maintain N copies of the same intent; a report can then no longer say "provisioning is 90% covered", because the capability has been shredded into pieces. Collapse everything into one case and generate the script at run time, and a result can no longer be explained, because nothing records which steps actually ran.

DK Studio separates what is being verified from how it is done. That separation is the whole model.

The layers

Capability          "provision and enrol a digital key"
  └─ Variant        CCC + Apple — a concrete implementation of that capability
       └─ Procedure an ordered list of steps
            └─ Expectation  a portable matcher over a value
  └─ Driver         the bench-side backend that executes the procedure

Each layer answers one question and delegates the rest downwards.

Capability

A capability states the business intent in domain terms: a stable identifier and title, a description of the intent, the preconditions that must hold before the case means anything, and the acceptance criteria — the business-level outcome. It carries no protocol name, no device model, no script and no port.

A capability is stored in test_cases with id, title, description, suite_id, priority and status, alongside created_at and updated_at. Priority defaults to medium, status to active. The preconditions and the acceptance criteria are part of the intent the case file carries.

A case belongs to a suite, and a case can be part of more than one — the same capability sits in a smoke suite and a regression suite without being copied.

The capability is the part that survives longest. When a protocol changes, the variant changes and the capability is untouched — which is exactly why a report can still group results by capability.

Variant

A variant is a capability's concrete implementation for one target. Its identity is:

(capability, protocol family, device type, OEM)

The project is deliberately not part of the identity. A project supplies data; it does not create a new flow. That is the rule that keeps the number of variants proportional to the number of real process differences rather than to the number of projects.

The protocol family is one of CCC, ICCE, ICCOA or COT. The device type distinguishes the device families that differ in behaviour, such as apple and google. The OEM distinguishes implementations that differ in how they answer; * means the variant does not distinguish an OEM.

Two variants of the same capability may inherit from one another, so the steps shared by CCC on Apple and CCC on Google are written once. A variant that extends another overrides the steps it changes and inherits the rest.

Procedure

A procedure is an ordered list of steps. Each step names the command to run, the script or payload to run it with, an optional timeout in milliseconds, and the expectations attached to it.

The commands a step can invoke are the agent's own command vocabulary — shell, get_system_info, get_serial_ports, open_serial, close_serial, serial_write — described in Remote Shell and Serial.

Steps are reused across capabilities where the mechanics are the same: opening a serial port is the same step wherever it appears.

Expectation

An expectation is a matcher, deliberately separated from the code that reads the value.

• The extractor is protocol-specific. It knows that the status byte is at offset 0 for one OEM and offset 4 for another, and that the reply arrives as a hex string.
• The matcher is portable. It knows only that the value equals 0x00, that it is ACTIVE, or that it arrived within 30 seconds.

Splitting them means the assertion logic can be shared, reviewed and re-run offline, while the protocol-specific reading stays with the variant that owns it. An expectation is anchored to a step; several expectations may target the same step and are told apart by a sub-index.

- ordinal: 4
  kind: action
  command: serial_write
  script: "$ENROL_FRAME"
  assertions:
    - index: 0
      extractor: "reply.hex[0:2]"
      matcher: "equals"
      expected: "0x00"
    - index: 1
      extractor: "reply.elapsed_ms"
      matcher: "less_than"
      expected: 15000

Driver

The driver is the execution backend: it owns a bench, knows how to run a procedure on it, and returns a result. The model expects more than one.

• The agent driver runs procedures over the command channel — the same shell, open_serial and serial_write commands described in Remote Shell and Serial.
• A local driver runs a procedure directly on a workstation, without a remote agent.

Drivers are the only layer allowed to be specific to a lab, a toolchain or a project — which is why the layers above them are kept free of that detail.

How a run resolves a capability

A run turns a capability into a concrete procedure like this:

1. Take the capabilities selected by the suite.
2. For each capability, resolve the variant matching the run's target — protocol family, device type and OEM.
3. Bind the parameters for the run's project, with variant-level overrides applied on top.
4. Render the procedure, substituting parameter placeholders into every step, and compute a hash of the result so a later script edit is visible in the history.
5. Hand the rendered steps to the driver and collect the per-step outcomes.

A capability with no matching variant is skipped with a reason, not failed. A missing mandatory parameter is an error, because it is a configuration problem rather than a product failure.

The case file

The model is expressed as a case file: the capability at the top, its variants beneath it, and the steps of each variant with the expectations attached to them.

id: tc.provision
title: Provision and enrol a digital key
description: The device completes enrolment and the platform can query the key state.
category: functional
priority: high
status: active
preconditions:
  - Device is powered on and attached to the bench
  - SIM is active and entitled
acceptance: The device acknowledges the enrolment and the platform reports ACTIVE
suite: [smoke, regression]
 
variants:
  - id: v.provision.ccc.apple
    protocol: CCC
    device_type: apple
    oem: "*"
    steps:
      - ordinal: 1
        kind: check
        command: shell
        script: "ping -c2 $BENCH_HOST"
        timeout_ms: 10000
      - ordinal: 2
        kind: action
        command: serial_write
        script: "$ENROL_FRAME"
        timeout_ms: 15000
        assertions:
          - index: 0
            extractor: "reply.hex[0:2]"
            matcher: "equals"
            expected: "0x00"

See also

• Writing a Test Case — the shape of a case and its assertions.
• Data-driven matrices — generating cases from data.
• Runs and scheduling — how a run drives a capability.
• Results and reports — what a run produces.
• Remote shell and serial — the commands a step invokes.