Unstable connections

This page introduces the faults that come and go: commands that time out on some runs and not others, an agent that drops off the list and comes back, a serial write that reports success while nothing comes back. Three layers produce these symptoms — USB and serial, the network, and timeouts — and each has its own checks and its own fixes.

Symptoms

What you see	Most likely layer
A command runs, sometimes times out	Timeout, or a script that outlives its own assumptions
The agent appears in the list, then vanishes	Network, or a proxy that closes idle sockets
`serial_write` reports `Failed to send to /dev/ttyUSB0`	The port is not open, or the device re-enumerated
`get_serial_ports` returns an empty array	The optional native `serialport` bindings are missing
The agent log shows `Device disconnected` every few seconds	The device is re-enumerating, or the port is being closed
Commands arrive with a delay of about five seconds	The socket is down and the agent is using the polling path

USB and serial flakiness

The device re-enumerates. The agent watches the serial ports and reports devices coming and going as device_event messages. A device that disappears has its port closed, and when it comes back it may have a different path (/dev/ttyUSB0 becomes /dev/ttyUSB1). The old path stays dead until you open the new one.
The agent's user cannot reach the port. On Linux the user running the agent needs read and write access to the device node — the dialout group is the usual answer. Without it, open_serial fails and the command status is error.
The native serial bindings are optional. The agent loads them lazily and degrades if they are missing: get_serial_ports then returns an empty array — its output.stdout is [], not an error — and every open fails. The agent log says [Serial] SerialPort bindings not available. Reinstall the workspace on the bench so serialport and @serialport/bindings-cpp build:

pnpm install

serial_write refuses an unopened port. The write is refused unless the port is open and writable, and the result reads Failed to send to <path>. Open it first with open_serial, which takes path and baudRate (default 115200) — and nothing else. dataBits, stopBits and parity are ignored on the socket path; they only exist on the agent's own local serial API at http://localhost:3001.
A successful write is not evidence of a delivered response. Serial bytes are printed to the agent's console and are never forwarded to the studio. The studio receives device metadata only — path, manufacturer, serial number, vendor and product id — through a device_event message. There is no serial read command in the protocol, so a device's answer can only be read on the bench.

Network flakiness

The polling fallback. The agent polls GET /api/agent/commands?agentId= every 5000 ms and posts each outcome to POST /api/agent/commands/result, whether or not the socket is up. Both paths write the same agent_commands rows, so a command picked up by the poll tick instead of the socket looks identical afterwards — it just arrived up to five seconds later.
The socket reconnect. The agent reconnects on its own while the link is unstable, and the polling loop keeps the platform working in the meantime. When the client stops reconnecting, restarting the agent process is what resumes the socket.
The studio queues against a live socket. POST /api/agents/command answers Agent is offline when the agent has no live socket, and no command row is created. Work for an agent is queued against the agent, not against a bench.
Proxies close idle sockets. The connection is idle between commands. A reverse proxy with a short read timeout will cut it, and the agent will fall back to polling for a while. Keep proxy_read_timeout at 300 s or more.
Presence flips immediately. "Online" is derived from the live socket map, so an agent disappears from the list the moment the socket drops and reappears the moment it is re-authenticated. The lastSeen column next to it is written only by the heartbeat endpoint.

Timeouts

The shell command writes your script to <cwd>/.agent_script_<timestamp>.sh (mode 0755), spawns it with /bin/bash in that working directory, and runs it as the agent's own user. The default timeout is 300000 ms — five minutes; the studio's REST command endpoint sets none, so the default applies unless you pass one in args.

Read the result at GET /api/agents/commands/{id}; the JSON in result holds stdout, stderr, exitCode and durationMs.

Exit code	Meaning
`0`	Success. The command status is `success`.
any other positive code	The script's own exit code. The status is `error`, and `stderr` holds whatever the script wrote.
`-1`	The process could not be spawned at all — `/bin/bash` missing, or the working directory not writable. The spawn error is in `stderr`.
`-2`	The timeout fired. The script is sent `SIGTERM` and `stderr` reads `Command timed out after <ms>ms`.

A timeout only signals the shell

SIGTERM reaches the script, not its whole process tree. A child that keeps the serial port open will survive the timeout, and the next open_serial will fail until it exits. The script file itself is left behind in the working directory.

Recover

Read the result before retrying
Fetch the command result and look at exitCode, durationMs and stderr first — a retry without that only duplicates the failure.
```
curl -sS http://localhost:3000/api/agents/commands/<command-id>
```
Re-check the socket, not just the REST API
The REST API stays healthy when the socket is down. Check the socket endpoint directly, and restart the agent process if the client has stopped reconnecting.
```
curl -sS http://localhost:3000/api/agents
curl -sS "http://localhost:3000/api/socket.io/?EIO=4&transport=polling"
pnpm --filter @dkstudio/agent dev
```

Re-open the serial port after a re-enumeration

List the ports again, then open the path that is actually there, then write.

curl -sS -X POST http://localhost:3000/api/agents/command \
  -H 'Content-Type: application/json' \
  -d '{"agentId":"<agent-id>","command":"get_serial_ports","args":{}}'