Harness Engineering: Building Safe and Reliable Tooling for AI

A model can suggest a code change. A coding agent also needs tools to read files, make the change, run checks, and see the results. That surrounding set of tools and controls is its harness.

Harness engineering designs that environment: available tools, permissions, and feedback from checks. Its purpose is to support editing, testing, and other actions. There is no single architecture for every agent.

Context Engineering addresses the information available to the agent. Harness engineering addresses what it can do and how those actions execute.

Separate runtime controls from project guidance

Two useful layers are:

Layer	Responsibilities	Examples
Platform runtime	Dispatch tools, manage sessions, enforce configured execution limits	File editor, terminal runner, sandbox, approval policy
Project environment	Make development tasks reproducible and explain local conventions	Build scripts, pinned dependencies, fixtures, repository instructions

The boundary depends on the product and deployment. Sandboxing, network restrictions, and approval requirements are configuration choices, not properties guaranteed by every coding agent. Verify the actual permissions in the environment you use.

An instruction such as “do not write to production” is guidance to the model. Credentials without production access provide a separate, enforceable control.

flowchart LR
    accTitle: Agent tool execution loop
    accDescr: The model proposes an action, the runtime checks permissions, and a tool executes in the configured environment. Exit status and bounded output return to the model.
    Model["Model proposes an action"] --> Runtime["Runtime checks permissions"]
    Runtime --> Tool["Tool executes in configured environment"]
    Tool --> Result["Exit status and bounded output"]
    Result --> Model

Pair each important rule with observable feedback

Birgitta Böckeler’s Harness engineering for coding agent users distinguishes guides that steer work before an action from sensors that inspect its result. This extends the runtime view: a project harness also makes engineering expectations executable.

For the checkout example, pair guidance with a check:

Guidance	Feedback mechanism	Remaining judgment
Pricing owns discount calculation	Dependency rules reject imports of pricing internals from web	Review for duplicated formulas that imports cannot reveal
Preserve the accepted quote	Acceptance fixtures compare API, UI, and persisted amounts	Confirm that fixture expectations express product intent
Follow the existing module boundaries	Structural checks detect forbidden dependencies or cycles	Assess whether the boundaries still suit the next change

Run cheap, relevant checks during editing, then rerun required checks in CI on the integrated change. Use model review for questions that need interpretation, with evidence for a human reviewer. Passing deterministic checks establishes only their encoded properties; passing a model review supplies a fallible judgment. Neither replaces acceptance criteria.

Make maintainability feedback actionable

Böckeler’s follow-up, Maintainability sensors for coding agents, experiments with linting, dependency rules, coupling analysis, and mutation testing. A practical lesson is to explain how to respond to a finding, rather than returning an unexplained score.

For an illustrative dependency rule, return a diagnostic such as:

Violation: web/checkout imports pricing/internal/discountRules.
Boundary: web consumes the published quote contract.
Next step: inspect the pricing client and reuse its quote operation.
If the boundary must change, record the reason for review.

Keep exceptions and threshold changes visible in the diff. Review them before accepting a green result: an agent can silence a useful warning by relaxing the rule. Also inspect trade-offs between rules. Splitting a long component into many pieces may reduce line counts while increasing coupling and the number of parameters passed between components.

Start with an observed maintenance problem and a small check. Track whether it helps subsequent changes; a cleaner dashboard alone does not establish a more maintainable design. For an architecture-level treatment of fitness functions and automated governance, see Chapter 2, “Fitness Functions” and Chapter 4, “Automating Architectural Governance” in Building Evolutionary Architectures.

Make commands work without personal shell setup

A common failure is a command that works in a developer’s terminal but fails in automation. The agent may use a different shell, working directory, environment, or startup mode.

For Bash, interactive and login modes affect which startup files are read; non-interactive execution does not generally read ~/.bashrc automatically. Some .bashrc files also return early outside an interactive shell. This is about shell mode, not simply whether a terminal screen exists. See the Bash startup-file rules.

Wrapping every command in bash -i -c can introduce aliases, prompts, and machine-specific behavior. Prefer a documented runtime setup and commands that work in CI as well as locally.

For an illustrative Node project with a lockfile and existing package scripts:

# AGENTS.md

## Setup
- Run commands from the repository root.
- Use the Node version declared in .node-version.
- Install the locked dependencies with npm ci.
- Unit tests use local fixtures and need no external services.

## Verification
- Type check: npm run typecheck
- Unit tests, one run: npm run test:ci
- Lint: npm run lint
- Production build: npm run build

## Editing conventions
- Follow nearby code and keep changes within the requested scope.
- Regenerate files in generated/ using npm run generate.
- Report checks that could not run and the reason.

These commands assume the corresponding scripts exist. Replace them with commands verified in your repository, including service setup and teardown where needed. Keep the authoritative command definitions in scripts so humans, agents, and CI use the same implementation.

Use MCP where it helps connectivity

The Model Context Protocol (MCP) standardizes how clients discover and call server-provided tools. It can connect an agent to a schema inspector, issue tracker, or browser tool. A compatible client, supported capabilities, and appropriate configuration are still required. The MCP tools specification describes discovery, calls, and error reporting.

MCP does not make a server safe or its database access read-only. Enforce read-only access through credentials and server configuration. Similarly, a tool description is not proof that the implementation has no side effects.

Use native terminal tools or a CLI when they already solve the problem. A feedback loop does not require MCP; the requirement is reliable execution and observable results.

Package repeated workflows as skills

A skill can bundle instructions, scripts, and templates for a recurring task, such as generating a client from an API contract. The Agent Skills format uses a SKILL.md entry point, with additional resources loaded as needed by supporting tools.

A skill makes procedures reusable. It does not guarantee that an agent follows every step or that the procedure remains correct. Version the scripts, verify their outputs, and update the instructions when project conventions change.

Example: a bounded checkout verification tool

For the running discount feature, a team could expose verify_checkout with the contract below. This is an illustrative tool the team must implement; it is not a built-in agent command.

tool: verify_checkout
inputs:
  run_id: identifier of an allocated disposable environment
  suite: enum [discount-valid, discount-expired, checkout-no-code]
  candidate_manifest: immutable reference to repository commits and image digests
execution:
  timeout_seconds: 120
  prerequisites: candidate revisions deployed; services ready; fixtures seeded
  side_effects: creates synthetic quotes and orders in this run only
  concurrency: reject a second active invocation for the same run and suite
outputs:
  status: passed | failed | invalid_arguments | environment_unavailable | timed_out
  executed_cases: list of named checks
  assertions: expected and observed values for each check
  elapsed_ms: measured duration
  artifact_refs: references to sanitized logs and traces
  cleanup_status: completed | pending | failed

The runner validates the suite and manifest, enforces environment ownership and timeout, and supplies credentials outside the tool arguments. It must verify the deployed revisions against the manifest rather than echoing the requested revisions as proof. Persist full sanitized artifacts even when the model receives only a bounded summary.

Result	Agent response
invalid_arguments: unknown suite	Correct the call from the tool schema; do not edit application code
environment_unavailable: pricing readiness failed	Inspect setup evidence and restore the prerequisite within budget
failed: displayed total differs from persisted total	Use assertion values and candidate source to diagnose the implementation
timed_out: browser stopped responding	Inspect trace and cleanup status before another invocation
passed with no executed cases	Treat the result as invalid verification evidence

A timeout does not prove that no order was created. The runner should allocate isolated fixtures for another attempt or reset the previous fixtures after confirming termination. Never retry a mutation blindly because the tool returned no final response.

Design tool results for diagnosis

A useful command result includes the working directory, exit status, elapsed time, and relevant output. Preserve full logs as artifacts when the model receives only a summary. Distinguish “the test failed” from “the test runner could not start.”

Set timeouts for commands that can hang, and ensure timed-out child processes are cleaned up. Separate local build permissions from deployment credentials. For mutations of shared systems, make the allowed scope explicit and record what happened.

Model choice is another configuration decision. Compare candidate models on representative tasks, including successful completion, review effort, latency, and total cost. Context capacity alone does not establish suitability.

A useful first harness is modest: one reproducible setup, a few reliable checks, and permissions matched to the task. The next post explains how to define the behavior those tools should help deliver: Spec-Driven Development.