Tbench runner protocol | Tbench Actions

Execution and verification, separately.

tbench actions executes a versioned Actions TOML policy. tbench verify grades repository-native checks from a separately approved verification TOML policy, with required/advisory status and commit-bound JSON evidence. Neither command requires the Harbor benchmark harness.

The signed queue currently accepts Actions execution policies. Weighted application verification runs through its separate local command; a queued verifier adapter needs a separately matched report contract. Application checks are not benchmark audit certification.

Install the reviewed Python package with its portable-signatures extra for signed queue execution. This source implementation is separate from the existing downloadable GitHub worker release.

Inspect a plan before executing.

tbench actions doctor

tbench actions plan /path/to/repository \
  --trusted-base <approved-base-sha> --head <exact-commit-sha> \
  --policy-sha256 <owner-approved-actions-policy-sha256>

tbench verify plan /path/to/repository \
  --trusted-base <approved-base-sha> --head <exact-commit-sha>

Actions reads .tbench/actions.toml; local verification reads .tbench/verify.toml. A plan does not execute commands. Replace plan with run and explicitly add --trust-image only after reviewing the policy and locally installed image. Run on the approved host or CI agent with its local Docker engine, not inside a nested job container.

Your 15-day approved organization trial includes these execution and local verification paths on your compute. Optional provider/model charges and separately approved hosted benchmark allowances are not contributor-seat credits.

The owner approves what a device may execute.

Choose an approved logical repository name and a trusted base commit.
Pin a locally present Linux Docker image by its full sha256: image ID. Select its native architecture.
Review named argv steps, working directories, CPU, memory, timeout and log caps. Keep secrets out of policy documents.
Register the immutable queue policy in the execution console. A change creates a new policy revision and digest.
On the device, map that policy ID to the approved local repository, trusted base and policy-file hash. PR content cannot change this mapping.

The queue does not receive repository source or clone credentials. The exact requested commit must already be available locally. Source synchronization stays with the organization’s own Git access controls.

{
  "name": "Repository checks",
  "repository": "team/service",
  "mode": "actions",
  "image": "sha256:<locally-installed-image-id>",
  "platform": "linux",
  "arch": "amd64",
  "steps": [
    {
      "id": "tests",
      "argv": [
        "python",
        "-m",
        "pytest",
        "-q"
      ],
      "cwd": ".",
      "env": {}
    }
  ],
  "limits": {
    "cpus": 2,
    "memory_mib": 2048,
    "timeout_sec": 300,
    "log_bytes": 65536
  }
}

Enrollment is not automatic participation.

An owner issues scoped, expiring device access in the console. Download the one-time enrollment file, transfer it securely to the device owner and verify its signing-key fingerprint through a separate administrator-approved channel.

tbench actions enroll --enrollment /private/enrollment.json \
  --pin <administrator-verified-key-fingerprint> --state /private/device.json

tbench actions serve --state /private/device.json \
  --repositories /private/repositories.toml --once --trust-image

Keep enrollment, state and repository mappings outside Git and shared folders. The device owner explicitly starts the session. Starting it does not grant access to another person’s computer or add a new device to the organization.

Docker runs Linux jobs on supported hosts. Windows and Mac hosts do not turn Linux containers into native Windows or Xcode jobs; native Apple execution needs its own disposable backend on Apple hardware.

Use the pipeline you already have.

Custom scripts, GitLab jobs and Jenkins stages can invoke the local Actions command with their own checkout and approved policy. The adapter records provider-native job identity and converts the local result to an exit code and JSON report. Your existing CI engine remains responsible for pipeline scheduling, credentials and protected-branch rules.

The direct queue API accepts only an approved policy ID, exact commit SHA, adapter and idempotent external job ID. It does not accept arbitrary commands, host paths or source URLs from a contributor.

These invocation adapters are not native replacements for GitLab Runner or Jenkins agents. GitHub’s runs-on integration continues to use the separate existing GitHub worker.

Bounded jobs. Explicit evidence.

Independent Ed25519-signed assignments, a pinned signing key and fenced expiring leases.
A fresh tracked-file snapshot, non-root execution, read-only container root, disabled network and bounded writable storage.
No host Docker socket, arbitrary host mounts, automatic image pulls or implicit architecture emulation.
Cancellation, heartbeat and completion are bound to the same device, job, commit, policy and lease.
An expired active job goes to operator attention rather than running again on another device.

Device-reported results are not remote attestation. A compromised device can fabricate its own report. Independent certification and high-trust merge enforcement require separately qualified evidence controls.