How Tbench Actions works

The one-line change

GitHub assigns a job to any idle runner whose label set is a superset of the labels the job requested. So a runner provider only has to register runners carrying a stable, memorable label, and the customer changes exactly one line:

-    runs-on: ubuntu-latest
+    runs-on: tbench-2vcpu-ubuntu-2404

No step is added and no workflow logic changes. The label is the whole product surface: it names both a size and an image. Your machine, not ours, executes the job — this beta has no paid cloud overflow, so a job waits while every enrolled device is offline.

Requests, the queue, and leases

Every eligible delivery becomes a request with a stable identity: installation, repository, run, attempt, and job. Duplicate webhook deliveries do not create extra requests, and a delayed event cannot reopen a completed job.

Two different resources are bounded independently — conflating them is what makes an idle queue look busy:

A device claims a fenced 15-minute lease: while it holds the lease it is the only device that may act on that request, and a stale device that returns late cannot act on a request another device has since taken.

A job’s life

1. A compatible job is queued by GitHub and becomes a request.
2. An enrolled device polls, finds work it is allowed to take, and claims a lease on it.
3. The worker asks the hosted broker to prepare a one-use runner. The broker registers an ephemeral runner and returns short-lived JIT configuration, never a long-lived credential.
4. The worker starts the runner container inside an isolated network. GitHub’s scheduler — not the device’s queue order — decides which matching job it takes.
5. The job runs, then the runner deregisters itself. The worker removes only the containers and networks it recorded for that request.
6. The outcome is reported, and the request reaches a terminal state.

Credentials never reach the device

The hosted broker — a Vercel server — is the only party holding the GitHub App ID and its private key. The App requests repository Administration write, Actions write, and Metadata read, and subscribes to workflow_run and workflow_job events.

• Installation tokens are requested for exactly one repository with those three permissions, kept only in server memory for that broker request, and revoked afterwards.
• A device stores only its revocable, installation-bound enrollment bearer.
• Runner configuration is one-use and passed in a transient environment, never as a CLI argument or into logs.

Isolation on the device

Jobs run on the pinned Ubuntu 24.04 x64 image, in a container started with all capabilities dropped, no-new-privileges, a process limit, and enforced CPU and memory caps. The runner sits on an internal Docker network with no direct route; the only egress path is a small, separately bounded proxy.

Default network access is GitHub-only. Opting into TBENCH_EGRESS_PROFILE=package-registries adds only the npm and PyPI registries; arbitrary hosts stay blocked. Every DNS answer must be globally routable, and private, LAN and metadata addresses remain denied.

What is deliberately not enforced

• A container is not a security boundary for hostile code. It shares the host kernel. Run code you already trust on your own machine.
• A workflow allowlist is policy, not isolation. A repository-scoped runner can pick up another same-label job in that repository before Tbench observes it, so a workflow restriction is not a hard pre-execution boundary.
• Cancellation is scoped to the owned runner. Tbench stops only its own device runner; GitHub’s REST cancel ends the whole workflow, so sibling jobs are never silently cancelled.

These are the reasons the beta asks for trusted private repositories rather than public or outside-fork code. See the reference for the exact variables and commands, or troubleshooting when a job stops unexpectedly.