Skip to the content

Stop agents across a fleet

Suspend, terminate or lock down agents on one machine, one agent, one person or the whole fleet, with an order two approver keys sign and each node checks.

Built, not yet claimed preview: this guide describes a package that has not passed its release gate.

Know what a kill order stops and what it leaves running, set up the approver keys before an incident, rehearse on one test machine, request an order and read what it reaches, approve it with two keys, read each node's answer, lift it, and stop agents by hand when Central Lite is down.

1. Know what an order does and doesn't

A kill order names a scope: one machine (host: and its host id), a list of them (hosts:, up to 4,096 ids, 1,024 in the console), one agent (agent: and its id, such as agent:claude-code), one person by the pseudonym nodes upload (principal:kh1: and its hash), or fleet. Its action is suspend (the tasks are stopped and denied every decision until a lift), terminate (suspended, then killed), or lockdown (every governed task in scope is terminated and no governed agent may start or act on that machine until the order is lifted). A suspend or terminate also refuses new tasks in its scope until the order expires, at most 24 hours after it was requested. One task on one machine is not an order: use the single-task kill.

Each node verifies the order itself before it acts: the approvers' signatures against the org root metadata it trusts, the window, and that it has not seen the order before. How firmly it stops a task depends on the platform: A on Linux for a task launched with agentgarm run and its own cgroup; B on macOS with the Endpoint Security client, where a process of the same user can wake a stopped process until the next sweep; C on macOS without it, and for hooks-only sessions (every decision is denied, and the editor is never signalled). On Windows, suspend and terminate through the task's Job Object are built, not yet run on Windows hardware; resume and lift there come later. A no-root install takes no kill order, from Central Lite or carried by hand: it holds no org root to verify one against.

An order does not stop CI jobs, your users' browsers, edge connectors or any process outside a governed task, and a principal: order cannot find a machine on the non-monitoring profile. Under Central Lite's 1-of-1 software root, whoever controls Central Lite's host can add an approver and so issue an order alone; only an offline root of two or more keys stops that, and no production one exists yet.

See also

Stop an agent task on any machine

Kill or suspend one agent task from the Central Lite host, check that the node applied it, and revoke a machine you no longer trust.

2. Set up approvers before you need them

Orders are approved by the remote approver keys listed in your org root metadata, the set CI approvals use, never by an admin's sign-in passkey, which Central Lite refuses as an approver. Write the approvers file as the CI approvals guide shows, with at least two keys, and sign it in with garm-central root lite --approvers; garm-central approver list shows each with active or revoked. Do this before an incident: Central Lite counts a key only 24 hours after it joined the set, a rule nodes cannot check but Central Lite always applies.

How many keys an order needs comes from the root metadata's threshold: 2 for one machine and 2 for anything wider unless you set it. --kill-threshold-wide (2 to 16) and --kill-threshold-host (1 to the wide value, 1 only with a single approver key) on garm-central root lite set it; left out, the installed value is kept. A request is refused while fewer keys than that count. Keep the keys with different people, on security keys or passkeys that ask for a fingerprint or PIN: each approval needs user presence and user verification.

Central Lite records which admin holds each key, at its first answer in the console or with garm-central approver hold, and refuses that key from anyone else's session. When your identity provider deprovisions a holder, Central Lite stops counting their keys at once, and the request's count of usable keys drops with it; it revokes a key still suspended after 7 days, or at once when the holder is deleted, unless that would leave fewer keys than the threshold (who holds each approver key). Until it is revoked, a machine still accepts the key.

garm-central root lite --approvers approvers.json
garm-central approver list

Checked by the walk: this command ran as written on a Central Lite host, and the output showed active.

See also

Approve a CI job's risky step from your laptop

Mark the commands a CI job may wait on a person for, and approve the job's request from the console with your passkey, for exactly that call, once.

3. Rehearse on one test machine

Before you need it for real, go through the steps below once with a host: order and the suspend action, against one machine you can spare, with a short wait. Find its host id with garm-central node list. Start an agent there with agentgarm run first, so you see it stop, and lift the order at the end. A suspend of one machine leaves every other machine alone, and the lift lets the stopped task carry on.

Write the reason for the people whose agents stop: it is shown on every machine in scope, so say what is happening and never name a person. garm-central kill list shows every order and lift, newest first, with its state.

garm-central node list
garm-central kill list

Checked by the walk: this command ran as written on a Central Lite host.

4. Request an order and read what it reaches

On the Central Lite host, garm-central kill request prints kill order … requested with its action, its scope and how many approvals it needs by when, then what it reaches as Central Lite last heard of the fleet: reaches: N node(s) for a host, host-list or principal: scope, or reaches: every enrolled node … for fleet or an agent (with any node that subscribes before it lapses), too old for nodes whose agentgarmd cannot apply an order, not covered for CI nodes, and the non-monitoring nodes a principal: scope cannot find. Under a software root it says so, and it ends with the command each approver runs. The sample sets TEST_MACHINE to the hostname garm-central node list shows for the machine you can spare and takes its host id from there; a name that matches no node leaves the scope empty, and the request is refused. --expires-in sets the window (default 1 hour, at most 24). --required M asks for more approvals than the threshold, never fewer.

In the console, open Kill orders, fill in the scope, the action, the reason and the wait, and choose “Show what it reaches”: “Request with a passkey” stays disabled until it has shown the radius for the scope as typed, and while too few approver keys count. The request asks for a fresh passkey assertion over exactly what you typed. Either way the request is chained and raises a kill_order_pending alert; it waits for approvals until its window ends, and garm-central kill cancel or “Cancel with a passkey” withdraws it before then.

In the console, requesting, cancelling and asking for a lift need the security or owner role. One bound to a host group may request a host: or hosts: order for machines in its group; an order for the fleet, an agent or a person needs the role for the whole org (who may request a kill order). garm-central on the host is outside roles.

TEST_MACHINE=${TEST_MACHINE:?set it to the hostname garm-central node list shows for the test machine}
HOST_ID=$(garm-central node list | awk -v name="$TEST_MACHINE" '$2 == name { print $1; exit }')
garm-central kill request --scope "host:$HOST_ID" --action suspend --reason "suspected credential leak: agents paused for review" --expires-in 30m
garm-central kill request --scope fleet --action lockdown --reason "incident response: governed agents stopped fleet-wide"

Checked by the walk: this command ran as written on a Central Lite host, and the output showed requested.

The console's Kill orders: a request with the blast radius it reaches, then each node's answer to an issued order

5. Approve it with two keys

Each approver signs the order's digest, not the page's text. In the console, open the order: it recomputes the digest from the order's own fields and checks that the challenge Central Lite sent is that order's, then shows digest checked and enables “Approve with an approver key”. If either check fails it says “Don't sign it”. A Central Lite that was broken into could show one order and ask for a signature over another, so the console's own check is only as good as the console.

The check that does not trust Central Lite: download the order (“Download the order file”, or garm-central kill export), run garm-central kill show --verify on a machine Central Lite does not control, which prints the order and recomputes its digest, and sign with fido2-assert using the input garm-central kill challenge prints. Each approver keeps their credential id, as the approver set lists it, in a file (alice.credential), and sets FIDO_DEVICE to their key as fido2-token -L lists it; ORDER is the pending order garm-central kill list shows. Then garm-central kill approve answers approval counted: 1 of 2, and the second key order … issued: 2 of 2 …; queued for N node(s). Each key counts once. Once issued, the order goes to every subscribed node in scope within about a second, and to a node in a fleet or agent scope that connects later, until it expires.

ORDER=$(garm-central kill list | awk '$2 == "kill" && $4 == "pending" { print $1; exit }')
garm-central kill export "$ORDER" --out order.json
garm-central kill show --verify order.json
garm-central kill challenge "$ORDER"
ORDER=$(garm-central kill list | awk '$2 == "kill" && $4 == "pending" { print $1; exit }')
garm-central kill challenge "$ORDER" --credential "$(cat alice.credential)" > assert.in
fido2-assert -G -p -v -i assert.in -o assert.out "$FIDO_DEVICE"
garm-central kill approve "$ORDER" --assertion assert.out
ORDER=$(garm-central kill list | awk '$2 == "kill" && $4 == "pending" { print $1; exit }')
garm-central kill challenge "$ORDER" --credential "$(cat bob.credential)" > assert.in
fido2-assert -G -p -v -i assert.in -o assert.out "$FIDO_DEVICE"
garm-central kill approve "$ORDER" --assertion assert.out

Checked by the walk: these commands ran as written on a Central Lite host, and the output showed approval counted: 1 of 2 and issued: 2 of 2.

6. Read each node's answer

garm-central kill status shows the order, its window and its approvals, then nodes: with a count of each answer and a line per node with its detail. A node answers applied (with how many tasks it stopped), partial (some tasks in scope could not be stopped), refused (its own check failed: the order is not applied there, its agents keep running, and the refusal is alerted at once), root_behind (its root metadata is older than the order needs; Central Lite sends the root and then the order again), failed or lifted_locally. A node that has not answered stays queued, and gets the order when it next connects, until the order expires. Nodes too old to apply an order were counted in the request and are never sent one. The console's order view shows the same answers, with each node's task counts. After a restore, the console's “Orders only nodes know” card lists orders nodes report in force that Central Lite has no record of; lift each with garm-central kill lift (see Back up and restore Central Lite).

An order that matches nothing on a node is applied with no tasks and stays in force there for later launches. On the machine, agentgarm status shows each order in force, a lockdown first: LOCKDOWN, “this machine is locked down: no governed agent starts or acts”, or a kill order line, then the order, its scope, who approved it, the root's custody and its reason.

garm-central kill status ORDER
agentgarm status

Checked by the walk: this command ran as written on a Linux node enrolled in Central Lite, and the output showed kill order.

See also

Find out why your agent stopped

Tell whether your agent was suspended, killed, locked out by a kill order or stopped by its budget, get your work back, and find out who can lift it.

7. Lift it

A lift is approved like an order, at the order's own threshold. garm-central kill lift prints lift … of order … requested: 2 approval(s) needed; the approvers sign the lift's id, as in the approval step, taking it from garm-central kill list (the pending lift): the first answers approval counted: 1 of 2, the second issued: 2 of 2. In the console, “Request a lift with a passkey” on an issued order does the same. Once issued, the lift goes to every node the order reached.

A suspended task runs again only when no other hold is left on it, such as a budget breach. A terminated task stays ended: its user gets their work back from the checkpoint taken at launch, with rollback. A lockdown ends only with its lift, while a suspend or terminate stops refusing new tasks once its window ends, though the tasks it suspended stay suspended until the lift.

ORDER=$(garm-central kill list | awk '$2 == "kill" && $4 == "issued" { print $1; exit }')
garm-central kill lift "$ORDER" --reason "review finished: agents may resume"
LIFT=$(garm-central kill list | awk '$2 == "lift" && $4 == "pending" { print $1; exit }')
garm-central kill challenge "$LIFT" --credential "$(cat alice.credential)" > assert.in
fido2-assert -G -p -v -i assert.in -o assert.out "$FIDO_DEVICE"
garm-central kill approve "$LIFT" --assertion assert.out
LIFT=$(garm-central kill list | awk '$2 == "lift" && $4 == "pending" { print $1; exit }')
garm-central kill challenge "$LIFT" --credential "$(cat bob.credential)" > assert.in
fido2-assert -G -p -v -i assert.in -o assert.out "$FIDO_DEVICE"
garm-central kill approve "$LIFT" --assertion assert.out

Checked by the walk: these commands ran as written on a Central Lite host, and the output showed 2 approval(s) needed, approval counted: 1 of 2 and issued: 2 of 2.

8. Stop or lift by hand when Central Lite is down

A node Central Lite cannot reach takes an issued order by hand: write it with garm-central kill export (or “Download the order file”), carry the file to the machine and run sudo agentgarm kill-order apply there, from outside every governed task. agentgarmd checks it exactly as one from Central Lite and answers order … applied with the tasks found and stopped (exit 0), or order … refused: and why (exit 1). A lift file is applied the same way. It needs an enrolled machine; a no-root install takes no order, even carried by hand.

When the approver keys or Central Lite itself are lost, root on the machine can lift an order in force there: sudo agentgarm kill-order lift with a reason. It is recorded in the machine's audit chain, alerted to Central Lite on the next connection and shown there as lifted_locally. Both commands are not built on Windows yet: they refuse there until a later release.

sudo agentgarm kill-order apply order.json
sudo agentgarm kill-order lift ORDER --reason "Central Lite unreachable: lifted on the machine"