Guide 40 of 61 · Investigating and recovering
Send the fleet's security events to Splunk or Sentinel
Have Central Lite forward the fleet's security and audit events, its own audit log and its alerts as OCSF to Splunk or Microsoft Sentinel.
Built, not yet claimed preview: this guide describes a package that has not passed its release gate.
Know what Central Lite sends and what stays on the machines, connect Splunk's HTTP Event Collector or Microsoft Sentinel's Logs Ingestion API, search what arrives, read a sink's lag, rejects and gaps, and rotate its credential. At the end one Central Lite feeds your SIEM for the whole fleet, with one credential to look after.
1. Know what Central Lite sends and what stays on the machines
Machines already upload their security and audit events to Central Lite, so Central Lite can forward the fleet's record with one SIEM credential instead of one on every machine. A sink sends, as OCSF 1.9.0: the uploaded security and audit events (operational if you ask; never telemetry), Central Lite's own audit log (sign-ins, admin actions, SCIM and role changes, kill orders and legal holds) and its alerts. A new sink starts from what arrives after it is added: nothing older is sent.
Every personal field arrives as the same keyed pseudonym the machines use (kh1:), so an event from Central Lite and the same person's events from a machine's own export join in your SIEM. Prompt and response text never leave the machines, and a machine under the non-monitoring profile uploads less in the first place. Delivery is at least once: deduplicate on metadata.uid.
A sink never holds anything up: machines keep uploading, and enforcement never waits on a SIEM. Every problem with a sink is a Central Lite alert, forwarded too once the sink recovers. Adding, changing and pausing sinks needs the owner role in the console.
See also
Send agent events to your SIEMTurn on agentgarmd's exporters: OCSF events to your SIEM, OTel GenAI spans and token metrics to your collector, and notice when an export stops.
2. Connect Splunk
In Splunk, make an HTTP Event Collector token allowed to write only to the index you choose, and save it in a file only Central Lite's account can read. garm-central siem add splunk takes HEC's https:// URL (a port such as :8088 is allowed), the token file, the index, and --ack to move on only once Splunk confirms each batch is indexed, which Splunk Enterprise offers and Splunk Cloud does not. The token is sealed in Central Lite's keystore, never in its database, its logs or its audit log. Batches stay under 1,000,000 bytes, Splunk Cloud's limit, and are sent uncompressed. Splunk's certificate is checked against the platform's trusted authorities. A self-hosted Splunk with a certificate of its own can be pinned by its key instead: --pin-spki takes the SHA-256 of the certificate's public key in hex, and Central Lite then accepts that key alone, whoever issued the certificate (openssl x509 -in hec.crt -pubkey -noout | openssl pkey -pubin -outform der | sha256sum prints it). When the certificate is renewed with a new key, garm-central siem set-pin pins the new one (or --clear goes back to the trusted authorities) and the sink carries on from where it was. Its host name must still resolve to a public address: Central Lite never sends events to a private one.
garm-central siem test splunk sends one test finding and prints Splunk's answer. Events arrive with sourcetype ocsf:agentgarm, so Splunk's OCSF-CIM add-on can map them, and Splunk indexes every OCSF field for searches like the one below. In the console, Settings → SIEM adds, tests, pauses and edits a sink the same way, each with a passkey.
garm-central siem add splunk --url https://splunk.example:8088 --token-file hec.token --index agentgarm --ack
garm-central siem test splunkindex=agentgarm sourcetype="ocsf:agentgarm" class_uid=2004 earliest=-24h | stats count by severity, finding_info.title3. Connect Microsoft Sentinel
Central Lite writes to one custom table, AgentGarmOCSF_CL, through the Logs Ingestion API. Deploy the table and its data collection rule from Central Lite's template, deploy/sentinel/agentgarm-ocsf.bicep (or the ARM JSON beside it), into your Sentinel workspace's resource group; it outputs the rule's immutable id and its logs ingestion endpoint. Register an application in Entra ID, upload a certificate to it as for single sign-on, and give it the Monitoring Metrics Publisher role on that rule; the role can take 30 minutes to apply.
garm-central siem add sentinel takes the endpoint, the rule's id, your tenant, the application's client id, and the certificate and its RSA key, of which only the key is sealed. A client secret (--secret-file) also works, but a certificate is preferred. Data can take 15 minutes to appear. Sentinel has no OCSF support of its own, so the table promotes the class, category, severity, activity, ids of the event, machine and actor to columns and keeps the whole event in RawData; query it in KQL as below. For long retention, the Basic and Auxiliary table plans cost less.
az deployment group create --resource-group <resource group> --template-file deploy/sentinel/agentgarm-ocsf.bicep --parameters workspaceName=<workspace>garm-central siem add sentinel --endpoint <logs ingestion endpoint> --dcr <immutable id> --tenant <tenant id> --client-id <client id> --cert-file siem.crt --key-file siem.key
garm-central siem test sentinelAgentGarmOCSF_CL
| where TimeGenerated > ago(1d) and class_uid == 2004
| project TimeGenerated, severity_id, device_uid, title = tostring(RawData.finding_info.title)4. Read lag, rejects and gaps
garm-central siem list, or Settings → SIEM, shows each sink's state (active, paused, or stopped), how far behind it is, its rejects and gaps, its last delivery and its last error. A SIEM that is down or busy is retried, backing off up to 5 minutes; after 5 failed tries Central Lite alerts siem_sink_failing, and a sink more than 15 minutes behind raises siem_lag.
A refused credential (401 or 403) stops the sink (siem_credential_refused), and so does a setting the SIEM refuses for every event, such as an index the token may not write (siem_sink_misconfigured); fix it, then garm-central siem resume, or Resume in the console, carries on from where it stopped. An event the SIEM refuses on its own is set aside as a reject (siem_rejects) and the rest go on. If a sink stays stopped longer than retention keeps its events, retention wins: the range it could not send is recorded as a gap (siem_gap), chained in the audit log, and skipped. garm-central siem show lists a sink's newest rejects, with the SIEM's message, and its gaps; the console shows only their counts.
garm-central siem list
garm-central siem show splunk
garm-central siem resume splunk5. Rotate a credential
In the console, Settings → SIEM, edit the sink and give the new HEC token or client secret, with a passkey: Central Lite seals the new one, erases the old one, and the sink carries on from where it was. Leave the field empty to keep the stored credential. On the host, garm-central siem set-credential does the same from a file, and is how a Sentinel sink gets a certificate (or goes back to a client secret): the sink keeps its settings and carries on from where it was, so nothing is skipped. Upload the new certificate to the application before you switch, and remove the old one after.
Credentials are not in Central Lite's backups: after a restore, each sink stops with “credential not in this Central's keystore” until you set its credential again, which also restarts it. Settings → SIEM also removes a sink and shows its rejects and gaps in detail.
garm-central siem set-credential sentinel --cert-file siem-new.crt --key-file siem-new.keySee also
Back up and restore Central LiteWrite Central Lite's encrypted backups by hand and on a schedule, and restore one so that no enrolled node refuses what Central Lite signs next.