Validation status

deadair is tested against live Elastic, OpenSearch, and Microsoft Sentinel APIs, with local contract fixtures covering paths that are unsafe or impractical to reproduce live. This page says what runs in CI, what was checked manually, and what remains unproven. For finding terminology and triage, see Read the findings.

Test coverage

The live CI matrix covers Elastic Security 8.19.19 and 9.4.4, plus OpenSearch Security Analytics 2.19.6 and 3.7.0. Sentinel unit and contract tests run with the normal Go suite. The live Sentinel conformance test is opt-in and does not run on a schedule.

The MSSP lab includes healthy, failed, and unreachable tenants. It exercises partial failures, per-instance state, schema drift, downtime suppression, redaction, and exporter labels.

Sentinel live conformance

The Sentinel adapter was exercised against a disposable workspace in UK South using a custom workspace-scoped reader limited to the operations in the Sentinel credential guide. The read-only conformance test is gated by DEADAIR_IT_SENTINEL=1; it expects pre-seeded fixtures and never creates or changes Azure resources.

The workspace used for the 2026-08-22 pass deliberately included missing, stale, late, and incompatible data paths. The recorded scan found them and exited 1.

On 2026-09-05, a fresh UK South lab tested two synthetic firewall feeds in CommonSecurityLog. The London feed stopped while the Manchester feed kept the table fresh. Only London exceeded its producer threshold; restoring it produced a confirmed recovery in the report diff. A maintenance window suppressed London’s alert while retaining the stale observation. The certificate-backed reader also passed all three write-denial checks. A separate live exporter run observed quiet feeds and cleared both stale gauges after resumed traffic became queryable. The full read-only conformance run passed again, including separate Scheduled and NRT clocks and a successful summary run whose output matched the exact Basic-table input bin.

On 2026-09-06, a GitHub-hosted OIDC run tested revision 6a2e74a against two disposable UK South workspaces. The identity trusted one repository branch and had a custom read-only role scoped to those workspaces. Available, missing and denied table inputs returned exits 0, 1 and 2 respectively. Denied access failed the Action even with fail-on-findings: false. The run checked redacted artifacts, the rendered job summaries, all three write-denial probes, a two-workspace fleet candidate scan, and Azure CLI session cleanup.

The same table names had different schemas in each workspace. A missing ClientIP column and a text-valued DestinationPort both passed deadair’s dependency gate but failed native zero-row query checks. Fallback columns and a workspace parser compiled successfully in both workspaces. These cases confirm the boundary: Sentinel candidate checks assess dependencies, not full query compatibility. The run used synthetic data in one subscription; it did not test Azure Lighthouse or GitHub Environment approval rules.

The 2026-08-22 live workspace covered:

• enabled Scheduled rules from the stable API and an NRT rule from the preview API;
• direct local tables, join, union, let, and mixed present/missing inputs;
• bare and called zero-parameter saved workspace functions plus a closed scalar parameterized function;
• a literal watchlist, a configured same-subscription workspace, and exact remote Sentinel onboarding before remote table or Logs evidence;
• a native ASIM call whose semantic PartialError remains deliberately unassessed;
• structural Basic-table to Analytics-summary-table to analytics-rule lineage;
• a fresh shared network table with a stale Palo Alto Networks/PAN-OS slice isolated by a closed literal filter;
• a successful native summary run after the current ARM definition became visible, plus an exact 20-minute Basic-source to Analytics-output bin and count match;
• rule-aware zero-row table permission probes;
• Scheduled event-time freshness, NRT ingestion-time freshness, and paired TimeGenerated/ingestion_time() lag samples;
• explicit incomplete evidence when Scheduled and NRT rules share one source;
• missing, stale, bounded-empty, Basic-plan, and Auxiliary-plan table cases; and
• the exact empty installed-package and no-positive-template-link provenance path.

This is the scope of the recorded 2026-08-22 live pass. The certificate-backed scanner also passed separate exact 403 checks for alert-rule DELETE, table DELETE, and shared-key POST. The native ASIM case intentionally remains unassessed on PartialError. The empty Content Hub inventory proves the no-link path only; it does not prove a positive installed-package association.

Separate manual lab checks used a restricted identity that could query one table but not another. The permitted scan stayed partial instead of treating the denied table as missing. The same lab also exercised candidate gates for Scheduled JSON, NRT JSON, and Azure-Sentinel YAML, covering healthy, missing, and deliberately unassessed inputs without installing a rule. Those one-off checks are not executed by TestSentinelReadOnlyLab. Basic-table Scheduled and NRT rules were also accepted while disabled but rejected when enabling them because their queries could not execute; the temporary rules were removed without creating an incident.

Run the same read-only conformance test with your own pre-seeded disposable workspace. The test requires a certificate-backed scanner identity and deliberately rejects Azure CLI credentials:

unset AZURE_CLIENT_SECRET
AZURE_TOKEN_CREDENTIALS=EnvironmentCredential \
AZURE_TENANT_ID=<tenant-id> \
AZURE_CLIENT_ID=<scanner-app-client-id> \
AZURE_CLIENT_CERTIFICATE_PATH=/restricted/path/scanner.pem \
DEADAIR_SENTINEL_SCANNER_CLIENT_ID=<same-scanner-app-client-id> \
DEADAIR_IT_SENTINEL=1 \
DEADAIR_AZURE_SUBSCRIPTION_ID=<subscription-id> \
DEADAIR_AZURE_RESOURCE_GROUP=<resource-group> \
DEADAIR_SENTINEL_WORKSPACE=<workspace-resource-name> \
DEADAIR_SENTINEL_REMOTE_WORKSPACE=<remote-workspace-resource-name> \
go test -tags=integration ./integration -run '^TestSentinelReadOnlyLab$' -v

See the integration guide for the scanner identity contract and required fixture names. Authentication uses the EnvironmentCredential path in DefaultAzureCredential; no tenant, subscription, workspace ID, or credential is embedded in the test.

Current limits

Microsoft Sentinel

• Sentinel live conformance is manual. A successful run proves the tested workspace and identity, not every Sentinel cloud, region, RBAC layout, or KQL construct.
• The Sentinel GitHub Action reuses a caller-established Azure CLI login through DefaultAzureCredential. The GitHub-hosted OIDC path was tested with branch-scoped trust in Azure public cloud. Other trust policies, clouds and cross-tenant layouts need their own checks.
• The Sentinel CLI currently targets Azure public cloud endpoints.
• Sentinel assesses direct local Analytics tables, saved workspace functions with closed scalar arguments, literal watchlists, metadata-backed ASIM functions with complete evidence, native ASIM calls with positive table and permission evidence, and configured literal workspace() tables. Tabular or runtime parameters, dynamic table/watchlist/workspace selection, external data, app(), resource(), ADX/cluster, and ARG remain unassessed. A native ASIM PartialError is unassessed.
• Filtered source activity was exercised against a stale Palo Alto Networks/PAN-OS slice while other vendors kept the shared table fresh. It requires one direct local Analytics table followed by a closed, parser-supported literal filter. Escaped and verbatim literals fail closed because their exact KQL value cannot be reconstructed safely. The bounded result is informational and does not replace table-wide health or create a finding. Each rule-filter collection is capped at 20 distinct queries. Tests verify that identical filters share an observation without merging rule identities, clocks or policy thresholds, and that later collections query again. Producer expectations use a separate collection.
• Summary-rule lineage remains structural ARM metadata. A separate path reads the latest completed LASummaryLogs run in a bounded seven-day window for relevant active summary rules, capped at 50 rules per scan. Fixtures reject missing or malformed ARM modification times, runs that predate the current ARM definition, inconsistent native timestamps, invalid provisioning, and invalid schedules. A successful run becomes incomplete once it exceeds its cadence and delay plus the documented eight-hour retry allowance. Failed or overdue runs create summary-pipeline findings; the policy decides whether they fail the gate. Recovery requires a new successful run, not a repeated read of an old result. The live lab proved structural lineage, a successful bounded runtime record, and the matching source/output bin under the scanner identity. On 5 September 2026, a second disposable lab removed a summary rule’s raw input table, observed a native failed run, then restored the table without changing the summary rule. A newer successful run recovered the finding in ordinary and redacted report diffs; the exporter changed from unhealthy to successful. The successful retry returned zero rows for an older bin, so this test proved runtime recovery, not current output freshness. Overdue-run transitions remain covered by contract tests. Azure advanced LASummaryLogs.RuleLastModifiedTime between bins while the ARM definition stayed unchanged, so deadair does not present that field as an exact revision ID. It requires the run to start after the ARM modification time and rejects native timestamps that are too old or later than the run. binDelay is measured in minutes, matching Azure Monitor’s summary-rule scheduling model and the Sentinel portal.
• Native-template and Content Hub provenance uses exact IDs and versions and is informational. A positive installed-package association has not been tested in a live lab.
• The configured cross-workspace lab uses two workspaces in one subscription. Cross-tenant Azure Lighthouse behavior has not been proven. Same-subscription source evidence remains conclusive, but scanner access alone cannot prove the creator credential still works across subscriptions. A fixture-tested path accepts only an exact successful SentinelHealth event after the latest rule change and within the expected run cadence plus scheduling delay for an eligible installed rule that references another subscription. Candidate, absent, old, ambiguous, mismatched, and non-successful evidence remains unassessed. deadair does not identify tenant boundaries separately, so Azure Lighthouse and other cross-tenant topologies remain unidentified and have not been live-tested. The corroboration path itself has not been tested live.
• Cross-workspace fixture tests require an exact GA onboardingStates/default resource before any remote table or Logs probe. Missing, denied, malformed, or mismatched onboarding evidence is unavailable. Microsoft’s hard limit is 20 workspaces per analytics-rule query; the fixtures pin deadair’s conservative choice to count the home workspace, so a verified rule with more than 19 distinct remotes is incompatible. Microsoft recommends no more than five total for performance. Azure Monitor separately warns at five workspace regions and blocks a scope spanning 20 or more; fixtures require every workspace location, normalize distinct regions, leave missing location unavailable, and make 20 or more regions incompatible after any alias proof. deadair has no separate five-region finding. Exact onboarding and the two-workspace same-subscription path were tested live; high workspace-count and region-count thresholds remain fixture-tested.
• Basic and Auxiliary tables are reported as incompatible. Required-field and unused-telemetry findings are unavailable because Sentinel does not provide authoritative rule field metadata or a bounded, authoritative per-table event and storage inventory for this scan.

All backends

• The version matrix does not cover every hosted service, SIEM topology, or parser convention.
• Fleet behavior is lab-tested, but production scan time and API limits depend on the number and size of the connected tenants.
• Low-volume and cadence checks need enough local scan history to establish a useful baseline.
• Redaction hides rule, source, dependency, lineage, provenance, workspace, watchlist, template, and package identifiers but does not remove sensitive counts, versions, timing, severity, lag, or coverage information. Dependency probe expressions are not serialized.
• Results are limited to the rules, sources, fields, and metadata visible to the scan credential. A known-good rule and source pair should be checked after changing credentials or scope.

Verify a deployment

Create the documented read-only credential, then run:

deadair check
deadair scan --redact --json --json-out deadair-redacted.json --html-out deadair-report.html

When schema visibility and persistent local state are approved:

deadair scan \
  --redact \
  --schema \
  --state-file deadair-state.json \
  --json \
  --json-out deadair-redacted.json \
  --html-out deadair-report.html

Start with one enabled rule whose expected source you know. Check its declared input, resolution, source state, and available runtime evidence. If that rule-source pair is wrong, fix credential scope, namespace selection, or source visibility before relying on the rest of the report.

The exporter binds to 127.0.0.1:9317 by default. Put it behind authenticated scraping before exposing it to shared infrastructure. Review redacted output before sharing it; do not put unredacted rule, source, field, tenant, workspace, watchlist, parser, summary-rule, template, package, or customer names in a public issue.