Incidents

Incidents represent periods where a monitor is considered unhealthy. Testable opens incidents after the monitor's failure rules are met and resolves them after the recovery rules are met.

Lifecycle

A scheduled check fails, a confirmation check fails, or a heartbeat condition is violated.
The monitor reaches its configured consecutive failure threshold.
An incident opens and notifications are sent according to monitor or organization settings.
Checks, pings, activity, and comments accumulate while the incident is ongoing.
The monitor reaches its configured consecutive success threshold or heartbeat recovery condition.
The incident resolves and the duration is recorded.

Incident Detection Settings

Non-heartbeat monitors can configure down consecutive tries, up consecutive tries, and additional confirmation checks. Confirmation checks are immediate retries on a different runner when possible, reducing noise from a single runner or region.

Incident Details

Incident detail pages show the monitor, current incident state, start and resolution time, duration, affected regions, activity, and comments. The central details panel changes by monitor type.

Monitor-Type Details

HTTP

HTTP incidents show root cause, summary, assertions, error details, affected regions, and request details. Request details can include packets sent and received for the selected region.

Ping and Port

Ping and Port incidents show root cause, summary, assertions, error details, affected regions, and traceroute details when available from the check execution.

Custom

Custom incidents show scenario execution details. If both caused-by and resolved-by executions exist, the page lets you switch between them.

Heartbeat

Heartbeat incidents show a heartbeat-specific panel with pings at incident start, pings at resolution, and configuration values such as heartbeat frequency, acceptable delay, minimum active sources, and maximum active sources.

Activity and Comments

Activity records incident state changes and related events. Comments support Markdown and can be added on monitors and incidents when the comments feature is enabled. New comment notifications can be configured in monitor or organization notification settings.

Best Practices

Use confirmation checks for flaky external dependencies before raising failure thresholds too high.
Set recovery thresholds so incidents do not flap open and closed during partial recovery.
Use comments for investigation notes, handoff context, and post-incident summaries.
Review incidents by monitor type because HTTP, custom, heartbeat, ping, and port failures expose different diagnostic data.