Vortos
Alerts

Alerts

Alert dedupe, flap damping, escalation with quiet hours and on-call rotation, SSRF-hardened webhook delivery, and acknowledgement.

Alerts

vortos-alerts is the framework's paging layer — the thing that turns "something is wrong" signals from Health, Backup, Deploy, and your own rules into an actual notification, sent to the right channel, to the right person, without flooding anyone the moment a flaky check starts flapping.

A notifier outage must never block the system it's monitoring

NotifierInterface::notify() is contractually forbidden from throwing into the dispatcher. A failed delivery comes back as NotificationResult::failed(), and OutboxNotifier handles retry and fallback from there. Whatever you're monitoring keeps running even if every notification channel is down.

Dedupe — one incident, one notification

Without dedupe, a flapping health check or a burst of identical errors pages someone once per occurrence. Dedupe groups alert events by a stable Fingerprint and tracks an AlertState (open, resolved) per fingerprint — the same underlying problem firing repeatedly inside a dedupe window produces exactly one notification, not one per event.

Flap damping

A signal that flips open → resolved → open rapidly (a database connection that drops for two seconds every few minutes, say) is the specific failure mode dedupe alone doesn't solve — each re-open after a resolve is a new incident from dedupe's perspective. FlapDamper exists for exactly this:

$outcome = $flapDamper->recordTransition($state, now: new \DateTimeImmutable(), window: $dedupeWindow);

$outcome->shouldEscalate; // true exactly once — the first time the flap threshold is crossed
$outcome->isDamped;       // true on every subsequent flap within the same window

After more than maxTransitions open→resolve→open cycles within a window, the first crossing notifies once ("this is flapping"), and every cycle after that within the same window is silently damped — not because the problem stopped, but because you already know about it and another page adds nothing.

Escalation

EscalationEngine::tick() is a pure function of (event, state, ack, active silences, now) — every escalation timer is deterministically testable because time is always an explicit input, never read from the system clock inside the engine itself.

[$decision, $state] = $engine->start($event, now: new \DateTimeImmutable());
// decision is EscalationDecision::notify(tier: 0, reason: 'initial page')
//          or  EscalationDecision::suppress(...) if quiet hours / a silence applies

Escalation respects two suppression mechanisms before ever paging:

  • Quiet hours (QuietHoursPolicy) — configured windows where non-critical alerts don't page a human; they still get recorded, just not delivered urgently.
  • Maintenance silences (MaintenanceSilence) — an explicit, time-boxed "we know, don't page" declared ahead of planned work.

If neither applies, the engine consults OnCallRotation to decide who tier 0 actually pages, and escalates to the next tier if nobody acknowledges within the policy's window.

Acknowledgement

php bin/console vortos:alerts:ack <fingerprint> --by=alice

An acknowledgement halts escalation for that fingerprint — AckTokenSigner issues a signed token so a one-click "acknowledge" link in a notification (Slack, email) can ack without requiring the responder to be logged into anything; the signature is what prevents a guessed or forged ack token from silencing someone else's incident.

Silencing for planned maintenance

php bin/console vortos:alerts:silence --rule=database-latency --duration=2h --reason="planned failover test"

A silence is explicit and time-boxed — it expires automatically, so a silence someone forgot to lift doesn't quietly suppress a real incident weeks later.

Notifiers and the SSRF-hardened webhook

NotifierInterface is the Ops Kit port. Shipped drivers: slack, telegram, ses (email), and a generic webhook for anything else.

The generic webhook driver is the one that needs the most care — it sends to a URL an operator configures, which means it needs to refuse to be turned into an internal network probe. SsrfGuard is the single place this validation lives:

  • Scheme — only https is allowed outside local development.
  • Address ranges — every private, link-local, loopback, multicast, and cloud-metadata range is denied, including 169.254.169.254 (the address that serves cloud instance credentials on AWS/GCP/Azure) and IPv6 equivalents (ULA, link-local).
  • Redirects — disabled at the HTTP transport layer entirely, not just checked once at the start. SsrfGuard validates the declared destination; redirect-following is turned off so a benign-looking URL can't 302 its way to an internal address after the check passes.
$guard = new SsrfGuard();
$guard->assertSafe('https://hooks.example.com/webhook'); // fine
$guard->assertSafe('http://169.254.169.254/latest/meta-data/'); // throws SsrfViolationException

This exists because webhook URLs are operator input

A webhook notifier's destination is configuration, set by whoever administers your alerting — not a hardcoded, trusted value. Treat it the same way you'd treat any other user-supplied URL: validate before you fetch.

Outbound rate limiting

SlidingWindowOutboundRateLimiter caps how many notifications can go out to a channel in a window, independent of dedupe and escalation — a defense against a misconfigured rule or a runaway loop generating thousands of distinct fingerprints, each individually deduped but collectively still capable of hammering a Slack channel or an SMS gateway into rate-limit hell on the receiving end.

Rules

AlertRule pairs a condition with a severity and routing target. Conditions are pluggable:

ConditionEvaluates
ThresholdConditionA numeric sample crossing a threshold
ResourceConditionA resource-usage sample (disk, memory, connection pool)
CertExpiryConditionA TLS certificate's remaining validity window
SloBurnConditionAn SLO error-budget burn rate
NoConditionAlways fires — for manual or always-on rules

Declare your rules in config/alerts.php — a list of AlertRule (or a closure returning one). This is loaded by AlertRuleSetFactory; you no longer override the AlertRuleSet service definition. Without the file the rule set is empty and alerting does nothing.

config/alerts.php
use Vortos\Alerts\Rule\AlertRule;
use Vortos\Alerts\Rule\AlertRuleKind;
use Vortos\Alerts\Severity;

return [
    new AlertRule(
        id: 'db-latency',
        severity: Severity::Warning,
        kind: AlertRuleKind::Threshold,
        condition: $latencyCondition,
        forDuration: 120,
    ),
];
# validate rule definitions before deploying them
php bin/console vortos:alerts:rules:validate

# send a synthetic test alert through the full pipeline (dedupe, routing, delivery)
php bin/console vortos:alerts:test --rule=database-latency

Integrations

Alerts wires into other packages as event sinks rather than the other way around, so those packages never depend on Alerts directly:

SourceWhat triggers an alert
HealthHealthProbeAlertSource, CapacityAlertSource, CertExpiryAlertSource, SyntheticUptimeAlertSource
BackupBackupEventAlertSink — a failed backup or failed restore drill
DeployDeployAuditAlertSink — deploy and rollback audit events
SLO burn rateSloBurnAlertSource, via a pluggable SloBurnRateProviderInterface

CLI

php bin/console vortos:alerts:drain              # flush queued outbox deliveries
php bin/console vortos:alerts:ack <fp> --by=me   # acknowledge an open alert
php bin/console vortos:alerts:silence --rule=... --duration=2h
php bin/console vortos:alerts:rotation:show       # who is currently on call
php bin/console vortos:alerts:rules:validate
php bin/console vortos:alerts:test --rule=...

Step-by-step: wiring a Slack channel

Configure the notifier and a channel:

config/alerts.php
use Vortos\Alerts\DependencyInjection\VortosAlertsConfig;

return static function (VortosAlertsConfig $config): void {
    $config->channel('ops')
        ->notifier('slack')
        ->webhookUrl($_ENV['SLACK_OPS_WEBHOOK']);
};

Define a rule routing to it:

$config->rule('database-latency')
    ->condition(new ThresholdCondition(metric: 'db.p99_ms', operator: ThresholdOperator::GreaterThan, value: 500))
    ->severity(Severity::Critical)
    ->route('ops');

Validate and test before relying on it:

php bin/console vortos:alerts:rules:validate
php bin/console vortos:alerts:test --rule=database-latency

On this page