Alert policies

Alert policies run server-side the moment an alert arrives, before it pages anyone. Match on what the alert looks like, then suppress it, delay it, auto-ack it, auto-close it, change its priority, or add tags — so the noise never reaches a phone at 3 a.m. in the first place.

1. Create a policy: match conditions plus actions

   A policy has two halves. The match side says which incoming alerts it applies to; the actions side says what happens to them. You can match on any combination of these fields:

   • messageRegex — regular expression tested against the alert message
   • priorities — P1 through P5
   • sources — the integration or pipeline that sent the alert
   • tags — matches if the alert carries any of the listed tags
   • entities — the alert's entity (host, service, job)

   Conditions you leave empty always pass, and multiple conditions combine with AND: an empty match is a catch-all that hits every alert. One action per policy is the usual shape, but you can combine them (e.g. set priority and add tags in one policy).

   The available actions:

   • suppress — drop the alert entirely. Nothing is stored, nothing pages.
   • delayMinutes — hold the alert before escalation starts (0–1440). If it closes on its own while held, nobody was ever paged.
   • autoAck — store the alert, mark it acknowledged, skip paging.
   • autoClose — store the alert, close it immediately, skip paging.
   • setPriority — rewrite the priority (P1–P5), then continue normal processing.
   • addTags — attach tags, then continue normal processing.

   The same policy over the API:

   curl -X POST https://api.ops-ping.com/v2/alert-policies \
     -H "Authorization: OpsPingKey YOUR_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{
       "name": "Drop synthetics below P3",
       "match": { "sources": ["synthetics"], "priorities": ["P4","P5"] },
       "actions": { "suppress": true }
     }'

   Note

   Match conditions are field-based only — there are no time-based conditions (nothing like "only outside business hours") and no rewriting of the message or other payload fields. If you need that behavior today, it has to live in whatever sends the alert.

2. Order matters — first match wins

   Policies are evaluated in order, and evaluation stops at the first match. If policy #1 suppresses an alert, policy #2 never sees it. That's the whole model: put specific rules on top, broad rules below. A catch-all at position #1 makes every policy under it dead code.

   The admin page reorders with up/down buttons; the API takes a full ordering in one call — the ids array must name every policy exactly once, and the new position in the array is the new order:

   curl -X POST https://api.ops-ping.com/v2/alert-policies/reorder \
     -H "Authorization: OpsPingKey YOUR_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{ "ids": ["POLICY_C", "POLICY_A", "POLICY_B"] }'

   The response is the policy list in its new evaluation order. An unknown id fails the whole request (404) and changes nothing — you never end up in a half-reordered state.

3. Dry-run before you commit

   A suppression rule that eats real alerts is worse than the noise it removed. Dry-run evaluates your whole policy chain — in its current order — against an alert and reports which policy would win and what it would do, without storing anything or paging anyone. The admin page pastes a sample payload; the API takes the same body:

   curl -X POST https://api.ops-ping.com/v2/alert-policies/dry-run \
     -H "Authorization: OpsPingKey YOUR_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{
       "alert": {
         "message": "Disk usage 91% on db-1",
         "priority": "P4",
         "source": "monitoring",
         "tags": ["db"],
         "entity": "db-1"
       }
     }'

   Every field has a default, so a minimal {"alert":{"source":"monitoring"}} is enough to test a source-scoped policy. The response names the winning policy and its effect:

   {
     "matched": { "id": "…", "name": "Drop synthetics below P3", "order": 0 },
     "outcome": "suppressed"
   }

   outcome is one of none (no policy matched — the alert flows through untouched), continue (matched, only mutated: setPriority/addTags — the response carries the resulting mutations), suppressed (dropped), or handled (stored but never paged — delay, auto-ack, or auto-close; a delay includes delayedUntil with the release timestamp).

   To replay history instead of inventing a sample, pass an existing alert by id — handy for "why didn't this one page us yesterday?"

   curl -X POST https://api.ops-ping.com/v2/alert-policies/dry-run \
     -H "Authorization: OpsPingKey YOUR_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{ "alertId": "ALERT_UUID" }'

   Tip

   Dry-run uses the same evaluator as the live alert pipeline — the preview can't drift from reality. But it previews the chain as it exists now: a policy you're still composing in the UI isn't in the chain until saved, so save first (disabled if you like), dry-run, then enable.

4. Scope it: team or tenant-wide

   A policy with a teamId only ever matches that team's alerts. A policy with no team is tenant-wide — it applies to every alert in the tenant, which is exactly what you want for tenant-level hygiene rules ("drop all P5 synthetics") and exactly wrong for a rule that only makes sense for one team. When in doubt, bind the team.

   Scoping interacts with ordering the way you'd expect: team policies and tenant-wide policies share one ordered list, so a tenant-wide rule above a team rule can shadow it. Keep broad, safe rules (suppression of obvious noise) high and specific reshaping rules below them.

   Every change is audited: alert_policy.create, alert_policy.update, alert_policy.delete, and alert_policy.reorder each write an audit log entry with the acting user, so you can always answer "who silenced our disk alerts?" And a disabled policy (enabled: false) stays in the list for reference but is skipped entirely during evaluation.