Skip to content
Back to solution packs catalog
Signed artifact v0.1.0 Storage & NVR Infrastructure Apache-2.0

Camera Semantic Search Health

io.serviceradar.camera-semantic-search-health

Monitors camera visual indexing queue backlog lag via SRQL and probes media-worker readiness endpoints for semantic clip search pipelines.

What's inside: 2 checks · 2 alert rules · 1 dashboard

Overview

Camera Semantic Search Health

First-party ServiceRadar Solution Pack (io.serviceradar.camera-semantic-search-health) monitoring camera visual indexing job backlog via SRQL and probing media-worker /readyz readiness endpoints.

Included Resources

Declarative Checks (2)

  • visual-index-queue-lag -- Core srql_query check monitoring queued visual indexing job backlog against configurable warning and critical thresholds.
  • media-worker-readiness -- Edge http_request check probing the media-worker /readyz endpoint (service_targets -> extract_field -> threshold_assert -> alert_output).

Stateful Alert Rules (2)

  • camera_visual_index_queue_lag (enabled) -- Opens a high-priority alert when visual indexing backlog exceeds the critical threshold.
  • camera_media_worker_unready (enabled) -- Opens a critical alert when the media-worker readiness probe transitions to down.

Authored Dashboards (1)

  • camera-semantic-search-overview -- Unified SRQL overview dashboard tracking semantic indexing pipeline availability, failing media worker probes, and open pipeline alerts.

Declarative Checks

Service & posture check DAGs (2)

Visual Index Queue Lag

visual-index-queue-lag

target: singleton Every 120s 4 nodes · 3 edges

Monitors queued camera visual indexing jobs via SRQL to detect embedding pipeline backlog or stalled workers.

signal value verdict SRQL_QUERY Visual Index Backlog Q... in:events log_name:"came... EXTRACT_FIELD Extract Pending Job Count pending_jobs THRESHOLD_ASSERT Backlog Thresholds pending_jobs ALERT_OUTPUT Visual Index Queue Ver... verdict -> service_state
Canonical check JSON (checks/visual-index-queue-lag.json)
{
  "alert_policy": {
    "cooldown_seconds": 300,
    "emit_ocsf_transition": true,
    "open_on_statuses": [
      "down"
    ],
    "priority": 2,
    "renotify_seconds": 21600,
    "threshold": 1,
    "window_seconds": 300
  },
  "credential_slots": [],
  "description": "Monitors queued camera visual indexing jobs via SRQL to detect embedding pipeline backlog or stalled workers.",
  "edges": [
    {
      "id": "e1",
      "source": "srql_1",
      "source_handle": "signal",
      "target": "extract_lag",
      "target_handle": "in"
    },
    {
      "id": "e2",
      "source": "extract_lag",
      "source_handle": "value",
      "target": "assert_lag",
      "target_handle": "in"
    },
    {
      "id": "e3",
      "source": "assert_lag",
      "source_handle": "verdict",
      "target": "out_1",
      "target_handle": "in"
    }
  ],
  "inputs": [
    {
      "default": {
        "$pack_param": "queue_lag_warn_count"
      },
      "description": "Warning threshold for queued visual indexing jobs",
      "key": "queue_lag_warn_count",
      "max": 100000,
      "min": 1,
      "type": "integer"
    },
    {
      "default": {
        "$pack_param": "queue_lag_crit_count"
      },
      "description": "Critical threshold for queued visual indexing jobs",
      "key": "queue_lag_crit_count",
      "max": 100000,
      "min": 1,
      "type": "integer"
    }
  ],
  "managed": true,
  "name": "Visual Index Queue Lag",
  "nodes": [
    {
      "config": {
        "limit": 100,
        "query": "in:events log_name:\"camera.visual_index.queued\" time:last_15m stats:\"count() as pending_jobs\"",
        "result_mode": "first_row",
        "timeout_ms": 5000
      },
      "id": "srql_1",
      "kind": "srql_query",
      "label": "Visual Index Backlog Query"
    },
    {
      "config": {
        "cast": "integer",
        "field": "pending_jobs",
        "output_key": "pending_jobs",
        "source_type": "srql_field"
      },
      "id": "extract_lag",
      "kind": "extract_field",
      "label": "Extract Pending Job Count"
    },
    {
      "config": {
        "crit": {
          "op": ">=",
          "value": "{{inputs.queue_lag_crit_count}}"
        },
        "field": "pending_jobs",
        "warn": {
          "op": ">=",
          "value": "{{inputs.queue_lag_warn_count}}"
        }
      },
      "id": "assert_lag",
      "kind": "threshold_assert",
      "label": "Backlog Thresholds"
    },
    {
      "config": {
        "verdict_map": {
          "degraded": "visual_index_queue_elevated",
          "down": "visual_index_queue_stalled",
          "healthy": "visual_index_queue_healthy",
          "unknown": "visual_index_queue_unknown"
        }
      },
      "id": "out_1",
      "kind": "alert_output",
      "label": "Visual Index Queue Verdict"
    }
  ],
  "pack_id": "io.serviceradar.camera-semantic-search-health",
  "pack_version": "0.1.0",
  "parameter_overrides": {},
  "schedule": {
    "interval_seconds": 120,
    "jitter_seconds": 15,
    "timeout_seconds": 30
  },
  "schema_version": 1,
  "slug": "visual-index-queue-lag",
  "state": "enabled",
  "target_binding": {
    "mode": "singleton"
  },
  "ui_layout": {
    "nodes": {
      "assert_lag": {
        "x": 640,
        "y": 120
      },
      "extract_lag": {
        "x": 360,
        "y": 120
      },
      "out_1": {
        "x": 920,
        "y": 120
      },
      "srql_1": {
        "x": 80,
        "y": 120
      }
    }
  }
}

Media Worker Readiness

media-worker-readiness

target: service_targets Every 60s 4 nodes · 3 edges

Probes the media-worker /readyz endpoint to confirm semantic clip inference readiness and response latency.

signal value verdict HTTP_REQUEST GET media-worker /readyz EXTRACT_FIELD Extract Readiness Latency latency_ms THRESHOLD_ASSERT Readiness Latency Thre... latency_ms ALERT_OUTPUT Media Worker Verdict verdict -> service_state
Canonical check JSON (checks/media-worker-readiness.json)
{
  "alert_policy": {
    "cooldown_seconds": 300,
    "emit_ocsf_transition": true,
    "open_on_statuses": [
      "down"
    ],
    "priority": 2,
    "renotify_seconds": 21600,
    "threshold": 1,
    "window_seconds": 300
  },
  "credential_slots": [],
  "description": "Probes the media-worker /readyz endpoint to confirm semantic clip inference readiness and response latency.",
  "edges": [
    {
      "id": "e1",
      "source": "probe_readyz",
      "source_handle": "signal",
      "target": "extract_ready",
      "target_handle": "in"
    },
    {
      "id": "e2",
      "source": "extract_ready",
      "source_handle": "value",
      "target": "assert_ready",
      "target_handle": "in"
    },
    {
      "id": "e3",
      "source": "assert_ready",
      "source_handle": "verdict",
      "target": "out_1",
      "target_handle": "in"
    }
  ],
  "inputs": [
    {
      "default": {
        "$pack_param": "worker_latency_warn_ms"
      },
      "description": "Warning latency threshold in milliseconds for media-worker readiness probes",
      "key": "worker_latency_warn_ms",
      "max": 15000,
      "min": 50,
      "type": "integer"
    },
    {
      "default": {
        "$pack_param": "worker_latency_crit_ms"
      },
      "description": "Critical latency threshold in milliseconds for media-worker readiness probes",
      "key": "worker_latency_crit_ms",
      "max": 30000,
      "min": 100,
      "type": "integer"
    }
  ],
  "managed": true,
  "name": "Media Worker Readiness",
  "nodes": [
    {
      "config": {
        "expected_statuses": [
          200
        ],
        "follow_redirects": false,
        "method": "GET",
        "timeout_ms": 5000,
        "url": {
          "$pack_param": "media_worker_readyz_url"
        }
      },
      "id": "probe_readyz",
      "kind": "http_request",
      "label": "GET media-worker /readyz"
    },
    {
      "config": {
        "cast": "number",
        "field": "latency_ms",
        "output_key": "latency_ms",
        "source_type": "field"
      },
      "id": "extract_ready",
      "kind": "extract_field",
      "label": "Extract Readiness Latency"
    },
    {
      "config": {
        "crit": {
          "op": ">=",
          "value": "{{inputs.worker_latency_crit_ms}}"
        },
        "field": "latency_ms",
        "warn": {
          "op": ">=",
          "value": "{{inputs.worker_latency_warn_ms}}"
        }
      },
      "id": "assert_ready",
      "kind": "threshold_assert",
      "label": "Readiness Latency Thresholds"
    },
    {
      "config": {
        "verdict_map": {
          "degraded": "media_worker_slow",
          "down": "media_worker_not_ready",
          "healthy": "media_worker_ready",
          "unknown": "media_worker_probe_unknown"
        }
      },
      "id": "out_1",
      "kind": "alert_output",
      "label": "Media Worker Verdict"
    }
  ],
  "pack_id": "io.serviceradar.camera-semantic-search-health",
  "pack_version": "0.1.0",
  "parameter_overrides": {},
  "schedule": {
    "interval_seconds": 60,
    "jitter_seconds": 10,
    "timeout_seconds": 20
  },
  "schema_version": 1,
  "slug": "media-worker-readiness",
  "state": "enabled",
  "target_binding": {
    "mode": "service_targets",
    "target_ids": [
      "http://serviceradar-media-worker.example.internal:8096/readyz"
    ],
    "url": "http://serviceradar-media-worker.example.internal:8096/readyz"
  },
  "ui_layout": {
    "nodes": {
      "assert_ready": {
        "x": 640,
        "y": 120
      },
      "extract_ready": {
        "x": 360,
        "y": 120
      },
      "out_1": {
        "x": 920,
        "y": 120
      },
      "probe_readyz": {
        "x": 80,
        "y": 120
      }
    }
  }
}

Observability Rules

Alert & event rules (2)

Camera Visual Index Queue Lag

camera_visual_index_queue_lag

stateful_alert_rule install: enabled severity: 20

Alerts when queued visual indexing jobs exceed the configured critical lag threshold.

threshold: 1 window: 300s cooldown: 300s
Canonical rule JSON (rules/visual-index-backlog-alert.json)
{
  "alert": {
    "severity": "critical",
    "summary": "Camera visual indexing queue backlog exceeded critical threshold"
  },
  "bucket_seconds": 60,
  "cooldown_seconds": 300,
  "description": "Alerts when queued visual indexing jobs exceed the configured critical lag threshold.",
  "event": {
    "log_name": "health.camera_visual_index_queue_lag"
  },
  "group_by": [
    "check_slug"
  ],
  "install_state": "enabled",
  "kind": "stateful_alert_rule",
  "match": {
    "check_slug": "visual-index-queue-lag",
    "status": "down"
  },
  "name": "camera_visual_index_queue_lag",
  "priority": 20,
  "renotify_seconds": 21600,
  "signal": "event",
  "threshold": 1,
  "window_seconds": 300
}

Camera Media Worker Unready

camera_media_worker_unready

stateful_alert_rule install: enabled severity: 20

Alerts when the media-worker /readyz readiness probe transitions to down.

threshold: 1 window: 300s cooldown: 300s
Canonical rule JSON (rules/media-worker-unready-alert.json)
{
  "alert": {
    "severity": "critical",
    "summary": "Camera semantic search media-worker /readyz endpoint is unready"
  },
  "bucket_seconds": 60,
  "cooldown_seconds": 300,
  "description": "Alerts when the media-worker /readyz readiness probe transitions to down.",
  "event": {
    "log_name": "health.camera_media_worker_unready"
  },
  "group_by": [
    "check_slug"
  ],
  "install_state": "enabled",
  "kind": "stateful_alert_rule",
  "match": {
    "check_slug": "media-worker-readiness",
    "status": "down"
  },
  "name": "camera_media_worker_unready",
  "priority": 20,
  "renotify_seconds": 21600,
  "signal": "event",
  "threshold": 1,
  "window_seconds": 300
}

Authored Dashboards

Portable dashboards (1)

Camera Semantic Search Health

camera-semantic-search-overview

4 panels

Overview of camera semantic search visual indexing queue backlog, media-worker /readyz health, and active pipeline alerts.

Monitored Semantic Search Services Unhealthy Semantic Search Checks Open Semantic Search Alerts Semantic Search Service Availability Breakdown
Canonical dashboard JSON (dashboards/camera-semantic-search-overview.json)
{
  "default_time_range": "last_24h",
  "description": "Overview of camera semantic search visual indexing queue backlog, media-worker /readyz health, and active pipeline alerts.",
  "metadata": {
    "category": "Storage & NVR"
  },
  "panels": [
    {
      "data_binding": {
        "value": "total"
      },
      "display_config": {
        "unit": "checks"
      },
      "layout": {
        "h": 3,
        "w": 4,
        "x": 0,
        "y": 0
      },
      "position": 0,
      "srql_query": "in:services stats:\"count() as total\"",
      "title": "Monitored Semantic Search Services",
      "visual_config": {},
      "visual_type": "stat"
    },
    {
      "data_binding": {
        "value": "failing"
      },
      "display_config": {
        "unit": "checks"
      },
      "layout": {
        "h": 3,
        "w": 4,
        "x": 4,
        "y": 0
      },
      "position": 1,
      "srql_query": "in:services available:false stats:\"count() as failing\"",
      "title": "Unhealthy Semantic Search Checks",
      "visual_config": {},
      "visual_type": "stat"
    },
    {
      "data_binding": {
        "value": "open_alerts"
      },
      "display_config": {
        "unit": "alerts"
      },
      "layout": {
        "h": 3,
        "w": 4,
        "x": 8,
        "y": 0
      },
      "position": 2,
      "srql_query": "in:alerts status:open stats:\"count() as open_alerts\"",
      "title": "Open Semantic Search Alerts",
      "visual_config": {},
      "visual_type": "stat"
    },
    {
      "data_binding": {
        "category": "available",
        "value": "total"
      },
      "display_config": {},
      "layout": {
        "h": 5,
        "w": 12,
        "x": 0,
        "y": 3
      },
      "position": 3,
      "srql_query": "in:services stats:\"count() as total by available\"",
      "title": "Semantic Search Service Availability Breakdown",
      "visual_config": {},
      "visual_type": "category"
    }
  ],
  "slug": "camera-semantic-search-overview",
  "title": "Camera Semantic Search Health",
  "variables": {},
  "version": 1
}

Parameters & Bindings

Config inputs & credential slots

Configuration parameters (5)

Key Type Default Bounds Description
queue_lag_warn_count integer 50 min: 1, max: 100000 Warning threshold for queued camera visual indexing jobs.
queue_lag_crit_count integer 250 min: 1, max: 100000 Critical threshold for queued camera visual indexing jobs.
worker_latency_warn_ms integer 500 min: 50, max: 15000 Warning latency threshold in milliseconds for media-worker readiness probes.
worker_latency_crit_ms integer 2000 min: 100, max: 30000 Critical latency threshold in milliseconds for media-worker readiness probes.
media_worker_readyz_url string "http://serviceradar-media-worker.example.internal:8096/readyz" — Readiness endpoint URL for the semantic search media-worker service.

Abstract credential slots (0)

100% secret-free archive

Solution Packs never embed secret material or environment UUIDs. Declared slots bind locally to CNPG unified credentials (network_credential_secrets or network_credential_rules) during installation.

No external credentials required by this pack.

ServiceRadar Developer Portal

Docs, SDKs, solution packs, plugins, and native add-ons for building on ServiceRadar.

© 2026 Carver Automation Corporation. All rights reserved.

ServiceRadar® is a registered trademark of Carver Automation Corporation.