Sindh IT Portal — Facilitation DeskSpecification documents
Englishاردوسنڌي
← All documents

Analytics & KPIs

The complete measurement system for the Sindh IT Portal — Facilitation Desk (SITP): the nine metric families, seven role dashboards, GIS heatmap, public transparency layer, scheduled digests, exports, anomaly detection, self-serve reporting, and the master KPI catalog.

Field Value
Doc ID 17
Status Draft
Owner S&ITD / MAAHIR
Languages EN (master) · UR · SD
Related docs 03-non-functional-requirements/en.md, 04-functional-requirements/en.md, /specs/en/04-roles-permissions/, /specs/en/06-ticket-workflow/, 09-ai-ocr-architecture/en.md, /specs/en/15-tech-architecture/, 16-security-and-compliance/en.md

1. Purpose & Scope

This document specifies every metric, dashboard, report, and control the SITP analytics plane must produce. It is the single reference used by the engineering team (MAAHIR) to build materialized views, by the S&ITD product owner to accept dashboards, by departmental stakeholders to interpret their performance, and by the public to understand what is and is not published.

The analytics plane is responsible for three things:

  1. Operational insight — let officers, DGs, department admins, secretaries, the SACM/Minister, and the super admin see the live and historical state of the portal relative to SLAs and targets.
  2. Accountability & transparency — let companies track their own interactions and let the public see anonymized aggregates about how departments are serving the IT industry.
  3. Governance & cost control — let S&ITD govern data access, AI spend, retention, and audit completeness from the same plane.

This document covers the nine metric families, the seven role dashboards (including the public transparency dashboard), the GIS/district heatmap, scheduled digests, exports, anomaly detection and SLA-breach prediction, saved views / favorites / embeddable widgets, a definitions glossary, data freshness and accuracy controls, and the self-serve report builder (Metabase). The underlying architecture (MariaDB materialized views/cubes, Metabase embedding, ECharts/Recharts, WebSocket live push, server-side export, cron digests) is summarized in §3 and detailed in /specs/en/15-tech-architecture/ §14.


2. Analytics Principles

Every metric, dashboard, and report in SITP obeys the following ten principles. They are referenced by ID (AP-1 … AP-10) throughout this document and in acceptance criteria.

ID Principle What it means in practice
AP-1 Real-time + historical, together The same metric is computable both "live" (current value over a WebSocket push, ≤ 5 s freshness for count-style KPIs) and "historical" (any bounded date range, with day/week/month/year grain). A dashboard tile never silently mixes grains; the grain is labelled on the tile.
AP-2 Drill-down to the ticket Every aggregate (count, average, rate) is a hyperlink through the filtered ticket list. A user who sees "47 breaching tickets in Labour dept" can click through to the 47 tickets, scoped by their RBAC, without leaving the dashboard.
AP-3 Role-scoped everywhere Every query carries the viewer's role, department, section, and ABAC confidential/VIP scope. There are no "god queries" in dashboards. The Public role sees only anonymized aggregates (see §11).
AP-4 Anonymization before public exposure Any metric reachable by the public dashboard is sourced from a separate anonymized aggregate table (see §3.3) populated by a scheduled worker, never computed live from ticket rows. Suppression rules (see §11.3) apply at publish time.
AP-5 One definition per metric Each KPI has exactly one canonical definition, formula, unit, and rounding rule in the glossary (§16). Two dashboards showing "CSAT" show the same number for the same period and scope because they read the same materialized view.
AP-6 Targets are configurable, not hard-coded Targets (SLA windows, CSAT goals, breach thresholds) live in the configuration tables and flow into targets shown on tiles. Changing a target does not require a code change.
AP-7 Cost-aware AI metrics Every AI-family KPI is paired with the engine and cost dimension. Token/cost totals roll up per engine, per capability, per department, and per day, so AI spend is governable from the same dashboards that show AI quality.
AP-8 Drift and freshness are visible Each dashboard carries a "last refreshed" timestamp and a freshness SLA badge. If the underlying worker is lagging beyond its SLA, the tile is flagged stale rather than reported as current (see §17).
AP-9 Auditable exports Every export (PDF/Excel/CSV) is itself an audited event: who, what filters, what grain, when, and a presigned download link logged. Sensitive exports (e.g., confidential-ticket extracts) require step-up auth.
AP-10 Multilingual labels, language-agnostic IDs Dashboard labels render in EN/UR/Sindhi per the viewer's locale; KPI IDs (KPI-VOL-007) and metric keys never translate, so cross-language reports are comparable (see _glossary.md).

3. Analytics Architecture (Recap)

This section recapitulates the analytics-specific portion of /specs/en/15-tech-architecture/ §14 so this document is self-contained. The full component-by-component rationale lives in the technical architecture doc.

3.1 Aggregation strategy

Dashboard reads never hit transactional tables directly. The Analytics module maintains a set of materialized views and lightweight cubes in MariaDB, refreshed by scheduled BullMQ workers on cadences defined per KPI (see §6). The pattern is:

Heavy historical queries and Metabase ad-hoc reads run against a read replica once read replicas are introduced (see /specs/en/15-tech-architecture/ §5.6). Until then, Metabase is pointed at a low-priority connection pool with its own throttle so analyst reads cannot starve the request path.

3.2 Visualization stack

3.3 Public transparency separation

Public-facing dashboards read only from anl_pub_*. No live query path connects the public route to the transactional or even the internal aggregate tables. This guarantees that a bug in the public UI cannot leak a ticket row, a company name, or an officer name. The nightly worker that populates anl_pub_* applies the suppression rules in §11.3.

3.4 Live-push and staleness contract

Live tiles subscribe to channels scoped by RBAC; the gateway never trusts a client-asserted scope. Pushes are debounced (e.g., a count tile re-renders at most once every 5 s even under burst). Each tile carries a last_refreshed_at timestamp and a freshness SLA (see §17). If the worker falls behind, the tile is marked stale with a visible badge; stale data is never silently presented as current.


4. Analytics Data Flow (Diagram + Description)

The diagram shows how a domain event (a ticket state change, an AI call, a notification delivery, an audit write) flows through the analytics plane and surfaces on dashboards, digests, exports, and the public page.

flowchart LR subgraph Src["Sources (system of record)"] TKT["Tickets module<br/>(MariaDB: tkt_*)"] ORG["Org/RBAC module<br/>(org_*)"] AI["AI service<br/>(ai_* audit rows)"] NOT["Notifications<br/>(not_*)"] AUD["Audit log<br/>(aud_event)"] COM["Comms<br/>(com_*)"] end subgraph ETL["Aggregation plane (BullMQ workers)"] EVT["Domain-event fan-out<br/>(Redis pub-sub)"] MV["Materialized-view refreshers<br/>(anl_mv_* · 5 min / hourly)"] CUBE["Cube refreshers<br/>(anl_cube_* · hourly / nightly)"] PUB["Public anonymizer worker<br/>(anl_pub_* · nightly)"] end subgraph Store["Analytics store (MariaDB)"] AMV[("anl_mv_*")] ACB[("anl_cube_*")] APB[("anl_pub_*")] CFG[("anl_cfg_*<br/>targets, thresholds, digests")] end subgraph Surf["Surfaces"] DASH["Role dashboards<br/>(ECharts/Recharts)"] WS["WebSocket live push"] DIG["Scheduled digests<br/>(email/SMS/WA)"] EXP["Exports<br/>(Puppeteer / ExcelJS)"] MB["Metabase embedded"] PUBDASH["Public transparency dashboard"] ANOM["Anomaly & SLA-breach predictor"] end TKT --> EVT ORG --> EVT AI --> EVT NOT --> EVT AUD --> EVT COM --> EVT EVT --> MV EVT --> CUBE MV --> AMV CUBE --> ACB AMV --> PUB PUB --> APB AMV --> DASH ACB --> MB AMV --> EXP AMV --> DIG AMV --> ANOM EVT --> WS WS --> DASH APB --> PUBDASH CFG --> DASH CFG --> DIG CFG --> ANOM ANOM -. breach alert .-> DIG ANOM -. breach alert .-> WS

Written description. Domain events from every stateful module — ticket transitions, org/RBAC changes, AI call audit rows, notification delivery receipts, audit-log writes, and comms activity — are published onto a Redis pub-sub fan-out. Three classes of BullMQ worker consume these events and maintain the analytics store: materialized-view refreshers keep anl_mv_* fresh at 5-minute or hourly cadence; cube refreshers rebuild anl_cube_* hourly or nightly; and the public anonymizer worker reads anl_mv_* once nightly and writes the fully-anonymized, suppression-applied anl_pub_* table that only the public dashboard ever reads. Configuration — targets, thresholds, digest schedules, anomaly rules — lives in anl_cfg_* and feeds both the surfaces (so tiles show target lines) and the anomaly/digest workers.

Six surfaces consume the analytics store: the role dashboards (custom ECharts/Recharts, fed by anl_mv_* plus WebSocket live push for count tiles); Metabase embedded for ad-hoc exploration over anl_cube_*; the scheduled digests rendered from anl_mv_*; the export workers producing PDF/Excel/CSV from anl_mv_*; the anomaly detector and SLA-breach predictor reading trends in anl_mv_* against thresholds in anl_cfg_*; and the public transparency dashboard reading only from anl_pub_*. Breach-prediction alerts flow back through the notifications pipeline (digests, in-app, WebSocket) and never expose raw ticket data to recipients outside their RBAC scope.


5. Conventions Used in the KPI Catalog

The KPI catalog (§6) and the master table (§18) use these conventions:


6. The Nine Metric Families — KPI Catalog

6.1 Family 1 — Volume & Throughput (VOL)

Volume and throughput measure how much demand the portal is absorbing and how quickly that demand is being closed out, by every meaningful cut (department, category, district, channel, severity, company-size tier).

ID Name Definition / Formula Unit Target Source · Audience Refresh
KPI-VOL-001 Tickets Created Count of tickets entering New state in the period. count tkt_* · SA, DA, DG, OF, SEC, MIN live
KPI-VOL-002 Tickets Resolved Count of tickets reaching Resolved in the period. count tkt_* · SA, DA, DG, OF, SEC, MIN 5m
KPI-VOL-003 Tickets Closed Count of tickets reaching terminal Closed (auto-close or accepted) in the period. count tkt_* · SA, DA, DG, SEC, MIN 5m
KPI-VOL-004 Throughput Rate Tickets Resolved ÷ Tickets Created over the period. ratio ≥ 0.95 tkt_* · SA, DA, DG, SEC, MIN hourly
KPI-VOL-005 Open Backlog Count of tickets in non-terminal states at the instant of measurement. count tkt_* · SA, DA, DG, OF live
KPI-VOL-006 Net Backlog Change (Open Backlog_end − Open Backlog_start) over the period. count ≤ 0 tkt_* · SA, DA, DG, SEC, MIN daily
KPI-VOL-007 Volume by Channel Ticket count grouped by intake channel (web, email, SMS, WhatsApp, IVR, walk-in). count tkt_*, int_* · SA, DA, DG, SEC, MIN hourly
KPI-VOL-008 Volume by Category Ticket count grouped by service category/sub-category. count tkt_* · SA, DA, DG, SEC, MIN hourly
KPI-VOL-009 Volume by District Ticket count grouped by requester district (GIS feed). count tkt_*, org_* · SA, DA, DG, SEC, MIN, PUB hourly
KPI-VOL-010 Volume by Department Ticket count grouped by assigned department/section. count tkt_*, org_* · SA, DA, DG, SEC, MIN hourly
KPI-VOL-011 Volume by Company-Size Tier Ticket count grouped by entity size tier (freelancer / startup / SME / enterprise / foreign branch). count tkt_*, org_* · SA, SEC, MIN daily
KPI-VOL-012 Peak-Hour Volume Maximum tickets-created-in-an-hour in the period, with the hour timestamp. count tkt_* · SA, DA, DG, SEC, MIN hourly
KPI-VOL-013 Volume Growth (MoM / YoY) ((Tickets Created this period − Tickets Created prior comparable period) ÷ prior). percent anl_mv_* · SA, SEC, MIN daily
KPI-VOL-014 Draft → Submitted Conversion Tickets Submitted ÷ (Drafts Started) over the period. ratio ≥ 0.70 tkt_* · SA, DA daily
KPI-VOL-015 Reopened Volume Count of tickets transitioned Resolved → Reopened in the period. count tkt_* · SA, DA, DG, SEC, MIN hourly

6.2 Family 2 — Speed (SPD)

Speed measures latency across the ticket lifecycle: how fast the company hears back, how fast the issue is resolved, and where time accumulates inside the workflow.

ID Name Definition / Formula Unit Target Source · Audience Refresh
KPI-SPD-001 First Response Time (FRT) Time from New to first outbound company-facing message (any channel) on a ticket. Reported as median, p90, p95, mean. hours median ≤ 4 h tkt_*, not_* · SA, DA, DG, OF, SEC, MIN hourly
KPI-SPD-002 Turnaround Time (TAT) Working time from creation to Resolved, SLA-pause-aware (weekends, holidays, await-on-company excluded). Reported as median, p90, p95. days median ≤ 5 d tkt_* · SA, DA, DG, SEC, MIN hourly
KPI-SPD-003 Time-to-Triage Time from New to Triaged (categorized + routed). hours ≤ 2 h tkt_* · SA, DA, DG hourly
KPI-SPD-004 Time-to-Assign Time from Triaged to Assigned (owner set). hours ≤ 4 h tkt_* · SA, DA, DG hourly
KPI-SPD-005 Time-to-Acknowledge Time from Assigned to first internal note or status change by the assignee. hours ≤ 8 h tkt_* · SA, DA, DG, OF hourly
KPI-SPD-006 Mean Handle Time Sum of officer-active working time ÷ tickets resolved, in the period. Excludes await-on-company. hours ≤ 8 h tkt_*, aud_* · SA, DA, DG, OF daily
KPI-SPD-007 Time-in-State (per state) Median dwell time in each state (New, Triaged, Assigned, InProgress, AwaitingCompany, Resolved). hours tkt_*, aud_* · SA, DA, DG, OF hourly
KPI-SPD-008 Aging Buckets Count of open tickets bucketed by age: 0–2 d, 3–5 d, 6–10 d, 11–20 d, 21+ d. count 0 in 21+ bucket tkt_* · SA, DA, DG, OF, SEC, MIN hourly
KPI-SPD-009 Await-on-Company Duration Median cumulative paused time awaiting company response, per ticket. hours tkt_* · SA, DA, DG, CO daily
KPI-SPD-010 Time-in-Escalation Median cumulative time a ticket spends on an escalation rung before being picked up. hours ≤ 24 h tkt_*, aud_* · SA, DA, DG, SEC, MIN hourly
KPI-SPD-011 p95 Resolution Time 95th-percentile SLA-aware TAT, reported per department and per category. days p95 ≤ 10 d tkt_* · SA, DA, DG, SEC, MIN daily
KPI-SPD-012 Reassignment Latency Time lost due to internal reassignment (time a ticket sits after a re-route before the new owner acts). hours median ≤ 4 h tkt_*, aud_* · SA, DA, DG daily
KPI-SPD-013 Resolution Cycle Time by Category Median SLA-aware TAT grouped by service category (for benchmarking). days tkt_* · SA, DA, DG, SEC, MIN daily

6.3 Family 3 — SLA & Escalation (SLA)

SLA metrics measure conformance to the configurable response/resolution windows (default 2 / 5 / 10 days) and the behaviour of the escalation ladder.

ID Name Definition / Formula Unit Target Source · Audience Refresh
KPI-SLA-001 SLA Compliance Rate Tickets meeting SLA ÷ Tickets resolved in the period (SLA-aware). percent ≥ 95 % tkt_*, anl_cfg_* · SA, DA, DG, SEC, MIN, PUB hourly
KPI-SLA-002 SLA Breach Count Count of tickets whose SLA clock exceeded the configured window. count tkt_* · SA, DA, DG, OF, SEC, MIN live
KPI-SLA-003 SLA At-Risk Count Count of open tickets projected to breach within the next 24 h (see §14). count tkt_*, predictor · SA, DA, DG, OF live
KPI-SLA-004 Escalation Rate Tickets escalated (any rung) ÷ Tickets created in the period. ratio ≤ 0.20 tkt_*, aud_* · SA, DA, DG, SEC, MIN hourly
KPI-SLA-005 Escalation Rung Distribution Count of tickets currently on rung 1 (2 d), rung 2 (5 d), rung 3 (10 d / SACM). count tkt_*, aud_* · SA, DA, DG, SEC, MIN hourly
KPI-SLA-006 Repeat Escalation Rate Tickets re-escalated after de-escalation ÷ Tickets escalated. ratio ≤ 0.10 tkt_*, aud_* · SA, DA, DG daily
KPI-SLA-007 SLA Pause Events Count of SLA-clock pauses triggered (await-on-company, weekend, Sindh public holiday). count tkt_*, aud_* · SA, DA, DG hourly
KPI-SLA-008 RTI SLA Compliance For the RTI special category: RTI tickets meeting statutory deadline ÷ RTI tickets resolved (Sindh Transparency & RTI Act 2016 deadlines). percent 100 % tkt_* · SA, DA, DG, SEC, MIN, PUB daily
KPI-SLA-009 SLA Compliance by Severity KPI-SLA-001 grouped by severity (low / normal / high / urgent). percent ≥ 95 % per sev tkt_* · SA, DA, DG, SEC, MIN hourly
KPI-SLA-010 SLA Compliance by Department KPI-SLA-001 grouped by department (drives the dept leaderboard). percent ≥ 95 % per dept tkt_*, org_* · SA, DA, DG, SEC, MIN, PUB hourly
KPI-SLA-011 Breach Recovery Time Median time from a breach occurring to the ticket being resolved. hours ≤ 48 h tkt_*, aud_* · SA, DA, DG daily
KPI-SLA-012 Holiday/Weekend Pause Share SLA pauses due to non-business time ÷ total SLA pause events. ratio tkt_*, aud_* · SA, DA, DG daily

6.4 Family 4 — Quality & Outcome (QLT)

Quality metrics measure whether resolutions actually satisfied the company, whether the proof-of-resolution gate held, and whether the first contact was enough.

ID Name Definition / Formula Unit Target Source · Audience Refresh
KPI-QLT-001 Customer Satisfaction (CSAT) Mean of post-resolution company rating (1–5) on resolved tickets. score (1–5) ≥ 4.2 tkt_* (feedback) · SA, DA, DG, OF, SEC, MIN, PUB daily
KPI-QLT-002 CSAT Response Rate CSAT responses received ÷ CSAT surveys sent. percent ≥ 40 % tkt_*, not_* · SA, DA, DG daily
KPI-QLT-003 Appeal Rate Tickets appealed ÷ Tickets resolved in the period. ratio ≤ 0.05 tkt_* · SA, DA, DG, SEC, MIN daily
KPI-QLT-004 Appeal Uphold Rate Appeals upheld in company's favour ÷ Appeals decided. ratio ≤ 0.20 tkt_*, aud_* · SA, DA, DG, SEC, MIN daily
KPI-QLT-005 Reopen Rate Tickets reopened ÷ Tickets resolved. ratio ≤ 0.08 tkt_* · SA, DA, DG, OF, SEC, MIN daily
KPI-QLT-006 Resolution Acceptance Rate Tickets accepted by company (no appeal, no reopen) ÷ Tickets resolved. ratio ≥ 0.85 tkt_* · SA, DA, DG, SEC, MIN, PUB daily
KPI-QLT-007 Proof-of-Resolution Completeness Tickets reaching Resolved with ≥1 evidence attachment + resolution note ÷ Tickets reaching Resolved. Enforced by the proof gate; should be ~1.00 by construction; measures gate bypass attempts. ratio = 1.00 tkt_*, fil_*, aud_* · SA, DA, DG daily
KPI-QLT-008 First-Contact Resolution (FCR) Tickets resolved without reassignment, reopen, or second touch ÷ Tickets resolved, restricted to FCR-eligible categories. ratio ≥ 0.60 tkt_*, aud_* · SA, DA, DG, OF daily
KPI-QLT-009 FCR-Eligible Share Tickets in FCR-eligible categories ÷ all tickets (context for KPI-QLT-008). ratio tkt_* · SA, DA, DG daily
KPI-QLT-010 Resolution Note Quality Score Heuristic 0–100 score from note length, presence of action verb, and attachment-of-proof flag (configurable rubric). score (0–100) ≥ 70 tkt_* · SA, DA, DG daily
KPI-QLT-011 MoM Action-Item Closure Rate MoM action items closed by due date ÷ MoM action items created in the period. ratio ≥ 0.80 mtg_* (MoM) · SA, DA, DG, SEC, MIN daily
KPI-QLT-012 Net Resolution Score Composite = (Resolution Acceptance × 0.5) + (CSAT/5 × 0.3) + ((1 − Reopen Rate) × 0.2). Single headline outcome score. score (0–1) ≥ 0.85 tkt_* · SA, DA, DG, SEC, MIN daily

6.5 Family 5 — Department & Staff Performance (PERF)

Department and staff performance metrics drive leaderboards, workload balancing, and bottleneck identification. All PERF KPIs are scoped by department unless the viewer is Super Admin (cross-department) or Public (anonymized department ranking only — see §11).

ID Name Definition / Formula Unit Target Source · Audience Refresh
KPI-PERF-001 Department Leaderboard Departments ranked by composite = (SLA Compliance × 0.4) + (Resolution Acceptance × 0.3) + (CSAT/5 × 0.2) + ((1 − Reopen Rate) × 0.1). rank tkt_*, org_* · SA, DA, DG, SEC, MIN, PUB daily
KPI-PERF-002 Workload Balance (Gini) Gini coefficient of open-ticket counts across officers in a department/section. 0 = perfectly balanced, 1 = concentrated. ratio (0–1) ≤ 0.30 tkt_*, org_* · SA, DA, DG hourly
KPI-PERF-003 Officer CSAT Mean CSAT (1–5) attributed to the resolving officer's tickets. score (1–5) ≥ 4.0 tkt_*, org_* · SA, DA, DG, OF daily
KPI-PERF-004 Officer Resolution Rate Tickets resolved by officer ÷ Tickets assigned to officer in the period. ratio ≥ 0.90 tkt_*, org_* · SA, DA, DG, OF daily
KPI-PERF-005 Officer Throughput Resolved-ticket count per officer per period. count tkt_*, org_* · SA, DA, DG, OF daily
KPI-PERF-006 Officer Mean Handle Time KPI-SPD-006 computed per officer. hours ≤ 8 h tkt_*, aud_* · SA, DA, DG, OF daily
KPI-PERF-007 Active-vs-Idle Time Officer time in active states vs. idle (no ticket touched), from session + audit signals. Excludes leave. ratio active ≥ 0.60 of on-duty aud_*, org_* · SA, DA, DG 15m
KPI-PERF-008 Department Backlog Open-ticket count per department/section. count tkt_*, org_* · SA, DA, DG, OF live
KPI-PERF-009 Bottleneck Department Department with the highest TAT p95 and/or lowest SLA Compliance in the period (flagged). flag anl_mv_* · SA, SEC, MIN daily
KPI-PERF-010 Bottleneck Section Section within a department with the longest median Time-in-State on the critical path. flag anl_mv_* · SA, DA, DG daily
KPI-PERF-011 Top Performers Officers ranked in the top decile by composite of throughput, CSAT, and SLA compliance (department-scoped). rank tkt_*, org_* · SA, DA, DG weekly
KPI-PERF-012 Underperformers (watch list) Officers below configured thresholds on SLA compliance or CSAT for 2 consecutive weeks. list tkt_*, org_* · SA, DA, DG weekly
KPI-PERF-013 Reassignments per Ticket Sum of internal reassignments ÷ Tickets resolved, per department. ratio ≤ 0.50 tkt_*, aud_* · SA, DA, DG daily

6.6 Family 6 — AI Module Metrics (AIM)

AI metrics measure both the quality of each AI capability and the cost of running it. Every quality KPI is paired with its engine and token/cost dimension so S&ITD can govern AI spend from the same dashboards that show AI quality.

ID Name Definition / Formula Unit Target Source · Audience Refresh
KPI-AIM-001 OCR Success Rate OCR calls returning usable structured output ÷ OCR calls attempted. percent ≥ 95 % ai_* · SA hourly
KPI-AIM-002 OCR Field-Level Accuracy Fields correctly extracted ÷ fields expected, on a sampled gold set (weekly eval). percent ≥ 90 % ai_* (eval) · SA weekly
KPI-AIM-003 Summary Acceptance Rate AI summaries accepted (≤ threshold edit) by officer ÷ Summaries generated. ratio ≥ 0.70 ai_*, aud_* · SA, DA, DG daily
KPI-AIM-004 Auto-Routing Accuracy AI-suggested route matching the final accepted route ÷ Routing suggestions. percent ≥ 85 % ai_*, tkt_* · SA, DA hourly
KPI-AIM-005 Classification Accuracy AI category matching the final accepted category ÷ Classifications. percent ≥ 85 % ai_*, tkt_* · SA, DA hourly
KPI-AIM-006 Draft Reply Adoption Rate Drafts sent with ≤ threshold edit distance ÷ Drafts generated. ratio ≥ 0.50 ai_*, not_* · SA, DA, DG, OF daily
KPI-AIM-007 Draft Edit Distance Mean normalized Levenshtein distance between draft and sent message (lower = better). ratio (0–1) ≤ 0.30 ai_*, not_* · SA, DA, DG daily
KPI-AIM-008 Chatbot Deflection Rate Chatbot sessions resolved without escalation to a human ÷ Chatbot sessions. ratio ≥ 0.45 ai_* · SA, SEC, MIN, PUB hourly
KPI-AIM-009 Chatbot Satisfaction Mean thumbs-up/down rating on chatbot answers (1–5). score (1–5) ≥ 3.8 ai_* · SA daily
KPI-AIM-010 Chatbot Escalation-to-Human Count of chatbot sessions escalated to a live officer / ticket. count ai_*, tkt_* · SA hourly
KPI-AIM-011 Translation Acceptance Rate Translations accepted ≤ threshold edit ÷ Translations generated (per locale pair). ratio ≥ 0.75 ai_*, aud_* · SA daily
KPI-AIM-012 MoM Action-Item Precision Sampled precision of extracted action items (correct items ÷ extracted). ratio ≥ 0.85 ai_* (eval), mtg_* · SA, DA weekly
KPI-AIM-013 MoM Action-Item Recall Sampled recall (correct items ÷ true items). ratio ≥ 0.80 ai_* (eval), mtg_* · SA, DA weekly
KPI-AIM-014 Urgency/Sentiment Accuracy AI urgency/sentiment matching reviewer label ÷ Labels, on a sampled set. percent ≥ 80 % ai_* (eval) · SA weekly
KPI-AIM-015 PII Redaction Precision Correctly redacted entities ÷ Entities redacted, on a sampled set. percent ≥ 99 % ai_* (eval) · SA weekly
KPI-AIM-016 Duplicate Detection Precision True duplicates ÷ Detected duplicate pairs. ratio ≥ 0.80 ai_*, tkt_* · SA, DA daily
KPI-AIM-017 Per-Engine Token Usage Sum of input + output tokens per engine (Azure OpenAI / Gemini / Bedrock / Llama / Qwen), per capability, per day. tokens ai_* · SA hourly
KPI-AIM-018 Per-Engine Cost Token usage × unit price per engine per day; rollup by capability and department. PKR ai_*, anl_cfg_* (price) · SA, SEC, MIN daily
KPI-AIM-019 AI Calls per Ticket Mean AI calls (any capability) per resolved ticket. ratio ai_*, tkt_* · SA daily
KPI-AIM-020 On-Prem vs Cloud Share AI calls served by on-prem engines ÷ all AI calls (data-residency governance). ratio ai_* · SA daily

6.7 Family 7 — Engagement & Communications (COM)

Engagement and comms metrics measure whether the portal's notification and chat channels are actually reaching and being used by their audiences.

ID Name Definition / Formula Unit Target Source · Audience Refresh
KPI-COM-001 Notification Delivery Rate Messages delivered ÷ Messages attempted, all channels combined and per channel. percent ≥ 98 % not_* · SA, DA 15m
KPI-COM-002 Email Delivery Rate Emails delivered ÷ Emails attempted (Mailjet). percent ≥ 98 % not_*, int_* · SA 15m
KPI-COM-003 SMS Delivery Rate SMS delivered ÷ SMS attempted (Jazz/Telenor). percent ≥ 95 % not_*, int_* · SA 15m
KPI-COM-004 WhatsApp Delivery Rate WA messages delivered ÷ attempted. percent ≥ 95 % not_*, int_* · SA 15m
KPI-COM-005 Read Receipt Rate Messages read ÷ Messages delivered, where read tracking exists (email open, WA read, in-app view). percent ≥ 60 % not_* · SA, DA, DG hourly
KPI-COM-006 Inbound Reply Rate Outbound notifications that received an inbound reply ÷ Outbound notifications (two-way engagement). ratio not_*, int_* · SA, DA daily
KPI-COM-007 Notification Opt-Out Rate Recipients who opted out of a channel ÷ Recipients (preference-center signal). ratio ≤ 0.05 not_*, org_* · SA daily
KPI-COM-008 Chat Activity (messages/day) Count of internal-comms messages per day, per tier (ticket thread / inbox / channel). count com_* · SA, DA, DG hourly
KPI-COM-009 Channel Activity Active channel count and messages per channel in the period. count com_* · SA, DA hourly
KPI-COM-010 Chat Median Response Time Median time from an officer mention/DM to first reply, on internal comms. minutes ≤ 30 min com_*, aud_* · SA, DA, DG daily
KPI-COM-011 Presence Hours Sum of officer presence hours (online + available) per department per day. hours com_*, aud_* · SA, DA, DG 15m
KPI-COM-012 KB Deflection Rate KB "was this helpful" sessions that did not result in a ticket ÷ KB sessions. ratio ≥ 0.30 kb_*, tkt_* · SA, SEC, MIN, PUB daily

6.8 Family 8 — Governance & Compliance (GOV)

Governance and compliance metrics give S&ITD assurance that access is reviewed, audit is complete, retention is honoured, and statutory categories (RTI) are handled on time.

ID Name Definition / Formula Unit Target Source · Audience Refresh
KPI-GOV-001 Audit Events Captured Count of aud_event rows written in the period (overall and by event class). count aud_* · SA hourly
KPI-GOV-002 Audit Completeness State-changing actions with a matching aud_event ÷ all state-changing actions (gap detection). percent = 100 % aud_*, app logs · SA daily
KPI-GOV-003 Access Review Completion Roles/access reviewed on schedule ÷ Roles/access due for review. percent = 100 % org_*, aud_* · SA, SEC monthly
KPI-GOV-004 Role Change Events Count of role/permission changes in the period (with reason captured). count org_*, aud_* · SA daily
KPI-GOV-005 Retention Rule Executions Count of retention-driven purges/archives executed on schedule. count fil_*, aud_* · SA weekly
KPI-GOV-006 Data-Class Tagging Coverage Records with a valid data-class tag ÷ all records (Public/Internal/Confidential/Restricted). percent = 100 % org_*, fil_* · SA daily
KPI-GOV-007 RTI Tickets Handled Count of RTI-category tickets in the period (statutory category). count tkt_* · SA, DA, DG, SEC, MIN, PUB daily
KPI-GOV-008 RTI Deadline Compliance RTI tickets resolved within statutory deadline ÷ RTI tickets resolved. percent 100 % tkt_* · SA, DA, DG, SEC, MIN, PUB daily
KPI-GOV-009 Export Events Count of analytics exports generated in the period (with who/what filters). count aud_* · SA daily
KPI-GOV-010 Failed Login Attempts Count of failed authentications (and locked-out accounts). count aud_*, Keycloak logs · SA hourly
KPI-GOV-011 Step-Up Auth Events Count of step-up challenges issued and passed/failed (sensitive actions). count aud_* · SA daily
KPI-GOV-012 Privileged Action Count Count of privileged admin actions (role grants, flag toggles, data purges, config edits). count aud_* · SA daily
KPI-GOV-013 Data-Subject Requests Handled Count of access/correction/erasure requests handled within SLA. count aud_*, org_* · SA weekly

6.9 Family 9 — Public Transparency (PUB)

The Public family is the only set of KPIs exposed on the public transparency dashboard (see §11). Every KPI here is sourced from anl_pub_* (nightly anonymized), never from a live transactional query, and obeys the suppression rules in §11.3.

ID Name Definition / Formula Unit Target Source · Audience Refresh
KPI-PUB-001 Total Tickets Received Cumulative count of tickets created (anonymized). count anl_pub_* · PUB daily
KPI-PUB-002 Total Tickets Resolved Cumulative count of tickets resolved (anonymized). count anl_pub_* · PUB daily
KPI-PUB-003 Currently Open Count of tickets in non-terminal states (anonymized). count anl_pub_* · PUB daily
KPI-PUB-004 Overall SLA Compliance Anonymized platform-wide SLA compliance rate. percent ≥ 95 % anl_pub_* · PUB daily
KPI-PUB-005 Median Resolution Time Platform-wide median SLA-aware TAT. days anl_pub_* · PUB daily
KPI-PUB-006 Volume by Department Anonymized ticket count by department (department names visible — they are public bodies). count anl_pub_* · PUB daily
KPI-PUB-007 SLA Compliance by Department Anonymized SLA compliance per department (the public leaderboard). percent anl_pub_* · PUB daily
KPI-PUB-008 Volume by Category Anonymized ticket count by service category. count anl_pub_* · PUB daily
KPI-PUB-009 Volume by District (heatmap) Anonymized ticket count by district (drives the GIS heatmap — see §10). count anl_pub_* · PUB daily
KPI-PUB-010 Public CSAT Anonymized platform-wide mean CSAT (1–5). score (1–5) ≥ 4.0 anl_pub_* · PUB daily

7. The Seven Role Dashboards

Each role gets a dashboard tuned to its decisions. Every tile satisfies AP-1 (live + historical) and AP-2 (drill-down to the scoped ticket list). The role codes used here match /specs/en/04-roles-permissions/.

7.1 Super Admin Dashboard (SA)

Cross-department, cross-channel, system-wide view. The only role with infrastructure + AI-cost + governance tiles alongside operational KPIs.

Operational tiles: KPI-VOL-001..006, 013, 015 · KPI-SPD-001, 002, 008, 011 · KPI-SLA-001, 002, 003, 004 · KPI-QLT-001, 006, 012 · KPI-PERF-001, 009. Volume cuts: volume by channel, category, district, department, company-size tier (KPI-VOL-007..011). AI spend & quality: KPI-AIM-001, 004, 006, 008, 014, 015, 017, 018, 019, 020 (full AI family accessible). Engagement/comms: KPI-COM-001..004, 008, 010. Governance/compliance: full GOV family (KPI-GOV-001..013), including access-review completion, audit completeness, retention executions, step-up events, privileged-action count. Config surfaces: targets (anl_cfg_*), feature-flag status summary, digest schedule health, freshness/staleness monitor for all anl_mv_* workers (see §17). Special: system health badges (portal/API/docs/notifications uptime from the public status page), MariaDB/Redis/MinIO exporter summaries (read-only), Metabase launch link.

7.2 Department Admin Dashboard (DA)

Scoped to one department and its nested sections. Drives staffing and workload decisions.

Operational tiles: KPI-VOL-001..003, 005, 007, 008 (dept-scoped) · KPI-SPD-001, 003, 004, 007, 012 · KPI-SLA-001, 002, 003, 009 · KPI-QLT-001, 002, 005, 008. Workload & staff: KPI-PERF-002 (Gini), KPI-PERF-005, 006, 007, 008, 013, KPI-PERF-010 (bottleneck section), KPI-PERF-011, 012 (top/under performers). Section drill: every aggregate drillable to the section and to the individual officer's queue. Engagement: dept-scoped KPI-COM-008, 009, 010, 011. Exports: one-click daily dept pack (PDF) for circulation.

7.3 DG / Director Dashboard (DG)

Oversight of departments under the DG's purview, focused on SLA risk and escalation. Notify-only versus action powers are honoured per the configurable oversight matrix (/specs/en/04-roles-permissions/ §9).

At-risk focus: KPI-SLA-002, 003 (breach + at-risk, live), KPI-SLA-005 (escalation rung distribution), KPI-SPD-010 (time-in-escalation), KPI-SLA-011 (breach recovery). Outcome: KPI-QLT-001, 005, 006, 012, KPI-PERF-001 (leaderboard for purview). Department comparison: side-by-side dept tiles under purview, with KPI-SPD-011, 013 per dept. Action surface: approve sensitive/VIP closures, push tickets down the escalation ladder — both gated by oversight config and step-up auth.

7.4 Officer Dashboard (OF)

Personal productivity view: "my queue, my SLA clock, my CSAT."

My queue: open tickets assigned to me (live), sorted by SLA-due; aging buckets KPI-SPD-008 (mine); KPI-SLA-003 (my at-risk tickets). My speed: KPI-SPD-001, 005, 006 (my FRT, time-to-ack, handle time). My quality: KPI-PERF-003, 004, 005, 006 (my CSAT, resolution rate, throughput, handle time), KPI-QLT-008 (my FCR). Drafts & AI assist: my draft-adoption rate (KPI-AIM-006, personal view), AI suggestions pending my review (summary, routing, draft). Comms: my mentions/DMs (KPI-COM-010 for my responsiveness), my presence hours (KPI-COM-011).

7.5 Secretary / Minister Dashboard (SEC / MIN)

Executive-summary view for the political and administrative leadership (Secretary S&ITD, Minister/SACM). Numbers are platform-wide or purview-wide; detail is one click away but not in your face.

Headline tiles: KPI-VOL-001, 002, 005, KPI-SLA-001, 010, KPI-QLT-001, 012, KPI-PUB-010. Accountability: KPI-PERF-001 (department leaderboard), KPI-PERF-009 (bottleneck dept), KPI-SLA-005 (how many on each escalation rung, including rung 3 = SACM). Trend: KPI-VOL-013 (MoM/YoY growth), 90-day trend lines on SLA compliance and CSAT. AI & cost (MIN/SEC only): KPI-AIM-018 (per-engine cost rollup), KPI-AIM-008 (chatbot deflection), KPI-AIM-020 (on-prem/cloud share — data-residency posture). RTI: KPI-GOV-007, 008 (RTI volume + statutory compliance). Public transparency preview: a read-only mirror of what the public currently sees (§11), so leadership can compare internal vs published numbers. Delivery: the dashboard content is also pushed as the scheduled digest (§8).

7.6 Company Dashboard (CO)

The company representative's own analytics: my tickets, my response times, my satisfaction — scoped strictly to my company's tickets (RBAC-enforced; never another company's data).

My tickets: open/closed/resolved counts, aging buckets, current SLA status per ticket, at-risk flag. My responsiveness: KPI-SPD-009 (await-on-company duration — time the company itself added), inbound reply rate from the company side. My satisfaction: CSAT I have given, appeals I have filed, appeal status. My interactions: notifications I received and read, chat sessions with the desk, KB articles I found helpful. Action surface: file a ticket, respond to an await, request a TRI meeting, submit CSAT.

7.7 Public Transparency Dashboard (PUB)

The only dashboard visible without authentication. Detailed in §11. Sourced exclusively from anl_pub_*. Tiles: KPI-PUB-001..010, the GIS heatmap (§10), a plain-language FAQ explaining each metric, and a "data published on" timestamp.


8. Scheduled Digests

A cron-driven BullMQ job renders per-role, per-department digests on configurable cadences and pushes them through the notifications pipeline (email/SMS/WhatsApp/in-app, per recipient preference). All digests are multilingual (recipient's locale) and carry Gregorian + Hijri dates.

8.1 Default digest schedule

Audience Cadence (default) Contents Channel
Minister / SACM Weekly (Mon 08:00) Platform headline KPIs, dept leaderboard, bottleneck dept, RTI compliance, AI spend, top escalations (rung 3). Email (primary) + WA summary
Secretary S&ITD Daily (08:00) Yesterday's volume/resolved/backlog, SLA compliance, at-risk & breaching tickets, dept leaderboard deltas, RTI. Email
Department Admin Daily (07:30) Dept-scoped volume, SLA, workload balance, bottleneck section, top/under performers, today's at-risk. Email + in-app
DG / Director Daily (08:00) Purview SLA, at-risk & escalation rung distribution, breach recovery, outcome scores. Email + in-app
Officer Per-shift (start-of-shift + mid-shift) My queue, my at-risk tickets, my CSAT trend, pending AI suggestions to review. In-app + WA
Company (opt-in) Weekly My open tickets, SLA status, awaits-on-me, CSAT reminders. Email
Super Admin (opt-in) Daily System health, freshness/staleness, audit completeness, AI cost, top anomalies. Email

Every cadence above is configurable per recipient, per department, and per environment via anl_cfg_*. The preference center lets any user mute, switch to digest-only, or change channel per digest type.

8.2 Sample digest layout (Secretary, daily)

Sindh IT Portal — Facilitation Desk
Daily Digest · S&ITD Secretary
Gregorian: Mon 17 Jul 2026 · Hijri: 2 Safar 1448

1. HEADLINE  (yesterday / rolling 7-day)
   - Tickets created:        312   (7-day avg 298)
   - Tickets resolved:       287   (7-day avg 271)
   - Open backlog:          1,453
   - SLA compliance:         96.2% (target 95%) ▲
   - CSAT:                   4.3 / 5
   - At-risk (next 24h):      18   [drill]
   - Breaching:                6   [drill]

2. DEPARTMENT LEADERBOARD (top 5 / bottom 3)
   Top:    Labour (98.4%) · IT (97.9%) · Investment (97.1%)
   Bottom: Excise (89.2%) · Revenue (90.1%) · Works (91.4%)

3. ESCALATION RUNG
   Rung 1 (2d, DG):   42
   Rung 2 (5d, Sec):   9
   Rung 3 (10d, SACM): 2   [drill]

4. RTI  (statutory)
   RTI handled:           7
   RTI within deadline: 100%

5. AI & COST
   AI calls yesterday: 2,114   cost: PKR 41,320
   On-prem share: 38%  Chatbot deflection: 47%

6. ANOMALY ALERTS  (from §14)
   - Excise dept SLA down 6.1pts vs 7-day avg.
   - FBR intake volume +220% vs baseline (spike).

[View full dashboard]   [Drill at-risk]   [Manage preferences]

8.3 Digest controls


9. Exports

9.1 Formats and engines

Format Engine Use case
PDF Puppeteer (server-side, headless Chromium) Letter-quality reports on S&ITD/Sindh letterhead; used for digests circulated to ministerial/secretary audiences, RTI responses, audit packs.
Excel (.xlsx) ExcelJS Multi-tab analyst workbooks; one tab per KPI family or per department; formulas live where useful.
CSV Node stream writer Raw extracts for downstream BI / statistical tools; always scoped by RBAC.

All exports are generated by BullMQ workers; the user is notified (in-app + email) when the file is ready and downloads via a presigned MinIO URL with a short TTL.

9.2 Export contract


10. GIS / District Heatmap

The GIS/district heatmap visualizes KPI-VOL-009, KPI-PUB-009, and KPI-SLA-010/KPI-PUB-007 on a map of Sindh's districts.

Specification:

Implementation note: no PostGIS dependency — the heatmap uses bounding-box math and a lightweight spatial index (see /specs/en/15-tech-architecture/ §5.1).


11. Public Transparency Dashboard

The public dashboard is the portal's accountability commitment: any citizen, journalist, or company can see, at a glance, how the Sindh IT Portal is performing for the IT industry — without logging in and without exposing any individually-identifiable data.

11.1 What is shown

11.2 Anonymization rules

  1. No PII, ever. No company name, no officer name, no requester name, no ticket ID, no free-text appears on the public dashboard.
  2. Separate table. All public KPIs read from anl_pub_*, populated by the nightly anonymizer worker from anl_mv_*. No live query path connects the public route to transactional tables.
  3. Aggregation only. Every public number is an aggregate over ≥ n tickets (default n = 5). Smaller cells are suppressed or rolled into "Other / insufficient data".
  4. No small-cell leakage. Cross-tabbing two public dimensions (e.g., department × category) is suppressed where any cell falls below n, to prevent re-identification by subtraction.
  5. Lag. Public data is published with a 24-hour lag (yesterday's anonymized rollup), so a filer cannot recognize their own just-filed ticket in the numbers.
  6. No raw exports. The public dashboard offers no CSV/Excel export of underlying rows — only the published aggregates.
  7. Audit. Every publish event (worker run, row counts before/after suppression) is written to aud_event and visible to Super Admin.

11.3 Suppression parameters (configurable)

Parameter Default Effect
pub.min_cell_size 5 Minimum ticket count for any published cell.
pub.lag_hours 24 Minimum age of data before publication.
pub.allow_district_drill false Whether district-level drill below the top number is shown publicly.
pub.allow_dept_ranking true Whether department names are ranked publicly (they are public bodies).
pub.crosstab_suppression true Suppress cross-tab cells below min_cell_size.

11.4 Refresh

Nightly (01:00 PKT) via the anonymizer worker. The "data published on" timestamp on the page reflects the last successful publish; a failed publish leaves the previous day's data visible and raises a Super Admin alert (§17).


12. Saved Views, Favorites, and Embeddable Widgets

12.1 Saved views

Any internal user can save a dashboard state (tiles + filters + grain + date range) as a named view. Saved views are private by default and shareable within the user's department. Super Admin can promote a saved view to a department default.

12.2 Favorites

Any tile or KPI can be pinned as a favorite; favorites aggregate into a personalized "My view" tile strip at the top of the user's dashboard. Favorites persist across sessions and locales.

12.3 Embeddable widgets

Selected tiles can be exported as embeddable widgets (signed, time-limited iframe URLs) for embedding in:

Embeddable widgets are signed with a rotating key, scoped to a specific KPI + filter, and rate-limited. They never carry credentials; they only display anonymized or already-authorized aggregates.


13. Anomaly Detection & SLA-Breach Prediction

13.1 Anomaly detection

A scheduled worker scans anl_mv_* for statistically unusual changes and raises alerts:

13.2 SLA-breach prediction

KPI-SLA-003 (at-risk count) is computed by a lightweight predictor:

13.3 Alert hygiene


14. Data Freshness & Accuracy Controls

14.1 Freshness SLAs

Each anl_mv_* table has a published freshness SLA (the Refresh column in the KPI catalog). A monitor compares actual last-refresh to the SLA; breach raises a stale badge on affected tiles and a Super Admin alert.

Refresh tier Max staleness Examples
live 5 s open-ticket count, breach count
5m / 15m 5 / 15 min volume, delivery rates, presence
hourly 60 min FRT/TAT distributions, SLA compliance
daily 24 h + 1 run CSAT, reopen/acceptance rates, leaderboards
weekly / monthly 7 / 30 d AI evals, access reviews

14.2 Reconciliation

Every nightly run reconciles anl_mv_* against the transactional source totals (row counts and sum checks). Discrepancies above a threshold raise a Super Admin alert and a re-compute job; the affected tiles are marked stale until reconciled.

14.3 Accuracy controls


15. Self-Serve Report Builder (Metabase)

Internal roles (primarily Super Admin, Dept Admin, DG, Secretary) get embedded Metabase for ad-hoc exploration and bespoke reports.


16. Definitions Glossary (Metrics)

This glossary is the canonical source for every metric term. It is mirrored (metric-key-only) into _glossary.md so the same definitions surface in UR/Sindhi dashboards. Where a definition is non-obvious, the formula is given.

Term Definition
FRT (First Response Time) Time from ticket creation (New) to the first outbound message the company receives on that ticket, on any channel. Working-time aware (excludes SLA-pause windows).
TAT (Turnaround Time) Working time from creation to Resolved, SLA-pause-aware. Reported as median, p90, p95 over a population.
SLA Compliance Tickets meeting the configured SLA window ÷ Tickets resolved in the period.
SLA Pause A period during which the SLA clock is held: await-on-company, weekend, or Sindh public holiday.
Escalation Rung Step on the ladder: rung 1 = DG (2 d), rung 2 = Department Secretary (5 d), rung 3 = S&ITD Secretary / SACM (10 d). Configurable per department.
CSAT Post-resolution rating by the company, scale 1–5. Reported as mean over surveys returned.
CSAT Response Rate Surveys returned ÷ Surveys sent.
Appeal A formal request by a company to reconsider a resolution, after Resolved.
Reopen A ticket transitioned from Resolved to Reopened (company rejected the resolution).
Resolution Acceptance A resolved ticket that was neither appealed nor reopened within the acceptance window.
Proof of Resolution ≥1 evidence attachment + resolution note required before Resolved; the proof gate enforces it.
FCR (First-Contact Resolution) A resolved ticket with no reassignment, no reopen, and no second internal touch, within FCR-eligible categories.
Net Resolution Score Composite: 0.5·(Resolution Acceptance) + 0.3·(CSAT/5) + 0.2·(1 − Reopen Rate).
Active-vs-Idle Officer on-duty time spent actively working tickets vs. idle, from session + audit signals.
Gini (workload balance) Gini coefficient of open-ticket counts across officers; 0 = perfectly balanced, 1 = all on one officer.
Bottleneck Department / Section The unit with the worst SLA/TAT in the period (flagged).
Throughput Rate Tickets Resolved ÷ Tickets Created over the period.
Aging Bucket Age-of-open-ticket bucket: 0–2 d, 3–5 d, 6–10 d, 11–20 d, 21+ d.
OCR Success An OCR call returning usable structured output (no error, non-empty, schema-valid).
Draft Adoption A draft reply sent with edit distance ≤ the configured threshold.
Chatbot Deflection A chatbot session that resolved without escalating to a human officer or creating a ticket.
Token A model usage unit (input + output), recorded per engine per capability per call in ai_*.
Per-Engine Cost Token usage × configured unit price per engine, per day.
Delivery Rate Messages delivered ÷ Messages attempted, per channel.
Read Receipt A delivered message that was opened/viewed (email open, WA read, in-app view).
KB Deflection A KB "was this helpful" session that did not result in a ticket creation.
Audit Completeness State-changing actions with a matching aud_event ÷ all state-changing actions.
Anonymized Aggregate A KPI value computed over ≥ pub.min_cell_size tickets, with no PII, sourced from anl_pub_*.
Suppression Withholding a public cell whose sample size is below pub.min_cell_size, or that would allow re-identification by cross-tab subtraction.
Stale Tile A dashboard tile whose underlying data is older than its freshness SLA; shown with a visible badge.
Saved View A named, restorable dashboard state (tiles + filters + grain + date range).

17. Configuration Surface (summary)

All tunables live in anl_cfg_* and are editable by Super Admin (step-up auth). Changes are audited (aud_event) and versioned.

Config group Examples
targets.* SLA windows, CSAT goal, FCR target, breach thresholds.
pub.* Public suppression parameters (§11.3).
digests.* Per-audience cadence, sections, channels, quiet hours.
anomaly.* Threshold rules, z-score window, predictor confidence p.
refresh.* Per-table refresh cadence and freshness SLAs.
engines.* Per-engine unit prices (for KPI-AIM-018), on-prem/cloud policy.
embedding.* Signed-URL TTLs, allowed embed scopes.

18. Master KPI Catalog

The complete catalog: 9 families, 120 KPIs. Counts per family: VOL 15 · SPD 13 · SLA 12 · QLT 12 · PERF 13 · AIM 20 · COM 12 · GOV 13 · PUB 10.

ID Family Name Unit Refresh
KPI-VOL-001 Volume & Throughput Tickets Created count live
KPI-VOL-002 Volume & Throughput Tickets Resolved count 5m
KPI-VOL-003 Volume & Throughput Tickets Closed count 5m
KPI-VOL-004 Volume & Throughput Throughput Rate ratio hourly
KPI-VOL-005 Volume & Throughput Open Backlog count live
KPI-VOL-006 Volume & Throughput Net Backlog Change count daily
KPI-VOL-007 Volume & Throughput Volume by Channel count hourly
KPI-VOL-008 Volume & Throughput Volume by Category count hourly
KPI-VOL-009 Volume & Throughput Volume by District count hourly
KPI-VOL-010 Volume & Throughput Volume by Department count hourly
KPI-VOL-011 Volume & Throughput Volume by Company-Size Tier count daily
KPI-VOL-012 Volume & Throughput Peak-Hour Volume count hourly
KPI-VOL-013 Volume & Throughput Volume Growth (MoM/YoY) percent daily
KPI-VOL-014 Volume & Throughput Draft → Submitted Conversion ratio daily
KPI-VOL-015 Volume & Throughput Reopened Volume count hourly
KPI-SPD-001 Speed First Response Time (FRT) hours hourly
KPI-SPD-002 Speed Turnaround Time (TAT) days hourly
KPI-SPD-003 Speed Time-to-Triage hours hourly
KPI-SPD-004 Speed Time-to-Assign hours hourly
KPI-SPD-005 Speed Time-to-Acknowledge hours hourly
KPI-SPD-006 Speed Mean Handle Time hours daily
KPI-SPD-007 Speed Time-in-State (per state) hours hourly
KPI-SPD-008 Speed Aging Buckets count hourly
KPI-SPD-009 Speed Await-on-Company Duration hours daily
KPI-SPD-010 Speed Time-in-Escalation hours hourly
KPI-SPD-011 Speed p95 Resolution Time days daily
KPI-SPD-012 Speed Reassignment Latency hours daily
KPI-SPD-013 Speed Resolution Cycle Time by Category days daily
KPI-SLA-001 SLA & Escalation SLA Compliance Rate percent hourly
KPI-SLA-002 SLA & Escalation SLA Breach Count count live
KPI-SLA-003 SLA & Escalation SLA At-Risk Count count live
KPI-SLA-004 SLA & Escalation Escalation Rate ratio hourly
KPI-SLA-005 SLA & Escalation Escalation Rung Distribution count hourly
KPI-SLA-006 SLA & Escalation Repeat Escalation Rate ratio daily
KPI-SLA-007 SLA & Escalation SLA Pause Events count hourly
KPI-SLA-008 SLA & Escalation RTI SLA Compliance percent daily
KPI-SLA-009 SLA & Escalation SLA Compliance by Severity percent hourly
KPI-SLA-010 SLA & Escalation SLA Compliance by Department percent hourly
KPI-SLA-011 SLA & Escalation Breach Recovery Time hours daily
KPI-SLA-012 SLA & Escalation Holiday/Weekend Pause Share ratio daily
KPI-QLT-001 Quality/Outcome Customer Satisfaction (CSAT) score (1–5) daily
KPI-QLT-002 Quality/Outcome CSAT Response Rate percent daily
KPI-QLT-003 Quality/Outcome Appeal Rate ratio daily
KPI-QLT-004 Quality/Outcome Appeal Uphold Rate ratio daily
KPI-QLT-005 Quality/Outcome Reopen Rate ratio daily
KPI-QLT-006 Quality/Outcome Resolution Acceptance Rate ratio daily
KPI-QLT-007 Quality/Outcome Proof-of-Resolution Completeness ratio daily
KPI-QLT-008 Quality/Outcome First-Contact Resolution (FCR) ratio daily
KPI-QLT-009 Quality/Outcome FCR-Eligible Share ratio daily
KPI-QLT-010 Quality/Outcome Resolution Note Quality Score score (0–100) daily
KPI-QLT-011 Quality/Outcome MoM Action-Item Closure Rate ratio daily
KPI-QLT-012 Quality/Outcome Net Resolution Score score (0–1) daily
KPI-PERF-001 Dept/Staff Performance Department Leaderboard rank daily
KPI-PERF-002 Dept/Staff Performance Workload Balance (Gini) ratio (0–1) hourly
KPI-PERF-003 Dept/Staff Performance Officer CSAT score (1–5) daily
KPI-PERF-004 Dept/Staff Performance Officer Resolution Rate ratio daily
KPI-PERF-005 Dept/Staff Performance Officer Throughput count daily
KPI-PERF-006 Dept/Staff Performance Officer Mean Handle Time hours daily
KPI-PERF-007 Dept/Staff Performance Active-vs-Idle Time ratio 15m
KPI-PERF-008 Dept/Staff Performance Department Backlog count live
KPI-PERF-009 Dept/Staff Performance Bottleneck Department flag daily
KPI-PERF-010 Dept/Staff Performance Bottleneck Section flag daily
KPI-PERF-011 Dept/Staff Performance Top Performers rank weekly
KPI-PERF-012 Dept/Staff Performance Underperformers (watch list) list weekly
KPI-PERF-013 Dept/Staff Performance Reassignments per Ticket ratio daily
KPI-AIM-001 AI Module Metrics OCR Success Rate percent hourly
KPI-AIM-002 AI Module Metrics OCR Field-Level Accuracy percent weekly
KPI-AIM-003 AI Module Metrics Summary Acceptance Rate ratio daily
KPI-AIM-004 AI Module Metrics Auto-Routing Accuracy percent hourly
KPI-AIM-005 AI Module Metrics Classification Accuracy percent hourly
KPI-AIM-006 AI Module Metrics Draft Reply Adoption Rate ratio daily
KPI-AIM-007 AI Module Metrics Draft Edit Distance ratio (0–1) daily
KPI-AIM-008 AI Module Metrics Chatbot Deflection Rate ratio hourly
KPI-AIM-009 AI Module Metrics Chatbot Satisfaction score (1–5) daily
KPI-AIM-010 AI Module Metrics Chatbot Escalation-to-Human count hourly
KPI-AIM-011 AI Module Metrics Translation Acceptance Rate ratio daily
KPI-AIM-012 AI Module Metrics MoM Action-Item Precision ratio weekly
KPI-AIM-013 AI Module Metrics MoM Action-Item Recall ratio weekly
KPI-AIM-014 AI Module Metrics Urgency/Sentiment Accuracy percent weekly
KPI-AIM-015 AI Module Metrics PII Redaction Precision percent weekly
KPI-AIM-016 AI Module Metrics Duplicate Detection Precision ratio daily
KPI-AIM-017 AI Module Metrics Per-Engine Token Usage tokens hourly
KPI-AIM-018 AI Module Metrics Per-Engine Cost PKR daily
KPI-AIM-019 AI Module Metrics AI Calls per Ticket ratio daily
KPI-AIM-020 AI Module Metrics On-Prem vs Cloud Share ratio daily
KPI-COM-001 Engagement/Comms Notification Delivery Rate percent 15m
KPI-COM-002 Engagement/Comms Email Delivery Rate percent 15m
KPI-COM-003 Engagement/Comms SMS Delivery Rate percent 15m
KPI-COM-004 Engagement/Comms WhatsApp Delivery Rate percent 15m
KPI-COM-005 Engagement/Comms Read Receipt Rate percent hourly
KPI-COM-006 Engagement/Comms Inbound Reply Rate ratio daily
KPI-COM-007 Engagement/Comms Notification Opt-Out Rate ratio daily
KPI-COM-008 Engagement/Comms Chat Activity (messages/day) count hourly
KPI-COM-009 Engagement/Comms Channel Activity count hourly
KPI-COM-010 Engagement/Comms Chat Median Response Time minutes daily
KPI-COM-011 Engagement/Comms Presence Hours hours 15m
KPI-COM-012 Engagement/Comms KB Deflection Rate ratio daily
KPI-GOV-001 Governance/Compliance Audit Events Captured count hourly
KPI-GOV-002 Governance/Compliance Audit Completeness percent daily
KPI-GOV-003 Governance/Compliance Access Review Completion percent monthly
KPI-GOV-004 Governance/Compliance Role Change Events count daily
KPI-GOV-005 Governance/Compliance Retention Rule Executions count weekly
KPI-GOV-006 Governance/Compliance Data-Class Tagging Coverage percent daily
KPI-GOV-007 Governance/Compliance RTI Tickets Handled count daily
KPI-GOV-008 Governance/Compliance RTI Deadline Compliance percent daily
KPI-GOV-009 Governance/Compliance Export Events count daily
KPI-GOV-010 Governance/Compliance Failed Login Attempts count hourly
KPI-GOV-011 Governance/Compliance Step-Up Auth Events count daily
KPI-GOV-012 Governance/Compliance Privileged Action Count count daily
KPI-GOV-013 Governance/Compliance Data-Subject Requests Handled count weekly
KPI-PUB-001 Public Transparency Total Tickets Received count daily
KPI-PUB-002 Public Transparency Total Tickets Resolved count daily
KPI-PUB-003 Public Transparency Currently Open count daily
KPI-PUB-004 Public Transparency Overall SLA Compliance percent daily
KPI-PUB-005 Public Transparency Median Resolution Time days daily
KPI-PUB-006 Public Transparency Volume by Department count daily
KPI-PUB-007 Public Transparency SLA Compliance by Department percent daily
KPI-PUB-008 Public Transparency Volume by Category count daily
KPI-PUB-009 Public Transparency Volume by District (heatmap) count daily
KPI-PUB-010 Public Transparency Public CSAT score (1–5) daily

19. Open Questions / TBD

# Item Status
1 Final unit prices per AI engine for KPI-AIM-018 (depends on contracted rates). TBD
2 Public-dashboard suppression defaults (pub.min_cell_size, district drill) — pending S&ITD sign-off. TBD
3 Whether department names are ranked publicly by default (legal/comms review). TBD
4 Breach-predictor confidence threshold p (default 0.70) — tune after first month of real data. TBD
5 Metabase sandbox row-level scope rules for cross-dept DG purviews. TBD
6 Hijri-aware grain for monthly digests (lunar vs Gregorian month). TBD

End of document.