Analytics & KPIs
The complete measurement system for the Sindh IT Portal — Facilitation Desk (SITP): the nine metric families, seven role dashboards, GIS heatmap, public transparency layer, scheduled digests, exports, anomaly detection, self-serve reporting, and the master KPI catalog.
| Field | Value |
|---|---|
| Doc ID | 17 |
| Status | Draft |
| Owner | S&ITD / MAAHIR |
| Languages | EN (master) · UR · SD |
| Related docs | 03-non-functional-requirements/en.md, 04-functional-requirements/en.md, /specs/en/04-roles-permissions/, /specs/en/06-ticket-workflow/, 09-ai-ocr-architecture/en.md, /specs/en/15-tech-architecture/, 16-security-and-compliance/en.md |
1. Purpose & Scope
This document specifies every metric, dashboard, report, and control the SITP analytics plane must produce. It is the single reference used by the engineering team (MAAHIR) to build materialized views, by the S&ITD product owner to accept dashboards, by departmental stakeholders to interpret their performance, and by the public to understand what is and is not published.
The analytics plane is responsible for three things:
- Operational insight — let officers, DGs, department admins, secretaries, the SACM/Minister, and the super admin see the live and historical state of the portal relative to SLAs and targets.
- Accountability & transparency — let companies track their own interactions and let the public see anonymized aggregates about how departments are serving the IT industry.
- Governance & cost control — let S&ITD govern data access, AI spend, retention, and audit completeness from the same plane.
This document covers the nine metric families, the seven role dashboards (including the public transparency dashboard), the GIS/district heatmap, scheduled digests, exports, anomaly detection and SLA-breach prediction, saved views / favorites / embeddable widgets, a definitions glossary, data freshness and accuracy controls, and the self-serve report builder (Metabase). The underlying architecture (MariaDB materialized views/cubes, Metabase embedding, ECharts/Recharts, WebSocket live push, server-side export, cron digests) is summarized in §3 and detailed in /specs/en/15-tech-architecture/ §14.
2. Analytics Principles
Every metric, dashboard, and report in SITP obeys the following ten principles. They are referenced by ID (AP-1 … AP-10) throughout this document and in acceptance criteria.
| ID | Principle | What it means in practice |
|---|---|---|
| AP-1 | Real-time + historical, together | The same metric is computable both "live" (current value over a WebSocket push, ≤ 5 s freshness for count-style KPIs) and "historical" (any bounded date range, with day/week/month/year grain). A dashboard tile never silently mixes grains; the grain is labelled on the tile. |
| AP-2 | Drill-down to the ticket | Every aggregate (count, average, rate) is a hyperlink through the filtered ticket list. A user who sees "47 breaching tickets in Labour dept" can click through to the 47 tickets, scoped by their RBAC, without leaving the dashboard. |
| AP-3 | Role-scoped everywhere | Every query carries the viewer's role, department, section, and ABAC confidential/VIP scope. There are no "god queries" in dashboards. The Public role sees only anonymized aggregates (see §11). |
| AP-4 | Anonymization before public exposure | Any metric reachable by the public dashboard is sourced from a separate anonymized aggregate table (see §3.3) populated by a scheduled worker, never computed live from ticket rows. Suppression rules (see §11.3) apply at publish time. |
| AP-5 | One definition per metric | Each KPI has exactly one canonical definition, formula, unit, and rounding rule in the glossary (§16). Two dashboards showing "CSAT" show the same number for the same period and scope because they read the same materialized view. |
| AP-6 | Targets are configurable, not hard-coded | Targets (SLA windows, CSAT goals, breach thresholds) live in the configuration tables and flow into targets shown on tiles. Changing a target does not require a code change. |
| AP-7 | Cost-aware AI metrics | Every AI-family KPI is paired with the engine and cost dimension. Token/cost totals roll up per engine, per capability, per department, and per day, so AI spend is governable from the same dashboards that show AI quality. |
| AP-8 | Drift and freshness are visible | Each dashboard carries a "last refreshed" timestamp and a freshness SLA badge. If the underlying worker is lagging beyond its SLA, the tile is flagged stale rather than reported as current (see §17). |
| AP-9 | Auditable exports | Every export (PDF/Excel/CSV) is itself an audited event: who, what filters, what grain, when, and a presigned download link logged. Sensitive exports (e.g., confidential-ticket extracts) require step-up auth. |
| AP-10 | Multilingual labels, language-agnostic IDs | Dashboard labels render in EN/UR/Sindhi per the viewer's locale; KPI IDs (KPI-VOL-007) and metric keys never translate, so cross-language reports are comparable (see _glossary.md). |
3. Analytics Architecture (Recap)
This section recapitulates the analytics-specific portion of /specs/en/15-tech-architecture/ §14 so this document is self-contained. The full component-by-component rationale lives in the technical architecture doc.
3.1 Aggregation strategy
Dashboard reads never hit transactional tables directly. The Analytics module maintains a set of materialized views and lightweight cubes in MariaDB, refreshed by scheduled BullMQ workers on cadences defined per KPI (see §6). The pattern is:
- Transactional tables (
tkt_*,org_*,aud_*,ai_*,not_*, …) are the system of record. - Aggregation tables (
anl_mv_*) are denormalized, pre-grouped rollups (by day × dept × category × district × severity × …) keyed for the dashboard's hottest group-by queries. Eachanl_mv_*is refreshed on its own schedule (every 5 min for live-ish; hourly for operational; nightly for historical completeness). - Cube-style tables (
anl_cube_*) hold multi-dimensional sums and counts that allow the UI to pivot by any of the pre-cubed dimensions without re-querying the transactional tables. - Public aggregates (
anl_pub_*) are an entirely separate, anonymized, denormalized table populated by a single nightly worker fromanl_mv_*, with suppression rules applied (see §11). The public dashboard reads only fromanl_pub_*.
Heavy historical queries and Metabase ad-hoc reads run against a read replica once read replicas are introduced (see /specs/en/15-tech-architecture/ §5.6). Until then, Metabase is pointed at a low-priority connection pool with its own throttle so analyst reads cannot starve the request path.
3.2 Visualization stack
- Custom dashboards — ECharts/Recharts in the Next.js portal for the seven role dashboards, the GIS/district heatmap, and the public transparency dashboard. Tiles support filters (date range, department, category, district, channel, severity, locale).
- Metabase (embedded) — self-hosted; embedded via signed URLs for internal roles. Used for ad-hoc exploration, deep-dive operational reporting, and the self-serve report builder (see §15). Metabase connects read-only to the analytics schema; it can never write.
- WebSocket live push — selected tiles (open-ticket count, breaching-tickets, this-hour volume, officer active/idle) subscribe to the WebSocket gateway. Counts are pushed by BullMQ workers and the API on domain events; full re-renders are debounced (see §3.4).
- Server-side exports — PDF via Puppeteer (for letter-quality, letterhead-capable reports) and Excel via ExcelJS (for analyst-friendly multi-tab workbooks). CSV for raw extracts. All exports are generated by BullMQ workers and delivered via presigned MinIO URLs (see §12).
3.3 Public transparency separation
Public-facing dashboards read only from anl_pub_*. No live query path connects the public route to the transactional or even the internal aggregate tables. This guarantees that a bug in the public UI cannot leak a ticket row, a company name, or an officer name. The nightly worker that populates anl_pub_* applies the suppression rules in §11.3.
3.4 Live-push and staleness contract
Live tiles subscribe to channels scoped by RBAC; the gateway never trusts a client-asserted scope. Pushes are debounced (e.g., a count tile re-renders at most once every 5 s even under burst). Each tile carries a last_refreshed_at timestamp and a freshness SLA (see §17). If the worker falls behind, the tile is marked stale with a visible badge; stale data is never silently presented as current.
4. Analytics Data Flow (Diagram + Description)
The diagram shows how a domain event (a ticket state change, an AI call, a notification delivery, an audit write) flows through the analytics plane and surfaces on dashboards, digests, exports, and the public page.
Written description. Domain events from every stateful module — ticket transitions, org/RBAC changes, AI call audit rows, notification delivery receipts, audit-log writes, and comms activity — are published onto a Redis pub-sub fan-out. Three classes of BullMQ worker consume these events and maintain the analytics store: materialized-view refreshers keep anl_mv_* fresh at 5-minute or hourly cadence; cube refreshers rebuild anl_cube_* hourly or nightly; and the public anonymizer worker reads anl_mv_* once nightly and writes the fully-anonymized, suppression-applied anl_pub_* table that only the public dashboard ever reads. Configuration — targets, thresholds, digest schedules, anomaly rules — lives in anl_cfg_* and feeds both the surfaces (so tiles show target lines) and the anomaly/digest workers.
Six surfaces consume the analytics store: the role dashboards (custom ECharts/Recharts, fed by anl_mv_* plus WebSocket live push for count tiles); Metabase embedded for ad-hoc exploration over anl_cube_*; the scheduled digests rendered from anl_mv_*; the export workers producing PDF/Excel/CSV from anl_mv_*; the anomaly detector and SLA-breach predictor reading trends in anl_mv_* against thresholds in anl_cfg_*; and the public transparency dashboard reading only from anl_pub_*. Breach-prediction alerts flow back through the notifications pipeline (digests, in-app, WebSocket) and never expose raw ticket data to recipients outside their RBAC scope.
5. Conventions Used in the KPI Catalog
The KPI catalog (§6) and the master table (§18) use these conventions:
- KPI ID —
KPI-<FAM>-<nnn>where<FAM>is the three-letter family code (VOL, SPD, SLA, QLT, PERF, AIM, COM, GOV, PUB). IDs are stable across EN/UR/SD and never renumbered. - Unit — the canonical unit (
count,ratio(0–1),percent,minutes,hours,days,PKR,tokens,events/day,score(0–5 or 0–100 as noted)). - Target — the default V1 target. Targets are configurable per department via
anl_cfg_*; the table shows the platform default. - Source — the primary source table or event stream.
tkt_*= ticketing;org_*= org/RBAC;aud_*= audit;ai_*= AI call audit;not_*= notifications;com_*= comms;int_*= integrations;fil_*= files. All KPIs are read fromanl_mv_*rollups derived from these sources. - Audience — role codes:
SASuper Admin,DADept Admin,DGDirector General,OFOfficer,SECSecretary,MINMinister/SACM,COCompany,PUBPublic. A KPI's audience is the set of roles whose dashboards show it (see §7). - Refresh — the maximum staleness the tile will show before being marked stale:
live(≤ 5 s via WebSocket),5m,15m,hourly,daily,weekly,monthly. Nightly historical completeness reconciliation is implied for all. - All time-windowed KPIs are computable over any bounded date range and any of the standard grains (hour, day, ISO week, Gregorian month, Gregorian year). Hijri equivalents are derived at presentation only.
6. The Nine Metric Families — KPI Catalog
6.1 Family 1 — Volume & Throughput (VOL)
Volume and throughput measure how much demand the portal is absorbing and how quickly that demand is being closed out, by every meaningful cut (department, category, district, channel, severity, company-size tier).
| ID | Name | Definition / Formula | Unit | Target | Source · Audience | Refresh |
|---|---|---|---|---|---|---|
| KPI-VOL-001 | Tickets Created | Count of tickets entering New state in the period. |
count | — | tkt_* · SA, DA, DG, OF, SEC, MIN |
live |
| KPI-VOL-002 | Tickets Resolved | Count of tickets reaching Resolved in the period. |
count | — | tkt_* · SA, DA, DG, OF, SEC, MIN |
5m |
| KPI-VOL-003 | Tickets Closed | Count of tickets reaching terminal Closed (auto-close or accepted) in the period. |
count | — | tkt_* · SA, DA, DG, SEC, MIN |
5m |
| KPI-VOL-004 | Throughput Rate | Tickets Resolved ÷ Tickets Created over the period. |
ratio | ≥ 0.95 | tkt_* · SA, DA, DG, SEC, MIN |
hourly |
| KPI-VOL-005 | Open Backlog | Count of tickets in non-terminal states at the instant of measurement. | count | — | tkt_* · SA, DA, DG, OF |
live |
| KPI-VOL-006 | Net Backlog Change | (Open Backlog_end − Open Backlog_start) over the period. |
count | ≤ 0 | tkt_* · SA, DA, DG, SEC, MIN |
daily |
| KPI-VOL-007 | Volume by Channel | Ticket count grouped by intake channel (web, email, SMS, WhatsApp, IVR, walk-in). | count | — | tkt_*, int_* · SA, DA, DG, SEC, MIN |
hourly |
| KPI-VOL-008 | Volume by Category | Ticket count grouped by service category/sub-category. | count | — | tkt_* · SA, DA, DG, SEC, MIN |
hourly |
| KPI-VOL-009 | Volume by District | Ticket count grouped by requester district (GIS feed). | count | — | tkt_*, org_* · SA, DA, DG, SEC, MIN, PUB |
hourly |
| KPI-VOL-010 | Volume by Department | Ticket count grouped by assigned department/section. | count | — | tkt_*, org_* · SA, DA, DG, SEC, MIN |
hourly |
| KPI-VOL-011 | Volume by Company-Size Tier | Ticket count grouped by entity size tier (freelancer / startup / SME / enterprise / foreign branch). | count | — | tkt_*, org_* · SA, SEC, MIN |
daily |
| KPI-VOL-012 | Peak-Hour Volume | Maximum tickets-created-in-an-hour in the period, with the hour timestamp. | count | — | tkt_* · SA, DA, DG, SEC, MIN |
hourly |
| KPI-VOL-013 | Volume Growth (MoM / YoY) | ((Tickets Created this period − Tickets Created prior comparable period) ÷ prior). |
percent | — | anl_mv_* · SA, SEC, MIN |
daily |
| KPI-VOL-014 | Draft → Submitted Conversion | Tickets Submitted ÷ (Drafts Started) over the period. |
ratio | ≥ 0.70 | tkt_* · SA, DA |
daily |
| KPI-VOL-015 | Reopened Volume | Count of tickets transitioned Resolved → Reopened in the period. |
count | — | tkt_* · SA, DA, DG, SEC, MIN |
hourly |
6.2 Family 2 — Speed (SPD)
Speed measures latency across the ticket lifecycle: how fast the company hears back, how fast the issue is resolved, and where time accumulates inside the workflow.
| ID | Name | Definition / Formula | Unit | Target | Source · Audience | Refresh |
|---|---|---|---|---|---|---|
| KPI-SPD-001 | First Response Time (FRT) | Time from New to first outbound company-facing message (any channel) on a ticket. Reported as median, p90, p95, mean. |
hours | median ≤ 4 h | tkt_*, not_* · SA, DA, DG, OF, SEC, MIN |
hourly |
| KPI-SPD-002 | Turnaround Time (TAT) | Working time from creation to Resolved, SLA-pause-aware (weekends, holidays, await-on-company excluded). Reported as median, p90, p95. |
days | median ≤ 5 d | tkt_* · SA, DA, DG, SEC, MIN |
hourly |
| KPI-SPD-003 | Time-to-Triage | Time from New to Triaged (categorized + routed). |
hours | ≤ 2 h | tkt_* · SA, DA, DG |
hourly |
| KPI-SPD-004 | Time-to-Assign | Time from Triaged to Assigned (owner set). |
hours | ≤ 4 h | tkt_* · SA, DA, DG |
hourly |
| KPI-SPD-005 | Time-to-Acknowledge | Time from Assigned to first internal note or status change by the assignee. |
hours | ≤ 8 h | tkt_* · SA, DA, DG, OF |
hourly |
| KPI-SPD-006 | Mean Handle Time | Sum of officer-active working time ÷ tickets resolved, in the period. Excludes await-on-company. | hours | ≤ 8 h | tkt_*, aud_* · SA, DA, DG, OF |
daily |
| KPI-SPD-007 | Time-in-State (per state) | Median dwell time in each state (New, Triaged, Assigned, InProgress, AwaitingCompany, Resolved). |
hours | — | tkt_*, aud_* · SA, DA, DG, OF |
hourly |
| KPI-SPD-008 | Aging Buckets | Count of open tickets bucketed by age: 0–2 d, 3–5 d, 6–10 d, 11–20 d, 21+ d. | count | 0 in 21+ bucket | tkt_* · SA, DA, DG, OF, SEC, MIN |
hourly |
| KPI-SPD-009 | Await-on-Company Duration | Median cumulative paused time awaiting company response, per ticket. | hours | — | tkt_* · SA, DA, DG, CO |
daily |
| KPI-SPD-010 | Time-in-Escalation | Median cumulative time a ticket spends on an escalation rung before being picked up. | hours | ≤ 24 h | tkt_*, aud_* · SA, DA, DG, SEC, MIN |
hourly |
| KPI-SPD-011 | p95 Resolution Time | 95th-percentile SLA-aware TAT, reported per department and per category. | days | p95 ≤ 10 d | tkt_* · SA, DA, DG, SEC, MIN |
daily |
| KPI-SPD-012 | Reassignment Latency | Time lost due to internal reassignment (time a ticket sits after a re-route before the new owner acts). | hours | median ≤ 4 h | tkt_*, aud_* · SA, DA, DG |
daily |
| KPI-SPD-013 | Resolution Cycle Time by Category | Median SLA-aware TAT grouped by service category (for benchmarking). | days | — | tkt_* · SA, DA, DG, SEC, MIN |
daily |
6.3 Family 3 — SLA & Escalation (SLA)
SLA metrics measure conformance to the configurable response/resolution windows (default 2 / 5 / 10 days) and the behaviour of the escalation ladder.
| ID | Name | Definition / Formula | Unit | Target | Source · Audience | Refresh |
|---|---|---|---|---|---|---|
| KPI-SLA-001 | SLA Compliance Rate | Tickets meeting SLA ÷ Tickets resolved in the period (SLA-aware). |
percent | ≥ 95 % | tkt_*, anl_cfg_* · SA, DA, DG, SEC, MIN, PUB |
hourly |
| KPI-SLA-002 | SLA Breach Count | Count of tickets whose SLA clock exceeded the configured window. | count | — | tkt_* · SA, DA, DG, OF, SEC, MIN |
live |
| KPI-SLA-003 | SLA At-Risk Count | Count of open tickets projected to breach within the next 24 h (see §14). | count | — | tkt_*, predictor · SA, DA, DG, OF |
live |
| KPI-SLA-004 | Escalation Rate | Tickets escalated (any rung) ÷ Tickets created in the period. |
ratio | ≤ 0.20 | tkt_*, aud_* · SA, DA, DG, SEC, MIN |
hourly |
| KPI-SLA-005 | Escalation Rung Distribution | Count of tickets currently on rung 1 (2 d), rung 2 (5 d), rung 3 (10 d / SACM). | count | — | tkt_*, aud_* · SA, DA, DG, SEC, MIN |
hourly |
| KPI-SLA-006 | Repeat Escalation Rate | Tickets re-escalated after de-escalation ÷ Tickets escalated. |
ratio | ≤ 0.10 | tkt_*, aud_* · SA, DA, DG |
daily |
| KPI-SLA-007 | SLA Pause Events | Count of SLA-clock pauses triggered (await-on-company, weekend, Sindh public holiday). | count | — | tkt_*, aud_* · SA, DA, DG |
hourly |
| KPI-SLA-008 | RTI SLA Compliance | For the RTI special category: RTI tickets meeting statutory deadline ÷ RTI tickets resolved (Sindh Transparency & RTI Act 2016 deadlines). |
percent | 100 % | tkt_* · SA, DA, DG, SEC, MIN, PUB |
daily |
| KPI-SLA-009 | SLA Compliance by Severity | KPI-SLA-001 grouped by severity (low / normal / high / urgent). | percent | ≥ 95 % per sev | tkt_* · SA, DA, DG, SEC, MIN |
hourly |
| KPI-SLA-010 | SLA Compliance by Department | KPI-SLA-001 grouped by department (drives the dept leaderboard). | percent | ≥ 95 % per dept | tkt_*, org_* · SA, DA, DG, SEC, MIN, PUB |
hourly |
| KPI-SLA-011 | Breach Recovery Time | Median time from a breach occurring to the ticket being resolved. | hours | ≤ 48 h | tkt_*, aud_* · SA, DA, DG |
daily |
| KPI-SLA-012 | Holiday/Weekend Pause Share | SLA pauses due to non-business time ÷ total SLA pause events. |
ratio | — | tkt_*, aud_* · SA, DA, DG |
daily |
6.4 Family 4 — Quality & Outcome (QLT)
Quality metrics measure whether resolutions actually satisfied the company, whether the proof-of-resolution gate held, and whether the first contact was enough.
| ID | Name | Definition / Formula | Unit | Target | Source · Audience | Refresh |
|---|---|---|---|---|---|---|
| KPI-QLT-001 | Customer Satisfaction (CSAT) | Mean of post-resolution company rating (1–5) on resolved tickets. | score (1–5) | ≥ 4.2 | tkt_* (feedback) · SA, DA, DG, OF, SEC, MIN, PUB |
daily |
| KPI-QLT-002 | CSAT Response Rate | CSAT responses received ÷ CSAT surveys sent. |
percent | ≥ 40 % | tkt_*, not_* · SA, DA, DG |
daily |
| KPI-QLT-003 | Appeal Rate | Tickets appealed ÷ Tickets resolved in the period. |
ratio | ≤ 0.05 | tkt_* · SA, DA, DG, SEC, MIN |
daily |
| KPI-QLT-004 | Appeal Uphold Rate | Appeals upheld in company's favour ÷ Appeals decided. |
ratio | ≤ 0.20 | tkt_*, aud_* · SA, DA, DG, SEC, MIN |
daily |
| KPI-QLT-005 | Reopen Rate | Tickets reopened ÷ Tickets resolved. |
ratio | ≤ 0.08 | tkt_* · SA, DA, DG, OF, SEC, MIN |
daily |
| KPI-QLT-006 | Resolution Acceptance Rate | Tickets accepted by company (no appeal, no reopen) ÷ Tickets resolved. |
ratio | ≥ 0.85 | tkt_* · SA, DA, DG, SEC, MIN, PUB |
daily |
| KPI-QLT-007 | Proof-of-Resolution Completeness | Tickets reaching Resolved with ≥1 evidence attachment + resolution note ÷ Tickets reaching Resolved. Enforced by the proof gate; should be ~1.00 by construction; measures gate bypass attempts. |
ratio | = 1.00 | tkt_*, fil_*, aud_* · SA, DA, DG |
daily |
| KPI-QLT-008 | First-Contact Resolution (FCR) | Tickets resolved without reassignment, reopen, or second touch ÷ Tickets resolved, restricted to FCR-eligible categories. |
ratio | ≥ 0.60 | tkt_*, aud_* · SA, DA, DG, OF |
daily |
| KPI-QLT-009 | FCR-Eligible Share | Tickets in FCR-eligible categories ÷ all tickets (context for KPI-QLT-008). |
ratio | — | tkt_* · SA, DA, DG |
daily |
| KPI-QLT-010 | Resolution Note Quality Score | Heuristic 0–100 score from note length, presence of action verb, and attachment-of-proof flag (configurable rubric). | score (0–100) | ≥ 70 | tkt_* · SA, DA, DG |
daily |
| KPI-QLT-011 | MoM Action-Item Closure Rate | MoM action items closed by due date ÷ MoM action items created in the period. |
ratio | ≥ 0.80 | mtg_* (MoM) · SA, DA, DG, SEC, MIN |
daily |
| KPI-QLT-012 | Net Resolution Score | Composite = (Resolution Acceptance × 0.5) + (CSAT/5 × 0.3) + ((1 − Reopen Rate) × 0.2). Single headline outcome score. |
score (0–1) | ≥ 0.85 | tkt_* · SA, DA, DG, SEC, MIN |
daily |
6.5 Family 5 — Department & Staff Performance (PERF)
Department and staff performance metrics drive leaderboards, workload balancing, and bottleneck identification. All PERF KPIs are scoped by department unless the viewer is Super Admin (cross-department) or Public (anonymized department ranking only — see §11).
| ID | Name | Definition / Formula | Unit | Target | Source · Audience | Refresh |
|---|---|---|---|---|---|---|
| KPI-PERF-001 | Department Leaderboard | Departments ranked by composite = (SLA Compliance × 0.4) + (Resolution Acceptance × 0.3) + (CSAT/5 × 0.2) + ((1 − Reopen Rate) × 0.1). |
rank | — | tkt_*, org_* · SA, DA, DG, SEC, MIN, PUB |
daily |
| KPI-PERF-002 | Workload Balance (Gini) | Gini coefficient of open-ticket counts across officers in a department/section. 0 = perfectly balanced, 1 = concentrated. | ratio (0–1) | ≤ 0.30 | tkt_*, org_* · SA, DA, DG |
hourly |
| KPI-PERF-003 | Officer CSAT | Mean CSAT (1–5) attributed to the resolving officer's tickets. | score (1–5) | ≥ 4.0 | tkt_*, org_* · SA, DA, DG, OF |
daily |
| KPI-PERF-004 | Officer Resolution Rate | Tickets resolved by officer ÷ Tickets assigned to officer in the period. |
ratio | ≥ 0.90 | tkt_*, org_* · SA, DA, DG, OF |
daily |
| KPI-PERF-005 | Officer Throughput | Resolved-ticket count per officer per period. | count | — | tkt_*, org_* · SA, DA, DG, OF |
daily |
| KPI-PERF-006 | Officer Mean Handle Time | KPI-SPD-006 computed per officer. | hours | ≤ 8 h | tkt_*, aud_* · SA, DA, DG, OF |
daily |
| KPI-PERF-007 | Active-vs-Idle Time | Officer time in active states vs. idle (no ticket touched), from session + audit signals. Excludes leave. | ratio | active ≥ 0.60 of on-duty | aud_*, org_* · SA, DA, DG |
15m |
| KPI-PERF-008 | Department Backlog | Open-ticket count per department/section. | count | — | tkt_*, org_* · SA, DA, DG, OF |
live |
| KPI-PERF-009 | Bottleneck Department | Department with the highest TAT p95 and/or lowest SLA Compliance in the period (flagged). |
flag | — | anl_mv_* · SA, SEC, MIN |
daily |
| KPI-PERF-010 | Bottleneck Section | Section within a department with the longest median Time-in-State on the critical path. | flag | — | anl_mv_* · SA, DA, DG |
daily |
| KPI-PERF-011 | Top Performers | Officers ranked in the top decile by composite of throughput, CSAT, and SLA compliance (department-scoped). | rank | — | tkt_*, org_* · SA, DA, DG |
weekly |
| KPI-PERF-012 | Underperformers (watch list) | Officers below configured thresholds on SLA compliance or CSAT for 2 consecutive weeks. | list | — | tkt_*, org_* · SA, DA, DG |
weekly |
| KPI-PERF-013 | Reassignments per Ticket | Sum of internal reassignments ÷ Tickets resolved, per department. |
ratio | ≤ 0.50 | tkt_*, aud_* · SA, DA, DG |
daily |
6.6 Family 6 — AI Module Metrics (AIM)
AI metrics measure both the quality of each AI capability and the cost of running it. Every quality KPI is paired with its engine and token/cost dimension so S&ITD can govern AI spend from the same dashboards that show AI quality.
| ID | Name | Definition / Formula | Unit | Target | Source · Audience | Refresh |
|---|---|---|---|---|---|---|
| KPI-AIM-001 | OCR Success Rate | OCR calls returning usable structured output ÷ OCR calls attempted. |
percent | ≥ 95 % | ai_* · SA |
hourly |
| KPI-AIM-002 | OCR Field-Level Accuracy | Fields correctly extracted ÷ fields expected, on a sampled gold set (weekly eval). | percent | ≥ 90 % | ai_* (eval) · SA |
weekly |
| KPI-AIM-003 | Summary Acceptance Rate | AI summaries accepted (≤ threshold edit) by officer ÷ Summaries generated. |
ratio | ≥ 0.70 | ai_*, aud_* · SA, DA, DG |
daily |
| KPI-AIM-004 | Auto-Routing Accuracy | AI-suggested route matching the final accepted route ÷ Routing suggestions. |
percent | ≥ 85 % | ai_*, tkt_* · SA, DA |
hourly |
| KPI-AIM-005 | Classification Accuracy | AI category matching the final accepted category ÷ Classifications. |
percent | ≥ 85 % | ai_*, tkt_* · SA, DA |
hourly |
| KPI-AIM-006 | Draft Reply Adoption Rate | Drafts sent with ≤ threshold edit distance ÷ Drafts generated. |
ratio | ≥ 0.50 | ai_*, not_* · SA, DA, DG, OF |
daily |
| KPI-AIM-007 | Draft Edit Distance | Mean normalized Levenshtein distance between draft and sent message (lower = better). | ratio (0–1) | ≤ 0.30 | ai_*, not_* · SA, DA, DG |
daily |
| KPI-AIM-008 | Chatbot Deflection Rate | Chatbot sessions resolved without escalation to a human ÷ Chatbot sessions. |
ratio | ≥ 0.45 | ai_* · SA, SEC, MIN, PUB |
hourly |
| KPI-AIM-009 | Chatbot Satisfaction | Mean thumbs-up/down rating on chatbot answers (1–5). | score (1–5) | ≥ 3.8 | ai_* · SA |
daily |
| KPI-AIM-010 | Chatbot Escalation-to-Human | Count of chatbot sessions escalated to a live officer / ticket. | count | — | ai_*, tkt_* · SA |
hourly |
| KPI-AIM-011 | Translation Acceptance Rate | Translations accepted ≤ threshold edit ÷ Translations generated (per locale pair). |
ratio | ≥ 0.75 | ai_*, aud_* · SA |
daily |
| KPI-AIM-012 | MoM Action-Item Precision | Sampled precision of extracted action items (correct items ÷ extracted). | ratio | ≥ 0.85 | ai_* (eval), mtg_* · SA, DA |
weekly |
| KPI-AIM-013 | MoM Action-Item Recall | Sampled recall (correct items ÷ true items). | ratio | ≥ 0.80 | ai_* (eval), mtg_* · SA, DA |
weekly |
| KPI-AIM-014 | Urgency/Sentiment Accuracy | AI urgency/sentiment matching reviewer label ÷ Labels, on a sampled set. |
percent | ≥ 80 % | ai_* (eval) · SA |
weekly |
| KPI-AIM-015 | PII Redaction Precision | Correctly redacted entities ÷ Entities redacted, on a sampled set. |
percent | ≥ 99 % | ai_* (eval) · SA |
weekly |
| KPI-AIM-016 | Duplicate Detection Precision | True duplicates ÷ Detected duplicate pairs. |
ratio | ≥ 0.80 | ai_*, tkt_* · SA, DA |
daily |
| KPI-AIM-017 | Per-Engine Token Usage | Sum of input + output tokens per engine (Azure OpenAI / Gemini / Bedrock / Llama / Qwen), per capability, per day. | tokens | — | ai_* · SA |
hourly |
| KPI-AIM-018 | Per-Engine Cost | Token usage × unit price per engine per day; rollup by capability and department. | PKR | — | ai_*, anl_cfg_* (price) · SA, SEC, MIN |
daily |
| KPI-AIM-019 | AI Calls per Ticket | Mean AI calls (any capability) per resolved ticket. | ratio | — | ai_*, tkt_* · SA |
daily |
| KPI-AIM-020 | On-Prem vs Cloud Share | AI calls served by on-prem engines ÷ all AI calls (data-residency governance). |
ratio | — | ai_* · SA |
daily |
6.7 Family 7 — Engagement & Communications (COM)
Engagement and comms metrics measure whether the portal's notification and chat channels are actually reaching and being used by their audiences.
| ID | Name | Definition / Formula | Unit | Target | Source · Audience | Refresh |
|---|---|---|---|---|---|---|
| KPI-COM-001 | Notification Delivery Rate | Messages delivered ÷ Messages attempted, all channels combined and per channel. |
percent | ≥ 98 % | not_* · SA, DA |
15m |
| KPI-COM-002 | Email Delivery Rate | Emails delivered ÷ Emails attempted (Mailjet). |
percent | ≥ 98 % | not_*, int_* · SA |
15m |
| KPI-COM-003 | SMS Delivery Rate | SMS delivered ÷ SMS attempted (Jazz/Telenor). |
percent | ≥ 95 % | not_*, int_* · SA |
15m |
| KPI-COM-004 | WhatsApp Delivery Rate | WA messages delivered ÷ attempted. |
percent | ≥ 95 % | not_*, int_* · SA |
15m |
| KPI-COM-005 | Read Receipt Rate | Messages read ÷ Messages delivered, where read tracking exists (email open, WA read, in-app view). |
percent | ≥ 60 % | not_* · SA, DA, DG |
hourly |
| KPI-COM-006 | Inbound Reply Rate | Outbound notifications that received an inbound reply ÷ Outbound notifications (two-way engagement). |
ratio | — | not_*, int_* · SA, DA |
daily |
| KPI-COM-007 | Notification Opt-Out Rate | Recipients who opted out of a channel ÷ Recipients (preference-center signal). |
ratio | ≤ 0.05 | not_*, org_* · SA |
daily |
| KPI-COM-008 | Chat Activity (messages/day) | Count of internal-comms messages per day, per tier (ticket thread / inbox / channel). | count | — | com_* · SA, DA, DG |
hourly |
| KPI-COM-009 | Channel Activity | Active channel count and messages per channel in the period. | count | — | com_* · SA, DA |
hourly |
| KPI-COM-010 | Chat Median Response Time | Median time from an officer mention/DM to first reply, on internal comms. | minutes | ≤ 30 min | com_*, aud_* · SA, DA, DG |
daily |
| KPI-COM-011 | Presence Hours | Sum of officer presence hours (online + available) per department per day. | hours | — | com_*, aud_* · SA, DA, DG |
15m |
| KPI-COM-012 | KB Deflection Rate | KB "was this helpful" sessions that did not result in a ticket ÷ KB sessions. |
ratio | ≥ 0.30 | kb_*, tkt_* · SA, SEC, MIN, PUB |
daily |
6.8 Family 8 — Governance & Compliance (GOV)
Governance and compliance metrics give S&ITD assurance that access is reviewed, audit is complete, retention is honoured, and statutory categories (RTI) are handled on time.
| ID | Name | Definition / Formula | Unit | Target | Source · Audience | Refresh |
|---|---|---|---|---|---|---|
| KPI-GOV-001 | Audit Events Captured | Count of aud_event rows written in the period (overall and by event class). |
count | — | aud_* · SA |
hourly |
| KPI-GOV-002 | Audit Completeness | State-changing actions with a matching aud_event ÷ all state-changing actions (gap detection). |
percent | = 100 % | aud_*, app logs · SA |
daily |
| KPI-GOV-003 | Access Review Completion | Roles/access reviewed on schedule ÷ Roles/access due for review. |
percent | = 100 % | org_*, aud_* · SA, SEC |
monthly |
| KPI-GOV-004 | Role Change Events | Count of role/permission changes in the period (with reason captured). | count | — | org_*, aud_* · SA |
daily |
| KPI-GOV-005 | Retention Rule Executions | Count of retention-driven purges/archives executed on schedule. | count | — | fil_*, aud_* · SA |
weekly |
| KPI-GOV-006 | Data-Class Tagging Coverage | Records with a valid data-class tag ÷ all records (Public/Internal/Confidential/Restricted). |
percent | = 100 % | org_*, fil_* · SA |
daily |
| KPI-GOV-007 | RTI Tickets Handled | Count of RTI-category tickets in the period (statutory category). | count | — | tkt_* · SA, DA, DG, SEC, MIN, PUB |
daily |
| KPI-GOV-008 | RTI Deadline Compliance | RTI tickets resolved within statutory deadline ÷ RTI tickets resolved. |
percent | 100 % | tkt_* · SA, DA, DG, SEC, MIN, PUB |
daily |
| KPI-GOV-009 | Export Events | Count of analytics exports generated in the period (with who/what filters). | count | — | aud_* · SA |
daily |
| KPI-GOV-010 | Failed Login Attempts | Count of failed authentications (and locked-out accounts). | count | — | aud_*, Keycloak logs · SA |
hourly |
| KPI-GOV-011 | Step-Up Auth Events | Count of step-up challenges issued and passed/failed (sensitive actions). | count | — | aud_* · SA |
daily |
| KPI-GOV-012 | Privileged Action Count | Count of privileged admin actions (role grants, flag toggles, data purges, config edits). | count | — | aud_* · SA |
daily |
| KPI-GOV-013 | Data-Subject Requests Handled | Count of access/correction/erasure requests handled within SLA. | count | — | aud_*, org_* · SA |
weekly |
6.9 Family 9 — Public Transparency (PUB)
The Public family is the only set of KPIs exposed on the public transparency dashboard (see §11). Every KPI here is sourced from anl_pub_* (nightly anonymized), never from a live transactional query, and obeys the suppression rules in §11.3.
| ID | Name | Definition / Formula | Unit | Target | Source · Audience | Refresh |
|---|---|---|---|---|---|---|
| KPI-PUB-001 | Total Tickets Received | Cumulative count of tickets created (anonymized). | count | — | anl_pub_* · PUB |
daily |
| KPI-PUB-002 | Total Tickets Resolved | Cumulative count of tickets resolved (anonymized). | count | — | anl_pub_* · PUB |
daily |
| KPI-PUB-003 | Currently Open | Count of tickets in non-terminal states (anonymized). | count | — | anl_pub_* · PUB |
daily |
| KPI-PUB-004 | Overall SLA Compliance | Anonymized platform-wide SLA compliance rate. | percent | ≥ 95 % | anl_pub_* · PUB |
daily |
| KPI-PUB-005 | Median Resolution Time | Platform-wide median SLA-aware TAT. | days | — | anl_pub_* · PUB |
daily |
| KPI-PUB-006 | Volume by Department | Anonymized ticket count by department (department names visible — they are public bodies). | count | — | anl_pub_* · PUB |
daily |
| KPI-PUB-007 | SLA Compliance by Department | Anonymized SLA compliance per department (the public leaderboard). | percent | — | anl_pub_* · PUB |
daily |
| KPI-PUB-008 | Volume by Category | Anonymized ticket count by service category. | count | — | anl_pub_* · PUB |
daily |
| KPI-PUB-009 | Volume by District (heatmap) | Anonymized ticket count by district (drives the GIS heatmap — see §10). | count | — | anl_pub_* · PUB |
daily |
| KPI-PUB-010 | Public CSAT | Anonymized platform-wide mean CSAT (1–5). | score (1–5) | ≥ 4.0 | anl_pub_* · PUB |
daily |
7. The Seven Role Dashboards
Each role gets a dashboard tuned to its decisions. Every tile satisfies AP-1 (live + historical) and AP-2 (drill-down to the scoped ticket list). The role codes used here match /specs/en/04-roles-permissions/.
7.1 Super Admin Dashboard (SA)
Cross-department, cross-channel, system-wide view. The only role with infrastructure + AI-cost + governance tiles alongside operational KPIs.
Operational tiles: KPI-VOL-001..006, 013, 015 · KPI-SPD-001, 002, 008, 011 · KPI-SLA-001, 002, 003, 004 · KPI-QLT-001, 006, 012 · KPI-PERF-001, 009.
Volume cuts: volume by channel, category, district, department, company-size tier (KPI-VOL-007..011).
AI spend & quality: KPI-AIM-001, 004, 006, 008, 014, 015, 017, 018, 019, 020 (full AI family accessible).
Engagement/comms: KPI-COM-001..004, 008, 010.
Governance/compliance: full GOV family (KPI-GOV-001..013), including access-review completion, audit completeness, retention executions, step-up events, privileged-action count.
Config surfaces: targets (anl_cfg_*), feature-flag status summary, digest schedule health, freshness/staleness monitor for all anl_mv_* workers (see §17).
Special: system health badges (portal/API/docs/notifications uptime from the public status page), MariaDB/Redis/MinIO exporter summaries (read-only), Metabase launch link.
7.2 Department Admin Dashboard (DA)
Scoped to one department and its nested sections. Drives staffing and workload decisions.
Operational tiles: KPI-VOL-001..003, 005, 007, 008 (dept-scoped) · KPI-SPD-001, 003, 004, 007, 012 · KPI-SLA-001, 002, 003, 009 · KPI-QLT-001, 002, 005, 008.
Workload & staff: KPI-PERF-002 (Gini), KPI-PERF-005, 006, 007, 008, 013, KPI-PERF-010 (bottleneck section), KPI-PERF-011, 012 (top/under performers).
Section drill: every aggregate drillable to the section and to the individual officer's queue.
Engagement: dept-scoped KPI-COM-008, 009, 010, 011.
Exports: one-click daily dept pack (PDF) for circulation.
7.3 DG / Director Dashboard (DG)
Oversight of departments under the DG's purview, focused on SLA risk and escalation. Notify-only versus action powers are honoured per the configurable oversight matrix (/specs/en/04-roles-permissions/ §9).
At-risk focus: KPI-SLA-002, 003 (breach + at-risk, live), KPI-SLA-005 (escalation rung distribution), KPI-SPD-010 (time-in-escalation), KPI-SLA-011 (breach recovery).
Outcome: KPI-QLT-001, 005, 006, 012, KPI-PERF-001 (leaderboard for purview).
Department comparison: side-by-side dept tiles under purview, with KPI-SPD-011, 013 per dept.
Action surface: approve sensitive/VIP closures, push tickets down the escalation ladder — both gated by oversight config and step-up auth.
7.4 Officer Dashboard (OF)
Personal productivity view: "my queue, my SLA clock, my CSAT."
My queue: open tickets assigned to me (live), sorted by SLA-due; aging buckets KPI-SPD-008 (mine); KPI-SLA-003 (my at-risk tickets).
My speed: KPI-SPD-001, 005, 006 (my FRT, time-to-ack, handle time).
My quality: KPI-PERF-003, 004, 005, 006 (my CSAT, resolution rate, throughput, handle time), KPI-QLT-008 (my FCR).
Drafts & AI assist: my draft-adoption rate (KPI-AIM-006, personal view), AI suggestions pending my review (summary, routing, draft).
Comms: my mentions/DMs (KPI-COM-010 for my responsiveness), my presence hours (KPI-COM-011).
7.5 Secretary / Minister Dashboard (SEC / MIN)
Executive-summary view for the political and administrative leadership (Secretary S&ITD, Minister/SACM). Numbers are platform-wide or purview-wide; detail is one click away but not in your face.
Headline tiles: KPI-VOL-001, 002, 005, KPI-SLA-001, 010, KPI-QLT-001, 012, KPI-PUB-010.
Accountability: KPI-PERF-001 (department leaderboard), KPI-PERF-009 (bottleneck dept), KPI-SLA-005 (how many on each escalation rung, including rung 3 = SACM).
Trend: KPI-VOL-013 (MoM/YoY growth), 90-day trend lines on SLA compliance and CSAT.
AI & cost (MIN/SEC only): KPI-AIM-018 (per-engine cost rollup), KPI-AIM-008 (chatbot deflection), KPI-AIM-020 (on-prem/cloud share — data-residency posture).
RTI: KPI-GOV-007, 008 (RTI volume + statutory compliance).
Public transparency preview: a read-only mirror of what the public currently sees (§11), so leadership can compare internal vs published numbers.
Delivery: the dashboard content is also pushed as the scheduled digest (§8).
7.6 Company Dashboard (CO)
The company representative's own analytics: my tickets, my response times, my satisfaction — scoped strictly to my company's tickets (RBAC-enforced; never another company's data).
My tickets: open/closed/resolved counts, aging buckets, current SLA status per ticket, at-risk flag.
My responsiveness: KPI-SPD-009 (await-on-company duration — time the company itself added), inbound reply rate from the company side.
My satisfaction: CSAT I have given, appeals I have filed, appeal status.
My interactions: notifications I received and read, chat sessions with the desk, KB articles I found helpful.
Action surface: file a ticket, respond to an await, request a TRI meeting, submit CSAT.
7.7 Public Transparency Dashboard (PUB)
The only dashboard visible without authentication. Detailed in §11. Sourced exclusively from anl_pub_*. Tiles: KPI-PUB-001..010, the GIS heatmap (§10), a plain-language FAQ explaining each metric, and a "data published on" timestamp.
8. Scheduled Digests
A cron-driven BullMQ job renders per-role, per-department digests on configurable cadences and pushes them through the notifications pipeline (email/SMS/WhatsApp/in-app, per recipient preference). All digests are multilingual (recipient's locale) and carry Gregorian + Hijri dates.
8.1 Default digest schedule
| Audience | Cadence (default) | Contents | Channel |
|---|---|---|---|
| Minister / SACM | Weekly (Mon 08:00) | Platform headline KPIs, dept leaderboard, bottleneck dept, RTI compliance, AI spend, top escalations (rung 3). | Email (primary) + WA summary |
| Secretary S&ITD | Daily (08:00) | Yesterday's volume/resolved/backlog, SLA compliance, at-risk & breaching tickets, dept leaderboard deltas, RTI. | |
| Department Admin | Daily (07:30) | Dept-scoped volume, SLA, workload balance, bottleneck section, top/under performers, today's at-risk. | Email + in-app |
| DG / Director | Daily (08:00) | Purview SLA, at-risk & escalation rung distribution, breach recovery, outcome scores. | Email + in-app |
| Officer | Per-shift (start-of-shift + mid-shift) | My queue, my at-risk tickets, my CSAT trend, pending AI suggestions to review. | In-app + WA |
| Company (opt-in) | Weekly | My open tickets, SLA status, awaits-on-me, CSAT reminders. | |
| Super Admin (opt-in) | Daily | System health, freshness/staleness, audit completeness, AI cost, top anomalies. |
Every cadence above is configurable per recipient, per department, and per environment via anl_cfg_*. The preference center lets any user mute, switch to digest-only, or change channel per digest type.
8.2 Sample digest layout (Secretary, daily)
Sindh IT Portal — Facilitation Desk
Daily Digest · S&ITD Secretary
Gregorian: Mon 17 Jul 2026 · Hijri: 2 Safar 1448
1. HEADLINE (yesterday / rolling 7-day)
- Tickets created: 312 (7-day avg 298)
- Tickets resolved: 287 (7-day avg 271)
- Open backlog: 1,453
- SLA compliance: 96.2% (target 95%) ▲
- CSAT: 4.3 / 5
- At-risk (next 24h): 18 [drill]
- Breaching: 6 [drill]
2. DEPARTMENT LEADERBOARD (top 5 / bottom 3)
Top: Labour (98.4%) · IT (97.9%) · Investment (97.1%)
Bottom: Excise (89.2%) · Revenue (90.1%) · Works (91.4%)
3. ESCALATION RUNG
Rung 1 (2d, DG): 42
Rung 2 (5d, Sec): 9
Rung 3 (10d, SACM): 2 [drill]
4. RTI (statutory)
RTI handled: 7
RTI within deadline: 100%
5. AI & COST
AI calls yesterday: 2,114 cost: PKR 41,320
On-prem share: 38% Chatbot deflection: 47%
6. ANOMALY ALERTS (from §14)
- Excise dept SLA down 6.1pts vs 7-day avg.
- FBR intake volume +220% vs baseline (spike).
[View full dashboard] [Drill at-risk] [Manage preferences]
8.3 Digest controls
- Composability: each digest's sections map to KPI IDs in
anl_cfg_*, so an admin can add/remove sections without code. - Suppression: digests respect the same anonymization/suppression rules as the public dashboard where a section crosses into PUB-family KPIs.
- Failure handling: if a digest job fails, the failed-recipient list is retried with backoff and surfaced on the Super Admin freshness/staleness monitor.
9. Exports
9.1 Formats and engines
| Format | Engine | Use case |
|---|---|---|
| Puppeteer (server-side, headless Chromium) | Letter-quality reports on S&ITD/Sindh letterhead; used for digests circulated to ministerial/secretary audiences, RTI responses, audit packs. | |
| Excel (.xlsx) | ExcelJS | Multi-tab analyst workbooks; one tab per KPI family or per department; formulas live where useful. |
| CSV | Node stream writer | Raw extracts for downstream BI / statistical tools; always scoped by RBAC. |
All exports are generated by BullMQ workers; the user is notified (in-app + email) when the file is ready and downloads via a presigned MinIO URL with a short TTL.
9.2 Export contract
- Filters preserved: every export records the exact filters and grain used (date range, department, category, district, channel, severity, locale) in both the file header and the
aud_eventrow. - RBAC-enforced: the export worker re-checks the requesting user's scope at generation time (defence in depth — a stale session cannot widen scope).
- Step-up for sensitive: exports containing confidential/VIP tickets or PII require step-up auth before the job is queued.
- Audit:
KPI-GOV-009(export events) captures who/what/when/filters for every export. - Retention: export artifacts expire from MinIO per the data-class retention rule; the
aud_eventrow is retained for the audit window.
10. GIS / District Heatmap
The GIS/district heatmap visualizes KPI-VOL-009, KPI-PUB-009, and KPI-SLA-010/KPI-PUB-007 on a map of Sindh's districts.
Specification:
- Layer: choropleth over Sindh district polygons (GeoJSON sourced from a versioned, government-approved boundary file in
_diagrams/gis/). - Metric selector: volume / SLA compliance / median TAT / CSAT — selectable by the viewer (internal) or fixed to volume + SLA compliance (public).
- Grain: district; optional drill to tehsil where data volume permits (suppression applies below district for the public layer).
- Time selector: any bounded range; defaults to last 30 days.
- Colour scale: sequential single-hue (Ajrak indigo ramp); colour-blind safe; threshold bands configurable in
anl_cfg_*. - Interactivity: hover → district name + value + sample size; click → drill-down list (internal roles) or district summary card (public).
- Anonymization (public): the public heatmap reads from
anl_pub_*and obeys the suppression rules in §11.3 (no district with <ntickets is shown with a value; it is grouped into an "Other / insufficient data" bucket —nis configurable, default 5). - Performance: rendered client-side with ECharts geo; geometry served pre-simplified; data payload kept under 50 KB.
Implementation note: no PostGIS dependency — the heatmap uses bounding-box math and a lightweight spatial index (see /specs/en/15-tech-architecture/ §5.1).
11. Public Transparency Dashboard
The public dashboard is the portal's accountability commitment: any citizen, journalist, or company can see, at a glance, how the Sindh IT Portal is performing for the IT industry — without logging in and without exposing any individually-identifiable data.
11.1 What is shown
KPI-PUB-001Total received,KPI-PUB-002total resolved,KPI-PUB-003currently open.KPI-PUB-004overall SLA compliance,KPI-PUB-005median resolution time,KPI-PUB-010public CSAT.KPI-PUB-006/KPI-PUB-007volume and SLA compliance by department (department names are public bodies and are shown).KPI-PUB-008volume by category.KPI-PUB-009GIS district heatmap (§10).- A plain-language "what do these numbers mean?" FAQ (multilingual) and a "data published on" timestamp.
11.2 Anonymization rules
- No PII, ever. No company name, no officer name, no requester name, no ticket ID, no free-text appears on the public dashboard.
- Separate table. All public KPIs read from
anl_pub_*, populated by the nightly anonymizer worker fromanl_mv_*. No live query path connects the public route to transactional tables. - Aggregation only. Every public number is an aggregate over ≥
ntickets (defaultn = 5). Smaller cells are suppressed or rolled into "Other / insufficient data". - No small-cell leakage. Cross-tabbing two public dimensions (e.g., department × category) is suppressed where any cell falls below
n, to prevent re-identification by subtraction. - Lag. Public data is published with a 24-hour lag (yesterday's anonymized rollup), so a filer cannot recognize their own just-filed ticket in the numbers.
- No raw exports. The public dashboard offers no CSV/Excel export of underlying rows — only the published aggregates.
- Audit. Every publish event (worker run, row counts before/after suppression) is written to
aud_eventand visible to Super Admin.
11.3 Suppression parameters (configurable)
| Parameter | Default | Effect |
|---|---|---|
pub.min_cell_size |
5 | Minimum ticket count for any published cell. |
pub.lag_hours |
24 | Minimum age of data before publication. |
pub.allow_district_drill |
false | Whether district-level drill below the top number is shown publicly. |
pub.allow_dept_ranking |
true | Whether department names are ranked publicly (they are public bodies). |
pub.crosstab_suppression |
true | Suppress cross-tab cells below min_cell_size. |
11.4 Refresh
Nightly (01:00 PKT) via the anonymizer worker. The "data published on" timestamp on the page reflects the last successful publish; a failed publish leaves the previous day's data visible and raises a Super Admin alert (§17).
12. Saved Views, Favorites, and Embeddable Widgets
12.1 Saved views
Any internal user can save a dashboard state (tiles + filters + grain + date range) as a named view. Saved views are private by default and shareable within the user's department. Super Admin can promote a saved view to a department default.
12.2 Favorites
Any tile or KPI can be pinned as a favorite; favorites aggregate into a personalized "My view" tile strip at the top of the user's dashboard. Favorites persist across sessions and locales.
12.3 Embeddable widgets
Selected tiles can be exported as embeddable widgets (signed, time-limited iframe URLs) for embedding in:
- Departmental intranets and the S&ITD public site.
- The Minister/SACM briefing pages.
- Partner portals (PSEB, P@SHA) for cross-industry visibility — only PUB-family KPIs are embeddable externally.
Embeddable widgets are signed with a rotating key, scoped to a specific KPI + filter, and rate-limited. They never carry credentials; they only display anonymized or already-authorized aggregates.
13. Anomaly Detection & SLA-Breach Prediction
13.1 Anomaly detection
A scheduled worker scans anl_mv_* for statistically unusual changes and raises alerts:
- Threshold rules (configurable in
anl_cfg_*): e.g., "department SLA compliance drops > 5 points vs 7-day average", "intake volume > 200% of baseline for a channel", "CSAT for a department < 3.5 for 3 days". - Statistical anomalies: simple z-score / rolling-median-plus-MAD over a trailing window, per KPI per cut. No black-box ML at V1 — explainable rules first.
- Alert routing: anomalies flow into the notifications pipeline (in-app, WebSocket, and into the next relevant digest) and onto the Super Admin freshness/anomaly monitor.
13.2 SLA-breach prediction
KPI-SLA-003 (at-risk count) is computed by a lightweight predictor:
- For each open ticket, compute remaining SLA budget vs. remaining SLA clock, factoring the ticket's category-historical TAT distribution and the assignee's recent velocity.
- If projected resolution > SLA due with >
pconfidence, flag the ticket at-risk. pis configurable (default 0.70). The predictor is recalculated every refresh cycle forKPI-SLA-003and on every relevant domain event for the live tile.- Accuracy of the predictor is itself tracked (
predicted_at_risk_that_breached ÷ predicted_at_risk) as a governance check; target ≥ 0.70 precision to limit alert fatigue.
13.3 Alert hygiene
- Every anomaly/breach alert is deduplicated within a window and rate-limited per recipient to avoid fatigue.
- Alerts carry a drill link into the scoped ticket list (AP-2) so the recipient can act immediately.
- Alert definitions are versioned; changes are audited like any other config change.
14. Data Freshness & Accuracy Controls
14.1 Freshness SLAs
Each anl_mv_* table has a published freshness SLA (the Refresh column in the KPI catalog). A monitor compares actual last-refresh to the SLA; breach raises a stale badge on affected tiles and a Super Admin alert.
| Refresh tier | Max staleness | Examples |
|---|---|---|
| live | 5 s | open-ticket count, breach count |
| 5m / 15m | 5 / 15 min | volume, delivery rates, presence |
| hourly | 60 min | FRT/TAT distributions, SLA compliance |
| daily | 24 h + 1 run | CSAT, reopen/acceptance rates, leaderboards |
| weekly / monthly | 7 / 30 d | AI evals, access reviews |
14.2 Reconciliation
Every nightly run reconciles anl_mv_* against the transactional source totals (row counts and sum checks). Discrepancies above a threshold raise a Super Admin alert and a re-compute job; the affected tiles are marked stale until reconciled.
14.3 Accuracy controls
- One definition per KPI (AP-5) — see glossary §16.
- Sampled audits: Super Admin can trigger a sampled re-computation of any KPI against raw rows; results attached to the freshness monitor.
- Reproducibility: every export and every saved view records its filter set, so any number in a report can be reproduced from
anl_mv_*+ the filter set + the version of the definition.
15. Self-Serve Report Builder (Metabase)
Internal roles (primarily Super Admin, Dept Admin, DG, Secretary) get embedded Metabase for ad-hoc exploration and bespoke reports.
- Access: signed-URL embedding in the portal; users never see the Metabase login screen. RBAC scopes are passed through as embedding parameters so a Dept Admin's Metabase session is already scoped to their department.
- Data boundary: Metabase connects read-only to
anl_cube_*(and selectedanl_mv_*); never to transactional tables; never toanl_pub_*(the public writer is the only writer there). - Saved questions & dashboards: Metabase questions/dashboards can be saved privately or shared within role; sharing is governed by the same RBAC.
- Exports from Metabase: produce CSV/Excel through the same audited export path (§9) so they appear in
KPI-GOV-009. - Sandboxing: row-level sandboxing is enforced via Metabase sandboxing on top of the DB-level scope, so a Dept Admin literally cannot select another department's rows.
- Public embedding: disabled for ad-hoc questions. Only curated PUB-family tiles (§12.3) are embeddable externally.
16. Definitions Glossary (Metrics)
This glossary is the canonical source for every metric term. It is mirrored (metric-key-only) into _glossary.md so the same definitions surface in UR/Sindhi dashboards. Where a definition is non-obvious, the formula is given.
| Term | Definition |
|---|---|
| FRT (First Response Time) | Time from ticket creation (New) to the first outbound message the company receives on that ticket, on any channel. Working-time aware (excludes SLA-pause windows). |
| TAT (Turnaround Time) | Working time from creation to Resolved, SLA-pause-aware. Reported as median, p90, p95 over a population. |
| SLA Compliance | Tickets meeting the configured SLA window ÷ Tickets resolved in the period. |
| SLA Pause | A period during which the SLA clock is held: await-on-company, weekend, or Sindh public holiday. |
| Escalation Rung | Step on the ladder: rung 1 = DG (2 d), rung 2 = Department Secretary (5 d), rung 3 = S&ITD Secretary / SACM (10 d). Configurable per department. |
| CSAT | Post-resolution rating by the company, scale 1–5. Reported as mean over surveys returned. |
| CSAT Response Rate | Surveys returned ÷ Surveys sent. |
| Appeal | A formal request by a company to reconsider a resolution, after Resolved. |
| Reopen | A ticket transitioned from Resolved to Reopened (company rejected the resolution). |
| Resolution Acceptance | A resolved ticket that was neither appealed nor reopened within the acceptance window. |
| Proof of Resolution | ≥1 evidence attachment + resolution note required before Resolved; the proof gate enforces it. |
| FCR (First-Contact Resolution) | A resolved ticket with no reassignment, no reopen, and no second internal touch, within FCR-eligible categories. |
| Net Resolution Score | Composite: 0.5·(Resolution Acceptance) + 0.3·(CSAT/5) + 0.2·(1 − Reopen Rate). |
| Active-vs-Idle | Officer on-duty time spent actively working tickets vs. idle, from session + audit signals. |
| Gini (workload balance) | Gini coefficient of open-ticket counts across officers; 0 = perfectly balanced, 1 = all on one officer. |
| Bottleneck Department / Section | The unit with the worst SLA/TAT in the period (flagged). |
| Throughput Rate | Tickets Resolved ÷ Tickets Created over the period. |
| Aging Bucket | Age-of-open-ticket bucket: 0–2 d, 3–5 d, 6–10 d, 11–20 d, 21+ d. |
| OCR Success | An OCR call returning usable structured output (no error, non-empty, schema-valid). |
| Draft Adoption | A draft reply sent with edit distance ≤ the configured threshold. |
| Chatbot Deflection | A chatbot session that resolved without escalating to a human officer or creating a ticket. |
| Token | A model usage unit (input + output), recorded per engine per capability per call in ai_*. |
| Per-Engine Cost | Token usage × configured unit price per engine, per day. |
| Delivery Rate | Messages delivered ÷ Messages attempted, per channel. |
| Read Receipt | A delivered message that was opened/viewed (email open, WA read, in-app view). |
| KB Deflection | A KB "was this helpful" session that did not result in a ticket creation. |
| Audit Completeness | State-changing actions with a matching aud_event ÷ all state-changing actions. |
| Anonymized Aggregate | A KPI value computed over ≥ pub.min_cell_size tickets, with no PII, sourced from anl_pub_*. |
| Suppression | Withholding a public cell whose sample size is below pub.min_cell_size, or that would allow re-identification by cross-tab subtraction. |
| Stale Tile | A dashboard tile whose underlying data is older than its freshness SLA; shown with a visible badge. |
| Saved View | A named, restorable dashboard state (tiles + filters + grain + date range). |
17. Configuration Surface (summary)
All tunables live in anl_cfg_* and are editable by Super Admin (step-up auth). Changes are audited (aud_event) and versioned.
| Config group | Examples |
|---|---|
targets.* |
SLA windows, CSAT goal, FCR target, breach thresholds. |
pub.* |
Public suppression parameters (§11.3). |
digests.* |
Per-audience cadence, sections, channels, quiet hours. |
anomaly.* |
Threshold rules, z-score window, predictor confidence p. |
refresh.* |
Per-table refresh cadence and freshness SLAs. |
engines.* |
Per-engine unit prices (for KPI-AIM-018), on-prem/cloud policy. |
embedding.* |
Signed-URL TTLs, allowed embed scopes. |
18. Master KPI Catalog
The complete catalog: 9 families, 120 KPIs. Counts per family: VOL 15 · SPD 13 · SLA 12 · QLT 12 · PERF 13 · AIM 20 · COM 12 · GOV 13 · PUB 10.
| ID | Family | Name | Unit | Refresh |
|---|---|---|---|---|
| KPI-VOL-001 | Volume & Throughput | Tickets Created | count | live |
| KPI-VOL-002 | Volume & Throughput | Tickets Resolved | count | 5m |
| KPI-VOL-003 | Volume & Throughput | Tickets Closed | count | 5m |
| KPI-VOL-004 | Volume & Throughput | Throughput Rate | ratio | hourly |
| KPI-VOL-005 | Volume & Throughput | Open Backlog | count | live |
| KPI-VOL-006 | Volume & Throughput | Net Backlog Change | count | daily |
| KPI-VOL-007 | Volume & Throughput | Volume by Channel | count | hourly |
| KPI-VOL-008 | Volume & Throughput | Volume by Category | count | hourly |
| KPI-VOL-009 | Volume & Throughput | Volume by District | count | hourly |
| KPI-VOL-010 | Volume & Throughput | Volume by Department | count | hourly |
| KPI-VOL-011 | Volume & Throughput | Volume by Company-Size Tier | count | daily |
| KPI-VOL-012 | Volume & Throughput | Peak-Hour Volume | count | hourly |
| KPI-VOL-013 | Volume & Throughput | Volume Growth (MoM/YoY) | percent | daily |
| KPI-VOL-014 | Volume & Throughput | Draft → Submitted Conversion | ratio | daily |
| KPI-VOL-015 | Volume & Throughput | Reopened Volume | count | hourly |
| KPI-SPD-001 | Speed | First Response Time (FRT) | hours | hourly |
| KPI-SPD-002 | Speed | Turnaround Time (TAT) | days | hourly |
| KPI-SPD-003 | Speed | Time-to-Triage | hours | hourly |
| KPI-SPD-004 | Speed | Time-to-Assign | hours | hourly |
| KPI-SPD-005 | Speed | Time-to-Acknowledge | hours | hourly |
| KPI-SPD-006 | Speed | Mean Handle Time | hours | daily |
| KPI-SPD-007 | Speed | Time-in-State (per state) | hours | hourly |
| KPI-SPD-008 | Speed | Aging Buckets | count | hourly |
| KPI-SPD-009 | Speed | Await-on-Company Duration | hours | daily |
| KPI-SPD-010 | Speed | Time-in-Escalation | hours | hourly |
| KPI-SPD-011 | Speed | p95 Resolution Time | days | daily |
| KPI-SPD-012 | Speed | Reassignment Latency | hours | daily |
| KPI-SPD-013 | Speed | Resolution Cycle Time by Category | days | daily |
| KPI-SLA-001 | SLA & Escalation | SLA Compliance Rate | percent | hourly |
| KPI-SLA-002 | SLA & Escalation | SLA Breach Count | count | live |
| KPI-SLA-003 | SLA & Escalation | SLA At-Risk Count | count | live |
| KPI-SLA-004 | SLA & Escalation | Escalation Rate | ratio | hourly |
| KPI-SLA-005 | SLA & Escalation | Escalation Rung Distribution | count | hourly |
| KPI-SLA-006 | SLA & Escalation | Repeat Escalation Rate | ratio | daily |
| KPI-SLA-007 | SLA & Escalation | SLA Pause Events | count | hourly |
| KPI-SLA-008 | SLA & Escalation | RTI SLA Compliance | percent | daily |
| KPI-SLA-009 | SLA & Escalation | SLA Compliance by Severity | percent | hourly |
| KPI-SLA-010 | SLA & Escalation | SLA Compliance by Department | percent | hourly |
| KPI-SLA-011 | SLA & Escalation | Breach Recovery Time | hours | daily |
| KPI-SLA-012 | SLA & Escalation | Holiday/Weekend Pause Share | ratio | daily |
| KPI-QLT-001 | Quality/Outcome | Customer Satisfaction (CSAT) | score (1–5) | daily |
| KPI-QLT-002 | Quality/Outcome | CSAT Response Rate | percent | daily |
| KPI-QLT-003 | Quality/Outcome | Appeal Rate | ratio | daily |
| KPI-QLT-004 | Quality/Outcome | Appeal Uphold Rate | ratio | daily |
| KPI-QLT-005 | Quality/Outcome | Reopen Rate | ratio | daily |
| KPI-QLT-006 | Quality/Outcome | Resolution Acceptance Rate | ratio | daily |
| KPI-QLT-007 | Quality/Outcome | Proof-of-Resolution Completeness | ratio | daily |
| KPI-QLT-008 | Quality/Outcome | First-Contact Resolution (FCR) | ratio | daily |
| KPI-QLT-009 | Quality/Outcome | FCR-Eligible Share | ratio | daily |
| KPI-QLT-010 | Quality/Outcome | Resolution Note Quality Score | score (0–100) | daily |
| KPI-QLT-011 | Quality/Outcome | MoM Action-Item Closure Rate | ratio | daily |
| KPI-QLT-012 | Quality/Outcome | Net Resolution Score | score (0–1) | daily |
| KPI-PERF-001 | Dept/Staff Performance | Department Leaderboard | rank | daily |
| KPI-PERF-002 | Dept/Staff Performance | Workload Balance (Gini) | ratio (0–1) | hourly |
| KPI-PERF-003 | Dept/Staff Performance | Officer CSAT | score (1–5) | daily |
| KPI-PERF-004 | Dept/Staff Performance | Officer Resolution Rate | ratio | daily |
| KPI-PERF-005 | Dept/Staff Performance | Officer Throughput | count | daily |
| KPI-PERF-006 | Dept/Staff Performance | Officer Mean Handle Time | hours | daily |
| KPI-PERF-007 | Dept/Staff Performance | Active-vs-Idle Time | ratio | 15m |
| KPI-PERF-008 | Dept/Staff Performance | Department Backlog | count | live |
| KPI-PERF-009 | Dept/Staff Performance | Bottleneck Department | flag | daily |
| KPI-PERF-010 | Dept/Staff Performance | Bottleneck Section | flag | daily |
| KPI-PERF-011 | Dept/Staff Performance | Top Performers | rank | weekly |
| KPI-PERF-012 | Dept/Staff Performance | Underperformers (watch list) | list | weekly |
| KPI-PERF-013 | Dept/Staff Performance | Reassignments per Ticket | ratio | daily |
| KPI-AIM-001 | AI Module Metrics | OCR Success Rate | percent | hourly |
| KPI-AIM-002 | AI Module Metrics | OCR Field-Level Accuracy | percent | weekly |
| KPI-AIM-003 | AI Module Metrics | Summary Acceptance Rate | ratio | daily |
| KPI-AIM-004 | AI Module Metrics | Auto-Routing Accuracy | percent | hourly |
| KPI-AIM-005 | AI Module Metrics | Classification Accuracy | percent | hourly |
| KPI-AIM-006 | AI Module Metrics | Draft Reply Adoption Rate | ratio | daily |
| KPI-AIM-007 | AI Module Metrics | Draft Edit Distance | ratio (0–1) | daily |
| KPI-AIM-008 | AI Module Metrics | Chatbot Deflection Rate | ratio | hourly |
| KPI-AIM-009 | AI Module Metrics | Chatbot Satisfaction | score (1–5) | daily |
| KPI-AIM-010 | AI Module Metrics | Chatbot Escalation-to-Human | count | hourly |
| KPI-AIM-011 | AI Module Metrics | Translation Acceptance Rate | ratio | daily |
| KPI-AIM-012 | AI Module Metrics | MoM Action-Item Precision | ratio | weekly |
| KPI-AIM-013 | AI Module Metrics | MoM Action-Item Recall | ratio | weekly |
| KPI-AIM-014 | AI Module Metrics | Urgency/Sentiment Accuracy | percent | weekly |
| KPI-AIM-015 | AI Module Metrics | PII Redaction Precision | percent | weekly |
| KPI-AIM-016 | AI Module Metrics | Duplicate Detection Precision | ratio | daily |
| KPI-AIM-017 | AI Module Metrics | Per-Engine Token Usage | tokens | hourly |
| KPI-AIM-018 | AI Module Metrics | Per-Engine Cost | PKR | daily |
| KPI-AIM-019 | AI Module Metrics | AI Calls per Ticket | ratio | daily |
| KPI-AIM-020 | AI Module Metrics | On-Prem vs Cloud Share | ratio | daily |
| KPI-COM-001 | Engagement/Comms | Notification Delivery Rate | percent | 15m |
| KPI-COM-002 | Engagement/Comms | Email Delivery Rate | percent | 15m |
| KPI-COM-003 | Engagement/Comms | SMS Delivery Rate | percent | 15m |
| KPI-COM-004 | Engagement/Comms | WhatsApp Delivery Rate | percent | 15m |
| KPI-COM-005 | Engagement/Comms | Read Receipt Rate | percent | hourly |
| KPI-COM-006 | Engagement/Comms | Inbound Reply Rate | ratio | daily |
| KPI-COM-007 | Engagement/Comms | Notification Opt-Out Rate | ratio | daily |
| KPI-COM-008 | Engagement/Comms | Chat Activity (messages/day) | count | hourly |
| KPI-COM-009 | Engagement/Comms | Channel Activity | count | hourly |
| KPI-COM-010 | Engagement/Comms | Chat Median Response Time | minutes | daily |
| KPI-COM-011 | Engagement/Comms | Presence Hours | hours | 15m |
| KPI-COM-012 | Engagement/Comms | KB Deflection Rate | ratio | daily |
| KPI-GOV-001 | Governance/Compliance | Audit Events Captured | count | hourly |
| KPI-GOV-002 | Governance/Compliance | Audit Completeness | percent | daily |
| KPI-GOV-003 | Governance/Compliance | Access Review Completion | percent | monthly |
| KPI-GOV-004 | Governance/Compliance | Role Change Events | count | daily |
| KPI-GOV-005 | Governance/Compliance | Retention Rule Executions | count | weekly |
| KPI-GOV-006 | Governance/Compliance | Data-Class Tagging Coverage | percent | daily |
| KPI-GOV-007 | Governance/Compliance | RTI Tickets Handled | count | daily |
| KPI-GOV-008 | Governance/Compliance | RTI Deadline Compliance | percent | daily |
| KPI-GOV-009 | Governance/Compliance | Export Events | count | daily |
| KPI-GOV-010 | Governance/Compliance | Failed Login Attempts | count | hourly |
| KPI-GOV-011 | Governance/Compliance | Step-Up Auth Events | count | daily |
| KPI-GOV-012 | Governance/Compliance | Privileged Action Count | count | daily |
| KPI-GOV-013 | Governance/Compliance | Data-Subject Requests Handled | count | weekly |
| KPI-PUB-001 | Public Transparency | Total Tickets Received | count | daily |
| KPI-PUB-002 | Public Transparency | Total Tickets Resolved | count | daily |
| KPI-PUB-003 | Public Transparency | Currently Open | count | daily |
| KPI-PUB-004 | Public Transparency | Overall SLA Compliance | percent | daily |
| KPI-PUB-005 | Public Transparency | Median Resolution Time | days | daily |
| KPI-PUB-006 | Public Transparency | Volume by Department | count | daily |
| KPI-PUB-007 | Public Transparency | SLA Compliance by Department | percent | daily |
| KPI-PUB-008 | Public Transparency | Volume by Category | count | daily |
| KPI-PUB-009 | Public Transparency | Volume by District (heatmap) | count | daily |
| KPI-PUB-010 | Public Transparency | Public CSAT | score (1–5) | daily |
19. Open Questions / TBD
| # | Item | Status |
|---|---|---|
| 1 | Final unit prices per AI engine for KPI-AIM-018 (depends on contracted rates). |
TBD |
| 2 | Public-dashboard suppression defaults (pub.min_cell_size, district drill) — pending S&ITD sign-off. |
TBD |
| 3 | Whether department names are ranked publicly by default (legal/comms review). | TBD |
| 4 | Breach-predictor confidence threshold p (default 0.70) — tune after first month of real data. |
TBD |
| 5 | Metabase sandbox row-level scope rules for cross-dept DG purviews. | TBD |
| 6 | Hijri-aware grain for monthly digests (lunar vs Gregorian month). | TBD |
End of document.