Availability Measurement

How a CampusCore deployment tracks that the system is available, and how the availability percentage is calculated, excluding scheduled maintenance. This document states what "available" means, where the numbers come from, and the exact formula and queries that produce the percentage. The companion artifact is the availability CloudWatch dashboard in the deployment's AWS account, created by Terraform (infrastructure/app/availability.tf).

The availability definition

The system is available when both of these hold:

1. The deployment is reachable and its core dependencies answer. An external Route 53 health check requests the public /readiness endpoint every 30 seconds from multiple AWS regions outside the deployment's own network. Each successful probe proves, in one request: the web service is serving, PostgreSQL is answering queries, the pgvector extension is present, and the cache is answering a read/write round-trip. A green probe is therefore not a self-report - it is the request path a real user's browser takes, exercised end to end.
2. The assistant is answering. A reachable site whose chat requests are failing does not count as up. Real-traffic chat metrics carry this term: any 5-minute interval in which more than 5% of chat requests errored counts as unavailable time, using the same 5% floor the chat-error-rate alarm pages on.

Background processing (document ingestion, scheduled jobs) is monitored and alarmed but does not gate the availability number. A delayed document ingest is an operational issue, not "the system is down".

The measurement source

• Probe: an aws_route53_health_check per deployment, requesting https://{env}.campuscoreai.com/readiness (HTTPS with SNI; an environment without a public subdomain is probed at its load-balancer DNS name over HTTP). Interval 30 seconds, failure threshold 3, verdicts aggregated by Route 53 across its checker fleet.
• Probe metric: AWS/Route53 HealthCheckStatus (1 = healthy, 0 = unhealthy), published in the us-east-1 region regardless of where the deployment runs.
• Chat metrics: CampusCore namespace, chat.request.count and chat.request.error_count, dimensioned by DeploymentId, published from the application in the deployment's own region.
• Retention: all terms are computed from CloudWatch metric data, which is retained at 5-minute resolution for 63 days - comfortably past a monthly reporting cycle. Alarm state history is never used in the calculation (AWS retains it for only 14 days).

The formula

For a reporting period of T seconds:

• U = the average of HealthCheckStatus over the period (the fraction of time the probe found the deployment up).
• B = 300 x the number of 5-minute intervals in which the chat error rate exceeded 5%.
• M = the total seconds of scheduled maintenance announced to the institution in advance (see the maintenance-window log below).

availability % = ((U x T) - B) / (T - M) x 100

An interval in which the probe was down and the chat floor was also breached is deducted by both terms. The mechanical method accepts that double deduction as the conservative reading - it can only understate availability, never overstate it. When a reported number sits at a contractual boundary, the operator refines it by excluding from B the intervals in which the probe was already down, and states that refinement in the report.

When a maintenance window exists inside the period, the probe and chat queries are run piecewise around the window (the same queries with adjusted start/end times), so downtime inside an announced window neither penalizes the numerator nor inflates the denominator.

The no-traffic convention: with zero chat traffic, the error-rate expression yields no intervals over the floor, so B = 0 and the probe alone carries the number. Missing chat data never counts against availability - only the probe, which never stops sampling, can register downtime in a quiet period.

Reproducing the number

Both terms are single CLI queries against CloudWatch, runnable by any operator with read access to the deployment's account. Substitute the health-check ID (visible on the dashboard widget or via aws route53 list-health-checks), the deployment ID (the sanitized environment name, e.g. vsu-troy-pilot), and the reporting period. The probe publishes continuously, so its 5-minute buckets are complete and the average below is the period average; the chat query counts only over-floor buckets, so buckets CloudWatch omits (no traffic) correctly contribute nothing.

The probe term U (note the region is always us-east-1):

aws cloudwatch get-metric-data --region us-east-1 \
  --start-time 2026-08-01T00:00:00Z --end-time 2026-09-01T00:00:00Z \
  --metric-data-queries '[{"Id":"uptime","MetricStat":{"Metric":{"Namespace":"AWS/Route53","MetricName":"HealthCheckStatus","Dimensions":[{"Name":"HealthCheckId","Value":"<health-check-id>"}]},"Period":300,"Stat":"Average"}}]' \
  --query 'MetricDataResults[0].Values' | jq 'add / length'

The chat term (the count of bad 5-minute intervals; multiply by 300 for B), run in the deployment's region:

aws cloudwatch get-metric-data \
  --start-time 2026-08-01T00:00:00Z --end-time 2026-09-01T00:00:00Z \
  --metric-data-queries '[
    {"Id":"error_rate","Expression":"IF(requests > 0, (errors / requests) * 100, 0)"},
    {"Id":"errors","MetricStat":{"Metric":{"Namespace":"CampusCore","MetricName":"chat.request.error_count","Dimensions":[{"Name":"DeploymentId","Value":"<deployment-id>"}]},"Period":300,"Stat":"Sum"},"ReturnData":false},
    {"Id":"requests","MetricStat":{"Metric":{"Namespace":"CampusCore","MetricName":"chat.request.count","Dimensions":[{"Name":"DeploymentId","Value":"<deployment-id>"}]},"Period":300,"Stat":"Sum"},"ReturnData":false}
  ]' \
  --query 'MetricDataResults[0].Values' | jq '[.[] | select(. > 5)] | length'

Alarming

The probe is alarmed like every other critical signal: availability-probe-failing fires when HealthCheckStatus drops below 1 for three consecutive minutes, delivering to the deployment's SNS topic (email) and Slack channel. A deleted or misconfigured health check reads as an outage (treat_missing_data = breaching), never as silence. The chat floor is alarmed by chat-error-rate at the same 5% threshold the formula uses, so an interval that would deduct from the availability number also pages when it happens.

One caveat: AWS/Route53 metrics exist only in us-east-1, and a CloudWatch alarm can only notify a same-region SNS topic. Every current deployment runs in us-east-1, where this is moot; a future deployment in another region keeps the probe and the dashboard but needs a us-east-1 notification fabric before the probe alarm can be created for it (the alarm is region-gated in infrastructure/app/monitoring.tf).

Reviewing availability

To review availability for any period, in the deployment's AWS account:

1. Open CloudWatch -> Dashboards -> availability. The first widget shows the availability percentage over the selected time range, derived live from the probe.
2. Walk the panels: probe status, chat success rate with per-provider model metrics, HTTP, retrieval (hard errors split from empty results), ingestion queues, compute.
3. For a precise number over a fixed reporting period, run the two queries above and apply the formula.

The dashboard lives in the institution's own AWS account, so granting anyone a read-only console view of it is an institution-side decision.

Maintenance windows

Scheduled maintenance announced to the institution in advance is excluded from the denominator. This log is the record the formula's M term is computed from: one line per window, appended when the window is announced, never edited afterward (git history timestamps every entry).

Date	Window (UTC)	Announced	Notes

No scheduled maintenance window has been taken to date.