Skip to content

04 — Post-Deployment

Scope: Per-client. Configure the application, set up integrations, and verify everything works.

Part A: With the Client

These steps are done together with the client or require their input.

4a. Complete Setup Wizard

Time: ~10 minutes

  1. Navigate to the deployed URL (e.g., https://ai.howard.edu)
  2. Log in with the superuser account
  3. You'll be redirected to the Setup Wizard (since is_configured=False)
  4. Fill in:
  5. Institution name
  6. Institution abbreviation
  7. Assistant name
  8. Logo image (optional upload)
  9. Welcome message (optional)
  10. Primary brand color (optional)
  11. Submit — the workspace is now configured and accessible to users

4b. Configure SSO

Time: ~15 minutes

SSO is configured via the Django admin panel — no redeployment needed.

Gather from University IT

Item Description
IdP Entity ID Identity Provider entity identifier
IdP Metadata URL Metadata endpoint (optional)
SSO URL Single Sign-On endpoint
SLO URL Single Logout endpoint
X509 Certificate IdP signing certificate (PEM)
Attribute mappings email, firstName, lastName
Login button name e.g., "Howard SSO"

Create SSOProvider in Django Admin

  1. Log in to https://{domain}/cc_admin/django/ with the superuser account
  2. Go to Authentication > SSO Providers > Add
  3. Fill in the shared fields:
  4. Name: Display name for the login button (e.g., "Howard SSO")
  5. Protocol: SAML 2.0 or OpenID Connect
  6. Client ID: Organisation slug (e.g., howard-sso). This is the URL slug, not the IdP's OIDC client id.
  7. Fill in the protocol-specific fields (the other section can be left blank):

For SAML 2.0: - IdP Entity ID, SSO URL, SLO URL: From university IT (or set IdP Metadata URL to auto-discover) - X.509 Certificate: Paste the PEM cert (encrypted at rest)

For OpenID Connect: - OIDC Provider URL: The IdP's discovery base, e.g. https://login.university.edu (serves /.well-known/openid-configuration) - OIDC Client ID: The client identifier the IdP issued for CampusCore (the token audience — not the slug) - OIDC Client Secret: Issued alongside the client id (encrypted at rest) - OIDC Scopes: defaults to openid email profile 5. Map your IdP claims to CampusCore fields in Map your IdP claims to CampusCore fields (Email, External ID, First/Last name, and the Groups claim if you want role sync). The defaults (email, sub, given_name, family_name) are the OIDC standard claim names. 6. Check Is enabled 7. Save — the SSO provider is immediately active (no deploy needed)

Give the Service-Provider Details to University IT

After saving, the admin panel shows read-only URLs to register with the IdP:

SAML (under "Service Provider URLs"): - SP Entity ID: https://{domain}/accounts/saml/{client_id}/metadata/ - ACS URL: https://{domain}/accounts/saml/{client_id}/acs/

OIDC (under "OIDC Configuration"): - Redirect URI: https://{domain}/accounts/oidc/{client_id}/login/callback/

University IT configures their IdP with the relevant value(s) plus the released claims (email, first name, last name, and the groups claim if role sync is used).

4c. Set Up Connectors

Connectors (e.g., the Google products) are configured post-deployment via the Django admin panel.

Additional connector guides will be added to references/ as integrations are built.

4d. Upload Initial Knowledge Base

  1. Log in as admin
  2. Go to Settings > Knowledge Admin
  3. Create knowledge folders for different content categories
  4. Upload documents (PDF, DOCX, etc.) to each folder
  5. Documents are automatically processed and indexed

Part B: Internal

These steps are done by us without client involvement.

4e. Configure Scraper (if needed)

Time: ~30 minutes

If the university needs web content scraped:

  1. Create a scraper config file at campuscore_app/scrapers/configs/{client}.py
  2. Define the scraping targets (URLs, selectors, schedules)
  3. Commit and push with a deployment

4f. Run First Scrape

After the scraper config is deployed:

# Find the running task
TASK_ARN=$(aws ecs list-tasks --cluster campuscore-howard \
  --service-name campuscore-howard --query 'taskArns[0]' --output text)

# Run the scraper
aws ecs execute-command \
  --cluster campuscore-howard \
  --task $TASK_ARN \
  --container campuscore-web \
  --interactive \
  --command "python manage.py scrape_webpages"

4g. Verify Sentry is Receiving Events

Time: ~5 minutes

If the tenant has a SENTRY_DSN set in its GitHub Environment (see Sentry Setup for the full operator playbook), confirm telemetry is flowing before declaring the deploy done.

  1. Check the deploy workflow log. Find the deploy-app job → Verify Sentry config landed in TF_VAR_sentry_dsn step. Expected:
::notice::SENTRY_DSN secret is present (length=…, prefix=https://...)

If you see ::warning::SENTRY_DSN secret resolved to empty, stop and fix the secret before continuing — the ECS task will boot with Sentry disabled.

  1. Check the running container's boot log in CloudWatch (/ecs/campuscore-<tenant>). Filter to the most recent task start. You should see three lines on startup:
Sentry initialized: env=<tenant-env> release=<sha> traces_sample_rate=0.20 logs=True instrumenter=otel
OTel tracing initialized — Postgres exporter + Sentry bridge (OTel is single trace source)
OTel auto-instrumentation enabled for: psycopg, httpx, logging
  1. Fire a smoke event from inside the running task to confirm round-trip:
aws ecs execute-command \
  --cluster campuscore-<tenant> \
  --task $TASK_ARN \
  --container campuscore-web \
  --interactive \
  --command "python -c 'import logging, sentry_sdk; logging.getLogger(\"smoke\").error(\"DEPLOY_VERIFY_SMOKE\"); sentry_sdk.flush(timeout=5); print(\"flushed\")'"
  1. Confirm in Sentry: open the tenant's Sentry project, set the environment filter to <tenant-env>, and within ~30 seconds you should see:
  2. Issues tab: a DEPLOY_VERIFY_SMOKE event
  3. Logs tab: the same log record (separate product, same event)

If anything is missing, see the Troubleshooting section of Sentry Setup.

4h. Verify Slack workflow_runs is Receiving Events

Time: ~5 minutes

If the tenant has SLACK_CHANNEL_WORKFLOW_RUNS set in its GitHub Environment (see Slack Setup for the full operator playbook including bot install + channel creation + invite), confirm the bot is posting before declaring the deploy done.

  1. Check the deploy workflow log. Find the deploy-app job → Export Terraform variables step. Expected:
echo "TF_VAR_slack_bot_token=***"
echo "TF_VAR_slack_channel_workflow_runs=C0XXXXXXXX"

=*** (masked, non-empty) on the bot token and =C0XXXXXXXX (visible, non-empty) on the channel ID. If either is blank, fix the GitHub Environment secret/variable before continuing — the ECS task will run with Slack disabled.

  1. Confirm the running container has the env vars (length check only — never print the token):
aws ecs describe-task-definition \
  --task-definition campuscore-<tenant-env> \
  --query 'taskDefinition.containerDefinitions[0].environment[?name==`SLACK_BOT_TOKEN` || name==`SLACK_CHANNEL_WORKFLOW_RUNS`].{name: name, has_value: length(value) > `0`}' \
  --output table \
  --profile <tenant>

Both rows should show has_value: True.

  1. Trigger a real Slack event from the dashboard:
  2. Open https://<tenant-domain>/admin/observability/vector/
  3. Switch to the Maintenance tab

  4. Click Run check now

  5. Confirm in Slack within ~10 seconds:

  6. Open the channel the environment's SLACK_CHANNEL_WORKFLOW_RUNS variable points at (by convention the shared #client_workflow_runs)
  7. Expected pair of messages from @campuscoreplatform (the CampusCorePlatform app's bot user): 
    ▶ Index health check starting
✓ Index health check — all metrics ok   (or ⚠ / ✗ depending on state)

If the message doesn't arrive, see the Troubleshooting section of Slack Setup. The most common cause is the channel value set as a secret instead of a variable, or the bot not invited to a private channel.

4i. Verify the Auto-Rebuild Schedule (or Disable It)

Time: ~2 minutes

The EventBridge auto-rebuild schedule runs daily and conditionally triggers REINDEX when an index-health metric trips. It is on by default, so the deploy you just ran created it. See Vector Index Observability for the metrics + thresholds.

Verify it exists:

aws scheduler get-schedule \
  --name "campuscore-<tenant-env>-auto-rebuild" \
  --profile <tenant>

The schedule fires at 06:00 UTC daily by default. Override with INDEX_MAINTENANCE_SCHEDULE_CRON if a different cron expression makes sense for the tenant's traffic pattern.

To turn the schedule off for this tenant, set the toggle and re-deploy:

gh variable set ENABLE_INDEX_MAINTENANCE_SCHEDULE \
  --env <tenant-env> \
  --repo CampusCoreAI/campuscore \
  --body 'false'

4j. Delete Onboarding Superuser

Important: Never grant is_superuser (or is_staff) to an institution admin. CampusCore reserves the superuser flag exclusively for CampusCore operators - it is the only marker that opens Django admin, the /cc_admin/ tools, and CampusCore-controlled feature availability. Institution admins are provisioned by granting the institution_admin role (via an SSO group mapping, or a manual grant in Settings → Users made by an operator); they administer the workspace entirely through the SPA's Settings surface and carry no Django flags. A stray is_staff=True grants nothing, but set neither flag - the account state should say what it means.

The bootstrap superuser created by ensure_superuser is for CampusCore use during onboarding. Never share its credentials with the institution.

Once SSO is confirmed working and a university admin holds the institution_admin role (confirm they can open Settings → Users), delete the onboarding superuser:

aws ecs execute-command \
  --cluster campuscore-howard \
  --task $TASK_ARN \
  --container campuscore-web \
  --interactive \
  --command "python manage.py shell -c \"from django.contrib.auth.models import User; User.objects.filter(username='admin').delete()\""

If CampusCore staff need ongoing platform-admin access for support, create a dedicated CampusCore-owned superuser account before deleting the bootstrap one.

Part C: Verification

Verification Checklist

Time: ~15 minutes

  • Branding: Logo, assistant name, and colors appear correctly in header, sidebar, and login page
  • SSO: Login with university SSO works (if configured)
  • Email/password auth: Local auth works for admin accounts
  • Chat: Ask a question — the assistant responds with the correct name and personality
  • Search: Upload a test document, then ask a question about it — verify retrieval works
  • Settings: Admin can access and modify all workspace settings
  • Non-admin access: Regular users see the chat interface (not admin settings)
  • Health check: GET /health_check returns 200
  • HTTPS: SSL certificate is valid and all traffic is encrypted
  • Slack (if configured): #client_workflow_runs received the smoke "Run check now" message pair stamped with this deployment's name
  • Auto-rebuild schedule: aws scheduler get-schedule --name "campuscore-<tenant-env>-auto-rebuild" returns a schedule

Troubleshooting

Deployment fails at "Configure AWS (admin) with OIDC"

  • Verify ADMIN_AWS_ROLE_ARN repo-level secret is correct
  • Check that the admin role's OIDC trust policy allows the correct repo

Deployment fails at "Configure AWS (client) for ECR"

  • Verify AWS_ROLE_ARN variable is correct in the GitHub Environment
  • Check that the client deploy role trusts our admin role ARN

ECS tasks keep crashing

  • Check CloudWatch Logs: /ecs/campuscore-{client}
  • Common issues: missing environment variables, database connection failure
  • Verify all secrets are set in the GitHub Environment

Setup wizard doesn't appear

  • The unconfigured AppConfig row is created lazily on first read (apps/main_app/models/config.py - get_active_config), so a missing row is never the cause
  • Verify the AppConfig has is_configured=False
  • Verify the logged-in account is the bootstrap superuser - anyone else sees the "setup pending" page, not the wizard

SSO not working

  • Check the SSO Provider in Django Admin (/admin/campuscore_auth/ssoprovider/) — ensure it's enabled
  • Verify the IdP's ACS URL matches {domain}/accounts/saml/{client_id}/acs/
  • Check that the X.509 certificate is valid PEM format
  • Review Django logs for SAML assertion errors

Database migration errors

  • The ECS entrypoint runs python manage.py run_boot_sequence, which applies migrations under a cross-container advisory lock
  • When several containers start at once, exactly one migrates; the others log Another container is running the boot sequence - waiting for its lock... until it finishes. That line during a slow boot means waiting, not stuck
  • Check CloudWatch Logs for migration output
  • A startup failure mentioning pg_advisory_unlock returned false means the container's DB connection was replaced mid-boot and its steps ran unserialized; ECS replaces the container, but investigate the connection drop
  • If stuck, use ECS Exec to run migrations manually with plain python manage.py migrate

Sentry not receiving events after a successful deploy

  • Run through the boot-log check in Step 4g above. If you see Sentry disabled: SENTRY_DSN is empty, the env var didn't reach the container.
  • Most common cause: the SENTRY_DSN secret is set at repo-level instead of environment-level. Re-set with gh secret set SENTRY_DSN --env <tenant-env> ....
  • See Sentry Setup → Troubleshooting for the full diagnosis tree.