Skip to content

Parallel Dev Stacks

Run several fully isolated dev stacks at once - one per git worktree - so parallel work items never collide on containers, ports, databases, workers, or browser cookies. Every work item gets its own worktree regardless of whether it brings up a stack - that rule lives in work/README.md; this doc owns the stack side. scripts/stack.sh is the front door; the main checkout keeps working exactly as before (docker compose up, containers named campuscore_*, ports 8000/5173/9000/...).

Quick reference

git worktree add -b <issue#>-<slug> ../campuscore-<slug> main   # a worktree per work item, branched off main
cd ../campuscore-<slug>

bash scripts/stack.sh up --seed    # isolated stack, data cloned from the main stack
bash scripts/stack.sh status       # registry + this stack's containers
bash scripts/stack.sh url          # this stack's base URL (for curl etc.)
bash scripts/stack.sh down         # tear down: containers, volumes, registry entry

up prints the stack's URLs when it finishes:

Surface URL
App (Django + built SPA) http://cc-<slug>.localhost:<base+0>
Vite dev server (live reload) http://cc-<slug>.localhost:<base+1>
Worker health http://localhost:<base+8>/health

How isolation works

Every stack is a separate Docker Compose project. stack.sh up writes a repo-root .env (compose interpolation: COMPOSE_PROJECT_NAME=cc-<slug>, hostname, a port block) and campuscore_app/.stack.env (Django-side ALLOWED_HOSTS / CSRF_TRUSTED_ORIGINS for the stack hostname), then starts the core services: db, s3, s3-init, elasticmq, redis, web, worker, web_ui. s3 is the local S3 server, Versity Gateway, and s3-init is a one-shot run of manage.py prepare_local_upload_bucket that creates the upload bucket and sets its browser-upload CORS rule before web and worker start. Compose project-scopes the network and data volumes automatically, each stack builds and runs its own worktree's code (including its own worker), and docker compose exec -T web ... from the worktree targets that stack - so gate.sh, the pre-commit hooks, and build_spa_local.sh all work unmodified.

A local web container, in the main checkout or a worktree stack, runs manage.py run_boot_sequence as it starts, the same as a cloud container does. That command migrates, creates a superuser from DJANGO_SUPERUSER_EMAIL and DJANGO_SUPERUSER_PASSWORD when none exists, then syncs the role, feature, connector action policy and skill registries, so a fresh stack has its features and roles without a hand-run sync. It runs each time the web container starts and again during each stack.sh up, but not when uvicorn --reload restarts the server after a code change. A failing step stops the web container, and docker compose logs web shows which step failed.

A test run in a stack leaves the stack's queues and upload bucket alone. Every SQS call a test makes lands in an in-memory fake from campuscore_app/conftest.py, so the stack's worker log after a test run shows only real work, and a test that needs the real client opts out with @pytest.mark.real_sqs_client. Tests also read and write a bucket of their own, <FILE_UPLOAD_BUCKET>-tests, on the stack's S3 server.

Hostnames do the browser-side isolation. Chrome resolves *.localhost to 127.0.0.1 natively and cookies are host-only, so a logged-in session on cc-a.localhost never interferes with cc-b.localhost or localhost:8000.

The port block is deterministic: each slug gets an index in <main-checkout>/.stack-registry.json (git-ignored, allocation serialized by a lockdir), and the block is 10000 + 100*index with offsets web +0, vite +1, s3 +2, db +4, redis +5, sqs +6, sqs ui +7, worker health +8, keycloak +9, canvas +10, canvas mail +11. Offset +3 is unused, so every existing stack keeps its other ports.

.stack.env also overrides CAMPUSCORE_SUBDOMAIN (blanked) and CUSTOM_DOMAIN_WITH_PROTOCOL (the stack's own URL), so the SP/SSO URLs the app builds for itself are honest per stack instead of claiming the main .env's domain.

Auxiliary services stay with the main stack only: tailwind, dozzle and the docs sites. Keycloak is per-stack and opt-in: stack.sh up --sso starts this stack's own Keycloak and seeds both SSO providers (local-sso-testing.md); the container then stays with the stack until down. Canvas LMS is likewise per-stack and opt-in: stack.sh up --canvas (implies --sso) starts a real Canvas wired to the stack's Keycloak, seeds personas and coursework, and configures the CampusCore connector against it (local-canvas-testing.md); budget 3-4 GiB of extra Docker RAM and expect a slow first bootstrap on a cold cache.

Seeding

A fresh stack has an empty database - no searchable content, and the setup wizard gates the app. stack.sh seed (or up --seed) clones the main stack's data volumes (pgdata, s3_data) into the stack, carrying schema, embeddings, uploaded files, the configured AppConfig, and the superuser - the stack is immediately logged-in-able and searchable, and up applies the worktree's own migrations on top. The s3 service keeps each object's metadata and the bucket's CORS rule as ordinary files in s3_data, so the byte-for-byte volume copy carries them and the seeded stack needs no bucket setup of its own.

The seed clones from the compose project that SEED_SOURCE_PROJECT names, which defaults to the main checkout's campuscore. Any other worktree stack's project (cc-<slug>) works too, as in SEED_SOURCE_PROJECT=cc-other bash scripts/stack.sh up --seed, and the seed refuses to clone a stack into itself. The seed refuses to run when a source volume is missing or empty, which is what a main checkout still on the pre-PostgreSQL-18 compose file looks like. A main checkout in that state carries its existing data onto the new image with bash scripts/pg-upgrade-local.sh - run that once before seeding, because a bare docker compose up -d would initialise an empty database instead.

Known limitation: a consistent copy of a running Postgres data directory is not possible, so the seed pauses the source stack's db and s3 containers for the few seconds the copy takes. Concurrent seeds serialize through the registry lock.

Caveats

  • A worktree that runs a stack must be brought up through stack.sh. Running plain docker compose up in a worktree that has no generated .env falls back to the default project name and collides with the main stack.
  • The SPA's browser tests run on demand, in the stack they are started from. The browser_tests service sits under the compose tools profile, so stack.sh up never starts it. The gate runs it once per run with docker compose --profile tools run --rm -T browser_tests, in this worktree's compose project, sharing web_ui's node_modules and pnpm store volumes. Its image, mcr.microsoft.com/playwright, is about 2.5 GB; the first gate run on a machine pulls it, and every stack after that reuses it.
  • The SPA bundle is per-worktree. templates/spa/index.html and static/spa/ are git-ignored build artifacts; a fresh worktree serves a 503 "Frontend not built" on the app URL until the bundle is built (up does this unless --no-spa; the Vite URL works regardless).
  • Docker Desktop VM memory is the ceiling. One stack idles around 2 GiB and the db requests 2 GiB of shared memory for HNSW index builds. The default VM allocation (~8 GiB) fits the main stack plus one worktree stack; raise it to ~24 GiB (Docker Desktop -> Settings -> Resources) for main plus 2-3 stacks.
  • Agent permission rules for the repo's own commands are tracked. .claude/settings.json carries allow rules for scripts/gate.sh, scripts/stack.sh, the SPA build, the sync and check scripts, and the container test, type and lint commands, so a session on a machine without local rules still runs them without the auto-mode classifier. The per-machine .claude/settings.local.json adds to that list and already reaches worktree sessions. A rule matches a single command prefix: a compound line with a redirect or a trailing command matches nothing, which is why the gate is run as bash scripts/gate.sh --detach and then bash scripts/gate.sh --wait.
  • Seeds inherit the main stack's embedding state. A clone is a snapshot: if the main stack's corpus or embedding model moves, re-seed (stack.sh seed is idempotent - it replaces the stack's data volumes).

Moving from MinIO

The stack used to run MinIO as its S3 server, and MinIO's images no longer pull, so a machine that ran it has uploads in a minio_data volume that the s3 service cannot read. The move is a one-time step per checkout.

On the main checkout, pull, then run docker compose up -d --remove-orphans so the old minio containers go and s3 starts, and then run bash scripts/migrate_minio_uploads.sh. The script starts a temporary MinIO container on campuscore_minio_data, copies every object in the upload bucket into s3 with its content type, and skips objects already there at the same size, so a rerun resumes an interrupted copy. It refuses before changing anything when the volume is absent, when a running container still mounts it, when the stack's s3 or web is not running, when s3 has no upload bucket, or when the MinIO image is not in the local Docker cache. Without the cached image the old files cannot be read and must be uploaded again; MINIO_IMAGE=<image> points the script at a different cached MinIO release. The old volume is kept, and the script prints the docker volume rm command that deletes it once the files show up in the app.

A worktree stack is disposable, so it is re-seeded rather than migrated. Seed a worktree only after the main checkout has migrated. Seeding earlier clones database rows whose files are missing, and nothing reports an error. After pulling, run bash scripts/stack.sh up --seed in the worktree; its --remove-orphans removes the old minio container, which would otherwise keep the S3 host port. Then run docker volume rm cc-<slug>_minio_data, because a compose file that no longer declares the volume never removes it. If the main checkout still has its minio container, the seed refuses and prints the main checkout's steps above.

web, worker and s3-init run one image, <project>-app, so the <project>-web, <project>-worker and <project>-s3-init images an earlier compose file built are unused; docker image rm deletes them.

After the PR merges

If a stack was brought up, stack.sh down from the worktree removes its containers, network, volumes, registry entry, and generated env files. Then, from the main checkout, git worktree remove <path> deletes the checkout - this happens for every item, stack or not. The post-merge cleanup step in the implement skill runs this sequence.