Agent & Retrieval System

The agentic RAG pipeline is the core of CampusCore's chat experience. A ReAct-style reasoning agent decides what tools to call, observes results, and generates a grounded response. This document covers the agent architecture, retrieval tools, and attachment handling.

Architecture Overview

User message
    │
    ▼
ChatService.generate_response()
    │
    ├── 1. Get/create conversation
    ├── 2. Link file attachments
    ├── 3. Save user message
    ├── 4. Build system prompt (AppConfig + guardrails)
    └── 5. agent.run()
            │
            ├── Build attachment context (files + folders → system prompt)
            ├── Wire tools with user/conversation context
            └── ReAct loop:
                 ├── LLM call → tool calls or text response
                 ├── Execute tools → observe results
                 ├── Token management (prevent context overflow)
                 └── Repeat until done or max iterations

Key Files

File	Role
services/chat/chat_service.py	Orchestrates chat flow, bridges views and agent
services/chat/	The chat turn machinery around it: chat_user.py (the CurrentChatUser boundary value), conversation_access.py (ownership-door conversation access), cross_turn_notes.py (prior searches / connector reads / action decisions re-presented per turn), turn_reporting.py (audit + trace linkage), assistant_turn_recorder.py, conversation_folder_service.py
services/agent/agent_core.py	The ReAct reasoning loop and its factories
services/agent/prompt_context.py	Initial-message assembly: folder resolution, attachment cadence block, the cached/volatile system-message split
services/agent/tool_wiring.py	Per-run context wiring and the per-iteration connector-tool rebuild
services/agent/tool_execution.py	Traced, metered execution of one tool call
services/agent/observation_rendering.py	The observation boundary: citation stamp + render, per-result truncation
services/agent/source_attribution.py	Sources-panel rows from a turn's cited refs
services/agent/search_audit.py	The per-turn search-audit system message
services/agent/lazy_referral.py	Observability-only lazy-referral detection
services/agent/connector_reflection.py	Unused-connector reflection before a response commits
services/agent/connector_previews.py	Capped connector-read previews for cross-turn recall
services/agent/tool_narration.py	Fallback thinking text for tool calls without a model thought
services/agent/tools/search_attachments.py	Search/retrieve user-provided files and folders
services/agent/tools/search_knowledge.py	Search the institutional knowledge base
services/agent/tools/search_knowledge_folder.py	Direct folder search (used when auto-routing is inactive)
services/agent/tools/utilities.py	GetCurrentDateTool
services/agent/tools/connector_tools.py	ConnectorActionTool (external integrations)
services/agent/tools/base.py	BaseTool, ToolRegistry, create_default_registry()
services/agent/tools/presentation.py	Knowledge-layer vocabulary: entity_label, entity_entry_phrase, the entity-count caveat strings
services/agent/tools/describe_knowledge.py	Corpus inventory tool; its format_for_llm renders caveated entity counts
services/agent/middleware/	Token management, tool-result budget, compaction
services/folder_routing.py	The single home for folder-description similarity (used by search routing and the admin matrix)
services/agent/events.py	Streaming event types for real-time UI
prompts/agent_system_prompt.py	Agent system prompt template
prompts/prompt_builder.py	Reads AppConfig, formats the template

Model Configuration

The agent's model is settings.AGENT_MODEL - a typed default in campus_core/settings_envs.py, overridable per environment via the env var. It is the single model variable for the reasoning loop. The registered tool surface is search / find_entity / read_file plus the inventory and connector tools - exactly the set create_default_registry() builds (pinned by the exact-set contract test in tests/test_agent_core.py). The five former "reasoning support" meta-tools were removed after an evidence-gated keep/cull decision (issue #24: the agent never invoked them, and the loop's own prompt-taught behavior covers every shape they claimed). Per-model request behavior (Responses vs chat-completions endpoint, reasoning effort, temperature omission) resolves from the model id in services/agent/model_request_profile.py. Model swaps are gated on the A/B eval harness: run evals/offline/model_ab_eval.py (see evals/README.md) and change the default only on parity or better.

Two Retrieval Tools

Staleness note: this section predates the retrieval consolidation (9960903, 2026-07-14), which replaced the registered tool surface with search / find_entity / read_file (see services/agent/tools/base.py create_default_registry()). The tools below still exist and are composed internally by search, but the agent no longer registers them directly. The full rewrite is tracked in the agent tool-surface redesign work item.

The agent has two separate retrieval tools that map to two fundamentally different concerns:

search_attachments - User-Provided Context

Handles everything the user explicitly brought into the conversation: uploaded files and selected knowledge folders.

Parameters:

Param	Type	Description
query	str	Search query (required)
retrieve_all	bool	When True, return all content in document order instead of semantic search. For summaries/overviews. Default False.
scope	str \| None	Optional filename or folder name to target. Omit to use the current-turn default (see Scoping logic). Pass 'all' to search every attachment in the conversation.
num_results	int	Max results for search mode (1-20, default 10).

Two modes:

• Search mode (retrieve_all=False): Hybrid semantic + full-text search within attachment scope. Uses RetrievalService.retrieve() with conversation and/or folder scoping, then reranks results.
• Retrieve-all mode (retrieve_all=True): Direct ORM query returning all chunks ordered by document grouping + position. No semantic ranking. Capped at 50 chunks (~25k tokens) with a truncation note if exceeded.

Scoping logic: - No scope → search/retrieve across ALL of the conversation's attachments (files + folders). There is no automatic current-turn narrowing: the agent focuses on the just-attached file by naming it, which it knows from the cadence context's "added just now" marker. - scope = filename → filter to that file's chunks. A name that matches nothing returns nothing, never a silent widen. - scope = folder name → filter to that folder's chunks - scope = 'all' → explicitly search every attachment in the conversation (the widen for comparative questions)

Builder methods (called by agent_core.py before each run; access control is the EndUser principal bound at construction): - with_conversation_id(id) - scope to conversation's file attachments - with_folder_ids(ids, names) - scope to the conversation's folders. These are the standing set, every folder ever added to the conversation, not the current message's selection: the composer's folder picker adds to that set, and ChatService reads the whole set back on every turn. A turn that sends no folders still gets it. - with_attachment_filenames(filenames) - for scope name matching

search_knowledge - Institutional Knowledge Base

Searches the broader knowledge pool (website data, public/personal knowledge folders). No attachment awareness.

Parameters:

Param	Type	Description
query	str	Search query (required)
num_results	int	Max results (default 10)

Two execution paths: - With auto-routed folders: on each call, the tool's folder-discovery step scores every embedded folder against the query via services/folder_routing.score_folders (pgvector cosine, threshold 0.50, top 5) → searches the matched folders only → reranks. An empty folder result is reported honestly, never silently broadened. - Broad search: no folder cleared the threshold (or disable_routing=True) → searches the full index.

Builder methods: none - access control is the EndUser principal bound at construction.

When the agent uses which tool

Scenario	Tool	Notes
No attachments, normal question	search_knowledge	Auto-routing scopes to relevant folders
File attached this turn, "what is this about" / "summarize this"	search_attachments(scope="<that file>", retrieve_all=True)	The agent names the just-attached file, known from the cadence context
File attached this turn, specific question	search_attachments(query="...", scope="<that file>")	Same naming; falls back to search_knowledge if insufficient
Follow-up with no new attachment	search_attachments(retrieve_all=True)	No scope → all attachments (whole conversation)
"How does this compare to what I sent earlier?"	search_attachments(scope="all")	Explicit widen to every attachment
"What's in report.pdf?"	search_attachments(scope="report.pdf")	Scoped to a specific named file
Attachment doesn't have answer	search_knowledge	Agent calls as second tool

Attachment Handling

File Attachments

Files are conversation-scoped for storage (not per-message). The ConversationAttachment model tracks each file with a processing status pipeline: uploading → extracting → indexing → ready. Once a file is ready, it stays available for all subsequent messages in that conversation.

An unscoped search_attachments reads the whole conversation's attachments; "what is this about" reaches the file the user just attached because the agent names it, using the cadence context's "added just now" marker (see Scoping logic above). The uploaded_with_message FK is display only, not a scope boundary.

Folder Attachments

Folder selections are ephemeral - sent as query parameters per message, not persisted. The user selects which KnowledgeFolder IDs to scope to for each message.

Recency Context

When files are attached with the current message, the system prompt labels them distinctly:

## User Attachments

### Files
- report.pdf **<-- attached with this message**
- syllabus.docx

### Folders
- Financial Aid (id: 42)

This tells the agent which file is the current-turn subject. Retrieval already defaults to it (Scoping logic above), so generic references ("this", "what is this about", "summarise this") target the just-attached file without the agent naming it. The agent widens only for explicit historical or comparative references, via scope='all' or a named earlier file.

Data flow for recency: 1. Frontend sends attachment_ids (IDs of files uploaded with this message) in the POST /api/chat/stream body 2. ChatAttachmentService.get_filenames_by_ids() resolves IDs → display names, scoped to the turn's conversation - a claimed id the conversation does not own resolves to nothing 3. chat/chat_service.py passes attachment_filenames (all) and new_attachment_filenames (this message) to agent.run() 4. prompt_context.build_attachment_context() labels the current-turn file(s)

Connector Files Loaded into the Chat

A connector action (canvas__load_course_file_into_chat) can pull an upstream file into the conversation. Unlike a hand upload, the processed copy is shared: one SourceFile with access_scope="connector" per upstream file version, linked into conversations by ConversationAttachment rows. The link is the access grant - KnowledgePolicy.chunk_q grants a user the connector-scoped chunks of files they hold a live link to - and the my_context scope includes the chunks of files linked into the current conversation. Corpus searches exclude connector-file chunks exactly like attachment chunks; the shared chunks carry metadata.source_type = "connector_file" plus the connector identity and never any conversation or user key. These files show in the Context panel only, never as message chips. See connectors.md, "Chat-Attachment File Bridge".

Document Indexing for Attachments

File attachments use lightweight metadata (filename as title, first 200 chars as summary) instead of LLM-generated metadata. This speeds up processing from ~12-18s to ~5s per file.

Attachment chunks are isolated via metadata filters: - metadata.source_type = "conversation_attachment" - metadata.conversation_id - scopes to the conversation - metadata.user_id - scopes to the user

Knowledge Routing

Folder routing happens inside the search tool at search time, not before the loop. SearchKnowledgeTool's folder-discovery step (services/agent/tools/search_knowledge.py) calls services/folder_routing.score_folders, one pgvector CosineDistance query over KnowledgeFolder.description_embedding (threshold 0.50, at most 5 folders). The same function feeds the admin Folder Routing matrix (services/observability/folder_discoverability.py), so the dashboard shows the live path's scores by construction. A pre-loop KnowledgeRoutingMiddleware was built but never wired into the loop and has been deleted.

System Prompt Design

The system prompt uses a two-message pattern for caching efficiency:

1. Static system message - the agent prompt template (cached across requests, ~5k tokens saved on cache hits)
2. Dynamic system message - per-request context:
3. User identity (## User Identity: preferred name, role names, department - signed-in only)
4. Currently Connected Services and the Knowledge Inventory (signed-in only)
5. Attachment context (files, folders, recency labels)
6. Prior-turn search summary
7. Today's date

Audience-aware assembly

The static template is composed per (stage, audience) pair by _compose in prompts/agent_system_prompt.py: @@AUDIENCE_MISSION@@ and @@AUDIENCE_SCOPE@@ slots select member or guest constants, and every other section stays byte-identical across audiences. Members get the general-assistant stance - ## General Assistance & Academic Work makes general help first-class, states the grounding hard rule (an institution-specific fact is never answered from model memory), and carries the coursework teach-don't-do stance. Guests get ## Scope of Help: institution questions only, a one-sentence decline for everything else, and a sign-in pointer. The max-iterations fallback template has the same audience treatment (@@AUDIENCE_FALLBACK_SCOPE@@), because the fallback swap replaces the first system message - exactly where a guest's scope restriction lives. build_agent_system_prompt(stage, audience) in prompts/prompt_builder.py resolves all four combinations and validates a per-audience required-sections list.

User identity in the dynamic context

build_user_identity_section(user, role_slugs) (prompts/prompt_builder.py) renders the ## User Identity block from display_name(user) (apps/auth/services/identity_service.py - preferred name, official name fallback), Role.display_name values resolved from the principal's role slugs, and UserProfile.department. Absent fields are omitted; a user with nothing known renders no section at all. The block deliberately excludes student_id, external_id, and every other FERPA-adjacent identifier - the dynamic context is stored verbatim in the trace store (see security-and-compliance.md) - and a contract test pins the exclusion.

Entity counts are index statistics

Entity counts describe what extraction indexed - duplicates and cross-listings included - never how many courses, programs, or anything else the institution offers. The contract is enforced at the data layer first. The one caveat string lives in services/agent/tools/presentation.py as ENTITY_COUNT_CAVEAT, and both surfaces that render corpus counts attach it to the same text as the numbers - the Knowledge Inventory block in prompts/prompt_builder.py and describe_knowledge's format_for_llm - so the model never receives a bare "2630 courses" it could quote. Result-set summaries (find_entity, list_entities) use the same "N course entries" phrasing so a page of results cannot read as a total either. The prompt half is _ENTITY_COUNT_RULE in prompts/agent_system_prompt.py, a stage-neutral constant spliced into both the reasoning-loop and fallback templates. The routing table splits "how many do YOU have indexed" (describe_knowledge) from "how many does the university OFFER" - the latter answered from an authoritative source: a connected service whose capability covers it (a student information system, a registrar API) or a published statement found via search.

Tool descriptions self-describe; the prompt routes

A tool's description states what the tool does, its inputs, and the properties of its output - including caveats about its own data - and nothing about sibling tools. Cross-tool routing ("for X use Y", "call A before B") lives only in the system prompt's Routing Framework, Tool Catalogue, and question-shape table, so routing policy has one home and tool descriptions stay true wherever the tool is mounted (default registry, guest registry, future surfaces). An institutional total is answered from a published figure with a citation, or the agent says the indexed sources do not state one and offers to narrow - the count-trap probes in evals/data/reasoning_probe_questions.json exercise exactly this.

Streaming Events

The agent emits typed events during execution for real-time UI updates:

Event	Purpose
ThinkingEvent	Agent reasoning step indicator
ToolCallEvent	Tool invocation with name and arguments
ToolResultEvent	Tool result (success/failure); its data_preview never streams
ActionConfirmationEvent	A connector write held for the user's approve/deny decision
TextEvent	Response text (delta or full)
SourcesEvent	The sources the finished answer cited (emitted once, only when non-empty)
ConversationEvent	The conversation created or resumed for this turn, so the client can address it
MessageSavedEvent	A message row was persisted, with its id and role
DoneEvent	Agent completed with summary
ErrorEvent	The turn failed; carries the user-safe message

Events are streamed as SSE-formatted strings through chat/chat_service.py to the frontend, where web/src/lib/streaming/useChatStream.ts processes them.

The persisted trace

The thinking, tool call, tool result and action confirmation events of a turn, with the DoneEvent counts and the cited sources, are stored on the assistant row as Message.agent_metadata. The stored shape has one home, PersistedTurnTrace in services/chat/turn_trace.py, built from the SSE classes above so the wire and the stored forms cannot drift; the one difference is StoredToolResult, which keeps the data_preview the wire excludes because cross_turn_notes reads it back on the next turn. Every reader - the transcript API, cross_turn_notes, analytics_rollup and the guest API - parses the row through parse_stored_trace, which raises StoredTraceInvalid naming only the row id, never the row; the Django admin's read-only field is the one raw viewer, for a superuser. A row the model rejects is a bug to hear about, not a row to tolerate: the signed-in and guest transcripts answer the generic 500 with an error-level log line naming the row, the next turn of that conversation fails before the agent runs, and the nightly rollup aborts that day and names the row in its failure notice. The transcript returns TranscriptTrace, the same fields over the plain SSE classes, so data_preview stays off the API exactly as it stays off the stream. Migration 0145 parses every stored row and refuses the deploy while one does not fit; python manage.py check_agent_traces runs the same audit on demand and prints the repair.

The error path

LLM failures raise at their raise sites; nothing in the LLM layer converts a failure into a normal-looking end-of-stream event. A provider exception propagates out of LLMWithTools.generate_with_tools on both endpoint paths, and the Responses adapter (services/agent/responses_adapter.py) raises LLMStreamError for the failures the API delivers as data: a response.failed event, or a stream that ends having produced no content. The one deliberate exception is partial output: response.incomplete and a stream that dies after producing content both complete with what streamed, because a partial answer beats an error message. The backstop is in the loop: a turn that finishes with neither text nor tool calls raises in AgentCore._handle_finish_turn - the single enforcement point of the no-silent-empty-completion invariant, whatever produced the empty turn.

The raise lands in AgentCore.run's outer handler (_emit_run_error), which emits a sanitized ErrorEvent (services/error_handling/sanitize.py - two user-safe strings, technical detail only in logs) followed by DoneEvent(status="error") carrying the run's trace_id. ChatService.generate_response's own boundary emits the same two frames for failures outside the agent (the turn timeout, a permission error), but only when the agent never sent its own done - a trailing error pair after a delivered answer would repaint it as failed. The SPA (web/src/lib/streaming/useChatStream.ts) holds an errored turn on screen: done with status="error" moves the turn to its failed phase, which keeps the live turn and the red alert mounted instead of firing the refetch-and-reset that follows a completed turn, mirroring how a stopped turn behaves. When the failed question was saved before the failure, the banner offers "Try again"; the server announces the saved row's id on every turn, re-runs included, so a failed retry keeps the offer. The button replays that message through the edit/regenerate path, so the transcript never gains a duplicate question. It hides while a turn is in flight and when there is nothing saved to replay - a failure before the question was persisted leaves retyping as the only recovery.

Source citations ([S#])

The Sources strip under an answer lists only the sources the answer actually cited. The contract lives in one module - services/agent/citations.py - and is enforced at one boundary; the chain is:

1. Declare - every tool class sets cites_sources: bool, a required BaseTool field (a class that does not declare cannot be instantiated, so the registry fails to build). A citing tool (search, find_entity, read_file) returns its citable content as typed Citation objects on ToolResult.citations; the SourceIdentity union (file / web / uncitable) makes the source identity - or the explicit opt-out - required by construction. citations.source_key is the single home of the file-before-web decision: a FileIdentity that also carries a url is still a file.
2. Stamp + render - the loop's observation boundary (services/agent/observation_rendering.py) is the single attribution point. It stamps each citation's turn-local ref (AgentState.add_citations, key file:<id> / web:<url>, stable for the whole turn), renders the blocks through citations.render_citations (the only Ref:-rendering code path: Ref:/Title:/Content: plus the tool's extras lines), composes them with the tool's own format_for_llm text, and truncates the composite. A tool that returns citations while declaring cites_sources=False aborts the turn with an error - a developer mistake, surfaced loudly.
3. Cite - the system prompt's citation rule (prompts/agent_system_prompt.py, _SOURCE_CITATION_RULE) tells the model to tag retrieved facts with the ref and never to invent one.
4. Filter - after the answer, extract_cited_refs(answer) + build_sources(state, cited_refs) (services/agent/source_attribution.py) keep only the cited citations from state.citations and resolve them to Source rows; a SourcesEvent is emitted only when the result is non-empty. A cited ref that resolves to nothing emits the agent.sources.unresolved_citation metric and is dropped - never back-filled.
5. Persist/strip - chat/chat_service.py strips the [S#] markers from every persisted write (strip_citation_markers) and stores the sources as PersistedTurnTrace.sources (services/chat/turn_trace.py), which the transcript returns as TranscriptTrace.sources for history rendering and the panel. The live SSE text keeps the markers; the SPA strips them at render (web/src/lib/render/citations.ts, the lockstep mirror of services/agent/citations.py).

What each tool cites: search attaches one citation per hit; find_entity attaches one citation per backing source file on its full-payload modes only (candidate and compact-browse rows are navigation, not citable facts); read_file attaches one citation carrying the returned text.

A new citing tool needs zero wiring beyond the declaration and its Citation list - that contract is pinned by tests/test_source_attribution.py (TestObservationBoundary, which drives a never-registered fake tool through the loop's observation boundary; TestSearchObservationByteStability pins the exact block bytes; registry construction proves every class declares). The persistence wiring is pinned by TestCitedTurnPersistsSources.