Dependency-direction audit of core subsystems¶
Audited at commit 287da3a (2026-07-19) for issue #12.
Full seam-by-seam evidence lives in work/archive/12-dependency-direction-audit/research.md; this report is the actionable distillation.
Paths are relative to campuscore_app/ unless noted.
How to read this¶
The rubric is the dependency-direction rule in .claude/skills/software_design_philosophy/SKILL.md: the consumer that enforces a behavior owns the contract; implementations depend inward; no cycles.
Findings are named via its two red flags - inverted contract ownership (the contract lives with implementations, so each privately re-decides behavior the consumer relies on) and upward or cyclic dependency - plus information leakage where one design decision is encoded in several places that must agree by hand.
Severity scale:
- High - silent cross-component breakage is actively possible today (the issue-#9 shape: green tests, changed behavior).
- Medium - drift is possible but the blast radius is contained or detection is likely.
- Low - structural debt; no near-term breakage path.
Verdicts:
- issue - a refactor issue is filed and linked; the fix has its own pipeline run.
- trade-off - the shape is deliberate and the rationale is recorded here; no action.
- note - recorded for awareness; action belongs to already-tracked work or none is warranted.
Findings register¶
| # | Finding | Red flag | Severity | Verdict |
|---|---|---|---|---|
| F1 | Ingestion step order encoded twice | Information leakage | Medium | issue #13 |
| F2 | Three independent priority mappings | Information leakage | Medium | issue #13 |
| F3 | Untyped inter-step metadata contract | Inverted contract ownership | High | resolved in #14 |
| F4 | Four independent enqueue-message build sites | Inverted contract ownership | High | issue #15 |
| F5 | Parser error wording selects retry category | Inverted contract ownership | Medium | issue #16 |
| F6 | Residual file-format lists outside the registry | Information leakage | Medium | issue #17 |
| F7 | Extraction prompt hand-duplicates entity schemas | Information leakage | Medium | issue #18 |
| F8 | identity_primary derived twice; sentinel hand-built twice | Information leakage | Low | issue #19 |
| F9 | Credential encryption is call-site discipline | Inverted contract ownership | Medium | issue #20 |
| F10 | Embedding contract in ~6 places; folder similarity in 4 | Information leakage | Medium | issue #21 |
| F11 | Extraction subsystems import upward into ingestion infra | Upward or cyclic dependency | Low | resolved in #22 |
| F12 | Agent tool surface: per-tool presentation/error/metadata shapes | Inverted contract ownership | High | citation half resolved by #10; rest noted |
| F13 | Two call-time cycles broken by lazy imports | Upward or cyclic dependency | Low | trade-off |
| F14 | Scrape run transitions decided in two places | - | Low | trade-off |
| F15 | URL normalization re-applied at ~8 sites | - | Low | note |
| F16 | Attachment isolation lives only in the vector store | Information leakage | Medium | note (Phase D RLS work) |
| F17 | Written-but-unread legacy: is_public columns, "n/a" api_version | - | Low | note |
F1 - ingestion step order encoded twice (Medium, issue #13)¶
Resolved by #13: the order now lives once in services/document_ingestion/steps.py (state machine and runner both derive from it) and the dead PERSISTED member is gone. The text below records the pre-fix state.
The pipeline's step order is declared in STEP_ORDER (services/document_ingestion/state_machine.py:48-59) and again in PIPELINE (services/document_ingestion/pipeline/runner.py:62-82), which must match 1:1 by hand; runner.py:1-9 claims to be the single home while STEP_ORDER independently re-encodes it.
A deprecated PERSISTED step survives in the enum (models/document_ingestion.py:49) belonging to neither.
Risk: adding or reordering a step in one place but not the other breaks idempotency/resume logic (next_step, _is_at_or_past) with no test failing - nothing pins the two declarations equal.
Suggested shape: one declaration owns the order (the runner, per its own docstring) and the state machine derives from it - or a contract test pins them identical and the dead enum member is dropped.
F2 - three independent priority mappings (Medium, issue #13)¶
Resolved by #13: queue.PRIORITY_BY_RUN_KIND is now the single mapping, every publisher routes through the dispatcher, and the legacy QueueService publish path is gone. The text below records the pre-fix state.
Source/kind-to-priority is decided in three unrelated maps: trigger._priority_for_kind (trigger.py:313-329), queue.SOURCE_PRIORITY + priority_for (queue.py:43-66), and trigger.lazy_create_for_legacy_message (trigger.py:352-356; since removed in the legacy-drain cleanup).
The production write path uses only the first (trigger.py:293-304), leaving the second as a fallback branch and the third for legacy messages.
Risk: a new source kind gets interactive-vs-bulk routing from whichever map the author found first; the other maps silently disagree.
Suggested shape: one mapping owned by the queue layer, consumed by trigger and worker alike.
F3 - untyped inter-step metadata contract (High, issue #14)¶
StepContext is typed (pipeline/context.py:36-80) except for its most load-bearing field: ctx.extracted_metadata is a mutable dict whose keys extract_content stamps privately (pipeline/extract.py:104-126 - original_filename, source_url, source_type, conversation_id, ...) and four later steps read (chunk_and_embed.py:55-56,149-150, persist.py:57-59, entity_extract.py:80-88, finalize.py:49).
The orchestrator cannot see this contract; a renamed or unstamped key changes downstream behavior (summary chunk creation, attachment skip logic, entity provenance) with green tests.
This is the ingestion-side analog of the #9 citation regression.
Suggested shape: promote the load-bearing keys to typed StepContext fields (or a typed model on the context), producer-stamped and consumer-read, so the type checker owns the seam.
F4 - four independent enqueue-message build sites (High, issue #15)¶
DocumentProcessingMessage is hand-built at four sites, each choosing its own keys: fresh crawl (management/commands/scrape_webpages.py:923-932), unprocessed-file re-enqueue (:751-762), first-run-calibration deferred batch (:389-402), and reprocess (management/commands/process_content.py:250-258).
The registrar path omits source_type, silently inheriting the knowledge_folder default (services/folder_file_registrar.py:153-162, default at services/queue_service.py:33) - connector imports ride this too.
The calibration path names files from the storage-key basename instead of derive_scraped_display_name (scrape_webpages.py:389-402), so first-run-calibrated files get file_<id>.pdf-style display names.
The scraped URL is stored under SourceFile.metadata["url"] (scrape_webpages.py:868) while the pipeline stamps chunks with source_url (extract.py:113) and readers check both (scrape_webpages.py:397).
Risk: this seam already produced the synthetic-display-name bug fixed on the pilot branch; each new ingest source re-invents the message and inherits a different subset of the conventions.
Suggested shape: the trigger owns one message factory per source kind; source_type is explicit everywhere; one canonical URL metadata key; the calibration path uses the shared display-name helper.
F5 - parser error wording selects retry category (Medium, issue #16)¶
Leaf parsers raise bare exceptions (parsers/image/gemini_ocr.py:137-139, parsers/pdf/image_based_pdf.py:206-208, parsers/office/python_docx.py:70-77), and the runner categorizes them by string heuristic: "parse" in msg maps to SchemaInvalid (terminate), anything else to Transient (retry) (pipeline/runner.py:179-198).
Only the LibreOffice converter and the dispatch boundary raise categorized errors (converters/libreoffice.py:71-114, content_extractor.py:466-469).
Risk: rewording an exception message inside a parser flips a failure between retry-forever and terminate-to-inbox - retry policy decided by prose.
Suggested shape: the parser contract (content_extraction/interfaces.py:7) requires categorized exceptions; the runner heuristic becomes a guarded last-resort fallback.
F6 - residual file-format lists outside the registry (Medium, issue #17)¶
SUPPORTED_FORMATS (utils/content_types.py:93-192) is the consolidated source of truth and most lists derive from it, but three surfaces still re-encode the decision by hand: the SPA's upload allowlist (web/src/lib/upload/uploader.ts:40 - comment claims mirroring, no mechanical link, no pinning test), the per-parser _check_extension sets (six parsers, e.g. parsers/office/python_docx.py:19-21 - dormant, dispatch never consults them, nothing pins them), and the hand-maintained MIME_TO_CONTENT_TYPE axis (content_types.py:576).
Risk: adding a format updates the registry and quietly leaves the SPA rejecting the upload, or vice versa - the exact drift class the registry was built to end.
Suggested shape: generate or test-pin the SPA list from the server registry; pin or delete _check_extension; derive or pin the MIME map.
F7 - extraction prompt hand-duplicates entity schemas (Medium, issue #18)¶
The entity-extraction prompt writes all 13 entity type shapes as free prose (prompts/scraper/entity_extraction.py:24-184) that must agree with the Pydantic union in schemas/entities.py by discipline; the only programmatic link is one EntityRef import (:20), and no test pins the prose to the schema.
Risk: a schema field change (add, rename, requiredness) leaves the prompt teaching the model the old shape; extraction quality degrades with no failing test - drift is invisible until entities go missing.
Suggested shape: render the per-type field rules from the schemas (the @@SENTINEL@@ single-sourced prompt-constants pattern used by the agent prompts is the in-repo reference).
F8 - identity_primary derived twice; sentinel hand-built twice (Low, issue #19)¶
Resolved by #19: format_identity_primary and build_generic_page_batch in schemas/entities.py are now the single construction sites; the writer, the reconciler, and both sentinel producers consume them. The text below records the pre-fix state.
The writer computes identity_primary = f"{typed.type}:{key.primary}" (services/documents/writer.py:289-290) and the reconciler re-derives the same format independently (services/entity_reconciliation/reconciler.py:469).
The GenericPage "no confident entity" sentinel is hand-constructed at two producer sites with duplicated field choices (services/entity_extraction/entity_extractor.py:227-250; pipeline/entity_extract.py:96-107 - both summary=markdown[:600], title="").
Risk: a format or field change in one site diverges dedup/reconciliation keys or sentinel semantics.
Suggested shape: one shared derivation helper; one sentinel factory.
F9 - credential encryption is call-site discipline (Medium, issue #20)¶
Connection.credentials is a plain TextField (apps/connectors/models.py:161); encryption-on-store happens independently at each boundary - OAuth callback (apps/connectors/views.py:62-68), manual-connection API (apis/connectors_api.py:239,264), token refresh (apps/connectors/services/token_updater.py:28) - while decrypt is centralized (connectors/base.py:186).
Risk: the next write path (a new auth flow, an admin tool, a migration) can persist plaintext credentials without any contract violation surfacing; this is SOC2-adjacent.
Suggested shape: encryption guaranteed at the model boundary (encrypting descriptor or a save() invariant), so a plaintext write is impossible rather than merely avoided.
F10 - embedding contract in ~6 places; folder similarity in 4 (Medium, issue #21)¶
The embedding model/dimension decision has a canonical home (clients/openai.py:14, settings.py:492-493) but is re-declared at pipeline/chunk_and_embed.py:36-37, documents/writer.py:97, vector_stores/vector_index_registry.py:73-74, and three model fields' dimensions=1536 (models/vector_index.py:41, document.py:147, knowledge_folder.py:87).
Folder-routing similarity over KnowledgeFolder.description_embedding is implemented four separate ways: ORM CosineDistance (services/agent/middleware/knowledge_routing.py:59; services/agent/tools/search_knowledge.py:226) and pure-Python cosine (services/agent/tools/discover_knowledge_sources.py:102-136; services/observability/folder_discoverability.py:84-115).
Relatedly, BaseVectorStore declares less than the concrete store offers - scope kwargs and search_hybrid exist only on DocumentChunkVectorStore (document_chunk_store.py:263-273), so RetrievalService bridges by signature introspection (retrieval_service.py:95-153).
Risk: an embedding-model migration must find every re-declaration; the four similarity implementations can rank folders differently for the same query; the introspection bridge means the abstract contract no longer describes reality.
Suggested shape: one embedding constant consumed everywhere; folder similarity in one module; the ABC reconciled with the concrete surface.
Resolution: #21 shipped all three consolidations - EMBEDDING_MODEL_ID + EMBEDDING_DIMENSIONS in clients/openai.py are the contract's single home (the DOCUMENT_EMBEDDING_DIMENSION setting is gone), folder similarity lives only in services/folder_routing.score_folders (the never-wired KnowledgeRoutingMiddleware and the matrix's pure-Python cosine were deleted), and the single-implementation BaseVectorStore ABC was collapsed so RetrievalService calls DocumentChunkVectorStore directly with no introspection.
F11 - extraction subsystems import upward into ingestion infra (Low, issue #22)¶
content_extraction and entity_extraction import document_ingestion's infra modules at module level (content_extractor.py:54-66, entity_extractor.py:46-53, plus every PDF parser and the chunker importing span_tracker), while ingestion's pipeline steps import the extractors downward (pipeline/extract.py:39-43, entity_extract.py:28) - a mutual package dependency avoided at import time only because the downward edges are lazy.
Risk: change amplification - the extraction subsystems cannot be understood, tested, or reused without the ingestion package, and vice versa.
Suggested shape: move the shared kernel (cost_tracker, rate_budget, rate_budget_singleton, span_tracker, exceptions) to a neutral package both depend on, so all arrows point one way.
Resolution: #22 shipped the extraction into two neutral packages - services/llm_infra/ (cost, cost_tracker, rate_budget, rate_budget_singleton, span_tracker) and services/error_handling/ (the categorized exceptions plus ErrorKind, which moved out of step_result, ending that pair's lazy cycle).
content_extraction and entity_extraction now import nothing from document_ingestion; both new packages are leaves relative to the three subsystems.
F12 - agent tool surface: per-tool presentation/error/metadata shapes (High, note - owned by #10)¶
Each tool privately decides what the model sees (format_for_llm overrides at services/agent/tools/search.py:303-331, search_knowledge.py:534-588; JSON-dump default at tools/base.py:189), its own error shape, and its own metadata keys the consumer reads back (agent_core.py:1270-1277); builder wiring is hasattr duck-typing (agent_core.py:1508-1621); the file-before-web citation precedence is duplicated between agent_core.py:413-438 and agent_state.py:319-334 with a keep-in-sync comment.
This is the seam that produced the #9 citation regression, and its structural fix - a typed evidence contract owned by the agent core with one render boundary - is already tracked as issue #10, with the broader surface redesign in work/archive/ENTITY_SOURCE_MODEL_AND_TOOL_SURFACE_SPEC.md.
No new issue is filed; this register row is the cross-link, and the per-component section below is written to serve as #10's research input.
Resolution: #10 shipped the citation half - typed Citation contract in services/agent/citations.py, required cites_sources declaration on BaseTool, single stamp+render boundary in AgentCore._render_observation, and the duplicated precedence collapsed into citations.source_key; the non-citation shapes (error/metadata/builder duck-typing) remain per-tool.
F13 - two call-time cycles broken by lazy imports (Low, trade-off)¶
The runner imports the worker command's _step_result_from_tracker in-function (runner.py:161-163,248-250) while the worker imports run_pipeline at module load (run_document_worker.py:29); pipeline/persist.py:29 imports DocumentWriter while the writer imports pipeline.context/pipeline.embedding_input (writer.py:88,101), broken by the PEP-562 lazy __getattr__ (pipeline/__init__.py:31-36).
Accepted: both are deliberate, commented in code, and confined to call-time; the cost is that the modules cannot be layered cleanly.
Issue #22's shared-kernel extraction removed the third lazy cycle this register once grouped here (exceptions <-> step_result); these two remain as accepted trade-offs.
F14 - scrape run transitions decided in two places (Low, trade-off)¶
The reconciler owns crash/stranded detection (services/scraping/run_monitor.py:88-171) while the scrape command owns its own clean-finish and SIGTERM transitions (scrape_webpages.py:243-285); every transition on both sides is a racer-safe conditional update gated on active status.
Accepted: the owner-plus-janitor split is intentional - the command reports its own outcome when alive, the reconciler rules only on provably-dead runs.
F15 - URL normalization re-applied at ~8 sites (Low, note)¶
normalize_url is applied independently in the scrape command (scrape_webpages.py:512-659) and inside every ScrapingProcess method (services/scraping/process.py:202,273,314,321-328).
The function is idempotent and single-sourced (scrapers/utils/url_normalizer.py:19-119); the repetition is defensive, not divergent.
F16 - attachment isolation lives only in the vector store (Medium, note - Phase D RLS work)¶
Conversation-attachment isolation is enforced solely inside the store (document_chunk_store.py:208-257, keyed on chunk metadata fields) - it is not part of KnowledgePolicy (services/access/policy.py:116-155) and not mirrored in the dormant RLS spike (services/access/rls.py:54-93).
A read path that bypasses the store (as apis/documents_api.py:61 legitimately does via the policy adapter) gets campus/folder scoping but not attachment isolation unless it re-implements it.
Flagged to the access-control/RLS (Phase D) work rather than a new issue: any move toward a provable floor must absorb this rule into the policy, not leave it store-private.
F17 - written-but-unread legacy (Low, note)¶
is_public columns survive as written-but-unread (models/vector_index.py:69, written at writer.py:247,263; the policy filters on denormalized scope columns instead, explicitly per policy.py:11), kept transitionally in sync (knowledge_folder.py:146-152).
api_version="n/a" satisfies the connector conformance check, which asserts only non-emptiness (postgres/connector.py:38-39, test_connector_contract.py:84-90).
Compliant seams catalog¶
These are the in-repo reference shapes - build new seams like these.
- Connector plugin registry (
apps/connectors/connectors/base.py:28,registry.py:22-110) - the contract defines what a connector is; auto-discovery registers implementations; broken imports are quarantined, not fatal; an exhaustive grep confirms zero core imports of any concrete connector. The conformance harness parametrizes over the live registry (apps/connectors/tests/test_connector_contract.py:56-244), so every future connector is tested by existing tests - the ideal answer to "what stops the next implementation from violating it?". - Crawl launcher (
services/scraping/launcher.py:58-285) - consumer-owned interface, both implementations, and the environment-selecting factory in one module; consumers import only the factory. Its handle type is theLaunchHandlethe model persists, imported fromapps.main_app.models, so the launcher and the row it is launched from cannot disagree about what a handle is. - FolderFileRegistrar (
services/folder_file_registrar.py:40-163) - one seam owning SourceFile creation + enqueue for both browser and connector paths, with one access gate and one canonical metadata shape. - Citation contract (
services/agent/citations.py:22-70) - a leaf module three distinct consumers (stamp, parse, strip) import; it imports nothing from the subsystem; pinned by registry-anchored tests (tests/test_source_attribution.py). - Access policy (
services/access/policy.py:116-155) - one ORM authority for chunk visibility, enforced in the store through a thin adapter (document_chunk_store.py:94-119) and reused verbatim by the API layer (apis/documents_api.py:61) rather than re-implemented. - File-format registry (
utils/content_types.py:93-192) - one frozen-dataclass table from which the extension, parser-routing, and upload allowlists derive, holding parser names as strings so the registry imports nothing from implementations; the string indirection is pinned bytests/test_file_format_registry.py:58. (Issue #17 finishes the remaining edges - see F6.) - Ingestion boundary types (
services/document_ingestion/step_result.py,services/queue_service.py:19-65,services/error_handling/exceptions.py) -StepResult, the wire message, and the categorized-error taxonomy (ErrorKind+ exception classes, one module since #22) are each declared exactly once and consumed by state machine, runner, and worker alike. - Retrieval contract placement (
services/retrieval_service.py:48-61,services/vector_stores/base.py:44-64) -RetrievalServiceandSearchResultlive outside the agent package as a shared contract with multiple consumers importing downward and no cycles; this location is the compliant shape (the finding at this seam is the ABC lag, F10, not the placement).
Per-component context¶
Agent core and retrieval are distinct components: the agent core owns the agent's behavior; retrieval is a capability it consumes through a contract, a replaceable dependency like any other.
Agent core¶
Owns the reasoning loop, middleware policy, event contract, prompt assembly, citation surface, and the tool surface.
The loop's contracts point mostly the right way - the event union is producer-owned and consumed downward by ChatService (services/agent/events.py:187-197; chat_service.py:293-362), the citation module is a leaf (compliant catalog above), and no import cycles exist anywhere in the subsystem.
The structural weakness is concentrated at the tool seam (F12): the contract module tools/base.py is co-located with implementations, and presentation, error shape, per-tool metadata, and builder wiring are all decided per-implementation while the consumer relies on them.
Middleware has no shared base class; ordering and the required set are consumer-enforced inline (agent_core.py:700-735,945-994) - consumer-owned, but by convention rather than by type.
For issue #10: the inventory to start from is F12 plus the citation-chain facts - stamp at agent_state.py:336-350, render per-tool, parse at agent_core.py:1203,1450, filter at _build_sources (agent_core.py:373-495), strip at chat_service.py:336-425 - and the duplicated file-before-web precedence.
The entity/source tool-surface spec (work/archive/ENTITY_SOURCE_MODEL_AND_TOOL_SURFACE_SPEC.md) carries the presentation-contract design; note its Phase A writer-side changes are already merged and its line references into writer.py/persist.py are stale.
Retrieval¶
The capability the agent consumes: RetrievalService orchestrating a vector store behind SearchResult.
Its placement is compliant (catalog above); its finding is contract lag - the ABC under-declares the concrete store's surface and the service bridges by introspection (F10).
The cross-component boundary with the agent core is the tool -> RetrievalService seam: tools inject or lazily build the service (search_knowledge.py:180-186) and consume SearchResult fields plus implicit metadata keys.
The load-bearing metadata chain - extract.py:113 stamps source_url, search_knowledge.py:508-514 reads it into hit dicts, agent_core.py:407 turns it into Source.url - crosses ingestion, retrieval, and agent core; it is the concrete thread issue #10's typed evidence contract should formalize.
Document ingestion¶
The state machine and its boundary types are the strongest contracts in the codebase (compliant catalog), and step implementations import nothing upward.
The findings cluster where the orchestrator's knowledge is re-encoded or bypassed: the twice-declared step order (F1), the three priority maps (F2), the untyped inter-step dict (F3), and the four message build sites (F4).
The two call-time cycles (F13) are accepted.
Note the two distinct step contracts - the state machine's StepCallable(item) (state_machine.py:98) and the pipeline's StepFn(ctx) (runner.py:49) - are reconciled only inside the runner's _wrap adapter (runner.py:153-176).
Content extraction¶
The parser protocol and dispatch are registry-shaped and clean (interfaces.py:7; two-layer table dispatch, no if/elif chains); the file-format registry consolidation largely shipped, superseding work/archive/FILE_FORMAT_SUPPORT_SPEC.md (its line references are stale).
Findings: bare-exception error categorization by string heuristic (F5), the residual hand-maintained lists (F6), and the upward infra imports (F11).
A soft coupling to watch: office parsers emit ## headings engineered against the chunker's separators (spreadsheet.py:82-84; markdown_chunker.py:109) with no shared constant.
Scraping¶
The launcher is a reference shape (catalog); config, URL utilities, and display-name derivation are shared leaf modules.
The findings live at the handoff into ingestion: the four message build sites and the metadata["url"]-vs-source_url key split (F4).
Run-lifecycle authority is deliberately split (F14).
Connectors¶
The registry is the codebase's best example of the rule (catalog) - contract-owned dispatch, middleware, introspection; per-connector freedom confined to auth kind, API version pinning, and capability opt-ins (s3_bucket/connector.py:175).
Two soft spots: no shared error taxonomy in the contract (the consumer imposes one after the fact, cloud_file_import/service.py:55,248-267; resolved by #65 - apps/connectors/errors.py declares the taxonomy and the action boundary translates) and encryption-on-store enforced per call site (F9).
The ingestion handoff is compliant: connectors produce bytes/metadata; main_app owns the SourceFile/message contract via the registrar.
Entity extraction¶
The identity contract is schema-owned and consumed type-agnostically by the writer (schemas/entities.py:243-506; writer.py:289-290) - the right direction.
Findings are duplication at the edges: the prompt's hand-written type rules (F7), the second identity_primary derivation in the reconciler and the twice-built GenericPage sentinel (F8), and the upward infra imports shared with content extraction (F11).
The agent's read side consumes Django models plus the shared normalizer and never touches the extraction schemas - the write and read contracts are fully decoupled.
The entity/source spec's Phase A writer changes are merged; a lone-GenericPage batch already writes zero rows (writer.py:130).
Vector stores¶
Naming correction: the store is DocumentChunkVectorStore (services/vector_stores/document_chunk_store.py:142); DocumentIndexVectorStore does not exist (stale references corrected in this PR).
The store owns the DocumentChunk.embedding search path, HNSW tuning, and the access-control enforcement point; SearchResult is the single cross-layer DTO.
Findings: the re-declared embedding contract and the four folder-similarity implementations outside the store (F10), attachment isolation living only here (F16), and the transitional is_public legacy (F17).
The RLS spike (services/access/rls.py) is dormant - referenced only by its test - and mirrors KnowledgePolicy but not the store-private attachment rule.
Staleness appendix¶
Corrected in this PR:
CLAUDE.mdrepository layout listedapps/document_extraction/; parsing and chunking live underapps/main_app/services/content_extraction/(plusservices/document_ingestion/for the pipeline).CLAUDE.mdand.claude/skills/software_design_philosophy/SKILL.mdnamedDocumentIndexVectorStore; the class isDocumentChunkVectorStore.docs/project/document-ingestion-state-machine.mdshowed a stale step list (PERSISTED, wrong parallel group) and claimed the worker collapses the sequence into a singleENQUEUED -> DONEtransition; the code drives per-step transitions (state_machine.py:48-59,runner.py:62-82).
Recorded, not changed here:
work/archive/FILE_FORMAT_SUPPORT_SPEC.mdandwork/archive/ENTITY_SOURCE_MODEL_AND_TOOL_SURFACE_SPEC.mdboth describe pre-implementation states; their consolidations largely shipped and their file:line references are stale - read them for intent, verify against code.docs/project/agent-retrieval.mdalready carries its own staleness note pending the tool-surface redesign.