Conversation-first delivery and collections¶
This is a proposed architecture for follow-on work. The current UI delivery is tracked in the task queue. A catalog entry, a dependency's capability or a design example is not a working Humagen integration.
Status: partly built. Checked against main on 2026-11-19. The "Current
boundary" column below was corrected on that date. The
corpus model is superseded by RFC 0010 (Corpora and
the workspace graph), which is kept in the humagen-cloud repository.
| Decision | State | Evidence |
|---|---|---|
| Corpus, CorpusRun, CorpusFile, FileVersion, RunFile and Chunk records | Not built, superseded by RFC 0010 | A corpus holds no content (pkg/corpora/corpora.go); no run, file or version records on main |
| Lexical and vector retrieval | Built, in Humagen's own store | pkg/knowledge/vectors.go, pkg/dense (#132). Not through a Saige retrieval adapter |
| Additional collection providers | Built | OneDrive (#284); GCS and S3 in pkg/catalog/objectstorage (#197) |
| Reviewed improvement from production cases | Partly built | The eval candidate review queue (internal/platform/eval_candidates.go). Candidate comparison and rollback: not confirmed |
| Browser workspace; A2UI, A2A and AG-UI adapters; capability-gap response | Not confirmed | No code found on main |
Stay with the work¶
The project is the home object for work, scope and permissions. Within that scope, the conversation is the primary interaction: ask for a result, inspect its evidence, revise it and review publication in the same place. Specialized views earn their place by making a concrete review or management task easier. Library and Operations use the same authoritative resources; they must not introduce a second execution path or a separate set of access rules.
Skills are immutable packages with files. Memory chains are managed file views over scoped records. Personal, Agent, Project and Company identify real owners; folders do not create new permissions. A chat action must preserve its scope and open an editable native draft before any work runs.
| Capability | Current boundary | Follow-on work |
|---|---|---|
| Conversation delivery | Native runs, SSE, shared presentation and immutable deliverables | More registered generators and inline views over existing result versions |
| Skills and Memory | Scoped file browsers over native stores and actions | Additional managed resource types through those same stores and policies |
| Sources | Collections searched by BM25, fused with vectors when an embedder is set (pkg/knowledge, pkg/dense); Drive, Notion, OneDrive, GCS and S3 providers; recorded refresh attempts |
A permission-aware Saige retrieval adapter and additional collection providers |
| Browser access | No embedded Chrome service | An authenticated, isolated browser workspace and phone interaction |
| Agent protocols | The saige/flow runtime with saige definitions, and pkg/call |
Explicit A2UI, A2A and AG-UI adapters with conformance evidence |
| Improvement loop | Native eval suites, an eval candidate review queue and reviewed publication primitives | Reviewed production fixtures, candidate comparison and human-approved release/rollback |
Proposed corpus model¶
Superseded (2026-11-19): RFC 0010 (Corpora and the workspace graph, in the humagen-cloud repository) replaces this model. On main a corpus is a named set of sources and catalog connectors and holds no content (
pkg/corpora/corpora.go). None of the records below exist. The section is kept as the record of the earlier design.
Organize managed knowledge as corpora, corpus runs, files and file-version chunks. A file belongs to the corpus and is attached to each relevant run; refreshing it does not create a second file identity. These are proposed contracts for the next backend delivery, not entities already implemented by the current source store.
| Record | Identity and relationship | Retained facts |
|---|---|---|
Corpus |
Stable tenant-scoped ID | Provider/origin, selected resources, schedule, policy, published manifest and freshness |
CorpusRun |
Attempt ID belonging to one corpus | Trigger, status, checkpoints, provider cursors, enumeration completeness, staged manifest and observed outcome |
CorpusFile |
Stable corpus file ID, unique by provider authority and upstream file ID | Current published version, metadata pointers, deletion state and independent access/revocation revision |
FileVersion |
Immutable version belonging to one corpus file | Provider revision/ETag, metadata and content hashes, captured bytes or reference, extraction revision and access observation at collection |
RunFile |
Run/file membership referencing the exact observed file version | Planned action, observed change, outcome, error and whether bytes were fetched or reused |
Chunk |
Exact file version plus chunker revision and chunk position | Text/hash, offsets and source location |
| Derived embedding or fact | Exact supporting chunks plus pipeline/model revision | Vector or assertion, provenance, index generation and current visibility/tombstone state |
Every record carries tenant confinement. Run membership does not transfer file ownership or permission. Historical versions and a version's saved access observation are evidence of what was collected, never current reader grants. Facts spanning files retain every supporting chunk/source so authorization can be checked before a fact is returned.
Pull, publish and invalidate¶
- Build a bounded pull plan from verified provider IDs, cursors, revisions, ETags and prior hashes. Separate content, metadata, permission and deletion observations. Fetch bytes only when the provider evidence or required pipeline revision makes them necessary; compute content hashes to confirm reuse without substituting hashes for upstream file identity.
- Persist attempt checkpoints and idempotent
RunFileoutcomes. Partial or failed enumeration cannot imply file absence, deletion or a complete new corpus. Resume/retry within explicit item, byte, time and attempt bounds. - Stage file versions, chunks, vectors and graph updates. Atomically publish the complete successful manifest, its provider cursor and index generation. On failure, retain the last-good manifest and its actual freshness; a run's partial progress is not a successful refresh.
- Replace current derived descendants when content or pipeline changes. Tombstone deleted files and remove their chunks, embeddings and facts from current retrieval. Retain immutable historical versions only under the configured retention and access rules.
| Change | Required invalidation |
|---|---|
| Metadata only | Update observed metadata and filters; invalidate metadata-dependent results without automatically re-embedding unchanged text |
| Content/provider revision | Publish the exact new version; replace affected chunks, embeddings, graph facts and result-cache entries |
| Permissions or revocation | Advance authorization state independently of content; deny affected results and invalidate reader/result caches immediately |
| Confirmed deletion | Tombstone the file and derived descendants in the next complete publication; withhold any separately observed revoked access immediately |
| Chunker, extractor, model or retrieval revision | Rebuild affected derived data or query results under a new generation before publication |
Derived cache keys include tenant, corpus, file version, chunker/extractor, model and retrieval revisions. Query caches also bind the query and published index/graph generation. Cached results retain current-reader authorization requirements and are checked again before use; no cache entry can outlive revocation merely because its content hash is unchanged.
Map from the current sources¶
Keep existing Source APIs compatible while adding the corpus contracts behind
the native source/retriever seams. Today's Source configuration maps toward
Corpus, lifecycle attempts toward CorpusRun, and current document IDs and
passages toward stable files and version-bound chunks. The current store has
bounded attempts and atomic last-good copies; it does not yet implement the
full run/file/version/manifest graph above. A live query remains a current-reader
operation and does not silently create a shared corpus publication.
Compose retrieval rather than fork it¶
Corrected (2026-11-19): main pins saige v0.45.1 (
go.mod), andpkg/knowledgenow fuses BM25 with vectors itself (RFC 0005, change 1). The links below stay at v0.41.0, the version this section was written against.
Humagen currently retrieves local passages through its own BM25 store. The pinned Saige v0.41.0 already exposes lexical/vector composition, reciprocal rank fusion, reranking and incremental source synchronization. Reuse those interfaces behind the native retriever; Humagen keeps tenant identity, source configuration, authorization, scheduling and atomic publication. See Saige's retrieval configuration and the ecosystem plan.
The adapter must fix the pipeline to the host's tenant scope, preserve external file identity independently of content deduplication, and authorize each passage before it reaches a result or model context. A project grant, content hash, vector index or graph namespace cannot replace the provider's file check. Stage partial sync results and publish a complete successful collection atomically; a failed refresh must retain the last-good version without serving unauthorized cached content.
Saige's graph interfaces expose fact search and episode provenance. A future Humagen fact must resolve to readable supporting sources at query time. Unattributed facts from protected collections are withheld. Entity merging and cross-document relationships must preserve source authority and deletion lineage. See Saige's graph contract.
Slack has two separate paths¶
Slack's official MCP service uses Streamable HTTP at https://mcp.slack.com/mcp,
a registered app and user authorization. It does not support Dynamic Client
Registration. Humagen's planned live-search adapter must retain those app,
scope and reader checks. See Slack's MCP documentation.
An internal channel collector is a separate proposed connector. Establish its eligibility, approved channels, scopes, retention and intended use before collecting data. Record bounded backfills and checkpoints, honor rate limits, and propagate edits, deletions and membership revocation through documents, vectors and graph facts. A different API method is not a way around policy. Slack distinguishes internal organization apps from third-party distribution; that distinction does not remove authorization obligations. See Slack's internal-app guidance and API terms.
Website retrieval and a browser workspace¶
Start website retrieval with explicit URLs, bounded responses and a guarded HTTP client. Saige's HTTP source accepts an injected client; its default client does not itself establish Humagen's network policy. Compose it with the guarded fetch client and the deployment's authorized network policy. See the HTTP source and safe HTTP client.
Interactive Chrome requires a separate isolated service, authenticated session ownership, controlled navigation and a view usable on a phone. Tailscale can provide private reachability; it does not replace Humagen authentication or provider permissions. The loopback development preview is not that service.
The requested Cerebras knowledge-base article returned HTTP 500 during research. Its body remains unverified, so this design does not attribute an architecture to it.
Missing capabilities stay explicit¶
Define a bounded capability-gap response containing the missing renderer, generator or protocol, its version, the target surface, the reason it is unavailable and a supported text, table or file fallback. The conversation should show the useful fallback and a clear request to build the missing view.
Building a renderer or generator uses the existing child-run path, ordinary tool grants and a reviewed component registration. Generated code or a model's claim cannot register a capability or prove a successful outcome. Protocol adapters must preserve native actor, run, approval and result identities; native SSE alone is not A2UI, A2A or AG-UI conformance. AgentSpec remains the current deployment/runtime boundary. Unifying definitions is still the separate ecosystem step 17.
Human-reviewed improvement¶
A proposed improvement starts with an authorized production case. A person reviews a redacted fixture, a bounded repair run creates a candidate, and evals compare it with the retained baseline. Missing measurements remain inconclusive. Only explicit human publication promotes the candidate, with its evidence and a rollback version retained. Saige provides comparison and regression gates; Humagen owns the review and deployment decision. See Saige's comparison gate.
This loop does not authorize unreviewed model changes, deployments or broader access. Acceptance needs reproducible fixtures, permission regressions, native run evidence, measured candidate behavior and an exercised rollback.