Native AI collaboration and Conversation Brain delivery

This document records the complete native desktop change set developed after commit d9dab9f. It is the release and review map for five connected features:

1. authenticated local and remote Codex App Server control;
2. a native human-collaboration room with cooperative controller leases;
3. Native Chat and Codex message actions, branching, speech, and response details;
4. offline emoji, reactions, and contextual transcript tail navigation; and
5. the Human-AI Conversation Brain.

All application UI remains original Rust/GPUI. The implementation does not embed a browser, import the reference React/Bun/OpenCode frontends, intercept a private Codex Desktop transport, provision public TLS infrastructure, or claim access to an AI model's hidden reasoning or literal internal mental state.

Architecture at a glance

The Codex, collaboration, Native Chat, and Conversation Brain state machines remain separate. They communicate only through explicit, typed commands and observable lifecycle events.

Delivered changes

Dual-server Codex transport and identity

The owned Codex App Server now uses an authenticated loopback WebSocket rather than stdio. EXEPERT can keep that local server and one configured remote server connected concurrently.

• CodexServerId and CodexTaskKey qualify all tasks, commands, pending requests, transcript projections, approvals, mirrors, diagnostics, and UI selection so identical thread IDs cannot collide across servers.
• The owned process binds ws://127.0.0.1:0, publishes only a SHA-256 token verifier in its argument list, passes readiness checks, and remains governed by the existing Windows child-process lifecycle.
• Remote profiles require trusted wss://; plaintext non-loopback endpoints and certificate bypasses fail closed.
• The remote initialize response must identify the exact supported Codex Desktop 0.149.1 protocol peer.
• Transient failures use bounded reconnect generations and backoff. Pending writes are discarded, persisted projections remain visible, and controlled remote tasks return to mirror mode. Authentication, TLS, schema, and version failures stop automatic retry.
• Remote task creation accepts a manually entered remote absolute working directory. Controller-local attachments and file-opening actions remain unavailable for remote tasks.
• The undocumented remoteControl/* protocol surface remains disabled.

Settings, credentials, and controller grants

Codex settings migrate deterministically to version 2. Existing task IDs become LOCAL task keys. The JSON store contains stable profile metadata, selected server state, task keys, and remote default paths, but no capability secrets.

Windows Credential Manager stores one stable local controller token and one token for the remote profile. Token debug/display/error implementations are redacted. The local child receives only the token hash verifier in argv.

Controller setup copy, rotation, import, and remote-forget operations are explicit. A controller grant uses the separate exepert-codex-v1:<base64url-json> envelope. It is never confused with a human room invitation.

Mirror promotion and cooperative ownership

The former read-only mirror strip is now an explicit control bar containing Copy Visible Transcript, Take Control, Fork, and Stop Mirror.

Take Control remains read-only until confirmation, any required collaboration lease, and the matching thread/resume acknowledgement have all succeeded. Failure releases the lease and leaves the task mirrored. A peer-held lease can only be waited on or avoided by forking; EXEPERT never revokes another peer's live lease.

Leases are keyed by server and thread, renewed every five seconds, and expire after fifteen seconds. They are cooperative between EXEPERT clients: another client with the raw App Server capability token can bypass them, which the UI states explicitly.

Native human collaboration

The global bottom-right collaboration launcher and popup implement a private, memory-only human room:

• original native speech-bubble, collapse, and right-facing share actions;
• host-assigned peer identity, sequence numbers, and timestamps;
• authenticated welcome/history, presence, typing, text, lease, disconnect, and room-end messages;
• at most eight peers, 500 in-memory messages, 4,000 characters per message, and 32 KiB WebSocket frames;
• typing throttling, chat and lease rate limits, bounded queues, and slow-peer removal; and
• responsive 390 by 520 px popup geometry that clamps to a bottom sheet on narrow windows without disconnecting when collapsed.

The room token and exepert-collab-v1:<base64url-json> invitation remain in memory and expire when the host ends the room. Only the display handle and popup preference persist. Completed visible Codex answers can be shared after preview confirmation; prompts, reasoning, tools, approvals, attachments, credentials, and whole transcripts are never shared automatically.

Native Chat history and message actions

Native Chat persistence is now a version-2 multi-chat store with stable chat and message IDs. Legacy message arrays and session keys migrate into one active conversation.

The store is bounded to 20 chats, 50 messages per chat, and 8 MiB total. It prunes the oldest inactive conversation when required, protects the active and fork-source conversations, and keeps an oversized active chat memory-only rather than deleting it silently. Attachment bytes retain their existing redaction boundary.

Role-appropriate message actions include:

• Native assistant: copy, local edit, regenerate, confirmed delete, read aloud, fork, Markdown export, and response details;
• Native user: copy, edit and resend, fork, and confirmed delete; and
• authoritative Codex assistant: copy, read aloud, human share, Markdown export, response details, and server-supported fork from the selected turn.

Mutating Native actions stage their transcript changes and restore the prior history/draft when the replacement request fails. Codex does not expose edit, delete, or regenerate for authoritative history because the pinned App Server does not provide those operations.

Windows read-aloud runs the default SAPI voice on a dedicated COM worker. It accepts one bounded visible-text utterance at a time and stops on replacement, message removal, chat switching, shutdown, or explicit user action. Reasoning, tool payloads, credentials, attachments, and speech content stay out of logs.

The non-modal response-details sheet shows only reported model, provider, identity, timing, usage, context, and Codex task metadata. Unsupported metrics remain unavailable instead of being inferred.

Contextual icons, emoji, reactions, and transcript navigation

Original 1.8 px outline SVGs provide compact 32 px contextual actions. Stable element IDs, accessible labels, tooltips, keyboard focus, destructive hover, and responsive wrapping remain part of the GPUI contract.

Both composers have a lazy offline emoji picker backed by a compact bundled Emojibase 16.0.2 English/Malay catalog. Windows renders the Unicode emoji; the application downloads no artwork or metadata during normal operation. Search, categories, skin tones, recent choices, keyboard navigation, responsive grid clamping, and selection replacement are local.

Native assistant reactions persist in the bounded chat store. Codex reactions are task-qualified, local-only annotations capped at 500 targets and cleared on restart. Neither reaction path reaches model requests, Codex wire traffic, collaboration peers, telemetry, exports, logs, or diagnostics.

Native and Codex transcripts now use a shared 42 px floating Scroll to Bottom action. It appears only outside the near-bottom threshold and marks unseen content below without consuming layout height. Codex rows receive 12 px side gutters while retaining task-local virtual-list bookmarks and tail-follow behavior.

Conversation Brain

The Chat context panel now contains a persistent two-brain card, and Brain mode contains separate SIMULATION and CONNECTION views.

The Human brain is labeled HUMAN · PREDICTED ANALOGY; the AI brain is labeled AI · INTERFACE ACTIVITY. A bounded reducer maps observable typing, submit, response, tool, approval, completion, cancellation, and failure events into visual activity. Human-to-AI and AI-to-Human bridge pulses are presentation signals, not measurements of cognition.

Optional local semantic analysis uses only bounded user-visible or authoritative agent text. It recognizes supported English/Malay evidence, fails closed for unsupported languages or low confidence, and never reads raw reasoning, reasoning summaries, tools, approvals, attachments, or credentials. Derived conversation state is process-local and capped at 40 conversations.

Empathy mode is separately opt-in. It produces a visible, user-overridable fixed tone/depth suggestion after a 400 ms local debounce. Native requests send only validated enum fields; Codex receives a visible presentation cue before the user's text. The guidance cannot change facts, uncertainty, safety, permissions, tools, approvals, coding correctness, or requested output format.

The UI and bundled atlas manifest state that this is an interaction visualization—not consciousness, mind-reading, hidden reasoning, literal thoughts, clinical emotion assessment, or measured human brain activation. Commercial v1 intentionally exposes no activation-derived evidence provider.

Supabase Edge Function boundary

run-chat accepts only the bounded optional interaction-guidance enum contract. The server maps accepted values to trusted fixed copy and rejects unknown fields, unsupported languages, or low-confidence values.

Input messages and output observability copies redact the private fixed guidance fragment. The provider response returned to the user is not rewritten by that observability redaction. Attachment bytes retain their existing redaction path, and safe persisted context excludes the guidance object.

Persistence and lifetime matrix

Data	Lifetime	Storage
Native chats, safe response metadata, Native reactions	Durable and bounded	Atomic local Native Chat store v2
Codex server/profile/task UI metadata	Durable and bounded	codex-settings.json v2
Codex capability tokens	Durable	Windows Credential Manager only
Collaboration handle and popup preference	Durable	Collaboration settings v1
Human room messages, peer IDs, typing, room token, invite	Current process/room	Memory only
Codex reactions and transcript bookmarks	Current process	Memory only
Conversation Brain derived activity and semantics	Current process	Memory only
Conversation Brain semantic/empathy/collapse preferences	Durable	Conversation Brain settings v1

Source and asset inventory

Area	Files and responsibility
GPUI integration	src/app.rs owns all source-specific state, overlays, actions, lifecycle adapters, and responsive rendering
Native Chat contract	src/chat.rs owns multi-chat v2 persistence, stable identities, reactions, metadata, and request DTOs
Codex server scope	src/codex/server.rs, settings.rs, and state.rs qualify profiles, task keys, persisted settings, and reducer state
Codex credentials	src/codex/credentials.rs owns token generation, redacted types, and Windows Credential Manager access
Codex runtime	src/codex/runtime.rs and transport.rs own dual supervisors, authenticated WebSockets, reconnection, and protocol correlation
Human collaboration	src/collaboration/ owns invites, wire protocol, host/client runtime, leases, settings, and UI-facing state
Message support	src/speech.rs, src/emoji.rs, and src/ui/chat.rs own SAPI, offline emoji/reactions, and shared controls
Conversation Brain	src/conversation_brain.rs and assets/brain/conversation-atlas-v1.json own the reducer, inference, settings, evidence labels, and mapping disclosure
Assets	assets/activity/*.svg and assets/emoji/emojibase-16.0.2-en-ms.json provide original controls and pinned offline metadata
Catalog build	scripts/build_emoji_catalog.ps1 reproduces the bounded bilingual emoji asset from pinned source data
Edge trust boundary	supabase/functions/run-chat/ validates guidance, inserts trusted prompt copy, and redacts observability
Contracts	tests/*_ux_contract.rs plus Rust unit tests cover transport, storage, actions, collaboration, emoji, and Conversation Brain invariants
Documentation	Native Chat, Codex, collaboration, Conversation Brain architecture pages, implementation specs, deferred work, notices, and activity metadata

The Cargo workspace adds tungstenite 0.30.0 with native-root Rustls support, Windows Credential APIs, and Windows SAPI/COM bindings. Lockfile additions are limited to their resolved transitive packages.

The repository ignore rules now exclude the root desktop runtime log and service-level Python bytecode/test/lint caches. These generated local artifacts remain on disk but are not release source.

Security and licensing invariants

• Raw tokens never enter JSON settings, process arguments, logs, diagnostics, errors, room history, or implementation documents.
• Plain WebSockets are loopback-only; remote control requires trusted WSS and user-managed TLS/DNS/routing.
• Mirror polling remains read-only and automatic approvals remain forbidden.
• Collaboration leases coordinate EXEPERT clients but are not a security boundary against clients holding the App Server token.
• Chat content, prompts, reasoning, approvals, attachment bytes, and protocol frames are not added to runtime diagnostics.
• Kairu/hl-plugins and Frimousse are credited as MIT-licensed interaction inspiration; their implementations and assets are not copied.
• Emojibase 16.0.2 metadata is included under its MIT terms. Emoji artwork is supplied by the Windows native emoji renderer.
• Conversation Brain publications and atlases are citations only; no referenced model weights, atlas coordinates, meshes, or artwork are redistributed.

See the root THIRD_PARTY_NOTICES.md for the complete notices and provenance.

Verification record

The final Windows desktop validation used the pinned toolchain:

cd desktop
cargo +1.96.0-x86_64-pc-windows-msvc fmt --all -- --check
cargo +1.96.0-x86_64-pc-windows-msvc test --workspace --target x86_64-pc-windows-msvc --locked
cargo +1.96.0-x86_64-pc-windows-msvc clippy --workspace --all-targets --target x86_64-pc-windows-msvc --locked -- -D warnings
cargo +1.96.0-x86_64-pc-windows-msvc build -p exepert-desktop --release --target x86_64-pc-windows-msvc --locked

The result was 318 passing Rust unit tests plus all integration contracts. Two existing native GPUI renderer smoke tests remain ignored by the default suite. Clippy passed with warnings denied, Rustfmt passed, and the target-specific Windows release executable built successfully.

The Edge guidance/redaction harness passed 6 of 6 tests through tsx because a standalone Deno executable was unavailable. The Edge function also bundled successfully with JSR imports kept external. git diff --check reported no whitespace errors.

Automated contracts do not replace manual release interaction. Human room behavior behind a real trusted WSS proxy, remote Codex control against an external server, Windows voice playback, native picker/focus behavior, and the Conversation Brain's final visual presentation remain manual checks. No Supabase deployment is implied by this source delivery.

Maintainer reading order

1. Read Native Codex live client.
2. Read Native human collaboration.
3. Read Native desktop Chat.
4. Read Conversation Brain.
5. Review the five completed implementation specs under _bmad-output/implementation-artifacts/ for acceptance and deferred edges.
6. Review THIRD_PARTY_NOTICES.md before redistribution.