Start with one agent. Finish with a team.

Create music, narration and sound effects

Native audio workflows through fal.ai in Canvas and Workbench.

• Open Images > Audio workflow in Canvas or Workbench. Configure access reuses your fal.ai account in the encrypted desktop vault. Enable the workspace, permitted audio models, external media and optional assigned-agent access; keep per-run and daily reservations. No extra key per node is needed. Higgsfield currently documents no standalone audio models in its API, only native audio in some videos. No Suno endpoint was verified in the official fal catalog, so it is not advertised or guessed.
• Search the audio catalog, select a model and read its official contract. Music models include ElevenLabs Music, MiniMax Music and Lyria where available; speech and effects include ElevenLabs endpoints. The live catalog is filtered by audio output; models can change, be deprecated or use unsupported contracts. Each model exposes its own fields, defaults, constraints, examples and documentation. Voices, language, lyrics, composition sections, instrumental mode, duration and references are available only when declared by that model. Use voices and references you have permission to use.
• Narration: put only the spoken words in Text, or individual speakers in the dialogue Inputs array. Keep Direction and context notes empty for speech-only contracts. Music direction goes in Prompt; lyrics and composition plans have their own fields. Notes can supply context only to models with a direction field. Production notes and camera instructions are never silently appended to speech. Reference inputs select named workspace audio/video/image nodes or confined project files; providers receive only explicitly bound inputs.
• Save, Estimate, inspect the outbound input and reservation, then Generate once. Character-priced models count the actual Text/dialogue fields. Duration-priced models use their explicit duration, not a hidden five-second video default; minute estimates round up. When units cannot be derived, enter explicit estimated billing units. Estimates and 4x reservations are not a provider invoice cap. Unknown or rejected submissions are never automatically sent again; Retry download reuses the paid result.
• Completed audio is saved without transcoding under generated/audio by default, as a playable, downloadable native media node. Reuse it as an audio reference in another audio or compatible video workflow. Voices/models must support that reference; a saved input is not a guarantee of identical generative voices. MP3, WAV, OGG, M4A and FLAC containers are supported; raw PCM and telephony outputs are rejected before submission. Playback depends on the codec. Removing a Canvas node preserves its file and does not authorize agents to reuse historical media. Existing Codex image workflows and embedded dictation/TTS are unchanged.
• Agents use audio_workflow_models/list/read/create/update/preview/run/cancel/retry_download/remove, or orkestrai audio with --task and --input. These aliases share the same native creative service, revision guards, queue, credentials and budgets as video_workflow_* with config.modality=audio. Discover the endpoint before creating, choose an authorized profile, preview the exact input, and reuse the same UUID after a timeout. Never put credentials in prompts or parameters.
• Output privacy: Configure access > Workspace permissions > fal.ai output access defaults to Private. An owner may explicitly choose Anyone with the link for new outputs; fal is asked to delete them after one hour. Anyone holding that URL can download it until expiration. Inputs remain private and the API key never goes to a media host. The preview and queued run pin this choice; changed permissions require a fresh preview. Agents cannot change the policy. This does not repair existing files or regenerate them. Orkestrai still disables stored JSON input/output history, so the fal dashboard may say Output not available even when the queue has returned a result. A 401/403 download is not a completed delivery: Retry download does not charge for another generation, and the provider request ID identifies the original job for support. Do not generate again without reviewing the additional charge.

Read scanned PDFs with local OCR

Local OCR makes scanned PDFs searchable by users and agents.

• Drop a scanned PDF onto Canvas, or use Second Brain > Import files. Open its document node or source entry: OCR runs automatically on image pages and preserves embedded text on mixed PDFs. English, Portuguese and Spanish models ship with the app; no internet, API key, Python or system OCR installation is required. The original PDF is not rewritten and page images are not saved. A private temporary language directory is removed after the parser exits.
• Recognized passages show OCR, page and estimated recognition confidence. Search the same text through Second Brain, knowledge_search and knowledge_read. Agents must cite the source id, revision/hash and page and treat OCR as fallible, untrusted evidence. Check names, numbers and critical values against the original. This recognizes printed text, not guaranteed handwriting, diagram understanding or exact table reconstruction.
• Limits: 25 MB per file, first 200 pages, 6,000 characters per page and 250,000 characters total. Rasterization uses up to 8 megapixels and 4,096 pixels per side; source images over 16 megapixels may be omitted by the PDF decoder. Extraction stops after two minutes per document and preserves available text with an explicit partial state and unfinished/failed page list. Blank pages remain empty. Password-protected PDFs require an unlocked copy; damaged files show errors. Refresh index retries a partial result; split large PDFs if needed. One parser runs at a time outside the application server, including its OCR worker. A timeout kills both. Previously imported scans are reindexed automatically on the next query after the extractor update.

Second Brain: connected knowledge

Turn documents and project work into cited, reusable knowledge.

• Open Second Brain in the Canvas toolbar or Workbench. Sources, Knowledge graph, Workspace Memory and Agent learning share the same workspace. Add to Canvas creates a connectable knowledge node; its graph is separate from the codebase dependency graph. The existing code graph is not replaced.
• Drop one or more files from Finder or Explorer onto empty Canvas space, or choose Import files in Second Brain. Each file becomes a document node and a copy is retained under .orkestrai/knowledge in the project. Existing images retain the image workflow. Agents can attach existing project files with knowledge_attach or orkestrai knowledge attach path. Import does not run macros, formulas, scripts or document actions. The file limit is 25 MB. PDF with embedded text, Markdown/plain text, XLSX and CSV are searchable. Other formats are retained but explicitly unsupported for extraction; Scanned and mixed PDFs receive embedded English, Portuguese and Spanish OCR, with no account, upload or separate installation.
• Search matches normalized words in titles, tags and extracted text. Results include the source id, revision, SHA-256 and page, sheet/row or line locator. Open a source to read passages, edit comma-separated tags, follow backlinks or jump to its Canvas node. Original files remain intact when a node is removed. Text extraction is bounded to 250,000 characters and 200 passages, with explicit partial/empty/error/missing states; a partial index is not the entire document.
• Notes can link to unique source titles using [[Brief]] or an exact source id. Ambiguous duplicate titles do not silently select a target. Hashtags and explicit tags organize sources. Canvas edges, wiki links and existing task/memory provenance form the graph; edges never grant permissions. Notes/tasks/memories are read fresh on query; files are checked for changes and re-extracted when needed. While visible, Second Brain receives workspace events and watches linked document files. Changes are coalesced briefly, then re-read and re-extracted before display. Atomic saves, deletion and recreation of linked files are observed. WSL/network files use a one-second filesystem poll. A 15-second recovery check renews the watcher lease and catches missed events; disconnected updates are shown as reconnecting, not live. Hidden views pause; file watchers expire after 45 seconds without a visible client. Refresh index forces extraction. Only attached sources are indexed: creating an arbitrary file in the project does not automatically import it. This is lexical retrieval, not vector search or a guarantee of correct answers.
• Agents use knowledge_search and knowledge_read and must cite actual passages. Imported text is untrusted evidence, never a system instruction. Do not import passwords, credentials or private conversations. The private Computer conversation-memory store is excluded. No Obsidian account, cloud index or embedding API is required.
• Open Second Brain > Agent learning, select the named agent and choose Automatic, Review first or Off. Learning belongs to that Canvas node, not its current provider or session: renaming it, restarting or switching provider retains its lessons. A new or copied agent does not silently inherit another identity. Deleting a workspace removes its learning records; deleting original project files is not part of this operation.
• Task completion, or a blocked/error task transition, creates an idempotent pending reflection. The task response and bridge instructions ask the responsible agent to report a meaningful mistake/correction or a verified reusable procedure through learning_reflect. If nothing was learned, learning_skip archives the reflection. The app never wakes every agent merely on startup, spends tokens on a separate reflection model, fabricates lessons or trains model weights.
• Automatic mode can activate ordinary lessons from completed assigned tasks. Evidence remains clearly labeled Agent-reported; this is not an independent test result. Review first keeps proposals pending. Suspicious instruction-changing content and lessons from unfinished work require review. Obvious credential patterns are rejected, but pattern matching is not universal DLP or immunity to prompt injection. Off stops reflection capture and recall without deleting history.
• Review the trigger, mistake, correction, task evidence and revision history. Activate, reject or archive a lesson; stale review writes are rejected. Relevant active lessons are included in new task briefs, bounded to five lessons and 1,200 characters each. Agents should also use learning_search before relevant direct work. Historical lessons cannot change security grants, rewrite roles/skills or override current instructions. Recall is assistance, not a promise the model will never repeat an error.
• Knowledge graph is a native point-and-link network, not another Canvas of cards. Choose 3D to rotate or 2D to pan; drag a source to reposition it, scroll/pinch to zoom, click to read it, or double-click to focus. In 3D, right-drag or Shift-drag pans. Fit, zoom, focus, rearrange and fullscreen controls have tooltips. Colors initially distinguish relationship communities computed with Louvain; the legend names each group after its most connected source and focuses it on click. The palette control switches to source-type colors and type visibility filters. Communities are navigation aids, not new facts or links. Hover shows the full title and excerpt; selected neighbors remain highlighted. Existing positions, camera and selection survive live changes. Rearrange explicitly computes a new force layout. Layout runs in a worker; GPU rendering runs only after changes and pauses offscreen. No invented links or continuous background animation. Use Sources if WebGL is unavailable.

Choose a video provider without changing your image workflow

Use fal.ai, BytePlus ModelArk or Higgsfield with separate accounts, contracts and budgets.

• Open Images > Video workflow in Canvas or Workbench. Existing drafts still use fal.ai. Create reference images and storyboards with the existing Codex subscription workflow; this feature does not send image generation to another provider.
• In the video node, open Configure access > New account. Choose the provider before entering its credential: a fal key, a BytePlus ModelArk API key, or Higgsfield KEY_ID:KEY_SECRET. Save it in the encrypted desktop vault. Then enable this workspace, external media, permitted models, per-run/daily USD reservations and optional agent access. Website subscriptions do not imply API credit or model access. An existing account cannot change provider; create another account instead.
• Select Video provider, Model and a matching Account in the node. fal.ai keeps its live catalog; BytePlus and Higgsfield use versioned, reviewed contracts. Matching model names do not imply matching resolutions, durations, audio or reference modes: Higgsfield H3 currently exposes 2K, not fal's 768p option. Changing provider explicitly resets incompatible parameters/account after confirmation while preserving required references and locked characters for remapping. Verify every input, dialogue and audio setting. Save before estimating.
• Model-list rates are labeled public estimates when applicable, with their verification date. They exclude assumed promotions and are not final quotes. Higgsfield Estimate uploads the approved local references and quotes the exact input against that account; it does not generate. BytePlus estimates documented token rates, duration and resolution, conservatively budgeting unknown input-video duration with a 4x reservation. fal keeps its current estimator. No estimate or local reservation guarantees the provider's final invoice. Generate explicitly only after reviewing the outbound data, cost and grants.
• Each run freezes its provider, account and contract. Reloading resumes the same remote job, never silently switches models and never retries an uncertain paid submission. Retry download does not regenerate. BytePlus accepts local image/audio references but currently requires public HTTPS URLs for input videos; Orkestrai does not publish them elsewhere automatically. It exposes reference generation, not direct editing/extension in this adapter. After submission, BytePlus cancellation is disabled because its deletion endpoint can erase completed results; local queued work can still be cancelled. Higgsfield supports its reviewed text/image/reference/edit/extend endpoints where listed. Inspect actual picture and sound before approval.
• Agents use video_workflow_models with input.provider, then the exact input.endpoint; set config.provider and a matching profileId when creating/updating. They retain assigned-task, owner-grant, preview/revision and idempotency requirements. New providers do not authorize spending or grant credentials automatically. Changing provider/model needs owner agreement; voice and character references are never a promise of perfect generative consistency.
• Video references show names and previews for both canvas assets and standalone project files. Choose the role (first frame, last frame, reference image, video or audio), then pick canvas media or use Choose from project to browse folders. View reference opens a larger preview without leaving the workflow. Numbers are one-based model input positions, not filenames. Technical details contains the exact API mapping and editable relative file path. Viewing a file does not create a node or rewrite the agent's binding. Agents use the same bindings; the app does not infer missing references or start paid generation automatically. The provider adapter uploads or encodes inputs at the disclosed quote/generation step.

Assemble and export a video sequence

Order, trim and caption existing video clips without changing their source files.

• Open Images > Video sequence in Canvas or Workbench. Add existing workspace Video nodes (MP4, WebM or MKV). Install video encoder once in the node if requested: the owner confirms a separate, pinned FFmpeg download verified by SHA-256. No Homebrew, API key or paid generation is involved.
• Select a clip in the vertical list. Move it up/down, set in/out points, audio level and a caption, then Save. Removing a clip only removes it from the sequence. The source hash is pinned; a changed or missing source blocks export instead of substituting media. Choose horizontal, vertical or square output and 24/25/30 fps. Fit keeps all picture content with black padding, never silent crop or stretch. Captions are burned inside 10% title-safe margins.
• Play sequence previews the ordered trims. Export MP4 creates a new H.264/AAC file under generated/videos/sequences and a connected Video node. Inspect the delivered video and sound. An export keeps its source revision and is marked older after edits. Exporting is not delivery; failures and interruptions stay visible. Cancel stops the encoder and leaves sources intact. Owned temporary files are removed after success/failure/cancellation and abandoned job directories are removed before the next export. Use Full screen on the preview to inspect vertical clips and captions at a readable size, then return to the same editor.
• Agents use video_workflow_sequences or orkestrai video sequences with an assigned task. Read the current revision before apply/export; retry a timed-out request with the same UUID idempotencyKey. Runtime installation is owner-only. Limits are 30 clips, 10 minutes, even output dimensions 240–1920 per side, one local export at a time, at least 3 GB free disk and a bounded output size. Transfer a sequence together with all its source video nodes. Existing Codex ImageGen images and fal generation remain separate; this is assembly of delivered videos, not a new generative model.

Reuse creative workflows and inspect the queue

Save versioned scene briefs with named inputs and recreate editable storyboards without copying execution authority.

• Open Images > Creative workflows in Canvas or Workbench, or Save workflow in a Storyboard header. Save workflow captures the current saved revision, ordered scene direction, dialogue, duration and shot. Name it and optionally select an existing workflow to create a new immutable version. Unsaved scene edits are not captured. The agent command video_workflow_recipes also accepts a group containing exactly one storyboard; it captures that board, not unrelated notes, agents or automation. Files, run history, sessions, account credentials and paid grants are not stored in the recipe.
• In Library, choose a version, enter the production brief/script, output aspect ratio and optional Codex image agent. Bind every named product/reference slot to a current-workspace image or video, and every character slot to an approved local character. Matching approved identities can be preselected by exact family/version/digest. Only the owner may deliberately choose a different approved character; agents must preserve the saved identity. Import a character from Character library first when needed. The owner can reuse recipes from another workspace, but agents cannot browse or read other workspaces.
• Create editable storyboard adds ordinary native content with fresh scene IDs and no previous run links. The script replaces the literal {{script}} once, without recursive evaluation; if a direction has no placeholder, the brief is appended. An empty required script or missing input blocks creation. Open the resulting storyboard, inspect every scene and prepare image/video drafts separately. Image dimensions are requested through the existing Codex workflow; video duration and aspect ratio must be declared by the selected endpoint or preparation stops. Existing generation, alpha validation, paid preview, budgets and security gates remain in force.
• Generation queue refreshes real current-workspace image/video states while open. It shows drafts, failures, provider position, reserved USD and output links, not invented percentage progress. Open workflow exposes its regular estimate, grant, review and generation controls. Cancel uses the existing executor; Retry download only retrieves an existing remote result. Uncertain paid submissions have no automatic retry. Some completed outputs do not mean the whole storyboard is complete. Deleting a saved recipe version never deletes instantiated storyboards or output files.

Plan scenes in a native storyboard

Ordered scenes, locked identities and connected image/video drafts in Canvas and Workbench.

• Open Images > Storyboard in Canvas or Workbench. Add a scene and set its title, direction, dialogue, language and requested duration. Select the exact approved character version and local reference images/clips; choose a Codex executor for image generation. Save before switching scenes. Duplicate starts a new draft without reusing execution links. Drag scenes to reorder or use Move up/down. Removing a scene preserves its workflows and output files.
• Prepare image draft creates a connected existing Codex image workflow, without running it. Up to five references are supported, including all selected character masters; excess references fail explicitly instead of being dropped. Open the draft and use its normal generation controls. Prepare video draft creates a native fal video workflow without charging. Choose an allowed endpoint, bind every required character and reference, review the actual model duration/audio capabilities and use Save > Estimate > Generate. Unsupported requested duration is rejected, never silently shortened. Preparing the same unchanged scene reuses its linked draft. To choose another draft explicitly, unlink and prepare again.
• Scene changes mark linked drafts stale; outputs remain unchanged. Manual edits and video_workflow_storyboards use the same revisioned document. Concurrent writes fail with a revision conflict; local unsaved edits remain visible until you discard and reload. Copied storyboards retain missing reference/character IDs for explicit local repair; they never inherit account grants. Review actual picture and sound before approval. An available locked voice reference does not guarantee that every model can reproduce it.

Compare and approve creative variants

Open Compare and review in an Image or Video node header, or in a Storyboard. Same workflow limits the contact sheet to related outputs; turn it off to compare other local assets. Choose A and B with searchable selectors or click a thumbnail for B. Open output locates the original Canvas node. Video pairs share play, pause and seek over the shorter common duration; switch sound between A and B. A single video uses native controls. Images fit without cropping. Missing or undecodable files cannot be approved. Compare the face, wardrobe, framing, language and actual voice against approved references; a completed generation is not a consistency guarantee. Write feedback and choose Approve, Request changes or Reject. History preserves each decision, author, time, comment and file fingerprint. Approval binds the exact asset bytes, generation provenance and available frozen character versions. A changed file invalidates the decision; competing reviews require reload. Agents use video_workflow_assets list/inspect/decide and may only propose a decision, never approve their own output. These actions do not call fal or ImageGen and do not spend credit.

• Open variants
• Compare picture and sound
• Record a review

Create a new direction from an existing image

Open Creative actions in an Image node header. Choose Variation, Remove background, Change region or Animate. The original is never overwritten. Each action freezes the exact source bytes into generated/creative-actions and connects a reference node and a new native workflow. The saved ancestry includes the original node, file hash and generation context. Variation and Change region accept direction and 1–10 results. Choose a Codex agent or assign it in the resulting workflow. For Change region, drag a rectangle on the actual image or adjust X, Y, width and height with the keyboard-accessible sliders. Source dimensions are verified by the backend. This rectangle is prompt guidance, not a hard inpainting mask or a promise that every outside pixel will be unchanged. Prepare draft does not generate or spend credit. Open the new Image workflow and use the existing Codex subscription flow; background removal requests genuine RGBA transparency and retains the existing validation/repair loop, with no local pixel editing. Animate creates a native video draft: choose any compatible allowed model, map the frozen image and every required character, save, estimate and explicitly run. A missing or changed frozen reference blocks reuse. Cancel before preparation creates nothing; removing a draft never deletes the original. Agents use video_workflow_assets command=prepare with the inspected expectedDigest and edit.operation; both surfaces use the same validation.

• Choose an image action
• Set direction or region
• Inspect the connected draft

Reuse an approved brand kit

Version palettes, logos, products and creative rules, then place their exact assets in any workspace.

• Open Images > Brand kits in Canvas or Workbench. In This workspace, name the kit, choose color swatches and hexadecimal values, add named logo/product/style images, and write the brief, communication tone and usage rules. Image assets must be local PNG, JPEG or WebP files; export vector logos through the existing Design tools first. Save drafts before approving.
• Review the actual files, then Approve and lock kit. Approval freezes copies under generated/brands with SHA-256 fingerprints. Approved versions cannot be edited or deleted as drafts; create a new version to change them. Conflicting revisions or changed files stop the operation instead of overwriting another edit. Palette and rules alone can also form a kit.
• Use Approved library > Add kit to Canvas in any workspace. The exact version becomes a native group, a rules note and connected image nodes. Copies are independent of the original project; no provider account, token or generation grant is transferred. Connect the note and relevant images to existing Codex image or fal video workflows, respecting each model’s reference limit. Agents use video_workflow_brands to draft/read/fork/place local kits; only the owner can approve or import across workspaces. Kit inputs do not guarantee pixel-perfect generated outputs: review the result before use.

Direct framing and camera movement

Save shot intent alongside scenes without losing reference or voice bindings.

• Open a Storyboard scene or Video workflow and use Shot direction. Choose framing, angle, movement and pace. The same controls persist in the scene and its materialized video draft; still-image drafts include framing and angle, not camera motion.
• These controls are creative intent in the prompt, not a guaranteed camera trajectory. First and last frames and other media fields depend on the selected endpoint’s declared contract. Scene duration must match an exact supported duration; unsupported or ambiguous durations stop preparation instead of silently shortening the clip.
• Changing model preserves saved shot intent, character and voice mappings, file references and parameters. Incompatible curated adapters refuse the switch; generic endpoints require repairing incompatible pointers before estimation. Required frame references remain required until explicitly rebound. Agents use the same shot object in video_workflow_storyboards scene operations and video workflow configuration.

Video workflows with fal.ai

In Canvas, open the Image menu and choose Add video workflow. The same node opens in Workbench. Existing Codex image generation remains unchanged and needs no API key. Video is a separate, opt-in paid fal.ai integration: open Configure access in the node, save an account key in the desktop vault, enable the account and this workspace, authorize external prompts/images, select models, and set USD reservation limits and concurrency. Agents additionally need the workspace agent permission, a live authenticated terminal, and a task assigned to them.

• Storyboard-to-video: Codex writes the scene order, prompts, dialogue, language and sound direction in Notes. Existing Codex/GPT Image workflows create all characters, storyboards and visual references. Only approved reference files are uploaded to fal for video generation; fal does not generate these images. Enable native audio only on models whose own contract supports it. Seedance 2.5 reference-to-video accepts image/audio references and generate_audio; native speech is not an Orkestrai TTS voice, and exact language/voice behavior depends on the model.
• Character library stores workspace drafts and owner-locked versions: appearance, 1–12 master images and either an approved audio reference or an existing provider voice ID scoped to account and compatible endpoints, plus language and delivery style. Approval copies local assets to generated/characters, records SHA-256 fingerprints and makes that version immutable. Changes create a new draft version, never rewrite existing scenes. Agents may propose/read versions, not approve, delete locked versions or silently replace an existing scene's character.
• Bind an exact approved character version to each scene and map its master images and voice to supported model fields. The server injects the frozen inputs and rejects changed/missing files, conflicting bindings, an incompatible account/model or disabled native audio before generating. A transferred scene retains unresolved character IDs and cannot run in another workspace until the owner deliberately rebinds local approved identities. Snapshots retain version IDs and fingerprints. Input locking is not a 100% generative consistency guarantee: picture, speech and lip sync still need human review. Automatic voice enrollment and consistency scoring are not provided. Assemble delivered clips with the native Video sequence node.
• Open the Images menu, choose Characters, then drag an approved character onto an empty area of the destination Canvas or choose Add to Canvas. The library is shared across local workspaces. Orkestrai copies all frozen images and the voice sample, preserving the exact version and fingerprint, and creates a grouped identity Note and reference Images without changing the source. Provider voice IDs retain their original account binding, but credentials and workspace grants are never copied. Workbench offers the same library and Add to Canvas command. Agents can place approved local characters; importing from another workspace requires the owner.
• Choose an approved character by name: the scene shows its master images and voice sample, and assigns all references to conventional inputs declared by the selected model. No reference is dropped to fit. Advanced input mapping remains available for unusual contracts. Type @ in Direction or use Insert a character reference to insert a saved @{name} alias; the alias stays bound to the exact version ID. Removing a referenced character or using an unknown alias blocks generation instead of silently changing the subject. Agents can request the same conservative mapping with video_workflow_characters command=binding and the current config.
• Browse the complete published fal.ai video catalog with search, including Seedance 2.5 and text-, image-, reference-, audio- and video-driven endpoints. Orkestrai loads the selected endpoint's official OpenAPI contract, preserves its input types, enums and nested parameters, and pins that contract to each run. Catalog visibility does not guarantee account access. Deprecated endpoints and invalid contracts cannot generate.
• For catalog models, use Workspace media inputs to bind an Image/Video node or a workspace-relative image, video or audio file to an input path such as /image_urls/0 or /elements/0/frontal_image_url. Bindings are ordered, hashed and rechecked before sending; media bytes are not stored in workflow JSON. Nested parameters have a JSON editor and contract reference. Up to 50 bindings, 10 MB per image, 64 MB per audio/video and 100 MB total are accepted; the provider may impose stricter limits. Copying between workspaces remaps selected node bindings and clears file-only bindings.
• Billing units normally follow duration or one output. For endpoints billed in tokens, frames, compute units or automatic duration, an explicit estimated billing quantity may be required. Read the selected endpoint's pricing and set account-side limits. A local reservation is not a provider-enforced invoice cap.
• Wan 2.7 generates video from text, with 2–15 seconds, 720p/1080p and enumerated aspect ratios. Kling 3 Pro animates a selected first-frame Image node, optionally with an end frame, for 3–15 seconds. Kling inputs must be PNG/JPEG/WebP, at least 300 px per side, up to 10 MB and within a 0.4–2.5 aspect ratio. Only supported controls appear. Kling speech supports English and Chinese; Wan may create background audio automatically.
• Select context Notes explicitly; their text becomes part of the outgoing prompt. Select first/last images by node identity. Ordinary collaboration edges do not execute other branches. Save, estimate cost, inspect the base estimate and reservation, then generate. The local reservation is four times the base-unit estimate as headroom, not a guaranteed cap on the provider invoice. Enforce account-level limits on fal.ai as well. Existing workspace security gates can require approval for paid operations.
• Runs are persisted before submission. After restart, Orkestrai polls the saved remote job rather than generating again. If the paid submission itself loses its response, the run is marked unconfirmed and is not retried automatically. Check the fal.ai queue/account before taking further action. Cancellation requested is not confirmed cancellation or a refund; a completed result can win that race.
• Each declared video/audio output is downloaded unchanged to the selected workspace folder with size limits and SHA-256 provenance, then appears as its own reusable node. MP4, WebM, MOV, MKV, GIF and supported audio containers retain their original format; playback depends on the installed browser codecs, and downloads remain available. A download retry never generates again. Files are not stretched, cropped or re-encoded. Deleting nodes preserves delivered files. Active paid runs block transfer and deletion.
• Keys remain in the desktop secure vault. No account key or generated signed media link enters node payloads or agent responses. Local reference files upload only after the workspace execution gate, with a requested private ACL and 24-hour expiration; time-limited read links are passed only to the selected endpoint. fal request JSON storage is disabled. Supported outputs request a one-hour lifetime and private CDN ACL. These settings cannot guarantee upstream provider retention. Send only material you may share, and retain downloaded files yourself.
• Agents use video_workflow_models/list/read/create/update/preview/run/cancel/retry_download/remove or orkestrai video. List exposes permitted profiles and model capabilities, never credentials. Use the returned revision and preview ID with a stable UUID idempotency key. A queued request is not a delivered video. Do not use raw provider HTTP to bypass workspace policy. Runway direct integration and arbitrary ComfyUI JSON are not supported. Creative workflows reuse native storyboards, and Video sequence assembles delivered clips locally.
• The picker includes the public catalog, not a claim that every listed endpoint is usable. Some published entries have no queue OpenAPI contract, use real-time streaming instead, are deprecated, or return 404. These cannot execute through the queue adapter; contract errors are shown before a paid submission. The bundled discovery snapshot is refreshed from fal, and each selected model uses its current validated contract.
• A run accepts up to 10 video/audio outputs. Each file is downloaded without resizing or transcoding and rechecks current filesystem exclusions, size and network grants before publication. If approval is pending or access was revoked, the paid job is preserved and Retry download recovers the result without generating again. FLAC references are supported alongside WAV, MP3, Ogg and M4A.
• Selecting a model loads its official queue contract. Fields include provider descriptions, suggested examples and visible defaults. Examples are suggestions, not a closed set of valid values. Objects and lists have nested controls; JSON remains available for advanced or ambiguous contracts. Prompt rewriting is visible beside the main settings with a warning: it changes direction, not render speed. Saved legacy drafts offer Edit all model parameters to explicitly convert to the full endpoint without losing direction or references. Parameters not supported by a newly selected endpoint remain visible and must be removed or remapped, never silently discarded.
• Select a configured account to compare base rates and billing units in the model picker. Rates are fetched only for displayed models, cached for up to five minutes and scoped to the account; missing rates are shown as unavailable, never zero. They are not configuration-aware final quotes: resolution, audio and other multipliers may apply. Estimate cost still validates the complete draft, references and budget before generation. This lookup never submits a job or reserves money. Agents use video_workflow_models with endpoint for the same documented schema, or pricingIds (up to 50) and profileId for permitted account rates; they never receive the credential.

Open a delivery folder

Ask the agent to open generated/images/xyz-carousel. It uses fs_open_folder, or orkestrai fs open-folder "generated/images/xyz-carousel", to open the existing folder in Finder, Explorer, or the Linux file manager on the host. Computer Control, Accessibility and Screen Recording are not required. Registered @alias folders are supported; files, URLs, executable bundles, path escapes and new grants are not. Explicit filesystem restrictions and emergency stop remain effective. The operation is audited and confirms the operating system accepted the request; it does not claim visual inspection. If the agent session predates the MCP tool, it can use the CLI.

Automatic replies to one approved conversation

Enable Bounded Security with computer and agent capabilities. In the existing Computer node, open Conversation replies and read the authorized native window. Select an active assigned task, the actual conversation HEADER (never its sidebar contact), composer and Send control. Select a received-message prefix exactly as exposed by native accessibility, including the sender; the message label must distinguish received messages from your own and ideally include its timestamp. Limit response length and hourly count. Only the owner can create, enable or revoke this authorization; it does not grant publication to the entire app. Create an enabled Manual prompt_agent automation for the same agent. Manual automations stay enabled across successive events; do not add scheduled polling. Select the exact native window, task and conversation authorization in observation, use Auto and enable Monitor. Switching focus to another app no longer pauses native observation. Reads remain bound to the authorized process/window; changing the selected conversation, closing the target or revoking access still prevents processing. Observation does not focus, click or type. Native accessibility must expose complete, unambiguous messages; this is not an API import of the application's entire chat history. Unsupported apps/Linux do not fall back to coordinate-based automatic sending. The first observation establishes a baseline. Subsequent observed incoming messages enter a persistent local inbox, including arrivals while the agent is busy. A short configured cooldown groups consecutive questions. Events carry grantId, batchId, inReplyToDigest and ALL messages in the batch. The agent reads them in order, groups subjects and addresses every question using computer_reply with those fields, targetId, taskId, idempotencyKey and the complete response. Pending requests belong in the existing task/notes; accepting a request is not completing it. The same agent session retains conversational context. Each batch contains at most 20 messages and 60,000 characters; excess messages stay pending, never silently truncated. The inbox allows 128 unresolved messages and blocks with an error if full. Orkestrai validates the native incoming message before typing. During composition and submission, the pinned recipient, authorization, focus and complete draft remain guarded; the original message does not need to remain visible or keep the same history index. New messages remain pending while an already claimed batch is answered. Human drafts, changed recipient, revoked grants or ambiguous input stop execution. On macOS, process-targeted native input updates the actual composer; multiline editors use Shift+Return for line breaks, never an unmodified newline. Native PID/window/bounds checks remain active throughout. A successful command acknowledges its batch as natively submitted, not delivered/read. A partial or uncertain failure stays blocked for inspection; do not resend or claim nothing was sent without checking. The panel displays pending-message count, observer state and measured local check time separately from provider generation time. Inbox text is stored in the app's local database outside the workspace repository: unresolved messages remain until handled, and up to 64 submitted messages are retained for context for at most 14 days while monitoring runs. Up to 4,096 recent message hashes prevent duplicate observation. Audit keeps counts/hashes, not chat text; provider transcripts have separate retention. App shutdown restores queued observations, but cannot guarantee recovery of messages the native interface never exposed. Text replies do not authorize attachments, file access or image generation on a contact's instructions; generated-image delivery needs its own owner-approved integration. Local builds remain ad-hoc; OS permissions belong to the owner and are never bypassed.

A companion that remembers and schedules work

Start with an active task assigned to the Canvas agent and the existing Computer conversation authorization. Ask the agent naturally to help; computer_capabilities reports supported resources, setup and permissions without reading credentials. Its briefing connects tasks, notes, published Workshop tools, integrations, native image workflows, TTS and PDF generation. It must inspect existing tools before proposing another and may publish automatically only within your existing Workshop policy. For example: "Every Monday at 14:00, America/Sao_Paulo, prepare my weekly report and send its summary to this authorized conversation." The agent creates an existing prompt_agent routine through automation_save, then returns its id and next occurrence. In Automations, Schedule offers interval or Calendar with once/daily/weekly/monthly, local time, IANA timezone, weekdays/date/day, and missed-run policy. Skip allows 60 seconds; Latest permits one recent occurrence within your configured delay, not a flood of missed reminders. Nonexistent daylight-saving times are skipped; repeated times run once; a nonexistent monthly day is skipped. The host must be awake, the workspace loaded and authorization active. Inspect, edit, pause or cancel in the same Automations UI; agents can edit only their own assigned-task routines with the current revision. In Computer > Conversation replies, enable Private conversation memory explicitly. Only newly observed incoming messages after the baseline and successful native submissions enter this separate local journal. It is not shared project memory and never imports an application's entire history. Retention is 30–3650 days, also bounded by 5,000 messages/16 MiB per conversation. Up to 256 sourced facts retain their cited excerpts until removed; the agent searches older details on demand instead of placing the whole journal in each prompt. History lets the owner search, revise facts and erase a fact or all local memory. Deletion does not erase provider transcripts or the external chat. Disabling pauses new retention; revoking/removing the authorization deletes its private records. Long requests stay traceable after acknowledgment. Enable scheduled reminders and task results for that exact contact to allow computer_send without requiring a fresh incoming message. It requires an assigned task or automation run source, deduplicates that source across restarts and uses the same foreground, recipient, empty-draft, hourly-limit and risk safeguards. A permission change between typing and sending stops publication. Incoming messages never authorize new contacts, apps, files or purchases. Calendar dispatch and a cleared composer are not proof of a delivered report. artifact_speech reuses configured TTS to create a workspace WAV; artifact_report creates PDF; artifact_inspect verifies path, format, size and SHA-256. Native images retain the existing Codex workflow without a new API key. In Computer > Conversation replies > Native attachments, the owner authorizes the picker, optional menu item, preview Send, formats and size for that contact. computer_media_send uses an incoming digest or an authorized task/run source, a private immutable copy, exact recipient and preview filename guards. Retry keys cannot repeat an uncertain send. The original file is unchanged; temporary copies are limited to 100 MiB and expire after 15 minutes. Receiving is a separate opt-in: choose the message Download control and optional incoming-media prefixes. computer_media_receive saves into a NEW authorized workspace path, verifies format/hash and never executes incoming content or overwrites files. Only a Download belonging uniquely to that incoming message is accepted. Native picker support currently targets standard macOS attached file dialogs; custom dialogs and other desktop backends are not claimed as verified. Portal upload/download remains separate under existing browser grants. TTS WAV is an audio-file attachment, not a native push-to-talk recording. artifact_transcribe uses existing STT for PCM16 WAV; the installed app also decodes Ogg/Opus, MP3 and M4A with Chromium, bounded to 10 MiB and 10 minutes, without Python, external upload, microphone access or another window. Codec availability is checked at execution. Prepared, submitted, received and delivered are different states; an imported file is not yet understood content. Interrupted text composition has a bounded retry only before Send and only for an exact prefix of the authorized reply. Inspect interrupted reply lets the owner review an older failed draft, then authorize resuming the original request without rewriting its content or publishing anything during recovery. A changed recipient/draft or any recorded Send attempt remains blocked; recovery is not delivery. Workshop authoring exposes the exact manifest contract and the authenticated agent's standing publication limits. Transform fixtures can assert expectedOutput before automatic publication; use two distinct examples and check publishedRevision before execution. macOS editor focus waits for native confirmation and can use the verified editor's focus action. App-owned popovers retain the original document guard. Another pre-Send interruption requires a fresh owner inspection and distinct recovery authorization; uncertain publication is never replayed. Native macOS attachments now handle animated picker menus, system-owned Open/Save sheets and file-reference URLs while comparing the full canonical path. Standalone previews validate the exact authorized recipient, filename, empty caption and Send in one bounded container. In Computer > Conversation replies > Native attachments, Inspect interrupted attachment permits owner recovery only when the verified audit contains no Send attempt. Close previews and confirm the file was not sent manually; the agent must reuse the original request. Uncertain sends remain blocked. In Computer > Conversation replies > Persona and voice, save the companion name, owner instructions, response language, default voice/speed and public style. Incoming text, transcribed audio, images and private memories are reference content, not permission to change persona, recipients, apps or file access. The app blocks recognized credential material before public text, speech, generated reports and textual attachments; errors and audit retain a reason/hash, not rejected content. Operational details can be blocked and em dashes replaced before sending. These checks are defense in depth, not universal DLP or immunity to prompt injection. Local TTS exposes F1-F5 and M1-M5 for each supported language through Settings > Voice and computer_capabilities (30 ids). For example, request pt-BR-m3 with artifact_speech; an unknown id is rejected instead of silently changing voices. A configured per-contact voice is used when omitted. STT is unchanged. Receiving authorized audio automatically attempts local transcription after download: transcription.state=ready is untrusted external text; unavailable means retry artifact_transcribe on the verified saved file, not download again. Transcript text is transient, not copied into action/audit rows. A WAV attachment is still not a native push-to-talk voice note. Native attachments has separate Photo and Document picker controls configured by the owner. computer_media_send defaults to photo for images and document for other formats; an unconfigured photo route fails clearly instead of silently sending a document. presentation=document explicitly retains file delivery. Recipient, immutable file/hash, preview identity, empty draft and single-submit safeguards remain mandatory. A custom preview that cannot prove file identity is rejected, not guessed from a screenshot. Canvas agent keeps the existing autonomous tools and unrestricted shell; app policy cannot isolate or audit actions bypassing its bridge. Restricted conversation is an opt-in separate, tool-free Codex 0.154.x inference with owner policy, only this contact's messages/memory, no development transcript, shell, patch, browser, plugins or MCP. It can answer text but does not run schedules or generate/send media. Unsupported provider/version/WSL stops rather than falling back to the free terminal. This limited mode is not a replacement for the full autonomous agent. OS permission and native-app acceptance still require local verification. WhatsApp on macOS can omit filenames from photo previews. For that specific preview, Orkestrai requires a fresh native picker receipt for the exact staged path, app and window, then checks the authorized recipient, a single photo, one-item count and empty caption again at Send. A receipt from another file/window or an unverified preview is rejected. This does not relax Document checks or turn submission into delivery confirmation. A deliberate no-reply decision must also resolve its dispatched inbox batch. After reading every message, the agent uses computer_inbox_acknowledge with its exact batchId/inReplyToDigest and reason already_answered or no_response_needed. The decision is audited as skipped with sent=false, not as a delivered response. Pending arrivals are preserved and can run immediately; another contact/task, an uncertain send or an answered batch cannot be cleared this way. Merely saying in the terminal that no reply is needed leaves the queue pending. In Computer > Conversation replies, enable Temporary focus for sending and Find and reopen this contact automatically to authorize native conversation navigation on macOS. If another chat is selected, the observer locates the exact approved recipient through native search and verifies its conversation header before reading messages. Reply preflight repeats this check if selection changed while the agent prepared its response. computer_open_conversation exposes the same bounded operation to the assigned agent; no manual search or screenshot polling is required. Navigation never types in a message composer or sends, and is audited separately. Ambiguous duplicate contacts, unsupported native search controls, revoked grants and unverified headers remain blocked rather than choosing someone else. Background monitoring still requires the host awake and the approved application running. Messages already captured in the conversation inbox remain actionable after restart or reopening the chat, even if the app only renders recent history. The assigned agent replies using the original batch id and digest; it does not need to scroll back to rediscover saved questions. The live authorized contact and composer are still verified before publication. Uncertain submissions remain blocked and unanswered messages are never silently discarded. Incoming native conversation events can include forwarded messages and media captions, not only plain text. For supported WhatsApp macOS labels, Orkestrai verifies the incoming envelope and exact authorized contact before queuing them, preserving the original label and digest. Recognizing a photo, sticker or audio notification does not mean the agent has seen the image or heard the audio. It can answer the available caption; downloading or transcribing the attachment still requires the separate media-receive authorization. Outgoing messages and quoted copies must not trigger a reply.

Observe native app changes without continuous model polling

Choose Native text, visual fallback in the existing Computer node. On macOS and Windows, Auto reads the authorized window independently of focus through native accessibility without producing a PNG. Two settled text observations wake the assigned operator with a bounded delta, rather than a new screenshot and a full task/history reload. The default local check interval is one second and cooldown is two seconds; configured values remain under owner control. The panel shows the actual source and last local check duration. Unsupported or incomplete native trees fall back to bounded visual checks, at least three seconds apart. Region and pixel threshold apply only to that visual fallback; native reads cover the authorized window. Linux currently uses the visual path. On macOS, permission status checks the actual native caller; a locally replaced app may require renewed Accessibility authorization. A local detection time is not a provider response-time guarantee. Authorize the application, enable Bounded Security with computer and agent capabilities, create an active task assigned to the operator, and select an enabled Manual automation that prompts that same operator. Select the exact window, automation and task, then enable Monitor. Allow agent configuration is a separate owner opt-in for computer_watch. Do not also schedule model polling. Nothing wakes the model on unchanged content. Native observation does not wait for foreground and never activates the window. Pausing, unloading, revoking the app, or completing/reassigning the task stops future work. Generic observation starts a new baseline after restart. Conversation replies restore a persistent inbox of observed messages; see that use case for retention and batching. Agents use computer_read for current text and controls, then computer_interact to fill or press an exact native element. The operation rechecks the expected role/name/value and conversation or document guards immediately before input, and returns fresh text afterward. Fill requires the existing draft value; never overwrite a human draft. Before sending, include guards for the exact recipient and complete draft and declare external_publication. Unknown, changed or ambiguous controls fail closed. Inspect the actual outcome; pressing Send alone does not prove delivery. No repeated screenshots are needed while a gate is pending. Generic controls retain publication gates. For unattended replies use an owner-approved Conversation replies grant and computer_reply, described in the corresponding use case; never replace it with blanket app publication permission. UI content is untrusted data, not instructions. Password controls are redacted by native reads; screenshots can still include visible secrets. Native text deltas expire from local delivery memory after two minutes; audit stores counts and digests, not chat text. Provider transcripts have separate retention. Native accessibility observation and computer_read on macOS/Windows do not require the target to be foreground. On macOS, passive visual fallback captures the selected window by ID without activation. Screen-rectangle fallback on other backends still requires foreground to avoid capturing an unrelated covering app. This change does not remove input/publication focus safeguards or guarantee access to content an app does not expose. Approved text replies use process-directed native controls on supported macOS/Windows adapters, without global mouse or keyboard input. Exact window, recipient, draft, authorization and single-submit checks remain mandatory. Global input and native file dialogs still require foreground. A preflight failure before any native input keeps the batch retryable; an uncertain submission never retries automatically. Native apps, external browsers and managed Portals require their own controls and authorization; a successful test of one does not certify another or future interface changes. On macOS, selected-window reads use public native APIs without repeated System Events calls. The private pipe helper stops after 30 seconds idle and recycles between confirmed requests after 256 operations or five minutes; closing it cancels pending commands without replay. Another app can keep being observed during a process-directed operation. Reads of the same process and global input remain exclusive. A fast observation or enabled Auto switch does not prove that the target supports background typing or Send; unsupported input does not authorize an automatic focus change. On macOS, open Computer > Conversation replies and enable Temporary focus for sending for the exact contact. It is off by default and only the owner can change it. computer_reply and computer_send may then bring the approved window forward for guarded text composition and submission. Recent typing, clicks or held modifiers delay admission. The previous application/window is restored unless the user switched elsewhere; closed or replaced targets are not guessed. A restoration failure is audited without replaying the message. Observation remains in background. This opt-in covers text sending, not file dialogs or unrestricted desktop control; existing permissions, publication gates and uncertain-send recovery remain enforced.

• Use temporary screenshot retention for operational checks and evidence for important milestones. Temporary images expire after 15 minutes and are capped at 32 MiB / 64 files per workspace. Evidence defaults to 14 days and 256 MiB, with controls in the same node and a 1,500-file cap. All registered Computer workspaces share a 2 GiB / 5,000-file cap. Oldest images are deleted first; links then return unavailable. Cleanup runs every five minutes while Core is running and before/after retained captures, including when monitoring is paused. Below 1 GiB free, capture stops before writing. Cleanup never follows storage symlinks or touches creative image assets. Audit metadata and provider conversation copies have separate retention; these settings do not delete those records.
• computer_batch accepts steps: [{input: {command: 'type', targetId: '<window>', text: '<complete text>'}}, {input: {command: 'screenshot', target: 'window', targetId: '<window>', retention: 'temporary'}}]. Use a stable batch idempotency key and taskId; up to 12 steps are validated before any effect. Inspect completed and each step: failed or gated stops subsequent steps and includes gateId. Retry an approved gate only after inspecting the current target, with the same key; completed steps are not repeated. A potentially partial failure requires inspection and a deliberate new action. Never combine composition and publication without first verifying the recipient and content.

Interactive, on-demand, and persistent agents

Open an agent terminal menu and choose Agent runtime. Interactive preserves the existing Canvas lifecycle. On demand wakes through a human message or durable automation, resumes the exact provider conversation, and sleeps after the configured idle period when it has no active run or assigned work. Persistent is supervised by the background Core and restarts after process failure or system sleep without requiring a rendered Canvas. Per-agent safeguards cap concurrent automation runs and pause automatic starts when a known provider quota reaches the chosen threshold; a manual wake always remains available. The runtime, last wake/sleep, errors, active run count, task, and session state use the same persisted node and Control Center history across Canvas, Workbench, and Remote. Existing agents remain Interactive until explicitly changed.

Autonomy policy, approvals, and encrypted credentials

Open Automations → Security to define one standing grant for unattended work. Observe records activity without restricting the current shell; Prepare and Ask before mutations pause writes; Bounded autonomous lets agents keep a free shell and execute brokered actions inside approved capabilities, workspace roots, hosts, operating hours, and concurrency limits. High-risk boundaries such as force push, production deploy, purchases, external publication, account changes, bulk deletion, and access outside the grant can be pre-approved or require the workspace owner, a named reviewer, or Council consensus. Emergency stop disables every routine and aborts active runs immediately. The Vault stores values through the operating-system encrypted store and gives agents only a SecretRef bound to explicit integrations, operations, and destinations. OAuth or an interactive Portal login is completed once by the user; later jobs receive a credential only inside the trusted connector executor. Every brokered action is exact semantic evidence in a hash-chained audit export. Free-shell effects remain labeled as inferred because Orkestrai does not pretend to provide syscall-level observation.

Automations

Open Automations from the Canvas toolbar, the Workbench explorer, or Command/Ctrl+K. A trigger can be manual, scheduled, a task change, a confirmed agent message, a Git commit, a GitHub pull request, a webhook, a file or folder change, or a provider usage threshold. Actions send a prompt to one agent, create a traceable Kanban task, or show an explicit desktop notification. Development, design, marketing, research, and operations recipes provide safe starting points. Every run records trigger input, target agent/provider, quota snapshots, output acknowledgement, duration, attempt, and recoverable failure; retries are bounded and duplicate event deliveries are idempotent. Settings → Autonomy & 24/7 Core can keep the local Core and active work in the system tray after every window closes, start it silently when you sign in, show its live uptime, and restart it without restarting the whole desktop app. Choosing Quit Orkestrai remains the explicit stop. GitHub tokens are encrypted by Electron safeStorage and never stored in the workspace database. Legacy scheduled Routines remain compatible and appear here automatically. 24/7 means the enabled Core waits for future events or schedules, not that a model thinks continuously. For example, schedule an inbox check every 15 minutes, prepare a report, and gate sending it. Each run finishes and the next trigger starts another. Keep the computer awake, Core active, credentials valid, and provider quota available; sleep and shutdown stop local execution. Closing a window differs from Quit Orkestrai. Verify run history and approvals rather than assuming that a persistent terminal guarantees completed work.

Durable automation execution

Every automation trigger is written to the workspace database before execution. The Core claims it with a renewable lease, stores checkpoints, and prevents two workers from running the same delivery. If Orkestrai, the computer, or a queue backend stops, an expired lease returns to the queue automatically. Transient failures use bounded exponential backoff; the final failure moves to Needs intervention instead of looping forever. A prompt action starts or resumes its target agent when no PTY is open. Execution history shows the current attempt, next retry, cancellation, output, and dead-letter state; queued and running work can be cancelled, and final failures can be replayed as a new traceable run. Suspended workspaces do not enqueue work.

Integration Center and encrypted accounts

Open Automations -> Integrations to connect Gmail, Slack, Telegram, WhatsApp, GitHub, or an HTTPS webhook. Gmail uses the official system-browser OAuth flow with PKCE; the other providers accept their revocable bot or app token directly into the operating-system encrypted Vault. The workspace database stores only a SecretRef and bounded account metadata. Choose the exact operations that account may perform and, when applicable, its default channel, chat, recipient, repository, or endpoint. Agents never receive the resolved value: integration_list exposes the account manifest, while integration_execute resolves the SecretRef only inside the trusted connector, requires the agent's active assigned task, applies the workspace autonomy policy, and records a sanitized idempotent event. Supported actions cover Gmail search/read/draft/send/labels and workspace attachments, Slack channels/messages, Telegram messages/updates/workspace documents, WhatsApp messages/document links, GitHub pull-request reads, and typed webhooks. Use a persistent or on-demand agent plus a durable Automation to classify an inbox, prepare a report or PDF in the workspace, and deliver it on schedule. Use a Managed Portal instead when a service has no connector; complete its login yourself so cookies and passwords remain outside prompts. In Automations > Integrations > Activity, Accepted by provider means a provider message ID or an HTTP acceptance was received, not that the recipient read or received the message. Expand Provider receipt to inspect the returned IDs. Send unconfirmed means publication may have occurred before its response was lost; inspect the destination before taking any further action, and never invent a new retry key. The request remains blocked across restarts. A failure confirmed before publication may retry the same unchanged request and key after its authorization or configuration is repaired. Historical records without a request digest cannot be replayed automatically. An idempotency key cannot be reused for different text, recipients, attachments or operations. Native Computer sending has its own independent confirmation and recovery path.

Visible browser workflows and automatic tool publication

Sign in privately, or ask the agent to use an account authorized for this site. Password inputs marked writeOnly accept portal_type over HTTPS or loopback with same-origin forms. The agent submits the login and verifies the authenticated page in the same Portal. Passwords remain redacted; OTP, CAPTCHA, private fields and explicit security gates still require their existing authorization. Credentials sent in chat can remain in provider transcripts; use test accounts or sign in manually for private credentials. New and previously unconfigured Portals allow the whole workspace team to read and interact, including newly recruited agents. In the gear, choose All workspace agents or Selected agents only. Existing explicit lists, Manual only and Read only settings are preserved. The agent controls the same native page and tabs visible in Canvas or Workbench; opening another view does not create a second browser session. Visible control is the default. Enable background control explicitly to continue with that Portal hidden, and use Pause to stop further brokered actions immediately. Normal Portal use does not require enabling Automations > Security or assigning a task. An optional taskId must belong to the authenticated agent. Explicitly enabled enforcing policies still apply their boundaries and risk gates; disabled or Observe policies preserve interactive behavior and audit it. Emergency stop always applies. Password, token, OTP, and marked-private fields are withheld from semantic snapshots and DOM reads and masked in captures; arbitrary agent JavaScript is disabled. Re-snapshot after changes instead of reusing stale element references. In Security, enable Agent-created tools only for selected agents and executor types in Bounded mode. Valid fixtures, limits, existing Portal or integration permissions, and destination-bound SecretRefs are required. Commands still require owner publication. Browser tools bind to a Portal id and use ordered typed steps with exact semantic role/name targets and {{input}} templates. Gate approvals resume the same tool run from saved completed steps using its original revision and idempotency key; do not invent a new key for a retry. An interrupted effect whose outcome is unknown is not automatically replayed. Audit expands operation, actor, target, risk decision, safe output, and correlation details and refreshes without overwriting policy edits. Emergency stop remains latched even if the policy switch is turned off; prepare and save an explicit resume. Webhook triggers live in Automations > New > Webhook; after saving, reopen the editor to see the complete local endpoint and send X-Orkestrai-Webhook-Secret or Bearer authentication. Outbound webhooks are configured separately in Integrations. Internet ingress needs a private tunnel or a proxy limited to the webhook route, never public access to the entire Core. Browser permissions are broker controls, not an OS sandbox: a free shell, another browser, or unmediated program can act outside this audit. The app must remain running and the computer awake for local 24/7 work.

Tool Workshop: reusable capabilities without exposed credentials

Tool manifests declare schemaVersion, executor, inputSchema, outputSchema, capabilities, secretRefs, timeoutMs, maxOutputBytes and fixtures. Executors are browser, transform, http, integration, and workspace_command. Browser steps use {action, target: {role, name}, args}; targets must match exactly one element in a fresh snapshot. Templates resolve declared input fields and previous steps. Dry run validates structure and input without performing external effects; it is not a live endpoint test. Run controlled fixtures before enabling unattended effects. A saved revision is immutable, the published revision stays active across later drafts, and rollback creates a new draft. Automatic publication requires the explicit standing grant described above; workspace commands always require owner review. SecretRefs bind to tool:<slug>, tool integration, and the exact destination. Resume waiting runs with the same idempotency key; unknown interrupted effects require inspection before a new run.

Computer node and bounded desktop control

Tell an existing agent, locally by text or dictation or through Remote: "Open Calculator and calculate 73 times 19; record and verify the result." The agent creates a briefing note and an active task assigned to itself, calls computer_prepare to create/reuse and connect the Computer node, and inspects the host computer. Preparation inherits only an existing enabled Bounded grant with the computer capability and exact allowedApps; it never widens access or reactivates a paused node. Otherwise authorize the required app and enable control yourself. Grant macOS Accessibility, Screen Recording and Automation when requested. computer_launch focuses an existing app window, preserving its session, or opens its registered app; then use wait/inspect, window-bound input and screenshot, inspect the real result, update the note and close the task. Example launch IDs: com.apple.calculator or com.google.Chrome on macOS; CalculatorApp or chrome on Windows; a registered .desktop ID or matching WM_CLASS on Linux X11. App authorization is possible by ID even before the app opens; closed authorized apps stay visible. Agent input/captures require an allowed window and a live assigned task. Use a stable idempotencyKey per exact action; running or possibly partial failed actions are not automatically replayed. Inspect first and create a new key only for a deliberate new action. Desktop input is serialized across workspaces and focus is checked before typing. Declare risk external_publication before sending email or posting, purchase before buying, and the appropriate destructive/credential risk; enforcing Security gates bind approval to the exact request and attempt. Pixel-only desktop operations cannot infer every business risk or browser origin: this is not a shell sandbox, nor automatic password masking. Prefer Portal or Integration Center when a structured contract is available. SecretRefs must bind computer.type_secret and the exact app; values travel through protected stdin and are not returned to the agent or audit. Linux secret typing is explicitly unsupported. Captures under .orkestrai/computer/evidence reject symlink escapes, refresh in the node, and remain read-only evidence; visible sensitive content can appear in a native screenshot. Remote sends the request to the existing host agent, not a new phone browser; the host must be awake with Core, an interactive desktop and required grants available.

Managed Portal automation

The Portal gear selects a workspace-shared or Portal-private browser profile, allowed hosts, and a download folder confined to the workspace. Sign in directly in the visible Portal: cookies and passwords never enter prompts, while the Core reuses the protected profile with the Canvas closed. Agents call portal_snapshot for semantic references and then use typed tabs, navigate, click, type, select, upload, download, wait, extract, and screenshot operations by unique Portal name or id. Host changes are denied unless allowlisted, files cannot escape the workspace, and bounded results plus every navigation or mutation enter Control Center with actor, Portal, profile reference, outcome, and correlation data. Arbitrary agent scripts are refused; use typed operations with explicit Portal grants. Background control requires a separate opt-in. Automations can schedule the same typed operations durably.

• Canvas zoom scales the page visually without changing its responsive viewport. Resize the node or use Test responsiveness to change the actual CSS dimensions.
• Menus and Canvas controls take precedence over the live page. During motion, open overlays or left/top origin clipping, an in-node preview preserves visual placement; moving the page back into view restores the same interactive session without reloading.

Installation

Download the latest release for macOS, Windows, or Linux. The desktop app includes the server and runtimes required to operate the canvas.

• Install at least one supported CLI: Claude Code, Codex, Kimi Code, OpenCode, Cursor, Antigravity, Cline, Devin, or GitHub Copilot.
• On Linux, use the AppImage on any supported distribution or the RPM package on Fedora, RHEL, CentOS, and compatible systems.
• On macOS, use version 0.1.4 or newer: it is signed with Developer ID and notarized by Apple, so it opens normally through Finder after being copied to Applications.
• Keep Git available for history, diff, and floor features.
• Local voice models are optional and download only after your confirmation.

Your first workspace

A workspace points to a real project folder. Orkestrai keeps visual configuration separate and provisions the local bridge agents use to operate the canvas.

Canvas and connections

The canvas brings together real PTY terminals, notes, tasks, workflows, images, portals, and files. Connections represent actual collaboration, show activity when agents talk, and automatically simplify physics outside the viewport or in dense workspaces.

• Connect notes and images to their owners to make context explicit.
• Assign work through the kanban; each task goes directly to the chosen terminal.
• Open Portals to view and test local applications without leaving the workspace; links requesting a browser tab create another Portal node in the same Canvas.
• Use Cmd/Ctrl+K to search the in-app documentation.

Native image generation workflows

Build reusable creative graphs directly on the canvas. A workflow combines its prompt with connected briefs, up to five ordered image references, and a live Codex executor.

• Drag one or several images from Finder or File Explorer onto empty Canvas to create a separate Image reference for each file, without changing the original image. Imports accept up to 100 images, 10 MB each, and place them without overlap. Drop files onto an agent header, terminal body or quick prompt to attach workspace file references to that prompt instead, up to 12 per drop. Review the prompt and send it yourself; dropping does not press Enter or start generation. Invalid files are reported, and successfully imported references stay visible.
• An authenticated Codex account or plan with ImageGen available is required; no OpenAI API key is needed.
• Request one to ten outputs in one logical run; Codex automatically performs the native image_gen.imagegen calls required by its tool contract.
• Choose exact Instagram, Stories/Reels, TikTok, or custom delivery dimensions. Measurable safe areas protect important content, native masters remain untouched, and an incompatible aspect ratio returns to ImageGen for safe recomposition instead of being cropped.
• Transparent outputs are decoded and checked for real alpha pixels. With multiple references, ImageGen first composes on a uniform white matte and removes it in a second native edit; a proven checkerboard-cutout prompt provides another native fallback. Python and local pixel manipulation remain forbidden.
• A connected Codex can create or revise drafts, attach, remove, or reorder inputs, run or cancel generation, and remove the workflow through typed tools.
• Orkestrai never asks for or stores an image API key and never calls an image provider endpoint directly.
• Every validated result is written to its preallocated workspace path and returns as a connected Image node with bounded history and provenance.
• Canvas, Workbench, CLI, and MCP operate the same persisted workflow without parallel automation state.
• Canvas-started Codex terminals use the MCP from the current Orkestrai installation as a session-only override without rewriting user configuration.

Native API Client

Create, import, and test HTTP/REST, GraphQL, WebSocket, and gRPC in the same workspace as the implementation, without moving routine API work to another application.

• Use assisted OAuth 2.0, Bearer, Basic, or API-key authentication with a cookie jar, proxy, custom CA, client certificates, and TLS controls.
• Edit JSON, JavaScript, GraphQL, and XML with syntax color, search, formatting, and runtime-aware completion; explore JSON/XML response trees and WebSocket/gRPC transcripts.
• Synchronize linked Bruno and OpenCollection sources with pull, push, watch mode, and explicit conflict resolution; Postman and OpenAPI remain pull-only.
• Import Bruno, OpenCollection YAML, Postman v2.1, Swagger 2.0, or OpenAPI 3.x; export Bruno, OpenCollection, Postman, or OpenAPI 3.1 JSON/YAML with explicit fidelity notes.
• Import and export Postman environments independently, or back up the complete native state with the versioned Orkestrai format.
• Connect the node to agents and leads so they can safely create or edit complete collections, run requests and runners, and export Bruno or Postman through typed MCP and CLI tools without exposing stored secrets.

Workbench and editor

Workbench turns the same workspace into a focused desktop surface. Its explorer opens agents, tasks, notes, portals, mobile devices, and workspace files in up to eight resizable panes.

• Choose vertical or horizontal tabs and split any pane to the right or below.
• Edit workspace files in Monaco with persistent models, find and replace, symbols, formatting, and protected dirty buffers.
• Open Markdown, PDF, image, and binary previews without creating disconnected canvas nodes.
• Use Cmd/Ctrl+K to search files, content, agents, tasks, tools, settings, and commands.

Operational code intelligence

The native graph connects indexed code, live work, bounded evidence, and delivery actions without sending repository source to another service.

• Use Assisted mode for near-real-time incremental refresh and agent reads that wait for the freshest known revision; Manual reserves explicit indexing for you, while Disabled stops watchers and access without deleting history.
• Build a traceable context package with an explicit 500 to 16,000 token budget, review its exact symbols, relationships, tests, findings, and redacted excerpts, then hand it to the leader, an available agent, a Council, or the task board.
• Inspect live agent, task, and Floor ownership on the graph and surface overlapping paths or symbols before parallel work collides.
• Select a relationship to see whether it is static, inferred, or runtime evidence together with provenance and confidence.
• Compare persisted code and relationship revisions, and save named investigations that restore the repository, mode, filters, selected symbol, source location, and graph camera.
• Graph selection and Monaco remain synchronized, while CLI and MCP expose the same context, operations, explanation, revision, comparison, and investigation contracts.

Native Design Studio and code delivery

Design documents stay editable in Orkestrai while people and agents use the same structured scene graph. The Code area connects visual work to the actual project without making a disconnected export.

• Import HTML/Tailwind, Svelte, React/JSX, or Vue structure as editable native layers without executing project scripts.
• Generate Svelar/Svelte 5, React, Next.js, Vue 3, or HTML/Tailwind only after reviewing the complete file.
• Reuse compatible Code Connect mappings and open every linked artifact directly in Monaco.
• Compare a selected frame against a live Portal or attached mobile device through normalized reference, implementation, overlay, and pixel diff views.
• Turn visual evidence into a Kanban task or Review Center entry tied to the actual Git change.

Interactive prototypes and native motion

Prototype flows, interactions, presentation, and animation remain inside the same native Design document used for manual editing, Figma interoperability, agents, and delivery.

• Create multiple starting flows and use click, press, hover, or timed triggers for navigation, overlays, back, scrolling, and variable modes.
• Preview transitions, hotspots, fixed layers, horizontal or vertical overflow, device framing, and fullscreen in the focused player.
• Share a self-contained read-only HTML prototype without exposing the workspace.
• Build reusable duration and easing tokens, layer tracks, and keyframes, then copy CSS animations or Motion.dev code.
• Use the same revision-safe command bus from the editor, universal search, Orkestrai CLI, or typed MCP agents.

Live Design collaboration

People and connected agents review the same revisioned Design document with visible presence and explicit decisions instead of exchanging disconnected screenshots.

• See participant cursors, selections, pages, and follow mode while reviewing together.
• Anchor versioned comment threads, replies, mentions, and resolution to a page or layer.
• Preview agent proposals and inspect their structural diff before an explicit human approval applies them atomically.
• Use short layer leases to block conflicting writes without locking the entire document.
• Move uncertain work into Council or an isolated Floor, and grant Remote devices independent View, Comment, Propose, or Edit permissions over sanitized summaries.

Design quality, templates, and recovery

The native document carries production checks and recovery controls beside the same layers people and agents edit.

• Audit generic or duplicate names, clipped text and content, unexpected overlap, WCAG contrast, and missing accessibility metadata.
• Select any issue to jump directly to the affected layer.
• Start product, marketing, mobile, or design-system work from complete editable native templates with variables and semantics.
• Recover a damaged document from its automatic backup, inspect recent revision history, and compact bounded history explicitly.
• Large documents render incrementally around the viewport while the full document remains available to search, agents, export, and audit.

Leader and Orkestrai bridge

The leader coordinates the team through the orkestrai CLI and MCP tools provisioned in the workspace. It can talk to agents, write notes, create tasks, open portals, and organize floors.

orkestrai list
orkestrai ask Frontend "Implement the screen from the spec note"
orkestrai task add "Review responsiveness" --assign Reviewer
orkestrai notify "Delivery ready for review"

Workflows and parallel work

Workflows chain agents, human approvals, and reusable outputs. Floors use git worktrees to isolate parallel efforts and preview diffs and conflicts before landing on the main branch.

• Sync a workflow with its connections to turn the drawing into a pipeline.
• See each task title, stage, and assignee for ground and every worktree.
• Create one floor per feature when teams need to change the same project in parallel.
• Review the preview before landing; conflicts are never resolved silently.

Dictation and spoken replies

Dictation uses Parakeet and replies use Supertonic 3, both locally. Presets are available for Brazilian Portuguese, US English, and Latin American Spanish. While the microphone is opening or transcribing, click its Cancel control to stop the attempt. Late responses cannot open the microphone or insert text afterwards. Startup is limited to 15 seconds and transcription to three minutes; recording automatically stops and transcribes after 15 minutes. Cancelling discards that attempt without sending text to the agent.

• Focus any task, role, note, or form field and use the global orb to insert dictation at the cursor.
• Optionally submit terminal dictation with Enter; regular text fields remain insert-only.
• With no active field on the canvas, the orb sends the transcript to the leader.
• Choose and test the microphone and speaker, then set voice and speed from 0.75x to 1.50x in Settings.
• Removed audio devices fall back to the system default, with distinct guidance for permission, missing hardware, interrupted capture, or likely input contention.
• The first download requires confirmation; processing then stays on-device.

Operations and continuity

Workspaces keep running in the background. Resumable sessions preserve each terminal conversation, while native notifications distinguish task completion, project completion, and required attention.

• Change an agent provider in place while preserving its role, floor, and connections.
• Pin Usage to the canvas and let the leader consult its source, fallback, monitored window, and threshold before assigning new work.
• Ports lists only listeners tied to local Portals in the current workspace.
• Choose a built-in light or dark theme, or edit and share validated semantic color tokens.
• Canvas data, sessions, and local models remain on your machine.

Decisions, automation, devices, and sharing

Control Center and Review Center make activity and delivery evidence explicit. Council, Automations, Mobile Device, and encrypted workspace sharing extend the same traceable operating model.

• Ask two to five agents for independent Council perspectives, then keep the final selection human.
• Run idempotent automations from schedules, tasks, messages, Git events, webhooks, file changes, or usage limits.
• Control iOS Simulator and Android emulators or approved physical devices from Canvas and Workbench.
• Choose a browser/mobile or installed-app invitation, approve the exact device and role, and follow the sanitized workspace projection in the installable Remote PWA.
• Remote keeps every area reachable on phones and shows each agent's current focus, recent semantic activity, and coordination totals without exposing internal message bodies.
• Operators can hold a traceable agent conversation. Administrators can start or restore agents, while raw terminal control requires a separate per-device switch that is disabled by default and audited.

Coordination, attention, and durable context

Orkestrai turns delivery signals into a connected operating history instead of scattering them across terminal output, cards, and transient notifications.

• Durable message receipts and semantic activity preserve who sent what, its delivery state, outcome, source, and correlation across restarts.
• Attention Center and Universal Search surface unresolved questions, permissions, failures, messages, and activity across workspaces with structured filters.
• Workstreams connect a Kanban task to its agent, Floor, branch, Council, Git revision, review, evidence, tests, risks, and linked files without duplicating records.
• Workspace Memory stores sourced decisions, constraints, facts, preferences, references, and lessons with immutable revisions and conflict protection.
• Annotation Center unifies code-review and Design comments against their canonical revisions and flags feedback that became stale.
• Versioned Team Packs preserve reusable roles, skills, notes, and team structure with release notes, integrity checks, and no leaked runtime state.
• Persistent Huddles let a person address selected agents, dictate turns, recover the transcript, create a linked task, and continue securely from the Remote PWA.

API Client scripts and tests

A practical reference for the official Postman Runtime, Bruno’s official safe QuickJS runtime, and native Orkestrai declarative tests.

• Execution order includes collection, folder, and request pre-request scripts, the network call, request, folder, and collection post-response scripts, then native assertions.
• Postman keeps pm.globals, pm.collectionVariables, pm.environment, pm.iterationData, and pm.variables separate. Bruno exposes the equivalent environment, global, collection, runtime, secret, and runner iteration APIs.
• Postman supports sendRequest, runRequest, collection flow, cookies, vault, visualizer, legacy globals, pm.require for bundled libraries, accurate iteration metadata, and bundled Chai. Bruno supports sendRequest, runRequest, req/res, request and folder variables, post-response variables, declarative assertions, tests blocks, cookies, runner flow, visualizations, bundled libraries, and global test/expect/assert.
• Imported JavaScript executes unchanged in its selected source runtime. Postman team Package Library, hosted datasets, mocks, and cloud-owned state still require Postman services because they are not part of a portable collection file. Bruno stays in its official safe QuickJS runtime, with unsafe NodeVM access to the host filesystem, processes, and arbitrary local modules deliberately disabled.

const requestId = 'req-' + Date.now();

const tenant = pm.iterationData.get('tenant');

pm.variables.set('requestId', requestId);
pm.globals.set('lastTenant', tenant);
pm.request.headers.upsert({
  key: 'X-Request-Id',
  value: requestId,
});

pm.vault.get('apiKey').then((apiKey) => {
  pm.request.headers.upsert({ key: 'X-API-Key', value: apiKey });
});

console.log('Request prepared:', requestId, tenant);

let body;

pm.test('Status is 200', () => {
  pm.expect(pm.response.code).to.equal(200);
});

pm.test('Body is valid JSON', () => {
  body = pm.response.json();
});

if (body) {
  pm.test('Response contains access_token', () => {
    pm.expect(body).to.have.property('access_token');
  });

  pm.test('Response contains a user id', () => {
    pm.expect(body).to.have.property('user');
    pm.expect(body.user).to.have.property('id');
  });

  if (body.access_token) {
    pm.environment.set('accessToken', body.access_token);
  }

  if (body.user?.id) {
    pm.environment.set('userId', body.user.id);
  }

  pm.execution.setNextRequest('Load user');
  console.log('Authenticated user:', body.user?.id);
}

GET {{baseUrl}}/users/{{userId}} HTTP/1.1
Authorization: Bearer {{accessToken}}
X-Request-Id: {{requestId}}

const login = await bru.runRequest('Auth / Login');
const token = login.data.access_token || bru.getVar('accessToken');

if (!token) {
  throw new Error('The accessToken variable is missing');
}

req.setHeader('Authorization', 'Bearer ' + token);
req.setHeader('Accept', 'application/json');

console.log('Authenticated request');

const body = res.getBody();

test('User was created', () => {
  expect(res.getStatus()).to.equal(201);
  expect(body).to.have.property('id');
});

bru.setVar('createdUserId', body.id);
bru.setVar('lastStatus', res.getStatus());
bru.setNextRequest('Load user');

console.log('Created user:', body.id);

Source          Path             Operator       Expected
Status          —                Equals         200
Body            data.user.id     Exists         —
Header          content-type     Contains       application/json
Response time   —                Less than      1000

Collection variable: baseUrl = https://api.example.com
Environment variable: accessToken = <active environment token>
Script-created variable: userId = 42

URL: {{baseUrl}}/users/{{userId}}
Header: Authorization = Bearer {{accessToken}}

Visible browser workflows and automatic tool publication

Watch an agent work in your signed-in browser

• Portal agents now operate the same visible native page with named-agent grants, read/interact modes, explicit background opt-in, immediate pause, protected-field masking, stale-reference checks, and no arbitrary scripts. Preauthorized agents can publish bounded browser/integration/HTTP/transform tools; gate resumptions preserve checkpoints and the original revision. Audit details, local webhook setup, and latched emergency stop are explicit in the UI.
• Tool manifests declare schemaVersion, executor, inputSchema, outputSchema, capabilities, secretRefs, timeoutMs, maxOutputBytes and fixtures. Executors are browser, transform, http, integration, and workspace_command. Browser steps use {action, target: {role, name}, args}; targets must match exactly one element in a fresh snapshot. Templates resolve declared input fields and previous steps. Dry run validates structure and input without performing external effects; it is not a live endpoint test. Run controlled fixtures before enabling unattended effects. A saved revision is immutable, the published revision stays active across later drafts, and rollback creates a new draft. Automatic publication requires the explicit standing grant described above; workspace commands always require owner review. SecretRefs bind to tool:<slug>, tool integration, and the exact destination. Resume waiting runs with the same idempotency key; unknown interrupted effects require inspection before a new run.