Skip to content

feat: agenstra live browser preview #515

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
me-marcel wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

feat: agenstra live browser preview #515

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 23 additions & 2 deletions apps/agenstra/backend-agent-manager/Dockerfile.vnc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,10 @@
# Build: npx nx run backend-agent-manager:vnc-container-image
# Run: nx docker:run agenstra-backend-agent-manager -p 6080:6080
# Notes: Required runtime env: VNC_PASSWORD
# Optional: VNC_DISPLAY, VNC_RESOLUTION, VNC_DEPTH
# Optional: VNC_DISPLAY, VNC_RESOLUTION, VNC_DEPTH, BROWSER_PREVIEW_ENABLED
# When BROWSER_PREVIEW_ENABLED=true, Chromium CDP is available on internal port 9222 via socat
# (Chromium binds loopback :9223; 9222 is not published to the host)

# Agent workspace mount path: /home/agenstra/environment
# -----------------------------------------------------------------------------
#
Expand Down Expand Up @@ -40,6 +43,7 @@ RUN apt-get update && \
supervisor \
net-tools \
openssl \
socat \
novnc \
websockify \
nano \
Expand Down Expand Up @@ -78,9 +82,22 @@ RUN mkdir -p /home/agenstra/.config/xfce4/xfconf/xfce-perchannel-xml && \
> /home/agenstra/.config/xfce4/xfconf/xfce-perchannel-xml/xfce4-settings.xml && \
chown -R agenstra:agenstra /home/agenstra/.config/xfce4

RUN sed -i 's|Exec=/usr/bin/chromium %U|Exec=/usr/bin/chromium --no-sandbox --disable-dev-shm-usage --display=:1 %U|g' \
RUN sed -i 's|Exec=/usr/bin/chromium %U|Exec=/usr/local/bin/chromium-launch %U|g' \
/usr/share/applications/chromium.desktop

# Chromium CDP only binds loopback; expose it on the container network via socat (see entrypoint).
RUN printf '%s\n' \
'#!/bin/bash' \
'set -euo pipefail' \
'CHROMIUM_FLAGS=(--no-sandbox --disable-dev-shm-usage --display=:1 --window-size=1910,865 --window-position=0,0 --start-maximized)' \
'if [ "${BROWSER_PREVIEW_ENABLED:-}" = "true" ] || [ "${BROWSER_PREVIEW_ENABLED:-}" = "1" ]; then' \
' CHROMIUM_FLAGS+=(--remote-debugging-port=9223 --remote-allow-origins=*)' \
'fi' \
'exec /usr/bin/chromium "${CHROMIUM_FLAGS[@]}" "$@"' \
> /usr/local/bin/chromium-launch && \
chmod 755 /usr/local/bin/chromium-launch && \
chown agenstra:agenstra /usr/local/bin/chromium-launch

RUN printf '%s\n' \
'#!/bin/bash' \
'set -e' \
Expand Down Expand Up @@ -145,6 +162,10 @@ RUN printf '%s\n' \
'fi' \
'vncserver "${VNC_DISPLAY}" -geometry "${VNC_RESOLUTION}" -depth "${VNC_DEPTH}" -localhost no -SecurityTypes VncAuth' \
'websockify -D --web /usr/share/novnc/ --cert "$HOME/.novnc/ssl/novnc.crt" --key "$HOME/.novnc/ssl/novnc.key" 6080 localhost:5901' \
'if [ "${BROWSER_PREVIEW_ENABLED:-}" = "true" ] || [ "${BROWSER_PREVIEW_ENABLED:-}" = "1" ]; then' \
' # Chromium listens on 127.0.0.1:9223; proxy to 0.0.0.0:9222 for manager CDP over the agent network.' \
' socat TCP-LISTEN:9222,fork,reuseaddr,bind=0.0.0.0 TCP:127.0.0.1:9223 &' \
'fi' \
'exec tail -f /dev/null' \
> /usr/local/bin/docker-entrypoint.sh && \
chmod 755 /usr/local/bin/docker-entrypoint.sh && \
Expand Down
2 changes: 2 additions & 0 deletions apps/agenstra/backend-agent-manager/docker-compose.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ services:
WEBSOCKET_NAMESPACE: ${WEBSOCKET_NAMESPACE:-agents}
WEBSOCKET_CORS_ORIGIN: ${WEBSOCKET_CORS_ORIGIN:-*}
NODE_ENV: ${NODE_ENV:-development}
# Used to join per-agent Docker networks for browser Preview (CDP). Defaults to this service's container_name.
MANAGER_CONTAINER_ID: ${MANAGER_CONTAINER_ID:-agent-manager-api}
# Cursor agent configuration
CURSOR_API_KEY: ${CURSOR_API_KEY:-}
AGENT_DEFAULT_IMAGE: ${AGENT_DEFAULT_IMAGE:-ghcr.io/forepath/agenstra-manager-worker:latest}
Expand Down
1 change: 1 addition & 0 deletions apps/agenstra/backend-agent-manager/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
"ssh2": "1.17.0",
"sshpk": "1.18.0",
"uuid": "11.1.1",
"ws": "8.18.0",
"zod": "4.3.6",
"@opentelemetry/api": "1.9.1",
"@opentelemetry/auto-instrumentations-node": "0.79.0",
Expand Down
31 changes: 31 additions & 0 deletions ...ckend-agent-manager/src/migrations/1780000000000_AddBrowserPreviewEnabledToAgentsTable.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
import { MigrationInterface, QueryRunner, TableColumn } from 'typeorm';

/**
* Adds browser_preview_enabled to agents.
* Existing agents with a VNC sidecar get preview enabled (VNC implies preview).
*/
export class AddBrowserPreviewEnabledToAgentsTable1780000000000 implements MigrationInterface {
name = 'AddBrowserPreviewEnabledToAgentsTable1780000000000';

public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.addColumn(
'agents',
new TableColumn({
name: 'browser_preview_enabled',
type: 'boolean',
isNullable: false,
default: false,
}),
);

await queryRunner.query(`
UPDATE agents
SET browser_preview_enabled = true
WHERE vnc_container_id IS NOT NULL
`);
}

public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.dropColumn('agents', 'browser_preview_enabled');
}
}
84 changes: 84 additions & 0 deletions apps/agenstra/frontend-agent-console/src/i18n/messages.de.xlf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1285,6 +1285,90 @@
<source>Create SSH Connection</source>
<target>SSH-Verbindung erstellen</target>
</trans-unit>
<trans-unit id="featureChat-enableBrowserPreview" datatype="html">
<source>Enable Browser Preview</source>
<target>Browser-Vorschau aktivieren</target>
</trans-unit>
<trans-unit id="featureChat-createBrowserPreviewHint" datatype="html">
<source>Stream and control Chromium in the virtual workspace without full desktop VNC access. Defaults to enabled. Enabling VNC always includes Preview.</source>
<target>Chromium im virtuellen Workspace streamen und steuern, ohne vollen Desktop-VNC-Zugriff. Standardmäßig aktiviert. VNC schließt die Vorschau immer mit ein.</target>
</trans-unit>
<trans-unit id="featureChat-openBrowserPreview" datatype="html">
<source>Open browser Preview</source>
<target>Browser-Vorschau öffnen</target>
</trans-unit>
<trans-unit id="featureChat-openVirtualDesktop" datatype="html">
<source>Open virtual desktop</source>
<target>Virtuellen Desktop öffnen</target>
</trans-unit>
<trans-unit id="featureChat-browserPreviewTitle" datatype="html">
<source>Browser Preview</source>
<target>Browser-Vorschau</target>
</trans-unit>
<trans-unit id="featureChat-browserPreviewClose" datatype="html">
<source>Close</source>
<target>Schließen</target>
</trans-unit>
<trans-unit id="featureChat-browserPreviewConnecting" datatype="html">
<source>Connecting to browser…</source>
<target>Verbinde mit dem Browser…</target>
</trans-unit>
<trans-unit id="featureChat-browserPreviewBack" datatype="html">
<source>Back</source>
<target>Zurück</target>
</trans-unit>
<trans-unit id="featureChat-browserPreviewForward" datatype="html">
<source>Forward</source>
<target>Vor</target>
</trans-unit>
<trans-unit id="featureChat-browserPreviewReload" datatype="html">
<source>Reload</source>
<target>Neu laden</target>
</trans-unit>
<trans-unit id="featureChat-browserPreviewAddress" datatype="html">
<source>Address</source>
<target>Adresse</target>
</trans-unit>
<trans-unit id="featureChat-browserPreviewAddressPlaceholder" datatype="html">
<source>https://example.com</source>
<target>https://example.com</target>
</trans-unit>
<trans-unit id="featureChat-browserPreviewGo" datatype="html">
<source>Go</source>
<target>Los</target>
</trans-unit>
<trans-unit id="featureChat-browserPreviewGuideTitle" datatype="html">
<source>How to use Browser Preview</source>
<target>So nutzt du die Browser-Vorschau</target>
</trans-unit>
<trans-unit id="featureChat-browserPreviewGuideNav" datatype="html">
<source>Enter a URL in the address bar and press Go (or Enter) to open a page in this virtual browser.</source>
<target>Gib eine URL in die Adressleiste ein und drücke Los (oder Enter), um eine Seite in diesem virtuellen Browser zu öffnen.</target>
</trans-unit>
<trans-unit id="featureChat-browserPreviewGuideInteract" datatype="html">
<source>Click, scroll, and type on the stream below to control the remote browser. Use Back, Forward, and Reload like a normal browser.</source>
<target>Klicke, scrolle und tippe auf dem Stream darunter, um den Remote-Browser zu steuern. Nutze Zurück, Vor und Neu laden wie in einem normalen Browser.</target>
</trans-unit>
<trans-unit id="featureChat-browserPreviewGuideClose" datatype="html">
<source>Close this window when you are done. Each open starts a fresh browser tab.</source>
<target>Schließe dieses Fenster, wenn du fertig bist. Jedes Öffnen startet einen neuen Browser-Tab.</target>
</trans-unit>
<trans-unit id="featureChat-browserPreviewGuideWorkspaceTitle" datatype="html">
<source>Open an app running in the workspace</source>
<target>App im Workspace öffnen</target>
</trans-unit>
<trans-unit id="featureChat-browserPreviewGuideWorkspaceBody" datatype="html">
<source>Preview shares a private Docker network with the workspace container. <x id="START_TAG_CODE" ctype="x-code" equiv-text="&lt;code&gt;"/>localhost<x id="CLOSE_TAG_CODE" ctype="x-code" equiv-text="&lt;/code&gt;"/> here is the Preview browser itself—not the workspace. Bind your app to <x id="START_TAG_CODE_1" ctype="x-code" equiv-text="&lt;code&gt;"/>0.0.0.0<x id="CLOSE_TAG_CODE_1" ctype="x-code" equiv-text="&lt;/code&gt;"/> (all interfaces), then open it with the workspace hostname and port.</source>
<target>Die Vorschau teilt sich ein privates Docker-Netzwerk mit dem Workspace-Container. <x id="START_TAG_CODE" ctype="x-code" equiv-text="&lt;code&gt;"/>localhost<x id="CLOSE_TAG_CODE" ctype="x-code" equiv-text="&lt;/code&gt;"/> meint hier den Vorschau-Browser selbst—nicht den Workspace. Binde deine App an <x id="START_TAG_CODE_1" ctype="x-code" equiv-text="&lt;code&gt;"/>0.0.0.0<x id="CLOSE_TAG_CODE_1" ctype="x-code" equiv-text="&lt;/code&gt;"/> (alle Interfaces) und öffne sie dann mit dem Workspace-Hostnamen und Port.</target>
</trans-unit>
<trans-unit id="featureChat-browserPreviewGuideWorkspaceExampleLabel" datatype="html">
<source>Example:</source>
<target>Beispiel:</target>
</trans-unit>
<trans-unit id="featureChat-browserPreviewGuideWorkspaceFallback" datatype="html">
<source>Use <x id="START_TAG_CODE" ctype="x-code" equiv-text="&lt;code&gt;"/>http://&lt;workspace-container-name&gt;:&lt;port&gt;<x id="CLOSE_TAG_CODE" ctype="x-code" equiv-text="&lt;/code&gt;"/>. Ask the agent or run <x id="START_TAG_CODE_1" ctype="x-code" equiv-text="&lt;code&gt;"/>hostname<x id="CLOSE_TAG_CODE_1" ctype="x-code" equiv-text="&lt;/code&gt;"/> in a workspace terminal to get the container name.</source>
<target>Verwende <x id="START_TAG_CODE" ctype="x-code" equiv-text="&lt;code&gt;"/>http://&lt;workspace-container-name&gt;:&lt;port&gt;<x id="CLOSE_TAG_CODE" ctype="x-code" equiv-text="&lt;/code&gt;"/>. Frage den Agenten oder führe <x id="START_TAG_CODE_1" ctype="x-code" equiv-text="&lt;code&gt;"/>hostname<x id="CLOSE_TAG_CODE_1" ctype="x-code" equiv-text="&lt;/code&gt;"/> in einem Workspace-Terminal aus, um den Containernamen zu erhalten.</target>
</trans-unit>
<trans-unit id="featureChat-createSshConnectionHint" datatype="html">
<source>Enable SSH access to the environment container. Defaults to disabled.</source>
<target>SSH-Zugang zum Umgebungscontainer aktivieren. Standardmäßig deaktiviert.</target>
Expand Down
63 changes: 63 additions & 0 deletions apps/agenstra/frontend-agent-console/src/i18n/messages.xlf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -963,6 +963,69 @@
<trans-unit id="featureChat-createVirtualWorkspaceHint" datatype="html">
<source>Enable VNC access to the environment&apos;s virtual workspace. Defaults to disabled.</source>
</trans-unit>
<trans-unit id="featureChat-enableBrowserPreview" datatype="html">
<source>Enable Browser Preview</source>
</trans-unit>
<trans-unit id="featureChat-createBrowserPreviewHint" datatype="html">
<source>Stream and control Chromium in the virtual workspace without full desktop VNC access. Defaults to enabled. Enabling VNC always includes Preview.</source>
</trans-unit>
<trans-unit id="featureChat-openBrowserPreview" datatype="html">
<source>Open browser Preview</source>
</trans-unit>
<trans-unit id="featureChat-openVirtualDesktop" datatype="html">
<source>Open virtual desktop</source>
</trans-unit>
<trans-unit id="featureChat-browserPreviewTitle" datatype="html">
<source>Browser Preview</source>
</trans-unit>
<trans-unit id="featureChat-browserPreviewClose" datatype="html">
<source>Close</source>
</trans-unit>
<trans-unit id="featureChat-browserPreviewConnecting" datatype="html">
<source>Connecting to browser…</source>
</trans-unit>
<trans-unit id="featureChat-browserPreviewBack" datatype="html">
<source>Back</source>
</trans-unit>
<trans-unit id="featureChat-browserPreviewForward" datatype="html">
<source>Forward</source>
</trans-unit>
<trans-unit id="featureChat-browserPreviewReload" datatype="html">
<source>Reload</source>
</trans-unit>
<trans-unit id="featureChat-browserPreviewAddress" datatype="html">
<source>Address</source>
</trans-unit>
<trans-unit id="featureChat-browserPreviewAddressPlaceholder" datatype="html">
<source>https://example.com</source>
</trans-unit>
<trans-unit id="featureChat-browserPreviewGo" datatype="html">
<source>Go</source>
</trans-unit>
<trans-unit id="featureChat-browserPreviewGuideTitle" datatype="html">
<source>How to use Browser Preview</source>
</trans-unit>
<trans-unit id="featureChat-browserPreviewGuideNav" datatype="html">
<source>Enter a URL in the address bar and press Go (or Enter) to open a page in this virtual browser.</source>
</trans-unit>
<trans-unit id="featureChat-browserPreviewGuideInteract" datatype="html">
<source>Click, scroll, and type on the stream below to control the remote browser. Use Back, Forward, and Reload like a normal browser.</source>
</trans-unit>
<trans-unit id="featureChat-browserPreviewGuideClose" datatype="html">
<source>Close this window when you are done. Each open starts a fresh browser tab.</source>
</trans-unit>
<trans-unit id="featureChat-browserPreviewGuideWorkspaceTitle" datatype="html">
<source>Open an app running in the workspace</source>
</trans-unit>
<trans-unit id="featureChat-browserPreviewGuideWorkspaceBody" datatype="html">
<source>Preview shares a private Docker network with the workspace container. <x id="START_TAG_CODE" ctype="x-code" equiv-text="&lt;code&gt;"/>localhost<x id="CLOSE_TAG_CODE" ctype="x-code" equiv-text="&lt;/code&gt;"/> here is the Preview browser itself—not the workspace. Bind your app to <x id="START_TAG_CODE_1" ctype="x-code" equiv-text="&lt;code&gt;"/>0.0.0.0<x id="CLOSE_TAG_CODE_1" ctype="x-code" equiv-text="&lt;/code&gt;"/> (all interfaces), then open it with the workspace hostname and port.</source>
</trans-unit>
<trans-unit id="featureChat-browserPreviewGuideWorkspaceExampleLabel" datatype="html">
<source>Example:</source>
</trans-unit>
<trans-unit id="featureChat-browserPreviewGuideWorkspaceFallback" datatype="html">
<source>Use <x id="START_TAG_CODE" ctype="x-code" equiv-text="&lt;code&gt;"/>http://&lt;workspace-container-name&gt;:&lt;port&gt;<x id="CLOSE_TAG_CODE" ctype="x-code" equiv-text="&lt;/code&gt;"/>. Ask the agent or run <x id="START_TAG_CODE_1" ctype="x-code" equiv-text="&lt;code&gt;"/>hostname<x id="CLOSE_TAG_CODE_1" ctype="x-code" equiv-text="&lt;/code&gt;"/> in a workspace terminal to get the container name.</source>
</trans-unit>
<trans-unit id="featureChat-createSshConnectionLabel" datatype="html">
<source> Create SSH Connection </source>
</trans-unit>
Expand Down
2 changes: 2 additions & 0 deletions docs/agenstra/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ Agenstra is a full-stack agent management platform that lets you:
- **Version Control Integration** Full Git operations (status, branches, commit, push, pull, rebase) directly from the web interface
- **Container Management** Monitor and interact with agent containers, view logs, and manage container lifecycle
- **VNC Browser Access** Graphical browser access via VNC with XFCE4 desktop and Chromium browser
- **Browser Preview** Browser-only Chromium control via CDP without publishing extra ports

## Documentation Structure

Expand Down Expand Up @@ -54,6 +55,7 @@ Feature documentation:
- [Web IDE](./features/web-ide.md) Monaco Editor integration for code editing
- [Chat Interface](./features/chat-interface.md) AI chat functionality and message flow
- [VNC Browser Access](./features/vnc-browser-access.md) Graphical browser access via VNC and noVNC
- [Browser Preview](./features/browser-preview.md) Browser-only Chromium Preview via CDP
- [Authentication](./features/authentication.md) Multiple authentication methods with configurable user registration
- [Atlassian import](./features/atlassian-import.md) Jira and Confluence imports into controller tickets and knowledge (admin)
- [Dynamic provider plugins](./features/dynamic-provider-plugins.md) Runtime provider extensions for controller and manager (baked-in or mounted)
Expand Down
1 change: 1 addition & 0 deletions docs/agenstra/deployment/environment-configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,7 @@ Optional runtime extensions for provisioning and context import. See [Dynamic pr
- `PORT` - HTTP API port (default: `3000`)
- `WEBSOCKET_PORT` - WebSocket gateway port (default: `8080`)
- `NODE_ENV` - Environment mode (`development` or `production`)
- `MANAGER_CONTAINER_ID` - Docker container ID or name of the manager API container. Used to temporarily join per-agent networks for browser Preview CDP. Defaults to compose `container_name` (`agent-manager-api`); falls back to `HOSTNAME` when unset.

### Database Configuration

Expand Down
15 changes: 15 additions & 0 deletions docs/agenstra/features/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ Agenstra provides a complete set of features for managing distributed AI agent i
- **Web IDE** Monaco Editor integration for code editing
- **Chat Interface** AI chat functionality with real-time responses
- **VNC Browser Access** Graphical browser access via VNC and noVNC
- **Browser Preview** Browser-only Chromium stream/control via CDP (no extra published ports)
- **Deployment** CI/CD pipeline management and deployment functionality
- **Authentication** Multiple authentication methods with configurable user registration
- **Tickets and Workspaces** Ticket boards, migration, and automation on the controller
Expand Down Expand Up @@ -133,6 +134,18 @@ Graphical browser access via VNC and noVNC. Access a Chromium browser running in
- Dedicated Docker network for container isolation
- Shared workspace volume between agent and VNC containers

### [Browser Preview](./browser-preview.md)

Browser-only Preview of Chromium via CDP over existing Socket.IO. Separately gated from full VNC; VNC implies Preview.

**Key Capabilities**:

- Stream and control Chromium without publishing extra host ports
- Screencast frames and input over the existing Socket.IO path
- Fresh Chromium tab per Preview session with fixed viewport
- Separately gated from full desktop VNC; enabling VNC always includes Preview
- Reach workspace apps over the shared Docker network (not `localhost` in Preview)

### [Deployment](./deployment.md)

CI/CD pipeline management and deployment functionality. Configure CI/CD providers (GitHub Actions), trigger pipeline runs, monitor their status, and view logs directly from the Agenstra console.
Expand Down Expand Up @@ -246,6 +259,7 @@ graph TB
IDE[Web IDE]
Chat[Chat Interface]
VNC[VNC Browser Access]
BP[Browser Preview]
DEP[Deployment]
AUTH[Authentication]
TK[Tickets and Workspaces]
Expand All @@ -261,6 +275,7 @@ graph TB
AM --> IDE
AM --> Chat
AM --> VNC
AM --> BP
AM --> DEP
WS --> Chat
WS --> TK
Expand Down
Loading
Loading