Self-hosted AI companion for iPhone and iPad
_{Ancient signal. Modern interface.
Voice mode · Mimo TTS · Sensors · Rich Chat · Notes · Session management · Remote MCP}

---

What is HERALD?

HERALD is a native iOS client for the Hermes Agent framework. It connects to your self-hosted Hermes instance through a native WebSocket relay channel, giving you a polished mobile experience — streaming chat, voice mode, health/location/motion sensors, notes, and session management — without your data leaving your infrastructure.

HERALD is not the AI. It is the phone interface for your Hermes agent.

---

What's new in 2.4.1

HERALD 2.4.1 makes conversations more reliable when the model or network has an unlucky moment.

• One chat per conversation — compose IDs reconcile to the actual Hermes session, avoiding duplicate rows with competing titles.
• Clear interrupted-turn recovery — upstream interruptions are shown as retryable failures, never as successful replies.
• Reasoning survives refresh — available reasoning is retained on past messages and shown as a collapsed “Thought process” summary.
• Optional live reasoning — when enabled, the connector uses the existing Hermes dashboard gateway to stream the model’s reasoning as it is produced.

Architecture

iOS App ← HTTP/SSE → Caddy (:443) ← Connector HTTP Facade (:8010)
                                        ├── Native Relay WS (:8765) ← Hermes Gateway
                                        ├── MCP HTTP (:8767)
                                        └── Hermes API Server (:8642)

• http_facade.py — FastAPI HTTP/SSE server for the iOS app (~300 lines)
• relay_server.py — Native Hermes relay protocol WebSocket server for the gateway (305 lines)
• client.py — Connector core: job execution, model/profile RPCs, streaming bridge
• The Docker relay container is stopped — all iOS traffic goes directly to the connector

Screens

Welcome

Endpoint

Paired

---

iOS Platform Integrations

HERALD is a deeply native iOS app that uses platform APIs the way Apple intended. Every integration is a first-class citizen, not a wrapper.

HealthKit HERALD syncs real-time health data from Apple Health so your agent can reason about your body alongside your conversations. Heart rate, resting heart rate, HRV Step count, distance, flights climbed Sleep analysis (time in bed, time asleep) Active energy, exercise minutes Mindful session data Background delivery — data pushes to your AI even when the app is closed	CoreLocation HERALD tracks your position so your agent knows where you are, where you have been, and where you are going. Continuous background location updates Significant location change monitoring Visit detection (arrival/departure) Geofence awareness Location data piped to your AI in real-time All data stays on your relay
CoreMotion HERALD reads accelerometer, gyroscope, and activity data so your agent knows your current activity state. CMMotionActivity (walking, running, cycling, driving, stationary) Step counting via CMPedometer Cadence, pace, distance Altitude changes via barometric altimeter Fall detection awareness Motion data synced to your AI context	Widgets and Live Activities HERALD ships widget extensions that keep your AI connection visible at a glance. HeraldHealthWidget — latest heart rate, step count, sleep summary HeraldStatusWidget — host online/offline, connection state, model name Live Activities — real-time streaming status on the Lock Screen Dynamic Island integration for voice sessions Widget data refreshed via App Group container Timeline provider with relevance-based updates	Camera and Photos HERALD lets you attach images from your camera or photo library, and voice mode can stream live camera context to your agent. Camera capture via UIImagePickerController Photo library picker with PHPickerViewController Image compression and base64 encoding for relay transport Live camera feed during voice mode sessions Image preview with fullscreen viewer
Push Notifications HERALD uses APNs with silent push to wake the app when your agent has something to deliver, even in the background. APNs device token registration via relay Silent push for background conversation sync Rich notifications with message previews Notification actions (reply, dismiss) Push broker architecture for token relay Per-device registration with Keychain storage	AVFoundation and Speech Voice mode uses MiMo ASR for speech recognition and MiMo TTS for synthesis, with Hermes processing. MiMo ASR for streaming speech-to-text MiMo TTS for text-to-speech synthesis Push-to-talk mode via HermesTalkCoordinator Audio session management (speaker, receiver, Bluetooth) Voice transcript display with live streaming
Share Extension and Siri Share content directly to HERALD from any app, and use Siri Shortcuts to trigger your AI hands-free. Share sheet integration for text and images Siri Shortcuts support NSUserActivity for Spotlight search Universal links for deep linking URL scheme for inter-app communication	SwiftUI and UIKit HERALD uses SwiftUI for the interface with UIKit where it matters — haptics, pasteboard, activity view controllers, and precise gesture handling. SwiftUI NavigationSplitView for iPad UIKit haptics via UIImpactFeedbackGenerator UIPasteboard for copy/paste UIActivityViewController for share sheets UIDevice orientation and model detection Scene-based lifecycle (UISceneDelegate)

Keychain and Security

All sensitive data lives in the Keychain, not UserDefaults.

• APNs device token stored as ThisDeviceOnly
• Session access tokens with AfterFirstUnlock protection
• Biometric-protected secure storage
• App Attest for push broker authentication
• No data leaves your infrastructure

---

Features

Rich Chat Real-time streaming with markdown rendering Syntax-highlighted code blocks (Swift, Python, JS, TS, SQL, Bash) Thinking blocks — stream live when the gateway transport is enabled, then collapse into a “Thought process” summary Tool call bubbles — expandable args/result Markdown tables with grid-based rendering Canvas — edit AI-generated code in a dedicated panel Long-press context menus (copy, share, retry, delete) Inline diffs and image previews

Session Management Pin, archive, rename, search sessions Device-scoped session isolation Context window usage ring Model switching via direct RPC Slash command autocomplete Context compaction with budget warnings Cron job scheduling from your phone Skills browser and profile switching

Notes PencilKit handwriting editor with tool picker On-device handwriting recognition Relay CRUD with optimistic concurrency SHA-256 content hashing and monotonic revisions PDF export with document directives iPad split-view navigation

Inbox and Action Center Push-driven action items from your agent Dismiss, snooze, and filter controls Refresh on push wake Directive progress tracking Enriched document previews

---

Pairing Props

Copy-paste setup prompts that any AI assistant can use to walk you through configuring Herald end-to-end.

Prompt	Method	Best for
tailscale.md	Tailscale tailnet (private mesh)	Home lab, single-network, privacy-first
remote-relay.md	Public URL (Fly.io, VPS, etc.)	Mobile use, travel, always-on access

How it works: copy the contents of either prompt, paste it into a conversation with your Hermes agent (or Claude, ChatGPT, etc.), and the assistant will check your prerequisites, deploy the relay, configure the connector, and pair your phone step by step.

See docs/pairing-props/ for a detailed comparison, architecture diagram, and troubleshooting guide.

---

Quick Start

1. Deploy the connector

pip install herald-connector
herald configure-mcp   # registers MCP tools in ~/.hermes/config.yaml
herald run             # starts all services

The connector runs four services in one process:

• HTTP facade on port 8010 — iOS app API (SSE streaming, models, profiles, sessions)
• Native relay WS on port 8765 — Hermes gateway connects here
• MCP HTTP server on port 8767 — Streamable HTTP for remote Hermes access
• FastAPI host WS — optional, for legacy pairing flow

For live reasoning, HERALD can also connect to your existing Hermes dashboard gateway over its JSON-RPC WebSocket. That transport streams answer deltas, tool activity, and model reasoning; it does not require a second gateway process. Enable it per installation with HERALD_TRANSPORT=tui_ws. If the dashboard is auth-gated, provide the connector with its configured gateway credentials via a protected environment file so it can mint short-lived WebSocket tickets.

# Keep this file out of source control and readable only by the connector user.
umask 077
cat > ~/.config/herald-gateway-auth.env <<'EOF'
HERALD_TRANSPORT=tui_ws
HERALD_GW_URL=http://127.0.0.1:9119
HERALD_GW_AUTH_PROVIDER=basic
HERALD_GW_USERNAME=your-dashboard-user
HERALD_GW_PASSWORD=your-dashboard-password
EOF
chmod 600 ~/.config/herald-gateway-auth.env

Reference that file from the connector service with an EnvironmentFile. Never put the password in config.yaml, a service unit, shell history, or the repository. The connector caches the dashboard session and mints a new short-lived, single-use ticket for each WebSocket connection.

2. Point Caddy at the connector

herald.example.com {
    reverse_proxy localhost:8010 {
        header_up Connection {>Connection}
        header_up Upgrade {>Upgrade}
        transport http { response_header_timeout 0 }
    }
}

3. Build and install HERALD

git clone https://github.com/[user]/Herald.git
cd Herald
xcodegen generate
open Herald.xcodeproj

Build to your device from Xcode, enter https://herald.example.com in the onboarding flow, and start chatting.

See docs/BUILDING.md for detailed signing and entitlements instructions.

---

Tech Stack

Layer	Technology
iOS App	Swift 6.2, SwiftUI, UIKit, iOS 18+
Connector	Python, WebSockets, FastMCP (Streamable HTTP), Hermes Relay Protocol
Project Config	XcodeGen (project.yml)
Build	Xcode 26+, macOS 26+

---

Project Structure

Herald/
├── App/                    # App entry, scene delegate
├── Core/                   # MarkdownParser, Design system, networking
├── Features/
│   ├── Chat/               # Chat screen, message bubbles, renderers
│   │   └── Renderers/      # Code, thinking, tool call, table views
│   ├── Canvas/             # Canvas panel for code artifacts
│   ├── Capture/            # Camera and photo capture
│   ├── Cron/               # Cron job scheduling
│   ├── Inbox/              # Action Center and push items
│   ├── Notes/              # PencilKit editor, recognition, relay sync
│   ├── Onboarding/         # Setup wizard (endpoint, permissions, pairing)
│   ├── Permissions/        # Health, location, notification grants
│   ├── Settings/           # App settings
│   ├── Sidebar/            # iPad right panel
│   ├── Skills/             # Skills browser and profile switching
│   └── Talk/               # Voice mode (MiMo ASR/TTS + Hermes)
├── Models/                 # Data models (Message, Artifact, etc.)
├── Stores/                 # State management (ChatStore, etc.)
├── Services/
│   ├── Live/               # HermesTalkCoordinator, MimoASRService, MimoTTSService
│   └── Protocols/          # Service protocols
├── Widgets/                # Home Screen widgets + Live Activities
└── Resources/              # Assets, entitlements, Info.plist
connector/                  # Python connector + relay server

---

Contributing

See CONTRIBUTING.md for guidelines.

---

Acknowledgements

Built on the foundation of Hermes-iOS by Dylan Buck and the Nous Research community. Original work licensed under MIT.

---

License

MIT

---

_{Your AI. Your server. Your rules.}