Build with MemorieAI
Everything you need to add a voice-powered guide to your website in under 5 minutes. No build tools. No framework lock-in. One script tag.
Quickstart
Paste this script tag before </body> on any page. That's the entire integration.
Installation
No API key yet? Request early access and we'll set you up within 24 hours.
The MemorieAI bubble appears in the bottom-right corner. Your users click it, speak or type their request, and the assistant guides them through your site in real time.
Verify It Works
Open your browser console after loading the page with the script. You should see:
If you see a 403 domain_not_allowed error, your domain has not been registered for your API key. See Domain Locking.
How It Works
MemorieAI operates as a thin layer on top of your existing website. It reads the live DOM, understands intent through voice or text, selects the right tool, and executes it. Nothing is pre-trained on your site.
Semantic Map
On every page load, the embed script builds a semantic map: a structured index of visible interactive elements. This map is what the AI "sees" about your page.
Each element in the map includes: tag name, visible text, ARIA labels, role, viewport position (above / visible / below), and estimated area. The map is rebuilt automatically after any navigation.
Elements with descriptive ARIA labels, meaningful button text, and semantic HTML are indexed much more reliably than unlabeled icon buttons or generic "click here" links.
Tool Pipeline
When a user speaks or types a request, this sequence runs:
Intent detection: The AI parses the user's request and maps it to one or more browser tools.
Selector resolution: The AI picks a target element from the semantic map using the best available descriptor (text, role, position).
Tool execution: The tool runs on the live page: navigate, highlight, click, fill, scroll, or read.
Map refresh: After any navigation, the semantic map is rebuilt for the new page before the next tool can run.
Voice feedback: The assistant speaks the result back to the user.
Configuration
All configuration is done through data-* attributes on the script tag. No separate config file, no dashboard required for basic setup.
Data Attributes
| Attribute | Default | Description |
|---|---|---|
| data-key | — | Your MemorieAI API key. Required. Obtain from the dashboard after signing up. |
| data-lang | auto | Force a language: en, es. Defaults to browser locale detection. |
| data-position | bottom-right | Bubble position: bottom-right or bottom-left. |
| data-theme | auto | Color theme: light, dark, or auto (follows system preference). |
| data-color | #004488 | Primary accent color (hex). Applied to the bubble, highlights, and panel accents. |
| data-label | MemorieAI | Name shown in the panel header and in the assistant's speech. |
| data-assistant | — | Pro / Enterprise. Your custom Vapi assistant ID, overriding the default MemorieAI assistant. |
Custom Prompt
Pro and Enterprise customers can write a custom system prompt per site from the dashboard. The prompt tells the assistant about your site's structure, key workflows, and personality. The live page map is injected automatically — you don't need to describe every page.
Include exact URL paths for your most common workflows. The assistant will navigate directly to them when user intent matches, instead of relying on the page map alone.
Prompt Template
For the Miramar, FL deployment, the system prompt includes all 17 city departments, 40+ services, and their exact URLs — enabling the assistant to navigate directly without searching.
Browser Tools
MemorieAI has six tools it can call on any page. Selectors are always drawn from the live semantic map — the AI never invents IDs or guesses URLs.
Parameters: url (string, required)
Navigates the current tab to any URL — relative (/pay-bills) or absolute (https://…). After the page loads, the semantic map is automatically rebuilt before any subsequent tool can run. This prevents race conditions when navigate and highlight are called in the same batch.
Parameters: selector (string, required)
Places a glowing animated border on the matched element so the user can see exactly where to look or click. The selector is a natural-language description ("Pay Now button", "Search field") resolved by a 5-stage fuzzy matching pipeline: exact ID → exact text → fuzzy text → scoring → best candidate.
Parameters: selector (string, required)
Programmatically clicks a button, link, tab, or any interactive element. Uses the same selector resolution as highlight_element. Fires a native browser click event — works with React, Vue, Angular, and vanilla DOM listeners.
Parameters: selector (string), text (string, required)
Types text into an input, textarea, or contenteditable element. Fires input and change events so framework bindings stay in sync. Does not submit the form — the user retains control over submission.
Parameters: selector (string, required)
Smooth-scrolls the page until the matched element is in the viewport. Useful for directing attention without clicking. Respects prefers-reduced-motion: uses instant scroll when the user has animations disabled.
Parameters: selector (string, required)
Reads the text content of an element aloud to the user via the voice interface. Useful for reading table values, error messages, or form summaries that the user may not see clearly.
Advanced
Multi-Tenant Architecture
Each customer gets a unique data-key. The key is validated server-side against the requesting origin on every session start. A key registered for acme.com will not work on any other domain.
The Vapi API key and assistant ID are never exposed to the browser. They live only on the MemorieAI backend and are returned as short-lived session tokens scoped to the request origin.
Domain Locking
From your dashboard, register every origin your key is allowed to serve. Requests from unlisted origins are rejected with a 403 domain_not_allowed error before any Vapi session is created.
localhost, 127.0.0.1, and localhost:* are always allowed in development — no registration needed.
Languages
MemorieAI detects the user's preferred language from their browser locale and from the language they speak or type. The assistant responds in the same language automatically.
Force a specific language with data-lang:
Currently well-tested languages:
- English (en) — full support
- Spanish (es) — full support; primary language for Miramar, FL deployment
- Additional languages on request — contact us
Knowledge Base
For government and enterprise deployments, MemorieAI supports a structured knowledge base that maps user intents directly to pages and elements without relying on the AI alone.
Two JSON files power the Miramar deployment:
- site-index.json — Maps URLs to department names, sections, and available actions. Used to resolve "where is X" questions.
- intent-directory.json — Maps intent keywords to direct navigate + highlight actions. Bypasses the AI for the 40 most common city service requests.
Enterprise customers can provide their own site-index and intent-directory via the dashboard. Contact sales for the schema documentation.
Architecture
MemorieAI is a three-layer platform. The Browser layer runs in the user's device (embed script or Chrome extension), the Gateway layer is a Cloudflare Worker with Durable Objects that authenticates and routes every session, and the Agent layer is whatever AI is driving the session (Vapi voice AI, the MCP Server, or any custom agent over the WebSocket protocol).
The Gateway is the only layer that holds long-lived secrets. Browser code never sees the Vapi private key, the master API key, or the customer's raw memai_sk_* credential. Agents only see what their scoped API key allows.
Communication: Agents speak JSON-RPC 2.0 over WebSocket to the Gateway. The Gateway forwards browser commands to whichever browser is connected to the same SessionDO and streams DOM change notifications back. Voice sessions take a different path — the Gateway mints a short-lived Vapi JWT and the browser opens a direct WebRTC connection to Vapi.
Session Types
A MemorieAI session is a triplet of browser, agent, and policy. There are four session types — pick the one that matches your deployment.
| Session Type | Browser | Agent | Human Present | Use Case |
|---|---|---|---|---|
| voice | End user's browser, via embed.js | Vapi voice AI | Yes | Voice-first assistant on a public website (current Miramar deployment). |
| agent-assist | End user's browser | Customer's AI agent (MCP or custom) | Yes | An AI support bot remotely guides a live user through their session while they watch. |
| headless BYOB | Customer-hosted headless Chrome | Customer's AI agent | No | Customer runs their own headless browser with embed.js — automation on their infra. |
| headless managed | MemorieAI cloud browser | Customer's AI agent | No | Turnkey — we spin up the browser, customer just sends commands. |
Same execution engine, different control plane. All session types use the same embed.js DOM tools (navigate, click, highlight, fill, scroll, read). What differs is who spawns the browser and who sends the commands.
MCP Server
The MemorieAI MCP Server exposes browser sessions as tools to any MCP-compatible AI client. Drop it into Claude Code, Cursor, or any custom MCP client and you immediately get 15 browser-automation tools backed by the MemorieAI Gateway.
What is MCP?
The Model Context Protocol (MCP) is an open standard for connecting AI assistants to external tools and data. It uses JSON-RPC 2.0 over stdio or SSE. Clients like Claude Code and Cursor speak MCP natively, so any MCP server they install becomes a first-class tool surface in the assistant.
The MemorieAI MCP Server is a thin protocol adapter — it translates MCP tool calls into JSON-RPC commands on the MemorieAI Gateway and streams browser results back to the client.
Installation
| Variable | Required | Description |
|---|---|---|
| MEMORIEAI_API_KEY | yes | API key in the format memai_sk_<customer>_<hex>. Issued by the Admin CLI. |
| MEMORIEAI_GATEWAY_URL | no | Defaults to https://memorieai-embed-api.josuedpinedad.workers.dev. Override for self-hosted gateway. |
Claude Code Setup
Add the server to claude_desktop_config.json (location varies by OS — usually under ~/Library/Application Support/Claude/ on macOS or %APPDATA%\Claude\ on Windows).
Restart Claude Code. The 15 memorieai_* tools become available in any conversation.
Cursor Setup
Same shape, different config file. Cursor reads from .cursor/mcp.json in your project root (or the global Cursor settings).
MCP Tools Reference
All 15 tools are listed below. Every browser-action tool requires the session_id returned by memorieai_create_session. Selectors are natural-language descriptors ("Pay Now button"), not CSS selectors.
Session Management
| Tool | Parameters | Description |
|---|---|---|
| memorieai_create_session | type, target_url | Create a new browser automation session. type is agent-assist or headless. Returns session_id, ws_url_agent, ws_url_browser. The browser must connect separately before commands work (auto for managed, manual for BYOB). |
| memorieai_list_sessions | — | List all active browser sessions created in this MCP connection. |
| memorieai_close_session | session_id | Close a browser session and release all resources. Returns { closed: true, session_id }. |
Browser Actions
| Tool | Parameters | Description |
|---|---|---|
| memorieai_navigate | session_id, url | Navigate the browser to a URL. Must be within the session's URL sandbox (policy.navigable_domains). Returns { success: true } or error -32001. |
| memorieai_click | session_id, selector, confirm? | Click an element by semantic selector. For destructive actions (delete, cancel, sign out), set confirm: true or the gateway returns -32002. |
| memorieai_highlight | session_id, selector | Visually highlight an element on the page. Most useful in agent-assist mode where a human is watching. |
| memorieai_fill | session_id, selector, text | Fill a form field with text. Fires input and change events so framework bindings stay in sync. Never submits the form. |
| memorieai_scroll | session_id, selector | Smooth-scroll to an element on the page. |
| memorieai_read | session_id, selector | Read the text content of an element. Returns up to ~500 chars of trimmed text. |
| memorieai_read_dom | session_id | Get the full semantic map of the current page — all visible elements with text, tag, role, and viewport position. |
Advanced
| Tool | Parameters | Description |
|---|---|---|
| memorieai_screenshot | session_id | Take a screenshot of the current page. Headless sessions only. Returns a base64-encoded PNG. |
| memorieai_wait_for | session_id, selector, timeout_ms? | Wait for an element to appear on the page. Default timeout: 10000 ms. Useful before clicking elements that load asynchronously. |
| memorieai_evaluate | session_id, expression | Evaluate a JavaScript expression in the browser. Headless only. Requires the evaluate scope on the API key — disabled by default. |
| memorieai_get_state | session_id | Get the current session state: URL, title, session metadata, viewport. |
| memorieai_set_viewport | session_id, width, height | Set browser viewport dimensions in pixels. Headless only. |
Example Flow
A typical conversation where Claude uses MCP to navigate to a Miramar city service:
The MCP Server also exposes memorieai://sessions/{id}/dom and memorieai://sessions/{id}/state as MCP resources — the assistant can subscribe to live DOM changes instead of polling.
Agent API
If you can't or don't want to use MCP, talk to the Gateway directly. The Agent API is REST for session creation and WebSocket for command execution. Every session is bracketed by one POST /v1/sessions and one WebSocket connection per side (agent + browser).
Authentication
Every Agent API request authenticates with an API key in the Authorization: Bearer header. Keys are issued via the Admin CLI and look like:
The key has three parts: memai_sk_ prefix (so they're easy to spot in logs), the customer slug, and 32 random hex characters. The Gateway hashes every key with SHA-256 and stores only the hash in KV — if the KV namespace ever leaked, attackers could not extract raw keys.
Scopes attached to each key control what it can do:
sessions:create— required to callPOST /v1/sessionsbrowser:*— required to connect the agent WebSocket and send anybrowser.*commandbrowser:evaluate— required separately forbrowser.evaluate(JavaScript execution)
Raw keys are only shown once — at the moment of creation. Save them to your secrets manager immediately. If you lose a key, revoke it and create a new one; there is no recovery path.
REST Endpoints
POST /v1/sessions
Create a new browser-automation session. Returns the session ID and the two WebSocket URLs.
| Code | Body | Trigger |
|---|---|---|
| 201 | session info | Session created. Connect to ws_url_agent to start sending commands. |
| 400 | { error: "invalid_json" } | Body could not be parsed as JSON. |
| 400 | { error: "invalid_session_type" } | type must be agent-assist or headless. |
| 400 | { error: "missing_target_url" } | target_url is required and must be a string. |
| 401 | { error: "unknown_key" } | API key does not exist in KV. |
| 401 | { error: "inactive_key" } | Key was revoked. |
| 401 | { error: "scope_denied" } | Key lacks the sessions:create scope. |
GET /v1/ws/agent?session=<id>
WebSocket upgrade for the agent side of a session. The agent sends JSON-RPC commands and receives results plus browser notifications.
Requires the browser:* scope. The Gateway authenticates the key, looks up the SessionDO by ID, and pipes the WebSocket through.
GET /v1/ws/browser?session=<id>
WebSocket upgrade for the browser side. embed.js (or the Chrome extension) opens this connection. Browser-side authentication is via the session token returned in the create-session response, not the long-lived API key.
WebSocket Protocol
The Gateway speaks JSON-RPC 2.0. Every frame is a single JSON object. Pick globally-unique id values for requests — the Gateway uses them to route responses back to the right caller.
Request (agent → gateway → browser)
Response (browser → gateway → agent)
Error
Notification (browser → agent, no id)
The browser pushes DOM state changes whenever the page navigates or the semantic map changes meaningfully. Notifications have no id — they don't expect a response.
Available Methods
The full set of JSON-RPC methods accepted on the agent WebSocket.
| Method | Params | Description |
|---|---|---|
| browser.navigate_to | url | Navigate to a URL. Sandboxed against policy.navigable_domains. |
| browser.click_element | selector, confirm? | Click an element. Destructive selectors require confirm: true. |
| browser.highlight_element | selector | Show a highlight overlay on an element. |
| browser.fill_field | selector, text | Type text into an input or textarea. |
| browser.scroll_to | selector | Smooth-scroll until the element is in view. |
| browser.read_content | selector | Return the text content of an element. |
| browser.read_dom | — | Return the full semantic map of the current page. |
| browser.wait_for | selector, timeout? | Wait for an element to appear. Default timeout 10000 ms. |
| browser.screenshot | — | Return a base64 PNG screenshot. Headless only. |
| browser.evaluate | expression | Evaluate JavaScript. Headless only. Requires browser:evaluate scope. |
| browser.set_viewport | width, height | Set viewport dimensions in pixels. Headless only. |
| session.get_state | — | Return session metadata: id, type, target_url, created_at. |
| session.close | — | Tear down the session and close both WebSockets. |
Error Codes
The Gateway extends the JSON-RPC 2.0 error code range with MemorieAI-specific codes from -32001 through -32008.
| Code | Message | Meaning & Recovery |
|---|---|---|
| -32700 | parse_error | Frame was not valid JSON. Fix the encoder. |
| -32600 | invalid_request | Missing required field (method or id). Fix request format. |
| -32601 | method_not_found | Method is not on the available-methods list. |
| -32001 | blocked_by_url_sandbox | navigate_to target is outside policy.navigable_domains. Don't retry with the same URL. |
| -32002 | destructive_action_requires_confirm | Selector text matches a destructive keyword. Resend with confirm: true. |
| -32003 | element_not_found | Selector did not resolve to any element. Try a different descriptor or call read_dom first. |
| -32004 | browser_not_connected | No browser is connected to the session yet. Wait, then retry. |
| -32005 | rate_limited | Per-session or per-customer command cap exceeded. Back off using retry_after_ms in the error data. |
| -32006 | browser_crashed | The browser process died. Create a new session. |
| -32007 | session_expired | Session exceeded idle timeout. Create a new session. |
| -32008 | scope_denied | API key lacks the required scope for this command. |
Security
The Gateway enforces four layers of defense on every session — independent of any client-side check in embed.js.
- URL Sandbox:
browser.navigate_tois validated againstpolicy.navigable_domainsserver-side. Same-origin and fragment-only destinations are always allowed; cross-origin requires an explicit allow-list entry. - Destructive Click Guard:
browser.click_elementon a target whose text matches a destructive keyword (delete, remove, cancel, reset, clear, sign out, log out, logout, unsubscribe, deactivate) requiresconfirm: true. Per-customer extras and excludes can be configured. - Rate Limiting: Per-session and per-customer command rate caps prevent runaway loops and abuse. Limits vary by plan (Starter: 60/min, Pro: 300/min, Enterprise: custom).
- Idle Timeout: Sessions are torn down automatically after a configurable idle window — default 5 min for agent-assist, 30 min for headless.
browser.evaluate runs arbitrary JavaScript inside the browser session. It is disabled unless the API key carries the explicit browser:evaluate scope, and even then it should only be granted to first-party automation under a security review.
Admin CLI
The @memorieai/cli package wraps the Gateway's admin endpoints in an ergonomic command-line tool. Use it to mint, list, and revoke API keys without touching the KV namespace directly.
Installation
Managing API Keys
Three subcommands cover the full lifecycle: create, list, revoke.
memorieai keys create
Save the key now. It is printed once and never stored in clear text. The Gateway holds only the SHA-256 hash — there is no recovery if you lose it.
memorieai keys list
memorieai keys revoke
Revocation deletes the apikey:<hash> record from KV. KV propagation is eventually consistent across Cloudflare's edge — allow up to ~60 seconds for the change to roll out globally.
Environment Variables
| Variable | Required | Description |
|---|---|---|
| MEMORIEAI_MASTER_KEY | yes | The master admin key. Authenticates every CLI request to /v1/admin/keys. Stored only as a Cloudflare Worker secret on the server side; compared in constant time to prevent timing attacks. |
| MEMORIEAI_GATEWAY_URL | no | Override the default Gateway URL. Useful when running against a staging Worker or self-hosted deployment. |
The master key is distinct from per-customer memai_sk_* keys. It grants the ability to mint and revoke other keys — keep it in a secrets manager, never in source control.
Chrome Extension
The MemorieAI Chrome extension is the most feature-complete deployment surface. It uses the same DOM tools and Gateway protocol as the embed script, plus a persistent sidepanel, a background service worker, and Chrome-specific UX (the bubble, the iframe widget, the navigation queue).
Installation
Open
chrome://extensionsand toggle Developer mode in the top-right corner.Click Load unpacked and select the
extension/directory from this repository.Start the widget server so the iframe served from
localhost:8080can load (a CSP requirement): runstart-demo.baton Windows orcd extension && python -m http.server 8080elsewhere.Visit
miramarfl.gov. The MemorieAI bubble appears in the bottom-right; click it to open the sidepanel.
No build step. The extension is plain JavaScript — reload Chrome after edits to content.js, widget.js, or sidepanel.js.
Architecture
Four components communicate via Chrome message-passing and postMessage.
| Component | File | Responsibility |
|---|---|---|
| Content Script | content.js | Injected into every page. Holds findEl(), buildSemanticMap(), the six DOM tools, and the liquid UI animations. |
| Sidepanel | sidepanel.js | Bridge between the widget iframe and the active tab. Holds the navigation queue and injects the content script when needed. |
| Widget | widget.js | Voice / text UI inside a sandboxed iframe served from localhost:8080. Manages the Vapi call lifecycle, prompt assembly, and tool-result routing. |
| Service Worker | background.js | Routes BUBBLE_CLICK to sidePanel.open(), sends AUTO_START after 1500 ms, forwards PAGE_READY to the sidepanel. |
Content Script
content.js is the only script that touches the live DOM. It exposes the six browser tools to the rest of the extension via chrome.runtime.onMessage and runs the semantic map builder that the AI uses for context.
findEl — 5-stage element matching. The AI sends a semantic descriptor like "Pay Now button". findEl() resolves it to the actual DOM node in five stages:
- Exact ID match (
#pay-now-btn) - Exact text match (whole element text)
- Fuzzy text match (substring + typo tolerance)
- Scoring algorithm (text similarity + viewport bonus + area bonus)
- Fallback to the best-scored candidate, or null
buildSemanticMap — Page context for the AI. Returns at most 25 elements visible on screen, each truncated to 30 characters. The map is rebuilt on every navigation and sent to Vapi as a system message before each call so the assistant always sees the current page state.
Navigation Queue
A key race condition: when Vapi sends navigate_to + highlight_element in the same tool-call batch, both arrive at sidepanel.js simultaneously. Without protection, the highlight races to the old page before the new one loads.
The fix is a synchronous guard set before any await: navigate flips a _navigating flag, and sibling DOM tools (highlight_element, click_element, fill_field, scroll_to, read_content) check the flag and get queued. The queue is flushed 400 ms after the new page fires REFRESH_PAGE_MAP.
Liquid UI
Visual feedback when the AI acts on the page. Three primitives live in content.js:
- liquidFlow() — animated Bézier path from the bubble to the target element. Runs a callback (e.g. the click) at the end of the animation.
- applyLiquid() — morphing border overlay on the target element to draw the eye.
- showContextCard() — tooltip rendered below the element with a brief AI context line.
FAQ
Does MemorieAI store voice recordings?
No. Voice is processed in real time via Vapi's WebRTC pipeline. Transcripts are not stored by MemorieAI. Vapi's own data retention policy applies to the voice session; see their documentation for details.
Does it work with SPAs (React, Vue, Angular)?
Yes. The semantic map is rebuilt after every URL change via the History API. Navigation tool calls use window.location and trigger a map rebuild on the load event of the new page. All click and fill events fire native DOM events that framework bindings respect.
What is the performance impact of the embed?
The embed script is under 12 KB gzipped. It loads with defer and does not block rendering. The semantic map is built asynchronously on load. The Vapi WebRTC connection opens only when the user activates the bubble — there is no persistent connection on page load.
Is it GDPR / CCPA compliant?
MemorieAI does not store personally identifiable information from user sessions. The only data collected is anonymous session telemetry (session count, tool usage counts) used for billing. Enterprise customers can request a Data Processing Agreement (DPA).
Can I white-label the assistant?
Yes on Pro and Enterprise. Use data-label to set the display name, data-color for the brand color, and data-assistant to connect your own Vapi assistant with a fully custom voice and persona.
What happens if the embed script fails to load?
Nothing. The embed is fully additive — it adds the MemorieAI bubble to the page but does not modify any existing elements. If the script fails, your site works exactly as before. There are no fallback dependencies.