Browser Automator

Give it a URL and plain-English instructions — it drives the browser step by step, streaming live.

Checking agent-browser…

Configuration

OpenAI API Key Used to interpret each instruction. Never stored on the server.

Model

Automation

1. Target URL The page to open before running your instructions.

2. Instructions One instruction per step, separated by ;. Be specific — describe what to click, type, or select in plain English.

Error

Live Automation 0 / 0 Running…

Action Log

about:blank

Live browser feed will appear here
once the automation starts

API Reference

POST /api/automate/start

Starts a new automation job. Navigates to the URL, then for each semicolon-separated instruction uses an iterative loop: snapshot → AI plans the next single action → execute → screenshot → repeat until done. Returns a jobId immediately.

Request Body

Field	Type		Description
url	string	required	URL to open before running instructions
instructions	string	required	Semicolon-separated plain-English steps
openaiApiKey	string	optional	Falls back to `OPENAI_API_KEY` in `.env`
model	string	optional	OpenAI model. Default: `gpt-4o-mini`

Example

curl -X POST http://localhost:3000/api/automate/start \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/login",
    "instructions": "type user@email.com in the email field; type password123 in the password field; click Sign In"
  }'

GET /api/automate/stream/:jobId

Server-Sent Events stream. Connect with new EventSource(url) to receive live browser frames and log entries.

Events: frame — base64 PNG screenshot after each action | log — log entry | status — job state update | ping — keepalive every 15 s.

Frame event payload

{ "image": "<base64 PNG>", "action": "Clicking Sign In button", "ts": 1705312801234 }

POST /api/automate/stop/:jobId

Sends a stop signal. The current action completes cleanly, then the job halts and the browser session closes. No body needed.

GET /api/automate/status/:jobId

REST fallback — returns full log array and current status. Use the SSE stream endpoint instead for real-time updates.

Tips · Each instruction is executed with a live snapshot-act-snapshot loop — the AI sees the current page state before every action. · Use semicolons to separate steps: click the Login button; type "hello" in the search box; press Enter. · Be specific: "click the blue Export button" works better than "click Export". · For popups, add "in the popup" or "in the dialog" to help the AI scope correctly.