LocalPilot

Record a browser workflow, or describe it in plain language, and replay it — without sending the page anywhere.

---

LocalPilot is a Chrome extension (Manifest V3) for automating repetitive work in web apps — especially the internal, server-rendered kind that never got an API. Nothing about the page leaves the browser: the default planner is a rule-based engine that runs locally, and the optional model providers are ones you host yourself.

What it does

• Record and replay. Click through the flow once; LocalPilot captures the steps, the pauses you took, and three independent ways of finding each control again.
• Plain-language commands. Type "preencher email com user@test.com" or "clicar em Editar da terceira linha" in the popup and it resolves the element on the page.
• Generate an automation from a description. Write the flow as a list of sentences and LocalPilot turns it into steps — including repetitions from an inline array.
• Loops with variables. Run the same flow once per row of a small table, with {{placeholders}} in any field, and screenshots named per iteration.
• Screenshots into folders. Capture the visible tab to a subfolder of Downloads, with a filename that can carry the loop's values.
• Export and import. Move automations between machines, back them up, or share one as a single JSON file.

Install

No store listing yet. Grab a build or make your own:

git clone https://github.com/athosbes/localpilot.git
cd localpilot/extension
npm install
npm run package

Then open chrome://extensions, turn on Developer mode, choose Load unpacked, and select extension/dist. Released builds are attached to each GitHub release as a zip of that same folder.

Recording a flow

Open the popup and hit Gravar interações na tela. An overlay appears in the corner of the page with four controls beyond the recorder itself:

Button	What it adds
+2s / +5s	An explicit wait step, for spinners and async refreshes
📸	A screenshot step, numbered so several captures do not collide
Pausar	Stops capturing without ending the session
Finalizar	Saves the recording as an automation

Pauses longer than ~1s between two interactions are captured automatically and attributed to the step before them, so the replay waits where you waited.

Loops

An automation can repeat once per row of a table defined in the Editor. Columns are variable names, rows are the values bound on each pass:

cliente	centro_custo
ACME	1001
Globex	1002

Any step's value, selector, target text, screenshot filename or destination folder can reference those names as {{cliente}}. {{iteracao}} and {{total}} are always available. An unknown name is left in the text verbatim, so a typo shows up in the log instead of silently becoming an empty string.

Filling the table

You rarely want to type that table by hand. Two shortcuts:

• Variáveis usadas nos passos sits above the loop and lists every {{name}} your steps reference, marking the ones no column fills. One click creates the missing columns, keeping the values already entered.

• Gerar as repetições a partir de dados takes the data in whatever shape you have it and builds the rows:

  cliente,centro_custo          [{"cliente": "ACME", "centro_custo": "1001"}]
  ACME,1001                     {"cliente": ["ACME", "Globex"]}
  Globex,1002                   cliente = ACME, Globex
  

  A spreadsheet paste (tab, comma or semicolon, quoted fields honoured), JSON in four shapes, or plain assignment lines. Replacing the data is how you reuse one automation against another dataset — the rows change, the loop's switches stay. If a column name does not match a placeholder your steps use, the panel says which, instead of running with empty values.

Two switches govern a loop:

• Voltar para a URL inicial a cada repetição (on by default when the automation has a start URL) re-navigates before each pass. Without it, pass two starts on whatever page pass one ended on, which is rarely what a form-filling loop wants.
• Continuar mesmo se uma iteração falhar keeps going after a bad row.

Generating an automation from text

The Criar com IA tab turns a description into steps. One action per line:

Acessar https://portal.exemplo.com/pedidos
Para cada cliente em [ACME, Globex, Initech]
Preencher cliente com {{cliente}}
Clicar em Buscar
Aguardar 2 segundos
Na linha do {{cliente}} clicar em Editar
Tirar print da tela como pedido-{{cliente}}.png

The Para cada … em [ … ] line becomes the loop table. Generated steps are anchored to the wording of each control rather than to a selector, because no page is open while you write the description — the replay's text lookup resolves them when it runs.

Pointing at one row of a table

Pages that repeat the same button per row are where naive matching goes wrong. LocalPilot accepts three ways of narrowing down, in order of how well they survive the data changing:

You say	It uses
na linha do Globex clicar em Excluir	The content of the enclosing row
clicar em Editar da terceira linha	The position among equal matches
clicar no último botão Abrir	The last match

A named row is a hard constraint: if no row matches, the run reports it instead of acting on some other record.

Replaying against legacy pages

Server-rendered Java stacks (JSF, PrimeFaces, GWT) regenerate element ids on every render and often reveal controls only after a round-trip. So:

• Ids and classes that look generated — j_idt45, form:j_idt12, css-1x2y3z, purely numeric ids — are rejected when building a selector.
• Old submit buttons are matched on their value caption.
• Every recorded step stores a CSS selector, an XPath and the control's visible wording, and resolveTarget falls through them in that order.
• A click is recorded against the control itself, not the <form> that received the submit event or the <span> that happened to be painted inside the button.
• During replay each target is polled until it is both present and enabled, so a control that appears after an AJAX refresh no longer fails the step.

Screenshot destination

Chrome extensions can only write beneath the browser's own Downloads directory. The Pasta de destino field is therefore a relative subfolder (relatorios/2026), not an arbitrary path — there is no directory picker available to an extension's download API. Tick Perguntar onde salvar to get the native save dialog instead. Folder and filename are both placeholder-aware, which is what makes per-iteration naming work.

AI providers

Provider	Where it runs	Notes
Smart Local NLP (default)	In the extension	Rule-based, instant, no download. Also the fallback for the other two.
LiteRT / Gemma	Your GPU, via WebGPU	Prompt assembly and model discovery are in place; the generation step still falls back to the local planner.
Ollama / LiteLLM	A server you run	The only provider that leaves the machine, and it is opt-in from Settings.

Editing steps

Each step carries controls to move it up or down and to duplicate it. A duplicate keeps the original's target, which is the quick way to build "fill this field, now the next one" without re-picking a selector. An automation can also be duplicated whole from its card, so a new variant starts from a working flow rather than from an empty one.

Export and import

Exportar writes a JSON file — one automation from its card, or all of them from the header. Importar reads that file back.

An imported file is data from outside your browser, so it is not trusted. Every step is rebuilt field by field: unknown actions are rejected, unknown fields are dropped rather than carried along, numbers are clamped to their valid ranges, and anything that could smuggle script into a step — a javascript: selector or a non-http navigation target — fails that automation with a message naming it. Imports always get a fresh id, so they can never overwrite something already saved.

Development

cd extension
npm install
npm run typecheck   # tsc --noEmit
npm test            # node:test over the built suite
npm run build       # emits dist/
npm run package     # all three

Tests live in extension/tests and cover the pure logic: intent parsing, the composer, loop expansion, recorded-step normalisation and download-path sanitising. DOM-dependent code (src/content/selectors.ts, src/content/dom.ts) is exercised by hand against a fixture page — see CONTRIBUTING.md.

Layout

extension/
  src/ai/          intent parsing, planners, provider adapters
  src/automation/  step execution, loops, recording, text composer
  src/background/  service worker: tab orchestration, downloads
  src/content/     page context, selector strategy, in-page recorder
  src/options/     dashboard (automations, composer, editor, history, settings)
  src/popup/       toolbar popup
  tests/           node:test suites

Privacy

The content script only ever performs structured, allow-listed DOM actions. There is no eval, no new Function, no telemetry, and no page data transmission. The one exception is the LiteLLM provider, which you enable yourself and point at your own endpoint.

Contributing

Issues and pull requests are welcome — see CONTRIBUTING.md.

License

MIT © athosbes