Agent Bridge

Lets an AI agent read and operate page controls, each step gated.

Source
$npx @dinachi/cli@latest add agent
Region
Volume
40
Switch
Select
Slider
data-agent-hidden

Usage

import { AgentProvider, useAgent } from "@/components/ui/agent"

function App() {
  const agent = useAgent({
    allow: (step) => step.action !== "press" || confirm(`Press ${step.label}?`),
  })
  return <AgentProvider agent={agent}>{/* your app */}</AgentProvider>
}

agent.snapshot()
// [{ ref: "c1", component: "input", label: "Project name", value: "Apollo", actions: ["setValue", "clear"] }, ...]

await agent.act({
  steps: [
    { ref: "Project name", action: "setValue", args: ["Atlas"] },
    { ref: "Region", action: "select", args: ["Europe"] },
  ],
})

Examples

Region
Press Snapshot to see what an agent sees.
Take a snapshot, then fill the input and pick a region.
import { useState } from "react";
import {
  AgentProvider,
  isAgentChange,
  useAgent,
  type AgentActResult,
  type AgentComponent,
} from "@/components/ui/agent";
import { Button } from "@/components/ui/button";
import { Input } from "@/components/ui/input";
import {
  Select,
  SelectContent,
  SelectFieldLabel,
  SelectItem,
  SelectTrigger,
  SelectValue,
} from "@/components/ui/select";

const regions = { us: "United States", eu: "Europe", ap: "Asia Pacific" };

export function Example() {
  const [log, setLog] = useState<string>("");
  const [changedBy, setChangedBy] = useState<string>("");
  const [root, setRoot] = useState<HTMLDivElement | null>(null);
  // Scoped to the demo, so the agent does not also see the docs site around it.
  const agent = useAgent({ root: root ?? undefined });

  const show = (value: AgentComponent[] | AgentActResult) =>
    setLog(JSON.stringify(value, null, 2));

  return (
    <AgentProvider agent={agent}>
      <div className="grid w-full max-w-2xl gap-6 md:grid-cols-2">
        <div className="space-y-4" ref={setRoot}>
          <div className="space-y-2">
            <label htmlFor="agent-project" className="text-sm font-medium">
              Project name
            </label>
            <Input id="agent-project" defaultValue="Apollo" />
          </div>
          <div className="space-y-2">
            <Select
              items={regions}
              defaultValue="us"
              onValueChange={(_, details) =>
                setChangedBy(isAgentChange(details) ? "the agent" : "you")
              }
            >
              <SelectFieldLabel>Region</SelectFieldLabel>
              <SelectTrigger>
                <SelectValue />
              </SelectTrigger>
              <SelectContent>
                {Object.entries(regions).map(([value, label]) => (
                  <SelectItem key={value} value={value}>
                    {label}
                  </SelectItem>
                ))}
              </SelectContent>
            </Select>
            {changedBy && (
              <p className="text-xs text-muted-foreground">
                Region last changed by {changedBy}.
              </p>
            )}
          </div>
          <div className="flex flex-wrap gap-2" data-agent-hidden>
            <Button
              variant="outline"
              size="sm"
              onClick={() => show(agent.snapshot())}
            >
              Snapshot
            </Button>
            <Button
              size="sm"
              onClick={async () =>
                show(
                  await agent.act({
                    steps: [
                      {
                        ref: "Project name",
                        action: "setValue",
                        args: ["Atlas"],
                      },
                      { ref: "Region", action: "select", args: ["Europe"] },
                    ],
                  }),
                )
              }
            >
              Run agent steps
            </Button>
          </div>
        </div>
        <pre
          className="max-h-72 overflow-auto rounded-md border bg-muted/40 p-3 text-xs"
          data-agent-hidden
        >
          {log || "Press Snapshot to see what an agent sees."}
        </pre>
      </div>
    </AgentProvider>
  );
}

Notes

How it works

Inside AgentProvider, form controls and dialogs register their state and actions. An action calls the component's own state setter, so it runs through your onValueChange and onOpenChange like any other change. Unregistered elements are still reached through the accessibility tree. The agent gets a short list of labelled controls and names them by label.

What the agent sees

  • Input, Textarea, OTPField: value, and length for OTPField. setValue, clear.
  • NumberField: value, min, max, step. setValue, increment, decrement.
  • Slider, CompareSlider: value, a number or [low, high], with min, max, step. setValue.
  • Select: value, open, items. select, open, close.
  • Combobox: value, open, items, multiple. select, search, open, close, and setValue when multiple.
  • Autocomplete: value (the text), items. setValue, search.
  • RadioGroup, ToggleGroup: value, items, multiple. select, and setValue when multiple.
  • CheckboxGroup: value, items. setValue; each checkbox in it also takes toggle and setChecked.
  • Checkbox, Switch, Toggle: checked, "mixed" when indeterminate. toggle, setChecked.
  • Dialog, AlertDialog, Drawer, Popover, PreviewCard, Collapsible, ExpandableCard: open. open, close.
  • Accordion item: open, and value as the panel's text while open. open, close.
  • Menu, ContextMenu, Menubar menu: open. open, close; each item takes press, and setChecked when it can be checked.
  • NavigationMenu: open for an item with a trigger. open, close; links take press.
  • Tabs, AnimatedTabs: value, items. select.
  • Carousel: value, items. goTo, by label or by number from 1.
  • Sortable: items in order. move, an item to a position from 1.
  • SwipeableRow: its content as a region, its actions as buttons. press, and dismiss with onDismiss.
  • HoldToConfirm: needs approval by default. press completes the hold.
  • Button: open for dialog triggers. press.
  • Link: value (the href). press.
  • Progress, ProgressRing, Meter: value, min, max. Read only.
  • Toast: value (the description), type. dismiss.

Inside a Field, a control also reports required, invalid and the Field's errors, and in names its Fieldset or Form. A popup with no title of its own goes by its trigger's name.

Naming a control

A step names its control by ref, exact label, partial label, or kind when only one of that kind is on the page. Choices match by label, then value; a miss lists the options. Steps stop at the first failure; earlier steps are not undone.

Agent changes

<Select
  onValueChange={(value, details) => {
    if (isAgentChange(details)) track("agent-picked-region", value)
    if (value === "ap") details.cancel()
  }}
/>

isAgentChange(details) is true for an agent's change. details.cancel() refuses it; the step fails with "The app refused the change."

Controlling access

  • data-agent-hidden removes a region from the snapshot.
  • data-agent-part marks an element belonging to a useAgentControl control, so it is not listed twice. Controls inside it are still found.
  • data-agent-secret reports a value as [redacted]. Password, one-time-code and card fields are secret already.
  • data-agent-confirm="Reason" marks a control that needs a person. Without allow the step fails with that reason; with it, allow gets the reason as confirm.
  • data-agent-label names a control when its accessible name is not what an agent should use.
  • data-agent-dismiss marks the button dismiss presses in a toast.
  • Content behind an open modal is left out of the snapshot.

WebMCP

useAgent({ webmcp: { allow: (step) => confirm(`Let the browser agent ${step.action} ${step.label}?`) } })

Registers snapshot, query, act and one tool per app action with the browser's agent. Does nothing without WebMCP.

For other model SDKs, agentToolsFor(agent) returns the tool definitions and runAgentTool(agent, name, input) runs one.

API

rootParentNode = document
Where to look for controls. The default finds portalled popups.
allow
agent
(step, { signal }) => boolean | Promise<boolean>
Approves or refuses each step. Without it, every step runs except ones needing a person, which fail.
timeoutnumber = 300000
Milliseconds allow may take before the step fails.
onAction(step) => void
Runs after each step that ran.
webmcp
agent
{ allow }
Registers the tools with the browser's agent. Its allow is required and runs after the bridge's own.

Accessibility

  • The agent reads the same accessible names a screen reader does.
  • Label every control. Two unnamed controls of one kind cannot be told apart.