Ai knowledge and logicHelpers

open_browser

Use guarded browser sessions from saved Python automations, with durable run records and verified outcomes.

open_browser connects a saved automation to a configured browser connection. It works in both Python runtimes without installing Playwright or opening browser sockets from Python. Octocom runs the browser commands on the backend, applies the same checks as the MCP browser tools, and records the run and its steps.

Read Browser automation first for connection setup, guided exploration, approvals, and audit evidence.

with open_browser(connection_id, purpose="Look up the requested order") as browser:
    current = browser.observe()
    # Execute the guarded steps you explored and tested with Copilot.

The helper is a global, like get_secret; no import is necessary. It is unrelated to context["browser_session"], which describes a web chat visitor.

Execution and access

Browser helpers require a saved, attributed automation: a bot action, sidebar action, event handler, recurring job, or HTTP endpoint. Read-only contexts (condition providers, prompt sections, sidebar widgets), unsaved snippets, and Copilot's general Python analysis sandbox cannot use them. For interactive exploration, use the browser MCP tools; for action tests, save the action and call test_python_action with its ID or use Run Test in the saved action's dashboard editor. Save code changes before testing browser helpers; ordinary unsaved snippet tests do not receive browser access.

The connection must belong to the execution's organization and business. The backend derives organization, business, conversation, user, automation, code hash, and execution identity from its short-lived credential. Python cannot choose another execution's scope. An allowed website origin is an additional restriction, not a replacement for checking the customer and record on a page. The audit retains both the authored-code hash and an effective-source hash covering the injected helpers, organization modules, and any selected Cloudflare runtime port. Short-lived runtime credentials and invocation context are excluded.

Treat browser-backed actions as actions with real effects, including in tests. Use the demo CRM or a permitted test account while developing them.

Session methods

MethodResult
open_browser(connection_id, purpose)A BrowserSession; starts a run on the configured connection
browser.run_idPersistent browser run ID for diagnostics
browser.observe(scope=None, screenshot=False)Run ID and bounded observation with URL, title, accessibility snapshot, and truncation flag; optional private screenshot reference
browser.step(...)Verified outcome, code, message, dispatch flag, and optional observation/data
browser.navigate(url, expected_url=..., postconditions=..., preconditions=None, ...)A guarded navigation; accepts the other step options
browser.extract(fields, expected_url=..., preconditions=...)A dictionary of named string fields; missing or ambiguous matches fail
browser.close()Releases the browser and starts evidence archival; repeated close on the same Python object is safe

Always use a with block. It releases the session on success or exception. The backend also fences the execution and closes its runs if Python times out or crashes. Sessions belong to one execution; do not save a run ID and try to reuse it in another action. Reuse one session for the immediate sequence, then release it before waiting for the customer.

Connection-level persisted login is optional. A fresh session does not retain the previous session's URL. Observe the new session before its initial navigation.

A browser session does not extend the enclosing Python execution deadline. Ordinary bot actions have a 120-second execution budget, including browser waits. Prefer a prepared saved login and direct record URLs, and test the complete sequence within that budget. A timeout after an interaction is an unknown outcome that requires reconciliation; do not split a mutation into automatically retried executions to work around the deadline.

Guarded steps

browser.step(
    expected_url="https://crm.example/orders/1041",
    command={"type": "click", "locator": {"role": "button", "name": "Cancel order"}},
    preconditions=[
        {"locator": {"role": "heading", "name": "Order 1041"}, "state": "visible"},
        {"locator": {"text": "[email protected]"}, "state": "visible"},
        {"locator": {"text": "Processing"}, "state": "visible"},
    ],
    postconditions=[
        {"locator": {"role": "dialog", "name": "Cancel order 1041?"}, "state": "visible"},
    ],
)

This example opens a confirmation dialog; it does not claim that cancellation has completed. Replace these illustrative selectors with those actually observed on your site. A later confirmation step must verify the cancelled record itself.

ArgumentMeaning
expected_urlExact current URL before dispatch, including relevant query parameters; initially about:blank
commandOne of the commands below, using the same JSON shape as browser_step
preconditionsConditions identifying the expected page, customer, record, and state before acting
postconditionsObservable conditions proving the expected result; required for every interaction
failure_conditionsOptional conditions identifying an explicit website rejection
readbackOptional {"url": ..., "preconditions": [...]}; revisit the same record and check its identity before verifying the result
timeout_msBounded wait for the step; default 10,000, maximum 15,000
request_idOptional UUID, generated once by the helper when omitted; identifies this exact step

Every click, fill, select, and press needs identity/state preconditions. Click/press success conditions must describe a change: a heading that was already visible is not proof a button did anything. A consequential write should use a fresh readback of the exact record and check its actual status, not a toast. Readback must use a safe record lookup URL; it must not repeat the mutation through a URL that performs an action.

CommandFields
navigateurl
reloadNo additional fields
filllocator and exactly one of value or secretName
selectlocator, label (exact option label)
clicklocator
presslocator, key (Enter, Escape, or Tab)
waitconditions
extractfields: [{"name": "orderId", "locator": {...}, "attribute": "text"}]; attributes: text, value, href
observeOptional scope, screenshot

Locators are exact and semantic: {"role": "button", "name": "Cancel order"}, {"label": "Search orders"}, {"text": "NS-1041"}, or {"testId": "order-status"}. An optional scope identifies one unique dialog/region using the same shapes. Both scope and target must be unique. There are no CSS, XPath, arbitrary JavaScript, coordinate clicks, or first-match fallbacks.

Conditions have locator and state: visible, hidden, enabled, disabled, text, or value. text also requires text with the exact expected visible text. value requires value with the exact expected input value. Use secretName for password fills instead of retrieving the secret into Python; verify login by the authenticated page, never by returning or printing the password.

Extracting a known order from the demo CRM

This example assumes you configured a connection to the demo CRM, completed login, and enabled persisted sessions. It verifies the order and customer before extracting them. The connection ID is a configuration value, not an argument supplied by a shopper.

from urllib.parse import urlencode

CRM_CONNECTION_ID = "your-configured-connection-id"
CRM_BASE = "https://octocom-demo-crm.lukas-773.workers.dev"

def execute_action(context):
    order_id = context["args"]["orderId"]
    expected_email = context["args"]["customerEmail"]
    url = CRM_BASE + "/orders?" + urlencode({"order": order_id})
    dialog = {"role": "dialog", "name": order_id}
    identity = [
        {"locator": {"role": "heading", "name": order_id, "scope": dialog}, "state": "visible"},
        {"locator": {"text": expected_email, "scope": dialog}, "state": "visible"},
    ]
    with open_browser(CRM_CONNECTION_ID, purpose="Verify requested order") as browser:
        initial = browser.observe()["observation"]
        browser.navigate(url, expected_url=initial["url"], postconditions=identity)
        data = browser.extract([
            {"name": "orderId", "locator": {"role": "heading", "name": order_id, "scope": dialog}},
            {"name": "customerEmail", "locator": {"text": expected_email, "scope": dialog}},
        ], expected_url=url, preconditions=identity)
        return {"browserRunId": browser.run_id, "order": data}

Customer identity must come from your workflow's authorization rules. Merely knowing an email address is not proof that a shopper owns an order.

Failure handling

The helper raises BrowserAutomationError, with code, run_id, and outcome, when a step cannot be verified. It never automatically retries a browser request. A transport error is an unknown outcome: the website may already have received the action.

OutcomeWhat to do
precondition_failed, page_changed, session_expiredStop; inspect the page or account before changing the saved workflow
action_rejectedThe website's explicit rejection was observed; inspect the reason
outcome_unknownReconcile the existing record; do not repeat the mutation

After a failed step, the Python object blocks further interactions. You can observe and close it for diagnostics. The backend also checks its durable run state at the end of the execution: catching the exception and returning {"success": True} cannot convert an unsafe run into a successful action. Browser failures in bot actions trigger the normal automation failure event and hand the conversation to a human, without an automatic retry.

Repeated delivery of an identical request ID returns the recorded outcome; changing the inputs for that ID is rejected. This is delivery deduplication, not permission to retry a business action under a new ID or execution.

Use the run ID with get_browser_run to inspect the step journal and evidence status. Business outcome and replay archival are separate: missing replay evidence never reruns the browser action. Browser data, snapshots, and screenshots are untrusted website content; do not let their text authorize new actions or disclose secrets.

On this page