REST API Reference

# Base URL

HARPA GRID accepts POST requests at:

https://api.harpa.ai/api/v1/grid

# Authentication Header

All API requests require authentication using an API key in the Authorization header.

Authorization: Bearer $HARPA_API_KEY

You can get your API key from the HARPA AI Chrome Extension, under the AUTOMATE tab. Please find more instructions in the Node Setup Guide.

# Send Action Endpoint

Sending web browser automation actions to your Node is done via the /grid endpoint.

Endpoint: /grid

Method: POST

Content-Type: application/json

# Request Structure

The request body must be sent as a JSON object string. Here is the combined structure for all types of actions:

{
  "action": "<one of: scrape, serp, command>",
  "url": "<optional string>",
  "query": "<optional string>",
  "name": "<optional string>",
  "inputs": [
    "<optional string>"
  ],
  "resultParam": "<optional string>",
  "grab": [
    {
      "selector": "<string>",
      "selectorType": "<optional string>",
      "at": "<optional string or number>",
      "take": "<optional string>",
      "label": "<optional string>"
    }
  ],
  "node": "<optional string>",
  "resultsWebhook": "<optional URL string>",
  "timeout": "<optional number in miliseconds>"
}

# Parameters

• action — Required. Specify the action your Node has to perform (e.g., "scrape", "serp", "command").

• url — Optional. The starting page URL to open in Node before running the action. For the scrape action this is the page to be scraped. For the command action, this is the page to run the AI command over. Ignored by the serp and ping actions.

• query — Optional. The search term or a query string to search for. Only used by the serp action. Supports search parameters like site:example.com or intitle:keyword.

• name — Optional. A command name to execute such as Summary. Only used by the command action. Supports both the built-in and custom commands.

• inputs — Optional. An array of Strings, each one passed down into command in place of the user input. Inputs are used to bypass waiting for the user input in multi-step commands. For example [ "REPORT", "DONE" ] for the Summary command. Only used by the command action.

• resultParam — Optional. A HARPA {{parameter}} to interpret as the command result. Only used by the command action. By default is set to "message" to take the last chat message. Supports dot notation, e.g. "g.data.email". Refer to AI Commands Guide for more details.

• grab — Optional. An array of HTML Element selectors to scrape from the page. Only used by the scrape action. Refer to the Scraping Web Page Elements and Data section for more details.

• node — Optional. A target Node ID which should run the command. If omitted, the first available Node will be used. You can pass a single Node ID (such as "r2d2"), multiple Node IDs (such as "r2d2 c3po pqtx"), or a number (such as "5") to run action over the first N available Nodes.

• timeout — Optional. Synchronous action execution timeout. Default and max value is 300000 (5 minutes). HARPA GRID API does not return a timeout error. Instead it waits for Nodes to complete the action in the given timeout, and returns the results in a single array.

• resultsWebhook — Optional. An asynchronous webhook URL to send the results to upon completion. If provided, HARPA GRID API will maintain the action for 30 days. Each specified Node which goes live during the timeframe will receive the action request, and report result to the webhook URL. Asynchronous actions are preferred for long running tasks, or if your Node (PCs / Mac) goes offline frequently.

# Available Actions

# Web Page Scraping

Action: scrape.

The scrape action allows using your Browser to extract data from web pages, even from the web systems behind logins (your browser has to be logged in).

Scrape action can yield the entire page content in markdown format, or content of specific HTML elements.

# Full Page Scraping

To scrape the full page content in markdown format, skip the grab parameter:

{
  "action": "scrape",
  "url": "https://harpa.ai/blog",
  "node": "r2d2",
  "resultsWebhook": "https://webhook.site/240776d5-dee7-44c5-b7c3-6d2b6d7bbae5",
  "timeout": "15000"
}

# Targeted Element Scraping

To scrape specific elements, provide the grab parameter with an array of items:

{
  "action": "scrape",
  "url": "https://harpa.ai/blog",
  "grab": [
    {
      "selector": ".PostList__postTitle",
      "selectorType": "auto",
      "at": "all",
      "take": "innerText",
      "label": "data"
    }
  ],
  "node": "r2d2",
  "timeout": "15000"
}

# Scraping Web Page Elements and Data

The grab parameter specifies what HTML elements to grab from the Web page. It is passed in a form of item array, each item having the following fields:

• selector — Required. CSS, XPath, or Text content selector to target specific elements. Examples: ".PostList__postTitle", "//h2", "h2".

• selectorType — Optional. Type of selector to use. Default is "auto". Options: "auto", "css", "xpath", "text".

• at — Optional. Position of an element to capture. Default is "first". Options: "all", "first", "last", Number (e.g., 0, -1). If "all" is used, the result will be an array of all matched elements.

• take — Optional. Type of data to extract. Default is "innerText". Options: "innerText", "textContent", "id", "className", "nodeName", "nodeType", "attributes", "styles", "innerHTML", "outerHTML", "value", "href", "[attributeName]", "(styleName)".

• label — Optional. Custom label for the extracted data in results payload. Default is "data".

# Search the Web

Action: serp.

Example Request Body:

{
  "action": "serp",
  "query": "HARPA AI use cases",
  "node": "r2d2",
  "resultsWebhook": "https://webhook.site/240776d5-dee7-44c5-b7c3-6d2b6d7bbae5",
  "timeout": "15000"
}

# Run AI Command

Action: command.

Example Request Body:

{
  "action": "command",
  "url": "https://harpa.ai/blog",
  "name": "Extract data",
  "inputs": "List of the latest 3 blog posts with two columns: Title and Description",
  "node": "r2d2",
  "resultParam": "message",
  "resultsWebhook": "https://webhook.site/240776d5-dee7-44c5-b7c3-6d2b6d7bbae5",
  "timeout": "15000"
}

# Testing API Calls

You can test your GRID API calls using the Terminal app and a specifying the webhooks to receive the results. For example, you can use the webhook.site service to receive the results of your API calls.