Run JS Step

# RUN JS

This is an advanced step that allows running JavaScript code in scope of the opened web page. The JavaScript code you declare in the “js code to run” will be sent to the page along with the parameters for evaluation.

If you are familiar with JavaScript, you can do a lot with the help of this command, e.g. scroll the page up or down based on parameter input:

Typical use-case scenarios include:

• Performing arithmetic operations.
• Clearing text content with regular expressions.
• Extracting values from the page DOM.
• Scrapping data from the page DOM.
• Hiding elements on page.

RUN JS step will store the result of the JavaScript code execution in the parameter you specify in the “store response in” field, or "param" in YAML file. By default, the result is stored in the {{param}} parameter.

# Example A: Summing Values

Here is an example that sums up two values:

ChatML declaration

meta:
  title: Sum values
  description: ''
  category: User
  name: cmd-sum-values
steps:
  - type: ask
    message: 'Please provide p1:'
    param: p1
    default: ''
  - type: ask
    message: 'Please provide p2:'
    param: p2
    default: ''
  - type: js
    code: return Number(args.p1) + Number(args.p2)
    args: p1, p2
    param: sum
  - type: say
    message: The sum is {{sum}}
version: 1

# Example B: Hiding Elements by Text

This command hides elements containing text:

ChatML declaration

meta:
  title: Hide elements by text
  description: ''
  category: User
  name: cmd-hide-elements-by-text
steps:
  - type: ask
    message: What to hide?
    param: text
    default: ''
  - type: js
    code: |-
      const elements = $harpa.query.run(
        { $text: args.text }, { mult: true })

      elements.forEach(e => e.remove())
    args: text
version: 1

# Example C: Extracting Data from Page

And this command extracts and returns all prices from the page:

ChatML declaration

meta:
  title: Find prices
  description: ''
  category: User
  name: cmd-find-prices
steps:
  - type: js
    code: |-
      const items = await $harpa.parser
        .parse(null, { extract: true })

      return items
        .filter(i => i.type === 'price')
        .map(p => ({
          value: p.value,
          metric: p.metric,
        }))
    param: prices
  - type: say
    message: 'Prices are: ```{{prices}}```'
version: 1

# Example D: Extracting YouTube Video Links

A more elaborate example of a single RUN JS step that extracts YouTube video links from a web page:

- type: js
  code: |-
    return Array
      .from(document.querySelectorAll('a[href*="/watch"]'))
      .map(a => {
        const ariaLabel = a.getAttribute('aria-label') || ''
        const viewsMatch
          = ariaLabel.match(/(\d[\d,]*) views/)
        const timeAgoMatch
          = ariaLabel.match(/views (.+) ago/)
        const url = new URL(a.href)

        return {
          url: 'https://www.youtube.com'
            + url.pathname
            + '?v='
            + url.searchParams.get('v'),
          title:
            a.getAttribute('title') || a.innerText.trim(),
          views: viewsMatch ? parseInt(viewsMatch[1].replace(/,/g, '')) : null,
          publishedTimeAgo: timeAgoMatch ? timeAgoMatch[1].trim() : null
        }
      })
      .filter(item =>
        item.url.includes('/watch?v=') &&
        item.views !== null &&
        item.publishedTimeAgo !== null
      )
  param: videos
  silent: true

# Async JS Functions

RUN JS step supports async JS functions. You can use await keyword in your RUN JS code e.g. to perform HTTP requests.

However, HARPA AI will terminate the RUN JS step after 15 seconds if it doesn't receive a response.

# Available JS API

You can console.log($harpa) in the RUN JS command to inspect the in-page API HARPA provides. Useful methods include:

• $harpa.page.click to click elements on page.
• $harpa.page.scroll to scroll page up and down.
• $harpa.page.parse to parse data from page.
• $harpa.page.query to query elements on page (e.g. by text content, css).
• $harpa.page.fetch to send data over HTTP or request HTTP resources.
• $harpa.page.idle to wait for page to stop updating.
• $harpa.inspector.* to inspect document nodes.