Click & Paste Steps

# CLICK

Select an interface element for HARPA to click on, simulating a mouse button press. Click "Grab" and choose the element to be clicked.

# Example

You can automate sending comments on YouTube. First, use CLICK to activate the text insertion window, followed by the PASTE step.

ChatML declaration

Under the hood, HARPA AI uses AI Element Selectors to find the element to click on. You can use the "Grab" button to select the element to click on or customize the selector manually in the YAML file.

- type: click
  selectorType: ai
  selector: null
  item: null
  showMore: false
  onFailure: skip
  waitForIdle: true

- type: click
  selectorType: ai
  selector:
    - $matches:
        - $tag: A
        - $role: link
        - $class: Button
        - $class: GetPluginButton
        - $attribute: href=https://get.harpa.ai
        - $attribute: target=_blank
        - $attribute: type=button
        - $style: Poppins:24px:500:normal
        - $content: ADD TO BROWSER
        - $class: LandingHero__buttonContainer
          traverse: '0'
        - $class: LandingHero__content
          traverse: '0:1:0'
        - $class: LandingHero
          traverse: '0:0:1:0'
        - $class: LandingContent
          traverse: '0:2:0:0:1:0'
        - $class: Page__content
          traverse: '0:0:2:0:0:1:0'
        - $anchor: >
            Bring AI to your browser. Chat with websites, PDFs, videos, write emails, SEO articles, tweets, automate workflows, monitor prices & data. Bing AI & Notion AI alternative.
          shift: '-12:-106'
        - $text: >
            Bring AI to your browser. Chat with websites, PDFs, videos, write emails, SEO articles, tweets, automate workflows, monitor prices & data. Bing AI & Notion AI alternative.
          traverse: '-1:1:0'
        - $anchor: '#1'
          shift: '18:250'
        - $text: '#1'
          traverse: '-2:1:0'
        - $anchor: AI AUTOMATION AGENT FOR CHROME
          shift: '-40:250'
        - $text: AI AUTOMATION AGENT FOR CHROME
          traverse: '-2:1:0'
      min: 9
    - $size: 1
  showMore: false
  onFailure: skip
  waitForIdle: true

# PASTE

This step allows you to insert a given text into any page's text field. It defaults to the largest input field, but manually selecting a target input is possible.

Some websites, like YouTube, may require a CLICK step before pasting to activate the input field.

ChatML declaration

# full version
- type: paste
  text: '{{gpt}}'
  close: false

# short version
- paste: '{{gpt}}'

# target selector
- paste: '{{gpt}}'
  selector:
    - $matches:
        - $tag: SPAN
        - $role: text
        - $class: input__email
        - $class: input-box
        - $attribute: role=text
        - $style: Roboto:14px:500:normal
        - $content: Email
      min: 5
    - $size: 1

# Example

Automate a thank-you comment for helpful content on a YouTube video using a combination of steps: COMMAND (or GPT) + CLICK + PASTE + CLICK.

For example, you can automate logging into a Mail service using NAVIGATE + PASTE + PASTE + CLICK steps in HARPA AI. To automate this, create a custom command with the specified steps with inputs.

Each PASTE and CLICK step lets you target a specific element for interaction.

ChatML declaration

meta:
  title: ProtonMail Login
  description: ''
  category: User
  name: pqiqyO9Tl7blDTycH14sM
steps:
  - type: navigate
    url: https://account.proton.me/login
    waitForIdle: true
  - type: paste
    text: [email protected]
    selectorType: ai
    selector:
      - $matches:
          - $tag: INPUT
          - $id: username
          - $class: input-element
          - $class: w100
          - $attribute: autocomplete=username
          - $attribute: autocapitalize=off
          - $attribute: autocorrect=off
          - $attribute: spellcheck=false
          - $attribute: aria-invalid=false
          - $attribute: aria-describedby=id-3
          - $attribute: value
          - $attribute: data-last-active-input
          - $testId: input-input-element
          - $style: '"Inter var":14px:400:normal'
          - $role: form
            traverse: '2:1:0:0:0'
          - $class: sign-layout-main-content
            traverse: '0:2:1:0:0:0'
          - $class: ui-standard
            traverse: '1:0:2:1:0:0:0'
          - $class: sign-layout
            traverse: '1:0:2:1:0:0:0'
          - $class: shadow-color-primary
            traverse: '1:0:2:1:0:0:0'
          - $anchor: Email or username
            shift: '17:36'
          - $text: Email or username
            traverse: '-2:1:0:0:0'
          - $anchor: Password
            shift: 17:-61
          - $text: Password
            traverse: '-3:2:1:0:0:0'
          - $anchor: Enter your Proton Account details.
            shift: '17:80'
          - $text: Enter your Proton Account details.
            traverse: '-2:1:0:2:1:0:0:0'
        min: 11
      - $size: 1
    item:
      type: element
      container:
        __$ht: element
        id: 1
      value: Element
    showMore: false
    onFailure: skip
    waitForIdle: false
  - type: paste
    text: YouWontGuessThisOne
    selectorType: ai
    selector:
      - $matches:
          - $tag: INPUT
          - $role: textbox
          - $id: password
          - $class: input-element
          - $class: w100
          - $attribute: autocomplete=current-password
          - $attribute: autocapitalize=off
          - $attribute: autocorrect=off
          - $attribute: spellcheck=false
          - $attribute: aria-invalid=false
          - $attribute: aria-describedby=id-4
          - $attribute: type=text
          - $attribute: value
          - $attribute: data-last-active-input
          - $testId: input-input-element
          - $style: '"Inter var":14px:400:normal'
          - $role: form
            traverse: '3:1:0:0:0'
          - $class: sign-layout-main-content
            traverse: '0:3:1:0:0:0'
          - $class: ui-standard
            traverse: '1:0:3:1:0:0:0'
          - $class: sign-layout
            traverse: '1:0:3:1:0:0:0'
          - $class: shadow-color-primary
            traverse: '1:0:3:1:0:0:0'
          - $anchor: Password
            shift: '17:36'
          - $text: Password
            traverse: '-2:1:0:0:0'
          - $anchor: Keep me signed in
            shift: '-11:-60'
          - $text: Keep me signed in
            traverse: '-4:3:1:0:0:0'
          - $anchor: >-
              Not your device? Use a private browsing window to sign in and
              close it when done. Learn more
            shift: '-11:-80'
          - $text: >-
              Not your device? Use a private browsing window to sign in and
              close it when done. Learn more
            traverse: '-3:3:1:0:0:0'
        min: 11
      - $size: 1
    item:
      type: element
      container:
        __$ht: element
        id: 2
      value: Element
    showMore: false
    onFailure: skip
    waitForIdle: false
  - type: click
    selectorType: ai
    selector:
      - $matches:
          - $tag: BUTTON
          - $role: button
          - $class: button
          - $class: w100
          - $class: button-large
          - $class: button-solid-norm
          - $class: mt-6
          - $attribute: aria-busy=false
          - $attribute: type=submit
          - $style: '"Inter var":16px:400:normal'
          - $content: Sign in
          - $role: form
            traverse: '5'
          - $class: sign-layout-main-content
            traverse: '0:5'
          - $class: ui-standard
            traverse: '1:0:5'
          - $class: sign-layout
            traverse: '1:0:5'
          - $class: shadow-color-primary
            traverse: '1:0:5'
          - $anchor: New to Proton? Create account
            shift: 0:-51
          - $text: New to Proton? Create account
            traverse: '-1:5'
          - $anchor: Trouble signing in?
            shift: 0:-105
          - $text: Trouble signing in?
            traverse: '-2:5'
          - $anchor: >-
              Not your device? Use a private browsing window to sign in and
              close it when done. Learn more
            shift: '163:73'
          - $text: >-
              Not your device? Use a private browsing window to sign in and
              close it when done. Learn more
            traverse: '-3:5'
        min: 5
      - $size: 1
    showMore: false
    onFailure: skip
    waitForIdle: true
version: 1

# SCROLL

SCROLL step is not implemented in HARPA AI yet. Please use RUN JS step to scroll the page until SCROLL step is implemented:

# scrolls one viewport
- type: runjs
  code: return $harpa.page.scroll()
# scrolls to the bottom of the page
- type: runjs
  code: return $harpa.page.scroll('bottom')
# scrolls to the top of the page
- type: runjs
  code: return $harpa.page.scroll('top')
# scrolls to position 1000px from the top of the page
- type: runjs
  code: 'return $harpa.page.scroll({ y: 1000 })'