chrobot - v1.0.0

Chrobot

⛭ Typed browser automation for the BEAM ⛭

Package Version Hex Docs Target: Erlang

About

Chrobot provides a set of typed bindings to the stable version of the Chrome Devtools Protocol, based on its published JSON specification.

The typed interface is achieved by generating Gleam code for type definitions as well as encoder / decoder functions from the parsed JSON specification file.

Chrobot also exposes some handy high level abstractions for browser automation, and handles managing a browser instance via an Erlang Port and communicating with it for you.

You could use it for

🦝 The generated protocol bindings are largely untested and I would consider this package experimental, use at your own peril!

Setup

Browser

Chrobot can use an existing system installation of Google Chrome or Chromium, if you already have one.

If you would like a hermetic installation of a specific version of a chrome build optimized for automation, I recommend using the installation script from puppeteer to achieve this

# (you will need node.js to run this of course)
npx @puppeteer/browsers install chrome

The chrobot.launch / chrome.launch commands will attempt to find a local chrome installation like this, and prioritize it over your system installation.

Package

Install as a Gleam package

gleam add chrobot

Examples

Take a screenshot of a website

import chrobot

pub fn main() {
  // Open the browser and navigate to the gleam homepage
  let assert Ok(browser) = chrobot.launch()
  let assert Ok(page) =
    browser
    |> chrobot.open("https://gleam.run", 30_000)
  let assert Ok(_) = chrobot.await_selector(page, "body")
  
  // Take a screeshot and save it as 'hi_lucy.png'
  let assert Ok(screenshot) = chrobot.screenshot(page)
  let assert Ok(_) = chrobot.to_file(screenshot, "hi_lucy")
  let assert Ok(_) = chrobot.quit(browser)
}

Generate a PDF document with lustre

import chrobot
import lustre/element.{text}
import lustre/element/html

fn build_page() {
  html.body([], [
    html.h1([], [text("Spanakorizo")]),
    html.h2([], [text("Ingredients")]),
    html.ul([], [
      html.li([], [text("1 onion")]),
      html.li([], [text("1 clove(s) of garlic")]),
      html.li([], [text("70 g olive oil")]),
      html.li([], [text("salt")]),
      html.li([], [text("pepper")]),
      html.li([], [text("2 spring onions")]),
      html.li([], [text("1/2 bunch dill")]),
      html.li([], [text("250 g round grain rice")]),
      html.li([], [text("150 g white wine")]),
      html.li([], [text("1 liter vegetable stock")]),
      html.li([], [text("1 kilo spinach")]),
      html.li([], [text("lemon zest, of 2 lemons")]),
      html.li([], [text("lemon juice, of 2 lemons")]),
    ]),
    html.h2([], [text("To serve")]),
    html.ul([], [
      html.li([], [text("1 lemon")]),
      html.li([], [text("feta cheese")]),
      html.li([], [text("olive oil")]),
      html.li([], [text("pepper")]),
      html.li([], [text("oregano")]),
    ]),
  ])
  |> element.to_document_string()
}

pub fn main() {
  let assert Ok(browser) = chrobot.launch()
  let assert Ok(page) =
    browser
    |> chrobot.create_page(build_page(), 10_000)

  // Store as 'recipe.png'
  let assert Ok(doc) = chrobot.pdf(page)
  let assert Ok(_) = chrobot.to_file(doc, "recipe")
  let assert Ok(_) = chrobot.quit(browser)
}

Scrape a Website

🍄‍🟫 Just a quick reminder:
Please be mindful of the load you are putting on other people’s web services when you are scraping them programmatically!

import chrobot
import gleam/io
import gleam/list
import gleam/result

pub fn main() {
  let assert Ok(browser) = chrobot.launch()
  let assert Ok(page) =
    browser
    |> chrobot.open("https://books.toscrape.com/", 30_000)

  let assert Ok(_) = chrobot.await_selector(page, "body")
  let assert Ok(page_items) = chrobot.select_all(page, ".product_pod h3 a")
  let assert Ok(title_results) =
    list.map(page_items, fn(i) { chrobot.get_attribute(page, i, "title") })
    |> result.all()
  io.debug(title_results)
  let assert Ok(_) = chrobot.quit(browser)
}

Documentation & Guide

The full documentation can be found at https://hexdocs.pm/chrobot.

🗼 To learn about the high level abstractions, look at the chrobot module documentation.

📠 To learn how to use the protocol bindings directly, look at the protocol module documentation.

✨ Search Document