Vibe.WebTools (vibe v0.2.0)

Copy Markdown View Source

Provider-neutral web search and fetch API exposed to eval as Web.

Web.search/2 and Web.fetch/2 keep agent-facing code stable while concrete providers handle vendor-specific APIs. Exa is the default search provider; a local Req implementation is the default fetch provider.

Summary

Types

fetch_result()
search_result()

Functions

fetch(url, opts \\ [])

Fetches a URL through the configured fetch provider.

fetch!(url, opts \\ [])

Raises on URL fetch errors.

filter_domain(search, domain)

Filters search results to URLs containing a domain substring.

parse_html(value)

Parses HTML from a fetch result or binary string with Floki.

parse_html!(value)

Parses HTML from a fetch result or binary string with Floki, raising on failure.

search(query, opts \\ [])

Searches the web through the configured search provider.

search!(query, opts \\ [])

Raises on web search errors.

select(result, selector)

Extracts matching HTML from a fetched page with a CSS selector.

select!(result, selector)

Extracts matching HTML from a fetched page with a CSS selector, raising on failure.

take(search, count)

Keeps at most count search results.

truncate(value, opts \\ [])

Truncates fetched content or plain text using the shared tool-output byte limit.

Types

fetch_result()

@type fetch_result() :: Vibe.WebTools.FetchResult.t()

search_result()

@type search_result() :: Vibe.WebTools.SearchResult.t()

Functions

fetch(url, opts \\ [])

@spec fetch(
  String.t(),
  keyword()
) :: {:ok, Vibe.WebTools.FetchResult.t()} | {:error, term()}

Fetches a URL through the configured fetch provider.

fetch!(url, opts \\ [])

@spec fetch!(
  String.t(),
  keyword()
) :: Vibe.WebTools.FetchResult.t()

Raises on URL fetch errors.

filter_domain(search, domain)

@spec filter_domain(
  {:ok, Vibe.WebTools.SearchResult.t()} | Vibe.WebTools.SearchResult.t(),
  String.t()
) ::
  {:ok, Vibe.WebTools.SearchResult.t()} | Vibe.WebTools.SearchResult.t()

Filters search results to URLs containing a domain substring.

parse_html(value)

@spec parse_html(Vibe.WebTools.FetchResult.t() | String.t()) ::
  {:ok, Vibe.WebTools.HTML.html_tree()} | {:error, term()}

Parses HTML from a fetch result or binary string with Floki.

parse_html!(value)

@spec parse_html!(Vibe.WebTools.FetchResult.t() | String.t()) ::
  Vibe.WebTools.HTML.html_tree()

Parses HTML from a fetch result or binary string with Floki, raising on failure.

search(query, opts \\ [])

@spec search(
  String.t(),
  keyword()
) :: {:ok, Vibe.WebTools.SearchResult.t()} | {:error, term()}

Searches the web through the configured search provider.

search!(query, opts \\ [])

@spec search!(
  String.t(),
  keyword()
) :: Vibe.WebTools.SearchResult.t()

Raises on web search errors.

select(result, selector)

@spec select(Vibe.WebTools.FetchResult.t(), String.t()) ::
  {:ok, Vibe.WebTools.FetchResult.t()} | {:error, term()}

Extracts matching HTML from a fetched page with a CSS selector.

select!(result, selector)

@spec select!(Vibe.WebTools.FetchResult.t(), String.t()) ::
  Vibe.WebTools.FetchResult.t()

Extracts matching HTML from a fetched page with a CSS selector, raising on failure.

take(search, count)

@spec take(
  {:ok, Vibe.WebTools.SearchResult.t()} | Vibe.WebTools.SearchResult.t(),
  non_neg_integer()
) ::
  {:ok, Vibe.WebTools.SearchResult.t()} | Vibe.WebTools.SearchResult.t()

Keeps at most count search results.

truncate(value, opts \\ [])

@spec truncate(Vibe.WebTools.FetchResult.t(), keyword() | pos_integer()) ::
  Vibe.WebTools.FetchResult.t()
@spec truncate(String.t(), keyword() | pos_integer()) :: String.t()

Truncates fetched content or plain text using the shared tool-output byte limit.