Crawler v1.0.0 Crawler.Linker.PathFinder

Finds different components of a given URL, e.g. its domain name, directory path, or full path.

The safe option in some the functions indicates whether the return value should be transformed in order to be safely used as folder and file names.

Link to this section Summary

Functions

Finds the base path of a given page

Finds the domain name with port number (e.g. example.org:8080)

Finds the full path of a given page

Finds the URL scheme (e.g. https://)

Link to this section Functions

Link to this function find_base_path(url, safe \\ true)

Finds the base path of a given page.

Examples

iex> PathFinder.find_base_path("http://hi.hello")
"hi.hello"

iex> PathFinder.find_base_path("https://hi.hello:8888/dir/world")
"hi.hello-8888/dir"

iex> PathFinder.find_base_path("https://hi.hello:8888/dir/world", false)
"hi.hello:8888/dir"
Link to this function find_domain(url, safe \\ true)

Finds the domain name with port number (e.g. example.org:8080).

Examples

iex> PathFinder.find_domain("http://hi.hello")
"hi.hello"

iex> PathFinder.find_domain("https://hi.hello:8888/world")
"hi.hello-8888"

iex> PathFinder.find_domain("https://hi.hello:8888/world", false)
"hi.hello:8888"
Link to this function find_path(url, safe \\ true)

Finds the full path of a given page.

Examples

iex> PathFinder.find_path("http://hi.hello")
"hi.hello"

iex> PathFinder.find_path("https://hi.hello:8888/world")
"hi.hello-8888/world"

iex> PathFinder.find_path("https://hi.hello:8888/world", false)
"hi.hello:8888/world"
Link to this function find_scheme(url)

Finds the URL scheme (e.g. https://).

Examples

iex> PathFinder.find_scheme("http://hi.hello")
"http://"

iex> PathFinder.find_scheme("https://hi.hello:8888/")
"https://"