Euros v0.2.1 Euros.Core View Source

Provides the function for crawl web page

Link to this section Summary

Link to this section Functions

Link to this function crawl(url, callback, option \\ %Euros.HTTPOption{}, pattern \\ ~r".*") View Source

crawl web page

Example

iex> url = "https://euros-test.blogspot.jp/" 
"https://euros-test.blogspot.jp/"
iex> Euros.Core.crawl(url, fn(page) -> page |> inspect |> IO.puts end)

if set cookie

iex> url = "https://euros-test.blogspot.jp/" 
"https://euros-test.blogspot.jp/"
iex> option = %Euros.HTTPOption{cookie: "foo=bar;"}
%Euros.HTTPOption{cookie: "foo=bar;", recv_timeout: 60000, timeout: 60000}
iex> Euros.Core.crawl(url, fn(page) -> page |> inspect |> IO.puts end, option)

if cusotm url pattern

iex> url = "https://euros-test.blogspot.jp/" 
"https://euros-test.blogspot.jp/"
iex> option = %Euros.HTTPOption{cookie: "foo=bar;"}
%Euros.HTTPOption{cookie: "foo=bar;", recv_timeout: 60000, timeout: 60000}
iex> pattern = ~r/test1/
~r/test1/
iex> Euros.Core.crawl(url, fn(page) -> page |> inspect |> IO.puts end, option, pattern)