Euros v0.2.1 Euros.Core View Source
Provides the function for crawl web page
Link to this section Summary
Link to this section Functions
Link to this function
crawl(url, callback, option \\ %Euros.HTTPOption{}, pattern \\ ~r".*")
View Source
crawl web page
Example
iex> url = "https://euros-test.blogspot.jp/"
"https://euros-test.blogspot.jp/"
iex> Euros.Core.crawl(url, fn(page) -> page |> inspect |> IO.puts end)
if set cookie
iex> url = "https://euros-test.blogspot.jp/"
"https://euros-test.blogspot.jp/"
iex> option = %Euros.HTTPOption{cookie: "foo=bar;"}
%Euros.HTTPOption{cookie: "foo=bar;", recv_timeout: 60000, timeout: 60000}
iex> Euros.Core.crawl(url, fn(page) -> page |> inspect |> IO.puts end, option)
if cusotm url pattern
iex> url = "https://euros-test.blogspot.jp/"
"https://euros-test.blogspot.jp/"
iex> option = %Euros.HTTPOption{cookie: "foo=bar;"}
%Euros.HTTPOption{cookie: "foo=bar;", recv_timeout: 60000, timeout: 60000}
iex> pattern = ~r/test1/
~r/test1/
iex> Euros.Core.crawl(url, fn(page) -> page |> inspect |> IO.puts end, option, pattern)