Crawly v0.7.0 Crawly.Middlewares.UserAgent View Source
Set/Rotate user agents for crawling. The user agents are read from :crawly, :user_agents sessions.
The default value for the user agent is: Crawly Bot 1.0
Rotation is determined through Enum.random/1
.
Options
:user_agents
, optional. A list of user agent strings to rotate. Defaults to "Crawly Bot 1.0".
Example Declaration
middlewares: [
{UserAgent, user_agents: ["My Custom Bot] }
]
Link to this section Summary
Link to this section Functions
Link to this function