SpiderMan.Component.ItemProcessor behaviour (spider_man v0.4.1) View Source
Store items.
Life cycle of request:
- insert requests to ets of downloader.
- downloader's producer get pass out to processes.
- processes handle message.
- handle by pre pipelines.
- call
Requester.request/3
. - handle by post pipelines.
- pass out message.
- if success, pass out to Batchers.
- if failed, maybe try again.
- batcher get enough message and call
handle_batch/4
.- call
Storage.store/3
to save items. - if success, done.
- if failed, maybe try again.
- call
Link to this section Summary
Link to this section Callbacks
Specs
process_name(Broadway.name(), base_name :: String.t()) :: Broadway.name()