conn_config() = brod:conn_config()
connection() = kpro:connection()
endpoint() = brod:endpoint()
fetch_fun() = fun((offset()) -> {ok, {offset(), [brod:message()]}} | {error, any()})
group_id() = brod:group_id()
offset() = brod:offset()
offset_time() = brod:offset_time()
partition() = brod:partition()
req_fun() = fun((offset(), kpro:count()) -> kpro:req())
topic() = brod:topic()
assert_client/1 | Assert client_id is an atom(). |
assert_group_id/1 | Assert group_id is a binary(). |
assert_topic/1 | Assert topic is a binary(). |
assert_topics/1 | Assert a list of topic names [binary()]. |
bytes/1 | Return message set size in number of bytes. |
describe_groups/3 | Send describe_groups_request and wait for describe_groups_response. |
epoch_ms/0 | Milliseconds since beginning of the epoch (midnight Jan 1, 1970 (UTC)). |
fetch/4 | Fetch a message-set. |
fetch/5 | Fetch a single message set from the given topic-partition. |
fetch_committed_offsets/3 | Fetch commited offsets for the given topics in a consumer group. |
fetch_committed_offsets/4 | Fetch committed offsets for the given topics in a consumer group. |
flatten_batches/3 | Make a flat message list from decoded batch list. |
fold/8 | |
get_metadata/1 | Try to connect to any of the bootstrap nodes and fetch metadata for all topics. |
get_metadata/2 | Try to connect to any of the bootstrap nodes and fetch metadata for the given topics. |
get_metadata/3 | Try to connect to any of the bootstrap nodes using the given connection options and fetch metadata for the given topics. |
get_stable_offset/1 | last_stable_offset is added in fetch response version 4 This function takes high watermark offset as last_stable_offset in case it's missing. |
group_per_key/1 | Group values per-key in a key-value list. |
group_per_key/2 | Group values per-key for the map result of a list. |
init_sasl_opt/1 | Hide sasl plain password in an anonymous function to avoid the plain text being dumped to crash logs. |
is_normal_reason/1 | Check terminate reason for a gen_server implementation. |
is_pid_alive/1 | |
list_all_groups/2 | List all groups in the given cluster. |
list_groups/2 | List all groups in the given coordinator broker. |
log/3 | simple wrapper around error_logger. |
make_batch_input/2 | Make batch input for kafka_protocol. |
make_fetch_fun/4 | Make a fetch function which should expand max_bytes when
it is not big enough to fetch one signle message. |
make_part_fun/1 | |
os_time_utc_str/0 | Get now timestamp, and format as UTC string. |
parse_rsp/1 | Parse decoded kafka response (#kpro_rsp{} ) into a more generic
representation. |
request_sync/2 | |
request_sync/3 | |
resolve_offset/4 | Resolve timestamp or semantic offset to real offset. |
resolve_offset/5 | Resolve timestamp to real offset. |
assert_client(Client::brod:client_id() | pid()) -> ok | no_return()
Assert client_id is an atom().
assert_group_id(GroupId::group_id()) -> ok | no_return()
Assert group_id is a binary().
assert_topic(Topic::topic()) -> ok | no_return()
Assert topic is a binary().
assert_topics(Topics::[topic()]) -> ok | no_return()
Assert a list of topic names [binary()].
bytes(Msgs::bord:batch_input()) -> non_neg_integer()
Return message set size in number of bytes. NOTE: This does not include the overheads of encoding protocol. such as magic bytes, attributes, and length tags etc.
describe_groups(CoordinatorEndpoint::endpoint(), ConnCfg::conn_config(), IDs::[brod:group_id()]) -> {ok, kpro:struct()} | {error, any()}
Send describe_groups_request and wait for describe_groups_response.
epoch_ms() -> kpro:msg_ts()
Milliseconds since beginning of the epoch (midnight Jan 1, 1970 (UTC)).
fetch(Conn::connection(), ReqFun::req_fun(), Offset::offset(), MaxBytes::kpro:count()) -> {ok, {offset(), [brod:message()]}} | {error, any()}
Fetch a message-set. If the given MaxBytes is not enough to fetch a single message, expand it to fetch exactly one message
fetch(Hosts::connection() | brod:client_id() | brod:bootstrap(), Topic::topic(), Partition::partition(), Offset::offset(), Opts::brod:fetch_opts()) -> {ok, {offset(), [brod:message()]}} | {error, any()}
Fetch a single message set from the given topic-partition.
fetch_committed_offsets(Client::brod:client(), GroupId::group_id(), Topics::[topic()]) -> {ok, [kpro:struct()]} | {error, any()}
Fetch commited offsets for the given topics in a consumer group.
1. Get broker endpoint by calling
brod_client:get_group_coordinator
2. Establish a connecton to the discovered endpoint.
3. send offset_fetch
request and wait for response.
If Topics is an empty list, fetch offsets for all topics in the group
fetch_committed_offsets(BootstrapEndpoints::[endpoint()], ConnCfg::conn_config(), GroupId::group_id(), Topics::[topic()]) -> {ok, [kpro:struct()]} | {error, any()}
Fetch committed offsets for the given topics in a consumer group.
1. try find out the group coordinator broker from the bootstrap hosts
2. send offset_fetch
request and wait for response.
If Topics is an empty list, fetch offsets for all topics in the group
flatten_batches(BeginOffset::offset(), Header::map(), Batches0::[kpro:batch()]) -> {offset(), [kpro:message()]}
Make a flat message list from decoded batch list. Return the next beging-offset together with the messages.
fold(Hosts::connection() | brod:client_id() | brod:bootstrap(), Topic::topic(), Partition::partition(), Offset::offset(), Opts::brod:fetch_opts(), Acc, Fun::brod:fold_fun(Acc), Limits::brod:fold_limits()) -> brod:fold_result()
get_metadata(Hosts::[endpoint()]) -> {ok, kpro:struct()} | {error, any()}
Try to connect to any of the bootstrap nodes and fetch metadata for all topics
get_metadata(Hosts::[endpoint()], Topics::all | [topic()]) -> {ok, kpro:struct()} | {error, any()}
Try to connect to any of the bootstrap nodes and fetch metadata for the given topics
get_metadata(Hosts::[endpoint()], Topics::all | [topic()], ConnCfg::conn_config()) -> {ok, kpro:struct()} | {error, any()}
Try to connect to any of the bootstrap nodes using the given connection options and fetch metadata for the given topics.
get_stable_offset(Header) -> any()
last_stable_offset is added in fetch response version 4 This function takes high watermark offset as last_stable_offset in case it's missing.
group_per_key(List::[{Key, Val}]) -> [{Key, [Val]}]
Group values per-key in a key-value list.
group_per_key(MapFun::fun((term()) -> {Key, Val}), List::[term()]) -> [{Key, [Val]}]
Group values per-key for the map result of a list.
init_sasl_opt(Config::brod:client_config()) -> brod:client_config()
Hide sasl plain password in an anonymous function to avoid the plain text being dumped to crash logs
is_normal_reason(X1) -> any()
Check terminate reason for a gen_server implementation
is_pid_alive(Pid) -> any()
list_all_groups(Endpoints::[endpoint()], Options::conn_config()) -> [{endpoint(), [brod:cg()] | {error, any()}}]
List all groups in the given cluster. NOTE: Exception if failed against any of the coordinator brokers.
list_groups(Endpoint::endpoint(), ConnCfg::conn_config()) -> {ok, [brod:cg()]} | {error, any()}
List all groups in the given coordinator broker.
log(X1::info | warning | error, Fmt::string(), Args::[any()]) -> ok
simple wrapper around error_logger. NOTE: keep making MFA calls to error_logger to 1. allow logging libraries such as larger parse_transform 2. be more xref friendly
make_batch_input(Key::brod:key(), Value::brod:value()) -> brod:batch_input()
Make batch input for kafka_protocol.
make_fetch_fun(Conn::pid(), Topic::topic(), Partition::partition(), FetchOpts::brod:fetch_opts()) -> fetch_fun()
Make a fetch function which should expand max_bytes
when
it is not big enough to fetch one signle message.
make_part_fun(F::brod:partitioner()) -> brod:partition_fun()
os_time_utc_str() -> string()
Get now timestamp, and format as UTC string.
parse_rsp(Kpro_rsp::kpro:rsp()) -> ok | {ok, term()} | {error, any()}
Parse decoded kafka response (#kpro_rsp{}
) into a more generic
representation.
Return ok
if it is a trivial 'ok or not' response without data fields
Return {ok, Result}
for some of the APIs when no error-code found in
response. Result could be a transformed representation of response message
body #kpro_rsp.msg
or the response body itself.
For some APIs, it returns {error, CodeOrMessage}
when error-code is not
no_error
in the message body.
NOTE: Not all error codes are interpreted as {error, CodeOrMessage}
tuple.
for some of the complex response bodies, error-codes are retained
for caller to parse.
request_sync(Conn::connection(), Req::kpro:req()) -> ok | {ok, term()} | {error, any()}
request_sync(Conn::connection(), Kpro_req::kpro:req(), Timeout::infinity | timeout()) -> ok | {ok, term()} | {error, any()}
resolve_offset(Pid::pid(), Topic::topic(), Partition::partition(), Time::offset_time()) -> {ok, offset()} | {error, any()}
Resolve timestamp or semantic offset to real offset. The give pid should be the connection to partition leader broker.
resolve_offset(Hosts::[endpoint()], Topic::topic(), Partition::partition(), Time::offset_time(), ConnCfg::conn_config()) -> {ok, offset()} | {error, any()}
Resolve timestamp to real offset.
Generated by EDoc