cth_skip_is_failure (macula_testkit v0.2.0)

View Source

A Common Test hook that turns every skip nobody allowed into a failure.

rebar3 ct exits 0 when a case skips itself ({skip, Reason} from the case, init_per_testcase, init_per_group or init_per_suite): the run reads as green although the test never ran. This hook rewrites such a skip into {fail, {skip_not_allowed, Reason}}, so the case is reported as failed (or its group or suite as auto skipped) and rebar3 exits 1.

A skip that is genuinely wanted is allowed explicitly, per suite, with the reason it is safe:

{ct_opts, [{ct_hooks, [{cth_skip_is_failure, [{allow, [{my_SUITE, a_case, "why skipping it here is safe"}, {dist_SUITE, '*', "needs distribution; run by ct_dist.sh"}]}]}]}]}.

The second element is the case name, or '*' for any skip in that suite, including its init_per_suite and init_per_group. An allowance without a reason is refused when the hook starts, so the run does not start.

Summary

Functions

The hook's id: one instance per run.

Read the allow-list. Every allowance must carry a non-empty reason.

A case that returns {skip, Reason} skipped itself.

A skip from init_per_group skips every case in the group.

A skip from init_per_suite skips every case in the suite.

A skip from init_per_testcase skips the case.

Types

allowance/0

-type allowance() :: {module(), where(), string()}.

where/0

-type where() :: atom().

Functions

id(Opts)

-spec id(term()) -> cth_skip_is_failure.

The hook's id: one instance per run.

init(Id, Opts)

-spec init(term(), [{allow, [allowance()]}]) -> {ok, [allowance()]} | {error, term()}.

Read the allow-list. Every allowance must carry a non-empty reason.

post_end_per_testcase(Suite, Case, Config, Return, Allow)

-spec post_end_per_testcase(module(), atom(), term(), term(), [allowance()]) -> {term(), [allowance()]}.

A case that returns {skip, Reason} skipped itself.

post_init_per_group(Suite, Group, Config, Return, Allow)

-spec post_init_per_group(module(), atom(), term(), term(), [allowance()]) -> {term(), [allowance()]}.

A skip from init_per_group skips every case in the group.

post_init_per_suite(Suite, Config, Return, Allow)

-spec post_init_per_suite(module(), term(), term(), [allowance()]) -> {term(), [allowance()]}.

A skip from init_per_suite skips every case in the suite.

post_init_per_testcase(Suite, Case, Config, Return, Allow)

-spec post_init_per_testcase(module(), atom(), term(), term(), [allowance()]) -> {term(), [allowance()]}.

A skip from init_per_testcase skips the case.