benchee_json v1.0.0 Benchee.Formatters.JSON View Source

Functionality for converting Benchee benchmarking results to JSON so that they can be written to file or just generated for your usage.

The most basic use case is to configure it as one of the formatters to be used by Benchee.run/2.

list = Enum.to_list(1..10_000)
map_fun = fn(i) -> [i, i * i] end

Benchee.run(%{
    "flat_map"    => fn -> Enum.flat_map(list, map_fun) end,
    "map.flatten" => fn -> list |> Enum.map(map_fun) |> List.flatten end
  },
  formatters: [
    {Benchee.Formatters.JSON, file: "my.json"},
    Benchee.Formatters.Console
  ],
)

Link to this section Summary

Functions

Simple wrapper for encoding a map/benchee struct to JSON

Formats the output of benchee to a list of maps, where each map represents a given benchmark scenario

Uses the return value of Benchee.Formatters.JSON.format/2 to write it to the JSON file defined in the initial configuration

Link to this section Functions

Simple wrapper for encoding a map/benchee struct to JSON.

Decouples it from the actual JSON library (currently Jason) used by benchee_json so that other plugins can rely on it just working with a general Benchee suite structure.

Formats the output of benchee to a list of maps, where each map represents a given benchmark scenario.

Each map includes the name, input name, run times, run time statistics, memory usages and memory usage statistics for each scenario.

Examples

iex> suite = %Benchee.Suite{
...>      scenarios: [
...>        %Benchee.Scenario{
...>          job_name: "My Job",
...>          name: "My Job",
...>          input_name: "Some Input",
...>          input: "Some Input",
...>          run_time_data: %Benchee.CollectionData{
...>            samples: [200, 400, 400, 400, 500, 500, 700, 900],
...>            statistics: %Benchee.Statistics{
...>              average:       500.0,
...>              ips:           2000.0,
...>              std_dev:       200.0,
...>              std_dev_ratio: 0.4,
...>              std_dev_ips:   800.0,
...>              median:        450.0,
...>              minimum:       200,
...>              maximum:       900,
...>              mode:          400,
...>              sample_size:   8,
...>              percentiles:   %{99 => 900}
...>            }
...>          },
...>          memory_usage_data: %Benchee.CollectionData{
...>            samples: [200, 400, 400, 400, 500, 500, 700, 900],
...>            statistics: %Benchee.Statistics{
...>              average:       500.0,
...>              ips:           nil,
...>              std_dev:       200.0,
...>              std_dev_ratio: 0.4,
...>              std_dev_ips:   nil,
...>              median:        450.0,
...>              minimum:       200,
...>              maximum:       900,
...>              mode:          nil,
...>              sample_size:   8,
...>              percentiles:   %{99 => 900}
...>            }
...>          }
...>        },
...>      ]
...>    }
iex> Benchee.Formatters.JSON.format(suite, %{file: "my_file.json"})
"[{\"name\":\"My Job\",\"job_name\":\"My Job\",\"input_name\":\"Some Input\",\"run_time_data\":{\"statistics\":{\"absolute_difference\":null,\"average\":500.0,\"ips\":2.0e3,\"maximum\":900,\"median\":450.0,\"minimum\":200,\"mode\":400,\"percentiles\":{\"99\":900},\"relative_less\":null,\"relative_more\":null,\"sample_size\":8,\"std_dev\":200.0,\"std_dev_ips\":800.0,\"std_dev_ratio\":0.4},\"samples\":[200,400,400,400,500,500,700,900]},\"memory_usage_data\":{\"statistics\":{\"absolute_difference\":null,\"average\":500.0,\"ips\":null,\"maximum\":900,\"median\":450.0,\"minimum\":200,\"mode\":null,\"percentiles\":{\"99\":900},\"relative_less\":null,\"relative_more\":null,\"sample_size\":8,\"std_dev\":200.0,\"std_dev_ips\":null,\"std_dev_ratio\":0.4},\"samples\":[200,400,400,400,500,500,700,900]},\"tag\":null}]"
Link to this function

write(data, arg2) View Source
write(String.t(), %{file: String.t()}) :: :ok

Uses the return value of Benchee.Formatters.JSON.format/2 to write it to the JSON file defined in the initial configuration.