ExKafkaLogger v0.2.5 ExKafkaLogger View Source

ExKafkaLogger

ExKafkaLogger is a Elixir library that works using Poison to log data, parse to JSON and send it to Kafka.

This library automatically gets data from the default Elixir Logger and from every request/response from your project. This data (and any data you might want to send) is then parsed and sent to Kafka with a specified topic.

Documentation is avaiable online at https://hexdocs.pm/ex_kafka_logger.

Installation

The package can be installed by adding ex_kafka_logger to your list of dependencies in mix.exs:

def deps do
  [{:ex_kafka_logger, "~> 0.1"}]
end

After installing the library you will need to run the following command

$ mix deps.get

Then you will also need to follow the subsequent configuration steps.

Configuration in 3 steps

Step 1

Add the ExKafkaLogger Plug on your Phoenix endpoint.ex file just above plug YourApp.Router

It should looks like:

# ...

  plug Plug.Session,
    store: :cookie,
    key: "_my_app_key",
    signing_salt: "SECRET_SALT"

  plug ExKafkaLogger.Plug
  plug MyApp.Router
end

Step 2

Use Plug.ErrorHandler and create a handle_error function in the end of your router.ex file to catch the errors without let Phoenix send the default errors response, like the example below.

use YourApp.Web, :router
use Plug.ErrorHandler

# ... your code ...

defp handle_errors(conn, _) do
  response = ExKafkaLogger.HttpError.template(conn.status)
  json(conn, response)
end

Step 3

Configure your application to make the Elixir Logger to use ExKafkaLogger as one of your backend and let ExKafkaLogger knows how to connect in your Kafka instance.

In your config/ENV.exs file add the lines like the example below.

config :logger, backends: [:console, ExKafkaLogger]

config :ex_kafka_logger,
  kafka_topic: "your_logging_topic",
  service_name: "your_app_name"

 config :kafka_ex,
  brokers: [
    {"127.0.0.1", 9092},
  ],
  consumer_group: "consumer-group",
  disable_default_worker: false,
  sync_timeout: 3000,
  max_restarts: 10,
  max_seconds: 60,
  kafka_version: "0.9.0"

Example Application

Here is an example of how to use this library to log a simple Phoenix Application.

Link to this section Summary

Functions

Alias to ExKafkaLogger.log(:debug, log) The argument could be a String or a Map

Example

iex> ExKafkaLogger.debug("Something to debug and log on Kafka")
:ok

Alias to ExKafkaLogger.log(:error, log) The argument could be a String or a Map

Example

iex> ExKafkaLogger.error("Some error to log on Kafka")
:ok

Alias to ExKafkaLogger.log(:info, log) The argument could be a String or a Map

Example

iex> ExKafkaLogger.info("Some info to log on Kafka")
:ok

The function receive two arguments the log level that is an Atom and could be (:info, :debug, :warn or :error), the second argument is the log that is a String

Example

iex> ExKafkaLogger.log(:info, "Some info to log on Kafka")
:ok

Alias to ExKafkaLogger.log(:warn, log) The argument could be a String or a Map

Example

iex> ExKafkaLogger.warn("Some warn to log on Kafka")
:ok

Link to this section Functions

Alias to ExKafkaLogger.log(:debug, log) The argument could be a String or a Map

Example

iex> ExKafkaLogger.debug("Something to debug and log on Kafka")
:ok

Alias to ExKafkaLogger.log(:error, log) The argument could be a String or a Map

Example

iex> ExKafkaLogger.error("Some error to log on Kafka")
:ok

Alias to ExKafkaLogger.log(:info, log) The argument could be a String or a Map

Example

iex> ExKafkaLogger.info("Some info to log on Kafka")
:ok

The function receive two arguments the log level that is an Atom and could be (:info, :debug, :warn or :error), the second argument is the log that is a String

Example

iex> ExKafkaLogger.log(:info, "Some info to log on Kafka")
:ok

Alias to ExKafkaLogger.log(:warn, log) The argument could be a String or a Map

Example

iex> ExKafkaLogger.warn("Some warn to log on Kafka")
:ok