Create your own version of fluent-bit image

Posted on | by Rishitha Minol

In this post we are planning to create our own version of docker image using rockylinux/rockylinux:8. In this image all of the configurations pass by environment variables. Environment variable configuration passing is a handy way to manage your container environment.

first of all we have to think fluent-bit perspective. It has several configuration files needed.

  1. fluent-bit.conf – main configuration file
  2. parser.conf – log line parsing information (This file contains how to properly parse your log lines)

Thes are the main files needed by fluent-bit to perform it’s function.

First of all we have to keep a container up and running. That way we can log into that container at any time. In order to fulfill this we are using a docker compose file to start rockylinuxx/rockylinux:8 container. Create a file named dummy-container.yml using below configuration.

version: '3'

services:
  dummy_container:
    container_name: 'dummy_container'
    image: 'rockylinux/rockylinux:8'
    command: ["tail", "-f", "/dev/null"]

And use below command to start the container.

docker-compose up -d

You can get the shell using below command.

docker exec -it dummy_container bash

Install below packages into the container.

  1. libpq-devel – which is needed by fluent-bit to push logs into postgresql. We are planning a way of pushing logs into a postgresql instance in next post.
  2. cmake – building
  3. git – pulling code
  4. Install development tools. Group install. (dnf groupinstall "Development Tools")
dnf install epel-release -y
dnf install cmake git vim libpq-devel supervisor -y
dnf groupinstall "Development Tools" -y

Clone fluent-bit project.

git clone https://github.com/fluent/fluent-bit.git

By default fluent-bit codebase disabled postgresql library support. we have to enable it manually. In CMakeLists.txt file. It has an option called FLB_OUT_PGSQL. Find it and make it’s value Yes.

Goto build directoyr and start building your application.

cd build
cmake ..
make
make install

This package will install into your container. The fluent-bit directory can be removed once above steps are complete.

After installing fluent-bit, check the location using which command and use ldd to check whether the executable properly work with the postgresql library.

You have installed fluent-bit properly and remove unwanted packages to reduce docker image size.

dnf groupremove "Development Tools" -y
dnf remove cmake git vim -y

# Remove fluent-bit directory cloned previously
rm -rf fluent-bit

rm -rf /var/cache/dnf

Commit your docker container to create new image.

docker commit dummy_container

Tag your new image with a name and tag using the sha256 code returned from above command.

docker tag <sha256> rockylinux-fluent-bit:latest