in Resources

Logging with Log4j2: A Comprehensive Guide for Java Developers

default image

A developer writing code on his laptop

Logging is an essential part of any software development process. It helps developers debug applications, understand code flow, and identify issues in production. With the emergence of microservices and distributed architectures, the need for effective logging has increased manifold.

Log4j2 is one of the most popular open-source logging frameworks used by Java developers. It is the successor to Log4j 1.x and provides significant improvements in terms of performance, configurability, and extensibility.

In this comprehensive guide, we will dive deep into Log4j2 and understand how it can be leveraged to build robust logging in Java applications.

What is Log4j2?

Log4j2 is an open-source logging framework developed by the Apache Software Foundation. It is distributed under the Apache License 2.0.

Log4j2 builds on top of the solid foundation laid by its predecessor Log4j. It provides the following improvements:

  • Better performance: Log4j2 is significantly faster than Log4j 1.x due to optimizations in garbage collection, multi-threading, and asynchronous logging.

  • More flexibility: It provides multiple configuration options including XML, JSON, YAML, and programmatic configuration. Configuration changes can be automatically reloaded at runtime.

  • Extensibility: Log4j2 provides a plugin architecture to extend its capabilities by creating custom components like Appenders, Filters, Layouts etc.

  • Reliability: It can function properly even when exceptions are thrown from Appenders and Filters. Automatic reconfiguration helps avoid losing events on failure.

  • Support for Lambda expressions: From version 2.4 onwards, Log4j2 supports Java 8 lambda expressions for lazy logging.

Compared to basic System.out.println(), using a Logger provides better control and management over your logs. Log4j2 gives immense power and flexibility to build tailor-made logging in your applications.

Core Components of Log4j2

Log4j2 is composed of several interrelated components working together to enable logging. Understanding these building blocks is key to unlocking the full potential of Log4j2.

Block diagram showing the different components that build up Log4j2

LoggerContext

The LoggerContext represents the anchor point of the logging system. It contains the configuration and all the loggers requested by the application.

Configuration

The Configuration holds all the settings required for logging like loggers, appenders, filters etc. Log4j2 supports configuration in XML, JSON, YAML and programmatic configuration.

Logger

The Logger represents the main entity that performs the actual logging. It generates log events based on the severity like debug, info, warn, error etc.

LoggerConfig

LoggerConfig encapsulates the configuration for a Logger such as logging level, appender and filter settings. This allows configuring loggers selectively.

Filter

Filter helps control flow of log events through the system. Events are allowed or blocked based on filter criteria. This provides fine-grained control over logging behavior.

Appender

An Appender controls the output destination of log events. It could be a file, database, console etc. A Logger can contain multiple Appenders sending copies of log events to different destinations.

Layout

A Layout formats the output of log events within an Appender. You can configure layouts like JSON, XML, HTML etc.

Now that we understand the core components, let‘s see how Log4j2 can be set up in a project.

Setting up Log4j2

Log4j2 artifacts can be added to your project using build tools like Maven and Gradle or manually as a dependency JAR.

Using Maven

For Maven-based projects, add the following dependencies in pom.xml:

<dependencies>
  <dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-api</artifactId>
    <version>2.20.0</version>
  </dependency>

  <dependency>
    <groupId>org.apache.logging.log4j</groupId>  
    <artifactId>log4j-core</artifactId>
    <version>2.20.0</version>
  </dependency>
</dependencies>

To avoid repeating versions, you can import the Log4j Bill of Materials (BOM):

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.apache.logging.log4j</groupId>
      <artifactId>log4j-bom</artifactId>
      <version>2.20.0</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

<dependencies>
  <dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-api</artifactId>
  </dependency>

  <dependency>
   <groupId>org.apache.logging.log4j</groupId>
   <artifactId>log4j-core</artifactId>
  </dependency>
</dependencies>

Using Gradle

For Gradle, add the dependencies in build.gradle:

dependencies {
  implementation ‘org.apache.logging.log4j:log4j-api:2.20.0‘
  implementation ‘org.apache.logging.log4j:log4j-core:2.20.0‘ 
}

To utilize the BOM:

dependencies {
  implementation platform(‘org.apache.logging.log4j:log4j-bom:2.20.0‘)
  implementation ‘org.apache.logging.log4j:log4j-api‘
  runtimeOnly ‘org.apache.logging.log4j:log4j-core‘
}

Manual Installation

You can also manually download the Log4j jars and add them to your application‘s classpath:

  • log4j-api-2.20.0.jar
  • log4j-core-2.20.0.jar

This allows you to use Log4j2 without any build tool.

Top Features of Log4j2

A person sitting and learning different techniques in Java

Log4j2 is packed with features that make it the go-to logging framework for Java developers. Let‘s explore some of its powerful capabilities:

Extensibility using Plugins

Log4j2 promotes extensibility through its plugins architecture. You can create custom plugins for components like Appenders, Filters, Lookups etc. using the @Plugin annotation. This avoids having to modify library code to add new functionality.

Support for Lambda Expressions

From version 2.4 onwards, Log4j2 supports Java 8 lambda expressions. This allows deferred execution of expensive log message construction until the logging level is met.

logger.debug("Result is {}", () -> expensiveOperation());

Asynchronous Logging

Log4j2 provides asynchronous loggers that log events on a separate thread than the application thread. This improves application performance by avoiding I/O blocking for logging.

Garbage-free Logging

The default Garbage-free logging mode minimizes object creation resulting in reduced pressure on the garbage collector. This ensures high throughput and low logging overhead.

Lookups

Log4j2 supports Lookups that allow pulling data from the context map, system properties etc. This helps enrich log messages with dynamic runtime information.

Built-in Custom Log Levels

You can define custom log levels in the configuration and code. The ExtendedLogger helper tool generates logger classes with methods for custom levels like logger.notice().

YAML and JSON Configuration

In addition to XML, Log4j2 supports YAML and JSON for configuration. This allows programmatic construction of configs.

Automatic Configuration Reloading

Any changes to the configuration files are automatically detected and applied at runtime without restarting the application.

Lambda Support for Custom Filters

From version 2.12 onwards, Log4j2 has first-class support for lambda expressions in custom filters.

These features make Log4j2 extremely flexible and enterprise-ready for diverse logging requirements.

Implementing Custom Log Levels

Log levels in Log4j2 allow classifying log events based on their severity. By default, Log4j2 provides pre-defined log levels like ERROR, INFO, DEBUG etc.

However, you can also define custom log levels and use them in your applications. Let‘s see how to do that.

Via Configuration

Custom log levels can be defined in the configuration file like:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
  <CustomLevels>
    <CustomLevel name="DIAGNOSTIC" intLevel="450" />
  </CustomLevels>

  ...
</Configuration>

This adds a new log level DIAGNOSTIC with priority 450.

Via Code

You can programmatically create custom levels:

Level DIAGNOSTIC = Level.forName("DIAGNOSTIC", 450);

Logging with Custom Levels

To log with custom levels:

// Create logger 
Logger logger = LogManager.getLogger();

// Log event with custom level
logger.log(DIAGNOSTIC, "Custom level log message");

For convenience, you can use the ExtendedLogger helper:

// Generates logger with method for custom level
ExtendedLogger logger = ExtendedLogger.wrap(LogManager.getLogger());

logger.diagnostic("Custom level log message");

This allows logging via dedicated methods like logger.diagnostic() instead of logger.log().

Log4j2 Configuration Overview

Log4j2 provides many configuration options to build flexible logging for your specific use case.

Configuration File Formats

Log4j2 supports configuration in XML, JSON, YAML, and Java properties. For example:

XML

<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
  <Appenders>
    <Console name="Console" target="SYSTEM_OUT">
      <PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/> 
    </Console>
  </Appenders>

  <Loggers>
    <Root level="info">
      <AppenderRef ref="Console"/>
    </Root>
  </Loggers>  
</Configuration>

JSON

{
  "configuration": {
    "appenders": {
      "Console": {
        "name": "Console",
        "target": "SYSTEM_OUT",
        "PatternLayout": {
          "pattern": "%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"
        }  
      }
    },
    "loggers": {
      "Root": {
        "level": "info",
        "appenderRefs": {
          "Console": {
            "ref": "Console"
          }
        }
      }
    }
  }
}

YAML

Configuration:
  Appenders:
    Console:
      name: Console
      target: SYSTEM_OUT
      PatternLayout: 
        Pattern: "%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"

  Loggers:
    Root:
      level: info
      AppenderRef:
        - 
          ref: Console

Programmatic Configuration

You can also programmatically create configurations in Java code:

Configuration config = ConfigurationBuilder.newBuilder()
  .setConfigurationName("BuilderConfig")
  .setStatusLevel(Level.ERROR)
  .add(builder.newFilter("ThresholdFilter", Filter.Result.ACCEPT, Filter.Result.NEUTRAL).
    addCriteria(LevelRangeFilter.createFilter(Level.ERROR, Level.FATAL)))
  .add(builder.newAppenderRef("Stdout").
    addAttribute("layout", builder.newLayout("PatternLayout").
      addAttribute("pattern", "%d [%t] %level: %msg%n")))
  .add(builder.newRootLogger(Level.ERROR).add(builder.newAppenderRef("Stdout")))
  .build();

LoggerContext ctx = Configurator.initialize(config);

This creates a configuration programmatically for custom use cases.

Loggers, Appenders and Layouts

  • Loggers output the actual log events based on the logging level. LoggerConfig allows configuring logging behavior for different loggers.

  • Appenders determine the log event destination like console, file, database etc. Each Appender requires a Layout.

  • Layouts format the log event output as per the syntax specified like JSON, XML, HTML etc.

Together they enable sending formatted logs to multiple destinations based on configurable logging logic.

Filters

Filters are used to selectively process log events based on criteria like logging level, marker, MDC key-value pairs etc.

Some built-in filters are:

  • BurstFilter: Rate limits logging by suppressing burst events above a threshold.
  • LevelRangeFilter: Logs events between specified minimum and maximum levels.
  • MarkerFilter: Filters events based on marker presence.
  • ThreadContextMapFilter: Filters events based on values in the thread context map.

Filters provide a powerful way to fine-tune logging behavior.

Best Practices for Log4j2

Here are some tips to follow when using Log4j2:

  • Avoid storing sensitive information like passwords in log events. Use masks or hashes instead.

  • Set appropriate logging levels across environments like DEBUG for dev, INFO for QA, and ERROR for production.

  • Use Asynchronous Loggers where possible to reduce application slow down due to logging overhead.

  • Enable Garbage-free Logging to minimize temporary object creation and reduce GC pressure.

  • Use filters judiciously to log only relevant events rather than all events.

  • Follow log rotation and archival policies to avoid huge log files eating up disk space.

  • Monitor log metrics occasionally to identify abnormal activity like sudden spikes in errors.

  • Mask sensitive data before logging using transformers for security and compliance.

Following these best practices will result in an efficient and effective logging infrastructure using Log4j2.

Conclusion

Robust logging is crucial for developing high-quality applications with visibility into runtime behavior. Log4j2 is one of the most comprehensive open-source logging frameworks for Java.

With its rich feature set, extensibility, and flexibility, Log4j2 empowers developers to tailor logging specifically to their application‘s needs. Its high-performance architecture makes it suitable even for large-scale enterprise systems.

This guide covers the end-to-end aspects of Log4j2 – from core concepts and setup to configuration and best practices. With the explained techniques and examples, you should be able to fully leverage Log4j2‘s capabilities for your logging needs and build more observable and resilient systems.

Written by