Category Archives: Java Logging

Java :How to convert a stack trace to a string?


In this blog you will learn way to convert a stack trace to a String which includes all the Stack Trace message of an exception or error.

Why this conversion is required?

This conversion is required if you want to save the message in custom log file for further analysis of log message. The e.getMessage() in Java program only returns small portion of the error, but if you want to see the detailed error then StackTrace will give you complete detail about the error in the program.

Example

To convert stackTrace to a string using printWriter as input for stackTrace to convert in String.

import java.io.PrintWriter;
import java.io.StringWriter;

public class TestExample {

	public static void main(String[] args) {
		try
		{
			int y=5/0;
		}
		catch(Exception ex)
		{
			System.out.println(convertStackTraceToString(ex));
		}

	}
    /**
     * method to convert stack trace to String
     * @param ex
     * @return
     */
	private static String convertStackTraceToString(Exception ex) {

		StringWriter sw = new StringWriter();
		PrintWriter pw = new PrintWriter(sw);
		ex.printStackTrace(pw);
		String sStackTrace = sw.toString();
		return sStackTrace;
	}

}

Output


java.lang.ArithmeticException: / by zero
    at com.exceptions.errors.TestExample.main(TestExample.java:11)
Advertisements

Log4j2: How to Mask Logs Personal/Confidential/SPI Information


You can configure Log4j2 LogEventConverter plugin for masking personal/confidential/SPI data. For example here we are masking credit card number, SSN and CVV of log statements.

Below Log4j2 dependencies you need to add with your application.

Pre-Requisite

Log4j2 Dependency

pom.xml entry


         <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-api</artifactId>
            <version>2.6.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-core</artifactId>
            <version>2.6.1</version>
        </dependency>
   

Log4j2.xml Configuration File

Below is basic console based configuration for masking where packages used for where loggers need to apply and masking plugin annotation as in below program. In PatternLayout used key as spi which is map with plugin which will mask log text and print as message.


<?xml version="1.0" encoding="UTF-8"?>
<configuration xmlns="http://logging.apache.org/log4j/2.0/config"
       status="OFF" packages="com.mask.logs">
       <Appenders>
    <Console name="STDOUT" target="SYSTEM_OUT">
     <PatternLayout pattern="%d{yyyyMMdd HH:mm:ss.SSS} [%t] %-5level %logger{36} - %spi%n"/>
   
    </Console>
  </Appenders>
  <loggers>
              <Logger name="org.apache.log4j.xml" level="all" />
              <root level="all">
                     <appender-ref ref="STDOUT" level="TRACE" />
                     </root>
  </loggers>
</configuration>

To know more about log4j2 configuration follow below link:

Log4j2 Java Logging Example Tutorial – XML Configuration, Severity Levels, Formatting and Appenders

Log4j2 Plugin Code for Masking

Created this LogMakingConverter Plug in for masking logs statement
Here masking for Credit card number, CVV and SSN . You can also implement for password and rest depend on your application needs. Here we have used Java regular expression Mater and Pattern apis for maskin and also different character as +++ for CVV,********* for SSN and **************** for credit card by matching number of digits in log lines.

package com.mask.logs;
/**
 * Created this LogMakingConverter Plug in for masking logs statement
 * Here masking for Credit card number, CVV and SSN .
 * You can also implement for password and rest depend on your application needs.
 */
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import org.apache.logging.log4j.core.LogEvent;
import org.apache.logging.log4j.core.config.plugins.Plugin;
import org.apache.logging.log4j.core.pattern.ConverterKeys;
import org.apache.logging.log4j.core.pattern.LogEventPatternConverter;

@Plugin(name="LogMaskingConverter", category = "Converter")
@ConverterKeys({"spi","trscId"})
public class LogMaskingConverter extends LogEventPatternConverter{
	private static final String CREDIT_CARD_REGEX = "([0-9]{16})";;
	private static final Pattern CREDIT_CARD_PATTERN = Pattern.compile(CREDIT_CARD_REGEX);
	private static final String CAREDIT_CARD_REPLACEMENT_REGEX = "XXXXXXXXXXXXXXXX";

	private static final String CVV_REGEX = "([0-9]{3})";
	private static final Pattern CVV_PATTERN = Pattern.compile(CVV_REGEX);
	private static final String CVV_REPLACEMENT_REGEX = "+++";

	private static final String SSN_REGEX = "([0-9]{9})";
	private static final Pattern SSN_PATTERN = Pattern.compile(SSN_REGEX);
	private static final String SSN_REPLACEMENT_REGEX = "*********";

    protected LogMaskingConverter(String name, String style) {
        super(name, style);
    }

    public static LogMaskingConverter newInstance(String[] options) {
        return new LogMaskingConverter("spi",Thread.currentThread().getName());
    }

    @Override
    public void format(LogEvent event, StringBuilder outputMessage) {
    	String message = event.getMessage().getFormattedMessage();
		String maskedMessage = message;
		try {
			maskedMessage = mask(message);
		} catch (Exception e) {
			System.out.println("Failed While Masking");
			maskedMessage = message;
		}
		outputMessage.append(maskedMessage);	

    }

    private String mask(String message) {
    	Matcher matcher =null;
		StringBuffer buffer = new StringBuffer();

		matcher = CREDIT_CARD_PATTERN.matcher(message);
		maskMatcher(matcher, buffer,CAREDIT_CARD_REPLACEMENT_REGEX);
		message=buffer.toString();
		buffer.setLength(0);

		matcher = SSN_PATTERN.matcher(message);
		maskMatcher(matcher, buffer,SSN_REPLACEMENT_REGEX);
		message=buffer.toString();
		buffer.setLength(0);

		matcher = CVV_PATTERN.matcher(message);
		maskMatcher(matcher, buffer,CVV_REPLACEMENT_REGEX);

		return buffer.toString();
	}

    private StringBuffer maskMatcher(Matcher matcher, StringBuffer buffer, String maskStr)
    {
    	while (matcher.find()) {
			matcher.appendReplacement(buffer,maskStr);
		}
		matcher.appendTail(buffer);
    	return buffer;
    }

}

Masking Test Program

Below is test program to print some log statements which are converting by Log4j2.

package com.mask.logs;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

public class MaskingTest {
	  private static final Logger logger = LogManager.getLogger(MaskingTest.class);
      public static void myTest(){
             logger.info("this is my info message");
             logger.debug("This is debug message");
             logger.debug("Passed to server::0084USER:17603,IP:0:0:0:0:0:0:0:1,3425,Credit Card 1:1000002367844224,3425,Credit Card2:1000002367844224 , CVV:234,SSN:123456789");

      }
      public static void main(String[] args) {
             myTest();
      }
}

Output

Below is mask logs by log4j2 for confidential/personal/spi data.

20180607 11:20:43.558 [main] INFO  com.mask.logs.MaskingTest - this is my info message
20180607 11:20:43.559 [main] DEBUG com.mask.logs.MaskingTest - This is debug message
20180607 11:20:43.560 [main] DEBUG com.mask.logs.MaskingTest - Passed to server::+++4USER:+++03,IP:0:0:0:0:0:0:0:1,+++5,Credit Card 1:XXXXXXXXXXXXXXXX,+++5,Credit Card2:XXXXXXXXXXXXXXXX , CVV:+++,SSN:
*********

Summary

  • Step by step program for masking sensitive information in logs.
  • Log4j2 XML Configuration for loggers and appenders for masking.
  • LogMaskingConverter Log4j2 plugin for masking log statements.
  • Regular expression for Creditcard, SSN and CVV.
  • Test program to write logs  by java logging api to write in logs file.

More 

Below are some more masking ways for different type of data like XML, JSON and printing objects before logging , sending to page or transferring over network.

How to MASK XML Confidential/Personal Data : JAVA

How to Mask JSON Confidential/Personal Information in logs :JAVA

How to mask JAVA Object confidential/personal information in logs while Printing

Log4j2 JSON Configuration for Java Logging Severity Levels, Formatting and Appenders


In below previous post you read about Log4j2 XML configuration for Appenders, formatters and Loggers for Console and File Logging. I have also explained about RollingFile appeneders and there management.

Below is Log4j2 JSON configuration equivalent to XML configuration and dependency required for JSON. For more info in detail and configuration steps follow previous post for Log4J2 XML configuration.

You have to add below JSON dependency in your class path or pom.xml

POM.XML

<!-- basic Log4j2 dependency -->
<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-api</artifactId>
    <version>2.6.1</version>
</dependency>
<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-core</artifactId>
    <version>2.6.1</version>
</dependency>
<!-- Asynchronous logging for multithreaded env -->
<dependency>
    <groupId>com.lmax</groupId>
    <artifactId>disruptor</artifactId>
    <version>3.3.4</version>
</dependency>

<!-- Jackson JSON Processor -->
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
        <version>2.4.1</version>
    </dependency>

create file log4j2.json in your root folder or resource folder.

{
   "configuration": {
       "status": "info",
       "monitorInterval": "60",
       "name": "FacingIssuesOnIT",
      "properties": {
         "property": [
            {
               "name": "filename",
               "value": "target/FacingIssueOnIT.log"
            },
            {
               "name": "log-path",
               "value": "C:/logs/"
            }
         ]
      },
      "appenders": {
         "Console": {
            "PatternLayout": {
               "pattern": "%d{yyyyMMdd HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"
            },
            "name": "STDOUT",
            "target": "SYSTEM_OUT"
         },
         "File": {
            "PatternLayout": {
               "pattern": "%d %p %c{1.} [%t] %m%n"
            },
            "name": "file",
            "fileName": "${filename}"
         },
         "RollingFile": {
            "PatternLayout": {
               "pattern": "%d{yyyyMMdd HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"
            },

                "Policies": {
               "SizeBasedTriggeringPolicy": {
                  "size": "100 MB"
               },
               "TimeBasedTriggeringPolicy": {
                  "interval": "1",
                  "modulate": "true"
               }
            },
            "DefaultRolloverStrategy": {
               "Delete": {
                  "IfFileName": {
                     "glob": "*/FacingIssueOnIT-*.log.gz"
                  },
                  "IfLastModified": {
                     "age": "1h"
                  },
                  "basePath": "${log-path}",
                  "maxDepth": "2"
               }
            },
            "name": "RollingFile",
            "fileName": "${log-path}/FacingIssueOnIT.log",
            "filePattern": "${log-path}/$${date:yyyy-MM-dd}/FacingIssueOnIT-%d{yyyy-MM-dd}-%i.log.gz"
         }
      },
      "Loggers": {
         "Logger": [
            {
               "name": "com.logging",
               "level": "debug"
            },
            {
               "appender-ref": {
                  "ref": "RollingFile",
                  "level": "debug"
               },
               "name": "root",
               "level": "debug",
               "additivity": "false"
            }
         ],
         "Root": {
            "AppenderRef": [
               {
                  "ref": "file",
                  "level": "error"
               },
               {
                  "ref": "STDOUT",
                  "level": "debug"
               },
               {
                  "ref": "RollingFile"
               }
            ],
            "level": "trace"
         }
      }

   }
}

Log4j2 New Features,Compare with Log4j and other Logging Framework


Log4j2 New Features

  • Java 8-style lambda support for lazy logging.
  • Log4j2 is garbage-free or least low-garbage since version 2.6
  • Log4j2 support configuration via XML, JSON, YAML, properties configuration files or programmatically.
  • Async Loggers – performance similar to logging switched off on multithreaded.
  • Concurrency improvements: log4j2 uses java.util.concurrent libraries to perform locking at the lowest level possible. Log4j-1.x has known deadlock issues.
  • Filtering: filtering based on context data,  regular expressions, markers and other components in the Log event. Filters can be associated with Loggers. You can use a common Filter class in any of these circumstances.
  • Custom Log Level
  • Plugin Architecture – easy to extend by building custom components
    Supported APIs: SLF4J, Commons Logging, Log4j-1.x and java.util.logging.

Why people are moving?

  • Community support: Log4j2 has an active community where question, answered, features are added and bugs are fixed.
  • Automatically reload it’s configuration upon modification without losing log events while reconfiguring.

Drawback

  • log4j 2.0 is very different than log4j 1.x, and the API is mostly incompatible.
  • Java 6 required for version 2.0 to 2.3. Java 7 is required for Log4j 2.4 and later.

Steps to upgrade Log4j1 to Log4j2

  1. add log4j2 dependency  log4j-api-2.6.2.jar and log4j-core-2.6.2.jar in your class path or pom.xml .
  2. Log4j2 looks for file log4j2.xml in config file. Add log4j2.xml file either in the class path or specify path with log4j configuration file system properties.
  3. To debug the log4j.xml configuration use in the beginning of configuration file.
  4. To add more configuration for Console, File and RollingFile appenders follow Log4j2 Tutorial and Configuration.

Comparison with other Logging Framework

Follow the below link to compare different logging framework on benchmark.

https://www.loggly.com/blog/benchmarking-java-logging-frameworks/