Log4j2: How to Mask Logs Personal/Confidential/SPI Information


You can configure Log4j2 LogEventConverter plugin for masking personal/confidential/SPI data. For example here we are masking credit card number, SSN and CVV of log statements.

Below Log4j2 dependencies you need to add with your application.

Pre-Requisite

Log4j2 Dependency

pom.xml entry


         <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-api</artifactId>
            <version>2.6.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-core</artifactId>
            <version>2.6.1</version>
        </dependency>
   

Log4j2.xml Configuration File

Below is basic console based configuration for masking where packages used for where loggers need to apply and masking plugin annotation as in below program. In PatternLayout used key as spi which is map with plugin which will mask log text and print as message.


<?xml version="1.0" encoding="UTF-8"?>
<configuration xmlns="http://logging.apache.org/log4j/2.0/config"
       status="OFF" packages="com.mask.logs">
       <Appenders>
    <Console name="STDOUT" target="SYSTEM_OUT">
     <PatternLayout pattern="%d{yyyyMMdd HH:mm:ss.SSS} [%t] %-5level %logger{36} - %spi%n"/>
   
    </Console>
  </Appenders>
  <loggers>
              <Logger name="org.apache.log4j.xml" level="all" />
              <root level="all">
                     <appender-ref ref="STDOUT" level="TRACE" />
                     </root>
  </loggers>
</configuration>

To know more about log4j2 configuration follow below link:

Log4j2 Java Logging Example Tutorial – XML Configuration, Severity Levels, Formatting and Appenders

Log4j2 Plugin Code for Masking

Created this LogMakingConverter Plug in for masking logs statement
Here masking for Credit card number, CVV and SSN . You can also implement for password and rest depend on your application needs. Here we have used Java regular expression Mater and Pattern apis for maskin and also different character as +++ for CVV,********* for SSN and **************** for credit card by matching number of digits in log lines.

package com.mask.logs;
/**
 * Created this LogMakingConverter Plug in for masking logs statement
 * Here masking for Credit card number, CVV and SSN .
 * You can also implement for password and rest depend on your application needs.
 */
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import org.apache.logging.log4j.core.LogEvent;
import org.apache.logging.log4j.core.config.plugins.Plugin;
import org.apache.logging.log4j.core.pattern.ConverterKeys;
import org.apache.logging.log4j.core.pattern.LogEventPatternConverter;

@Plugin(name="LogMaskingConverter", category = "Converter")
@ConverterKeys({"spi","trscId"})
public class LogMaskingConverter extends LogEventPatternConverter{
	private static final String CREDIT_CARD_REGEX = "([0-9]{16})";;
	private static final Pattern CREDIT_CARD_PATTERN = Pattern.compile(CREDIT_CARD_REGEX);
	private static final String CAREDIT_CARD_REPLACEMENT_REGEX = "XXXXXXXXXXXXXXXX";

	private static final String CVV_REGEX = "([0-9]{3})";
	private static final Pattern CVV_PATTERN = Pattern.compile(CVV_REGEX);
	private static final String CVV_REPLACEMENT_REGEX = "+++";

	private static final String SSN_REGEX = "([0-9]{9})";
	private static final Pattern SSN_PATTERN = Pattern.compile(SSN_REGEX);
	private static final String SSN_REPLACEMENT_REGEX = "*********";

    protected LogMaskingConverter(String name, String style) {
        super(name, style);
    }

    public static LogMaskingConverter newInstance(String[] options) {
        return new LogMaskingConverter("spi",Thread.currentThread().getName());
    }

    @Override
    public void format(LogEvent event, StringBuilder outputMessage) {
    	String message = event.getMessage().getFormattedMessage();
		String maskedMessage = message;
		try {
			maskedMessage = mask(message);
		} catch (Exception e) {
			System.out.println("Failed While Masking");
			maskedMessage = message;
		}
		outputMessage.append(maskedMessage);	

    }

    private String mask(String message) {
    	Matcher matcher =null;
		StringBuffer buffer = new StringBuffer();

		matcher = CREDIT_CARD_PATTERN.matcher(message);
		maskMatcher(matcher, buffer,CAREDIT_CARD_REPLACEMENT_REGEX);
		message=buffer.toString();
		buffer.setLength(0);

		matcher = SSN_PATTERN.matcher(message);
		maskMatcher(matcher, buffer,SSN_REPLACEMENT_REGEX);
		message=buffer.toString();
		buffer.setLength(0);

		matcher = CVV_PATTERN.matcher(message);
		maskMatcher(matcher, buffer,CVV_REPLACEMENT_REGEX);

		return buffer.toString();
	}

    private StringBuffer maskMatcher(Matcher matcher, StringBuffer buffer, String maskStr)
    {
    	while (matcher.find()) {
			matcher.appendReplacement(buffer,maskStr);
		}
		matcher.appendTail(buffer);
    	return buffer;
    }

}

Masking Test Program

Below is test program to print some log statements which are converting by Log4j2.

package com.mask.logs;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

public class MaskingTest {
	  private static final Logger logger = LogManager.getLogger(MaskingTest.class);
      public static void myTest(){
             logger.info("this is my info message");
             logger.debug("This is debug message");
             logger.debug("Passed to server::0084USER:17603,IP:0:0:0:0:0:0:0:1,3425,Credit Card 1:1000002367844224,3425,Credit Card2:1000002367844224 , CVV:234,SSN:123456789");

      }
      public static void main(String[] args) {
             myTest();
      }
}

Output

Below is mask logs by log4j2 for confidential/personal/spi data.

20180607 11:20:43.558 [main] INFO  com.mask.logs.MaskingTest - this is my info message
20180607 11:20:43.559 [main] DEBUG com.mask.logs.MaskingTest - This is debug message
20180607 11:20:43.560 [main] DEBUG com.mask.logs.MaskingTest - Passed to server::+++4USER:+++03,IP:0:0:0:0:0:0:0:1,+++5,Credit Card 1:XXXXXXXXXXXXXXXX,+++5,Credit Card2:XXXXXXXXXXXXXXXX , CVV:+++,SSN:
*********

Summary

  • Step by step program for masking sensitive information in logs.
  • Log4j2 XML Configuration for loggers and appenders for masking.
  • LogMaskingConverter Log4j2 plugin for masking log statements.
  • Regular expression for Creditcard, SSN and CVV.
  • Test program to write logs  by java logging api to write in logs file.

More 

Below are some more masking ways for different type of data like XML, JSON and printing objects before logging , sending to page or transferring over network.

How to MASK XML Confidential/Personal Data : JAVA

How to Mask JSON Confidential/Personal Information in logs :JAVA

How to mask JAVA Object confidential/personal information in logs while Printing

Advertisements
Posted in Java, Java Logging, LOG4J2 | Tagged , , , , , , , , , , , , , , | 3 Comments

How to MASK XML Confidential/Personal Data : JAVA


Here you will see all steps to mask confidential/ information in XML like credit card, CVV, Exp date,  SSN, password etc. So that it will print in mask form as ****** so that unauthorize use will not misuse of others information.

Pre- Requisite

Use below library or add in pom.xml.


<dependency>
    <groupId>org.jsoup</groupId>
    <artifactId>jsoup</artifactId>
    <version>1.10.2</version>
</dependency>

Here is same XML file as AccountDetail.xml where need to mask cardNumber, cvv and expDate .



<AccountList>
    <Account>
        <id>E001</id>
        <FirstName>Saurabh</FirstName>
        <LastName>Gupta</LastName>
        <AddressDetail>
            <AddressLine1>Noida City Center</AddressLine1>
            <City>Noida</City>
            <State>UP</State>
            <Pincode>201301</Pincode>
            <Contry>India</Contry>
        </AddressDetail>
        <CreditCardDetail>
            <CardNumber>1233454565676567</CardNumber>
            <CVV>456</CVV>
            <ExpDate>12/90</ExpDate>
        </CreditCardDetail>
    </Account>
    <Account>
        <id>E002</id>
        <FirstName>Ankur</FirstName>
        <LastName>Mehrotra</LastName>
        <AddressDetail>
            <AddressLine1>New Delhi Metro Station</AddressLine1>
            <City>New Delhi</City>
            <State>UP</State>
            <Pincode>210345</Pincode>
            <Contry>India</Contry>
        </AddressDetail>
        <CreditCardDetail>
            <CardNumber>8967452312123456</CardNumber>
            <CVV>876</CVV>
            <ExpDate>09/83</ExpDate>
        </CreditCardDetail>
    </Account>
    <Account>
        <id>E003</id>
        <FirstName>Shailesh</FirstName>
        <LastName>Nagar</LastName>
        <AddressDetail>
            <AddressLine1>Dwarka Metro Station</AddressLine1>
            <City>Delhi</City>
            <State>Delhi</State>
            <Pincode>345876</Pincode>
            <Contry>India</Contry>
        </AddressDetail>
        <CreditCardDetail>
            <CardNumber>9078563412345678</CardNumber>
            <CVV>986</CVV>
            <ExpDate>08/99</ExpDate>
        </CreditCardDetail>
    </Account>
</AccountList>

Java code to mask above XML. In below code cardNumber is masking partially so that show last four digits only and hide rest of numbers while cvv and exp date digits are masked completely.

package com.mask.xml;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.parser.Parser;
import org.jsoup.select.Elements;

public class MaskXML {

	public static void main(String[] args) {

		try {
			FileInputStream inputStream = new FileInputStream(
					new File("D:\\Saurabh Gupta\\Workspace\\JavaTestExamples\\src\\main\\resources\\UserAccountDetail.xml"));
			Document doc = Jsoup.parse(inputStream, "UTF-8", "", Parser.xmlParser());
			Elements toMaskTagCompletely = doc.select("Pincode,ExpDate,CVV");
			Elements toMaskTagPartially = doc.select("CardNumber");
			for (Element element : toMaskTagCompletely) {
				element.text(replaceDigits(element.text()));
			}
			for (Element element : toMaskTagPartially) {
				element.text("XXXXXXXXXXXX" + element.text().substring(element.text().length() - 4));
			}
			System.out.println(doc.toString());
		} catch (FileNotFoundException ex) {
			ex.printStackTrace();
		} catch (IOException ex) {
			ex.printStackTrace();
		}

	}

	private static String replaceDigits(String text) {
		StringBuffer buffer = new StringBuffer(text.length());
		Pattern pattern = Pattern.compile("\\d");
		Matcher matcher = pattern.matcher(text);
		while (matcher.find()) {
			matcher.appendReplacement(buffer, "X");
		}
		return buffer.toString();
	}

}

Result : Masked XML


<?xml version="1.0" encoding="UTF-8"?> 
<AccountList> 
 <Account> 
  <id>
   E001
  </id> 
  <FirstName>
   Saurabh
  </FirstName> 
  <LastName>
   Gupta
  </LastName> 
  <AddressDetail> 
   <AddressLine1>
    Noida City Center
   </AddressLine1> 
   <City>
    Noida
   </City> 
   <State>
    UP
   </State> 
   <Pincode>
    XXXXXX
   </Pincode> 
   <Contry>
    India
   </Contry> 
  </AddressDetail> 
  <CreditCardDetail> 
   <CardNumber>
    XXXXXXXXXXXX6567
   </CardNumber> 
   <CVV>
    XXX
   </CVV> 
   <ExpDate>
    XX/XX
   </ExpDate> 
  </CreditCardDetail> 
 </Account> 
 <Account> 
  <id>
   E002
  </id> 
  <FirstName>
   Ankur
  </FirstName> 
  <LastName>
   Mehrotra
  </LastName> 
  <AddressDetail> 
   <AddressLine1>
    New Delhi Metro Station
   </AddressLine1> 
   <City>
    New Delhi
   </City> 
   <State>
    UP
   </State> 
   <Pincode>
    XXXXXX
   </Pincode> 
   <Contry>
    India
   </Contry> 
  </AddressDetail> 
  <CreditCardDetail> 
   <CardNumber>
    XXXXXXXXXXXX3456
   </CardNumber> 
   <CVV>
    XXX
   </CVV> 
   <ExpDate>
    XX/XX
   </ExpDate> 
  </CreditCardDetail> 
 </Account> 
 <Account> 
  <id>
   E003
  </id> 
  <FirstName>
   Shailesh
  </FirstName> 
  <LastName>
   Nagar
  </LastName> 
  <AddressDetail> 
   <AddressLine1>
    Dwarka Metro Station
   </AddressLine1> 
   <City>
    Delhi
   </City> 
   <State>
    Delhi
   </State> 
   <Pincode>
    XXXXXX
   </Pincode> 
   <Contry>
    India
   </Contry> 
  </AddressDetail> 
  <CreditCardDetail> 
   <CardNumber>
    XXXXXXXXXXXX5678
   </CardNumber> 
   <CVV>
    XXX
   </CVV> 
   <ExpDate>
    XX/XX
   </ExpDate> 
  </CreditCardDetail> 
 </Account> 
</AccountList>

Summary

  • Example for mask XML.
  • Shared API and source code for masking XML in less code.
  • Shared code for Mask complete and partial text data for credit card, SSN, CVV etc.

Related Posts

Below are some more masking ways for different type of data like XML, JSON and printing objects before logging , sending to page or transferring over network.

Log4j2: How to Mask Logs Personal/Confidential/SPI Information

How to Mask JSON Confidential/Personal Information in logs :JAVA

How to mask JAVA Object confidential/personal information in logs while Printing

 

Posted in Java, JSON, Mask | Tagged , , , , , , , , , , , , , | 2 Comments

YAML and JAVA Configuration


Below are Tools which support YAML for JAVA API’s. Here I will focus only on SnakeYAML which widely used by industry.

  • JvYaml
  • SnakeYAML
  • YamlBeans
  • JYaml
  • Camel

SnakeYAML Configuration

JAVA


<dependency>
    <groupId>org.yaml</groupId>
    <artifactId>snakeyaml</artifactId>
    <version>1.20-SNAPSHOT</version>
  </dependency>

Android


<dependency>
  <groupId>org.yaml</groupId>
  <artifactId>snakeyaml</artifactId>
  <version>1.20-SNAPSHOT</version>
  <classifier>android</classifier>
</dependency>

For any operation with SnakeYAML api’s you have to use below steps:
package :

org.yaml.snakeyaml.Yaml

Intialization:
Yaml yaml = new Yaml();

Loading YAML :

  • yaml.load(String) accepts a String.
  • yaml.load(InputStream) accepts a InputStream.

yaml.load(InputStream) detects the encoding by checking the BOM (byte order mark) sequence at the beginning of streams. If no BOM presents the utf-8 encoding is assumed..

More

To know more about YAML Syntax, Configuration with Java and other supporting language, frameworks and tools, Sample configuration files and JSON and YAML conversion follow below YAML Tutorials and YAML related exceptions follow YAML Issues.

Posted in Java, JSON, YAML | Tagged , , , , , | Leave a comment

JAVA : How to convert YAML Documents to JSON List?


Here is code to convert YAML documents to JSON objects by Jackson and snakeyml apis. Jackson also support YAML support.

Pre-Requisite



        <dependencies>
        <!-- Jackson JSON Processor -->
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
            <version>2.4.1</version>
        </dependency>
        <!-- For YAML -->
        <dependency>
            <groupId>org.yaml</groupId>
            <artifactId>snakeyaml</artifactId>
            <version>1.21</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.dataformat</groupId>
            <artifactId>jackson-dataformat-yaml</artifactId>
            <version>2.1.2</version>
        </dependency>
    </dependencies>   

Sample YAML Documents File


---
# My personal record
name: Saurabh Kumar Gupta
Title: Sr. Project Lead
skill: JAVA/J2EE
employed: True
domains:
    - Telecom
    - Finance
    - Banking
    - Healthcare
languages:
    ELK: Medium
    JAVA: Expertize
    Scripting: Comfortable
education: |
    MCA
    B.Sc
    Diploma

---
# Gaurav personal record
name: Gaurav Gupta
Title: Project Lead
skill: ELK
employed: True
domains:
    - Telecom
    - Banking
    - Healthcare
languages:
    ELK: Medium
    JAVA: Expertize
    Scripting: Comfortable
    Bigdata: Expertize
education: |
    MCA
    B.Sc

Code to Convert YAML documents to JSON Objects

package com.fiot.examples.yaml;

import java.io.File;
import java.io.FileInputStream;
import java.io.InputStream;
import java.util.Iterator;
import org.yaml.snakeyaml.Yaml;
import org.yaml.snakeyaml.constructor.SafeConstructor;

public class ConvertYAMLObjectsToJSON {
	public static void main(String[] args) {
		try (InputStream input = new FileInputStream(new File(
				"F:\\Workspace-Blog\\TestExamples\\src\\main\\resources\\YAMLDocument2.yaml"))) {
			Yaml yaml = new Yaml(new SafeConstructor());
			Iterator iterator = yaml.loadAll(input).iterator();
			while (iterator.hasNext()) {
				System.out.println(iterator.next());
			}
		} catch (Throwable e) {
			System.out.println("ERROR: " + e.getMessage());
		}
	}
}

Output


{name=Saurabh Kumar Gupta, Title=Sr. Project Lead, skill=JAVA/J2EE, employed=true, domains=[Telecom, Finance, Banking, Healthcare], languages={ELK=Medium, JAVA=Expertize, Scripting=Comfortable}, education=MCA
B.Sc
Diploma
}
{name=Gaurav Gupta, Title=Project Lead, skill=ELK, employed=true, domains=[Telecom, Banking, Healthcare], languages={ELK=Medium, JAVA=Expertize, Scripting=Comfortable, Bigdata=Expertize}, education=MCA
B.Sc}

More

To know more about YAML Syntax, Configuration with Java and other supporting language, frameworks and tools, Sample configuration files and JSON and YAML conversion follow below YAML Tutorials and YAML related exceptions follow YAML Issues.

Posted in Java, JSON, Uncategorized, YAML | Tagged , , , , , | Leave a comment

JAVA : How to convert YAML To JSON?


Here is code to convert YAML document to JSON by Jackson and snakeyml apis. Jackson also support YAML support.

Pre-Requisite


<dependencies>
        <!-- Jackson JSON Processor -->
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
            <version>2.4.1</version>
        </dependency>
        <!-- For YAML -->
        <dependency>
            <groupId>org.yaml</groupId>
            <artifactId>snakeyaml</artifactId>
            <version>1.21</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.dataformat</groupId>
            <artifactId>jackson-dataformat-yaml</artifactId>
            <version>2.1.2</version>
        </dependency>
    </dependencies>

Sample YAML File


---
# My personal record
name: Saurabh Kumar Gupta
Title: Sr. Project Lead
skill: JAVA/J2EE
employed: True
domains:
    - Telecom
    - Finance
    - Banking
    - Healthcare
languages:
    ELK: Medium
    JAVA: Expertize
    Scripting: Comfortable
education: |
    MCA
    B.Sc
    Diploma
...

Code to convert YAML to JSON data

package com.fiot.examples.yaml;

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.dataformat.yaml.YAMLFactory;

public class ConvertYAMLToJSON {
	public static void main(String[] args) {
		String content = "";
		try {
			content = new String(Files.readAllBytes(Paths.get(
					"F:\\Workspace-Blog\\TestExamples\\src\\main\\resources\\YAMLDocument.yaml")));
			System.out.println("*********Content from YAML File ****************");
			System.out.println(content);
			String json = convertYamlToJson(content);
			System.out.println("*********Cnverted JSON from YAML File ****************");
			System.out.println(json);
		} catch (IOException e) {
			e.printStackTrace();
		}
	}

	private static String convertYamlToJson(String yaml) {
		try {
			ObjectMapper yamlReader = new ObjectMapper(new YAMLFactory());
			Object obj = yamlReader.readValue(yaml, Object.class);
			ObjectMapper jsonWriter = new ObjectMapper();
			return jsonWriter.writerWithDefaultPrettyPrinter().writeValueAsString(obj);
		} catch (JsonProcessingException ex) {
			ex.printStackTrace();
		} catch (IOException ex) {
			ex.printStackTrace();
		}
		return null;
	}
}

Output


*********Content from YAML File ****************
---
# My personal record
name: Saurabh Kumar Gupta
Title: Sr. Project Lead
skill: JAVA/J2EE
employed: True
domains:
    - Telecom
    - Finance
    - Banking
    - Healthcare
languages:
    ELK: Medium
    JAVA: Expertize
    Scripting: Comfortable
education: |
    MCA
    B.Sc
    Diploma
...
*********Cnverted JSON from YAML File ****************
{
  "name" : "Saurabh Kumar Gupta",
  "Title" : "Sr. Project Lead",
  "skill" : "JAVA/J2EE",
  "employed" : true,
  "domains" : [ "Telecom", "Finance", "Banking", "Healthcare" ],
  "languages" : {
    "ELK" : "Medium",
    "JAVA" : "Expertize",
    "Scripting" : "Comfortable"
  },
  "education" : "MCA\nB.Sc\nDiploma\n"
}

Below are some online tools to convert YAML/YML to JSON.
https://codebeautify.org/yaml-to-json-xml-csv
http://convertjson.com/yaml-to-json.htm

More

To know more about YAML Syntax, Configuration with Java and other supporting language, frameworks and tools, Sample configuration files and JSON and YAML conversion follow below YAML Tutorials and YAML related exceptions follow YAML Issues.

Posted in Java, JSON, Uncategorized, YAML | Tagged , , , , , | Leave a comment

Difference between YAML and JSON


 

“YAML is superset of JSON”

Below are comparison between YAML and JSON by conceptually and writing differences

YAML vs JSON

  • YAML is best suited for configuration while JSON is better as a serialization format or serving up data for your APIs.
  • YAML is by no means  a replacement for JSON .You should use the data format that makes the most sense for what you are trying to accomplish.

YAML Advantage

  • YAML has a couple of big advantages  including the ability to self reference, support for complex datatypes, embedded block literals, comments, and more.
  • Write your configuration files in YAML format where you have the opportunity – it is designed to be readable and easily editable by humans.

JSON Disadvantage

  • JSON designed to be human readable – intentionally lacking features to support editing.
  • JSON doesn’t support comments – this is intentionally left out of the JSON specification because its not what the format was designed for.

JSON vs YAML

  • JSON is well suited for  serialization format to data interchange between apis over network.
  • JSON ships with a far simpler specification than YAML.
  • JSON  learning is faster in comparison to YAML, because it is not nearly as robust in its feature set.
  • YAML is a superset of JSON, which means you can parse JSON with a YAML parser.

JSON Advantage

  • JSON is  best to data interchange.

Disadvantage of YAML

  • YAML parsers are younger and  known to be less secure.
  • YAML is mainly designed for configuration when use for data interchange , many of YAMLs features lose their appeal.

Syntax Difference between YAML and JSON

Below are some syntax difference in YAML and JSON while writing files:

JSON Syntax

  • JSON is a subset of the JavaScript object notation syntax.
  • JSON data stored in name/value pairs.
  • JSON records separated by commas.
  • JSON field names & strings are wrapped by double quotes.

YAML Syntax

  • YAML stands for ain’t markup language and is a superset of JSON – You Convert YAML to JSON
  • YAML files begin with ‘—‘, marking the start of the document.
  • YAML documents end with ‘…’ but it’s optional.
  • YAML key value pairs are separated by colon.
  • YAML lists begin with a hyphen.

More

To know more about YAML Syntax, Configuration with Java and other supporting language, frameworks and tools, Sample configuration files and JSON and YAML conversion follow below YAML Tutorials and YAML related exceptions follow YAML Issues.

 

Posted in Java, JSON, Uncategorized, YAML | Tagged , , , , , , , | Leave a comment

YAML Introduction


YAML(Ain’t Markup Language) is a human friendly , cross language, unicode based data serialization format designed for interaction with all programming languages.

File Name Extension : .yml or .yaml

Latest Release : 1.2

It is broadly use for programming:

  • Configuration Files
  • Internet Messaging
  • Object Persistence to data auditing
  • Also support for Unicode standards characters.

YAML allow data to show itself in a natural and meaningful way and provide a unique cleanness by minimizing the amount of structural characters.
For example: indentation may be used for structure, colons use to separate “mapping key: value” pairs, and dashes are used to create “bullet” lists for showing collections of data.

YAML is a superset of JSON : you can convert YAML to JSON by different API’s

YAML Goals

Below are main design goals for YAML :

  • YAML is easily readable and understandable by humans.
  • YAML is expressive and extensible.
  • YAML is easy to implement and use.
  • YAML matches the native data structures of agile languages.
  • YAML data is portable between programming languages.
  • YAML has a consistent model to support generic tools.
  • YAML supports one-pass processing.

YAML Documents Processing

A YAML document is parsed only when the iterator is invoked (lazy evaluation).

More

To know more about YAML Syntax, Configuration with Java and other supporting language, frameworks and tools, Sample configuration files and JSON and YAML conversion follow below YAML Tutorials and YAML related exceptions follow YAML Issues.

 

Image | Posted on by | Tagged , , , , , , , | Leave a comment

How to validate YAML/YML?


You can follow YAML syntax and semantics to create YAML files as explained in previous post.

YAML Syntax

Below are some online tools to validate YAML/YML file . you just need to copy and paste there.

http://www.yamllint.com/

https://codebeautify.org/yaml-validator

http://beautifytools.com/yaml-validator.php

https://jsonformatter.org/yaml-validator

More

To know more about YAML Syntax, Configuration with Java and other supporting language, frameworks and tools, Sample configuration files and JSON and YAML conversion follow below YAML Tutorials and YAML related exceptions follow YAML Issues.

 

Posted in YAML | Tagged , , , | Leave a comment

YAML Syntax


By using below example of YAML file content will explain about YAML syntax so that easily understandable.

Example :

---
# My personal record
name: Saurabh Kumar Gupta
Title: Sr. Project Lead
skill: JAVA/J2EE
employed: True
domains:
    - Telecom
    - Finance
    - Banking
    - Healthcare
languages:
    ELK: Medium
    JAVA: Expertize
    Scripting: Comfortable
education: |
    MCA
    B.Sc
    Diploma
...

Comments  

YAML documents comments start with #.

Example  : #My personal record

Documents

YAML documents start with (- – -)  and ends with (. . .) (optional)

Example : as above YAML content having only one document.

List

YAML all members of a list are lines beginning at the same indentation level starting with a “-” (a dash and a space).

Example: as above YAML content having domains list as below

domains:
    - Telecom
    - Finance
    - Banking
    - Healthcare

Dictionary

A YAML dictionary is represented in a simple key: value form (the colon must be followed by a space).

Example : as above YAML content having dictionary as languages.

languages:
    ELK: Medium
    JAVA: Expertize
    Scripting: Comfortable

List and Dictionary Together

---
# Employee records
- Saurabh
   name: Saurabh Kumar Gupta
   Title: Sr. Project Lead
   skill: JAVA/J2EE
   employed: True
   domains:
     - Telecom
     - Finance
     - Banking
     - Healthcare
- Gaurav
   name: Gaurav Kumar Gupta
   Title: Project Lead
   skill: ELK
   employed: True
   domains:
     - Telecom
     - Finance
     

Boolean

you can also specify a boolean value (true/false) in several forms.

employed: yes
employed: no
employed: True
employed: false
employed: TRUE

If need to use Boolean value as String Literal use as below

employed: "yes"
employed: "no"
employed: "True"
employed: "false"
employed: "TRUE"

Multiline Value

YAML values can span multiple lines using two ways by  | or >.

  •  “Literal Block Scalar” | will include the newlines and any trailing spaces.
  • Folded Block Scalar” > will fold newlines to spaces;

It’s used to make what would otherwise be a very long line easier to read and edit. In either case the indentation will be ignored.

education: |
    MCA
    B.Sc
    Diploma

and

education: '>
     MCA
     B.Sc
     Diploma'

Special Characters

YAML allowed anything to put unquoted but there are some special cases where need to show value with quote.

Character (:) represents as mapping and Character (#) for comments. if these characters are occurs with space in any text value will use with (‘ ‘ or ” “) quotes.

Example :

description: 'you can write your code here: so that we can copy.'
or
description: "you can write your code here: so that we can copy."

The difference between single quotes and double quotes is that in double quotes you can use escapes.

description: "it\'s time to go home."

Variables

YAML uses “{{ var }}” for variables. If a any value after a colon starts with a “{” (curly bracket), YAML will think it is a dictionary, so you must quote it, like so:

Example :

log_file :"{{ LOG_DIR}}\\apps\\app_logs-*.log"

Note:

These reserved or special characters (‘ ” :[] {} > | * & ! % # ` @ ,) can not be used as first character of unquoted scalar. Only allowed characters are ( ? :  ) on beginning of string if a non-space character follows. Better always use quote for these type of scalar values.

Posted in Java, YAML | Tagged , , , , , , , | 3 Comments

YAML Supporting Language, Framework and Tools


YAML Supports so many languages and frameworks below are YAML Supporting languages, frameworks and respective tools :

C/C++ Libraries:

  • libyaml
  • libcyaml
  • Syck
  • yaml-cpp

Crystal:

  • YAML

Ruby:

  • psych
  • RbYaml
  • yaml4r

Python:

  • PyYAML
  • ruamel.yaml
  • PySyck

Java:

  •  JvYaml
  • SnakeYAML
  • YamlBeans
  • JYaml
  • Camel

Perl Modules:

  • YAML
  • YAML::XS
  • YAML::Syck
  • YAML::Tiny
  • PlYaml

C#/.NET:

  • YamlDotNet
  • yaml-net
  • yatools.net

Golang:

  •  Go-yaml
  • Go-gypsy

PHP:

  • php-yaml
  • syck
  • spyc

OCaml:

  • ocaml-syck

Javascript:

  • JS-YAML
  • JS-YAML Online

Actionscript:

  • as3yaml

Haskell:

  • YamlReference

Dart:

  • yaml

Rust:

  • yaml-rust
  • serde-yaml

Nim:

  • NimYAML

Others:

  • yamlvim (src)

Related Projects:

  • Rx
  • Kwalify
  • yaml_vim
  • yatools.net
  • JSON
  • Pygments

References

http://yaml.org/

More

To know more about YAML Syntax, Configuration with Java and other supporting language, frameworks and tools, Sample configuration files and JSON and YAML conversion follow below YAML Tutorials and YAML related exceptions follow YAML Issues.

 

Posted in Java, YAML | Tagged , , , , , | Leave a comment