Adding Conversion Characters to PatternLayout

"Paul Glezen" January 2001

Abstract

This article describes a systematic way to extend the log4j API to include additional attributes formatted using the PatternLayout class.

Contents

Introduction

This article assumes familiarity with the log4j User Manual. It builds on fundamental classes described in both the User Manual and the Javadoc API. To assist in illustrating the concepts, a simple case study will be developed along side the explanations. The resulting classes may be used as a template for your own extensions. Condenced (i.e. statements compressed, comments removed) snipets of the case study code are included in this document.

The Case Study

The case study was developed in a CORBA environment in which the following information for each log entry was needed. The letters in parenthesis represent the corresponding character to be used by the PatternLayout class for formatting.

It seems odd to use "b" for the component name. Presently PatternLayout already defines both "C" and "c" for class name and category name respectively.

A Peek Under the Hood

In principle, if the steps described below are followed closely, there is not a need to understand how the extended classes will be used by log4j. But sometimes software development can be entirely unprincipled. You may wish to extend log4j in a different manner than describe here or you may make a mistake that requires knowledge of what is really going on. (Heaven forbid there be a mistake in this document). In any case, it doesn't hurt to get an idea of what's going on.

The following describes a "typical" logging scenario in the un-extended log4j case.

  1. Application code invokes a log request on a Category object. Let's say the info method was invoked.

  2. The first thing info does is to check if logging has been turned off entirely for the info level. If so, it returns immediately. We'll assume for this scenario that logging has not been turned off for the info level.

  3. Next info compares the Priority level for this category against Priority.INFO. Assuming the priority warrants logging the message, the category instantiates a LoggingEvent object populated with information available for logging.

  4. The Category instance passes the LoggingEvent instance to all its Appender implementations.

  5. Most (but not all) Appender implementations should have an associated subclass of Layout. The Layout subclass is passed the LoggingEvent instance and returns the event's information formatted in a String according to the configuration of the Layout.

    When the Layout subclass is PatternLayout, the format of the event's information is determined by a character sequence similar to the C language library's printf routine. PatternLayout delegates the parsing of this character sequence to a PatternParser instance.

    When the PatternLayout was constructed, it created a PatternParser to tokenize the character sequence. Upon recognizing a token, the PatternParser constructs an appropriate PatternConverter subclass, passing it formatting information from the token. Often the PatternConverter subclasses are implemented as static inner classes of PatternParser. The parse method of the PatternParser returns a linked list of these PatternConverter subclasses.

  6. PatternLayout.format() passes the LoggingEvent to each PatternConverter subclass in the linked list. Each link in the list selects a particular item from the LoggingEvent, converts this item to a String in the proper format and appends it to a StringBuffer.

  7. The format method returns the resulting String to the Appender for output.

The above discussing involved most of the classes that we must extend or implement.

The Process

 Below are steps required to add additional attributes available for logging by extending log4j. This will allow you to specify their output formats in the same manner as those provided by the PatternLayout class. The steps are numbered for reference only. It makes no difference in which order they are followed.

It's helpful if you know the attributes you wish to add and a PatternLayout symbol for each one before you begin. Be sure to consult the PatternLayout documentation to ensure the symbols you select are not already in use.

Before we dig in, I should give the standard lecture on comments. If the log4j library were not well documented, it would be useless to everyone but the log4j creators; likewise with your extensions. Much like eating vegetables and saving the environment, we all agree commenting code properly should be done. Yet it is often sacrificed for more immediate pleasures. We all write code faster without comments; especially those pesky Javadoc comments. But the reality is that the utility of undocumented code fades exponentially with time.

Since the log4j product comes with Javadoc comments together with the documentation it produces, it makes sense to include Javadoc comments in your extensions. By their very nature, logging tools are strong candidates for re-use. They can only be independently re-used if they are supported by strong documentation component.

This all having been said, I have elected to remove most comments from examples in the interest of space rather than including them to serve as a nagging reminder. The reader is referred to the case study source code files for a Javadoc version and a Javadoc website for more information on Javadoc conventions.

1. Extending LoggingEvent

Extending the LoggingEvent class should be one of the trivial steps. All that is needed in the extension is the addition of public data members representing the new attributes and a new constructor to populate them. 

import org.apache.log4j.Category;
import org.apache.log4j.Priority;
import org.apache.log4j.spi.LoggingEvent;

public class AppServerLoggingEvent extends LoggingEvent
                                   implements java.io.Serializable 
{
   public String hostname;
   public String component;
   public String server;
   public String version;
   
   public AppServerLoggingEvent( String    fqnOfCategoryClass, 
                                 AppServerCategory  category, 
                                 Priority  priority, 
                                 Object    message, 
                                 Throwable throwable) 
   {
      super( fqnOfCategoryClass,
             category,
             priority,
             message,
             throwable );

      hostname  = category.getHostname();
      component = category.getComponent();
      server    = category.getServer();
      version   = category.getVersion();
   }  
}

The constructor demonstrates that in most cases, the Category subclass will contain most of the information necessary to populate the attributes of the LoggingEvent subclass. Extensions to LoggingEvent seem no more than a collection of strings with a constructor. Most of the work is done by the super class.

2. Extending PatternLayout

Extending the PatternLayout class should be another simple matter. The extension to PatternLayout should differ from its parent only in the creation of a PatternParser instance. The extended PatternLayout should create an extended PatternParser class. Fortunately, this task in PatternLayout is encapsulated within a single method. 

import org.apache.log4j.PatternParser;
import org.apache.log4j.PatternLayout;

public class AppServerPatternLayout extends PatternLayout 
{
   public AppServerPatternLayout() 
   {
      this(DEFAULT_CONVERSION_PATTERN);
   }

   public MyPatternLayout(String pattern) 
   {
      super(pattern);
   }
    
   public PatternParser createPatternParser(String pattern) 
   {
      PatternParser result;
      if ( pattern == null )
         result = new AppserverPatternParser( DEFAULT_CONVERSION_PATTERN );
      else
         result = new AppServerPatternParser ( pattern );

      return result;
  }
}

3. Extend PatternParser and PatternConverter

Recall from our peek under the hood that the PatternParser does much of its work in its parse method. The PatternLayout object instantiates a PatternParser object by passing it the pattern string. The PatternLayout then invokes the parse method of PatternParser to produce a linked list of PatternConverter subclass instances. It is this linked list of converters that is used to convert an event instance into a string used by appenders.

Our job will be to subclass PatternParser to properly interpret formatting characters we wish to add. Fortunately, PatternParser has been designed so that only the one step in the parsing process differing for each formatting character has to be overridden. The grunt work of parsing is still performed by the PatternParser.parse() method. Only the PatternParser.finalizeConverter method has to be overridden. This is the method that decides which PatternConverter to create based on a formatting character.

The extension to PatternParser, AppServerPatternParser, is similar to its super class. It uses

AppServerPatternParser differs principally by dedicating a separate converter type for each logging attribute to be formatted. Rather than placing switch logic in the converter, like its parent class, each converter only converts one format character. This means the decision of which converter subclass to instantiate is made at layout instantiation time rather than in a switch statement at logging time.

It also differs in that the format constants are characters rather than integers. 

import org.apache.log4j.*;
import org.apache.log4j.helpers.FormattingInfo;
import org.apache.log4j.helpers.PatternConverter;
import org.apache.log4j.helpers.PatternParser;
import org.apache.log4j.spi.LoggingEvent;

public class AppServerPatternParser extends PatternParser 
{
   static final char HOSTNAME_CHAR  = 'h';
   static final char SERVER_CHAR    = 's';
   static final char COMPONENT_CHAR = 'b';
   static final char VERSION_CHAR   = 'v';

   public AppServerPatternParser(String pattern) 
   {
      super(pattern);
   }
    
   public void finalizeConverter(char formatChar) 
   {
      PatternConverter pc = null;
      switch( formatChar )
      {
         case HOSTNAME_CHAR:
            pc = new HostnamePatternConverter( formattingInfo );
            currentLiteral.setLength(0);
            addConverter( pc );
            break;
         case SERVER_CHAR:
            pc = new ServerPatternConverter( formattingInfo );
            currentLiteral.setLength(0);
            addConverter( pc );
            break;
         case COMPONENT_CHAR:
            pc = new ComponentPatternConverter( formattingInfo );
            currentLiteral.setLength(0);
            addConverter( pc );
            break;
         case VERSION_CHAR:
            pc = new VersionPatternConverter( formattingInfo );
            currentLiteral.setLength(0);
            addConverter( pc );
            break;
         default:
            super.finalizeConverter( formatChar );
      }
   }
  
   private static abstract class AppServerPatternConverter extends PatternConverter 
   {
      AppServerPatternConverter(FormattingInfo formattingInfo) 
      {
         super(formattingInfo);     
      }

      public String convert(LoggingEvent event) 
      {
         String result = null;
         AppServerLoggingEvent appEvent = null;

         if ( event instanceof AppServerLoggingEvent )
         {
            appEvent = (AppServerLoggingEvent) event;
            result = convert( appEvent );
         }
         return result;
      }

      public abstract String convert( AppServerLoggingEvent event );
   }

   private static class HostnamePatternConverter extends AppServerPatternConverter
   {
      HostnamePatternConverter( FormattingInfo formatInfo )
      {  super( formatInfo );  }

      public String convert( AppServerLoggingEvent event )
      {  return event.hostname;  }
   }

   private static class ServerPatternConverter extends AppServerPatternConverter
   {
      ServerPatternConverter( FormattingInfo formatInfo )
      {  super( formatInfo );  }

      public String convert( AppServerLoggingEvent event )
      {  return event.server;  }
   }

   private static class ComponentPatternConverter extends AppServerPatternConverter
   {
      ComponentPatternConverter( FormattingInfo formatInfo )
      {  super( formatInfo );  }

      public String convert( AppServerLoggingEvent event )
      {  return event.component;  }
   }

   private static class VersionPatternConverter extends AppServerPatternConverter
   {
      VersionPatternConverter( FormattingInfo formatInfo )
      {  super( formatInfo );  }

      public String convert( AppServerLoggingEvent event )
      {  return event.version;  }
   }
}

4. Extending Category

Extending Category and its factory will be more straight forward than extending PatternParser and the converters. The following tasks are involved in overridding Category for our purposes.

Most of the code below is standard getter/setter verbage which has been somewhat abbreviated. The notable parts are in bold. We add five more attributes to Category: the four new logging attributes plus a static AppServerCategoryFactory reference. This is pre-initialized to an instance with attributes set to null as a precautionary measure. Otherwise the getInstance method will result in a null pointer exception if invoked before the setFactory method.

The getInstance method simply invokes its parent class method that accepts a CategoryFactory reference in addition to the category name.

The forcedLog method follows closely the corresponding parent class method. The most important difference is the instantiation of the AppServerLoggingEvent. A minor yet necessary difference is the use of the getRendererMap() method rather than accessing the data member directory as in Category. Category can do this because the rendererMap is package level accessible.

The setFactory method is provided to allow application code to set the factory used in the getInstance method. 

import org.apache.log4j.Priority;
import org.apache.log4j.Category;
import org.apache.log4j.spi.CategoryFactory;
import org.apache.log4j.spi.LoggingEvent;

public class AppServerCategory extends Category
{
   protected String component;
   protected String hostname;
   protected String server;
   protected String version;
   private static CategoryFactory factory = 
                  new AppServerCategoryFactory(null, null, null);

   protected  AppServerCategory( String categoryName,
                                 String hostname,
                                 String server,
                                 String component,
                                 String version )
   {
      super( categoryName );
      instanceFQN = "org.apache.log4j.examples.appserver.AppServerCategory";
      
      this.hostname  = hostname;
      this.server    = server;
      this.component = component;
      this.version   = version;
   }

   public String getComponent()
   { return (component == null ) ? "" : result; }

   public String getHostname()
   {  return ( hostname == null ) ? "" : hostname; }

   public static Category getInstance(String name)
   {
      return Category.getInstance(name, factory);
   }

   public String getServer()
   {  return ( server == null ) ? "" : server; }

   public String getVersion()
   {  return ( version == null ) ? "" : version; }

   protected void forcedLog( String    fqn, 
                             Priority  priority, 
                             Object    message, 
                             Throwable t) 
   {
      LoggingEvent event = new AppServerLoggingEvent(fqn, this, priority, message, t);
      callAppenders( event );
   }

   public void setComponent(String componentName)
   { component = componentName; }

   public static void setFactory(CategoryFactory factory)
   { AppServerCategory.factory = factory; }

   public void setHostname(String hostname)
   { this.hostname = hostname; }

   public void setServer(String serverName)
   { server = serverName; }

   public void setVersion(String versionName)
   { version = versionName; }
}

5. Extending CategoryFactory

The last step is to provide an implementation of the CategoryFactory interface that will correctly instantiate our AppServerCategory objects. It will obtain the hostname of the machine on which it runs using the java.net API. Aside from providing getters and setters for the attributes introduced, the only method to be implemented is the makeNewCategoryInstance.

Below is a snipet from AppServerCategoryFactory with getters, setters and comments removed. 

import org.apache.log4j.Category;
import org.apache.log4j.spi.CategoryFactory;
import java.net.InetAddress;
import java.net.UnknownHostException;

public class AppServerCategoryFactory implements CategoryFactory
{
   protected String hostname;
   protected String server;
   protected String component;
   protected String version;
   protected ResourceBundle messageBundle;

   protected  AppServerCategoryFactory( String serverName,
                                        String componentName,
                                        String versionName )
   {
      try
      {
         hostname = java.net.InetAddress.getLocalHost().getHostName();
      }
      catch ( java.net.UnknownHostException uhe )
      {
         System.err.println("Could not determine local hostname.");
      }

      server    = serverName;
      component = componentName;
      version   = versionName;
   }

   public Category makeNewCategoryInstance(String name)
   {
       Category result = new AppServerCategory( name, 
                                                hostname, 
                                                server, 
                                                component, 
                                                version);
       return result;
   }
}

Usage

 We now arrive at how to use what we have created. We must remember to initialize log4j by creating an instance of AppServerCategoryFactory and passing it to AppServerCategory. Once done, we can obtain a AppServerCategoryInstance anytime by using the static getInstance method of AppServerCategory. This will ensure that AppServerLoggingEvent instances are generated by the category logging methods. 

import org.apache.log4j.*;
import org.apache.log4j.appserver.AppServerCategory;
import org.apache.log4j.appserver.AppServerCategoryFactory;
import org.apache.log4j.appserver.AppServerPatternLayout;

public class test
{
   private static String formatString = 
      "---------------------------------------------------%n" +
      "Time:      %d%n" +
      "Host:      %h%n" +
      "Server:    %s%n" +
      "Component: %b%n" +
      "Version:   %v%n" +
      "Priority:  %p%n" +
      "Thread Id: %t%n" +
      "Context:   %x%n" +
      "Message:   %m%n";

   public static void main(String[] args)
   {
      AppServerCategoryFactory factory;
      factory = new AppServerCategoryFactory("MyServer", "MyComponent", "1.0");
      AppServerCategory.setFactory( factory );
      Category cat = AppServerCategory.getInstance("some.cat");

      PatternLayout layout = new AppServerPatternLayout( formatString );
      cat.addAppender( new FileAppender( layout, System.out) );

      cat.debug("This is a debug statement.");
      cat.info("This is an info statement.");
      cat.warn("This is a warning statement.");
      cat.error("This is an error statement.");
      cat.fatal("This is a fatal statement.");
   }
}

Configurators

There is one a word of caution concerning the use of configurators that may create Category instances (such as PropertyConfigurator and DOMConfigurator). Since these configurators do not know about our extensions, any Category instances they create will not be AppServerCategory instances. To prevent this problem, any AppServerCategory that one might want to be configured through a configurator should be instantiated before the configure method is invoked. In this way, the configurator will configure the AppServerCategory that already exists rather than creating an instance of its super class.

The consequence of a configurator creating the super class by mistake is merely that the extra attributes will not appear in the log output. All other attributes are conveyed properly.

Further Enhancements

 There are some other directions in which this log4j extension may be enhanced.

  1. The hostname attribute could incorportate a formatting convention similar to that of class and category names whereby only a certain number of the more significant components are displayed. But whereas with class and category names, the most significant component is on the right, with host names, it is on the left.

  2. Specifying a version number could be dangerous since programmers are apt to change versions of the code without changing the string constant in the code which specifies the version. Some source control programs provide for insertion of a version number into source. For those that don't, including the version number as a constant is likely to lead to confusion later on. It would be nice to see this short-coming addressed.