Apache Tamaya — Extension: Model Documentation and Validation

Overview

The Tamaya model module provides support for documenting configuration and validating configuration read and processed against this model. Documentation and config models can be provided in different ways:

• as separate meta-model documents

• by providers that check classes/packages for configuration annotations (planned)

Compatibility

The module is based on Java 7, so it will not run on Java 7 and beyond.

Installation

To benefit from configuration builder support you only must add the corresponding dependency to your module:

<dependency>
  <groupId>org.apache.tamaya.ext</groupId>
  <artifactId>tamaya-model</artifactId>
  <version>{tamayaVersion}</version>
</dependency>

Describing the Configuration Meta-Model

Basically configuration is modelled using key, value-pairs. Looking at a keys a.b.c.param1 and a.b.c.param2 the following concepts can be used to defined/describe configuration:

1. the configuration section: In our case this equals to a.b.c, which itself also includes the transitive entries a.b and a.

2. the configuration parameter: Basically parameters are adressed using their fully qualified names, which equals to the containing section name and the relative parameter name, separated by the dor separator. In the above example a.b.c.param1 and a.b.c.param2 are the fully qualified parameter names.

Now with only these 2 concepts a simple configuration meta-model can be defined as

• a meta-model’s name, used just for grouping different meta-models and entries to better separate descriptions, e.g. in a management console or generated configuration documentation site.

• a set of sections.

• a set of parameters.

• Both, sections (.model.target=Section) as well as parameter models (.model.target=Parameter)

  • can be modelled by a meta-data entry, by default _my.config.key.model.

  • may be required, or optional (.model.required=true|false)

  • may have an optional description

• Parameters additionally have

  • a type (.model.type=Classname), described by the fully qualified class name, into which any configured (String) value must be convertable into. If no type is configured java.lang.String is assumed as default.

  • an optional regular expression that can be used to validate the String values returned from a configuration (.model.expression=regexpression).

Given these concepts a configuration can be fully described. Entries that are not contained in one of the given sections (or its children), or parameters not described or marked as valid (e.g. for dynamic configModels of a section), are called undefined. Undefined parameters should be grouped with its parent section. Each section, as well as all parent sections, including transitive) of any parametet read, should similarly marked as undefined, if and only if

1. the section itself is (directly) undefined

2. the section is not a super section of a defined section.

As en example the section definition of a.b.c also implicitly includes the sections a.b and a to be defined sections, despite the fact that section properties, such as description and custom configModels are not inherited to its parent, or child section.

Defining Meta-Configuration Model

The configuration meta-model is defined by simple configuration meta-data entries. The section for all model configuration by default is called model, which results in entries such as _my.config.key.model.target=Section. Within this section fully qualified configuration keys defines which part of the configuration is targeted by certain entries.

# Metamodel information
_model.provider=ConfigModel Extension

# org.mycompany.root (optional section)
_org.mycompany.root.model.target=Section
_org.mycompany.root.model.description=Root section defining my company configuration.

# org.mycompany.security (required section)
_org.mycompany.security.model.target=Section
_org.mycompany.security.model.required=true
_org.mycompany.security.model.description=Security related settings.\
         refer for further details to XXX.

# minmal section
_minimal.model.target=Section

# custom validated section
_validated.model.target=Section
_validated.model.validator=org.apache.tamaya.model.TestValidator

# org.mycompany.root.name (required parameter)
_org.mycompany.root.name.model.target=Parameter
_org.mycompany.root.name.model.required=true
_org.mycompany.root.name.model.description=The company's name, also used in the application's main header.

# org.mycompany.security (required parameters)
_org.mycompany.security.uid.model.required=true
_org.mycompany.security.uid.model.description=The user id.
_org.mycompany.security.realm.model.required=true
_org.mycompany.security.realm.model.validator=org.apache.tamaya.model.RealmValidator
_org.mycompany.security.realm.model.description=The security realm required.
_org.mycompany.security.tokenid.model.description=The token id, if the token service is used (optional).

# A minmal parameter
_minimalClass.model.target=Class

Tracking Configuration Access

The model module also allows tracking which code accesses configuration properties or configuration parameters. It checks the stacktrace to evaluate the calling code location, hereby any unwanted packages can be implicitly ommitted from the stacktrace. Also the maximal length of the stacktrace retained can be constraint in length. The usages are recorded as Usage instances. Hereby for each parameter accessed a corresponding Usage instance is created. It can be accessed by calling Usage ConfigUsageStats.getUsage(String key). Usage statistics for calling Configuration.getProperties() can be obtained calling Usage getUsageAllProps();.

Usage tracking is disabled by default. It can be enabled by calling ConfigUsageStats.enableUsageTracking(true);. ConfigUsageStats.isUsageTrackingEnabled() returns the current tracking status.

The Usage class itself provides access to further fainer grained usage data (AccessDetail) containing:

• the access point (fqn.ClassName#method(line: xxx)).

• the number of accesses

• the first an last access

• the values read

• the access stacktrace (filtered by ignored packages).

public final class Usage {
    [...]
    public String getKey();
    public void clearMetrics();
    public int getReferenceCount();
    public int getUsageCount();
    public Collection<AccessDetail> getAccessDetails(Class type);
    public Collection<AccessDetail> getAccessDetails(Package pack);
    public Collection<AccessDetail> getAccessDetails(String lookupExpression);
    public Collection<AccessDetail> getAccessDetails();
    public void trackUsage(String value);
    public void trackUsage(String value, int maxTraceLength);


    public static final class AccessDetail {
        [...]
        public void clearStats();
        public long trackAccess(String value);
        public long getAccessCount();
        public String getAccessPoint();
        public long getFirstAccessTS();
        public long getLastAccessTS();
        public String[] getStackTrace();
        public Map<Long, String> getTrackedValues();
    }

}

With ConfigUsageStats.clearUsageStats() the collected statistics can be reset at any time. Summarizing the main singleton for configuration statistics is defined as follows:

public final class ConfigUsageStats{
    public static Set<String> getIgnoredUsagePackages();
    public static void addIgnoredUsagePackages(String... packageName);
    public static void enableUsageTracking(boolean enabled);
    public static Usage getUsage(String key);
    public static Collection<Usage> getUsages();
    public static void clearUsageStats();
    public static Usage getUsageAllProperties();
    public static boolean isUsageTrackingEnabled();
    public static String getUsageInfo();
}

Accessing Usage Statistics

Bascially usage statistics are available in two forms:

• The Usage/AccessDetail object tree can be accessed programmatically from the ConfigUsageStats singleton.

• With ConfigUsageStats.getUsageInfo() also a textual representation of the usage statistics can be obtained, as illustrated below (a snipped from the current test output):

Apache Tamaya Configuration Usage Metrics
=========================================
DATE: Sat Apr 30 21:51:09 CEST 2016

220    <<all>>:
  - 220   <unknown/filtered/internal>                       , first=Sat Apr 30 21:51:09 CEST 2016, last=Sat Apr 30 21:51:09 CEST 2016
3      java.version:
  - 2     test.model.TestConfigAccessor#readProperty(line:43), first=Sat Apr 30 21:51:09 CEST 2016, last=Sat Apr 30 21:51:09 CEST 2016
  - 1     <unknown/filtered/internal>                       , first=Sat Apr 30 21:51:09 CEST 2016, last=Sat Apr 30 21:51:09 CEST 2016

public final class ConfigModelManager {

    private ConfigModelManager() {}

    public static Collection<ConfigModel> getModels();
    public static Collection<ConfigModel> findModels(ModelType type, String namePattern);
    public static <T extends ConfigModel> T getModel(String name, Class<T> modelType);
    public static Collection<ConfigModel> findModels(String namePattern);

    public static Collection<ValidationResult> validate();
    public static Collection<ValidationResult> validate(boolean showUndefined);
    public static Collection<ValidationResult> validate(Configuration config);
    public static Collection<ValidationResult> validate(Configuration config, boolean showUndefined);

    public static void registerMBean();
    public static void registerMBean(String context);

}

public interface ConfigModel {

    ModelTarget getType();
    String getName();
    String getProvider();
    boolean isRequired();
    String getDescription();
    Collection<ValidationResult> validate(Configuration config);
}

public enum ModelTarget {
    /**
     * A configuration section.
     */
    Section,
    /**
     * A configuration paramter.
     */
    Parameter,
    /**
     * ConfigModel that is a container of other validations.
     */
    Group
}

public final class ValidationResult {

    public static ValidationResult ofValid(ConfigModel configModel);
    public static ValidationResult ofMissing(ConfigModel configModel);
    public static ValidationResult ofMissing(ConfigModel configModel, String message);
    public static ValidationResult ofError(ConfigModel configModel, String error);
    public static ValidationResult ofWarning(ConfigModel configModel, String warning);
    public static ValidationResult ofDeprecated(ConfigModel configModel, String alternateUsage);
    public static ValidationResult ofDeprecated(ConfigModel configModel);
    public static ValidationResult ofUndefined(final String key);
    public static ValidationResult of(ConfigModel configModel, ValidationState result, String message);

    public ConfigModel getConfigModel();
    public ValidationState getResult();
    public String getMessage(),
}

Auto-Documentation of Classes with Configuration Injection

A special feature of this module is that it observes ConfigEvent published through Tamaya’as event channel (tamaya-events module). If no metaconfiguration model is found the model manager by default automatically creates models for all injected instances on the fly. In the case of CDI integration this happens typically during deployment time, since CDI initializes during deployment time. Other runtime platforms, such as OSGI, may have rather different behaviour. Nevertheless this means that after your system has been started you should have access to a complete set of ConfigModel instances that automatically document all the classes in your system that consume configuration (through injection).

Registering Configuration Models

The model extension also provides an SPI where customized functionality can be added. The main abstraction hereby is the ModelProviderSpi interface, which allows any kind of additional config models to be added to the system:

public interface ModelProviderSpi {

    Collection<ConfigModel> getConfigModels();

}

New instances implementing this interface must be registered into the current ServiceContext, by default the ServiceLoader is used.

The ConfigUsageStatsSpi

The methods for managing and tracking of configuration changes are similarly delegated to an implementation of the org.apache.tamaya.model.spi.ConfigUsageStatsSpi SPI. By implementing this SPI and registerting it with the ServiceContext the usage tracking logic can be adapted or replaced.

Other Utility Classes

The module also provides further utility classes that may be useful for implementing models or testing:

• AbstractModel provides a base class that can be extended, when implementing ConfigModel.

• AreaConfigModel provides a ConfigModel implementation (with a corresponding Builder) to model the requirement of certain configuration sections being present, or opionally present, in the model.

• ParameterModel provides an implementation base class for validating parameters on existence and compliance with a regular expression.

• ConfigDocumentationMBean is the MBean registered that models similar functionality as ConfigModelManager.

• ConfigModelGroup provides a ConfigModel that groups several child models.

• ConfigModelReader allows to read ConfigModels from properties files as described at the beginning of this document.

Switches to enable/disable functionality

The model module provides different switches that can be used to activate or deactivate features:

• tamaya.model.integrated.enabled allows to deactivate reading inline metaconfiguration delivered with the normal Tamaya Configuration. By default inline entries (_.abcd.model.*) are evaluated.

• tamaya.model.default.enabled allows to deactivate reading metamodel information from classpath:META-INF/configmodel.properties. By default it is active.

• tamaya.model.resources allows to define additional resources (loaded through the resources extension), that can be used to read metamodel information in any format using Tamaya’s format module.

• the system property tamaya.model.autoModelEvents allows to activate/deactivate the automatic documentation of classes configured and published by Tamaya ConfiguredType event instances (e.g. published by Tamaya’s injection modules).