Filters allow Log Events to be evaluated to determine if or how they should be published. A Filter will be called on one if its filter methods and will return a Result, which is an Enum that has one of 3 values - ACCEPT, DENY or NEUTRAL.
Filters may be configured in one of four locations;
The BurstFilter provides a mechanism to control the rate at which LogEvents are processed by silently discarding events after the maximum limit has been reached.
Parameter Name | Type | Description |
---|---|---|
level | String | Level of messages to be filtered. Anything at or below this level will be filtered out if maxBurst has been exceeded. The default is WARN meaning any messages that are higher than warn will be logged regardless of the size of a burst. |
rate | float | The average number of events per second to allow. |
maxBurst | integer | The maximum number of events that can occur before events are filtered for exceeding the average rate. The default is 10 times the rate. |
omMatch | String | Action to take when the filter matches. May be ACCEPT, DENY or NEUTRAL. The default value is NEUTRAL. |
omMismatch | String | Action to take when the filter does not match. May be ACCEPT, DENY or NEUTRAL. The default value is DENY. |
A configuration containing the BurstFilter might look like:
<?xml version="1.0" encoding="UTF-8"?> <configuration status="warn" name="MyApp" packages=""> <appenders> <RollingFile name="RollingFile" fileName="logs/app.log" filePattern="logs/app-%d{MM-dd-yyyy}.log.gz"> <BurstFilter level="INFO" rate="16" maxBurst="100"/> <PatternLayout> <pattern>%d %p %C{1.} [%t] %m%n</pattern> </PatternLayout> <TimeBasedTriggeringPolicy /> </RollingFile> </appenders> <loggers> <root level="error"> <appender-ref ref="RollingFile"/> </root> </loggers> </configuration>
The CompositeFilter provides a way to specify more than one filter. It is added to the configuration as a filters element and contains other filters to be evaluated. The filters element accepts no parameters.
A configuration containing the CompositeFilter might look like:
<?xml version="1.0" encoding="UTF-8"?> <configuration status="warn" name="MyApp" packages=""> <filters> <Marker marker="EVENT" onMatch="ACCEPT" onMismatch="NETURAL"/> <DynamicThresholdFilter key="loginId" defaultThreshold="ERROR" onMatch="ACCEPT" onMismatch="NEUTRAL"> <KeyValuePair key="User1" value="DEBUG"/> </DynamicThresholdFilter> </filters> <appenders> <File name="Audit" fileName="logs/audit.log"> <PatternLayout> <pattern>%d %p %C{1.} [%t] %m%n</pattern> </PatternLayout> </File> <RollingFile name="RollingFile" fileName="logs/app.log" filePattern="logs/app-%d{MM-dd-yyyy}.log.gz"> <BurstFilter level="INFO" rate="16" maxBurst="100"/> <PatternLayout> <pattern>%d %p %C{1.} [%t] %m%n</pattern> </PatternLayout> <TimeBasedTriggeringPolicy /> </RollingFile> </appenders> <loggers> <logger name="EventLogger" level="info"> <appender-ref ref="Audit"/> </logger> <root level="error"> <appender-ref ref="RollingFile"/> </root> </loggers> </configuration>
The DynamicThresholdFilter allows filtering by log level based on specific attributes. For example, if the user's loginid is being captured in the ThreadContext Map then it is possible to enable debug logging for only that user.
Parameter Name | Type | Description |
---|---|---|
defaultThreshold | String | Level of messages to be filtered. If there is no matching key in the key/value pairs then this level will be compared against the event's level. |
keyValuePair | KeyValuePair[] | One or more KeyValuePair elements that define the matching value for the key and the Level to evaluate when the key matches. |
omMatch | String | Action to take when the filter matches. May be ACCEPT, DENY or NEUTRAL. The default value is NEUTRAL. |
omMismatch | String | Action to take when the filter does not match. May be ACCEPT, DENY or NEUTRAL. The default value is DENY. |
Here is a sample configuration containing the DynamicThresholdFilter:
<?xml version="1.0" encoding="UTF-8"?> <configuration status="warn" name="MyApp" packages=""> <DynamicThresholdFilter key="loginId" defaultThreshold="ERROR" onMatch="ACCEPT" onMismatch="NEUTRAL"> <KeyValuePair key="User1" value="DEBUG"/> </DynamicThresholdFilter> <appenders> <RollingFile name="RollingFile" fileName="logs/app.log" filePattern="logs/app-%d{MM-dd-yyyy}.log.gz"> <BurstFilter level="INFO" rate="16" maxBurst="100"/> <PatternLayout> <pattern>%d %p %C{1.} [%t] %m%n</pattern> </PatternLayout> <TimeBasedTriggeringPolicy /> </RollingFile> </appenders> <loggers> <root level="error"> <appender-ref ref="RollingFile"/> </root> </loggers> </configuration>
The MapFilter allows filtering against data elements that are in a MapMessage.
Parameter Name | Type | Description |
---|---|---|
keyValuePair | KeyValuePair[] | One or more KeyValuePair elements that define the key in the map and the value to match on. |
operator | String | If the operator is "or" then a match by any one of the key/value pairs will be considered to be a match, otherwise all the key/value pairs must match. |
omMatch | String | Action to take when the filter matches. May be ACCEPT, DENY or NEUTRAL. The default value is NEUTRAL. |
omMismatch | String | Action to take when the filter does not match. May be ACCEPT, DENY or NEUTRAL. The default value is DENY. |
As in this configuration, the MapFilter can be used to log particular events:
<?xml version="1.0" encoding="UTF-8"?> <configuration status="warn" name="MyApp" packages=""> <MapFilter onMatch="ACCEPT" onMismatch="NEUTRAL" operator="or"> <KeyValuePair key="eventId" value="Login"/> <KeyValuePari key="eventId" value="Logout"/> </MapFilter> <appenders> <RollingFile name="RollingFile" fileName="logs/app.log" filePattern="logs/app-%d{MM-dd-yyyy}.log.gz"> <BurstFilter level="INFO" rate="16" maxBurst="100"/> <PatternLayout> <pattern>%d %p %C{1.} [%t] %m%n</pattern> </PatternLayout> <TimeBasedTriggeringPolicy /> </RollingFile> </appenders> <loggers> <root level="error"> <appender-ref ref="RollingFile"/> </root> </loggers> </configuration>
This sample configuration will exhibit the same behavior as the preceding example since the only logger configured is the root.
<?xml version="1.0" encoding="UTF-8"?> <configuration status="warn" name="MyApp" packages=""> <appenders> <RollingFile name="RollingFile" fileName="logs/app.log" filePattern="logs/app-%d{MM-dd-yyyy}.log.gz"> <BurstFilter level="INFO" rate="16" maxBurst="100"/> <PatternLayout> <pattern>%d %p %C{1.} [%t] %m%n</pattern> </PatternLayout> <TimeBasedTriggeringPolicy /> </RollingFile> </appenders> <loggers> <root level="error"> <MapFilter onMatch="ACCEPT" onMismatch="NEUTRAL" operator="or"> <KeyValuePair key="eventId" value="Login"/> <KeyValuePari key="eventId" value="Logout"/> </MapFilter> <appender-ref ref="RollingFile"> </appender-ref> </root> </loggers> </configuration>
This third sample configuration will exhibit the same behavior as the preceding examples since the only logger configured is the root and the root is only configured with a single appender reference.
<?xml version="1.0" encoding="UTF-8"?> <configuration status="warn" name="MyApp" packages=""> <appenders> <RollingFile name="RollingFile" fileName="logs/app.log" filePattern="logs/app-%d{MM-dd-yyyy}.log.gz"> <BurstFilter level="INFO" rate="16" maxBurst="100"/> <PatternLayout> <pattern>%d %p %C{1.} [%t] %m%n</pattern> </PatternLayout> <TimeBasedTriggeringPolicy /> </RollingFile> </appenders> <loggers> <root level="error"> <appender-ref ref="RollingFile"> <MapFilter onMatch="ACCEPT" onMismatch="NEUTRAL" operator="or"> <KeyValuePair key="eventId" value="Login"/> <KeyValuePari key="eventId" value="Logout"/> </MapFilter> </appender-ref> </root> </loggers> </configuration>
The MarkerFilter compares the configured Marker value against the Marker that is included in the LogEvent. A match occurs when the Marker name matches either the Log Event's Marker or one of its parents.
Parameter Name | Type | Description |
---|---|---|
marker | String | The name of the Marker to compare. |
omMatch | String | Action to take when the filter matches. May be ACCEPT, DENY or NEUTRAL. The default value is NEUTRAL. |
omMismatch | String | Action to take when the filter does not match. May be ACCEPT, DENY or NEUTRAL. The default value is DENY. |
A sample configuration that only allows the event to be written by the appender if the Marker matches:
<?xml version="1.0" encoding="UTF-8"?> <configuration status="warn" name="MyApp" packages=""> <appenders> <RollingFile name="RollingFile" fileName="logs/app.log" filePattern="logs/app-%d{MM-dd-yyyy}.log.gz"> <MarkerFilter marker="FLOW" onMatch="ACCEPT" onMismatch="DENY"/> <PatternLayout> <pattern>%d %p %C{1.} [%t] %m%n</pattern> </PatternLayout> <TimeBasedTriggeringPolicy /> </RollingFile> </appenders> <loggers> <root level="error"> <appender-ref ref="RollingFile"/> </root> </loggers> </configuration>
The RegexFilter allows the formatted or unformatted message to be compared against a regular expression.
Parameter Name | Type | Description |
---|---|---|
regex | String | The regular expression. |
useRawMsg | boolean | If true the unformatted message will be used, otherwise the formatted message will be used. The default value is false. |
omMatch | String | Action to take when the filter matches. May be ACCEPT, DENY or NEUTRAL. The default value is NEUTRAL. |
omMismatch | String | Action to take when the filter does not match. May be ACCEPT, DENY or NEUTRAL. The default value is DENY. |
A sample configuration that only allows the event to be written by the appender if it contains the word "test":
<?xml version="1.0" encoding="UTF-8"?> <configuration status="warn" name="MyApp" packages=""> <appenders> <RollingFile name="RollingFile" fileName="logs/app.log" filePattern="logs/app-%d{MM-dd-yyyy}.log.gz"> <RegexFilter regex=".* test .*" onMatch="ACCEPT" onMismatch="DENY"/> <PatternLayout> <pattern>%d %p %C{1.} [%t] %m%n</pattern> </PatternLayout> <TimeBasedTriggeringPolicy /> </RollingFile> </appenders> <loggers> <root level="error"> <appender-ref ref="RollingFile"/> </root> </loggers> </configuration>
The StructuredDataFilter is a MapFilter that also allows filtering on the event id, type and message.
Parameter Name | Type | Description |
---|---|---|
keyValuePair | KeyValuePair[] | One or more KeyValuePair elements that define the key in the map and the value to match on. "id", "id.name", "type", and "message" should be used to match on the StructuredDataId, the name portion of the StructuredDataId, the type, and the formatted message respectively. |
operator | String | If the operator is "or" then a match by any one of the key/value pairs will be considered to be a match, otherwise all the key/value pairs must match. |
omMatch | String | Action to take when the filter matches. May be ACCEPT, DENY or NEUTRAL. The default value is NEUTRAL. |
omMismatch | String | Action to take when the filter does not match. May be ACCEPT, DENY or NEUTRAL. The default value is DENY. |
As in this configuration, the StructuredDataFilter can be used to log particular events:
<?xml version="1.0" encoding="UTF-8"?> <configuration status="warn" name="MyApp" packages=""> <StructuredDataFilter onMatch="ACCEPT" onMismatch="NEUTRAL" operator="or"> <KeyValuePair key="id" value="Login"/> <KeyValuePari key="id" value="Logout"/> </StructuredDataFilter> <appenders> <RollingFile name="RollingFile" fileName="logs/app.log" filePattern="logs/app-%d{MM-dd-yyyy}.log.gz"> <BurstFilter level="INFO" rate="16" maxBurst="100"/> <PatternLayout> <pattern>%d %p %C{1.} [%t] %m%n</pattern> </PatternLayout> <TimeBasedTriggeringPolicy /> </RollingFile> </appenders> <loggers> <root level="error"> <appender-ref ref="RollingFile"/> </root> </loggers> </configuration>
The ThreadContextMapFilter allows filtering against data elements that are in the ThreadContext Map.
Parameter Name | Type | Description |
---|---|---|
keyValuePair | KeyValuePair[] | One or more KeyValuePair elements that define the key in the map and the value to match on. |
operator | String | If the operator is "or" then a match by any one of the key/value pairs will be considered to be a match, otherwise all the key/value pairs must match. |
omMatch | String | Action to take when the filter matches. May be ACCEPT, DENY or NEUTRAL. The default value is NEUTRAL. |
omMismatch | String | Action to take when the filter does not match. May be ACCEPT, DENY or NEUTRAL. The default value is DENY. |
A configuration containing the ThreadContextMapFilter might look like:
<?xml version="1.0" encoding="UTF-8"?> <configuration status="warn" name="MyApp" packages=""> <DynamicThresholdFilter key="loginId" defaultThreshold="ERROR" onMatch="ACCEPT" onMismatch="NEUTRAL"> <KeyValuePair key="User1" value="DEBUG"/> </DynamicThresholdFilter> <appenders> <RollingFile name="RollingFile" fileName="logs/app.log" filePattern="logs/app-%d{MM-dd-yyyy}.log.gz"> <BurstFilter level="INFO" rate="16" maxBurst="100"/> <PatternLayout> <pattern>%d %p %C{1.} [%t] %m%n</pattern> </PatternLayout> <TimeBasedTriggeringPolicy /> </RollingFile> </appenders> <loggers> <root level="error"> <appender-ref ref="RollingFile"/> </root> </loggers> </configuration>
This filter returns the onMatch result if the level in the LogEvent is the same or more specific than the configured level and the onMismatch value otherwise. For example, if the ThresholdFilter is configured with Level ERROR and the LogEvent contains Level DEBUG then the onMismatch value will be returned since ERROR events are more specific than DEBUG.
Parameter Name | Type | Description |
---|---|---|
level | String | A valid Level name to match on. |
omMatch | String | Action to take when the filter matches. May be ACCEPT, DENY or NEUTRAL. The default value is NEUTRAL. |
omMismatch | String | Action to take when the filter does not match. May be ACCEPT, DENY or NEUTRAL. The default value is DENY. |
A sample configuration that only allows the event to be written by the appender if the level matches:
<?xml version="1.0" encoding="UTF-8"?> <configuration status="warn" name="MyApp" packages=""> <appenders> <RollingFile name="RollingFile" fileName="logs/app.log" filePattern="logs/app-%d{MM-dd-yyyy}.log.gz"> <ThresholdFilter level="TRACE" onMatch="ACCEPT" onMismatch="DENY"/> <PatternLayout> <pattern>%d %p %C{1.} [%t] %m%n</pattern> </PatternLayout> <TimeBasedTriggeringPolicy /> </RollingFile> </appenders> <loggers> <root level="error"> <appender-ref ref="RollingFile"/> </root> </loggers> </configuration>
The time filter can be used to restrict filter to only a certain portion of the day.
Parameter Name | Type | Description |
---|---|---|
start | String | A time in HH:mm:ss format. |
end | String | A time in HH:mm:ss format. Specifying an end time less than the start time will result in no log entries being written. |
timezone | String | The timezone to use when comparing to the event timestamp. |
omMatch | String | Action to take when the filter matches. May be ACCEPT, DENY or NEUTRAL. The default value is NEUTRAL. |
omMismatch | String | Action to take when the filter does not match. May be ACCEPT, DENY or NEUTRAL. The default value is DENY. |
A sample configuration that only allows the event to be written by the appender from 5:00 to 5:30 am each day using the default timezone:
<?xml version="1.0" encoding="UTF-8"?> <configuration status="warn" name="MyApp" packages=""> <appenders> <RollingFile name="RollingFile" fileName="logs/app.log" filePattern="logs/app-%d{MM-dd-yyyy}.log.gz"> <TiemFilter start="05:00:00" end="05:30:00" onMatch="ACCEPT" onMismatch="DENY"/> <PatternLayout> <pattern>%d %p %C{1.} [%t] %m%n</pattern> </PatternLayout> <TimeBasedTriggeringPolicy /> </RollingFile> </appenders> <loggers> <root level="error"> <appender-ref ref="RollingFile"/> </root> </loggers> </configuration>