UIMA Version 3 User's Guide

The major improvements in version 3 include:

Just to give a small taste of the kinds of things Java 8 integration provides, here's an example of using the new select framework, where the task is to compute

Here is the Java code using the new select framework together with Java 8 streaming functions:

Set<Type> foundTypes =
   myIndex.select(MyType.class) 
   .coveredBy(myBoundingAnnotation)
   .nonOverlapping()
   .map(fs -> fs.getType())
   .collect(Collectors.toCollection(TreeSet::new));

Another example: to collect, by category, the average length of the annotations having that category. Here we assume that MyType is an Annotation and that it has a feature called category which returns a String denoting the category:

Map<String, Double> freqByCategory =
   myIndex.select(MyType.class)
   .collect(Collectors
     .groupingBy(MyType::getCategory,
                 Collectors.averagingDouble(f -> 
                   (double)(f.getEnd() - f.getBegin()))));

The UIMA Java SDK Version 3 requires Java 8.

The JCas class changes include no longer needing or using the Xyz_Type sister classes for each main JCas class. User code is unlikely to access these sister classes. The JCas API method to access this sister class now throws a UnsupportedOperation exception.

The non-JCas Java cover classes for the built-in UIMA types remain, for backwards compatibility. So, if you have code that casts a Feature Structure instance to AnnotationImpl (a now deprecated version 2 non-JCas Java cover class), that will continue to work.

The backwards compatibility extends to the serialized forms, so that it should be possible to have a UIMA-AS services working with a client, where the client is a version 3 instance, but the server is still a version 2 (or vice versa).

There are 3 sets of APIs for creating and modifying Feature Structures; all are supported in V3.

Version 3 retains all 3 sets, to enable backward compatibility.

The low level CAS interface was originally provided to enable a extra-high-performance (but without compile-time type safety checks) mode. In Version 3, this mode is actually somewhat slower than the others, and no longer has any advantages.

Using the low level CAS interface also sometimes blocks one of the new features of Version 3 - namely, automatic garbage collection of unreachable Feature Structures. This is because creating a Feature Structure using the low level API creates the Java object for that Feature Structure, but returns an "int" handle to it. In order to be able to find the Feature Structure, given that int handle, an entry is made in an internal map. This map holds a reference to this Feature Structure, which prevents it from being garbage collected (until of coursse, the CAS is reset).

The normal CAS APIs allow writing Annotators where the type system is unknown at compile time; these are fully supported.

Some V2 applications make use of the Feature Structure address, using these as an integer identifier and using the low level CAS APIs to access the Feature Structure, given this integer. These applications also often use the stability of these ids across some serialization/deserializations.

Normally in V3, deserialization of CASs having these IDs occurs without preserving the IDs, and without setting up the low level CAS APIs to be able to access these using them. If an existing application depends on the low level access via the address, a special mode, called V2IdRefs, can be specified, which will support this. It comes at a cost however, which is that all new Feature Structures created (or deserialized) will be added to an internal table to enable the low level CAS getFSForRef(int) method to work. As a result, these Feature Structures are not eligible for garbage collection.

This mode is set on individual CASs via a new API; a default value may optionally be specified. Once set on a CAS, it remains until set to a different value; CAS Reset does not affect the setting, nor does checking it into / out of a CAS Pool.

When a new CAS is created, this mode is set according to two sources:

The APIs for this are part of the LowLevelCAS. The controlling APIs all return an instance of AutoClosableNoException, which can be used to reset the setting to its previous value. A recommended way of using these is with the Java try with resources construct:

try (AutoClosableNoException w = llcas.ll_enableV2IdRefs) {
  ... some operations
} // automatically restores previous value 

LowLevelCas instance APIs for enabling/disabling this mode on a particular CAS:

// set the mode    
AutoClosable ll_enableV2IdRefs()

// same, but with explicit set or reset of the mode
AutoClosableNoException ll_enableV2IdRefs(true/false)

// return true if the mode is enabled
boolean is_ll_enableV2IdRefs() 

Static LowLevelCas APIs for setting the default value for this mode for new CASs on a particular thread:

// set the default    
AutoClosableNoException LowLeveCas.ll_defaultV2IdRefs()

// same, but with explicit set or reset of the mode
AutoClosableNoException LowLeveCas.ll_defaultV2IdRefs(true/false)

// return true if the mode is enabled
boolean LowLeveCas.is_ll_defaultV2IdRefs() 

This mode modifies multiple things in the operation of UIMA V3.

Pears are supported in Version 3. If they use JCas, their JCas classes need to be migrated.

When a PEAR contains a JCas class definition different from the surrounding non-PEAR context, each Feature Structure instance within that PEAR has a lazily-created "dual" representation using the PEAR's JCas class definition. The UIMA framework things storing references to Feature Structures are modified to store the non-PEAR version of the Feature Structure, but to return (when in a particular PEAR component in the pipeline) the dual version. The intent is that this be "invisible" to the PEAR's annotators. Both of these representations share the same underlying CAS data, so modifications to one are seen in the other.

If a user builds code that holds onto Feature Structure references, outside of annotators (e.g., as a shared External Resource), and sets and references these from both outside and inside one (or more) PEARs, they should adopt a strategy of storing the non-PEAR form. To get the non-PEAR form from a Feature Structure, use the method myFeatureStructure._maybeGetBaseForPearFs().

The new Java Object support does not support multiple, different JCas class definitions for the same UIMA Type, inside and outside of the PEAR context. If this is detected, a runtime exception is thrown.

The workaround for this is to manually merge any JCas class definitions for the same class.

The formatting of various UIMA artifacts, including Feature Structures, has changed somewhat, to be more informative. This may impact situations such as testing, where the exact string representations are being compared.

A special global Java property, -Duima.v2_pretty_print_format can be set to have the toString() operation for Feature Sructures print in the V2 style.

The default logging configuration in v2 was to use Java Util Logging (the logger built into Java). For v3, the default is to use SLF4J which, in turn, picks a back-end logger, depending on what it finds in the class path.

This change was done to permit easier integration of UIMA as a library running within other frameworks.

V3 UIMA logger includes the APIs like info(..), warn(..) etc., that are part of the SLF4j APIs. In addition, these are augmented with the Java 8 style lambda arguments that were introduced in log4j-2, for more concise and efficient log message computation.

The new UIMA Logger APIs (e.g. logger.info(...), logger.warn(...)) use the SLF4j and other modern logger substitutable notation of "{}", as opposed to the style adopted by the original Java logger, of "{nnn}". All modern loggers have switched to this.

The technique for (optionally) reporting the class and method (and sometimes, line number) was changed to conform to current logger conventions - whereby the loggers themselves obtain this information from the call stack. The V2 calls which pass in the sourceClass and sourceMethod information have this information ignored, but replaced with what the loggers obtain from the stack track. In some cases, where the callers in V2 were not actually passing in the correct class/method information, this will result in a different log record.

For more details, please see the logging chapter.

Type System definitions are shared when they are equal. After type systems have been built up from type definitions, at "commit" time, a check is made to see if an identical type system already exists (same types and features). This is often the case when a UIMA application is scaling up by adding multiple pipelines, all using the same type system.

If an identical committed type system already exists, then the commit operation returns it, and the one just built is discarded. Normally, this is not an issue. However, some application code may save references to the type system object or to defined types and features. These references end up pointing to the discarded version, when the commit operation finds an already committed equal version.

Application code may code around this by re-acquiring references to the type system object, and to any type and feature objects, if the type system instance object returned from commit is not identical (==) to the one being committed. The type system commit APIs are changed to return the type system - either the one being committed, or an already existing equal committed type system. So when coding myTypesystem.commit(); if you later refer to myTypesystem, change this to myTypesystem = myTypesystem.commit();, to keep the variable myTypesystem always referring to to the committed type system.

In the interest of performance, some duplicate checks, such as whether an array index is within bounds, have been removed from UIMA when they are already being checked by the underlying Java runtime. This has affected some of the internal APIs, such as the JCas's checkArrayBounds which was removed because it was no longer being used.

The various JCas Classes implementing the built-ins for arrays have some additional interfaces added, grouping them into CommonPrimitiveArray or CommonArray. These changes are internal, and should not affect users.

Some applications may use a JCas class definition, defining for type T features f1, f2, f3 (for example), in a mode where under a single class loader (for example, in one Java application), multiple CASs are loaded and processed, where each CAS might have other versions of the type system, defining for type T a subset of the features in the JCas.

In order to make this scenario possible, v3 takes an extra step, right before type system commit time, of loading the JCas classes corresponding to the types, and then augmenting the type definitions with additional features defined in the JCas but not in the type description. After this is done, the type system is committed, and offsets are assigned to the JCas class that are constant, even when a subsequent type system is loaded that defines more features (provided that no new features are introduced).

This feature represents a trade-off between highly efficient, locked-down offsets for features, and some limited flexibility to handle a somewhat common use case where additional features exist in the JCas. The JCas loading code always checks to insure compatibility between the offsets in the JCas classes, as first set up, and any subsequent type system being used with that JCas.

This accommodation doesn't handle many possible scenarios. Some of these include situations where a supertype might subsequently add extra feature slots, or the features end up after merging to have a different ordering.

For cases where this accommodation is insufficient, the workaround is to run separate UIMA applications, each under its own class loader, for the incompatible situations.

PEARs, because they are loaded lazily after the type system has been committed, do not support this kind of augmentation of types from the Pear-specific JCas class definition.

The FSIndex interface implements Collection, so you can now write for (MyType item : myIndex) to iterate over an index.

Because it implements Collection, the FSIndex interface includes a stream() method, so you can now write myIndex.stream().any-stream-operations, which will use the items in the index as the source of the stream.

The FSIterator interface now implements the Java ListIterator Interface, and supports the methods there except for add, nextIndex, previousIndex, and set; the remove() method's meaning is changed to remove the item from all of the UIMA indexes.

The iterators over indexes no longer throw concurrent modification exceptions if the index is modified while it is being iterated over. Instead, the iterators use a lazily-created copy-on-write approach that, when some portion of the index is updated, prior to the update, copies the original state of that portion, and continues to iterate over that. While this is helpful if you are explicitly modifying the indexes in a loop, it can be especially helpful when modifying Feature Structures as you iterate, because the UIMA support for detecting and avoiding possible index corruption if you modify some feature being used by some index as a key, is automatically (under the covers) temporarily removing the Feature Structure from indexes, doing the modification, and then adding it back.

Similarly to version 2, iterator methods moveToFirst, moveToLast, and moveTo(a_positioning_Feature_Structure) "reset" the iterator to be able to "see" the current state of the indexes. This corresponds to resetting the concurrent modification detection sensing in version 2, when these methods are used.

Note that the phrase Concurrent Modification is being used here in a single threading to the indexes. UIMA does not support multi-threaded write access to the CAS; it does support multi-threaded read access to a set of CAS Views, concurrent with one thread having write access (to different views).

The remove() API for iterators is now implemented for FSIterators. Its meaning is slightly different from the normal Java meaning - it doesn't remove the item from the collection being iterated over; rather it removes the Feature Structure returned by get() from all indexes in the view.

The FSIterator methods that normally check for iterator validity have versions which skip that check. This may be a performance optimization in cases where you can guarantee the iterator is valid, for example if you have a loop which is checking hasNext() and following it with a next(), which is only executed if the hasNext() was true. The non-checking versions are suffixed with Nvc (stands for No Validity Check).

The FSIndex API has a new method, subType(type-spec), which returns an FSIndex for the same index, but specialized to elements which are a subtype of the original index. The type-spec can be either a JCas class, e.g. MyToken.class, or a UIMA type instance.

A versatile new Select framework for accessing and acting on Feature Structures selected from the CAS or from Indexes or from other collection objects is documented in a separate chapter. This API is integrated with Java 8's Stream facility.

There is a new framework that supports allowing you to add your own custom Java objects as objects transportable in the CAS. A following chapter describes this facility, and some new semi-built-in types that make use of it.

The built-in FSArray JCas class is now parameterized with the type of its elements.

UIMA Array and List types implement Iterable, so you can use them like this: for (MyType item : myArray) ....

UIMA Arrays and Lists support contains(element) and isEmpty().

UIMA Array and List types support a stream() method returning a Stream or a type-specialized sub interface of Stream for primitives (IntStream, LongStream, DoubleStream) over the objects in the collection. Omitted are stream types where boxing would occur - Arrays of Byte, Short, Float, Boolean.

The iterator() methods for IntegerList IntegerArrayList, IntegerArray, DoubleArray, and LongArray return an OfInt / OfDouble / OfLong instances. These are subtypes of Iterator with an additional methods nextInt / nextLong / nextDouble which avoid the boxing of the normal iterator.

The new select framework supports stream operations; see the "select" chapter for details.

A new set of methods on UIMA built-in lists, createNonEmptyNode() and emptyList(), creates a non-empty node of the type, or retrieves a (shared) empty node of the type. These methods are not static, and create or get the instance in the same CAS as the object instance. These methods are callable on both the empty and non-empty node instances, or on their shared super interface, for example, on NonEmptyFloatList, EmptyFloatList, and FloatList (the common super interface).

A new set of static methods on UIMA built-in lists and arrays, create(jcas, array_source) take a Java array of items, and creates a corresponding UIMA built-in list or array populated with items from the array_source.

For UIMA Lists and Arrays, the CAS and JCas has emptyXXXList/Array methods, which return a shared instance of the immutable empty object. The Cas and JCas have generic emptyArray/List, taking an argument JCas class identifying the type, e.g. FloatArray.class, StringList.class, etc.

For lists, there are some new common APIs for all list kinds.

In Java 8, classes which implement Collection can be converted to streams using the xxx.sream() method. To better integrate with Java 8, the following UIMA classes and interfaces now implement Stream or Collection:

Some APIs were reorganized. Some of the reorganizations include altering the super class and implements hierarchies, making some classes abstract, making use of Java 8's new default mechanisms to supply default implementations in interfaces, and moving methods to more common places. Users of the non-internal UIMA APIs should not be affected by these reorganizations.

As an example, version 2 had two different Java objects representing particular Feature Structures, such as "Annotation". One was used (org.apache.uima.jcas.tcas.Annotation) if the JCas was enabled; the other (org.apache.uima.cas.impl.AnnotationImpl)otherwise. In version 3, there's only one implementation; the other (AnnotationImpl) is converted to an interface. Annotation now "implements AnnotationImpl.

Several APIs require a UIMA type to be specified. For instance, the API to remove all Feature Structures of a particular type requires the type to be specified. Instead of a UIMA Type object, if there is a JCas cover class for that type, you can pass that as well, as (for example) Annotation.class.

JCasgen is modified to generate the v3 style of JCas cover classes. It no longer generates the the xxx_Type.java classes, as these are not used by UIMA Version 3.

public final static String _TypeName = "org.apache.uima.jcas.cas.Sofa";
public final static String _FeatName_sofaNum    = "sofaNum";
public final static String _FeatName_sofaID     = "sofaID";
public final static String _FeatName_mimeType   = "mimeType";
public final static String _FeatName_sofaArray  = "sofaArray";
public final static String _FeatName_sofaString = "sofaString";
public final static String _FeatName_sofaURI    = "sofaURI";

Version 3 adds generic typing to several structures, and makes use of this to enable users to unclutter their code by taking advantage of Java's type inferencing, in many cases.

Generic types are added to:

The convenience methods in the JCas have been duplicated in the CAS, e.g. getAllIndexFS.

New methods getIndexedFSs(myUimaType) and getIndexedFSs(MyJCas.class) return unmodifiable, unordered Collections of all indexed Feature Structures of the specified type and its subtypes in this CAS's view. This collection can be used in a Java extended-for loop construction. getIndexedFSs() is the same but is for all Feature Structures, regardless of type. These are methods on the CAS, JCas, FSIndexRepository interfaces, and return the Feature Structures of the specified type (including subtypes).

The TypeSystemMgr Interface has a variation of the commit method, which has a parameter that specifies the class loader to be used when loading JCas class. This should be used whenever there are user-specified JCas classes associated with the type system. If not specified, it defaults to the class loader used to load the UIMA framework.

The utility class org.apache.uima.util.FileUtils has a new method writeToFile(path, string), which efficiently writes a string using UTF-8 encoding to path.

The StringArray class has a new contains(a_string) method.

The CAS protectIndexes method returns an instance of AutoClosableNoException which is a subtype where the close method doesn't throw an exception. This allows writing the try-with-resources form without a catch block for Exception.

Sometimes Annotators may log excessively, causing problems in production settings. Although this could be controlled using logging configuration, sometimes when UIMA is embedded into other applications, you may not have easy access to modify those.

For this case, the produceAnalysisEngine's "additionalParameters" map supports a new key, AnalysisEngine.PARAM_THROTTLE_EXCESSIVE_ANNOTATOR_LOGGING. This key specifies that throttling should be applied to messages produced by annotators using loggers obtained by Annotator code using the getLogger() API.

The value specified must be an Integer, and is the number of messages allowed before logging is suppressed. This number is applied to each logging level, separately. To suppress all logging, use 0.

The Type interface has new methods subsumes(another_type), isStringOrStringSubtype(), and isStringSubtype().

The FlowController_ImplBase supports a getLogger() API, which is shorthand for getContext().getLogger().

Many error messages were changed or added, causing changes to localization classes. For coding efficiency, some of the structure of the internal error reporting calls was changed to make use of Java's variable number of arguments syntax.

The UIMA Logger implementation has been extended with both the SLF4J logger APIs and the Log4j APIs which support Java 8's Supplier Functional Interfaces.

The TypeSystem and Type object implementations implement Iterable and will iterate over all the defined types, or, for a type, all the defined Features for that type.

The various options and specifications are specified using the builder pattern. Each specification has a name, which is a Java method name, sometimes having further parameters. These methods return an instance of SelectFSs; this instance is updated by each builder method.

A common approach is to chain these methods together. When this is done, each subsequent method updates the SelectFSs instance. This means that the last method in case there are multiple method calls specifying the same specification is the one that is used.

For example,

a_cas.select().typePriority(true).typePriority(false).typePriority(true)

would configure the select to be using typePriority (described later).

Some parameters are specified as positional parameters, for example, a UIMA Type, or a starting position or shift-offset.

Feature Structures are kept in the CAS, and may be accessed using UIMA Indexes. Note that not all Feature Structures in the CAS are in the UIMA indexes; only those that the user had "added to the indexes" are. Feature Structures not in the indexes are not included when using the CAS as the source for the select framework.

Feature Structures may, additionally, be kept in FSArrays, FSLists, and many additional collection-style objects that implement SelectViaCopyToArray interface. This interface is implemented by the new semi-built-in types FSArrayList, FSHashSet and FSLinkedHashSet; user-defined JCas classes for user types may also choose to implement this. All of these sources may be used with select.

Figure 4.2. select method with type

For CAS sources, if Views are being used, there is a separate set of indexes per CAS view. When there are multiple views, only one view's set of indexed Feature Structures is accessed - the view implied by the CAS being used. Note that there is a way to specify aggregating over all views; see allViews described later.

For CAS sources, users may specify all Feature Structures in a view, or restrict this in two ways:

It is possible to specify both of these, using the form myIndex.select(myType); in that case the type must be the type or a subtype of the index's top most type.

If no index is specified, the default is

Note that the non-CAS collection sources (e.g. the FSArray and FSList sources are considered ordered, but non-sorted, and therefore cannot be used for an operations which require a sorted order.

There are 4 kinds of sources of Feature Structures supported:

UIMA Collection sources have somewhat limited configurability, because they are considered non-sorted, and therefore cannot be used for an operations which require a sorted order, such as the various bounding selections (e.g. coveredBy) or positioning operations (e.g. startAt).

Each of these sources has a new API method, select(...), which initiates the select specification. The select method can take an optional parameter, specifying the UIMA type to return. If supplied, the type must must be the type or subtype of the index (if one is specified or implied); it serves to further restrict the types selected beyond whatever the index (if specified) has as its top-most type.

// this works
// the generic type for Token is passed as an argument to select
FSIterator<Token> token_it = cas.select(Token.class).fsIterator();

FSIndex<Token> token_index = ... ; // generically typed

// this next fails because the
// Token generic type from the index variable being assigned
// doesn't get passed to the select().
FSIterator<Token> token_iterator = token_index.select().fsIterator();

// You can overcome this in two ways:
// pass in the type as an argument to select
// using the JCas cover type.  
FSIterator<Token> token_iterator = 
    token_index.select(Token.class).fsIterator();

// You can also use the static form of select
// to avoid repeating the type information
FSIterator<Token> token_iterator = 
    SelectFSs.select(token_index).fsIterator();

// Finally, you can also explicitly set the generic type 
// that select() should use, like a special kind of type cast, like this:
FSIterator<Token> token_iterator =
    token_index.<Token>select().fsIterator();

FSIterator<Token> token_iterator = 
    annotation_index.select(Token.class).fsIterator();

There are four sets of sub-selection and ordering specifications, grouped by what they apply to:

With some exceptions, configuration items to the left also apply to items on the right.

When the same configuration item is specified multiple times, the last one specified is the one that is used.

Figure 4.3. Selection and Ordering

After the sources and selection and ordering options have been specified, one terminal form action may be specified. This can be an getting an iterator, array or list, or a single value with various extra checks, or a Java stream. Specifying any stream operation (except limit) converts the object to a stream; from that point on, any stream operation may be used.

Figure 4.4. Select Terminal Form Actions

Here's a tutorial example on how to design and implement your own special Java object. For this example, we'll imagine we need to implement a map from FeatureStructures to FeatureStructures.

Figure 5.1. Creating a custom Java CAS-stored Object

Step 1 is deciding on the Java Object implementation to use. We can define a special class, but in this case, we'll just use the ordinary Java HashMap<TOP, TOP> for this.

Step 2 is deciding on the CAS Feature Structure representation of this. For this example, let's design this to represent the serialized form of the hashmap as 2 FSArrays, one for the keys, and one for the values. We could also use just one array and intermingle the keys and values. It's up to the designer of this new JCas class to decide how to do this.

Step 3 is defining the UIMA Type for this. Let's call it FS2FSmap. It will have 2 Features: an FSArray for the keys, and another FSArray for the values. Let's name those features "keys" and "values". Notice that there's no mention of the Java object in the UIMA Type definition.

Step 4 is to run JCasGen on this class to get an initial version of the class. Of course, it will be missing the Java HashMap, but we'll add that in the next step.

For custom types which hold collections of Feature Structures, you can have those participate in the Select framework, by implementing the optional Interface SelectViaCopyToArray.

For more examples, please see the implementations of the semi-built-in classes described in the following section.

Some additional semi-built-in UIMA types are defined in Version 3 using this new mechanism. They work fully in Java, and are serialized or transported to non-Java frameworks as ordinary CAS objects.

Semi-built-in means that the JCas cover classes for these are defined as part of the core Java classes, but the types themselves are not "built-in". They may be added to any tyupe system by importing them by name using the import statement:

<import name="org.apache.uima.semibuiltins"/>

If you have a Java project whose classpath includes uimaj-core, and you run the Component Descriptor Editor Eclipse plugin tool on a descriptor which includes a type system, you can configure this import by selecting the Add on the Import type system subpanel, and import by name, and selecting org.apache.uima.semibuiltins. (Note: this will not show up if your project doesn't include uimaj-core on its build path.)

While it is possible to have a single custom JCas class implement multiple Java Objects, this is typically not a good design practice, as it reduces reusability. It is usually better to implement one custom Java object per JCas class, with an associated UIMA type, and have that as the reusable entity.

There are 2 logging level schemes, and there is a mapping between them. Either of them may be used when using the UIMA logger. One of the schemes is the original UIMA v2 level set, which is the same as the built-in-to-java logger levels. The other is the scheme adopted by SLF4J and many of its back ends.

Log statements are "filtered" according to the logging configuration, by Level, and sometimes by additional indicators, such as Markers. Levels work in a hierarchy. A given level of filtering passes that level and all higher levels. Some levels have two names, due to the way the different logger back-ends name things. Most levels are also used as method names on the logger, to indicate logging for that level. For example, you could say aLogger.log(Level.INFO, message) but you can also say aLogger.info(message)). The level ordering, highest to lowest, and the associated method names are as follows:

The CONFIG and FINEST levels are merged with other levels, but distinguished by having Markers. If the filtering is configured to pass CONFIG level, then it will pass the higher levels (i.e., the INFO/WARN/ERROR or their alternative names WARNING/SEVERE) levels as well.

Context data is kept in SLF4j MDC maps; there is a separate map per thread. This information is set before calling Annotator's process or initialize methods. The following table lists the keys and the values recorded in the contexts; these can be retrieved by the logging layouts and included in log messages.

Because the keys for context data are global, the ones UIMA uses internally are prefixed with "uima_".

By default, UIMA is configured so that the UIMA logger is hooked up to the SLF4j facade, which may or may not have a logging back-end. If it doesn't, then any use of the UIMA logger will produce one warning message stating that SLF4j has no back-end logger configured, and so no logging will be done.

When UIMA is run as an embedded library in other applications, slf4j will use those other application's logging frameworks.

Each logging back-end has its own way of being configured; please consult the proper back-end documentation for details.

For backwards compatibility, the binary distribution of UIMA includes the slf4j back-end which hooks to the standard built-in Java logging framework, so out-of-the-box, UIMA should be configured and log by default as V2 did.

Although UIMA V3 is designed to be backwards compatible with UIMA V2, there are some migration steps needed. These fall into two broad use cases:

UIMA V3 is designed to be binary compatible with existing UIMA V2 pipelines, so compiled and/or JAR-ed up classes representing a V2 pipeline should run with UIMA v3, with three changes:

Some adjustments may need to be made to logging setup, typically by including additional Jars (provided in the UIMA Binary distribution) in your application's classpath. If you are using the standard UIMA Launch scripts, this is already done. For custom application setups, insure that the classpath includes the (now) required jar "slf4j-api-xxxx.jar" (replace xxxx with the version). If you were using the standard UIMA based logging, to get the similar behavior, include the slf4j-jdk14-xxxx.jar; this enables the standard Java Utility Logging facility.

Some Maven projects use the JCasGen maven plugin; these projects' JCasGen maven plugin, if switched to UIMA V3, automatically generate the V3 versions. For proper operation, please run maven clean install; the clean operation ought to remove the previously generated JCas class, including the UIMA V2 xxx_Type classes. These are no longer used, and won't compile in V3.

You can use any of the methods of invoking JCasGen to generate the new V3 versions. If using the Eclipse plugins (i.e., pushing the JCasGen) button in the configuration editor, etc.), the V3 version of the plugin must be the one installed into Eclipse.

If you have the source or class files, you can also migrate those using the migration tool described in this section. This approach is useful when you've customized the JCas class, and wish to preserve those customizations, while converting the v2 style to the v3 style.

If you have customized JCasGen classes, these can be migrated by running the migration tool, which is available as a stand-alone command line tool (runV3migrateJCas.sh or ...bat), or as Eclipse launch configurations.

This tool can migrate either sets of

Usually, if you have the source code it is best to migrate the sources. Otherwise, you can migrate the compiled classes.

When migrating source files, you specify one or more "roots" - places in a file directory, and the tool scans those directories recursively (including inside Jars and PEARs), looking for JCas source files. When it finds one, it copies it to the output spot and migrates it. The output is arranged in parallel directories (before and after migration), for easy side-by-side comparing in a tool such as Eclipse file compare.

After checking the migration results, including comparing the files, you replace the original source with the migrated versions. Also, the original V2 source would contain a source file for each JCas class ending in "_Type"; these are not used in version 3 and should be deleted.

You may also migrate class files; this can be used when the source files are not available. This option has a decompilation step, to produce the source to be migrated and requires a classpath (passed as the migrationClasspath parameter); this classpath is used to resolve symbols during the decompilation, and should be the classpath used when running those classes. For class files, the migration tool attempts to compile the results and, for Jars and PEARs, to update those migrated classes in a copy of the original packaging (meaning, within Jars or PEARs):

When scanning directories from source or class roots, if a Jar or a PEAR is encountered, it is recursively scanned.

When migrating from compiled classes:

The results of the migration include the migrated files, a set of logs, and for classesRoots: the compiled classes, and repackaging of them into copies of original Jars and/or PEARs. The migration operation is summarized in the console output, detailing anything that might need inspection to verify the migration was done correctly.

To complete the migration, fix any reported issues that need fixing, and then update your UIMA application to use these classes/Jars/PEARs in place of the version 2 ones.

The actual migration step is a source-to-source transformation, done using a parse of the source files. The parts in the source which are version 2 specific are replaced with the equivalent version 3 code. Only those parts which need updating are modified; other code and comments which are part of the source file are left unchanged. This is intended to preserve any user customization that may have been done.

7.3.2. Understanding the reports

The output directory contains a logs directory with additional information. A summary is also written to System.out.

Each file translated has both a v2 source and a v3 source. When the input is ".class" files, the v2 source is the result of the decompilation step, prior to any migration.

The process of scanning directories to find JCas class to migrate may come across multiple instances of the same class. There are two subcases:

• The instances are the same.

• The instances are different (two non-identical definitions for the same class). Sometimes these arise when migrating from compiled classes, where the compilation was done by different versions of the Java compiler, and the resulting decompilations are logically equal but have some fields or methods in a different order.

This diagram illustrates some of the potentials for identical and non-identical duplicate definitions for the same classname, that the tool may encounter. The blue boxes represent ordinary file directories or Jars, and the other boxes with labels Cn1 and Cn2 represent the definitions for a classes named Cn1 and Cn2; the different colors represent non-identical definitions, as an example. Note that a definition for a class might appear sometimes not within a Jar (or a PEAR, not shown here), as well as with that.

The migration tool allows for all of these variants. It will migrate all versions, and will (when migrating from compiled Jars and PEARs) compile and reassemble these.

The output directories prefix the package/classname holding the source code with a prefix of "a0", "a1", etc. The "a" stands for alternative, and the 0 is for the first alternative, and the 1, 2, ... are for other non-equal alternatives.

When the migration is run from compiled classes, then, if possible, the resulting migrated classes are recompiled and if from Jars or PEARs, reassembled into copies of those artifacts. The compilation for the same classname, with the same sourcecode, could be different for different containers because each compilation is done with that container's classpath (e.g. Jar or Pear) and with respect to the compilation units of that container.

Because of this, the compiled results for a given source instance, are done separately, and kept in output directories, indexed additionally by the container number, as "c0", "c1", ... . A list of all container numbers and the migrated classes within those containers, is printed out to enable correlating these by hand when necessary.

The overall directory output directory tree looks like:

Directory structure, starting at -outputDirectory
   converted/
     v2/
       a0/pkg/name.../Classname.java   
                           /Classname2.java etc.
       a1/pkg/name.../Classname.java  if there are multiple
                                              different versions 
       ...
     v3/
       a0/pkg/name.../Classname.java
                     /Classname2.java etc.
       a1/pkg/name.../Classname.java  if there are multiple
                                              different versions
       ...
       
     v3-classes/   for Jars and PEARs, the compiled class
       // xyz is the path in the container to the
       //       start of the pkg/name.../Classname.class
       // the "a0", "a1", ... is extra but serves to 
       //      identify which alternative of the source
       23/a0/xyz/pkg/name.../Classname.class
       33/a0/xyz/pkg/name.../Classname.class
       42/a0/xyz/pkg/name.../Classname.class
         ...    

     pears/
       // xyz_updated_pear_copy is the path
       //    relative to the container, of the PEAR
       33/xyz_updated_pear_copy.pear  
         ...
     
     jars/
       // xyz_updated_jar_copy is the path
       //    relative to the container, of the Jar
       42/xyz_updated_jar_copy.jar
         ...
       
   not-converted/
   
   logs/
     processed.txt
     failed.txt
     skippedBuiltins.txt
     nonJCasFiles.txt
     workaroundDir.txt
     deletedCheckModified.txt
     manualInspection.txt
     pearFileUpdates.txt
     jarFileUpdates.txt
     
     ...

The converted subtree holds all the sources and migrated versions that were successfully migrated. The not-converted subtree hold the sources that failed in some way the migration. The logs contain many kinds of entries for different issues encountered:

processed.txt

List of successfully processed classes

failed.txt

List of classes that failed to migrate

skippedBuiltins.txt

List of classes representing built-ins that were skipped. These need manual inspection to see how to merge with new v3 built-ins.

NonJCasFiles.txt

List of files that were thought to be JCas classes but upon further analysis appear to not be. These need manual inspection to confirm.

deletedCheckModified.txt

List of class where a version 2 if statement doing the "featOkTst" was apparently modified. In the migrated code, this statement was deleted, perhaps incorrectly. These need manual inspection to confirm.

manualInspection.txt

List of files where the migration found a get or set method, where the version 2 code was accessing a casFeatCode with the feature name not matching. These need manual inspection.

jarsFileUpdates.txt

List of Jar files and classes which were replace in them.

pearsFileUpdates.txt

List of Pear files and classes which were replace in them.

cd $UIMA_HOME
      
bin/runV3migrateJCas.sh

  -migrateClasspath /home/me/myproj/xyz.jar:$UIMA_HOME/lib/uima-core.jar
    
  -classesRoots /home/me/myproj/xyz.jar:/home/me/myproj/target/classes
  
  -outputDirectory /temp/migratejcas  

Startup an Eclipse workspace containing the project 
with JCas source files to be migrated.
      
Select the Java project with the JCas sources to be migrated.

Eclipse -> menu -> Run -> Run configurations
  Use the search box to find 
  "UIMA run V3 migrate JCas from sources" launcher.

Projects may have tests which write to the UIMA log. Because V3 switched to SLF4J as the default logger, unless SLF4J can find an adapter to some back-end logger, it will issue a message and substitute a "NO-OP" back-end logger. If your test cases depend on having the V2 default logger (which is the one built into Java), you need to add a "test" dependency that specifies the SLF4J-to-JDK14 adapter to your POM. Here's the xml for that:

<dependency>
  <groupId>org.slf4j</groupId>
  <artifactId>slf4j-jdk14</artifactId>
  <version>1.7.24</version>  <!-- or some version you need -->
  <scope>test</scope>
</dependency>

JCas classes defining Java implementations for UIMA Types may be defined within a PEAR. These are loaded using the isolating Classloader, just like all the other PEAR resources. As a result, this may cause some issues if the same JCas class is also defined outside the PEAR boundary, and loaded with the normal UIMA classloader. The result of having the same JCas class both on the PEAR classloader and outside that classloader will be that Java will have both classes loaded, and code within the PEAR will be linked with one of them, and code outside the PEAR will be linked with the other.

Sometimes, this is exactly what you might want. For example, you might have in the pear, a special JCas definition of a UIMA type "Token" which the PEAR uses, while you might have another JCas definition for that same UIMA type outside of the PEAR. Note that UIMA will always merge Type definitions from inside and outside of PEARs, when it sets up a pipeline - it merges all type definitions found for the whole pipeline.

A consequence of having two loaded class definitions in two contexts for the same UIMA type means that the classes have the same names, but are different (because of different loading classloaders), and assigning one to the other in Java will produce a ClassCast exception.

Othertimes, you may not want different classes. For instance, the class definitions might be identical, and you want to create some "Token" annotations within the PEAR, and have them used by JCas references outside of the PEAR.

In this case, the simplest thing to do is to install the PEAR, but then update its classpath so it no longer includes the JCas classes that came with the PEAR. When classes are not found with the special PEAR class loader, that loader delegates to its parent, which is the normal UIMA class loader. This action will cause the PEAR to use the identically same JCas class within the PEAR as is used outside of the PEAR, and no Class Cast Exception issues will arise. This is the most efficient way to run with PEARs that use JCas classes where you want to share results inside and outside of PEARs.

Version 3 has special support for the case where there are different definitions of JCas classes for the same UIMA type, inside and outside the PEAR. It does this using what are called PEAR Trampolines. When there are multiple JCas definitions, the one defined outside of the PEAR is the one stored internally in UIMA's indexes and types that have references to Feature Structures. Accessing the Feature Structures checks (by asking the CAS) to see if its in a particular PEAR context (there may be several in one pipeline), and if so, a trampoline instance of the Feature Structure is created / used / accessed. The trampoline instance shares internally the CAS data with the base instance, but is a separate instance of the PEAR's JCas class definition. This allows seamless access both inside and outside of the PEAR context to the particular JCas class definition needed.

Custom Java Objects may store references to Feature Structures. If it is desired to create these inside a PEAR, and yet have the references work outside a PEAR, the implementor of these must insure that the actual stored JCas class for a Feature Structure is the base version, not the PEAR version, and also insure that any references are properly converted (while within a PEAR context).

Refer to the implementation of FSHashSet and FSArrayList to see what needs to be done to make these "Pear aware".

This table describes the various JVM defined properties; specify these on the Java command line using -Dxxxxxx, where the xxxxxx is one of the properties starting with uima. from the table below.