Implementation and Low-Level API for the CAS Interfaces.

Internal APIs. Use these APIs at your own risk. APIs in this package are subject to change without notice, even in minor releases. Use of this package is not supported. If you think you have found a bug in this package, please try to reproduce it with the officially supported APIs before reporting it.

Internals documentation

NOTE: This documentation is plain HTML, generated from a WYSIWIG editor "tinymce".   The way to work on this:  after setting up a small web page with the tinymce (running from a local file), use the Tools - source code to cut/paste between this file's source and that editor.

Java Cover Objects

It is possible to run UIMA without creating Java cover objects for FSs.  However, for convenience, many of the APIs return Java objects that provide, in turn, various APIs for accessing features, updating in indexes, etc.

There are two kinds of Java cover objects:

• Basic - this provides a generic API that works for all FSs.
• JCas - this provides a custom Java Class for a type that gives get/set style accessors for the Features.

Both of these inherit from the FeatureStructure Interface.  Use of the JCas is optional; if the JCas cover classes are available (in the class path), they are used. 

UIMA Indexes

Indexes are defined for a pipeline, and are kept as part of the general CAS definition.

Each CAS View has its own instantiation of the defined indexes (there's one definition for all views), and as a result, a particular FS may be added-to-indexes and indexed in some views, and not in others.

There are 3 kinds of indexes: Sorted, Set, and Bag.  The basic object type for an index is FSLeafIndexImpl. This has 3 subtypes, one for each of the index types:

• FSBagIndex
• FSIntArrayIndex (for Sorted)
• FSRBTSetIndex (for Set)

The leaf index is just for one type (and doesn't include entries for any subtypes).

Indexes are connected to specific index definitions; these definitions include a type which is the top type for elements of this index. The index definition logically includes that type and all of its subtypes.

An additional data struction, the IndexIteratorCachePair, is associated with each index definition.  It holds references to the subtype FSLeafIndexImpls for all subtypes of an index; this list is created lazily, only when an iterator is created over this index at a particular type level (which can be the type the index was defined for, or any subtype).  This lazy aspect is important, because UIMA is often used in cases where there's a giant type system, with lots of subtypes, only a few of which are used in a particular pipeline instance.

There are two tasks that indexes accomplish:

• updating the index with adds and removes of FSs.  This update operation is optimized by
  • keeping each type indexed separately, so only that data structure for the particular type need be updated (this design choice has a cost in iteration, though)
  • treating more common use cases efficiently - the main one being that of adding something "to the end" of the items in the index.
• iterating over an index for a type and its subtypes. 
  • For indexes having no subtypes, this is done by iterating over the FSLeafIndexImpl for that index and type. 
  • For indexing with subtypes, this is done by creating individual iterators for the type and all of its subtypes, each iterating over the FSLeafIndexImpl for that type.  These iterators are then logically combined into one iterator.

Iterators

There are two main kinds of iterators:

• Iterators over UIMA Indexes
• Iterators over other UIMA objects, such as Views, or internal structures.

Iterators over UIMA indexes

There are two main kinds of iterators over UIMA indexes:

• those returning int values representing the location of the FS in the heap.
• those returning Java cover objects representing the FS.  This is typically implemented as a wrapper around the one returning the int value.  (It can't be a subclass, overriding the get() method, because you can't change the return type when overriding).

The basic int iterators are implemented with instances of the classes:

• FSIntIteratorImplBase<T extends FeatureStructure> - the common superclass
  • IntIterator4bag<T extends FeatureStructure>
  • IntIterator4set<T extends FeatureStructure>
  • IntIterator4sorted<T extends FeatureStructure>

All of these implement an iterator over the corresponding FSLeafIndexImpl for one type.

The class PointerIterator in FSIndexRepositoryImpl is an int iterator that combines iterators for type and their subtypes, into one aggregated iterator, taking into account the comparator sorting order among the various iterators. So, for instance, if you do a moveTo operation, it does a move to on all the individual iterators, and then figures out which of those is the left-most one in the comparator ordering.

PointerIteratorUnordered is a variant that also combines iterators for a type and its subtypes, but doesn't try to keep these in order.  It is designed to be used when iterating through all instances of a type and its subtypes, in an arbitrary order, such as what the method getAllIndexedFS(type) does.

SnapshotPointerIterator is a variant which creates a snapshot when the iterator is created, and then (ignoring any subsequent index updates) iterates over that.  This iterator won't throw ConcurrentModificationException.

The basic impls of IntIterator4bag/set/sorted are created by calls to pointerIterator; this method is implemented in each of the IntIterator4bag/set/sorted classes.

The 2nd argument passed is a ref to this FSIndexRepositoryImpl's int[] used to detect concurrent modification exception.  If null is passed, then no testing for this is done.  This kind of call happens with the use of the refIterator() methods, which are used internally when it is known that the iteration will not be modifying the indexes in any way.

Iterators which return Java cover object:

• FSIteratorImplBase<T> abstract class implements FSIterator
  • FSIteratorWrapper<T> - this is the standard wrapper around the above int iterators.
  • FSIteratorAggregate<T> - created with a list of iterators, operates like PointerIteratorUnordered, except the underlying iterators are ones returning Java cover objects
  • FSIteratorFlat<T> - iterates over a corresponding "flattened" iterator of a type and its subtypes (an optimization).
  • FSIteratorWrapperDoublecheck<T> - only used for debugging
  • FilteredIterator<T> - wraps an FSIterator, and applies a FSMatchConstraint
  • Subiterator<T> - created from an AnnotationIndex; wraps a plain FSIterator
  • UnambiguousIteratorImpl<T> - created from an AnnotationIndex; wraps a plain FSIterator

Plain FSIterators are created from index instances via the iterator() method; corresponding int iterators are created from low-level indexes via ll_iterator(). This method picks the appropriate underlying iterator based on

• the index kind (BAG/SET/SORTED)
• whether or not the index has subtypes
• index extended function flags (snapshot)
• whether or not the index has a currently valid "flat" version (for SORTED indexes)

Iterator Interfaces

There are several overlapping interfaces (probably due to historical reasons) for these iterators.

First, interfaces for iterators returning ints:

• IntPointerIterator - returns an int, implements isValid, inc/dec, moveToFirst, moveToLast, get, copy
• ComparableIntPointerIterator<T> - combines IntPointerIterater and the standard Java Comparable. This is the main interface for a full-function instance iterators over a FSLeafIndexImpl.
  • The comparable is to allow the iterators to be compared (which means to compare the values of the FSs the iterators are positioned at), which is only used when ordering multiple iterators in use when iterating over a type and its subtypes (one iterator for each).
• LowLevelIterator - like IntPointerIterator, has moveToFirst/Last, and isValid; difference: has: ll_get, moveToNext/Previous, and moveTo(fs), also ll_indexSize and ll_getIndex.

Next, interfaces for iterators returning Java cover objects:

• FSIterator - extends the standard Java interface: hasNext, next, remove, with isValid, get, moveToNext/Previous/First/Last, moveTo(fs), copy.