Revision 150990

Added support for analyzers that produce overlapping tokens
PR:
Obtained from:
Submitted by:	Mark Harwod
Reviewed by:	
CVS: ----------------------------------------------------------------------
CVS: PR:
CVS:   If this change addresses a PR in the problem report tracking
CVS:   database, then enter the PR number(s) here.
CVS: Obtained from:
CVS:   If this change has been taken from another system, such as NCSA,
CVS:   then name the system in this line, otherwise delete it.
CVS: Submitted by:
CVS:   If this code has been contributed to Apache by someone else; i.e.,
CVS:   they sent us a patch or a new module, then include their name/email
CVS:   address here. If this is your work then delete this line.
CVS: Reviewed by:
CVS:   If we are doing pre-commit code reviews and someone else has
CVS:   reviewed your changes, include their name(s) here.
CVS:   If you have not had it reviewed then delete this line.