opennlp.tools.tokenize
Class DictionaryDetokenizer

java.lang.Object
  opennlp.tools.tokenize.DictionaryDetokenizer

All Implemented Interfaces:

Detokenizer

public class DictionaryDetokenizer
extends Object
implements Detokenizer

A rule based detokenizer. Simple rules which indicate in which direction a token should be moved are looked up in a DetokenizationDictionary object.

See Also:

Detokenizer, DetokenizationDictionary

Nested Class Summary

Nested classes/interfaces inherited from interface opennlp.tools.tokenize.Detokenizer

Detokenizer.DetokenizationOperation

Constructor Summary

DictionaryDetokenizer(DetokenizationDictionary dict)

Method Summary

Detokenizer.DetokenizationOperation[]

detokenize(String[] tokens) Detokenize the input tokens.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DictionaryDetokenizer

public DictionaryDetokenizer(DetokenizationDictionary dict)

Method Detail

detokenize

public Detokenizer.DetokenizationOperation[] detokenize(String[] tokens)

Description copied from interface: Detokenizer

Detokenize the input tokens.

Specified by:

detokenize in interface Detokenizer

Parameters:

tokens - the tokens to detokenize.

Returns:

the merge operations to detokenize the input tokens.