Tokenizer implementation for common texts, using whitespaces as word seperators.
Source for this file: /Document/src/document/pdf/tokenizer/default.php
ezcDocumentPdfTokenizer | --ezcDocumentPdfDefaultTokenizer
Version: | //autogen// |
From ezcDocumentPdfTokenizer: | |
---|---|
ezcDocumentPdfTokenizer::FORCED
|
Constant indicating a forced breaking point without rendering a space character. |
ezcDocumentPdfTokenizer::SPACE
|
Constant indicating a breaking point, including a rendered space. |
ezcDocumentPdfTokenizer::WRAP
|
Constant indicating a possible breaking point without rendering a space character. |
public array |
tokenize(
$string
)
Split string into words |
From ezcDocumentPdfTokenizer | |
---|---|
public abstract array |
ezcDocumentPdfTokenizer::tokenize()
Split string into words |
Split string into words
This function takes a string and splits it into words. There are different mechanisms which indicate possible splitting points in the resulting word stream:
Non breaking spaces should not be splitted into multiple words, so there will be no break applied.
Name | Type | Description |
---|---|---|
$string |
string |
Method | Description |
---|---|
ezcDocumentPdfTokenizer::tokenize() |
Split string into words |