## Apache Zeta Components Manual :: Docs For Class ezcDocumentPdfLiteralTokenizer

### Document::ezcDocumentPdfLiteralTokenizer

### Class ezcDocumentPdfLiteralTokenizer

Tokenizer implementation for literal blocks, preserving whitespaces.

Source for this file: /Document/src/document/pdf/tokenizer/literal.php

    ezcDocumentPdfTokenizer
       |
       --ezcDocumentPdfLiteralTokenizer

Version: | //autogen//

### Inherited Constants

From ezcDocumentPdfTokenizer:
`ezcDocumentPdfTokenizer::FORCED` | Constant indicating a forced breaking point without rendering a space character.
`ezcDocumentPdfTokenizer::SPACE` | Constant indicating a breaking point, including a rendered space.
`ezcDocumentPdfTokenizer::WRAP` | Constant indicating a possible breaking point without rendering a space character.

### Method Summary

protected string | `convertTabs( $string , [ $tabwidth = 8] , [ $offset = 0] )` Convert tabs to spaces.
public array | `tokenize( $string )` Split string into words.

### Inherited Methods

From ezcDocumentPdfTokenizer
public abstract array | `ezcDocumentPdfTokenizer::tokenize()` Split string into words

### Methods

#### convertTabs

string convertTabs( string $string , [int $tabwidth = 8] , [int $offset = 0] )

Convert tabs to spaces.

Convert all tabs to spaces, using a 8 spaces for a tab.

##### Parameters:

Name | Type | Description
--- | --- | ---
`$string` | string |
`$tabwidth` | int |
`$offset` | int |

#### tokenize

array tokenize( string $string )

Split string into words.

This function takes a string and splits it into words. There are different mechanisms which indicate possible splitting points in the resulting word stream:

- self:SPACE: The renderer might render a space
- self:WRAP: The renderer might wrap the line at this position, but will not render spaces.

A possible splitting of an english sentence might look like:

1. array(
2. 'Hello',
3. self:SPACE ,
4. 'world!',
5. );

Non breaking spaces should not be splitted into multiple words, so there will be no break applied.

##### Parameters:

Name | Type | Description
--- | --- | ---
`$string` | string |

##### Redefinition of:

Method | Description
--- | ---
`ezcDocumentPdfTokenizer::tokenize()` | Split string into words

Documentation generated by phpDocumentor 1.4.3