Class Tokenizer

simple tokenizer for DlangUI ML

Inherits from

Object (base class)

Constructors

Name	Description
`this`

Fields

Name	Type	Description
`_filename`	`string`
`_len`	`int`
`_line`	`ushort`
`_lineText`	`dchar[]`
`_pos`	`ushort`
`_prevChar`	`dchar`
`_token`	`Token`
`_lines`	`LineStream`

Methods

Name	Description
`emitError`
`emitError`
`emitUnknownObjectError`
`emitUnknownPropertyError`
`getContextSource`
`isAlpha`
`isAlphaNum`
`isNum`
`nextToken`	get next token
`nextChar`
`parseEof`
`parseEol`
`parseError`
`parseFloating`
`parseHex`
`parseIdent`
`parseMultiLineComment`
`parseNumber`
`parseOp`
`parseSingleLineComment`
`parseString`
`parseWhiteSpace`
`peekChar`
`peekNextChar`
`setTokenStart`
`skipChar`

Authors

Vadim Lopatin, coolreader.org@gmail.com

Copyright

Vadim Lopatin, 2015

License

Boost License 1.0