Module dparse.lexer

Functions

Name	Description
byToken(range)	Creates a token range from the given source code. Creates a default lexer configuration and a GC-managed string cache.
byToken(range, cache)	Creates a token range from the given source code. Uses the given string cache.
byToken(range, config, cache)	Creates a token range from the given source code. Uses the provided lexer configuration and string cache.
getTokensForParser(sourceCode, config, cache)
isBasicType(type)
isIntegerLiteral(type)
isKeyword(type)
isLiteral(type)
isNumberLiteral(type)
isOperator(type)
isProtection(type)
isSpecialToken(type)
isStringLiteral(type)
optimalBucketCount(size)	Helper function used to avoid too much allocations while lexing.
rangeMatch()
rangeMatch()
skip(, , )	Skips between 0 and 16 bytes that match (or do not match) one of the given chars.
skip(, , )	Skips between 0 and 16 bytes that match (or do not match) one of the given chars.
unDecorateComment(comment, outputRange)	Removes "decoration" such as leading whitespace, leading + and * characters, and places the result into the given output range

Structs

Name	Description
DLexer	The D lexer struct.
LexerConfig	Lexer configuration struct
StringCache	The string cache is used for string interning.

Enums

Name	Description
CommentBehavior
StringBehavior	Configure string lexing behavior
WhitespaceBehavior	Configure whitespace handling

Manifest constants

Name	Type	Description
ByteCombine
ByteCombine

Aliases

Name	Type	Description
BasicTypes		Basic type token types.
IdType	ubyte	Token ID type for the D lexer.
IntegerLiterals		Number literal token types.
Keywords		Keyword token types.
Literals
NumberLiterals		Number literal token types.
Operators		Operator token types.
Protections		Protection token types.
SpecialTokens
str		Function used for converting an IdType to a string.
StringLiterals		String literal token types
tok	TokenId!(IdType,operators,dynamicTokens,keywords,token)	Template used to refer to D token types.
Token	TokenStructure!(ubyte,"\x0a string comment;\x0a string trailingComment;\x0a\x0a int opCmp(size_t i) const pure nothrow @safe {\x0a if (index < i) return -1;\x0a if (index > i) return 1;\x0a return 0;\x0a }\x0a\x0a int opCmp(ref const typeof(this) other) const pure nothrow @safe {\x0a return opCmp(other.index);\x0a }\x0a")	The token type in the D lexer