Module dparse.lexer
Functions
Name | Description |
---|---|
byToken(range)
|
Creates a token range from the given source code. Creates a default lexer configuration and a GC-managed string cache. |
byToken(range, cache)
|
Creates a token range from the given source code. Uses the given string cache. |
byToken(range, config, cache)
|
Creates a token range from the given source code. Uses the provided lexer configuration and string cache. |
getTokensForParser(sourceCode, config, cache)
|
|
isBasicType(type)
|
|
isIntegerLiteral(type)
|
|
isKeyword(type)
|
|
isLiteral(type)
|
|
isNumberLiteral(type)
|
|
isOperator(type)
|
|
isProtection(type)
|
|
isSpecialToken(type)
|
|
isStringLiteral(type)
|
|
optimalBucketCount(size)
|
Helper function used to avoid too much allocations while lexing. |
rangeMatch()
|
|
rangeMatch()
|
|
skip(, , )
|
Skips between 0 and 16 bytes that match (or do not match) one of the given chars. |
skip(, , )
|
Skips between 0 and 16 bytes that match (or do not match) one of the given chars. |
Structs
Name | Description |
---|---|
DLexer
|
The D lexer struct. |
LexerConfig
|
Lexer configuration struct |
StringBehavior
|
Configure string lexing behavior |
StringCache
|
The string cache is used for string interning. |
Enums
Name | Description |
---|---|
CommentBehavior
|
|
WhitespaceBehavior
|
Configure whitespace handling |
Templates
Name | Description |
---|---|
TokenTriviaFields
|
Manifest constants
Name | Type | Description |
---|---|---|
ByteCombine
|
||
ByteCombine
|
Aliases
Name | Type | Description |
---|---|---|
BasicTypes
|
|
Basic type token types. |
IdType
|
ubyte
|
Token ID type for the D lexer. |
IntegerLiterals
|
|
Number literal token types. |
Keywords
|
|
Keyword token types. |
Literals
|
|
|
NumberLiterals
|
|
Number literal token types. |
Operators
|
|
Operator token types. |
Protections
|
|
Protection token types. |
SpecialTokens
|
|
|
str
|
|
Function used for converting an IdType to a string. |
StringLiterals
|
|
String literal token types |
tok
|
TokenId!(IdType,operators,dynamicTokens,keywords,token)
|
Template used to refer to D token types. |
Token
|
TokenStructure!(ubyte,"import dparse.lexer:TokenTriviaFields; mixin TokenTriviaFields;")
|
The token type in the D lexer |