Function getTokensForParser


						
				Token[] getTokensForParser(R)
				(
				

				  R sourceCode,
				

				  LexerConfig config,
				

				  StringCache* cache
				

				)
				

				if (is(Unqual!(ElementEncodingType!R) : ubyte) && isDynamicArray!R);

Returns

an array of tokens lexed from the given source code to the output range. All whitespace, comment and specialTokenSequence tokens (trivia) are attached to the token nearest to them.

Trivia is put on the last token as trailingTrivia if it is on the same line as the trivia, otherwise it will be attached to the next token in the leadingTrivia until there is the EOF, where it will be attached as trailingTrivia again.