lezer-lr/com.monkopedia.kodemirror.lezer.lr/InputStream

InputStream

Tokenizers interact with the input through this interface. It presents the input as a stream of characters, tracking lookahead and hiding the complexity of ranges from tokenizer code.

Properties

var chunk: String

var chunkOff: Int

var chunkPos: Int

var end: Int

val input: Input

var next: Int

The character code of the next code unit in the input, or -1 at EOF.

pos

var pos: Int

The current position of the stream.

ranges

val ranges: List<TextRange>

Functions

acceptToken

fun acceptToken(token: Int, endOffset: Int = 0)

Accept a token. By default, the end of the token is set to the current stream position, but you can pass an offset (relative to the stream position) to change that.

acceptTokenTo

fun acceptTokenTo(token: Int, endPos: Int)

Accept a token ending at a specific given position.

advance

fun advance(n: Int = 1): Int

Move the stream forward N (defaults to 1) code units. Returns the new value of next.

clipPos

fun clipPos(pos: Int): Int

peek

fun peek(offset: Int): Int

Look at a code unit near the stream position. .peek(0) equals .next, .peek(-1) gives you the previous character, and so on.

read

fun read(from: Int, to: Int): String

resolveOffset

fun resolveOffset(offset: Int, assoc: Int): Int?