Writing a Language Package¶

This example shows how to define a language for Kodemirror, either using a Lezer grammar (for tree-sitter-style parsing) or a StreamParser (for simpler line-by-line tokenization).

Language and LanguageSupport¶

A language package typically exports a LanguageSupport that bundles the language definition with supporting extensions (autocompletion, folding, etc.):

import com.monkopedia.kodemirror.language.*

fun myLanguage(): LanguageSupport {
    return LanguageSupport(
        language = myLang,
        support = myCompletions  // optional supporting extension
    )
}

Install it with:

val state = EditorState.create(EditorStateConfig(
    extensions = myLanguage().extension
))

Defining a language with a Lezer parser¶

If you have a Lezer grammar, create an LRLanguage:

import com.monkopedia.kodemirror.language.LRLanguage

val myLang = LRLanguage.define(
    parser = myParser,  // a com.monkopedia.kodemirror.lezer.common.Parser
    name = "myLang"
)

Defining a language with StreamParser¶

For simpler languages, implement StreamParser for line-by-line tokenization:

private data class SimpleState(val inString: Boolean = false)

private val simpleParser = object : StreamParser<SimpleState> {
    override val name = "simple-lang"

    override fun startState(indentUnit: Int) = SimpleState()

    override fun token(stream: StringStream, state: SimpleState): String? {
        // Comments
        if (stream.match("//")) {
            stream.skipToEnd()
            return "comment"
        }
        // Strings
        if (stream.match("\"")) {
            while (!stream.eol()) {
                if (stream.next() == "\"") break
            }
            return "string"
        }
        // Numbers
        if (stream.match(Regex("\\d+")) != null) return "number"
        // Keywords
        if (stream.match(Regex("\\b(fn|let|if|else|return|while|for|true|false)\\b")) != null) {
            return "keyword"
        }
        // Types
        if (stream.match(Regex("\\b(Int|String|Bool|Float)\\b")) != null) return "typeName"
        // Operators
        if (stream.match(Regex("[+\\-*/=<>!]+")) != null) return "operator"
        // Identifiers and other
        stream.next()
        return null
    }

    override fun copyState(state: SimpleState) = state.copy()
}

private val simpleLang = StreamLanguage.define(simpleParser)

StreamParser interface¶

Method	Description
`startState(indentUnit)`	Create the initial parser state
`token(stream, state)`	Consume one token, return its type (or `null`)
`copyState(state)`	Deep-copy the parser state
`blankLine(state, indentUnit)`	Called for empty lines
`indent(state, textAfter, context)`	Compute indentation for a line

Token types are standard CodeMirror highlighting tags: "keyword", "variableName", "string", "comment", "number", "operator", etc.

StringStream¶

The StringStream class provides methods for consuming input:

Method	Description
`next()`	Consume and return the next character
`peek()`	Look at the next character without consuming
`eat(ch)` / `eat(regex)`	Consume if matching
`eatWhile(ch)` / `eatWhile(regex)`	Consume while matching
`eatSpace()`	Skip whitespace
`match(string, consume, caseInsensitive)`	Match a string, returns `Boolean`
`match(regex, consume)`	Match a regex, returns `MatchResult?`
`skipToEnd()`	Move to end of line
`skipTo(ch)`	Skip to a character
`current()`	Get the text consumed since `start`
`eol()` / `sol()`	At end/start of line?
`column()`	Current column

match() return types

stream.match(String) returns Boolean. stream.match(Regex) returns MatchResult?. Don't use != null on the string overload.

Adding indentation¶

Provide an indent method in your StreamParser:

override fun indent(state: MyState, textAfter: String, context: IndentContext): Int? {
    // Return the number of spaces for indentation, or null for default
    return null
}

For tree-based languages, use indentNodeProp:

val indentNodeProp: NodeProp<(TreeIndentContext) -> Int?>

Built-in strategies:

delimitedIndent() — indent between matching delimiters
continuedIndent() — indent continuation lines
flatIndent — no indentation change

Adding code folding¶

Use foldNodeProp to define foldable regions:

val foldNodeProp: NodeProp<(SyntaxNode, EditorState) -> FoldRange?>

The foldInside(node) helper returns a FoldRange for the region between a node's first and last children (useful for brace-delimited blocks).

Using legacy modes¶

The :legacy-modes module provides over 100 languages ported from CodeMirror 5. Each is a StreamParser that you wrap with StreamLanguage.define():

import com.monkopedia.kodemirror.legacy.modes.python

val pythonLanguage = StreamLanguage.define(python)

LRLanguage — LR-parser-based language
LanguageSupport — language extension bundle
HighlightStyle — syntax highlighting style
syntaxHighlighting — highlighting extension
StreamParser — stream-based parser interface

Based on the CodeMirror Language Package example.