chroma fork
  • Go 91.6%
  • JavaScript 4.7%
  • Python 2.5%
  • Just 0.5%
  • Shell 0.3%
  • Other 0.4%
Find a file
Haïm Dimer a14b489502
Some checks failed
CI / Test (push) Has been cancelled
CI / Lint (push) Has been cancelled
fix(docker): tokenize newlines as whitespace instead of Error (#1332)
## Summary

The Dockerfile lexer emits an `Error` token for the trailing newline of
every instruction, even for a minimal `FROM alpine`:

```
FROM alpine
```
tokenises as `Keyword("FROM")`, `TextWhitespace(" ")`,
`LiteralString("alpine")`, `Error("\n")`. A realistic multi-instruction
Dockerfile produces one `Error` token per line.

## Root cause

The `root` state has no rule matching a bare newline, so once an
instruction rule stops at the end of a line, the `\n` falls through to
no rule and becomes an `Error`.

This lexer is ported from Pygments, whose `RegexLexer` has an implicit
fallback: when no rule matches and the current character is `\n`, it
emits `Text` and resets to `root` (see `get_tokens_unprocessed`). Chroma
has no such implicit newline handling, so a Pygments-ported lexer that
relied on it needs the rule spelled out.

## Fix

Add a whitespace rule to the `root` state:

```xml
<rule pattern="\s+">
  <token type="TextWhitespace" />
</rule>
```

Newlines (and blank lines) now tokenise as `TextWhitespace`; `FROM ...
AS`, `\`-continuations, `HEALTHCHECK`, `CMD [...]`, etc. are unaffected,
and there are no more `Error` tokens.

## Tests

There was no Docker test fixture, so this adds one
(`lexers/testdata/docker.actual` + recorded `docker.expected`) covering
`FROM ... AS`, `WORKDIR`, `COPY`, `ENV`, `ARG`, a line-continuation
`RUN`, `EXPOSE`, `HEALTHCHECK`, and a JSON-form `CMD`. `go test ./...`
passes.

Note: the same "unmatched newline becomes Error" pattern likely affects
other Pygments-ported lexers; this PR fixes only Docker, happy to look
at a broader sweep separately if that's wanted.

---

Disclosure: this fix was developed with AI assistance; I reviewed it and
stand behind it.
2026-07-30 11:48:09 +10:00
.github chore: add theme request link 2026-07-02 23:20:33 +10:00
_tools chore: remove unbuildable _tools/exercise and _tools/css2style 2026-07-08 12:06:37 +10:00
bin chore(deps): update all non-major dependencies (#1323) 2026-07-13 19:47:06 +10:00
cmd chore(deps): update all non-major dependencies (#1323) 2026-07-13 19:47:06 +10:00
formatters fix: use sync.OnceValue for FontLiberationMono 2026-07-08 12:42:55 +10:00
internal/playground refactor: share playground rendering between chromad and WASM 2026-07-08 12:06:37 +10:00
lexers fix(docker): tokenize newlines as whitespace instead of Error (#1332) 2026-07-30 11:48:09 +10:00
quick feat!: replace custom Iterator with iter.Seq[Token] 2026-06-17 15:47:45 +10:00
styles fix: case-normalise Counterpart lookups and serialisation 2026-07-08 12:06:37 +10:00
.editorconfig Major updates to Caddyfile lexer (#932) 2024-02-20 20:08:27 +11:00
.gitignore feat: WASM playground 2025-07-01 09:59:45 +10:00
.golangci.yml chore: upgrade golangci-lint 2026-05-12 16:48:00 +10:00
.goreleaser.yml fix: switch to jsdelivr as unpkg was throwing CORS errors 2025-04-03 20:43:09 +11:00
AGENTS.md Fix AGENTS.md referencing a non-existent scripts directory (#1231) 2026-05-02 08:56:15 +10:00
biome.json chore: add JS formatting with biome 2025-07-01 10:18:33 +10:00
BUILD.bit fix: repair README language-table generation 2026-07-05 19:47:36 +10:00
chroma.jpg docs: add logo 2025-04-03 21:48:12 +11:00
coalesce.go fix: forward SetTracing through Coalesce and RemappingLexer 2026-07-08 12:06:37 +10:00
coalesce_test.go Upgrade project to Go 1.19 and switch to alecthomas/assert/v2 2022-10-18 08:55:38 +11:00
colour.go fix: reject malformed colours in ParseColour 2026-07-08 12:06:37 +10:00
colour_test.go fix: reject malformed colours in ParseColour 2026-07-08 12:06:37 +10:00
COPYING Include embedded font license in COPYING file (#1233) 2026-03-22 17:46:02 +11:00
delegate.go feat!: replace custom Iterator with iter.Seq[Token] 2026-06-17 15:47:45 +10:00
delegate_test.go feat!: replace custom Iterator with iter.Seq[Token] 2026-06-17 15:47:45 +10:00
doc.go Fix typo: translater → translator (#79) 2017-10-27 09:19:54 +11:00
Dockerfile chore(deps): update all non-major dependencies (#1285) 2026-06-13 16:44:35 +10:00
emitters.go chore: remove commented-out panic, allocate trace encoder lazily 2026-07-08 12:06:37 +10:00
formatter.go refactor!: remove unused RecoveringFormatter 2026-07-06 15:03:45 +10:00
go.mod chore(deps): update all non-major dependencies (#1323) 2026-07-13 19:47:06 +10:00
go.sum chore(deps): update all non-major dependencies (#1323) 2026-07-13 19:47:06 +10:00
iterator.go feat!: replace custom Iterator with iter.Seq[Token] 2026-06-17 15:47:45 +10:00
Justfile fix: repair README language-table generation 2026-07-05 19:47:36 +10:00
lexer.go refactor!: remove redundant Analyser interface 2026-07-08 12:06:37 +10:00
lexer_test.go Upgrade project to Go 1.19 and switch to alecthomas/assert/v2 2022-10-18 08:55:38 +11:00
mutators.go fix: clamp state-stack pops instead of panicking 2026-07-08 12:06:37 +10:00
mutators_test.go feat!: replace custom Iterator with iter.Seq[Token] 2026-06-17 15:47:45 +10:00
pygments-lexers.txt Update pygments-lexers.txt (#1070) 2025-04-21 08:47:35 +10:00
README.md feat(lexers): add Garden lexer (#1322) 2026-07-14 19:32:14 +10:00
regexp.go chore: remove commented-out panic, allocate trace encoder lazily 2026-07-08 12:06:37 +10:00
regexp_test.go fix: guard against zero-width match loops in the lexer 2026-07-08 12:06:37 +10:00
registry.go docs: document registration concurrency contract 2026-07-08 12:06:37 +10:00
remap.go fix: forward SetTracing through Coalesce and RemappingLexer 2026-07-08 12:06:37 +10:00
remap_test.go feat!: replace custom Iterator with iter.Seq[Token] 2026-06-17 15:47:45 +10:00
renovate.json5 chore: some house-keeping 2026-06-17 11:43:19 +10:00
serialise.go refactor: extract lexer config glob validation 2026-07-08 12:06:37 +10:00
serialise_test.go chore: upgrade golangci-lint 2026-05-12 16:48:00 +10:00
style.go docs: correct stale style and HTML formatter doc comments 2026-07-08 12:06:37 +10:00
style_test.go feat(styles): add light/dark mode support 2026-05-26 12:48:10 +10:00
tokentype_enumer.go feat!: replace custom Iterator with iter.Seq[Token] 2026-06-17 15:47:45 +10:00
types.go docs: document StandardTypes parent-class fallback 2026-07-08 12:06:37 +10:00

Chroma

Note

v3 alpha available. Chroma v3 replaces the custom Iterator type with Go's built-in iter.Seq[Token], removes the EOF sentinel, and bumps the module path to github.com/alecthomas/chroma/v3. See the v3.0.0-alpha.1 release for migration details.

A general purpose syntax highlighter in pure Go

Go Reference CI Slack chat

Chroma takes source code and other structured text and converts it into syntax highlighted HTML, ANSI-coloured text, etc.

Chroma is based heavily on Pygments, and includes translators for Pygments lexers and styles.

Table of Contents

  1. Supported languages
  2. Try it
  3. Using the library
    1. Quick start
    2. Identifying the language
    3. Formatting the output
    4. The HTML formatter
  4. More detail
    1. Lexers
    2. Formatters
    3. Styles
  5. Command-line interface
  6. Testing lexers
  7. What's missing compared to Pygments?

Supported languages

Prefix Language
A ABAP, ABNF, ActionScript, ActionScript 3, Ada, Agda, AL, Alloy, AMPL, Angular2, ANTLR, ApacheConf, APL, AppleScript, ArangoDB AQL, Arduino, ArmAsm, Arturo, Astro, ATL, AutoHotkey, AutoIt, Awk
B Ballerina, Bash, Bash Session, Batchfile, Beef, BibTeX, Bicep, BlitzBasic, BNF, BQN, Brainfuck
C C, C#, C++, C3, Caddyfile, Caddyfile Directives, Cap'n Proto, Cassandra CQL, Ceylon, CFEngine3, cfstatement, ChaiScript, Chapel, Cheetah, Clojure, CMake, COBOL, CoffeeScript, Common Lisp, Coq, Core, Crystal, CSS, CSV, CUE, Cython
D D, Dart, Dax, Desktop file, Devicetree, Diff, Django/Jinja, dns, Docker, DTD, Dylan
E EBNF, Elixir, Elm, EmacsLisp, ERB, Erlang
F Factor, Fennel, Fish, Forth, Fortran, FortranFixed, FSharp
G Garden, GAS, GDScript, GDScript3, Gemfile.lock, Gemtext, Genshi, Genshi HTML, Genshi Text, Gettext, Gherkin, Gleam, GLSL, Gnuplot, Go, Go HTML Template, Go Template, Go Text Template, GraphQL, Groff, Groovy
H Handlebars, Hare, Haskell, Haxe, Hexdump, HLB, HLSL, HolyC, HTML, HTTP, Hurl, Hy, Hyprlang
I Idris, Igor, INI, Io, ISCdhcpd
J J, Janet, Java, JavaScript, JQ, JSON, JSONata, Jsonnet, Julia, Jungle
K Kakoune, KDL, Kotlin
L Lateralus, Lean4, Lighttpd configuration file, LilyPond, LLVM, lox, Lua, Luau
M Makefile, Mako, markdown, Markless, Mason, Materialize SQL dialect, Mathematica, Matlab, MCFunction, Meson, Metal, microcad, MiniZinc, MLIR, Modelica, Modula-2, Mojo, MonkeyC, MoonBit, MoonScript, MorrowindScript, Myghty, MySQL
N NASM, Natural, NDISASM, Newspeak, Nginx configuration file, Nim, Nix, NSIS, Nu
O Objective-C, ObjectPascal, OCaml, Octave, Odin, OnesEnterprise, OpenEdge ABL, OpenSCAD, Org Mode
P PacmanConf, Perl, PHP, PHTML, Pig, PkgConfig, PL/pgSQL, plaintext, Plutus Core, Pony, PostgreSQL SQL dialect, PostScript, POVRay, PowerQuery, PowerShell, Prolog, Promela, PromQL, properties, Protocol Buffer, Protocol Buffer Text Format, PRQL, PSL, Puppet, Python, Python 2
Q QBasic, QML
R R, Racket, Ragel, Raku, react, ReasonML, reg, Rego, reStructuredText, Rexx, RGBDS Assembly, Ring, RPGLE, RPMSpec, Ruby, Rust
S SAS, Sass, Scala, scdoc, Scheme, Scilab, SCSS, Sed, Sieve, Smali, Smalltalk, Smarty, SNBT, Snobol, Solidity, SourcePawn, Spade, SPARQL, SQL, SquidConf, Standard ML, stas, Stylus, Svelte, Swift, SYSTEMD, systemverilog
T TableGen, Tal, TASM, Tcl, Tcsh, Templ, Termcap, Terminfo, Terraform, TeX, Thrift, TOML, TradingView, Transact-SQL, Turing, Turtle, Twig, TypeScript, TypoScript, TypoScriptCssData, TypoScriptHtmlData, Typst
U ucode
V V, V shell, Vala, VB.net, verilog, VHDL, VHS, VimL, vue
W WDTE, WebAssembly Text Format, WebGPU Shading Language, WebVTT, Whiley
X XML, Xorg
Y YAML, YAML+Jinja, YANG
Z Z80 Assembly, Zed, Zig

I will attempt to keep this section up to date, but an authoritative list can be displayed with chroma --list.

Try it

Try out various languages and styles on the Chroma Playground.

Using the library

This is version 2 of Chroma, use the import path:

import "github.com/alecthomas/chroma/v3"

Chroma, like Pygments, has the concepts of lexers, formatters and styles.

Lexers convert source text into a stream of tokens, styles specify how token types are mapped to colours, and formatters convert tokens and styles into formatted output.

A package exists for each of these, containing a global Registry variable with all of the registered implementations. There are also helper functions for using the registry in each package, such as looking up lexers by name or matching filenames, etc.

In all cases, if a lexer, formatter or style can not be determined, nil will be returned. In this situation you may want to default to the Fallback value in each respective package, which provides sane defaults.

Quick start

A convenience function exists that can be used to simply format some source text, without any effort:

err := quick.Highlight(os.Stdout, someSourceCode, "go", "html", "monokai")

Identifying the language

To highlight code, you'll first have to identify what language the code is written in. There are three primary ways to do that:

  1. Detect the language from its filename.

    lexer := lexers.Match("foo.go")
    
  2. Explicitly specify the language by its Chroma syntax ID (a full list is available from lexers.Names()).

    lexer := lexers.Get("go")
    
  3. Detect the language from its content.

    lexer := lexers.Analyse("package main\n\nfunc main()\n{\n}\n")
    

In all cases, nil will be returned if the language can not be identified.

if lexer == nil {
  lexer = lexers.Fallback
}

At this point, it should be noted that some lexers can be extremely chatty. To mitigate this, you can use the coalescing lexer to coalesce runs of identical token types into a single token:

lexer = chroma.Coalesce(lexer)

Formatting the output

Once a language is identified you will need to pick a formatter and a style (theme).

style := styles.Get("swapoff")
if style == nil {
  style = styles.Fallback
}
formatter := formatters.Get("html")
if formatter == nil {
  formatter = formatters.Fallback
}

Then obtain an iterator over the tokens:

contents, err := io.ReadAll(r)
iterator, err := lexer.Tokenise(nil, string(contents))

And finally, format the tokens from the iterator:

err := formatter.Format(w, style, iterator)

The HTML formatter

By default the html registered formatter generates standalone HTML with embedded CSS. More flexibility is available through the formatters/html package.

Firstly, the output generated by the formatter can be customised with the following constructor options:

  • Standalone() - generate standalone HTML with embedded CSS.
  • WithClasses() - use classes rather than inlined style attributes.
  • ClassPrefix(prefix) - prefix each generated CSS class.
  • TabWidth(width) - Set the rendered tab width, in characters.
  • WithLineNumbers() - Render line numbers (style with LineNumbers).
  • WithLinkableLineNumbers() - Make the line numbers linkable and be a link to themselves.
  • HighlightLines(ranges) - Highlight lines in these ranges (style with LineHighlight).
  • WithLinePrompts(prompt, ranges) - Render non-selectable prompts before lines in these ranges.
  • LineNumbersInTable() - Use a table for formatting line numbers and code, rather than spans.
  • BaseLineNumber(n) - Set the first line number; highlight and prompt ranges follow it.
  • WrapLongLines(true) - Wrap long lines instead of scrolling.
  • PreventSurroundingPre(true) - Emit the highlighted spans without the surrounding <pre>.
  • InlineCode(true) - Generate output suitable for inline <code> elements.
  • WithPreWrapper(wrapper) - Customise the wrapping <pre> element.
  • WithCustomCSS(css) - Add custom CSS per token type.
  • WithModeClasses(true) - Scope generated CSS by the style's light/dark mode class.

If WithClasses() is used, the corresponding CSS can be obtained from the formatter with:

formatter := html.New(html.WithClasses(true))
err := formatter.WriteCSS(w, style)

For light/dark theme switching, enable WithModeClasses(true) and write the CSS for both styles into one stylesheet. Each rule is scoped by its style's mode class (eg. .chroma.dark), so the theme can be toggled at runtime by swapping the mode class on the wrapper element.

More detail

Lexers

Lexers are community maintained. Please use Discussions for lexer requests and lexer highlighting bugs.

See the Pygments documentation for details on implementing lexers. Most concepts apply directly to Chroma, but see existing lexer implementations for real examples.

In many cases lexers can be automatically converted directly from Pygments by using the included Python 3 script pygments2chroma_xml.py. I use something like the following:

uv run --script _tools/pygments2chroma_xml.py \
  pygments.lexers.jvm.KotlinLexer \
  > lexers/embedded/kotlin.xml

A list of all lexers available in Pygments can be found in pygments-lexers.txt.

Formatters

Chroma supports HTML output, as well as terminal output in 8 colour, 256 colour, and true-colour.

A noop formatter is included that outputs the token text only, and a tokens formatter outputs raw tokens. The latter is useful for debugging lexers.

Styles

Chroma styles are defined in XML. The style entries use the same syntax as Pygments. All Pygments styles have been converted to Chroma using the _tools/style.py script.

Style names are case-insensitive. For example, monokai and Monokai are treated as the same style.

When you work with one of Chroma's styles, know that the Background token type provides the default style for tokens. It does so by defining a foreground color and background color.

For example, this gives each token name not defined in the style a default color of #f8f8f8 and uses #000000 for the highlighted code block's background:

<entry type="Background" style="#f8f8f2 bg:#000000"/>

Also, token types in a style file are hierarchical. For instance, when CommentSpecial is not defined, Chroma uses the token style from Comment. So when several comment tokens use the same color, you'll only need to define Comment and override the one that has a different color.

For a quick overview of the available styles and how they look, check out the Chroma Style Gallery.

Command-line interface

A command-line interface to Chroma is included.

Binaries are available to install from the releases page.

The CLI can be used as a preprocessor to colorise output of less(1), see documentation for the LESSOPEN environment variable.

The --fail flag can be used to suppress output and return with exit status 1 to facilitate falling back to some other preprocessor in case chroma does not resolve a specific lexer to use for the given file. For example:

export LESSOPEN='| p() { chroma --fail "$1" || cat "$1"; }; p "%s"'

Replace cat with your favourite fallback preprocessor.

When invoked as .lessfilter, the --fail flag is automatically turned on under the hood for easy integration with lesspipe shipping with Debian and derivatives; for that setup the chroma executable can be just symlinked to ~/.lessfilter.

Projects using Chroma

Testing lexers

If you edit some lexers and want to try it, open a shell in cmd/chromad and run:

go run . --csrf-key=securekey

A Link will be printed. Open it in your Browser. Now you can test on the Playground with your local changes.

If you want to run the tests and the lexers, open a shell in the root directory and run:

go test ./lexers

When updating or adding a lexer, please add tests. See lexers/README.md for more.

What's missing compared to Pygments?

  • Quite a few lexers, for various reasons (pull-requests welcome):
    • Pygments lexers for complex languages often include custom code to handle certain aspects, such as Raku's ability to nest code inside regular expressions. These require time and effort to convert.
    • I mostly only converted languages I had heard of, to reduce the porting cost.
  • Some more esoteric features of Pygments are omitted for simplicity.
  • Though the Chroma API supports content detection, very few languages support them. I have plans to implement a statistical analyser at some point, but not enough time.