TodayTools

Code syntax highlighters flashcards

Shiki, Prism, highlight.js and CodeMirror with Lezer: how they tokenise code, and which to ship where.

Donate
Beginner friendly, starts from zero
Section
How to study
0
Code syntax highlightersStart from zero · 1 of 12

What is a code highlighter, in one plain sentence?

A tool that colours the words in a piece of code by the job each word does, so people can read it faster.

Like a highlighter pen, but it knows which words to mark.

All topics

Code syntax highlighters at a glance

A code highlighter colours each word of code by its job, so it is easy to read.

Like a teacher colouring the nouns and verbs in a sentence: the words stay the same, the colours show their roles.

Words you will meet

Token
One small piece of code, like a word or a symbol
Keyword
A word the language keeps for itself, like let or const
Grammar
The rule book that says what kind each token is
Theme
A set of colours for each kind of token
Class
A label name on an HTML element that CSS can colour
Editor
A box you type code into that recolours as you type
Escape
Write < and > as harmless text so code is shown, not run

The steps

  1. 1. Start from zero (12 cards): What highlighting is, where you see it, and the plain words behind it.
  2. 2. Core basics (14 cards): How highlighting works, the four main tools, and putting coloured code on a page safely.
  3. 3. Next steps (15 cards): Shiki in production, dual themes, bundle size, Prism and hljs done well, CodeMirror basics.
  4. 4. Go deeper (17 cards): Accuracy, transformers, incremental parsing, huge files, custom grammars and choosing under constraints.
All 12 cards in Start from zero
What is a code highlighter, in one plain sentence?
A tool that colours the words in a piece of code by the job each word does, so people can read it faster.
What was reading code like before highlighters?
Every word was the same colour, like a page with no paragraphs. You had to read each character to find where a piece of text or a note started and ended.
Where have you already seen highlighted code?
On help pages for apps and software (people call these docs sites), in blog posts that teach coding, in chat apps that show code blocks, and inside code editor programs.
What everyday thing works like a highlighter?
A teacher who colours a sentence: nouns in blue, verbs in red. Nothing in the sentence changes; the colours just show what each word is doing.
What are code, a programming language and syntax?
Code is written instructions for a computer. Each programming language, such as Python or JavaScript, has its own spelling and grammar rules, called syntax.
Your first tiny win: which parts of this line get different colours?
let is a keyword, a word the language reserves for itself. "Chennai" is a string, which means plain text in quotes, and everything after // is a comment, a note for humans that the computer skips.
What are a token, a grammar and a regex?
A token is one small piece of code, like a single word or symbol. A grammar is the rule book that tells a highlighter what kind each token is, and a regex is a short search pattern, like 'anything between quotes', that some grammars use.
What are a browser, HTML, an element, CSS and the DOM?
A browser shows web pages. HTML is the language pages are written in, made of elements marked by tags like <p>; CSS sets their colours, and the DOM is the browser's live copy of the page that code can change.
How does a colour actually reach each word on a web page?
Each token is wrapped in a small span element with a class, which is a label name like keyword. A theme is a set of CSS rules that says which colour each class gets.
What is the difference between a highlighter and a code editor?
A highlighter colours code that just sits on a page. An editor is a box you type code into, and it recolours as you type, using a parser: a program that reads code and works out its structure, called a syntax tree.
What do you need before you start, and what words about tools will you meet?
A browser and a text editor, nothing to install. Highlighters come as libraries, which are ready-made code you import (pull into your file), and run in the browser or earlier, on a server or at build time (when the site is prepared).
What will you be able to do after this topic?
Pick the right highlighter for a page, colour every code block on it, make code look like it does in the editor, and show code safely so it is never run by mistake.
Code syntax highlighters cheatsheet

Shiki

  • codeToHtml(code, { lang: 'ts', theme: 'github-dark' }) Shorthand: code to HTML
  • themes: { light: 'github-light', dark: 'github-dark' } Dual theme via --shiki-dark
  • createHighlighterCore({ themes, langs, engine }) Fine-grained, small bundle
  • createJavaScriptRegexEngine() No WASM, for browsers
  • hl.loadLanguage('rust') Add a language later
  • transformers: [transformerNotationDiff()] // [!code ++] diff markers

Prism

  • Prism.highlightAll() Highlight every block
  • Prism.highlightElement(codeEl) One element
  • Prism.highlight(code, Prism.languages.js, 'js') String to HTML
  • window.Prism = { manual: true } Stop auto-run on load
  • import 'prismjs/components/prism-python' Load one grammar

highlight.js v11

  • hljs.highlight(code, { language: 'sql' }) Known language
  • hljs.highlightAuto(code, ['js', 'python']) Guess from a short list
  • hljs.highlightElement(codeEl) One element
  • hljs.registerLanguage('python', python) With lib/core for small bundles
  • hljs.configure({ languages: ['js', 'bash'] }) Limit auto-detect

CodeMirror 6

  • new EditorView({ doc, extensions, parent }) Create an editor
  • [basicSetup, json()] Defaults plus a language
  • new Compartment() Swap theme or language live
  • view.dispatch({ changes: { from: 0, insert: 'x' } }) Edit through a transaction
  • EditorView.updateListener.of(u => u.docChanged) React to changes
  • EditorState.readOnly.of(true) Read-only view

Lezer

  • HighlightStyle.define([{ tag: t.keyword, color }]) Token colours
  • syntaxHighlighting(style) Use a style as an extension
  • highlightCode(code, tree, classHighlighter, put, br) Static highlighting
  • styleTags({ Field: t.propertyName }) Map nodes to tags
  • StreamLanguage.define(legacyMode) Wrap a CodeMirror 5 mode

Pick by job

  • Docs and blogs Shiki at build time
  • Tiny client widget Prism or hljs core
  • Unknown language highlight.js auto-detect
  • Typing and editing CodeMirror 6 with Lezer
  • Huge read-only text Worker plus virtualised lines
Donate