adapters #

Each library is measured through one adapter, and the adapter is the extension point. The contract is in bench/adapter.ts, the adapters are under bench/libraries/, and bench/libraries.ts lists them.

interface HighlighterAdapter<TTokens = unknown> {
	id: string;
	label: string;
	package: string; // its manifest is read for the version
	by_maintainer: boolean;
	note: string | null;
	extra_packages?: ReadonlyArray<string>; // separately versioned packages whose versions are recorded too
	langs: Partial<Record<Lang, string>>; // harness id to library id; absent means unsupported
	via: Partial<Record<Lang, string>>; // a coverage footnote for a claimed language
	output: 'classes' | 'inline_styles';
	engine: 'scanner' | 'grammar_vm' | 'regex' | 'textmate';
	theme_css: ReadonlyArray<string> | null; // the default theme's stylesheets; null when styles are inline
	theme_schemes: ReadonlyArray<'light' | 'dark'>; // the color schemes that theme covers
	wasm?: ReadonlyArray<{import: string; binary: string}>; // the wasm binaries it loads, counted apart from the JS
	setup(langs: ReadonlyArray<Lang>): Promise<void>; // once per process, before any timing
	tokenize(src: string, lang: Lang): TTokens; // timed: the library's own result
	count_tokens(tokens: TTokens): number; // never timed
	html(src: string, lang: Lang): string; // timed: the string consumers ship
	bundle_entry(langs: ReadonlyArray<Lang>): string; // module source importing exactly these languages
	bundle_html(exports: Record<string, any>, src: string, lang: Lang): string; // highlights through a built bundle's exports
	bundle_html_source(lang: Lang): string; // that same call as an expression over `exports` and `src`
}

Rules
#

Same stage. tokenize and html are the narrowest public entries doing the same work as the other libraries': tokens out, and the HTML string a consumer ships.
Explicit languages. A library is measured only on the languages its adapter lists. A call for any other language, or for one that wasn't set up, throws. A library that returned escaped plain text for a language it didn't know would benchmark as very fast.
Token counts are recorded, not equalised. Libraries split the same source differently, so each count is the library's own, and counting happens outside the timed call.
Synchronous timed paths. A library with an async API does its loading in setup and is timed through its synchronous core.
Browser-safe. An adapter uses no Node-only API, so it can run in a page unchanged. The harness reads versions, stylesheets, and wasm binaries from disk on its behalf.

bundle_entry and bundle_html serve the bundle sizes of deterministic metrics: the first writes the module a bundle is built from, and the second highlights through that bundle's exports, which is how each bundle is shown to hold its languages. bundle_html_source is the second as text: a cold-start process, which method describes, must load nothing but the module it measures, so it can't load the adapter to make the call. The check of every bundle evaluates the text and requires the same HTML.

What each adapter calls
#

library	tokenize	html	one token is
fuz_code	`SyntaxStyler#lex`	`SyntaxStyler#stylize`	a typed leaf or container
Twinkleplop	the language package's `tokenize()`	the language package's `language()`	a typed token after reclassification
Prism	`Prism.tokenize`	`Prism.highlight`	a `Token` at any depth
Shiki, both engines	`codeToTokensBase`	`codeToHtml`	a themed token, whitespace included

Each adapter has a notes.md beside it with the entry points, the import shape, how its tokens are counted, and its caveats, written so the library's maintainers can review or replace it.

One default is changed. Shiki stops tokenizing a line after half a second and returns the rest of it as one token, so what a call returns depends on how fast the machine is at that moment: a first call on a busy machine can come back with fewer tokens than the next. Both Shiki adapters raise that limit out of reach rather than turning it off, so every call does the same work, and still reads the clock as Shiki's default does.

fuz_code's tokenize result is valid only until its next call: for performance, it lexes into one buffer that every call reuses, where most other libraries return a result the caller keeps. A consumer that keeps one copies it, and the timed call doesn't. Its notes.md says how the harness reads the result in time.

What the HTML looks like differs, and the results say so:

fuz_code and Prism return bare <span> elements with classes, and the consumer supplies the wrapper
Twinkleplop returns a <pre> with a span for each line and classes on each token
Shiki returns a <pre> with a span for each line and an inline color on each token, so it needs no stylesheet and its tokenizing includes resolving colors

Coverage
#

Every adapter claims the timed languages. xml is claimed by all but Twinkleplop, which has no XML language. Where support is indirect, the coverage matrix carries a footnote:

fuz_code runs its TypeScript lexer for JS
Prism's Svelte grammar is the third-party prism-svelte, and its XML is its markup grammar under another name
Shiki's XML grammar loads the Java grammar it embeds

setup loads exactly the languages it is given, plus what a language embeds structurally, like the script and style languages of HTML and Svelte. A timed process sets up only the language it measures, so every library has the same languages loaded.

That decides how Markdown is measured: without highlighting inside fenced code blocks. fuz_code, Prism, and Shiki highlight a fenced block only when its language is loaded, and Twinkleplop never does, so loading one language puts them on the same footing. Two exceptions come from what loads with Markdown itself: Prism's Markdown depends on its markup grammar, so Prism still highlights a fence tagged html, xml, svg, or markup, and fuz_code and Prism highlight a fence tagged as Markdown.

The output check
#

Before a library is measured on a language, check_output runs it on the language's harness snippet, a few lines written dense in syntax. The library must produce tokens, and HTML with several kinds of styled span. A plain-text fallback has none, or only structural ones: a wrapper for each line, and in Shiki one span in the default color. A library that fails is recorded as excluded with the reason, which the results file keeps apart from unsupported.

Each input is then checked against a lower floor. It is lower because prose-heavy Markdown legitimately has few kinds of span, which is why the snippet decides and the floor only guards.

The tests run both for every adapter on every input of the corpus in every language it claims.

Adding a library
#

Add the library as a devDependency.
Write bench/libraries/<id>/adapter.ts, exporting an adapter, and notes.md beside it.
Add the adapter to ADAPTERS in bench/libraries.ts.

The tests then cover it, and a results file carries the roster of libraries, so the site needs no change.