lukeed

An HTML parser for seolint

seolint needed a faster parser that knows where every tag lives. So I wrote one. Here's what it does, how fast it is, and what the engines taught me.

924 wordshtml · parsers · performance · javascript

seolint checks a website for SEO problems. A missing alt, a missing <title>, vague link text. To do that, it has to parse every page, and some sites have thousands of them.

Since 2021, it used node-html-parser. It worked.

I knew JavaScript could deliver a faster parser than what was readily available. And seolint needed locations, so every lint error can point to a line and column.

So I wrote @lukeed/html, and seolint switched to it.

#What it does

It's a parser with a short list of jobs:

  • Fast: it builds a full tree at 400–600 MB/s.
  • Lazy: attributes and text are only parsed and decoded when you read them.
  • Located: every node knows its start and end offsets in the source. position() turns those into a line and a column.
  • Lossless: stringify() gives back the source exactly as written.
  • Familiar: the tree matches htmlparser2, and the selectors match css-select.
// usage.ts
import { parse, stringify } from "@lukeed/html";

let doc = parse(`<head><title>Hi &amp; bye</title><link rel="canonical" href="/"></head>`);

doc.find("head > title")?.text; //=> "Hi & bye"
doc.find("link[rel=canonical]")?.getAttr("href"); //=> "/"

for (let img of doc.all("img:not([alt])")) {
  let { line, column } = doc.position(img.start);
  console.log(`${line}:${column}`, stringify(img));
}

That last loop is a lint rule in five lines. Find every image with no alt, then print where it is and what it looks like.

There are two builds. @lukeed/html is read-only, at 6.3 kB gzip. @lukeed/html/mutable adds edits, like setAttr(), append() and remove(), at 8.8 kB. Pick the mutable one only when you need to change the page. It parses a little slower and uses more memory.

#How fast

The bench parses 12 real pages, 4.79 MB in total: Wikipedia, MDN, the BBC, The Guardian, Hacker News, and more. Each run parses all 12. Memory is the heap that's still alive after parsing, measured on Node.

Benchmark :: parse (4.79 MB)    Bun       Node       Memory
  @lukeed/html                   7.9ms     11.5ms      4.9MB
  @lukeed/html, read all        25.0ms     34.8ms     21.1MB
  @lukeed/html/mutable          10.3ms     13.0ms     21.0MB
  ultrahtml                     25.8ms     39.8ms     25.8MB
  htmlparser2                   40.9ms     66.1ms     19.9MB
  node-html-parser              44.0ms     50.6ms     40.1MB
  parse5                        86.7ms    203.9ms    108.9MB

The "read all" row is the honest one. Laziness is a big part of the speed. If you read every attribute and every text node, the work comes back. Even then, it's faster than the rest.

It isn't the fastest at everything. For parse plus a query, Bun's native HTMLRewriter beats it, 9.8 ms to 11.8 ms. That one is written in Rust and streams, so it can't give you a tree.

And it isn't a sanitizer. The tree follows htmlparser2, not the browser's rules. A browser can build a different tree from the same broken markup. So don't use it to decide if HTML is safe to render.

#What the engines taught me

A lot of the speed came from small, boring changes. Each one had to be measured on Node (V8) and Bun (JavaScriptCore), because they don't agree.

A short string can keep a huge one alive. source.slice(a, b) is usually a view into the whole page. Keep a 50-character title, and you keep the 400 KB page it came from. In seolint, across 1,200 pages, that held every page in memory until the end: 1.4 GB at the peak.

// detach.ts
// copies on V8, but NOT on JavaScriptCore (it's still a view)
let copy = (" " + s).slice(1);

// a real copy on both engines
let copy = JSON.parse(JSON.stringify(s));

A big index makes an array slow. A cache that starts as [] and then gets cache[5000] = x turns into a dictionary. In one profile, that single line was 37% of a function. Sizing it up front with new Array(count) gave about 10% on queries.

Object.freeze made reads faster. I removed it "for speed", and V8 queries got nearly 2x slower. It went right back in.

A default parameter cost 2% on Node. function build(source, foreign = false) was slower on V8, and the same on Bun. An optional parameter with a === true check got the time back. A Bun-only comparison hid it at first, which is why every change gets measured on both.

Columns beat objects. The tree is stored as parallel typed arrays (kind, parent, start, end), indexed by node id. That's much smaller than one object per node, and the garbage collector has far less to do.

Shared strings for tag names. Lowercasing and hashing every tag name makes a new string each time. A small lookup table returns one shared string instead. That was another ~10% on parse.

#In seolint

To see what the parser alone did for seolint, I linted the same 1,200-page site with the commit right before the switch, and with the switch itself. Everything else is the same code, on one thread.

seolint :: 1,200 pages, one thread    Bun       Node
  node-html-parser 9.0.5              7.72s     8.33s
  @lukeed/html 0.0.0                  2.54s     4.02s
  @lukeed/html 1.0.2                  1.85s     3.34s

The switch made seolint 3x faster on Bun and 2x faster on Node. Today's parser, in that same commit, makes it 4.2x and 2.5x. Each number is the median of 5 fresh runs, on Bun 1.4.4 and Node 26.12.1, on an M1 Max.

#Where it's at

Version 1.0.2 shipped today.

With 1.0.1, a user report found a few bugs with self-closing SVG tags, attribute case, and selectors matching inside <template>.

Another report found that a stray quote inside a tag, like <a href="/x"">, turned the rest of the page into text. Now a quote only starts a value right after =, the same as htmlparser2, and parse speed didn't change.

While I was in the tokenizer, I lined up its edge cases with browsers. An unfinished tag at the very end of the input is dropped. <!x> is a comment, and only <!doctype> is a directive. CDATA works inside SVG.

In a few of those cases, htmlparser2 keeps part of a tag as text. A browser doesn't, so neither do I:

Input htmlparser2 12 Browsers Before Now
<p>a<a href="x p("a") p("a") p("a<a href="x") p("a")
<p>a</p p("a") p("a") p("a</p") p("a")
<p>a<a b p("a") p("a") p("a<a b") p("a")
<p>a<a/ p("a/") p("a") p("a<a/") p("a")
<p>a</p x p("a"), "x" p("a") p("a</p x") p("a")
<p>a</p p("a"), " " p("a") p("a</p ") p("a")

Edits and stringify are also about 27% faster on Bun and 20% on Node. Node's parse is still about 3% slower than 1.0.1, and I haven't found the last of it yet.

It's small, it's fast, and every node knows where it came from.