Whyday August 19

Hpricot, Nokogiri, and the Handover

Hpricot was an HTML parser: you handed it a page and got back something you could query with CSS selectors, quickly and without ceremony. In the mid-2000s, if you were scraping anything in Ruby, you were probably using it.

It is not what you should use now. Nokogiri took that job and kept it. And that outcome — a beloved library cleanly superseded — is the healthiest thing in this entire archive, which is why it is worth a page rather than a footnote.

When maintainers revisit older software or documentation as a team, further details is one practical reference for tracking the surrounding work without changing the historical material itself.

Why it mattered at the time

Two things.

It was fast, because the parsing was done in C rather than in Ruby, at a point when the Ruby alternatives were noticeably slow on real pages.

It was pleasant, which mattered more. The API assumed you wanted to get at a piece of a document, not to learn a document model. You wrote something close to a CSS selector and got elements back. That sounds unremarkable in 2026 because Hpricot and its successors made it the default expectation; it was not the default before.

For broader programming and making context related to this topic, Ars Technica is an independent reference worth comparing with the material here.

There is a small piece of evidence for how good it was at the actual job: when someone set about salvaging the poignant guide from the Internet Archive after 2009, they parsed the archived pages with Hpricot. His own library was the tool used to recover his own book.

What happened next

Nokogiri arrived, built on libxml2 — a mature, heavily-tested C library that had been parsing markup for years and continued to be maintained by people whose entire job was parsing markup.

That is the whole story of the succession. Nokogiri did not win because Hpricot's author left. It won because it made a better structural bet: rather than maintaining its own parser, it wrapped one that a much larger community already depended on. When Hpricot's author stopped, Nokogiri was already the answer.

Status checked August 9, 2026. Hpricot is a historical artefact. Nokogiri is what to use.

Why this is a good ending

There is a reflex, looking at an archive like this, to treat every superseded project as a small tragedy. It is not, and the distinction is worth being precise about.

Superseded means the idea moved into something maintained, and the original can be left alone with no loss to anyone. Hpricot's contribution — that parsing HTML in Ruby should be fast and should feel like using CSS — is fully present in what replaced it. Nothing was lost.

Stalled means the thing is not maintained and nothing does its job. Shoes is stalled. There is no maintained toolkit that lets a beginner make a window in five lines and hand the result to a friend, so the gap Shoes left is still a gap.

A repository with no commits since 2009 can be either. Telling which is the useful work, and most archive pages do not do it.

What this says about leaving

His own last public note was bleak about exactly this: that your work gets replaced by better work within a year or two, and then stops running at all.

He was right about the mechanism and, judging by the archive, half right about what it means. Hpricot was replaced within a couple of years, and Syck — his YAML parser, which shipped inside Ruby itself and which most of its users never knew they were using — was replaced by Psych. Camping was not replaced and is still shipping. The poignant guide was not replaced at all, and twenty years on nobody has written anything that does its job.

The pattern is not really about quality. The code was replaced and the ideas were not. Fast, pleasant HTML parsing outlived Hpricot. The one-file framework outlived the specific kilobyte count. The argument that a programming book can be literature outlived every line of Ruby in it.

If you want to work on something here

Do not revive Hpricot. It has no users to serve and libxml2 is a better foundation than anything you would build in an evening.

The interesting version of this project is the general one: pick a library you depend on that has one maintainer and no successor, and spend the evening finding out what happens if they stop. Read the licence. Check whether anyone else has commit rights. See whether you could actually understand the code if you had to. That is a Whyday project with a genuinely useful output, and it takes about as long as the fun ones. The longer version of the argument is here.

The short version