The CEOL Universe: a word list you define yourself

Published 2026-11-01

10,020,150 candidate English strings, the evidence for each one, and no verdicts

“How do you know that a word is valid?”

“If it’s in the dictionary, obviously.”

“Which dictionary?”

You might name the usual suspects: Merriam-Webster, Oxford English, maybe even Wiktionary, and so on.

But these sources don’t always agree: at any time, a word on one might not be on another.

A disagreement could reflect several things, e.g.,

• Timing: Sources aren’t always updated at the same times, so one might be less current than the other. Some may be retired altogether.
• Focus: Specialized, niche, and technical terminology is sometimes deemed out-of-scope by mainstream word repositories (and vice versa), despite meeting other criteria for word validity.
• Standards: Some platforms refuse to publish certain words (slang, profanity, etc.), even if they otherwise qualify as “valid.”
• Definitions (most important): Word inclusion criteria and algorithms vary by source. What qualifies as “valid” isn’t the same across platforms, and some might even disagree on the definition of the word “word” itself – are spaces allowed? what about hyphens, symbols, or special characters? And the method by which a new candidate string is introduced might differ: some sources are community-maintained (e.g., Wiktionary), whereas others are institutional (e.g., Merriam-Webster, OED).

So, if you want to know whether a word is legit, where do you look? Is finding it on a single source enough, or must the string be attested by more than one platform? Which sources are the most credible?

I won’t pretend to have an answer to any of these questions.

But in case somebody does, it might be necessary to know which words are on which sources at a given time, where the platforms agree and disagree, and other metadata about English strings, and that is something that I can help with.

The CEOL Universe (the Common English OpenList Universe) is a free dataset of 10,020,150 strings that might be English words: single words and their forms, hyphenated compounds, accented loanwords, contractions, and multiword terms. For each one, it records which of 8 published word lists include it, whether it turns up in a large corpus of books, and how it got into the dataset in the first place.

But the CEOL Universe is not another word list. It never decides what counts as a word. You do:

• Evidence, not verdicts: Every row says which sources list the string, which sources couldn’t have listed it, and which never measured it. No row claims that a string is a word.
• Your definition, your list: Pick your criteria (listed by at least 4 lists, plain a to z, no language model involved, etc.), and get the exact count, the code that reproduces the list, and a datasheet that describes it.
• Versions that never change: A released version is fixed. A correction comes as a new version with a change log, and every file’s hash is on the dataset’s card.
• Free and open: The data is CC BY-SA 4.0, and the code is MIT. Every source’s notice ships with the data.

Why I built it

The CEOL Universe started as a side effect of my other word list. I built the English OpenList as a free English word list for word games, and it now supplies Amordle, Ars Magna, and Anagram Tiles. Building it meant taking in nearly 10 million strings and ruling on every one of them: on the list, or off it.

Every word list makes those calls. Almost none of them shows its work. A list tells you that a string is in; it doesn’t tell you why, who else agrees, or what was never checked.

And what should count changes with the use. A word game needs one list, a spell-checker needs another, and a study of how language models cope with uncommon or made-up strings needs a third. No single list fits all three.

So I separated the evidence from the verdicts. The CEOL Universe keeps every candidate, along with everything recorded about where it came from, and leaves the definition to whoever uses it. You choose the rule, and the dataset gives you the list that meets it, with the evidence for every entry.

What’s already out there

There’s no shortage of English word lists. There’s a shortage of lists that tell you anything beyond the string.

• Lexicons: WordNet and Wiktionary are rich (senses, definitions, relations), but each one records only its own entries. Princeton says WordNet is no longer developed, and Wiktionary is edited every day, so a reading of it is only good for its date.
• Spell-checker lists: SCOWL sorts words into sizes by how common they are, for spell-checkers, and the hunspell dictionaries are built from it. SCOWL’s original version left out hyphenated and open compounds, and as of February 2026, its original format is no longer maintained.
• Word-game lists: ENABLE (173,528 words) and YAWL (264,097) are in the public domain, but ENABLE reflects the Official Scrabble Players Dictionary as of January 1, 1998. The official tournament lists themselves are proprietary.
• Big text files: dwyl/english-words has over 466,000 entries, but its own README says the copyright still belongs to the original publisher, and no entry says where it came from. Moby Words II, released in 2002, has 354,984 single words and nothing else about them.
• Frequency lists: wordfreq and the Google Books Ngrams tell you how often a string occurs, not whether anyone lists it, and wordfreq’s author has said it won’t be updated again.

Each of these records its own membership. None of them records which other lists hold a string, clearly distinguishes “not measured” from “not listed,” and is checkpointed by date.

That’s the gap the CEOL Universe fills:

• A listing per source: For each candidate, its status in 8 published lists: WordNet 3.0, a dated Wiktionary dump, 4 hunspell dictionaries (American, British, Canadian, and Australian), ENABLE, and SCOWL.
• “Not measured” isn’t “no”: A list that can’t hold a class of string (a list with no hyphens can’t hold a hyphenated one) is recorded as unable to, not as leaving it out. For comparison, wordfreq returns 0 for a word it doesn’t know.
• A grammar, not a guess: The candidates are the output of a spelling grammar that is code you can run. It admits hyphens, accents, apostrophes, ligatures, a diaeresis, and spaces, and every row is marked with the classes it carries.
• The same ID in every version: Each row’s ID is derived from the string alone, so a string keeps its ID from one version to the next.
• Models and generators, marked: The rows the English OpenList decided with a language model, and the rows its synthetic engine made, are marked, so a list can keep them or drop them.

How the lists compare

Because every candidate is checked against every list, the lists can finally be measured against each other. Of the 9,725,285 single-token rows, 1,089,968 (11.2%) are listed by at least 1 of the 8 lists. Most of those are listed by a single family of lists, and 43.7% of them by Wiktionary and nothing else. The consensus core, the rows listed by 2 or more independent families, is 417,864 rows.

Some of it is what you’d expect: of ENABLE’s words that Wiktionary could hold, Wiktionary holds 96%. Some of it isn’t: WordNet holds only 8% of what Wiktionary holds, because Wiktionary is about 10 times bigger, inflected forms included, and WordNet lists base forms.

How it was built

Version 1.0 starts from the English OpenList’s whole intake, the 9,653,999 strings it took in, whichever way it ruled. Then the sources are read again under the spelling grammar, every source’s coverage is reconciled until nothing a source lists is missing, and multiword terms come in from WordNet and Wiktionary. Every step has its own committed record, and the release build checks the result before anything ships.

The catch

A dataset like this is only useful if you know its weak spots, so here they are up front. (The dataset’s card puts them first, too.)

• Most rows rest on a single corpus. 8,024,536 rows (80.1%) came in through the English OpenList, are on none of the 8 lists, and appear in Google Books. For 7,900,248 of them, Google Books is the only evidence, and Google Books is scanned text, with recognition errors.
• Some rows were made, not found. 330,386 rows came from the English OpenList’s synthetic engine (affixes, chemical names, scientific prefixes, and Greek plurals), and 137,485 rows of its list were decided with a language model. Both are marked.
• Some lists can’t be published. The build measured 13 lists, and 5 of them are withheld for license reasons, so the published listing counts 8.
• No definitions. The CEOL Universe records no senses, parts of speech, pronunciations, or etymologies. WordNet, Wiktionary, and the CMU Pronouncing Dictionary already do that well.

Use it

The dataset lives on Hugging Face in 7 configs: every row (universe, 10,020,150 rows), the single-token rows with their features (features), the rows a published list holds (panel-listed, 1,384,833), the English OpenList at a pinned revision (eol-valid, 344,713), multiword rows (spaced, 294,865), rows with a hyphen, an accent, a diaeresis, an apostrophe, or a ligature (exceptions, 209,883), and a small sample (38,988) for trying it out.

from datasets import load_dataset

rows = load_dataset(
    "ryanjosephkamp/ceol-universe",
    "universe",
    revision="v1.0",
    split="candidates",
)
print(rows.num_rows)  # 10,020,150

Or, query it straight from the hub with DuckDB:

SELECT count(*)
FROM 'hf://datasets/ryanjosephkamp/ceol-universe@v1.0/data/universe.parquet';

Or, skip the code entirely. On the site, you can look a string up and read its record, compose a list and take its code, chart every dimension of every row, and read the dataset’s documentation, limitations first. A list you compose there comes with a datasheet, so whatever you ship says what’s in it.

For research, the site also carries a technical report with the method and every figure, a review of comparable resources, and a study kit for drawing a reproducible sample of candidates and labeling it under a definition of your own.

Under the hood

The CEOL Universe is 3 pieces of software: a Python pipeline that builds the dataset, a static site that explores it, and a small Python kit for studies. Here’s what each one uses, and why.

The pipeline: Python, with Parquet out

• Python 3.11: The build, the analysis, and the checks are Python, because the sources already have good Python tools. nltk reads WordNet, wordfreq scores frequency, and spylls, a pure-Python port of hunspell, checks candidates against the 4 hunspell dictionaries, affix rules included.
• A grammar you can run: What counts as a candidate is pipeline/ceol/grammar.py, not a paragraph of prose. Every string in the dataset passes it, and every row records which of its classes (hyphenated, accented, spaced, and so on) the string carries.
• Parquet, 1 file per config: Columnar, compressed, and readable by pandas, Polars, DuckDB, Arrow, and Hugging Face’s datasets library with no glue code. The release build writes the files with pyarrow, then checks each one before it ships: every ID distinct, each config exactly its rule’s filter, and nothing withheld left in.
• IDs from the string: A row’s ID is the first 16 hexadecimal digits of the SHA-256 hash of its string, so a string has the same ID in every version, whatever order the build happened to meet it in.
• Every figure from code: No number in the card, the report, or the site is typed by hand. A check re-derives every quoted figure from the committed records, and the repository’s checks fail if one drifts.

Hosting: Hugging Face, with a tag per version

The dataset lives on Hugging Face because it gives a dataset a free, versioned home. Each release is a tag (v1.0), every file is addressable at that tag, and the same address works from Python, from DuckDB’s hf:// paths, and from a browser. A dataset card and a Croissant metadata file describe it, in the 2 formats dataset tools already read.

The site: Astro, React islands, and no server

• Astro 7 and React 19, in TypeScript: Most of the site is documentation, which Astro renders to plain HTML at build time. Only the interactive parts (the composer, the profile, and the lookup) load React, and only on the pages that use them.
• A count cube instead of a database: Counting 10 million rows in a browser isn’t practical, and a server would cost money forever. So the build precomputes the number of rows for every combination of 12 columns: 10,174 cells, in a 325 KB file. The site answers any combination of criteria exactly by adding up cells. The composer above runs on a folded copy of it.
• Code that’s tested against the counts: The composer writes the same list as DuckDB SQL, Polars, and pandas, and a test runs all 3 on version 1.0’s files and holds each one to the cube’s count.
• Downloads without a server: “Download the list” reads only the columns it needs from the Parquet file on Hugging Face, by HTTP range requests (hyparquet), then zips the list, its datasheet, and the notices in the browser (fflate). In testing, the whole universe came out as a 30 MB zip in about 15 seconds.
• Static hosting: The built site is plain files on Cloudflare Pages, deployed with wrangler. There’s no backend, no database, and nothing to keep running.

The study kit and the checks

ceolkit is a small Python package, with Jupyter notebooks, that loads a pinned version and draws a stratified sample by a public rule, so 2 people who use the same seed get the same sample. Every pull request runs the pipeline’s and the kit’s tests (pytest), the figure check, and a link check on GitHub Actions, and the site has its own tests (Vitest).

And the English OpenList

The English OpenList, the list that started all of this, is still its own project, with its own site, repository, and nightly updates. The CEOL Universe holds a dated copy of it as one of its configs. I wrote about it in its own post.

Project links

• The CEOL Universe’s site
• Compose a list
• The dataset’s limitations
• The dataset on Hugging Face
• The code on GitHub
• The English OpenList