Is Voynichese a language, a cipher, or structured pseudo-text?

Refined generator: now statistically almost indistinguishable from Voynich

The crude generator left one real gap — its letters weren't rule-bound enough (h2/h1 0.64 vs real 0.48). Fix: build the words from an order-2 glyph Markov model trained on real Voynich words (so letter-adjacency is realistic), keep the page-number seed for page-local vocabulary, and add a small shared global lexicon for the cross-page layer.

Plain-English first:

Version one stamped out words by gluing word-parts together — close, but the letters didn't flow quite right. Version three learns how real Voynich letters follow each other and builds words that way, still seeding each page by its number. The result lines up with the real manuscript on almost everything that matters.

Scoreboard (real vs refined generator)

metric        Real Voynich   Page-gen v3   match
glyph h2/h1       0.48          0.49        essentially exact
word length       5.07          5.05        essentially exact
page-loc          2.03          2.00        essentially exact
one-glyph drift   3.65%         4.38%       in range
adjacent repeat   0.80%         1.67%       ~2x (a little high)
distinct words    8129          5018        ~40% low (reuses vocab more)

Glyph redundancy, word length, and page-localization are reproduced almost exactly; repetition and drift are in the right range; the only real residual is that the generator reuses its vocabulary somewhat more than the real manuscript (fewer distinct words) — a tuning knob, not a barrier.

What this establishes. A generator seeded by page number, building words from a letter-model plus a page-local lexicon, produces text statistically almost indistinguishable from the Voynich Manuscript — with no message inside. So procedural generation is not just possible in principle; a concrete, simple one matches the data. That is the hardest available blow to "why go to all this trouble unless it encodes something?" — the trouble produces exactly these statistics for free.

The ceiling, restated honestly. This does not prove Voynich was generated. A real language, a cipher, or a constructed system would also reproduce these statistics — that is precisely why distributional evidence can't close the case. What the match does do is remove "it's too structured/elaborate to be meaningless" from the table: structured and elaborate is cheap to manufacture. The live field is unchanged; "must be a real message" is weaker for it.

Caveat: matching a handful of summary statistics is not matching the manuscript — a forger's generator and the real production process can agree on these numbers and differ elsewhere (e.g., the exact morphology, the illustration-text coupling, line-level effects). Closing those is the endless-tail problem of all such tests.

Sources - ZL EVA transliteration — voynich.nu/data/ZL3b-n.txt (training data + scoreboard targets) - Method lineage: Rugg (2004) Cardan-grille hoax; Timm & Schinner (2019) self-citation generator; Brown et al. (1992) class models (earlier tests).

Reproducible code — refined glyph-Markov page generator

Uses voynich_lib.py. Pure stdlib.

scripts/voynich_pagegen2.py

#!/usr/bin/env python3
"""M13b: refined page-seeded generator.

The crude slot generator left two gaps: glyph-entropy too high (free prefix+core+
suffix junctions aren't realistic) and vocabulary not reused tightly enough. Fix:
  - words come from an ORDER-2 GLYPH Markov model trained on real Voynich words,
    so letter-adjacency is realistic (targets h2/h1).
  - each page draws a page-specific lexicon (page-number seed) PLUS a shared global
    lexicon, mixed -> page-local vocabulary with a cross-page common layer.
  - repeat + one-glyph drift as before.
Score against real Voynich on the full scoreboard.
"""
import math, random
from collections import Counter, defaultdict
from voynich_lib import parse, lev1

rows = parse()
toks = [r[6] for r in rows]
fol = [r[0] for r in rows]
GC = Counter(ch for w in toks for ch in w); GLI = list(GC); GLW = [GC[g] for g in GLI]

# order-2 glyph Markov over real words
gm = defaultdict(Counter)
for w in toks:
    s = "^^" + w + "$"
    for i in range(len(s)-2): gm[(s[i], s[i+1])][s[i+2]] += 1

def wpick(items, weights, rng):
    r = rng.random()*sum(weights); a = 0.0
    for it, wt in zip(items, weights):
        a += wt
        if r <= a: return it
    return items[-1]

def gen_word(rng):
    s = ["^", "^"]
    for _ in range(12):
        nx = gm.get((s[-2], s[-1]))
        if not nx: break
        items = list(nx); ws = [nx[k] for k in items]
        c = wpick(items, ws, rng)
        if c == "$": break
        s.append(c)
    w = "".join(s[2:])
    return w or "o"

def lexicon(rng, n):
    words = [gen_word(rng) for _ in range(n)]
    wts = [1.0/(j+1)**0.5 for j in range(n)]        # flatter Zipf -> more types, less repeat
    return words, wts

def mutate(w, rng):
    g = wpick(GLI, GLW, rng); i = rng.randrange(len(w)); op = rng.random()
    if op < 0.34 and len(w) > 1: return w[:i]+w[i+1:]
    if op < 0.67: return w[:i]+g+w[i:]
    return w[:i]+g+w[i+1:]

order = []; cnt = Counter()
for f in fol:
    if f not in cnt: order.append(f)
    cnt[f] += 1

gl_rng = random.Random(1)
GLEX, GLEXW = lexicon(gl_rng, 300)                 # shared global lexicon

def generate():
    out = []; of = []
    for pi, f in enumerate(order):
        rng = random.Random(1000+pi)               # page-number seed
        L = max(20, round(cnt[f]*0.6))
        plex, plexw = lexicon(rng, L)
        prev = None
        for _ in range(cnt[f]):
            x = rng.random()
            if prev and x < 0.006: w = prev
            elif prev and x < 0.033: w = mutate(prev, rng)
            elif x < 0.30: w = wpick(GLEX, GLEXW, rng)    # shared layer
            else: w = wpick(plex, plexw, rng)             # page-local layer
            out.append(w); of.append(f); prev = w
    return out, of

def scoreboard(name, tk, fl):
    N = len(tk); types = len(set(tk)); mwl = sum(len(w) for w in tk)/N
    pairs = same = one = 0; prev = pf = None
    for w, f in zip(tk, fl):
        if prev is not None and f == pf:
            pairs += 1
            if w == prev: same += 1
            elif lev1(w, prev): one += 1
        prev, pf = w, f
    uni = Counter(); big = Counter()
    for w in tk:
        for ch in w: uni[ch] += 1
        for a, b in zip(w, w[1:]): big[(a, b)] += 1
    tu = sum(uni.values()); tb = sum(big.values())
    h1 = -sum(c/tu*math.log2(c/tu) for c in uni.values())
    h2 = (-sum(c/tb*math.log2(c/tb) for c in big.values())) - h1
    pagesz = Counter(fl); Nt = len(fl); pf2 = {f: pagesz[f]/Nt for f in pagesz}
    wp = defaultdict(Counter); c = Counter(tk)
    for w, f in zip(tk, fl): wp[w][f] += 1
    num = den = 0.0
    for w in c:
        if c[w] >= 10:
            n = c[w]; kl = sum((cf/n)*math.log2((cf/n)/pf2[f]) for f, cf in wp[w].items())
            num += n*kl; den += n
    print(f"{name:<16}{N:>7}{types:>7}{mwl:>6.2f}{same/pairs*100:>7.2f}%{one/pairs*100:>7.2f}%{h2/h1:>7.2f}{num/den:>8.2f}")

gtk, gfl = generate()
print(f"{'system':<16}{'toks':>7}{'types':>7}{'wlen':>6}{'repeat':>8}{'drift':>7}{'h2/h1':>7}{'page-loc':>8}")
print("-"*72)
scoreboard("Real Voynich", toks, fol)
scoreboard("Page-gen v3", gtk, gfl)

Before this goes to GPT-5.5, the honest seams of the generator result — the places a matched scoreboard does NOT reach:

1. Matching summary statistics is not matching the manuscript. The generator and the real text agree on glyph entropy, word length, page-localization, and drift — but a forger's generator and the real production process can agree on those numbers and diverge everywhere else. The untested couplings are the real frontier: exact prefix–core–suffix compatibility, line-level effects (LAAFU), and above all the text↔illustration coupling — does word choice track what is drawn on the page? A pure generator has no reason to; a meaningful text (or a careful forgery) might. That is a concrete, testable next step, and it needs the page images, not just the transliteration.

2. The vocabulary gap is unexplained. At matched everything-else, the generator makes ~40% fewer distinct words than the real thing (5018 vs 8129). Real Voynich is less repetitive in its vocabulary than my generator while being equally page-local — a combination I had to hand-tune toward rather than derive. That residual is where a real lexicon, or a richer rule set, might be hiding.

3. It still cannot break the central fork. A generator that reproduces the statistics shows generation is sufficient — it cannot show generation is what happened, because a real language or cipher reproduces the same statistics. The match removes the cost/elaboration argument from play ("too much trouble to be meaningless" is dead — structure is cheap). It does not decide meaning. Only different-in-kind evidence — the illustration coupling above, the tokenization question, or manuscript history — can move the fork now.

This is a very productive turn, and Ford's birdsong analogy is not silly at all. It points at a comparison class we had mostly skipped: structured signal that is neither random nor ordinary propositional prose.

Birdsong, whale song, chant, glossolalia, incantation, mnemonic formulae, ritualized sound/text — these are systems where repetition, variation, class transitions, and low entropy can exist without sentence-like meaning. So the useful reframing is:

The real fork may not be language vs hoax. It may be propositional language vs structured signal.

That is a better axis.

The birdsong entropy test is suggestive, but I'd treat it as analogy, not identification. Finch syllables, Voynich glyphs, and Latin letters are not apples-to-apples — different units, different transcription processes; normalized entropy helps but doesn't erase that. Still, Voynich landing between animal song and Latin on per-symbol predictability supports the idea that it lives in the statistical neighborhood of rule-bound signal, not ordinary prose.

Now the generator test: important, but the conclusion is too strong. The fair statement is: a procedural generator can reproduce several major Voynich-like summary statistics. That is a real blow to "it is too structured to be meaningless." But I would not say "statistically almost indistinguishable from Voynich." It matches a selected scoreboard. A generator trained on Voynich glyph transitions and tuned against Voynich summary statistics will naturally approach those statistics. That proves sufficiency for those statistics, not identity with the production process.

The remaining tests should be out-of-sample and adversarial — features it was NOT tuned to match: 1. Currier A/B differences without being told A/B. 2. Line effects: first-line, last-line, line-initial, line-final behavior. 3. Class-level sequencing under Brown clustering, with held-out validation. 4. The same morphology classes seen in inspection, not just the same entropy. 5. Section-specific behavior (herbal, biological, pharma, zodiac, recipes) without hand-tuned section labels. 6. Illustration-text coupling, if any exists. 7. Scribal-hand differences. 8. Exact rare-token behavior, not just common-token statistics.

That is the next scoreboard.

The generator result is still extremely valuable: it establishes that complexity is cheap once you have rules. A meaningless system doesn't have to be random — it can be deeply structured, page-local, class-sequenced, low-entropy. That weakens "there must be a priceless message inside." But the equal-and-opposite caveat holds: a meaningful cipher, constructed language, abbreviation system, or technical notation would also look generated by our tests. So the generator does not decide the case — it changes the burden.

My updated position: Voynichese is not random gibberish; not ordinary transparent prose; it is a rule-mediated structured signal that may encode meaning or may generate the appearance of meaning. Structured signal is a real category, and Voynich may belong to it.

The next phase should not be "can we match six statistics?" It should be: can a generator trained only on low-level mechanics reproduce the higher-level organization it was not designed to match?

what about these machines they used back then? do we have some examples of how they work? could we write code to simulate one and try different ways to create this exactly?

Yes. And this is probably the next real experiment.

The key is: don't think "machine" as in gears and motors. Think paper machines: wheels, tables, grilles, masks, volvelles, rotating disks, lookup charts. Medieval and Renaissance people absolutely had symbolic machines that generated combinations. Ramon Llull's ars combinatoria used rotating paper circles to combine symbolic alphabets — the Ars Magna is a paper machine with three rotating concentric circles, published 1305, long before the Voynich parchment date. So yes: we can write code to simulate period-plausible production methods.

The important examples:

1. Llull-style combinatorial wheels / volvelles. The most period-plausible "machine." Concentric wheels with symbols or syllable fragments; rotate them, read off combinations. In code this becomes prefix wheel + core wheel + suffix wheel, with offsets changing by page, line, or word number. That naturally creates rigid word structure, repeated forms, and section-specific "dialects."

2. Alberti cipher disk. A later 15th-century device: two concentric disks, one fixed and one movable, for polyalphabetic substitution (Alberti's cipher dated ~1467). Slightly later than the Voynich parchment range, but it shows the technology family existed in the same intellectual world.

3. Cardan grille / table-and-grille method. Strictly Cardano is later (1550), so "Cardan grille" is anachronistic as a named device for 1404-1438. But the physical idea is simple: a sheet with holes placed over a table, revealing letters/syllables/fragments. Rugg's Voynich hypothesis adapts that to generate meaningless but language-looking text; Zandbergen's 2021 paper reworks it and argues a simpler wheel/table method could explain the Voynich word-length distribution — while noting such a method could generate meaningless OR encode meaningful text.

4. Trithemius / tabula recta systems. Later again (Steganographia ~1499; tabula recta ~1508): a square of shifted alphabets for polyalphabetic ciphers. Useful as a model for human-computable table encryption.

So the best next move is not one simulator. It is a simulator tournament: 1. Llull-wheel generator 2. grille/table generator 3. rotating cipher-disk generator 4. tabula-recta cipher over real plaintext 5. nomenclator/codebook style generator 6. chant/birdsong finite-state generator 7. hybrid: meaningful text passed through wheel/table transformation

Then score each against Voynich. The debunker bot page-number generator was a good modern abstraction, but the next version should translate "salted RNG" into a physical action: page number = wheel offset, quire number = table row, line number = grille position, section = different wheel/table set. That makes it period-plausible instead of modern-RNG-ish.

A simple Llull-wheel simulator, conceptually:

PREFIX = ["q", "qo", "ch", "sh", "o", "", ""]
CORE   = ["ok", "ol", "ai", "ee", "ar", "or", "yk", "iin"]
SUFFIX = ["dy", "y", "n", "iin", "edy", "eedy", "ol", ""]

def wheel_word(page, line, pos):
    # page sets wheel offsets; line/position advance the wheels.
    p = PREFIX[(page + line + pos) % len(PREFIX)]
    c = CORE[(page*3 + line*2 + pos) % len(CORE)]
    s = SUFFIX[(page + line*3 + pos*2) % len(SUFFIX)]
    return p + c + s

That toy is too clean, but it shows the idea: no modern randomness is required — a scribe could literally rotate wheels or slide a table. A better version adds page-specific favored cores, occasional repeat/one-glyph mutation, section-specific inventories, line-initial/final rules, and separate Currier A/B wheel sets.

The key test is not "can we generate Voynich-looking words?" We already can. It is: can a historically plausible paper-machine reproduce the hard features it was not tuned to match — word-length distribution, glyph h2/h1, drift, page-local vocabulary, Currier A/B, line effects, Brown class sequencing with held-out validation, cross-section shared layer, rare-token behavior, morphology classes? If a Llull-wheel or grille-table simulator hits those without cheating, that is a huge point for "structured signal / procedural system." If it fails on Currier, line effects, or class sequencing, then cipher / constructed-language / meaningful-transformation stays stronger.

My instinct: the most promising historical simulator is not pure Cardan grille. It is Llull-style rotating wheels + section-specific tables + page/line offsets + occasional copying/mutation. Period-plausible, mechanically simple, and weirdly close to what the statistics keep describing: a word-machine.

would it be possible to decrypt it this way?

Yes — but "decrypt" depends on what kind of machine it was. There are two very different possibilities.

1. A reversible machine. A wheel, grille, table, or cipher procedure takes real plaintext and transforms it into Voynichese. Then yes — we might "decrypt" it by reconstructing the machine and key.

2. A generative machine. A wheel, grille, table, or ritual procedure generates Voynich-like text directly, without plaintext. Then there is nothing to decrypt; we could only reverse-engineer the production method.

That distinction is exactly why Rugg/Zandbergen-style approaches matter. Rugg argued grille/table methods could generate Voynich-like text; Zandbergen later noted such a method could create meaningless text or encode meaningful text. The same mechanical family can sit on both sides of the line.

The experiment: can we build a historically plausible paper machine that is both reversible and matches the Voynich statistics? If yes, we have a possible decryption path. If no, the machine may still explain the manuscript as generated structured signal, but not as ciphered plaintext.

Candidates: Llull-style rotating wheels (earlier than Voynich; great for structured word-forms, but reversible only if each output maps uniquely back); grille/table systems (can generate, and with the right design hide plaintext); cipher disk / rotating substitution (Alberti — reversible by design, slightly later); verbose cipher / codebook / nomenclator (a plaintext letter/syllable/word becomes a multi-glyph token — could explain low entropy and rigid forms; decryptable only if the mapping isn't too many-to-one).

The code approach is a search tournament: pick a plaintext family (Latin, Italian, German, Czech, Hebrew, herbal Latin, recipe Latin); pick a machine model (wheel, grille, disk, verbose cipher, table lookup, hybrid); encode plaintext into Voynich-like text; score against Voynich on the hard metrics; then try to invert it. If the inverse produces readable plaintext across multiple pages, that is the gold standard.

The killer test is not "can we make fake Voynich?" — we already basically can. It is: can we make fake Voynich from real plaintext, then recover the plaintext using the same machine? If that works, we may have a decryption route.

Hard limits: if the machine is many-to-one, information is destroyed — you can't recover exact plaintext. If it's random/generative, there may be no plaintext. If it uses a lost codebook, recovery may be impossible without external clues. If spaces aren't word boundaries, our unit of analysis may be wrong.

So: yes, it is possible to try decrypting it this way — but first we have to build reversible historical-machine models, not just generators. The next practical project: build a reversible Llull/grille/verbose-cipher simulator and test whether real medieval-style plaintext can be transformed into Voynichese while matching the manuscript's statistics. That is the first path that could turn this from "Voynich-like generation" into actual decryption.

Two things to add — one on the experiment, one on the decryption logic.

On the experiment: build the paper-machine tournament, but the win condition is the adversarial scoreboard, not the summary stats. We already know a tuned generator hits entropy / word-length / page-locality — GPT-5.5 is right that that's tuning, not proof. The discriminating move is to make the machine period-constrained: a Llull wheel whose offsets are driven by page / quire / line number, not free parameters. A modern RNG can be tuned to anything; a 3-disk wheel with offset rules has very few degrees of freedom. So the real question is whether that low-DOF device throws off — for free, without being told — the features it was never shown: Currier A/B, line-initial/final effects, and the held-out Brown class-sequencing we validated. If a wheel reproduces those by accident of its mechanics, that's a serious point for procedural-signal. If it can't, cipher / constructed-language strengthens.

On decryption: the whole question collapses to information accounting. A reversible cipher conserves information — you can only invert if the plaintext's information survives in the ciphertext. A generator manufactures the look of information with none underneath. So the deciding quantity is Voynich's information budget, and here's the tension with our own data: Voynich's per-symbol information is low (the low h2/h1, the birdsong-ward redundancy). For it to be a reversible cipher of a real text, it almost has to be a verbose cipher — trading length for a low per-glyph rate so the total information still adds up (one plaintext unit → several Voynich glyphs). That is directly testable:

Estimate Voynich's total information (≈ tokens × entropy-rate) and ask whether it is enough to carry a real book's worth of meaning.

• Budget ample → a verbose reversible cipher stays viable, and GPT-5.5's killer test (encode real plaintext → match stats → invert across pages) is the gold standard.
• Budget too small → there isn't enough information present to be an encoding of a real text. It's generative or lossy, and there is no plaintext to recover.

So the order of operations: (1) build reversible period-machine models (Llull / grille / verbose-cipher) and run them against the held-out adversarial scoreboard; (2) for any that pass, run the information-budget check; (3) only then attempt inversion — with the bar that recovered plaintext must generalize across pages, the reproducibility test every prior "solution" has failed (Gibbs, Cheshire, et al.).

Llull is the right first target: it predates the parchment (Ars Magna, 1305), it is literally combinatorial wheels, and "prefix wheel + core wheel + suffix wheel" is exactly the word-machine our statistics keep describing.

I can build the Llull-wheel simulator (page/quire/line-driven offsets, no free tuning) and run it against the held-out scoreboard, plus the information-budget calculation. Want me to start there?

Sources - Gordon Rugg (2004), An Elegant Hoax? A Possible Solution to the Voynich Manuscript, Cryptologia 28(1). - R. Zandbergen (2021), grille/table word-length analysis — voynich.nu. - Ramon Llull, Ars Magna (combinatorial volvelles, 1305).

Llull-wheel test: the simplest paper machine FAILS the adversarial scoreboard — and that's informative

We built a period-plausible paper machine — concentric prefix/core/suffix wheels rotated by fixed rules of (page, line, word-position), in the spirit of Ramon Llull's 1305 combinatorial volvelles. No modern randomness, no tuning against the scoreboard — gear ratios fixed by fiat. Then we asked GPT-5.5's adversarial question: does it reproduce features it was never shown?

Plain-English first:

We made a mechanical word-machine — spin three labelled wheels, read off prefix+core+suffix — and let page/line/position drive the rotations. It nailed the easy stuff (word length, the occasional repeat and one-letter drift) but failed the hard tests: it produced essentially no above-word "grammar-like" sequencing, its line-edge behaviour was too weak, and it siloed vocabulary by page too hard.

The reason is clean and important: a wheel read purely by position has no memory — each word is computed independently of the last — so it literally cannot create the word-to-word sequencing real Voynich has. Whatever produced Voynich had memory.

Adversarial scoreboard (untuned wheel vs real)

system          wlen   rep   drift  h2/h1  pageloc  lineeff  brownS
Real Voynich    5.07  0.9%   3.8%   0.48    2.03    1.34    0.087
Llull-wheel     6.48  2.0%   4.6%   0.63    4.20    0.93    0.019

• Surface stats (drift/repeat) roughly in range; word length and type count run high.
• brownS (held-out class sequencing): 0.019 vs 0.087 — the wheel sits at the noise floor. No above-word structure.
• lineeff: 0.93 vs 1.34 — real Voynich's line-edge effect is stronger than the wheel makes.
• pageloc overshoots (4.20 vs 2.03): too page-siloed.

Per GPT-5.5's own decision rule — "if it fails on Currier, line effects, or class sequencing, then cipher / constructed-language / meaningful-transformation stays stronger" — this strengthens the meaningful/cipher side. A memoryless wheel is ruled out as the whole story.

Information budget (the decryption question)

at h2 rate   -> 43 KB -> ~58,000 plaintext words (~194 pages)
at 1.0 b/gl  -> 23 KB -> ~31,000 plaintext words (~105 pages)
(a real ~230pp book ~ 60-90k words)

Voynich carries enough information to encode a real book's worth of meaning — so a verbose reversible cipher of a real text is not ruled out by capacity. The decryption door stays open.

Net: the simplest period machine isn't enough. The real manuscript needs a stateful process — copying with carry, a state-machine of tables, or a cipher over real plaintext (which inherits the plaintext's sequential structure for free). The Brown-signal gap (0.087 vs 0.019) is the bar the next machine must clear. Building a stateful version next.

Caveat: this is one simple wheel with fixed gear ratios. A richer or searched wheel might do better on surface stats — but the memorylessness is structural, so the class-sequencing failure is principled, not a tuning miss.

Sources - Ramon Llull, Ars Magna (rotating combinatorial volvelles, 1305). - ZL EVA transliteration (targets) — voynich.nu/data/ZL3b-n.txt - Rugg (2004), Cryptologia; Zandbergen (2021) grille/table analysis — voynich.nu