About TorahGPT

Two tiny transformers, about 800,000 parameters each, trained from scratch on a laptop: TorahGPT on the Hebrew Torah of the Leningrad Codex, and KJVGPT on the King James Bible. They run entirely in your browser. Expect scripture-flavoured nonsense.

How they write

Each model has learned one thing: given the text so far, how likely each possible next character is. To write, it picks a character at random according to those odds, adds it to the text, and repeats, one character at a time. Nothing else guides it. There's no meaning behind the words, only patterns learned from the text.

Temperature changes the odds. Lower values stick closely to the likeliest characters, so the models repeat familiar phrases; higher values give unlikely characters more of a chance, and the text gets stranger.

The texts

TorahGPT learned from the five books of Moses in the Westminster Leningrad Codex, as published by the Open Scriptures Hebrew Bible project: 304,850 letters in 5,853 verses. The vowel points and cantillation marks were removed, leaving only the letters, as in a Torah scroll. Divine names are written with the substitutes customary in Orthodox writing, such as ה׳ and אלקים, so TorahGPT never saw the names themselves.

KJVGPT learned from the 31,102 verses of the King James Bible, from the thiagobodruk/bible collection.

About a tenth of each text was held back from training and used to check that the models were learning the style of the text rather than memorising it. TorahGPT rarely reproduces more than a few words of the Torah at a time; what it writes is new, in the Torah's vocabulary.

The lengths

The three length options are the traditional counts of the Torah's commandments.

The Talmud (Makkot 23b) pairs the 365 prohibitions with the days of the solar year, and the 248 positive commandments with the limbs of the human body.

Translation

TorahGPT's English comes from the free MyMemory translation service. English you type into TorahGPT, and the Hebrew it writes, is sent to MyMemory to be translated; nothing else leaves your browser. Machine translation tends to smooth nonsense into fluent-sounding English, so the Hebrew is always the more honest of the two.

The design

The colours come from the curtains of the Tabernacle in Exodus 26:1: blue, purple and scarlet on fine twined linen. TorahGPT writes in Frank Ruhl Libre, a revival of a Hebrew book face from 1908, and KJVGPT in IM Fell English, a revival of 17th-century English printing types. The large ב on TorahGPT's opening screen echoes the enlarged letter that traditionally begins a Torah scroll.

Code

Everything, from preparing the texts to training the models, is on GitHub.