Blogger Built CraftGPT in Minecraft — a Language Model Like ChatGPT, Powered by Redstone and Blocks

Players continue to discover ever more inventive ways to use redstone.

The blocky sandbox Minecraft offers abundant opportunities for creative expression. The creator behind the YouTube channel sammyuri set out to assemble a compact language model.

The dimensions of his “small” model, CraftGPT, measure 1020 × 260 × 1656 blocks. He clarified that, because of this scale, he had to use the Distant Horizons mod for filming, so some elements may look a bit odd (redstone is rendered with reduced detail).

For training the model, sammyuri used Python and employed the “TinyChat” dataset, which contains basic English conversational data.

The model “has a vocabulary of 1920 tokens and consists of 6 layers. Its context window is 64 tokens, which is adequate for (very) short exchanges.”

Source: iXBT.games