What is GPT-2 and How it works in a Spreadsheet

In the fast-paced world of generative AI and LLM models, software engineer Ishan Anand has jammed GPT-2 into Microsoft Excel. This spreadsheet aimed to learn more about the concept of LLM. Even non-developers and non-technical people now can understand how the concept of LLM actually works in AI models like Chatgpt, Gemini, and others.

Anand calls himself a ‘Spreadsheet Addict’ and he said that if someone can understand Spreadsheet, they can easily understand AI. He said in his interviews that previously he was going to design GPT-2 into javascript, but later being fond of spreadsheets he came up ending with spreadsheets.

“By using a spreadsheet anyone (even non-developers) can explore and play directly with how a ‘real’ transformer works under the hood with minimal abstractions to get in the way,” said Anand on the official website for the sheet.

Also read: Copilot Pro vs. ChatGPT Plus

What is GPT-2?

GPT-2 is a LLM based model but the users cannot make conversations with it like ChatGPT. The reason for the same is that GPT-2 was designed previously before the chat era with AI models. The spreadsheet is available on GitHub in the form of an xlsb (Excel binary) file.

To access the spreadsheets of GPT-2, users need the latest version of Excel in their system.

The users need to enter the inputs into another cell and then they will see the predictive output in another cell of the spreadsheet. “Spreadsheets-are-all-you-need implements the forward pass of GPT2 (an ancestor of ChatGPT that was state of the art only a few years ago) entirely in Excel using standard spreadsheet functions.”- He said on its official site named ‘spreadsheets-are-all-you-need’.

“I walked away from Karpathy’s videos realizing GPT is mostly just a big computational graph (like a spreadsheet),” he says, “And [I] loved how Jeremy often uses spreadsheets in his course to make the material more approachable. After watching those two, it suddenly clicked that it might be possible to do the whole GPT-2 model in a spreadsheet.”- He added

How does GPT-2 work?

GPT-2 mainly works on ‘next-token prediction’, in which the transformer model completes the input with the most likely next part. The spreadsheet can handle up to 10 tokens and 10 characters per word limit, which is much less as compared to GPT-4 which can handle 128,000 tokens.  Although it is not that bad to start with.

Also read: xAI Recently Open-Sources Base Model of Grok

Anand said that-” This same Transformer architecture is the foundation for OpenAI’s ChatGPT, Anthropic’s Claude, Google’s Bard/Gemini, Meta’s Llama, and many other LLMs.”

Anand has also uploaded some YouTube videos for the users so that they can easily understand the concept of LLM with these videos.