Build a Large Language Model (from Scratch) by Sebastian Raschka

What people said (7)

• LatencyKills · 2026-06-06 in How LLMs work

  I have a BS in CS (and have been in the field for 25 years). I couldn't understand the transformer architecture until I built a few myself. Here are the books I worked through. I now feel I have a very good understanding of modern LLMs. https://www.amazon.com/Build-Large-Language-Model-Scratch/dp... https://www.amazon.com/Build-DeepSeek-Scratch-Abhijit-Dandek...

• LatencyKills · 2026-06-06 in How LLMs work

  Not OP but I worked through Sebastian Raschka's "Build a Large Language Model (From Scratch)" [0] and Raj Abhijit Dandekar's "Build a DeepSeek Model (From Scratch)" [1] books. I don't think there is anything in a transformer I couldn't explain in the smallest detail now. [0]: https://www.amazon.com/Build-Large-Language-Model-Scratch/dp... [1]: https://www.amazon.com/Build-DeepSeek-Scratch-Abhijit…

• JoeDaDude · 2026-05-05 in Train Your Own LLM from Scratch

  Coincidentally, I just started on Build a Large Language Model (From Scratch), a repo/book/course by Sebastian Raschka [0][1][2]. Maybe it is a good problem to have to have to decide which learning resource to use. [0] https://github.com/rasbt/LLMs-from-scratch [1] https://www.manning.com/books/build-a-large-language-model-f... [2] https://magazine.sebastianraschka.com/p/coding-llms-from-the...

• auraham · 2026-02-26 in I don't know how you get here from “predict the next word”

  I highly recommend Build a large language model from scratch [1] by Sebastian Raschka. It provides a clear explanation of the building blocks used in the first versions of ChatGPT (GPT 2 if I recall correctly). The output of the model is a huge vector of n elements, where n is the number of tokens in the vocabulary. We use that huge vector as a probability distribution to sample the next token gi…

• encyclopedism · 2026-01-20 in Scaling long-running autonomous coding

  We know what an LLM is in fact you can build one from scratch if you like. e.g. https://www.manning.com/books/build-a-large-language-model-f... It's an algorithm and a completely mechanical process which you can quite literally copy time and time again. Unless of course you think 'physical' computers have magical powers that a pen and paper Turing machine doesn't? > Many people are throwing aroun…

• js8 · 2025-10-16 in Writing an LLM from scratch, part 22 – training our LLM

  It's based on a book https://www.manning.com/books/build-a-large-language-model-f..., is it a good book?

• ryanchants · 2025-10-10 in Ask HN: Build Your Own LLM?

  I'd get it straight from Manning and save a few bucks and take out the middle man: https://www.manning.com/books/build-a-large-language-model-f...