Five stages, in order. Each one assumes the one before it, and every resource is free and clonable, so the whole path can sit in a single folder on your disk.
| Stage | What you can do after it | Start with |
|---|---|---|
| 1. Foundations | Read a training loop and say what each line does | Dive into Deep Learning |
| 2. Build a model | Train a GPT-class model you wrote yourself | LLMs-from-scratch, then nanochat |
| 3. Use models well | Fine-tune and prompt on purpose, not by guessing | Hugging Face LLM Course |
| 4. Agents | Build something that calls tools in a loop | Hugging Face Agents Course |
| 5. Read papers | Open an arXiv link and get the point | A venue’s orals, read together |
Every star count, file count and push date below came from the GitHub API on September 17, 2026. Those numbers move. What each repository is for does not, and that is the part worth choosing on.
Who this path is for
You write Python and you remember enough linear algebra to read a matrix multiplication without stopping. You have called an AI API and want to know what is behind it.
If you have never trained a model, start at stage 1. If you have, stage 2.
This is not a route into programming. Several of these stages hand you a file of math and expect you to sit with it.
How we picked
Four rules:
- Free, and readable without an account.
- Complete in itself, not a list of links to other people’s work.
- Clonable, under a license that lets you keep your copy.
- A known place in the order, so you can tell what it assumes and what it prepares you for.
A last-push date needs reading with some care. A repository that stopped changing is not automatically abandoned. A finished textbook stops getting commits. Two entries below are years old for that reason, and one candidate turned out to be genuinely dead. Both cases are marked where they appear.
Stage 1: Foundations
After this you can read a training loop and explain every line.
Dive into Deep Learning is the spine here: 29,632 stars, 1,027 files, 209 of them Markdown, 137 MB. The book is CC BY-SA 4.0 with a separate license for its sample code, so your clone is properly yours. It runs from linear regression through attention and transformers, with notebooks you can execute as you read.
Its last push was August 18, 2024. That is a finished textbook rather than a stalled project, and nothing about backpropagation has changed since. The later chapters on current architectures are where the age shows, and stage 2 covers that ground anyway.
Neural Networks: Zero to Hero belongs in this stage, with one thing worth knowing before you clone it: the repository is 9 files, one of which is Markdown. The teaching is in Karpathy’s video lectures, and the repo holds the notebooks that accompany them. It is the clearest explanation of backpropagation I have found, and it is also the one part of this path that does not live in your folder. Watch it next to D2L rather than instead of it.
After cloning D2L, try:
Read the attention chapter. Explain what the scaled dot-product
attention code computes, line by line, in the notation the chapter
uses. If I'm lost, tell me which earlier section to reread.Stage 2: Build a model
After this you have trained a GPT-class model yourself and can account for every file that went into it.
LLMs-from-scratch is the one to work through first: 105,143 stars, Apache-2.0, 358 files, last push September 17, 2026. Tokenizer, attention, the full model, pretraining, fine-tuning, in that order. It is the most heavily commented option here, which matters if stage 1 was hard work.
nanochat is the shortest complete pipeline: 58,088 stars, MIT, 53 files, last push September 7, 2026. Tokenizer through pretraining, fine-tuning and a chat interface, small enough to read end to end in an afternoon once the Raschka book has given you the vocabulary. nanoGPT (63,173 stars, MIT, last push November 12, 2025) is its smaller predecessor if you only want the training loop.
Stanford CS336 is the university version of this stage, and the org is easy to misread. The current run is in the unprefixed repositories: lectures (3,779 stars, 420 files, last push September 12, 2026) and assignment1-basics (2,768 stars, MIT, last push April 7, 2026). The spring2024- repositories next to them are the older offering. The lectures repository ships no license file, so keep your clone to yourself.
Once both are in the folder:
LLMs-from-scratch and nanochat both implement attention. Show me the
two implementations next to each other and explain where they differ
and what each author was optimizing for.Stage 3: Use models well
After this you can fine-tune a model on your own data, and recognize when you should not.
Hugging Face LLM Course is the largest body of maintained teaching material in this article: 1,444 files, 1,139 of them Markdown, 16.4 MB of text, Apache-2.0, last push September 17, 2026, which is the day we checked. Its 4,223 stars understate it badly, because people take the course on the website and never visit the repository. This is a good reminder that a star count measures attention, not substance.
Prompt Engineering Guide (78,412 stars, last push March 11, 2026) pairs techniques with the papers they came from, which makes it useful again at stage 5.
Generative AI for Beginners (119,941 stars, last push September 17, 2026) covers the same ground more gently if the Hugging Face course moves too fast.
I want to fine-tune a small model on [describe your data]. Find the
chapters in the HF course that apply, put them in reading order, and
flag anything that assumes a dataset shape I don't have.Stage 4: Agents
After this you can build something that calls tools in a loop, and explain why it breaks when it breaks.
Hugging Face Agents Course is the main text: 32,623 stars, Apache-2.0, 421 Markdown files, last push September 15, 2026. Current frameworks and MCP, with units you can run.
AI Agents for Beginners (74,989 stars, last push September 10, 2026) is shorter and lesson-shaped. It works well as a second pass over the same ideas.
You now have two courses in one folder that sometimes disagree about how an agent should be structured. Ask across both when they do.
Stage 5: Read current papers
After this you can open an arXiv link and get the point without opening three tabs of background first.
What makes papers hard here is rarely the reading. It is that a paper assumes you know what the last two years argued about, and no course hands you that.
The exercise that builds it: take one venue, read a full set of its orals, and write down what they have in common. We did this for computer vision. ECCV 2026 oral trends is the result of reading 26 oral papers, and the Gallery entry is the same material as a folder you can open and search, with paper summaries, a collaboration map and the source manifest.
Do it for a venue in your own area. It doubles as the test of whether stages 1 through 4 actually took.
What we left out, and why
Anthropic’s courses repository (22,841 stars) was on our list until we opened it. It is archived, and its notebooks teach against Claude 3 Haiku, a model Anthropic scheduled for retirement in April 2026. The prompting ideas in it are still worth reading. It does not belong in a sequence you expect someone to run.
Hosted video courses with no repository. Several are excellent. None of them can keep the promise in the title.
fastbook (25,305 stars, last push September 13, 2026) is a genuine alternative to D2L at stage 1, and better if you would rather train something before reading the theory. It is not in the spine because working through two textbooks at once is how people stall.
Keeping the path in one folder
The five spine repositories come to roughly 170 MB and about 1,800 Markdown files. That is more than anyone reads front to back, which is the situation where asking an agent beats browsing a directory tree.
Clone them into one directory as you go, and point whatever agent you already use at it. Claude Code and Codex both read a folder of Markdown perfectly well, and a folder of notes plus an agent is enough to get value from this on day one.
Two things get awkward as the folder grows. Grep only finds your exact wording, so a question about “switching costs” misses the chapter that says “lock-in.” And the PDFs of the papers you collect at stage 5 sit there contributing nothing to search.
How does StashBase help with a folder like this?
Disclosure: StashBase is our product.
- Open the folder and it prepares the files in the background: text extracted from PDFs and DOCX, OCR for scanned pages, local transcripts for recorded lectures. Keyword search then covers all of it, with no account or key required.
- Search by meaning finds the passage that used different words than your question. It stays off until you add your own OpenAI or OpenRouter key under Settings → Advanced.
- Pick Claude Code or Codex from the agent picker on a subscription you already pay for, and read the chapter beside the chat instead of switching windows. See the agent setup guide.
- Ask the agent to build a Wiki over the folder when the stages start cross-referencing each other. The pages are ordinary Markdown linked back to the sources, and your cloned repositories are left untouched.
The ECCV 2026 Gallery entry is a folder assembled this way if you want to see the result before building your own.
Start at the stage you’re actually at
The common mistake is starting at stage 3 because that is where the interesting work is, then discovering at the first fine-tuning failure that stage 1 was load-bearing.
Clone D2L and read the attention chapter. If it makes sense, you have saved yourself weeks. If it does not, you have found out cheaply, and you know exactly what to read.