Introduction

These are my notes on Large Language Models. Below is a list of content I want to study.

• Andrej Karpathy’s video on micrograd. \(\rightarrow\) Done!

• Stanford lecture: “Building LLMs.” \(\rightarrow\) Done!

• 3Blue1Brown’s LLM series. \(\rightarrow\) Done!

  • First video.

  • Second video.

  • Third video.

• Andrej Karpathy’s makemore series.

  • First video.

  • Second video.

  • Third video.

  • Fourth video.

  • Fifth video.

• Mechanistic Interpretability.

  • Tutorial on TransformerLens.

  • Neel’s Glossary.

  • Chris Olah’s LessWrong article.

  • Cohere’s 1h intro to Mechanistic Interpretability.

  • Check out the ICML workshop.

• Stanford CS336 course:

  • First lecture/homework: basics.

  • Second lecture/homework: systems.

  • Third lecture/homework: scaling.

  • Fourth lecture/homework: data.

  • Fifth lecture/homework: alignment.