Am I ready to read AI papers?

I understand gradient descent.

• ○Derivatives and Gradients (not started)
• ○Gradient Descent (not started)
• ○Momentum and Adam (not started)

I understand backpropagation.

• ○The Chain Rule (not started)
• ○Computational Graphs and Autodiff (not started)
• ○Backpropagation (not started)

I understand embeddings.

• ○Vectors (not started)
• ○Dot Product (not started)
• ○Embeddings (not started)
• ○Word2Vec (not started)

I understand probability in language modeling.

• ○Probability and Distributions (not started)
• ○Probability of Sequences (not started)
• ○Language Modeling (not started)
• ○Autoregressive Next-Token Prediction (not started)

I understand cross-entropy.

• ○Entropy (not started)
• ○Cross-Entropy Loss (not started)
• ○Perplexity (not started)

I understand attention.

• ○Attention (not started)
• ○Self-Attention (not started)
• ○Multi-Head Attention (not started)
• ○Causal Masking (not started)

I understand the Transformer architecture.

• ○Positional Encoding (not started)
• ○Residual Connections (not started)
• ○Layer Normalization (not started)
• ○Feed-Forward Sublayer (MLP) (not started)
• ○The Transformer Block (not started)
• ○Encoder, Decoder & Encoder–Decoder (not started)

I understand pretraining.

• ○Tokenization (not started)
• ○Pretraining at Scale (not started)
• ○Scaling Laws (Preview) (not started)

I understand fine-tuning.

• ○Pretrain, Then Fine-Tune (not started)
• ○From Base Model to ChatGPT (not started)

I understand evaluation.

• ○Generalization, Overfitting and Underfitting (not started)
• ○Evaluation Metrics for Classifiers (not started)
• ○Data Leakage (not started)
• ○Sampling and Uncertainty (not started)

I understand RAG.

Taught fully in Chapter 12: Embeddings, RAG & the LLM Application Stack (not written yet).

I understand basic agent architecture.

Taught fully in Chapter 13: Agents (not written yet).

I know how to read experiments and ablations.

• ○Sampling and Uncertainty (not started)

Taught fully in Epilogue: Reading Research (not written yet).