Back to Module 1: LLM FoundationIn Progress
AI Syllabus Module

Module 1.9: Self-Attention

Deconstruct dot-product attention steps, QKV matrices, and context calculations mathematically.

Lessons & Submodules

Submodules mapping coming soon.

Key Skills

  • •Compute Query, Key, and Value vectors from raw inputs
  • •Map scaled dot-product attention matrices mathematically

Interview Value

  • Explain how causal masking prevents models from looking at future token values during autoregressive generation.