Foundations of Large Language Models

This course introduces the foundations of large language models, from language models and Transformers through pre-training, post-training, inference, advanced architectures, and agents.

The course is offered by the Master Vision Apprentissage and is open to MVA students.

• Hugging Face course organization
• Course material on GitHub
• Course Discord server

The preliminary plan for the first semester of 2026-2027 is:

• Session 1 - 1 October: Language models (David Louapre)
• Session 2 - 29 October: Transformers (Nathanaël Fijalkow)
• Session 3 - 5 November: Pre-training and scaling laws (David Louapre)
• Session 4 - 12 November: Post-training and alignment (Nathanaël Fijalkow)
• Session 5 - 19 November: Decoding and inference optimization (David Louapre)
• Session 6 - 26 November: Advanced architectures (Nathanaël Fijalkow)
• Session 7 - 3 December: Agents (David Louapre)
• Session 8 - 10 December: Advanced topics (Nathanaël Fijalkow)

All course material is maintained in the official GitHub repository. Available material includes:

• Slides: Transformers 101, Post-training, and Function Calling & Agents
• Notebooks: the corresponding hands-on notebooks
• Hugging Face organization: announcements and course resources