Pair programming
Each topic starts with coding exercises that you work through with a partner.
A technical AI safety curriculum, based on ARENA
We teach from ARENA’s open curriculum. Week 0 covers deep learning fundamentals, Week 1 mechanistic interpretability, Week 2 reinforcement learning and alignment science, and Week 3 is a capstone project.
Fundamentals
PyTorch, then CNNs, ResNets and a transformer you build yourself. It gives everyone the same footing before the safety topics.
Topics
- CNNs and ResNets
- Optimization and hyperparameters
- Backpropagation
- Transformers from scratch
How we teach
Open-ended work
Once the basics are in place, you pick one direction within the topic and go further, with teaching assistants on hand.
An example week
Full-time, Monday to Friday. The exact schedule will be confirmed before the program starts.
Mon
Pair programming
Tue
Pair programming
Wed
Pair programming
Thu
Open-ended work
Fri
Open-ended work
Tools
- Python
- PyTorch
- TransformerLens
- Gymnasium
- Weights & Biases
- Jupyter
- Git
Example capstone projects
Editing a stored fact
Find where a small transformer stores a fact, and check the finding by editing it.
Reward hacking
Train an RL agent on a slightly wrong reward, find where it exploits the gap, and measure how often.
Testing two explanations
Come up with two accounts of what a model component does, and design an experiment that tells them apart.