Track

Physical AI from scratch

From one neuron to world action models, one interactive class at a time.

55 classes · 8 modules · 1 open

M1Neurons and learning

  1. 01One neuron, by hand: weights, bias, step vs sigmoid, binary cross-entropy, one gradient step by the chain rule, and why one neuron can't learn XOROpen
  2. 02Hidden layers and backpropagation: an MLP learns XOR; gradient descent on the loss landscape and the learning-rate limitSoon
  3. 03Derivatives as a graph: the chain rule on a computation graphSoon
  4. 04Backpropagation I: build a tiny autograd engine (micrograd style)Soon
  5. 05Backpropagation II: train a small MLP with your engineSoon
  6. 06Layers and hidden units: width, depth and universal approximationSoon
  7. 07Classification: softmax, cross-entropy, and digits on CPUSoon

M2The deep learning toolkit

  1. 08Tensors and PyTorch autograd: the same ideas, at scaleSoon
  2. 09Optimizers: SGD, momentum, Adam, learning-rate schedulesSoon
  3. 10Generalization: overfitting, regularization, dropout, normalizationSoon
  4. 11Convolutions: filters, receptive fields, a small CNNSoon
  5. 12Embeddings and sequences: tokens, positions, why RNNs struggledSoon
  6. 13Attention from scratch: queries, keys, valuesSoon
  7. 14The transformer block: build and train a tiny GPTSoon
  8. 15Training in practice: data loading, mixed precision, checkpoints, debugging on 6 GBSoon

M3Generative models

  1. 16Autoencoders and latent spacesSoon
  2. 17Variational autoencoders and the ELBOSoon
  3. 18Diffusion I: adding noise and learning to remove itSoon
  4. 19Diffusion II: train and sample a small DDPMSoon
  5. 20Flow matching: straight paths from noise to dataSoon
  6. 21Conditioning and classifier-free guidanceSoon
  7. 22Tokenizing images and video: patches (ViT) and discrete codes (VQ)Soon

M4Robots and data

  1. 23Robot crash course: frames, rotations, forward kinematics [L24, L25]Soon
  2. 24Jacobians and inverse kinematics [L26, L27]Soon
  3. 25Trajectories and control loops [L20, L28]Soon
  4. 26Cameras and depth for robots [L30, L31]Soon
  5. 27Teleoperation: leader and follower arms on the SO-101Soon
  6. 28LeRobot datasets: episodes, frames, recording your ownSoon
  7. 29Action representations: joints vs end effector, chunks, frames of referenceSoon
  8. 30Simulation and sim-to-real with orbesimSoon

M5Imitation learning and RL

  1. 31Behavior cloning and compounding error [L33]Soon
  2. 32ACT I: action chunking and the CVAESoon
  3. 33ACT II: train ACT on a LeRobot datasetSoon
  4. 34Diffusion PolicySoon
  5. 35Evaluating policies: rollouts, success rate, honest statisticsSoon
  6. 36RL foundations: MDPs, returns, value, policy gradients [L34]Soon
  7. 37PPO in orbesimSoon
  8. 38Mixing the two: RL fine-tuning of imitation policiesSoon

M6VLMs and VLAs

  1. 39Contrastive vision-language learning: CLIP and SigLIPSoon
  2. 40Vision-language models: vision encoder plus language modelSoon
  3. 41From VLM to VLA: actions as outputs [L35]Soon
  4. 42SmolVLA I: the architecture and the action expertSoon
  5. 43SmolVLA II: fine-tune on your own SO-101 dataSoon
  6. 44π0 and flow-matching action expertsSoon
  7. 45GR00T: a fast and a slow systemSoon
  8. 46Deploying a VLA: latency, async inference, the real armSoon

M7Video world models

  1. 47What is a world model: from World Models to DreamerSoon
  2. 48Video diffusion: latent video models (Wan and friends)Soon
  3. 49World foundation models: NVIDIA CosmosSoon
  4. 50Synthetic robot data from video models: DreamGenSoon
  5. 51Latent actions: learning actions from video (LAPA, Genie)Soon
  6. 52Evaluating world modelsSoon

M8World action models

  1. 53DreamZero: predicting video and actions togetherSoon
  2. 54Dream-1 dissectedSoon
  3. 55Capstone: train a tiny world action model end to endSoon