Skydio · Anduril · Dedrone · Apptronik · 20 weeks
The 20-week path from "I can run YOLO on a webcam" to "I ship robust perception on a robot." Targets perception roles at the top autonomy companies, where the bar includes multi-view geometry, sensor fusion, and real-time deployment.
Image formation, multi-view geometry, classical features.
Pinhole + distortion, what camera calibration actually computes.
You will read camera_info.yaml every day. Know what every field means.
Pre-deep-learning detection. Still used inside many SLAM stacks.
ORB-SLAM3 is built on classical features. You cannot debug it without knowing them.
How two cameras give you depth. The math behind stereo and SfM.
Hartley & Zisserman chapters 9-12 are the canonical reference. Heavy but worth it.
Detection, segmentation, depth, NeRF, transformers.
CNNs, attention, modern training tricks.
The Stanford course is still the best free intro. Lectures + assignments take ~6 weeks if you do them properly.
YOLO for speed, transformer-based detectors for quality.
YOLOv8/v9 is the workhorse on real robots. DETR variants are the research direction.
Per-pixel labeling for scene understanding.
SAM 2 (Meta, 2024) is the default for new pipelines. Mask R-CNN still ships in legacy stacks.
Stereo block matching, monocular depth networks (MiDaS, Depth Anything).
Stereo is the reliable workhorse; mono is the cheap option that's gotten surprisingly good.
Neural radiance fields, Gaussian splatting, novel view synthesis.
The hot research area. Real production use is still emerging but worth being fluent for senior interviews.
Camera × LiDAR × IMU — what separates demo perception from production.
How to align camera + IMU + LiDAR coordinate frames.
Kalibr is the open-source standard. Reading the source is half the job.
Project LiDAR points onto images, project image features into 3D.
Autonomous vehicles have done this for a decade. Robotics is catching up.
TensorRT, ONNX, real-time pipelines, edge inference.
NVIDIA's optimizer for running models fast on Jetson.
Senior perception engineers can convert a PyTorch model to TensorRT in a day. Junior engineers can't. Be the senior.
Convert between PyTorch, TF, JAX, edge runtimes.
ONNX is the duct tape that holds modern MLOps together for vision.