AlphaZero is the successor to the famous AlphaGo algorithm that beat a champion Go player. Unlike AlphaGo, it requires no human games to initialize its training, and can learn to play other games, such as Chess. This talk will break down how AlphaZero works without assuming any prior reinforcement learning knowledge. It will also look at a recent extension, MuZero, that lifts one of the main limitations of AlphaZero: the need to have an explicit model of the environment. MuZero has achieved stunning results on various reinforcement learning domains, such as learning to play Atari games.