I'd note that the Fleuret books gives the foundation for basic deep learning whereas the OP seems to give the necessary foundations for Transformers and related machinery. These later addition what is now "AI" are in total at least as complicated as the original deep learning concepts. Not that just learning basic neural networks but the two book don't appear to cover the same ground.
Only had a quick skim, but looks like the content of this book has very little to do with transformers. Might be more relevant for diffusion models (image/video generation) though.