The bar for entry is surprisingly low, you just need to brush up on intro abstract algebra. I recommend the following:
1. Kaye, LaFlamme, and Mosca - An Introduction to Quantum Computing
2. Nielsen and Chuang - Quantum Computation and Quantum Information (The Standard reference source)
3. Andrew Childs's notes here [1]. Closest to the state-of-the-art, at least circa ~3 years ago.