I believe SVG is not accessible for visually impaired. Not sure what is the current status tho.
Still, the one-SVG-to-have-it-all might be an overkill for a web page. Both semantically and syntactically...