The starting point of casting a horoscope is calculating the apparent locations (this means "where you would have seen them had you looked up there and then") of a whole bunch of celestial objects at the time and place where a particular person was born.
You won't (write software to) do that without knowing a whole bunch of linear algebra and ODEs.
The fortune-telling part is not what needs the math degree.
Does the quality of the astrology depend on the correctness of celestial object positioning? I always assumed it had more to do with the "bedside manner" of the practitioner, and less to do with any measurable falsifiable phenomenon.