Nonstandard analysis [0] [1] uses infinitesimals but is still completely rigorous. I haven't ever really used nonstandard analysis myself, but there's a fairly well-regarded textbook available online [2].

[0]: https://en.wikipedia.org/wiki/Nonstandard_analysis

[1]: https://math.stackexchange.com/questions/51453/is-non-standa...

[2]: https://people.math.wisc.edu/%7Ehkeisler/keislercalc-06-03-2...

Various algebras of dual numbers are used in most automatic derivative routines.

This is treated more rigorously and generically in the subject of synthetic differential geometry.

There's no alternative that's significantly easier to understand and to use. The so-called "nonstandard analysis" hasn't caught on, because it's mostly the exact same arguments wrapped in slightly different language, not making them any simpler or shorter.

The language used by mathematicians is subject to constant evolution. 18th and 19th century results in analysis are not expressed and taught in the same way their original authors did it. Newton, Leibniz, Euler, Lagrange, Fourier, Riemann -- none of them expressed their results in terms of epsilons and deltas. These only caught on in the second half of 19th century, and they did so, because they were a better tool to rigorously prove the ideas.

New terminology inventions that make the subjects easier to understand take the field by storm. Some of the relatively recent examples are category theory, homological algebra, or, for that matter, the notion of sheafs, popularized by J.P. Serre himself. Mathematicians are very open to innovation, and intransigence is not the reason why we're stuck with epsilon-delta.

The reason is that nobody has yet come up with a better way of talking about these concepts. I repeatedly observe many people who seem to believe that their difficulty in understanding math stems from mathematicians gatekeeping their results. I think that this belief is just a coping mechanism. Mathematics is genuinely hard, and when people have trouble understanding something, it's easier to think that it's someone else's fault, rather than accepting one's own deficiencies.