It's rarely understood that infinity isn't something mathematicians made up to make things more complex, it's an abstraction that makes a lot of ideas vastly simpler.

This is alluded to in the article; it's challenging to prove a+b=b+a without infinity (though if you do modular/wraparound arithmetic it becomes straightforward).

It seems to me (not an expert in this area by a long stretch) that ultrafinite mathematics could basically be a branch of theoretical computer science in the sense that people seem interested in procedures to generate the numbers. In this regard, it's a bit surprising that TCS wasn't mentioned in the article.