It seems likely that there are an infinite number of math problems but only a finite number of interesting ones.
Trivially false. Let P be the set of maths problems and I be the interesting subset of P. If I is finite, then there exists an element x belonging to P\I whose description is minimal among P\I. Then x is interesting. QED.
I think it really depends on what the universe looks like as you drill down into it. It seems like the further down into smaller systems you get, the more analytically complex it gets. And then there will always be more value in enhancing the generalisations you have.
Interesting is a bit in the eye of the beholder. Some people probably find maths boring full stop, some probably find all of it interesting.
That's a good question that can be answered by methods in mathematics.