Just use cursor based pagination. How is this any different whether it’s infinite scroll or separate pages?
I guess the argument is that, while it's the same whether the user asks for "give me page 3" or "give me scrollbar Y coordinate 2160", the user is more likely to do the first or at least to care about the correctness of its result?
Cursor based pagination only works if the items are sorted in strict chronological order. Otherwise, the problem remains. Consider the list
where the brackets represent the first page. If the user clicks the next page button (asking for the next page after C), but meanwhile the order changed, they get this: Notice how they now see B twice and completely miss out on D. In constrast, infinite scroll will take the new infinite list and remove A B C, leaving D F E …