The biggest problem is that it's good if your images are all landscape or all portrait, but not when mixed.
What?
The defining feature of masonry is that it supports mixed aspect ratios. That's its whole thing. If you aren't mixing landscape and portrait images, you shouldn't be using masonry layout.
The whole point of a masonry layout is if you have different aspect ratios. Otherwise a masonry layout is just a normal grid.