Lots of reasons:

The code is not portable between architectures.

You can’t actually define your data structure. You can pretend with your compiler’s version of “pack” with regrettable results.

You probably have multiple kinds of undefined behavior.

Dealing with compatibility between versions of your software is awkward at best.

You might not even get amazing performance. mmap is not a panacea. Page faults and TLB flushing are not free.

You can’t use any sort of advanced data types — you get exactly what C gives you.

Forget about enforcing any sort of invariant at the language level.

No defined binary encoding, no guarantee about concurrent modifications, performance trade-offs (mmap is NOT always faster than sequential reads!) and more.

Because a struct might not serialize the same way from a CPU architecture to another.

The sizes of ints, the byte order and the padding can be different for instance.