A plastic pellet is typically 3-5mm in diameter. I think I'd notice that in my food. Even if I did enjoy swallowing fish guts whole, a plastic pellet is just going to pass straight through my digestive system.

Additives can leach out of plastics and enter the food chain, but pellets lost at sea are a completely insignificant factor because the total volume of waste produced by this route is so small. The majority of marine plastic is either post-consumer waste dumped in rivers in developing countries, or fishing gear that is lost at sea. If you're really worried about this, then you really need to take it up with the government of the Philippines and the global fishing industry.

https://ourworldindata.org/ocean-plastics

Well, probably not the nurdles themselves unless they're scooped from the oceans and used as a food additive, but they'll break down into microplastics and enter the food chain that way. The damage of said microplastics is still being researched, at the moment (I believe) it's still fairly vague, not unlike asbestos or smoking. IIRC they have been found to mimic hormones though.