Depending on your definition of poison of course, but at least for LSD and magic mushrooms, all that's known is a lower bound for their biological toxicity. For LSD IIRC it's somewhere in the 30mg (!) range, for mushrooms it's more than can comfortably fit a stomach. I'm afraid I don't have sources handy though.

That said, the state of mind must be so wild that "is it poisonous in this specific/usual sense" is probably not the right thing to discuss.