I did this too! For months (almost a year) I used descriptions, pictures, and measurements of food to get rough calorie counts. My diet is pretty simple and repetitive.
I would occasionally check the estimates, maybe once every few days for meals I wasn't already pretty sure of, and it was generally accurate. Where it was extremely inaccurate was on portions, and anyone who has dealt with computer vision could tell you, you can't get scale from a picture. So I'd have to weigh some meals or ingredients, which would generally make things more accurate again.
So, I think it's possible, but you need multimodal data and grounded with regular checks.
About once a week I ask ChatGPT to give me a reasonable diet for recomp with weight loss. It consistently insisted I have at least 7 meals consisting of at least 30g of protein per meal, but the protein source can't be whey or casein. When I ask "why" it cites a bunch of studies ... but most of those "studies" are N=1 of a college or Olympic level athlete. If, instead, I grab a large scale lateral analysis, it says "3 meals" with about 1/2 of the protein.
It'll defend both sides (mutually contradictory) to the death. NOTHING will budge it from its initial stance.