> I gave it a very simple repository, said to write tests, and the result wasn't usable at all. Really wonder if I'm doing it wrong.

I think so. The humans should be writing the spec. The AI can then (try to) make the tests pass.