I used an old Motorola phone for this, and yes, if it dies it won’t turn back on again until the power button is pressed. I Googled around and there’s a way to disable this behavior though, through an ADB command. The bad part is that it supposedly might get stuck in a boot loop – it tries to boot but there’s not enough battery yet, so it dies and keeps trying to boot. Over and over.

I made a second Tasker automation, so it shuts down with less than 15 percent battery. It might still get stuck in a boot loop, but eventually the solar panel will quickly charge it above 15% so that it won’t be for very long.

When the phone is "off" but showing the charging animation, it's actually booted and the animation is a program it's running. There used to be a hack, I don't know if it works on modern Android, where you'd essentially edit the init scripts and tell it that the charging animation task should be the rest of the boot process, or your specific app of interest.