As a parent of young children I've found that I need my phone on any time my children are not with me. Calls from school or day care don't always come from the same number, so I answer every call when my kids are in the care of others (but none otherwise).
Why? What happens if you don’t have it, just as parents didn’t have them for centuries?
Then practice keeping your phone in your pocket for increasingly long periods of time. You may need to build up to this and to develop some level of control.