No, you certainly don’t have to. Just pull yourself back into focus when you become aware of your distractions.
St Thomas Aquinas, I think, said something beautiful about our souls being like a loyal dog which naturally sometimes wanders away from it master, but always returns to him when he calls. Can’t remember the exact quote, unfortunately.