I personally -as a Catholic- do not hold such logic.
Those whom I love the most I desire to be with and spend time with and speak with, the most. But this is not always possible- as you have communicated. However- with prayer - this is always possible.
It is possible that when I set myself to pray - I can pray to Jesus every time - can I not? And if I desire Jesus more than all - then to pray to someone
other than him would be to cheat myself of the most precious, blood-bought communion that exists; that between the bride and her husband - Christ and his Church. This only shows where my passions and desires truly are.
If the Church has given us permission to pray to Mary - then great. But does the Church force us to pray to her over our Savior - or is this a matter of personal practice? Where is it commanded from the Church that we MUST pray to Mary at all? Yet Christ has commanded us to pray to him - and to call on his name.
If you as a Catholic find yourself praying to Mary more than you pray to JESUS CHRIST - though God keeps no “score-sheets” it is plain enough whom you desire more. Yes I am Catholic - but it is time for Catholics to be real, and point our fellow brothers and sisters back to the feet of Jesus!