The question is why do we call our Pope the Holy Father.
One might ask instead: why do we call the Church “holy” (as in, “One, Holy, Catholic, and Apostolic Church”). Because the answer is the same.
The Pope is our “Holy” Father even though he is (always, more or less) a sinful man, just as the Church is always “holy” even when She is filled with (nothing but) sinners.
The Church (like the Pope) is “holy” because the Church has a holy origin and ministry (as pronounced by Jesus). Just like Peter. It was Peter that satan desired to sift like flour, and it was Peter for whom Jesus prayed in particular, and it was Peter who would lead the Church on earth.
Peter is to the Church as Joseph was to the Holy Family, and the Pope is heir to this very same Office and ministry.
I doubt many Christians would deny that St. Peter could rightly be called “holy,” even though his first words to Our Lord included the prohibition, “Depart from me, Lord, for I am a sinful man.”