All the above answers are very good and well stated. There is also a historical development. For centuries secular priests were not called Father and in some countries they are not called Father. The title Father came into popular use in the religious orders of monks where the superior of the monastery was called Abba. Today we call him the Abbot. His house is called the abbey.
When the mendicants (Franciscans, Carmelites, and Dominicans) were founded they too called their superiors Father. For example, in my Franciscan tradition we still refer to the superior as Father, even if he’s not a priest. Francis of Assisi is always called Holy Father or Seraphic Father. The priests who joined the mendicants and the monastic orders were called Brother. The secular priests were called by their secular titles. In the USA we called them Mr. until the late 1800s. In Italy they are still called Don (Sir). That’s how we get saints such as Don Bosco. Don is not his name, but his secular title.
Gradually, many orders were clericalized, meaning that the number of priests was larger than the number of lay men. Rome changed the rules for some orders. It required that the superiors always be priests, even if they had both lay and clerical brothers. Gradually, the title Father became associated with the priests, because they were the superiors and when their term was up, they left the post to a successor, but they retained the title, Father, just as the early founders have retained the title Father.
Over the years, priests who were also religious were known as Father, except for the Franciscans, Carmelites and Dominicans. They always called their members Friar. When the Jesuits were founded, they were religious, but they were not monks or friars. They were the first religious order of men who were founded as a clerical order. They were all called Father, because that was the title for the priests who ran the religious houses. It was the expansion of the Jesuits that also expanded the use of the title Father for priests who were not religious superiors.
Now, why were religious superiors called Abba (Father). The tradition begins in the Old Testament with the Patriarchs. They were called Father. These religious superiors were the patriarchs of their communities. That’s how the custom carries over from Judaism into Christianity.
In the early 20th century America, many religious orders were asked to take over parishes, because there was a shortage of parish priests. As the number of parish priests who were secular priests grows and replaces the religious, the laity continued to call them Father. Gradually the title Mr. is lost and replaced by Father. The laity did not know the difference between a religious and a secular priest. When a Carmelite superior left a parish and was replaced by a secular priest, that priest simply inherited the title, Father. Secular priests gradually organize themselves around bishops and are bound to dioceses. Eventually the title secular is replaced by diocesan. Today we know them better as diocesan priests. Once the secular identity of the diocesan priest disappears into the background, not in Canon Law, but in daily life, so do the secular titles such as Mr., Don, Dr., Lord, Sir, etc. They all become Father, just like the religious superiors.
Today, there is a strong movement within the Benedictine and Franciscan families to de-clericalize, strip away the false impression that the priesthood is essential to the religious life. Many monasteries of Benedictines (Benedictines, Trappists, Cistercians, Camaldolese and others) and many provinces (regions) of Franciscans have gone back to the title Brother. My own region uses Brother. Father is reserved for the elected successor of our Holy Father Francis. I believe that with time, we will see the monks and mendicants called Brother, except for the superiors who are the patriarchs of these communities. Father will probably stay with the rest of the priests in the Church.
It’s interesting to note that in the Anglican countries (not the Anglican Church) bishops are still called Doctor (especially in Ireland and Canada). We Americans refer to them as Bishop. In the French, Spanish and Italian cultures Bishops are still called My Lord (Monsignor, Monseñor, Monsignore)
The titles are not a doctrinal or biblical phenomenon as much as a cultural one.
Fraternally,
Br. JR, OSF