My understanding is that the concept of celibacy comes from Mathew 19:12 -
“For there are eunuchs, who were born so from their mother’s womb: and there are eunuchs, who were made so by men: and there are eunuchs, who have made themselves eunuchs for the kingdom of heaven. He that can take, let him take it.”
Paul mentions in 1 Cor 7:32-35 “~ But I would have you to be without solicitude.** He that is without a wife, is solicitous for the things that belong to the Lord**, how he may please God.”
The Church Fathers wrote about it, such as St. Augustine:
Christian Doctrine
On Christian Doctrine (Book III)
25. … for example,** if he has embraced a life of celibacy and made himself a eunuch for the kingdom of heaven’s sake, he contends that the commands given in Scripture about loving and ruling a wife are not to be taken literally, but figuratively; and if he has determined to keep his virgin unmarried**, he tries to put a figurative interpretation on the passage where it is said, “Marry thy daughter, and so shall thou have performed a weighty matter.” Accordingly, another of our rules for understanding the Scriptures will be as follows,–to recognize that some commands are given to all in common, others to particular classes of persons, that the medicine may act not only upon the state of health as a whole, but also upon the special weakness of each member. For that which cannot be raised to a higher state must be cared for in its own state.
I believe that, if Peter’s wife was still alive by the time he had started his active ministry, they would have been living as “brother and sister”. Celibacy was a tradition in the early church, as St. Paul wrote in his letters, and was backed up by St. Augustine, St. Cyril and other church fathers. In 385, Pope Siricius issued the first papal decree on the subject, saying that “clerical continence” was a tradition reaching as far back as apostolic times.
A bit long winded, but I hope this helps a bit.