The early Church recognized that there was something intrinsically unique about the Church in Rome:
St. Irenaeus:
But since it would be too long to enumerate in such a volume as this the succession of all the churches, we shall confound all those who, in whatever manner, whether through self-satisfaction or vainglory, or through blindness and wicked opinion, assemble other than where it is proper, by pointing out here the successions of the bishops of the greatest and most ancient church known to all, founded and organized at Rome by the two most glorious apostles, Peter and Paul, that church which has the tradition and the faith which comes down to us after having been announced to men by the apostles. With that church, because of its superior origin, all the churches must agree, that is, all the faithful in the whole world, and it is in her that the faithful everywhere have maintained the apostolic tradition" (Against Heresies 3:3:2 [A.D. 189]). (Emphasis added)
Optatus of Milevus:
“In the city of Rome the episcopal chair was given first to Peter; the chair in which Peter sat, the same who was head—that is why he is also called Cephas ‘Rock’]—of all the apostles, the one chair in which unity is maintained by all. Neither do the apostles proceed individually on their own, and anyone who would [presume to] set up another chair in opposition to that single chair would, by that very fact, be a schismatic and a sinner. . . . Recall, then, the origins of your chair, those of you who wish to claim for yourselves the title of holy Church” (The Schism of the Donatists2:2 [A.D. 367]).
Pope Damasus I:
“Likewise it is decreed . . . that it ought to be announced that . . . the holy Roman Church has been placed at the forefront not by the conciliar decisions of other churches, but has received the primacy by the evangelic voice of our Lord and Savior, who says: ‘You are Peter, and upon this rock I will build my Church, and the gates of hell will not prevail against it; and I will give to you the keys of the kingdom of heaven, and whatever you shall have bound on earth will be bound in heaven, and whatever you shall have loosed on earth shall be loosed in heaven’ [Matt. 16:18–19]. The first see, therefore, is that of Peter the apostle, that of the Roman Church, which has neither stain nor blemish nor anything like it” (Decree of Damasus 3 [A.D. 382]).