First, the fact that a doctrine develops over time holds no bearing upon its truth. Part of the basis of Catholicism is “This must be true because we have
always believed it, and God would not have allowed us to remain in error”. Obviously, that could not be said at the beginning of the Church, because there had not yet been an ‘always’.
As regards Protestant-accepted doctrines, there might be a problem: one of the fundamental tenets of Protestantism is that any doctrine which is not
explicit in the
Bible is a doctrine which may be considered but cannot be trusted. I.e., anything which developed later, such as the celibacy of the priesthood, will always be regarded with deep suspicion.
Regarding the papacy, the issue is not whether it existed, but what its role was, and particularly what its influence was outside of Rome. There were five patriarchs: in Rome, Constantinople, Alexandria, Antioch and Jerusalem. These cities were the centres of administrative secular power, locations of apostolic martyrdoms, and long-established churches. Each of these metropolitan (‘mother city’) churches had authority over the churches around it,
but not over the churches under another patriarch.
All too often, Catholic apologists confuse the acknowledged Primacy of the See of Peter, i.e., Rome’s status as “first among equals” of the five patriarchates, with a claimed
supremacy of the See of Peter. They quote comments from early Church Fathers about the significance of Rome, but read supremacy where only primacy exists: note especially the fact that, in the writings of
non-Latin Fathers, the references are to Rome being “first in love”, “first in honour”, etc, not “first in power”. As is clearly shown in the edicts and letters of the
Seven Ecumenical Councils, the Greek-speaking churches did not accept any Roman claim to power over them. In fact, they believed that it was the Councils whose word was to be obeyed, not any patriarch’s.
(I think that I might do a post on those Councils and the relevant portions, because those texts are quite long, and people seem to keep missing that important point.)
However, returning to the role of the papacy, it was ultimately the idea of the Supremacy of Rome upon which the unity of the Church was shattered. When the Latin churches inserted one word, “filioque”, into the Niceno-Constantinopolitan Creed, in order to prevent a heresy which was growing in Spain, the Greeks became upset.
From the Greek perspective, the Creed was sacrosanct: no one below an Ecumenical Council (and all five patriarchs were below them) should ever contemplate altering it. To do so was regarded as heresy

. Unfortunately, political manoeuvring
on both sides blew the issue up into a prideful contest of wills, and a challenge of authorities. When the Greeks saw the Bishop of Rome as claiming sovereign authority over all of Christendom, they vehemently rejected the claim, and the Church separated into those churches which did follow Rome, and those which did not.
It was, I think, one of the saddest moments in Church history.