If Jesus walked up to me and told me that the Pope was infallible, I assure you I would believe it. The comparison is not even close to the same.
Firstly - the Apostles did NOT agree on whether Gentiles should be bound to the Jewish law. That is why they needed a council. Peter spoke, and even those who disagreed with him listened and obeyed.
Not all the bishops agreed on Christ’s nature during the Arian controversy. So a council was called, again. The majority of attendees concurred with the then Pope that bishop Arius was wrong. Some, including Arius of course, didn’t. The disagreement of some of the successors of the Apostles didn’t mean that the issue was unresolved, it was resolved. It meant that those who disagreed with the Pope and those bishops who were in communion with him were wrong.
By the way, Jesus didn’t walk up to you and say ‘Matthew, Mark, Luke and John are
inspired scripture. In particular Matthew 18 reports my words exactly, so believe in them implicitly. The Gospel of Thomas, the Protoevangelion of James, the letters of Clement … well, some in the early church will consider them scripture, but they’re not, so feel free to ignore them.’
No. He said to the Apostles ‘what you bind on earth will be bound in heaven’, and more
significantly ‘he who hears you hears Me, he who rejects you rejects me.’ He gave them full authority to determine the Canon of that scripture you love so much.
Why on earth do you think that we can have a functioning church with a bunch of Appstles or bishops all holding equal sway and equal weight? Even if there were only two, they would be bound to disagree on something important. Result? Chaos. The split between Orthodoxy and Catholicism was chaos, as was the Reformation and everything that has happened outside of Catholicism since.
Jesus told Peter - no-one else - that Satan especially wanted his soul, that Perer had been specially prayed for, and that it was Peter’s role, no-one else’s, to ‘strengthen your brethren’. This clearly shows a very special leadership role for Peter and his successors that none of the other Apostles had - one where he was the final go-to person intimes of doubt or trouble, including disputes about the faith.