This is incorrect. There were heretics right from the start of the Church. Look at what Augustine said in the 4th century:"[T]here are many other things which most properly can keep me in [the Catholic Church’s] bosom. The unanimity of peoples and nations keeps me here. Her authority, inaugurated in miracles, nourished by hope, augmented by love, and confirmed by her age, keeps me here. The succession of priests, from the very see of the apostle Peter, to whom the Lord, after his resurrection, gave the charge of feeding his sheep [John 21:15–17], up to the present episcopate, keeps me here.
And last, the very name Catholic, which, not without reason, belongs to this Church alone, in the face of so many heretics, so much so that, although all heretics want to be called ‘Catholic,’ when a stranger inquires where the Catholic Church meets, none of the heretics would dare to point out his own basilica or house" (Against the Letter of Mani Called “The Foundation” 4:5 [A.D. 397]). (emphasis mine).
I’m not saying you don’t have a point, I just think we should be clear on this. I guess the difference is that nobody today disputes that these groups were all heretics, as the groups are no longer around. If, for example, the Lutherans were to cease as a church, then people in a 100 years time would probably call them undisputed heretics as well. And I’m only using the Lutherans as an example, I have nothing against them personally