I think we can probably all agree that certain Fathers consenting to a doctrine weigh more than others. Irenaeus and Justin Martyr give us, early on, the seed from which a couple of Marian doctrines could grow. In Against Heresies he states, “It was right and necessary that Adam be restored in Christ…that Eve be restored in Mary, so that a Virgin, become advocate of a virgin, might erase and abolish the disobedience of a virgin by her obedience as Virgin.” In reflecting on Mary as the New Eve, along with certain passages of Scripture (Gen 3, Lk 1), the Church was able to discern the truth of Mary’s Immaculate Conception.
"[Jesus] became man by the Virgin so that the course which was taken by disobedience in the beginning through the agency of the serpent might be also the very course by which it would be put down. Eve, a virgin and undefiled, conceived the word of the serpent and bore disobedience and death. But the Virgin Mary received faith and joy when the angel Gabriel announced to her the glad tidings that the Spirit of the Lord would come upon her and the power of the Most High would overshadow her, for which reason the Holy One being born of her is the Son of God. And she replied ‘Be it done unto me according to your word’ [Luke 1:38]" (Dialogue with Trypho the Jew 100 [A.D. 155]).
Thus, the roots of Mary’s perfect obedience have already been in discussion by this time.
Tertullian expands upon this concept as well: “It was while Eve was still a virgin that the word of the devil crept in to erect an edifice of death. Likewise through a virgin the Word of God was introduced to set up a structure of life. Thus what had been laid waste in ruin by this sex was by the same sex reestablished in salvation. Eve had believed the serpent; Mary believed Gabriel. That which the one destroyed by believing, the other, by believing, set straight” (The Flesh of Christ 17:4 [A.D. 210].
Ephraim the Syrian talks about this thought more clearly: “You alone and your Mother are more beautiful than any others, for there is no blemish in you nor any stains upon your Mother. Who of my children can compare in beauty to these?” (Nisibene Hymns 27:8 [A.D. 361]).
It starts to get more clear with St. Ambrose: “Mary, a Virgin not only undefiled but a Virgin whom grace has made inviolate, free of every stain of sin.” Ambrose,Sermon 22:30(A.D. 388),in JUR,II:166
One of the greatest theologians of all time, Augustine, had this to say: “Having excepted the holy Virgin Mary, concerning whom, on account of the honor of the Lord, I wish to have absolutely no question when treating of sins—for how do we know what abundance of grace for the total overcoming of sin was conferred upon her, who merited to conceive and bear him in whom there was no sin?—so, I say, with the exception of the Virgin, if we could have gathered together all those holy men and women, when they were living here, and had asked them whether they were without sin, what do we suppose would have been their answer?” (Nature and Grace 36:42 [A.D. 415]).
And again with Theodotus: “A virgin, innocent, spotless, free of all defect, untouched, unsullied, holy in soul and body, like a lily sprouting among thorns.” Theodotus of Ancrya,Homily VI:11(ante A.D. 446),in THEO,339
Or how about this? “As he formed her without my stain of her own,so He proceeded from her contracting no stain.” Proclus of Constantinople,Homily 1(ante A.D. 446),in ULL,97
So, in response to your claim that there is virtually no evidence to suggest Mary was sinless, I say bologna. I think there is a great deal of evidence in the pre-Nicene through the post-Nicene Fathers that Mary was indeed understood as sinless. It is true that we don’t have a lot written on the topic. Yet, I believe that is a good thing. In the Early Church, they had all they could handle to hold to the orthodox teachings on Christ’s nature and the Trinity. As we got that under control, we were able to reflect more on other parts of the Scriptures revealed to us, and gained better understandings there as well. This is called the development of doctrine, not the invention of it. If you can’t see the roots of the Immaculate Conception in Mary as the “New Eve,” then I suggest you take it to prayer. Let God reveal it to you. God bless!