The Ascension of Isaiah (c. AD 90, possibly a Christian revision of an earlier Jewish work) and Odes of Solomon (c. AD 125, sometimes bound with an earlier collection of Jewish songs) both speak of Mary as having given birth to Jesus without birth pangs. Since birth pangs were/are believed to be the result of original sin, this may be a reflection of an implicit or explicit belief that Mary was exempted from what we now call original sin.
St. Ephraim the Syrian wrote in Nisibene Hymns (c. AD 370), I assume addressing Jesus Christ, “You alone and your Mother are more beautiful than any others, for there is no blemish in you nor any stains upon your Mother. Who of my children can compare in beauty to these?” As an aside that one’s interesting for those Eastern Christians who dislike the idea of a “stain” on the soul, but let’s pass over that.
St. Ambrose of Milan, in Virgins (AD 377) described Mary in this way: “She was a virgin not only in body but also in mind, who stained the sincerity of its disposition by no guile, who was humble in heart, grave in speech, prudent in mind, sparing of words, studious in reading, resting her hope not on uncertain riches, but on the prayer of the poor, intent on work, modest in discourse; wont to seek not man but God as the judge of her thoughts, to injure no one, to have goodwill towards all, to rise up before her elders, not to envy her equals, to avoid boastfulness, to follow reason, to love virtue. When did she pain her parents even by a look? When did she disagree with her neighbors? When did she despise the lowly? When did she avoid the needy?”
St. Augustine of Hippo, in Nature and Grace (AD 415) wrote: “We must except the holy Virgin Mary, concerning whom I wish to raise no question when it touches the subject of sins, out of honor to the Lord; for from him we know what abundance of grace for overcoming sin in every particular was conferred upon her who had the merit to conceive and bear him who undoubtedly had no sin. Well, then, if, with this exception of the Virgin, we could only assemble together all the forementioned holy men and women, and ask them whether they lived without sin while they were in this life, what can we suppose would be their answer?”
In Falling Asleep of Mary (c. AD 550) the author, in describing the transfer of Mary’s body to heaven, described it as “spotless and precious”. Granted this is her body not her soul but like the idea that she did not suffer birth pangs the integrity of her body could be considered as reflecting the integrity of her soul.
The source for all this is The Fathers Know Best; Your Essential Guide to the Teachings of the Early Church by Jimmy Akin. The book contains a lot of other quotes concerning Mary but these are the ones that seemed most relevant to the Immaculate Conception to me.