Back in the 4th and 5th centuries, the Catholic Church seemed to believe, as reflected in the writings of its brightest luminaries of those times, that relations with one’s spouse for the purpose of slaking physical desire was a venial sin. This teaching is found frequently in the writings of St. Augustin. The Church no longer teaches that.
(The question of whether the Church was wrong then and is right now, or was right then and is wrong now is a topic for a different thread.)
By my reading, the Church Fathers in the periods you mention did not teach anything contrary to what the Church says today.
**Church Fathers on the Goodness of Sex Within Marriage **
historyandapologetics.com/2015/08/church-fathers-on-goodness-of-sex.html
307 A.D. - St. Lactantius - “When God had formulated the plan of the two sexes, he instilled in them the desire for each other along with delight in intercourse. He therefore mixed into the bodies of all living things a most ardent [sexual] desire… Our adversary knows, then, how great is the power of this desire…and he causes it to change from something right and good to something evil and depraved…so that foreign [desires] contaminate the proper ones that are themselves sinless. … If someone cannot restrain these impulses, he should control them within the prescribed limit of a legitimate marriage. In this way he will attain what he eagerly desires and yet not fall into sin. … Certainly pleasure is a consequence of honorable works; if [married people] seek it properly, they are permitted to enjoy right and legitimate pleasure.” (Divine Institutes Book 6 Chapter 23)
Before 354 A.D. - St. Athanasius - “[There is a] lawful use [of the sexual organs] which God permitted when He said, ‘Increase and multiply, and replenish the earth,’ and which the Apostle approves in the words, ‘Marriage is honourable and the bed undefiled.’ … He is blessed who, being freely yoked in his youth, naturally begets children.” (Letter 48)
393 A.D. - St. Jerome - “[We] do not follow the views of Marcion and Manichaeus, and disparage marriage; nor, deceived by the error of Tatian, the leader of the Encratites, do we think all intercourse impure… We are not ignorant of the words, Marriage is honourable among all, and the bed undefiled. We have read God’s first command, Be fruitful, and multiply, and replenish the earth; but while we honour marriage we prefer virginity… Will silver cease to be silver, if gold is more precious than silver?” (Against Jovinianus Book 1 Paragraph 3)
Before 397 A.D. - St. John Chrysostom - “[T]he [sexual] company of your own wife…is no blame at all… Have you not heard Paul, saying, Marriage is honorable, and the bed undefiled?” (Homily 51 on Matthew 15:1 Paragraph 5)
418 A.D. - St. Augustine - “Marriage, therefore, is a good in all the things which are proper to the married state. And these are three: it is the ordained means of procreation, it is the guarantee of chastity, it is the bond of union. … In relation to them all the Scripture has this general praise: Marriage is honourable in all, and the bed undefiled. … [For] by virtue of [marriage] the conjugal embrace is not only allowable, but is even useful and honourable… [T]hose words of God, Be fruitful and multiply, are not prophetic of sins to be condemned, but a benediction upon the fertility of marriage. For by these ineffable words of His…God endowed the [original] pair with their seminal power. … [T]he chastity of virginity [was then] submissive to the power of the gentlest love.” (On the Grace of Christ and on Original Sin Book 2 Chapters 39-40)
.
