There are very, very few legitimate reasons to deny the marital debt. They all include grave physical reasons, e.g., the six-weeks after birth (and even this cannot always excuse, if there is no real cause for concern of the health of the woman), plague, infidelity (exceptions to this mentioned in the Summa:
newadvent.org/summa/5062.htm#article1), etc. If you are very serious about this, I will see if I can find the time to translate St. Alphonsus’s entire treatment of this issue in his
Theologia Moralis. Let me know if you’d like that or not. Since you already have a good grounding in the issue, it seems that a comprehensive approach would be best to clear up any particulars that you may find foggy.
For a start, I recommend the section on the marital debt from the supplement to the Summa:
newadvent.org/summa/5064.htm. I would add the caveat that articles 3 and 4 are not included in the official edition (the Leonine edition, which published the definitive editions of all the works of St. Thomas; as you probably know, the supplement was not actually written by St. Thomas himself but by one of his students after his death mid-way through the work). You don’t necessarily need to read all the objections, etc., but at least the “On the contrary” and “I answer that” for each article.
What’s more, the mediaeval position that it was not permitted to ask the debt during holy days was not usually followed after that time, especially because there is nothing in the law of the Church that forbids it, and it doesn’t appear to be a matter of divine law (the priests in the Old Law were actually directed to engage in the marital act on the Sabbath, for example). The mediaevals often also claimed that the marital debt was forbidden during Lent, though, again, this is not the position of theologians since the time of Trent (perhaps because Trent codified so many practices of the Church into law and teachings of the Church into definitive dogma that were simply commonly practiced or believed till that time).
Note well, however, that even though he says that it is not permissible to ask the debt during those times, he still holds that the other spouse absolutely must pay the debt when asked. There are a lot more cases where you must pay the debt than when you may or ought ask it. This is because even if the other spouse may be abusing his right, it is still his right. (This is why, for example–and I’m sorry to be so graphic on a public forum of all places–St. Alphonsus holds that one is bound to pay the debt even multiple times in one evening, even being woken up by the other spouse, to pay the debt up to as many as 3 times before the spouse can refuse the debt.)
In any case, this particular passage struck me as relevant regarding the nature of the marital debt. As you noted, it is not about the husband lording it over the wife. In the marital debt, as to the proportional equality, the spouses are equal. They have given themselves to each other in matrimony, and each has right over the other’s body, as St. Paul says and St. Thomas notes immediately before the following passge. (N.B. When he says that the husband is more noble regarding the marital act, he is answering an objection against the equality of proportion in the marital debt, for in any act whatever is the agent is more noble than the patient, and in the marital act, the husband is agent while the wife patient. So, while the two are equal regarding proportion, the husband is clearly more noble in the obedience he deserves from his wife and also in his role in procreation.)
I answer that, Equality is twofold, of quantity and of proportion. Equality of quantity is that which is observed between two quantities of the same measure, for instance a thing two cubits long and another two cubits in length. But equality of proportion is that which is observed between two proportions of the same kind as double to double. Accordingly, speaking of the first equality, husband and wife are not equal inmarriage ; neither as regards the marriage act, wherein the more noble part is due to the husband, nor as regards the household management, wherein the wife is ruled and the husband rules. But with reference to the second kind of equality, they are equal in both matters, because just as in both the marriage act and in the management of the household the husband is bound to the wife in all things pertaining to the husband, so is the wife bound to the husband in all things pertaining to the wife. It is in this sense that it is stated in the text (Sent. iv, D, 32) that they are equal in paying and demanding the debt. (Suppl., 64, v.)