At the risk of sounding vaguely Freudian, I think sometimes our dreams act out desires that we have repressed in our waking consciousness. For example, I’ll sometimes dream that a dead pet is alive again or that a lost love has returned to me. While awake I don’t waste time and emotion wishing for dead pets back, and in the other case I emphatically do not want it to happen, yet I suspect there is some suppressed, irrational part of me that still desires both.
I would hazard to guess that there is some part of you that is still interested in other women besides your wife. I imagine this is the case with virtually every married man. Except perhaps in the most intense emotional phase of a young relationship, we don’t really lose eyes for anyone else. But we make a decision to be faithful, and that is what really matters.
If it’s possible to avoid the coworker you might consider that, but for the most part I would not worry about it. St. Augustine did think about the same problem, about whether there is sin involved in immoral dreams since one does seem to give one’s consent in them. Myself, I’ve experienced sexual dreams in which my will rebelled against what was happening and forced the dream to change, or at least that is how it seemed, leading me to suspect venial sin in cases when I did not rebel. But lucidity while dreaming is something that varies between people and between individual dreams, and even in a more lucid dream I doubt our control over intellect and will is ever sufficient to commit a mortal sin.