I think you’re making a little too much of this. There is an old saying that “a roof constitutes an introduction.” That referred to the roof of a private party–that is, no one else at the party of a friend ought to be considered a stranger–but to some extent it also applies to fellow parishioners. You are about as close as two stranger invited to the same party.
That does not mean by any means that you ought to trust everyone who belongs to your parish! No, it just means that you should accept that they ought to feel free to approach you without a third party to introduce you. Just as if you met them at a wedding reception, you have ever right to set your distance and let them cope with the boundaries you set.
In your place, I’d probably just smile and say I was not looking to meet any men at church–yes, you’re busy, but you didn’t really turn him down because you were busy, did you?–but “thank you” for thinking I looked like a person worth getting to know and that perhaps I’d see him around the parish. After the second incident, I’d smile and say, “Oh, we’ve met. I’m not looking to meet any men at church. Thanks, but no thanks!”
Still, I don’t think you did anything wrong. It was an awkward situation; those happen when people try to meet new people, sometimes. No harm done. I agree that looking for help with the issues that will interfere with your hope to get married and have a family ought to be a top priority. No matter what your vocation, you’ll want to get to work on that.