I actually did my first confession in 30+ years with the priest that helped me come back to the Church quite a few years ago. Face to face of course, and I let it ALL out. Every bit of it and then some. Was it uncomfortable? Amazingly, no, not really, and in fact, I prefer it that way, but I can absolutely see why someone would go to a different parish, and no, I do not, as others have said, believe there is anything wrong with it at all.
With that said, for the past year or so, I myself have been going to another parish\priest to confess. Not really for “fear” of seeing my parish priest every few weeks, but because they offer 8am Saturday confessions which are very convenient with my busy schedule. My own parish has them Saturday only, 3:30pm, when I’m rarely available. (I also take advantage of Wednesday afternoon confession from time to time at a local retreat house).
Anyway, I would not feel bad about it. You are confessing to God, and He knows you mean well. However, again, as others have said, I think you can find a sort of comfort and growing in faith by seeing the same priest over and over again.