I think the excuse that it’s distracting is not valid.
I’d like to add my own *gentle *two cents on this question.
My church never did hold hands while I was in Catholic school, and I didnt see it happen until I joined the “adult” Sunday choir. The choir held hands religiously, but the congregation didn’t. About five years later when I came back to the Church, everyone in the pews did it, and it felt…trite and awkward. I feel left out of the uniformed structure if I don’t go along.
As far as the sign of peace, I find that it is distracting, simply because you go from a solemn moment to a greeting which is more cordial and something that would be better suited under the Liturgy of the Word, or during the opening blessings. I am not against it per se, but where it is in the order of mass.
Here’s the only analogy I can think of right now: you’re listening to a beautiful concerto and you get to the climax of the piece, (at the Mass you’re getting carried along in the solemnity, reverence, awe, majesty, and worship that accompanies this music). Suddenly, it all stops to the sound of a hurdy-gurdy.
It’s disjointed and a little jarring and although I love offering the sign of peace, it’s not where my attention should be. It should be that we are together focusing on Christ present - or about to be present - a the altar, not focusing on each other.