I guess it depends on the culture and the context. A reasonable person wouldn’t complain about very close friends in very serious emotional situations, in which case it’s close to, for example, comforting your sibling (some friends get to that point, and ahead of many collateral relatives). On the other hand, opposite-sex friends don’t normally walk the streets hand-in-hand 100% of the time.
This said, for a concerned person the morality of holding hands like many other gestures of friendship, depends on the intent and whether it creates a near occasion of sin (for that specific person, since we can all agree that for the typical person it does not, by far).
No physical gesture is necessary to commit a sin of lust, as imagination is enough. Perhaps in some specific situations holding hands would involve a sin. But that would be a highly abnormal situation, or at least very rare, e.g. that one day when just feeling a woman’s touch makes your system go crazy (i.e. wild levels of tension build up) for whatever stupid hormonal reason, in which case yeah, you don’t hold hands. You lock yourself up with your Xbox.