I dunno, I would have to say in football and hockey and lacrosse (and increasingly basketball), the “violence”, a.k.a. hard physical contact, is ancillary to the objective of scoring a touchdown, or a goal, or a basket, or what have you. It’s not the main point.
Wrestling (of the “real” variety), also the objective is not really to hurt your opponent so much as to pin him.
Now boxing and ultimate fighting I’m not so sure about. Maybe I’m willing to allow boxing and disallow ultimate fighting based on historical reasons. But for martial arts in general I tend to think that, as forms of self defense, they are fine, and sparring and such is required as part of training, but your intent should not be to hurt your sparring partner. So where the competition involves trying to inflict injury on your opponent, I think it’s immoral. But that’s just my opinion.