There’s definitely a reason for hate crimes to be punished differently. I used to agree with what you just said, but then I noticed what the bigger issue was.
A murder/mugging, etc. is usually part of organized crime. A Mafioso whacks one of his stool pigeon drug dealers. The murder is still wrong, but the drug dealer put himself in the situation by his involvement with the mafia.
A hate crime is convicted against a random person. Matthew Shepherd and James Byrd were both minding their own business, and were tortured by violent sadists for no reason.
A gang jumping or mugging that results in death is motivated by money. Same with a gang war. But a hate crime is pure psychopathic sadism at the root, usually committed against someone who was unfortunate enough to cross paths with the sadist. It is far worse.