I think that at the very least it should be retained as a symbol on papal coats of arms.
I would retain it for the very reason why people hate it-it is a symbol of absolute power, the only real absolute power, the power of the Keys.
We are sensual people, getting rid of the “baubles”, “smells and bells”, and other “flashy” things often works to kill piety and reverence. Should priests wear gunnysack chausibles (like they don’t already) because of this odd “WWJD?” attitude of liturgical/ecclesiastical minimalism?
Humility? Sometimes humility is better shown by accepting something you might not personally like for the sake of greater causes. I’m not a big fan of the “aw, shucks…” humility that gets rid of outward symbolism in order not to offend the sensibilities of people that hate us no matter what we do. If the popes started wearing sackcloth and ashes, there would be plenty of people calling them hypocrites and this, that and the other thing.