Catholics are brought up to not only love Christ, and God, but also each other, the entirety of humanity. Some symbols the are a representation of negative acts or may cause spiritual harm are as in must Christian faith's frowned upon. We are also taught to be accepting and understanding to each others faith, as mislead as some may be.
Meditate and answer this; Q. If you had an anchor wrapped around your neck, and this anchor was the only thing causing your descent from the gates of heaven what would you do?
As Catholics we are all given the knowledge and resources for our own salvation, if if we make a conscious decision to ignore the guides and tools provided for our salvation, we are in effect condemning ourself.
If I were on that bus giving my knowledge and the history of the charm you wear around your neck I would not place judgment, for it is not my place to do so. I would want to make sure you understood the spiritual negativity that the charm represents.
If you decided to continue wearing it, you would now be consciously make a choice, by the power of free will bestowed upon us by our creator who will pass judgment when our day comes.