I think you would have to get much more clarifying information from the person who said it as to how it is selfish and why they think that.
Buying only certain brands is not inherently wrong. What might be wrong is the underlying reason for doing so, but that would be situation specific.