That’s actually a very complicated issue, especially when added to your comment about castes.
Some groups people self identify as, and can remain quiet about as desired, e.g. a Country & Western fan.
Other groups that are identified by appearance can not really be hidden, although some choose to dress in a way that identifies with a certain group, which is interesting.
But to be part of a group by birth and immediately be held in higher or lower regard in society than others, via wealth, assets or caste/ class is intriguing. The people that judge others in that way as are interesting too.
Personally I judge a person by how they act and live, but of course you only know so much about any person. Going to Uni with a rich alcoholic who kept writing off nice cars with personalised number plates taught me not to respect people just because they have money. Knowing someone who didn’t want to go home and run their family estate made me realise the freedom that having nothing gives you.
Sure labelling groups can lead to issues, but it’s human nature. Better to develop an understanding and try and get along, when we can, I’d say.

I don't really buy into names when I can avoid it, like I'm not a member of a political party, I like to keep my option open. Still group names are useful, like I need a Doctor, so I can Google a list of Doctors.
The real danger isn't the group names it's the amount of personal identity people invest in those groups, as we see in the MAGA crowd. Pure mental illness or intrinsic fascism, which it is is still a bit of an argument.