Just not going to agree with what you’re having an issue with here. It all stems from the idea, “treat others the way they want to be treated, as long as that treatment doesn’t interfere with the fundamental requests of others that are universal to the human experience.” Such as, if person A wants to go into a boxing ring and work out their stress for the day, that’s fine as long as there is someone volunteering to enter that ring with them. They have no right to force that request on anyone else.
I really don’t see where you’re not getting this. A “right” to me is the universal common request of a species that transverses any group identity and time period. Such as wanting to live, wanting community, wanting safety, etc. We draw a line in the sand for those “wants” and demand that they be taken as a “right”. How else do you learn what a “right” is from someone without asking them? You just assume that it comes from a list from a deity? or just that you know what your rights are so you assume they are the same rights for everyone else?
Slavery interferes with the rights of other people, just that culture A has a different understanding of the hierarchy of these rights than culture B. But neither culture denies that these are rights that everyone desires. So the pro-slave culture tries to dehumanize the people they enslave so that they can claim to not have to apply human rights to these humans. “They’re not human but sub-human.”, etc. Or “my slave, my right.” response that the religious genital mutilation community uses on boys and girls still today. We can understand where they have their hierarchy of rights wrong and have discourse about why we believe they have it wrong. Because you can’t know your list is wrong if you live in an echo chamber. But if they still don’t change, then force is required to make the change.