It was called "chattel" slavery for a reason. Under the law, “slaves” were “owned.” In a Platonic universe, people were enslaved by enslavers, but here on earth, an entire infrastructure “enslaved” enslaved people, whereas, within that infrastructure, one human being “owned” another. That distinction is worth preserving.

The essence of slavery, the essence of its wrongness, and the lessons of its ubiquity, all depend on legal ownership. By “rightfully” placing the “enslaver” with the “rapist and colonizer,” (a parallelism only an elitist who has never been raped could find useful), you make enslavement just one more thing that humans do to each other in the less than best of all possible worlds. “Owner” and “slave” are highly evocative words. Erase them at your anodyne peril.