Not even close.
You've heard of reparations, right?
But you entirely miss the point. I am telling you what people are hearing, and you are telling me they are wrong to hear it. Maybe they are, but that's not their fault. You can say "Fuck you, no offense" all day long, but people will take offense.
You can't "explain" the privilege part of it and not make anyone feel like an asshole. That's how the species is wired, and, I repeat, I believe that the word was chosen for that purpose, whether or not the conditions you describe prevail, which I agree they do. The Monopoly metaphor is excellent sociology, but it doesn't make "privilege" the right word.