I think the misunderstanding comes from the word “privilege.”

True, but the entire lexicon was available, and whoever came up with "white privilege" knew exactly what they were doing. The word "privilege" is intended to evoke guilt and consideration (in the legal sense of a willingness to give something up as compensation for the privilege).

So, yes, there is an inherent bias in favor of certain groups. Now all we need is a name for that bias that doesn't make them sound like assholes. Instead of defending this obnoxious locution, how about suggesting something better?