Of course there is. It may not be morally justified, but it’s still logical.

Their logic isn’t “We need to better society so that all may live in harmony”, it’s “I must control as much as possible to stave off the inevitable nothingness of death”.

Assuming all things that are repugnant to you have no logic to them at all is an easy way to misunderstand and underestimate those who can do the most harm to you.