Partially. I think its fine to have that kind of thing. But not all the time. Bandits who are actually good people will avoid murder if possible. And while bad people can also have loved ones, that does not invalidate self defense.

Just as you said: Self defense is not murder-hoboing. If we are talking murder-hoboing then we should apply that list to city guards and commoners, who are not meant to be fought.