They explain in the game, HL, that when you accio people, you are actually accioing their clothes.

Or it was levioso? I don’t really remember