they are an invasive species

Thats mostly false in my experience, though that will depend on the locality i guess. On the contrary, a lot of garden plants are invasive in the local ecosystem