I mentioned in a post a few days ago that when milkweed plants reproduce naturally such as mine have over the last three or four years, the plants grow in patches where they are very close together. It is much more difficult to keep the plants has free when they are so close together. Although I have been diligent about keeping beetles and aphids from reproducing on the milkweed plants, ants are hard to get rid of once they invade a patch. Because of their appetite for monarch eggs, their presence damages the monarch reproductive cycle.
I have had success keeping potted milkweed plants pest-free. That is a milkweed plant in the middle of the flower bed that is in the front of my house.