Oh, well, it's not something I'd be deterred by, but it's definitely something that needs to be taken into account in this scenario. I wouldn't be surprised if it's part of the reason the vigilante groups that pop up aren't as effective as they could be-Team Rocket, by all accounts I've heard, existed in this world before people from other worlds arrived, so they probably have some sort of stable native leadership in place. It's something of a natural advantage.