Returned to game...

Maybe word's gotten around that you have displayed Union tendencies in the past, so people are wary of you?

I think that is probably the most likely explanation.