I can see that being my problem since in the places I have been to I actually do seem to be a local. I also happen to stay only for a couple of weeks max and that doesn't help.

One curious thing I have seen is that some ex-pats don't want to meet other ex-pats from their home country. I tend to agree with that since not doing so decreases your chances of meeting new local people which can be really interesting.