Come to think of it, I think a good deal of the non-Japanese I met when I was in Tokyo were there by accident, meaning that they hadn't aimed for Japan, but their company (Ikea was the only one I remember) had thrown them there for a few years. Likewise, these people were always moving around - I remember one of them had to go to Scotland for three months of training, then back to Japan. Another suddenly had to move to Hong Kong.
My Japanese friends (and Taiwanese and Chinese) always say I should move to Japan (or Taiwan or China) and that I can get a job easy. I always figure they haven't read up on immigration laws. I'm not sure if it's easier to find work in Taiwan or China, but I'll probably find out since I may be studying in one of them.
To the OP, I think it's better to keep an open mind about Japan - Tokyo is amazing but there's so much more, even in the relatively little I have seen. I think the Osaka-US comparison wasn't intended to say they are alike, but just that the little things like what you mentioned remind you a little too much of home? I can sympathise with that feeling but really it could happen anywhere... best to let it go I say.
I think the key is knowing people - especially if you're talking about a long-term stay rather than travelling. Of course friends are very important, and I was very much into Tokyo because all my Japanese friends were either from Tokyo or living there - and I was just travelling! But now I've met a lot more people from different parts of Japan, I have a very good feeling about seeing more of the country. Really, you could meet a good friend anywhere.
edit - since Gifu was mentioned
I met a girl from Gifu here a few weeks ago (typical timing on my part, she's been here a year and I meet her three weeks before she leaves - which she did this afternoon). Anyway, I wouldn't have even thought about visiting Gifu before, but she wants me to go meet her grandmother who has never met a foreigner before, and see a 'conservative Japanese family' (her best friend's words), next time I go over. And now that is something I really look forward to doing.
I guess the point is you don't realise what's out there until you go and find out.