After I wrote that post I was wondering why I feel that (some) loan words sound dumb in English, but English loan words seem almost essential in Japanese. I think the reason is that in Japanese the loan words actually have different nuances than the Japanese equivalents.
For a related example:
If you were to say "ブラー" on a film set, it has nothing to do with the current camera's settings; ブラー means a blur effect added in post-production (when Hollywood's special effects software first came to Japan it was all in English, and when it was finally localized they simply katakana-ized the "blur" effect
The name of the effect became ブラー and blurring done with a camera remained ボケ[る]).
I can think of similar differences in nuance for just about all loan words in Japanese. ミルク came up in this thread--ミルク is never (hardly ever?) used as a substitute for 牛乳. You can't walk into a grocery store and find a carton of ミルク (unless it's part of a name). ミルク is only used to describe foods that have had milk added (ミルク入り缶コーヒー, etc.), whereas 牛乳 is used for the drink (milk). In English we don't differentiate between these two types of milk (a difference that is created by usage of the item), but that doesn't mean it isn't a valid use of the loan word to create a distinction.
To address astaroth's examples...
Anko-pan doesn't exist in the western world, so the Japanese name is the only name it has (even if you say "it's just bread with anko in it," what is anko???).
Panini is specific type of sandwich, so that makes sense. (I'm not sure about Italian, but in French "Panini" doesn't simply mean "any small sandwich")
Using the word "latte" for "just milk" is dumb... but I've never heard that before
ボケ already exists very much in the western world, and the Japanese word isn't used to differentiate from the English word, which makes it a different case from many of the English words on loan to the Japanese language.