Well...it would take a semester to explain it accurately, I’m afraid. Because it contains a fundamental question which is`where the Japanese language came from?`.
Roughly speaking, 外来語(loanwords) are written in katakana. But there are at least four kinds of loanwords in our modern Japanese language.
1. Simple exotic words (that nobody knows the Japanese words)
インク (ink), シリコン (silicon), マザーボード (motherboard), ハンバーガー (hamberger)
2. Japanized words (that many people realize they were originally foreign but altered, usually we don’t have appropriate translation : we call them 和製英語)
テレビ (television), シャープペンシル (mechanical pencil), ガソリンスタンド (gas station), サラリーマン (office worker)
3. Popular words rather than Japanese words (many people know the Japanese word but prefer the English word because it sounds simple: sometimes shortened)
プレゼント (present / 贈り物), ノート (notebook / 帳面), デパート (department store / 百貨店), ワイン (wine / 葡萄酒), ソファ (sofa / 長椅子)
4. Considered to be Japanese words (but actually not)
ミシン (sewing machine), ジグザグ (zigzag), 煙草 (タバコ / tobacco)
As you see, we use katakana for all the words above (except for tobacco). However, we actually have many kanji for foreign words which were imported in 19th century, such as 珈琲 (coffee) or 倫敦 (London). But we no longer use them mainly because it’s a pain in the ass. In a sense, we don’t “segregate” the katakana words.
Also, there are quite a few loanwords from China, Korea, Holland, or Portugal.
Did you know Tenpura, one of the most famous Japanese foods, was originally Portuguese word “tempero”?
Japanese language has a complex history. And it is still changing. I don’t know which direction it is going to. But being a member of this forum gives me a good opportunity to think about it.
(Sorry. Totally off-topic now...