Not true-use the word Jewish. Thats not offensive. Believe me they have offensive words available. The Jews have been targets of anti-Semitism for generations.

I commented on the first part of your question, somewhat, in a reply below. As to hating all religions, well, I do "label" myself as an antitheist, so I'm not a fan, but it is more their effects than the absurdities of the religions themselves that I abhor. Also, I think one has to be very careful to separate the religion from the religious. Most folk are born into a religion/culture & a lot of their life & society may be tied up in it & its practices. People are people & some of these guys are atheists in the making. It doesn't make sense to be hostile & make enemies (more than atheists already have).

Perhaps some people are more primed or inclined to taking offense and will often seize any opportunity to bed offended?

This is one of those subtle points I think you may have just missed. ...kind of like when Ross Perot referred to "You people" and then was baffled that people took offense (I'm showing my age with that reference from 1992). The Offensive part is not so much in the word "Jew," but in how you added "A" before it, like an item rather than a person. Did you even notice you didn't do that for "Muslim" or "Christian?" It makes a difference. Both those words also are easily understood as adjectives and not just as nouns, whereas the appropriate equivalent adjective is "Jewish," not "Jew."

People can be ethnically Jewish or may just be of that religion. I cannot say for certain, but I think calling a member of either group a "Jew" is just inconsiderate and/or tone deaf.

I think it is only offensive when Borat says it

I've never talked to a Muslim person before. I have only seen them.

I've never had that experience. Being anti-religion hasn't ever gotten me called anti-semitic.
Granted, it's gotten me called a lot of things, that's just never been one of them.

1. You created a false equivalency. Muslim:Muslim not equal to Jewish:A Jew.
2. Connotation, and in vocalization, inflection, mean a lot. How do you not know that?

Half of my family is Jewish (I'm not). It isn't offensive in polite conversation.

That is not my experience and observation. I've never known a Jew who is offended by being called a Jew. I can imagine that if one is intending to be hateful (anti-semitic) it will come across as hateful (anti-semitic) and not as simply conveying a cultural fact. That's another thing, while Judaism is indeed a religion, as my Jewish boss, who is non-religious said just a few days ago, being Jewish is a culture with or without practicing the religion..

I've never noticed it, but I've never really heard someone called "a Jew" unless they were using it in a disparaging way. I've mostly heard the term "she's Jewish" or "this Jewish friend of mine". My sister is Jewish, and she never worries about what people call her, she's pretty cool about religion in general.

I think the offensive use goes back much further, to when jews were the money lenders and so not liked.

I think it's people are way to sensitive. I will tell you a true that happened to some friends of my mine who play a black metal bsnd from Seattle named Inquisition they have been around for close two decades now and have toured the world and play hateful music aimed at the Christian faith. In 2012 the released a song entitled "Crush The Jewish Prophet" and were all of sudden tagged with being Nazis! Don't people know Jesus was a Jew!? I really think certain words just trigger the asshole in some idiots. Made no sense to me. Anywho that's my experience with that.

Context. Words themselves are not offensive. Their use becomes offensive when, in context of all our known history, they were used to be offensive in the past.

Recently, a player in my footy team called an opponent a very racist slur. He tried to defend it when I spoke to him after the game with "it was not meant to be racist but as a sledge". For me, intent does not excuse the slur. I had to kick him off the team.

Context gives words meaning.

"Jew" is technically the correct for someone who is Jewish. But the word "Jew" has also accumulated a ton of baggage through the years in ways that the words "Christian" or "Muslim" have not. The word "Jew" has been widely used by antisemites as a pejorative, so it's no surprise that some take immediate offense. I tend to say that someone is Jewish and not "a Jew." But it should not be offensive in the right context.

Is it offensive in all contexts? I wouldn't think so.