Apparently the whole dislike of Joe Frazier was an act. During his ban and when a return was possible, but still a ways off, he met privately with Joe Frazier about the possibility of making a fight between them and Frazier offered to help him out financially. Whether this true or not I don't know but Ali was the best for a variety of reasons

@ipdg77 My understanding is that the entire thing started as a way to hype the fights, but Ali took it a bit too far. Joe Frazier resented Ali characterizing him as an ape, going so far as to bring a toy gorilla to a press conference and smacking it around, and he remained bitter until the day he died. It was the one blind spot in Ali's otherwise stellar life.

@zeuser Perhaps he did take it too far. But after the Thriller in Manila there had to be genuine mutual respect...surely?

@ipdg77 Here's a good summary:

[chicagotribune.com]

An excerpt:
"Frazier was the worst of Ali's bad guys, but Ali was the real villain. In three memorable fights, Ali callously referred to Frazier in appalling racial terms — specifically as a "gorilla" and an "Uncle Tom" — while portraying himself as the hero of the oppressed African-American. In Ali's narrative, Frazier was simply a tool of white America."

Some of his reasons and reasonable to me:
Viet Cong never called him 'nigger"
Viet Cong never sicked dogs on him or turned fire hoses on him.