Does anyone else see the difference in not believing in a god and believing there is no god?
Of course. Its called deism. Its belief in a deity without subscription to any dogmatic system or religion.
Yes. Much happier and a lot less guilt.
Yes, there is a difference. Believing that there is no god takes a certain amount of, for lack of a better word, "faith", in that the non-existence of god (or gods) is no more proven by empirical evidence than the existence of god. As an agnostic I do not believe in god, while an atheist believes that there is no god. These are the definitions that I usually go by. I have not doubt that some will disagree.
Agnosticism has to do with knowledge. Atheism has to do with faith. If you are agnostic then you don't know if there is a god. If you are an Atheist then you don't believe there is a god.