When I did toastmasters, there were no prayers.
It's very effective at making you a better speaker. Bear in mind that a lot of the people who run most of the clubs don't have experience at running or administering things, so be patient. With that said, it's also a lot of fun. I'd encourage just about anyone to try it.
I'm with Dwight on this one. As a military officer assigned to the Pentagon, with responsibilities for briefings to senior officials, I joined Toastmasters to clean up those irritating 'um's' and 'uh's.' Never once heard a single incursion of religion, however.