They claim to be right because they believe in a fairytale? Believing ‘in nothing’ will always be closer to the truth than never questioning anything because you think you’ve got all the answers. Actually, ‘we know’ a LOT … and if there’s any part of it you don’t understand, you can find out. In their world, you can’t.

The argument from ignorance just makes god a placeholder (and rightly so). It reduces god to the avoidance of saying "I don't know". Then, when an answer is found, the "god" answer moves on to the next question.

Take medicine, for example, before and after the germ theory of disease was discovered.

Don’t worry about it and don’t invest your time in concerning yourself. Leave that to the mathematician and the physicist. If you want to explore how infinite regress resolves, I.e. when you face two mirrors to each other, you will probably lose all sense of functional existence. Life is for living, the rest is irrelevant!

If they are talking about, say the creation, then (sorry I will try to keep it short) list as many other unproven possibilities as you can, infinite multi-verses, limited multi-verses, universe an illusion, universe created by alien technology in another universe, universe goes round in a cycle big bang to big crunch and starts again, or big bang to big fade, at the end of the universe time goes backward and it keeps going back and forth, universe created by unknown deist god, and so on. Then when they ask you which one you believe give the honest. "I don't know, without evidence I can not." answer. Honesty usually throws them since it is not something they encounter in church. Then make your point that if you do not know you can not just drop god in with no evidence. You will not persuade them, because they have been taught to believe that they are given knowledge of everything because they are the special chosen ones of god, but you will be left with the high ground.