A Very Strange Defence

I'm involved in a very strange discussion with a believer right now. We're discussing Christianity as myth, and one of his retorts is that I cannot support my own worldview. He is referring to my atheism.
It's a strange line of apologetics. Always was. What he appears to suggest is that because I cannot categorically disprove the existence of his Deity, he can continue to stand squarely behind his own beliefs.
But can he? Let's consider the following. All religions make specific claims about how the world is. Sometimes these claims are vague, sometimes specific. Either way they can be subject to a degree of examination, and tentative conclusions about the likelihood that they are true. I'm unclear how I have to prove my position before rejecting his?
A word about atheism. It isn't a belief, nor a religion, any more than baldness is a hair colour. As Sam Harris remarks, it is no more than the noises people make in the face of bad claims. And be under no illusion, the claims of Christianity are bad claims. The Bible, it's primary source is brimming with forgeries, contradictions, and data that science has deemed inaccurate. It has proven itself untrustworthy, errant, and a poor guide to morality. It's core claims about Jesus are unsupported by any non biblical writings, and in many cases we see parallels with other faith systems. In short, one can reasonably conclude that the Bible alone provides ample reason to reject the Christian God. Yet I am still required to prove atheism? I don't think so, fella. To be sure, somewhere out there in the vastness of the cosmos may lurk beings of incredible power, but to the best of my knowledge we are not in contact with them. As such, I take the view that the claims of the religious remain unfounded, and I am perfectly entitled to reject them.