The above is basically a quippy way of describing oneself as an atheist from a Christian upbringing. I've been quoting it for years with the vague recollection that I'm quoting someone famous, but recently realized that I have no idea who. In its quippy knowingness it reminds me of G.K. Chesterton; but then, so do most things.
At least the way I use the phrase (whether this reflects the intent of the original or not), it means, like, someone who knows the stories from the Christian Bible and just doesn't believe they're true nor uniquely useful. As opposed to someone who feels the same way about Midrashic stories, or hadith, or Hindu stories, or whatever: those folks might say "I'm an atheist, but it's the (Jewish, Islamic, Hindu) god(s) in whom I don't believe."
(This is different from cultural Christianity or humanistic Judaism, which are basically celebrating the unique value of that particular religious milieu without actually believing in the religion. It's also different from the idea of positively rejecting a specific religion's God ("I could never worship a God who..."). It's more like in the movie Little Miss Sunshine, when Dwayne takes a vow of silence, so technically he's not speaking in any language, but honestly, the specific language he's not speaking is English.)
So the question is, what is the origin of this phrase/meme? FWIW, my brain is pretty confident in the inverted phrasing at the end of the quote: "...it's the Christian God in (whom|which) I don't believe." But I could also be completely wrong, and/or it could have been said in some other language and then translated.