I am currently reading God Collar by Marcus Brigstocke. It's brilliant, hilarious, witty and heartfelt. I laughed out loud at several bits, especially the one about the turgidity of school assemblies, and the bit about the desire to fart in church and mix up the shoes in the foyer of mosques. I also agreed wholeheartedly with his views on The God Delusion.
The main premise of God Collar is that the author experiences a "God-shaped hole" but can't manage to actually believe in God (especially not the smitey "Old Testament" God), which is quite understandable - I can't manage to believe in a supernatural creator deity, especially not the smitey variety. I can manage a sort of mystical energy (but not a person) or maybe it's just an experience, like love - in which case, should we even call it God?
I was quite surprised at the degree to which the book focusses on the vengeful deity depicted in the early books of the "Old Testament", especially as the author acknowledges Karen Armstrong in the introduction, and Karen Armstrong has done much to bring biblical criticism to the non-specialist, and to point out how you get a different picture of God from different authors of the Bible.
The author wishes that there was a religion where his atheism would be respected, and that the religion was just about being nice to people. He quite likes Jesus but cannot see why Jesus' death is supposed to save anyone. He would like to have a religion that is compatible with reason and science, where the wonder of the universe as discovered by science is appreciated. He would also like a religion that does not consider all the other religions to be doing it wrong. And he would like a religion that doesn't persecute women and gays.
Well several such religions exist. There's Unitarianism (welcoming towards atheists and humanists since at least the 1920s, ordaining women since 1904, respecting other religions since it began in the 16th century, and welcoming LGBT people since 1970).
Alternatively, if you don't like Christian symbolism served with your religious smörgåsbord, try Paganism (LGBT-friendly, has priestesses and goddesses; respects other religions; though Pagans are less inclusive towards atheism).
And if you don't like singing, try the Quakers.
Or if you like meditation, try Buddhism (most Buddhists are non-theist).
None of these religions mind if you're an atheist, and they won't try to change your mind about it. They also acknowledge the validity of other religions, are welcoming towards women and gay people, politically left-leaning and environmentally friendly.