Belief is faith. You have here a negative belief: you're believing that a thing is not so -- that's the atheist position, that the proposition that there is some sort of deity isn't so. I suppose not believing in a broader sense of not bothering to believe anything about the subject could be construed as atheism, but I'd call it more properly agnostic. The religious believer and the atheist are two who have bought a product off the metaphysical shelf, so to speak, one labeled "I believe A" and one "I believe not-A".
Some atheists will say they didn't take either product, but that doesn't fit what they're claiming to be: a-theism is a positive assertion that there is no deity, and since there is no proof, a positive assertion is a matter of faith. Someone choosing not to take a product from the belief shelf would need a different tag -- "apistos" would come close, "not-believer", would work.
Do I believe that purple unicorns exist? No, but I have not chosen to proceed on faith and believe positively that they do not exist, which is the position of the atheist regarding deity. While I do no expect to ever encounter a purple unicorn, I would be delighted if I should -- and that shows forth the mark of the real atheist, that he does not hold out the prospect of delight at finding that there really is deity "out there" somewhere. If the proposition is "Purple unicorns exist" (PUE), I neither believe nor disbelieve PUE -- but the position of the a-PUEist would be believing not-PUE.
If you're calling yourself an atheist and defining it differently, fine for you -- just confusing for communication.