I don't know your situation, so I will give example with myself, just for exactness sake. The point I am making doesn't care about personal history, so it applies to all of us, I just want the example to not be open to argument, which is why I'm using myself:
I came out at 25. I spent most of my life being distinctly aware that I was attracted to guys. As a kid, I just tried to ignore it. I only searched for pictures where there was ALSO a girl present, so I could delude myself that she had anything to do with the effect the picture had on me. I was ashamed of the prospect of being attracted to men, so I pushed it aside, ignored it, compartmentalized my attraction so as to make it not related to me in any way. Bullshit of course. But I grew up with this shame of who I was, this secret that horrified me, that had to be hidden from everyone and that I had to deny to myself because admitting it would make it true. I tried to tell myself that I also liked girls, I just hadn't found the right one. If I found her and ended up having sex with her, all of heterosexuality would pour itself into me.
It took until my 25th year to actually be able to admit I was gay. Anyone who thinks my life up to that point hasn't been DRAMATICALLY influenced by my sexuality, is, quite frankly, a moron. EVERYTHING I am is a result of the driving feelings and impulses I've had growing up. All my insecurities, all my relationships with people are COMPLETELY defined by the giant secret I've lived with for most of my adult life. I came out two years ago, and everything I've done since then has also been heavily influenced by my sexuality. My growth as a person in that period has everything to do with finding my true self, figuring out what mattered to me, what I wasn't ever again going to hide or deny because it would lower my self-worth.
Tell me anyone's life-story and I will tell you EXACTLY how being gay shaped most of who he is as a person today. Sexuality is one of the most primal, strongest driving forces in a person's life, especially in his formative years. And ESPECIALLY when it's alternative sexuality, which has so much social crap poured on top of it.
So yes, I thoroughly disagree with you that you are not defined by your sexuality. You might not want to be seen as "the gay guy" by society (though I wear it with pride, but to each their own), but being gay has shaped who you are, how you behave and how you interact with that society in a trillion different ways. Whether you're going to hide it or not is your own choice, but to try and deny that is burying your head in the sand.