You know, I thought I had already weighed in on this one, but apparently I have not. I've already voted "no," but I feel that in many ways the question is just too simple. What do you mean by "closeted," for example? And what do you mean by "bothered?"
I don't consider closeted gay men to be any less worthy of life or happiness or respect than myself, but I think I'd have a hard time holding up a relationship with a guy who's in the closet, especially if he has a wife or girlfriend who's not cool with the whole idea. But that's in fact a separate matter. First, there's the issue of cheating (but hey, if she doesn't mind then neither do I) and second there's the matter of possibly using people. Even were he single, I think I might have a problem with any guy willing to fuck me but not willing to be seen with me.
On the other hand, do I mind a guy who has never disclosed his sexuality and refuses to answer when asked? No. Unless I intend to date him, it's not my business who he finds attractive. So a much simpler question to ask would be "Would you like to have dinner some time?" Or I'm actively trying to set him (or someone else) up on a date. In which case I'm meddling anyway, and strictly speaking it's still not my business.
Do I mind the guy who's always making gay jokes and trashing the queers to distract himself from how long his gaze lingers on his neighbor's johnson at the gym? Yeah, but it's his behavior that bothers me, not its cause. If I knew the cause, then the hypocrisy of it would also bother me, but not the basic fact that he's in the closet.
There are a few pet peeves I might have. If a closeted lesbian still refers to her partner of three years as her "roommate," I might feel mildly offended because she thinks I'm still fooled by the charade and that's possibly a little insulting. There was a guy I knew in High School who changed his sexual identity almost as often as he changed socks, and that kinda bothered me, but not so much because he kept jumping out of the closet and back in, as that he expected the rest of us to come along for the ride.
But my opinion on guys who just don't or haven't come out remains neutral. I remained in the closet until I was halfway through college, and it's worth noting that of all the gay guys I grew up with, I think I'm the only one who doesn't have a story about getting my ass kicked for it. These days I'm out at work and at home, but there are still a few members of my family we've decided (that is, my more immediate family decided a s group) ought to remain in the dark as a matter of convenience until I bring home a man I think I might keep to meet the relatives.
I'm fully aware of the price many gay and bi individuals paid for coming out, and I am thankful that I did not have to pay it.
Even so, sometimes it's a fight. Among my friends, my sexuality is more often the butt of jokes than that of others and when we weer all younger and stupider a few of us even came to blows on the matter. Some of my friends and family used to think being gay made me somehow frail or at-risk, and do weird (occasionally stupid) things to "protect" me. Whether they realized it or not, they respected me less.
Sometimes I get tired of it, and I'm certain I'm getting a much lighter treatment than some of the other people here. I don't really miss being in the closet, but I can see how someone who's not me could still like it there.
What does that theoretical closeted guy owe me anyway? Nothing, really. Or at least no more than any other human being. And by staying in the closet he loses from me pretty much the same things I would gain from him (the security of a larger queer community, a friendlier environment, and maybe a potential date) so it more or less evens out in the end. And besides, you don't need to be "out" to support the gay cause. Back when I was still in the closet, I left my old church after the pastor (whom I had previously respected) went off on a homophobic rant, and so did my brother (who's straight) - and I made it known that we wouldn't be back so long as that guy was delivering sermons.