I'll share this experience here...it's long. (no pun intended)
JQ,
I know how this goes and how you feel more or less rejected and betrayed by people you liked and wished could get along with better.
While my situation is somewhat complicated and a little more involved, I'll share to show that even though it was just an off remark you overheard, it's a representation of a larger issue...lack of respect among your peers.
At my previous long term job, office politics were beyond ridiculous. Felt like I was in junior high school. Somehow, not by my doing I assure you, the subject of my penis became the topic of discussion. Women, gay guys, even the so-called 'straight' guys would be talking. Wanting to see it, wondering why they could never catch me in the main bathroom at the urinal hoping to catch a glimpse etc. Several female co-workers came to me and explained this is what even one of my bosses was saying. I thought it was rather odd that my dick would be the topic of discussion among 'straight' men...but whatever.
Fast forward to me becoming close friends with one of these 'straight' guys and him having a hot wife who expressed an interest in me to both of us that she was curious about me, wanted to "try me out". Her hubby thought it was hot and even initiated things. I fucked her while he watched but he kept his distance as she was one of those chicks who refused to suck him off or get anywhere near his cum. He ended up just wasting his load on his stomach and I had not yet developed a close enough relationship with them at this point to admit my cum fetish to them.
Now, he was already enamored with me before I was with his wife but after that he became even more interested, becoming more vocal about wanting to do more stuff in the bedroom, dropping hints that he wanted to try stuff with just me, without the wife. Eventually I caught on to his hints and what he was trying to do and just admitted to him that I would never do anything unless his wife was either involved or ok with it. He basically understood and took that as a rejection in me not wanting to mess around with him. TBH, I actually thought I was starting to develop feelings for both of them on different levels and I did want to mess around with him, but felt it wrong to do so behind his wife's back.
At this point the wife was starting to get suspicious and jealous of how much time we were spending together anyway and she was right in thinking there was something starting to develop with her hubby and I. She started treating me like 'the other woman'. Soon after I turned him down out of respect for her, he did not take that rejection well and turned it against me at work and it turned into a huge popularity battle. Pitting co-workers against each other. He started projecting things onto me and spreading rumors about me being gay or at least bi and saying that I wanted to have sex with him etc.
Turned out, many people in the office took my side, didn't believe him for the most part but also started coming out as bi themselves to prove that it was no big deal and that there was nothing wrong with it. Even people I wasn't friends with. The biggest issue was in all the talking and gossiping, the terms bisexual and gay were being used as derogatory slurs. To me, that was the most offensive thing about it all. The fact they were using those terms as an insult...I didn't know what to make of it, felt I was in the twilight zone. What hurt the most was these people WERE my friends or I at least considered them to be and I actually was developing feelings for them or at least felt a close bond with them. Although I didn't want to have sex with the guy, I would have messed around with him like he wanted provided his wife knew and was ok with it...given that I had already been with her. He wasn't about to come clean to his wife though about his level of curiosity with guys, and specifically his intense feelings towards me.
So the friendship fell apart, the office environment became extremely hostile and my solution was to just leave. The AVP of my division was a lesbian and once I put in for a transfer to a different department, she jumped on approving it quickly, understanding how detrimental the situation was becoming in affecting the quality of people's work, plus it was rife for a massive lawsuit waiting to happen.
It was much better when I moved to a new area, but then was soon faced with a hot, young supervisor who made cheating on her husband a recreational sport. She was already sleeping with another guy in the office who I admit I was hotter than by 1,000%. When she started hitting on me, making passes and dropping hints and expressing an interest in wanting to sleep with me, I politely turned her down as best I could and that made life hell for me in the office and sure enough she soon found reasons to let me go.
While I contemplated many times going to HR over all these things and even considered a lawsuit, I saw first hand what happened with others who did as they were put on a shit list and eventually let go, not even going to get into the few who attempted a lawsuit and failed miserably to their financial detriment.
My point JQ,
a toxic environment is not worth your mental health even for a good income. Not saying your office environment will turn into the circus that mine became, but it goes back to the feeling of dread and anger when you walk in that door, it's because that nagging feeling of disrespect and disgust that apparently some may have towards you will make you bitter, resentful and depressed about even being there, let alone trying to interact with them on even just a professional level.
"Don't shit where you eat"