In moral terms, you're right, but in legal terms your argument is meaningless.  It's pretty settled in precedent that the government cannot require a citizen to stick to every belief his or her professed religion asserts, it can only assess whether or not a belief in question actually is a belief of that religion (or specific church body).
And again, it isn't about the people, it's about a particular action.  To use one of your examples, it's irrelevant if a customer is a member of NAMBLA, but it would be relevant if that NAMBLA member requested a cake celebrating sex with the pre-pubescent.  The former is already settled law:  you can't discriminate on the basis of a customer's personal beliefs.  And in fact you can discriminate on the basis of some customer actions; for example, if I were to walk into an Office Depot and start singing Christmas carols and playing my guitar I could be asked to desist or leave, or if I were to enter a Safeway grocery with a pet I could be asked to leave.
So the question isn't whether a business can discriminate; it's almost always been ruled that they can't.  The legal question here is whether an individual can be obligated under threat of government force to act contrary to the free exercise of his religion.  And since he can't, under the First Amendment free-exercise clause, SCOTUS' job here is to draw a line allowing the individual his free exercise while forbidding the business to discriminate.
Your final paragraph brings in another point, which is the matter of discrimination on the basis of class, and that's so settled as to be more solid than the Washington Monument -- it was in fact part of the driving force behind some of the careful wording in the Fourteenth Amendment.  But that isn't in play here, by the baker's own testimony:  he isn't objecting to a class of people, he's objecting to being required to perform a specific action.  When I had my own business I didn't have to worry about that (I can't think of any handyman job that would require an action anyone could think of as immoral), but it does give me the perspective to differentiate between the two issues of disapproving of the customer v refusing to do a certain action -- I did handyman work for more than a few people I had reason to despise, but that was no grounds for refusing a job.