Many heterosexuals never thought about their identity. They never stopped for a moment to self reflect. Since they were children they were told men are like this and women are like that. Your purpose in life is to build a family and have children.

When they see someone living outside gender norms their brains explode, it goes against everything their parents taught them. So the fault must be ours, we are perverts and attention seekers. We go against their culture and the very basis of civilization (their words, not mine).

As you already figured, the problem people have with same sex attracted people is about gender. Men aren't supposed to fuck men, that's a woman thing. We are "unnatural" because we don't act like men are supposed to do.

When presented with this problem a straight person can take three paths:

One, they spew hatred and homophobia wishing we were dead so the unnatural thing dissappears and they can go back to mindlessly drifting through life. In the case of straight men they can use homophobia to strengthen their bonds.

Two, they make an effort to seek the truth. They realize they have been lied, homosexuality is a natural thing and their parents have filled their heads with gender bullshit. Its very unlikely, too much effort for the average straight person.

Three, they decide to embrace us as unnatural beings, completely misunderstanding us. They see us as freaks, but they are progressive and kind, not like these conservative bigots. Being unnatural is good, fashionable and sexy. Being gay is all about being gender non conforming, throwing bricks to the police and making your fetishes a political statement. Trans and gay are the same because both of them are unnatural and piss off conservatives.

To sum it up, the vast majority of people take gender norms as reference. Enforcing them or defying them. But no matter what, we are still singled out because they can't understand us.