After a lifetime of socialization, I'd be more surprised if you couldn't tell the difference. There being a difference doesn't say anything about the cause of that difference.

How about we support LGB rights because it doesn't harm anyone and there's no reason not to? Just because something is inborn doesn't mean it's good, and just because something isn't doesn't mean it's bad. The cause of homosexuality and bisexuality are irrelevant to the validity of gay rights.

If you think saying "no such thing as brain sex" damages support for gay rights, then you'd also have to argue that the "born this way" arguement supports things like pedophilia. Because the pedophile movement does compare itself to the gay rights movement in this way. And to preempt any knee-jerk responses, no, I am not comparing pedophilia to homosexuality.

And so what if it's fringe? The mainstream accepts a lot of nonsensical bullshit about women, and it isn't right just because it's mainstream. There are lots of things that don't look good to outsiders (like being against gender identity, for example), that doesn't mean we shouldn't say them.