HRT only goes so far in masculinizing appearance. While it reduces buccal fat around the eyes, it doesn't lead to the development of a stronger brow bone (giving transmen that "methhead" look), it doesn't change the size of your hands or feet, etc. Transmen have noticeably female faces the longer you look at them. When gay men and lesbians say we are attracted to the WHOLE PERSON of the same-sex, we mean it.

It's not that brow bone or buccal fat (whatever that is) is the direct cause to gay men finding males attractive, even if some sex characteristics are attractive (on a man). It's the feeling that something is off. It's that once the realization hits that this person is not male after all, whatever fleeting emotion that is mistaken for attraction vanishes. Losing interest/"mistaken attraction" based on new information, and having your assumptions corrected, is very normal.