I totally see where you're coming from.

the reason people have responded differently to the concept of transracial rather than transgender is because of the perceived materiality of experience. a visibly black person has a very specific experience of racism that no white person, regardless of their ancestral history, can understand or simply 'identify' into. of course it is the same with transgender at the end of the day BUT the reason that people have felt differently about gender is because gender has such strong visual elements tied to it in our culture than can be adapted by either sex that it forces something of a mind trick when we see it. if that makes sense. sex is binary and this is obvious once our clothes are off. but gendered fashion and modes of dress are just the trappings we put on top, and the significance attached to them socially is very important to many. I think this is PARTLY why transgender has been more readily accepted than other outrageously offensive identity appropriating.

also, wheras a white woman braiding her hair and tanning will NEVER lead her to experience what black women do, people dressing as the opposite sex do experience stigma and discrimination that others are able to observe and recognise - which then lends extra credence to the claim they are an oppressed population.