I think the flaw (for lack of better word, sorry) in your theory is that any female typical feature on a man or male typical feature on a woman is the opposite of the expectation. And because it’s the opposite, they don’t elicit the same reactions or responses that they would on a person who is the sex typically associated with the feature(s). Does that make sense? Meaning, it’s the very fact that they aren’t the sex typical of whatever feature that makes their experience different.

I'm trying very hard not to get too wrapped up in semantics or nitpick words, because what I've been trying to describe is a paradox that I think you're actually finding the words to (good on you! And thank you! 😊). A feature or attribute could be not sex-typical for a person, and indeed because it isn't sex-typical it would cause that person to be unable to experience it as the opposite sex does, yet embodying that feature means they appear to be sharing a feature more common to the other sex, which may be pointed out by others by doing things such as teasing. The person isn't really experiencing something the way the opposite sex does, but others, whether maliciously or not, point out the commonality.