Drawing on Anne Lawrence here:

In AGP, the idea is to use womens' spaces such as locker rooms or bathrooms. Because only women are meant to use those sex-specific facilities. It's not voyeuristic. The idea is to go unnoticed--which validates the concept of oneself as a woman. That is the ideal. They would prefer it that way. However, because passing isn't achieved, the AGP person can suffer "narcissistic injury" to the self. This is where the aggression comes in, because women are not playing into the fantasy for the individual.

I don't think AGP are deliberately setting out to make women uncomfortable, that is of course the outcome, and for some, not all, they don't care. The paraphilic interest is to be a woman, anything woman-typical or woman-exclusive (such as lesbian) is up for grabs.