I'm also seeing this as mostly a political problem and agree that the more we apply words like "misogynist" the more diluted the meaning becomes. I believe gender identity ideology is just one aspect of identity politics that's deranging us (at least, until it's eaten by a new one)

However, the effects of gender identity ideology do have different flavors for women than for men. Material changes in policy such as allowing men on women's sports teams effectively eliminates women's fair participation in sports, whereas men's sports are unaffected. Making changing rooms effectively co-ed is dangerous for women, but not for men.

There also seems to be a special type of vitriol reserved for women who speak up against this ideology ("punch TERFs", rape threats, death threats), whereas men who speak up are relatively unscathed. And, there seems to the double standard of keeping subreddits that are harmful to women but taking down the largest feminist subreddit that was heavily moderated to follow the rules.

I'm not sure if these points constitute "proof of misogyny" but I can see why one could draw that conclusion. But in any case, the gender ideologues have already one-upped women in the victimhood narrative, so a good start to solving these problems is what you said: supporting claims with facts.