Men in women's spaces is the problem. E.g. men's sports, toilets, prisons etc. were already unisex. Anyone can use/join them. It's the female spaces where men aren't allowed. There is actually no such thing as a "men's space", so women using them doesn't affect anyone except themselves. It's for their own good not to use them, not men's.

Not true. Women historically were explicitly barred from many sports and spaces such as schools, universities, clubs, most of the clergy. Still many clubs, fraternities and institutions like the Catholic hierarchy of the priesthood are for males only. In most of the Islamic world women can't be imams, and they certainly can't be mullahs and ayatollahs, and they are only allowed in one small part of mosques behind a curtain. Same goes for orthodox Jewish sects; only males get to become rebbes and devote their lives to study of the sacred texts, and at worship and social events like weddings women & girls are kept off to one side - or to a balcony - separate from the boys & men.

When previously all-male institutions like the university I attended decide to admit females, many had to amend their charters and other founding documents which said they were only for men. After I graduated, the alumni clubs only permitted women in certain parts of the premises - we were barred from the pools and gym, for example. When I was growing up, women were explicitly barred from applying for a vast number of jobs and positions. Women were also were not permitted to participate in distance running at the community or club level, or to partake in marathons.

There are still many private clubs around that are male-only, or if they are mixed-sex they have male-only spaces like the Men's Grill and Men's Lounge.