Some parts of fashion are arbitrary and artificial (gender), some are a result of men and women being shaped differently (sex).

Well-put. An example of the former would be colors and patterns deemed appropriate for each sex: there's no reason why men can't wear pink, or soft "sweet"-looking pastels, or floral prints-- it's not as though these things require that you have a bosom, or wide hips! But they're all "feminine" by gender. Somehow, "masculine" means colors that are simple (red, blue, and green in their pure forms), dark, or neutral. The more formal the setting, the more limited men's palette: if it's a "traditional businesswear" kinda place, you can get away with some color-- like the classic red necktie-- but for the dressiest occasions, it's black-and-white only. Basically men must be SERIOUS: never too expressive, playful, or decorative. Those things are gendered "female". Thus perfectly fine-- even required-- for silly lesser beings like us women, but where men are concerned? Anything with associations like that automatically has cooties.