They basically mean "femininity" and "masculinity" are sexual dimorphism

Which one could argue, they kind of are.

Behavioural trends like aggression vs. submission are obviously dimorphic since it stems from physical stature. Then there's fashion, which was created to only highlight the dimorphism, making women look more delicate by comparison. Modern beauty standards, where women are expected to be slender and not too heavy/muscular, unlike men. It's too facile to say this is all "just" a social construct unanchored from any biological/evolutionary roots: one could argue that femininity is also somewhat of a defense mechanism under sexual dimorphism.

It certainly doesn't have to mean all those things need to exist in modern society, but in some situations you can't easily get away from them. We can say gender is unhelpful when it does things like hold women back and we should thus dispense with differentiation completely—but we still need language to describe "femininity" and "masculinity" in those cases where it obviously exists.

"Beautiful", is a term that itself more aligned with the feminine. "Handsome" is the term that is aligned with men.

I wonder if this is just something specific to our English language, or if there are similar differences in other languages. Words can easily change, or new words can arise: people used to "feel so gay" but it means something different now...

I think words like "handsome" include some nuance where they imply other specific qualities beyond just beauty. If you picture a "Hercules" archetype that someone might hold up as an example of male beauty, yes, "beautiful" could certainly fit—but you wouldn't say "cute" there—since that's a different quality of beauty.