No, not all anomalies are disorders! David Bowie's eyeballs were anomalous in color, but his particular condition is not a disorder.

Then there is no issue with calling intersex a disorder?

There is a problem with referring to DSDs as "a disorder." Coz DSDs are actually a constellation of 40 (or thereabouts) very different conditions, each one of which is entirely unique.

These conditions have been variously called disorders of sex development and differences of sex development, or DSDs for short. Some people prefer calling them VSCs, which is short for such terms as variations in sex characteristics or variable sex conditions.

Many people avoid the term "intersex" because it misleadingly suggests that people with DSDs/VSCs are in between the two sexes, or a combination of the two. This is not true. They are all either male or female.

Not all DSDs are disorders. Many people with DSDs are very healthy people whose DSDs do not require any medical treatment. Olympians Caster Semenya, Margaret Wambui and Francine Niyonsaba, for example, are extremely healthy human beings - they just have male-specific DSDs that caused their male genitals not to develop typically in utero and thus to look atypical for males when they were born.

Similarly, there are many different kinds of medical conditions and diseases that impair function in some way and require medical treatment and therefore are considered disorders, but they're not disabilities - medically, legally or colloquially.