Humans are a sexually dimorphic species.

In sexually dimorphic species, the two sexes have differing traits that, in general identify male from female members.

There can be exceptions. Just as a random male robin may be lighter red than a rando female robin, in general, males have darker color.

In humans, the dimorphism includes height, muscle mass, bone density, strength, endurance, and many secondary sexual characteristics (for instance breasts in females and Adam's apples in men).

All of this is descriptive, not proscriptive. All of this is based off clear and present observation of human beings.

Now, moving beyond humans specifically, all sexually reproductive species have at least two sexes. That is generally part of the definition. How, exactly, each species is divided varies, as do what chromosomes trigger each sex to differentiate. Humans are an XY species. XX triggers female traits and reproductive organs, XY triggers male traits and reproductive organs. This is not particularly complicated unless you choose to believe that 0.1% exceptions invalidate a descriptive generalized observation, which is hogwash. Any exception is precisely that - an exception - and has nothing to do with trangenderism or any other such nonsense.

So yes, it is a binary, because our sexual reproduction is a binary process. Two gametes, two sets of chromosomes, dimorphism. There is no fallacy here, just observational fact.