Is sex determined by phenotype or genotype?

Phenotype, strictly speaking. There is a lot of talk about primary and secondary sexual characteristics, and I think people are getting confused about it all. Male and female, biologically, are defined by the gamete-producing organs of an organism. In humans, that means ovaries or testes. This is fundamental because it's what sexual reproduction is based on and what the entire concept of sex is built upon.

Nearly every human was born with structures to produce one of the types of gametes: sperm or ova. Despite any disorder of sexual development or intersex condition, it's virtually certain they have either ovaries or testes.

Having none at birth is a very rare anomaly. From strictly biological view, such a person is neuter. (Despite this, an argument can be made that they still have a sex that can be determined by which organs "should" have developed if whatever blocking factor were removed.) Having both at birth is beyond rare and biologically such a person would be a true hermaphrodite. These two kinds of people are the only ones who can possibly claim to be neither male nor female, or both, and there are maybe a few thousands or tens of thousands of them in the entire world.

For example: Caster Semenya was born with testes, just ones that didn't develop properly and remained internal. Despite outward appearance, Caster Semenya is male, biologically.