I absolutely agree with you. I could be that any given asexual person simply has a naturally extremely low sex drive. It's more likely that they're experiencing physical issues which are impacting their normal function. For OP it sounds like its a mental issue more than anything.

But i understand the knee jerk reaction you're getting too. People begin to identify with whatever makes them different, partly as a way of coping. I read about deaf people who refuse a simple treatment because they refuse the see being deaf as something to be 'cured'. They'd rather their identity than an extra sense.