All you have to do to be antifa is to be against fascism.

Wrong. Antifa are groups that use violence and intimidation to silence political opponents THEY label as fascists, most of whom arent fascists, but even if they were it would be irrelevant. I'm generally against Fascism, but self described fascists have as much a right to political speech and assembly as self described communists, who I also oppose, and whose ideology killed far more people. If you are against these rights, then you are a fascist. Which means if we use your own criteria people would then be justified in using violence and intimidation to silence you. See how that works?