But they can be used by shill teams to completely control certain narratives. You would think genuine users would be enough to keep things balanced, but when it comes to defending controversial ideas that require research to understand, well upvotes and downvotes can be used to fool people.

People also underestimate the level in which shills are organized. This is the CIA we are fighting against... they can beat regular users 1 vs 100 and they aren't fools. I dare say they see most users over here as a joke at best and even if they can't control the site itself, they can subvert the narratives as is to the point that it stops becoming a problem. The downvote option would give them even more power and completely destroy the site.