clip demonstrates that there are ways around it and links to a nytimes article pointing out basically a grandma neuron. and if you look at visualizations for resnet by zhu or the more recent (i believe) visualization by openai lab, you'd see a lot of grandma neurons.