After reading this I am seriously considering moving elsewhere.

There's a term for this: white flight. When the middle and upper class, who historically are predominantly white, flee an area.