“the specific circumstances around and the political movements involved in brexit make it a bad thing” and “the EU is a horrible institution that keeps pushing free market liberalism and destroying local democracy” are ideas that can coexist like you don’t have to choose one or the other
Plus: isolationism is not actually a solution to the things that are terrible about the EU.
