What it’s about, in a nutshell: In some areas of intellectual life, you need to already be an expert to find the other experts. This opens a door to a horrible possibility: if you misunderstand things and use that misunderstanding to pick out who you trust, then that trust will simply compound your misunderstanding. Morally flawed people will pick morally flawed advisors and gurus, and bootstrap themselves into being worse people. But we have to trust. So we might just be screwed.
The long version: For some kinds of experts, there’s an easy test: you can tell a good mechanic because they can fix your car. You don’t really need to know anything about cars to sort the real mechanics from the posers. Call these the obvious cognitive domains. A total novice has some hope of figuring out who the right advisors and teachers are. But in some kinds of cognitive domains, you already have to be an expert to recognize the experts. And no other kind of expertise will do — you need to share expertise to recognize a real expert. Call these cognitive islands. On a cognitive island, you need to already be some kind of expert to figure out who the experts are. The novice in that domain has no idea who to trust. Plausible candidates for cognitive islands include the moral and the aesthetic domains, and maybe even philosophy and economics and more.
Some people think being on a cognitive island makes it impossible to use experts. Only novices need the help of experts, and they can’t find any. I think this kind of pessimism is wrong, and I think if we look at how we actually trust each other, how we use other experts, we’ll see why. All the time, we use our own expertise to help find other experts who can help fill in our own gaps — our blind spots, our biases. We need others to help us triangulate on when we’re reasoning well and when we’ve made mistakes.
The real problem for cognitive islands isn’t that we can’t use other experts at all. It’s that there’s no safety net. If your own understanding is flawed, there’s no test for that. If other experts are flawed, there’s no independent check. You can only figure out who to trust by applying your own abilities. But we have to trust each other – we have to use each other to corroborate and check on our own thinking. And this means that if you’re deeply flawed, those flaws will simply compound themselves through expert selection. KKK members will pick racist advisors, who will corroborate their racism.
This leads to a kind of epistemic trap, which I call runaway personal echo chambers. On a cognitive island, the only way to figure out who to trust is to use your own abilities. So if you start out with deep problems in your understanding, you’ll just bootstrap yourself into something worse. And it doesn’t seem like there’s any way out.