Judging Experts by the Objective State of the Debate

Human civilization has more knowledge than any one person. We have a division of intellectual labor. Some people specialize in chemistry, others law, others fashion, others history, others football. A specialist in a type of knowledge is called an “expert” or even an “authority” for his field. The division of intellectual labor has progressed to the point of narrow specialities – e.g. we have experts in ancient Greek history, or WWII history, rather than all of history. There are different kinds of scientists, and then within a kind, e.g. physicist, there are sub-kinds, e.g. astrophysicist.

People accept expert advice from car mechanics, doctors, lawyers, scientists, tech support people, sports coaches and more. You may be able to learn about a few topics, in detail, yourself, but not all the topics that come up in your life. There’s too much to know it all yourself.

If you didn’t use other people’s expert knowledge – if you didn’t participate in the intellectual division of labor – you’d be handicapped, have a limited life and not accomplish much compared to people who do (just the same as a person who doesn’t participate in the economic division of labor cannot produce much compared to people who do participate).

The intellectual division of labor raises problems to be addressed. How do you know which ideas from other people to use? How do you judge an expert’s claim when you don’t know much about the field? How can decide what to think when experts in a field disagree with each other?

One attempted solution is credentials. Some people perform the task of judging experts. But the people saying which experts are good are themselves experts (in the field of judging expertise), so you’re left with the same problem of deciding which experts to listen to. That's just moved the problem: instead of deciding whether to listen to a scientist saying humans evolved, you decide whether to listen to a guy telling you he knows which scientists to listen to. And normally the qualifications of the people giving out credentials in a field are that they are experts in that field (not that they actually have any special expertise at judging experts), so it’s really just “Listen to me about which physicists you should listen to, because I’m a good physicist.”

Another attempted solution is reputation. Some people have a bunch of success in some visible way and then people listen to them more. And reputations can partially carry over to their associates, and to a lesser degree to their associate’s associates.

Another attempted solution – which is how a lot of reputation works – is to judge by popularity. But great ideas usually start out unpopular.

Another way people judge expertise is by charisma, social status, social skill, and stuff like that (including dressing well and speaking in a “smart” sounding way). This is a poor method. It leads to competitions not at field expertise but at expertise in impressing people and presenting as credible to them.

Another way people judge experts is by which ones create material (articles, books, videos, etc.) for a general audience that they like. This isn’t very good at figuring out who is the best at the details of the field because it looks for skills like being able to communicate well about the basics of the field.

I propose a better way to judge experts. This solution is especially meant for intellectuals rather than, e.g., bike repair experts. Experts should provide public information which can be evaluated by lay people. It’s their job to prove their own case if they want to be considered an expert. But how? Specifically by being open to debate. Experts should be open to questions and criticism, in public, and organize the information in a way that people can look it over and see who blocked further progress on resolving the disagreement. The public should favor experts who have addressed all outstanding criticism of their knowledge over experts who have withdrawn from that kind of discussion, ignored criticisms, refused to answer questions, derailed debates, etc. Experts should be judged by the current state of the debate in the field, and should organize that debate so it isn’t a mess with no clear answers.

People who don’t know how to do this aren’t fit to be experts in a fields that deal with controversies (but maybe they can successfully be an expert accountant). If your field has ongoing disagreements and debate, then you need to know how to organize and evaluate disagreements and debate in order to do effective work in your field.

The starting point of clarifying the state of the debate is to invite debate. The people who decline debate are the people blocking resolution of the issues. The people who are unwilling to try to address questions and criticisms should be presumed wrong, even though they might be right about some particular issues, because their methodology – their way of dealing with knowledge – is not oriented towards truth-seeking. People who reject intellectual collaboration, on principle, are limiting their participation in the intellectual division of labor and thereby limiting their effectiveness (just like a business that won’t consider any business deals with other businesses).

A good expert has the general attitude: “If I’m wrong, tell me what I’m wrong about. And I’ve told you what you’re wrong about and I’m still waiting for you to respond.” And he thinks of debate as primarily a matter of writing, over time, not verbal debate in person. So he can write a blog post criticizing something, and that advances the state of the debate, and if it’s not answered then that shows the other guy isn’t debating (or discussing, which should be the same thing). And it shows the other guy also lacks proxies to discuss for him. And lacks sources he could cite that address the issue with no new work. (Or else he has the perfect answer, already written, and just won’t bother to share the link, and none of his fans will share the link either? Not a plausible story.)

Openness to debate is a well known criterion, so many people pretend to meet it. But most don’t pretend in more than a token way. Suppose I wrote a blog post with some questions and criticisms for an expert. You, right now, could predict that most experts would ignore me. For example, Richard Dawkins would ignore me (and that’s not mere speculation, I have actually contacted him and been ignored, even though I’m an expert who has written serious criticism of some of his work). His openness to debate is limited in some ways.

What are the limits on the openness to debate of Dawkins and the large majority of other supposed experts? I could try to analyze and criticize them and talk about some Paths Forward stuff. But there’s a much simpler way for lay people to evaluate the matter. Has Dawkins written down what his limits on debate are, himself? Has he publicly shared a policy stating his openness to debate, including the limits and the reasons for those limits? Has he asked if anyone knows any ways to remove or reduce any of those limits? No he has not. Because he isn’t seriously interested in discussing and getting disagreements resolved.

Many experts were more open to debate when they were younger, and they get disillusioned after many bad, ineffective discussions. They give up and decide talking with people is mostly a waste of time. What they should have done is learned better methods that better conserve their time, get to the point faster, and so on (see Paths Forward for info on how to do that). Organize the debate better instead of giving up on debate (and then dishonestly pretending you’re still open to debate). Learn enough philosophy – methods of dealing with ideas, learning, resolving disagreements between ideas, etc. – to be an effective intellectual. Sure that’s hard (most philosophy is crap) but if you want to be a good intellectual you need to deal with that problem and find or create and then use actual good methods for making intellectual progress (and if you think you have those, write them down and expose those to criticism and debate, and also make them available for others to learn and use if you think they actually work well! As I have done.).

By the way, what if all the experts in a field are bad? What if none of them are really open to debate? Then it’s hard to evaluate, so you should ask a philosopher (general purpose expert) to evaluate the field (and you can judge which philosophers are experts by their openness to debate).