If you think I'm mistaken or ignorant about something important, I want to hear it. I am open to public comments and criticism. See Paths Forward for an explanation of my methodology for not blocking error correction (always having some Path Forward so that if I'm mistaken, and someone knows it, and they're willing to tell me, then I can be told and I won't ignore it).

I do not reply to everything addressed to me, at all venues. I do reply to a fair amount, but I don't have time to answer everything. However, I will guarantee you some attention if you follow a method of getting my attention which anyone can follow with predictable success. Here's what you do:

1. Post your issue to my Critical Fallibilism Forum.

2. If you don't receive a reply from anyone within a few days, post a self-reply with some followup points. Try again. If the first post didn't have them, follow up with a brief statement of why this is important and a brief summary (one paragraph max, each). Also make sure you're providing a clear question or call to action. What do you want to happen next? What sort of response do you want? Mention you want a Path Forward from me.

3. If you still don't receive a reply within a few days, write a self-reply asking why you didn't receive a reply, and include a brief statement of why replying to you matters and what you're looking for.

4. If you still don't receive a reply within a few days, email me personally ([email protected]) and ask for an answer and say that you've read Paths Forward. Link the CF Forum topic.

Summary: Post to CF. Follow up on why it matters and what reply you want. Follow up asking why no one is answering. Follow up by emailing me. You will get an answer by the end of this process.

Notes

You're welcome to try contacting me in other ways, and that often works, but no promises.

I don't answer everything the first time, but if you are persistent as stated above, then I can guarantee you an answer.

The reasons I want you to post on my public forum are that I want other forum readers to benefit from my answer, I want my answer to have a public permalink so I can refer other people to it in the future, and I want other people to be able to answer you (instead of me).

If you receive an answer from another person, and you think it's inadequate and really want an answer from me personally, you can continue with the steps outlined above and explain this (say why the answers from the other people are inadequate and why you want my personal attention).

I (or someone else) commonly will answer a point before reaching step 4. Often at step 1. (I'm most responsive on the CF forum, so just posting there is frequently enough to get a reply.)

Like many busy people, I am less inclined to answer if I think something is low quality. I certainly don't want to reply to every low quality thing addressed to me. However, if you follow the steps then you'll get a reply from someone, including from me if necessary. (Often other people are fully capable of answering issues, especially the comments I consider lower quality, so I don't always want to do it personally if someone else will do it.)

If you don't want your content to be exclusive to my forum, that's fine. You're welcome to put it on your own website and post a link or copy/paste.

If you want me to address something which costs money, offer me a free copy somewhere within the first 3 steps. If you won't do that, say why.

If I still don't answer after step 4, your personal email went in my spam folder. I don't think this is a common problem, but if it happens feel free to post to CF again and bring it up and I'll see it or someone else will who can contact me. Or it'd be fine to post 10 blog comments in a row or tweet me or something until I notice. Say that you did the 4 Paths Forward steps and I didn't reply, so maybe the email went in spam, and identify the CF posts in question so I can find them. Or you can email Justin or Alan and they'll get my attention. I mention this because spam filtering is a conceivable problem that could get in the way of Paths Forward, and I don't want that to happen. Email is not 100% reliable for contacting me, but it's pretty good and there are solutions if it fails.

Alternatives

What if you don't want to be so demanding and challenging as to ask for a Path Forward from Elliot/CR/CF? Maybe you expect you're wrong (rather than offering a correction), but you're still interested in pursuing the issue and learning something and getting it resolved? Perhaps you want some Path Forward for yourself to make progress?

Follow up on your own posts with new questions, new explanations of the issues and their importance, new angles and perspectives. Rewrite what you're saying a different way. And report what you've done to make progress, what effort you've put in (and what the result was), what you're planning to do, and if you're running out of ideas and if you'd like help with something. Keep at it over time. Be persistent, honest and curious and CF people will want to help. And make it easy for them: take short advice/comments/suggestions and then do a bunch more on your own initiative (and share this so they see giving you help was worthwhile), rather than expecting them to guide you step by step. Put effort into your learning as independently as you can, e.g. by taking book and link suggestions and doing series of blog posts about them as you read. Be a pleasure to help and offer more value than you ask for. (If you don't know how to offer value, but want to, ask.)

---

Update My Paths Forward Policy is a now a backup for my debate policy. The debate policy should handle most issues. Use it first. If something goes wrong with that, the Paths Forward Policy is still available.

Gyrodiot commented defending instrumentalism.

I'm going to clarify what I mean about "instrumentalism" and "empiricism". I don't know if we actually disagree or there's a misunderstanding.

FI has somewhat of a mixed view here (reason and observation are both great), and objects to an extreme focus on one or the other. CR and Objectivism both say you don't have to, and should not, choose between reason and observation. We object to the strong "rationalists" who want to sit in an armchair and reason out what reality is like without doing any science, and we object to the strong "empiricists" who want to look at reality and do science without thinking.

Instrumentalism means that theories are only or primarily instruments for prediction, with little or no explanation or philosophical thought. Our view is that observation and prediction are great and valuable, but aren't alone in being so great and valuable. Some important ideas – such as the theory of epistemology itself – are primarily non-empirical.

There's a way some people try to make philosophy empirical. It's: try different approaches and see what the results are (and try to predict the results of acting according to different philosophies of science). But how do you judge the results? What's a good result? More accurate scientific predictions, you say. But which ones? How do you decide which predictions to value more than others? Or do you say every prediction is equal and go for sheer quantity? If quantity, why, and how do you address that with only empiricism and no philosophical arguments? And you want more accurate predictions according to which measures? (E.g. do you value lower error size variance or lower error size mean, or one of the infinitely many possible metrics that counts both of them in some way?)

How do you know which observations to make, and which portion of the available facts to record about what you observe? How do you interpret those observations? Is the full answer just to predict which way of making observations will lead to the most correct predictions later on? But how do you predict that? How do you know which data will turn out useful to science? My answer is you need explanations of things like which problems science is currently working on, and why, and the nature of those problems – these things help guide you in deciding what observations are relevant.

Here are terminology quotes from BoI:

Instrumentalism   The misconception that science cannot describe reality, only predict outcomes of observations.

Note the "cannot" and "only".

Empiricism   The misconception that we ‘derive’ all our knowledge from sensory experience.

Note the "all" and the "derive". "Derive" refers to something like: take a set of observation data (and some models and formulas with no explanations, philosophy or conceptual thinking) and somehow derive all human knowledge, of all types (even poetry), from that. But all you can get that way are correlations and pattern-matching (to get causality instead of correlation you have to come up with explanations about causes and use types of criticism other than "that contradicts the data"). And there are infinitely many patterns fitting any data set, of which infinitely many both will and won't hold in the finite future, so how do you choose if not with philosophy? By assuming whichever patterns are computable by the shortest computer programs are the correct ones? If you do that, you're going to be unnecessarily wrong in many cases (because that way of prediction is often wrong, not just in cases where we had no clue, but also in cases when explanatory philosophical thinking could have done better). And anyway how do you use empiricism to decide to favor shorter computer programs? That's a philosophy claim, open to critical philosophy debate (rather than just being settled by science), of exactly the kind empiricism was claiming to do without.

Finally I'll comment on Yudkowsky on the virtue of empiricism:

The sixth virtue is empiricism. The roots of knowledge are in observation and its fruit is prediction.

I disagree about "roots" because, as Popper explained, theories are prior to observations. You need a concept of what you're looking for, by what methods, before you can fruitfully observe. Observation has to be selective (like it or not, there's too much data to record literally all of it) and goal-directed (instead of observing randomly). So goals and ideas about observation method precede observation as "roots" of knowledge.

Note: this sense of preceding does not grant debating priority. Observations may contradict preceding ideas and cause the preceding ideas to be rejected.

And note: observations aren't infallible either: observations can be questioned and criticized because, although reality itself never lies, our ideas that precede and govern observation (like about correct observational methods) can be mistaken.

Do not ask which beliefs to profess, but which experiences to anticipate.

Not all beliefs are about experience. E.g. if you could fully predict all the results of your actions, there would still be an unanswered moral question about which results you should prefer or value, which are morally better.

Always know which difference of experience you argue about.

I'd agree with often but not always. Which experience is the debate about instrumentalism and empiricism about?

---

See also my additional comments to Gyrodiot about this.

Gyrodiot wrote at the Less Wrong Slack Philosophy chatroom:

I was waiting for an appropriate moment to discuss epistemology. I think I understood something about curi's reasoning about induction After reading a good chunk of the FI website. Basically, it starts from this:

He quotes from: http://fallibleideas.com/objective-truth

There is an objective truth. It's one truth that's the same for all people. This is the common sense view. It means there is one answer per question.

The definition of truth here is not the same as The Simple Truth as described in LW. Here, the important part is:

Relativism provides an argument that the context is important, but no argument that the truth can change if we keep the context constant.

If you fixate the context around a statement, then the statement ought to have an objective truth value

Yeah. (The Simple Truth essay link.)

In LW terms that's equivalent to "reality has states and you don't change the territory by thinking differently about the map"

Yeah.

From that, FI posits the existence of universal truths that aren't dependent on context, like the laws of physics.

More broadly, many ideas apply to many contexts (even without being universal). This is very important. DD calls this "reach" in BoI (how many contexts does an idea reach to?), I sometimes go with "generality" or "broader applicability".

The ability for the same knowledge to solve multiple problems is crucial to our ability to deal with the world, and for helping with objectivity, and for some other things. It's what enabled humans to even exist – biological evolution created knowledge to solve some problems related to survival and mating, and that knowledge had reach which lets us be intelligent, do philosophy, build skyscrapers, etc. Even animals like cats couldn't exist, like they do today, without reach – they have things like behavioral algorithms which work well in more than one situation, rather than having to specify different behavior for every single situation.

The problem with induction, with this view is that you're taking truths about some contexts to apply them to other contexts and derive truths about them, which is complete nonsense when you put it like that

Some truths do apply to multiple contexts. But some don't. You shouldn't just assume they do – you need to critically consider the matter (which isn't induction).

From a Bayesian perspective you're just computing probabilities, updating your map, you're not trying to attain perfect truth

Infinitely many patterns both do and don't apply to other contexts (such as patterns that worked in some past time range applying tomorrow). So you can't just generalize patterns to the future (or to other contexts more generally) and expect that to work, ala induction. You have to think about which patterns to pay attention to and care about, and which of those patterns will hold in what ranges of contexts, and why, and use critical arguments to improve your understanding of all this.

We do [live in our own map], which is why this mode of thought with absolute truth isn't practical at all

Can you give an example of some practical situation you don't understand how to address with FI thinking, and I'll tell you how or concede? And after we go through a few examples, perhaps you'll better understand how it works and agree with me.

So, if induction is out of the way, the other means to know truth may be by deduction, building on truth we know to create more. Except that leads to infinite regress, because you need a foundation

CR's view is induction is not replaced with more deduction. It's replaced with evolution – guesses and criticism.

So the best we can do is generate new ideas, and put them through empirical test, removing what is false as it gets contradicted

And we can use non-empirical criticism.

But contradicted by what? Universal truths! The thing is, universal truths are used as a tool to test what is true or false in any context since they don't depend on context

Not just contradicted by universal truths, but contradicted by any of our knowledge (lots of which has some significant but non-universal reach). If an idea contradicts some of our knowledge, it should say why that knowledge is mistaken – there's a challenge there. See also my "library of criticism" concept in Yes or No Philosophy (discussed below) which, in short, says that we build up a set of known criticisms that have some multi-context applicability, and then whenever we try to invent a new idea we should check it against this existing library of known criticisms. It needs to either not be contradicted by any of the criticisms or include a counter-argument.

But they are so general that you can't generate new idea from them easily

The LW view would completely disagree with that: laws of physics are statements like every other, they are solid because they map to observation and have predictive power

CR says to judge ideas by criticism. Failure to map to observation and lack of predictive power are types of criticism (absolutely not the only ones), which apply in some important range of contexts (not all contexts – some ideas are non-empirical).

Prediction is great and valuable but, despite being great, it's also overrated. See chapter 1 of The Fabric of Reality by David Deutsch and the discussion of the predictive oracle and instrumentalism.

http://www.daviddeutsch.org.uk/books/the-fabric-of-reality/excerpt/

Also you can use them to explain stuff (reductionism) and generate new ideas (bottom-up scientific research)

From FI:

When we consider a new idea, the main question should be: "Do you (or anyone else) see anything wrong with it? And do you (or anyone else) have a better idea?" If the answers are 'no'and 'no' then we can accept it as our best idea for now.

The problem is that by having a "pool of statements from which falsehoods are gradually removed" you also build a best candidate for truth. Which is not, at all, how the Bayesian view works.

FI suggests evolution is a reliable way to suggest new ideas. It ties well into the framework of "generate by increments and select by truth-value"

It also highlights how humans are universal knowledge machines, that anything (in particular, an AGI) created by a human would have knowledge than humans can attain too

Humans as universal knowledge creators is an idea of my colleague David Deutsch which is discussed in his book, The Beginning of Infinity (BoI).

http://beginningofinfinity.com

But that's not an operational definition : if an AGI creates knowledge much faster than any human, they won't ever catch up and the point is moot

Yes, AGI could be faster. But, given the universality argument, AGI's won't be more rational and won't be capable of modes of reasoning that humans can't do.

The value of faster is questionable. I think no humans currently maximally use their computational power. So adding more wouldn't necessarily help if people don't want to use it. And an AGI would be capable of all the same human flaws like irrationalities, anti-rational memes (see BoI), dumb emotions, being bored, being lazy, etc.

I think the primary cause of these flaws, in short, is authoritarian educational methods which try to teach the kid existing knowledge rather than facilitate error correction. I don't think an AGI would automatically be anything like a rational adult. It'd have to think about things and engage with existing knowledge traditions, and perhaps even educators. Thinking faster (but not better) won't save it from picking up lots of bad ideas just like new humans do.

That sums up the basics, I think The Paths Forwards thing is another matter... and it is very, very demanding

Yes, but I think it's basically what effective truth-seeking requires. I think most truth-seeking people do is not very effective, and the flaws can actually be pointed out as not meeting Paths Forward (PF) standards.

There's an objective truth about what it takes to make progress. And separate truths depending on how effectively you want to make progress. FI and PF talk about what it takes to make a lot of progress and be highly effective. You can fudge a lot of things and still, maybe, make some progress instead of going backwards.

If you just wanna make a few tiny contributions which are 80% likely to be false, maybe you don't need Paths Forward. And some progress gets made that way – a bunch of mediocre people do a bunch of small things, and the bulk of it is wrong, but they have some ability to detect errors so they end up figuring out which are the good ideas with enough accuracy to slowly inch forwards. But, meanwhile, I think a ton of progress comes from a few great (wo)men who have higher standards and better methods. (For more arguments about the importance of a few great men, I particularly recommend Objectivism. E.g. Roark discusses this in his courtroom speech at the end of The Fountainhead.)

Also, FYI, Paths Forward allows you to say you're not interested in something. It's just, if you don't put the work into knowing something, don't claim that you did. Also you should keep your interests themselves open to criticism and error correction. Don't be an AGI researcher who is "not interested in philosophy" and won't listen to arguments about why philosophy is relevant to your work. More generally, it's OK to cut off a discussion with a meta comment (e.g. "not interested" or "that is off topic" or "I think it'd be a better use of my time to do this other thing...") as long as the meta level is itself open to error correction and has Paths Forward.

Oh also, btw, the demandingness of Paths Forward lowers the resource requirements for doing it, in a way. If you're interested in what someone is saying, you can be lenient and put in a lot of effort. But if you think it's bad, then you can be more demanding – so things only continue if they meet the high standards of PF. This is win/win for you. Either you get rid of the idiots with minimal effort, or else they actually start meeting high standards of discussion (so they aren't idiots, and they're worth discussing with). And note that, crucially, things still turn out OK even if you misjudge who is an idiot or who is badly mistaken – b/c if you misjudge them all you do is invest less resources initially but you don't block finding out what they know. You still offer a Path Forward (specifically that they meet some high discussion standards) and if they're actually good and have a good point, then they can go ahead and say it with a permalink, in public, with all quotes being sourced and accurate, etc. (I particularly like asking for simple things which are easy to judge objectively like those, but there are other harder things you can reasonably ask for, which I think you picked up on in some ways your judgement of PF as demanding. Like you can ask people to address a reference that you take responsibility for.)

BTW I find that merely asking people to format email quoting correctly is enough barrier to entry to keep most idiots out of the FI forum. (Forum culture is important too.) I like this type of gating because, contrary to moderators making arbitrary/subjective/debatable judgements about things like discussion quality, it's a very objective issue. Anyone who cares to post can post correctly and say any ideas they want. And it lacks the unpredictability of moderation (it can be hard to guess what moderators won't like). This doesn't filter on ideas, just on being willing to put in a bit of effort for something that is productive and useful anyway – proper use of nested quoting improves discussions and is worth doing and is something all the regulars actively want to do. (And btw if someone really wants to discuss without dealing with formatting they can use e.g. my blog comments which are unmoderated and don't expect email quoting, so there are still other options.)

It is written very clearly, and also wants to make me scream inside

Why does it make you want to scream?

Is it related to moral judgement? I'm an Objectivist in addition to a Critical Rationalist. Ayn Rand wrote in The Virtue of Selfishness, ch8, How Does One Lead a Rational Life in an Irrational Society?, the first paragraph:

There's a lot of reasoning for this which goes beyond the one essay. At present, I'm just raising it as a possible area of disagreement.

There are also reasons about objective truth (which are part of both CR and Objectivism, rather than only Objectivism).

The issue isn't just moral judgement but also what Objectivism calls "sanction": I'm unwilling to say things like "It's ok if you don't do Paths Forward, you're only human, I forgive you." My refusal to actively do anti-judgement stuff, and approve of PF alternatives, is maybe more important than any negative judgements I've made, implied or stated.

It hits all the right notes motivation-wise, and a very high number of Rationality Virtues. Curiosity, check. Relinquishment, check. Lightness, check. Argument, triple-check.

Yudkowsky writes about rational virtues:

Haha, yeah, no wonder a triple check on that one :)

Simplicity, check. Perfectionism, check. Precision, check. Scholarship, check. Evenness, humility, precision, Void... nope nope nope PF is much harsher than needed when presented with negative evidence, treating them as irreparable flaws (that's for evenness)

They are not treated as irreparable – you can try to create a variant idea which has the flaw fixed. Sometimes you will succeed at this pretty easily, sometimes it’s hard but you manage it, and sometimes you decide to give up on fixing an idea and try another approach. You don’t know in advance how fixable ideas are (you can’t predict the future growth of knowledge) – you have to actually try to create a correct variant idea to see how doable that is.

Some mistakes are quite easy and fast to fix – and it’s good to actually fix those, not just assume they don’t matter much. You can’t reliably predict mistake fixability in advance of fixing it. Also the fixed idea is better and this sometimes helps leads to new progress, and you can’t predict in advance how helpful that will be. If you fix a bunch of “small” mistakes, you have a different idea now and a new problem situation. That’s better (to some unknown degree) for building on, and there’s basically no reason not to do this. The benefit of fixing mistakes in general, while unpredictable, seems to be roughly proportional to the effort (if it’s hard to fix, then it’s more important, so fixing it has more value). Typically, the small mistakes are a small effort to fix, so they’re still cost-effective to fix.

That fixing mistakes creates a better situation fits with Yudkowsky’s virtue of perfectionism.

(If you think you know how to fix a mistake but it’d be too resource expensive and unimportant, what you can do instead is change the problem. Say “You know what, we don’t need to solve that with infinite precision. Let’s just define the problem we’re solving as being to get this right within +/- 10%. Then the idea we already have is a correct solution with no additional effort. And solving this easier problem is good enough for our goal. If no one has any criticism of that, then we’ll proceed with it...")

Sometimes I talk about variant ideas as new ideas (so the original is refuted, but the new one is separate) rather than as modifying and rescuing a previous idea. This is a terminology and perspective issue – “modifying" and “creating" are actually basically the same thing with different emphasis. Regardless of terminology, substantively, some criticized flaws in ideas are repairable via either modifying or creating to get a variant idea with the same main points but without the flaw.

PF expects to have errors all other the place and act to correct them, but places a burden on everyone else that doesn't (that's for humility)

Is saying people should be rational burdensome and unhumble?

According to Yudkowsky's essay on rational virtues, the point of humility is to take concrete steps to deal with your own fallibility. That is the main point of PF!

PF shifts from True to False by sorting everything through contexts in a discrete way.

The binary (true or false) viewpoint is my main modification to Popper and Deutsch. They both have elements of it mixed in, but I make it comprehensive and emphasized. I consider this modification to improve Critical Rationalism (CR) according to CR's own framework. It's a reform within the tradition rather than a rival view. I think it fits the goals and intentions of CR, while fixing some problems.

I made educational material (6 hours of video, 75 pages of writing) explaining this stuff which I sell for $400. Info here:

https://yesornophilosophy.com

I also have many relevant, free blog posts gathered at:

http://curi.us/1595-rationally-resolving-conflicts-of-ideas

Gyrodiot, since I appreciated the thought you put into FI and PF, I'll make you an offer to facilitate further discussion:

If you'd like to come discuss Yes or No Philosophy at the FI forum, and you want to understand more about my thinking, I will give you a 90% discount code for Yes or No Philosophy. Email [email protected] if interested.

Incertitude is lack of knowledge, which is problematic (that's for precision)

The clarity/precision/certitude you need is dependent on the problem (or the context if you don’t bundle all of the context into the problem). What is your goal and what are the appropriate standards for achieving that goal? Good enough may be good enough, depending on what you’re doing.

Extra precision (or something else) is generally bad b/c it takes extra work for no benefit.

Frequently, things like lack of clarity are bad and ruin problem solving (cuz e.g. it’s ambiguous whether the solution means to take action X or action Y). But some limited lack of clarity, lower precision, hesitation, whatever, can be fine if it’s restricted to some bounded areas that don’t need to be better for solving this particular problem.

Also, about the precision virtue, Yudkowsky writes,

FI/PF has no issue with this. You can specify required precision (e.g. within plus or minus ten) in the problem. Or you can find you have multiple correct solutions, and then consider some more ambitious problems to help you differentiate between them. (See the decision chart stuff in Yes or No Philosophy.)

PF posits time and again that "if you're not achieving your goals, well first that's because you're not faillibilist". Which is... quite too meta-level a claim (that's for the Void)

Please don't put non-quotes in quote marks. The word "goal" isn't even in the main PF essay.

I'll offer you a kinda similar but different claim: there's no need to be stuck and not make progress in life. That's unnecessary, tragic, and avoidable. Knowing about fallibilism, PF, and some other already-known things is adequate that you don't have to be stuck. That doesn't mean you will achieve any particular goal in any particular timeframe. But what you can do is have a good life: keep learning things, making progress, achieving some goals, acting on non-refuted ideas. And there's no need to suffer.

For more on these topics, see the FI discussion of coercion and the BoI view on unbounded progress:

http://beginningofinfinity.com

(David Deutsch, author of BoI, is a Popperian and is a founder of Taking Children Seriously (TCS), a parenting/education philosophy created by applying Critical Rationalism and which is where the the ideas about coercion come from. I developed the specific method of creating a succession of meta problems to help formalize and clarify some TCS ideas.)

I don't see how PF violates the void virtue (aspects of which, btw, relate to Popper's comments on Who Should Rule? cuz part of what Yudkowsky is saying in that section is don't enshrine some criteria of rationality to rule. My perspective is, instead of enshrining a ruler or ruling idea, the most primary thing is error correction itself. Yudkowsky says something that sorta sounds like you need to care about the truth instead of your current conception of the truth – which happily does help keep it possible to correct errors in your current conception.)

(this last line is awkward. The rationalist view may consider that rationalists should win, but not winning isn't necessarily a failure of rationality)

That depends on what you mean by winning. I'm guessing I agree with it the way you mean it. I agree that all kinds of bad things can happen to you, and stuff can go wrong in your life, without it necessarily being your fault.

(this needs unpacking the definition of winning and I'm digging myself deeper I should stop)

Why should you stop?

---

Justin Mallone replied to Gyrodiot:

hey gyrodiot feel free to join Fallible Ideas list and post your thoughts on PF. also, could i have your permission to share your thoughts with Elliot? (I can delete what other ppl said). note that I imagine elliot would want to reply publicly so keep that in mind.

Gyrodiot replied:

@JUSTINCEO You can share my words (only mine) if you want, with this addition: I'm positive I didn't do justice to FI (particularly in the last part, which isn't clear at all). I'll be happy to read Elliot's comments on this and update in consequence, but I'm not sure I will take time to answer further.

I find we are motivated by the same "burning desire to know" (sounds very corny) and disagree strongly about method. I find, personally, the LW "school" more practically useful, strikes a good balance for me between rigor, ease of use, and ability to coordinate around.

Gyrodiot, I hope you'll reconsider and reply in blog comments, on FI, or on Less Wrong's forum. Also note: if Paths Forward is correct, then the LW way does not work well. Isn't that risk of error worth some serious attention? Plus isn't it fun to take some time to seriously understand a rival philosophy which you see some rational merit in, and see what you can learn from it (even if you end up disagreeing, you could still take away some parts)?

---

For those interested, here are more sources on the rationality virtues. I think they're interesting and mostly good:

https://wiki.lesswrong.com/wiki/Virtues_of_rationality

https://alexvermeer.com/the-twelve-virtues-of-rationality/

http://madmikesamerica.com/2011/05/the-twelve-virtues-of-rationality/

That last one says, of Evenness:

Maybe. Depends on how "cautious" would be clarified with more precision. This could be interpreted to mean something I agree with, but also there are a lot of ways to interpret it that I disagree with.

I also think Occam's Razor (mentioned in that last link, not explicitly in the Yudkowsky essay), while having some significant correctness to it, is overrated and is open to specifications of details that I disagree with.

And I disagree with the "burden of proof" idea (I cover this in Yes or No Philosophy) which Yudkowsky mentions in Evenness.

The biggest disagreement is empiricism. (See the criticism of that in BoI, and FoR ch1. You may have picked up on this disagreement already from the CR stuff.)

I emailed this to some MIRI people and others related to Less Wrong.

---

I believe I know some important things you don't, such as that induction is impossible, and that your approach to AGI is incorrect due to epistemological issues which were explained decades ago by Karl Popper. How do you propose to resolve that, if at all?

I think methodology for how to handle disagreements comes prior to the content of the disagreements. I have writing about my proposed methodology, Paths Forward, and about how Less Wrong doesn't work because of the lack of Paths Forward:

http://curi.us/1898-paths-forward-short-summary

http://curi.us/2064-less-wrong-lacks-representatives-and-paths-forward

Can anyone tell me that I'm mistaken about any of this? Do you have a criticism of Paths Forward? Will any of you take responsibility for doing Paths Forward?

Have any of you written a serious answer to Karl Popper (the philosopher who refuted induction – http://fallibleideas.com/books#popper )? That's important to address, not ignore, since if he's correct then lots of your research approaches are mistakes.

In general, if someone knows a mistake you're making, what are the mechanisms for telling you and having someone take responsibility for addressing the matter well and addressing followup points? Or if someone has comments/questions/criticism, what are the mechanisms available for getting those addressed? Preferably this should be done in public with permalinks at a venue which supports nested quoting. And whatever your answer to this, is it written down in public somewhere?

Do you have public writing detailing your ideas which anyone is taking responsibility for the correctness of? People at Less Wrong often say "read the sequences" but none of them take responsibility for addressing issues with the sequences, including answering questions or publishing fixes if there are problems. Nor do they want to address existing writing (e.g. by David Deutsch – http://fallibleideas.com/books#deutsch ) which contains arguments refuting major aspects of the sequences.

Your forum ( https://agentfoundations.org ) says it's topic-limited to AGI math, so it's not appropriate for discussing criticism of the philosophical assumptions behind your approach (which, if correct, imply the AGI math you're doing is a mistake). And it states ( https://agentfoundations.org/how-to-contribute ):

It’s important for us to keep the forum focused, though; there are other good places to talk about subjects that are more indirectly related to MIRI’s research, and the moderators here may close down discussions on subjects that aren’t a good fit for this forum.

But you do not link those other good places. Can you tell me any Paths-Forward-compatible other places to use, particularly ones where discussion could reasonably result in MIRI changing?

If you disagree with Paths Forward, will you say why? And do you have some alternative approach written in public?

Also, more broadly, whether you will address these issues or not, do you know of anyone that will?

If the answers to these matters are basically "no", then if you're mistaken, won't you stay that way, despite some better ideas being known and people being willing to tell you?

The (Popperian) Fallible Ideas philosophy community ( http://fallibleideas.com ) is set up to facilitate Paths Forward (here is our forum which does this http://fallibleideas.com/discussion-info ), and has knowledge of epistemology which implies you're making big mistakes. We address all known criticisms of our positions (which is achievable without using too much resources like time and attention, as Paths Forward explains); do you?

---

Update (Dec 2019):

One person from MIRI responded the day I sent out the letter (Nov 9, 2017). He didn't answer anything I asked, but I decided to add the quotes for better completeness and record keeping. Below are Rob Bensinger's 3 emails, quoted, and my replies. After that he stopped responding.

Hi, Elliot. My short answer is that I think Popper is wrong; inductive reasoning works just as well as deductive in principle, though in practice we often have to rely on heuristic approximations of ideal inductive and deductive reasoning. The traditional problems with in-principle inductive reasoning (e.g., infinite hypothesis spaces) are well-addressed by Solomonoff's theory of algorithmic probability (http://world.std.com/~rjs/tributes/rathmannerhutter.pdf).

Have you written a serious and reasonably complete answer to Popper, or do you know of one that you will endorse, take responsibility for (if it's mistaken, you were mistaken), and address questions/criticisms/etc regarding?

And where is the Path Forward if you're mistaken?

I feel comfortable endorsing Solomonoff induction and Garrabrant induction (https://intelligence.org/2016/09/12/new-paper-logical-induction/) as philosophically unproblematic demonstrations that inductive reasoning works well in principle.

So you're disagreeing with Popper, but without addressing his arguments. If you're mistaken, and your mistakes have already been explained decades ago, you'll stay mistaken. No Paths Forward. Right?

I've read Popper before, and I believe the SEP when it says that he considered infinite hypothesis spaces a major underlying problem for induction (if not the core problem):

Popper gave two formulations of the problem of induction; the first is the establishment of the truth of a theory by empirical evidence; the second, slightly weaker, is the justification of a preference for one theory over another as better supported by empirical evidence. Both of these he declared insoluble, on the grounds, roughly put, that scientific theories have infinite scope and no finite evidence can ever adjudicate among them (LSD, 253–254; Grattan-Guiness 2004).

My claim is that Solomonoff induction addresses this problem handily, and that it more generally provides a good formal framework for understanding how and why inductive reasoning works well in practice.

I think we're getting off-topic. Do you agree or disagree with Paths Forward? Why? Do you have alternative written procedures for having issues like this addressed? Do you have e.g. a forum which would be a good place for me to reply to what you've said?

If I point out that you're mistaken about Popper's arguments and how they may be addressed, what happens next? BTW this would be much easier if there was a direct written answer to Popper by you, or by anyone else, that you were willing to take responsibility for. Why isn't there? That would also save effort for both of us – because responding to your unwritten views will require back-and-forth emails where I ask questions to find out what they are and get clarifications on what you're actually claiming. If your reasoning is that Popper is mistaken, so you don't want to bother properly answering him ... then your fallible criticism of Popper isn't itself being exposed to error correction very well.

In my understanding, there’s no one who speaks for Less Wrong (LW), as its representative, and is responsible for addressing questions and criticisms. LW, as a school of thought, has no agents, no representatives – or at least none who are open to discussion.

The people I’ve found interested in discussion on the website and slack have diverse views which disagree with LW on various points. None claim LW is true. They all admit it has some weaknesses, some unanswered criticisms. They have their own personal views which aren’t written down, and which they don’t claim to be correct anyway.

This is problematic. Suppose I wrote some criticisms of the sequences, or some Bayesian book. Who will answer me? Who will fix the mistakes I point out, or canonically address my criticisms with counter-arguments? No one. This makes it hard to learn LW’s ideas in addition to making it hard to improve them.

My school of thought (Fallible Ideas – FI) has representatives and claims to be correct as far as is known (like LW, it’s fallibilist, so of course we may discover flaws and improve it in the future). It claims to be the best current knowledge, which is currently non-refuted, and has refutations of its rivals. There are other schools of thought which say the same thing – they actually think they’re right and have people who will address challenges. But LW just has individuals who individually chat about whatever interests them without there being any organized school of thought to engage with. No one is responsible for defining an LW school of thought and dealing with intellectual challenges.

So how is progress to be made? Suppose LW, vaguely defined as it may be, is mistaken on some major points. E.g. Karl Popper refuted induction. How will LW find out about its mistake and change? FI has a forum where its representatives take responsibility for seeing challenges addressed, and have done so continuously for over 20 years (as some representatives stopped being available, others stepped up).

Which challenges are addressed? All of them. You can’t just ignore a challenge because it could be correct. If you misjudge something and then ignore it, you will stay wrong. Silence doesn’t facilitate error correction. For information on this methodology, which I call Paths Forward. BTW if you want to take this challenge seriously, you’ll need to click the link; I don’t repeat all of it. In general, having much knowledge is incompatible with saying all of it (even on one topic) upfront in forum posts without using references.

My criticism of LW as a whole is that it lacks Paths Forward (and lacks some alternative of its own to fulfill the same purpose). In that context, my criticisms regarding specific points don’t really matter (or aren’t yet ready to be discussed) because there’s no mechanism for them to be rationally resolved.

One thing FI has done, which is part of Paths Forward, is it has surveyed and addressed other schools of thought. LW hasn’t done this comparably – LW has no answer to Critical Rationalism (CR). People who chat at LW have individually made some non-canonical arguments on the matter that LW doesn’t take responsibility for (and which often involve conceding LW is wrong on some points). And they have told me that CR has critics – true. But which criticism(s) of CR does LW claim are correct and take responsibility for the correctness of? (Taking responsibility for something involves doing some major rethinking if it’s refuted – addressing criticism of it and fixing your beliefs if you can’t. Which criticisms of CR would LW be shocked to discover are mistaken, and then be eager to reevaluate the whole matter?) There is no answer to this, and there’s no way for it to be answered because LW has no representatives who can speak for it and who are participating in discussion and who consider it their responsibility to see that issues like this are addressed. CR is well known, relevant, and makes some clear LW-contradicting claims like that induction doesn’t work, so if LW had representatives surveying and responding to rival ideas, they would have addressed CR.

BTW I’m not asking for all this stuff to be perfectly organized. I’m just asking for it to exist at all so that progress can be made.

Anecdotally, I’ve found substantial opposition to discussing/considering methodology from LW people so far. I think that’s a mistake because we use methods when discussing or doing other activities. I’ve also found substantial resistance to the use of references (including to my own material) – but why should I rewrite a new version of something that’s already written? Text is text and should be treated the same whether it was written in the past or today, and whether it was written by someone else or by me (either way, I’m taking responsibility. I think that’s something people don’t understand and they’re used to people throwing references around both vaguely and irresponsibly – but they haven’t pointed out any instance where I made that mistake). Ideas should be judged by the idea, not by attributes of the source (reference or non-reference).

The Paths Forward methodology is also what I think individuals should personally do – it works the same for a school of thought or an individual. Figure out what you think is true and take responsibility for it. For parts that are already written down, endorse that and take responsibility for it. If you use something to speak for you, then if it’s mistaken you are mistaken – you need to treat that the same as your own writing being refuted. For stuff that isn’t written down adequately by anyone (in your opinion), it’s your responsibility to write it (either from scratch or using existing material plus your commentary/improvements). This writing needs to be put in public and exposed to criticism, and the criticism needs to actually get addressed (not silently ignored) so there are good Paths Forward. I hoped to find a person using this method, or interested in it, at LW; so far I haven’t. Nor have I found someone who suggested a superior method (or even any alternative method to address the same issues) or pointed out a reason Paths Forward doesn’t work.

Some people I talked with at LW seem to still be developing as intellectuals. For lots of issues, they just haven’t thought about it yet. That’s totally understandable. However I was hoping to find some developed thought which could point out any mistakes in FI or change its mind. I’m seeking primarily peer discussion. (If anyone wants to learn from me, btw, they are welcome to come to my forum. It can also be used to criticize FI.) Some people also indicated they thought it’d be too much effort to learn about and address rival ideas like CR. But if no one has done that (so there’s no answer to CR they can endorse), then how do they know CR is mistaken? If CR is correct, it’s worth the effort to study! If CR is incorrect, someone better write that down in public (so CR people can learn about their errors and reform; and so perhaps they could improve CR to no longer be mistaken or point out errors in the criticism of CR.)

One of the issues related to this dispute is I believe we can always proceed with non-refuted ideas (there is a long answer for how this works, but I don’t know how to give a short answer that I expect LW people to understand – especially in the context of the currently-unresolved methodology dispute about Paths Forward). In contrast, LW people typically seem to accept mistakes as just something to put up with, rather than something to try to always fix. So I disagree with ignoring some known mistakes, whereas LW people seem to take it for granted that they’re mistaken in known ways. Part of the point of Paths Forward is not to be mistaken in known ways.

Paths Forward is a methodology for organizing schools of thought, ideas, discussion, etc, to allow for unbounded error correction (as opposed to typical things people do like putting bounds on discussions, with discussion of the bounds themselves being out of bounds). I believe the lack of Paths Forward at LW is preventing the resolution of other issues like about the correctness of induction, the right approach to AGI, and the solution to the fundamental problem of epistemology (how new knowledge can be created).

I wrote this in Feb 2009 and emailed it to Yudkowsky. He didn't reply.

---

Dear Eliezer Yudkowsky,

I am writing to criticize some of your statements regarding Karl Popper. I hope this will be of interest.

http://yudkowsky.net/rational/bayes

Previously, the most popular philosophy of science was probably Karl Popper's falsificationism - this is the old philosophy that the Bayesian revolution is currently dethroning. Karl Popper's idea that theories can be definitely falsified, but never definitely confirmed, is yet another special case of the Bayesian rules

That isn't Popper's idea because he doesn't believe in definite falsifications. Falsifications are themselves tentative conjectures which must be held open to criticism and reconsidering.

Popper also doesn't assert that confirmations are never definite, rather he denies there is confirmation at all. The reason is that any given confirming evidence for theory T is logically consistent with T being false.

More generally, Popper's philosophy is not about what we can do definitely. He does not address himself to the traditional philosophical problem of what we can and can't be certain of, or what is and isn't a justified, true belief. While he did comment on those issues, his epistemic philosophy is not an alternative answer to those questions. Rather, his positive contributions focus on a more fruitful issue: conjectural knowledge. How do people acquire conjectural knowledge? What is its nature? And so on.

BTW, conjectural knowledge does not mean the probabilistic knowledge that Bayesians are fond of. Probabilistic knowledge is just as much of an anathema to Popper as certain knowledge, because the same criticisms (for example that attempting justification leads to regress or circularity) apply equally well to each.

Your claim at the end of the quote that Popperian epistemology is a special case of Bayesian epistemology is especially striking. Popper considered the Bayesian approach and told us where he stands on it. On page 141 of Objective Knowledge he states, "I have combated [Bayesian epistemology] for thirty-three years."

To say that something which Popper combatted for over three decades is a more general version of his own work is an extraordinary claim. It should be accompanied with extraordinary substantiation, and some account of where Popper's arguments on the subject go wrong, but it is not.

Popper was a hardworking, academic person who read and thought about philosophy extensively, including ideas he disagreed with. He would often try to present the best possible version of an idea, as well as a history of the problem in question, before offering his criticism of it. I would ask that a similar approach be taken in criticizing Popper. Both as a matter of respect, and because it improves discussion.

I created https://elliottemple.com

I particularly like the header image I made which depicts the intellectual traditions I care most about:

I also created https://elliottemple.com/reason-and-morality

It had an outline of philosophy with tons of links to learn more details.

If I were president I'd cancel most of the meetings, travel, etc, etc, and make some forums which are publicly readable.

There'd be a forum where all the countries have an account with write access. And one where all US politicians have write access. And one with a lot of media and intellectuals.

I think forum discussions are actually the best thing the US president could do.

Imagine if all the politicians, media personalities, etc., with bad ideas had to actually write about them on the record on a daily basis? Imagine if you just kept following up on discussions. What would they do? What most people do currently with me is just stop responding to things, which they can get away with socially because I have low prestige. But just refusing to answer forever wouldn't be a viable answer to the president's forum, and arguments/questions from the president and his staff. That'd look really bad to the world: Nancy Pelosi has been asked the same question for 5 days in a row and just won't answer at all.

But if they did answer they'd get pinned down.

They'd have to do evasive tactics: missing the point, playing dumb, trying to create confusion, saying unclear things, trying to make the discussion go in circles, etc. All that stuff can be called out, pointed out, and basically made to look as foolish as it is. People get away with that stuff in verbal formats with little followup, and behind closed doors, but not against the best debaters over a period of weeks with every word of it in the record. None of the bad guys have any method of dealing with that level of intellectual scrutiny.

They can lie, but the lies can be documented and the canonical links documenting lies can be repetitively posted every single time a lie is repeated. Staff can be hired to do that. That would cost a hell of a lot less than a wide variety of current, unimportant government departments. It's very easy by government standards.

And how do you deal with media questions? Press briefings are so incomplete that it's hard for people to see who's right and why. What if all the bullshit the press kept bugging Trump about was on a forum where some staff members replied with canonical links over and over so everything was getting answered? How would the media continue to ignore the main points, which they currently ignore, if it was being linked in reply to them every time they talked?