How should we think about gradations of certainty in Critical Rationalist terms?
don't.
there are the following 3 situations regarding one single unambiguous problem. this is complete.
1) you have zero candidate solutions that aren't refuted by criticism.
gradations of certainty won't help. you need to brainstorm!
2) you have exactly one candidate solution which is not refuted by criticism.
tentatively accept it. gradations of certainty won't help anything.
(if you don't want to tentatively accept it – e.g. b/c you think it'd be better to brainstorm and criticize more – then that is a criticism of accepting it at this time.)
3) you have more than one candidate solution which is not refuted by criticism.
this is where gradations of certainty are mainly meant to help. but they don't for several reasons. here are 6 points, 3A-3F:
3A) you can convert this situation (3) into situation (1) via a criticism like one of these 2:
3A1) none of the ideas under consideration are good enough to address their rivals.
3A2) none of these ideas under consideration tell me what to do right now given the unsettled dispute between them.
(if no criticisms along those lines apply, then that would mean some of the ideas you have solve your problem. they tell you what to do or think given the various ideas and criticism. in which case, do/think that. it's situation (2).)
3B) when it comes to taking action in life, you can and should come up with a single idea about what to do, which you have no criticism of, given the various unresolved issues.
3C) if you aren't going to take any actions related to the issue, then there's no harm in leaving it unresolved for now and not knowing the answer. you don't have to rate gradations of certainty, you can just say there's several candidates and you haven't sorted it out yet. you would only need to rank them, or otherwise decide which to pursue, if you were going to take some action in relation to the truth of this matter (in which case see 3B)
3D) anything you could use to rank one idea ahead of another (in terms of more gradations of certainty, more justification, more whatever kind of score) either does or doesn't involve a criticism.
if it doesn't involve a criticism of any kind, then why/how does it provide a reason to rank one uncriticized reason above another one (or add to the score of one over another)?
if it does involve a criticism, then the criticism should be addressed. criticisms are explanations of problems. addressing it requires conceptual thinking such as counter-arguments, explanations of why it's not a problem after all in this context, explanations of how to improve the idea to also address this criticism, etc. either you can address the criticism or you can't. if you can't that's a big deal! criticisms you see no way to address are show stoppers.
one doesn't ever have to act on or believe an idea one knows an unanswered criticism of. and one shouldn't.
also to make criticism more precise, you want to look at it like first you have:
- problem
- context (background knowledge, etc)
- idea proposed to solve that problem
then you criticize whether the idea solves the problem in the context. (i consider context implied as part of a problem, so i won't always mention it.)
if you have a reason the idea does not solve the problem, that's a show stopper. the idea doesn't work for what it's supposed to do. it doesn't solve the problem. if you don't have a criticism of the idea successfully solving the problem, then you don't have a criticism at all.
this differs from some loose ways to think about criticism which are often good enough. like you can point out a flaw, a thing you'd like to be better, without any particular problem in mind. then when you consider using the idea as a solution to some problem, in some context, you will find either the flaw does or doesn't prevent the idea from solving that problem.
in general, any flaw you point out ruins an idea as a solution to some problems and does not ruin it as a solution to some other problems.
3E) ranking or scoring anything using more than one variable is very problematic. it often means arbitrarily weighting the factors. this is a good article: http://www.newyorker.com/magazine/2011/02/14/the-order-of-things
3F) suppose you have a big pile of ideas. and then you get a list of criticisms. (it could be pointing out some ideas contradict some evidence. or whatever else). then you go through and check which ideas are refuted by at least one criticism, and which aren't. this does nothing to rank ideas or give gradations. it only divides ideas into two categories – refuted and not refuted. all the ideas in the non-refuted category were refuted by NONE of the criticism, so they all have equal status.
i think what some people do is basically believe all their ideas are wrong, bad, refuted. and then they try to approach gradations of certainty by which ones are less wrong. e.g. one idea is refuted by 20 criticisms, and another idea is only refuted by 5 criticisms. so the one that's only refuted 5 times has a higher degree of certainty. this is a big mistake. we can do better. and also the way they count how much is one criticism (with or without weighing how much each criticism counts) is arbitrary and fruitless.
something they should consider instead is forming a meta idea: "Idea A is refuted in like a TON of ways and seems really bad and show-stopping to me b/c... Idea B has some known flaws but i think there's a good shot they won't ruin everything, in regards to this specific use case, b/c... And all the other ideas I know of are even worse than A b/c... So i will use idea B for this specific task."
then consider this meta idea: do you have a criticism of it, yes or no? if no, great, you've got a non-refuted idea to proceed with. if you do have a criticism of this meta idea, you better look at what it is and think about what to do about it.
for a lot more info, see this post: http://curi.us/1595-rationally-resolving-conflicts-of-ideas