The following is intended as a statement of my position but does not attempt to argue for it in detail.
The concept of a critical preference makes the common sense assumption that there are strong and weak arguments, or in other words that arguments or ideas can be evaluated on a continuum based on their merit.
The merit of an idea is often metaphorically stated in terms of its weight (e.g. Popper wrote "weighty though inconclusive arguments", Objective Knowledge p 41). It's also commonly stated in terms of probability or likeliness. And it's also stated in terms of ranking or scoring ideas to see which is best.
Ideas do have merit, and they can be closer or further from the truth (more or less truthlike, if you prefer). However, we never know how much merit an idea has. We can't evaluate ideas that way.
(BTW suppose we could evaluate how much merit ideas have. A second assumption is that doing so would be useful and that it would make sense to prefer the idea with more merit. That should not be assumed uncritically.)
Popper did not give detailed arguments for the idea that we can or should evaluate arguments by their strength or amount of merit. That's why I call it an assumption. I think he uncritically took it for granted without discussion, as have most (all?) other philosophers.
In the strength based approach, an idea could score a 1, or a 2, or a 20. In Popper's view, the numbers don't have an absolute meaning; they can only be compared with the scores of other ideas. Or in other words, we never know how close to the truth we've come on an absolute scale. In this approach, an idea can have infinitely many different evaluations.
In my approach, an idea can only have three possible evaluations. An idea can be unproblematic (non-refuted), problematic (refuted), or we're unsure. Ignoring the possibility of not taking a stance, which isn't very important, an idea gets a boolean evaluation: it's either OK or not OK.
If we see a problem with an idea, then it's no good, it's refuted. We should never accept, or act on, ideas we know are flawed. Or in other words, if we know about an error it's irrational to continue with the error anyway.
On the other hand, if we have two ideas and we can see no problem with either, then we can have no reason to prefer one over the other. This way of assessing ideas does not allow for the middle ground of "weighty though inconclusive arguments".
If an idea is flawed, it may have a close variant which is unproblematic. Whenever we refute an idea, we should look for variants of the idea which have not been refuted. There may be good parts which can be rescued.
My approach is in significant agreement with Popper's epistemology because it does not allow for the possibility of ideas having support. Some people would say we can differentiate non-refuted ideas by how much support each has, but I follow Popper in denying that.
Popper's alternative to support is criticism. I accept the critical approach. Where I differ is in not allowing an idea to be both criticized and non-refuted. I don't think it makes sense to simultaneously accept a criticism of an idea, and accept the idea. We should make up our mind (keeping open the possibility of changing our mind at any time), or say we aren't sure.
As I see it, a criticism either points out a flaw in an idea or it doesn't. And we either have a criticism of the criticism, or we don't. A criticism can't contradict a theory and be itself non-refuted, but also fail to be decisive. On what grounds would it fail to be decisive, given we see no flaw in it?
Let's now consider the situation where we have conflicting non-refuted ideas, which is the problem that critical preferences try to solve. How should we approach such a conflict? We can make progress by criticizing ideas. But it may take us a while to think of a criticism, and we may need to carry on with life in the meantime. In that case, the critical preferences approach attempts to compare the non-refuted ideas, evaluate their merit, and act on the best one.
My approach to solving this problem is to declare the conflict (temporarily) undecided (pending a new idea or criticism) and then to ask the question, "Given the situation, including that conflict being undecided, what should be done?" Answering this new question does not depend on resolving the conflict, so it gets us unstuck.
When approaching this new question we may get stuck again on some other conflict of ideas. Being stuck is always temporary, but temporary can be a long time, so again we'll need to do something about it. What we can do is repeat the same method as before: declare that conflict undecided and consider what to do given that the undecided conflicts are undecided.
A special case of this method is discussed here. It discusses avoiding coercion. Coercion is an active conflict between ideas within one mind with relevance to a choice being made now. But the method can be applied in the general case of any conflict between ideas.
My approach accepts what we do not know, and seeks a good explanation of how to proceed given our situation. It is always possible to find such an explanation. It may sound difficult, but actually you already do it dozens of times per day without realizing it. Just like people must use conjectures and refutations to understand each other in English conversations (and must use them in all their thinking), and when they first hear that idea it sounds bizarre, but they already do it quickly, reliably, and without realizing what they are doing.