You should eliminate rival theories until you have exactly one candidate theory and then act on that. We thus dodge any issues of comparing two still-standing theories using some sort of criterion.
And that's it. The problem is solved. When there is only one candidate theory the solution is dead easy. But this solution raises a new problem which is how to deal with all the rival theories (in short order).
If you act on a theory while there are any active rivals that is coercion, which is the cause of distress. You are, roughly, forsaking a part of yourself (without having dealt with it in a rational fashion first).
Often we don't know how to resolve a controversy between two theories promptly, perhaps not even in our lifetime. But that does not mean we are doomed to any coercion. We can adopt a *single theory* with *no active rivals* which says "I don't know whether A or B is better yet, but I do know that I need to choose what to do now, and thus I will do C for reason D." A and B talk about, say, chemistry, and don't contain the means to argue with this new theory proposed -- they don't address the issue now being considered of what to do given the unsettled dispute between A and B -- so are not relevant rival theories, so we end up with only one theory about what to do, this new one we just invented. And acting on this new theory clearly does not forsake A or B; it's not in conflict with them.
We might invent two new theories, one siding more with A, and one more with B, and thus have a new conflict to deal with. But then we have *new problem* which does not depend on resolving the dispute between A and B. And we can do as many layers of reinterpretations and meta-theorizing like this as we want. Coercion is avoidable and practical problems of action are soluble promptly.
If it really comes down to it, just formulate one new theory "I better just do C for reason D *and* all arguments to the contrary are nothing but attempts to sabotage my life because I only have 3 seconds left to act." That ought to put a damper on rival theories popping up -- it'd now have to include a reason it's not sabotage.
One could still think of a rival theory which says it wants to do E because of B and this isn't sabotage b/c the A-based C will be disastrous and it's trying to help" or whatever. There is no mechanical strategy for making choices or avoiding coercion. What I mean to illustrate is we have plenty of powerful tools at our disposal. This process can go wrong, but there is plenty of hope and possibility for it to go right.
BTW this does not only apply to resolving rival theories *for purpose of action* or *within oneself*. It also works for abstract theoretical disputes between different people.
Suppose I believe X and you believe Y, *and we are rational*. Think of something like MWI vs copenhagen -- theories on that scale -- except something that we don't already know the answer to.
So we argue for a while, and it's clear that you can't answer some of my questions and criticisms, and I can't answer some of yours.
Most people would say "you haven't proven me wrong, and i see problems with your theory, so i am gonna stick with mine". That's called bias.
Some people might be tempted to analyze, objectively, which theory has more unanswered questions (weighted by importance of the question), and which theory has troubled with how many criticisms (weighted by amount of trouble and important of criticism). And thus they'd try to figure out which theory is "better" (which doesn't imply it's true, or even that the other is false -- well of course strictly they are both false, but the truth might be a minor modification of the worse theory).
What I think they've done there is abandon the critical rationalist process and replace it with a misguided attempt to measure which theories are good.
What we should do is propose *new theories* like "the current state of the debate leaves open the question of X and Y, but we should be able to all agree on Z so for issue A we should do B, and for issue C we should do D, and that's something we can all agree on. We can further agree about what research ought to be done and is important to do to resolve questions about both X and Y." Thus we can make a new *single theory* that *everyone on both sides can agree to* which does not forsake X or Y. This is the *one rational view to take of the field*, unlike the traditional approach of people being in different and incommensurable camps. This view will leave them with *nothing to argue about* and *no disagreement*.
Of course, someone might say it's mistaken and propose another view of the same type. And so we could have an argument about that. But this new argument does not depend on your view in the X vs Y dispute. It's a new problem. Just like above. And if it gets stuck, we can make another meta-theory.
This approach I advocate follows the critical rationalist process through and through. It depends on constructing new theories (which are just guesses and may go wrong) and criticizing them. It never resorts to static criteria of betterness.