ASSUMPTIONS AND MODELS

The lack of appreciation by those involved in the creation of imaginary value claims in health technology assessment is not only a lack of appreciation of the need to meet the standards of normal science, including the axioms of fundamental evidence but also a failure to appreciate the role of assumptions in creating models; models which should be abandoned unless a case can be made that the modeling is short term and the value claims empirically evaluable.

The focus on imaginary models illustrates one of the fundamental weaknesses in the rejection of normal science where claims are credible, evaluable, and replicable, the apparent lack of interest in what may be called ‘progress’; the discovery of new, yet provisional, facts. Discovery through a well tried application in the physical sciences and the mature social sciences of ‘conjecture and refutation’ has been accepted for almost a century, since the seminal contribution of Karl Popper to overturn logical positivism (or logical measurement) in the 1920s, culminating with his masterwork in the philosophy of science  The Logic of Scientific Discovery, 1934 (as Logik der Forschung, English translation in 1969) which pointed to the role of falsification. This addressed the asymmetry in the scientific method where proof is not possible, but statements or value claim can be falsified.

Progress and falsification are the main reasons why the approximate information belief, held globally by a majority of those working in the field, is described as a meme rather than a paradigm. A paradigm admits to the possibility of it being overthrown, with a successor paradigm resolving puzzles which have hitherto been unresolvable. Thus progress to a new set of claims and a deeper understanding of the real world, including therapy impacts, is possible. A meme admits of no successor. Believers in a meme, to paraphrase Richard Dawkin’s A Devil’s Chaplain are impelled by a deep inner conviction that something is true or right despite the possibility of appeals to evidence or reason. The positive virtue in seen in an unshakeable belief and strength of convictions as made clear in CHEERS 22. Indeed, lack of evidence may be a virtue; if the agreed end product is a set of non-evaluable yet self-referential value claims that can be conclusively validated, then the believer occupies the high ground. The greater the mystery of the mystical ratio preference score with ordinal properties, the closer the believer clings to that obviously weird interpretation. For believers truth is consensus; appeals to evidence are irrelevant.

There is a further element in this memetic belief system that also defies common sense (and elementary logic): the belief that assumptions driving a model structure and the parameters within the model can be justified because they are realistic. You may claim an assumption based on prior observation is justifiable for modeling the future, but that is a fact of your psychology not of logic. To make such a claim, justifying the list of ‘preferred’ assumptions that typically populate a model discussion, fails Hume’s problem of induction and the belief in verification in logical positivism. The fact that all past futures have resembled past pasts does not mean that future futures will resemble future pasts. The future is unknown; if we want to make claims then we cannot justify modeled assumptions on these grounds. Any set of assumptions has the same status as any other set. This leads, of course, to the possibility of hundreds of competing non-evaluable value claims for the same product and comparator. The CHEERS 22 guidance justifies this; if there is no way in logic or in empirical evaluation, to justify one model over another then all are equally valid (or invalid).

The solution to this absurd multiverse of competing models is quite clear: any model that fails to produce empirically evaluable value claims in a meaningful timeframe must be abandoned, together with the methodology that supports such constructs.

REFERENCES

Langley P. Peter Rabbit is a Badger in Disguise: Deconstructing the Belief System of the Institute for Clinical and Economic Review in Health Technology Assessment. InovPharm. 2021; 12(2): No.20 https://pubs.lib.umn.edu/index.php/innovations/article/view/3992/2855

Langley P. Nothing to cheer about: Endorsing imaginary economic evaluations and value claims with CHEERS 22. Maimon Working Papers No 2, January 2022 https://maimonresearch.com/nothing-to-cheer-about-endorsing-imaginary-economic-evaluations-and-value-claims-with-cheers-22