If a patient is suffering from broken legs due to brittle bones we may hypothesise a causal chain and attempt treatment. For example, oral administration of a calcium supplement, hoping that the level of calcium in the blood will rise, hoping that the extra calcium in the blood will lead to denser bones, hoping that the denser bones will be stronger and not break.

We can measure various links in this chain. Do blood tests show that the level of calcium has risen? If not we could try adding a buffer to stop stomach acid interferring with absorption. Does ultrasound bounce differently due to the increased density of bone? If bone density is not increased we might try adding a nutritional supplement to aid the action of the osteoblasts. Is the micro-structure poor, with cracks that leave the denser bone no stronger than before and just as prone to fractures. Perhaps weight bearing exercise would improve the micro-structure.

Clearly following the steps in the causal chain is important and from a neutral, scientific perpsective we can see each link on its own as a treatment that may or may not have an effect. However, from the medical perspective we need to distinguish between measurements that check whether our attempts to deliver treatment are succeeding in delivering treatment and measurements that check whether our attempts to deliver treatment are succeeding in having a beneficial effect.

Some-one has to check directly that the bone density gains one gets from calcium tables do in fact translate to stronger bones and fewer fractures. Improving the bone density is a treatment for recurring fractures. Does it work? Do fractures stop occurring? That is an empirical question that needs to be researched. If the outcome of the research is sufficiently clear cut practitioners may subsequently use bone density as a proxy for bone strength.

The deep problem is that proxies do not possess universal validity. If medical researchers come up with a new way of encouraging the body to increase bone density, the question of micro-structure and the question of whether the proxy is valid recurr. Perhaps the new treatment is giving rise to dense bone that is cracked and weak. Worse still, there is the risk of rejecting and abandoning the new treatment because it did not improve bone density when it did improve micro-structure and strength and reduce fractures. Medical research is difficult because its essential novelty is always putting the validity of important proxy outcomes in question.

Treatfection is the confusion of treatment with effect. We follow the causal chain of a medical treatment, but not to the very end, and forget that we are measuring treatment and not yet effect. The treatfected medical researcher declares that his treatment for recurring fractures is effective because bone density has improved, without waiting to see if his patients have stopped suffering fractures. Perhaps follow-up in a year or two will confirm his claims. Perhaps follow-up will refute his claims. Perhaps, due to a severe case of treatfection, follow-up will not be seen as necessary.

Treatfection is fatal to scientific medicine. Some quack remedies are pure bonkers but many have a causal chain with a valid initial segment. Homeopaths really do dilute their remedies. If you are content to measure concentrations and use the weakness of the remedy as a proxy for effectiveness, then you will find that homeopathy works. Astrologers use the same orbital parameters that astronomers do. If you are content to check that the planets have reached the postions that the astrologer predicted, and use those positions as proxy for human fate, then you will conclude that astrology works.

On the other hand, being too much the purist and revalidating every proxy, every time, would be so expensive that medical research would grind to a halt. Treating treatfection too agressively will kill the patient! Some judgement is needed as to whether a new treatment puts the validity of a proxy in question.

Consider the rival therapies for depression, fluoxetine (tradename Prozac) and Cognitive Behavioural Therapy (CBT).

Does popping a pill treat depression? Has the patient's health improved? We could ask his employer. Perhaps we learn that the patient used to sit blankly at work, starting out of the window, and now he gets on with his tasks, performing to an acceptable level of quality. We could ask his wife. Perhaps we learn that he used to sit on the sofa, staring at a TV that was off, but now he takes out the trash and walks the dog. In practise we do neither. We have the patient fill in a questionnaire, and fill it in again after the treatment. Is he still depressed? We take his word for it.

At first sight that looks a bit odd. We have all met folk who grumble, whine, and go on and on about how bad life is. If we depend on self-reporting are we not in fact treating whining rather than depression? Are we really willing to declare victory when the patient stops whining, even if he loses his job and his marriage breaks up?

The underlying logic of our approach is that although the questionnaire is merely a proxy we do not think that fluoxetine will damage its validity. We simply don't believe that fluoxetine is so narrowly and strangely specific that it could cure the patients verbal behaviour while leaving him dysfunctional, failing at work and at home. Yes, we are cutting corners on evaluating the treatment, but medical research is resource-constrained, and it is important to cut some corners least the money run out before we get to testing the treatment that works. The value of medical research is not guaranteed to be positive. Cut too many corners and you end up creating false facts that do more harm than good, so cutting corners is fraught. Nevertheless I think most people accept that administering questionnaires to patients in trials of pharmacological treatments is a reasonable risk to run, given the likely costs of more thorough approaches.

Does Cognitive Behavioural Therapy (CBT) treat depression? Here the theory is that the repetitions and self-reinforcements of our inner dialogue play a key role, creating vulnerability to adverse life events, perpetuating depression, tightening the downward spiral. CBT targets this cognitive behaviour, encouraging patients to notice and challenge their habitual responses. Again we administer questionnaires, and this time we can see the questionnaires as measuring treatment, analogous to measuring whether calcium in the patients stomach is ending up in their blood. The therapist is teaching the patient to stop repeating to himself how crap life is, and the questionnaire is arguably measuring the therapists success in delivering the treatment.

Does the treament do any good? We want to know the effect of the treatment. The danger of treatfection is often missed. The usual approach is to understand the questionnaire as measuring the effectiveness of the therapy at relieving the patients depression. Now we have a single questionnaire whose role is ambiguous between measuring the therapist's effectiveness at delivering the therapy (did the patient's cognitive behaviour change?) and measuring the therapy's effectiveness at lifting the patient's depression (what did the employer and the spouse say?).

I cannot see a good reply to this methodological critique. It is certainly possible that the questionnaires used for assessing depression are robust against the interventions of Cognitive Behavioural Therapy, but this is a conjecture, not an argument. I cannot see a way of assessing depression without asking, and if CBT is to be science and not transfection this will involve asking family, friends and employers. This will be difficult and expensive.

Worse, I cannot see the problem going away. One hopes that CBT will prove modestly useful and provide insights leading to the proposal of a CBT2 that may be more useful. Suppose we have questionnaires that we give to patients and which retain their validity as proxies for depression during treatment using CBT techniques. Will they retain their validity during treatment using CBT2 techniques? The point of CBT2 is to be more potent, but what will that enhanced potency achieve, beating depression or breaking the proxy? The expensive and difficulty of third party questionnaires looks unavoidable, both today and tommorrow.