Cost/Benefit Simplification

Introduction

Cost benefit analysis is an attempt to make a rational choice when faced with choosing between n mutually exclusive plans of action P₁,...,P_n. The key assumption is that for each course of action, laid out in plan P_i, there is both a cost C_i and a benefit B_i. Cost benefit analysis advocates choosing plan P_i such that B_i - C_i is maximal.

This is uncontentious in theory, but problematic in practice. The main practical difficulty lies in quantifying the benefits. This difficulty is rarely faced. The procedures adopted are usually sufficiently muddled to obscure the way in which the problem has been overcome.

First we describe the two common ways of overcoming the difficulties of estimating the benefits, then we say why this approaches are typically covert rather than overt. Finally we offer an example, and a postscript.

Method one: Average benefit.

While the B_i's are individually difficult to estimate, a decision maker may feel more confident of the average level of benefit, B. The plan P_i is then chosen to maximize B-C_i. Notice that even if B is no easier to estimate than the B_i, the decision is insensitive to errors in estimating B, which appears to be a factor in favour of the procedure.

Method two: Proportional benefit

Without loss of generality

B_i = A_i C_i

AC_i - C_i = (A-1) C_i

Again, a sensitivity analysis reveals that the decision taken is insensitive to the value of A, reassuring those who are concerned that estimating A may be no easier than estimating the B_i.

Covert, not overt

A cost benefit analysis is attempted in response to the uncomfortable realization that it is adequate neither to pick the cheapest, nor to pick the "best".

If one responds to the difficulties of estimating the B_i by adopting method one, one has in fact fallen back to choosing the cheapest. This is not something that one does deliberately. It comes about accidently. One may for instance become horrified by the uncertainties inherent in estimating the B_i. Suppose each B_i is a sum of various benefits B_i = B_i,1 + B_i,2 + B_i,3 + ... of widely varying degrees of uncertainty. It is natural to drop the most uncertain terms. Typically though, it is the large terms that are the uncertain ones. One ends up with an analysis of the various benefits that is intended to focus on the certain benefits, but which in practice only includes the small benefits. The large benefits that cause the B_i to differ, one from another, are omitted because they are too hard to estimate. On has the form of a cost/benefit analysis, because the B_i are not literally identical, but one has the substance of method one, with a single value for B uniform across the plans, P_i

If one responds to the difficulties of estimating the B_i by adopting method two, one has in fact fallen back to choosing the "best". Notice the quotes around the word "best". One is literally choosing the most costly, but it doesn't seem like that. Psychologically, one feels that one is merely using cost as a proxy for quality. One is formally maximising B_i - C_i. The assumption of an A such that B_i= AC_i, means that one is just as much maximising B_i as C_i, and can in a sense say that one is choosing the best. It is the decision to abandon the attempt to make a direct estimate of B_i, and to use AC_i as a proxy that puts the quote marks around "best". Naturally the decision to abandon the attempt to estimate B_i directly is seldom taken overtly. Just as in the previous paragraph, the elements of B_i that are easy to estimate are included in the cost/benefit analysis, thus preserving the form of the analysis, but the elements of B_i that are dropped as too difficult to estimate are typically the large items that make one plan better value for its cost than another. Dropping them eliminates the substance of the cost/benefit analysis, leading to the choice of the most costly plan of action.

The example of big computer languages versus small computer languages

It is common in computing to enthuse about the number of features that a software package offers. There is something strange about this. There is a useful jargon word: affordances. The cool things that you can do using a software package are its "affordances". The program's "affordances" are what it lets you do. They are the benefits of using the program. Unfortunately, there is a price to pay. To obtain the benefits you must master the features by which one operates the program. Perhaps one can indeed produce tuneful music using the a composition package, but perhaps there are numerous features to be mastered in order to get it to work. The features are the cost. It is strange to enthuse about the features. You have to spend valuable time learning them. Yet it is not that strange. The features and the affordances are intimately linked. One cannot expect to compose polyphonic music, unless the package has the feature of multiple voices. Certainly one will spend time learning how to use the multiple voice feature, that is its cost. One never imagines that one could obtain the benefits afforded by a software package suitable for writing polyphonic music, without paying the cost of learning how to use the multiple voice feature.

When one counts the features, the more the merrier, one is using method two, proportional benefit, and using cost as a proxy for benefit.

The opposite trend is visible when computer programming languages are praised for being small and elegant. The praise is justified as far as it goes. Small, elegant languages are easier and faster to learn than big bloated languages. These are real costs, and nobody doubts that they count on the minus side when computing B_i - C_i. What are the benefits of a big bloated language. They are often contingent. Common Lisp is notorious for having a format command that has an option for writing numbers using Roman numerals. Actually, if you are writing a program to render the ordered list of HTML, you need to cope with numbering types "I" and "i", upper and lower case Roman numerals. It is very convenient that Common Lisp provides them for you. Notice the "if". "if not", well the feature is just bloat. It depends. It depends on what programs you are going to write in the future, and the future is uncertain. This uncertainty undermines the practically of cost benefit analysis. One easily ends up discounting the uncertain benefits of features might never use, One ends up with method one, average benefit, minimising cost, and never counting lost benefits.

Social Dynamics

I see an underlying social dynamic here that keeps both extreme camps well stocked with true believers.

Suppose that one is force fed minimalism. One's experience is that on is deprived of valuable features. It is the features one does not have, but which would have been very useful, that are foregrounded by the experience of using minimalist software. One is primed to believe that all features are useful.

Suppose instead that one suffers obscure bugs due to features that one is unaware of, or which one learnt once, but have forgotten through long misuse. One's experience of big, bloated systems is an experience of the cost of extra features. One does not miss the features one has. One is primed to shrug ones shoulders at extra features, and see an average benefit model as reasonable.

Thus there is a traffic between the two camps, each one training up a new generation of adherents for the other. It will not stop, not while the distinction between features and affordances is invisible.

Next time you are reading documentation, ask yourself "Am I reading about what this software lets me do, or about how I go about doing it?". "How would I disentangle the costs and the benefits?". For example, Object Oriented Programming is a popular new programming paradigm. An Object Oriented programming language lets you write Object Oriented programs. Is that an affordance? No, it is a feature. You have to write Object Oriented programs in order to get "the benefit". That is to say you have to pay the price of learning the Object Oriented Programming style, and then use it. This then yields a benefit, which is not stated, and is presumably hard to quantify. All benefits are hard to quantify. That is the rock on which cost/benefit analysis often founders. It is not a problem unique to Object Oriented programming.

The software industry has slipped into talking of features not affordances. It does this generally, not just for OOP. How does one respond? This is a matter of temperament not logic. If one is cynical and jaundiced, one doubts the existence of the unexplained benefits and falls into Method One: Average Benefit. If one is naive and optimistic, one has faith that effort mastering new features will be rewarded with corresponding expressive power, and users Method Two: Proportional Benefit.

There is no right answer to this conundrum. If a cost/benefit analysis is attempted, the cost of performing it is likely to escalate, due to unmanageable uncertainties in estimating benefits. All that is known beyond doubt is that the cost/benefit analysis must be abandoned before its own costs exceed the maximum difference between the net benefits of the various plans between which it is attempting to chose. Otherwise the cost of planning will exceed the maximum possible benefit that the best decision could yield over the worst. Notice the circularity. Should we persevere with a cost/benefit analysis, even though it is hard to estimate the benefits, our should we fall back on one or other of the two worthless approximations discussed above? We have three plans P₁, P₂, P₃, with costs C_i and benefits B_i. The benefits of the three different plans are all proving hard to estimate ...