Robert McNamara's Quantitative Fallacy
Introduction
Robert McNamara is a Harvard graduate who headed Ford and served as US Secretary of Defense from 1961 to 1968, then as President of the World Bank from 1968 to 1981.
The fallacy concerns decision-making that relies completely and exclusively on quantitative data while ignoring all other sources — on the pretext that those other sources of information cannot be proven.
McNamara’s words: “The first step is to measure whatever can be easily measured. This is OK as far as it goes. The second step is to disregard that which can’t be easily measured or to give it an arbitrary quantitative value. This is artificial and misleading. The third step is to presume that what can’t be measured easily really isn’t important. This is blindness. The fourth step is to say that what can’t be easily measured really doesn’t exist. This is suicide.”
McNamara believed that the United States lost its war in Vietnam because of this fallacy, which was embraced by the generals of that era. They relied mainly on enemy body-count statistics without taking any other information sources into consideration when making decisions, assuming such other sources could not be quantitatively proven.
This fallacy is one that many decision-makers in the business world fall into — which is why I’m discussing it in this article.
Unpacking the Fallacy
The text of the fallacy is very realistic, and although it’s old, it still applies literally to many decision-makers today. Decisions made on the basis of statistics and quantitative research can be good, but it ultimately depends on the quality of those data and the data-collection methodology the research followed. Anyone who uses quantitative reports to make decisions, before looking at any research’s recommendations, must look at the data-collection methodology and the quality control applied to data collectors in that research — to judge whether the results are reliable and accurate or not.
As for the part that what cannot be (or is difficult to) measure does not exist: in many cases this is no more than an assumption by decision-makers. With research, more than one way is found to measure those things they assumed didn’t exist — either through ignorance or intent. There’s an excellent English book on this subject titled How to Measure Anything.
As for the part that what cannot be (or is difficult to) measure isn’t important: this is very common, and here the role of the CX function comes in — setting clear markers for what matters to measure and what doesn’t, in terms of those indicators’ linkage to and direct or indirect impact on the customer experience, and in terms of their effect on behavioral customer loyalty. That way the door isn’t left open for just anyone to offer their opinion on what should or shouldn’t be measured.
The more dangerous matter among decision-makers — one the fallacy doesn’t explicitly state — is that some decision-makers manipulate the things being measured to serve their interests and to make it appear their performance is good, so they obtain incentives tied to service-level achievement. When questioned, some excuse themselves on the grounds that they found the measurement mechanism this way when they started, or that they believed it was being measured this way. We may excuse them for ignorance, but if it’s intentional on any party’s part, that’s false testimony. If the mistake is confirmed, the measurement mechanism must be corrected immediately to reflect reality — because the point of measurement to begin with is to know the current state in order to manage and improve it, not to measure for measurement’s sake.
That’s why establishing a centralized department for any kind of measurement may be a good idea — to govern and institutionalize all the indicators being measured in terms of definition, frequency, sources, and methodology. Also, measuring the Voice of Customer (VoC) through market research may be valuable for challenging the results of operational indicators that can be manipulated, and senior leadership always places greater credibility on the customer’s voice: even if operational indicators say things are fine and the customer doesn’t see it that way, it means things are not fine.
Closing
Managing measurement — and everything related to it, from distributing results to escalating them to executive leadership — is a sensitive matter that requires specialists. For this reason, it is good for the work of this department to be audited by internal audit or by external auditors, to ensure this sensitive unit is managed effectively. Otherwise the saying may apply: “a weapon in the hands of an unskilled man wounds the wielder.”
← Articles MOC