The Management Series

To sign up for our newsletter, please click here.

Jun 2018

Data v. Judgment

Having the information available to measure things like performance is one of the boons of modern management.

But Jerry Muller, a history professor at Catholic University in Washington who specializes in the history of capitalism, argues allowing metrics to supplant human judgment could turn out to be one of this century’s gravest management mistakes.

“In general a focus on measured targets discourages innovation—if the goals are specified in advance, there is little room for initiative, not to speak of risk-taking,” Muller writes in “Who Needs Judgment When You Have Data? You Do,” for Knowledge@Wharton. (Muller adopted it from his book The Tyranny of Metrics. An audio of an interview he did with Wharton on the book is available at the same link.)

Muller sees the reliance on metrics as a natural extension of the rise of “managerialism” preached in business schools.

“Managerialism is the belief that all organizations are fundamentally the same, and can be managed by the same tools. For devotees of managerialism, management requires not experience or judgment but technique,” he writes. “Advocates of managerialism see no problem in top executives moving from a corporation in one field to a corporation in another field; or the head of a corporation becoming the president of a university or the head of a department of the federal government.”

All you really need as a manager, he explains, is “a mastery of the same metric tools.”

Muller says it has become popular to “replace judgment, acquired by personal experience and talent, with numerical indicators of comparative performance based upon standardized data” and to “motivate people…by attaching rewards and penalties to their measured performance.”

And it is with these rewards and punishments that Muller takes issue.

Metrics can be useful as a source of information for making changes or altering strategy, he argues. But directly tying performance to metrics all too often results in gaming the system.

In policing, for example, “if the rate of major crimes in a district becomes the metric according to which officers are promoted, some officers will respond by simply not recording crimes or downgrading them from major offences to misdemeanors,” he says. “Or take the case of surgeons…[W]hen metrics of success and failure are made public—and hence affect their reputation and income—some respond by refusing to operate on patients with more complex problems, whose surgical outcomes are more likely to be negative.”

A second problem is “what gets measured and rewarded is more likely to get done—but at the expense of other essential functions that get overlooked,” Muller warns. “Thus, teachers ‘teach to the test’—shortchanging elements of English and math that are not tested, not to speak of other subjects…CEOs trying to maximize stock prices boost quarterly profits by cutting back spending on R&D and on the development of their work force.”

Muller reiterates that standardized measurement has a useful role—but as a tool, not as an ultimate measure of success or failure.

“The effective use of metrics requires judgment, and judgment is a matter of discrimination: of knowing which cases are harder, more challenging or more important than can be captured by a standardized measurement,” Muller writes. “And that discrimination is only possible when the judgment is linked to a sense of the overall purposes and peculiarities of one’s office, department, clinic or organization.”