Human Judgment Still Matters: Why Analytics and AI Can’t Replace Common Sense

The Limits of Analytics and AI

The search for a way to replace human judgment with analytics and Artificial Intelligence (AI) will never get us to the promised land. Why? Because there will always be human judgment involved. Some human made the decision of how to look at things. Plus, we have first-hand evidence that we still get plenty of things wrong.

The argument of “all we did was look at the numbers and situation” without further human analysis is doomed to be lacking. At the end of the day, the idea that outside forces tell us what is right or wrong in every situation is just another form of hubris. This codifies turning over power to others, because someone, somewhere, will create the perfect basis to decide things for us. In other words, humans will always be involved in one way or the other. We can either trust ourselves to make a sufficient number of correct decisions or we cede that power to others.

The Myth of Perfect Judgment

The idea that we will ever get things all right is predicated on a false premise—the perfectibility of man. My argument is that there will never be a perfect basis to decide matters that involve weighing and balancing information. This requires human judgment. Instead of looking for the perfect measurement, or machine, or formula to decide things for us, we need to embrace the things that make for good judgments and decisions.

Lessons from the Playing Field

Let’s start with an easy one for me—sports. You cannot listen to an athletic event without some talking head saying, “The analytics say…” To a limited degree, that is truthful. However, it lacks any perspective. When it is fourth and two on your own 38-yard line in the second quarter, some version of analytics will say go for it, since your chances of succeeding are higher than failing. True enough.

I would argue that is a silly way to make the decision. What about items, such as is the difference of five percentage points or fifty? What about how successful your team has been in such situations? How about the team defending? There are other questions that should be asked. Most important is what the numbers are concerning victory and defeat? Suppose the analytics say your chances of success and failure are 60-40. Suppose when you succeed, your chances of winning become 60 percent, but your chances of losing should you fail is also 60 percent. Do you go for it or not? I say no.

Beyond Numbers: The Human Element in Sports

While on the subject, can we please get rid of the talking heads saying, “I love his aggressiveness?” You need to be able to distinguish between aggressive and reckless in the name of the numbers on that single play. You need a human to keep asking questions, so the most informed decision can be made. You need a lot of numbers, not just one, and you will still be wrong some of the time.

There are plenty of other examples of this. Going for two points after a touchdown in the second quarter, since success makes it a three or seven-point game. I will go to my grave arguing that it is too early to do so. There are too many other possibilities for the rest of the game not to just kick the damn extra point.

Using machines to decide if it is a touchdown or not. Using other machines to call balls and strikes. We know they still get things wrong.

The Role of AI: Tool, Not Master

The dangers of relying on others to make the final decision are fully captured by AI. This is something I use multiple times a day. I will ask it to give me the facts, but not how to weigh them. I will ask it to tell me what both sides of a given issue are arguing on a given topic. I will not ask it which side is right. In other words, I will weigh the facts as I see fit and not give much of a hoot if that is different than a machine. I trust me more than them.

To the extent analytics and AI provide a way to quickly factor in a multitude of information, it is a good thing. To the extent we oversimplify or turn over evaluative responsibility to them, I argue that is a bad thing.