The Data Within

Why algorithms can never replace intuition.

Updated November 14, 2025 | Reviewed by Hara Estroff Marano

A few weeks ago, I found myself coughing and shivering under a blanket as the latest fall virus settled in for an extended stay. Unable to do much else, I turned on the TV and started looking for a movie to watch. But everything the streaming services suggested just felt … off. I recognized myself and my interests reflected back at me in the suggestions: a documentary about the financial system; a movie about food; a show about travel in East Asia. But none of it clicked.

Then, suddenly, I knew what I wanted to watch: a movie I’d seen with my wife and son 15 years ago. Undemanding, funny, brimming with cozy associations. It was the cinematic equivalent of chicken soup. Almost any other day, I wouldn’t have been interested. But just then, it was exactly what I needed.

The data the streaming services have on my viewing habits is more than impressive. It is clean and complete. It captures almost everything I have watched over the last decade, with the exception of a couple of hours of viewing on flights or in hotel rooms. Normally, the algorithm serves up a menu of options that includes something that will satisfy me. And that’s the thing about algorithms: They are tuned to normality. They make predictions based on statistical likelihoods, past behavior, and expectations about the continuation of trends.

I don’t get ill very often. Faced with an outlier like that, the algorithm was useless. And that was an important reminder. I spend a lot of my time enabling organizations to make data-driven decisions. But sometimes, the past can’t steer you. Sometimes, probability is just a roll of the dice. Sometimes, you have to go with your gut.

Too Fast and Too Much

Making decisions based on what feels right or on what your gut is telling you can be a risky business. Get it wrong and you could find yourself betting your house on red in Vegas or quitting your job on a whim. That’s why I firmly believe that following the data and applying rational thought to a problem is the best way to make a decision—most of the time.

But there are all sorts of situations in which data-driven calculations won’t lead to the best outcomes. The boxer who stops to work through possible responses to an incoming blow is going to get hit in the face. Similarly, the emergency room doctor facing a patient with ambiguous symptoms and deteriorating vital signs can't wait for every test result. Sometimes, the sheer volume of potentially relevant data points would take hours to process systematically. The experienced physician's gut feeling that "something's seriously wrong here" can trigger life-saving interventions faster than any attempt to work through all the data.

Gut instinct can be priceless when there’s a mismatch between urgency and the resources we have for analyzing the data. It’s not always going to be right, but letting your gut lead will almost always be better than not making a choice in time at all.

There is nothing mystical or irrational about going with your gut in situations like this. You are just tapping into a sophisticated form of pattern recognition that all higher animals have evolved. Our intuition compresses years of experience, thousands of subtle cues, and complex contextual factors into an immediate sense of what's right or wrong, safe or dangerous, promising or futile.

When You Are the Data

Sometimes, the problem isn’t that the data comes at you too fast or that there’s too much of it. Data-driven decision-making can also fail if the dataset is missing something critical. That's exactly what happened when I was searching for a movie. The streaming algorithm had years of viewing history, but it couldn't know I was ill. And even if it did, it still didn’t have much data on the kind of stuff I want to watch when I feel bad.

In situations like this, you have to turn inward, because only you have the missing piece of the puzzle. In fact, when it comes to decisions about you and how you fit into the world, you are the data the answers are going to turn on. And processing your internal states and feeding them into your choices will almost always mean letting your gut do a big part of the work.

Your gut instinct pulls from a pool of internal signals that no external algorithm can account for. And that internal complexity is often going to be impossible for you to break down and rationalize . The subtle tightness in your chest in response to a suggestion; the memories triggered by a certain color or sound; the intensity of feeling that one job offer brings but another does not. This is your intuition processing signals that matter deeply, even if they can't be quantified.

The ability to tap into that internal landscape becomes especially important when you are making creative choices or navigating major life decisions. The artist choosing their next project, the entrepreneur deciding whether to pivot, the person contemplating a cross-country move; these aren't optimization problems waiting to be solved. They're expressions of identity and agency. The data might tell you that San Francisco offers better job prospects, but your gut knows something about what kind of morning light fills you with joy.

These aren't irrational preferences; they're acknowledgments that some of the most important factors in our decisions live in dimensions that spreadsheets can't capture. When we honor this internal data, we're not abandoning logic— we're recognizing that our choices express who we are and who we're becoming.

Making More of Our Instincts

Like any other guide to making decisions, your gut is far from infallible. Here are some easy exercises to help you use it more effectively:

Sharpen your instincts so your gut is a better guide
Learn to look inward

As algorithms increasingly shape our world and our choices, it is important to maintain the tools we need to make decisions for ourselves. Our gut instinct is one of those tools. The situations in which AI is the least help—moments of crisis, creative decisions, and deeply personal choices—are often the ones that matter most.

If we want to continue to be the authors of our own lives rather than passengers on an optimized journey, we need to cultivate our gut instincts with the same intentionality we bring to skills like critical thinking and adopting new technologies.

Share this post Facebook Bluesky Linkedin Email

There was a problem adding your email address. Please try again.

By submitting your information you agree to the Psychology Today Terms & Conditions and Privacy Policy

Faisal Hoque is the author of 11 books, including TRANSCEND (2025) and REINVENT (2023). He is the founder of SHADOKA and NextChapter, among other companies, and serves as an Executive Fellow at IMD Business School in Switzerland.

Get the help you need from a therapist near you–a FREE service from Psychology Today.

This article is part of the Bringwise Psychology Journal — daily insights on human behavior, mental health, and personal growth.