I love driving but sometimes I hate my car. It has been known to slam on the brakes for no reason, misinterpreting a bend in the road or a right turn lane as an obstruction. Eventually I found the emergency brake setting and turned it off. Call me old school, but if I’m driving, I’m driving. I don’t want a computer having more control of the car than I do. But I know a lot of people who put complete trust in those systems, and are so “busy” they’d prefer the car to do all the driving. 

This makes me think about how we are living our lives and working our work. The rapid increase in using AI to do our thinking at work has led me to investigate the concept of Automation Bias. A phenomenon first identified in the 1990s, it is becoming increasingly obvious that we need to be careful with it right now. 

Automation bias is the tendency to believe automated decision making systems even to the point of ignoring contradictory information. Studies have included professional pilots, doctors, and financial markets, and they reveal multiple potential sources of error, resulting in plane crashes, mistaken prescriptions, and massive financial losses. 

The risks are relevant to any type of organization. Key ones to consider include: 

Reduction in human intuition and expertise 

We talk about having a human in the loop to check what the AI automation is producing, but if we inherently trust the AI, we may start to ignore our intuition and our years of experience, and lazily lose our critical thinking skills.  

If we are engaging AI agents in place of juniors, there’s going to be a big hole in expertise in a very short time. Organisations need to be clear what skills they need long term, even if AI does a lot of the ‘gruntwork’. 

Misapplication of information 

Guilty! I’ve caught myself blindly applying information straight from ChatGPT, thinking that I have fed it all the information with a great bunch of questions. But alas, that has caught me out. The confidence with which I assumed the answer proved that I was not being critical enough. 

If we are then basing decisions on information that is either not relevant or applicable or complete, how good can our decisions be? 

Complacency – who’s accountable anyway? 

A false sense of security can mean we are not adequately monitoring performance, and not owning decisions and outputs that may be from AI, even though they fall under our accountability or responsibility. 

Loss of agency 

Ask people what they love about their work. For most people, it will be some combination of them making decisions that make a difference, contributing to something bigger than themselves, connecting with others and learning as they go.  

Helping our team members build and retain the ability to make good decisions is fast becoming one of the biggest challenges of this era. 

We are risk takers, and I love helping organizations take risks. Sometimes to move forward the fastest we need to get the basics right first. We need to be teaching our people to take control of the wheel again, to feel the revs of the engine and listen for the gear changes, before they are ready for autonomous vehicle.