Why Do AI Projects Fail Even With Working Technology?
There is a particular discussion around certain types of AI projects.
The demo is impressive — the model makes a prediction, the outcome is surprising, the automation is quick, and everyone in the room sees the potential.
Right after, the project meets the real life business case, where there are spreadsheets with missing information, disconnected legacy systems, untrained employees, and inefficient work flows that the new solution does not account for.
The technology was not at fault — the project simply underwhelmed.
The Wrong Question Can Be The Most Common Setup To Failure
The most common pitfall is asking the wrong question.
Not “where can we apply AI?” (the answer to which is often not useful) — but rather asking “what can AI do for us?”, when a better question would be “what do we want AI to help us with?”
The former leads to neat but useless experiments, while the latter asks for a specific business case that AI can meaningfully affect.
Some examples of meaningful questions are:
- What repetitive process do our employees tediously perform?
- What information is currently difficult to access or derive?
- What decisions are consistently made incorrectly?
Once the question is set up, the response to it can often be engineered with simpler technologies than a machine learning model. Sometimes it is automation, sometimes it is a rule-based algorithm, and sometimes it turns out that human judgment is actually the most reliable factor.
Predictions Need To Come With Something Else
An AI demo often assumes a perfect dataset — the kind that is far removed from the real-life application.
In practice, data is messy, redundant, duplicated, unlabeled, old, inconsistently formatted, or just completely unavailable.
Industrial environments add an additional challenge through disconnected legacy systems and sensors that rarely speak to each other.
Ultimately, an AI model is only as good as the data that feeds it, which means that data preparation takes time and resources.
It is not an exercise to simply get out of the way before “real work” begins — rather, it is a part of the AI project just like any other.
A similar pitfall is waiting for a prediction to somehow fix itself.
If an AI produces an insight, it has to go somewhere — a dashboard that no one looks at, or a report that gets buried under other priorities are examples of outcomes that miss their target.
The actual value of an AI project comes from supporting a business process, such as a work order or report that gets automatically populated by the model, or a human operator that reviews the AI suggestion before implementing it.
A useful AI project looks like this: data -> ai -> insight -> workflow -> human -> action.
Get Nayantara P S’s stories in your inbox
Join Medium for free to get updates from this writer.
It is a complete loop that has value at every stage.
Employees Are Humans, And Humans Are Not Always Logical
Training employees to use a new technology is an obvious pitfall.
If they do not understand the result that the AI produces, it is simply ignored.
If they have to spend time understanding why the AI produces that result, they will find another way to get it done.
Good AI automation completes the existing workflow, rather than getting in its way.
People need to understand the changes that the AI brings, and what role they play in it.
Likewise, people should not have to constantly review or approve every single suggestion that AI makes, unless it genuinely benefits from human intuition at every step.
A well-designed AI will let humans offload their own repetitive and error-prone tasks, and will only ask for human input when judgement or intuition is needed.
Big Ideas Are Often Not The Best Way To Start
There is pressure in any industry to “think big”, but successful AI projects start small.
A single well-documented process will teach you more about your business than any amount of disconnected data.
Pick a goal and a metric, measure how well your employees perform before implementing AI, then use automation to assist them and see how much they improve after.
Did the work become easier? Was the accuracy higher? Did people even use the new technology?
These are more interesting questions than “did our AI work?”
A Working Model Often Has Little Connection To Real-World Results
AI is a tool, but the tool is only as useful as the support around it.
This includes the data that feeds the model, the business processes that the model alters, the software stack that implements the model, and the employees that operate this software.
It does not matter nearly as much whether the machine learning architecture is state-of-the-art, because it will need to connect to other systems that are often not as advanced.
This is especially true for industrial environments that involve physical machinery, automation, and sensors, where most other parts of the system predate modern machine learning techniques.
Aperture Venture Studio works with AI and IoT ventures that are focused on building solutions around real-world operational processes, and the best results come from focusing on tangible improvements.
A working AI model is not a goal in and of itself, but rather a means to an end.
The end goal should be connected to a business metric, and the project should be designed around it.
A functioning model is not enough — the output has to be useful to people, and the value has to be measurable.
