AI Implementation in Companies: The Biggest Mistakes and How to Avoid Them

Why the biggest danger in AI transformation in companies is hesitation

Many executives ask themselves the same question: If AI makes a mistake in my company and I am responsible as management, do I really want to take that risk? The honest answer is usually: not really.

However, this question is too short-sighted. Ignoring current technological possibilities poses another risk: losing touch with competitors who are already using AI. The risk of a carefully planned implementation is generally much smaller than the risk of doing nothing at all.

Additionally, there is a second, very human hurdle: the fear of admitting to one’s own team or management that one is not yet familiar with AI. This concern is unfounded. No one is fully knowledgeable about AI; even well-known figures in the tech industry often miss the mark with their predictions. Therefore, the crucial question is less about whether, but how an AI transformation in the company is structured: rolled out slowly, with clear control mechanisms instead of blind trust in every response from a language model.

How Zillow Lost 500 Million Dollars Due to a Flawed AI Pricing Model

The US real estate company Zillow implemented an AI pricing model in 2018 to automate the purchase of properties and resell them for profit. For several years, the model worked reliably. Then the situation changed. By 2021, the US real estate market had developed so dynamically that the AI model could no longer keep pace with reality. The result: over 500 million US dollars in losses and the reduction of about a quarter of the workforce.

“The cause wasn’t a malicious AI, I’d say. It worked for a few years.” — Stefan, Moderator

The actual cause lay elsewhere. The system continued to operate based on assumptions that had long been outdated by real market developments. The market became too volatile for a static model that made decisions based on outdated data. A timely threshold, where humans could manually intervene if exceeded or not met, would likely have significantly limited this loss.

Zillow is not an isolated case of a small startup, but an established, billion-dollar corporation with experienced data teams. This makes the case particularly instructive for any company that believes that having enough budget automatically puts them on the safe side.

Why IBM Watson Failed in Cancer Therapy Despite a 4 Billion Dollar Investment

Even very large companies are not immune to AI mistakes. IBM launched the program Watson for Oncology in 2012, an AI system designed to assist doctors with treatment recommendations for cancer therapy. About four billion US dollars were invested in the project.

In the end, the model delivered inaccurate, uncertain, and sometimes incorrect treatment recommendations. The cause was not a lack of ambition, but a too narrow data basis. The training data was limited, and there were too few real patient data available, which left the model too underdeveloped to work reliably in real clinical practice. In the end, the entire program was sold for a fraction of the invested amount.

The case shows: A large budget is no safeguard against AI mistakes. Risk management in AI implementation is just as important for small and medium-sized enterprises as it is for a global corporation. Unlike the Zillow example, the IBM project was also not a classic language model, but an early big data system from a time before modern LLMs. This changes the technical starting point, but not the central lesson: insufficient or inappropriate data leads to unreliable results, regardless of the budget.

For companies working with modern language models today, the good news is: The entry barrier is significantly lower than in 2012, as pre-trained models already possess broad foundational knowledge and no longer need to be trained from scratch with proprietary data. This reduces the risk of a too narrow data basis but does not replace the need for continuous verification of results.

The Three Most Common Mistakes in AI Implementation in Companies

In addition to the well-known AI fails, there are typical mistakes that occur in almost every company that introduces AI without a clear structure. Three of these are particularly common in practice.

Blind Trust in AI Responses Without Verification

Many teams assume that a prompt to a language model is sufficient and that the response is automatically correct. In reality, processes are needed that actively verify information from a language model instead of blindly relying on it. Effective countermeasures include automated matching processes and guardrails that check AI system responses against reliable data sources before they lead to a decision.

Underestimating AI Transformation as a Side Project

A genuine transformation process that actually delivers measurable added value cannot be handled on the side by someone who just has two weeks less to do. Those who want sustainable impact need time for analysis, expertise, and a well-thought-out roadmap. Those who start slowly instead, with a clearly defined pilot project, gain valuable experience for the next steps without exposing the entire company to an unpredictable risk.

Not Reviewing Results After Implementation

Implementing an AI solution once and then never checking again whether it actually relieves the workload and delivers added value is one of the most common and easily avoidable mistakes. A short but regular review rhythm is often sufficient to recognize early on whether a use case actually delivers the expected value or needs adjustment.

How Guardrails and Human-in-the-Loop Processes Limit AI Risks in Companies

The solution for both cases, Zillow and IBM, lies not in avoiding AI but in considering control mechanisms from the start.

“It’s about consciously involving humans in the control function at certain thresholds or specific actions, allowing certain topics to be evaluated.” — Simon Auer, CEO Markably

Specifically, this means: Automatically cross-checking the results of an AI agent, setting confidence intervals that indicate how certain a system is about a response, and defining clear upper and lower limits beyond which a human must intervene. In dynamic processes with high market volatility, like in the Zillow example, such thresholds are particularly important.

In addition, it is worthwhile to take a look at the data currency of the models used. Most language models have a fixed training cut-off date. If a model is not actively prompted for a current web search, it operates with knowledge from several months ago, which creates additional risks, especially in dynamic processes like in the Zillow example.

Who is Liable When AI Makes a Mistake in the Company

The responsibility for AI decisions ultimately always lies with the person who implements and approves them. That’s why it’s worthwhile to treat AI responsibility from the beginning as a strategic question and not just a pure IT decision.

This does not mean that every decision must be checked by a person. It means clearly defining at which thresholds, processes, or decisions a human must be involved. In practice, this often means having a clearly designated role in the company responsible for AI topics, combined with documented escalation levels for critical decisions. This keeps the responsibility traceable instead of being torn between IT, specialist departments, and management in critical situations.

Does AI Implementation Cost Jobs? What Employees Really Expect

The elephant in the room with any AI introduction in a company is the fear that it will ultimately cost jobs. This doesn’t have to be the case. The goal should always be a combination of increasing efficiency and preserving the knowledge that employees have built up over the years, applied in areas where AI cannot operate.

A concrete example: If four people take on certain tasks and their efficiency or quality can be improved through AI, capacities are freed up. These can be used to be closer to the customer, provide better advice, and take on tasks that deliver direct human value—tasks that were previously simply unaffordable. AI helps save costs without necessarily having to reduce staff. On the contrary: The existing potential of employees can often be fully utilized.

“The biggest mistake you can actually make in AI transformation is to miss it.” — Stefan, Moderator

In the AI workshop at LEAN-CODERS, we analyze your specific use cases together and build an initial working prototype, including the control mechanisms that prevent mistakes like those at Zillow or IBM from the start. Also, read part 2 of this series for practical entry.