A Roadmap to a Successful AI Project: Planning, Execution & ROI
Introduction
Deloitte in 2019 commented that 91% of AI projects did not meet expectations, Forbes in 2020 stated that only 14.6% of companies have implemented AI capabilities in production and McKinsey reported that 64% of projects will not continue beyond the pilot stage in 2021.
In our experience, the number of frustrated AI projects is quite high. Normally, the majority of companies surveyed have a lot of investment capacity and the institutions that fail the most are not part of the census. It is also true that since 2021, the rate of AI projects has increased considerably with the Generative AI revolution but the success rate towards its transition to production remains to be seen.
The year 2023 was the year of Proof of Concepts (PoCs). Will 2024 be the year of transition to Production?"
Understanding the maturity of AI in companies
In the current business context, the implementation of AI is becoming a fundamental pillar for innovation and competitiveness. Gartner has identified five levels of AI maturity that help to understand the progress of organizations in this area.
Level 1: Awareness — In the initial stage, companies begin to explore AI, discussing possible solutions without embarking on pilot projects or proofs of concept. This phase is predominantly theoretical and awareness-raising oriented.
Level 2: Active — Organizations at this level run initial AI experiments and pilot projects. This is the state in which most companies are located, testing the possibilities and limitations of AI in controlled scenarios.
Level 3: Operational — Companies advance to the operational phase when at least one of your AI projects goes into production. This level is the objective that most companies aspire to achieve in the short term.
Level 4: Systematic — At this point, AI is no longer just a project or two; It has become part of almost all the company's digital processes. AI applications improve productive interaction both inside and outside the organization, evidencing mature and strategic adoption. It is a vision of the future in which organizations aspire to be in the medium term.
Level 5: Transformational — The most advanced level of AI maturity is achieved when organizations incorporate AI as an essential component of your business workflows. Is visionary and it may be more realistic for companies that will be born in this era, where AI is at the heart of everything they do from the beginning.
This Blog is aimed at companies that are mainly in Level 2 experimentation. The purpose is to guide the reader (within the limitations of a blog) to identify a valuable use case for business that can be moved to production and that, based on the good results, opens the way to a process transformation motivated by AI.
Doing AI projects requires an approach and work methodology that adapts to these types of projects. Artificial intelligence projects share more similarities with business initiatives than with mere technological implementations. In fact, they are more like a marketing strategy than the installation of a Salesforce-type CRM system.
The reason for this lies in the multiple levels of uncertainty that surround them, from the quality of the data to the acceptance by users, through the effectiveness of the algorithm. Despite an exhaustive analysis, the final result is partially unpredictable, which often contributes to the failure of these projects.
Likewise, technology consulting companies, often responsible for the transformation of their clients and accustomed to conventional IT implementations, encounter challenges when adapting to this type of project despite the strong technological component. The weight of success falls mainly on other factors.
How to plan and execute an AI project successfully
First of all, AI projects, much like data projects, are not merely IT initiatives but business ones. We want to identify relevant use cases that when presented to the CEO he can visualize the future impact on profits. At SunnyData we have an approach aimed at raising cases together with the departments or teams designated by the client and that normally does not require more than a few days of work. These business-oriented sessions involve an experimentation phase during which ideas are conceived, tested, and demonstrators for the most promising proofs of concept are built.
80% of the ideas should have already been eliminated during these sessions before proceeding to a PoC. The defined use case should be the one that offers the greatest cost-benefit ratio. This means that the interim results should look very promising, while the room for improvement remains high at a relatively low projected cost.
Once a case has been identified, whether through the previous process or due to internal initiative or popular demand, it should move forward with a defined scope and an evaluation process aimed at developing a PoC. The scope of the project may be undervalued but in the initial phases of a PoC it is key to making it successful and concise.
No time should be wasted on unnecessary integrations. Business rules that have already been evaluated to be feasible to implement should not be applied. Nothing should be done that does not raise doubts. The rule is: evaluate what should be evaluated and generate uncertainty. Be careful, many of the evaluations are not technical but business-related (such as user acceptance), and this can lead to a PoC being somewhat more extensive.
Regarding the evaluation process we must “Check” on:
The state of the art AI: Is the state of the art of AI sufficient to meet the objectives? Does the use of AI comply with laws and regulations? Is there quality data and, if not, is it possible to remedy it? Technical analysis
The processes: What are the processes? What sequentiality? What dependencies? What inefficiencies are detected/to be improved? Analysis of the AS-IS and TO-BE process with AI.
In the early stages of AI development, it is essential not to fear the possibility of stopping a PoC. Often, executives and managers, as well as some consulting firms, lack the courage to discontinue these projects, leading to overcommitment and premature scale expansion. Stopping a PoC should not be seen as a failure or a loss of investment. On the contrary, this action should be considered a valuable opportunity to learn and fine-tune future investments, which may be more targeted and potentially offer a much more significant return on investment.
On the other hand, it is important to approach PoC with a proper perspective. A good PoC is one that is carried out responsibly and with a clear understanding of its feasibility. Unfortunately, many companies have promoted PoCs more as a marketing strategy to add logos to their portfolio, often with minimal accountability and full knowledge that certain projects are not viable. This practice has generated some distrust and rejection towards PoCs. In the field of AI projects, starting without an appropriate PoC involves a huge risk that SHOULD NOT BE TAKEN.
Below, we will examine some common reasons why AI projects do not achieve the expected success.
The other 7 reasons why AI projects fail
What are the reasons for failure? There is no order, and they are all equally relevant, in fact, I have ordered them randomly so that it is not a totally subjective fact.
State of the Art of AI: This point is very simple. Technology advances, and there are things that can be done today that could not be done 3 years ago.If you have a good use case but it is not possible to do it, it is best to park it and focus on others. The time to do this project will surely come later, but you will save yourself from having wasted the funds on a bad project that might require redoing (if they allow it).
Not considering return on investment: Every project must generate an ROI and this approach should be a commandment for both clients and consultants. Unfortunately, AI projects are the ones that fail the most and where it is most empting to turn it into a research project”(things are not going as expected and investments continue). I have worked in many consulting firms, and I am surprised by the lack of sincerity that exists when talking to clients, and how contracts are signed for “projects'' that will never go into production and become an “investigation.”
Lack of global vision: This AI project that they are proposing to you, or that your company has chosen to carry outIt must be visualized in production and in conjunction with the rest of the system and processes. It is not enough to be technically brilliant. Example: A company wants to create an assistant for information security queries. Will workers use it? No, this application should be integrated into current processes and systems such as a Google/Microsoft Suite so that it can be used and increase productivity and security.
Bad choice of cases to implement: AI is not a solution to all problems. In fact, there are clear limitations to what AI can and cannot do. Typically a bad starting point for a project is when a client comes to you with a problem they can't solve and hopes AI can fix it. It is advisable to hold use case discovery workshops with the different departments and try to prioritize those identified that generate a higher ROI, have lower risk, are for internal use, the state of the art is advanced, has already been solved before, etc. SunnyData has a fairly efficient matrix that is used in workshops with clients (request via LinkedIn).
Unsuitable suppliers: All consulting firms are now experts in AI and have joined the wave of AI hype. This is good, because it generates more competition and greater development opportunities, however, they do not have the experience, the talent and above all they lack the methodology for the development of these types of projects. AI projects are treated like any other project, or worse, because there are more excuses to justify failure.
Technology that is not used for what it is: Generative AI is great and we love it, but it's just one branch. We cannot and should not solve all problems with Generative AI. It probably won't be long before the explosion, for example, of reinforcement training and what will we do? Do all the projects again? As experts we must be clear about what type of problems are solved with these solutions even if it has not yet exploded! This is what differentiates between a company that leads versus one that follows the flow.
Data problems: Data is a problem when it is not there or does not have the necessary quality. However, it depends on the use case. For certain projects it can be a limitation, but for others it is not so much. There are very refined techniques today to generate synthetic data that for some practical cases do not have excessive dependence on the quality of the global institutional data and also the possibility of correcting or adapting them, although it surely requires more time.
Our Conclusions
The large-scale adoption of artificial intelligence is not just a trend, but an imminent revolution that will redefine the business landscape. This technological change offers a window of opportunities for those companies that dare to innovate, allowing them not only to reposition themselves in their markets but also to lead new sectors.
This transition also poses significant challenges, primarily for companies that resist change or are slow to adapt. In this context, a company's ability to integrate AI into its business model and operations will determine its future success. Those that proactively adopt this technology could enjoy substantial competitive advantages, while those that do not could be left behind.
