Aug 8, 2024
Generative AI is revolutionizing critical components of enterprises. Since the advent of technologies like ChatGPT, businesses have been eager to incorporate these advancements to enhance productivity, customer experience, and overall efficiency. However, while many companies have invested significantly in developing Proof of Concept (POC) demos, only a few have successfully transitioned to production. Gartner estimates that more than 30% of Generative AI projects will be abandoned after POC by the end of 2025 . A report by Goldman Sachs titled “Gen AI: Too Much Spend, Too Little Benefit?” suggests the primary reason is overinflated development and operational costs.
After speaking with over 50 enterprises, including many from the Fortune 500, we have developed a template to help businesses maximize their ROI from POC to production. This blog will walk you through this template and provide actionable insights for your Generative AI strategy.
Understanding ROI for GenAI Projects
For a good investment, any dollar spent should yield more in return. Refer to Table 1 below. Without GenAI implementation, assume your revenue is 100, and all costs combined are 50, resulting in a profit of 50. After implementing GenAI to improve the same offering, let's assume your revenue increases by X and your costs also increase by Y, accounting for the GenAI development, maintenance, and operational costs. Your profit with GenAI implementation would then be 50 + (X - Y).
Thus, for a successful GenAI project, X should be significantly higher than Y.
Table 1. Profits before and after GenAI project implementation
How can X be high?
Generative AI and new-age LLMs have the potential to streamline processes, automate tasks, and enhance customer experiences. For example:
Shield Healthcare is leveraging Generative AI to build a co-pilot for healthcare insurance policy documents. This co-pilot assists customer service agents in understanding individual company benefits more accurately and efficiently. The AI ingests vast amounts of policy documents, extracts key information, and provides real-time suggestions during customer interactions. This improves customer experience, increases revenue and retention, and enhances agent productivity by reducing operational costs. Shield estimates cost benefits of $15M every year through this.
A Fortune 500 bank is addressing the high cost of manually responding to a large volume of customer emails. They are developing a Generative AI bot capable of understanding and responding to customer emails autonomously. The AI bot uses advanced NLP techniques to comprehend the context and intent of emails, automating 25% of responses and potentially saving $25,000 per month. This enhances scalability, reduces operational costs, and improves customer satisfaction by providing quicker responses.
Examples of such successful projects highlight that targeting the right use cases with the new suite of technologies is crucial. Ensuring that their GenAI investments (Y) are significantly lower than the realized gains (X) is the key to deriving real benefits from these initiatives.
How can Y be high?
However, many GenAI projects or POCs turn out to be more expensive than their realized benefits, leading to their abandonment. Any GenAI stack includes three major components:
Your data
The LLM (read GPT)
The Application
While your data is free (a coveted resource that ideally should remain private), the Y costs are primarily associated with the LLM and the applications, both in opex and capex capacities.
The major components of Y therefore include:
The LLM:
Capex: Zero for closed general LLMs like GPT, non-zero for owned LLMs like fine-tuned Llama.
Opex: Scales with usage for closed general LLMs like GPT, non-scaling for owned LLMs like fine-tuned Llama.
The Application:
Capex: Development costs.
Opex: Maintenance and server costs.
While application costs are manageable, most unsuccessful projects fail due to high LLM costs when scaled. For example, a consumer app tech platform spends $2 on GPT costs for every user to convert longer videos to shorter reels on their MVP with 100 users. They currently have 20,000 Monthly Active Users. The $40,000 per month extra overhead of LLM call costs is unbearable. Fine-tuned LLMs are the best option here, that would cost them only $2,000 to cater to all users, a 20x cost benefit.. Additionally, fine-tuned LLMs offer better control and performance compared to general models like GPT4.
Conclusion
For any of your Generative AI projects, ensure you understand your X and Y. You have to strategize to maximize your X and minimize your Y. Given the margin estimate for Y, choose the best option and proceed confidently.
About Genloop
Genloop is a US-based company with expertise in delivering fine-tuned private LLMs that outperform GPT4. They are saving enterprises $100K every month in their LLM costs by upgrading their GPT4 to a private specialized LLM without any development effort from the enterprises. Please visit genloop.ai or email founder@genloop.ai for more details.