If the screwdriver were invented by today’s tech industry, it would likely be touted as a versatile tool for a range of tasks, including hammering nails. This same overzealous enthusiasm can be seen with the growing interest in large language models (LLMs) since the launch of ChatGPT. While LLMs have undeniable potential, there has been a notable backlash as their capabilities are often overhyped, and their accuracy can be questionable. While there are certainly valuable applications for LLMs, it’s important to critically evaluate several factors before fully committing to their deployment.
First, you should ask whether an LLM will truly be better or at least equal to human responses in the specific application you’re considering. For example, customer service chatbots powered by LLMs are a common use case, but they often fall short in providing meaningful interactions, especially when the queries go beyond simple, scripted responses. On the other hand, relying on human agents who may be restricted to a script but lack the flexibility to offer personalized solutions can be equally frustrating. Any deployment of an LLM needs to be carefully tested to determine if it provides a comparable or superior user experience compared to the human or chatbot alternatives it may replace.
Another crucial consideration is the liability exposure that comes with deploying LLMs. In today’s litigious world, it’s essential to assess the legal risks of using AI in processes that could lead to harm, such as providing medical advice, legal guidance, or financial recommendations. An LLM could inadvertently provide incorrect or harmful advice, which could lead to significant legal consequences for a company. Even outside of traditionally high-risk industries, there’s a real risk of LLM-generated information causing confusion or spreading misinformation, which could ultimately result in lawsuits or reputational damage.
Finally, it’s important to evaluate whether deploying an LLM will truly be cheaper in the long run. While the costs of using a general-purpose LLM like ChatGPT may seem low upfront, custom-built systems often come with hidden expenses that go beyond the initial compute power. These can include the cost of specialized staff to manage and maintain the system, as well as the infrastructure needed for debugging and training. Moreover, as these services become more popular and the initial wave of investment subsides, the cost of LLM usage may rise. Before fully adopting an LLM, you should carefully consider whether it will remain a cost-effective solution for the duration of its use.