🤔Why is GenAI so Hard?

The Big Promise: You want an outcome. So rather than doing the work to generate that outcome, you task your AI-powered dream assistant to do that work for you … instantly, to perfection, at near-zero cost.

This vision has captured the imaginations of millions. Yet despite the billions that have been invested and the best efforts of many of the world’s top minds, it’s difficult to find examples of this promise in practice. Why?

Success Framework 🗝️

To achieve the big promise, it’s first critical to understand the obstacles that stand in the way. With this understanding we can begin the work to remove them. So what are they?

Prompt Engineering

First, it’s surprisingly difficult to describe precisely what you want. Like a lucky soul that finds Aladdin’s lamp🪔, the right wish is rarely as obvious as we imagine.

System Engineering

Even if you know what to ask, it’s really hard to build an AI system that reliably turns that request into exactly what you want. There are a lot of moving parts here so it’s helpful to break this down further according to how these systems operate.

1. LLM: These models may be magical, but they’re also quite dumb. They’re simply responding to a collection of electrons (the prompt) by organizing a second collection (the response) in the configuration that, statistically speaking, most commonly follows the first. If either your prompt or the response you want is not in the data used to train the model, you’re not likely to be satisfied with the result.
   🥡Takeaway: new LLMs are being released constantly; your domain expert should regularly evaluate a collection of relevant prompts using the latest models.

   

2. Data: So we use data (in a variety of ways) to shift the calculus [alter the weightings] inside the LLM. This helps to generate a response that’s more relevant to you. But there are a bunch of challenges here as well:

   • Existence: it’s possible that the data you need isn’t stored anywhere.

   • Access: even if it is stored, it may be inaccessible for security, privacy, technical or other reasons.

   • Structure: even if it is accessible, it’s unlikely to be stored in a form that mirrors human communication. Without a lot of work to restructure this data, it won’t help the LLM revise its estimation of the most likely response to your prompt.

   • Quality: even if you’re able to structure the data in a way that reflects how people actually communicate, it’s possible that the data is incomplete or incorrect. In this case, we’re still likely to get an unsatisfactory result.

   🥡Takeaway: don’t skip the data work.

   

3. Engineering: selecting the best LLM for the purpose and securing the data above both lead to a complex and specialized engineering task to configure the LLM and these data inputs to generate the best possible output. 

   🥡Takeaway: there are many ways to get this wrong and the techniques are constantly evolving. An experienced LLM Engineer can accelerate this step while reducing the risk of a bad solution.

   

4. Continuous Improvement: The LLM Engineer is seldom the best person to assess the “rightness” of the output and typically relies on a Domain Expert to help evaluate the quality of the responses and fine tune the system to clear the “Quality Bar”. This leads to the question of where to set this “Quality Bar”. The answer depends on the Use Case and especially on who the User is. If the User is internal, the bar can be set low to “Better Than Nothing”. In other words, if the system outputs something that provides a better starting point than what the internal stakeholder currently has, then that user will be happy. However, if the User is a customer, the bar has to be set to “Better (or faster) Than Expected”. As companies increasingly implement (and tout) their AI capabilities, this expectations bar will continue rising. Companies must find ways to keep pace with these rising expectations by continually improving their AI system.

   • 🥡Takeaway: at GenAI’s current pace of innovation, a continuous improvement plan is essential.

     

This need for rapid and continuous improvement is what drives Lamatic. It will also provide the theme for our upcoming series of posts, so stay tuned for more on this interesting and important topic.