In late July 2023, at the Unparsed Conference, Christoph Börner, Cyara’s Senior Director of Digital and renowned chatbot expert, met with Hans van Dam, founder of the Conversation Design Institute to discuss the present state of the chatbot industry, conversational chatbot design, large language models (LLMs) and the prospective developments within this field.
They began their discussion by shedding light on how all software including chatbots and conversational artificial intelligence (AI) need to undergo stringent testing. Christoph emphasized that best practices suggest organizations should allocate approximately ⅓ of the project budget to this testing. While Hans agreed with this advice, he warned that unfortunately, only a small number of organizations are doing so.
A key observation from their discussion was that the absence of adequate testing of chatbot functionality can hinder organizations from achieving their desired returns on investments. Additionally, it exposes them to substantial risks and vulnerabilities. These include the provision of incorrect information, creating a negative brand perception, and potentially driving customer churn because of subpar customer experiences (CX).
Large language models
Hans and Christoph delved deeper into the topic of large language models, highlighting how these models are simplifying and speeding up the process of generating and accessing vast volumes of data. This availability and convenience extends not only to organizations, but also to individuals, who can now easily leverage publicly accessible and free tools like ChatGPT. The result is a significant reduction in workloads. They also noted that organizations are now increasingly utilizing these extensive datasets to expedite the training of their own chatbots.
“LLMs are a great productivity tool if you are building chatbots. You could actually use ChatGPT to be more productive” – Hans van Dam
Hans elaborated on this by suggesting that while LLMs play a pivotal role in training chatbots, conversation designers remain indispensable for ensuring their ultimate success. These highly skilled professionals possess the ability to swiftly understand the strengths and weaknesses of chatbots and LLMs, identifying issues with conversational flows and other potential obstacles.
They suggested that there is a common misconception that technologies like LLMs can be effortlessly integrated and will yield immediate results for businesses. However, this idea is in fact far from the reality. While chatbots and LLMs do offer substantial benefits such as reducing workloads and aiding conversational designers in their daily tasks, they cannot serve as replacements.
This is partly because they lack a number of key skills, for example, the inherent understanding of how humans interact with one another, the capacity to fine-tune data as needed, and the ability to provide empathy and understanding when necessary. Hans advised that in order to avoid negative outcomes, only trained and knowledgeable professionals should be entrusted with the management of chatbots and LLMs.
“If you know what you’re doing, you can do the right thing faster. But, if you don’t know what you are doing, you can mess up much faster and with much greater impact” – Hans van Dam
The potential
As chatbots and LLMs continue to enhance their capabilities and accuracy, the potential and possibilities they can provide continue to increase, benefitting both organizations and individuals alike.
Hans explained how he was recently asked to present a TedX talk on chatbots and ChatGPT. During his preparation, he made a decision… he tasked ChatGPT with writing the script for his entire talk. This underscores the approaching reality where a considerable portion of the content we consume—whether visual, auditory, or written—will be generated by AI. In fact, we can already see many instances of this, for example ChatGPT is listed as a co-author for many books available for sale on platforms such as the Amazon Store.
However, this advancement raises valid concerns. As AI, chatbots, and LLMs are being trained on increasingly larger volumes of content, there is a greater likelihood that they may carelessly replicate this information, either wholly or partially. This has raised significant concerns, not only regarding the authenticity and validity of the information LLMs provide, but also in terms of potential copyright infringement on existing works.
For example, 9,000 authors, including James Patterson and Margaret Atwood have written an open letter which alleges that technology companies working with generative AI are exploiting their copyrighted works. They propose that consent should be actively sought from authors before any of their content is used for training purposes. They have also requested fair compensation for both the past and ongoing use of their works in training. Hans highlights that while all individuals are influenced by what they see and read, this influence is significantly amplified when it comes to AI.
Conclusion
Hans advised that we are still at the early stages of chatbot and LLM capabilities and that significant change and transformation is approaching. He drew a parallel between how App stores were initially launched with only very basic capabilities and applications (such as timers, countdowns and torches). Over time, they have now grown to provide a user with almost every type of application they could desire. Similarly, he believes the trajectory of chatbots and LLMs will lead them to evolve far beyond our current comprehension. Adding to this, Christoph suggested that we are likely to see LLMs becoming more tailored and targeted to specific industries and verticals, such as finance and travel. And he was right, with OpenAI announcing exactly this only a month later!
As this newly emerging world continues to evolve, the pair suggest that organizations need to remain mindful of the skills, resources, costs and challenges involved in implementing chatbots, AI and LLMs. Critically, they also need to recognize the necessity of allocating approximately ⅓ of their project budget to rigorously testing the performance of their chatbot systems to ensure they provide users with the best experience and don’t result in unforeseen negative consequences.
