Chatbots and Conversational AI

Before we dive into what chatbots are, let’s first understand their origins.

The Turing test

A pivotal moment occurred in 1950 when Alan Turing proposed the Turing test in his paper, “Computing Machinery and Intelligence.” Essentially, Turing questioned whether machines could think and suggested a test to determine this ability. In this test, also known as the imitation game, a human interrogator would engage in text-based conversations with a human and a machine. The machine was considered intelligent if the interrogator could not reliably tell the human from the machine. While the ability to carry a conversation doesn’t definitively equate to human-level thinking, it marked a significant step in exploring the potential of conversational AI.

Early chatbots

A chatbot called ELIZA was one of the earliest and most popular programs capable of even attempting the Turing test. ELIZA was programmed to simulate conversations by pattern matching (we’ll dive into pattern matching in a later lesson). ELIZA was mostly famous for its DOCTOR script, which allowed it to play the role of a Rogerian psychotherapist. It was able to convince some early users to be intelligent and able to understand conversations. We have resurrected ELIZA in most of its former glory. Do you think it would pass the Turing test?

The Turing test has been a valuable conceptual framework that has steered the development of chatbots toward a specific goal (in this case, to simulate a human conversation). However, as a subjective test, its accuracy relied heavily on the interrogator. Furthermore, the focus of the test was primarily to evaluate a machine’s ability to mimic conversation rather than true understanding or intelligence.

Why are chatbots needed?

So far, it might appear as if the sole purpose of chatbots was to simulate a human conversation without necessarily needing to be intelligent. That is most certainly not the case. Limited by computational power, most early chatbots used pattern matching to converse with humans. However, pattern matching worked well if the scope was narrowed to a particular domain or use case. Simple chatbots that aim to provide information based on a user’s query have been around for quite some time. In an upcoming lesson, we will see how pattern matching can be implemented to create a simple chatbot.

While the earliest chatbots were derived from answering the philosophical question of whether computers can think, they led to great research and development in natural language processing. One of the areas of focus was to develop techniques to process and understand human language. The intricacies of human language required some breakthroughs, both in the algorithmic and hardware side of things, to create chatbots that could pass the Turing test. However, even earlier approaches with rule-based systems were able to produce serviceable chatbots. Once the potential of chatbots with newer techniques was recognized, a lot of research was poured into this domain, and we ended up with where we are today: the age of generative AI.

The aim of chatbots was primarily to enhance efficiency and assist people. Here’s how:

• Efficiency: Chatbots can handle routine tasks and inquiries, allowing human agents to focus on more complex issues.

• Cost-effectiveness: Chatbots can reduce operational costs by automating tasks.

• 24/7 availability: Chatbots can assist around the clock, improving customer service.

• Data collection: Chatbots can gather valuable customer data for analysis and improvement.

Chatbots also led to the rise of assistants. Clippy, the Microsoft Office assistant, was a rudimentary form of virtual assistant designed to provide user assistance within Microsoft Office applications. While it was limited in its abilities and perhaps a bit too early for its time, it was certainly a welcome addition and proof of concept.

Modern chatbots

Now that we understand where chatbots came from and how helpful they can be let’s explore the current landscape. NLP techniques have come a long way, and it is now fairly easy to create a chatbot using existing libraries and packages. We will use Rasa to create a chatbot that uses traditional NLP techniques.

With generative AI, however, chatbots have seen exponential growth due to the new realm of possibilities modern LLMs have unlocked. It is now as simple as sending a text to an LLM to begin a conversation. Newer chatbots are more adaptable, can converse in a wider range of languages, and can take on more complex or nuanced roles.

Chatbots are now being used in different domains for different purposes. Let’s look at a few common ones:

• Websites now use chatbots to allow new users to quickly get the necessary information.

• Hospitals use chatbots to connect patients to relevant doctors by examining their symptoms.

• Recipe chatbots can suggest dishes you can cook based on the provided ingredients.

• Educational chatbots can help users learn new concepts or test existing ones.

• Airports and airline chatbots can quickly get flight statuses and assist with booking flights.

For the sake of simplicity, we will refer to non-generative AI chatbots as traditional chatbots.

Bot vs. chatbot

Bot is a broader term that includes any software application designed to automate tasks. Bots can range from simple scripts to complex systems performing various functions. Perhaps the most common bots are web crawlers or search engine bots that scrape websites for information. Sophisticated bots are also used for time-critical tasks such as stock trading.

Chatbot is a specific type of bot designed to simulate human conversation. Modern chatbots can perform various tasks as well. Customer service bots can help you diagnose device issues or file a complaint. Informational bots can help you search for a particular item you might be looking for. The list is fairly large.

In essence, all chatbots are bots, but not all bots are chatbots.