The History and Current Status of Research

The 1960s marked the beginning of research in generative AI, when Joseph Weizenbaum developed the chatbot ELIZA, one of the first examples of an NLP system. It was a simple rules-based interaction system aimed at entertaining users with responses based on text input, and it paved the way for further developments in both NLP and generative AI.

Artificial neural networks (ANNs)

However, we know that modern generative AI is a subfield of DL. Although the first artificial neural networks (ANNs) were introduced in the 1940s, researchers faced several challenges, including limited computing power and a lack of understanding of the biological basis of the brain.

Backpropagation

As a result, ANNs didn’t gain much attention until the 1980s. This changed due to the advancement in new hardware and neuroscience developments. The development of the backpropagation algorithm facilitated the training phase of ANNs. Before the advent of backpropagation, training neural networks was difficult because it was not possible to efficiently calculate the gradient of the error with respect to the parameters or weights associated with each neuron, while backpropagation made it possible to automate the training process and enabled the application of ANNs.