Explore the design of scalable generative AI systems guided by a structured framework and real-world systems in text, image, audio, and video generation.

This course will prepare you to design generative AI systems with a practical and structured approach. You will begin by exploring the foundational concepts, such as neural networks, transformers, tokenization, embedding, etc. This course introduces a 6-step SCALED framework, a systematic approach to designing robust GenAI systems.

Next, through real-world case studies, you will immerse into the design of GenAI systems like text-to-text (e.g., ChatGPT), text-to-image (e.g., Stable Diffusion), text-to-speech (e.g., ElevenLabs), and text-to-video (e.g., SORA). This course describes these systems from a user-focused perspective, emphasizing how user inputs interact with backend processes.

Whether you are an ML/software engineer, AI enthusiast, or manager, this course will equip you to design, train, and deploy generative AI models for various use cases. You will gain confidence to approach new challenges in GenAI and leverage advanced techniques to create impactful solutions.

Grokking the Generative AI System Design

\text{Model  size}= \text{3  billion} \times \text{2 bytes} =\text{6  GB}

\text{User data} = 100 \text{ M} \times 10 \text{ KB} = 1 \text{ TB}

\text{User interaction data} = 100 \text{ M} \times 10 \times 2 \text{ KB} = 2 \text{ TB}

\text{User interaction indexing storage} = 2 \text{ TB} \times 0.25 = 0.5 \text{ TB} 


\begin{align*}
\text{Storage required per day}
&=  2 \text{ TB} + 0.5 \text{ TB} \\ &=2.5 \text{ TB/day}

\end{align*}

\text{Monthly  storage  required} = \text{2.5 TB} \times 30 = \text{75  TB/month}

\text{Inference servers required} = \frac{\text{TRPS}}{\text{QPS}}= \frac{11574}{104} \approx 112 \text{ GPU servers}

\begin{align*}
\text{Ingress bandwidth}
&= 11574 \times \text{2 KB} \\ &=23.15 \text{ MBbps} \\ &=23.15 \text{ MBps} \times 8\approx 185 \text{ Mbps}


\end{align*}

\begin{align*}
\text{Egress bandwidth}

&= 11574 \times 10 \text{ KB} \\ &= 115.74 \text{ MBps} \\ 
& =115.74 \text{ MBps}\times 8\approx \   926 \text{ Mbps} 


\end{align*}

Understand the System Design for deploying a text-to-text generation model like ChatGPT. 

Deploying the System Design of a Text-to-Text Generation System

Understand the System Design for deploying a text-to-text generation model like ChatGPT.

Introduction to GenAI System Design

Fundamental Concepts in GenAI

Back-of-the-envelope Calculations

Systematic Framework for Designing GenAI Systems

System Design of a Text-to-Text Generation System

System Design of a Text-to-Image Generation System

System Design of a Text-to-Speech Generation System

System Design of a Text-to-Video Generation System

Conclusion

Deploying the System Design of a Text-to-Text Generation System