Gain insights into data processing and concurrency in Elixir. Explore GenServer, GenStage, Flow, and Broadway to enhance performance, set up pipelines, and handle increased workloads efficiently.

Concurrent Data Processing in Elxir by The Pragmatic Programmer-01.png

myDokcer.tar.gz

ch1_1

ch_3

ch_3.1

ch1_1-task supervisor

ch_4.1

ch_5.1 RabbitMQ

Ch_5.2 RabbitMQ

Most engineers don’t think that concurrency is distinct from programming. Data processing includes transforming information in some way, like reporting, data aggregation, or analytics. On the other hand, concurrency involves significantly improving performance.

This course is designed to help you navigate the world of concurrency tools available in the Elixir ecosystem. You’ll explore popular modules and libraries like GenServer, GenStage, Flow, and Broadway. You’ll be able to ingest data, set up pipelines, and run concurrent processes with Elixir.

By the end of the course, you’ll have a wide range of new techniques to simplify your product, improve code performance, and make your application more resilient to increased workloads.

Concurrent Data Processing in Elixir

# Supervisor

Just like the workplace, where supervisors are held responsible for groups of employees, **Elixir supervisors** are responsible for the processes assigned to them. **Subordinate processes** report back to their supervisor to ensure everything runs smoothly. To accomplish this, supervisors come with a set of features that allow them to manage other processes effectively.

* They can start processes.

* They can stop processes.

* They can restart processes in the case of unforeseen system errors.

* Supervisors are configured to trap exits, so when a supervised process exits with an error, that error will be isolated. Therefore, it won’t propagate further.

All this makes supervisors an important building block for fault-tolerant applications.

# Child processes

Supervised processes are called **child processes**. Any OTP process can be supervised, and we can also add a supervisor as a child of another supervisor. We have to ask the supervisor to start a process we want to manage. This allows us to conveniently build a hierarchy of processes, also called a **supervision tree**.

Our sender application already has a supervisor in place, which we can see in `application.ex`:


Learn about supervisors, their properties, and integrate them into our project.

Getting Started

Easy Concurrency with the Task Module

Long-Running Processes Using GenServer

Data Processing Pipelines with GenStage

Process Collections with Flow

Data Ingestion Pipelines with Broadway

Concluding the Course

Meet the Supervisor

Supervisor