Gain insights into data processing and concurrency in Elixir. Explore GenServer, GenStage, Flow, and Broadway to enhance performance, set up pipelines, and handle increased workloads efficiently.

Concurrent Data Processing in Elxir by The Pragmatic Programmer-01.png

myDokcer.tar.gz

ch1_1

ch_3

ch_3.1

ch1_1-task supervisor

ch_4.1

ch_5.1 RabbitMQ

Ch_5.2 RabbitMQ

Most engineers don’t think that concurrency is distinct from programming. Data processing includes transforming information in some way, like reporting, data aggregation, or analytics. On the other hand, concurrency involves significantly improving performance.

This course is designed to help you navigate the world of concurrency tools available in the Elixir ecosystem. You’ll explore popular modules and libraries like GenServer, GenStage, Flow, and Broadway. You’ll be able to ingest data, set up pipelines, and run concurrent processes with Elixir.

By the end of the course, you’ll have a wide range of new techniques to simplify your product, improve code performance, and make your application more resilient to increased workloads.

Concurrent Data Processing in Elixir

# Simple examples

To keep our project and examples simple, we won’t send real emails. However, we still need some business logic for our experiments. We use the **`Process.sleep/1`** function to pretend to send an email. This is usually a slow operation and can take a few seconds to complete. When called with an integer, `Process.sleep/1` stops the current process for the given amount of time in milliseconds. This is very handy because we can simulate code that takes a long time to complete. We can also test various edge cases, as we’ll see [later](https://www.educative.io/courses/concurrent-data-processing-elixir/use-task-supervisor). Of course, in real-world production applications, we replace this with our actual business logic. But for now, let’s pretend we’re doing some very intensive work.

# Sender example

We created the sender project in the previous lesson. Let’s open `sender.ex` and add the following:

Learn to execute the task of sending emails and receiving confirmations in the form of notifications.

Getting Started

Easy Concurrency with the Task Module

Long-Running Processes Using GenServer

Data Processing Pipelines with GenStage

Process Collections with Flow

Data Ingestion Pipelines with Broadway

Concluding the Course

Start Tasks and Retrieve Results

Simple examples