Gain insights into data processing and concurrency in Elixir. Explore GenServer, GenStage, Flow, and Broadway to enhance performance, set up pipelines, and handle increased workloads efficiently.

Concurrent Data Processing in Elxir by The Pragmatic Programmer-01.png

myDokcer.tar.gz

ch1_1

ch_3

ch_3.1

ch1_1-task supervisor

ch_4.1

ch_5.1 RabbitMQ

Ch_5.2 RabbitMQ

Most engineers don’t think that concurrency is distinct from programming. Data processing includes transforming information in some way, like reporting, data aggregation, or analytics. On the other hand, concurrency involves significantly improving performance.

This course is designed to help you navigate the world of concurrency tools available in the Elixir ecosystem. You’ll explore popular modules and libraries like GenServer, GenStage, Flow, and Broadway. You’ll be able to ingest data, set up pipelines, and run concurrent processes with Elixir.

By the end of the course, you’ll have a wide range of new techniques to simplify your product, improve code performance, and make your application more resilient to increased workloads.

Concurrent Data Processing in Elixir

# Name processes using the `Registry`

Until now, we used the process identifier obtained from starting the process to interact with it. However, we can optionally provide a `:name` value when starting a process. This can be used later instead of a PID. This way, we don’t have to keep track of the PIDs of the processes we start. Functions like `GenServer.cast/2` and `GenServer.call/3` also accept a name instead of a process identifier, and we’ve already seen how to use `Process.whereis/1` to find out the PID for `JobRunner`.

To make working with names easier, Elixir comes with a module called **`Registry`**. It’s an efficient key-value store with some useful functionality. Let’s see how we can use it.

Processes can be named when they start via the `:name` option.  This is what we did when creating `JobRunner`:

```Elixir
children = [
   {DynamicSupervisor, strategy: :one_for_one, name: Jobber.JobRunner},
]
```

Learn the registry processes, including how to name them, the types of naming techniques, and how to start them.

Getting Started

Easy Concurrency with the Task Module

Long-Running Processes Using GenServer

Data Processing Pipelines with GenStage

Process Collections with Flow

Data Ingestion Pipelines with Broadway

Concluding the Course

Registry Processes

Name processes using the `Registry`

Registry Processes

Name processes using the Registry

Name processes using the `Registry`