Gain insights into data processing and concurrency in Elixir. Explore GenServer, GenStage, Flow, and Broadway to enhance performance, set up pipelines, and handle increased workloads efficiently.

Concurrent Data Processing in Elxir by The Pragmatic Programmer-01.png

myDokcer.tar.gz

ch1_1

ch_3

ch_3.1

ch1_1-task supervisor

ch_4.1

ch_5.1 RabbitMQ

Ch_5.2 RabbitMQ

Most engineers don’t think that concurrency is distinct from programming. Data processing includes transforming information in some way, like reporting, data aggregation, or analytics. On the other hand, concurrency involves significantly improving performance.

This course is designed to help you navigate the world of concurrency tools available in the Elixir ecosystem. You’ll explore popular modules and libraries like GenServer, GenStage, Flow, and Broadway. You’ll be able to ingest data, set up pipelines, and run concurrent processes with Elixir.

By the end of the course, you’ll have a wide range of new techniques to simplify your product, improve code performance, and make your application more resilient to increased workloads.

Concurrent Data Processing in Elixir

The ________ module is a frequently used tool when working with data. 

The ________ module is a frequently used tool when working with data.

________ provides a function called `partition/2` that creates another layer of stages that then act as a router.

________ provides a function called <code>partition/2</code> that creates another layer of stages that then act as a router.

**(True or False)** The flow will guarantee that same-key items will be sent to the same reducer process. 

(True or False) The flow will guarantee that same-key items will be sent to the same reducer process.

**(True or False)** An accumulator could be of any data type.

(True or False) An accumulator could be of any data type.

**(True or False)** We use `:timer.tc/1` to measure how long a function takes to run.

(True or False) We use <code>:timer.tc/1</code> to measure how long a function takes to run.

The _______ module has a direct equivalent in Flow.

The _______ module has a direct equivalent in Flow.

All trigger functions take a/an ________ as their first argument.

All trigger functions take a/an ________ as their first argument.

(Select all that apply.) Which callback functions group the data by the given criteria?

(Select all that apply.) Which callback functions group the data by the given criteria?

**(True or False)** Operations like `Flow.reduce/3`, `group_by/2` and `take_sort/3` require all events to complete before producing a result.

(True or False) Operations like <code>Flow.reduce/3</code>, <code>group_by/2</code> and <code>take_sort/3</code> require all events to complete before producing a result.

Can we use both `Flow` and `GenStage` together?

Can we use both <code>Flow</code> and <code>GenStage</code> together?

Quiz: Processing Collections with Flow

Quiz: Processing Collections with Flow

Check what you've learned in this chapter.


Quiz: Process Collections with Flow

Check what you've learned in this chapter.

Getting Started

Easy Concurrency with the Task Module

Long-Running Processes Using GenServer

Data Processing Pipelines with GenStage

Process Collections with Flow

Data Ingestion Pipelines with Broadway

Concluding the Course

Quiz: Process Collections with Flow