Pipe and Filter

Overview

There are various systems that involve multiple stages of transformation of discrete data items from input to output. To make these transformations more efficient and practical, it’s often useful to create them as standalone and reusable components.

For example, a compiler takes input which performs lexical analysis, and tokens are generated. These tokens are used for syntax analysis, which generates a parse tree, followed by semantic analysis, and so on. An output of one function becomes the input of the next, and hence data flows.

Architecture description

To improve efficiency and flexibility, systems that handle data transformation should be divided into reusable, independent components that communicate with each other using simple and generic interaction mechanisms. This allows the components to be easily combined and reused. Additionally, since the components are independent and loosely coupled, they can execute in parallel.

The pipe and filter pattern ...