Map-Reduce Pattern

What is the map-reduce pattern?

The map-reduce pattern is a way of processing large data sets in a way that can be distributed amongst many computers.

The basic idea is to start by processing data elements individually, and, finally, combine them to give the required result.

To give a simple example, suppose we want to calculate the average word length of the words in a block of text:

The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code – not in reams of trivial code that bore the reader to death.

We can break this task down into two steps:

• Count the number of letters in each word.
• Sum the total number of letters in all the words.

Dividing the sum by the number of words will give us our result, the average word length.

Here is a list of our words, with the punctuation removed:

strings = ['the', 'joy', 'of', 'coding', 'Python', 'should', 
'be', 'in', 'seeing', 'short', 'concise',