Testing Mapper

This lesson demonstrates how to write a unit test case for a Mapper class.

We'll cover the following

Testing Mapper

In the previous section, we wrote our CarMapper class. It takes in a long offset and a string line as input key-value pairs. The class then splits out car brand names and a count (always 1) as intermediate key-value pairs for the reducer function to consume. When developing code, we need to test it along the way. However, debugging and testing map reduce take a departure from the traditional testing and debugging of Java applications because of its distributed nature. We’ll start with unit testing.

Unit testing

Fortunately, the Java library MRUnit can be used to unit test MapReduce jobs. MRUnit has been retired; frameworks like Mockito are used instead to unit test MapReduce jobs. Nevertheless, we use MRUnit to get a feel for unit testing MapReduce. The unit test to exercise our CarMapper is shown below:

CarMapper unit test

    @Test
    public void testMapper() throws IOException {

        MapDriver<LongWritable, Text, Text, IntWritable> driver =
                MapDriver.<LongWritable, Text, Text, IntWritable>newMapDriver()
                        .withMapper(new CarMapper())
                        .withInput(new LongWritable(0), new Text("BMW BMW Toyota"))
                        .withInput(new LongWritable(0), new Text("Rolls-Royce Honda Honda"))
                        .withOutput(new Text("bmw"), new IntWritable(1))
                        .withOutput(new Text("bmw"), new IntWritable(1))
                        .withOutput(new Text("toyota"), new IntWritable(1))
                        .withOutput(new Text("rolls-royce"), new IntWritable(1))
                        .withOutput(new Text("honda"), new IntWritable(2));

        driver.runTest();
    }

The above code is self-explanatory. The library uses the builder pattern to create an instance of a MapDriver, initialized with an instance of the CarMapper class. In the following lines, we specify the input key value pairs with the withInput(...) method and then specify the expected output using the withOutput(...) method.

The complete runnable unit test case appears in the code widget below:

Get hands-on with 1400+ tech skills courses.