Employee Module

The employee module is where we’ll bridge the parsing of records with filtering and searching through employees. Proper isolation of concerns should make it possible for both types of users, those who create and those who consume employee data, to do everything they need without knowing about the requirements of the other.

Setting requirements

Our challenge here is to come up with an internal data representation that can easily be converted to CSV, all the while also allowing us to use employee entries the way we would handle any other data structure. Of course, we’ll want to test this with properties. Let’s start with the transformations from what the CSV parser hands to us. This was:

last_name, first_name, date_of_birth, email
Doe, John, 1982/10/14, john.doe@foobar.com
Ann, Mary, 1975/09/18, mary.ann@foobar.com

Now the thing to notice is that there are a couple of issues with the format of the document:

• The fields are messy and have extra leading spaces.
• The dates are in YYYY/MM/DD format. This is an issue because Erlang by default works with {Year, Month, Day}.

The transformation requires additional processing after the conversion from CSV, which could usually be done or handled by a framework or adapter. For example, most PostgreSQL connection libraries will convert the internal data type for dates and time to Erlang’s {{Year,Month,Day}, {H,Min,Sec}} tuple format without much of a problem. In the case of CSV, the specification is really lax, and as such, it’s our responsibility to convert from a string to the appropriate type, along with some additional validation.

We’ll define the following functionality:

• An accessor for each field (last_name/1, first_name/1, date_of_birth/1, email/1).
• A function to search employees by birthday.
• A from_csv/1 function that takes a CSV string and returns a cleaned-up set of maps representing individual employees.

Adapting CSV Data

Since we have already tested CSV conversion itself in the CSV Parsing lesson, We need to take the output from the parser and hammer it into shape. For our tests, this means that we won’t have to work with generating CSV data but with generating the data that needs cleaning up.

Properties

Let’s start with the leading spaces. We know all the fields that are required, and we know that all of the fields but the first will start with whitespace, so a property only needs to ensure that no fields start with whitespace once handled. We can see the property in the code widget below. We rely on the raw_employee_map() generator, call the private adapting function, and then check that the first character is not a space in any key or value.

Generators

Let’s Look at how to implement the generator. The first trick here is that instead of generating a map with the map() generator provided by PropEr, we’ll build one from a proplist. The issue with the default generator is that it takes types for keys and values, and doesn’t let us set specific values easily. A proplist and a ?LET macro will do just fine. Let’s take a look at the generator in the code widget below.

We also used the prop_csv:field() generator that we defined while writing the CSV parser.

We’ll add an unconditional leading whitespace in front of the field. We’ll also write a generator for the arbitrary date format provided for us in the file sample, which will need to be properly cleaned up as well.